Advanced Pixel Search Library

[frank10]

I think that for this motion detection, the 24 bit is useless: it's far better to work with <=8bit.

The fastest method is to consider only one channel, having a monocromathic pic similar to the grayscale pics, only with luma variation. Let's say we use only the blue channel or the channel less sensible to noise (I think is blue indeed).

Next step is looking how much noise interfere with the luma blue, then make some filter to uniform the luma value if inside a reasonable threshold, such as the cartoonize example but only with luma. This should lower the bit resolution to 4-6 bit. Here, starting from only the blue channel it could be better your filter idea.

This is the most critical, as in this stage we need to eliminate the noise random variations.

Next, the most important is to search for drastic changes in the contiguos pixels as it shouldn't be any sparse ones.

Another complete different approach could be edge detection, this should always work also with noise because a profile is always detectable also in a noisy pic: at least it's possible to detect the variation and the area position, still if not perfectly exact in the shape because of noise.

Maybe this is the best option, but more difficult at the beginning to code.

[JohnOne]

The person in this post seems to get away with it, using a web cam.

[frank10]

It's different because he knows which uniform color to search for.

In our case we don't know which color to search: we only want to know which large area with a lot of different colors, changed suddenly.

[JohnOne]

So are uniform colours exempt from noise?

[CeramicWeasel]

If they're not, then you'll still get some noise even after filtering.

Using the filter method suggested by FastFrench, a Blue value of 63 (for example) that gets bumped to 64 by noise is still going to show up as different from one snapshot to the next after filtering.

I think the earlier suggestion of putting a ShadeVariation on FFLocalizeChanges as a noise filter is the simplest and most effective solution.

[frank10]

So are uniform colours exempt from noise?

No, but if you have a pic with no green color and you put in a green uniform ball, the noise will alter some of the green colors, but you surely will find at least one green pixel in that little area.

For that application it's sufficient to determine where the motors should turn the robot.

@CeramicWeasel

Yes, but it's possible to lower the bits and you will have always less differences.

As an extreme it's possible to use a 1bit pix BW. Anyway it's not possible eliminate all the noise, the point is to lower the noise as much as possible and the next step is looking only at the areas with big changes: these must be the contiguos pixels of the detection.

I attach an example of noise between 1sec snapshot: in fact there are a lot, also contiguos!

EDIT: I attached another pic with more changed px in a low-light condition.

BTW, the pixel not changed are black.

Edited January 19, 2012 by frank10

[JohnOne]

Crikey! I'd get a better camera.

[frank10]

It's not so bad, it's a Philips spc900. Anyway there are surely better now, but I don't think it will be better with another one:

I fear that on a reasonably noisy 24 bits per pixel picture recording, most - if not all - pixels will change from one picture to another. Even if the change is usually not perceptible, the color will usually not be exactly the same.

Edited January 19, 2012 by frank10

[JohnOne]

Time for some discreet colour stickers then I think.

[frank10]

I made some analysis on two screenshots to see how much different pixels there are and especially in which channel and how much shade variations:

So, in a 640x480 there are a total of 251.894 different pixels, but getting only one channel we have:

Blu: 226.371

Green: 186.288

Red: 190.205

So the best channel is green.

The shade variations for each channel (pic become 8bit) are:

Bvariation:
[0]|25523
[1]|81649
[2]|58393
[3]|40185
[4]|24354
[5]|12208
[6]|6371
[7]|2111
[8]|725
[9]|256
[10]|119
Gvariation:
[0]|65606
[1]|113874
[2]|49778
[3]|17525
[4]|3885
[5]|906
[6]|178
[7]|77
[8]|65
Rvariation:
[0]|61689
[1]|112991
[2]|51042
[3]|18838
[4]|5421
[5]|1519
[6]|273
[7]|89
[8]|32

So, it happens to be only a max of 8 levels of variation in green or red channel.

If we don't look at a variation <5, we get for the green channel only 320 different pixels with more than 5 levels of shade variations!

So we got from 251.894 to 320 px! And sparse too...

Just this analysis is a great noise filtering!

If we consider a motion detection, the pixels have surely a shade variation much more than 8 levels! And all contiguos too, so I think it's very easy from this point to detect it.

I made this analysis transforming the px colors in array strings and then compare them.

Using GDI+ on all pic isn't realtime, but it's possible to make it on a small portion of the pic, the motion detection area we want and it should be realtime.

I don't know what do you use for comparing px, but you could do a similar approach.

[FastFrench]

Very interesting, thank you.

If I can find the time for another release, then I'll add a ShadeVariation parameter on FFLocalizeChanges, maybe also something to retrieve a snapshot from a bitmap file, and also a Filtering function. The difficulty on the 1st point, it's to keep good performance on the function FFLocalizeChanges. Adding ShadeVariation involves much more computing. But in all cases, this solution would be much faster than any corresponding processing in Autoit language.

[frank10]

Ok.

I second the BMP snapshot: it could be very interesting.

I learned also how to solve the problem of snapshots of windows hidden: it's not necessary to hide the win, you could leave it normal and instead move it outside the screen, so it's not visible and there perform the snapshots.

Like:

WinMove("YourWin",'', @DesktopWidth + 1,0)

Edited January 23, 2012 by frank10

[frank10]

Hi, I'm on W7 64bit.

I tried to load the 32bit FF dll, also forcing autoit to work in 32bit with

#AutoIt3Wrapper_UseX64=n

because I have another include that doesn't work with 64bit.

So I modified the FastFind.au3 to load only the FastFind.dll (that is in the same dir) , but I get the error message "Failed to load etc"

Does it work the FastFind.dll ?

[frank10]

I'm sorry: it works.

I run the script from Scite and it launched in 64bit anyway, so the dll at 32 bit couldn't work well...

[garreh]

This library is excellent FastFrench.

One thing I'd like to see is statistical functions, like Photoshops histogram. In particular, mean/median of either red, green, blue or full RGB channels.

Edited January 28, 2012 by garreh

[FastFrench]

Hello and thanks.

Could you explain for what kind of usage you would need this kind of functions ?

Mean values would be very easy to have. Median is a bit more complicate.

BTW : Mean values on a specified rectangle would also provide an intersting alternative for Frank10 problem.

Edited January 30, 2012 by FastFrench

[garreh]

Could you explain for what kind of usage you would need this kind of functions ?

Mean values would be very easy to have. Median is a bit more complicate.

BTW : Mean values on a specified rectangle would also provide an intersting alternative for Frank10 problem.

Exactly, it will be useful to determine a change in colour (i.e. a rudimentary motion detection). This is how I would use it also... take a i.e. 150x100 pixel rectangle and determine the mean colour (I will do tests to work out which colour channel red, green, blue or full RGB gives the most accurate result) and compare this from the previous mean value. I've done a few tests in Photoshop using the histogram and a pattern definitely is there when there is movement in that area, it looks like a very promising solution.

Usage examples [idea]

FFMeanPixels(<colour channels>)

-- colour channels (string): R for red, G for green, B for blue, doesn't matter which order>

FFMeanPixels("RGB") or just FFMeanPixels()

--> Will return the mean colour of pixels for full RGB channel

--> Will calculate the mean of all the channels combined i.e. (R_mean+G_mean+B_mean)/3

FFMeanPixels("GB")

--> returns the mean colour pixels for green & blue channel

--> (G_mean+B_mean)/2

Edited January 30, 2012 by garreh

[garreh]

Actually should be syntax: FFMeanPixels(<colour channels>,

, [top],

, [bottom], [forceNewSnapshot=false])

[FastFrench]

Actualy I have not much time to make those changes (and also not using AutoIt some time ago). Would someone be ready to implement the AutoIt wrapper part if I provide him updated dll with corresponding prototype to use ?

[ChinaWild]

i would like to use this in c++, could you add something like a header or lib file?