Image Compare [lighting, motion detection]

[nullschritt]

A few days ago I wrote a script to get the relative difference between two sets of data.

 

When this comes to images we can detect all sorts of things, more accurately.

 

Take for example you have two images with different lighting.

 

http://prodynamics.co/link.php?CMD=file&ID=compare1 (1600x976)

http://prodynamics.co/link.php?CMD=file&ID=compare2 (1600x976)

 

(These are both jpg images, their format is stored in the database)

 

You will notice that the second image is considerably brighter than the first.

Every pixel of the second image is at least one shade brighter than the second, this means that there was an absolute change of 100% to the image, HOWEVER, lets say we only want to know how much light was added to the image, that's one of the things this script can detect. The script would say the image has only changed by 12%, that's because the new data is only 12% different than the old data, even though 100% of it has changed.

 

Now in order to be efficient, the script which converts the images into pixel data are set to skip by default ever 2nd row of pixels and read every other pixel in each row. This still provides pretty accurate results for changes larger than 1px(1px changes may be missed sometimes), and could still accurately display where a change occurred or track a moving object.(with modification to use processed data in said way)

 

In the case you are comparing images which have many 1px changes, you can set the step for both y and x to 1, and the function will compare every single pixel. Be warned though, this is very slow.

 

Large images can use a higher step, but using too high of a step may exclude certain changes from being observed.

 

That being said, if you want to compare large images quickly, without leaving out too much data, there is an alternative, you can size the image down to 1/4th 1/6th or 1/8th it's size and compare it. Scaling down two images keeps the relative data in them the same places, so the comparison will return almost identical results (off by maybe 2% max)Note: In most cases scaling will result in more accurate results than stepping.

 

The below example compares the above two images, and output the data to the console.

expandcollapse popup

;~ #include <_PixelGetColor.au3>
#include <GDIPlus.au3>
#include <inet.au3>
#include <array.au3>
$hDll = DllOpen("gdi32.dll")
_GDIPlus_Startup()

Global $image1, $image2

$f1 = binary(_INetGetSource("http://prodynamics.co/link.php?CMD=file&ID=compare1")) ;image 1
$f2 = binary(_INetGetSource("http://prodynamics.co/link.php?CMD=file&ID=compare2")) ;image 2 brighter

;here we use the default step options
$t1 = TimerInit()
$image1 = _Capturepixels($f1) ;get image 1 pixel
ConsoleWrite(TimerDiff($t1)/1000 &' Default Step'&@CRLF)
$t1 = TimerInit()
$image2 = _CapturePixels($f2) ;get image 2 pixels
ConsoleWrite(TimerDiff($t1)/1000& ' Default Step'&@CRLF)
$timer = TimerInit()
$compare = _datacompare($image1[0], $image2[0]);compare them
ConsoleWrite($compare[0]&'% Different(Localized) '&$compare[1]&'% Different(Global)'&@CRLF&'Took '&(TimerDiff($timer)/1000)&' seconds. Default Step'&@CRLF)

;here we double them, notice the preformance increase
$t1 = TimerInit()
$image1 = _Capturepixels($f1, 4, 8) ;get image 1 pixels
ConsoleWrite(TimerDiff($t1)/1000 &' Double Step'&@CRLF)
$t1 = TimerInit()
$image2 = _CapturePixels($f2,4 , 8) ;get image 2 pixels
ConsoleWrite(TimerDiff($t1)/1000& ' Double Step'&@CRLF)
$timer = TimerInit()
$compare = _datacompare($image1[0], $image2[0])
ConsoleWrite($compare[0]&'% Different(Localized) '&$compare[1]&'% Different(Global)'&@CRLF&'Took '&(TimerDiff($timer)/1000)&' seconds. Default Step'&@CRLF)
sleep(5000)


;~ $t1 = TimerInit()
;~ $diffarray = _mapchange($image1[0], $image2[0], $image1[1], $image1[2]) ;compare two images for difference
;~ ConsoleWrite(TimerDiff($t1)/1000& ' _mapchange'&@CRLF)
;~ $t1 = TimerInit()
;~ $image = _toimage($diffarray) ;here we turn the array of colors back into an image
;~ ConsoleWrite(TimerDiff($t1)/1000& ' _toimage'&@CRLF)
;~ _GDIPlus_ImageSaveToFile($image, @scriptdir&'\test.jpg') ;write it to a file
;~ shellexecute(@scriptdir&'\test.jpg')


#cs
Function _datacompare($data1, $data2, [$declimal])
    -$data1: A string of data, any length.
    -$data2: A string of data, must be the same length as $data1
    -$decimal: 1=Binary 9=Base-10, 15=Base-16, 31=Base-32

    Note: If you just want to compare two sets of binary data
    you probably want to use base-16. Unless you are sure your
    binary is in 1's and 0's.

    Returns: An array containing two floats. The first
    value is the localized change, and the second is the global change
#ce
func _datacompare($data1, $data2, $decimal=15)
    Local $difference
$data1 = StringSplit($data1, "")
$data2 = StringSplit($data2, "")
$difference = 0
$found = 0
for $i=1 to $data1[0]
if $data1[$i] <> $data2[$i] Then
    $temp = Abs(_tonum($data1[$i]) - _tonum($data2[$i]))
    $difference += $temp
    $found +=1
EndIf
Next
dim $ret[2]
$ret[0] = ((($difference/$found))/$decimal)*100
$ret[1] = ((($difference/$data1[0]))/$decimal)*100
Return $ret
EndFunc

#cs
Function: _mapchange($base, $new, $x, $y, [$decimal])
    $base: Base data to compare from
    $new: Data to compare
    $x: Width of output data (should be returned by capturepixels)
    $y: Height of outout data (should be returned by capturepixels)
    $decimal: Decimal system, you shouldn't change this

    Note: Use _toimage on data returned by this function to visually see
    a image of the change. (The lighter the color the more change the occured)

    Returns an 2D array of data. Each value of the array represents
    one pixel of the image. Each value is a percent between 0-100
    representing the change that occured in that pixel
#ce
func _mapchange($base, $new, $y, $x, $decimal = 10)
    Local $difference, $xx = 0, $yy = 0
    dim $result[1][$x+1]
    $t1 = TimerInit()
$data1 = _StringequalSplit($base, 8)
$data2 = _StringequalSplit($new, 8)
$difference = 0
for $i=1 to UBound($data1)-1
    if $xx > $x Then
        $xx=0
        $yy+=1
        ConsoleWrite($yy&'/'&$y&' ('&($yy/$y)*100&') '&@CRLF)
        redim $result[$yy+1][$x+1]
    EndIf
    if $data1[$i] <> $data2[$i] Then
    $values1 = StringSplit($data1[$i], "")
    $values2 = stringSplit($data2[$i], "")
    $diff = ""
    for $ix=1 to $values1[0]
    $diff += round((Abs(_tonum($values1[$ix]) - _tonum($values2[$ix]))/$decimal)*100)
    Next
    $diff = Round($diff/$values1[0])
    $result[$yy][$xx] = $diff
    Else
    $result[$yy][$xx] = 0
    EndIf
    $xx += 1
Next
return $result
EndFunc


#cs
Function _tonum($info)
    -$info: A single digit or carachter.

    Returns: A 0-based value.
#ce
func _tonum($info)
if $info+0 > 0 Then Return $info
$info = StringLower($info)
$return = asc($info)-87
switch $return
    Case -39
        Return 0
    Case Else
        Return $return
EndSwitch
EndFunc

#cs
Function _CapturePixels($data, [[$stepy], $stepx])
    -$data: Binary Data
    -$stepy: How often to skip a row of pixelxs. 1 = Never
    -$stepx: How often to skip a single pixel. 1 = Nevere
    Note: Use higher steps for larger images and lower steps
    for smaller images.
#ce
Func _CapturePixels($data, $stepy = 2, $stepx = 2)
    $ret = ""
    $HBITMAP2 = _GDIPlus_BitmapCreateFromMemory($data)
    $y=_GDIPlus_ImageGetWidth($HBITMAP2)
    $x=_GDIPlus_ImageGetHeight($HBITMAP2)

    For $iY = 0 To $x step $stepy
        For $iX = 0 To $y step $stepx
            $rety = StringRight(hex(_GDIPlus_BitmapGetPixel($hBitmap2, $ix, $iy)),8) ;get current pixel color
            $ret &= $rety
;~          ConsoleWrite($iy&'/'&$x&' '&$rety&@CRLF)
            Next
    Next


       ;For $x = 0 To _GDIPlus_ImageGetWidth($HBITMAP2)
       ;  For $y = 0 To _GDIPlus_ImageGetHeight($HBITMAP2)
       ;      $ret &= _PixelGetColor_GetPixel($vDC, $x, $y, $hDll)
      ;   Next
     ;Next
    _WinAPI_DeleteObject($HBITMAP2)
    dim $retx[3]
    $retx[0] = $ret
    $retx[1] = $x/$stepx
    $retx[2] = $y/$stepy
    Return $retx
EndFunc   ;==>Capturepixels

Func _toimage($colors)
    _GDIPlus_Startup()
    Local $hBitmap = _GDIPlus_BitmapCreateFromScan0(UBound($colors, 2), UBound($colors, 1)) ;create an empty bitmap
    Local $hBmpCtxt = _GDIPlus_ImageGetGraphicsContext($hBitmap) ;get the graphics context of the bitmap
    _GDIPlus_GraphicsSetSmoothingMode($hBmpCtxt, $GDIP_SMOOTHINGMODE_HIGHQUALITY)
    _GDIPlus_GraphicsClear($hBmpCtxt, 0x00000000) ;clear bitmap with color white

    for $i=0 to UBound($colors)-1
        for $i2=0 to UBound($colors,2 )-1
;~          if $colors[$i][$i2] > 30 Then
;~          ConsoleWrite($i&","&$i2&' - '&$colors[$i][$i2]&@CRLF)
        _GDIPlus_BitmapSetPixel($hBitmap, $i2, $i, $colors[$i][$i2]&'0')
;~      ConsoleWrite($i2&','&$i&' '&$colors[$i][$i2]&@CRLF)
;~          EndIf
        Next
    Next

    return $hBitmap ;return bitmap
    ;cleanup GDI+ resources
    _GDIPlus_GraphicsDispose($hBmpCtxt)
EndFunc   ;==>Example

Func _StringEqualSplit($sString, $iNumChars)
    If (Not IsString($sString)) Or $sString = "" Then Return SetError(1, 0, 0)
    If (Not IsInt($iNumChars)) Or $iNumChars < 1 Then Return SetError(2, 0, 0)
    Return StringRegExp($sString, "(?s).{1," & $iNumChars & "}", 3)
EndFunc

Cheers,

NullSchritt

 

PS: If I find the time, I'll add a motion tracking version of the script as well. (It can detect but not track motion in it's current form)

Edited February 14, 2014 by nullschritt

[nullschritt]

Notes: for simple motions detection, it may be faster and acceptable to use high step numbers instead of shrinking the image down. (unless you're trying to catch ninjas)

Further explanation on stepping:

A stepping value of 1, 1 will means every detail of change is detected, clear down to a small bug. Increasing the scaling value will create gaps between the scanned data, splitting it into a grid. For example a stepping value of 4, 12 would scan only scan every 4th horizontal line, and every 12th pixel. Imagine a grid laid over the image, and the higher the step values get the bigger the cell sizes get. A setting of 4, 12 would probably not detect a fly buzzing about, but could detect a person or animal walking past. If you only wanted to detect an adult human you could probably even use values of up to 16, 32.

If you want stepping type speed without losing any detail, you should scale your image down.

[jaberwacky]

So cool.  Has anyone seen that program that highlights movements red?  So you can see minor variations in motion?  I want to do that and this script might help do that.  But it's all on the back burner for now.

[nullschritt]

Updated

• added _trackchange()
• added _toimage()
• tonumber() to _tonumber
• capturepixels to _capturePixels()
• bug in capturepixels fixed.

You can now track motion using the arrays returned by _trackchange(). use _trackchange() on image 1 and 2 and then on 2 and 3. Then overlay the second array on the first, only replacing the values if they are above 20%. This will result in an array containing the path taken by any moving objects. _toimage can visually display the path on an image, among other uses.

 

See first post for updated code.

Note: 20% is a general threshold for ambient lighting changes, you should adjust this value for your personal needs.

Edited February 13, 2014 by nullschritt

[nullschritt]

Find Attached a mapped images displaying the lighting changes between the two pictures in the example.

This was produced with a step setting of 1,1, using _trackchange() and _toimage()

(click to enlarge)

This was produced with the default step setting(2, 2)[notice some detail loss]

(click to enlarge).

And Here's the same result using a step setting of 4, 4 (notice how the structure of the data is still nicely preserved, but detail is lost.)

(max size)

The darker blues represent areas of less change, and lighter blues represent areas of more change

Note: The above three images show how stepping works like a make-shift scaling system, but as you can see, fine detail is lost.

And because why not.

Here's a 8,8 step

and here's a 16,16 step

Edited February 13, 2014 by nullschritt

[Werty]

So cool.  Has anyone seen that program that highlights movements red?  So you can see minor variations in motion?  I want to do that and this script might help do that.  But it's all on the back burner for now.

 

You mean this ?...

http://bits.blogs.nytimes.com/2013/02/27/scientists-uncover-invisible-motion-in-video/?_php=true&_type=blogs&_r=0

Matlab source is available somewhere in one of the links on the page.

[nullschritt]

You mean this ?...

http://bits.blogs.nytimes.com/2013/02/27/scientists-uncover-invisible-motion-in-video/?_php=true&_type=blogs&_r=0

Matlab source is available somewhere in one of the links on the page.

 

That's amazing! My functions above could preform this same functionality(with the_trackchange() function)! Cool. (Though autoit is not fast enough to process the video in real time)

[jaberwacky]

You mean this ?...

http://bits.blogs.nytimes.com/2013/02/27/scientists-uncover-invisible-motion-in-video/?_php=true&_type=blogs&_r=0

Matlab source is available somewhere in one of the links on the page.

Yes that thank you.

But matlab though.  I dunno, honestly, autoit has made me kinda lazy about other languages.

[jaberwacky]

That's amazing! My functions above could preform this same functionality(with the_trackchange() function)! Cool. (Though autoit is not fast enough to process the video in real time)

Yeah I know right!  I think it might be a little more complex than I suspect.  Wouldn't I need to be able to find lines in an image?

For instance, I want to record my car engine running.  Then run a script which would highlight movements.  So if I aim it at the belts, I'm thinking that it will need to be able to identify outlines.  Dunno yet, just a seed of an idea so far.  ...

OK, thinking on it, I don't think so, any change is colored red for a short time.  I'm thinking it will keep track of which regions changed to red the most to identify possible trouble spots.

[nullschritt]

Update:

• Updated the _datacompare() function to be more accurate.

 

Also here:s an example of using _trackchange with an object.

[nullschritt]

Yeah I know right!  I think it might be a little more complex than I suspect.  Wouldn't I need to be able to find lines in an image?

 

For instance, I want to record my car engine running.  Then run a script which would highlight movements.  So if I aim it at the belts, I'm thinking that it will need to be able to identify outlines.  Dunno yet, just a seed of an idea so far.  ...

 

OK, thinking on it, I don't think so, any change is colored red for a short time.  I'm thinking it will keep track of which regions changed to red the most to identify possible trouble spots.

No you don't need to identify the outlines of anything. When motion occurs it slightly changes the observed color in that area. All you have to do is amplify the change.

 

For example if you ran a video of a belt turning through _mapchanges() each array would have a belt shaped set of (low)percentage changes. To amplify the change you only need to then amplify the color which shade changed. Though I cant pretend to be an expert, I think it would be easiest to identify change by boosting the color in that area by the percentage that is already changed times ten. for example if the color changed by 5% we would further increase the shade by 50% to make the motion easily apparent to the naked eye.

Edited February 14, 2014 by nullschritt

[nullschritt]

UPDATE

• _Datacompare() now returns an array with two values, the first value is localized change, which only accounts for pixels that have actually changed, and measures how much they have changed. Global calculates in all pixels in the photo, even ones that haven't changed. As such, to put them simply. Global calculates the difference that occurred overall in the whole photo, localized calculated how much has changed in the areas that have changed.

Note: These two values are relevant to detecting the difference between ambient light shifts, and on screen motion. (though a simple script that monitors the local changes should be enough for simple motion detection, anything above 25% local change should indicate motion rather than lighting change).

 

If you have any recommendations or suggestions, please throw them my way.

Edit: Tonight/Tomorrow I will be modifying the local analysis algorithm, in order to exclude most lighting updates. This way local results will only return moving objects or VERY bright lights. Global percentage will include all data, including local data. This does not affect motion tracking in any way. Motion tracking returns the raw percentage of change for each pixel, it is up to you to decide how to process that data/determine lighting/motion.

Edited February 14, 2014 by nullschritt

[JohnOne]

Yeah I know right!  I think it might be a little more complex than I suspect.  Wouldn't I need to be able to find lines in an image?

For instance, I want to record my car engine running.  Then run a script which would highlight movements.  So if I aim it at the belts, I'm thinking that it will need to be able to identify outlines.  Dunno yet, just a seed of an idea so far.  ...

OK, thinking on it, I don't think so, any change is colored red for a short time.  I'm thinking it will keep track of which regions changed to red the most to identify possible trouble spots.

You might be thinking of opencv library.

[jaberwacky]

Werty found the thing I was looking for, but that is pretty cool too.

[nullschritt]

Sorry I abandoned this project, I might take it up at some point.

Just thought I would leave an update saying that this code can be used to detect objects cloaked with the new light bending cloaking technologies we are seeing.

(as the object/person being cloaked will cause motion to occur, even if its invisible to the naked eye) [which I think is a fairly interesting use of the software]

If I do pick this back up, it'll be to add support for processing videos instead of just pictures/frames, and a better lighting filter(show only changes in motion)

[rony2006]

Hello,

I tried your example but I get the following erorr:

>"D:\rocimpma\Desktop\autoit-v3\install\SciTe\..\autoit3.exe" /ErrorStdOut "D:\rocimpma\Desktop\calculator.au3"    
0.000279591963674815 Default Step
0.00016856378677614 Default Step
-1.#IND% Different(Localized) -1.#IND% Different(Global)
Took 4.32294582322573e-005 seconds. Default Step
0.000165142750513155 Double Step
0.000156123654910742 Double Step
-1.#IND% Different(Localized) -1.#IND% Different(Global)
Took 3.17223362567643e-005 seconds. Default Step
>Exit code: 0    Time: 5.694

 

Can somebody help me please? I really need something like this

[ChesTeRcs]

Greetings!

Maybe I'm a bit late with that, but @rony2006
Your console error reason is the images are missing.
You can fix it by changing this

$f1 = binary(_INetGetSource("http://prodynamics.co/link.php?CMD=file&ID=compare1")) ;image 1
$f2 = binary(_INetGetSource("http://prodynamics.co/link.php?CMD=file&ID=compare2")) ;image 2 brighter

to that

$f1 = binary(_INetGetSource("https://www.imgonline.com.ua/examples/difference.jpg")) ;image 1
$f2 = binary(_INetGetSource("https://www.imgonline.com.ua/examples/difference2.jpg")) ;image 2 brighter

Or you can just use files locally

$f1 = binary(FileRead(@ScriptDir & "\difference.jpg"))
$f2 = binary(FileRead(@ScriptDir & "\difference2.jpg"))

 

Have fun with it!

Edited June 30, 2019 by ChesTeRcs