Advanced Pixel Search Library

[Mingre]

Hello,

it was initialy my idea to make it possible. Unfortunately, the only way to do that seems to be by using PrintWindow Windows function, and results were not reliable at all. In fact, it depends how printing is implemented within the application you want to capture, but you can't be sure it will work. It's also somewhat OS-dependant.

So, I'd say in some cases it may work, but you can't be sure it will, neither that refresh will be reliable when the Window is not visible.

For instance, when I've tried with Notepad on Seven 64, I guess it was OK. But a "flash" Window could'nt be captured when not visible.

FastFrench,

Then I guess I'll just have to give up in that case.

EDIT: Deleted my original question. Found the answer.

By the way, thanks for responding!

Edited May 19, 2011 by Lilbert

[monoscout999]

If you think that is useful you can add this functions.. or variants of this functions.. in C++... they will be much faster

expandcollapse popup

#RequireAdmin
#include <array.au3>
#include "FastFind.au3"
FFSetDebugMode(0x01)
$colors = _Getboxcolors(0, 0, 50, 50)
$mostcolors = _Showmostcolors($colors, 3)
_ArrayDisplay($colors)
_ArrayDisplay($mostcolors)
CloseFFDll()
Func _Getboxcolors($x1, $y1, $x2, $y2)
    Select
        Case $x1 > $x2
            SetError(1)
            ConsoleWrite("error number " & @error & " Your first x coor must be less than your second x coor")
            Return False
        Case $y1 > $y2
            SetError(2)
            ConsoleWrite("error number " & @error & " Your first y coor must be less than your second y coor")
            Return False
    EndSelect
    If $x2 = 0 Then $x2 = @DesktopWidth
    If $y2 = 0 Then $y2 = @DesktopHeight
    Local $y = $y1
    Local $n = (($x2 + 1) - $x1) * (($y2 + 1) - $y1)
    Local $colors[$n + 1][3] = [[0, "X", "Y"]]
    FFSnapShot($x1, $y1, $x2, $y2, 0)
    $start = TimerInit()
    For $y = $y1 To $y2
        For $x = $x1 To $x2
            $colors[0][0] += 1
            $n = $colors[0][0]
            $colors[$n][0] = Hex(FFGetPixel($x, $y, 0))
            $colors[$n][1] = $x
            $colors[$n][2] = $y
        Next
    Next
    $diff = TimerDiff($start)
    ConsoleWrite($diff & @CRLF)
    Return $colors
EndFunc   ;==>_Getboxcolors
Func _Showmostcolors($colorlist, $ncolors = 0)
    Local $lcol = _ArrayUnique($colorlist, 1, 1)
    If $ncolors = 0 Then $ncolors = $lcol[0]
    Local $rcol[$lcol[0] + 1] = [$lcol[0]]
    For $i = 1 To $lcol[0]
        For $n = 1 To $colorlist[0][0]
            If $lcol[$i] = $colorlist[$n][0] Then
                $rcol[$i] += 1
            EndIf
        Next
    Next
    Dim $enumcolors[$lcol[0] + 1][2]
    For $i = 0 To UBound($lcol) - 1
        $enumcolors[$i][0] = $lcol[$i]
        $enumcolors[$i][1] = $rcol[$i]
    Next
    $enumcolors[0][1] = "how many..."
    _ArraySort($enumcolors, 1, 1, 0, 1)
    Local $n = 1
    Local $mostcol[1]
    Local $howmany[1]
    While 1
        _ArrayAdd($howmany, $enumcolors[$n][1])
        _ArrayAdd($mostcol, $enumcolors[$n][0])
        If $n = $enumcolors[0][0] Then ExitLoop
        If $howmany[$n] = $enumcolors[$n + 1][1] Then
            $ncolors += 1
        EndIf
        If $n = $ncolors Then ExitLoop
        $n += 1
    WEnd
    Local $colorl[UBound($howmany)][2]
    For $i = 1 To UBound($howmany) - 1
        $colorl[$i][0] = $mostcol[$i]
        $colorl[$i][1] = $howmany[$i]
    Next
    $colorl[0][0] = UBound($colorl) - 1
    $colorl[0][1] = "How many..."
    Return $colorl
EndFunc   ;==>_Showmostcolors

one functio returns an array containing all the colors founding in a squad or rect... and his coord...

The other returns the most founded color and the followers in cuantity depends the parameter $ncolors...

[FastFrench]

(Update] Version 1.7 is available

[monoscout999]

(Update] Version 1.7 is available

good news...

anyway.. ¿what is the Raw Data?

[FastFrench]

good news...

anyway.. ¿what is the Raw Data?

It's the internal buffer that holds all the pixel data. 4 bytes per pixel. So if you have a 100x100 SnapShot, it's a 40000 bytes buffer. The 1st of the 4 bytes for each pixel is 0, then Red, Green and Blue. Honnestly, I've no idea how it may be used in AutoIt, but it probably can be useful for a somewhat advanced user

[FastFrench]

I've also added the link to FFShowPixels 1.7 in the first page :

Edited May 24, 2011 by FastFrench

[FastFrench]

If you think that is useful you can add this functions.. or variants of this functions.. in C++... they will be much faster

one functio returns an array containing all the colors founding in a squad or rect... and his coord...

The other returns the most founded color and the followers in cuantity depends the parameter $ncolors...

It's true that C++ can be much faster compared to autoIt, but it doesn't always make sense to do it in C++. For instance, I won't even consider doing something like the first function : it implies huge tranfert of data between C++ and AutoIt, and I don't like the idea. All I will do for that allready exists : FFGetPixel and RawData. The only reason to convert the color information into Hexa is if you want to display it on screen. And in that case, the time needed to display the information is usually greater than the time needed to compute those strings (even in AutoIt). So if you neeed this information in compact binary format, for efficient processing, then FFGetPixel and RawData do the job. If you need to convert that in displayable strings, then better do it in AutoIt.

For the second fonction, it would make sense for me to add something like that. One of the difficulty for this would be to search how to convert C/C++ arrays ou pointers into AutoIt Arrays.

[monoscout999]

It's true that C++ can be much faster compared to autoIt, but it doesn't always make sense to do it in C++. For instance, I won't even consider doing something like the first function : it implies huge tranfert of data between C++ and AutoIt, and I don't like the idea. All I will do for that allready exists : FFGetPixel and RawData. The only reason to convert the color information into Hexa is if you want to display it on screen. And in that case, the time needed to display the information is usually greater than the time needed to compute those strings (even in AutoIt). So if you neeed this information in compact binary format, for efficient processing, then FFGetPixel and RawData do the job. If you need to convert that in displayable strings, then better do it in AutoIt.

For the second fonction, it would make sense for me to add something like that. One of the difficulty for this would be to search how to convert C/C++ arrays ou pointers into AutoIt Arrays.

anyway... if i get the rawdata it will be much easier do what that functions do...

[Alandor]

(Update] Version 1.7 is available

Thank you very much.

[monoscout999]

It's the internal buffer that holds all the pixel data. 4 bytes per pixel. So if you have a 100x100 SnapShot, it's a 40000 bytes buffer. The 1st of the 4 bytes for each pixel is 0, then Red, Green and Blue. Honnestly, I've no idea how it may be used in AutoIt, but it probably can be useful for a somewhat advanced user

At the start i attemt to do it by my self and start using LockBits function... i do some progress but i start to look for the forum and i found this.. http://www.autoitscript.com/forum/index.php?app=forums&module=forums&section=findpost&pid=718522 .... so i addapt that to a funcition.. and all the credits for Malkey also i add a little test...

expandcollapse popup

#include <GDIPlus.au3>
#include <ScreenCapture.au3>
#include <Array.au3>
#include <GUIConstantsEx.au3>



$hBMP = _ScreenCapture_Capture("", 0, 0, 200, 200)
$pix = _getrawdata($hBMP)

_ArrayDisplay($pix)
$gui = GUICreate("test raw data")
$graph = GUICtrlCreateGraphic(100, 100, 200, 200)
GUISetState()
_printrawdata($pix, $graph)
GUICtrlSetState($graph, $GUI_SHOW) ; refresh graphic
Do
Until GUIGetMsg() = -3

Func _printrawdata($ArBMPRawdata, $hgraphic)
    For $x = 0 To UBound($ArBMPRawdata) - 1
        For $y = 0 To UBound($ArBMPRawdata, 2) - 1
            $color = "0x" & $ArBMPRawdata[$y][$x]
            GUICtrlSetGraphic($hgraphic, $GUI_GR_COLOR, $color)
            GUICtrlSetGraphic($hgraphic, $GUI_GR_PIXEL, $x, $y)
        Next
    Next

EndFunc   ;==>_printrawdata

Func _Getrawdata($hBMP) ; Credits to Malkey http://www.autoitscript.com/forum/index.php?app=forums&module=forums&section=findpost&pid=718522
    _GDIPlus_Startup()
    ; Getting the raw data of the pixels
    Local $hBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hBMP)
    Local $x = _GDIPlus_ImageGetWidth($hBitmap)
    Local $y = _GDIPlus_ImageGetHeight($hBitmap)
    Local $aSize = DllCall('gdi32.dll', 'int', 'GetBitmapBits', 'ptr', $hBMP, 'int', 0, 'ptr', 0)
    If $aSize[0] Then
        Local $tBits = DllStructCreate('byte[' & $aSize[0] & ']')
        DllCall('gdi32.dll', 'int', 'GetBitmapBits', 'ptr', $hBMP, 'int', $aSize[0], 'ptr', DllStructGetPtr($tBits))
        Local $sHex = Hex(DllStructGetData($tBits, 1))
        $sHex = StringRegExpReplace($sHex, "([[:xdigit:]]{6})(FF)", "\1 ")
        Local $pix[$y][$x], $aTemp
        Local $aArr = StringRegExp($sHex, "(.{" & ($x * 7) & "})", 3)

        For $x = 0 To UBound($aArr) - 1
            $aTemp = StringRegExp($aArr[$x] & " ", "(.{6}) ", 3)
            For $y = 0 To UBound($aTemp) - 1
                $pix[$x][$y] = StringRegExpReplace($aTemp[$y], "(.{2})(.{2})(.{2})", "\3\2\1") ; To RGB format
            Next
        Next
    EndIf
    _GDIPlus_BitmapDispose($hBitmap)
    _WinAPI_DeleteObject($hBMP)
    _GDIPlus_Shutdown()
    Return $pix
EndFunc   ;==>_Getrawdata

[FastFrench]

GetRawData is very similar, and the .au3 code to use it very close to this one with 'GetBitmapBits'.

[monoscout999]

i guess the screenshot is like your snapshot... if bout are the same.. with this function you can get the entire rawdata very quikly and you only search in the data for the value of the coord you want(pixelget) i m thinking in how to handle this entire data easily and do a query for example...

...I want to get the coord of all the colours that have in his red component a vaule > AA and the other compponents (Blue and green) < 55, then... you have a kind of shade variation!

Edited May 24, 2011 by monoscout999

[Alandor]

FastFrench, I would like to notice a bug when taking snapshots. There is always a one pixel vertical line (black most times but I have seen white too) added to the right side of the snapshot.

Greetings.

[FastFrench]

Hello,

thank you for the notice. Could you please tell me more precisely when it occurs ?

Do you use client or Window coordinates ? Do you capture the full Window ? Do you provide x1,y1,x2,y2 values or just keep 0 default values for a full window / screen SnapShot ?

Are you sure that your x2 value do not exceed width-1 ?

Do you see the problem on exported bitmaps or within you code ?

[Alandor]

Hello,

thank you for the notice. Could you please tell me more precisely when it occurs ?

Do you use client or Window coordinates ? Do you capture the full Window ? Do you provide x1,y1,x2,y2 values or just keep 0 default values for a full window / screen SnapShot ?

Are you sure that your x2 value do not exceed width-1 ?

Do you see the problem on exported bitmaps or within you code ?

It happened to me using FFSnapShot(0, 0, 0, 0, 1) or giving coordinates. Same exact coordinates used on the _ScreenCapture AutoIT built-in UDFs fit perfectly the screen capture region. I always notice this after use FFSaveBMP but really didn't checked if this happens on memory too.

[kmaworld586]

@FastFrench: This is indeed top-notch work.

Some questions:

1. Why is there a 1024 limit to exclusion rectangles?

Granted, to maintain speed a person would keep this number very low,

but in other applications where speed is not critical and broad accuracy may be,

the 1024 limit seems arbitrarily restrictive seeing as this number need not be hard-coded.

2. What about match small pattern inside larger square?

(example: looking at screen capture for a complete match of a 32x32

returns: co-ordinate of top-left)

3. Is this .dll multi-threaded?

If it were, speed could be dramatically increased

for more complex searches/matches.

Edited June 7, 2011 by kmaworld586

[FastFrench]

@Alandor : I'll check this when I have some time. I can't reproduce the problem when I restrict capture to the Client area. And with full Window captures, it's always a bit tricky to know the real border of the Window, on last versions of Windows and Aero active. Also it may depend on the width of the Window, as padding is sometime needed because each .BMP line must start on an even 32 bit boundary. So, if you have some time to investigate, can you check if you have the same problem when saving in JPG format ?

@kmaworld586 : Thank you for your interest in FastFind and questions/suggestions.

@FastFrench: This is indeed top-notch work.

Some questions:

1. Why is there a 1024 limit to exclusion rectangles?

Granted, to maintain speed a person would keep this number very low,

but in other applications where speed is not critical and broad accuracy may be,

the 1024 limit seems arbitrarily restrictive seeing as this number need not be hard-coded.

Granted, I will probably change those limits in some next version (same applies for colors in the list).

2. What about match small pattern inside larger square?

(example: looking at screen capture for a complete match of a 32x32

returns: co-ordinate of top-left)

I fear it would be complicate to use (and also, before you, no one seemed to be interested in this feature), that's the reason I didn't go forward on this. I think this would be even more useful with some alpha channel. But I can't think of an interface to make this easy to use (how to define the pattern... base on a bitmap file ? Another SnapShot ?)

3. Is this .dll multi-threaded?

If it were, speed could be dramatically increased

for more complex searches/matches.

Nop. And I'm not sure it would really make sense. The reason is that in most cases taking the Snapshot is the most time-consuming part, and this relies on a WinAPI call (runs on a single thread, even with a Seven 64 quad-core). Splitting this call into several smaller SnapShots would probably not save lot of time, and would even probably take more time than a single large call.

So, as searches take nearly no time compared to the SnapShot, I'm not sure the starting several threads for that would have any significant impact.

Probably on very large searches (like looking for the best 128x128 spot in a FullHD Snapshot, with a list of 1024 colors, 1024 exclusion rectangles, using shade variation), it would be useful. But who uses FastFind this way in a intensive way ? On my side, I hardly do over 10 searches on the same SnapShot, and those searches still take less time than the SnapShot itself.

Edited June 8, 2011 by FastFrench

[Alandor]

@Alandor : I'll check this when I have some time. I can't reproduce the problem when I restrict capture to the Client area. And with full Window captures, it's always a bit tricky to know the real border of the Window, on last versions of Windows and Aero active. Also it may depend on the width of the Window, as padding is sometime needed because each .BMP line must start on an even 32 bit boundary. So, if you have some time to investigate, can you check if you have the same problem when saving in JPG format ?

I will look into it.

I fear it would be complicate to use (and also, before you, no one seemed to be interested in this feature), that's the reason I didn't go forward on this. I think this would be even more useful with some alpha channel. But I can't think of an interface to make this easy to use (how to define the pattern... base on a bitmap file ? Another SnapShot ?)

Actually this is similar to the case I commented to you some posts ago about looking from a bitmap file.

[Alandor]

FastFrench, I have looked into it. I used this "code":

#include <FastFind.au3>
#include <ScreenCapture.au3>

Local $dir = "c:\nm5\temp\"

_ScreenCapture_Capture($dir & "prueba_udf.jpg", 0, 0, 40, 40)
_ScreenCapture_Capture($dir & "prueba_udf.bmp", 0, 0, 40, 40)
FFSaveBMP($dir & "prueba_FF", True, 0, 0, 40, 40)
FFSaveJPG($dir & "prueba_FF", 100, True, 0, 0, 40, 40)

The only difference is when using FFSaveBMP the result bitmap has 3 more horizontal pixels and a vertical black line at right. The other files are the same so the problem is only using FFSaveBMP. I attached the FFSaveBMP and the FFSaveJPG images (the FFSaveJPG is just the same the _ScreenCapture_Capture ones).

prueba_FF.bmp

[FastFrench]

Thank you very much Alandor for your investigation. So it's surely related with this padding in BMP files. I'll probably fix that in the next version.