Compare Images

[will88]

Is there anyway with autoit, that I can:

Take a picture and then compare it with multiple images until it finds the what image it matches with, and then type the word.

http://img370.imageshack.us/img370/7849/wordgt2.jpg

^ So basically what im trying to do is somehow recognize what the random word is and then type it.

Thanks

[jaberwacky]

Is there anyway with autoit, that I can:

Take a picture and then compare it with multiple images until it finds the what image it matches with, and then type the word.

http://img370.imageshack.us/img370/7849/wordgt2.jpg

^ So basically what im trying to do is somehow recognize what the random word is and then type it.

Thanks

I assume that you can do a PixelCheckSum on the initial picture. Store it to a variable. Then run a PixelCheckSum on the other pics. Finally, compare the variables for equality? That's probably pretty naive though. I don't foresee you breaking any CAPCHAs with this method though!

Edited January 8, 2009 by jaberwocky6669

[will88]

I assume that you can do a PixelCheckSum on the initial picture. Store it to a variable. Then run a PixelCheckSum on the other pics. Finally, compare the variables for equality? That's probably pretty naive though. I don't foresee you breaking any CAPCHAs with this method though!

Doesen't PixelCheckSum only check for changes on the screen?

Edited January 8, 2009 by will88

[jaberwacky]

Doesen't PixelCheckSum only check for changes on the screen?

But if the sum that it creates is unique then the sum could be used for future reference.

Try it:

dim $output1 = PixelCheckSum( 10 , 10 , 30 , 30 )
ConsoleWrite( $output1 & @CRLF )

dim $output2 = PixelCheckSum( 30 , 30 , 40 , 40 )
ConsoleWrite( $output2 & @CRLF )

If $output1 = $output2 then 
     ConsoleWrite( "Equality" & @CRLF )
Else
     ConsoleWrite( "Inequality" & @CRLF )
EndIf

Edited January 8, 2009 by jaberwocky6669

[will88]

I think this might work, didnt know before how to get it to show the sum.

Now I don't even have to do anything with pictures and just do this

dim $output = PixelCheckSum( 308 , 310 , 611 , 358 )
;ConsoleWrite( $output & @CRLF )
;MsgBox(0,"",$output)

If $output = "2534925219" Then
Send("bikhv")
Send("{Enter}")
Endif

Thanks

Edited January 8, 2009 by will88

[jaberwacky]

I think this might work, didnt know before how to get it to show the sum.

Now I don't even have to do anything with pictures and just do this

dim $output = PixelCheckSum( 308 , 310 , 611 , 358 )
;ConsoleWrite( $output & @CRLF )
;MsgBox(0,"",$output)

If $output = "2534925219" Then
Send("bikhv")
Send("{Enter}")
Endif

Thanks

You'd have to store up a huge amount of check sums. Seems like you should load each letter of the capcha alphabet into MS Paint. Delete the surrounding whitespace. Have AutoIt make a checksum of just the letter isolated by itself. Then after you have the entire alphabet and many many variations thereof you can then have autoit searh each capcha block looking for any matches.

[junkew]

Just some code to show how you can do this with string manipulation. BITBLT is more efficient but to demonstrate which way you could go.

Only part you have to do is segmentation of the different areas with letters.

Replace for debugging (longer than 1 char will show the hexdump of the colors)

local $whiteString= " "

local $blackString= "1"

;~ local $whiteString= " "

;~ local $blackString= "1 "

expandcollapse popup

Opt('MustDeclareVars', 1)
;http://www.autoitscript.com/forum/index.php?showtopic=86773&hl=
;White = R > 0x80, G > 0x80, B > 0x80
;#include <GUIConstants.au3>
#include <GDIPlus.au3>
#include <ScreenCapture.au3>
#include <string.au3>

; ** Example start **
Global Const $Bitmap1Filename = @TempDir & "\wordgt2.jpg"

Dim $BMP1Data = "", $BMP1Width = 0, $BMP1Height = 0, $BMP1LineWidth = 0
Dim $imgBytes, $image, $t

; Initialize GDI+ library
_GDIPlus_Startup()

; Load the bitmap to search in
GetImage($Bitmap1Filename, $BMP1Data, $BMP1Width, $BMP1Height, $BMP1LineWidth, 3)

_GDIPlus_Shutdown()

$BMP1Data = BinaryToString($BMP1Data)

dumpBlackWhite("test", $BMP1Data, $BMP1Height, $BMP1LineWidth)

;** Example end **

Func GetImage($BMPFile, ByRef $BMPDataStart, ByRef $Width, ByRef $Height, ByRef $Stride, $imgBytes = 3)
    Local $Scan0, $pixelData, $hbScreen, $pBitmap, $pBitmapCap, $handle, $bitmapdata, $pixelFormat

; Load the bitmap to search in
    If $BMPFile = "SCREEN" Then
        $hbScreen = _ScreenCapture_Capture("", 0, 0, -1, -1, False)
        $pBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hbScreen); returns memory bitmap
    Else
    ;try to get a handle
        $handle = WinGetHandle($BMPFile)
        If @error Then
        ;Assume its an unknown handle so correct filename should be given
            $pBitmap = _GDIPlus_BitmapCreateFromFile($BMPFile)
        Else
            $hbScreen = _ScreenCapture_CaptureWnd("", $handle, 0, 0, -1, -1, False)
            $pBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hbScreen); returns memory bitmap
        EndIf
    EndIf
    
;Get $tagGDIPBITMAPDATA structure
;~  ConsoleWrite("Bitmap Width:    " & _GDIPlus_ImageGetWidth($pBitmap) & @CRLF )
;~  ConsoleWrite("Bitmap Height:      " & _GDIPlus_ImageGetHeight($pBitmap) & @CRLF)

;~  24 bits (3 bytes) or 16 bits (2 bytes) comparison
    If ($imgBytes = 2) Then
        $BitmapData = _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF16RGB555)
    Else
        $BitmapData = _GDIPlus_BitmapLockBits($pBitmap, 0, 0, _GDIPlus_ImageGetWidth($pBitmap), _GDIPlus_ImageGetHeight($pBitmap), $GDIP_ILMREAD, $GDIP_PXF24RGB)
    EndIf
    
    If @error Then MsgBox(0, "", "Error locking region " & @error)
    
    $Stride = DllStructGetData($BitmapData, "Stride");Stride - Offset, in bytes, between consecutive scan lines of the bitmap. If the stride is positive, the bitmap is top-down. If the stride is negative, the bitmap is bottom-up.
    $Width = DllStructGetData($BitmapData, "Width");Image width - Number of pixels in one scan line of the bitmap.
    $Height = DllStructGetData($BitmapData, "Height");Image height - Number of scan lines in the bitmap.
    $pixelFormat = DllStructGetData($BitmapData, "PixelFormat");Pixel format - Integer that specifies the pixel format of the bitmap
    $Scan0 = DllStructGetData($BitmapData, "Scan0");Scan0 - Pointer to the first (index 0) scan line of the bitmap.
    
    $pixelData = DllStructCreate("ubyte lData[" & (Abs($Stride) * $Height - 1) & "]", $Scan0)
    $BMPDataStart = $BMPDataStart & DllStructGetData($pixelData, "lData")
    
    _GDIPlus_BitmapUnlockBits($pBitmap, $BitmapData)
    _GDIPlus_ImageDispose($pBitmap)
    _WinAPI_DeleteObject($pBitmap)

EndFunc  ;==>GetImage

Func dumpBlackWhite($sFname, $bmData, $h, $w)
    Local $i, $j, $tLine, $hexBytes
    Local $newLine, $newline2
    local $R, $G, $B
;Replace for debugging  
    local $whiteString= " "
    local $blackString= "1"
;~  local $whiteString= "     "
;~  local $blackString= "1   "

    For $i = 0 To $h - 1
        $tLine = StringMid($bmData, 1+($i * $w), $w)
        consolewrite(StringFormat("%06s:", 1 + ($i * $w)) )
        $newLine="" 
        for $j=0 to ($w / 3)
;~          $HexBytes=_StringToHex(stringmid($tLine, 1+($j * 3), 3))
            $HexBytes=_StringToHex(stringmid($tLine, 1+($j * 3), 3)) 
            $R=stringinstr("89ABCDEF",stringmid($hexbytes,1,1)) > 0 and stringinstr("0123456789ABCDEF",stringmid($hexbytes,2,1)) > 0
            $G=stringinstr("89ABCDEF",stringmid($hexbytes,3,1)) > 0 and stringinstr("0123456789ABCDEF",stringmid($hexbytes,4,1)) > 0
            $B=stringinstr("89ABCDEF",stringmid($hexbytes,5,1)) > 0 and stringinstr("0123456789ABCDEF",stringmid($hexbytes,6,1)) > 0
            
            if $r and $g and $b Then
                    $HexBytes = $whiteString 
            Else
                if stringlen($BlackString) = 1 then $HexBytes = $BlackString
            endif

            $newLine=$newline & $hexbytes
            if stringlen($BlackString) > 1 then $newline=$newline & " "
        next

            $newline2=stringleft($newline,1) 
        for $j=2 to stringlen($newline)-1 
            if stringmid($newline,$j-1,stringlen($Whitestring) * 3) = ($Whitestring & $Blackstring & $whiteString) Then
                $newline2=$newline2 & $Whitestring
            Else
                $newline2=$newline2 & stringmid($newline,$j,1)
            EndIf
        Next
        ConsoleWrite($newline2 & @CRLF)
    Next
EndFunc  ;==>dumpBlackWhite