Is there a faster way to DllStructSetData for pixel data

[mistersquirrle]

Using a slightly modified example from _GDIPlus_BitmapCreateFromScan0() :

expandcollapse popup

#include <GDIPlus.au3>
#include <GUIConstantsEx.au3>

Example()

Func Example()
    _GDIPlus_Startup()
    Local Const $iW = 500, $iH = 500

    Local $hGui = GUICreate("", $iW, $iH)
    GUISetState()
    Local $hGraphics = _GDIPlus_GraphicsCreateFromHWND($hGui)

    Local $tPixel = DllStructCreate("uint[" & $iW * $iH & "];")
    Local $iOffset

    Local $hBitmap, $hTimer

    While 1
        $hTimer = TimerInit()
        For $y = 0 To $iH - 1
            $iOffset = $y * $iW
            For $x = 0 To $iW - 1
;~              DllStructSetData($tPixel, 1, BitOR(0xFF000000, BitShift(Random(0, 255, 1), -16), BitShift(Random(0, 255, 1), -8), Random(0, 255, 1)), $iOffset + $x + 1)
                DllStructSetData($tPixel, 1, 0, $iOffset + $x + 1)
            Next
        Next
        ConsoleWrite('DllStructSetData = ' & Round(TimerDiff($hTimer), 2) & 'ms' & @CRLF)

        $hBitmap = _GDIPlus_BitmapCreateFromScan0($iW, $iH, $GDIP_PXF32ARGB, $iW, $tPixel)
        _GDIPlus_GraphicsDrawImage($hGraphics, $hBitmap, 0, 0)

        While 1
            Switch GUIGetMsg()
                Case $GUI_EVENT_CLOSE
                    ExitLoop 2

                Case $GUI_EVENT_RESTORE
                    ; to redraw the Bitmap
                    _GDIPlus_GraphicsDrawImage($hGraphics, $hBitmap, 0, 0)

                Case $GUI_EVENT_NONE
                    ExitLoop
            EndSwitch
        WEnd
    WEnd

    _GDIPlus_BitmapDispose($hBitmap)
    _GDIPlus_GraphicsDispose($hGraphics)
    _GDIPlus_Shutdown()
EndFunc   ;==>Example

 

It takes ~520ms or so per loop to do the DllStructSetData part itself on a 500x500 canvas. Obviously the size of it affects the speed, but it's the slowest part by far for a project I'm working on. What I'm trying to do is take data from a screenshot and overlay it onto the screen in a different position. Everything is working as intended, but it's just a bit slow. This example is basically exactly what I'm trying to do however.

 

So, is there a faster way to do DllStructSetData, as I need to use it with _GDIPlus_GraphicsDrawImage(), or is there a better way to read a byte array into a format to use with _GDIPlus_GraphicsDrawImage() ?

 

For some more reference, my current code is using LarsJ's DotNet.au3 to take a screenshot in C#, convert it to a byte array, and then send that array back to AutoIt for it to convert to the DllStruct and use with GDI+. If there's a better way in general, I'm open to it, or if you know if there's a way to return the data from C# it can very likely process it much faster than AutoIt.

 

[markyrocks]

Your example really doesn't show the problem bc it doesn't actually show where the bit array in question is coming from.  But I believe that if you set the dllstruct size properly you can just set the whole array in one shot.  Shouldn't be necessary to iterate over every element.   If that doesn't work for whatever reason you should be able to use something like a memcopy or write process memory function to write the whole thing to the pointer at once.  I will say that I think you're misinterpreting the point of the example.   I also think that you have needless steps in here.  Like why are you pulling the screenshot from c# and doing a conversion.   If you capture from autoit once you have the handle to the capture you can do whatever you need to do with it.  Maybe I'm missing something which is definitely possible bc of the example but that's just my 2c.

Edited October 15, 2021 by markyrocks

[mistersquirrle]

Do you have an example of how to 

4 hours ago, markyrocks said:

But I believe that if you set the dllstruct size properly you can just set the whole array in one shot.  Shouldn't be necessary to iterate over every element.

That was something that I was looking to do, but I'm not overall very familiar with DllStructs or how that would work.

 

The example is mostly just to show the speed of it, and it is as I mentioned exactly what I'm doing. Here's my actual code for that part (I'm not including the __PixelSearch* functions, which are in C#, but simulating the pixel data ($aScan) which is slow in AutoIt):

expandcollapse popup

#include-once
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <ScreenCapture.au3>
#include <WinAPIDiag.au3>

_GDIPlus_Startup()
Local $hGUI = GUICreate("GDI+ test", @DesktopWidth, @DesktopHeight, 0, 0, _
        $WS_POPUP, _
        $WS_EX_TOOLWINDOW + $WS_EX_TOPMOST)
GUISetStyle(BitOR($WS_POPUP, $WS_VISIBLE), BitOR($WS_DISABLED, $WS_EX_TRANSPARENT, $WS_EX_LAYERED))
GUISetBkColor(0x000000)

_WinAPI_SetLayeredWindowAttributes($hGUI, 0x000000)
Local $hGraphics = _GDIPlus_GraphicsCreateFromHWND($hGUI)

GUISetState()
HotKeySet("{ESC}", "__Exit")
OnAutoItExitRegister('__Exit')

WinSetTrans($hGUI, '', 128)

ConsoleWrite('Drawn GUI' & @CRLF)

While 1
    __UpdateGUI()
    Switch GUIGetMsg()
        Case $GUI_EVENT_CLOSE
            ExitLoop
    EndSwitch
WEnd

Func __UpdateGUI()
    Local $hUpdateTimer = TimerInit()
    Local $sMsg = ''
    Local $hTimer = TimerInit()
;~  __PixelSearchSSOnly(100, 500, 100 + 200, 500 + 200)
    Local $aStats[] = [True, 200, 200, 800, 4] ;= __PixelSearch_GetImgStats() ; True, Width, Height, Stride, BitsPerPixel (bpp)
    Local $iSize = ($aStats[1] * $aStats[2])
    $sMsg = 'SS - ' & Round(TimerDiff($hTimer), 2) & 'ms'

    Local $aScan[$iSize] ;= __PixelSearch_GetScan0()
    $hTimer = TimerInit()
    #Region Simulated Pixel/Byte Array
    For $i = 0 To $iSize - 1
        $aScan[$i] = BitOR(0xFF000000, BitShift(Random(0, 255, 1), -16), BitShift(Random(0, 255, 1), -8), Random(0, 255, 1))
    Next
    #EndRegion Simulated Pixel/Byte Array
    $sMsg = $sMsg & ', Scan0 - ' & Round(TimerDiff($hTimer), 2) & 'ms'

    $hTimer = TimerInit()

    Local $tPixel = DllStructCreate("uint[" & $aStats[1] * $aStats[2] & "];")

    For $x = 0 To $iSize - 1 ;step 2
        DllStructSetData($tPixel, 1 _
                , $aScan[$x] _
                , ($x + 1))
    Next

    $sMsg = $sMsg & ', StructSet - ' & Round(TimerDiff($hTimer), 2) & 'ms'

    $hTimer = TimerInit()
    Local $hBitmap = _GDIPlus_BitmapCreateFromScan0($aStats[1], $aStats[2], $GDIP_PXF32PARGB, $aStats[3], $tPixel)
    $sMsg = $sMsg & ', GDI - ' & Round(TimerDiff($hTimer), 2) & 'ms'

    $hTimer = TimerInit()
    _GDIPlus_GraphicsDrawImage($hGraphics, $hBitmap, (@DesktopWidth / 2), @DesktopHeight / 2)
    $sMsg = $sMsg & ', Draw - ' & Round(TimerDiff($hTimer), 2) & 'ms'

    $sMsg = $sMsg & ', Total - ' & Round(TimerDiff($hUpdateTimer), 2) & 'ms'

    ConsoleWrite($sMsg & @CRLF)
EndFunc   ;==>__UpdateGUI

Func __Exit()
    ;cleanup resources
    _GDIPlus_GraphicsDispose($hGraphics)
    _GDIPlus_Shutdown()
    GUIDelete($hGUI)
    Exit
EndFunc   ;==>__Exit

And here's what the output from that looks like when using my C# functions to generate the screenshot and pixel data (without them the Scan0 generation is ~100x slower):
 

SS - 7.86ms, Scan0 - 2.67ms, StructSet - 99.1ms, GDI - 0.32ms, Draw - 1.03ms, Total - 111.05ms
SS - 8.13ms, Scan0 - 3.18ms, StructSet - 92.98ms, GDI - 0.13ms, Draw - 0.43ms, Total - 104.91ms
SS - 3.85ms, Scan0 - 1.97ms, StructSet - 91.55ms, GDI - 0.17ms, Draw - 0.81ms, Total - 98.41ms
SS - 6.5ms, Scan0 - 2.71ms, StructSet - 93.83ms, GDI - 0.2ms, Draw - 0.6ms, Total - 103.91ms
SS - 9.05ms, Scan0 - 3.01ms, StructSet - 92.63ms, GDI - 0.11ms, Draw - 0.51ms, Total - 105.37ms
SS - 6.87ms, Scan0 - 2.93ms, StructSet - 91.61ms, GDI - 0.13ms, Draw - 0.68ms, Total - 102.27ms
SS - 7.54ms, Scan0 - 2.37ms, StructSet - 91.43ms, GDI - 0.19ms, Draw - 0.86ms, Total - 102.46ms
SS - 4.73ms, Scan0 - 2.42ms, StructSet - 90.61ms, GDI - 0.14ms, Draw - 0.43ms, Total - 98.38ms
SS - 9.11ms, Scan0 - 2.99ms, StructSet - 90.44ms, GDI - 0.11ms, Draw - 1ms, Total - 103.72ms
SS - 9.36ms, Scan0 - 3ms, StructSet - 108.1ms, GDI - 0.23ms, Draw - 0.6ms, Total - 121.35ms
SS - 3.39ms, Scan0 - 2.82ms, StructSet - 95.23ms, GDI - 0.14ms, Draw - 0.67ms, Total - 102.31ms
SS - 5.15ms, Scan0 - 3.28ms, StructSet - 103.75ms, GDI - 0.37ms, Draw - 1.25ms, Total - 113.88ms
SS - 6.85ms, Scan0 - 3.07ms, StructSet - 92.75ms, GDI - 0.18ms, Draw - 0.54ms, Total - 103.44ms

 

The reason for using C# to take the screenshot is that I plan on doing other actions in C# to select only parts of the screen that I actually need to further improve the performance and remove 'noise' from the screenshotted area by only searching for/selecting relevant areas.

[markyrocks]

ok I was wrong I know that i've managed to pull out a whole array at once as a binary string and using that string as an array of sorts.  I forgot how limited autoit is when it comes to this kinda stuff.  That being said any kinda work around would probably just take longer than actually just pushing the data to the struct.  Have you looked into just grabbing the capured image from the clipboard or making whatever modifications to the capture in c# and just putting the modified capture back in the clipboard and using the clipboard as a buffer between the 2 processes? Is the c# a process or a dll? Whatever it is that you have going on theres got to be a better way than returning a bytearray.  The speed is in the c# so that process should be quarterbacking and basically sending the autoit script the minimal amount of information possible for it to do whatever it is that you think you need it for.  Either that or bare minimum return a handle to the object.  

Edited October 15, 2021 by markyrocks

[mistersquirrle]

7 hours ago, markyrocks said:

Have you looked into just grabbing the capured image from the clipboard or making whatever modifications to the capture in c# and just putting the modified capture back in the clipboard and using the clipboard as a buffer between the 2 processes? Is the c# a process or a dll? Whatever it is that you have going on theres got to be a better way than returning a bytearray.

Indeed, and I have looked into a few things, since there's several ways to create the Bitmap, _GDIPlus_BitmapCreateFrom*

I had definitely looked at the options like *FromMemory, *FromResource, *FromScan0 (which I was using), *FromStream. Part of the problem though for me with those options is that I didn't understand how to use any of them, besides Scan0. I also didn't want to use *FromFile because I wanted to avoid read/writes since I wanted this to run at a high refresh rate and I'm not sure how that'd wear a drive (or maybe there's even a way around it).

 

However your comment on sending it to the clipboard did make me go back and take a look at it, and I think that we have a winner: https://www.autoitscript.com/autoit3/docs/libfunctions/_GDIPlus_BitmapCreateFromHBITMAP.htm

 

Using that with: 

1. https://docs.microsoft.com/en-us/dotnet/api/system.drawing.bitmap.-ctor?view=windowsdesktop-5.0#System_Drawing_Bitmap__ctor_System_Int32_System_Int32_System_Int32_System_Drawing_Imaging_PixelFormat_System_IntPtr_
2. https://docs.microsoft.com/en-us/dotnet/api/system.drawing.bitmap.gethbitmap?view=windowsdesktop-5.0

I am able in C# to create the bitmap using the ImgStats and Scan0 that I already have and return the IntPtr back to AutoIt, and it looks like that AutoIt likes it!

With using the HBitmap from C# directly the speed has vastly improved, so even if you didn't do anything directly, thanks for making me take another look at other options.

Log using the *FromScan0 function:

SS - 20.44ms, Scan0 - 17.35ms, StructSet - 608.05ms, GDI - 0.17ms, Draw - 3.05ms, Total - 649.13ms
SS - 19.58ms, Scan0 - 12.79ms, StructSet - 599.42ms, GDI - 0.26ms, Draw - 3.39ms, Total - 635.5ms
SS - 9.23ms, Scan0 - 16.72ms, StructSet - 608.09ms, GDI - 0.24ms, Draw - 2.89ms, Total - 637.24ms
SS - 13.15ms, Scan0 - 17.6ms, StructSet - 610.59ms, GDI - 0.22ms, Draw - 3.97ms, Total - 645.61ms
SS - 10.45ms, Scan0 - 17.92ms, StructSet - 607.26ms, GDI - 0.35ms, Draw - 4.11ms, Total - 640.19ms
SS - 8.82ms, Scan0 - 15.78ms, StructSet - 604.57ms, GDI - 0.22ms, Draw - 2.92ms, Total - 632.39ms
SS - 19.53ms, Scan0 - 16.65ms, StructSet - 616.01ms, GDI - 0.34ms, Draw - 3.81ms, Total - 656.43ms
SS - 21.7ms, Scan0 - 19.62ms, StructSet - 626.3ms, GDI - 0.17ms, Draw - 1.89ms, Total - 669.75ms

Log using the *FromHBITMAP function:

SS - 13.94ms, Scan0 - 2.2ms, StructSet - 0ms, GDI - 0.76ms, Draw - 4.66ms, Total - 21.62ms
SS - 17.52ms, Scan0 - 1.13ms, StructSet - 0ms, GDI - 0.4ms, Draw - 2.4ms, Total - 21.49ms
SS - 11.27ms, Scan0 - 2ms, StructSet - 0ms, GDI - 0.6ms, Draw - 5.06ms, Total - 19.01ms
SS - 10.3ms, Scan0 - 2.11ms, StructSet - 0.01ms, GDI - 1.26ms, Draw - 4.7ms, Total - 18.47ms
SS - 8.85ms, Scan0 - 1.75ms, StructSet - 0ms, GDI - 0.61ms, Draw - 2.7ms, Total - 13.97ms
SS - 11.85ms, Scan0 - 1.86ms, StructSet - 0ms, GDI - 1.06ms, Draw - 6.91ms, Total - 21.78ms
SS - 9.5ms, Scan0 - 2.6ms, StructSet - 0ms, GDI - 1.38ms, Draw - 5.92ms, Total - 19.46ms
SS - 19.34ms, Scan0 - 3.68ms, StructSet - 0.01ms, GDI - 1.7ms, Draw - 4.56ms, Total - 29.39ms

 

Massive change when testing a 500x500 area, from ~640ms --> ~20ms.

I'll leave this unsolved for a little bit incase anyone does want to come in with any suggestions for speeding up DllStructSetData as originally requested, but as far as want I wanted to do, this is basically solved.

Edited October 16, 2021 by mistersquirrle
Missing some words

[markyrocks]

It looks like I did help you bc to use the create bitmap from hbitmap you need a handle to a bitmap.  Which was one of my suggestions.   But I'm glad you figured it out.  

[junkew]

• Lowering the colordepth helps a lot
• fasm probably the fastest you can get (then C/C++ followed by C# probably as thats both compiled code)

 

[mistersquirrle]

I'm not necessarily looking to improve image/pixel searching performance, that I'm not doing with AutoIt, I'm doing that in C#. So C# is taking the screenshot, searching for what I want, and then returning the region/image/pixel data to AutoIt so I can use GDI+ to redraw the image/area in an overlay GUI on the screen (taking something happening on another screen that I'm not looking at and put important information where I am looking, or near my mouse).

So for example: 

In this screenshot I'm not doing any pixel/image searching, but this is how I was testing just the performance of it. Taking a screenshot in C#, sending it back to AutoIt and displaying it with GDI+. Originally I was passing the pixel data base as a Byte array, but GDI+ using _GDIPlus_BitmapCreateFromScan0() requires that data to be in DllStruct (from what I gathered). So martyrocks helped me realize that I could do it much better using _GDIPlus_BitmapCreateFromHBITMAP() instead.

 

So I'm not sure that those examples really apply here, but they are good examples of finding an image (unless I'm missing something else in them).

[junkew]

The examples are showing how to get to the different pixels of your bitmaps.

@LarsJyou probably can advice on this one 😉 

Edited October 18, 2021 by junkew

[LarsJ]

See this example for more information on the technique used by mistersquirrle.