Simple struct data copying

[weirddave]

If I have 2 simple identical structures and I want to copy the data in one to data in the other, what's the best way to do it? I want to keep the same memory in use when I do it, If I just do $b = $a, the pointer to $b changes, presumably because a new structure has been created

For example:

$a = DllStructCreate(String("struct;byte myData[100];endstruct"))
$b = DllStructCreate(String("struct;byte myData[100];endstruct"))
DllStructSetData($a, "myData",1,100)
DllStructSetData($b, "myData",2,100)
msgbox(0,DllStructGetData($a,"myData",100),DllStructGetData($b,"myData",100))
$t= DllStructGetPtr($b)
$b=$a
msgbox(0,DllStructGetData($a,"myData",100),DllStructGetData($b,"myData",100))
msgbox(0,$t,DllStructGetPtr($b))

The message boxed show the data in the last byte of both structs before and after the copy, followed by the change in pointer for $b
Is the correct procedure for the above example:
 

DllStructSetData($b, "myData", DllStructGetData($a, "myData"))

It appears to work (replacing $b=$a with the line above), but is this the best/correct method?

[jchd]

The last code box is correct, but the first isn't (no item in structure has item index 100) and can be simplified.

The _VarDump function is a little big to paste here (I've already posted it several times on the forum), but its output shows that things work as expected:

$a = DllStructCreate("byte myData[100]")
For $i = 1 To 100
    DllStructSetData($a, "myData", $i, $i)
Next
msgbox(0, "", DllStructGetData($a,"myData"))
ConsoleWrite(_vardump($a) & @LF)
$b = DllStructCreate("byte myData[100]")
DllStructSetData($b, "myData", DllStructGetData($a, "myData"))
msgbox(0, "", DllStructGetData($b,"myData"))
ConsoleWrite(_vardump($b) & @LF)

Console ouput of the above:

Struct       (100) @:02F65CA8 (structure alignment is unknown)
      byte[100]    0x0102030405060708090A0B0C0D0E0F10 ... 55565758595A5B5C5D5E5F6061626364
Struct       (100) @:02F65D88 (structure alignment is unknown)
      byte[100]    0x0102030405060708090A0B0C0D0E0F10 ... 55565758595A5B5C5D5E5F6061626364

 

[weirddave]

48 minutes ago, jchd said:

The last code box is correct, but the first isn't (no item in structure has item index 100) and can be simplified.

Not sure what you mean? "myData" has a range of 100 and is 1 based, so the last index is 100 isn't it? Or have I missed something?

[jchd]

Ah, sorry for my confusion, I never use field names, so I was confusing 2nd and 3rd parameters.

[AndyG]

Hi,

no, sorry, but that doesn´t work this way! You can´t do a "memcopy" with $b=$a, the only thing you do is to "copy" the struct to it´s own position! Aka, assign the structure to an other variable:

expandcollapse popup

$a = DllStructCreate(String("struct;byte myData[100000000];endstruct"))
$b = DllStructCreate(String("struct;byte myData[100];endstruct"))
$c = DllStructCreate(String("struct;byte myData[100];endstruct"))

$a_ptr = DllStructGetPtr($a)                                 ;a
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $a_ptr = ' & $a_ptr & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console
$a_size = DllStructGetSize($a)
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $a_size = ' & $a_size & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console

$b_ptr = DllStructGetPtr($b)
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $b_ptr = ' & $b_ptr & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console
$b_size = DllStructGetSize($b)
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $b_size = ' & $b_size & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console

$c_ptr = DllStructGetPtr($c)
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $c_ptr = ' & $c_ptr & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console
$c_size = DllStructGetSize($c)
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $c_size = ' & $c_size & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console


DllStructSetData($a, 1, 10, 1)                               ;set 10 at position 1
DllStructSetData($b, 1, 20, 1)                               ;set 20 at position 1

$b = $a                                                      ;memcopy? no, NOT a memcopy!

$b_ptr_after_copy = DllStructGetPtr($b)                      ;same ptr as $a-struct!
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $b_ptr_after_copy = ' & $b_ptr_after_copy & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console
$b_size_after_copy = DllStructGetSize($b)
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $b_size_after_copy = ' & $b_size_after_copy & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console


$is_b_10 = DllStructGetData($b, 1, 1)                        ;show the value at position 1
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $is_b_10 = ' & $is_b_10 & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console


;now it gets funny! the variable $b_ptr isn´t changed!!!!!!!
;is the previous structure $b deleted? or not? :o)
;if the memory at this position will be overwritten in the meantime, the script will crash here!

ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $b_ptr = ' & $b_ptr & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console
$t = DllStructCreate("byte [100]", $b_ptr)                   ;look at the "previous" $b-Struct
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $t = ' & $t & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console

$ret = "0x"
For $i = 1 To 100                                            ;show all bytes
    $ret &= Hex(DllStructGetData($t, 1, $i), 2)              ;tadaaaa
Next
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $ret = ' & $ret & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console

$ret = DllStructGetData($t, 1)                               ;show all bytes doesn´t work?!
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $ret = ' & $ret & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console

 

 

 

Edited December 19, 2016 by AndyG

[weirddave]

I think I understand that up until the re-creation of $b (as $t), why do the results from line 48 and 51 differ?

[AndyG]

3 hours ago, weirddave said:

why do the results from line 48 and 51 differ?

hmmm, exactly that´s the question....i don´t know! Possibly a cause of AutoIt´s internal access to the data... 

[LarsJ]

You could add

$d = $b

immediately before

$b = $a

 

[UEZ]

You can use memcpy to copy the struct block to another struct:

 

$a = DllStructCreate("struct;byte myData_a[100];endstruct")
$b = DllStructCreate("struct;byte myData_b[100];endstruct")
$a.myData_a((50)) = 100
$b.myData_b((50)) = 200


ConsoleWrite("Before: " & $b.myData_b((50)) & @CRLF)

$aRet = DllCall("MSVCRT.DLL", "ptr:cdecl", "memcpy", "struct*", $b, "struct*", $a, "uint", DllStructGetSize($b))

ConsoleWrite("After: "& $b.myData_b((50)) & @CRLF)

ConsoleWrite("Whole struct: " & $b.myData_b & @CRLF)

 

Edited December 20, 2016 by UEZ

[AndyG]

@LarsJ, yes of course you can "save" the struct respectively the pointer/names several times...

But this doesn´t answers the question (of weirddave) why dllstructgetdata() returns different values when reading a single byte(s) or reading the whole (byte)-content?! 

 

 

[Tekk]

Only a guess but I'm going to assume garbage collection.

$b = $a

After that assignment the memory that $b previously referenced should be given back to the heap, I doubt AutoIt cares that you still have a pointer to that memory.

... just a guess

[LarsJ]

AndyG, Yes it does. When you use structures and pointers to structures it's your responsibility to make sure that the pointers are valid. This means that the pointers in fact points to valid structures. When you set $b = $a these 100 bytes that previously was allocated to $b, are not allocated to any AutoIt variable any more and are not maintained by internal AutoIt code any more. In other words the 100 bytes is unused free memory that just contains random data. When you print out random data through $b_ptr, you get random data. If you print out random data once more it's still random data. Why should the data be the same as when you printed them out the first time? The 100 bytes are not treated as an AutoIt structure any more. They can be used for any purpose and changed at any time.

If you set $d = $b before $b = $a the 100 bytes are allocated to $d and $b_ptr does still point to a valid AutoIt structure.

[AndyG]

19 hours ago, LarsJ said:

If you set $d = $b before $b = $a the 100 bytes are allocated to $d and $b_ptr does still point to a valid AutoIt structure.

Yes, i know.

But...the question was, why do we get different values reading the memory "byte after byte" via dllstructgetdata($struct,1,$i) than reading an area/region via dllstructgetdata($struct,1)?!

I agree with you, probably the content of the (in the meanwhile allocated by any other program) memory changed after our $b=$a (as you mentioned above).

If i "point" a new struct to that memory using a pointer with dllstructcreate(), there should be no difference reading "byte by byte" or reading a 100 bytes segment. If the memory is protected in any case, dllstructcreate()/dllstructgetdata() should throw an error. There is no error thrown, the results are different.

I refuse to believe that AutoIt is "confused" by the $b=$a  but if you comment that line of code, everything works. The other thing is, whats going on with those 100 bytes of memory between the line $b=$a and the following dllstructcreate(). Which program could reserve/allocate exactly this little piece of memory?

$a = DllStructCreate(String("struct;byte myData[100];endstruct"))
$b = DllStructCreate(String("struct;byte myData[100];endstruct"))

$a_ptr = DllStructGetPtr($a)                                 ;a
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $a_ptr = ' & $a_ptr & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console
$a_size = DllStructGetSize($a)
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $a_size = ' & $a_size & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console

$b_ptr = DllStructGetPtr($b)
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $b_ptr = ' & $b_ptr & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console
$b_size = DllStructGetSize($b)
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $b_size = ' & $b_size & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console

DllStructSetData($a, 1, 10, 1)                               ;set 10 at position 1
DllStructSetData($b, 1, 20, 1)                               ;set 20 at position 1

$b = $a                                                      ;so far, so good


;creating a struct at the "old" $b-struct
$t = DllStructCreate("byte [100]", $b_ptr)                   ;look at the "previous" $b-Struct
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $t = ' & $t & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console

;reading byte by byte
$ret = "0x"
For $i = 1 To 100                                            ;show all bytes
    $ret &= Hex(DllStructGetData($t, 1, $i), 2)              ;tadaaaa
Next
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $ret = ' & $ret & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console

;reading 100 bytes
$ret = DllStructGetData($t, 1)                               ;show all bytes doesn´t work?!
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $ret = ' & $ret & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console

 

[LarsJ]

AutoIt is not confused. You are confused.

From the help file:
DllStructCreate( Struct [, Pointer] )
Pointer [optional]: If supplied the struct will not allocate memory but use the pointer supplied.

The pointer you are using below the line $b = $a is invalid. It's not pointing to an AutoIt structure. It's pointing to random data.

When you create a struct that does not allocate memory from a pointer that points to random data all you get is random data. You can repeat this as many times as you want. Every time you'll get nothing else but random data.

There is nothing wrong in the way that AutoIt handles structures. You have not understood how it works.

Edited December 21, 2016 by LarsJ

[weirddave]

It would seem that below $b=$a, the system is free to use data in the location of the old $b struct. I should think that recreating the struct with the old $b pointer is a dodgy thing to do, since that memory may be under another piece of software's control, or does the memory management take care of this and move the 'real' pointer to another area of memory? (I believe memory pointers are virtual and only applicable to the PID using it, I'm using a shared memory DLL, 2 scripts are sharing data but the pointers are different in each script).

[LarsJ]

There is no such memory management.

[weirddave]

Well, there's something sitting between the autoit pointers and the real memory location, otherwise my shared memory pointers would be the same in both scripts.
I think this explains it:
https://www.quora.com/Do-pointers-in-C-C++-contain-actual-memory-address

" Virtual means that the processor helds a “translation table” for pages (usually 4K each). So whenever you access one page, you get “translated” to another. "

I believe this is part of the memory management in the CPU.

[AndyG]

21 hours ago, weirddave said:

otherwise my shared memory pointers would be the same in both scripts.

Show your scripts, the call of a dll has nothing to to with TLB.

 

Compile this script, start it several times. You will see a "counter" running, each for its own process/window. You can stop the counter while holding the upper frame/border of the corresponding window. After release, the counter continues. After a click of ONE of the "Reset/sync"-Button, the counter of ALL Processes/windows will reset...

sharedmemdll.zip

[weirddave]

I have modified the code from your zip file to show the pointer in a new label in the GUI, they are different for each thread, which makes sense
I didn't know it was called TLB, ta for that. Of course, none of this explains what actually happens when recreating the struct using the old pointer. My best guess is that a new piece of real memory is allocated and mapped to the pointer, but I don't know a way to test this theory. It would be nice to know what's really happening either way, because it doesn't crash

expandcollapse popup

#Region                                      ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_UseUpx=n
#EndRegion                                     ;**** Directives created by AutoIt3Wrapper_GUI ****

;#AutoIt3Wrapper_UseUpx=n
#AutoIt3Wrapper_usex64=n

$x = Random(1, @DesktopWidth - 500)          ;zufällige Position der GUI
$y = Random(1, @DesktopHeight - 150)

$gui = GUICreate("Shared Mem Test, please compile and start this program several times!", 500, 150, $x, $y);GUI erstellen
$label = GUICtrlCreateLabel("", 20, 20, 100, 30) ;Label
$label2 = GUICtrlCreateLabel("", 20, 50, 100, 30) ;Label
$button = GUICtrlCreateButton("Reset/Sync", 20, 100);Button
GUISetState()                                ;GUI aktivieren

$dll = DllOpen("sharedmem.dll")              ;dll laden
$ptr = DllCall($dll, "dword", "GetPointer")  ;den Pointer zum shared memory zurückgeben

$struct = DllStructCreate("char[8];int", $ptr[0]);Struct an dieser Speicherposition erstellen, string und reset-flag
DllStructSetData($struct, 1, "AutoIt")       ;Daten schreiben
$i = 0                                       ;zähler, welcher in allen GUI´s GLEICHZEITIG resettet wird

GUICtrlSetData($label2, $ptr[0])
$t = TimerInit()
While 1                                      ;endlosschleife
    $msg = GUIGetMsg()                       ;event?
    If $msg = -3 Then ExitLoop               ;wenn GUI geschlossen, dann ende
    If $msg = $button Then                   ;wenn button, dann resetten
        DllStructSetData($struct, 2, 1)      ;reset-flag setzen
        DllStructSetData($struct, 1, "RESET") ;Text setzen

    EndIf
    If TimerDiff($t) > 100 Then              ;alle 100 Millisekunden die Structinhalte anzeigen
        $t = TimerInit()                     ;Timer starten
        If DllStructGetData($struct, 2) = 1 Then ;wenn in irgendeiner GUI der Resetbutton gedrückt wurde
            $i = 0                           ;zähler nullen
            GUICtrlSetData($label, DllStructGetData($struct, 1) & "  " & DllStructGetData($struct, 2) & "   " & $i);Structinhalte anzeigen
            Sleep(1000)                      ; reset-flag zeigen
            DllStructSetData($struct, 2, 0)  ;reset-flag zurücksetzen
            DllStructSetData($struct, 1, "AutoIt") ;Text zurücksetzen
        EndIf
        $i = $i + 1                          ;zahler erhöhen
        GUICtrlSetData($label, DllStructGetData($struct, 1) & "  " & DllStructGetData($struct, 2) & "   " & $i);Structinhalte anzeigen
    EndIf
WEnd