passing a pointer to a child process via command line

[AndyG]

16 hours ago, weirddave said:

What would make the DLL better (in my humble opinion) would be to have an 'Init (bytes)' function which actually allocates a piece of memory for sharing, that way you can ask for the amount you actually need

I tested that, but for some reason it doesn´t work.

16 hours ago, weirddave said:

(or make autoit create the asm and compile it I guess

The best solution i got was to create some DLL´s with several sizes, 1KB, 10KB, 100KB, 1MB and so on. 

Once these DLL´s are created, you can choose the one with the right size. But talking straight, WHAT IS THE REASON NOT TO USE ONLY A 100MB DLL?

100MB should be enough for any application....and such a file is easily compressible and can be (base64) compressed into an AutoIt-Function.

I think, this is a little job this evening

 

 

[weirddave]

One of the problems with my project is that there will sometimes be a need to run 2 copies of the parent/child process's, I could either have 2 Dll's or perhaps 2 'GetPointer' functions in the same dll.

I quite like the idea of generating the .asm text file from autoit and having that compile the dll with a name unique to the parent/child pair (PID in the name for example). creating the .asm could be from a template file which contains the dll code with this change:

_mem db [{numbytes}] dup 0

Then something like:

#include <File.au3>
$DLLname = "sharedmemDLL" & @AutoItPID & ".asm"
FileCopy ( "sharedmemDLLTemplate.asm", $DLLname )
$numSharedBytes = 1000
$sFilePath = $DLLname
_ReplaceStringInFile ( $sFilePath, "[{numbytes}]", $numSharedBytes)
;Now we compile it the .asm using some other commands

What do you think?  

[AndyG]

6 hours ago, weirddave said:

 

One of the problems with my project is that there will sometimes be a need to run 2 copies of the parent/child process's, I could either have 2 Dll's or perhaps 2 'GetPointer' functions in the same dll.

 

you could have 30 copies of your parent/child processes, or maybe 100... that doesn´t matter...

Let´s say, you need 1MB of Memory for some application A and B, 5MB of Memory for C and D, and 12MB for E and F.

Let´s imagine, you know what you are doing....

The sharedmem_100MB.DLL gives every program the same pointer, i.e. 0xDEADBEEF, where the 100MB of available memory starts.

YOU (remember, knowing what to do! ) have 3 pairs of programs AB, CD and EF. 

AB needs 1MB of shared memory. In AutoIt in the AB-Scripts, you declare a $struct=Dllstructcreate("byte [1000000]",0xDEADBEEF). At the position of 0xDEADBEEF, you work with the next 1000000bytes....

CD needs 5MB of shared memory. In AutoIt in the CD-Scripts, you declare a $struct=Dllstructcreate("byte [5000000]",0xDEADBEEF+1000000). The first 1MB is used by AB. At the position of 0xDEADBEEF+1000000, you work with the next 5000000bytes....

EF needs 12MB of shared memory. In AutoIt in the AB-Scripts, you declare a $struct=Dllstructcreate("byte [12000000]",0xDEADBEEF+6000000). The first 6MB(5+1) are used by AB and CD. At the position of 0xDEADBEEF+6000000, you work with the next 12000000bytes....

You can share the whole memory with x programs....either all can use the memory at the same position, or you can divide the memory in pieces/parts, every part  of them used by one (or more) program(s).

 

Once the DLL ( this is a short name of DYNAMIC LINK LIBRARY!) is loaded into memory the first time, every other program which opens this DLL (DllOpen() ) will be "linked"  to that DLL. The DLL will NOT BE LOADED AGAIN! This is the meaning/sense of a DLL. After loading into memory (only one time!!!) , all other programs can use the provided functions....and so you can do it also

 

//EDIT:

You could say, "Hey, this kind of "Memory sharing" is a really big security problem! Because every other program can easily "look" into the shared memory!"

And you are absolutely right! This is the reason why a DLL is not able to allocate "dynamically" memory....and when you ask a question about this topic in a Microsoft forum, very soon a M$-MVP arrives and states/claims "DOES NOT WORK!". But it works...nevertheless...

 

Edited October 4, 2016 by AndyG

[weirddave]

I can do it that way, but if they each create their own dll then I'm only using the memory I need.
The pairs of programs are AB, AB, AB, it's the same code in each pair. I could manage a single block of memory but it's more prone to me making a mistake  

Tomorrow I will have a go at generating the DLL automatically (really appreciate the .asm you shared)

[trancexx]

Ok, may I chime in?
Great, thanks!

So, what AndyG said about DLLs is not right. Process (or "program" if you like) is given memory space when it's run. Dll is loaded per process by mapping it into its memory space. Every process that loads the same dll, loads it only for itself. The code inside the dll then becomes the code of that process.
That's how dll can be loaded by any number of processes without any interaction whatsoever.

[weirddave]

Sure, you can chime in. Oh.
If that were true, the memory sharing wouldn't work, but it does.

[weirddave]

OK, yes, it maps it to the process memory space, but it doesn't actually load into another memory space for each load of the dll.

https://msdn.microsoft.com/en-us/library/windows/desktop/ms681914(v=vs.85).aspx
 

http://stackoverflow.com/questions/2846310/are-dll-files-loaded-once-for-every-program-or-once-for-all-programs

I think this is why memory reservation works but memory allocation won't.

[trancexx]

16 minutes ago, weirddave said:

OK, yes, it maps it to the process memory space, but it doesn't actually load into another memory space for each load of the dll.

https://msdn.microsoft.com/en-us/library/windows/desktop/ms681914(v=vs.85).aspx
 

http://stackoverflow.com/questions/2846310/are-dll-files-loaded-once-for-every-program-or-once-for-all-programs

I think this is why memory reservation works but memory allocation won't.

I'm not sure what do you mean by that.

Each time dll is loaded by another process, it's loaded only in the memory space of that process. Repetitive loading by the same process has no effect, except for incrementing reference count.

And that stackoverflow answer is wrong too.

[weirddave]

On the MS page:
"
Every process that loads the DLL maps it into its virtual address space. After the process loads the DLL into its virtual address, it can call the exported DLL functions.

The system maintains a per-process reference count for each DLL. When a thread loads the DLL, the reference count is incremented by one. 

"

It's loaded for real only once, it's mapped to each process memory as required. Can you show documentation to why the stackoverflow answer is incorrect, because evidence seems to suggest that it's right. If it is indeed wrong, by what method does the shared memory work?

[trancexx]

MS page says it. You're just not understanding what it says.

"Load" means "map" and "initialize". "Map" means read and copy binary data from the file on the disk (dll) and write it into processes virtual space (create image). "Initialize" means rewrite fixed pointers to match given address at which the image is written, to load possible dependencies and to call entry point function.

 

[Tekk]

https://msdn.microsoft.com/en-us/library/h90dkhs0(v=vs.90).aspx

The dll is taking advantage of IMAGE_SCN_MEM_SHARED.  Each process that loads the dll gets its own copy, that section is simply coherent across all processes that utilize the dll.  It works much the same way as memory mapped files.
 

section '.data' data readable writeable shareable

 

[weirddave]

https://msdn.microsoft.com/en-us/library/windows/desktop/ms681938(v=vs.85).aspx

"Multiple processes that load the same DLL at the same base address share a single copy of the DLL in physical memory. Doing this saves system memory and reduces swapping."
It's loaded once, is it not?

[trancexx]

8 hours ago, weirddave said:

https://msdn.microsoft.com/en-us/library/windows/desktop/ms681938(v=vs.85).aspx

"Multiple processes that load the same DLL at the same base address share a single copy of the DLL in physical memory. Doing this saves system memory and reduces swapping."
It's loaded once, is it not?

No, it's not. It's loaded for each process. Physical memory is one thing and virtual another. Yo have some insights at https://msdn.microsoft.com/en-us/library/windows/desktop/aa366785(v=vs.85).aspx

[weirddave]

From the link you kindly provided:
"These virtual memory pages are mapped to the same physical memory pages for both processes "

Which makes sense, you don't want to actually load it each time in real memory, isn't that the point of DLLs in the first place?
So, since everything in the DLL is directly visible to any process that 'loads' it, reserved memory is visible to all, and it's the same (because it's the same physical memory).
It's all about this bit:

section '.data' data readable writeable shareable

_mem db 1000 dup 0 ;this is shared memory in the DLL

 

Edited October 5, 2016 by weirddave
added the 'section' to the code snippet

[weirddave]

I think I've been too zoomed in on this particular DLL. In the wider world, if a process writes to some DLL memory that isn't explicitly shared then a copy of it is created for that process. I suspect this is a way to force naughty non re-entrant DLL code to work properly.

[weirddave]

I had to shelve this project for a while and have finally got the time to come back to it.

I'm having a bit of trouble understanding how to get a couple of structures into the shared memory. Is it possible to create 2 independent structures in the shared memory by specifying an offset?

When I do the call to get the shared memory pointer, is there a way to print the pointer to prove I've actually got a sensible answer? IsPtr($ptr) suggests it's not getting a valid pointer.

[AndyG]

4 hours ago, weirddave said:

I'm having a bit of trouble understanding how to get a couple of structures into the shared memory. Is it possible to create 2 independent structures in the shared memory by specifying an offset?

Yes. As mentioned above, you can "structure" your shared memory with structs as you need it. And place those structs into the memory where ever you like it.

Let's say your shared memory size is 1.000.000 bytes and your _independant_struct1 is 8.000bytes and _independant_struct2 is 30.000bytes.

You can determine/set the position of the independant structures while creating the struct(s). The "basepointer" is the pointer returned from the dll.

dllstructcreate($_independant_struct1, $basepointer+$offset_what_you_like_eg_60000_bytes) creates a struct into the shared memory with a size of 8000bytes at the position "offset" 60.000 (within the shared memory)

dllstructcreate($_independant_struct2, $basepointer+$offset_what_you_like_eg_110000_bytes) creates a struct into the shared memory with a size of 8000bytes at the position "offset" 110.000 (within the shared memory).

 

All you have to do is to care about the size of the structs so that they NOT overlap (except you want this for some reason!)

With dllstructcreate(struct,explicit_ptr) you NOT reserve memory, you only have access to the memory at the position "explicit_ptr"!

 

An example how to "overlap" structures to read/get results not obviously expected

;create struct
$struct =dllstructcreate("int;float[4];dword")

;fill struct with data
dllstructsetdata($struct,1,123456);int
dllstructsetdata($struct,2,4.80114160043301e+030,1);float
dllstructsetdata($struct,2,3.68584191575527e+024,2);float
dllstructsetdata($struct,2,7.71403089381636e+031,3);float
dllstructsetdata($struct,2,8.24605444209785e-019,4);float
dllstructsetdata($struct,3,0xDEADBEEF);dword

;read some data from struct at offset 4
$struct2=dllstructcreate("char[16]",dllstructgetptr($struct)+4)
msgbox(0,":o)",dllstructgetdata($struct2,1))

 

 

[weirddave]

I'm struggling to get this to work, I get the pointer to the shared memory then try to create 2 structures which fit in the memory space. 

$dll = DllOpen("sharedmem.dll")
if $dll= -1 then msgbox(0,"fail","no open dll")
$ptr = DllCall($dll, "dword", "GetPointer")
$Struct1 = DllStructCreate("byte[50]",$ptr+0)
if @error <> 0 then msgbox(0,"Struct Error", "Unable to create structure 1")
$Struct2 = DllStructCreate("byte[50]",$ptr+100)
if @error <> 0 then msgbox(0,"Struct Error", "Unable to create structure 2")

The DllStructCreate()'s both return errors. This is why I'd like to be able to see what the pointer is, to make sure I've actually got one (for testing and debugging purposes).

[AndyG]

Hi,

DllCall returns an array...

[weirddave]

After much strain I finally understand! I didn't realise DllCall always returns an array, thus $ptr[0] holds the value of the pointer. Phew  

Thanks for the help.