Which IPC is suitable for asynchronous messaging?

[Lion66]

I have a situation when the parent script launches 10-20 child scripts on the same computer, which send 2-3 short messages to the parent.
Messages from different scripts can come almost simultaneously.
It is important for me not to lose message and it is advisable not to change the order of messages from one script.

I chose MailSlot, but sometimes messages are lost.
What kind of UDF will you advise?
Thank you.

Edited March 8, 2023 by Lion66

[argumentum]

47 minutes ago, Lion66 said:

I chose MailSlot, but sometimes messages are lost.

Would you post code to prove that it happens ?. I use it and never had one lost.
Also, check my signature for other IPCs.

[mistersquirrle]

If you're just getting a couple of things back from the child, and not sending anything from the parent -> child (besides command line params), you could probably just get away with using StdoutRead: https://www.autoitscript.com/autoit3/docs/functions/StdoutRead.htm. When you use Run use $STDERR_MERGED, save the PID that's returned, and loop through all the children to read their output (from ConsoleWrite) to get their status or maybe results.

Otherwise like post some sample code so we can see what you're trying to accomplish really.

And check out the Wiki for some IPC options: https://www.autoitscript.com/wiki/User_Defined_Functions#Inter_Process_Communications

Personally I like: 

And about IPC specifically: 

 

 

Edited March 8, 2023 by mistersquirrle

[Nine]

Since everything is on the same computer, you could use my WCD IPC (see my signature).  It is very simple to use and hardly can be faster since it is based on Windows Messaging System.

If you have any problem with it, I will be glad to help you out.

[Lion66]

Thanks everyone for the responses.

I got an idea of what's relevant today.

argumentum

Yours "to prove" confused me a little, but I understand what you mean.

I saw a message in the table (pipetable.gif) about MailSlot:

"Message can get lost between sender and receiver".

Based on the my attempts and this information, I decided to ask the advice of experienced coders.

Now I found that MailSlot no response empty messages. This may explain the rare lost messages (in my case).

mistersquirrle

Each child script executes cmd.
As a result of StdoutRead, I will see much more than I would like.

Or I won't be able to multiple messages as the commands run.

I looked at IPCviaROTobjects and not found an example of receive several messages.

This is probably a powerful data transfer library, but my examples show that queues are not supported and message is lost, although I may not be using it correctly.

500ms delay is set on purpose and simulates message processing time.

Receiver:

Local $Message = "", $OldMessage = ""
While 1
    IPCROT_Receiver("Example", $Message)
    If $Message <> $OldMessage Then
        ConsoleWrite($Message & @CRLF)
        $OldMessage = $Message
        Sleep(500)
    EndIf
WEnd

Sender:

For $i = 0 to 9
    IPCROT_Sender( "Example", "Client1. Msg " & $i )
    Sleep(200)
Next

 

Nine

I liked your UDF:
-maintains a queue
-responds empty messages
-ready-made example of receiving multiple messages
-all the "excess" is hidden inside the UDF.

 If I am going to redo my program, then will use it.
And also in the future.
If I will have any questions, I'll ask in the UDF topic.

Edited March 9, 2023 by Lion66

[mistersquirrle]

Something to keep in mind (and I can't answer this about anything mentioned so far in this topic) is that when you have multiple writers/children, you may likely be overwriting data and "losing" messages. This is one good thing with just StdoutRead, you may get a lot more than you want (because of logging), but you can just adjust your children to put the "important" data/messages in a "container", like: [!This is an important message], and then parse it with something like _StringBetween. StdoutRead separates the messages by the PID of each child, so they can't overwrite each others data.

Nine would have to answer it about his UDF, but with most IPC that writes information to the same 'queue', you may get an instance where 2 are writing at the same time, one completes after the first and overwrites the data from the first one because it didn't exist when it started writing. Does that make sense? AutoIt is not thread safe, some IPC methods or UDFs may be, but that could also be why you're "losing" messages. You may want to look into something like: 

I've used this before when I had children scripts (around 8 ) all writing loglines to a file. This cause a lot of lost data and data combined into a single line as they were all competing to write data at the same place in the file. I used that UDF to have them wait until an existing writer was done before they did theirs. I then also prefixed the log lines with the PID of the child, so when they're all writing at the same time I can filter/search by a specific PID to see just its lines.

So I imagine that you can use this semaphore UDF (or just the AutoIt semaphore) to make sure that your children aren't putting data into whatever IPC you use. Also in terms of queues or stacks, I recommend checking out these two UDFs as well:

 

 

 

[Lion66]

1. I plan to cancel the write in the log (library from Nine) and use messages on the fly (using the queue).
Should I think about semaphore also when using only memory (or what is used there for Windows Messages ) ?

2. The library from Nine provides for a response to the received message.
This can be used so that the children script finds out that the message has been delivered.
Otherwise, send again.  A little redundant, but great reliability.

3. I am still interested to understand about StdoutRead. I use it inside a children script, but I do not see how it can be used in my parent.
I will show a very simplified example of a children script, which sends three messages to the parent. I want the parent to react (showing progress) after each message.
If in the parent script I will use StdoutRead, then I will have to run one script for each command (three scripts). Then I will prefer IPC. Correct me if I got lost.

#include <AutoItConstants.au3>
Local $iPID, $sOutput
$iPID = Run(@ComSpec & " /C DIR c:\test1", "", @SW_HIDE, $STDOUT_CHILD + $STDERR_MERGED)
ProcessWaitClose($iPID)
$sOutput = StdoutRead($iPID)
If StringInStr($sOutput, "Not found") > 0 Then
    SendMessageToParent("Fail") ; pseudo function
Else
    SendMessageToParent("Pass")
EndIf
Sleep(2000)     ; Imitation of processing time

$iPID = Run(@ComSpec & " /C ping 127.0.0.5", "", @SW_HIDE, $STDOUT_CHILD + $STDERR_MERGED)
ProcessWaitClose($iPID)
$sOutput = StdoutRead($iPID)
If StringInStr($sOutput, "100% loss") > 0 Then
    SendMessageToParent("Fail") ; pseudo function
Else
    SendMessageToParent("Pass")
EndIf
Sleep(2000)     ; Imitation of processing time

$iPID = Run(@ComSpec & " /C tree c:\test1", "", @SW_HIDE, $STDOUT_CHILD + $STDERR_MERGED)
ProcessWaitClose($iPID)
$sOutput = StdoutRead($iPID)
If StringInStr($sOutput, "Invalid path") > 0 Then
    SendMessageToParent("Fail") ; pseudo function
Else
    SendMessageToParent("Pass")
EndIf
Sleep(2000)     ; Imitation of processing time

Also, when I run children scripts through the FOR cycle, I need to somehow assign different output variable each iteration.
To do this, I probably have to do ObjCreate to use the iteration number, which greatly complicates the use.

 

Edited March 11, 2023 by Lion66

[mistersquirrle]

Here's an example of some basic child -> parent IPC with StdoutRead. This uses brackets [ ] to denote important lines that should be logged in the main script. You can use any type of delimiter/container characters and change it from just logging to doing some action.

Child process, make sure to compile:

#Region ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_UseUpx=y
#AutoIt3Wrapper_Change2CUI=y
#EndRegion ;**** Directives created by AutoIt3Wrapper_GUI ****

Global $iMax = 99
Global $iProgressPoints

If IsArray($cmdLine) And $cmdLine[0] > 0 Then
    $iMax = $cmdLine[1] ; We can set how many loops to do through the cmd line when we Run the child
EndIf

$iProgressPoints = Floor($iMax / 10) ; every 10%

ConsoleWrite('Logging every ' & $iProgressPoints & ', going to ' & $iMax & @CRLF)

For $iProgress = 0 To $iMax
;~  Sleep(Random(0, 1, 1)) ; Sleep min time is 10ms, so 1-9 = 10. This is basically just sleep 0 or sleep 10
    Sleep(Random(10, 100, 1)) ;
    ConsoleWrite($iProgress & @CRLF) ; 'Normal' log line
    If Mod($iProgress, $iProgressPoints) = 0 Then ; Log every certain percentage
        ConsoleWrite('[' & Round(($iProgress / $iMax) * 100, 0) & '% done, ' & $iProgress & '/' & $iMax & ']' & @CRLF) ; 'Important' log lines
    EndIf
Next

ConsoleWrite('[100% done, ' & $iProgress - 1 & '/' & $iMax & ', exiting]' & @CRLF) ; 'Important' log lines

Exit 0 ; exit code

 

And then the main/spawner that manages the children and logs important lines. This can be run from SciTE or compiled:

expandcollapse popup

#Region ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_UseUpx=y
#AutoIt3Wrapper_Change2CUI=y
#EndRegion ;**** Directives created by AutoIt3Wrapper_GUI ****

#include <AutoItConstants.au3>
#include <String.au3>

Global $iSpawnChildren = 3 ; Control how many children processes to spawn. Want to crash your computer? Run 50 :)
Global Enum $CHILD_PID, $CHILD_MSG, $CHILD_LASTMSG, $CHILD_MAX ; Just so we don't have to remember indexes
Global $aChildren[$iSpawnChildren][$CHILD_MAX] ; Array to hold our child information
Global $iSpawnMin = 99, $iSpawnMax = 199 ; How many loops each child should run

OnAutoItExitRegister('__Exit') ; Close any open children

HotKeySet('{END}', '__Exit') ; END key to close the program, don't just close the console window or this won't fire

; Main loop
While 1
    ; Spawn child in any slots that are open or replace children that no longer exist
    _CheckForChildrenToSpawn()
    ; Read any StdOut messages and check for 'important' ones
    _CheckForMsgsFromChildren()

    ; Avoid 100% cpu
    Sleep(10)
WEnd

Func _CheckForChildrenToSpawn()
    Local $iSpawned = 0 ; Not really needed, just how many new ones were spawned this check

    ; Loop through each available child slot
    For $iChild = 0 To UBound($aChildren) - 1
        ; Check if the child process still exists
        If ProcessExists($aChildren[$iChild][$CHILD_PID]) = 0 Then
            ; Check for any lingering messages
            _CheckForMsgsFromChildren($iChild)
            ; Spawn new process
            $aChildren[$iChild][$CHILD_PID] = Run('Child.exe ' & Random($iSpawnMin, $iSpawnMax, 1), '', @SW_HIDE, $STDERR_MERGED + $RUN_CREATE_NEW_CONSOLE)
            If @error Or $aChildren[$iChild][$CHILD_PID] = 0 Then
                __cLog('Unable to spawn a child process: ' & @error)
                $aChildren[$iChild][$CHILD_PID] = 0
                ContinueLoop
            EndIf
            ; Just a little sleep so you don't crash your system when spawning a lot of children
            Sleep(50)

            __cLog('Spawned new child PID: ' & $aChildren[$iChild][$CHILD_PID])
            ; Reset values for this child
            $aChildren[$iChild][$CHILD_MSG] = ''
            $aChildren[$iChild][$CHILD_LASTMSG] = TimerInit()
            $iSpawned += 1
        EndIf
    Next

    Return SetError(0, 0, $iSpawned)
EndFunc   ;==>_CheckForChildrenToSpawn

Func _CheckForMsgsFromChildren($iIndexOverride = Default)
    Local $aSplit, $aBetween
    Local $iMin = 0, $iMax = UBound($aChildren) - 1

    ; Allow checking a specific child, used for when the child process doesn't exist any more, get any remaning messages
    If Not $iIndexOverride = Default Then
        $iMin = $iIndexOverride
        $iMax = $iIndexOverride
    EndIf

    ; Loop through each process
    For $iChild = $iMin To $iMax
        ; Get any messages
        $aChildren[$iChild][$CHILD_MSG] = StdoutRead($aChildren[$iChild][$CHILD_PID])
        If @error Then ; Just most likely means that EOF was reached, since we're checking so often
            ContinueLoop
        EndIf

        If $aChildren[$iChild][$CHILD_MSG] == '' Then ContinueLoop ; No message
;~      __cLog('Child ' & StringFormat('%5s', $aChildren[$iChild][$CHILD_PID]) & ' full msg: ' & $aChildren[$iChild][$CHILD_MSG])

        ; Split the output by line, this means that multi-line messages aren't supported
        ; You can also skip this split and just use _StringBetween to get multi-line messages
        $aSplit = StringSplit($aChildren[$iChild][$CHILD_MSG], @CRLF, 3)

        ; Loop through each line of output and search for a [important message]
        For $iMsg = 0 To UBound($aSplit) - 1
            ; Get a message to output in the main script that was logged between as [] in the child
            $aBetween = _StringBetween($aSplit[$iMsg], '[', ']')
            If @error Then ContinueLoop

            For $iLog = 0 To UBound($aBetween) - 1
                __cLog('Child ' & StringFormat('%5s', $aChildren[$iChild][$CHILD_PID]) & ': ' & $aBetween[$iLog])
            Next
        Next

        $aChildren[$iChild][$CHILD_MSG] = ''
        $aChildren[$iChild][$CHILD_LASTMSG] = TimerInit()
    Next
EndFunc   ;==>_CheckForMsgsFromChildren

Func __Exit()
    ; Close any remaining open processes
    For $iChild = 0 To UBound($aChildren) - 1
        ProcessClose($aChildren[$iChild][$CHILD_PID])
    Next

    Exit
EndFunc   ;==>__Exit

Func __cLog($sMsg)
    ConsoleWrite($sMsg & @CRLF)
EndFunc   ;==>__cLog

 

I'm not trying to say that StdoutRead is the best option, and any IPC that you can use multiple outputs with (like if Mailslot allows you to use a different slot for each PID, or any other IPC allows multiple message queues instead of just 1) should have similar behavior.

[Lion66]

Thanks for the above examples. I was interested in studying them.
I remade the code for myself.
The main difference is that I move outside of the loop function _CheckForChildrenToSpawn().
I don't need to repeated run the child scripts.
And it looked good, but I found a serious problem:
If a child script finishes earlier than the Output survey occurs, then there is nothing to poll.
This theoretically can happen in some cases:
- If one of the processes in the launch cycle end before the cycle ends.
- if the message survey cycle is longer than the operating time of one of the processes.
I don’t know how to fix it yet. In my example, I designated the problem in two places as:

Sleep(5000) ; problem here !

Each of them will lead to problems. I attach files.

 

Child.au3 Receiver.au3

[Nine]

Here how to use my IPC :

Client (need to be compiled):

#include <Constants.au3>
#include <GUIConstants.au3>
#include "WCD_IPC.au3"

Opt ("MustDeclareVars", 1)

Local $iClientNumber = $cmdLine[1]

; $_WCD_Verbose = True

Global $hWnd = _WCD_CreateClient ("Test WCD Client " & $iClientNumber)
Global $hWndServer = _WCD_GetServerHandle ()

_WCD_Send($hWnd, $hWndServer, 1, '[Client' & $iClientNumber & ', Message1]' & @CRLF)
Sleep(Random(500, 5000, 1))
_WCD_Send($hWnd, $hWndServer, 1, '[Client' & $iClientNumber & ', Message2]' & @CRLF)
Sleep(Random(500, 5000, 1))
_WCD_Send($hWnd, $hWndServer, 1, '[Client' & $iClientNumber & ', Message3]' & @CRLF)
Sleep(Random(500, 5000, 1))

Parent (you can run it from Scite):

#include <Constants.au3>
#include <GUIConstants.au3>
#include "WCD_IPC.au3"

Opt ("MustDeclareVars", 1)

; $_WCD_Verbose = True

Local $hServer = _WCD_CreateServer ()
Local $aReq

For $i = 1 to 10
  Run("WCD_Client.exe " & $i, "", @SW_HIDE)
Next

While Sleep(100)
  If _WCD_Server_IsRequestAvail () Then
    $aReq = _WCD_Server_GetRequest ()
    ConsoleWrite($aReq[1])
  EndIf
WEnd

No worries - simple - fast 

[mistersquirrle]

9 hours ago, Lion66 said:

And it looked good, but I found a serious problem:
If a child script finishes earlier than the Output survey occurs, then there is nothing to poll.

That should not be a problem. If you look at the example of StdoutRead, they don't even read the output until the process is closed. As I understand it, the parent script keeps the read/write buffer/pipe open until it's cleared/read/closed, which happens on the parents end. So unless a new process is created under the same PID, the data should remain as long as the parent does (though I by no means am very familiar with how that all works, it's just my understanding). I'm not sure if the PID could be re-used while the parent has a 'handle' to it open.

9 hours ago, Lion66 said:

This theoretically can happen in some cases:
- If one of the processes in the launch cycle end before the cycle ends.
- if the message survey cycle is longer than the operating time of one of the processes.
I don’t know how to fix it yet. In my example, I designated the problem in two places as:

Sleep(5000) ; problem here !

 

 The problem is NOT there, the problem in your modifications is this:

; Get a message to output in the main script that was logged between as [] in the child
        $aBetween = _StringBetween($aChildren[$iChild][$CHILD_MSG], '[', ']')
        If @error Then ContinueLoop
        __cLog($aBetween[0])

You are specifically only displaying/processing one message. You need to log/process all occurrences that _StringBetween found (like in my original example):

; Get a message to output in the main script that was logged between as [] in the child
        $aBetween = _StringBetween($aChildren[$iChild][$CHILD_MSG], '[', ']')
        If @error Then ContinueLoop
        For $iMsg = 0 To UBound($aBetween) - 1
            __cLog($aBetween[$iMsg])
        Next

Also in your example you had:

Run('Child.au3 '...

Which for me didn't work, I had to compile the child.au3 and change it to child.exe. But then it worked fine with those changes, the output:

Client0, Message1
Client0, Message2
Client0, Message3
Client1, Message1
Client1, Message2
Client1, Message3
Client2, Message1
Client2, Message2
Client2, Message3

I also in my original example changed the main loop Sleep(10) to Sleep(10000) and a Sleep(5000) in the _CheckForMsgsFromChildren() function in the loop checking each child and I had no issues receiving data in either case.

 

@Nine I haven't used your UDF before, but I tried it out with your example, and it failed for me:

!>11:14:07 AutoIt3.exe ended.rc:-1073740771
+>11:14:07 AutoIt3Wrapper Finished.
>Exit code: 3221226525    Time: 2.62

I looked through the UDF some and enabled the verbose logging, and it only showed (with verbose enabled):

Warning : messages are now allowed with lower privilege windows

So I added #RequireAdmin to both, and it still failed with the same message after it launched 3-4 Clients.

I tried several things and I could not get it to work at all for me. Launching 1 client didn't crash the program, but it also didn't work. The .log only showed "messages are now allowed with lower privilege windows" which is odd because I compiled both with #RequireAdmin, I am admin, and running them with various flags (admin, upx, cui/gui, from scite vs both compiled, etc.) didn't make a difference.

I did finally get it to work by compiling the client as x86, which I normally don't do since I have x64 set as default. I then tested a mixed x64 parent and x86 client to see what would happen, and while it doesn't crash/fail, it does know that there's a message but it comes through blank. So your method only works if both are compiled/set as x86. I didn't see that mentioned in your UDF topic, and I don't know if it's an issue specific to me.

 

Edit: I see now that the error says "now allowed" and not "not allowed", so that wasn't part of the problem at all and works just fine when the client doesn't have admin but the server does. It did not appear to work in reverse if the parent was not an admin but the client was (it looked like that the parent could not start any clients in this case).

Edited March 11, 2023 by mistersquirrle

[Lion66]

Nine

Yes. exactly.
I have not found problems with your UDF,  although I have not yet used in really work.

mistersquirrle

Run('Child.au3 '...

This is a typo. Сorrectly .EXE.

After your remark, yes, I get all messages. But it looks like work consistently.

I would prefer to receive messages as they are sent (with the same sleeps). Like this, how would it be without a delay of 500:

Client1, msg 0
Client2, msg 0
Client3, msg 0
Client1, msg 1
Client2, msg 1
Client3, msg 1
Client2, msg 2
Client1, msg 2
Client3, msg 2

And I have never compile x64, as often get problems from security policy.

This interesting method can be applied, but this is not my choice.

Sorry 😎

Edited March 11, 2023 by Lion66

[mistersquirrle]

The messages from StdoutRead are in sent order of that client, and unless the clients are talking with each other to do things in a certain order, why does it matter? What is the purpose? Can you give an example (if not exact) use case of needing them in absolute order vs client order?

Is it a problem if you receive all messages from Client1, then Client2, then Client3 instead of

1. Client1 Msg1
2. Client2 Msg1
3. Client1 Msg2
4. Client3 Msg1
5. Client1 Msg3

?

If you really need them in absolute order then you'll likely need to include a timestamp in the message, load all messages first into a queue, sort the queue by the timestamp, then do whatever (log or action) in that order.

Nine can correct me, but their UDF also doesn't seem to do any type of ordering. However because of how it works with GUIRegisterMsg/SendMessage if two messages are sent at about the same time, the second one would get added first (though the timeframe for that to happen would be VERY slim). It does have a queue though from what I'm seeing. And because it works on interrupts as mentioned, it would be VERY unlikely that messages would be out of absolute order. Shouldn't be an issue with SendMessage, only PostMessage which can't be used with WM_COPYDATA, see Nine's comment about FIFO (First In First Out)

 

Also to be clear, I'm not trying to say or convince you that you should use my method/example over Nine's or anyone elses, especially since we still don't really know what your "real work" is for this. I'm just filling in any information/answering questions I might've missed on the information I've provided so far.

Honestly I started to look into doing some IPC things using the same methods that Nine is doing with WCD before I had looked at it, though I was just using WM_COMMAND, not WM_COPYDATA. I'm a fan of the 'instant' nature of GUIRegisterMsg being a callback and triggering as soon as the message is received (though with SendMessage it can take a while, up to ~20ms to send/return in the sender, vs PostMessage which sends/returns in under 0.1ms since it doesn't make sure the message is received/processed).

Edited March 12, 2023 by mistersquirrle
striked incorrect information

[Nine]

2 hours ago, mistersquirrle said:

Nine can correct me, but their UDF also doesn't seem to do any type of ordering.

False.  It orders in the queue of receiving/sending.  You need to understand that messages from Windows are treated on FIFO manner.  So if you need to reorder messages, it is the responsibility of the parent/child to do so. 

BTW, I do not think you need to answer every single thread in this forum, especially if you do not provide relevant information.