Pool - Post Office for Organised Labour

[RTFC]

Pool provides multiple AutoIt processes with an infrastructure for exchanging messages, data, and remote-control instructions, on a single machine or on a local area network. It is modelled partly on HTCondor, but also incorporates user-defined messages (UDM), LANchat, mails, data Container shipping management, and remote power control and remote execution. That last feature means one process can send lines of AutoIt script to another process, and run it there (including batch control and synchronisation). Consider it my take on inter-process communication (IPC).
 
Pool incorporates many contributions (a few are included in the bundle, other tiny snippets are part of the main script), from other forum members without whom this project simply would never even have been conceivable. In particular, I hereby want to acknowledge the excellent work of Zatorg, wraithdu, W0uter, Ward, ValeryVal, spudw2k, trancexx, This-is-Me, Nomad, Nine, Manadar, LazyCat, Lakes, Kip, Kealper, Jos, Greencan, FredAI, evilertoaster, dragan, Chimp, and AdamUL. Some codes merely helped to clarify my novice understanding of various IPC issues, others have been gratefully integrated. Three contributors need special mention: Kip & Zatorg for event-driven TCP, and trancexx for MailSlot; together these form the backbone of everything else. The only reason I still dare to publish this under my own name is because I did add over ten thousand lines of code myself. Nevertheless, Pool may serve as a showcase for how various members' example scripts can be combined into a comprehensive environment. Many, many thanks to everyone involved!
 
Pool is designed to be simple to use. Only a few functions are public, and of those, you really only need to study these four control UDFs:
_Pool_Send_Command(): Pool's Command & Control Centre (see list of commands below)
_Pool_Send_UserDefinedMsg(): define your own outgoing msgs
_Pool_Receive_UserDefinedMsg(): process your own incoming msgs
_Pool_Send_ExeQcall(): send one or more AutoIt instructions to target's Exe-Queue (ExeQ)
 
and these ones for data transfers:
_Pool_Container_Create(): associate shipping Container with data
_Pool_Container_Destroy(): release Container memory and all associated Sharing relations
_Pool_Container_CreateShare(): associate destination with Container
_Pool_Container_DestroyShare(): dissolve sharing association for destination
 
To get an idea of what _Pool_Send_Command() can do, here's the list of commands:
 

 

* Pool Control

"Ping" ; test presence, determine Pool-Msg turnaround time
"StandBy" ; defer TODOlist/ExeQ processing
"Proceed" ; resume TODOlist/ExeQ processing
"SyncWait" ; PARAM=syncID; Standby until SYNC_RESUME with same syncID received
"SyncResume" ; PARAM=syncID; Standby until SYNC_RESUME with same syncID received
"Identify" ; tell msg recipient to mail its own PML entry to msg sender
"Uptime" ; return uptime
"Mute" ; PARAM=T/F; T: defer outgoing messages; F: resume
"LeaveNow" ; leave Pool immediately; Exit unless $_POOL_STATUS_STAYALIVE = True
"LeaveWhenDone" ; leave Pool once all current tasks are completed (no reverse transfers afterwards)
"CountMembers" ; update various PML-based membership counters
"ListLocalMembers" ; return entire PML
"AddMembers" ; PARAM=number; tell MGR to start N more default processes locally
"AddMaxMembers" ; tell MGR to start new default Pool processes up to the number of CPU cores
"RemoveMember" ; tell TARGET process(!) to tell all COLLECTORS

* Power-related

"MachineWakeUp" ; triggers Wake-on-LAN magicpacket (untested)
"MachineStandBy" ; place target's machine in StandBy mode
"MachineSuspend" ; place target's machine in Hibernation
"MachineShutDown" ; power down target's machine (target calls Pool_Closedown too)
"MachineReboot" ; reboot target's machine (target calls Pool_Closedown too)

* MailSlot-related (call _Pool_Send_Mail() to send your own, user-defined mails)

"MailTest" ; send a Mail (with double echo)
"Scan4MailSlots" ; detect new MailSlots on localhost
"DestroyMailSlot" ; close MailSlot
"CreateMailSlot" ; create a new MailSlot

* ExeQ-related (call _Pool_Send_ExeQcall() to parse a single cmd or multi-line batch job)

"ClearExeQ" ; release all ExeQ contents
"IamYourMaster" ; for BDSM lovers
"IamYourSlave" ; ditto
"IamNotYourSlave" ; for the rest of us
"ExeQReportState" ; returns IPTR/ExeQ size, stored in PML
"ExeQReportBatch" ; returns target's current batch markers, stored in $_POOL_STATUS_EXEQ_BATCH_START/END
"ExeQIgnoreErrors" ; PARAM = T/F; break ExeQ running on @error
"ExeQBatchStart" ; use this ONLY before supplying batch contents one line at a time
"ExeQBatchEnd" ; use this ONLY after supplying batch contents one line at a time
"Run" ; start/resume execution until end of ExeQueue/currently defined batch
"Break" ; interrupt batch execution a.s.a.p.
"RunNext" ; execute the contents of $_Pool_ExeQueue[$_POOL_STATUS_EXEQ_IPTR]
"RunBatch" ; execute all lines of a defined ExeQ batch
"RunFrom" ; execute from the specified ExeQ line to the end
"RunPrev" ; execute last-previously executed line again
"RunAll" ; (re)run from first line of ExeQ to the end
"SetExeQptr" ; PARAM low word = IPTR (ExeQ instruction pointer)

* Container-related (requires prior creation of data Container and at least one Share)

"RequestContainerID" ; ask local MGR to provide IDs before creating Containers (much faster)
"LoadContainer" ; PARAM = structID; tell CLN to load specified Container with associated data, but do not send it (yet)
"LoadSendContainer" ; PARAM = structID; tell CLN to load specified Container with associated data, and send it
; if targetID = Creator: send out to all defined shares
; if targetID = sharer: return data to Creator
"SendContainer" ; PARAM = structID; tell CLN to send specified Container without reloading it first
; if targetID = Creator: send out to all defined shares
; if targetID = sharer: return data to Creator
"SendContainerAsMail"; PARAM = structID; tell CLN to send specified TINY Container via MailSlot, without reloading it first
"LoadSendContainerAsMail" ; PARAM = structID; tell CLN to reload and send specified TINY Container via MailSlot
"PrepContainerTtransfers" ; PARAM = structID; tell CLN to (re)activate all Shares for specified Container (sets all Share Transfers as "pending")
"DestroyContainer" ; PARAM = structID; release specified Container only
"ReleaseAllContainers" ; release all Containers (variables themselves remain intact)

* TCP/UDP-related

"NewPort" ; PARAM=new port; assign new port for TCP/UDP (requires both protocols to be currently disabled)
"TCPconnect" ; for MGRs/testing only; NB this would be impossible to receive without the MailSlot alternative
"TCPdisconnect" ; for MGRs/testing only
"TCPmultiMsgDelay" ; PARAM = delay in ms; adjust target's $_POOL_STATUS_TCP_MULTIMSG_DELAY
"WMmultiMsgDelay" ; PARAM = delay in ms; adjust target's $_POOL_STATUS_WMULTIMSG_DELAY
"UDPconnect" ; for MGRs/testing only
"UDPdisconnect" ; for MGRs/testing only
"UDPbaseDelay" ; PARAM = delay in ms; adjust target's $_POOL_STATUS_UDP_BASEDELAY
"UDPstepDelay" ; PARAM = delay in ms; adjust target's $_POOL_STATUS_UDP_STEPDELAY
"UseTCPforDataTransfers" ; switch to TCP packets per target MGR for shipping Containers
"UseUDPforDataTransfers" ; switch to UDP packet broadcasting for shipping Containers

 
Pool is big. It has got many dozens of fixed and dynamic user-defined settings to create specific applications with (see #Region Globals). Please take the time to read the extensive Remarks section at the top of the main script (Pool.au3), including the warnings and limitations. Furthermore, I've provided two example environments in the subdir Examples in the bundle: CLNtest (messages and remote execution) and CLNmaster + CLNslave (data Container shipping). If you run these on a single machine, you'll need to start PostOffice_Solo; on multiple machines, one should start first with PostOffice_Server, the rest with PostOffice_Client, once the Server is initialised. See the readme.txt in the Examples subdir for additional help. Another example (multi-processing a la Condor) is shipped with release 2.4+ of Eigen4AutoIt.

Since this is an Example script, you can find a number of interesting design ideas in Pool:

 

 

* Containers can be retrieved without ever having been sent out first

* TODO list processing can defer tasks with task-specific retry-expiry

* the three main Pool 2D arrays (PML, PSL, PSC) are accessed by column Name, without having to know the column index (like a database)

* you can switch between TCP (host-targeted) and UDP (broadcast) for data transfers

* handling of subnet masks other than 255.255.255.0 (look for _CIDR_PREFIX)

* using a hexstring for checksum bitflags (see _Pool_PrepChecksum, _Pool_SetChecksum, etc)

* TCP socket buffer + event processing can handle stacked and partial messages

* resend data slice msgs use slice multiplexing with data-dependent granularity

 

 Pool.v0.7.7z Pool bundle, second beta release, version 0.7

  IMPORTANT CAVEAT 

This is an experimental beta release. Some parts of Pool have never been tested, others only in the simplest possible setting (the largest network I ever tested comprised one desktop and three laptops (one with busted radio) on a crappy Wifi router; I've never tested it on a cabled network). The (W)LAN part still has many issues and may not be able to handle more than a few machines. There's no callstack error handler. Event-driven TCP remains glitchy, and some of my MailSlots keep malfunctioning. I also lack access to Windows 8.* test environments.

In addition, many issues you may encounter will be due to specific timings/event sequencing (which makes it hard to debug) and/or your specific infrastructure. That means that, most likely, I cannot recreate your Pool bugs (even with your scripts), so I cannot fix them either. Therefore, I will not be offering bug support for users at this point; there's still far too much I need to fix in my own environment to worry about yours, I'm sorry (also, I've been working on this for four months flat, and need a break from it). However, I am of course open to suggestions, remarks, criticism, and kind words (preferably the latter ).

Finally, I will be exceptionally busy with work for the foreseeable future, so it may take me more time than in the past to respond to your posts.

Have fun playing in the Pool.

Edited January 1, 2016 by RTFC

[TheDcoder]

Always the useful, RTFC you rule!!!

[RTFC]

High five, bro!

(and thanks for posting!)

[TheDcoder]

High five!

[RTFC]

I was given the impression of late that Pool.au3 looks too complicated to engage with, so I thought I'd post one of the bundled examples, to show that it's actually really simple to use.

A Pool script consists of a few steps:

1. define what type of Pool entity the script is going to be (a Post Office, client, master, slave...)
2. call _Pool_StartUp()
3. perform pool actions with _Pool_Send_Command ( $targetID, <pool command>, see full listing in Pool.au3 Remarks section)
4. call _Pool_CloseDown()

Per machine, you need one Post Office script (already running), and as many Pool clients as you like.

Furthermore- if you wish to send data from one script to another (on the same machine or on a network), you additionally need to:

1. create a shipping container for it (with _Pool_Container_Create());
2. define a "Share" relationship for the Container, that is, tell Pool where it needs to go (with _Pool_Container_CreateShare()); and
3. load the Container with data and send it, with _Pool_Send_Command ( $containerID, "LoadSendContainer")

That's basically it.

 

So here's a simple Pool client script (included in the bundle) that communicates with other Pool clients (start at least two instances of this), and which can send and remotely execute AutoIt calls at the destination. Below it you'll find a Post Office script that handles all comms between clients on a machine (with thanks to Melba23 for auxiliary function _MemoWrite).

expandcollapse popup

; POOL Messaging & ExeQueue Example
;
; Run one instance of PostOffice.exe per machine, and two or more instances of this script
;
;____________________________________________________________________________

#NoTrayIcon
#include "Pool.au3"

; set attributes (fixed)
$_POOL_ATTRIB_CLASS         ="Any"  ; cannot be empty; should match Pool MGR's Class
$_POOL_ATTRIB_OPERATOR      ="CLN"  ; CLN = main app/master/slave/collaborator/parent/child...
$_POOL_ATTRIB_SHAREDATA     =False  ; not tested here
$_POOL_ATTRIB_COLLECTOR     =True       ; keep PML up to date (for master/parent/collaborator)
$_POOL_ATTRIB_LAUNCH_MGR    =True       ; T: allow starting a new MGR instance ourselves
$_POOL_ATTRIB_KEY               ="<replace this string with your own encryption key in your apps>"
$_POOL_ATTRIB_MGR_DEFAULT   ="PostOffice_Solo.exe"  ; default MGR to launch if none found locally

; NB use these if testing on a (W)LAN (NB use one _Server, rest _Client)
;$_POOL_ATTRIB_MGR_DEFAULT  ="PostOffice_Server.exe"    ; default MGR to launch if none found locally
;$_POOL_ATTRIB_MGR_DEFAULT  ="PostOffice_Client.exe"    ; default MGR to launch if none found locally

; disable relevant timeouts
$_POOL_ATTRIB_TIMEOUT_CLN_SOLO=-1       ; positive value = msec without MGR before leaving Pool
$_POOL_ATTRIB_TIMEOUT_INACTIVE=-1       ; positive value = msec without msg/mail/TODOlist before leaving Pool
$_POOL_STATUS_TIMEOUT_UNCONDITIONAL=-1  ; positive value = msec before leaving Pool, regardless of state

; set status vars (dynamic)
$_POOL_STATUS_TESTING           =True       ; write events to handler
$_POOL_STATUS_CHAT_ENABLED      =True       ; quack, quack, quack...
$_POOL_STATUS_CHAT_ALIAS        ="CHAT " & @ScriptName & " / " & @AutoItPID & " says: "
$_POOL_STATUS_EXEQ_ENABLED      =True       ; T: allow other CLNs to send and Execute ANY AutoIt call here
$_POOL_STATUS_EXEQ_ENCRYPTED    =True       ; strongly recommended for general use
$_POOL_STATUS_IGNORE_ERRORS     =False  ; F: break ExeQ execution upon errors
$_POOL_STATUS_CHAT_INCOMING_UDF =_MemoWrite ; for this demo; write your own handler
$_POOL_STATUS_TESTING_UDF       =_MemoWrite ; for this demo; write your own handler
$_POOL_STATUS_STAYALIVE         =False  ; F: terminate when leaving Pool
$_POOL_STATUS_BEEP              =True       ; make some noise when pinged

; join Pool
_Pool_StartUp()

; variables/data for testing
$thisVarContained123=123                ; to be changed at destination by testbatch below
$targetID   =   $_POOL_BROADCAST        ; 0xffff = broadcast to all CLNs
$chatmsg    =   "Blah blah blah..." ; NB max.size=424 chars for long-distance chatmsgs (unencrypted)
$testCall   =   "Msgbox(0,'Test Call from PID " & @AutoItPID & " on " & @ComputerName & "','This MsgBox appears courtesy of ExeQ, running on PID: ' & @AutoItPID,5)"
$testBatch  =   "Beep(200,50)" & @CR & _
                    "Assign('thisVarContained123',456,2)" & @CR & _
                    "Beep(400,50)" & @CR & _
                    "Beep(800,50)" & @CR & _
                    "Beep(1600,50)" & @CR & _   ; should really be using asynchronous beep, because Beep() is a blocking function
                    "Beep(3200,50)" & @CR & _   ; but we get away with it here because it's only 50 ms per beep
                    "Run('Notepad.exe')" & @CR & _  ; never use RunWait in ExeQ!
                    "_TestUDF()"

; display status msgs and some action buttons
Global $hGUI=GUICreate("Msg/Mail/ExeQ Test CLN PID: " & @AutoItPID, 400, 410)
Global $hEdit=GUICtrlCreateEdit("Please wait for Post Office to connect..."  & @CRLF & @CRLF, 119, 10, 276, 390, BitOR($WS_VSCROLL,$ES_READONLY))
GUICtrlSetFont($hEdit, 9, 400, 0, "Courier New")

$buttonwidth=100
$ButtonPing         =GUICtrlCreateButton("Ping", 10, 10, $buttonwidth, 25)
GUICtrlSetState(-1,$GUI_DISABLE)
$ButtonStandby      =GUICtrlCreateButton("Standby", 10, 40, $buttonwidth, 25)
GUICtrlSetState(-1,$GUI_DISABLE)
$ButtonProceed      =GUICtrlCreateButton("Proceed", 10, 70, $buttonwidth, 25)
GUICtrlSetState(-1,$GUI_DISABLE)
$ButtonMailTest =GUICtrlCreateButton("Mail Test", 10, 100, $buttonwidth, 25)
GUICtrlSetState(-1,$GUI_DISABLE)
$ButtonChat         =GUICtrlCreateButton("Send Chat Msg", 10, 130, $buttonwidth, 25)
GUICtrlSetState(-1,$GUI_DISABLE)
$ButtonExeQline =GUICtrlCreateButton("Send ExeQ Line", 10, 160, $buttonwidth, 25)
GUICtrlSetState(-1,$GUI_DISABLE)
$ButtonExeQbatch    =GUICtrlCreateButton("Send ExeQ Batch", 10, 190, $buttonwidth, 25)
GUICtrlSetState(-1,$GUI_DISABLE)
$ButtonRunNext      =GUICtrlCreateButton("Run next line", 10, 220, $buttonwidth, 25)
GUICtrlSetState(-1,$GUI_DISABLE)
$ButtonRunLast      =GUICtrlCreateButton("Rerun last Line", 10, 250, $buttonwidth, 25)
GUICtrlSetState(-1,$GUI_DISABLE)
$ButtonRunBatch =GUICtrlCreateButton("Run Batch", 10, 280, $buttonwidth, 25)
GUICtrlSetState(-1,$GUI_DISABLE)
$ButtonExeQclear    =GUICtrlCreateButton("Clear ExeQ", 10, 310, $buttonwidth, 25)
GUICtrlSetState(-1,$GUI_DISABLE)

; always enabled
$ButtonShowPML      =GUICtrlCreateButton("Show PML", 10, 340, $buttonwidth, 25)
$ButtonQuit         =GUICtrlCreateButton("Quit", 10, 370, $buttonwidth, 25)

; GUI presets
$firsttime=True
$linesent=False
$batchsent=False
$ExeQcalled=False
$buttonsEnabled=False

GUISetState(@SW_SHOW,$hGUI)
While True
    Switch GUIGetMsg()
        Case $GUI_EVENT_CLOSE,$ButtonQuit
         ExitLoop

        Case $ButtonShowPML ; PML = Pool Member List
            _ArrayDisplay($_Pool_MemberList,"Pool Member List")

        ; remaining controls are enabled only if at least two CLNs are registered
        Case $ButtonPing
            _Pool_Send_Command($targetID,"Ping")

        Case $ButtonStandby
            _Pool_Send_Command($targetID,"StandBy")

        Case $ButtonProceed
            _Pool_Send_Command($targetID,"Proceed")

        Case $ButtonMailTest
            _Pool_Send_Command($targetID,"MailTest")

        Case $ButtonChat
            _Pool_Send_ChatMsg($targetID,$chatmsg)

        Case $ButtonExeQline
            _MemoWrite("Please be patient; this may take a few seconds to arrive at the destination(s)..." & @CRLF)
            _Pool_Send_ExeQcall($targetID,$testCall)
            $linesent=True
            $buttonsEnabled=False

        Case $ButtonExeQBatch
            _MemoWrite("Please be patient; this may take a few seconds to arrive at the destination(s)..." & @CRLF)
            _Pool_Send_ExeQcall($targetID,$testBatch)
            $batchsent=True
            $buttonsEnabled=False

        Case $ButtonRunNext
            _Pool_Send_Command($targetID,"RunNext")
            $ExeQcalled=True
            $buttonsEnabled=False

        Case $ButtonRunLast
            _Pool_Send_Command($targetID,"RepeatLast")

        Case $ButtonRunBatch
            _Pool_Send_Command($targetID,"RunBatch")
            $ExeQcalled=True
            $buttonsEnabled=False

        Case $ButtonExeQclear
            _Pool_Send_Command($targetID,"ExeQClear")
            $linesent=False
            $batchsent=False
            $ExeQcalled=False
            $buttonsEnabled=False
    EndSwitch

    ; multiple clients + registered by MGR = buttons enabled
    If $_POOL_STATUS_ID>0 And $_POOL_STATUS_TOTAL_ALLCLIENTS>0 Then
        If $buttonsEnabled=False Then
            GUICtrlSetState($ButtonPing     , $GUI_ENABLE)
            GUICtrlSetState($ButtonStandby  , $GUI_ENABLE)
            GUICtrlSetState($ButtonProceed  , $GUI_ENABLE)
            GUICtrlSetState($ButtonMailTest , $GUI_ENABLE)
            GUICtrlSetState($ButtonChat     , $GUI_ENABLE)
            GUICtrlSetState($ButtonExeQline , $GUI_ENABLE)
            GUICtrlSetState($ButtonExeQbatch    , $GUI_ENABLE)
            GUICtrlSetState($ButtonExeQclear    , $GUI_ENABLE)

            If $batchsent=True Then
                GUICtrlSetState($ButtonRunBatch,$GUI_ENABLE)
            Else
                GUICtrlSetState($ButtonRunBatch,$GUI_DISABLE)
            EndIf

            If $batchsent=True Or $linesent=True Then
                GUICtrlSetState($ButtonRunNext,$GUI_ENABLE)
            Else
                GUICtrlSetState($ButtonRunNext, $GUI_DISABLE)
            EndIf

            If $ExeQcalled=True Then
                GUICtrlSetState($ButtonRunLast,$GUI_ENABLE)
            Else
                GUICtrlSetState($ButtonRunLast,$GUI_DISABLE)
            EndIf

            $buttonsEnabled=True
            If $firsttime=True Then
                _MemoWrite("Ready for testing." & @CRLF)
                If Mod($_POOL_STATUS_ID,2)=0 Then
                    WinMove($hGUI,"",5,5,Default,Default,3)
                Else
                    WinMove($hGUI,"",@DesktopWidth-405,5, Default,Default,3)
                EndIf
                $firsttime=False
            EndIf
        EndIf
    Else
        If $buttonsEnabled=True Then
            GUICtrlSetState($ButtonPing     , $GUI_DISABLE)
            GUICtrlSetState($ButtonStandby  , $GUI_DISABLE)
            GUICtrlSetState($ButtonProceed  , $GUI_DISABLE)
            GUICtrlSetState($ButtonMailTest , $GUI_DISABLE)
            GUICtrlSetState($ButtonChat     , $GUI_DISABLE)
            GUICtrlSetState($ButtonExeQbatch    , $GUI_DISABLE)
            GUICtrlSetState($ButtonExeQclear    , $GUI_DISABLE)
            GUICtrlSetState($ButtonExeQline , $GUI_DISABLE)
            GUICtrlSetState($ButtonRunBatch , $GUI_DISABLE)
            GUICtrlSetState($ButtonRunNext  , $GUI_DISABLE)
            GUICtrlSetState($ButtonRunLast  , $GUI_DISABLE)
            $buttonsEnabled=False
        EndIf
    EndIf
WEnd
GUIDelete($hGUI)

_Pool_CloseDown()



; http://www.autoitscript.com/forum/topic/110948-add-text-to-edit-box-and-scroll-it-down/, post #2; Thanks, Melba23!
Func _MemoWrite($sMessage)
; display status updates in GUI

    ConsoleWrite($sMessage)

    If Not IsDeclared("hEdit") Then Return

   Local $iEnd = StringLen(GUICtrlRead($hEdit))
   _GUICtrlEdit_SetSel($hEdit, $iEnd, $iEnd)
    _GUICtrlEdit_Scroll($hEdit, $SB_SCROLLCARET)
    GUICtrlSetData($hEdit,$sMessage, 1)

EndFunc

Func _TestUDF()

        _MemoWrite("_TestUDF was called" & @CRLF & _
        "Updated $thisVarContained123 = " & $thisVarContained123 & @CRLF)

EndFunc

 

And the Post Office script:

expandcollapse popup

; DO NOT USE THIS SCRIPT IN A (W)LAN POOL, but ONLY on a single machine

#NoTrayIcon
#include "Pool.au3"

; set some attributes (permanent)
$_POOL_ATTRIB_CLASS             ="Any"  ; cannot be empty; should match CLNs' Class in this Pool
$_POOL_ATTRIB_OPERATOR          ="MGR"  ; MGR = your friendly local Post Office
$_POOL_ATTRIB_SHAREDATA         =True   ; T: handle incoming/outgoing vars, arrays, structs
$_POOL_ATTRIB_LAUNCH_SERVER     =False  ; boolean, T: start new instance of Server if original one dies
$_POOL_ATTRIB_LAUNCH_CLN        =False  ; enable this for grid computing functionality

; disable relevant timeouts
$_POOL_ATTRIB_TIMEOUT_UNCONDITIONAL =-1 ; positive value = always Exit when time's up (in msec)
$_POOL_ATTRIB_TIMEOUT_SERVER_SOLO   =-1 ; positive value = msec without any TCP clients (MGRs on other machines) before closedown
$_POOL_ATTRIB_TIMEOUT_MGR_SOLO      =-1 ; positive value = msec without local CLNs before closedown
$_POOL_ATTRIB_TIMEOUT_INACTIVE      =-1 ; positive value = msec without msg/mail/TCP/UDP/TODOlist before closedown

; set some status variables
$_POOL_STATUS_MGR_SOLO          =True   ; T: no LAN support (much faster); use this if all your CLNs run on a single machine
$_POOL_STATUS_SERVER_CESTMOI    =False  ; T: start as TCP server (required only on LAN)
$_POOL_STATUS_TCP_ENABLED       =False  ; required only for event-driven LAN msgs
$_POOL_STATUS_UDP_ENABLED       =False  ; required only for LAN data Sharing

; set Pool-related status vars
$_POOL_STATUS_EXEQ_ENABLED      =False  ; no point enabling this for a MGR (plus potential security risk)
$_POOL_STATUS_CHAT_ENABLED      =False  ; no point enabling this for a MGR (no user)
$_POOL_STATUS_STAYALIVE         =False  ; F: terminate when leaving Pool
$_POOL_STATUS_TESTING           =True

; Intialisation may take several minutes on a large LAN,
;   but because we here set $_POOL_STATUS_MGR_SOLO=True, no LAN scan is performed.

; initialise
_Pool_StartUp()

If @Compiled Then
    ; Everything a MGR does is periodic or event-driven.
    While True
        Sleep(10000)
    WEnd

Else    ; user interface for testing
    $hGUI=GUICreate("MGR", 160, 160)

    ; define some buttons
    $ButtonShowPML      =GUICtrlCreateButton("Show PML", 20, 10, 115, 25)
    $ButtonShowPSL      =GUICtrlCreateButton("Show PSL", 20, 40, 115, 25)
    $ButtonShowPSC      =GUICtrlCreateButton("Show PSC", 20, 70, 115, 25)
    $ButtonQuit         =GUICtrlCreateButton("Quit", 20, 120, 115, 25)

    WinMove($hGUI,"",Default,5)
    GUISetState(@SW_SHOW)
    While True
        Switch GUIGetMsg()
            Case $GUI_EVENT_CLOSE,$ButtonQuit
                ExitLoop

            Case $ButtonShowPML ; PML = Pool Member List
                _ArrayDisplay($_Pool_MemberList,"Pool Member List (PML)")

            Case $ButtonShowPSL ; PSL = Pool Struct List (list of Containers)
                _ArrayDisplay($_Pool_Structlist,"Pool Container List")

            Case $ButtonShowPSC ; PSC = Pool Shared Containers List
                _ArrayDisplay($_Pool_SharedContainers,"Pool Shares List")
        EndSwitch
    WEnd
EndIf

_Pool_CloseDown()
Exit

 

The master/slave example scripts in the bundle show how to parse variables, arrays, strings, structs, etc. between scripts.

Hope it helps.

PS this project is still in developmental limbo, so the example scripts and Remarks still constitute my only support for this, sorry.

[argumentum]

it fails and without support I'll find myself writing one from scratch based on your code.

---------------------------
AutoIt Error
---------------------------
Line 4490  (File "G:\au3s\Pool.v0.6\Pool\Pool.au3"):
$_Pool_MemberList[$memberindex][_Pool_GetPMLcol("Mail Errors")]=0
$_Pool_MemberList[^ ERROR
Error: Variable used without being declared.
---------------------------
OK   
---------------------------

=(

[RTFC]

Happy New Year, argumentum.

I've got to hand it to you, you're good at finding bugs in my code. You could earn good money with that kind of skill, you know? And because it's a special day today, and I'm feeling exceptionally generous, I've uploaded a bug-fixed beta version 0,7, (to be honest, it was a trivial oversight on my part, patched in 10 seconds).

 

Edited January 1, 2016 by RTFC

[noellarkin]

@RTFC are you still using/maintaining this UDF? I was looking for a single UDF that covers remote execution within a LAN, coprocess and IPC, and this seems to cover everything :)

[noellarkin]

At the moment I'm using the HTTPAPI server and have prescripted endpoints that I send requests to (each "agent" (slave in POOL terminology) computer on the LAN has the HTTPAPI server running as a service, and it catches the requests, adds them to queue.

Each request is a POST with a request body containing the AutoIt script I want to run as a string. The agent parses the request body, compiles it into an EXE file and 'RunWait's it. It sends the exit code and errors, if any, back as an HTTP response to the "master" system).

I've also figured out a way for the  "Master" to broadcast commands to all the systems on LAN (for example, to update an application) using ChildProc + HTTP requests (spawns a new process for each HTTP request).

How would POOL be an improvement over my rather rudimentary methods?

[RTFC]

On 10/3/2023 at 9:08 AM, noellarkin said:

are you still using/maintaining this UDF

Errrm, not actively, no, sorry (not much community interest over the last few years either, until your post). I now require a much faster, secure, high-throughput data transfer utility across WAN, one that an AutoIt solution simply cannot provide (it needs to run across platforms, not just on Windows, and without a single authority), so I cobbled together a C++ P2P solution that I am unable to make available for general use at this time.

On 10/3/2023 at 9:17 AM, noellarkin said:

How would POOL be an improvement over my rather rudimentary methods?

POOL-parsed scripts do not need to be compiled, and don't spawn separate processes for each request (but maybe that's part of your requirements, in which case it's a feature Pool lacks, it all depends on your use case). And it doesn't sound like your methods are rudimentary at all, more like you've developed a completely different, sophisticated way to achieve your goals.