Multi-Tasking UDF

Network_Guy · December 14, 2018

Hello AutoIT community ,

i would like to share my Multi Task UDF which enable execution of a function with or without parameters (let me call it task for simplicity) in a separated process as many times its needed with or without pausing the main script and create special shared variables all tasks can read &write from/to it , finally all Task will close automatically if Main process closed.

Requirements :-

1- MailSlot UDF (used by tasks to communicate)

2- #include <MultiTasking.au3> (Must be the last library to #include )

Functions :-

For managing Tasks :-

_Task_Create("mytask","myfunction",parameter1,parameter2,.................,parameter8) => create new task with name"mytask" which will execute "myfunc" function with max 8 parameter _Task_IsAlive("mytask") => return true if task "mytask" is alive ,false if dead
_Task_Kill("mytask") => force killing the task
_Task_Join("mytask") => Pause current process until task "mytask" is finished (can accept string array contain task names to pause current process untill all tasks have finished)
_Task_Info() => return array contain current Taskname ,executed function name,main process who called those task (PID)

Note :- using name is optional if u used blank string "" in _Task_create , task will have random name and u can use Process ID as returned from calling _Task_create function with _IsAlive ,_kill and _join methods insteed.

Example for managing Tasks

#include <WinAPISys.au3>
#include <Array.au3>
#include <MultiTasking.au3>       ;====> always last include

Opt("WinWaitDelay",100)                             ;====> _Task_Join use win&process , decreasing "winwaitdelay" from its default values increase things up

MsgBox(0,"Main Process PID: "&@AutoItPID,"Calling Func without parameters")

sleep(1000)
_Task_Create("Task1","Func_WITHOUT_parameters")      ;====> Only main process can use _Task_Create to prevent Process Loop
_Task_Create("Task2","Func_WITHOUT_parameters")
local $tasks[2]=["Task1","Task2"]
_TasK_Join($tasks)                                   ;====> Accept single name or Process ID or array contain both of them

sleep(2000)
MsgBox(0,"Main Process","Calling Func with parameters ")
local $Htimer=TimerInit()
 local $PID=_Task_Create("","Func_WITH_parameters",10,$Htimer);====> _Task_Create always return child process ID ( SToutread,STinwrite,STerrRead work with this PID )
_TasK_Join($PID)

sleep(2000)
msgBox(0,"Main Process","Lets call Endless Task")
_Task_Create("EndlessTask","FinalTask")


Sleep(2000)
local $state=_Task_IsAlive("EndlessTask")
MsgBox(0,"Main Process","EndlessTask live status is "&$state&" lets kill it")
_Task_kill("EndlessTask")                                       ;====>Not Recommended

sleep(2000)
$state=_Task_IsAlive("EndlessTask")
MsgBox(0,"Main Process","Now EndlessTask status is "&$state)

sleep(1000)
MsgBox(0,"Main Process","Thanx and Good Bye")


func Func_WITHOUT_parameters()
    local $infoarray=_Task_info()
    $Str="Hello I'm "&$infoarray[0]&@CRLF
    $Str&="I Executing "&$infoarray[1]&@CRLF
    $Str&="And my parent Process PID is :"&$infoarray[2]
    MsgBox(0,"Task : "&@AutoItPID,$Str)

EndFunc


func  Func_WITH_parameters($parameter1,$parameter2) ;====> $parameters always received as Strings
    $timetoexecutetask=TimerDiff($parameter2)
    sleep(2000)
    $Str="Hello my first parameter is "&$parameter1&" and its type is "&VarGetType($parameter1)&@CRLF
    $Str&="my second parameter is "&$parameter2&" and its type is "&VarGetType($parameter1)&@CRLF
    $Str&="and i tooked  "&$timetoexecutetask&" Msec after calling _Task_create function to start." ;====> Dont spam Task creation use it Wisly
    MsgBox(0,"Task: "&@AutoItPID,$Str)
EndFunc

Func FinalTask ()                               ;====> if Main process exit this task will exit too even if it has infinte loop
    While 1
    Sleep(100)
    Wend
    EndFunc

For Sharing Variables :-

_Task_SetVar(“shared_variable”,value) ==> create or set shared_variable = value
_Task_GetVar(“shared_variable”) ==> return shared_variable value

For Sharing Arrays :-

_Task_SetArray(“shared_Array”,$array) ==> Create or set Shared_array = $array (1D or 2D only)
_Task_Getarray(“shared_Array”) ==> return shared_array
_Task_SetArrayElement(“Shared_array”,value,Row,column) ==> set shared_array[Row][column] =value
_Task_GetArrayElement(“shared_array”, Row, column) ==> return shared_array[Row][column] value

Notes:-

for better performance use set or get array if you gonna deal with the whole array but use set or get arrayelement if u deal with a single array element only .
Many Tasks Read and Write to same Variable could lead to Race condition so its advisable to make sure that only one task write to one variable but if for some reason u need to write to same variable with many tasks same time u can use _Task_setvarEX or _task_SetarrayelementEX .

_Task_SetvarEX(“shared_variable”,”+=”,1) 
; it will check current shared_variable value and will increase it by 1 or equal $shared_variables +=1,Supported operators till now = ["+=", "-=", "/=", "*=", ;"&="]

if u faced "Error : Memory Buffer queue overload ." too many Input/Output operations and cannot be handled .solution is slow down Input/output by using sleep(1) and increase sleep value until your script be stable .
in case u need to sync between Tasks avoid using such code :-

Do
$boolean=_Task_GetVar(“shared_boolean”)
Sleep(1)
Untill  $boolean=false

Instead use:-

_Task_PauseUntilVar(“shared_boolean”,”=”,0) ; pause script until shared_boolean=0

cause it faster and much Less I/O,supported operators are ["=", "==", "<>", ">", ">=", "<", "<="]

Example for Sharing Variables

#include <Array.au3>
#include <MultiTasking.au3>

MsgBox(0,"Main Process","Lets Assign some shared variables "&@crlf&"My PID is :"&@AutoItPID)

_Task_SetVar("Shared_Var",0)
local $temparray[4]=[31,32,"String",True]
_Task_SetArray("Shared_Array",$temparray)


_Task_Create("Task1","GetSharedValues")
_Task_join("Task1")


msgbox(0,"Main Process","now we will start 5 Tasks of function 'RacingWrite_EXP'  same time which use _Task_SetVar , each one will loop 10 times, each loop increase 'shared_value' by 1")
MsgBox(0,"Main Process","So 'Shared_Value' should equal : 5 process * 10 loops = 50 "&@CRLF&"pls Check 'RacingWrite_EXP' function")
local $PIDarray[5]
for $i=0 to 4
$PIDarray[$i]=_Task_Create("","RacingWrite_EXP")
next
_Task_join($PIDarray)
MsgBox(0,"Main process","Tasks Finished But 'Shared_Value' = "&_Task_GetVar("Shared_Var"))


MsgBox(0,"Main process","Now lets Set 'Shared_value' to zero again and do the same but this time useing function 'SafeWrite_EXP' Which use _TasK_SetVarEX ,pls Check 'SafeWrite_EXP' function")
_Task_SetVar("Shared_Var",0)



local $PIDarray[5]
for $i=0 to 4
$PIDarray[$i]=_Task_Create("","SafeWrite_EXP")
next
_Task_join($PIDarray)
MsgBox(0,"Main process","Tasks Finished and Now 'Shared_Value' = "&_Task_GetVar("Shared_Var"))

MsgBox(0,"Main process","Finaly i will start 'Sync_EXP' function and will calculate how much time till 'Shared_Value' will Equal 100 ,pls Check 'Sync_EXP' function")
$hTimer=TimerInit()
_Task_Create("","Sync_EXP")
_Task_PauseUntilVar("Shared_Var","=",100)                       ;======>supported operator ["=", "==", "<>", ">", ">=", "<", "<="]
MsgBox(0,"Main process","Shared_Value Reached 100 in "&TimerDiff($hTimer))

MsgBox(0,"Main process","Thank you and Good bye")


func GetSharedValues()
    local $str="I'M Task with PID  : "&@AutoItPID&" lets retrieve those variables"&@CRLF
    $str&="Shared_Var ="&_Task_GetVar("Shared_Var")&@CRLF
    $str&="Shared_Array[0] ="&_Task_GetArrayElemenT("Shared_Array",0)&@CRLF
    $str&="Shared_Array[1] ="&_Task_GetArrayElemenT("Shared_Array",1)&@CRLF
    $str&="Shared_Array[2] ="&_Task_GetArrayElemenT("Shared_Array",2)&@CRLF
    $str&="Shared_Array[3] ="&_Task_GetArrayElemenT("Shared_Array",3)&@CRLF
    MsgBox(0,"Task1",$str)

    MsgBox(0,"Task1","Now we will Change Shared_Array[3] to :False  and will show all Array with _ArrayDispaly()" )
    _Task_SetArrayElement("Shared_Array",false,3)       ;====> false alway converted to 0
    local $temparray=_Task_GetArray("Shared_Array")
    _ArrayDisplay($temparray)

EndFunc

Func RacingWrite_EXP()

    for $i =0 to 9
    $tempint=_Task_GetVar("Shared_Var")
    _Task_SetVar("Shared_Var",$tempint+1)
    next

EndFunc

Func SafeWrite_EXP()

for $i =0 to 9
    _TasK_SetVarEX("Shared_Var","+=",1) ;===>supported opreators=["+=", "-=", "/=", "*=", "&="]
    next

endfunc

func Sync_EXP()
    sleep(3000)
    _Task_SetVar("Shared_Var","100")
    sleep(10000)
    endfunc

Download :- Attached File contain MultiTasking(UDF), and single task ping script vs multi task ping script.

Finally ,feel free to give suggestion , report bug ,or ask question

AutoIT-MultiTasking-UDF.rar

Edited August 2, 2019 by Network_Guy
some code optimization

joseLB · January 15, 2019

I didn´t test it yet, but seems to be great. I have some additional questions:

I suppose for a real function to be running continuously we need to do a loop inside it, correct? With some sleep() to not consume all cpu.
a function can call another one ?
probably global variables, delared in main program and that compiles oki, can not be used in the functions.
Do you have a clue on the performance of _Task_SetVar, get, array, arrayElement? I mean, probably is not a good idea to use them to Exchange lots of data or frequently changing data, or to use _Task_GetVar in a tight loop to check if there is any data to process, correct?

Thanks

Jose

Edited January 15, 2019 by joseLB

Network_Guy · January 16, 2019

14 hours ago, joseLB said:

I didn´t test it yet, but seems to be great. I have some additional questions:

I suppose for a real function to be running continuously we need to do a loop inside it, correct? With some sleep() to not consume all cpu.

a function can call another one ?

probably global variables, delared in main program and that compiles oki, can not be used in the functions.

Do you have a clue on the performance of _Task_SetVar, get, array, arrayElement? I mean, probably is not a good idea to use them to Exchange lots of data or frequently changing data, or to use _Task_GetVar in a tight loop to check if there is any data to process, correct?

Thanks

Jose

yes ,

if u mean calling function from created task then yes , but if u want to create task from a task then currently no to prevent process loops but it can be done in future .

#include <MultiTasking.au3>
_Task_Create("Task1","Working_Function")
_Task_Join("Task1")
_Task_Create("Task2","Not_Working_function")

func ToBeCalled()
Msgbox(0,"Task"," Working !")
endfunc

func Working_Function()
TobeCalled()
endfunc

func Not_Working_function()
_Task_Create("","ToBeCalled")
endfunc

To prevent process loops ,all main script code including the declared variables is ignored by Task . so u need to declare desired global variables again inside the task function with global keyword .(those global variables not shared between Main script & Tasks)

#include <MultiTasking.au3>

global $MyGlobal_Var="Global Variable Used by Main only ."

_Task_Create("Task1","Example")
MyGlobal_Var()

func Example()
global $MyGlobal_Var="Global Variable Used by Task1 only ."
MyGlobal_Var()
endfunc

func MyGlobal_Var()
msgbox(0,"Task","Task :"&$MyGlobal_Var)
endfunc

yes ,u need to decrease input & output (set or get ) as possible for better performance , u can neglect data size effect on performance , finally looping for variable u can use such code but it decrease performance alot cause u keep asking for "SaredVariable" value
```
while 1
if _TasK_GetVar("SharedVariable")<>"" then 
;do work
_TasK_SetVar("SharedVariable","")
endif
wend
```
u should instead use _Task_PauseUntilVar() which will just register an event once then pause the script untill event condition is true instead keep asking for "SharedVariable"
```
while 1
_Task_PauseUntilVar("SharedVariable","<>","")  ; event condtion= $sharedvariable <> ""
;do work
_TasK_SetVar("SharedVariable","")
wend
```

Dustbeen43 · May 16, 2019

Thanks for the work NetworkGuy.

I have a question for you. Is it possible to see ConsoleWrite() of the children's process in the main one ? To see actually what's going on under the wood I use log files. One for each children but it's not very easy to debug that way.

Thanks in advance.

Network_Guy · May 16, 2019

49 minutes ago, Dustbeen43 said:

Thanks for the work NetworkGuy.

I have a question for you. Is it possible to see ConsoleWrite() of the children's process in the main one ? To see actually what's going on under the wood I use log files. One for each children but it's not very easy to debug that way.

Thanks in advance.

Sure you can use PID from _Task_Create with stdoutread,stdinwrite,stderrRead, example:-

#include <MultiTasking.au3>

$PID=_Task_Create("Task1","Read_STD_Example")

_TasK_Join($PID)

$taskoutput=StdoutRead($PID)

MsgBox(0,"MainTask","Console out for Task1 :-"&@CRLF&$taskoutput)

func Read_STD_Example()
    for $i=1 to 10
    ConsoleWrite($i&@CRLF)
    next
EndFunc

Dustbeen43 · May 16, 2019

Thanks for your answer but I think I have missed something.

I'm trying to figure out how to manage threads with your libs.

I've made a script that represent two squads. Both squads are task and the main has no goal but to wait for the end of the story of both squads. One must cover the other and the other open the road to the first one. I use shared variable to communicate between the two squad.

The issue is that I log every moves and decisions in separate logs but both files are created but empty. I don't know where I have wrong.

I think I'm really close to have some pretty communications

JIJOE.au3

Network_Guy · May 16, 2019

@Dustbeen43 i checked your script , u forgot to include <File.au3> and variables u used in main program $logFileTango , $logFileCharly not defined in tasks and the only way to share variables by use _Task_SetVar or replace variable by its value like :-
_FileWriteLog(@ScriptDir & "\Tango.log", "my message")

finaly i dont know when both task will pass the first do ..... until , i didnt notic any thing changing threadTwinTango,threadTwinCharly to 0

Dustbeen43 · May 16, 2019

#include..... my bad. I've started to used AutoIt two days ago, I haven't all the reflex yet

But one little sentence makes no sense to me (at least for now) :

28 minutes ago, Network_Guy said:

finaly i dont know when both task will pass the first do ..... until , i didnt notic any thing changing threadTwinTango,threadTwinCharly to 0

So that means when I declared in the main

_Task_SetVar("threadTwinCharly",0)

And in Tango I set the value :

_Task_SetVar("threadTwinTango",1)

In Charly, when I get the value like :

_Task_GetVar("threadTwinCharly")

The result is always 0 until I change the value directly in Charly. But Tango won't see I've changed the value as well.

If I understand correctly the result I have it means that the variable is in fact not shared because each thread as a different value of the shared variable.

So (I think you have understand where I'm going) how can I shared one variable between thread like :

I declare the variable in main with 0

I set in thread_1 the variable to 1

and see in the thread_2 the value newly set 1 instead of 0 ?

Edited May 16, 2019 by Dustbeen43

Network_Guy · May 16, 2019

here is your script but :-
1- include file.au3

2- replace [$logFileTango] to [@ScriptDir & "\Tango.log"] and [$logFileCharly] to [@ScriptDir & "\Charly.log"] in all script lines

3- comment this code:-

Global Const $logFileTango = FileOpen(@ScriptDir & "\Tango.log", 1)
Global Const $logFileCharly = FileOpen(@ScriptDir & "\Charly.log", 1)

FileClose($logFileTango)
FileClose($logFileCharly)

after running it , the result is :-

Charly.log

2019-05-16 21:47:47 : Task : 1648Hello I'm Charly
I Executing TwinCharly
And my parent Process PID is :1676
2019-05-16 21:47:47 : Task : 1648 | I'm waiting TwinTango 
2019-05-16 21:47:48 : Task : 1648 | I'm waiting TwinTango 
2019-05-16 21:47:49 : Task : 1648 | I'm waiting TwinTango 
2019-05-16 21:47:50 : Task : 1648 | I'm waiting TwinTango 
2019-05-16 21:47:51 : Task : 1648 | I'm waiting TwinTango 
2019-05-16 21:47:52 : Task : 1648 | I'm waiting TwinTango 
2019-05-16 21:47:53 : Task : 1648 | I'm waiting TwinTango 
2019-05-16 21:47:54 : Task : 1648 | I'm waiting TwinTango 
2019-05-16 21:47:55 : Task : 1648 | I'm waiting TwinTango

Tang.log

2019-05-16 21:47:47 : Task : 6768Hello I'm Tango
I Executing Twintango
And my parent Process PID is :1676
2019-05-16 21:47:47 : Task : 6768 | I'm waiting TwinCharly
2019-05-16 21:47:48 : Task : 6768 | I'm waiting TwinCharly
2019-05-16 21:47:49 : Task : 6768 | I'm waiting TwinCharly
2019-05-16 21:47:50 : Task : 6768 | I'm waiting TwinCharly
2019-05-16 21:47:51 : Task : 6768 | I'm waiting TwinCharly
2019-05-16 21:47:52 : Task : 6768 | I'm waiting TwinCharly
2019-05-16 21:47:54 : Task : 6768 | I'm waiting TwinCharly

and from your code that is the expected result , cause :-

main script set threadTwinCharly , threadTwinTango = 0 then starting 2 task at same time
task charly set threadTwinCharly to 1 and task tango set threadTwinTango to 1 in the same time

they both will stuck in the do until loop cause threadTwinCharly , threadTwinTango will always be 1.

Task Charly:-
Do
      _FileWriteLog($logFileCharly, "Task : "&@AutoItPID & " | I'm waiting TwinTango")
      Sleep(1000)
Until _Task_GetVar("threadTwinTango") <> 1



Task Tango:-
 Do
      _FileWriteLog($logFileTango, "Task : "&@AutoItPID & " | I'm waiting TwinCharly")
      Sleep(1000)
 Until _Task_GetVar("threadTwinCharly") <> 1

JIJOE.au3

Dustbeen43 · May 16, 2019

I have made the changes too. And I'm glad to see I've made the same.

Concerning this :

2 minutes ago, Network_Guy said:

they both will stuck in the do until loop cause threadTwinCharly , threadTwinTango will always be 1.

You haven't understand my script. ThreadTwinCharly and Tango are initialized at 0 in the main script. When I create each task, after the intialisation tasks, I set each variable to 1 and they are both waiting the other thread to be ready (checking = 1). If they are = to 1 then the loop :

Do
      _FileWriteLog($logFile, "Task : "&@AutoItPID & " | I'm waiting TwinTango")
      Sleep(1000)
   Until _Task_GetVar("threadTwinTango") = 0

Must end because the variable isn't 0 anymore but 1... But it doesn't work that way apparently.

I post my code so you can see where I am now.

JIJOE.au3

Network_Guy · May 16, 2019

can u post what is your desired result and what u currently get cause last File not working at all

Dustbeen43 · May 16, 2019

Yeah my script is still not working properly ^^'

This is what's missing :

I just want to share data between two threads.

T1 and T2 has access to a shared variable.

T1 modify the variable and T2 can read all the changes T1 is doing.

You don't have to do it in my script. Just post an working example I will handle the rest and post my script as an other example for other users.

Thanks in advance. You help me a lot and I'm fully aware of that. 🙂

Network_Guy · May 16, 2019

please check this exp

EXP2.au3

Dustbeen43 · May 17, 2019

I just realize the file you gave me is the EXP2 in the archive I feel bad. I'm looking into it. Thanks for your patience.

Dustbeen43 · May 17, 2019

Ok I get it ! Now I know where my problems are.

See the script attached. I have a writer and a reader. It seams that Task_Join is snot waiting enough. It's acting like if the _Task_Join is waiting only 1 seconds and if the children don't answer the parent call, he just kill the children not matter what.

As a proof, modify the sleep in the WriteExp() function.

If you set 600, the result is OK (log files contains what I want) and the execution time is : 1.422s

If you set 700, the result is not OK (log files contains what I want) and the execution time is : 1.409s

If you set 1000, the result is not OK (log files don't contains what I want) and the execution time is : 1.429s

It bother to me because in C# (I'm using in my job) the Join has not the same meaning. Here is a sentence that illustrate my remark :

source (https://www.geeksforgeeks.org/joining-threads-in-c-sharp/)

Quote

In C#, Thread class provides the Join() method which allows one thread to wait until another thread completes its execution. If t is a Thread object whose thread is currently executing, then t.Join() causes the current thread to pause its execution until thread it joins completes its execution.

test2.au3

Edited May 17, 2019 by Dustbeen43

Network_Guy · May 17, 2019

you are right about _Task_Join Method and thanks for reporting it and sry for trouble , here is the fixed UDF :-

MultiTasking.au3

Dustbeen43 · May 17, 2019

No problem and thanks mate it works like a charm now. I can continue on my own.

Dustbeen43 · May 18, 2019

For people who wants to haver another example of the utility of this lib, here is my script : "GIJOE".

I'am launching 2 threads and both threads has its own procedure and they are communicating with shared variables.

The logic here is that the Thread 1 (Charly) is covering the back of the Thread 2 (Tango). If T1 is in combat then he changed his variable to say that he is in combat (so T2 is paused during the fight and he is resumed after the end of the combat). T1 has 6 HP ( while $loop < 7 but you can change it ) so if thread 2 has not finished before the secund fight (one fight every 3 rounds) then the thread 1 wil stopped. In that case T2 has to stopped because T1 has stopped.

The Thead 2 (Tango) is the twin of the first one. He's goal is to arrive in a destination which is at a distance of 9 ( while $loop < 10 but you can change it ) so if the T2 arrives before the thread 1 then T2 stops too.

With the original script, T1 will die before T2 but you can change the condition of both while to see the 2 different case.

NB : each script has it's own logfile so you can see actions both thread are doing.

NB2: "MultiTasking.au3" and its dependancies must be in the same directory of the file.

I want to thanks Network_guy for this lib and trancexx wich provide parts of this libs too.

GIJOE.au3

Multi-Tasking UDF

Recommended Posts

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Create an account or sign in to comment

Create an account

Sign in

Recently Browsing 0 members