Looking for a clean way to capture RunWait output

[tremolux66]

Initial Problem

I've written several scripts with the following sequence:

1. Execute a program using Run w/stdout+stderr captured
   • Typically processes all the files in one directory tree to populate a second tree
2. Execute a second program (also with Run) to monitor the products of the first program and
   • Display a progress bar (percentage of output files complete)
   • Also monitor the first program's process and exit when it terminates
3. The script then calls ProcessWaitClose (no timeout) on the first program's process and
   • Checks the first program's results
   • Kills the monitor program if it hasn't already exited on its own.

Sometimes, ProcessWaitClose returns 1 with @error = 0 and @extended = 0xCCCCCCCC (actually, 0xFFFFFFFFCCCCCCCC), which seems ambiguous: the documentation says that @error = non-zero and @extended = 0xCC... means an invalid PID (unclear what the return value is), and 1 is returned for non-existent processes (but no mention of @extended). The 1/0/0xCC... result seems to occur when the first program exits very quickly (with or without an error). Since the exit value is not available, the script scans the program's output and tries to determine whether it ran successfully. This has gotten complicated and unreliable.

Partial Fix

I've now implemented a much simpler approach that works for most cases:

1. Modify the monitor program so that it ignores the other program's process (the monitor always gets killed by the script anyway)
2. Execute the monitor program first using Run, then execute the processing program with RunWait
3. When RunWait returns, the child process exit value is available, so the script can ignore its output (which isn't available anyway)
4. If the monitor program is still running, kill it.

Remaining Issue

However, there are still a couple of cases where it's necessary to get both the exit value from the processing program and its output. Since RunWait doesn't capture stdout and stderr for the parent script, it's looking like I'll have to call RunWait and redirect the 2 streams to a temp file and then scan it. Also, to do the redirect, I think I'll have to use @ComSpec to execute the processing program, which adds an undesired layer.

Does anybody have a better (cleaner) way to handle these cases?

[Earthshine]

i certainly would never implement it in three separate processes. I would call processing functions that used the Progress bar udf and scan output as necessary until done.

Edited December 18, 2017 by Earthshine

[Jos]

This is what I do in AutoIt3Wrapper when I understand your question correctly.
It shells the program, captures the output constandly and then gets the return/exit code from the process closed.

These are the lines I use for all programs I shell:

$Pid = Run('"program" /parameter', '', @SW_HIDE, $STDOUT_CHILD + $STDERR_CHILD)
$Handle = _ProcessExitCode($Pid)
$Return_Text = ShowStdOutErr($Pid)
$ExitCode = _ProcessExitCode($Pid, $Handle)
_ProcessCloseHandle($Handle)
StdioClose($Pid)

You ofcourse need to get the UDF/Funcs as well from the source.

Jos

Edited December 18, 2017 by Jos

[Earthshine]

I would test that code but can't find the udf's. bunches of process udf but none with those 

Edited December 18, 2017 by Earthshine

[Jos]

They are in AutoIt3Wrapper itself as they were written quite some time ago.

SciTE has this great tool Ctrl+j when the carrot is on the Funcname which will jump strait to it. 
Ctrl+Shift+j will jump back.

Jos 

[Earthshine]

derp! thanks! I am going to make that a handy little function I can call

[tremolux66]

@Jos: Thanks for the suggestion; I'll look into it. It was frustrating to lose the exit code, but the simple fix I outlined takes care of that for the exit-only cases. For the capture+exit cases, your approach may very well do the trick.

@Earthshine: The main reason the monitoring programs are separate is that they're pretty generic and get used in other scenarios as well. For example, the one that tracks the number of output files also gets run in the background from Bash scripts, as does another that tracks the size of an output file.

Note that in some cases, the processing program that's being executed is pretty simple (e.g., using Cygwin's cp command for a large recursive copy). In other cases, the processing is more complex and involves multiprocessing (e.g., transforming thousands of files using up to 2 dozen simultaneous instances (edit: "threads" but not really) over several dozen processors [within a single computer]).

 

Edited December 20, 2017 by tremolux66
Reworded a portion for clarity.

[tremolux66]

Never mind - I figured it out.

I extracted the following from AutoIt3Wrapper.au3 into a separate file:

• __ConsoleWrite()
• _ProcessCloseHandle()
• _ProcessExitCode()
• ShowStdOutErr()

I renamed the functions (inserted a prefix), added/edited comments, etc. and it looks like I'm good to go. Passes syntax-checking; on to building the system and unit-testing.

Thanks again.

Original message:

On 12/18/2017 at 1:34 PM, Jos said:

You of course need to get the UDF/Funcs as well from the source.

@Jos Does this mean I need to extract the functions I want to use (along with any other required code) from AutoIt3Wrapper.au3 into a separate file?

(As opposed to:  #include <...\AutoIt3Wrapper.au3>  ?)

I suspect the answer is "yes", but want to make sure I understand correctly.

Edited December 20, 2017 by tremolux66
Answered my own question.

[Jos]

6 hours ago, tremolux66 said:

Jos Does this mean I need to extract the functions I want to use (along with any other required code) from AutoIt3Wrapper.au3 into a separate file?

(As opposed to:  #include <...\AutoIt3Wrapper.au3>  ?)

That is what indeed would do.

Jos

[AdamUL]

You can also look in the Help File at the example for _WinAPI_GetExitCodeProcess.  These are almost exactly the same functions as the ones that Jos uses, but are included in the WinAPI libraries, under slightly different names, that is included with AutoIt.  

 

Adam

 

[tremolux66]

8 hours ago, AdamUL said:

You can also look in the Help File at the example for _WinAPI_GetExitCodeProcess.  These are almost exactly the same functions as the ones that Jos uses, but are included in the WinAPI libraries, under slightly different names, that is included with AutoIt.  

 

Adam

 

@AdamUL: Thanks for the pointer.

Lazy person that I am, I'll probably continue using the sample code I got from @Jos since I've already incorporated it into my scripts.