[Solved] StdOutRead failing to read anything. No error but read 0 bytes.

EmilyLove · March 25, 2019

As the title says. I am trying to read the console window of Rclone.exe. I'm trying to get verbose stats, like files in transfer, their progress, errors, and other stuff. I've built a parser that works in practice on RegExr website. Unfortunately, I don't seem to be reading the console correctly. The Rclone console shows up in AutoIt log as if ConsoleWrite was used, but StdOutRead always succeeds to read, but it is always 0 bytes. Since it can't read the data correctly, the parser has nothing to work with.

TLDR: StdOutRead not reading anything. It succeeds, but 0 bytes read. Rclone console, the window I'm trying to read with StdOutRead, logs in AutoIt as if ConsoleWrite was used.

debug environment:

Download Rclone.exe and set in the same directory as script or in %Windir%\system32.
Run "Rclone.exe config" in cmd.exe. Create 2 remotes. "GDrive" (Google Drive option 12) & "GCache" (Cache option 7).
I: should not be used as a mount point. Can be changed by editing line 7

Test.au3

#include <Array.au3>
#include <AutoItConstants.au3>
#include <MsgBoxConstants.au3>
#include <StringConstants.au3>

;Run RClone Cache Mount on I:
Global $iPID_RClone = Run(@ComSpec & " /k rclone.exe mount --vfs-cache-mode full --vfs-cache-max-age 0h1m0s --vfs-cache-poll-interval 1m0s --cache-db-purge --cache-writes --cache-workers 16 --allow-other --acd-templink-threshold 0 --stats 1s --buffer-size 1G --low-level-retries 3600000 --retries 3600000 --retries-sleep 1ms --timeout 5s --contimeout 5s --drive-acknowledge-abuse --drive-skip-gdocs -v gcache:/ I:", "", @SW_SHOW, $STDOUT_CHILD)
OnAutoItExitRegister("_OnExit")
AdlibRegister("_Rclone_Read")

;Main Idle loop
While 1
    Sleep(60000)
WEnd

;This function gets ran when the script is exiting.
Func _OnExit()

    ;Unmount RClone from System before exiting.
    WinClose("[CLASS:ConsoleWindowClass]")

EndFunc   ;==>_OnExit

;This function parses the Rclone window, providing stats about what it is currently doing.
Func _Rclone_Parse()
    Local $sOutput = _Rclone_Read()


EndFunc   ;==>_Rclone_Parse

Func _Rclone_Read()
    ;debug
    ConsoleWrite("Read Function Executed!" & @CRLF)

    ; Read current stream.
    Local $sOutput = StdoutRead($iPID_RClone)

    ; Errors when nothing to do.
    If @error Then
        ConsoleWrite("Read Failed! ERROR!" & @CRLF)
        ;Return
    EndIf

    ;debug
    ConsoleWrite("Bytes read: " & @extended & @CRLF)

    ;Transferred
    ConsoleWrite("Transferred1 Start!" & @CRLF)
    Local $aTransferred1 = StringRegExp($sOutput, "^Transferred:[\s]*([\w\d\.]+[\s]*\/[\s]*[\d\.]+[\s]*[\w\/]+),[\s]*([\d\.\%\-]+),[\s]*([\d\.]+[\s]*[\w\/]+),[\s]*[\w]+[\s]*[\w\d\-]+$", $STR_REGEXPARRAYMATCH)

    ;debug
    Switch @error
        Case 0
            ConsoleWrite("Array is valid." & @CRLF)
            ConsoleWrite("Offset: " & @extended & @CRLF)
            _ArrayDisplay($aTransferred1)
        Case 1
            ConsoleWrite("Array is invalid. No matches." & @CRLF)
        Case 2
            ConsoleWrite("Bad pattern, array is invalid." & @CRLF)
            ConsoleWrite("Offset: " & @extended & @CRLF)
    EndSwitch

    ;Errors
    ConsoleWrite("Errors" & @CRLF)
    Local $aErrors = StringRegExp($sOutput, "^(Errors:)[\s]*([\d\.]+)", $STR_REGEXPARRAYMATCH)

    ;debug
    Switch @error
        Case 0
            ConsoleWrite("Array is valid." & @CRLF)
            ConsoleWrite("Offset: " & @extended & @CRLF)
            _ArrayDisplay($aErrors)
        Case 1
            ConsoleWrite("Array is invalid. No matches." & @CRLF)
        Case 2
            ConsoleWrite("Bad pattern, array is invalid." & @CRLF)
            ConsoleWrite("Offset: " & @extended & @CRLF)
    EndSwitch

    ;Checks
    ConsoleWrite("Checks" & @CRLF)
    Local $aChecks = StringRegExp($sOutput, "^(Checks:)[\s]*([\d\.]+)[\s]*\/[\s]*([\d\.]+),[\s]*([\w\d\-]+)", $STR_REGEXPARRAYMATCH)

    ;debug
    Switch @error
        Case 0
            ConsoleWrite("Array is valid." & @CRLF)
            ConsoleWrite("Offset: " & @extended & @CRLF)
            _ArrayDisplay($aChecks)
        Case 1
            ConsoleWrite("Array is invalid. No matches." & @CRLF)
        Case 2
            ConsoleWrite("Bad pattern, array is invalid." & @CRLF)
            ConsoleWrite("Offset: " & @extended & @CRLF)
    EndSwitch

    ;Transferred
    ConsoleWrite("Transferred2" & @CRLF)
    Local $aTransferred2 = StringRegExp($sOutput, "^(Transferred:)[\s]*([\d.]+)[\s]*\/[\s]*([\d.]+),[\s]*([\d\%\-]+)", $STR_REGEXPARRAYMATCH)

    ;debug
    Switch @error
        Case 0
            ConsoleWrite("Array is valid." & @CRLF)
            ConsoleWrite("Offset: " & @extended & @CRLF)
            _ArrayDisplay($aTransferred2)
        Case 1
            ConsoleWrite("Array is invalid. No matches." & @CRLF)
        Case 2
            ConsoleWrite("Bad pattern, array is invalid." & @CRLF)
            ConsoleWrite("Offset: " & @extended & @CRLF)
    EndSwitch

    ;Elapsed Time
    ConsoleWrite("Elasped Time" & @CRLF)
    Local $aTime = StringRegExp($sOutput, "^(Elapsed time:)[\s]*([\w\d\.]+)", $STR_REGEXPARRAYMATCH)

    ;debug
    Switch @error
        Case 0
            ConsoleWrite("Array is valid." & @CRLF)
            ConsoleWrite("Offset: " & @extended & @CRLF)
            _ArrayDisplay($aTime)
        Case 1
            ConsoleWrite("Array is invalid. No matches." & @CRLF)
        Case 2
            ConsoleWrite("Bad pattern, array is invalid." & @CRLF)
            ConsoleWrite("Offset: " & @extended & @CRLF)
    EndSwitch

    ;Files in Transfer
    ConsoleWrite("Files" & @CRLF)
    Local $aFiles = StringRegExp($sOutput, "^[\s\*]*([\s\w\d\!\@\#\$\%\^\&\(\)\-\=\_\+\/\`\~\'\;\,\.]+):[\s]*([\d\%]+)[\s]*\/[\s]*([\w\d\.]+),[\s]*([\w\d\.\/]+),[\s]*([\w\d]+)", $STR_REGEXPARRAYGLOBALMATCH)

    ;debug
    Switch @error
        Case 0
            ConsoleWrite("Array is valid." & @CRLF)
            ConsoleWrite("Offset: " & @extended & @CRLF)
            _ArrayDisplay($aFiles)
        Case 1
            ConsoleWrite("Array is invalid. No matches." & @CRLF)
        Case 2
            ConsoleWrite("Bad pattern, array is invalid." & @CRLF)
            ConsoleWrite("Offset: " & @extended & @CRLF)
    EndSwitch


    ;debug
    ConsoleWrite(@CRLF)
    ConsoleWrite(@CRLF)
EndFunc   ;==>_Rclone_Read

AutoIt Log:

>"C:\Program Files (x86)\AutoIt3\SciTE\..\AutoIt3.exe" "C:\Program Files (x86)\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.au3" /run /prod /ErrorStdOut /in "C:\Users\BetaL\Desktop\test.au3" /UserParams    
+>18:03:55 Starting AutoIt3Wrapper v.19.102.1901.0 SciTE v.4.1.2.0   Keyboard:00000409  OS:WIN_10/  CPU:X64 OS:X64  Environment(Language:0409)  CodePage:0  utf8.auto.check:4
+>         SciTEDir => C:\Program Files (x86)\AutoIt3\SciTE   UserDir => C:\Users\BetaL\AppData\Local\AutoIt v3\SciTE\AutoIt3Wrapper   SCITE_USERHOME => C:\Users\BetaL\AppData\Local\AutoIt v3\SciTE 
>Running AU3Check (3.3.14.5)  from:C:\Program Files (x86)\AutoIt3  input:C:\Users\BetaL\Desktop\test.au3
+>18:03:56 AU3Check ended.rc:0
>Running:(3.3.14.5):C:\Program Files (x86)\AutoIt3\autoit3.exe "C:\Users\BetaL\Desktop\test.au3"    
+>Setting Hotkeys...--> Press Ctrl+Alt+Break to Restart or Ctrl+Break to Stop
Read Function Executed!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


Read Function Executed!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


2019/03/25 18:03:57 INFO  : gcache: Cache DB path: C:\Users\BetaL\AppData\Local\rclone\cache-backend\gcache.db
2019/03/25 18:03:57 INFO  : gcache: Cache chunk path: C:\Users\BetaL\AppData\Local\rclone\cache-backend\gcache
Read Function Executed!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


2019/03/25 18:03:57 INFO  : gcache: Chunk Memory: true
2019/03/25 18:03:57 INFO  : gcache: Chunk Size: 5M
2019/03/25 18:03:57 INFO  : gcache: Chunk Total Size: 10G
2019/03/25 18:03:57 INFO  : gcache: Chunk Clean Interval: 1m0s
2019/03/25 18:03:57 INFO  : gcache: Workers: 16
2019/03/25 18:03:57 INFO  : gcache: File Age: 2d
2019/03/25 18:03:57 INFO  : gcache: Cache Writes: enabled
Read Function Executed!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


Read Function Executed!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


Read Function Executed!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


Read Function Executed!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


2019/03/25 18:03:58 INFO  : 
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 0
Checks:                 0 / 0, -
Transferred:            0 / 0, -
Elapsed time:          1s

Read Function Executed!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


Read Function Executed!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


Read Function Executed!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


Read Function Executed!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


2019/03/25 18:03:59 INFO  : 
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 0
Checks:                 0 / 0, -
Transferred:            0 / 0, -
Elapsed time:          2s

Read Function Executed!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


Read Function Executed!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


Read Function Executed!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


2019/03/25 18:04:00 INFO  : 
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 0
Checks:                 0 / 0, -
Transferred:            0 / 0, -
Elapsed time:          3s

Read Function Executed!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


Read Function Executed!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


Read Function Executed!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


Read Function Executed!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


2019/03/25 18:04:01 INFO  : 
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 0
Checks:                 0 / 0, -
Transferred:            0 / 0, -
Elapsed time:          4s

Read Function Executed!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


Read Function Executed!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


Read Function Executed!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


Read Function Executed!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


2019/03/25 18:04:02 INFO  : 
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 0
Checks:                 0 / 0, -
Transferred:            0 / 0, -
Elapsed time:          5s

2019/03/25 18:04:03 INFO  : 
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 0
Checks:                 0 / 0, -
Transferred:            0 / 0, -
Elapsed time:          6s

Read Function Executed!
Read Failed! ERROR!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


^CThe process tried to write to a nonexistent pipe.
Read Function Executed!
Read Failed! ERROR!
Bytes read: 0
Transferred1 Start!
Array is invalid. No matches.
Errors
Array is invalid. No matches.
Checks
Array is invalid. No matches.
Transferred2
Array is invalid. No matches.
Elasped Time
Array is invalid. No matches.
Files
Array is invalid. No matches.


+>18:04:03 AutoIt3.exe ended.rc:0
+>18:04:03 AutoIt3Wrapper Finished.
>Exit code: 0    Time: 8.886

test.au3

Edited March 26, 2019 by BetaLeaf

FrancescoDiMuro · March 26, 2019

8 hours ago, BetaLeaf said:

; Read current stream. Local $sOutput = StdoutRead($iPID_RClone)

You should read this in a While 1 loop, and check for error (no more lines to read).

Something like:

While 1
    $sOutput &= STDOutRead($iPID)
    If @error Then ExitLoop
WEnd

All the other errors appear because there is no data in the variable $sOutput.

Apply these changes, and try to display the content of the $sOutput variable

TheDcoder · March 26, 2019

@BetaLeaf I also recommend using $STDERR_MERGED with Run to capture the standard error stream as well, otherwise you would not get anything which might be written to STDERR

Nine · March 26, 2019

1 hour ago, FrancescoDiMuro said:
You should read this in a While 1 loop, and check for error (no more lines to read).

Something like:
While 1
    $sOutput &= STDOutRead($iPID)
    If @error Then ExitLoop
WEnd
All the other errors appear because there is no data in the variable $sOutput.

Apply these changes, and try to display the content of the $sOutput variable

I agree you will need to have some sort of a loop to catch all the data. But I don't believe you will receive an @error when there is no more data. Typically you get an error when the child is ending. Since your child (rclone.exe) is always running, you will need to find a way to know when it has finished sending. It can be a special ending character or string, or it might be because it has stop to send for a certain duration...I have tested it on a very small parent-child IPC, and if the child is still up and running, you don't get an error.

TheDcoder · March 26, 2019

1 hour ago, Nine said:

I agree you will need to have some sort of a loop to catch all the data. But I don't believe you will receive an @error when there is no more data. Typically you get an error when the child is ending.

This is correct, to be more accurate, StdoutRead sets the error if EOF has been reached, meaning the stream has been closed and there is no more data available. The application can still keep the stream open but the buffer might be empty, in this case you can check @extended for the number of bytes read, which would be 0 if the buffer is empty. Here is an ideal loop:

While True
    Sleep(10) ; Don't kill the CPU
    $sOutput &= StdOutRead($iPID)
    If @extended = 0 Then
        ContinueLoop
    ElseIf @error Then
        ExitLoop
    EndIf
WEnd

Nine · March 26, 2019

Nah, the perfect loop is this one :

$sOutput = ""
While True
    Sleep(10) ; Don't kill the CPU
    $sOutput &= StdOutRead($iPID)
    If @error Then Return SetError (1,0,0)
    If StringRight ($sOutput, ??) = "Something" Then ExitLoop  ; this is the tricky part, you need to find a way to know <End of Transmission>
WEnd

It can be any condition that will mark the end of transmission from rclone.exe

Edited March 26, 2019 by Nine

EmilyLove · March 26, 2019

Thanks for the information everyone. I got it working now.

1) Needed EOF detection. The StringRegExp for Elasped Time served this purpose.

2) Needed a proper loop. AdLibRegister caused undesirable nesting. While 1 solves this.

3) A few reg expression were invalid. Had to removing ^ and $ at the start and end.

4) Cleaned up the script. Still a WIP tho.

#include <Array.au3>
#include <AutoItConstants.au3>
#include <MsgBoxConstants.au3>
#include <StringConstants.au3>

;Run RClone Cache Mount on I:
Global $iPID_RClone = Run(@ComSpec & " /k rclone.exe mount --vfs-cache-mode full --vfs-cache-max-age 0h1m0s --vfs-cache-poll-interval 1m0s --cache-db-purge --cache-writes --cache-workers 16 --allow-other --acd-templink-threshold 0 --stats 1s --buffer-size 1G --low-level-retries 3600000 --retries 3600000 --retries-sleep 1ms --timeout 5s --contimeout 5s --drive-acknowledge-abuse --drive-skip-gdocs -v gcache:/ I:", "", @SW_SHOW, $STDERR_MERGED)
OnAutoItExitRegister("_OnExit")

;Main Idle loop
While 1
    Sleep(100)
    _Rclone_Read()
WEnd

;This function gets ran when the script is exiting.
Func _OnExit()
    ;Unmount RClone from System before exiting.
    WinClose("[CLASS:ConsoleWindowClass]")
EndFunc   ;==>_OnExit

;Read the Rclone stream.
Func _Rclone_Read()
    Local $sOutput, $iCountLoop

    While 1
        ; Read current stream.
        $sOutput &= StdoutRead($iPID_RClone)

        ;Elapsed Time
        Local $aTime = StringRegExp($sOutput, "(Elapsed time:)[\s]*([\w\d\.]+)", $STR_REGEXPARRAYMATCH)

        If Not @error Then
            ExitLoop
        EndIf

        ;Sleep so we don't overwork the cpu
        Sleep(10)

        ;debug
        $iCountLoop += 1
    WEnd

    ;Transferred
    Local $aTransferred1 = StringRegExp($sOutput, "Transferred:[\s]*([\w\d\.]+[\s]*\/[\s]*[\d\.]+[\s]*[\w\/]+),[\s]*([\d\.\%\-]+),[\s]*([\d\.]+[\s]*[\w\/]+),[\s]*[\w]+[\s]*[\w\d\-]+", $STR_REGEXPARRAYMATCH)
    _Rclone_Read_Debug_Check("Transferred 1", $aTransferred1)

    ;Errors
    Local $aErrors = StringRegExp($sOutput, "(Errors:)[\s]*([\d\.]+)", $STR_REGEXPARRAYMATCH)
    _Rclone_Read_Debug_Check("Errors", $aErrors)

    ;Checks
    Local $aChecks = StringRegExp($sOutput, "(Checks:)[\s]*([\d\.]+)[\s]*\/[\s]*([\d\.]+),[\s]*([\w\d\-]+)", $STR_REGEXPARRAYMATCH)
    _Rclone_Read_Debug_Check("Checks", $aChecks)

    ;Transferred
    Local $aTransferred2 = StringRegExp($sOutput, "(Transferred:)[\s]*([\d.]+)[\s]*\/[\s]*([\d.]+),[\s]*([\d\%\-]+)", $STR_REGEXPARRAYMATCH)
    _Rclone_Read_Debug_Check("Transferred 2", $aTransferred2)

    ;Elapsed Time
    Local $aTime = StringRegExp($sOutput, "(Elapsed time:)[\s]*([\w\d\.]+)", $STR_REGEXPARRAYMATCH)
    _Rclone_Read_Debug_Check("Elasped Time", $aTime)

    ;Files in Transfer
    Local $aFiles = StringRegExp($sOutput, "[\s\*]*([\s\w\d\!\@\#\$\%\^\&\(\)\-\=\_\+\/\`\~\'\;\,\.]+):[\s]*([\d\%]+)[\s]*\/[\s]*([\w\d\.]+),[\s]*([\w\d\.\/]+),[\s]*([\w\d]+)", $STR_REGEXPARRAYGLOBALMATCH) ;untested
    _Rclone_Read_Debug_Check("Files in Transfer", $aFiles)
    ConsoleWrite("Data Read: " & @CRLF & $sOutput & @CRLF)
    ConsoleWrite("EOF" & @CRLF & @CRLF)

EndFunc   ;==>_Rclone_Read

Func _Rclone_Read_Debug_Check($sName, $aData, $iError = @error, $iExtended = @extended)
    ConsoleWrite("Checking data for " & $sName & @CRLF)
    Switch $iError
        Case 0
            ConsoleWrite("Array is valid." & @CRLF)
            ConsoleWrite("Offset: " & $iExtended & @CRLF)
            _ArrayDisplay($aData)
        Case 1
            ConsoleWrite("Array is invalid. No matches." & @CRLF)
        Case 2
            ConsoleWrite("Bad pattern, array is invalid." & @CRLF)
            ConsoleWrite("Offset: " & $iExtended & @CRLF)
    EndSwitch
    ConsoleWrite("Checking data for " & $sName & " completed!" & @CRLF & @CRLF)
EndFunc   ;==>_Rclone_Read_Debug_Check

Edited March 26, 2019 by BetaLeaf

TheDcoder · March 27, 2019

If StringRight ($sOutput, ??) = "Something" Then ExitLoop  ; this is the tricky part, you need to find a way to know <End of Transmission>

@Nine This is nice to have, but not needed to detect EOT or EOL at all, since StdOutRead should set @error if the child has ended the transmission by closing the stream

Edited March 27, 2019 by TheDcoder

Nine · March 27, 2019

8 hours ago, TheDcoder said:
If StringRight ($sOutput, ??) = "Something" Then ExitLoop  ; this is the tricky part, you need to find a way to know <End of Transmission>
@Nine This is nice to have, but not needed to detect EOT or EOL at all, since StdOutRead should set @error if the child has ended the transmission by closing the stream

Unless you know something that I don't know, there is no way StdOutRead will set @error if the child process is still running. As you can see with BetaLeaf, he found a way to identify EOT by checking the string "Elapsed time". Then the script can exit the reading loop to continue its work. That is exactly what I was referring to.

TheDcoder · March 27, 2019

18 minutes ago, Nine said:

Unless you know something that I don't know, there is no way StdOutRead will set @error if the child process is still running.

Yes, it will set @error if the child process is still running, the way how it works is that the child process has a handle to write to stdout, if the child closes it, then StdOutRead will set @error because the stream has been closed.

The more technically accurate explanation is that, to read the stdout of a child process, you will have to use the ReadFile/ReadFileEx function. The child can use the CloseHandle function on its write handle without exiting, this will make the ReadFile function in the parent return an ERROR_BROKEN_PIPE error, which means that the other end of the "pipe" (write handle) has been closed. More details from the documentation:

Quote

If an anonymous pipe is being used and the write handle has been closed, when ReadFile attempts to read using the pipe's corresponding read handle, the function returns FALSE and GetLastError returns ERROR_BROKEN_PIPE.

That is how StdOutRead would detect EOT/EOL and set @error and how the child could still well be running

Though I should mentioned that it is rare for programs to close their stdout write handle without following up with an exit, but still a thing to note

29 minutes ago, Nine said:

As you can see with BetaLeaf, he found a way to identify EOT by checking the string "Elapsed time". Then the script can exit the reading loop to continue its work. That is exactly what I was referring to.

That is perfectly fine to do as well, I was just trying to provide a more generic loop, it is meant to be adjusted to your needs... like how in this case it might be a good idea to detect the elapsed time and exit the loop when it reaches 0

Nine · March 27, 2019

@TheDcoder Do you have a working example of using readfile that way ? I would be very curious of studying it.

TheDcoder · March 27, 2019

@Nine Yes, I do have an example

I learned all of this quite recently too, while porting one of my AutoIt projects to C, had to learn a lot about WinAPI. Here is the example in my actual code (which is part of a library):

// Source: https://github.com/DcodingTheWeb/liballium/blob/241fbf79cd709b1babed42a6c2cb2d1351844ce6/allium.c#L167

unsigned long bytes_read;
if (ReadFile(instance->stdout_pipe, buffer, 1, &bytes_read, NULL) == false || bytes_read == 0) return NULL;

// P.S The forum doesn't have highlighting for C :(

You can see that in the if statement I check if ReadFile has returned false (it will return false if there is an error), this condition will evaluate to true when the child process closes its write handle to the standard output.

A bit of warning if you actually want to study the whole function as an example, I have mixed POSIX and WinAPI functions with the help if #ifdef through out the code, so it can get confusing if you are not familiar...

Also sorry about hijacking your thread BetaLeaf :sweating:

EmilyLove · April 24, 2019

Thank you everyone for the additional information. I've learned a lot today.

Block all input without UAC	Save/Retrieve Images to/from Text	Monitor Management (VCP commands)
Tool to search in text (au3) files	Date Range Picker	Virtual Desktop Manager
Sudoku Game 2020	Overlapped Named Pipe IPC	HotString 2.0 - Hot keys with string
x64 Bitwise Operations	Multi-keyboards HotKeySet	Recursive Array Display
Fast and simple WCD IPC	Multiple Folders Selector	Printer Manager
GIF Animation (cached)	Screen Scraping	Multi-Threading Made Easy

Block all input without UAC	Save/Retrieve Images to/from Text	Monitor Management (VCP commands)
Tool to search in text (au3) files	Date Range Picker	Virtual Desktop Manager
Sudoku Game 2020	Overlapped Named Pipe IPC	HotString 2.0 - Hot keys with string
x64 Bitwise Operations	Multi-keyboards HotKeySet	Recursive Array Display
Fast and simple WCD IPC	Multiple Folders Selector	Printer Manager
GIF Animation (cached)	Screen Scraping	Multi-Threading Made Easy

Block all input without UAC	Save/Retrieve Images to/from Text	Monitor Management (VCP commands)
Tool to search in text (au3) files	Date Range Picker	Virtual Desktop Manager
Sudoku Game 2020	Overlapped Named Pipe IPC	HotString 2.0 - Hot keys with string
x64 Bitwise Operations	Multi-keyboards HotKeySet	Recursive Array Display
Fast and simple WCD IPC	Multiple Folders Selector	Printer Manager
GIF Animation (cached)	Screen Scraping	Multi-Threading Made Easy

Block all input without UAC	Save/Retrieve Images to/from Text	Monitor Management (VCP commands)
Tool to search in text (au3) files	Date Range Picker	Virtual Desktop Manager
Sudoku Game 2020	Overlapped Named Pipe IPC	HotString 2.0 - Hot keys with string
x64 Bitwise Operations	Multi-keyboards HotKeySet	Recursive Array Display
Fast and simple WCD IPC	Multiple Folders Selector	Printer Manager
GIF Animation (cached)	Screen Scraping	Multi-Threading Made Easy

[Solved] StdOutRead failing to read anything. No error but read 0 bytes.

Recommended Posts

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Create an account or sign in to comment

Create an account

Sign in

Recently Browsing 0 members