Stdoutread And Robocopy - How To Parse All The Info?

[CMarshy]

Alright Finished what I needed to do with the main changes made:-

• Seperated into functions
• 2 Progress bar ( File progressbar and Overall Progress Bar)
• Logging 4 log files 3 standard 1 Verbose(Progress bar) NoCopyLog.log is critical for overal progress bar
• Error checks before running (basic checks such as destination is not inside source, Valid drive paths, etc
• Disable of FormControls to ensure all form controls are used at the correct moment

Remider as previously stated the progress bar need decent size files inorder to work effectively, small files robocopy reports multiple percents at once so the bar will jump. or never appear to move if really small. Edited December 2, 2011 by CMarshy

[CMarshy]

expandcollapse popup

#include <GUIConstants.au3>
#include <file.au3>
#include<Constants.au3>
Global $FProgress
$GUI = GUICreate("Progress", 500, 200)
$SourceLabel = GUICtrlCreateInput("Select the Source Folder:", 10, 15, 380, 17)
$SourceBtn = GUICtrlCreateButton("...",400, 10, 25, 25)
$DestLabel = GUICtrlCreateInput("Select the Destination Folder:", 10, 40, 380, 17)
$DestBtn = GUICtrlCreateButton("...",400, 35, 25, 25)
GUICtrlCreateLabel("Current File: ", 10, 70)
$fileLbl = GUICtrlCreateLabel("N/A", 75, 70, 400)
GUICtrlCreateLabel("Current File Size: ", 10, 90)
$sizeLbl = GUICtrlCreateLabel("N/A", 95, 90, 105)
$progress = GUICtrlCreateProgress(10, 115, 480)
$Fullprogress = GUICtrlCreateProgress(10, 145, 480)
$GoBtn = GUICtrlCreateButton("GO", 205, 170, 40, 25)
$ExitBtn = GUICtrlCreateButton("EXIT", 275, 170, 40, 25)
$VerboseLogs=GUICtrlCreateCheckbox("Verbose Logging ON",25, 170, 150, 25)
_DisableFormControls()
GUISetState()
_EnableFormControl($SourceLabel)
_EnableFormControl($SourceBtn)
_EnableFormControl($ExitBtn)
_EnableFormControl($VerboseLogs)
$FProgress=0
$AppLogFile=@AppDataCommonDir &"AppLOG.LOG"
$NoCopyLog=@AppDataCommonDir &"NoCopyLOG.LOG"
$CopyLog=@AppDataCommonDir &"CopyLOG.LOG"
$ProcessBarLog=@AppDataCommonDir &"ProcessBarLOG.LOG"
Local $Destination, $Source, $tempOutput
_Writelog(1, "Application Launched")
While 1
$nMsg = GUIGetMsg()
Switch $nMsg
  Case $GUI_EVENT_CLOSE
  Case $SourceBtn
   _Writelog(1, "Source Button Clicked")
   $Source=_GetSourceDir()
   _Writelog(1, "Source set to" & $Source)
   GUICtrlSetData($SourceLabel,$Source)
   _Writelog(1, "GUI Source Input box updated to" & $Source)
   _EnableFormControl($DestLabel)
   _EnableFormControl( $DestBtn)
  Case $DestBtn
   _Writelog(1, "Destination Button Clicked")
   $Destination=_GetDestinationDir()
   _Writelog(1, "Source set to" & $Destination)
   GUICtrlSetData($DestLabel,$Destination)
   _Writelog(1, "GUI Source Input box updated to" & $Destination)
   _EnableFormControl($GoBtn)
  Case $GoBtn
   _Writelog(1, "Go Button Clicked")
   _Writelog(1, "Disable Form Controls")
   _DisableFormControls()
   $Source=GUICtrlRead($SourceLabel)
   $Destination=GUICtrlRead($DestLabel)
   _Writelog(1, "Launching Dummy Copier to determine number of files and size to be copied from " &$Source& " to " & $Destination)
   $errors=_ErrorChecking($Source, $Destination)
   _Writelog(1, "Error Level returned =" & $errors)
   If $errors = 0 Then
    $NumFiles=_DummyCopy($Source, $Destination)
    _Writelog(1, "The Number of files to be transfered at CASE = " &$NumFiles)
    _Writelog(1, "Launching Copier to copy from " &$Source& " to " & $Destination)
    _Copier($Source, $Destination, $NumFiles)
    _Writelog(1, "Copy function Completed")
    _EnableFormControl($SourceLabel)
    _EnableFormControl($SourceBtn)
    _EnableFormControl($ExitBtn)
    _EnableFormControl($VerboseLogs)
   
   Else
    MsgBox(1, "Copy", "Copy has NOT been run due to issues")
   EndIf
  Case $ExitBtn
   _Writelog(1, "Exit Button Clicked")
   _ExitProgram()
 
EndSwitch
WEnd
Func _ExitProgram()
_Writelog(1, "User has closed the application")
Exit
EndFunc

Func _GetSourceDir()
_Writelog(1, "Selecting Source Directory")
$SourceDir=FileSelectFolder("Get Source", "", 2, "EditPlus 2")
If Stringlen($SourceDir) = 3 Then
  If StringInStr($SourceDir, "") =3 Then
  _Writelog(1, "Detected a root drive" &$SourceDir & " Trimming the  of the end")
  $SourceDir=StringTrimRight($SourceDir,1)
  EndIf
EndIf
_Writelog(1, "Selected Source Directory - " &$SourceDir)
Return $SourceDir
_Writelog(1, "Returning Source Directory - " &$SourceDir)
EndFunc
Func _GetDestinationDir()
_Writelog(1, "Selecting Destination Directory")
$DestinationDir=FileSelectFolder("Select Destination", "", 1, "EditPlus 2")
If Stringlen($DestinationDir) = 3 Then
  If StringInStr($DestinationDir, "") =3 Then
  _Writelog(1, "Detected a root drive" &$DestinationDir & " Trimming the  of the end")
  $DestinationDir=StringTrimRight($DestinationDir,1)
  EndIf
EndIf
_Writelog(1, "Selected Destination Directory - " &$DestinationDir)
Return $DestinationDir
_Writelog(1, "Returning Destination Directory - " &$DestinationDir)
EndFunc
Func _ErrorChecking($Source, $Destination)
Local $Result
$result=0
_Writelog(1, "Checking for known Issues")
If $Source = "" Then
  _Writelog(1, "Source Directory has not been Selected")
  MsgBox(1, "WARNING", "Source Directory has not been Selected")
  $Result=1
EndIf
If $Destination = "" Then
  _Writelog(1, "Destination Directory has not been Selected")
  MsgBox(1, "WARNING", "Destination Directory has not been Selected")
  $Result=1
EndIf

$SourceColon=StringInStr($Source, ":")
$SourceSlash=StringInStr($Source, "")

If $SourceColon=0 or $SourceSlash=0 Then
  _Writelog(1, "Source Directory has not been Selected based on the second character being either a : or 1st character being a ")
  MsgBox(1, "WARNING", "Source Directory in not valid")
  $Result=1
EndIf

$DestColon=StringInStr($Destination, ":")
$DestSlash=StringInStr($Destination, "")

If $DestColon=0 or $DestSlash=0 Then
  _Writelog(1, "Destination Directory has not been Selected based on the second character being either a : or 1st character being a ")
  MsgBox(1, "WARNING", "Destination Directory in not valid" & $DestColon)
  $Result=1
EndIf
If StringInStr($Destination, $Source, 2, 1, 1) <> 0 Then
  _Writelog(1, "Destination Directory is inside the Source Directory")
  MsgBox(1, "WARNING", "Destination Directory  is inside the Source Directory")
  $Result=1
EndIf
  If Stringlen($Source) = 3 Then
  If StringInStr($Source, "") =3 Then
  _Writelog(1, "Detected a root drive" &$Source & " Trim the  of the end")
  MsgBox(1, "WARNING", "Source Directory  is a root directory remove the last ")
  $Result=1
  EndIf
EndIf
If Stringlen($Destination) = 3 Then
  If StringInStr($Destination, "") =3 Then
  _Writelog(1, "Detected a root drive" &$Destination & " Trim the  of the end")
  MsgBox(1, "WARNING", "Destination Directory  is a root directory remove the last ")
  $Result=1
  EndIf
EndIf
Return $Result
EndFunc
Func _Copier($sourcedir, $destdir, $NumFiles)
_Writelog(1, "Copy Function Launched")
_Writelog(1, "Ressetting Progress Bars to Zero")
GUICtrlSetData($progress, 0)
GUICtrlSetData($Fullprogress, 0)
$RoboJob=@ComSpec & " /c " & 'robocopy.exe "' & $sourcedir & '" "' & $destdir & '" /TEE /E /V /Log:'&$CopyLog
_Writelog(1, "Copy started " &$RoboJob )
_Writelog(1, "See Robocopy Log for more detailed information or the RoboCopy" )
$copyjob = Run ($RoboJob, @ScriptDir, @SW_Hide, 2 )
_UpdateFileProgress($copyJob , $sourcedir , $destdir, $NumFiles)
_Writelog(1, "Copy Finished " &$RoboJob )
 
EndFunc
Func _DummyCopy($sourcedir, $destdir)
Local $ReadArray
_Writelog(1, "Dummy Copy Function Launched")
$RoboDummyJob=@ComSpec & ' /c ' & 'robocopy.exe "' & $sourcedir & '" "' & $destdir & '" /NOCOPY /S /TEE /L /Log:'&$NoCopyLog
$Nocopyjob = Run ($RoboDummyJob, @ScriptDir, @SW_Hide, 2 )
_Writelog(1, "Dummy Copy started " &$RoboDummyJob )
While ProcessExists($Nocopyjob)
  _Writelog(1, "Sleeping while NoCopy Job is Running Process =" &$Nocopyjob )
  Sleep(10)
WEnd
_Writelog(1, "Job is Completed Process =" &$Nocopyjob )
If Not _FileReadToArray($NoCopyLog, $ReadArray) Then
   _Writelog(1, "ERROR reading log to Array  ERROR:" & @error)
   Exit
EndIf
For $x = 1 to $ReadArray[0]
_Writelog(1, "Reading " &$NocopyLog& " as Array ")
_Writelog(1, "Line "&$x&" = " &$ReadArray[$x])
  $Filesname=StringInStr($ReadArray[$x], "Files :")
  If $FilesName <> 0 Then
  _Writelog(1, "Line "&$x&" = " &$ReadArray[$x] & " Contains the word Files at position " &$Filesname)
  $allfiles=StringInStr($ReadArray[$x], "*.*")
  If $allfiles = 0 Then
   $TempFilesNum=StringTrimLeft($ReadArray[$x],Int($FilesName + 6))
   _Writelog(1,"String has been created" & $TempFilesNum)
   $TempFilesNumArray=StringRegExp($TempFilesNum, "S+",3)
    for $i = 0 to UBound($TempFilesNumArray) - 1
     _Writelog(1, "Split files line into array. Array " & $i & " =" & $TempFilesNumArray[$i])
    Next
   $FilesName=$TempFilesNumArray[0]
    _Writelog(1, $filesName & " The Number of files to be transfered at dummy copy")
    Return ($filesName)
  EndIF
EndIf
Next
EndFunc
Func _UpdateFileProgress($copyJob , $sourcedir , $destdir, $NumFiles)
_Writelog(4, "Update Progress Function Started" )
_Writelog(1, "The Number of files to be transfered at UpdateFileProress func =  " &$NumFiles)
$switch = 0; We don't want to run StringRegExp on lines that don't matter to us
$output = ""
While ProcessExists($copyjob)
_Writelog(4, "Running while Process Exists" & $copyJob )
$tempOutput =  StdoutRead($copyjob , False , False)
_Writelog(4, "StdoutRead($copyjob , False , False) = " & $tempOutput )
$tempOutput=StringRegExpReplace($tempOutput, "^( *)(.*)$", "1") ; Captures leading spaces in string
If not $tempOutput = "" or " " Then
_Writelog(4, "$tempOutput is not Blank it is = " & $tempOutput )
;ElseIf  StringInStr($tempOutput, "Started :") <> 0 Then
         $output = $tempOutput
   _Writelog(4, "StdoutRead to be processed = " & $output )
     $output = StringSplit($output, @CRLf)
            For $i = 1 To $output[0] - 1
    _Writelog(4, "StdoutRead Output Split by @CRLF= " & $output[$i] )
    If $output[$i] = " " Then
    _Writelog(4, "Determined as blank = " & $output[$i] )
    Else
     _Writelog(4, "Determined as not blank = " & $output[$i] )
     If  Not $switch Then
     _Writelog(4, "Output does not =  0")
     _Writelog(4, "Splitting Output["&$i& "] by any digit group and space groups" )
      If StringRegExp($output[$i], 'd+s+' & StringReplace($sourcedir, "", "")) == 1  Then
      _Writelog(4, "Splitting Output["&$i&"] or StringReplace  to  result = 1 (No matches found) ")
                        $switch = Not $switch
                        $i += 1
      EndIf
     _Writelog(4, "Splitting Output["&$i &"] or StringReplace  to  result <> 1 (matches found) ")
     EndIf
    EndIf
     _Writelog(4, "String  = "& $switch)
                If $switch Then
     _Writelog(4, "$switch $output["&$i&"]to be split by  = "& $switch)
                    $result = StringRegExp($output[$i], '([0-9.]+%)|([0-9.]+s*[mg]?)s*(.*)', 1)
     _Writelog(4, "split result  = "&$Result)
                    If IsArray($result) Then
                        If StringInStr($result[0], "%") Then; Current line is a progress update line
                        ; Set current progress bar to StringTrimRight($result[0], 1)
       GUICtrlSetData($progress, StringTrimRight($result[0], 1))
       _Writelog(4, "Set file complete percentage to  = "&$result[0])
                        ; Set current progress percent label to $result[0]
                        Else; Current line is a new file line
                        ; Reset progress bar to 0
      _FullProgress($NumFiles)
      _Writelog(4, "New File detected")
      _Writelog(4, "Reset percentage to 0%")
                            GUICtrlSetData($progress, 0)
                        ; Reset progress percent label to "0%"
                        ; Set current file size label to $result[0] (either a number in bytes or something like (4.2 m)
      _Writelog(4, "Setting File size to " & $result[1])
                            GUICtrlSetData($sizeLbl, $result[1])
                        ; Set current file name label to $result[1] (outputted as filename)
      _Writelog(4, "Setting File Name to " & $result[2])
                            GUICtrlSetData($fileLbl, $result[2])
                        EndIf
                    EndIf
                EndIf
            Next
            $output = $output[$output[0]]
  EndIf

WEnd
_Writelog(4, "Copy Process Finished")
_Writelog(4, "Reset percentage to 0%")
GUICtrlSetData($progress, 100)
_Writelog(4, "Set Size Lable to Complete")
GUICtrlSetData($sizeLbl, "Complete")
_Writelog(4, "Setting File Name to Complete")
GUICtrlSetData($fileLbl, "Complete")
GUICtrlSetData($FullProgress, 100)
EndFunc
Func _FullProgress($NumFiles)
_Writelog(1, "The Number of files to be transfered at FullProgress func =  " &$NumFiles)
$PerIncr=_Percentage($NumFiles)
_Writelog(1, "The Percent to Increase = " &$PerIncr)
_Writelog(1, "The Current Percent Level =  " &$FProgress)
$FProgress=Number($PerIncr+$FProgress)
GUICtrlSetData($Fullprogress, $FProgress)
_Writelog(1, "The New Percent Level =  " &$FProgress)

EndFunc
Func _Percentage($NumFiles)

$percent=Number((100)/($NumFiles))
Return $percent
 
EndFunc
Func _Writelog($logtype, $logMessage)
$Vebose=GUICtrlRead($VerboseLogs)
If $vebose <> 1 Then
If $logType = 1 Then
  _FileWriteLog($AppLogFile, $logMessage)
Else
  sleep(1)
EndIf
Else
If $logType = 1 Then
  _FileWriteLog($AppLogFile, $logMessage)
ElseIf $logType = 2 Then
  _FileWriteLog($CopyLog, $logMessage)
ElseIf $logType = 3 Then
  _FileWriteLog($NoCopyLog, $logMessage)
ElseIf $logType = 4 Then
  _FileWriteLog($ProcessBarLog, $logMessage)
Else
_FileWriteLog($AppLogFile, "Unknown Log Type message - " & $logMessage)
EndIf
EndIf
EndFunc
Func _DisableFormControls()
_FileWriteLog(1, "Disabling the Form Controls")
GUICtrlSetState ( $SourceLabel,$GUI_DISABLE)
GUICtrlSetState ( $SourceBtn,$GUI_DISABLE)
GUICtrlSetState ( $DestLabel,$GUI_DISABLE)
GUICtrlSetState ( $DestBtn,$GUI_DISABLE)
GUICtrlSetState ( $GoBtn,$GUI_DISABLE)
GUICtrlSetState ( $ExitBtn,$GUI_DISABLE)
GUICtrlSetState ( $VerboseLogs,$GUI_DISABLE)
EndFunc
Func _EnableFormControl($Control)
_FileWriteLog(1, "Enabling the Form Controls")
  _FileWriteLog(1, "Enabling the Form Controls - " &$Control)
  GUICtrlSetState( $Control,$GUI_Enable)
EndFunc

copied up the wrong version so it didn't work heres the working copy

[kylomas]

cmarchy,

I tried to copy the following:

c:tmp to c:tmptmp and received an error message saying that the target dir was inside the source dir...in fact it is NOT...

This logical test fails due to the above condition:

If StringInStr($Destination, $Source, 2, 1, 1) <> 0 Then
  _Writelog(1, "Destination Directory is inside the Source Directory")
  MsgBox(1, "WARNING", "Destination Directory  is inside the Source Directory")
  $Result=1
EndIf

kylomas

Edited December 2, 2011 by kylomas

[CMarshy]

So obvious now you've highlighted thanks! it but it never crossed my mind, oh well these things happen. Easily solved. The source is always going to be a folder in this so putting a "" after the source will clear that up nicely

If StringInStr($Destination , $Source & "", 2, 1, 1) <> 0 Then
  _Writelog(1, "Destination Directory is inside the Source Directory")
  MsgBox(1, "WARNING", "Destination Directory  is inside the Source Directory")
  $Result=1
EndIf

Thanks again for highlighting

CMarshy

Edited December 3, 2011 by CMarshy

[Valik]

Any particular reason you resurrected a dead thread and then bumped your own post within 4 hours of your necrophilia necromancy during off-peak hours? You don't have a good reason? Right. So don't do it again or you will find your ability to post here severely impacted.

[Valik]

Really? This is the only thread in the entire known universe that can help you? Even though there's a "Start New Topic" button on your screen? It must suck to be you.

[Valik]

It's called forum etiquette. I suggest you learn it before you find your ability to post on this forum significantly impacted. I am not the person you want to argue with.

[erroneous]

It's called forum etiquette. I suggest you learn it before you find your ability to post on this forum significantly impacted. I am not the person you want to argue with.

 

What a complete prick

[water]

Perfect for a first post

I suggest YOU read the forum rules and have a look at the forum Netiquette before you post any further.

That's the way for a long and pleasant stay here ;-)

[jfornango]

Wow... file that one in the Recycle Bin for later.

Personally, I'm glad to see someone resurect an old thread for a complete discussion (with answer!).  Beats having to read a dozen partial threads, where a responder will ultimately just link back to this original topic.

Oh, and before this thread gets totally hijacked, a very big "Thank You" to CMarshy for all the work getting a sweet chunk of code put together.  I know *I* will be using it for a few projects.