SFTPex.au3 File Download Scripting Issues

[BasementDweller]

Greetings!

I have never written a script of any kind before. Over the last week I have been trying to teach myself AutoIT scripting using varous Internet .pdf documents and YouTube tutorials. I'm trying to automate a download of four files from a remote server to my Windows 7 workstation's local directory (I'm currently using WS_FTP GUI to manually download these files). My AutoIT script also runs from this directory via SciTE.  I present an example script below (which is identical to my actual script without the sensitive stuff). My actual script seems to be syntactically correct gibberish because although the AU3Check (ctrl+F5) is successful, MsgBox shows SFTPEx.au3 errors at each step when the actual script is run. Windows 7 Task Manager shows psftp.exe does start when the script runs. I am hoping someone will be able to show me the problems with this script -- Thank you!

#include <SFTPEx.au3>
#include <MsgBoxConstants.au3>

$sTuser="username"
$sTpass="password"
$sTaddress="10.0.0.0"

$sPath = "psftp.exe"
$hSession = _SFTP_Open($sPath)
MsgBox(0, "_SFTP_Open", @error)
$hConnection = _SFTP_Connect ( $hSession, $sTaddress , $sTuser , $sTpass , 22  )

MsgBox(0, "_SFTP_Connect", @error)
$sRemoteDir = "NameOfTheRemoteDirectory"
_SFTP_DirSetCurrent ( $hConnection , $sRemoteDir)
MsgBox(0, "_SFTP_DirSetCurrent", @error)
sleep(30000)

Local $aFiles[4] = ["FileOne.txt", "FileTwo.txt", "FileThree.txt", "FileFour.txt"]

$sRemoteFile = $aFiles

_SFTP_FileGet( $hConnection, $sRemoteFile )
MsgBox(0, "_SFTP_FileGet", @error)
sleep (30000)

_SFTP_Close ( $hSession )

[BasementDweller]

Working on this still. I've modified the script by removing all MsgBox entries. I ran psftp.exe manually to see if I could find issues, and I did find one -- I needed a / before the $sRemoteDir name, so I added that. And I added a couple _ArrayDisplay lines:

*****SNIP*****
$sRemoteDir = "/NameOfTheRemoteDirectory" ;Added a / here
_SFTP_DirSetCurrent ( $hConnection , $sRemoteDir)

$sFileList = _SFTP_ListToArray ( $hConnection, $sRemoteDir) ;Added this line
sleep(120000) ;Added more time because 30000 files to list
_ArrayDisplay($sFileList)  ;Added this line

Local $aFiles[4] = ["FileOne.txt", "FileTwo.txt", "FileThree.txt", "FileFour.txt"]
$sRemoteFile = $aFiles
_ArrayDisplay($aFiles)  ;Added this line

_SFTP_FileGet( $hConnection, $sRemoteFile )
*****SNIP*****

What I find now is that the first _ArrayDisplay($sFileList) does show the 30000 file names in the directory that I want to be in, including the four files I'm trying to download. The next _ArrayDisplay($aFiles) does show the four file names in $aFiles[4]. Clearly I still don't have this right because the files aren't being downloaded. But I'm at the point where I know I'm in the directory where I want to be, and I can see the files I need listed in _ArrayDisplay($sFileList). I still need to figure out how to download multiple files. Nothing on the Internet about this that I've been able to find (hard to believe nobody on the planet has run into this -- my Search-Foo must suck). And nothing in the AutoIT forum that discusses multiple file downloads with SFTPEx.au3. So... Still trying.

[TheXman]

You need to add error checking after your commands to see which may be failing and why.  Add the following error checking after your _SFTP_FileGet to see why it may be failing.

_SFTP_FileGet( $hConnection, $sRemoteFile )
If @error Then
    Switch @error
        Case 1
            MsgBox($MB_ICONERROR, "_SFTP_FileGet ERROR", "The connection is closed")
        Case 2
            MsgBox($MB_ICONERROR, "_SFTP_FileGet ERROR", "Local file exists and $fFailIfExists = True")
        Case 3
            MsgBox($MB_ICONERROR, "_SFTP_FileGet ERROR", "Remote file not found")
        Case 4
            MsgBox($MB_ICONERROR, "_SFTP_FileGet ERROR", "Other error")
        Case Else
            MsgBox($MB_ICONERROR, "_SFTP_FileGet ERROR", "Unrecognized error - @error = " & @error)
    EndSwitch
    Exit -1
EndIf

 

Edited January 24, 2019 by TheXman

[BasementDweller]

Thanks TheXman, I appreciate the error checking script. I added this to my script and it didn't return any errors. So to troubleshoot further I simplified my script to focus on one file download instead of four, by changing $sRemoteFile = $aFiles to $sRemoteFile = FileOne.txt, and removing  Local $aFiles[4] = ["FileOne.txt", "FileTwo.txt", "FileThree.txt", "FileFour.txt"] . With that change I was able to download one FileOne.txt file. Expanding on that, I rewrote to download four files, as I show below, and this does work. So my file download issue is solved. My question now is (and asking only so I can better understand scripting in general and AutoIT in particular): Is what I wrote below the most efficient way to download multiple files, or is there a more elegant way to script this?

Thanks very much for your time.

$sTuser="username"
$sTpass="password"
$sTaddress="10.0.0.0"
$iServerPort="22"
$sPath = 'psftp.exe'
$hSession = _SFTP_Open($sPath)

$hConnection = _SFTP_Connect ( $hSession, $sTaddress , $sTuser , $sTpass , $iServerPort )

sleep(10000)

$sRemoteDir = "/NameOfTheRemoteDirectory"
_SFTP_DirSetCurrent ( $hConnection , $sRemoteDir)

$sRemoteFile = "FileOne.txt"
_SFTP_FileGet( $hConnection, $sRemoteFile )

$sRemoteFile = "FileTwo.txt"
_SFTP_FileGet( $hConnection, $sRemoteFile )

$sRemoteFile = "FileThree.txt"
_SFTP_FileGet( $hConnection, $sRemoteFile )

$RemoteFile = "FileFour.txt"
_SFTP_FileGet( $hConnection, $RemoteFile )

_SFTP_Close ( $hSession )

 

[TheXman]

55 minutes ago, BasementDweller said:

My question now is (and asking only so I can better understand scripting in general and AutoIT in particular): Is what I wrote below the most efficient way to download multiple files, or is there a more elegant way to script this?

The original creator of the SFTPEx.au3 UDF did not create a function for getting multiple files at once.  If you look at the functions in the UDF, you only see a _SFTP_FileGet() function.  It passes the "get" command to psftp, with the appropriate parameters to get a single file.  The "get" command only handles one file at a time.  PSFTP's  "mget" command allows you to download multiple files at a time.  So what I would suggest that you create and add a new function to the UDF called something like _SFTP_FileMGet() that uses the "mget" command with the appropriate parameters.

It would be relatively simple since the mget command is very similar to the get command.  The only difference is that the you give the mget command a list of files to retrieve, separated by spaces.  Therefore, you could basically copy the _SFTP_FileGet function, change the file parameter to an array of files, and then set up your command line using the files in the supplied array (and of course change the command from get to mget).

Edited January 28, 2019 by TheXman

[TheXman]

Here's what a function that retrieves multiple files at once may look like.  I haven't tested it.  But based on the _SFTP_FileGet function, it looks like it should work.  The only thing I'm not sure of is whether the responses from an mget command are the same as a get command.  Therefore, you may have to tweak the lines that alter the loop based on the command output.  I tried to stick as closely to the original _SFTP_FileGet function's structure and style as possible.  This function does not recurse through a directory structure looking for the files.  If you want additional functionality like that, just modify the function as needed.

 

expandcollapse popup

; #FUNCTION# ====================================================================================================================
; Name...........: _SFTP_FileGetMultiple
; Description ...: Get multiple files from a SFTP server.
; Syntax.........: _SFTP_FileGetMultiple( $hConnection, $aRemoteFiles )
; Parameters ....: $hConnection - as returned by _SFTP_Connect().
;                  $aRemoteFiles - An array of remote files to retrieve.
; Return values .: Success - 1
;                  Failure - 0, sets @error
;                  |1 - The connection is closed
;                  |2 - $aRemoteFiles is not a valid array
;                  |3 - Remote file not found
;                  |4 - Other error
; Author ........: TheXman
; Modified.......:
; Remarks .......:
; Related .......: _SFTP_Connect
; Link ..........:
; Example .......: No
; ===============================================================================================================================
Func _SFTP_FileGetMultiple($hConnection, $aRemoteFiles)
    If ProcessExists($hConnection) = 0 Then
        Return SetError(1, 0, 0)
    ElseIf Not IsArray($aRemoteFiles) Then
        Return SetError(2, 0, 0)
    EndIf

    Local $sLine, $sFileList = ""

    ;Create file list from array
    For $sFile in $aRemoteFiles
        If $sFileList <> "" Then $sFileList &= " "
        $sFileList &= StringFormat('"%s"', $sFile)
    Next

    StdinWrite($hConnection, 'mget -- ' & $sFileList & @CRLF)
    While 1
        $sLine = StdoutRead($hConnection)
        If ProcessExists($hConnection) = 0 Then
            Return SetError(1, 0, 0)
        ElseIf StringInStr($sLine, "psftp>") Then
            ExitLoop
        ElseIf StringInStr($sLine, "=> local:") Then
            ContinueLoop
        ElseIf StringInStr($sLine, "no such file or directory") Then
            Return SetError(3, 0, 0)
        ElseIf $sLine <> "" Then
            Return SetError(4, 0, 0)
        EndIf
        Sleep(10)
    WEnd

    Return 1
EndFunc   ; ==>_SFTP_FileGetMultiple

 

Edited January 29, 2019 by TheXman

[BasementDweller]

TheXman -- thanks for taking the time to write this. And thanks for your help overall.

When I get a chance, I'll see if it'll work. It's great that you've given me this example. I also appreciate the error checking example you wrote earlier.

Thank you!

[TheXman]

You're welcome!