WebDriver UDF - Help & Support (II)

[Danp2]

@RandalBY The follow options (set before calling _WD_Startup()) will tell geckodriver to attach to previously launched instance of Firefox --

_WD_Option('Driver', 'C:\Path\to\the\executable\geckodriver.exe')
_WD_Option('DriverParams', '--log trace --connect-existing  --marionette-port 2828')
_WD_Option('Port', 4444)

I haven't tried it myself, but others have posted about saving the sessionid and then using it to communicate with an existing instance of chromedriver.

[BuiTuanAnh]

Hello @Danp2
I have problems with gmail login
Please help me

[Danp2]

@BuiTuanAnh Sorry, but I can't read Vietnamese. Suggest that you post your script and then describe in detail the problems you are encountering.

[BuiTuanAnh]

Hi @Danp2

I cannot login to gmail when using webdriver

[Danp2]

9 minutes ago, BuiTuanAnh said:

I cannot login to gmail when using webdriver

Haven't tried to do this myself, but don't see why it shouldn't work. If you want further assistance, post your script or a short reproducer.

[BuiTuanAnh]

hi @Danp2

I just logged in normally, it seems that google has blocked the webdriver

My Script

Local $sDesiredCapabilities, $sSession
SetupChrome()
_WD_Startup()
$sSession = _WD_CreateSession($sDesiredCapabilities)
_WD_Navigate($sSession, "https://www.gmail.com")
_ChromeSetInputValueByClass($sSession,'whsOnd zHQkBf',$_user)
_ChromeClickSpan_Class($sSession,'RveJvd snByac')
_ChromeSetInputValueByClass($sSession,'whsOnd zHQkBf',$_pass)
_ChromeClickSpan_Class($sSession,'RveJvd snByac')

[Danp2]

9 hours ago, BuiTuanAnh said:

I just logged in normally, it seems that google has blocked the webdriver

No, that isn't accurate as can be seen by running the following --

#include "wd_core.au3"
#include "wd_helper.au3"

Local $sDesiredCapabilities, $sSession, $sElement
Local $sEmailAddress = 'yourname@gmail.com'
Local $sPassword = 'yourpassword'

SetupChrome()
_WD_Startup()
$sSession = _WD_CreateSession($sDesiredCapabilities)
_WD_Navigate($sSession, "https://www.gmail.com")

$sElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//input[@id='identifierId']")
_WD_ElementAction($sSession, $sElement, 'value', $sEmailAddress)
$sElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//span[@class='RveJvd snByac']")
_WD_ElementAction($sSession, $sElement, 'click')
_WD_LoadWait($sSession, 1000)


$sElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//input[@name='password']")
_WD_ElementAction($sSession, $sElement, 'value', $sPassword)
$sElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//div[@id='passwordNext']")
_WD_ElementAction($sSession, $sElement, 'click')
_WD_LoadWait($sSession)

Func SetupChrome()
_WD_Option('Driver', 'chromedriver.exe')
_WD_Option('Port', 9515)
_WD_Option('DriverParams', '--verbose --log-path="' & @ScriptDir & '\chrome.log"')

$sDesiredCapabilities = '{"capabilities": {"alwaysMatch": {"goog:chromeOptions": {"w3c": true, "excludeSwitches": [ "enable-automation"], "useAutomationExtension": false }}}}'
EndFunc

There's something wrong with your script. We can't test what you posted (non-standard function calls) and you didn't post the results from the Scite output panel, so we can only guess at what it might be.

[BuiTuanAnh]

hi @Danp2

I fixed it after turning on the less secure app in my account.

I have one more question

I want to get current url, how to do it
Please help me
Thank you

[Danp2]

31 minutes ago, BuiTuanAnh said:

I want to get current url, how to do it

Did you even try to figure this out on your own? 🙄 This comes straight from wd_demo --

ConsoleWrite("URL=" & _WD_Action($sSession, 'url') & @CRLF)

 

[BuiTuanAnh]

😭 Thank you very much, I have solved the problem

[RandalBY]

@Danp2 Is there a way to disable popup browser window on every action with it (_WD_Navigate, _WD_ExecuteScript etc.)?

[Danp2]

1 hour ago, RandalBY said:

@Danp2 Is there a way to disable popup browser window on every action with it (_WD_Navigate, _WD_ExecuteScript etc.)?

Not sure what you mean here. Is the browser jumping to the topmost window? What browser are you using? Can you post a short reproducer?

[RandalBY]

27 minutes ago, Danp2 said:

Is the browser jumping to the topmost window?

yes

Quote

What browser are you using?

Chrome latest version. I thought all browsers do the same . So not tested another. I was wrong?

[Danp2]

I predominantly use Firefox and don't recall this being an issue. Suggest that you give it s try and then report back with your results.

[crazycrash]

Thanks so much for this UDF! I am currently trying to convert some of my old browser automation scripts to Webdriver & Firefox. The only thing I couldn't find figure out yet at all is how to download a picture to a specific destination using Firefox. I basically would like to replace InetGet to get the following picture: Picture on Domain

What would be the simplest way to achieve this? An example would be greatly appreciated.

Cheers!

Edited January 4, 2020 by crazycrash

[Danp2]

I don't believe this functionality is included in the w3c specs, but it shouldn't be too hard to create a function to download and save a file using the winhttp functions. It will likely be Monday before I can look into this further.

[crazycrash]

With winhttp possibly something like the file download function from the below script could be used: (Quoted from careca). It seems to be working well.

Would it be possible to integrate this into your UDF as a simple function to effectively replace InetGet? Might be a nice additional feature. For me you answered my question already, thanks!

expandcollapse popup

#include "WinHttp.au3"
#include "Date.au3"
#include <FileConstants.au3>

$sServerAddress = "https://rda.ucar.edu"
$sGeneratorLocation = "/cgi-bin/login"

$sEmail = "xxxxx@xxx.co.za"
$sPassword = "xxxxxxx"

$hOpen = _WinHttpOpen()

; collect access cookie first
$hConnect = _WinHttpConnect($hOpen, $sServerAddress)
_WinHttpSimpleSSLRequest($hConnect)
_WinHttpCloseHandle($hConnect)

; build and fill the login form
$sForm = _
        '<form action="' & $sServerAddress & $sGeneratorLocation & '" method="post">' & _
        '   <input name="email" />' & _
        '   <input name="passwd" />' & _
        '   <input name="remember" />' & _
        '   <input name="do" />' & _
        '   <input name="url" />' & _
        '</form>'
$hConnect = $sForm

$sReturned = _WinHttpSimpleFormFill($hConnect, $hOpen, _
        Default, _
        "name:email", $sEmail, _
        "name:passwd", $sPassword, _
        "name:remember", "on", _
        "name:do", "login", _
        "name:url", "/")
If @error Then
    MsgBox(4096, "Error", @error)
Else


;=========================setup file naming according to date========================================

Local $Today = _Date_Time_GetSystemTime()
Local $CalDate = _Date_Time_SystemTimeToDateStr($Today)
Local $Yesterday = _DateTimeFormat(_DateAdd('d', -1, _NowCalcDate()), 2)
 ; Split DTG
 $aYesterday = StringSplit($Yesterday, "/")
 ; Reassemble in desired order
 $sYear = $aYesterday[1]
 $sMonth = _DateToMonth($aYesterday[2])
 $sYearMonth = $aYesterday[1] & "." & $aYesterday[2]
 $sYesterdayF = $aYesterday[1] & $aYesterday[2] & $aYesterday[3]
 $sFile1 = "fnl_" & $sYesterdayF & "_00_00.grib2"
 $sFile2 = "fnl_" & $sYesterdayF & "_06_00.grib2"
 $sFile3 = "fnl_" & $sYesterdayF & "_12_00.grib2"
 $sFile4 = "fnl_" & $sYesterdayF & "_18_00.grib2"

 $sTarget1 = "data/ds083.2/grib2/" & $sYear & "/" & $sYearMonth & "/" & $sFile1
 $sTarget2 = "data/ds083.2/grib2/" & $sYear & "/" & $sYearMonth & "/" & $sFile2
 $sTarget3 = "data/ds083.2/grib2/" & $sYear & "/" & $sYearMonth & "/" & $sFile3
 $sTarget4 = "data/ds083.2/grib2/" & $sYear & "/" & $sYearMonth & "/" & $sFile4

;create a folder where
 $sFldr = "C:\data\"& $sMonth & " " & $sYear &"\"
 DirCreate($sFldr)

 Download1()
  Download2()
   Download3()
    Download4()
EndIf

Func Download1()
;====================================================================================================
;========================================download file 1=============================================
  _WinHttpCloseHandle($hConnect)
  $hConnect = _WinHttpConnect($hOpen, $sServerAddress)
  $hRequest = _WinHttpSimpleSendSSLRequest($hConnect, "GET", $sTarget1)
  If _WinHttpQueryDataAvailable($hRequest) Then

    $headers = _WinHttpQueryHeaders($hRequest)
    $Length = StringRegExpReplace($headers, '(?s).*Content-Length:\h*(\d+).*', "$1")
    Local $bChunk, $bData = Binary("")

    ProgressOn($sFile1, "Downloading", "0 %")
    While 1
         $bChunk = _WinHttpReadData($hRequest, 2, 8192) ; binary
         If @error Then ExitLoop
         $bData &= $bChunk
         $percent = Int((BinaryLen($bData)/$Length)*100)
         ProgressSet($percent, "Downloading", $percent & " %")
     WEnd
     ProgressSet(100, "Done !", "100 %")

     FileWrite($sFldr & $sFile1, $bData )
     Sleep(1000)
     ProgressOff()
  Else
      MsgBox(48, "", "connection error")
  EndIf
  _WinHttpCloseHandle($hRequest)
  sleep(500)
EndFunc
;=============================================================================
Func Download2()
;====================================================================================================
;========================================download file 2=============================================
 $hRequest = _WinHttpSimpleSendSSLRequest($hConnect, "GET", $sTarget2)
  If _WinHttpQueryDataAvailable($hRequest) Then

    $headers = _WinHttpQueryHeaders($hRequest)
    $Length = StringRegExpReplace($headers, '(?s).*Content-Length:\h*(\d+).*', "$1")
    Local $bChunk, $bData = Binary("")

    ProgressOn($sFile2, "Downloading", "0 %")
    While 1
         $bChunk = _WinHttpReadData($hRequest, 2, 8192) ; binary
         If @error Then ExitLoop
         $bData &= $bChunk
         $percent = Int((BinaryLen($bData)/$Length)*100)
         ProgressSet($percent, "Downloading", $percent & " %")
     WEnd
     ProgressSet(100, "Done !", "100 %")
     FileWrite($sFldr & $sFile2, $bData)
     Sleep(1000)
     ProgressOff()
  Else; retry connecting

    $headers = _WinHttpQueryHeaders($hRequest)
    $Length = StringRegExpReplace($headers, '(?s).*Content-Length:\h*(\d+).*', "$1")
    Local $bChunk, $bData = Binary("")

    ProgressOn($sFile2, "Downloading", "0 %")
    While 1
         $bChunk = _WinHttpReadData($hRequest, 2, 8192) ; binary
         If @error Then ExitLoop
         $bData &= $bChunk
         $percent = Int((BinaryLen($bData)/$Length)*100)
         ProgressSet($percent, "Downloading", $percent & " %")
     WEnd
     ProgressSet(100, "Done !", "100 %")
     FileWrite($sFldr & $sFile2, $bData)
     Sleep(1000)
     ProgressOff()
  EndIf
  _WinHttpCloseHandle($hRequest)
sleep(500)
EndFunc
;=============================================================================
Func Download3()
;====================================================================================================
;========================================download file 3=============================================
  $hRequest = _WinHttpSimpleSendSSLRequest($hConnect, "GET", $sTarget3)
  If _WinHttpQueryDataAvailable($hRequest) Then

    $headers = _WinHttpQueryHeaders($hRequest)
    $Length = StringRegExpReplace($headers, '(?s).*Content-Length:\h*(\d+).*', "$1")
    Local $bChunk, $bData = Binary("")

    ProgressOn($sFile3, "Downloading", "0 %")
    While 1
         $bChunk = _WinHttpReadData($hRequest, 2, 8192) ; binary
         If @error Then ExitLoop
         $bData &= $bChunk
         $percent = Int((BinaryLen($bData)/$Length)*100)
         ProgressSet($percent, "Downloading", $percent & " %")
     WEnd
     ProgressSet(100, "Done !", "100 %")
     FileWrite($sFldr & $sFile3, $bData)
     Sleep(1000)
     ProgressOff()
  Else
      MsgBox(48, "", "connection error")
  EndIf
_WinHttpCloseHandle($hRequest)
sleep(500)
EndFunc
;=============================================================================
Func Download4()
;====================================================================================================
;========================================download file 4=============================================
  $hRequest = _WinHttpSimpleSendSSLRequest($hConnect, "GET", $sTarget4)
  If _WinHttpQueryDataAvailable($hRequest) Then

    $headers = _WinHttpQueryHeaders($hRequest)
    $Length = StringRegExpReplace($headers, '(?s).*Content-Length:\h*(\d+).*', "$1")
    Local $bChunk, $bData = Binary("")

    ProgressOn($sFile4, "Downloading", "0 %")
    While 1
         $bChunk = _WinHttpReadData($hRequest, 2, 8192) ; binary
         If @error Then ExitLoop
         $bData &= $bChunk
         $percent = Int((BinaryLen($bData)/$Length)*100)
         ProgressSet($percent, "Downloading", $percent & " %")
     WEnd
     ProgressSet(100, "Done !", "100 %")
     FileWrite($sFldr & $sFile4, $bData)
     Sleep(1000)
     ProgressOff()
  Else
      MsgBox(48, "", "connection error")
  EndIf
EndFunc
;====================================================================================================
Func Check()
    If FileExists($sFldr & $sFile1) = 0 Then
        Download1()
    EndIf
    If FileExists($sFldr & $sFile2) = 0 Then
        Download2()
    EndIf
    If FileExists($sFldr & $sFile3) = 0 Then
        Download3()
    EndIf
    If FileExists($sFldr & $sFile4) = 0 Then
        Download4()
    EndIf
EndFunc
;=============================================================================

; close handles
_WinHttpCloseHandle($hRequest)
_WinHttpCloseHandle($hConnect)
_WinHttpCloseHandle($hOpen)

 

Edited January 5, 2020 by crazycrash

[crazycrash]

Or maybe a function with this idea could be created (no gui popup). But how to save a file that can be displayed in the browser would have to be managed a little differently.

Quote

FirefoxProfile fxProfile = new FirefoxProfile();

fxProfile.setPreference("browser.download.folderList",2);
fxProfile.setPreference("browser.download.manager.showWhenStarting",false);
fxProfile.setPreference("browser.download.dir","c:\\mydownloads");
fxProfile.setPreference("browser.helperApps.neverAsk.saveToDisk","text/csv");

WebDriver driver = new FirefoxDriver(fxProfile);
driver.navigate().to("http://www.foo.com/bah.csv");

 

 

[Danp2]

My initial idea was for something like this --

_WD_DownloadFile($sURL, $sDest)

where $sURL represents the file to be downloaded and $sDest is the full path (including filename) for the file to be saved to disk. The underlying code would call an existing internal function (__WD_Get) to perform the download and then write the contents to disk, which is what the new _WD_UpdateDriver routine does.

Comments or suggestions anyone?

[Danp2]

Here's what I came up with --

expandcollapse popup

; #FUNCTION# ====================================================================================================================
; Name ..........: _WD_DownloadFile
; Description ...: Download file and save to disk
; Syntax ........: _WD_DownloadFile($sURL, $sDest)
; Parameters ....: $sURL                - URL representing file to be downloaded
;                  $sDest               - Full path, including filename, of destination file
; Return values .: True      - Download succeeded
;                  False     - Download failed
; Author ........: Dan Pollak
; Modified ......:
; Remarks .......:
; Related .......:
; Link ..........:
; Example .......: No
; ===============================================================================================================================
Func _WD_DownloadFile($sURL, $sDest)
    Local Const $sFuncName = "_WD_DownloadFile"
    Local $lResult = False

    ; Save current debug level and set to none
    Local $WDDebugSave = $_WD_DEBUG
    $_WD_DEBUG = $_WD_DEBUG_None

    Local $sData = __WD_Get($sURL)
    Local $iErr = @error

    If $iErr = $_WD_ERROR_Success Then
        Local $hFile = FileOpen($sDest, 18)

        If $hFile <> -1 Then
            FileWrite($hFile, $sData)
            FileClose($hFile)

            $lResult = True
        EndIf
    EndIf

    ; Restore prior setting
    $_WD_DEBUG = $WDDebugSave

    If $_WD_DEBUG = $_WD_DEBUG_Info Then
        ConsoleWrite($sFuncName & ': ' & $iErr & @CRLF)
    EndIf

    Return SetError(__WD_Error($sFuncName, $iErr), 0, $lResult)
EndFunc

and I tested it using this snippet of code --

_WD_Navigate($sSession, "http://google.com")

    ; Get the website's URL
    $sUrl = _WD_Action($sSession, 'url')

    ; Find the element and retrieve it's source attribute
    $sElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//img[@id='hplogo']")
    $sSource  = _WD_ElementAction($sSession, $sElement, "Attribute", "src")

    ; Combine the URL and element link
    $sURL = _WinAPI_UrlCombine($sUrl, $sSource)

    ; Download the file
    _WD_DownloadFile($sUrl, @ScriptDir & "\testimage.png")

Works in my basic test to download an image, but I haven't tried anything else yet. Please put this through it's paces and let me know if you encounter any issues.

Dan