Webdriverscript not working as expected...

[Jemboy]

Hi,

I have the below script.
It should return the second screenshot, however sometimes I get the first screenshot.
On some computers I almost 90% get screen 2, but on some others I get screen 1 (wrong screen) almost every time.

I have played around with "_WD_LoadWait", sleep or using "_WD_WaitElement", however I do not seem
to be able get the reason why my script is behaving so randomly.

Does anyone have some insights for me ?

expandcollapse popup

#include "wd_helper.au3"
#include "wd_capabilities.au3"

_Example()

Func _Example()
    # REMARK
    #   This is not functional script
    #   It only shows the concept how to use WebDriver UDF

    #Region ; initialize webdriver sesion


    _WD_UpdateDriver('chrome')

    ; specify driver, port and other options
    _WD_Option('Driver', 'chromedriver.exe')
    _WD_Option('Port', 9515)
    _WD_Option('DriverParams', '--port=9515 --verbose --log-path="' & @ScriptDir & '\chrome.log"')

    ; start the driver
    _WD_Startup()
    If @error Then Return SetError(@error, @extended, 0) ; always remember to check and handle error's

    ; create capabilites for session
    _WD_CapabilitiesStartup()
    _WD_CapabilitiesAdd('alwaysMatch', 'chrome')
    _WD_CapabilitiesAdd('w3c', True)
    _WD_CapabilitiesAdd('excludeSwitches', 'enable-automation')
    Local $sCapabilities = _WD_CapabilitiesGet()

    ; create session with given Capabilities
    Global $sSession = _WD_CreateSession($sCapabilities)
    If @error Then Return SetError(@error, @extended, 0) ; always remember to check and handle error's
    _WD_Window($sSession, 'maximize')

    #EndRegion ; initialize webdriver sesion

    #Region ; do your's stuff

    ; navigate to some website
    Local $sURL = "https://ordermanager.tecalliance.net/newapp/auth/login"
    _WD_Navigate($sSession, $sURL)
    If @error Then Return SetError(@error, @extended, 0) ; always remember to check and handle error's


    ; wait for loading process ends
    _WD_LoadWait($sSession, 1000)
    If @error Then Return SetError(@error, @extended, 0) ; always remember to check and handle error's

    ; find cookie popup
    Local $sElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//*[@class='mat-button-wrapper']")
    If @error Then Return SetError(@error, @extended, 0) ; always remember to check and handle error's

    ; click on default button from cookie popup (accept)
    _WD_ElementAction($sSession, $sElement, 'click')
    If @error Then Return SetError(@error, @extended, 0) ; always remember to check and handle error's


    ; wait max. 50 seconds for the login button (=aanmelden) to be available
    _WD_WaitElement($sSession, $_WD_LOCATOR_ByXPath, "//*[@class='mat-focus-indicator log-in-button mat-raised-button mat-button-base']", 100, 50 * 1000)
    If @error Then Return SetError(@error, @extended, 0) ; always remember to check and handle error's

    ; find this login (=aanmelden) button
    $sElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//*[@class='mat-focus-indicator log-in-button mat-raised-button mat-button-base']")
    If @error Then Return SetError(@error, @extended, 0) ; always remember to check and handle error's

    ; click this "login" button to start login process
    _WD_ElementAction($sSession, $sElement, 'click')
    If @error Then Return SetError(@error, @extended, 0) ; always remember to check and handle error's

    ; wait for the username (=gebruikersnaam) field to be available
    _WD_WaitElement($sSession, $_WD_LOCATOR_ByXPath, '//input[@id="input28"]', 100, 30 * 1000)
    If @error Then Return SetError(@error, @extended, 0) ; always remember to check and handle error's

    ; find this username field
    $sElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, '//input[@id="input28"]')
    If @error Then Return SetError(@error, @extended, 0) ; always remember to check and handle error's

    ; fill in user e-mail in the input field
    _WD_ElementAction($sSession, $sElement, 'value', "test@test.nl")
    If @error Then Return SetError(@error, @extended, 0) ; always remember to check and handle error's

    ; lookup orange [Volgende] (=next) button
    $sElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//*[@class='button button-primary']")
    If @error Then Return SetError(@error, @extended, 0) ; always remember to check and handle error's

    ; click on the orange [volgende] (=next) button to continue
    _WD_ElementAction($sSession, $sElement, 'click')
    If @error Then Return SetError(@error, @extended, 0) ; always remember to check and handle error's

    Exit    ; Exit so not to close his Chrome session.

    #EndRegion ; do your's stuff

    #Region ; Clean Up

    ; on the end session should be deleted
    _WD_DeleteSession($sSession)
    If @error Then Return SetError(@error, @extended, 0) ; always remember to check and handle error's

    ; and driver should be closed
    _WD_Shutdown()
    If @error Then Return SetError(@error, @extended, 0) ; always remember to check and handle error's

    #EndRegion ; Clean Up

EndFunc   ;==>_Example

[Danp2]

My recommendation is to run the script with $_WD_DEBUG_Full, and then share the output from a failed run of the script.

[gmmg]

Hello jemboy,

will you automate the login process or registration process?

for the Login, i think you can take the following URL: https://login.tecalliance.net

Greetings,

Marcel

[Jemboy]

2 minutes ago, gmmg said:

For the Login, i think you can take the following URL: https://login.tecalliance.net

I want to automate the login process.
Your link seems like a very nice shortcut to this🙂
I will have a look, if I can implement this.

Besides that it still  gnaws at me (Dutch: het knaagt aan me) that running the script multiple times
results in diffirent outcomes.
So I would love the learn what I am overlooking😁

[Jemboy]

@Danp2I will try to do that as soon as possible.
For now,  I have been able to capture (copy/paste) the Scite Output from both events.
And I want to analyse those with a text diff tool.... later today.
 

[Jemboy]

Haven;t had the time run and grab the full log, but I can post relevant Scite output here.
The websriver UDF is 1.3.1, however my full script (in another folder) uses the latest Webdriver UDF.

With the wrong response outputm I see "__WD_Post ==> Element interaction issue [14] : HTTP status = 400" where the diffirence is starting.
I didn't have any time left today to investigate why the script reproduce a diffirent result here while doing the same web page.

expandcollapse popup

;Script with GOOD results

_WD_Startup: OS:    WIN_10 X64 19045 
_WD_Startup: AutoIt:    3.3.16.1
_WD_Startup: Webdriver UDF: 1.3.1 (Update available)
_WD_Startup: WinHTTP:   1.6.4.2
_WD_Startup: Driver:    chromedriver.exe (32 Bit)
_WD_Startup: Params:    --port=9515 --verbose --log-path="O:\Autoit\Chromedriver Webdriver Tester\chrome.log"
_WD_Startup: Port:  9515
_WD_Startup: Command:   "chromedriver.exe" --port=9515 --verbose --log-path="O:\\Autoit\Chromedriver Webdriver Tester\chrome.log" 
_WD_Startup ==> Success [0]
_WD_CapabilitiesAdd ==> Success [0] : Successfully used [alwaysMatch]  with specified browser: chrome
_WD_CapabilitiesAdd ==> Success [0] : Successfully added capability
_WD_CapabilitiesAdd ==> Success [0] : Successfully added capability
__WD_Post ==> Success [0] : HTTP status = 200
_WD_CreateSession ==> Success [0] : cd526b06af4653b13ef84e7d0f493b61
__WD_Post ==> Success [0] : HTTP status = 200
_WD_Window ==> Success [0] : Parameters:   Command=maximize   Option=Default
__WD_Post ==> Success [0] : HTTP status = 200
_WD_Navigate ==> Success [0] : Parameters:   URL=https://ordermanager.tecalliance.net/newapp/auth/login
_WD_LoadWait ==> Success [0 / 4] : Parameters:    Delay=1000    Timeout=Default    Element=Default    DesiredState=complete    : ReadyState= complete (Fully loaded)
__WD_Post ==> Success [0] : HTTP status = 200
_WD_FindElement ==> Success [0] : Parameters:   Strategy=xpath   Selector=//*[@class='mat-button-wrapper']   StartNodeID=Default   Multiple=Default   ShadowRoot=Default
__WD_Post ==> Success [0] : HTTP status = 200
_WD_ElementAction ==> Success [0] : Parameters:   Command=click   Option=Default
_WD_WaitElement ==> Success [0] : Parameters:   Strategy=xpath   Selector=//*[@class='mat-focus-indicator log-in-button mat-raised-button mat-button-base']   Delay=100   Timeout=50000   Options=Default
__WD_Post ==> Success [0] : HTTP status = 200
_WD_FindElement ==> Success [0] : Parameters:   Strategy=xpath   Selector=//*[@class='mat-focus-indicator log-in-button mat-raised-button mat-button-base']   StartNodeID=Default   Multiple=Default   ShadowRoot=Default
__WD_Post ==> Success [0] : HTTP status = 200
_WD_ElementAction ==> Success [0] : Parameters:   Command=click   Option=Default
_WD_WaitElement ==> Success [0] : Parameters:   Strategy=xpath   Selector=//input[@id="input28"]   Delay=100   Timeout=30000   Options=Default
__WD_Post ==> Success [0] : HTTP status = 200
_WD_FindElement ==> Success [0] : Parameters:   Strategy=xpath   Selector=//input[@id="input28"]   StartNodeID=Default   Multiple=Default   ShadowRoot=Default
__WD_Post ==> Success [0] : HTTP status = 200
_WD_ElementAction ==> Success [0] : Parameters:   Command=value   Option=<masked>
__WD_Post ==> Success [0] : HTTP status = 200
_WD_FindElement ==> Success [0] : Parameters:   Strategy=xpath   Selector=//*[@class='button button-primary']   StartNodeID=Default   Multiple=Default   ShadowRoot=Default
__WD_Post ==> Success [0] : HTTP status = 200
_WD_ElementAction ==> Success [0] : Parameters:   Command=click   Option=Default
+>11:19:39 AutoIt3.exe ended.rc:0
+>11:19:39 AutoIt3Wrapper Finished.
>Exit code: 0    Time: 15.4

 

;Script with WRONG results

_WD_Startup: OS:    WIN_10 X64 19045 
_WD_Startup: AutoIt:    3.3.16.1
_WD_Startup: Webdriver UDF: 1.3.1 (Update available)
_WD_Startup: WinHTTP:   1.6.4.2
_WD_Startup: Driver:    chromedriver.exe (32 Bit)
_WD_Startup: Params:    --port=9515 --verbose --log-path="O:\Autoit\Chromedriver Webdriver Tester\chrome.log"
_WD_Startup: Port:  9515
_WD_Startup: Command:   "chromedriver.exe" --port=9515 --verbose --log-path="O:\Autoit\Chromedriver Webdriver Tester\chrome.log" 
_WD_Startup ==> Success [0]
_WD_CapabilitiesAdd ==> Success [0] : Successfully used [alwaysMatch]  with specified browser: chrome
_WD_CapabilitiesAdd ==> Success [0] : Successfully added capability
_WD_CapabilitiesAdd ==> Success [0] : Successfully added capability
__WD_Post ==> Success [0] : HTTP status = 200
_WD_CreateSession ==> Success [0] : 4fec888636b28ef675e1b0703e494bfa
__WD_Post ==> Success [0] : HTTP status = 200
_WD_Window ==> Success [0] : Parameters:   Command=maximize   Option=Default
__WD_Post ==> Success [0] : HTTP status = 200
_WD_Navigate ==> Success [0] : Parameters:   URL=https://ordermanager.tecalliance.net/newapp/auth/login
_WD_LoadWait ==> Success [0 / 4] : Parameters:    Delay=1000    Timeout=Default    Element=Default    DesiredState=complete    : ReadyState= complete (Fully loaded)
__WD_Post ==> Success [0] : HTTP status = 200
_WD_FindElement ==> Success [0] : Parameters:   Strategy=xpath   Selector=//*[@class='mat-button-wrapper']   StartNodeID=Default   Multiple=Default   ShadowRoot=Default
__WD_Post ==> Success [0] : HTTP status = 200
_WD_ElementAction ==> Success [0] : Parameters:   Command=click   Option=Default
_WD_WaitElement ==> Success [0] : Parameters:   Strategy=xpath   Selector=//*[@class='mat-focus-indicator log-in-button mat-raised-button mat-button-base']   Delay=100   Timeout=50000   Options=Default
__WD_Post ==> Success [0] : HTTP status = 200
_WD_FindElement ==> Success [0] : Parameters:   Strategy=xpath   Selector=//*[@class='mat-focus-indicator log-in-button mat-raised-button mat-button-base']   StartNodeID=Default   Multiple=Default   ShadowRoot=Default
__WD_Post ==> Element interaction issue [14] : HTTP status = 400
_WD_ElementAction ==> Element interaction issue [14] : Parameters:   Command=click   Option=Default
+>11:18:22 AutoIt3.exe ended.rc:0
+>11:18:22 AutoIt3Wrapper Finished.
>Exit code: 0    Time: 18.56

 

[SOLVE-SMART]

Hi @Jemboy 👋 ,

this is not enough. We need the full log of both versions. Because no information about your capabilities string which is essential to know.
Please simply set:

$_WD_DEBUG = $_WD_DEBUG_Full

... and run again. Post your logs and maybe we are able to help.

Also please consider to test your automation with different browsers. How is the behavior in MSEdge or how in Firefox?
Another question: Do you use _WD_JsonActionKey in your code? Because I reviewed the changes between v1.3.1 and v1.4.0 and there is nothing which should break somehow. Besides function _WD_JsonActionKey I am sure about this.

Best regards
Sven

Edited September 27 by SOLVE-SMART

[Danp2]

3 hours ago, Jemboy said:

_WD_ElementAction ==> Element interaction issue [14] : Parameters:   Command=click   Option=Default

Most likely caused by a popup, banner, etc that doesn't always appear.

[Jemboy]

@SOLVE-SMARTand @Danp2 I uploaded both FULL debug log files as requested :-)
The script was run under the following conditions:

1. for both runs, the same computer was used
2. webdriver UDF has been updated to the latest version (v1.4.0)
3. to my knowledge I am not using _WD_JsonActionKey
4. the script used, is 100% the script posted in the first post
5. the computer is running WIndows 11 Home x64 23H2 (Dutch)
   tested this also on my Windows 10 Pro x64 22H2 at work, with same script results!

One thing I noticed, that the first run always results in the "wrong" screen as showed in my first post.
From the second run on, the script is running as intended. (done 6 re-runs after each other).
After every run, I always kill the chromedriver DOS box, to have the "same" run conditions.

When I wait several minutes and start the script again, the first run is bad again.
Subsequent runs immediately after the first run are all good runs.

Last week I installed an AutoIt "framework" so I can update my script from centrally.
I have migrated 3 scripts so far.
One of the "enhancements" was, that every time a script is started, it is centrally logged.

I can see that this particular script has issues, because according to the logs,
on several computers (not all) the script is restarted again within seconds of the first start.
So my users just seem to do a restart after the script misfires after the first run....
 

chrome_good_output_20240930.txt chrome_bad_output_20240930.txt

Edited September 30 by Jemboy

[SOLVE-SMART]

Hi @Jemboy 👋 ,

thanks for the explanation and the log files. I reviewed the logs and also your webdriver script and my assumption is: it's not a issue of the au3WebDriver at all. Out of the network tab in the Browser DevTools you can see some redirects that will be handled (by the js file content (which I reviewed)) differently in case of timing issues. So if in case of a bad network speed (network throttling) the redirection could be wrong or incomplete.

But this isn't important:
I tried several times for the suggested URL by @gmmg in this post which is https://login.tecalliance.net and had no issues to get on your expected login page.

My recommendation is, switch the URL and you are fine (I hope). Also consider to add more capabilities like in this example.
I hope this will help you.

Best regards
Sven

[Jemboy]

@SOLVE-SMARTThanks for looking into my problem. Also thanks for pointing me to the scrtip with the extra capabilities options.
The script I posted in this thread was the modified UDF webdriver example.
My production script does contain more settinsgs, however I wanted to have a clean as possible script to tackle the odd behaviour.

The reason I am hesitant of using @gmmg solution, is that it is not an official URL and might stop working in the future.
Nevertheless if I do not find a solution, I probably go that way.
For now I gonna make some time comming weekls to see how I can defeat the script behaving so strangely.

I do find an solution I will post it in this thread🙂

[Jemboy]

Alas @gmmg your workaround  https://login.tecalliance.net only partly works.
After login in, I get to another portal page with a button/widget, I do not seems to click automatically yet.
After pressing the button, I am back at the screen with the black pop-up.

However a positive note would be that (if I get to click the button), and not the popup, at least the user would be alreadly logged in.
Normally the users run the script only 1x a day unless yhey close the webpage.

At the below screenshot there is a href="https://login.tecalliance.....".
Instead of clicking the button/widget, I might be able to navigate automatically to the URL.
Is there a way to extract the URL with the webdriver UDF ?

 

[SOLVE-SMART]

On thy fly:

Local Const $sSelector = '//a[@aria-label="app TecCom Portal starten"]'
Local Const $sElement  = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, $sSelector)
Local Const $sUrl      = _WD_ElementAction($sSession, $sElement, 'Attribute', 'href') ; <== 😀

Best regards
Sven

Edited October 1 by SOLVE-SMART

[Jemboy]

@Danp2, @SOLVE-SMART and @gmmg
Felt I needed to give everyone an update....

The problem I had, turned out to be timing related.
Doing a _WD_LoadWait ($sSession,1000) after navigating to the website does not seems enough  delay to always fully load the page.
And thus not all elements are alway available on slower computers or if other processes are slowing the computer down.

For future versions of my scripts, I am gonna research this _WD_LoadWait, to see how I can tweak it with the paramters, to just wait enough time to "fully" load the page.
But timeout after say 5 seconds.

My solution and how I fixed this for now was as following:

1. (re)check all elements I needed and note 100% the order they are appearing
2. determine which elements are required and which one are optional.
   (I categorized pop-ups as being optional).
3.  Do a _WD_WaitElement for the elements I am waiting for.
4. If the element is required and not found exit/stop the script with a clear error message.
    When expected optional elements (pop-ups) are not found, then just continue the script.

With the above I seem to have fixed the script and I am getting the same results 99% of the time.
(I might have to experiment with the _WD_WaitElement timeouts more to go to 100% 🙂)

Special thanks to @gmmg for the URL, to remind me to think out of the box and not just use my own url because it is!
Also a special thanks to @SOLVE-SMART for your link to better use webdriver capabilities  and showing me to use other selectors than just id and class.😉
And a very very special thanks to @Danp2 the webdriver wizard. for helping me, others and for creating and maintaining the webdriver UDF.🙂