Danp2 Posted April 29, 2021 Author Posted April 29, 2021 32 minutes ago, CodeWriter said: I've tried the binary path route for Firefox but no love. I'd be happy to post you the details and error logs when the binary path is added Please do. Quote Yes. _WD_NewTab() works as expected when loading Edge without a profile. With a profile, it doesn't work I'll try to duplicate here. Quote Here is the error code it generates Technically, there aren't any generated errors. The click just doesn't occur (or perhaps occurs in the wrong location). Quote alphabetize the function definitions. As soon as I do that, someone will want it back in the original order. 🤣 You could always use Scite Jump, which has an option to sort the function names. Or you could simply use the editor's search function to locate the desired function. Latest Webdriver UDF Release Webdriver Wiki FAQs
CodeWriter Posted April 30, 2021 Posted April 30, 2021 (edited) Editing the Firefox setup function in wd_demo.au3 with the following capabilities string that includes the binary path and profile path: Func SetupGecko() _WD_Option('Driver', 'geckodriver.exe') _WD_Option('DriverParams', '--log trace') _WD_Option('Port', 4444) ;~ $sDesiredCapabilities = '{"capabilities": {"alwaysMatch": {"browserName": "firefox", "acceptInsecureCerts":true}}}' $sDesiredCapabilities = '{"capabilities": {"alwaysMatch": {"moz:firefoxOptions": {"binary": "C:/Program Files/Mozilla Firefox/firefox.exe", "args": ["-profile", "C:/Users/MyName/AppData/Roaming/Mozilla/Firefox"]},"browserName": "firefox", "acceptInsecureCerts":true}}}' EndFunc ;==>SetupGecko Yields this error output: _WD_IsLatestRelease: True _WD_IsLatestRelease ==> Success _WD_Startup: OS: WIN_10 WIN32_NT 19041 _WD_Startup: AutoIt: 3.3.14.5 _WD_Startup: WD.au3: 0.4.0.3 (Up to date) _WD_Startup: WinHTTP: 1.6.4.2 _WD_Startup: Driver: geckodriver.exe _WD_Startup: Params: --log trace _WD_Startup: Port: 4444 __WD_Post: URL=HTTP://xxx.x.x.x:xxx/session; $sData={"capabilities": {"alwaysMatch": {"moz:firefoxOptions": {"binary": "C:/Program Files/Mozilla Firefox/firefox.exe", "args": ["-profile", "C:/Users/MyName/AppData/Roaming/Mozilla/Firefox"]},"browserName": "firefox", "acceptInsecureCerts":true}}} __WD_Post: StatusCode=0; ResponseText=WinHTTP request timed out before Webdriver... __WD_Post ==> Send / Recv error: WinHTTP request timed out before Webdriver _WD_CreateSession: WinHTTP request timed out before Webdriver _WD_CreateSession ==> Webdriver Exception: HTTP status = 0 __WD_Delete: URL=HTTP://xxx.x.x.x:xxx/session/ __WD_Delete: StatusCode=405; ResponseText=HTTP method not allowed... _WD_DeleteSession: HTTP method not allowed +>19:04:30 AutoIt3.exe ended.rc:0 +>19:04:30 AutoIt3Wrapper Finished. >Exit code: 0 Time: 71.25 Edited April 30, 2021 by CodeWriter
Danp2 Posted April 30, 2021 Author Posted April 30, 2021 @CodeWriter I can only replicate that behavior if I provide an invalid path to the Firefox executable. When that occurs, I get this in the geckodriver console -- 1619748790295 geckodriver INFO Listening on 127.0.0.1:4444 1619748790639 webdriver::server DEBUG -> POST /session {"capabilities": {"alwaysMatch": {"moz:firefoxOptions": {"binary": "C:/Program Files/Mozilla Firefoxx/firefox.exe"},"browserName": "firefox", "acceptInsecureCerts":true}}} 1619748790639 geckodriver::capabilities DEBUG Trying to read firefox version from ini files 1619748790639 geckodriver::capabilities DEBUG Trying to read firefox version from binary 1619748790640 geckodriver::capabilities DEBUG Failed to get binary version 1619748790640 webdriver::server DEBUG <- 400 Bad Request {"value":{"error":"invalid argument","message":"binary is not a Firefox executable","stacktrace":""}} Are you sure that your path is correct? Latest Webdriver UDF Release Webdriver Wiki FAQs
CodeWriter Posted April 30, 2021 Posted April 30, 2021 Here is the method shown in the Wiki that uses a function to find the correct path to the profile and it doesn't work either: a Firefox window appears but the wd_demo.au3 script doesn't run or executes any WebDriver UDF functions. Interestingly, if I use a capability string that has the binary path but no profile settings, Firefox will load. It's similar to the MS Edge situation in that it functions without a profile but load the driver with a profile and things grind to a halt. Func GetDefaultFFProfile() Local $sDefault, $sProfilePath = '' Local $sProfilesPath = StringReplace(@AppDataDir, '\', '/') & "/Mozilla/Firefox/" Local $sFilename = $sProfilesPath & "profiles.ini" Local $aSections = IniReadSectionNames($sFilename) If Not @error Then For $i = 1 To $aSections[0] $sDefault = IniRead($sFilename, $aSections[$i], 'Default', '0') If $sDefault = '1' Then $sProfilePath = $sProfilesPath & IniRead($sFilename, $aSections[$i], "Path", "") ExitLoop EndIf Next EndIf Return $sProfilePath EndFunc ;==>GetDefaultFFProfile Func SetupGecko() _WD_Option('Driver', 'geckodriver.exe') _WD_Option('DriverParams', '--log trace') _WD_Option('Port', 4444) ;~ $sDesiredCapabilities = '{"capabilities": {"alwaysMatch": {"browserName": "firefox", "acceptInsecureCerts":true}}}' $sDesiredCapabilities = '{"capabilities": {"alwaysMatch": {"moz:firefoxOptions": {"binary": "C:/Program Files/Mozilla Firefox/firefox.exe", "args": ["-profile", "' & GetDefaultFFProfile() & '"]}}}}' EndFunc ;==>SetupGecko And the resulting console/error report: _WD_IsLatestRelease: True _WD_IsLatestRelease ==> Success _WD_Startup: OS: WIN_10 WIN32_NT 19041 _WD_Startup: AutoIt: 3.3.14.5 _WD_Startup: WD.au3: 0.4.0.3 (Up to date) _WD_Startup: WinHTTP: 1.6.4.2 _WD_Startup: Driver: geckodriver.exe _WD_Startup: Params: --log trace _WD_Startup: Port: 4444 __WD_Post: URL=HTTP://xxx.x.x.x:xxx/session; $sData={"capabilities": {"alwaysMatch": {"moz:firefoxOptions": {"binary": "C:/Program Files/Mozilla Firefox/firefox.exe", "args": ["-profile", "C:/Users/MyName/AppData/Roaming/Mozilla/Firefox/Profiles/1eq1eg3d.default"]}}}} __WD_Post: StatusCode=0; ResponseText=WinHTTP request timed out before Webdriver... __WD_Post ==> Send / Recv error: WinHTTP request timed out before Webdriver _WD_CreateSession: WinHTTP request timed out before Webdriver _WD_CreateSession ==> Webdriver Exception: HTTP status = 0
Schmand Posted April 30, 2021 Posted April 30, 2021 (edited) Hello, Unfortunately I get a permanent error when executing the script. The json.au3 as well as wd_core.au3 and wd_helper.au3 are all in the script folder. I'm still at the very beginning and therefore don't have much code yet. However, no matter what I adjust here, the error remains the same. I get the following error message permanently: Here is the source code: #include "wd_core.au3" Local $sDesiredCapabilities SetupChrome() _WD_Startup() $sSession = _WD_CreateSession($sDesiredCapabilities) _WD_Navigate($sSession, "http://sit") ConsoleWrite("URL=" & _WD_Action($sSession, 'url') & @CRLF) _WD_Shutdown() Func SetupChrome() _WD_Option('Driver', 'chromedriver.exe') _WD_Option('Port', 9515) _WD_Option('DriverParams', '--verbose --log-path="' & @ScriptDir & '\chrome.log"') $sDesiredCapabilities = '{"capabilities": {"alwaysMatch": {"goog:chromeOptions": {"w3c": true, "excludeSwitches": [ "enable-automation"], "useAutomationExtension": false }}}}' EndFunc In my browser URL I got a data:, value in browser bar. I hope someone has an idea. Kind regards Edited April 30, 2021 by Schmand
mLipok Posted April 30, 2021 Posted April 30, 2021 (edited) @Danp2 Is it possible to using WD+GoogleChrome open few URL's in separate TAB's and manage them concurently ? If yes .. could you be so nice and give me any hint on: How to switch between already opened URL's/TAB's ? Edited April 30, 2021 by mLipok Schmand 1 Signature beginning:* Please remember: "AutoIt"..... * Wondering who uses AutoIt and what it can be used for ? * Forum Rules ** ADO.au3 UDF * POP3.au3 UDF * XML.au3 UDF * IE on Windows 11 * How to ask ChatGPT for AutoIt Code * for other useful stuff click the following button: Spoiler Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST API * ErrorLog.au3 UDF - A logging Library * Include Dependency Tree (Tool for analyzing script relations) * Show_Macro_Values.au3 * My contribution to others projects or UDF based on others projects: * _sql.au3 UDF * POP3.au3 UDF * RTF Printer - UDF * XML.au3 UDF * ADO.au3 UDF * SMTP Mailer UDF * Dual Monitor resolution detection * * 2GUI on Dual Monitor System * _SciLexer.au3 UDF * SciTE - Lexer for console pane * Useful links: * Forum Rules * Forum etiquette * Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * Wiki: * Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Good coding practices in AutoIt * OpenOffice/LibreOffice/XLS Related: WriterDemo.au3 * XLS/MDB from scratch with ADOX IE Related: * How to use IE.au3 UDF with AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * IE in TaskScheduler * IE Embedded Control Versioning (use IE9+ and HTML5 in a GUI) * PDF Related: * How to get reference to PDF object embeded in IE * IE on Windows 11 * I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions * EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *I also encourage you to check awesome @trancexx code: * Create COM objects from modules without any demand on user to register anything. * Another COM object registering stuff * OnHungApp handler * Avoid "AutoIt Error" message box in unknown errors * HTML editor * winhttp.au3 related : * https://www.autoitscript.com/forum/topic/206771-winhttpau3-download-problem-youre-speaking-plain-http-to-an-ssl-enabled-server-port/ "Homo sum; humani nil a me alienum puto" - Publius Terentius Afer"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming" , be and \\//_. Anticipating Errors : "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty." Signature last update: 2023-04-24
Danp2 Posted April 30, 2021 Author Posted April 30, 2021 @CodeWriter It appears that the issue is related to using an existing profile and not the binary path. Have you verified that no other instances of Firefox are running? That could explain this behavior. Latest Webdriver UDF Release Webdriver Wiki FAQs
seadoggie01 Posted April 30, 2021 Posted April 30, 2021 @Schmand There are two required UDFs for the WebDriver. You seem to be missing the JSON UDF: All my code provided is Public Domain... but it may not work. Use it, change it, break it, whatever you want. Spoiler My Humble Contributions:Personal Function Documentation - A personal HelpFile for your functionsAcro.au3 UDF - Automating Acrobat ProToDo Finder - Find #ToDo: lines in your scriptsUI-SimpleWrappers UDF - Use UI Automation more Simply-erKeePass UDF - Automate KeePass, a password managerInputBoxes - Simple Input boxes for various variable types
Danp2 Posted April 30, 2021 Author Posted April 30, 2021 @Schmand Are you using the full version of Scite? If so, you should be able to press Ctrl+F5 to perform a syntax check on the script. That will generally identify an issues such as missing #include files. When you run the script from Scite, you should get some helpful diagnostic information posted in the Output panel. Please post that here for review. Schmand 1 Latest Webdriver UDF Release Webdriver Wiki FAQs
Danp2 Posted April 30, 2021 Author Posted April 30, 2021 @mLipok Maybe I'm not understanding your request. Doesn't the DemoNavigation function in wd_demo.au3 already cover this? Latest Webdriver UDF Release Webdriver Wiki FAQs
Schmand Posted April 30, 2021 Posted April 30, 2021 It seems that I simply used the outdated version. With the new one it works now. Thank you
mLipok Posted April 30, 2021 Posted April 30, 2021 (edited) 32 minutes ago, Danp2 said: @mLipok Maybe I'm not understanding your request. Doesn't the DemoNavigation function in wd_demo.au3 already cover this? ufff. I forget to check. Must check again. Will back with answer and maybe with my own example when I check. Edited April 30, 2021 by mLipok Signature beginning:* Please remember: "AutoIt"..... * Wondering who uses AutoIt and what it can be used for ? * Forum Rules ** ADO.au3 UDF * POP3.au3 UDF * XML.au3 UDF * IE on Windows 11 * How to ask ChatGPT for AutoIt Code * for other useful stuff click the following button: Spoiler Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST API * ErrorLog.au3 UDF - A logging Library * Include Dependency Tree (Tool for analyzing script relations) * Show_Macro_Values.au3 * My contribution to others projects or UDF based on others projects: * _sql.au3 UDF * POP3.au3 UDF * RTF Printer - UDF * XML.au3 UDF * ADO.au3 UDF * SMTP Mailer UDF * Dual Monitor resolution detection * * 2GUI on Dual Monitor System * _SciLexer.au3 UDF * SciTE - Lexer for console pane * Useful links: * Forum Rules * Forum etiquette * Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * Wiki: * Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Good coding practices in AutoIt * OpenOffice/LibreOffice/XLS Related: WriterDemo.au3 * XLS/MDB from scratch with ADOX IE Related: * How to use IE.au3 UDF with AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * IE in TaskScheduler * IE Embedded Control Versioning (use IE9+ and HTML5 in a GUI) * PDF Related: * How to get reference to PDF object embeded in IE * IE on Windows 11 * I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions * EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *I also encourage you to check awesome @trancexx code: * Create COM objects from modules without any demand on user to register anything. * Another COM object registering stuff * OnHungApp handler * Avoid "AutoIt Error" message box in unknown errors * HTML editor * winhttp.au3 related : * https://www.autoitscript.com/forum/topic/206771-winhttpau3-download-problem-youre-speaking-plain-http-to-an-ssl-enabled-server-port/ "Homo sum; humani nil a me alienum puto" - Publius Terentius Afer"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming" , be and \\//_. Anticipating Errors : "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty." Signature last update: 2023-04-24
CodeWriter Posted April 30, 2021 Posted April 30, 2021 @Danp2There is only 1 instance of Firefox running on the machine--the one started by wd_dem0.au3. Yes, it's a bit dire with FireFox & Edge unable to run WebDriver with profiles and Chrome unable to run without zoom set at 100%.
Danp2 Posted April 30, 2021 Author Posted April 30, 2021 @CodeWriter I'm not aware of anyone else having these issues. FWIW, I am able to launch Firefox with an existing profile and control it with the UDF. It's possible that the issue is localized on your end. Latest Webdriver UDF Release Webdriver Wiki FAQs
Marlon13 Posted April 30, 2021 Posted April 30, 2021 Good day at all , I can't capture all the table inside the page, it's probably the syntax of the command that is wrong, could you tell me if it's okay or not in your opinion table in page <table data-barchart-table-show-symbol-details="" data-barchart-expland-new-rows="" data-ng-class="{'ordered-triggered': orderHasBeenATriggered, 'symbol-widgets-expanded': symbolWidgetsExpanded}"> command Local $aResult = _WD_GetTable($sSession, "//table['data-barchart-table-show-symbol-details']") _DebugArrayDisplay($aResult) return only a first nine line thanks at all
Danp2 Posted April 30, 2021 Author Posted April 30, 2021 5 minutes ago, Marlon13 said: //table['data-barchart-table-show-symbol-details'] That isn't the correct xpath. You might want to review this link for proper way to use the data attributes. FWIW, you should be able to access the tables with this xpath -- /descendant::table[1] ; first table /descendant::table[2] ; second table Latest Webdriver UDF Release Webdriver Wiki FAQs
Marlon13 Posted April 30, 2021 Posted April 30, 2021 I try to set in this mode , sorry but I don't know xpath Local $aResult = _WD_GetTable($sSession, '/descendant::table[1]') _DebugArrayDisplay($aResult) I hope is correct , in this mode return only 5 line
Danp2 Posted April 30, 2021 Author Posted April 30, 2021 @Marlon13 I just tried it and received back the entire table. Unsure why you would only get 5 rows returned, unless you encountered an error during the retrieve process. Did you examine the Scite output panel for any errors? Can you save this output to a file and then post it here for review? Latest Webdriver UDF Release Webdriver Wiki FAQs
agillbb Posted May 1, 2021 Posted May 1, 2021 Greetings - hope this is the correct place to ask a question. Fantastic WebDriver UDF, by the way. I am gathering data from a web page that uses a GET callback to grab JSON data from its server and then render it in the page. Rather than scrape the elements of the page, it would be great to simply save the JSON response to a variable and use the data directly. But I can't figure out how to access the GET response data using the WebDriver UDF. Is this possible, and how would you recommend doing it? Thanks.
Danp2 Posted May 1, 2021 Author Posted May 1, 2021 @agillbb Hard to say without further details. A few possibilities -- If the desired data is stored somewhere (like a javascript variable), then you could access it with _WD_ExecuteScript Depending on the website, you may be able to use _WD_ExecuteScript to execute a function that in turn retrieves the data If you know the correct URL, then you could try retrieving the data with __WD_Get If you are able to, share the details about the website and the specific GET so that we can take a look for ourselves. Latest Webdriver UDF Release Webdriver Wiki FAQs
Recommended Posts