WebDriver UDF (W3C compliant version) - 2024/09/21

[Danp2]

v0.3.0.8 just released with the following updates --

  - Changed (_WD_WaitElement): Added optional parameter to check elements enabled status
  - Changed (_WD_GetTable): Optionally support faster _HtmlTableGetWriteToArray
  - Changed (_WD_ElementAction): Allow retrieving element value with the 'value' command
  - Chore: Modified #include usage

[TheXman]

Below is my version of the _WD_GetTable() helper function (named _WD_GetTable_2) that I use for grabbing tables from web sites.  The difference between it and the existing _WD_GetTable() is that it spins thru the rows to find the maximum number of columns in the table.  As you know, the current _WD_GetTable() only checks the first <tr> to determine the number of columns.  I found that to be problematic for some of the tables that I have come across.  This is certainly not the fastest way to get table data, but it is a more reliable way to get table data and uses native WebDriver functions.  Maybe others may find it useful in those cases where the first row of the table doesn't reflect all of the columns in the body of the table.

expandcollapse popup

; #FUNCTION# ====================================================================================================================
; Name ..........: _WD_GetTable_2
; Description ...: Return all elements of a table
; Syntax ........: _WD_GetTable_2($sSession, $sBaseElement)
; Parameters ....: $sSession     - Session ID from _WDCreateSession
;                  $sTableXpath  - XPath of the table to return
; Return values .: Success      - 2D array
;                  Failure      - ""
;                  @ERROR       - $_WD_ERROR_Success
;                               - $_WD_ERROR_Exception
;                               - $_WD_ERROR_NoMatch
;                  @EXTENDED    - WinHTTP status code
; Author ........: TheXman
; Modified ......:
; Remarks .......: Number of rows and max cols are determined based on actual table data.
; Related .......:
; Link ..........:
; Example .......: No
; ===============================================================================================================================
Func _WD_GetTable_2($sSession, $sTableXpath)

    Const $sFuncName = "_WD_GetTable_2"

    Local $aTable[0], $aRowElements[0], $aColElements[0]
    Local $sRowColumns = ""
    Local $iRows = 0, $iCols = 0, $iChildCount = 0

    ;Get table row elements
    $aRowElements = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, $sTableXpath & "/tbody/tr", "", True)
    If @error <> $_WD_ERROR_Success Then Return SetError(__WD_Error($sFuncName, @error, "HTTP status = " & $_WD_HTTPRESULT), $_WD_HTTPRESULT, "")
    $iRows = UBound($aRowElements)

    ;Find max number of columns by checking each row's child count property
    For $iRow = 0 To UBound($aRowElements) - 1
        $iChildCount = _WD_ElementAction($sSession, $aRowElements[$iRow], "property", "childElementCount")
        If @error <> $_WD_ERROR_Success Then Return SetError(__WD_Error($sFuncName, @error, "HTTP status = " & $_WD_HTTPRESULT), $_WD_HTTPRESULT, "")

        ;If number of row columns is greater the previous value, then save new value
        If $iChildCount > $iCols Then $iCols = $iChildCount
    Next

    ;Dimension and load table data
    ReDim $aTable[0][$iCols]

    For $iRow = 0 To $iRows - 1
        ;Get column elements (td or th)
        $aColElements = _WD_FindElement($sSession, _
                                        $_WD_LOCATOR_ByXPath, _
                                        "./td|./th", _
                                        $aRowElements[$iRow], _
                                        True)
        If @error <> $_WD_ERROR_Success Then Return SetError(__WD_Error($sFuncName, @error, "HTTP status = " & $_WD_HTTPRESULT), $_WD_HTTPRESULT, "")

        ;Build pipe-separated row of column data
        $sRowColumns = ""
        For $iCol = 0 To UBound($aColElements) - 1
            If $iCol = 0 Then
                $sRowColumns &= _WD_ElementAction($sSession, $aColElements[$iCol], "text")
            Else
                $sRowColumns &= "|" & _WD_ElementAction($sSession, $aColElements[$iCol], "text")
            EndIf
        Next

        ;Add row to array
        _ArrayAdd($aTable, $sRowColumns)
    Next

    ;Return result table
    Return $aTable

EndFunc

 

Here's an example of a table and its definition that the current _WD_GetTable() has a problem with because the first <tr> does not accurately define the max number of columns.

 

Edited August 30, 2020 by TheXman
Updated script to handle "irregular" tables without throwing and error

[Danp2]

@TheXman Where can I access this table?

[TheXman]

https://fftoday.com/nfl/schedule.php

It is a 1200 cell table, so it takes a bit to build. It is 300 rows with 4 columns per row.

Edited August 29, 2020 by TheXman

[Danp2]

@Chimp's _HtmlTableGetWriteToArray seem to handle this table correctly. Just need to modify the call to filter out the non-breaking spaces.

[TheXman]

1 hour ago, Danp2 said:

@Chimp's _HtmlTableGetWriteToArray seem to handle this table correctly.

Of course this assumes that any given user has found and included that function.  If you plan to include it in one of your UDFs, then it really isn't an issue.  If not, then the current _WD_GetTable() helper function would not properly handle such tables by default.  I didn't offer the function as a replacement or to compete with Chimp's function.  I just threw it out there as an alternative and to show a relatively easy, native WebDriver, way to handle all tables including edge cases like my example.

Edited August 29, 2020 by TheXman

[Danp2]

@TheXman FWIW, I recently modified _WD_GetTable so that it will use _HtmlTableGetWriteToArray instead of the original coding if this UDF is available.

Appreciate your input, and I'll continue to do some testing to with your code.

[TheXman]

Yes, I noticed the updated _WD_Getable() function yesterday.  That's why I mentioned the part about "If a user has found and included the function".  Because if they hadn't, it would fall back to using the existing logic.

I hadn't started playing with the WebDriver UDFs until 2 days ago.  It is quite an impressive piece of work.  Kudos to you and all of the contributors! 

Edited August 29, 2020 by TheXman

[Danp2]

I updated the Readme on Github so that it shows the _HtmlTableGetWriteToArray UDF as optional. Probably need to do the same to the 1st post of the this thread.

Glad you are enjoying the UDF. Always willing to accept submissions for inclusion. 😉

Edit: 1st post updated

Edited August 29, 2020 by Danp2

[TheXman]

I noticed one more small issue with the current _WD_GetTable helper function.  If you run it against the Customers table on https://www.w3schools.com/html/html_tables.asp you will see that it fails with a NoMatch because the first <tr> does not have <td> tags, it has <th> tags defining the headers.  The _WD_GetTable_2() function, that I posted above, also handles this issue.

_WD_GetTable($gsSession, '//table[@id="customers"]')

Edited August 29, 2020 by TheXman

[Gianni]

Hi @TheXman,

from a quick test on the above table it seems that I get the same array using both_WD_GetTable () and _WD_GetTable_2 (), but _WD_GetTable_2 () is much slower.
I also did a test on the second table at this link (https://www.w3.org/WAI/tutorials/tables/irregular/), but _WD_GetTable_2 () seems to go wrong.

_WD_GetTable_2($sSession, "/html/body/div/main/article/div/figure[4]/div/table")

Can you take a test on that table too?

Edited August 29, 2020 by Chimp

[TheXman]

@Chimp

_WD_GetTable_2(), and WebDriver in general, will always be MUCH slower because WebDriver has to make a GET/POST request for each and every thing it does.  Of course it will be much slower than a function that processes the raw source as a whole.  That was never a question and I stated so in my earlier post.  My original point was that without your function, the WebDriver helper function did not correctly process a table that I encountered.  In addition, _WD_GetTable2() will be slower because it checks all table rows, before processing it, to determine the max number of columns in the table.  The current _WD_GetTable() function, without your UDF, only looks at the first <tr> in order to determine the number of columns.  As I pointed out, that logic didn't work on the table that I provided.  As I also pointed out, it doesn't work on the W3Schools table example either.  I'm not sure what you tested and what you are referring to when you say that you get the same array with the existing function and my function when I clearly showed that isn't the case (assuming the original logic and not using your function).

There are 2 tables on the page that you referenced.  Are you referring to a particular table?

 

Edited August 29, 2020 by TheXman

[Gianni]

8 minutes ago, TheXman said:

There are 2 tables on the page that you referenced.  Are you referring to a particular table?

I did a test on the second table using the following xpath

_WD_GetTable_2($sSession, "/html/body/div/main/article/div/figure[4]/div/table")

 

[TheXman]

2 hours ago, Chimp said:

I also did a test on the second table at this link (https://www.w3.org/WAI/tutorials/tables/irregular/), but _WD_GetTable_2 () seems to go wrong.

Yes, _WD_GetTable_2() encountered a NoMatch while processing that table on the irregular tables site.  Thanks for pointing out those irregular tables so that I can make sure that _WD_GetTable_2() can process them correctly.  _WD_GetTable_2() was written to handle a table that I encountered that _WD_GetTable() couldn't at the time.  Of course that table didn't have row spans, column spans, and multiple <tbody> tags like the tables on the irregular table site.  Making the required changes, to be able to handle them, shouldn't be too difficult.

Edited August 29, 2020 by TheXman

[TheXman]

I made an initial update to the script in my previous post.  Although it now pulls all of the "irregular" table data without throwing any errors, the exact placement of the spanned rows and columns are not maintained as they are when using the _HtmlTableGetWriteToArray() function.  I doubt I will continue to work on it since it really serves no purpose for me until I come across such a table in the course of my scripting.  And if I need a quick fix, _HtmlTableGetWriteToArray() is always there to save the day.

 

Edited August 30, 2020 by TheXman

[Bot]

Using the _WD_ElementActionEx function, I can "right click" on an element to show the context menu. But how can I interact with the context menu, e.g selecting an item.

I tried to send keystrokes but did not success:

$elementInput = _WD_FindElement($sessionID, $_WD_LOCATOR_ByXPath, "(//input[contains(@name,'q')])[1]")
_WD_ElementActionEx($sessionID, $elementInput, "rightclick")

$sAction = '{"actions":[{"type": "key", "id": "default keyboard", "actions": [{"type": "keyDown", "value": "\uE015"}, {"type": "keyUp", "value": "\uE015"}, {"type": "keyDown", "value": "\uE015"}, {"type": "keyUp", "value": "\uE015"}, {"type": "keyDown", "value": "\uE015"}, {"type": "keyUp", "value": "\uE015"}, {"type": "keyDown", "value": "\uE015"}, {"type": "keyUp", "value": "\uE015"}]}]}'

; $sAction = '{"actions":[{"type": "key", "id": "default keyboard", "actions": [{"type": "keyUp", "value": "Q"}, {"type": "keyDown", "value": "Q"}]}]}' ; Send the key Q which is shortcut for the Inspect Element menu

_WD_Action($sessionID, "actions", $sAction)

Can anyone help me please?

Edited September 1, 2020 by Bot

[Danp2]

@Bot Have you tried with _WD_LinkClickByText?

[Bot]

@Danp2 No I'm referring to the context menu of the browser, not the custom context menu of the webpage, so _WD_LinkClickByText just failed with a No match error.

Spoiler

_WD_LinkClickByText($sessionID, "Inspect Element")

 

Edited September 1, 2020 by Bot

[Danp2]

Don't know of any way to do it via Webdriver. You can use standard Autoit command to trigger the popup entry --

Send("Q")

but this obviously needs the browser to be topmost to work. You may be able to do it with ControlSend, but I haven't looked into that.

Just curious... Can you explain why you need to automate this action?

[Bot]

I want to access the context menu to select an installed add-on. However it seems that the key events will work with custom context menu of the webpage only and they are sent to the element, not the native context menu of the browser. So I think I will have to use standard AutoIT command instead. Anyway thanks for the help 🙂

Edited September 1, 2020 by Bot