WebDriver UDF - Help & Support

PoojaKrishna · October 31, 2018

@Danp2, Thank you so much. using _WD_GetMouseElement fixed the issue.

I'm adding the script using _WD_GetElementFromPoint here for your review. It is supposed to highlight an element when mouse is over an element and exits when you click some where on the window. But it always finds and highlights an element below the mouse position.

#Region header files
#Include <Misc.au3> ; Used for the _Singleton
#include "wd_core.au3"
#include "wd_helper.au3"
#include <WinAPIConv.au3>

#EndRegion header files
#RequireAdmin
$_WD_DEBUG = 0
;$_WD_DEBUG = false
_WD_Option('Driver', 'chromedriver.exe')
_WD_Option('Port', 9515)
_WD_Startup()
$sLastElement = ""
$sWinState = """start-maximized"","
$sProfile = ""
 $sDesiredCapabilities =   '{"capabilities": {"alwaysMatch": {"chromeOptions": {"w3c": true, "args":[' & $sWinState & " ""disable-infobars""" & $sProfile &'] }}}}'
    ;$sDesiredCapabilities = '{"capabilities": {"alwaysMatch": {"chromeOptions": {"w3c": true, "args":["start-maximized", "disable-infobars","--user-data-dir=C:\\Users\\USER\\AppData\\Local\\Google\\Chrome\\User Data\\Profile 1"] }}}}'
    $sSession = _WD_CreateSession($sDesiredCapabilities)
    $sRet = _WD_Navigate($sSession, "https://www.facebook.com/")
    $TempWinhandle = _WD_Window($sSession, 'window')
    AutoItSetOption("MouseCoordMode",2)

$hDLL = DllOpen("user32.dll") ;used for IsPressed
While 1
    $sElement = ""
    If _IsPressed("01") Then ; If left mouse is pushed.
        While (_IsPressed("01", $hDLL)) ;Wait until user releases button

        WEnd
        ExitLoop
    EndIf
    $Pos = MouseGetPos() ;get mouse position
    $sTempX =  $Pos[0]
    $sTempY = $Pos[1]
    ;$sElement = _WD_GetMouseElement($sSession)
    $sElement = _WD_GetElementFromPoint($sSession,$sTempX ,$sTempY )
    If $sLastElement <> "" and $sLastElement <> $sElement Then _WD_RemoveBorderElement($sSession, $sLastElement)
    If $sElement <> "" and $sLastElement <> $sElement Then
        _WD_HighlightElement($sSession, $sElement, 3)
        $sLastElement = $sElement
    EndIf
    Sleep(100)
WEnd
DllClose($hDLL)

Func _WD_RemoveBorderElement($sSession, $sElement)
    Local $sJsonElement = '{"' & $_WD_ELEMENT_ID & '":"' & $sElement & '"}'
    Local $sResponse = _WD_ExecuteScript($sSession, "arguments[0].style='" & "border: 0px" & "'; return true;", $sJsonElement)
    Local $sJSON = Json_Decode($sResponse)
    Local $sResult = Json_Get($sJSON, "[value]")
    Return ($sResult = "true" ? SetError(0, 0, $sResult) : SetError(1, 0, False))
EndFunc   ;==>_WD_HighlightElement

Danp2 · October 31, 2018

You may need to use AutoItSetOption to change the value of MouseCoordMode. Also, here's some code I've used in the past to do something similar pre-webdriver --

Func _IEGetMouseElement($oIE, $aPoint = MouseGetPos())

    Local $hWindow = _IEPropertyGet($oIE, "hwnd")
    Local $aIEPos = _ScreenToClient($hWindow, $aPoint[0], $aPoint[1])

    Local $nScreenY = _IEPropertyGet($oIE, "screeny")
    Local $nScreenTop = $oIE.document.parentWindow.screenTop

    ; Adjust Y value for height of Tabs, Toolbars, etc
    $aIEPos[1] -= ($nScreenTop - $nScreenY)

    $oElement = $oIE.document.elementFromPoint($aIEPos[0], $aIEPos[1])

    Return $oElement
EndFunc

You may need to make similar adjustment to the Y value for it to work.

PoojaKrishna · November 1, 2018

9 hours ago, Danp2 said:

You may need to use AutoItSetOption to change the value of MouseCoordMode. Also, here's some code I've used in the past to do something similar pre-webdriver --

Func _IEGetMouseElement($oIE, $aPoint = MouseGetPos())

    Local $hWindow = _IEPropertyGet($oIE, "hwnd")
    Local $aIEPos = _ScreenToClient($hWindow, $aPoint[0], $aPoint[1])

    Local $nScreenY = _IEPropertyGet($oIE, "screeny")
    Local $nScreenTop = $oIE.document.parentWindow.screenTop

    ; Adjust Y value for height of Tabs, Toolbars, etc
    $aIEPos[1] -= ($nScreenTop - $nScreenY)

    $oElement = $oIE.document.elementFromPoint($aIEPos[0], $aIEPos[1])

    Return $oElement
EndFunc

You may need to make similar adjustment to the Y value for it to work.

I have tried with AutoItSetOption and _ScreenToClient before posting it into the forum but it did not work for me. I'm sorry I do not have the code written with me now. The _ScreenToClient was always returning a failure.

Yes, need to make adjustment to the Y value. Will try again.

Thank you.

PoojaKrishna · November 2, 2018

I'm sorry. I couldn't make any progress for the _WD_GetElementFromPoint to get the element from mouse position.

This is what I have tried yet. The _WinAPI_ScreenToClient always fail to convert the coordinates. I can not use _WD_GetMouseElement as I'm placing a transparent layer above the web page.

Can anyone please help?

#Region header files
#include <Misc.au3> ;Is Pressed
#include "wd_core.au3"
#include "wd_helper.au3"
#include <WinAPIConv.au3>
#include <WinAPIConv.au3>
#EndRegion header files
#RequireAdmin
$_WD_DEBUG = 0
_WD_Option('Driver', 'chromedriver.exe')
_WD_Option('Port', 9515)
_WD_Startup()
$sLastElement = ""
$sSession = _WD_CreateSession('{"capabilities": {"alwaysMatch": {"chromeOptions": {"w3c": true, "args":[' & """start-maximized""," & " ""disable-infobars""" & "" &'] }}}}')
_WD_Navigate($sSession, "https://www.facebook.com/")
$hWinhandle = _WD_Window($sSession, 'window')
;AutoItSetOption("MouseCoordMode",2)
$hDLL = DllOpen("user32.dll") ;used for IsPressed
While 1
    $sElement = ""
    If _IsPressed("01") Then ; If left mouse is pushed.
        While (_IsPressed("01", $hDLL)) ;Wait until user releases button
        WEnd
        ExitLoop
    EndIf
    $MousePos = MouseGetPos() ;get mouse position
    Local $tPoint = DllStructCreate("int X;int Y")
    DllStructSetData($tPoint, "X", $MousePos[0])
    DllStructSetData($tPoint, "Y", $MousePos[1])
    $Ret = _WinAPI_ScreenToClient($hWinhandle, $tPoint)
    $sElement = _WD_GetElementFromPoint($sSession,DllStructGetData($tPoint, "X") ,DllStructGetData($tPoint, "Y") )
    If $sLastElement <> "" and $sLastElement <> $sElement Then _WD_RemoveBorderElement($sSession, $sLastElement)
    If $sElement <> "" and $sLastElement <> $sElement Then
        _WD_HighlightElement($sSession, $sElement, 3)
        $sLastElement = $sElement
    EndIf
    Sleep(100)
WEnd
DllClose($hDLL)
_WD_Shutdown()
Func _WD_RemoveBorderElement($sSession, $sElement)
    Local $sJsonElement = '{"' & $_WD_ELEMENT_ID & '":"' & $sElement & '"}'
    Local $sResponse = _WD_ExecuteScript($sSession, "arguments[0].style='" & "border: 0px" & "'; return true;", $sJsonElement)
    Local $sJSON = Json_Decode($sResponse)
    Local $sResult = Json_Get($sJSON, "[value]")
    Return ($sResult = "true" ? SetError(0, 0, $sResult) : SetError(1, 0, False))
EndFunc   ;==>_WD_HighlightElement

Danp2 · November 2, 2018

Can you show a website where _WD_GetMouseElement isn't returning the correct element?

PoojaKrishna · November 3, 2018

6 hours ago, Danp2 said:

Can you show a website where _WD_GetMouseElement isn't returning the correct element?

_WD_GetMouseElement returns the correct element only.

Danp2 · November 3, 2018

8 minutes ago, PoojaKrishna said:

_WD_GetMouseElement returns the correct element only.

Sorry, but I'm confused. Previously you stated this --

9 hours ago, PoojaKrishna said:

I can not use _WD_GetMouseElement as I'm placing a transparent layer above the web page.

so I was interested in seeing a website where _WD_GetMouseElement wasn't returning the desired element. That way, I could take a look and see if there was a way to make it work for the given situation.

PoojaKrishna · November 3, 2018

I'm sorry , If my statement confused you. I meant to tell you that in my program I want to use _WD_GetElementFromPoint. When my actual program executes a transparent window is placed over the web page. All the user actions including the mouse move are on the transparent window. so when user moves the mouse, the mouse pointer is not on the web page but on my transparent window. That is why I can not use the _WD_GetMouseElement.

Sorry for my bad English.

Danp2 · November 3, 2018

Yes, I understood that part. Can you reproduce this for me in a short script so that I can observe the behavior with the transparent window in place?

PoojaKrishna · November 3, 2018

The transparent layer is not an AutoIT one and not done by me, still I may be able to post an executable to reproduce the transparent layer . The script has control only on the session opened behind the transparent window. I was trying some other solutions but no luck yet.

Danp2 · November 3, 2018

Can you show the relevant HTML for this transparent layer? With that, I should be able to duplicate the scenario.

PoojaKrishna · November 4, 2018

Sorry, there is no HTML code, it is not a web page. The transparent one is a transparent window created using C#, I think. I can post it on Monday.

Edited November 4, 2018 by PoojaKrishna

Danp2 · November 4, 2018

@PoojaKrishna Try out this modified version of your code --

#include "wd_core.au3"
#include "wd_helper.au3"
#include <Misc.au3>
#include <WinAPIConv.au3>

Local $sDesiredCapabilities, $sSession

SetupChrome()

_WD_Startup()
$sLastElement = ""

$sSession = _WD_CreateSession($sDesiredCapabilities)

_WD_Navigate($sSession, "https://www.facebook.com/")
$hWinhandle = _WD_Window($sSession, 'window')

Local $aOffsets = _CalcMouseOffset()

;AutoItSetOption("MouseCoordMode",2)
$hDLL = DllOpen("user32.dll") ;used for IsPressed
While 1
    $sElement = ""
    If _IsPressed("01") Then ; If left mouse is pushed.
        While (_IsPressed("01", $hDLL)) ;Wait until user releases button
        WEnd
        ExitLoop
    EndIf
    $MousePos = MouseGetPos() ;get mouse position
    Local $tPoint = DllStructCreate("int X;int Y")
    DllStructSetData($tPoint, "X", $MousePos[0])
    DllStructSetData($tPoint, "Y", $MousePos[1])
    $Ret = _WinAPI_ScreenToClient($hWinhandle, $tPoint)


    $sElement = _WD_GetElementFromPoint($sSession, DllStructGetData($tPoint, "X") - $aOffsets[0], DllStructGetData($tPoint, "Y") - $aOffsets[1])
    If $sLastElement <> "" And $sLastElement <> $sElement Then _WD_RemoveBorderElement($sSession, $sLastElement)
    If $sElement <> "" And $sLastElement <> $sElement Then
        _WD_HighlightElement($sSession, $sElement, 3)
        $sLastElement = $sElement
    EndIf
    Sleep(100)
WEnd

DllClose($hDLL)
_WD_Shutdown()

Func _WD_RemoveBorderElement($sSession, $sElement)
    Local $sJsonElement = '{"' & $_WD_ELEMENT_ID & '":"' & $sElement & '"}'
    Local $sResponse = _WD_ExecuteScript($sSession, "arguments[0].style='" & "border: 0px" & "'; return true;", $sJsonElement)
    Local $sJSON = Json_Decode($sResponse)
    Local $sResult = Json_Get($sJSON, "[value]")
    Return ($sResult = "true" ? SetError(0, 0, $sResult) : SetError(1, 0, False))
EndFunc   ;==>_WD_RemoveBorderElement

Func _CalcMouseOffset()
    Local $sResponse = _WD_Window($sSession, "rect")
    Local $oJson = Json_Decode($sResponse)

    Local $aPos[4]
    $aPos[0] = Json_Get($oJson, ".value.x")
    $aPos[1] = Json_Get($oJson, ".value.y")
    $aPos[2] = Json_Get($oJson, ".value.width")
    $aPos[3] = Json_Get($oJson, ".value.height")

    Local $iXPos = $aPos[0] + $aPos[2] / 2, $iYPos = $aPos[1] + $aPos[3] / 2

    ; Center mouse on window and then trigger a MouseMove event
    ; so that we can capture the event's mouse coordinates
    MouseMove($iXPos + 1, $iYPos + 1, 0)
    _WD_ExecuteScript($sSession, "document.getElementsByTagName('html')[0].addEventListener( 'mousemove', function( event ) { window.mouseXPos = event.pageX; window.mouseYPos = event.pageY; this.removeEventListener('mousemove',arguments.callee,false); });")
    MouseMove($iXPos, $iYPos, 0)

    $sResponse = _WD_ExecuteScript($sSession, "return [mouseXPos, mouseYPos]")
    $oJson = Json_Decode($sResponse)

    Local $iWDXPos = Json_Get($oJson, ".value[0]")
    Local $iWDYPos = Json_Get($oJson, ".value[1]")

    Local $aResult[2]
    $aResult[0] = $iXPos - $iWDXPos
    $aResult[1] = $iYPos - $iWDYPos
    Return $aResult
EndFunc   ;==>_CalcMouseOffset

Func SetupChrome()
    _WD_Option('Driver', 'chromedriver.exe')
    _WD_Option('Port', 9515)
    _WD_Option('DriverParams', '--log-path="' & @ScriptDir & '\chrome.log"')

    $sDesiredCapabilities = '{"capabilities": {"alwaysMatch": {"chromeOptions": {"w3c": true, "args":[' & """start-maximized""," & " ""disable-infobars""" & "" & '] }}}}'
EndFunc   ;==>SetupChrome

Func SetupGecko()
    _WD_Option('Driver', 'geckodriver.exe')
    _WD_Option('DriverParams', '--log trace')
    _WD_Option('Port', 4444)

    $sDesiredCapabilities = '{"desiredCapabilities":{"javascriptEnabled":true,"nativeEvents":true,"acceptInsecureCerts":true}}'
EndFunc   ;==>SetupGecko

PoojaKrishna · November 5, 2018

Sorry, it is not working on my system. $Ret = _WinAPI_ScreenToClient($hWinhandle, $tPoint), always returns a zero. I'm trying to fix it. i will update my progress.

Edited November 5, 2018 by PoojaKrishna

PoojaKrishna · November 5, 2018

Dan,

Please see the attached code. Calculating the offset as Desktop height - ( Client Height + Task Bar height). Please review and let me know your suggestions.

The transparent layer executable is also attached. Please execute the TransparentLayer.exe after executing the script to see how it looks together. Please press the Esc key to exit the TrasparentLayer.exe

#cs
Reference
https://www.w3schools.com/jsref/prop_win_innerheight.asp
#ce
#include "wd_core.au3"
#include "wd_helper.au3"
#include <Misc.au3> ;Is pressed
#include <string.au3>

Local $sDesiredCapabilities, $sSession

SetupChrome()

_WD_Startup()
$sLastElement = ""
$sSession = _WD_CreateSession($sDesiredCapabilities)
_WD_Window($sSession, 'Maximize')
_WD_Navigate($sSession, "https://www.facebook.com/")
$sResponse = _WD_ExecuteScript($sSession, "return document.documentElement.clientHeight")
$pos = ControlGetPos("[CLASS:Shell_TrayWnd]","", "[CLASS:MSTaskListWClass; INSTANCE:1]")
$aClientheight = _StringBetween($sResponse,'{"value":' , '}')
$nOffset =  @DesktopHeight - ( $aClientheight[0] + $pos[3]) ;Desktop height - ( Client Height + TaskBar height)
$hDLL = DllOpen("user32.dll") ;used for IsPressed
While 1
    $sElement = ""
    $MousePos = MouseGetPos()
    $nX =  $MousePos[0]
    $nY = $MousePos[1] - $nOffset
    If $nX > -1 and $nY > -1 Then
        $sElement = _WD_GetElementFromPoint($sSession, $nX, $nY)
    EndIf
    If $sLastElement <> "" And $sLastElement <> $sElement Then _WD_RemoveBorderElement($sSession, $sLastElement)
    If $sElement <> "" And $sLastElement <> $sElement Then
        _WD_HighlightElement($sSession, $sElement, 3)
        $sLastElement = $sElement
    EndIf
    Sleep(100)
WEnd
DllClose($hDLL)
_WD_Shutdown()

Func _WD_RemoveBorderElement($sSession, $sElement)
    Local $sJsonElement = '{"' & $_WD_ELEMENT_ID & '":"' & $sElement & '"}'
    Local $sResponse = _WD_ExecuteScript($sSession, "arguments[0].style='" & "border: 0px" & "'; return true;", $sJsonElement)
    Local $sJSON = Json_Decode($sResponse)
    Local $sResult = Json_Get($sJSON, "[value]")
    Return ($sResult = "true" ? SetError(0, 0, $sResult) : SetError(1, 0, False))
EndFunc   ;==>_WD_RemoveBorderElement
Func SetupChrome()
    _WD_Option('Driver', 'chromedriver.exe')
    _WD_Option('Port', 9515)
    _WD_Option('DriverParams', '--log-path="' & @ScriptDir & '\chrome.log"')

    $sDesiredCapabilities = '{"capabilities": {"alwaysMatch": {"chromeOptions": {"w3c": true, "args":[' & """start-maximized""," & " ""disable-infobars""" & "" & '] }}}}'
EndFunc   ;==>SetupChrome

Func SetupGecko()
    _WD_Option('Driver', 'geckodriver.exe')
    _WD_Option('DriverParams', '--log trace')
    _WD_Option('Port', 4444)

    $sDesiredCapabilities = '{"desiredCapabilities":{"javascriptEnabled":true,"nativeEvents":true,"acceptInsecureCerts":true}}'
EndFunc   ;==>SetupGecko

TransparentLayer.exe

Edited November 5, 2018 by PoojaKrishna

Danp2 · November 5, 2018

I don't believe I made any changes related to the _WinAPI_ScreenToClient call, so unsure why this would suddenly stop working for you.

FWIW, I'm not sure that either of our solutions will work when the window isn't maximized.

PoojaKrishna · November 5, 2018

Yes, but in my program the window is always in a maximized state so I hope it will work..

Thank you for your help Dan. Will try to develop an equation for not maximized window also.

54 minutes ago, Danp2 said:

I don't believe I made any changes related to the _WinAPI_ScreenToClient call, so unsure why this would suddenly stop working for you

Is that anything related with the screen resolution? Mine is 1366 X 768.

LeBlanc · November 6, 2018

Hello sir,

thank for such a great UDF, however may you help with <input type="file"> ?, i try many _WD_ElementAction but cannot open the dialoge box.

i am doing like this :

$sElement = _WD_FindElement($sSession,$_WD_LOCATOR_ByXPath,"//input[@name='photo']")
_WD_ElementAction($sSession, $sElement, 'click');

even

$sElement = _WD_FindElement($sSession,$_WD_LOCATOR_ByXPath,"//input[@type='file']")
_WD_ElementAction($sSession, $sElement, 'click');

but no luck

thank you so much

Danp2 · November 6, 2018

Have you tried this?

_WD_ElementAction($sSession, $sElement, 'value', '<full path to file>');

bubbyw · November 14, 2018

Hi Dan

Having a problem using the command below. The "\" character seems to break the $sOption variable and the action inputs no text. Any help would be appreciated!

_WD_ElementAction($sSession, $sElement, 'value', "test.domain\user")

WebDriver UDF - Help & Support

Recommended Posts

Link to comment

Share on other sites

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Danp2

Danyfirex

Gianni

Posted Images

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Recently Browsing 0 members

Similar Content