Jump to content

WebDriver UDF - Help & Support

Danp2

By Danp2,
in AutoIt General Help and Support

 Share

Recommended Posts

PoojaKrishna Polymath

PoojaKrishna

Posted
Posted

@Danp2, Thank you so much. using  _WD_GetMouseElement fixed the issue.

I'm adding the script using _WD_GetElementFromPoint here for your review. It is supposed to highlight an element when mouse is over an element and exits when you click some where on the window. But it always finds and highlights an element below the mouse position.

  

#Region header files
#Include <Misc.au3> ; Used for the _Singleton
#include "wd_core.au3"
#include "wd_helper.au3"
#include <WinAPIConv.au3>

#EndRegion header files
#RequireAdmin
$_WD_DEBUG = 0
;$_WD_DEBUG = false
_WD_Option('Driver', 'chromedriver.exe')
_WD_Option('Port', 9515)
_WD_Startup()
$sLastElement = ""
$sWinState = """start-maximized"","
$sProfile = ""
 $sDesiredCapabilities =   '{"capabilities": {"alwaysMatch": {"chromeOptions": {"w3c": true, "args":[' & $sWinState & " ""disable-infobars""" & $sProfile &'] }}}}'
    ;$sDesiredCapabilities = '{"capabilities": {"alwaysMatch": {"chromeOptions": {"w3c": true, "args":["start-maximized", "disable-infobars","--user-data-dir=C:\\Users\\USER\\AppData\\Local\\Google\\Chrome\\User Data\\Profile 1"] }}}}'
    $sSession = _WD_CreateSession($sDesiredCapabilities)
    $sRet = _WD_Navigate($sSession, "https://www.facebook.com/")
    $TempWinhandle = _WD_Window($sSession, 'window')
    AutoItSetOption("MouseCoordMode",2)

$hDLL = DllOpen("user32.dll") ;used for IsPressed
While 1
    $sElement = ""
    If _IsPressed("01") Then ; If left mouse is pushed.
        While (_IsPressed("01", $hDLL)) ;Wait until user releases button

        WEnd
        ExitLoop
    EndIf
    $Pos = MouseGetPos() ;get mouse position
    $sTempX =  $Pos[0]
    $sTempY = $Pos[1]
    ;$sElement = _WD_GetMouseElement($sSession)
    $sElement = _WD_GetElementFromPoint($sSession,$sTempX ,$sTempY )
    If $sLastElement <> "" and $sLastElement <> $sElement Then _WD_RemoveBorderElement($sSession, $sLastElement)
    If $sElement <> "" and $sLastElement <> $sElement Then
        _WD_HighlightElement($sSession, $sElement, 3)
        $sLastElement = $sElement
    EndIf
    Sleep(100)
WEnd
DllClose($hDLL)

Func _WD_RemoveBorderElement($sSession, $sElement)
    Local $sJsonElement = '{"' & $_WD_ELEMENT_ID & '":"' & $sElement & '"}'
    Local $sResponse = _WD_ExecuteScript($sSession, "arguments[0].style='" & "border: 0px" & "'; return true;", $sJsonElement)
    Local $sJSON = Json_Decode($sResponse)
    Local $sResult = Json_Get($sJSON, "[value]")
    Return ($sResult = "true" ? SetError(0, 0, $sResult) : SetError(1, 0, False))
EndFunc   ;==>_WD_HighlightElement

 

 

Link to comment
Share on other sites
  • Replies 1.1k
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Danp2
Danp2

This is the "General Help and Support" thread for the WebDriver UDF. The UDF itself can be downloaded here. So if you have any questions, suggestions or errors please post here.

Danyfirex
Danyfirex

Hello @GregEisenberg You need to do this: Local $oJsonRect=_WD_ElementAction($sSession, $sElements, 'rect') ConsoleWrite("!" & Json_Get($oJsonRect, "[x]") & @CRLF) ConsoleWr

Gianni
Gianni

Time ago I wrote a function to extract data from  the source HTML code of tables. https://www.autoitscript.com/forum/topic/167679-read-data-from-html-tables-from-raw-html-source/ Seems that that fun

Posted Images

Danp2 Universalist

Danp2

Posted
  • Author
Posted

You may need to use AutoItSetOption to change the value of MouseCoordMode. Also, here's some code I've used in the past to do something similar pre-webdriver -- 

Func _IEGetMouseElement($oIE, $aPoint = MouseGetPos())

    Local $hWindow = _IEPropertyGet($oIE, "hwnd")
    Local $aIEPos = _ScreenToClient($hWindow, $aPoint[0], $aPoint[1])

    Local $nScreenY = _IEPropertyGet($oIE, "screeny")
    Local $nScreenTop = $oIE.document.parentWindow.screenTop

    ; Adjust Y value for height of Tabs, Toolbars, etc
    $aIEPos[1] -= ($nScreenTop - $nScreenY)

    $oElement = $oIE.document.elementFromPoint($aIEPos[0], $aIEPos[1])

    Return $oElement
EndFunc

You may need to make similar adjustment to the Y value for it to work.

Latest Webdriver UDF Release

Webdriver Wiki FAQs

Link to comment
Share on other sites
PoojaKrishna Polymath

PoojaKrishna

Posted
Posted
9 hours ago, Danp2 said:

You may need to use AutoItSetOption to change the value of MouseCoordMode. Also, here's some code I've used in the past to do something similar pre-webdriver -- 

Func _IEGetMouseElement($oIE, $aPoint = MouseGetPos())

    Local $hWindow = _IEPropertyGet($oIE, "hwnd")
    Local $aIEPos = _ScreenToClient($hWindow, $aPoint[0], $aPoint[1])

    Local $nScreenY = _IEPropertyGet($oIE, "screeny")
    Local $nScreenTop = $oIE.document.parentWindow.screenTop

    ; Adjust Y value for height of Tabs, Toolbars, etc
    $aIEPos[1] -= ($nScreenTop - $nScreenY)

    $oElement = $oIE.document.elementFromPoint($aIEPos[0], $aIEPos[1])

    Return $oElement
EndFunc

You may need to make similar adjustment to the Y value for it to work.

I have tried with AutoItSetOption and _ScreenToClient before posting it into the forum but it did not work for me. I'm sorry I do not have the code written with me now. The _ScreenToClient was always returning a failure.

Yes, need to make adjustment to the Y value. Will try again.

Thank you.

Link to comment
Share on other sites
PoojaKrishna Polymath

PoojaKrishna

Posted
Posted

I'm sorry. I couldn't make any progress for the _WD_GetElementFromPoint to get the element from mouse position.

This is what I have tried yet. The _WinAPI_ScreenToClient always fail to convert the coordinates. I can not use _WD_GetMouseElement  as I'm placing a transparent layer above the web page.

Can anyone please help? 

#Region header files
#include <Misc.au3> ;Is Pressed
#include "wd_core.au3"
#include "wd_helper.au3"
#include <WinAPIConv.au3>
#include <WinAPIConv.au3>
#EndRegion header files
#RequireAdmin
$_WD_DEBUG = 0
_WD_Option('Driver', 'chromedriver.exe')
_WD_Option('Port', 9515)
_WD_Startup()
$sLastElement = ""
$sSession = _WD_CreateSession('{"capabilities": {"alwaysMatch": {"chromeOptions": {"w3c": true, "args":[' & """start-maximized""," & " ""disable-infobars""" & "" &'] }}}}')
_WD_Navigate($sSession, "https://www.facebook.com/")
$hWinhandle = _WD_Window($sSession, 'window')
;AutoItSetOption("MouseCoordMode",2)
$hDLL = DllOpen("user32.dll") ;used for IsPressed
While 1
    $sElement = ""
    If _IsPressed("01") Then ; If left mouse is pushed.
        While (_IsPressed("01", $hDLL)) ;Wait until user releases button
        WEnd
        ExitLoop
    EndIf
    $MousePos = MouseGetPos() ;get mouse position
    Local $tPoint = DllStructCreate("int X;int Y")
    DllStructSetData($tPoint, "X", $MousePos[0])
    DllStructSetData($tPoint, "Y", $MousePos[1])
    $Ret = _WinAPI_ScreenToClient($hWinhandle, $tPoint)
    $sElement = _WD_GetElementFromPoint($sSession,DllStructGetData($tPoint, "X") ,DllStructGetData($tPoint, "Y") )
    If $sLastElement <> "" and $sLastElement <> $sElement Then _WD_RemoveBorderElement($sSession, $sLastElement)
    If $sElement <> "" and $sLastElement <> $sElement Then
        _WD_HighlightElement($sSession, $sElement, 3)
        $sLastElement = $sElement
    EndIf
    Sleep(100)
WEnd
DllClose($hDLL)
_WD_Shutdown()
Func _WD_RemoveBorderElement($sSession, $sElement)
    Local $sJsonElement = '{"' & $_WD_ELEMENT_ID & '":"' & $sElement & '"}'
    Local $sResponse = _WD_ExecuteScript($sSession, "arguments[0].style='" & "border: 0px" & "'; return true;", $sJsonElement)
    Local $sJSON = Json_Decode($sResponse)
    Local $sResult = Json_Get($sJSON, "[value]")
    Return ($sResult = "true" ? SetError(0, 0, $sResult) : SetError(1, 0, False))
EndFunc   ;==>_WD_HighlightElement

 

Link to comment
Share on other sites
Danp2 Universalist

Danp2

Posted
  • Author
Posted
8 minutes ago, PoojaKrishna said:

_WD_GetMouseElement  returns the correct element only.

Sorry, but I'm confused. Previously you stated this --

9 hours ago, PoojaKrishna said:

I can not use _WD_GetMouseElement  as I'm placing a transparent layer above the web page.

so I was interested in seeing a website where _WD_GetMouseElement  wasn't returning the desired element. That way, I could take a look and see if there was a way to make it work for the given situation. :)

Latest Webdriver UDF Release

Webdriver Wiki FAQs

Link to comment
Share on other sites
PoojaKrishna Polymath

PoojaKrishna

Posted
Posted

I'm sorry , If my statement confused you. I meant to tell you that in my program I want to use _WD_GetElementFromPoint. When my actual program executes a transparent window is placed over the web page. All the user actions including the mouse move are on the transparent window. so when user moves the mouse, the mouse pointer is not on the web page but on my transparent window. That is why I can not use the _WD_GetMouseElement.

Sorry for my bad English.

Link to comment
Share on other sites
PoojaKrishna Polymath

PoojaKrishna

Posted
Posted

The transparent layer is not an AutoIT one and not done by me, still I may be able to post an executable to reproduce the transparent layer . The script has control only on the session opened behind the transparent window. I was trying some other solutions but no luck yet.

Link to comment
Share on other sites
Danp2 Universalist

Danp2

Posted
  • Author
Posted

@PoojaKrishna Try out this modified version of your code -- 

#include "wd_core.au3"
#include "wd_helper.au3"
#include <Misc.au3>
#include <WinAPIConv.au3>

Local $sDesiredCapabilities, $sSession

SetupChrome()

_WD_Startup()
$sLastElement = ""

$sSession = _WD_CreateSession($sDesiredCapabilities)

_WD_Navigate($sSession, "https://www.facebook.com/")
$hWinhandle = _WD_Window($sSession, 'window')

Local $aOffsets = _CalcMouseOffset()

;AutoItSetOption("MouseCoordMode",2)
$hDLL = DllOpen("user32.dll") ;used for IsPressed
While 1
    $sElement = ""
    If _IsPressed("01") Then ; If left mouse is pushed.
        While (_IsPressed("01", $hDLL)) ;Wait until user releases button
        WEnd
        ExitLoop
    EndIf
    $MousePos = MouseGetPos() ;get mouse position
    Local $tPoint = DllStructCreate("int X;int Y")
    DllStructSetData($tPoint, "X", $MousePos[0])
    DllStructSetData($tPoint, "Y", $MousePos[1])
    $Ret = _WinAPI_ScreenToClient($hWinhandle, $tPoint)


    $sElement = _WD_GetElementFromPoint($sSession, DllStructGetData($tPoint, "X") - $aOffsets[0], DllStructGetData($tPoint, "Y") - $aOffsets[1])
    If $sLastElement <> "" And $sLastElement <> $sElement Then _WD_RemoveBorderElement($sSession, $sLastElement)
    If $sElement <> "" And $sLastElement <> $sElement Then
        _WD_HighlightElement($sSession, $sElement, 3)
        $sLastElement = $sElement
    EndIf
    Sleep(100)
WEnd

DllClose($hDLL)
_WD_Shutdown()

Func _WD_RemoveBorderElement($sSession, $sElement)
    Local $sJsonElement = '{"' & $_WD_ELEMENT_ID & '":"' & $sElement & '"}'
    Local $sResponse = _WD_ExecuteScript($sSession, "arguments[0].style='" & "border: 0px" & "'; return true;", $sJsonElement)
    Local $sJSON = Json_Decode($sResponse)
    Local $sResult = Json_Get($sJSON, "[value]")
    Return ($sResult = "true" ? SetError(0, 0, $sResult) : SetError(1, 0, False))
EndFunc   ;==>_WD_RemoveBorderElement

Func _CalcMouseOffset()
    Local $sResponse = _WD_Window($sSession, "rect")
    Local $oJson = Json_Decode($sResponse)

    Local $aPos[4]
    $aPos[0] = Json_Get($oJson, ".value.x")
    $aPos[1] = Json_Get($oJson, ".value.y")
    $aPos[2] = Json_Get($oJson, ".value.width")
    $aPos[3] = Json_Get($oJson, ".value.height")

    Local $iXPos = $aPos[0] + $aPos[2] / 2, $iYPos = $aPos[1] + $aPos[3] / 2

    ; Center mouse on window and then trigger a MouseMove event
    ; so that we can capture the event's mouse coordinates
    MouseMove($iXPos + 1, $iYPos + 1, 0)
    _WD_ExecuteScript($sSession, "document.getElementsByTagName('html')[0].addEventListener( 'mousemove', function( event ) { window.mouseXPos = event.pageX; window.mouseYPos = event.pageY; this.removeEventListener('mousemove',arguments.callee,false); });")
    MouseMove($iXPos, $iYPos, 0)

    $sResponse = _WD_ExecuteScript($sSession, "return [mouseXPos, mouseYPos]")
    $oJson = Json_Decode($sResponse)

    Local $iWDXPos = Json_Get($oJson, ".value[0]")
    Local $iWDYPos = Json_Get($oJson, ".value[1]")

    Local $aResult[2]
    $aResult[0] = $iXPos - $iWDXPos
    $aResult[1] = $iYPos - $iWDYPos
    Return $aResult
EndFunc   ;==>_CalcMouseOffset

Func SetupChrome()
    _WD_Option('Driver', 'chromedriver.exe')
    _WD_Option('Port', 9515)
    _WD_Option('DriverParams', '--log-path="' & @ScriptDir & '\chrome.log"')

    $sDesiredCapabilities = '{"capabilities": {"alwaysMatch": {"chromeOptions": {"w3c": true, "args":[' & """start-maximized""," & " ""disable-infobars""" & "" & '] }}}}'
EndFunc   ;==>SetupChrome

Func SetupGecko()
    _WD_Option('Driver', 'geckodriver.exe')
    _WD_Option('DriverParams', '--log trace')
    _WD_Option('Port', 4444)

    $sDesiredCapabilities = '{"desiredCapabilities":{"javascriptEnabled":true,"nativeEvents":true,"acceptInsecureCerts":true}}'
EndFunc   ;==>SetupGecko

 

Latest Webdriver UDF Release

Webdriver Wiki FAQs

Link to comment
Share on other sites
PoojaKrishna Polymath

PoojaKrishna

Posted
Posted (edited)

Dan,

Please see the attached code.  Calculating the offset as Desktop height - ( Client Height + Task Bar height). Please review and let me know your suggestions.

The transparent layer executable is also attached. Please execute the TransparentLayer.exe after executing the script to see how it looks together. Please press the Esc key to exit the TrasparentLayer.exe 

#cs
Reference
https://www.w3schools.com/jsref/prop_win_innerheight.asp
#ce
#include "wd_core.au3"
#include "wd_helper.au3"
#include <Misc.au3> ;Is pressed
#include <string.au3>

Local $sDesiredCapabilities, $sSession

SetupChrome()

_WD_Startup()
$sLastElement = ""
$sSession = _WD_CreateSession($sDesiredCapabilities)
_WD_Window($sSession, 'Maximize')
_WD_Navigate($sSession, "https://www.facebook.com/")
$sResponse = _WD_ExecuteScript($sSession, "return document.documentElement.clientHeight")
$pos = ControlGetPos("[CLASS:Shell_TrayWnd]","", "[CLASS:MSTaskListWClass; INSTANCE:1]")
$aClientheight = _StringBetween($sResponse,'{"value":' , '}')
$nOffset =  @DesktopHeight - ( $aClientheight[0] + $pos[3]) ;Desktop height - ( Client Height + TaskBar height)
$hDLL = DllOpen("user32.dll") ;used for IsPressed
While 1
    $sElement = ""
    $MousePos = MouseGetPos()
    $nX =  $MousePos[0]
    $nY = $MousePos[1] - $nOffset
    If $nX > -1 and $nY > -1 Then
        $sElement = _WD_GetElementFromPoint($sSession, $nX, $nY)
    EndIf
    If $sLastElement <> "" And $sLastElement <> $sElement Then _WD_RemoveBorderElement($sSession, $sLastElement)
    If $sElement <> "" And $sLastElement <> $sElement Then
        _WD_HighlightElement($sSession, $sElement, 3)
        $sLastElement = $sElement
    EndIf
    Sleep(100)
WEnd
DllClose($hDLL)
_WD_Shutdown()

Func _WD_RemoveBorderElement($sSession, $sElement)
    Local $sJsonElement = '{"' & $_WD_ELEMENT_ID & '":"' & $sElement & '"}'
    Local $sResponse = _WD_ExecuteScript($sSession, "arguments[0].style='" & "border: 0px" & "'; return true;", $sJsonElement)
    Local $sJSON = Json_Decode($sResponse)
    Local $sResult = Json_Get($sJSON, "[value]")
    Return ($sResult = "true" ? SetError(0, 0, $sResult) : SetError(1, 0, False))
EndFunc   ;==>_WD_RemoveBorderElement
Func SetupChrome()
    _WD_Option('Driver', 'chromedriver.exe')
    _WD_Option('Port', 9515)
    _WD_Option('DriverParams', '--log-path="' & @ScriptDir & '\chrome.log"')

    $sDesiredCapabilities = '{"capabilities": {"alwaysMatch": {"chromeOptions": {"w3c": true, "args":[' & """start-maximized""," & " ""disable-infobars""" & "" & '] }}}}'
EndFunc   ;==>SetupChrome

Func SetupGecko()
    _WD_Option('Driver', 'geckodriver.exe')
    _WD_Option('DriverParams', '--log trace')
    _WD_Option('Port', 4444)

    $sDesiredCapabilities = '{"desiredCapabilities":{"javascriptEnabled":true,"nativeEvents":true,"acceptInsecureCerts":true}}'
EndFunc   ;==>SetupGecko

 

TransparentLayer.exe

Edited by PoojaKrishna
Link to comment
Share on other sites
PoojaKrishna Polymath

PoojaKrishna

Posted
Posted

Yes, but in my program the window is always in a maximized state so I hope it will work..

Thank you for your help Dan. Will try to develop an equation for not maximized window also.

54 minutes ago, Danp2 said:

I don't believe I made any changes related to the _WinAPI_ScreenToClient call, so unsure why this would suddenly stop working for you

Is that anything related with the screen resolution? Mine is 1366 X 768.

Link to comment
Share on other sites
LeBlanc Seeker

LeBlanc

Posted
Posted

Hello sir,

thank for such a great UDF,  however may you help with <input type="file"> ?, i try many _WD_ElementAction but cannot open the dialoge box. 

i am doing like this :


$sElement = _WD_FindElement($sSession,$_WD_LOCATOR_ByXPath,"//input[@name='photo']")
_WD_ElementAction($sSession, $sElement, 'click'); 

even

$sElement = _WD_FindElement($sSession,$_WD_LOCATOR_ByXPath,"//input[@type='file']")
_WD_ElementAction($sSession, $sElement, 'click'); 

but no luck

thank you so much 

Link to comment
Share on other sites
  • 2 weeks later...
bubbyw Seeker

bubbyw

Posted
Posted

Hi Dan

Having a problem using the command below. The "\" character seems to break the $sOption variable and the action inputs no text. Any help would be appreciated!

_WD_ElementAction($sSession, $sElement, 'value', "test.domain\user")

 

 

Link to comment
Share on other sites
Guest
This topic is now closed to further replies.
 Share
Go to topic listing
×
×
  • Create New...