Spy on Internet Explorer Webbrowser

[junkew]

Some help is needed as I can not get the event information that is send when sinking events to autoIT of internet explorer.

I should be able to get all properties as described in

* http://msdn2.microsoft.com/en-us/library/a...876(VS.85).aspx

* Based on Help file of com

* Based on sample http://www.autoitscript.com/forum/index.ph...amp;hl=objevent

I added another mousehandler for mouseover by using code as below. This basically works but I am unable to do things like

consolewrite($e.clientX) or consolewrite($oIE.document.getparent.event.clientx)

Func IEEvent_DownloadComplete()
    GUICtrlSetData ( $GUIEdit, "IE has finished a navigation operation" & @CRLF  , "append" )
    $odoc=objcreate("IHtmlDocument2")
    $oDoc= $oIE.document
    $EventDoc=ObjEvent($oDoc,"Document_","HTMLDocumentEvents2")

EndFunc
Func Document_onmouseover($e)  
; This is an optional event function to catch non-defined events. 
; The parameter contains the name of the event being called.
    GUICtrlSetData ( $GUIEdit, "Doc mouseover" & @CRLF  , "append" )
EndFunc

Full code

=====

expandcollapse popup

; Example script, showing the usage of COM Event functions.
; Requires at least AutoIt beta version 3.1.1.104 !
;
; See also: [url="http://msdn.microsoft.com/workshop/browser/webbrowser/reference/objects/internetexplorer.asp"]http://msdn.microsoft.com/workshop/browser...netexplorer.asp[/url]
dim $odoc

; We use a very simple GUI to show the results of our Events.
#include "GUIConstants.au3"
$GUIMain=GUICreate            ( "Event Test",      600,500 )
$GUIEdit=GUICtrlCreateEdit    ( "Test Log:" & @CRLF,  10, 20, 580, 400)
$GUIProg=GUICtrlCreateProgress  (                      10,  5, 580,  10)
$GUIExit=GUICtrlCreateButton    ( " Close ",          250, 450, 80,  30)
GUISetState ()    ;Show GUI

; We prepare the Internet Explorer as our test subject
$oIE=ObjCreate("InternetExplorer.Application.1")
With $oIE
    .Visible=1
    .Top = (@DesktopHeight-400)/2
    .Height=400     ; Make it a bit smaller than our GUI.
    .Width=600
    .Silent=1         ; Don't show IE's dialog boxes
    $IEWnd=HWnd(.hWnd) ; Remember the Window, in case user decides to close it
EndWith

; We choose for a specific Internet Explorer interface 'DWebBrowserEvents' because the IE is subject
; to modifications by e.g. Visual Studio and Adobe Acrobat Reader. If you have IE-plugins installed, 
; AutoIt might not be able to find the correct interface automatically.
$EventObject=ObjEvent($oIE,"IEEvent_","DWebBrowserEvents")
if @error then 
   Msgbox(0,"AutoIt COM Test", _ 
    "ObjEvent: Can't use event interface 'DWebBrowserEvents'. Error code: " & hex(@error,8))
   exit
endif

; Now starting to load an example Web page.
$URL = "http://www.AutoItScript.com/"
$oIE.Navigate( $URL )          
sleep(1000)         ; Give it some time to load the web page

GUISwitch ( $GUIMain ) ; Switch back to our GUI in case IE stealed the focus

; Waiting for user to close the GUI.
While 1
   $msg = GUIGetMsg()
   If $msg = $GUI_EVENT_CLOSE or $msg = $GUIExit Then ExitLoop
Wend

$EventObject.Stop  ; Tell IE we don't want to receive events.
$EventObject=0   ; Kill the Event Object
If WinExists($IEWnd) then $oIE.Quit ; Close IE Window
$oIE=0           ; Remove IE from memory (not really necessary).

GUIDelete ()       ; Remove GUI

exit               ; End of our Demo.

; A few Internet Explorer Event Functions
; See also: [url="http://msdn.microsoft.com/workshop/browser/webbrowser/reference/objects/webbrowser.asp"]http://msdn.microsoft.com/workshop/browser.../webbrowser.asp[/url]

Func IEEvent_BeforeNavigate($URL, $Flags, $TargetFrameName, $PostData, $Headers, $Cancel)
;   Note: the declaration is different from the one on MSDN.
    GUICtrlSetData ( $GUIEdit, "BeforeNavigate: " & $URL & " Flags: " & $Flags & " tgframe: " & $TargetFrameName & " Postdat: " & $PostData & " Hdrs: " & $Headers & " canc: " & $Cancel  & @CRLF  , "append" )
EndFunc

Func IEEvent_ProgressChange($Progress,$ProgressMax)
    If $ProgressMax > 0 Then
        GUICtrlSetData($GUIProg, ($Progress * 100) / $ProgressMax )
    EndIf
EndFunc

Func IEEvent_StatusTextChange($Text)
    GUICtrlSetData ( $GUIEdit, "IE Status text changed to: " & $Text & @CRLF  , "append" )
EndFunc

Func IEEvent_PropertyChange( $szProperty)
    GUICtrlSetData ( $GUIEdit, "IE Changed the value of the property: " & $szProperty & @CRLF  , "append" )
EndFunc

Func IEEvent_DownloadComplete()
    GUICtrlSetData ( $GUIEdit, "IE has finished a navigation operation" & @CRLF  , "append" )
    $odoc=objcreate("IHtmlDocument2")
    $oDoc= $oIE.document
    $EventDoc=ObjEvent($oDoc,"Document_","HTMLDocumentEvents2")

EndFunc
Func Document_onmouseover($e)  
; This is an optional event function to catch non-defined events. 
; The parameter contains the name of the event being called.
    GUICtrlSetData ( $GUIEdit, "Doc mouseover" & @CRLF  , "append" )
EndFunc 

Func IEEvent_NavigateComplete($URL)  
;   Note: the declaration is different from the one on MSDN.
    GUICtrlSetData ( $GUIEdit, "IE has finished loading URL: " & $URL & @CRLF  , "append" )
EndFunc

Func IEEvent_($EventName)  
; This is an optional event function to catch non-defined events. 
; The parameter contains the name of the event being called.
    GUICtrlSetData ( $GUIEdit, "Uncatched event: " & $EventName & @CRLF  , "append" )
EndFunc

Edited March 10, 2008 by junkew

[DaleHohm]

The .event intrface is a problem due to the way that AutoIt handles COM events. You should view AutoIt as being "informed" of events rather than being in the event handling loop (you cannot block with AtuoIt and the .event items like keystate and mouse position have already changed by the time you get to act upon it).

My suggestion is to use _IEHeadInsertEventScript() (the the equivalent DOM code) to insert an event script for the element you want to monitor... in that script, write hidden custom data to an element on the page, then read data from that element with, perhaps an onchange event trigger.

Dale

[junkew]

Is it possible to get the event object the way I tried regardless if the values on clientX are still correct (I am more interested in the element I am hovering over like ID so I can use functions like $IE.document.getelementbyID($IDWeAreHoveringOver))

I am actually trying to accomplish a simple spying on webobjects to make my scripting life easier with the UDF supplied in IE.AU3.

With _IEHeadInsertEventScript() I have to add the onmouseover event to each object before I can actually spy.

(Then it comes more to bookmarklets/favlets like http://slayeroffice.com/tools/modi/v2.0/modi_help.html)

Basic thought is

1. to mouseover an HTML element

2. Write in a textbox the basic normal properties to identify

3. Have the spy program write me the code for the element I click on or when I am longer than n seconds above the same element it should write a piece of sourcecode to my editor or a temporary editbox within the GUI I create within AutoIT.

[DaleHohm]

Not so hard...

This is based on a concept put forth by big_daddy... it spits out info for each element you mouse over (ala ModIV2)

#include <IE.au3>

_IEErrorHandlerRegister()

$oIE = _IECreate("www.google.com")

$oBody = _IETagnameGetCollection($oIE, "body", 0)
$oTags = _IETagNameAllGetCollection($oBody)
Dim $oEvents[1]
$i = 0
For $oTag in $oTags
    $oEvents[$i] = ObjEvent($oTag, "IE_Evt_")
    $i += 1
    ReDim $oEvents[$i + 1]
Next

While True
    Sleep(100)
WEnd

Func IE_Evt_onmouseover()
    $o = @COM_EventObj
    ConsoleWrite($o.tagname & " : " & $o.id & @LF)
EndFunc

[junkew]

I only have one problem with this solution. I am actually changing than the HTML page where I preferabbly would not like to do this.

Is it possible without manipulating / adding script events to all elements to get the same information.

If you would write a spy like

if $WindowClass = "INTERNET_EXPLORER_SERVER" then

hookAllMouseOvers()

endIf

It will take a while before they are all hooked for all elements (could offcourse limit it to only INPUT and A tags or let user mark the tags he is interested in).

If on the opposite I could get to the event object with the properties its easier (and probably faster)

"The .event intrface is a problem due to the way that AutoIt handles COM events"

* Where can be read more on above statement and will it change in a future version of AutoIT?

* No other way to get to the event object? I am thinking about adding a small javascript to the page that gets me indirectly the information.

[DaleHohm]

I only have one problem with this solution. I am actually changing than the HTML page where I preferabbly would not like to do this.

Is it possible without manipulating / adding script events to all elements to get the same information.

I'm not sure I understand you. The previous example does not rewrite HTML.

If you would write a spy like

if $WindowClass = "INTERNET_EXPLORER_SERVER" then

hookAllMouseOvers()

endIf

It will take a while before they are all hooked for all elements (could offcourse limit it to only INPUT and A tags or let user mark the tags he is interested in).

Not that I am aware of. If you know how to do this from another language, the technique may be something that can be adapted.

If on the opposite I could get to the event object with the properties its easier (and probably faster)

"The .event intrface is a problem due to the way that AutoIt handles COM events"

* Where can be read more on above statement and will it change in a future version of AutoIT?

* No other way to get to the event object? I am thinking about adding a small javascript to the page that gets me indirectly the information.

There is no formal documentation on this. AutoIt is essentially "notified" of events and processes them on its own schedule. The application that generates the event moves on after the event notification is queued. Therefore, anything that relies on everything being in the same state as when the event was fired will be a problem -- for example, returning False in an onclick or onsubmit event handler will not abort the event because the source has already moved on.

For more context, see this discussion with the AutoIt COM architect where I raised a bug about this issue http://www.autoitscript.com/forum/index.ph...92&hl=event The short of it is that it is an architectural limitation, difficult to solve with no committment to do so.

Dale

Edit: added link

Edited March 12, 2008 by DaleHohm

[junkew]

For delphi code I feel that this is doing what I want (although not yet tried)

http://delphi.about.com/od/vclusing/a/wbsinkevents.htm

"Rewrite HTML" was not a full correct phrase. What I actually ment was that there is a (small) risk that I will influence the page/application behavior when I do insert javascripts and alter the events in such a way.

Thread on COM was clear and gave a good background. Thx.

[DaleHohm]

The event object is tied to the Window, so there is only one. By the time your event processor wakes up, your event is already stale.

If this is the type of processing you want to do, you must either write Javascript code to the page or you must choose another language.

Dale