junkew Posted August 3, 2014 Posted August 3, 2014 (edited) Probably all this stuff can work natively from AutoIT this is the result I have so far in W7 with Java 7 installed (worked similar in Java 6) expandcollapse popup;~ By inspecting the WindowsAccessBridge-32.dll it reveals some information about the hidden dialogs ;~ So it seems the hidden dialog is shown after you call windows_run() no clue if interaction is needed ;~ ;~ Somehow it sends a message unclear if this is to the JVM to respond to ;~ push SSZ6E73E320_AccessBridge_FromJava_Hello ;~ push SSZ6E73E300_AccessBridge_FromWindows_Hello ;~ db 'AccessBridge-FromWindows-Hello',0 ;~ db 'AccessBridge-FromJava-Hello',0 ;~ Copied from this NVDA reference and translated to AutoIT ;~ http://www.webbie.org.uk/nvda/api/JABHandler-pysrc.html ;~ ;~ def initialize(): ;~ global isRunning ;~ if not bridgeDll: ;~ raise NotImplementedError("dll not available") ;~ bridgeDll.Windows_run() ;~ #Accept wm_copydata and any wm_user messages from other processes even if running with higher privilages ;~*** ChangeWindowMessageFilter=getattr(windll.user32,'ChangeWindowMessageFilter',None) ;~*** if ChangeWindowMessageFilter: ;~*** if not ChangeWindowMessageFilter(winUser.WM_COPYDATA,1): ;~*** raise WinError() ;~*** for msg in xrange(winUser.WM_USER+1,65535): ;~*** if not ChangeWindowMessageFilter(msg,1): ;~*** raise WinError() ;~ #Register java events ;~ bridgeDll.setFocusGainedFP(internal_event_focusGained) ;~ bridgeDll.setPropertyActiveDescendentChangeFP(internal_event_activeDescendantChange) ;~ bridgeDll.setPropertyNameChangeFP(event_nameChange) ;~ bridgeDll.setPropertyDescriptionChangeFP(event_descriptionChange) ;~ bridgeDll.setPropertyValueChangeFP(event_valueChange) ;~ bridgeDll.setPropertyStateChangeFP(internal_event_stateChange) ;~ bridgeDll.setPropertyCaretChangeFP(internal_event_caretChange) ;~ isRunning=True ;create the structs ;~ ;~ http://docs.oracle.com/javase/accessbridge/2.0.2/api.htm ;~ ;~ #define MAX_STRING_SIZE 1024 ;~ #define SHORT_STRING_SIZE 256 ;~ struct AccessBridgeVersionInfo { ;~ wchar_t VMversion[SHORT_STRING_SIZE]; // version of the Java VM ;~ wchar_t bridgeJavaClassVersion[SHORT_STRING_SIZE]; // version of the AccessBridge.class ;~ wchar_t bridgeJavaDLLVersion[SHORT_STRING_SIZE]; // version of JavaAccessBridge.dll ;~ wchar_t bridgeWinDLLVersion[SHORT_STRING_SIZE]; // version of WindowsAccessBridge.dll ;~ }; ;~ Local $AccessBridgeVersionInfo=DllStructCreate("WCHAR VMversion[256];WCHAR bridgeJavaClassVersion[256];WCHAR bridgeJavaDLLVersion[256];WCHAR bridgeWinDLLVersion[256]") Local $AccessBridgeVersionInfo=DllStructCreate("WCHAR VMversion[256];WCHAR bridgeJavaClassVersion[256];WCHAR bridgeJavaDLLVersion[256];WCHAR bridgeWinDLLVersion[256]") ;~ struct AccessibleContextInfo { ;~ wchar_ name[MAX_STRING_SIZE]; // the AccessibleName of the object ;~ wchar_ description[MAX_STRING_SIZE]; // the AccessibleDescription of the object ;~ wchar_ role[SHORT_STRING_SIZE]; // localized AccesibleRole string ;~ wchar_ states[SHORT_STRING_SIZE]; // localized AccesibleStateSet string ;~ // (comma separated) ;~ jint indexInParent // index of object in parent ;~ jint childrenCount // # of children, if any ;~ jint x; // screen x-axis co-ordinate in pixels ;~ jint y; // screen y-axis co-ordinate in pixels ;~ jint width; // pixel width of object ;~ jint height; // pixel height of object ;~ BOOL accessibleComponent; // flags for various additional ;~ BOOL accessibleAction; // Java Accessibility interfaces ;~ BOOL accessibleSelection; // FALSE if this object doesn't ;~ BOOL accessibleText; // implement the additional interface ;~ BOOL accessibleInterfaces; // new bitfield containing additional ;~ // interface flags ;~ }; ;~ Local $AccessibleContextInfo=DllStructCreate("WCHAR name[1024];WCHAR description[1024];WCHAR role[256];WCHAR states[256];int indexInParent;int childrenCount;int x;int y;int width;int height;BOOL accessibleComponent;BOOL accessibleAction;BOOL accessibleSelection;BOOL accessibleText;BOOL accessibleInterfaces") Local $AccessibleContextInfo=DllStructCreate("WCHAR name[1024];WCHAR description[1024];WCHAR role[256];WCHAR states[256];int64 indexInParent;int64 childrenCount;int64 x;int64 y;int64 width;int64 height;byte accessibleComponent;byte accessibleAction;byte accessibleSelection;byte accessibleText;byte accessibleinterfaces") ;~ #AutoIt3Wrapper_UseX64=Y #include <EditConstants.au3> #include <GUIConstantsEx.au3> #include <WindowsConstants.au3> #include <constants.au3> #include <WinAPI.au3> #include <debug.au3> local $hwnd local $i local $result local $vmID local $ac ;~ Make messages elevated _ChangeWindowMessageFilter($WM_COPYDATA,1) for $i=$WM_USER to $WM_USER+65536 _ChangeWindowMessageFilter($i, 1) Next Func _ChangeWindowMessageFilter($iMsg, $iAction) Local $aCall = DllCall("user32.dll", "bool", "ChangeWindowMessageFilter", "dword", $iMsg, "dword", $iAction) If @error Or Not $aCall[0] Then Return SetError(1, 0, 0) Return 1 EndFunc local $bridgeDLL=DLLOPEN("WindowsAccessBridge-32.dll") ;~ sleep(500) If $bridgeDLL=true Then consolewrite("Windows accessbridge 32 opened " & $bridgeDLL & @CRLF) Else consolewrite("DLL not found try to change to WindowsAccessBridge-64.dll if you use 64 bits autoit and windows") endIf ;~ TODO: Handle messages received from initialize $result =dllcall($bridgeDLL,"NONE", "Windows_run") consolewrite($result & " " & @error & " initializeAccessBridge is finished") sleep(250) consolewrite("Windows_run passed :" & $result & @CRLF) Local $var = WinList() consolewrite("Before loading all Windows:" & $var[0][0] & @CRLF) For $i = 1 To $var[0][0] ; Only display visble windows that have a title If IsVisible($var[$i][1]) Then local $handle=wingethandle($var[$i][0]) $result =dllcall($bridgeDLL,"BOOL:cdecl", "isJavaWindow", "hwnd", $handle) if @error=0 Then if $result[0]=1 Then consolewrite( $i & " Java Window Title=" & $var[$i][0] & " Handle=" & $var[$i][1] & @TAB & " res: " & $result[0] & @CRLF) local $ac=0 local $vmID=0 $result =dllcall($bridgeDLL,"BOOL:cdecl", "getAccessibleContextFromHWND", "hwnd", $handle, "long*", $vmID, "ptr*", $ac) if @error=0 Then consolewrite("We have a context " & @CRLF) $vmID=$result[2] $ac=$result[3] consolewrite("We have a VMid " & $vmID & " ac " & $ac & @CRLF) if @error > 0 then consolewrite("Struct error") ;~ $result =dllcall($bridgeDLL, "BOOL:cdecl", "getVersionInfo", "long", $vmId, "struct", $AccessBridgeVersionInfo) $result =dllcall($bridgeDLL, "BOOL:cdecl", "getVersionInfo", "long", $vmId, "ptr", DllStructGetPtr($AccessBridgeVersionInfo)) consolewrite( @error & " context found of " & $vmID & @CRLF) if @error=0 Then $s1=dllstructgetdata($AccessBridgeVersionInfo, "VMVersion") $s2=dllstructgetdata($AccessBridgeVersionInfo, "bridgeJavaClassVersion") $s3=dllstructgetdata($AccessBridgeVersionInfo, "bridgeJavaDLLVersion") $s4=dllstructgetdata($AccessBridgeVersionInfo, "bridgeWinDLLVersion") consolewrite("VMVersion: <" & $s1 & ">" & @CRLF) consolewrite("bridgeJavaClassVersion: <" & $s2 & ">" & @CRLF) consolewrite("bridgeJavaDLLVersion: <" & $s3 & ">" & @CRLF) consolewrite("bridgeWinDLLVersion: <" & $s4 & ">" & @CRLF) EndIf ;~ Retrieves an AccessibleContextInfo object of the AccessibleContext object ac. $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "ptr", $ac, "ptr", DllStructGetPtr($AccessibleContextInfo)) ;~ Local $AccessibleContextInfo=DllStructCreate("WCHAR name[1024];WCHAR description[1024];WCHAR role[256];WCHAR states[256];int indexInParent;int childrenCount;int x;int y;int width;int height;BOOL accessibleComponent;BOOL accessibleAction;BOOL accessibleSelection;BOOL accessibleText;BOOL accessibleInterfaces") consolewrite("Result is " & $result ) if @error<>0 Then consolewrite("We have an error " & @CRLF) EndIf $s1=dllstructgetdata($AccessibleContextInfo, "name") $s2=dllstructgetdata($AccessibleContextInfo, "description") consolewrite("name: <" & $s1 & ">" & @CRLF) consolewrite("description: <" & $s2 & ">" & @CRLF) ;~ AccessibleContext GetAccessibleChildFromContext(long vmID, AccessibleContext ac, jint index); ;~ Returns an AccessibleContext object that represents the nth child of the object ac, where n is specified by the value index. Else consolewrite( @error & " No context found" & @CRLF) endif EndIf Else EndIf endif Next ;~ http://www.autohotkey.com/board/topic/95343-how-to-send-to-unseen-controls-in-a-java-app/ local $result =dllcall($bridgeDLL,"BOOL", "shutdownAccessBridge") Func IsVisible($handle) If BitAND(WinGetState($handle), 2) Then Return 1 Else Return 0 EndIf EndFunc ;==>IsVisible 1. start a java application 2. run above script 3. problem is still I am not getting the java context which should work Reference code from NVDA python https://github.com/ckundo/nvda/blob/master/source/JABHandler.py problem is somewhere here ;~ Retrieves an AccessibleContextInfo object of the AccessibleContext object ac. ;~ $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "ptr", $ac, "ptr", DllStructGetPtr($AccessibleContextInfo)) consolewrite("BEFORE BREAKING") $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","INT64", $vmId, "ptr", $ac, "ptr", DllStructGetPtr($AccessibleContextInfo)) consolewrite("AFTER BREAKING") Edited August 3, 2014 by junkew FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets
junkew Posted January 3, 2015 Posted January 3, 2015 still not fixed but with JDK 1.8 I seem to be having more information of the definitions  When you install JDK 1.8  definitions can be found there C:Program Files (x86)Javajdk1.8.0_25includewin32bridge ;~ ******* ;~ Every AccessibleContext IntPtr must be replaced by long, including but not limited to getAccessibleContextFromHWND, getAccessibleParentFromContext, getAccessibleChildFromContext, getAccessibleTextInf ;~ JOBJECT64 is defined as jlong on 64-bit systems and jobject on legacy versions of Java Access Bridge. For definitions, see the section ACCESSBRIDGE_ARCH_LEGACY in the AccessBridgePackages.h header file. ;~ #ifdef ACCESSBRIDGE_ARCH_LEGACY ;~ typedef jobject JOBJECT64; ;~ typedef HWND ABHWND64; ;~ #define ABHandleToLong ;~ #define ABLongToHandle ;~ #else ;~ typedef jlong JOBJECT64; ;~ typedef long ABHWND64; ;~ #define ABHandleToLong HandleToLong ;~ #define ABLongToHandle LongToHandle ;~ #endif ;~ a jobject is a 32 bit pointer for 32 bit builds ;~ ******* steps to reproduce 0. install jdk 1.8 1. run jdk-8u25-windows-i586-demosjdk1.8.0_25demojfcSwingSet2 2. its a little unclear if jabswitch /enable has to be called but to be sure its on it should be started from commandline 3. run script as posted 4. Validate with javamonkey and javaferret what you get (script should return the same result)  seems just a stupid incorrect dll call due to incorrect structure definition Any help would be appreciated  latest source trial and error expandcollapse popup#AutoIt3Wrapper_UseX64=Y consolewrite("We are using " & @OSArch & " at cpu " & @CPUArch & " Autoit 64 bit version "& @AutoItX64 & @CRLF) if @AutoItX64 = 1 then local $accessBridgeDLL="WindowsAccessBridge-64.dll" Else local $accessBridgeDLL="WindowsAccessBridge-32.dll" EndIf ;~ By inspecting the WindowsAccessBridge-32.dll it reveals some information about the hidden dialogs ;~ So it seems the hidden dialog is shown after you call windows_run() no clue if interaction is needed ;~ ;~ Somehow it sends a message unclear if this is to the JVM to respond to ;~ push SSZ6E73E320_AccessBridge_FromJava_Hello ;~ push SSZ6E73E300_AccessBridge_FromWindows_Hello ;~ db 'AccessBridge-FromWindows-Hello',0 ;~ db 'AccessBridge-FromJava-Hello',0 ;~ JABHANDLER.PY is a usefull source to study on internet ;~ ******* ;~ Every AccessibleContext IntPtr must be replaced by long, including but not limited to getAccessibleContextFromHWND, getAccessibleParentFromContext, getAccessibleChildFromContext, getAccessibleTextInf ;~ JOBJECT64 is defined as jlong on 64-bit systems and jobject on legacy versions of Java Access Bridge. For definitions, see the section ACCESSBRIDGE_ARCH_LEGACY in the AccessBridgePackages.h header file. ;~ #ifdef ACCESSBRIDGE_ARCH_LEGACY ;~ typedef jobject JOBJECT64; ;~ typedef HWND ABHWND64; ;~ #define ABHandleToLong ;~ #define ABLongToHandle ;~ #else ;~ typedef jlong JOBJECT64; ;~ typedef long ABHWND64; ;~ #define ABHandleToLong HandleToLong ;~ #define ABLongToHandle LongToHandle ;~ #endif ;~ a jobject is a 32 bit pointer for 32 bit builds ;~ 'bool' is a built-in C++ type while 'BOOL' is a Microsoft specific type that is defined as an 'int'. You can find it in 'windef.h' ;~ ******* ;~ Primitive Types and Native Equivalents ;~ Java Type Native Type Description ;~ boolean jboolean unsigned 8 bits ;~ byte jbyte signed 8 bits ;~ char jchar unsigned 16 bits ;~ short jshort signed 16 bits ;~ int jint signed 32 bits ;~ long jlong signed 64 bits ;~ float jfloat 32 bits ;~ double jdouble 64 bits ;~ void void not applicable ;~ Copied from this NVDA reference and translated to AutoIT ;~ http://www.webbie.org.uk/nvda/api/JABHandler-pysrc.html ;~ ;~ def initialize(): ;~ global isRunning ;~ if not bridgeDll: ;~ raise NotImplementedError("dll not available") ;~ bridgeDll.Windows_run() ;~ #Accept wm_copydata and any wm_user messages from other processes even if running with higher privilages ;~*** ChangeWindowMessageFilter=getattr(windll.user32,'ChangeWindowMessageFilter',None) ;~*** if ChangeWindowMessageFilter: ;~*** if not ChangeWindowMessageFilter(winUser.WM_COPYDATA,1): ;~*** raise WinError() ;~*** for msg in xrange(winUser.WM_USER+1,65535): ;~*** if not ChangeWindowMessageFilter(msg,1): ;~*** raise WinError() ;~ #Register java events ;~ bridgeDll.setFocusGainedFP(internal_event_focusGained) ;~ bridgeDll.setPropertyActiveDescendentChangeFP(internal_event_activeDescendantChange) ;~ bridgeDll.setPropertyNameChangeFP(event_nameChange) ;~ bridgeDll.setPropertyDescriptionChangeFP(event_descriptionChange) ;~ bridgeDll.setPropertyValueChangeFP(event_valueChange) ;~ bridgeDll.setPropertyStateChangeFP(internal_event_stateChange) ;~ bridgeDll.setPropertyCaretChangeFP(internal_event_caretChange) ;~ isRunning=True ;create the structs ;~ ;~ http://docs.oracle.com/javase/accessbridge/2.0.2/api.htm ;~ ;~ #define MAX_STRING_SIZE 1024 ;~ #define SHORT_STRING_SIZE 256 ;~ struct AccessBridgeVersionInfo { ;~ wchar_t VMversion[SHORT_STRING_SIZE]; // version of the Java VM ;~ wchar_t bridgeJavaClassVersion[SHORT_STRING_SIZE]; // version of the AccessBridge.class ;~ wchar_t bridgeJavaDLLVersion[SHORT_STRING_SIZE]; // version of JavaAccessBridge.dll ;~ wchar_t bridgeWinDLLVersion[SHORT_STRING_SIZE]; // version of WindowsAccessBridge.dll ;~ }; ;~ Local $AccessBridgeVersionInfo=DllStructCreate("WCHAR VMversion[256];WCHAR bridgeJavaClassVersion[256];WCHAR bridgeJavaDLLVersion[256];WCHAR bridgeWinDLLVersion[256]") Local $AccessBridgeVersionInfo=DllStructCreate("WCHAR VMversion[256];WCHAR bridgeJavaClassVersion[256];WCHAR bridgeJavaDLLVersion[256];WCHAR bridgeWinDLLVersion[256]") ;~ struct AccessibleContextInfo { ;~ wchar_ name[MAX_STRING_SIZE]; // the AccessibleName of the object ;~ wchar_ description[MAX_STRING_SIZE]; // the AccessibleDescription of the object ;~ wchar_ role[SHORT_STRING_SIZE]; // localized AccesibleRole string ;~ wchar_ states[SHORT_STRING_SIZE]; // localized AccesibleStateSet string ;~ // (comma separated) ;~ jint indexInParent // index of object in parent ;~ jint childrenCount // # of children, if any ;~ jint x; // screen x-axis co-ordinate in pixels ;~ jint y; // screen y-axis co-ordinate in pixels ;~ jint width; // pixel width of object ;~ jint height; // pixel height of object ;~ BOOL accessibleComponent; // flags for various additional ;~ BOOL accessibleAction; // Java Accessibility interfaces ;~ BOOL accessibleSelection; // FALSE if this object doesn't ;~ BOOL accessibleText; // implement the additional interface ;~ BOOL accessibleInterfaces; // new bitfield containing additional ;~ // interface flags ;~ }; ;~ typedef struct AccessibleContextInfoTag { ;~ wchar_t name[MAX_STRING_SIZE]; // the AccessibleName of the object ;~ wchar_t description[MAX_STRING_SIZE]; // the AccessibleDescription of the object ;~ wchar_t role[SHORT_STRING_SIZE]; // localized AccesibleRole string ;~ wchar_t role_en_US[SHORT_STRING_SIZE]; // AccesibleRole string in the en_US locale ;~ wchar_t states[SHORT_STRING_SIZE]; // localized AccesibleStateSet string (comma separated) ;~ wchar_t states_en_US[SHORT_STRING_SIZE]; // AccesibleStateSet string in the en_US locale (comma separated) ;~ jint indexInParent; // index of object in parent ;~ jint childrenCount; // # of children, if any ;~ jint x; // screen coords in pixels ;~ jint y; // " ;~ jint width; // pixel width of object ;~ jint height; // pixel height of object ;~ BOOL accessibleComponent; // flags for various additional ;~ BOOL accessibleAction; // Java Accessibility interfaces ;~ BOOL accessibleSelection; // FALSE if this object doesn't ;~ BOOL accessibleText; // implement the additional interface ;~ // in question ;~ // BOOL accessibleValue; // old BOOL indicating whether AccessibleValue is supported ;~ BOOL accessibleInterfaces; // new bitfield containing additional interface flags ;~ } AccessibleContextInfo; ;~ Local $AccessibleContextInfo=DllStructCreate("WCHAR name[1024];WCHAR description[1024];WCHAR role[256];WCHAR role_en_US[256];WCHAR states[256];WCHAR states_en_US[256];int indexInParent;int childrenCount;int x;int y;int width;int height;BOOL accessibleComponent;BOOL accessibleAction;BOOL accessibleSelection;BOOL accessibleText;BOOL accessibleInterfaces") const $cTypeStructAccessibleContextInfo="WCHAR name[1024];WCHAR description[1024];WCHAR role[256];WCHAR role_en_US[256];WCHAR states[256];WCHAR states_en_US[256];INT indexInParent;INT childrenCount;INT x;INT y;INT width;INT height;BOOL accessibleComponent;BOOL accessibleAction;BOOL accessibleSelection;BOOL accessibleText;BOOL accessibleInterfaces" ;~ const $cTypeStructAccessibleContextInfo="WCHAR data[8192]" Local $AccessibleContextInfo=DllStructCreate($cTypeStructAccessibleContextInfo) ; If an error occurred display the error code and return False. If @error Then consolewrite("Error in DllStructCreate $AccessibleContextInfo, Code: " & @error) exit EndIf consolewrite("We will call contextinfo with Struct Size: " & DllStructGetSize($AccessibleContextInfo) & @CRLF) ;~ Local $AccessibleContextInfo=DllStructCreate("WCHAR name[1024];WCHAR description[1024];WCHAR role[256];WCHAR states[256];int64 indexInParent;int64 childrenCount;int64 x;int64 y;int64 width;int64 height;byte accessibleComponent;byte accessibleAction;byte accessibleSelection;byte accessibleText;byte accessibleinterfaces") ;~ #AutoIt3Wrapper_UseX64=Y #include <EditConstants.au3> #include <GUIConstantsEx.au3> #include <WindowsConstants.au3> #include <constants.au3> #include <WinAPI.au3> #include <debug.au3> local $hwnd local $i local $result local $vmID local $ac ;~ Make messages elevated _ChangeWindowMessageFilter($WM_COPYDATA,1) for $i=$WM_USER to $WM_USER+65536 _ChangeWindowMessageFilter($i, 1) Next Func _ChangeWindowMessageFilter($iMsg, $iAction) Local $aCall = DllCall("user32.dll", "bool", "ChangeWindowMessageFilter", "dword", $iMsg, "dword", $iAction) If @error Or Not $aCall[0] Then Return SetError(1, 0, 0) Return 1 EndFunc local $bridgeDLL=DLLOPEN($AccessBridgeDLL) ;~ sleep(500) If $bridgeDLL=true Then consolewrite("Windows accessbridge 32 opened " & $bridgeDLL & @CRLF) Else consolewrite("DLL not found try to change to WindowsAccessBridge-64.dll if you use 64 bits autoit and windows") endIf ;~ TODO: Handle messages received from initialize $result =dllcall($bridgeDLL,"NONE", "Windows_run") consolewrite($result & " " & @error & " initializeAccessBridge is finished") sleep(250) consolewrite("Windows_run passed :" & $result & @CRLF) Local $var = WinList() consolewrite("Before loading all Windows:" & $var[0][0] & @CRLF) For $i = 1 To $var[0][0] ; Only display visble windows that have a title If IsVisible($var[$i][1]) Then local $handle=wingethandle($var[$i][0]) $result =dllcall($bridgeDLL,"BOOL:cdecl", "isJavaWindow", "hwnd", $handle) if @error=0 Then if $result[0]=1 Then consolewrite( "JAVA window found <" & $i & "> Java Window Title=" & $var[$i][0] & " Handle=" & $var[$i][1] & @TAB & " res: " & $result[0] & @CRLF) local $acParent=0 local $vmID=0 ;~AccessibilityContext (acParent above), which I had incorrectly mapped as an IntPtr, is actually an Int32 when using the "legacy" WindowsAccessBridge.dll library (used under x86), ;~and an Int64 when using the WOW64 WindowsAccessBridge-32.dll library. ;~WindowsAccessBridge-32.dll bool getAccessibleContextFromHWND(IntPtr hwnd, out Int32 vmID, out Int64 acParent); ;~WindowsAccessBridge.dll bool _getAccessibleContextFromHWND(IntPtr hwnd, out Int32 vmID, out Int32 acParent); $result =dllcall($bridgeDLL,"BOOL:cdecl", "getAccessibleContextFromHWND", "hwnd", $handle, "long*", $vmID, "INT64*", $acParent) ;~ $result =dllcall($bridgeDLL,"BOOL:cdecl", "getAccessibleContextFromHWND", "hwnd", $handle, "long*", $vmID, "ptr*", $acParent) ;~ $result =dllcall($bridgeDLL,"BOOL:cdecl", "getAccessibleContextFromHWND", "hwnd", $handle, "long*", $vmID, "INT64*", $acParent) if @error=0 Then consolewrite("Result getAccessibleContextFromHWND is <" & $result & "> ubound: " & ubound($result) & @crlf) $vmID=$result[2] $acParent=$result[3] consolewrite(" We have a VMid " & $vmID & " ac " & $acParent & @CRLF) if @error > 0 then consolewrite("Struct error") ;~ $result =dllcall($bridgeDLL, "BOOL:cdecl", "getVersionInfo", "long", $vmId, "struct", $AccessBridgeVersionInfo) ;~ typedef BOOL (*GetVersionInfoFP) (long vmID, AccessBridgeVersionInfo *info); $result =dllcall($bridgeDLL, "BOOL:cdecl", "getVersionInfo", "long", $vmId, "struct*", DllStructGetPtr($AccessBridgeVersionInfo)) if @error=0 Then $s1=dllstructgetdata($AccessBridgeVersionInfo, "VMVersion") $s2=dllstructgetdata($AccessBridgeVersionInfo, "bridgeJavaClassVersion") $s3=dllstructgetdata($AccessBridgeVersionInfo, "bridgeJavaDLLVersion") $s4=dllstructgetdata($AccessBridgeVersionInfo, "bridgeWinDLLVersion") consolewrite("Call version info: PASSED VMID found : " & $vmID & @CRLF) consolewrite(" VMVersion: <" & $s1 & ">" & @CRLF) consolewrite(" bridgeJavaClassVersion: <" & $s2 & ">" & @CRLF) consolewrite(" bridgeJavaDLLVersion: <" & $s3 & ">" & @CRLF) consolewrite(" bridgeWinDLLVersion: <" & $s4 & ">" & @CRLF) Else consolewrite("getVersionInfo error:" & @error & @CRLF) EndIf ;~ /** ;~ * Returns the Accessible Context for the top level object in ;~ * a Java Window. This is same Accessible Context that is obtained ;~ * from GetAccessibleContextFromHWND for that window. Returns ;~ * (AccessibleContext)0 on error. ;~ */ ;~ AccessibleContext getTopLevelObject (const long vmID, const AccessibleContext accessibleContext); ;~ typedef AccessibleContext (*getTopLevelObjectFP) (const long vmID, const AccessibleContext ac); local $topAC $result=dllcall($bridgeDLL, "BOOL:cdecl", "getTopLevelObject", "long", $vmId, "INT64", $topAC) if @error=0 Then consolewrite("getTopLevelObject Result is <" & $result & "> ubound: " & ubound($result) & @crlf) $vmID=$result[1] $topAC=$result[2] consolewrite("We have a VMid " & $vmID & " toplevel object ac " & $topAC & @CRLF) Else consolewrite("getTopLevelObjecterror:" & @error & @CRLF) endif consolewrite("BEFORE BREAKING" & @CRLF) consolewrite(" We still have a VMid " & $vmID & " acParent " & $acParent & @CRLF) ;~ Retrieves an AccessibleContextInfo object of the AccessibleContext object ac. ;~ $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "ptr", $ac, "ptr", DllStructGetPtr($AccessibleContextInfo)) ;~ typedef BOOL (*GetAccessibleContextInfoFP) (long vmID, AccessibleContext ac, AccessibleContextInfo *info); ;~ $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","INT64", $vmId, "int", $ac, "ptr*", DllStructGetPtr($AccessibleContextInfo)) ;~ $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "long", $ac, "ptr", DllStructGetPtr($AccessibleContextInfo),DllStructGetSize($AccessibleContextInfo)) $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "INT64", $acParent, "struct*", DllStructGetPtr($AccessibleContextInfo)) if @error<>0 Then consolewrite("We have an error " & @error & @CRLF) EndIf ;~ We should pass it by reference ;~ Exception code c0000005 is an access violation, also known as general protection fault. The program is reading from, or writing to, an address which is not part of the virtual address space. ;~ A very common cause is that you are de-referencing a stale pointer. In other words, the pointer was once valid, but you have subsequently freed it. Then later when you attempt to access it, ;~ an exception is raised. ;~ The problem is don't use out parameter. Use the IntPtr. ;~ So it should be getAccessibleContextInfo(Int32 vmID, IntPtr accessibleContext, IntPtr acInfo) consolewrite("AFTER BREAKING" & @CRLF) consolewrite("Result is <" & $result & "> ubound: " & ubound($result) & @crlf) $AccessibleContextInfo=$result[3] $s1=dllstructgetdata($AccessibleContextInfo, "name") $s2=dllstructgetdata($AccessibleContextInfo, "description") $s3=dllstructgetdata($AccessibleContextInfo, "role") $s4=dllstructgetdata($AccessibleContextInfo, "role_en_US") $s5=dllstructgetdata($AccessibleContextInfo, "states") $s6=dllstructgetdata($AccessibleContextInfo, "states_en_US") $s7=dllstructgetdata($AccessibleContextInfo, "indexInParent") $s8=dllstructgetdata($AccessibleContextInfo, "childrenCount") $s9=dllstructgetdata($AccessibleContextInfo, "x") $s10=dllstructgetdata($AccessibleContextInfo, "y") $s11=dllstructgetdata($AccessibleContextInfo, "width") $s12=dllstructgetdata($AccessibleContextInfo, "heigth") consolewrite("name: <" & $s1 & ">" & @CRLF) consolewrite("description: <" & $s2 & ">" & @CRLF) consolewrite("role: <" & $s3 & ">" & @CRLF) consolewrite("role_en_US: <" & $s4 & ">" & @CRLF) consolewrite("states: <" & $s5 & ">" & @CRLF) consolewrite("states_en_US: <" & $s6 & ">" & @CRLF) consolewrite("indexInParent: <" & $s7 & ">" & @CRLF) consolewrite("childrenCount: <" & $s8 & ">" & @CRLF) consolewrite("x: <" & $s9 & ">" & @CRLF) consolewrite("y: <" & $s10 & ">" & @CRLF) consolewrite("width: <" & $s11 & ">" & @CRLF) consolewrite("height: <" & $s12 & ">" & @CRLF) ;~ AccessibleContext GetAccessibleChildFromContext(long vmID, AccessibleContext ac, jint index); ;~ Returns an AccessibleContext object that represents the nth child of the object ac, where n is specified by the value index. Else consolewrite( @error & " No context found" & @CRLF) endif EndIf Else EndIf endif Next ;~ http://www.autohotkey.com/board/topic/95343-how-to-send-to-unseen-controls-in-a-java-app/ local $result =dllcall($bridgeDLL,"BOOL", "shutdownAccessBridge") Func IsVisible($handle) If BitAND(WinGetState($handle), 2) Then Return 1 Else Return 0 EndIf EndFunc ;==>IsVisible FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets
junkew Posted January 3, 2015 Posted January 3, 2015 (edited) yes, now it works (I get x and y and name information from a java window)and lots of stuff can be rewrittenCan someone explain me the difference between these 2 lines (DllStructGetPtr seems not needed or when used gives me not the expected result);~ $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "INT64", $acParent, "struct*", DllStructGetPtr($AccessibleContextInfo)) $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "INT64", $acParent, "struct*", $AccessibleContextInfo)output of below scriptexpandcollapse popupWe are using X64 at cpu X64 Autoit 64 bit version 1 We will call contextinfo with Struct Size: 6188 Windows accessbridge 32 opened 1 Windows_run passed : Before loading all Windows:569 JAVA window found <62> Java Window Title=SwingSet2 Handle=0x000000000017068A res: 1 Result getAccessibleContextFromHWND is <> ubound: 4 We have a VMid 1969346 ac 1539741304 Call version info: PASSED VMID found : 1969346 VMVersion: <1.8.0_25> bridgeJavaClassVersion: <1.8.0_25> bridgeJavaDLLVersion: <1.8.0_25> bridgeWinDLLVersion: <1.8.0_25> *** getTopLevelObject Result is <0> ubound: 3 We have a VMid 1969346 toplevel object ac 0 BEFORE BREAKING We still have a VMid 1969346 acParent 1539741304 AFTER BREAKING Result is <> ubound: 4 name: <SwingSet2> description: <> role: <frame> role_en_US: <frame> states: <enabled,focusable,visible,showing,resizable> states_en_US: <enabled,focusable,visible,showing,resizable> indexInParent: <-1> childrenCount: <1> x: <1152> y: <195> width: <736> height: <0> JAVA window found <68> Java Window Title=SwingSet2 Handle=0x0000000000080C7C res: 1 Result getAccessibleContextFromHWND is <> ubound: 4 We have a VMid 1969346 ac 1539741232 Call version info: PASSED VMID found : 1969346 VMVersion: <1.8.0_25> bridgeJavaClassVersion: <1.8.0_25> bridgeJavaDLLVersion: <1.8.0_25> bridgeWinDLLVersion: <1.8.0_25> *** getTopLevelObject Result is <0> ubound: 3 We have a VMid 1969346 toplevel object ac 0 BEFORE BREAKING We still have a VMid 1969346 acParent 1539741232 AFTER BREAKING Result is <> ubound: 4 name: <SwingSet2> description: <> role: <frame> role_en_US: <frame> states: <enabled,focusable,visible,showing,resizable> states_en_US: <enabled,focusable,visible,showing,resizable> indexInParent: <-1> childrenCount: <1> x: <1152> y: <195> width: <736> height: <0> and for completeness the whole testing sourceexpandcollapse popup#AutoIt3Wrapper_UseX64=Y consolewrite("We are using " & @OSArch & " at cpu " & @CPUArch & " Autoit 64 bit version "& @AutoItX64 & @CRLF) if @AutoItX64 = 1 then local $accessBridgeDLL="WindowsAccessBridge-64.dll" Else local $accessBridgeDLL="WindowsAccessBridge-32.dll" EndIf ;~ By inspecting the WindowsAccessBridge-32.dll it reveals some information about the hidden dialogs ;~ So it seems the hidden dialog is shown after you call windows_run() no clue if interaction is needed ;~ ;~ Somehow it sends a message unclear if this is to the JVM to respond to ;~ push SSZ6E73E320_AccessBridge_FromJava_Hello ;~ push SSZ6E73E300_AccessBridge_FromWindows_Hello ;~ db 'AccessBridge-FromWindows-Hello',0 ;~ db 'AccessBridge-FromJava-Hello',0 ;~ JABHANDLER.PY is a usefull source to study on internet ;~ ******* ;~ Every AccessibleContext IntPtr must be replaced by long, including but not limited to getAccessibleContextFromHWND, getAccessibleParentFromContext, getAccessibleChildFromContext, getAccessibleTextInf ;~ JOBJECT64 is defined as jlong on 64-bit systems and jobject on legacy versions of Java Access Bridge. For definitions, see the section ACCESSBRIDGE_ARCH_LEGACY in the AccessBridgePackages.h header file. ;~ #ifdef ACCESSBRIDGE_ARCH_LEGACY ;~ typedef jobject JOBJECT64; ;~ typedef HWND ABHWND64; ;~ #define ABHandleToLong ;~ #define ABLongToHandle ;~ #else ;~ typedef jlong JOBJECT64; ;~ typedef long ABHWND64; ;~ #define ABHandleToLong HandleToLong ;~ #define ABLongToHandle LongToHandle ;~ #endif ;~ a jobject is a 32 bit pointer for 32 bit builds ;~ 'bool' is a built-in C++ type while 'BOOL' is a Microsoft specific type that is defined as an 'int'. You can find it in 'windef.h' ;~ ******* ;~ #Appropriately set the return and argument types of all the access bridge dll functions ;~ if bridgeDll: ;~ _fixBridgeFunc(None,'Windows_run') ;~ _fixBridgeFunc(None,'setFocusGainedFP',c_void_p) ;~ _fixBridgeFunc(None,'setPropertyStateChangeFP',c_void_p) ;~ _fixBridgeFunc(None,'setPropertyCaretChangeFP',c_void_p) ;~ _fixBridgeFunc(None,'setPropertyActiveDescendentChangeFP',c_void_p) ;~ _fixBridgeFunc(None,'releaseJavaObject',c_long,JOBJECT64) ;~ _fixBridgeFunc(BOOL,'getVersionInfo',POINTER(AccessBridgeVersionInfo),errcheck=True) ;~ _fixBridgeFunc(BOOL,'isJavaWindow',HWND) ;~ _fixBridgeFunc(BOOL,'isSameObject',c_long,JOBJECT64,JOBJECT64) ;~ _fixBridgeFunc(BOOL,'getAccessibleContextFromHWND',HWND,POINTER(c_long),POINTER(JOBJECT64),errcheck=True) ;~ _fixBridgeFunc(HWND,'getHWNDFromAccessibleContext',c_long,JOBJECT64,errcheck=True) ;~ _fixBridgeFunc(BOOL,'getAccessibleContextAt',c_long,JOBJECT64,jint,jint,POINTER(JOBJECT64),errcheck=True) ;~ _fixBridgeFunc(BOOL,'getAccessibleContextWithFocus',HWND,POINTER(c_long),POINTER(JOBJECT64),errcheck=True) ;~ _fixBridgeFunc(BOOL,'getAccessibleContextInfo',c_long,JOBJECT64,POINTER(AccessibleContextInfo),errcheck=True) ;~ _fixBridgeFunc(JOBJECT64,'getAccessibleChildFromContext',c_long,JOBJECT64,jint,errcheck=True) ;~ _fixBridgeFunc(JOBJECT64,'getAccessibleParentFromContext',c_long,JOBJECT64) ;~ _fixBridgeFunc(BOOL,'getAccessibleRelationSet',c_long,JOBJECT64,POINTER(AccessibleRelationSetInfo),errcheck=True) ;~ _fixBridgeFunc(BOOL,'getAccessibleTextInfo',c_long,JOBJECT64,POINTER(AccessibleTextInfo),jint,jint,errcheck=True) ;~ _fixBridgeFunc(BOOL,'getAccessibleTextItems',c_long,JOBJECT64,POINTER(AccessibleTextItemsInfo),jint,errcheck=True) ;~ _fixBridgeFunc(BOOL,'getAccessibleTextSelectionInfo',c_long,JOBJECT64,POINTER(AccessibleTextSelectionInfo),errcheck=True) ;~ _fixBridgeFunc(BOOL,'getAccessibleTextAttributes',c_long,JOBJECT64,jint,POINTER(AccessibleTextAttributesInfo),errcheck=True) ;~ _fixBridgeFunc(BOOL,'getAccessibleTextLineBounds',c_long,JOBJECT64,jint,POINTER(jint),POINTER(jint),errcheck=True) ;~ _fixBridgeFunc(BOOL,'getAccessibleTextRange',c_long,JOBJECT64,jint,jint,POINTER(c_wchar),c_short,errcheck=True) ;~ _fixBridgeFunc(BOOL,'getCurrentAccessibleValueFromContext',c_long,JOBJECT64,POINTER(c_wchar),c_short,errcheck=True) ;~ _fixBridgeFunc(BOOL,'selectTextRange',c_long,JOBJECT64,c_int,c_int,errcheck=True) ;~ _fixBridgeFunc(BOOL,'getTextAttributesInRange',c_long,JOBJECT64,c_int,c_int,POINTER(AccessibleTextAttributesInfo),POINTER(c_short),errcheck=True) ;~ _fixBridgeFunc(JOBJECT64,'getTopLevelObject',c_long,JOBJECT64,errcheck=True) ;~ _fixBridgeFunc(c_int,'getObjectDepth',c_long,JOBJECT64) ;~ _fixBridgeFunc(JOBJECT64,'getActiveDescendent',c_long,JOBJECT64) ;~ _fixBridgeFunc(BOOL,'requestFocus',c_long,JOBJECT64,errcheck=True) ;~ _fixBridgeFunc(BOOL,'setCaretPosition',c_long,JOBJECT64,c_int,errcheck=True) ;~ _fixBridgeFunc(BOOL,'getCaretLocation',c_long,JOBJECT64,POINTER(AccessibleTextRectInfo),jint,errcheck=True) ;~ _fixBridgeFunc(BOOL,'getAccessibleActions',c_long,JOBJECT64,POINTER(AccessibleActions),errcheck=True) ;~ _fixBridgeFunc(BOOL,'doAccessibleActions',c_long,JOBJECT64,POINTER(AccessibleActionsToDo),POINTER(jint),errcheck=True) ;~ Primitive Types and Native Equivalents ;~ Java Type Native Type Description ;~ boolean jboolean unsigned 8 bits ;~ byte jbyte signed 8 bits ;~ char jchar unsigned 16 bits ;~ short jshort signed 16 bits ;~ int jint signed 32 bits ;~ long jlong signed 64 bits ;~ float jfloat 32 bits ;~ double jdouble 64 bits ;~ void void not applicable ;~ Copied from this NVDA reference and translated to AutoIT ;~ http://www.webbie.org.uk/nvda/api/JABHandler-pysrc.html ;~ ;~ def initialize(): ;~ global isRunning ;~ if not bridgeDll: ;~ raise NotImplementedError("dll not available") ;~ bridgeDll.Windows_run() ;~ #Accept wm_copydata and any wm_user messages from other processes even if running with higher privilages ;~*** ChangeWindowMessageFilter=getattr(windll.user32,'ChangeWindowMessageFilter',None) ;~*** if ChangeWindowMessageFilter: ;~*** if not ChangeWindowMessageFilter(winUser.WM_COPYDATA,1): ;~*** raise WinError() ;~*** for msg in xrange(winUser.WM_USER+1,65535): ;~*** if not ChangeWindowMessageFilter(msg,1): ;~*** raise WinError() ;~ #Register java events ;~ bridgeDll.setFocusGainedFP(internal_event_focusGained) ;~ bridgeDll.setPropertyActiveDescendentChangeFP(internal_event_activeDescendantChange) ;~ bridgeDll.setPropertyNameChangeFP(event_nameChange) ;~ bridgeDll.setPropertyDescriptionChangeFP(event_descriptionChange) ;~ bridgeDll.setPropertyValueChangeFP(event_valueChange) ;~ bridgeDll.setPropertyStateChangeFP(internal_event_stateChange) ;~ bridgeDll.setPropertyCaretChangeFP(internal_event_caretChange) ;~ isRunning=True ;create the structs ;~ ;~ http://docs.oracle.com/javase/accessbridge/2.0.2/api.htm ;~ ;~ #define MAX_STRING_SIZE 1024 ;~ #define SHORT_STRING_SIZE 256 ;~ struct AccessBridgeVersionInfo { ;~ wchar_t VMversion[SHORT_STRING_SIZE]; // version of the Java VM ;~ wchar_t bridgeJavaClassVersion[SHORT_STRING_SIZE]; // version of the AccessBridge.class ;~ wchar_t bridgeJavaDLLVersion[SHORT_STRING_SIZE]; // version of JavaAccessBridge.dll ;~ wchar_t bridgeWinDLLVersion[SHORT_STRING_SIZE]; // version of WindowsAccessBridge.dll ;~ }; ;~ Local $AccessBridgeVersionInfo=DllStructCreate("WCHAR VMversion[256];WCHAR bridgeJavaClassVersion[256];WCHAR bridgeJavaDLLVersion[256];WCHAR bridgeWinDLLVersion[256]") Local $AccessBridgeVersionInfo=DllStructCreate("WCHAR VMversion[256];WCHAR bridgeJavaClassVersion[256];WCHAR bridgeJavaDLLVersion[256];WCHAR bridgeWinDLLVersion[256]") ;~ struct AccessibleContextInfo { ;~ wchar_ name[MAX_STRING_SIZE]; // the AccessibleName of the object ;~ wchar_ description[MAX_STRING_SIZE]; // the AccessibleDescription of the object ;~ wchar_ role[SHORT_STRING_SIZE]; // localized AccesibleRole string ;~ wchar_ states[SHORT_STRING_SIZE]; // localized AccesibleStateSet string ;~ // (comma separated) ;~ jint indexInParent // index of object in parent ;~ jint childrenCount // # of children, if any ;~ jint x; // screen x-axis co-ordinate in pixels ;~ jint y; // screen y-axis co-ordinate in pixels ;~ jint width; // pixel width of object ;~ jint height; // pixel height of object ;~ BOOL accessibleComponent; // flags for various additional ;~ BOOL accessibleAction; // Java Accessibility interfaces ;~ BOOL accessibleSelection; // FALSE if this object doesn't ;~ BOOL accessibleText; // implement the additional interface ;~ BOOL accessibleInterfaces; // new bitfield containing additional ;~ // interface flags ;~ }; ;~ typedef struct AccessibleContextInfoTag { ;~ wchar_t name[MAX_STRING_SIZE]; // the AccessibleName of the object ;~ wchar_t description[MAX_STRING_SIZE]; // the AccessibleDescription of the object ;~ wchar_t role[SHORT_STRING_SIZE]; // localized AccesibleRole string ;~ wchar_t role_en_US[SHORT_STRING_SIZE]; // AccesibleRole string in the en_US locale ;~ wchar_t states[SHORT_STRING_SIZE]; // localized AccesibleStateSet string (comma separated) ;~ wchar_t states_en_US[SHORT_STRING_SIZE]; // AccesibleStateSet string in the en_US locale (comma separated) ;~ jint indexInParent; // index of object in parent ;~ jint childrenCount; // # of children, if any ;~ jint x; // screen coords in pixels ;~ jint y; // " ;~ jint width; // pixel width of object ;~ jint height; // pixel height of object ;~ BOOL accessibleComponent; // flags for various additional ;~ BOOL accessibleAction; // Java Accessibility interfaces ;~ BOOL accessibleSelection; // FALSE if this object doesn't ;~ BOOL accessibleText; // implement the additional interface ;~ // in question ;~ // BOOL accessibleValue; // old BOOL indicating whether AccessibleValue is supported ;~ BOOL accessibleInterfaces; // new bitfield containing additional interface flags ;~ } AccessibleContextInfo; ;~ Local $AccessibleContextInfo=DllStructCreate("WCHAR name[1024];WCHAR description[1024];WCHAR role[256];WCHAR role_en_US[256];WCHAR states[256];WCHAR states_en_US[256];int indexInParent;int childrenCount;int x;int y;int width;int height;BOOL accessibleComponent;BOOL accessibleAction;BOOL accessibleSelection;BOOL accessibleText;BOOL accessibleInterfaces") const $cTypeStructAccessibleContextInfo="WCHAR name[1024];WCHAR description[1024];WCHAR role[256];WCHAR role_en_US[256];WCHAR states[256];WCHAR states_en_US[256];INT indexInParent;INT childrenCount;INT x;INT y;INT width;INT height;BOOL accessibleComponent;BOOL accessibleAction;BOOL accessibleSelection;BOOL accessibleText;BOOL accessibleInterfaces" ;~ const $cTypeStructAccessibleContextInfo="WCHAR data[8192]" Local $AccessibleContextInfo=DllStructCreate($cTypeStructAccessibleContextInfo) ; If an error occurred display the error code and return False. If @error Then consolewrite("Error in DllStructCreate $AccessibleContextInfo, Code: " & @error) exit EndIf consolewrite("We will call contextinfo with Struct Size: " & DllStructGetSize($AccessibleContextInfo) & @CRLF) ;~ Local $AccessibleContextInfo=DllStructCreate("WCHAR name[1024];WCHAR description[1024];WCHAR role[256];WCHAR states[256];int64 indexInParent;int64 childrenCount;int64 x;int64 y;int64 width;int64 height;byte accessibleComponent;byte accessibleAction;byte accessibleSelection;byte accessibleText;byte accessibleinterfaces") ;~ #AutoIt3Wrapper_UseX64=Y #include <EditConstants.au3> #include <GUIConstantsEx.au3> #include <WindowsConstants.au3> #include <constants.au3> #include <WinAPI.au3> #include <debug.au3> local $hwnd local $i local $result local $vmID local $ac ;~ Make messages elevated _ChangeWindowMessageFilter($WM_COPYDATA,1) for $i=$WM_USER to $WM_USER+65536 _ChangeWindowMessageFilter($i, 1) Next Func _ChangeWindowMessageFilter($iMsg, $iAction) Local $aCall = DllCall("user32.dll", "bool", "ChangeWindowMessageFilter", "dword", $iMsg, "dword", $iAction) If @error Or Not $aCall[0] Then Return SetError(1, 0, 0) Return 1 EndFunc local $bridgeDLL=DLLOPEN($AccessBridgeDLL) ;~ sleep(500) If $bridgeDLL=true Then consolewrite("Windows accessbridge 32 opened " & $bridgeDLL & @CRLF) Else consolewrite("DLL not found try to change to WindowsAccessBridge-64.dll if you use 64 bits autoit and windows") endIf ;~ TODO: Handle messages received from initialize $result =dllcall($bridgeDLL,"NONE", "Windows_run") consolewrite($result & " " & @error & " initializeAccessBridge is finished") sleep(250) consolewrite("Windows_run passed :" & $result & @CRLF) Local $var = WinList() consolewrite("Before loading all Windows:" & $var[0][0] & @CRLF) For $i = 1 To $var[0][0] ; Only display visble windows that have a title If IsVisible($var[$i][1]) Then local $handle=wingethandle($var[$i][0]) $result =dllcall($bridgeDLL,"BOOL:cdecl", "isJavaWindow", "hwnd", $handle) if @error=0 Then if $result[0]=1 Then consolewrite( "JAVA window found <" & $i & "> Java Window Title=" & $var[$i][0] & " Handle=" & $var[$i][1] & @TAB & " res: " & $result[0] & @CRLF) local $acParent=0 local $vmID=0 ;~AccessibilityContext (acParent above), which I had incorrectly mapped as an IntPtr, is actually an Int32 when using the "legacy" WindowsAccessBridge.dll library (used under x86), ;~and an Int64 when using the WOW64 WindowsAccessBridge-32.dll library. ;~WindowsAccessBridge-32.dll bool getAccessibleContextFromHWND(IntPtr hwnd, out Int32 vmID, out Int64 acParent); ;~WindowsAccessBridge.dll bool _getAccessibleContextFromHWND(IntPtr hwnd, out Int32 vmID, out Int32 acParent); ;~ _fixBridgeFunc(BOOL,'getAccessibleContextFromHWND',HWND,POINTER(c_long),POINTER(JOBJECT64),errcheck=True) $result =dllcall($bridgeDLL,"BOOL:cdecl", "getAccessibleContextFromHWND", "hwnd", $handle, "long*", $vmID, "UINT64*", $acParent) ;~ $result =dllcall($bridgeDLL,"BOOL:cdecl", "getAccessibleContextFromHWND", "hwnd", $handle, "long*", $vmID, "ptr*", $acParent) ;~ $result =dllcall($bridgeDLL,"BOOL:cdecl", "getAccessibleContextFromHWND", "hwnd", $handle, "long*", $vmID, "INT64*", $acParent) if @error=0 Then consolewrite("Result getAccessibleContextFromHWND is <" & $result & "> ubound: " & ubound($result) & @crlf) $vmID=$result[2] $acParent=$result[3] consolewrite(" We have a VMid " & $vmID & " ac " & $acParent & @CRLF) if @error > 0 then consolewrite("Struct error") ;~ $result =dllcall($bridgeDLL, "BOOL:cdecl", "getVersionInfo", "long", $vmId, "struct", $AccessBridgeVersionInfo) ;~ typedef BOOL (*GetVersionInfoFP) (long vmID, AccessBridgeVersionInfo *info); $result =dllcall($bridgeDLL, "BOOL:cdecl", "getVersionInfo", "long", $vmId, "struct*", DllStructGetPtr($AccessBridgeVersionInfo)) if @error=0 Then $s1=dllstructgetdata($AccessBridgeVersionInfo, "VMVersion") $s2=dllstructgetdata($AccessBridgeVersionInfo, "bridgeJavaClassVersion") $s3=dllstructgetdata($AccessBridgeVersionInfo, "bridgeJavaDLLVersion") $s4=dllstructgetdata($AccessBridgeVersionInfo, "bridgeWinDLLVersion") consolewrite("Call version info: PASSED VMID found : " & $vmID & @CRLF) consolewrite(" VMVersion: <" & $s1 & ">" & @CRLF) consolewrite(" bridgeJavaClassVersion: <" & $s2 & ">" & @CRLF) consolewrite(" bridgeJavaDLLVersion: <" & $s3 & ">" & @CRLF) consolewrite(" bridgeWinDLLVersion: <" & $s4 & ">" & @CRLF) Else consolewrite("getVersionInfo error:" & @error & @CRLF) EndIf ;~ /** ;~ * Returns the Accessible Context for the top level object in ;~ * a Java Window. This is same Accessible Context that is obtained ;~ * from GetAccessibleContextFromHWND for that window. Returns ;~ * (AccessibleContext)0 on error. ;~ */ ;~ AccessibleContext getTopLevelObject (const long vmID, const AccessibleContext accessibleContext); ;~ typedef AccessibleContext (*getTopLevelObjectFP) (const long vmID, const AccessibleContext ac); ;~ _fixBridgeFunc(JOBJECT64,'getTopLevelObject',c_long,JOBJECT64,errcheck=True) local $topAC $result=dllcall($bridgeDLL, "UINT64:cdecl", "getTopLevelObject", "long", $vmId, "UINT64", $topAC) if @error=0 Then consolewrite(" *** getTopLevelObject Result is <" & $result[0] & "> ubound: " & ubound($result) & @crlf) $vmID=$result[1] $topAC=$result[2] consolewrite("We have a VMid " & $vmID & " toplevel object ac " & $topAC & @CRLF) Else consolewrite("getTopLevelObjecterror:" & @error & @CRLF) endif consolewrite("BEFORE BREAKING" & @CRLF) consolewrite(" We still have a VMid " & $vmID & " acParent " & $acParent & @CRLF) ;~ Retrieves an AccessibleContextInfo object of the AccessibleContext object ac. ;~ $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "ptr", $ac, "ptr", DllStructGetPtr($AccessibleContextInfo)) ;~ typedef BOOL (*GetAccessibleContextInfoFP) (long vmID, AccessibleContext ac, AccessibleContextInfo *info); ;~ _fixBridgeFunc(BOOL,'getAccessibleContextInfo',c_long,JOBJECT64,POINTER(AccessibleContextInfo),errcheck=True) ;~ $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","INT64", $vmId, "int", $ac, "ptr*", DllStructGetPtr($AccessibleContextInfo)) ;~ $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "long", $ac, "ptr", DllStructGetPtr($AccessibleContextInfo),DllStructGetSize($AccessibleContextInfo)) ;~ $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "INT64", $acParent, "struct*", DllStructGetPtr($AccessibleContextInfo)) $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "INT64", $acParent, "struct*", $AccessibleContextInfo) if @error<>0 Then consolewrite("We have an error " & @error & @CRLF) EndIf ;~ We should pass it by reference ;~ Exception code c0000005 is an access violation, also known as general protection fault. The program is reading from, or writing to, an address which is not part of the virtual address space. ;~ A very common cause is that you are de-referencing a stale pointer. In other words, the pointer was once valid, but you have subsequently freed it. Then later when you attempt to access it, ;~ an exception is raised. ;~ The problem is don't use out parameter. Use the IntPtr. ;~ So it should be getAccessibleContextInfo(Int32 vmID, IntPtr accessibleContext, IntPtr acInfo) consolewrite("AFTER BREAKING" & @CRLF) consolewrite("Result is <" & $result & "> ubound: " & ubound($result) & @crlf) $AccessibleContextInfo=$result[3] $s1=dllstructgetdata($AccessibleContextInfo, "name") $s2=dllstructgetdata($AccessibleContextInfo, "description") $s3=dllstructgetdata($AccessibleContextInfo, "role") $s4=dllstructgetdata($AccessibleContextInfo, "role_en_US") $s5=dllstructgetdata($AccessibleContextInfo, "states") $s6=dllstructgetdata($AccessibleContextInfo, "states_en_US") $s7=dllstructgetdata($AccessibleContextInfo, "indexInParent") $s8=dllstructgetdata($AccessibleContextInfo, "childrenCount") $s9=dllstructgetdata($AccessibleContextInfo, "x") $s10=dllstructgetdata($AccessibleContextInfo, "y") $s11=dllstructgetdata($AccessibleContextInfo, "width") $s12=dllstructgetdata($AccessibleContextInfo, "heigth") consolewrite("name: <" & $s1 & ">" & @CRLF) consolewrite("description: <" & $s2 & ">" & @CRLF) consolewrite("role: <" & $s3 & ">" & @CRLF) consolewrite("role_en_US: <" & $s4 & ">" & @CRLF) consolewrite("states: <" & $s5 & ">" & @CRLF) consolewrite("states_en_US: <" & $s6 & ">" & @CRLF) consolewrite("indexInParent: <" & $s7 & ">" & @CRLF) consolewrite("childrenCount: <" & $s8 & ">" & @CRLF) consolewrite("x: <" & $s9 & ">" & @CRLF) consolewrite("y: <" & $s10 & ">" & @CRLF) consolewrite("width: <" & $s11 & ">" & @CRLF) consolewrite("height: <" & $s12 & ">" & @CRLF) ;~ AccessibleContext GetAccessibleChildFromContext(long vmID, AccessibleContext ac, jint index); ;~ Returns an AccessibleContext object that represents the nth child of the object ac, where n is specified by the value index. Else consolewrite( @error & " No context found" & @CRLF) endif EndIf Else EndIf endif Next ;~ http://www.autohotkey.com/board/topic/95343-how-to-send-to-unseen-controls-in-a-java-app/ local $result =dllcall($bridgeDLL,"BOOL", "shutdownAccessBridge") Func IsVisible($handle) If BitAND(WinGetState($handle), 2) Then Return 1 Else Return 0 EndIf EndFunc ;==>IsVisible Edited January 3, 2015 by junkew FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets
LarsJ Posted January 4, 2015 Posted January 4, 2015 You use eitherDllCall( ... "struct*", $AccessibleContextInfo )orDllCall( ... "ptr", DllStructGetPtr($AccessibleContextInfo) )"struct*" tells DllCall to get the pointer by its internal code. I think the "struct*" method is the preferred method. It's shorter and easier to read. And probably also a little bit faster, because you don't need DllStructGetPtr. Controls,  File Explorer,  ROT objects,  UI Automation,  Windows Message MonitorCompiled code: Accessing AutoIt variables,  DotNet.au3 UDF,  Using C# and VB codeShell menus: The Context menu,  The Favorites menu. Shell related: Control Panel,  System Image ListsGraphics related: Rubik's Cube,  OpenGL without external libraries,  Navigating in an image,  Non-rectangular selectionsListView controls: Colors and fonts,  Multi-line header,  Multi-line items,  Checkboxes and icons,  Incremental searchListView controls: Virtual ListViews,  Editing cells,  Data display functions
junkew Posted January 4, 2015 Posted January 4, 2015 Tried many combinations of int*, struct*. to see what works and what not learning a lot but still not working continued in general help and support '?do=embed' frameborder='0' data-embedContent>> FAQ 31 How to click some elements, FAQ 40 Test automation with AutoIt, Multithreading CLR .NET Powershell CMDLets
pd Posted June 3, 2015 Posted June 3, 2015 Hi experts,I also tried with Java Access Bridge but seems like it is not able to read control IDs from Java AWT application. Any help is appreciated.Thanks,PD
pd Posted June 3, 2015 Posted June 3, 2015 Hi All,Does this UDF work with AWT application like SQL Devloper? I tried using it but it is not able to identify controls.Regards,PD
TheDcoder Posted June 21, 2015 Posted June 21, 2015 @pd This UDF was last updated in 2009... Since 2009 there were many script breaking changes, You need to work around the UDF to make it work  TD EasyCodeIt - A cross-platform AutoIt implementation - Fund the development! (GitHub will double your donations for a limited time) DcodingTheWeb Forum - Follow for updates and Join for discussion
ravi2015 Posted November 2, 2015 Posted November 2, 2015 Can you please send one small sample script/UDF to interact awt window?
sherek Posted December 15, 2017 Posted December 15, 2017 On 19/01/2009 at 9:10 AM, seangriffin said: This UDF provides automation support for Java applications.  This UDF currently automates standalone applications and browser-based applets, developed in either Swing or the Abstract Window Toolkit (AWT).  REQUIREMENTS:  AutoIt3 3.2 or higher Java Runtime Environment (JRE) 1.4 or above Java Access Bridge for the Microsoft Windows Operating System Microsoft Visual C++ 2008 Redistributable AutoIT for Java Access dynamically linked library (DLL) Tesseract UDF 0.6 Manual check and update of the Java Access Bridge INSTALLATION: To install the latest Java Runtime Environment (JRE):  Open the web page http://java.com/en/download/manual.jsp. Click Windows XP/Vista/2000/2003 Offline. Save the file to your computer and run it (ie. jre-6u11-windows-i586-p-s.exe). Follow the installation instructions. Restart your computer when prompted. To install the Java Access Bridge for the Microsoft Windows Operating System:  Open the web page Java Access Bridge. Click Download Java Access Bridge 2.0.1. Click I agree to the Software License Agreement. Click Continue. Click accessbridge-2_0_1.exe. Save the file to a location on your computer. Run the file accessbridge-2_0_1.exe from your computer. Follow the installation instructions. Restart your computer when prompted. To install the Microsoft Visual C++ 2008 Redistributable:  Open the web page http://www.microsoft.com/downloads/details.aspx?FamilyID=9b2da534-3e03-4391-8a4d-074b9f2bc1bf&displaylang=en. Click Download. Save the file to a location on your computer. Run the file vcredist_x86.exe from your computer. Follow the installation instructions. To install the AutoIT for Java Access dynamically linked library (DLL):  Right-click on the link AutoITJavaAccess.dll below. Save the file to your Windows System32 folder (ie. C:\WINDOWS\system32). To install the Tesseract UDF 0.6:  Open the link http://www.autoitscript.com/forum/index.php?showtopic=89542. Save the file to your AutoIT include folder (ie. C:\Program Files\AutoIT\Include). To manually check and update the Java Access Bridge:  Note:- This step should be performed in addition to the install of the Java Access Bridge above.  Using Windows Explorer, go to your Java folder C:\Program Files\Java. Open the first subfolder (ie. j2re1.4.2_19). Go to the subfolder lib\ext. If the files access-bridge.jar and jaccess.jar already exist then go to step 9 below. Copy the file access-bridge.jar to this subfolder (ie. C:\Program Files\Java\jre1.4.2_19\lib\ext). If the subfolder in step 2 above starts with j2re1.2 or jre1.2, then copy the file jaccess-1_2.jar to this subfolder (ie. C:\Program Files\Java\jre1.4.2_19\lib\ext). If the subfolder in step 2 above starts with j2re1.3 or jre1.3, then copy the file jaccess-1_3.jar to this subfolder (ie. C:\Program Files\Java\jre1.4.2_19\lib\ext). If the subfolder in step 2 above starts with j2re1.4 or jre1.4, then copy the file jaccess-1_4.jar to this subfolder (ie. C:\Program Files\Java\jre1.4.2_19\lib\ext). Repeat steps 3 to 8 for every subfolder within the Java folder. To confirm that the installation process was successful:  Using Windows Explorer, go to C:\Program Files\Java Access Bridge and double-click JavaFerret.exe. In the Java Ferret window, select the menu item JavaEvents -> Track Mouse Events. Drag your mouse over any Java application, and the Java Ferret window should dynamically update with information about the Java objects as you drag your mouse over them. LIST OF FUNCTIONS:  DEMONSTRATION: <Under Construction>  EXAMPLES: _JavaObjValueGet.au3 _JavaObjValueSet.au3_JavaObjValueSelect.au3 _JavaObjValuesGet.au3_JavaObjPropertyGet.au3 DOWNLOAD:  Latest Version - v0.7 (17/03/09) AutoITJavaAccess.dllJava.au3  I'm trying to get a value from a .jar file (generate a test with netbeans 8.1 and the javaferret generates the dump), where it is not getting value from a JTextField. It is reading the dll normally (returns the value 1 in the DllOpen ("AutoITJavaAccess.dll")). I am using windows 7 ultimate, 64 bit. The value of the JTextField comes empty, and the JEdit examples do not work. Some help? Thank you!
rlvitorino Posted February 24, 2019 Posted February 24, 2019 The error is happing in my java.au3 >"C:\Program Files (x86)\AutoIt3\SciTE\..\AutoIt3.exe" "C:\Program Files (x86)\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.au3" /run /prod /ErrorStdOut /in "C:\Users\Digão\Downloads\_JavaObjValueGet (1).au3" /UserParams +>16:11:42 Starting AutoIt3Wrapper v.18.708.1148.0 SciTE v.4.1.0.0 Keyboard:00000416 OS:WIN_10/ CPU:X64 OS:X64 Environment(Language:0416) CodePage:0 utf8.auto.check:4 +> SciTEDir => C:\Program Files (x86)\AutoIt3\SciTE UserDir => C:\Users\Digão\AppData\Local\AutoIt v3\SciTE\AutoIt3Wrapper SCITE_USERHOME => C:\Users\Digão\AppData\Local\AutoIt v3\SciTE >Running AU3Check (3.3.14.5) from:C:\Program Files (x86)\AutoIt3 input:C:\Users\Digão\Downloads\_JavaObjValueGet (1).au3 "C:\Users\Digão\Downloads\Java.au3"(871,41) : warning: $javaferret_obj_role: possibly used before declaration. if StringCompare($javaferret_obj_role, ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ "C:\Users\Digão\Downloads\Java.au3"(871,41) : error: $javaferret_obj_role: undeclared global variable. if StringCompare($javaferret_obj_role, ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ C:\Users\Digão\Downloads\_JavaObjValueGet (1).au3 - 1 error(s), 1 warning(s) !>16:11:43 AU3Check ended. Press F4 to jump to next error.rc:2 +>16:11:43 AutoIt3Wrapper Finished. >Exit code: 2 Time: 1.566 Â
rlvitorino Posted April 18, 2019 Posted April 18, 2019 Â I'm trying to automate the java process, although I'm having error below are you helping me?
BrewManNH Posted April 18, 2019 Posted April 18, 2019 How are you calling that function? If using a hotkey or an OnEvent trigger, it won't work. Those types of function calls don't use the parameters in the function, so the variables are undeclared. If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag GudeHow to ask questions the smart way! I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from. Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator
rlvitorino Posted April 18, 2019 Posted April 18, 2019 (edited) I just run java.au3 and this error appeared. Edited April 18, 2019 by rlvitorino
BrewManNH Posted April 18, 2019 Posted April 18, 2019 2 minutes ago, rlvitorino said: I just circled java.au3 I don't know what that means. If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag GudeHow to ask questions the smart way! I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from. Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator
BrewManNH Posted April 18, 2019 Posted April 18, 2019  He kind of explained that you have to create the variable yourself and assign an object to it, if I'm reading it right. Not too sure why it's set p like this though. If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag GudeHow to ask questions the smart way! I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from. Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator
rlvitorino Posted April 18, 2019 Posted April 18, 2019 Â Â Would not you have the latest version of JAVA UDF this version that is in the link is giving many problems?
BrewManNH Posted April 18, 2019 Posted April 18, 2019 I've never used it, so I don't have a copy of it. Have you looked through the whole thread to make sure it hasn't been updated by someone else in it? If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag GudeHow to ask questions the smart way! I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from. Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now