Java UDF

[junkew]

Probably all this stuff can work natively from AutoIT this is the result I have so far in W7 with Java 7 installed (worked similar in Java 6)

expandcollapse popup

;~ By inspecting the WindowsAccessBridge-32.dll it reveals some information about the hidden dialogs
;~   So it seems the hidden dialog is shown after you call windows_run() no clue if interaction is needed
;~
;~ Somehow it sends a message unclear if this is to the JVM to respond to
;~   push   SSZ6E73E320_AccessBridge_FromJava_Hello
;~   push   SSZ6E73E300_AccessBridge_FromWindows_Hello
;~   db 'AccessBridge-FromWindows-Hello',0
;~   db 'AccessBridge-FromJava-Hello',0

;~ Copied from this NVDA reference and translated to AutoIT
;~ http://www.webbie.org.uk/nvda/api/JABHandler-pysrc.html
;~
;~ def initialize():
;~         global isRunning
;~         if not bridgeDll:
;~                 raise NotImplementedError("dll not available")
;~         bridgeDll.Windows_run()
;~         #Accept wm_copydata and any wm_user messages from other processes even if running with higher privilages
;~***         ChangeWindowMessageFilter=getattr(windll.user32,'ChangeWindowMessageFilter',None)
;~***         if ChangeWindowMessageFilter:
;~***                 if not ChangeWindowMessageFilter(winUser.WM_COPYDATA,1):
;~***                         raise WinError()
;~***                 for msg in xrange(winUser.WM_USER+1,65535):
;~***                         if not ChangeWindowMessageFilter(msg,1):
;~***                                 raise WinError()
;~         #Register java events
;~         bridgeDll.setFocusGainedFP(internal_event_focusGained)
;~         bridgeDll.setPropertyActiveDescendentChangeFP(internal_event_activeDescendantChange)
;~         bridgeDll.setPropertyNameChangeFP(event_nameChange)
;~         bridgeDll.setPropertyDescriptionChangeFP(event_descriptionChange)
;~         bridgeDll.setPropertyValueChangeFP(event_valueChange)
;~         bridgeDll.setPropertyStateChangeFP(internal_event_stateChange)
;~         bridgeDll.setPropertyCaretChangeFP(internal_event_caretChange)
;~         isRunning=True

;create the structs
;~
;~  http://docs.oracle.com/javase/accessbridge/2.0.2/api.htm
;~
;~ #define MAX_STRING_SIZE     1024
;~ #define SHORT_STRING_SIZE    256
;~                  struct AccessBridgeVersionInfo {
;~  wchar_t VMversion[SHORT_STRING_SIZE];              // version of the Java VM
;~  wchar_t bridgeJavaClassVersion[SHORT_STRING_SIZE]; // version of the AccessBridge.class
;~  wchar_t bridgeJavaDLLVersion[SHORT_STRING_SIZE];   // version of JavaAccessBridge.dll
;~  wchar_t bridgeWinDLLVersion[SHORT_STRING_SIZE];    // version of WindowsAccessBridge.dll
;~ };

;~ Local $AccessBridgeVersionInfo=DllStructCreate("WCHAR VMversion[256];WCHAR bridgeJavaClassVersion[256];WCHAR bridgeJavaDLLVersion[256];WCHAR bridgeWinDLLVersion[256]")
Local $AccessBridgeVersionInfo=DllStructCreate("WCHAR VMversion[256];WCHAR bridgeJavaClassVersion[256];WCHAR bridgeJavaDLLVersion[256];WCHAR bridgeWinDLLVersion[256]")

;~ struct AccessibleContextInfo {
;~  wchar_ name[MAX_STRING_SIZE];        // the AccessibleName of the object
;~  wchar_ description[MAX_STRING_SIZE]; // the AccessibleDescription of the object
;~  wchar_ role[SHORT_STRING_SIZE];      // localized AccesibleRole string
;~  wchar_ states[SHORT_STRING_SIZE];    // localized AccesibleStateSet string
;~                                       //   (comma separated)
;~  jint indexInParent                   // index of object in parent
;~  jint childrenCount                   // # of children, if any
;~  jint x;                              // screen x-axis co-ordinate in pixels
;~  jint y;                              // screen y-axis co-ordinate in pixels
;~  jint width;                          // pixel width of object
;~  jint height;                         // pixel height of object
;~  BOOL accessibleComponent;            // flags for various additional
;~  BOOL accessibleAction;               // Java Accessibility interfaces
;~  BOOL accessibleSelection;            // FALSE if this object doesn't
;~  BOOL accessibleText;                 // implement the additional interface
;~  BOOL accessibleInterfaces;           // new bitfield containing additional
;~                                       //   interface flags
;~ };
;~ Local $AccessibleContextInfo=DllStructCreate("WCHAR name[1024];WCHAR description[1024];WCHAR role[256];WCHAR states[256];int indexInParent;int childrenCount;int x;int y;int width;int height;BOOL accessibleComponent;BOOL accessibleAction;BOOL accessibleSelection;BOOL accessibleText;BOOL accessibleInterfaces")
Local $AccessibleContextInfo=DllStructCreate("WCHAR name[1024];WCHAR description[1024];WCHAR role[256];WCHAR states[256];int64 indexInParent;int64 childrenCount;int64 x;int64 y;int64 width;int64 height;byte accessibleComponent;byte accessibleAction;byte accessibleSelection;byte accessibleText;byte accessibleinterfaces")


;~ #AutoIt3Wrapper_UseX64=Y

#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <constants.au3>
#include <WinAPI.au3>
#include <debug.au3>

local $hwnd
local $i
local $result
local $vmID
local $ac

;~ Make messages elevated
_ChangeWindowMessageFilter($WM_COPYDATA,1)
for $i=$WM_USER to $WM_USER+65536
    _ChangeWindowMessageFilter($i, 1)
Next

Func _ChangeWindowMessageFilter($iMsg, $iAction)
    Local $aCall = DllCall("user32.dll", "bool", "ChangeWindowMessageFilter", "dword", $iMsg, "dword", $iAction)
    If @error Or Not $aCall[0] Then Return SetError(1, 0, 0)
    Return 1
EndFunc

local $bridgeDLL=DLLOPEN("WindowsAccessBridge-32.dll")
;~ sleep(500)
If $bridgeDLL=true Then
    consolewrite("Windows accessbridge 32 opened " & $bridgeDLL & @CRLF)
Else
    consolewrite("DLL not found try to change to WindowsAccessBridge-64.dll if you use 64 bits autoit and windows")
endIf

;~ TODO: Handle messages received from initialize

$result =dllcall($bridgeDLL,"NONE", "Windows_run")
consolewrite($result & " " & @error & " initializeAccessBridge is finished")

sleep(250)

consolewrite("Windows_run passed :" & $result & @CRLF)

Local $var = WinList()
consolewrite("Before loading all Windows:" & $var[0][0] & @CRLF)

For $i = 1 To $var[0][0]
; Only display visble windows that have a title
    If IsVisible($var[$i][1]) Then
        local $handle=wingethandle($var[$i][0])
        $result =dllcall($bridgeDLL,"BOOL:cdecl", "isJavaWindow", "hwnd", $handle)
        if @error=0 Then
            if $result[0]=1 Then
                consolewrite( $i & " Java Window Title=" & $var[$i][0] &  " Handle=" & $var[$i][1] & @TAB & " res: " &  $result[0] & @CRLF)

            local $ac=0
            local $vmID=0

            $result =dllcall($bridgeDLL,"BOOL:cdecl", "getAccessibleContextFromHWND", "hwnd", $handle, "long*", $vmID, "ptr*", $ac)

                if @error=0 Then
                    consolewrite("We have a context " & @CRLF)
                    $vmID=$result[2]
                    $ac=$result[3]
                    consolewrite("We have a VMid " & $vmID & " ac " & $ac & @CRLF)


                    if @error > 0 then consolewrite("Struct error")

;~                  $result =dllcall($bridgeDLL, "BOOL:cdecl", "getVersionInfo", "long", $vmId, "struct", $AccessBridgeVersionInfo)
                    $result =dllcall($bridgeDLL, "BOOL:cdecl", "getVersionInfo", "long", $vmId, "ptr", DllStructGetPtr($AccessBridgeVersionInfo))
                    consolewrite( @error & " context found of " & $vmID & @CRLF)
                    if @error=0 Then

                        $s1=dllstructgetdata($AccessBridgeVersionInfo, "VMVersion")
                        $s2=dllstructgetdata($AccessBridgeVersionInfo, "bridgeJavaClassVersion")
                        $s3=dllstructgetdata($AccessBridgeVersionInfo, "bridgeJavaDLLVersion")
                        $s4=dllstructgetdata($AccessBridgeVersionInfo, "bridgeWinDLLVersion")

                        consolewrite("VMVersion: <" & $s1 & ">" & @CRLF)
                        consolewrite("bridgeJavaClassVersion: <" & $s2 & ">" & @CRLF)
                        consolewrite("bridgeJavaDLLVersion: <" & $s3 & ">" & @CRLF)
                        consolewrite("bridgeWinDLLVersion: <" & $s4 & ">" & @CRLF)
                    EndIf

;~ Retrieves an AccessibleContextInfo object of the AccessibleContext object ac.
                    $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "ptr", $ac, "ptr", DllStructGetPtr($AccessibleContextInfo))
;~ Local $AccessibleContextInfo=DllStructCreate("WCHAR name[1024];WCHAR description[1024];WCHAR role[256];WCHAR states[256];int indexInParent;int childrenCount;int x;int y;int width;int height;BOOL accessibleComponent;BOOL accessibleAction;BOOL accessibleSelection;BOOL accessibleText;BOOL accessibleInterfaces")
                    consolewrite("Result is " & $result )
                    if @error<>0 Then
                        consolewrite("We have an error " & @CRLF)
                    EndIf


                    $s1=dllstructgetdata($AccessibleContextInfo, "name")
                    $s2=dllstructgetdata($AccessibleContextInfo, "description")
                    consolewrite("name: <" & $s1 & ">" & @CRLF)
                    consolewrite("description: <" & $s2 & ">" & @CRLF)

;~ AccessibleContext GetAccessibleChildFromContext(long vmID, AccessibleContext ac, jint index);

;~ Returns an AccessibleContext object that represents the nth child of the object ac, where n is specified by the value index.


                Else
                    consolewrite( @error & " No context found" & @CRLF)
                endif
            EndIf
        Else
        EndIf
    endif
Next
;~ http://www.autohotkey.com/board/topic/95343-how-to-send-to-unseen-controls-in-a-java-app/
local $result =dllcall($bridgeDLL,"BOOL", "shutdownAccessBridge")

Func IsVisible($handle)
    If BitAND(WinGetState($handle), 2) Then
        Return 1
    Else
        Return 0
    EndIf

EndFunc   ;==>IsVisible

1. start a java application

2. run above script

3. problem is still I am not getting the java context which should work

Reference code from NVDA python

https://github.com/ckundo/nvda/blob/master/source/JABHandler.py

problem is somewhere here

;~ Retrieves an AccessibleContextInfo object of the AccessibleContext object ac.
;~                  $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "ptr", $ac, "ptr", DllStructGetPtr($AccessibleContextInfo))
consolewrite("BEFORE BREAKING")
                    $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","INT64", $vmId, "ptr", $ac, "ptr", DllStructGetPtr($AccessibleContextInfo))
consolewrite("AFTER BREAKING")

Edited August 3, 2014 by junkew

[junkew]

still not fixed but with JDK 1.8 I seem to be having more information of the definitions

 

When you install JDK 1.8

 

definitions can be found there

C:Program Files (x86)Javajdk1.8.0_25includewin32bridge

;~ *******
;~ Every AccessibleContext IntPtr must be replaced by long, including but not limited to getAccessibleContextFromHWND, getAccessibleParentFromContext, getAccessibleChildFromContext, getAccessibleTextInf
;~ JOBJECT64 is defined as jlong on 64-bit systems and jobject on legacy versions of Java Access Bridge. For definitions, see the section ACCESSBRIDGE_ARCH_LEGACY in the AccessBridgePackages.h header file.
;~ #ifdef ACCESSBRIDGE_ARCH_LEGACY
;~ typedef jobject JOBJECT64;
;~ typedef HWND ABHWND64;
;~ #define ABHandleToLong
;~ #define ABLongToHandle
;~ #else
;~ typedef jlong JOBJECT64;
;~ typedef long ABHWND64;
;~ #define ABHandleToLong HandleToLong
;~ #define ABLongToHandle LongToHandle
;~ #endif
;~ a jobject is a 32 bit pointer for 32 bit builds
;~ *******

steps to reproduce

0. install jdk 1.8

1. run jdk-8u25-windows-i586-demosjdk1.8.0_25demojfcSwingSet2

2. its a little unclear if jabswitch /enable has to be called but to be sure its on it should be started from commandline

3. run script as posted

4. Validate with javamonkey and javaferret what you get (script should return the same result)

 

seems just a stupid incorrect dll call due to incorrect structure definition

Any help would be appreciated

 

latest source trial and error

expandcollapse popup

#AutoIt3Wrapper_UseX64=Y
consolewrite("We are using " & @OSArch & " at cpu " & @CPUArch & " Autoit 64 bit version "& @AutoItX64 & @CRLF)
if @AutoItX64 = 1 then
    local $accessBridgeDLL="WindowsAccessBridge-64.dll"
Else
    local $accessBridgeDLL="WindowsAccessBridge-32.dll"
EndIf

;~ By inspecting the WindowsAccessBridge-32.dll it reveals some information about the hidden dialogs
;~   So it seems the hidden dialog is shown after you call windows_run() no clue if interaction is needed
;~
;~ Somehow it sends a message unclear if this is to the JVM to respond to
;~   push   SSZ6E73E320_AccessBridge_FromJava_Hello
;~   push   SSZ6E73E300_AccessBridge_FromWindows_Hello
;~   db 'AccessBridge-FromWindows-Hello',0
;~   db 'AccessBridge-FromJava-Hello',0

;~ JABHANDLER.PY is a usefull source to study on internet

;~ *******
;~ Every AccessibleContext IntPtr must be replaced by long, including but not limited to getAccessibleContextFromHWND, getAccessibleParentFromContext, getAccessibleChildFromContext, getAccessibleTextInf
;~ JOBJECT64 is defined as jlong on 64-bit systems and jobject on legacy versions of Java Access Bridge. For definitions, see the section ACCESSBRIDGE_ARCH_LEGACY in the AccessBridgePackages.h header file.
;~ #ifdef ACCESSBRIDGE_ARCH_LEGACY
;~ typedef jobject JOBJECT64;
;~ typedef HWND ABHWND64;
;~ #define ABHandleToLong
;~ #define ABLongToHandle
;~ #else
;~ typedef jlong JOBJECT64;
;~ typedef long ABHWND64;
;~ #define ABHandleToLong HandleToLong
;~ #define ABLongToHandle LongToHandle
;~ #endif
;~ a jobject is a 32 bit pointer for 32 bit builds
;~ 'bool' is a built-in C++ type while 'BOOL' is a Microsoft specific type that is defined as an 'int'. You can find it in 'windef.h'
;~ *******

;~ Primitive Types and Native Equivalents
;~ Java Type    Native Type Description
;~ boolean      jboolean    unsigned 8 bits
;~ byte         jbyte       signed 8 bits
;~ char         jchar       unsigned 16 bits
;~ short        jshort      signed 16 bits
;~ int          jint        signed 32 bits
;~ long         jlong       signed 64 bits
;~ float        jfloat      32 bits
;~ double       jdouble     64 bits
;~ void         void        not applicable

;~ Copied from this NVDA reference and translated to AutoIT
;~ http://www.webbie.org.uk/nvda/api/JABHandler-pysrc.html
;~
;~ def initialize():
;~         global isRunning
;~         if not bridgeDll:
;~                 raise NotImplementedError("dll not available")
;~         bridgeDll.Windows_run()
;~         #Accept wm_copydata and any wm_user messages from other processes even if running with higher privilages
;~***         ChangeWindowMessageFilter=getattr(windll.user32,'ChangeWindowMessageFilter',None)
;~***         if ChangeWindowMessageFilter:
;~***                 if not ChangeWindowMessageFilter(winUser.WM_COPYDATA,1):
;~***                         raise WinError()
;~***                 for msg in xrange(winUser.WM_USER+1,65535):
;~***                         if not ChangeWindowMessageFilter(msg,1):
;~***                                 raise WinError()
;~         #Register java events
;~         bridgeDll.setFocusGainedFP(internal_event_focusGained)
;~         bridgeDll.setPropertyActiveDescendentChangeFP(internal_event_activeDescendantChange)
;~         bridgeDll.setPropertyNameChangeFP(event_nameChange)
;~         bridgeDll.setPropertyDescriptionChangeFP(event_descriptionChange)
;~         bridgeDll.setPropertyValueChangeFP(event_valueChange)
;~         bridgeDll.setPropertyStateChangeFP(internal_event_stateChange)
;~         bridgeDll.setPropertyCaretChangeFP(internal_event_caretChange)
;~         isRunning=True

;create the structs
;~
;~  http://docs.oracle.com/javase/accessbridge/2.0.2/api.htm
;~
;~ #define MAX_STRING_SIZE     1024
;~ #define SHORT_STRING_SIZE    256
;~                  struct AccessBridgeVersionInfo {
;~  wchar_t VMversion[SHORT_STRING_SIZE];              // version of the Java VM
;~  wchar_t bridgeJavaClassVersion[SHORT_STRING_SIZE]; // version of the AccessBridge.class
;~  wchar_t bridgeJavaDLLVersion[SHORT_STRING_SIZE];   // version of JavaAccessBridge.dll
;~  wchar_t bridgeWinDLLVersion[SHORT_STRING_SIZE];    // version of WindowsAccessBridge.dll
;~ };

;~ Local $AccessBridgeVersionInfo=DllStructCreate("WCHAR VMversion[256];WCHAR bridgeJavaClassVersion[256];WCHAR bridgeJavaDLLVersion[256];WCHAR bridgeWinDLLVersion[256]")
Local $AccessBridgeVersionInfo=DllStructCreate("WCHAR VMversion[256];WCHAR bridgeJavaClassVersion[256];WCHAR bridgeJavaDLLVersion[256];WCHAR bridgeWinDLLVersion[256]")

;~ struct AccessibleContextInfo {
;~  wchar_ name[MAX_STRING_SIZE];        // the AccessibleName of the object
;~  wchar_ description[MAX_STRING_SIZE]; // the AccessibleDescription of the object
;~  wchar_ role[SHORT_STRING_SIZE];      // localized AccesibleRole string
;~  wchar_ states[SHORT_STRING_SIZE];    // localized AccesibleStateSet string
;~                                       //   (comma separated)
;~  jint indexInParent                   // index of object in parent
;~  jint childrenCount                   // # of children, if any
;~  jint x;                              // screen x-axis co-ordinate in pixels
;~  jint y;                              // screen y-axis co-ordinate in pixels
;~  jint width;                          // pixel width of object
;~  jint height;                         // pixel height of object
;~  BOOL accessibleComponent;            // flags for various additional
;~  BOOL accessibleAction;               // Java Accessibility interfaces
;~  BOOL accessibleSelection;            // FALSE if this object doesn't
;~  BOOL accessibleText;                 // implement the additional interface
;~  BOOL accessibleInterfaces;           // new bitfield containing additional
;~                                       //   interface flags
;~ };
;~     typedef struct AccessibleContextInfoTag {
;~         wchar_t name[MAX_STRING_SIZE];          // the AccessibleName of the object
;~         wchar_t description[MAX_STRING_SIZE];   // the AccessibleDescription of the object

;~         wchar_t role[SHORT_STRING_SIZE];        // localized AccesibleRole string
;~         wchar_t role_en_US[SHORT_STRING_SIZE];  // AccesibleRole string in the en_US locale
;~         wchar_t states[SHORT_STRING_SIZE];      // localized AccesibleStateSet string (comma separated)
;~         wchar_t states_en_US[SHORT_STRING_SIZE]; // AccesibleStateSet string in the en_US locale (comma separated)

;~         jint indexInParent;                     // index of object in parent
;~         jint childrenCount;                     // # of children, if any

;~         jint x;                                 // screen coords in pixels
;~         jint y;                                 // "
;~         jint width;                             // pixel width of object
;~         jint height;                            // pixel height of object

;~         BOOL accessibleComponent;               // flags for various additional
;~         BOOL accessibleAction;                  //  Java Accessibility interfaces
;~         BOOL accessibleSelection;               //  FALSE if this object doesn't
;~         BOOL accessibleText;                    //  implement the additional interface
;~                                                 //  in question

;~         // BOOL accessibleValue;                // old BOOL indicating whether AccessibleValue is supported
;~         BOOL accessibleInterfaces;              // new bitfield containing additional interface flags

;~     } AccessibleContextInfo;
;~ Local $AccessibleContextInfo=DllStructCreate("WCHAR name[1024];WCHAR description[1024];WCHAR role[256];WCHAR role_en_US[256];WCHAR states[256];WCHAR states_en_US[256];int indexInParent;int childrenCount;int x;int y;int width;int height;BOOL accessibleComponent;BOOL accessibleAction;BOOL accessibleSelection;BOOL accessibleText;BOOL accessibleInterfaces")
const $cTypeStructAccessibleContextInfo="WCHAR name[1024];WCHAR description[1024];WCHAR role[256];WCHAR role_en_US[256];WCHAR states[256];WCHAR states_en_US[256];INT indexInParent;INT childrenCount;INT x;INT y;INT width;INT height;BOOL accessibleComponent;BOOL accessibleAction;BOOL accessibleSelection;BOOL accessibleText;BOOL accessibleInterfaces"
;~ const $cTypeStructAccessibleContextInfo="WCHAR data[8192]"

Local $AccessibleContextInfo=DllStructCreate($cTypeStructAccessibleContextInfo)
 ; If an error occurred display the error code and return False.
If @error Then
    consolewrite("Error in DllStructCreate $AccessibleContextInfo, Code: " & @error)
    exit
EndIf
consolewrite("We will call contextinfo with Struct Size: " & DllStructGetSize($AccessibleContextInfo) & @CRLF)

;~ Local $AccessibleContextInfo=DllStructCreate("WCHAR name[1024];WCHAR description[1024];WCHAR role[256];WCHAR states[256];int64 indexInParent;int64 childrenCount;int64 x;int64 y;int64 width;int64 height;byte accessibleComponent;byte accessibleAction;byte accessibleSelection;byte accessibleText;byte accessibleinterfaces")


;~ #AutoIt3Wrapper_UseX64=Y

#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <constants.au3>
#include <WinAPI.au3>
#include <debug.au3>

local $hwnd
local $i
local $result
local $vmID
local $ac

;~ Make messages elevated
_ChangeWindowMessageFilter($WM_COPYDATA,1)
for $i=$WM_USER to $WM_USER+65536
    _ChangeWindowMessageFilter($i, 1)
Next

Func _ChangeWindowMessageFilter($iMsg, $iAction)
    Local $aCall = DllCall("user32.dll", "bool", "ChangeWindowMessageFilter", "dword", $iMsg, "dword", $iAction)
    If @error Or Not $aCall[0] Then Return SetError(1, 0, 0)
    Return 1
EndFunc

local $bridgeDLL=DLLOPEN($AccessBridgeDLL)
;~ sleep(500)
If $bridgeDLL=true Then
    consolewrite("Windows accessbridge 32 opened " & $bridgeDLL & @CRLF)
Else
    consolewrite("DLL not found try to change to WindowsAccessBridge-64.dll if you use 64 bits autoit and windows")
endIf

;~ TODO: Handle messages received from initialize

$result =dllcall($bridgeDLL,"NONE", "Windows_run")
consolewrite($result & " " & @error & " initializeAccessBridge is finished")

sleep(250)

consolewrite("Windows_run passed :" & $result & @CRLF)

Local $var = WinList()
consolewrite("Before loading all Windows:" & $var[0][0] & @CRLF)

For $i = 1 To $var[0][0]
; Only display visble windows that have a title
    If IsVisible($var[$i][1]) Then
        local $handle=wingethandle($var[$i][0])
        $result =dllcall($bridgeDLL,"BOOL:cdecl", "isJavaWindow", "hwnd", $handle)
        if @error=0 Then
            if $result[0]=1 Then
                consolewrite( "JAVA window found <" & $i & "> Java Window Title=" & $var[$i][0] &  " Handle=" & $var[$i][1] & @TAB & " res: " &  $result[0] & @CRLF)

            local $acParent=0
            local $vmID=0

;~AccessibilityContext (acParent above), which I had incorrectly mapped as an IntPtr, is actually an Int32 when using the "legacy" WindowsAccessBridge.dll library (used under x86),
;~and an Int64 when using the WOW64 WindowsAccessBridge-32.dll library.
;~WindowsAccessBridge-32.dll bool getAccessibleContextFromHWND(IntPtr hwnd, out Int32 vmID, out Int64 acParent);
;~WindowsAccessBridge.dll    bool _getAccessibleContextFromHWND(IntPtr hwnd, out Int32 vmID, out Int32 acParent);
            $result =dllcall($bridgeDLL,"BOOL:cdecl", "getAccessibleContextFromHWND", "hwnd", $handle, "long*", $vmID, "INT64*", $acParent)
;~          $result =dllcall($bridgeDLL,"BOOL:cdecl", "getAccessibleContextFromHWND", "hwnd", $handle, "long*", $vmID, "ptr*", $acParent)
;~          $result =dllcall($bridgeDLL,"BOOL:cdecl", "getAccessibleContextFromHWND", "hwnd", $handle, "long*", $vmID, "INT64*", $acParent)


                if @error=0 Then
                    consolewrite("Result getAccessibleContextFromHWND is <" & $result & "> ubound: " & ubound($result) & @crlf)
                    $vmID=$result[2]
                    $acParent=$result[3]
                    consolewrite("  We have a VMid " & $vmID & " ac " & $acParent & @CRLF)


                    if @error > 0 then consolewrite("Struct error")

;~                  $result =dllcall($bridgeDLL, "BOOL:cdecl", "getVersionInfo", "long", $vmId, "struct", $AccessBridgeVersionInfo)
;~   typedef BOOL (*GetVersionInfoFP) (long vmID, AccessBridgeVersionInfo *info);
                    $result =dllcall($bridgeDLL, "BOOL:cdecl", "getVersionInfo", "long", $vmId, "struct*", DllStructGetPtr($AccessBridgeVersionInfo))

                    if @error=0 Then
                        $s1=dllstructgetdata($AccessBridgeVersionInfo, "VMVersion")
                        $s2=dllstructgetdata($AccessBridgeVersionInfo, "bridgeJavaClassVersion")
                        $s3=dllstructgetdata($AccessBridgeVersionInfo, "bridgeJavaDLLVersion")
                        $s4=dllstructgetdata($AccessBridgeVersionInfo, "bridgeWinDLLVersion")

                        consolewrite("Call version info: PASSED VMID found : " & $vmID & @CRLF)
                        consolewrite("  VMVersion: <" & $s1 & ">" & @CRLF)
                        consolewrite("  bridgeJavaClassVersion: <" & $s2 & ">" & @CRLF)
                        consolewrite("  bridgeJavaDLLVersion: <" & $s3 & ">" & @CRLF)
                        consolewrite("  bridgeWinDLLVersion: <" & $s4 & ">" & @CRLF)
                    Else
                        consolewrite("getVersionInfo error:" & @error  & @CRLF)
                    EndIf

;~ /**
;~      * Returns the Accessible Context for the top level object in
;~      * a Java Window.  This is same Accessible Context that is obtained
;~      * from GetAccessibleContextFromHWND for that window.  Returns
;~      * (AccessibleContext)0 on error.
;~      */
;~     AccessibleContext getTopLevelObject (const long vmID, const AccessibleContext accessibleContext);
;~ typedef AccessibleContext (*getTopLevelObjectFP) (const long vmID, const AccessibleContext ac);
local $topAC
$result=dllcall($bridgeDLL, "BOOL:cdecl", "getTopLevelObject", "long", $vmId, "INT64", $topAC)
if @error=0 Then
    consolewrite("getTopLevelObject Result is <" & $result & "> ubound: " & ubound($result) & @crlf)
    $vmID=$result[1]
    $topAC=$result[2]
    consolewrite("We have a VMid " & $vmID & " toplevel object ac " & $topAC & @CRLF)
Else
    consolewrite("getTopLevelObjecterror:" & @error  & @CRLF)
endif



consolewrite("BEFORE BREAKING" & @CRLF)
                    consolewrite("  We still have a VMid " & $vmID & " acParent " & $acParent & @CRLF)
;~ Retrieves an AccessibleContextInfo object of the AccessibleContext object ac.
;~                  $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "ptr", $ac, "ptr", DllStructGetPtr($AccessibleContextInfo))
;~ typedef BOOL (*GetAccessibleContextInfoFP) (long vmID, AccessibleContext ac, AccessibleContextInfo *info);

;~                  $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","INT64", $vmId, "int", $ac, "ptr*", DllStructGetPtr($AccessibleContextInfo))
;~                  $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "long", $ac, "ptr", DllStructGetPtr($AccessibleContextInfo),DllStructGetSize($AccessibleContextInfo))
                    $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "INT64", $acParent, "struct*", DllStructGetPtr($AccessibleContextInfo))
                    if @error<>0 Then
                        consolewrite("We have an error " & @error & @CRLF)
                    EndIf
;~ We should pass it by reference
;~ Exception code c0000005 is an access violation, also known as general protection fault. The program is reading from, or writing to, an address which is not part of the virtual address space.
;~ A very common cause is that you are de-referencing a stale pointer. In other words, the pointer was once valid, but you have subsequently freed it. Then later when you attempt to access it,
;~ an exception is raised.

;~ The problem is don't use out parameter. Use the IntPtr.
;~ So it should be getAccessibleContextInfo(Int32 vmID, IntPtr accessibleContext, IntPtr acInfo)

consolewrite("AFTER BREAKING" & @CRLF)

                    consolewrite("Result is <" & $result & "> ubound: " & ubound($result) & @crlf)
$AccessibleContextInfo=$result[3]
                    $s1=dllstructgetdata($AccessibleContextInfo, "name")
                    $s2=dllstructgetdata($AccessibleContextInfo, "description")
                    $s3=dllstructgetdata($AccessibleContextInfo, "role")
                    $s4=dllstructgetdata($AccessibleContextInfo, "role_en_US")
                    $s5=dllstructgetdata($AccessibleContextInfo, "states")
                    $s6=dllstructgetdata($AccessibleContextInfo, "states_en_US")
                    $s7=dllstructgetdata($AccessibleContextInfo, "indexInParent")
                    $s8=dllstructgetdata($AccessibleContextInfo, "childrenCount")
                    $s9=dllstructgetdata($AccessibleContextInfo, "x")
                    $s10=dllstructgetdata($AccessibleContextInfo, "y")
                    $s11=dllstructgetdata($AccessibleContextInfo, "width")
                    $s12=dllstructgetdata($AccessibleContextInfo, "heigth")

                    consolewrite("name: <" & $s1 & ">" & @CRLF)
                    consolewrite("description: <" & $s2 & ">" & @CRLF)
                    consolewrite("role: <" & $s3 & ">" & @CRLF)
                    consolewrite("role_en_US: <" & $s4 & ">" & @CRLF)
                    consolewrite("states: <" & $s5 & ">" & @CRLF)
                    consolewrite("states_en_US: <" & $s6 & ">" & @CRLF)
                    consolewrite("indexInParent: <" & $s7 & ">" & @CRLF)
                    consolewrite("childrenCount: <" & $s8 & ">" & @CRLF)
                    consolewrite("x: <" & $s9 & ">" & @CRLF)
                    consolewrite("y: <" & $s10 & ">" & @CRLF)
                    consolewrite("width: <" & $s11 & ">" & @CRLF)
                    consolewrite("height: <" & $s12 & ">" & @CRLF)

;~ AccessibleContext GetAccessibleChildFromContext(long vmID, AccessibleContext ac, jint index);

;~ Returns an AccessibleContext object that represents the nth child of the object ac, where n is specified by the value index.


                Else
                    consolewrite( @error & " No context found" & @CRLF)
                endif
            EndIf
        Else
        EndIf
    endif
Next
;~ http://www.autohotkey.com/board/topic/95343-how-to-send-to-unseen-controls-in-a-java-app/
local $result =dllcall($bridgeDLL,"BOOL", "shutdownAccessBridge")

Func IsVisible($handle)
    If BitAND(WinGetState($handle), 2) Then
        Return 1
    Else
        Return 0
    EndIf

EndFunc   ;==>IsVisible

[junkew]

yes, now it works (I get x and y and name information from a java window)

and lots of stuff can be rewritten

Can someone explain me the difference between these 2 lines (DllStructGetPtr seems not needed or when used gives me not the expected result)

;~                  $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "INT64", $acParent, "struct*", DllStructGetPtr($AccessibleContextInfo))
                    $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "INT64", $acParent, "struct*", $AccessibleContextInfo)

output of below script

expandcollapse popup

We are using X64 at cpu X64 Autoit 64 bit version 1
We will call contextinfo with Struct Size: 6188
Windows accessbridge 32 opened 1
Windows_run passed :
Before loading all Windows:569
JAVA window found <62> Java Window Title=SwingSet2 Handle=0x000000000017068A  res: 1
Result getAccessibleContextFromHWND is <> ubound: 4
  We have a VMid 1969346 ac 1539741304
Call version info: PASSED VMID found : 1969346
  VMVersion: <1.8.0_25>
  bridgeJavaClassVersion: <1.8.0_25>
  bridgeJavaDLLVersion: <1.8.0_25>
  bridgeWinDLLVersion: <1.8.0_25>
 *** getTopLevelObject Result is <0> ubound: 3
We have a VMid 1969346 toplevel object ac 0
BEFORE BREAKING
  We still have a VMid 1969346 acParent 1539741304
AFTER BREAKING
Result is <> ubound: 4
name: <SwingSet2>
description: <>
role: <frame>
role_en_US: <frame>
states: <enabled,focusable,visible,showing,resizable>
states_en_US: <enabled,focusable,visible,showing,resizable>
indexInParent: <-1>
childrenCount: <1>
x: <1152>
y: <195>
width: <736>
height: <0>
JAVA window found <68> Java Window Title=SwingSet2 Handle=0x0000000000080C7C  res: 1
Result getAccessibleContextFromHWND is <> ubound: 4
  We have a VMid 1969346 ac 1539741232
Call version info: PASSED VMID found : 1969346
  VMVersion: <1.8.0_25>
  bridgeJavaClassVersion: <1.8.0_25>
  bridgeJavaDLLVersion: <1.8.0_25>
  bridgeWinDLLVersion: <1.8.0_25>
 *** getTopLevelObject Result is <0> ubound: 3
We have a VMid 1969346 toplevel object ac 0
BEFORE BREAKING
  We still have a VMid 1969346 acParent 1539741232
AFTER BREAKING
Result is <> ubound: 4
name: <SwingSet2>
description: <>
role: <frame>
role_en_US: <frame>
states: <enabled,focusable,visible,showing,resizable>
states_en_US: <enabled,focusable,visible,showing,resizable>
indexInParent: <-1>
childrenCount: <1>
x: <1152>
y: <195>
width: <736>
height: <0>
 

and for completeness the whole testing source

expandcollapse popup

#AutoIt3Wrapper_UseX64=Y
consolewrite("We are using " & @OSArch & " at cpu " & @CPUArch & " Autoit 64 bit version "& @AutoItX64 & @CRLF)
if @AutoItX64 = 1 then
    local $accessBridgeDLL="WindowsAccessBridge-64.dll"
Else
    local $accessBridgeDLL="WindowsAccessBridge-32.dll"
EndIf


;~ By inspecting the WindowsAccessBridge-32.dll it reveals some information about the hidden dialogs
;~   So it seems the hidden dialog is shown after you call windows_run() no clue if interaction is needed
;~
;~ Somehow it sends a message unclear if this is to the JVM to respond to
;~   push   SSZ6E73E320_AccessBridge_FromJava_Hello
;~   push   SSZ6E73E300_AccessBridge_FromWindows_Hello
;~   db 'AccessBridge-FromWindows-Hello',0
;~   db 'AccessBridge-FromJava-Hello',0

;~ JABHANDLER.PY is a usefull source to study on internet

;~ *******
;~ Every AccessibleContext IntPtr must be replaced by long, including but not limited to getAccessibleContextFromHWND, getAccessibleParentFromContext, getAccessibleChildFromContext, getAccessibleTextInf
;~ JOBJECT64 is defined as jlong on 64-bit systems and jobject on legacy versions of Java Access Bridge. For definitions, see the section ACCESSBRIDGE_ARCH_LEGACY in the AccessBridgePackages.h header file.
;~ #ifdef ACCESSBRIDGE_ARCH_LEGACY
;~ typedef jobject JOBJECT64;
;~ typedef HWND ABHWND64;
;~ #define ABHandleToLong
;~ #define ABLongToHandle
;~ #else
;~ typedef jlong JOBJECT64;
;~ typedef long ABHWND64;
;~ #define ABHandleToLong HandleToLong
;~ #define ABLongToHandle LongToHandle
;~ #endif
;~ a jobject is a 32 bit pointer for 32 bit builds
;~ 'bool' is a built-in C++ type while 'BOOL' is a Microsoft specific type that is defined as an 'int'. You can find it in 'windef.h'
;~ *******

;~ #Appropriately set the return and argument types of all the access bridge dll functions
;~ if bridgeDll:
;~  _fixBridgeFunc(None,'Windows_run')
;~  _fixBridgeFunc(None,'setFocusGainedFP',c_void_p)
;~  _fixBridgeFunc(None,'setPropertyStateChangeFP',c_void_p)
;~  _fixBridgeFunc(None,'setPropertyCaretChangeFP',c_void_p)
;~  _fixBridgeFunc(None,'setPropertyActiveDescendentChangeFP',c_void_p)
;~  _fixBridgeFunc(None,'releaseJavaObject',c_long,JOBJECT64)
;~  _fixBridgeFunc(BOOL,'getVersionInfo',POINTER(AccessBridgeVersionInfo),errcheck=True)
;~  _fixBridgeFunc(BOOL,'isJavaWindow',HWND)
;~  _fixBridgeFunc(BOOL,'isSameObject',c_long,JOBJECT64,JOBJECT64)
;~  _fixBridgeFunc(BOOL,'getAccessibleContextFromHWND',HWND,POINTER(c_long),POINTER(JOBJECT64),errcheck=True)
;~  _fixBridgeFunc(HWND,'getHWNDFromAccessibleContext',c_long,JOBJECT64,errcheck=True)
;~  _fixBridgeFunc(BOOL,'getAccessibleContextAt',c_long,JOBJECT64,jint,jint,POINTER(JOBJECT64),errcheck=True)
;~  _fixBridgeFunc(BOOL,'getAccessibleContextWithFocus',HWND,POINTER(c_long),POINTER(JOBJECT64),errcheck=True)
;~  _fixBridgeFunc(BOOL,'getAccessibleContextInfo',c_long,JOBJECT64,POINTER(AccessibleContextInfo),errcheck=True)
;~  _fixBridgeFunc(JOBJECT64,'getAccessibleChildFromContext',c_long,JOBJECT64,jint,errcheck=True)
;~  _fixBridgeFunc(JOBJECT64,'getAccessibleParentFromContext',c_long,JOBJECT64)
;~  _fixBridgeFunc(BOOL,'getAccessibleRelationSet',c_long,JOBJECT64,POINTER(AccessibleRelationSetInfo),errcheck=True)
;~  _fixBridgeFunc(BOOL,'getAccessibleTextInfo',c_long,JOBJECT64,POINTER(AccessibleTextInfo),jint,jint,errcheck=True)
;~  _fixBridgeFunc(BOOL,'getAccessibleTextItems',c_long,JOBJECT64,POINTER(AccessibleTextItemsInfo),jint,errcheck=True)
;~  _fixBridgeFunc(BOOL,'getAccessibleTextSelectionInfo',c_long,JOBJECT64,POINTER(AccessibleTextSelectionInfo),errcheck=True)
;~  _fixBridgeFunc(BOOL,'getAccessibleTextAttributes',c_long,JOBJECT64,jint,POINTER(AccessibleTextAttributesInfo),errcheck=True)
;~  _fixBridgeFunc(BOOL,'getAccessibleTextLineBounds',c_long,JOBJECT64,jint,POINTER(jint),POINTER(jint),errcheck=True)
;~  _fixBridgeFunc(BOOL,'getAccessibleTextRange',c_long,JOBJECT64,jint,jint,POINTER(c_wchar),c_short,errcheck=True)
;~  _fixBridgeFunc(BOOL,'getCurrentAccessibleValueFromContext',c_long,JOBJECT64,POINTER(c_wchar),c_short,errcheck=True)
;~  _fixBridgeFunc(BOOL,'selectTextRange',c_long,JOBJECT64,c_int,c_int,errcheck=True)
;~  _fixBridgeFunc(BOOL,'getTextAttributesInRange',c_long,JOBJECT64,c_int,c_int,POINTER(AccessibleTextAttributesInfo),POINTER(c_short),errcheck=True)
;~  _fixBridgeFunc(JOBJECT64,'getTopLevelObject',c_long,JOBJECT64,errcheck=True)
;~  _fixBridgeFunc(c_int,'getObjectDepth',c_long,JOBJECT64)
;~  _fixBridgeFunc(JOBJECT64,'getActiveDescendent',c_long,JOBJECT64)
;~  _fixBridgeFunc(BOOL,'requestFocus',c_long,JOBJECT64,errcheck=True)
;~  _fixBridgeFunc(BOOL,'setCaretPosition',c_long,JOBJECT64,c_int,errcheck=True)
;~  _fixBridgeFunc(BOOL,'getCaretLocation',c_long,JOBJECT64,POINTER(AccessibleTextRectInfo),jint,errcheck=True)
;~  _fixBridgeFunc(BOOL,'getAccessibleActions',c_long,JOBJECT64,POINTER(AccessibleActions),errcheck=True)
;~  _fixBridgeFunc(BOOL,'doAccessibleActions',c_long,JOBJECT64,POINTER(AccessibleActionsToDo),POINTER(jint),errcheck=True)


;~ Primitive Types and Native Equivalents
;~ Java Type    Native Type Description
;~ boolean      jboolean    unsigned 8 bits
;~ byte         jbyte       signed 8 bits
;~ char         jchar       unsigned 16 bits
;~ short        jshort      signed 16 bits
;~ int          jint        signed 32 bits
;~ long         jlong       signed 64 bits
;~ float        jfloat      32 bits
;~ double       jdouble     64 bits
;~ void         void        not applicable

;~ Copied from this NVDA reference and translated to AutoIT
;~ http://www.webbie.org.uk/nvda/api/JABHandler-pysrc.html
;~
;~ def initialize():
;~         global isRunning
;~         if not bridgeDll:
;~                 raise NotImplementedError("dll not available")
;~         bridgeDll.Windows_run()
;~         #Accept wm_copydata and any wm_user messages from other processes even if running with higher privilages
;~***         ChangeWindowMessageFilter=getattr(windll.user32,'ChangeWindowMessageFilter',None)
;~***         if ChangeWindowMessageFilter:
;~***                 if not ChangeWindowMessageFilter(winUser.WM_COPYDATA,1):
;~***                         raise WinError()
;~***                 for msg in xrange(winUser.WM_USER+1,65535):
;~***                         if not ChangeWindowMessageFilter(msg,1):
;~***                                 raise WinError()
;~         #Register java events
;~         bridgeDll.setFocusGainedFP(internal_event_focusGained)
;~         bridgeDll.setPropertyActiveDescendentChangeFP(internal_event_activeDescendantChange)
;~         bridgeDll.setPropertyNameChangeFP(event_nameChange)
;~         bridgeDll.setPropertyDescriptionChangeFP(event_descriptionChange)
;~         bridgeDll.setPropertyValueChangeFP(event_valueChange)
;~         bridgeDll.setPropertyStateChangeFP(internal_event_stateChange)
;~         bridgeDll.setPropertyCaretChangeFP(internal_event_caretChange)
;~         isRunning=True

;create the structs
;~
;~  http://docs.oracle.com/javase/accessbridge/2.0.2/api.htm
;~
;~ #define MAX_STRING_SIZE     1024
;~ #define SHORT_STRING_SIZE    256
;~                  struct AccessBridgeVersionInfo {
;~  wchar_t VMversion[SHORT_STRING_SIZE];              // version of the Java VM
;~  wchar_t bridgeJavaClassVersion[SHORT_STRING_SIZE]; // version of the AccessBridge.class
;~  wchar_t bridgeJavaDLLVersion[SHORT_STRING_SIZE];   // version of JavaAccessBridge.dll
;~  wchar_t bridgeWinDLLVersion[SHORT_STRING_SIZE];    // version of WindowsAccessBridge.dll
;~ };

;~ Local $AccessBridgeVersionInfo=DllStructCreate("WCHAR VMversion[256];WCHAR bridgeJavaClassVersion[256];WCHAR bridgeJavaDLLVersion[256];WCHAR bridgeWinDLLVersion[256]")
Local $AccessBridgeVersionInfo=DllStructCreate("WCHAR VMversion[256];WCHAR bridgeJavaClassVersion[256];WCHAR bridgeJavaDLLVersion[256];WCHAR bridgeWinDLLVersion[256]")

;~ struct AccessibleContextInfo {
;~  wchar_ name[MAX_STRING_SIZE];        // the AccessibleName of the object
;~  wchar_ description[MAX_STRING_SIZE]; // the AccessibleDescription of the object
;~  wchar_ role[SHORT_STRING_SIZE];      // localized AccesibleRole string
;~  wchar_ states[SHORT_STRING_SIZE];    // localized AccesibleStateSet string
;~                                       //   (comma separated)
;~  jint indexInParent                   // index of object in parent
;~  jint childrenCount                   // # of children, if any
;~  jint x;                              // screen x-axis co-ordinate in pixels
;~  jint y;                              // screen y-axis co-ordinate in pixels
;~  jint width;                          // pixel width of object
;~  jint height;                         // pixel height of object
;~  BOOL accessibleComponent;            // flags for various additional
;~  BOOL accessibleAction;               // Java Accessibility interfaces
;~  BOOL accessibleSelection;            // FALSE if this object doesn't
;~  BOOL accessibleText;                 // implement the additional interface
;~  BOOL accessibleInterfaces;           // new bitfield containing additional
;~                                       //   interface flags
;~ };
;~     typedef struct AccessibleContextInfoTag {
;~         wchar_t name[MAX_STRING_SIZE];          // the AccessibleName of the object
;~         wchar_t description[MAX_STRING_SIZE];   // the AccessibleDescription of the object

;~         wchar_t role[SHORT_STRING_SIZE];        // localized AccesibleRole string
;~         wchar_t role_en_US[SHORT_STRING_SIZE];  // AccesibleRole string in the en_US locale
;~         wchar_t states[SHORT_STRING_SIZE];      // localized AccesibleStateSet string (comma separated)
;~         wchar_t states_en_US[SHORT_STRING_SIZE]; // AccesibleStateSet string in the en_US locale (comma separated)

;~         jint indexInParent;                     // index of object in parent
;~         jint childrenCount;                     // # of children, if any

;~         jint x;                                 // screen coords in pixels
;~         jint y;                                 // "
;~         jint width;                             // pixel width of object
;~         jint height;                            // pixel height of object

;~         BOOL accessibleComponent;               // flags for various additional
;~         BOOL accessibleAction;                  //  Java Accessibility interfaces
;~         BOOL accessibleSelection;               //  FALSE if this object doesn't
;~         BOOL accessibleText;                    //  implement the additional interface
;~                                                 //  in question

;~         // BOOL accessibleValue;                // old BOOL indicating whether AccessibleValue is supported
;~         BOOL accessibleInterfaces;              // new bitfield containing additional interface flags

;~     } AccessibleContextInfo;
;~ Local $AccessibleContextInfo=DllStructCreate("WCHAR name[1024];WCHAR description[1024];WCHAR role[256];WCHAR role_en_US[256];WCHAR states[256];WCHAR states_en_US[256];int indexInParent;int childrenCount;int x;int y;int width;int height;BOOL accessibleComponent;BOOL accessibleAction;BOOL accessibleSelection;BOOL accessibleText;BOOL accessibleInterfaces")
const $cTypeStructAccessibleContextInfo="WCHAR name[1024];WCHAR description[1024];WCHAR role[256];WCHAR role_en_US[256];WCHAR states[256];WCHAR states_en_US[256];INT indexInParent;INT childrenCount;INT x;INT y;INT width;INT height;BOOL accessibleComponent;BOOL accessibleAction;BOOL accessibleSelection;BOOL accessibleText;BOOL accessibleInterfaces"
;~ const $cTypeStructAccessibleContextInfo="WCHAR data[8192]"

Local $AccessibleContextInfo=DllStructCreate($cTypeStructAccessibleContextInfo)
 ; If an error occurred display the error code and return False.
If @error Then
    consolewrite("Error in DllStructCreate $AccessibleContextInfo, Code: " & @error)
    exit
EndIf
consolewrite("We will call contextinfo with Struct Size: " & DllStructGetSize($AccessibleContextInfo) & @CRLF)

;~ Local $AccessibleContextInfo=DllStructCreate("WCHAR name[1024];WCHAR description[1024];WCHAR role[256];WCHAR states[256];int64 indexInParent;int64 childrenCount;int64 x;int64 y;int64 width;int64 height;byte accessibleComponent;byte accessibleAction;byte accessibleSelection;byte accessibleText;byte accessibleinterfaces")


;~ #AutoIt3Wrapper_UseX64=Y

#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <constants.au3>
#include <WinAPI.au3>
#include <debug.au3>

local $hwnd
local $i
local $result
local $vmID
local $ac

;~ Make messages elevated
_ChangeWindowMessageFilter($WM_COPYDATA,1)
for $i=$WM_USER to $WM_USER+65536
    _ChangeWindowMessageFilter($i, 1)
Next

Func _ChangeWindowMessageFilter($iMsg, $iAction)
    Local $aCall = DllCall("user32.dll", "bool", "ChangeWindowMessageFilter", "dword", $iMsg, "dword", $iAction)
    If @error Or Not $aCall[0] Then Return SetError(1, 0, 0)
    Return 1
EndFunc

local $bridgeDLL=DLLOPEN($AccessBridgeDLL)
;~ sleep(500)
If $bridgeDLL=true Then
    consolewrite("Windows accessbridge 32 opened " & $bridgeDLL & @CRLF)
Else
    consolewrite("DLL not found try to change to WindowsAccessBridge-64.dll if you use 64 bits autoit and windows")
endIf

;~ TODO: Handle messages received from initialize

$result =dllcall($bridgeDLL,"NONE", "Windows_run")
consolewrite($result & " " & @error & " initializeAccessBridge is finished")

sleep(250)

consolewrite("Windows_run passed :" & $result & @CRLF)

Local $var = WinList()
consolewrite("Before loading all Windows:" & $var[0][0] & @CRLF)

For $i = 1 To $var[0][0]
; Only display visble windows that have a title
    If IsVisible($var[$i][1]) Then
        local $handle=wingethandle($var[$i][0])
        $result =dllcall($bridgeDLL,"BOOL:cdecl", "isJavaWindow", "hwnd", $handle)
        if @error=0 Then
            if $result[0]=1 Then
                consolewrite( "JAVA window found <" & $i & "> Java Window Title=" & $var[$i][0] &  " Handle=" & $var[$i][1] & @TAB & " res: " &  $result[0] & @CRLF)

            local $acParent=0
            local $vmID=0

;~AccessibilityContext (acParent above), which I had incorrectly mapped as an IntPtr, is actually an Int32 when using the "legacy" WindowsAccessBridge.dll library (used under x86),
;~and an Int64 when using the WOW64 WindowsAccessBridge-32.dll library.
;~WindowsAccessBridge-32.dll bool getAccessibleContextFromHWND(IntPtr hwnd, out Int32 vmID, out Int64 acParent);
;~WindowsAccessBridge.dll    bool _getAccessibleContextFromHWND(IntPtr hwnd, out Int32 vmID, out Int32 acParent);
;~  _fixBridgeFunc(BOOL,'getAccessibleContextFromHWND',HWND,POINTER(c_long),POINTER(JOBJECT64),errcheck=True)
            $result =dllcall($bridgeDLL,"BOOL:cdecl", "getAccessibleContextFromHWND", "hwnd", $handle, "long*", $vmID, "UINT64*", $acParent)
;~          $result =dllcall($bridgeDLL,"BOOL:cdecl", "getAccessibleContextFromHWND", "hwnd", $handle, "long*", $vmID, "ptr*", $acParent)
;~          $result =dllcall($bridgeDLL,"BOOL:cdecl", "getAccessibleContextFromHWND", "hwnd", $handle, "long*", $vmID, "INT64*", $acParent)


                if @error=0 Then
                    consolewrite("Result getAccessibleContextFromHWND is <" & $result & "> ubound: " & ubound($result) & @crlf)
                    $vmID=$result[2]
                    $acParent=$result[3]
                    consolewrite("  We have a VMid " & $vmID & " ac " & $acParent & @CRLF)


                    if @error > 0 then consolewrite("Struct error")

;~                  $result =dllcall($bridgeDLL, "BOOL:cdecl", "getVersionInfo", "long", $vmId, "struct", $AccessBridgeVersionInfo)
;~   typedef BOOL (*GetVersionInfoFP) (long vmID, AccessBridgeVersionInfo *info);
                    $result =dllcall($bridgeDLL, "BOOL:cdecl", "getVersionInfo", "long", $vmId, "struct*", DllStructGetPtr($AccessBridgeVersionInfo))

                    if @error=0 Then
                        $s1=dllstructgetdata($AccessBridgeVersionInfo, "VMVersion")
                        $s2=dllstructgetdata($AccessBridgeVersionInfo, "bridgeJavaClassVersion")
                        $s3=dllstructgetdata($AccessBridgeVersionInfo, "bridgeJavaDLLVersion")
                        $s4=dllstructgetdata($AccessBridgeVersionInfo, "bridgeWinDLLVersion")

                        consolewrite("Call version info: PASSED VMID found : " & $vmID & @CRLF)
                        consolewrite("  VMVersion: <" & $s1 & ">" & @CRLF)
                        consolewrite("  bridgeJavaClassVersion: <" & $s2 & ">" & @CRLF)
                        consolewrite("  bridgeJavaDLLVersion: <" & $s3 & ">" & @CRLF)
                        consolewrite("  bridgeWinDLLVersion: <" & $s4 & ">" & @CRLF)
                    Else
                        consolewrite("getVersionInfo error:" & @error  & @CRLF)
                    EndIf

;~ /**
;~      * Returns the Accessible Context for the top level object in
;~      * a Java Window.  This is same Accessible Context that is obtained
;~      * from GetAccessibleContextFromHWND for that window.  Returns
;~      * (AccessibleContext)0 on error.
;~      */
;~     AccessibleContext getTopLevelObject (const long vmID, const AccessibleContext accessibleContext);
;~ typedef AccessibleContext (*getTopLevelObjectFP) (const long vmID, const AccessibleContext ac);
;~  _fixBridgeFunc(JOBJECT64,'getTopLevelObject',c_long,JOBJECT64,errcheck=True)
local $topAC
$result=dllcall($bridgeDLL, "UINT64:cdecl", "getTopLevelObject", "long", $vmId, "UINT64", $topAC)
if @error=0 Then
    consolewrite(" *** getTopLevelObject Result is <" & $result[0] & "> ubound: " & ubound($result) & @crlf)
    $vmID=$result[1]
    $topAC=$result[2]
    consolewrite("We have a VMid " & $vmID & " toplevel object ac " & $topAC & @CRLF)
Else
    consolewrite("getTopLevelObjecterror:" & @error  & @CRLF)
endif



consolewrite("BEFORE BREAKING" & @CRLF)
                    consolewrite("  We still have a VMid " & $vmID & " acParent " & $acParent & @CRLF)
;~ Retrieves an AccessibleContextInfo object of the AccessibleContext object ac.
;~                  $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "ptr", $ac, "ptr", DllStructGetPtr($AccessibleContextInfo))
;~ typedef BOOL (*GetAccessibleContextInfoFP) (long vmID, AccessibleContext ac, AccessibleContextInfo *info);
;~  _fixBridgeFunc(BOOL,'getAccessibleContextInfo',c_long,JOBJECT64,POINTER(AccessibleContextInfo),errcheck=True)

;~                  $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","INT64", $vmId, "int", $ac, "ptr*", DllStructGetPtr($AccessibleContextInfo))
;~                  $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "long", $ac, "ptr", DllStructGetPtr($AccessibleContextInfo),DllStructGetSize($AccessibleContextInfo))
;~                  $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "INT64", $acParent, "struct*", DllStructGetPtr($AccessibleContextInfo))
                    $result =dllcall($bridgeDLL, "BOOL:cdecl", "getAccessibleContextInfo","long", $vmId, "INT64", $acParent, "struct*", $AccessibleContextInfo)

                    if @error<>0 Then
                        consolewrite("We have an error " & @error & @CRLF)
                    EndIf
;~ We should pass it by reference
;~ Exception code c0000005 is an access violation, also known as general protection fault. The program is reading from, or writing to, an address which is not part of the virtual address space.
;~ A very common cause is that you are de-referencing a stale pointer. In other words, the pointer was once valid, but you have subsequently freed it. Then later when you attempt to access it,
;~ an exception is raised.

;~ The problem is don't use out parameter. Use the IntPtr.
;~ So it should be getAccessibleContextInfo(Int32 vmID, IntPtr accessibleContext, IntPtr acInfo)

consolewrite("AFTER BREAKING" & @CRLF)

                    consolewrite("Result is <" & $result & "> ubound: " & ubound($result) & @crlf)
$AccessibleContextInfo=$result[3]
                    $s1=dllstructgetdata($AccessibleContextInfo, "name")
                    $s2=dllstructgetdata($AccessibleContextInfo, "description")
                    $s3=dllstructgetdata($AccessibleContextInfo, "role")
                    $s4=dllstructgetdata($AccessibleContextInfo, "role_en_US")
                    $s5=dllstructgetdata($AccessibleContextInfo, "states")
                    $s6=dllstructgetdata($AccessibleContextInfo, "states_en_US")
                    $s7=dllstructgetdata($AccessibleContextInfo, "indexInParent")
                    $s8=dllstructgetdata($AccessibleContextInfo, "childrenCount")
                    $s9=dllstructgetdata($AccessibleContextInfo, "x")
                    $s10=dllstructgetdata($AccessibleContextInfo, "y")
                    $s11=dllstructgetdata($AccessibleContextInfo, "width")
                    $s12=dllstructgetdata($AccessibleContextInfo, "heigth")

                    consolewrite("name: <" & $s1 & ">" & @CRLF)
                    consolewrite("description: <" & $s2 & ">" & @CRLF)
                    consolewrite("role: <" & $s3 & ">" & @CRLF)
                    consolewrite("role_en_US: <" & $s4 & ">" & @CRLF)
                    consolewrite("states: <" & $s5 & ">" & @CRLF)
                    consolewrite("states_en_US: <" & $s6 & ">" & @CRLF)
                    consolewrite("indexInParent: <" & $s7 & ">" & @CRLF)
                    consolewrite("childrenCount: <" & $s8 & ">" & @CRLF)
                    consolewrite("x: <" & $s9 & ">" & @CRLF)
                    consolewrite("y: <" & $s10 & ">" & @CRLF)
                    consolewrite("width: <" & $s11 & ">" & @CRLF)
                    consolewrite("height: <" & $s12 & ">" & @CRLF)

;~ AccessibleContext GetAccessibleChildFromContext(long vmID, AccessibleContext ac, jint index);

;~ Returns an AccessibleContext object that represents the nth child of the object ac, where n is specified by the value index.


                Else
                    consolewrite( @error & " No context found" & @CRLF)
                endif
            EndIf
        Else
        EndIf
    endif
Next
;~ http://www.autohotkey.com/board/topic/95343-how-to-send-to-unseen-controls-in-a-java-app/
local $result =dllcall($bridgeDLL,"BOOL", "shutdownAccessBridge")

Func IsVisible($handle)
    If BitAND(WinGetState($handle), 2) Then
        Return 1
    Else
        Return 0
    EndIf

EndFunc   ;==>IsVisible

Edited January 3, 2015 by junkew

[LarsJ]

You use either

DllCall( ... "struct*", $AccessibleContextInfo )

or

DllCall( ... "ptr", DllStructGetPtr($AccessibleContextInfo) )

"struct*" tells DllCall to get the pointer by its internal code. I think the "struct*" method is the preferred method. It's shorter and easier to read. And probably also a little bit faster, because you don't need DllStructGetPtr.

[junkew]

Tried many combinations of int*, struct*. to see what works and what not learning a lot but still not working

continued in general help and support

'?do=embed' frameborder='0' data-embedContent>>

[pd]

Hi experts,

I also tried with Java Access Bridge but seems like it is not able to read control IDs from Java AWT application. Any help is appreciated.

Thanks,

PD

[pd]

Hi All,

Does this UDF work with AWT application like SQL Devloper? I tried using it but it is not able to identify controls.

Regards,

PD

[TheDcoder]

@pd This UDF was last updated in 2009... Since 2009 there were many script breaking changes, You need to work around the UDF to make it work

 

TD

[ravi2015]

Can you please send one small sample script/UDF to interact awt window?

[sherek]

On 19/01/2009 at 9:10 AM, seangriffin said:

This UDF provides automation support for Java applications.

 

This UDF currently automates standalone applications and browser-based applets, developed in either Swing or the Abstract Window Toolkit (AWT).

 

REQUIREMENTS:

 

• AutoIt3 3.2 or higher
• Java Runtime Environment (JRE) 1.4 or above
• Java Access Bridge for the Microsoft Windows Operating System
• Microsoft Visual C++ 2008 Redistributable
• AutoIT for Java Access dynamically linked library (DLL)
• Tesseract UDF 0.6
• Manual check and update of the Java Access Bridge

INSTALLATION:

To install the latest Java Runtime Environment (JRE):

 

• Open the web page http://java.com/en/download/manual.jsp.
• Click Windows XP/Vista/2000/2003 Offline.
• Save the file to your computer and run it (ie. jre-6u11-windows-i586-p-s.exe).
• Follow the installation instructions.
• Restart your computer when prompted.

To install the Java Access Bridge for the Microsoft Windows Operating System:

 

• Open the web page Java Access Bridge.
• Click Download Java Access Bridge 2.0.1.
• Click I agree to the Software License Agreement.
• Click Continue.
• Click accessbridge-2_0_1.exe.
• Save the file to a location on your computer.
• Run the file accessbridge-2_0_1.exe from your computer.
• Follow the installation instructions.
• Restart your computer when prompted.

To install the Microsoft Visual C++ 2008 Redistributable:

 

• Open the web page http://www.microsoft.com/downloads/details.aspx?FamilyID=9b2da534-3e03-4391-8a4d-074b9f2bc1bf&displaylang=en.
• Click Download.
• Save the file to a location on your computer.
• Run the file vcredist_x86.exe from your computer.
• Follow the installation instructions.

To install the AutoIT for Java Access dynamically linked library (DLL):

 

• Right-click on the link AutoITJavaAccess.dll below.
• Save the file to your Windows System32 folder (ie. C:\WINDOWS\system32).

To install the Tesseract UDF 0.6:

 

• Open the link http://www.autoitscript.com/forum/index.php?showtopic=89542.
• Save the file to your AutoIT include folder (ie. C:\Program Files\AutoIT\Include).

To manually check and update the Java Access Bridge:

 

Note:- This step should be performed in addition to the install of the Java Access Bridge above.

 

• Using Windows Explorer, go to your Java folder C:\Program Files\Java.
• Open the first subfolder (ie. j2re1.4.2_19).
• Go to the subfolder lib\ext.
• If the files access-bridge.jar and jaccess.jar already exist then go to step 9 below.
• Copy the file access-bridge.jar to this subfolder (ie. C:\Program Files\Java\jre1.4.2_19\lib\ext).
• If the subfolder in step 2 above starts with j2re1.2 or jre1.2, then copy the file jaccess-1_2.jar to this subfolder (ie. C:\Program Files\Java\jre1.4.2_19\lib\ext).
• If the subfolder in step 2 above starts with j2re1.3 or jre1.3, then copy the file jaccess-1_3.jar to this subfolder (ie. C:\Program Files\Java\jre1.4.2_19\lib\ext).
• If the subfolder in step 2 above starts with j2re1.4 or jre1.4, then copy the file jaccess-1_4.jar to this subfolder (ie. C:\Program Files\Java\jre1.4.2_19\lib\ext).
• Repeat steps 3 to 8 for every subfolder within the Java folder.

To confirm that the installation process was successful:

 

• Using Windows Explorer, go to C:\Program Files\Java Access Bridge and double-click JavaFerret.exe.
• In the Java Ferret window, select the menu item JavaEvents -> Track Mouse Events.
• Drag your mouse over any Java application, and the Java Ferret window should dynamically update with information about the Java objects as you drag your mouse over them.

LIST OF FUNCTIONS:

 

DEMONSTRATION:

<Under Construction>

 

EXAMPLES:

_JavaObjValueGet.au3

_JavaObjValueSet.au3_JavaObjValueSelect.au3

_JavaObjValuesGet.au3_JavaObjPropertyGet.au3

DOWNLOAD:

 

Latest Version - v0.7 (17/03/09)

AutoITJavaAccess.dllJava.au3

 

I'm trying to get a value from a .jar file (generate a test with netbeans 8.1 and
the javaferret generates the dump), where it is not getting value from a JTextField. 
It is reading the dll normally (returns the value 1 in the DllOpen ("AutoITJavaAccess.dll")). 
I am using windows 7 ultimate, 64 bit. The value of the JTextField comes empty, and the JEdit examples do not work. Some help?

Thank you!

[rlvitorino]

The error is happing in my java.au3

>"C:\Program Files (x86)\AutoIt3\SciTE\..\AutoIt3.exe" "C:\Program Files (x86)\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.au3" /run /prod /ErrorStdOut /in "C:\Users\Digão\Downloads\_JavaObjValueGet (1).au3" /UserParams    
+>16:11:42 Starting AutoIt3Wrapper v.18.708.1148.0 SciTE v.4.1.0.0   Keyboard:00000416  OS:WIN_10/  CPU:X64 OS:X64  Environment(Language:0416)  CodePage:0  utf8.auto.check:4
+>         SciTEDir => C:\Program Files (x86)\AutoIt3\SciTE   UserDir => C:\Users\Digão\AppData\Local\AutoIt v3\SciTE\AutoIt3Wrapper   SCITE_USERHOME => C:\Users\Digão\AppData\Local\AutoIt v3\SciTE 
>Running AU3Check (3.3.14.5)  from:C:\Program Files (x86)\AutoIt3  input:C:\Users\Digão\Downloads\_JavaObjValueGet (1).au3
"C:\Users\Digão\Downloads\Java.au3"(871,41) : warning: $javaferret_obj_role: possibly used before declaration.
			if StringCompare($javaferret_obj_role,
			~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
"C:\Users\Digão\Downloads\Java.au3"(871,41) : error: $javaferret_obj_role: undeclared global variable.
			if StringCompare($javaferret_obj_role,
			~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
C:\Users\Digão\Downloads\_JavaObjValueGet (1).au3 - 1 error(s), 1 warning(s)
!>16:11:43 AU3Check ended. Press F4 to jump to next error.rc:2
+>16:11:43 AutoIt3Wrapper Finished.
>Exit code: 2    Time: 1.566

 

[rlvitorino]

 

I'm trying to automate the java process, although I'm having error below are you helping me?

[BrewManNH]

How are you calling that function? If using a hotkey or an OnEvent trigger, it won't work. Those types of function calls don't use the parameters in the function, so the variables are undeclared.

[rlvitorino]

I just run java.au3 and this error appeared.

Edited April 18, 2019 by rlvitorino

[BrewManNH]

2 minutes ago, rlvitorino said:

I just circled java.au3

I don't know what that means.

[rlvitorino]

I just run java.au3 and this error appeared.

[rlvitorino]

would have a newer version for testing?

[BrewManNH]

 

He kind of explained that you have to create the variable yourself and assign an object to it, if I'm reading it right. Not too sure why it's set p like this though.

[rlvitorino]

 

 

Would not you have the latest version of JAVA UDF this version that is in the link is giving many problems?

[BrewManNH]

I've never used it, so I don't have a copy of it. Have you looked through the whole thread to make sure it hasn't been updated by someone else in it?