Changing all coordinates in a long script to work on different monitor?

[Nick3399]

I have built a rather long script that involves mouse clicks, pixel search etc that all reference the screen coordinates. These coordinates are all over my script. My old computer that I initially built this script on has become super sluggish so I am trying to run it on a new computer and monitor. When I try to run it on new machine, the mouse clicks and pixels are all wrong and I am not sure how to fix any of it without having to repull all of coordinates which would take forever. 
 

also wondering about the pixel search color — not sure if the colors would show exactly the same on the new monitor?

does anyone know how I can go about resolving this? If I change the screen resolution of the new monitor to the old would that do anything or would it still cause issues bc it’s higher quality and larger?

 

EDIT: I just checked and I can’t make the new monitor the same resolution as the old. the old monitor is 1680x1050 and the new is 1600x900

Edited October 4, 2022 by Nick3399

[ternal]

assuming they are hard-coded into your script, I don't think there is an easy solution for your problem. You could try to convert to the window-relative coordinate system if the app window does not take the full screen. if all coordinates just get shifted you can add/subtract integers from them. You could try to calculate the approximate position by scale factor, but at a certain point it will be less work to re-add all the coordinates I think.

if there would be a color difference you can compare the colors. if you convert them to hex you can get the "distance" between 2 colors by representing them in a 3d space where each color is an axis.

$iDifference = sqrt(($R1 - $R2)²+($G1 - $G2)²+($B1 - $B2)²)

I hope this helps.

[Nick3399]

13 hours ago, ternal said:

assuming they are hard-coded into your script, I don't think there is an easy solution for your problem. You could try to convert to the window-relative coordinate system if the app window does not take the full screen. if all coordinates just get shifted you can add/subtract integers from them. You could try to calculate the approximate position by scale factor, but at a certain point it will be less work to re-add all the coordinates I think.

if there would be a color difference you can compare the colors. if you convert them to hex you can get the "distance" between 2 colors by representing them in a 3d space where each color is an axis.

$iDifference = sqrt(($R1 - $R2)²+($G1 - $G2)²+($B1 - $B2)²)

I hope this helps.

So you are correct in that the program window is only a 800x600 and not the entire screen… 

 

if the old screen is 1680x1050 and new screen is 1600x900, but in both the actual program window is 800x600 and location is 100x100 (using winmove) do you know how I would adjust this? Or would I only need to change the program window location since the actual program window is still the same pixel size (but I don’t think this would work bc I’m using screen pixels)? 😫 

 

I’m trying to think through this… if I do:

1600/1680= .95238095

900/1050= .85714286
 

if I change the window location to 95.2380952x85.7142857 would that fix the pixel locations? 😳

or would I do it opposite 

1680/1600

1050/900

 

105x116.6667 as new window location

edit: the above didn’t seem to work as things are still misaligned

Edited October 5, 2022 by Nick3399

[Shark007]

In my opinion, you are going to have to change all of your screen coordinates to be relative to your gui.

I suggest you start familiarizing yourself with

WinGetPos

[ternal]

https://www.autoitscript.com/autoit3/docs/functions/AutoItSetOption.htm   (mousecoordmode)

the au3info tool can also be setup to use windowrelative coordinates. As shark007 suggested, wingetpos can also be used, just remember to pay attention to what system you are working with.

if it window is the same size, then just make sure the window gets placed in the same position like the old setup and everything should still work. If it doesend, depending on the app, check dpi setting and windows scale settings, but I think you would notice from the start that the scaling is off.

if you re-enter all the coordinates, please follow Shark007's advice, this will save you headaches in the future.

 

[Nick3399]

45 minutes ago, ternal said:

https://www.autoitscript.com/autoit3/docs/functions/AutoItSetOption.htm   (mousecoordmode)

the au3info tool can also be setup to use windowrelative coordinates. As shark007 suggested, wingetpos can also be used, just remember to pay attention to what system you are working with.

if it window is the same size, then just make sure the window gets placed in the same position like the old setup and everything should still work. If it doesend, depending on the app, check dpi setting and windows scale settings, but I think you would notice from the start that the scaling is off.

if you re-enter all the coordinates, please follow Shark007's advice, this will save you headaches in the future.

 

Would pixelcordmode work for pixelsearch and pixelgetcolor functions?

[ternal]

https://www.autoitscript.com/autoit3/docs/functions/PixelSearch.htm

"If PixelCoordMode is set to its default SCREEN (1) mode the hwnd parameter is ignored. Only if PixelCoordMode is set to WINDOW (0) or CLIENT (2) will it be honoured."

you can just pass the handle of the window to pixelsearch as well.

 

[Nick3399]

1 hour ago, ternal said:

https://www.autoitscript.com/autoit3/docs/functions/PixelSearch.htm

"If PixelCoordMode is set to its default SCREEN (1) mode the hwnd parameter is ignored. Only if PixelCoordMode is set to WINDOW (0) or CLIENT (2) will it be honoured."

you can just pass the handle of the window to pixelsearch as well.

AutoItSetOption("MouseCoordMode", 2)
AutoItSetOption("PixelCoordMode", 2)
AutoItSetOption("CaretCoordMode", 2)

okay, I think I was able to fix those but now ImageSearch is clicking on the wrong pixel using the client setting

I have the below set at the top

AutoItSetOption("MouseCoordMode", 2)
AutoItSetOption("PixelCoordMode", 2)
AutoItSetOption("CaretCoordMode", 2)

 

and imagesearch as :

        $T22= _ImageSearch("Images\T22.png", 0, $x3, $y3, 40)
    If $T22= 1 Then
            MouseClick("left", $X3 + 10, $y3)

 

 

I guess I could change the window location to 0,0 instead of 100,100 so imagesearch client and screen match, but I would have to redo what I just did though

how can I have imagesearch return coordinates on the client rather than screen? I have checked the documentation but can’t find anything for the normal imagesearch function

Edited October 6, 2022 by Nick3399

[ternal]

3 hours ago, Nick3399 said:

I guess I could change the window location to 0,0 instead of 100,100 so imagesearch client and screen match, but I would have to redo what I just did though

how can I have imagesearch return coordinates on the client rather than screen? I have checked the documentation but can’t find anything for the normal imagesearch function

I personally don't use imagesearch with autoit, I use it in python(faster). I am not smart enough to help you in this tho 😕

as for the coordinates, just add to each coordinate X_offset and Y_offset, and declare X_offset/Y_offset in the beginning of your script, as your lefttop window location changes you can update it with just altering 2 variables, this makes your script more flexible. but I am sorry I cannot help you further with imgsearch

 

[Nick3399]

20 hours ago, Shark007 said:

In my opinion, you are going to have to change all of your screen coordinates to be relative to your gui.

I suggest you start familiarizing yourself with

WinGetPos

 

Hi @Shark007 - would you have any idea how to get imagesearch working with pixelcoordmode to 2 for client? 
 

i’d prefer not to have to go back to each imagesearch line if at all possible as I have hundreds. 

[pixelsearch]

On 10/4/2022 at 9:28 PM, Nick3399 said:

I have built a rather long script that involves mouse clicks, pixel search etc that all reference the screen coordinates.

Just an idea to share. Let's keep the resolutions you indicated :

Old monitor resolution  = 1680 x 1050
New monitor resolution = 1600 x 900

If your program window is 800 x 600 on both monitors and the window is centered, then the upper left corner coords of the program window should be :

Old monitor :

$x_old = (1680 - 800) / 2 = 440
$y_old = (1050 - 600) / 2 = 225

New monitor :

$x_new = (1600 - 800) / 2 = 400
$y_new = (900  - 600) / 2 = 150

Difference :

$x_diff = 440 - 400 = 40
$y_diff = 225 - 150 = 75

Then you could apply these 2 variables ($x_diff and $y_diff) to all your pixel coords in the script and get the same pixelsearch (or mouse click) results inside your program window on both monitors ?

For example, if a pixelsearch was done on the old monitor with these screen coords : 440, 225 (this is the old upper left coord to make it easy) then this same pixelsearch will be done on the new monitor with these screen coords : 440 - $x_diff, 225 - $y_diff

Does this simple approach work for you ?

Edited October 6, 2022 by pixelsearch
typo

[Nick3399]

1 hour ago, pixelsearch said:

Just an idea to share. Let's keep the resolutions you indicated :

Old monitor resolution  = 1680 x 1050
New monitor resolution = 1600 x 900

If your program window is 800 x 600 on both monitors and the window is centered, then the upper left corner coords of the program window should be :

Old monitor :

$x_old = (1680 - 800) / 2 = 440
$y_old = (1050 - 600) / 2 = 225

New monitor :

$x_new = (1600 - 800) / 2 = 400
$y_new = (900  - 600) / 2 = 150

Difference :

$x_diff = 440 - 400 = 40
$y_diff = 225 - 150 = 75

Then you could apply these 2 variables ($x_diff and $y_diff) to all your pixel coords in the script and get the same pixelsearch (or mouse click) results inside your program window on both monitors ?

For example, if a pixelsearch was done on the old monitor with these screen coords : 440, 225 (this is the old upper left coord to make it easy) then this same pixelsearch will be done on the new monitor with these screen coords : 440 - $x_diff, 225 - $y_diff

Does this simple approach work for you ?

I tried looking at differences between pixels from old script and on new monitor but they didn’t seem to to align for everything unfortunately 😕 I decided to just redo everything with client coordinates so I don’t run into this issue again. 
 

the one issue I am facing with client coordinates though is imagesearch where it’s searching and returning screen coordinates instead of the client 😕 do you have any idea how to modify that by chance?

[pixelsearch]

There's a function named _WinAPI_ScreenToClient that could help you, fingers crossed.

[Nick3399]

10 hours ago, pixelsearch said:

There's a function named _WinAPI_ScreenToClient that could help you, fingers crossed.

I think for this I would have to modify of my mouseclicks referencing imagesearch to use the variable from screen to client which would be quite a bit as I have tons of different variables and variable names 😕 if there’s no single line way such as pixelcoordmode, I am prob going to just make my window location 0,0 so it will match the imagesearch coordinates between client and screen as that is just a 1 line fix 

Edited October 7, 2022 by Nick3399

[pixelsearch]

On 10/6/2022 at 1:15 AM, Nick3399 said:

how can I have imagesearch return coordinates on the client rather than screen?

To do that, you could modify directly a function in imagesearch udf (or in a copy of it, leaving the original untouched) and apply the _WinAPI_ScreenToClient function directly in imagesearch udf. This will modify the coords returned from the function, changing them automatically from screen coords to client coords,

I succeeded to do it by adding :
* 1 line in the calling script
* modifying a function in imagesearch udf

I never used imagesearch until now but it was an interesting challenge. The problem is that we're not using the same imagesearch udf. Looking at your code above, you're probably using the one found in this link  (dll's from 2010 which crash my computer) and call the function like this :

$T22 = _ImageSearch("Images\T22.png", 0, $x3, $y3, 40) ; 0 = return top & left coord, tolerance 40

My successful test was done on a more recent ImageSearch Udf found in this link, from user VIP Trong, which calls the function like  that :

Local $aRet = _ImageSearch($sImage_Name, 40, False) ; tolerance 40 , False = return top & left coord
; Array result : $aRet[0] = 1 (success, image found) => $aRet[1] = top coord, $aRet[2] = left coord

Anyway, no matter the ImageSearch UDF used, the logic should be same in both ImageSearch UDF

Calling script :

#include <MsgBoxConstants.au3>
#include "Copy of _ImageSearch_UDF.au3"

Local $sImage_Name = "notepad-logo.bmp"

Local $hGUI = GUICreate("Test with VIP's _ImageSearch", 800, 600, 100, 100) ; GUI left & top at 100, 100
Local $idPic = GUICtrlCreatePic($sImage_Name, 20, 40, 18, 25) ; 18 x 25 = original pic size
GUISetState(@SW_SHOW)

;==============================
$g_hGUI_ForClientCoords = $hGUI ; force _ImageSearch to return client cooords (this Global variable = 0 in Copy of _ImageSearch_UDF.au3)
;==============================

Local $aRet = _ImageSearch($sImage_Name, 40, False)
If $aRet[0] = 1 Then
    MsgBox($MB_TOPMOST, 'Success', 'Image found:' & " X=" & $aRet[1] & " Y=" & $aRet[2] & @CRLF & $sImage_Name)
Else
    MsgBox($MB_TOPMOST, 'Failed', 'Image not found')
EndIf

GUIDelete($hGUI)

VIP's ImageSearch UDF (the modified part) :

; 2 lines added at top of the UDF
#include <WinAPIConv.au3> ; for _WinAPI_ScreenToClient
Global $g_hGUI_ForClientCoords = 0 ; change it in calling script to return client coords

...

Func _ImageSearch($_ImagePath, $_Tolerance = 0, $_CenterPos = True)

    ; Commented 1 original line from VIP Trong
    ; Return _ImageSearch_Area($_ImagePath, 0, 0, @DesktopWidth, @DesktopHeight, $_Tolerance, $_CenterPos)

    ; Added all the following lines
    Local $aRet = _ImageSearch_Area($_ImagePath, 0, 0, @DesktopWidth, @DesktopHeight, $_Tolerance, $_CenterPos)
    If $aRet[0] = 1 And $g_hGUI_ForClientCoords Then ; $aRet[0] = 1 means success
        Local $tPoint = DllStructCreate("int X;int Y")
        DllStructSetData($tPoint, "X", $aRet[1])
        DllStructSetData($tPoint, "Y", $aRet[2])
        _WinAPI_ScreenToClient($g_hGUI_ForClientCoords, $tPoint)
        $aRet[1] = $tPoint.X
        $aRet[2] = $tPoint.Y
    EndIf
    Return $aRet

EndFunc   ;==>_ImageSearch
; * -----:| Dao Van Trong - TRONG.LIVE

So, as soon as this line in the calling script is active...

$g_hGUI_ForClientCoords = $hGUI

...then the same client coords will always be returned, no matter the GUI is displayed at 100, 100 or 200, 200 . If you comment that line, then screen coords will be returned.

Hope it helps 
 

[Nick3399]

On 10/7/2022 at 11:37 AM, pixelsearch said:

To do that, you could modify directly a function in imagesearch udf (or in a copy of it, leaving the original untouched) and apply the _WinAPI_ScreenToClient function directly in imagesearch udf. This will modify the coords returned from the function, changing them automatically from screen coords to client coords,

I succeeded to do it by adding :
* 1 line in the calling script
* modifying a function in imagesearch udf

I never used imagesearch until now but it was an interesting challenge. The problem is that we're not using the same imagesearch udf. Looking at your code above, you're probably using the one found in this link  (dll's from 2010 which crash my computer) and call the function like this :

$T22 = _ImageSearch("Images\T22.png", 0, $x3, $y3, 40) ; 0 = return top & left coord, tolerance 40

My successful test was done on a more recent ImageSearch Udf found in this link, from user VIP Trong, which calls the function like  that :

Local $aRet = _ImageSearch($sImage_Name, 40, False) ; tolerance 40 , False = return top & left coord
; Array result : $aRet[0] = 1 (success, image found) => $aRet[1] = top coord, $aRet[2] = left coord

Anyway, no matter the ImageSearch UDF used, the logic should be same in both ImageSearch UDF

Calling script :

#include <MsgBoxConstants.au3>
#include "Copy of _ImageSearch_UDF.au3"

Local $sImage_Name = "notepad-logo.bmp"

Local $hGUI = GUICreate("Test with VIP's _ImageSearch", 800, 600, 100, 100) ; GUI left & top at 100, 100
Local $idPic = GUICtrlCreatePic($sImage_Name, 20, 40, 18, 25) ; 18 x 25 = original pic size
GUISetState(@SW_SHOW)

;==============================
$g_hGUI_ForClientCoords = $hGUI ; force _ImageSearch to return client cooords (this Global variable = 0 in Copy of _ImageSearch_UDF.au3)
;==============================

Local $aRet = _ImageSearch($sImage_Name, 40, False)
If $aRet[0] = 1 Then
    MsgBox($MB_TOPMOST, 'Success', 'Image found:' & " X=" & $aRet[1] & " Y=" & $aRet[2] & @CRLF & $sImage_Name)
Else
    MsgBox($MB_TOPMOST, 'Failed', 'Image not found')
EndIf

GUIDelete($hGUI)

VIP's ImageSearch UDF (the modified part) :

; 2 lines added at top of the UDF
#include <WinAPIConv.au3> ; for _WinAPI_ScreenToClient
Global $g_hGUI_ForClientCoords = 0 ; change it in calling script to return client coords

...

Func _ImageSearch($_ImagePath, $_Tolerance = 0, $_CenterPos = True)

    ; Commented 1 original line from VIP Trong
    ; Return _ImageSearch_Area($_ImagePath, 0, 0, @DesktopWidth, @DesktopHeight, $_Tolerance, $_CenterPos)

    ; Added all the following lines
    Local $aRet = _ImageSearch_Area($_ImagePath, 0, 0, @DesktopWidth, @DesktopHeight, $_Tolerance, $_CenterPos)
    If $aRet[0] = 1 And $g_hGUI_ForClientCoords Then ; $aRet[0] = 1 means success
        Local $tPoint = DllStructCreate("int X;int Y")
        DllStructSetData($tPoint, "X", $aRet[1])
        DllStructSetData($tPoint, "Y", $aRet[2])
        _WinAPI_ScreenToClient($g_hGUI_ForClientCoords, $tPoint)
        $aRet[1] = $tPoint.X
        $aRet[2] = $tPoint.Y
    EndIf
    Return $aRet

EndFunc   ;==>_ImageSearch
; * -----:| Dao Van Trong - TRONG.LIVE

So, as soon as this line in the calling script is active...

$g_hGUI_ForClientCoords = $hGUI

...then the same client coords will always be returned, no matter the GUI is displayed at 100, 100 or 200, 200 . If you comment that line, then screen coords will be returned.

Hope it helps 
 

😮 Oh my gosh! edit so I have just tried modifying the au3 file I’m using and I can’t seem to get it working 😕
 

My variables are not $aRet — they have a bunch of different variables throughout the script at each imagesearch line

This is the imagesearch I am using also - its 2015 version

 

ImageSearch2015.au3

Edited October 9, 2022 by Nick3399

[orok]

I also tried with fun pixels. The best way is buy a monitor with resolution in which script works.  Also always remember to set DPI as 100%

[Nick3399]

hi @pixelsearch I’m sorry to bother again. I have tried to modify the ImageSearch au3 multiple times but cannot seem to get it working properly on my 2015 version