Jump to content

Mousemove not working properly with scaling in some windows

ahha

By ahha,
in AutoIt General Help and Support

 Share

Recommended Posts

ahha Universalist

ahha

Posted
Posted

This one has me baffled.  I can't seem to get MouseMove to work properly with scaling in some windows.

Run this program and use the arrow keys to move the mouse Up, Down, Left, Right at native (100%) scaling and it works as expected. 

Now go to 125%, for example, and move Up, Down, Left, Right on the desktop and it works.

Now open the MouseMove help window (cursor on MouseMove then F1).

Move the mouse into the MouseMove help window and try Up, Down, Left, Right and the movements are diagonal!

If you move, for example, right from desktop into the MouseMove window it goes from moving right to down and right (diagonal) as soon as the mouse hits the MouseMove help window.  I'm stumped.

How do I get MouseMove to work properly?  Any hints appreciated. 

;
;#AutoIt3Wrapper_run_debug_mode=Y   ; use this to debug in console window <--- LOOK


;Trying to figure out how to scale properly for MouseMove - moving mouse in Up, Down, Left, Right moves diagonally on some windows if scale <> 1 (100%).
;Test on main monitor and use 1920x1080 then scale to 125% as simple example.
;To change display scale in Windows 10, right click on desktop "Display Settings" --> Scale and layout

;arrow keys on arrow keypad
HotKeySet("{UP}", "__UpMouse")
HotKeySet("{DOWN}", "__DownMouse")
HotKeySet("{LEFT}", "__LeftMouse")
HotKeySet("{RIGHT}", "__RightMouse")

;v2g - cutting out superfluous code to show issue, added scaling

Opt("MouseCoordMode",1)         ;1 = absolute screen coordinates (default) - use this because otherwise active window mucks it up

$scale = _GetScale()
MsgBox(0,"DEBUG", "$scale = " & $scale)


While 1
    ;watch the console - the mouse is not moving correctly
    Sleep(10)   ;this works well and does not appear to use the CPU
WEnd


;================= Functions ==================

Func __UpMouse()
    Local $mousepos, $mouseposafter
    $mousepos = MouseGetPos()
    MouseMove($mousepos[0]+0, $mousepos[1]-1)
    $mouseposafter = MouseGetPos()
    ConsoleWrite("in  __UpMouse()" & @CRLF & "Before UP (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & " After UP (x,y) = " & $mouseposafter[0] & "," & $mouseposafter[1] & @CRLF & @CRLF)
EndFunc

Func __DownMouse()
    Local $mousepos, $mouseposafter
    $mousepos = MouseGetPos()
    MouseMove($mousepos[0]+0, $mousepos[1]+1)
    $mouseposafter = MouseGetPos()
    ConsoleWrite("in  __DownMouse()" & @CRLF & "Before DOWN (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & " After DOWN (x,y) = " & $mouseposafter[0] & "," & $mouseposafter[1] & @CRLF & @CRLF)
EndFunc

Func __LeftMouse()
    Local $mousepos, $mouseposafter
    $mousepos = MouseGetPos()
    MouseMove($mousepos[0]-1, $mousepos[1]+0)
    $mouseposafter = MouseGetPos()
    ConsoleWrite("in  __LeftMouse()" & @CRLF & "Before LEFT (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & " After LEFT (x,y) = " & $mouseposafter[0] & "," & $mouseposafter[1] & @CRLF & @CRLF)
EndFunc

Func __RightMouse()
    Local $mousepos, $mouseposafter
    $mousepos = MouseGetPos()
    MouseMove($mousepos[0]+1, $mousepos[1]+0)
    $mouseposafter = MouseGetPos()
    ConsoleWrite("in  __RightMouse()" & @CRLF & "Before RIGHT (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & " After RIGHT (x,y) = " & $mouseposafter[0] & "," & $mouseposafter[1] & @CRLF & @CRLF)
EndFunc



Func _GetScale()    ;returns a scale factor for PixelGetColor
    ;from: https://www.autoitscript.com/forum/topic/156251-mouse-coordinates/?tab=comments#comment-1130002
   $tDEVMODE = DllStructCreate('byte[32];dword[10];byte[32];dword[6]')
   $pDEVMODE = DllStructGetPtr($tDEVMODE)
   $i = 0

   while 1
      $ret = DllCall('user32.dll', 'int', 'EnumDisplaySettings', 'ptr', 0, 'dword', $i, 'ptr', $pDEVMODE)
      if ($ret[0] = 0) then exitloop
      $width = DllStructGetData($tDEVMODE, 4, 2) ; native width
      $i += 1
   wend

   $scale = $width / @DesktopWidth
   Return($scale)
EndFunc

 

ahha Universalist

ahha

Posted
  • Author
Posted (edited)

Can't as no controls are listed in AutoIt3 Info and tabbing does not work, so I'm stuck with moving the mouse.

Did you observe the strange behavior of MouseMove?

 

Edited by ahha
Malkey Universalist

Malkey

Posted
Posted

While running your script, I experienced the weird MouseMove behavior in both the AutoIt Help window and the AutoIt3 AU3Info window. I am running Win 10 and using a 125% (Recommended) scale.

I managed to fix this weird behavior by setting the "Advanced scaling settings".
The procedure I followed:-
Right mouse button click on a clear part of desktop;
Left mouse button click on "Display settings" on the menu;
Just below the recommended scale percentage input box, there is a link "Advanced scaling settings" which you click on;
Enter a custom scaling size in the box. (I entered 125, which was the recommended scale value on the previous window.); 
Then press the "Apply" button; 
Note the appearance of red writing "The custom scale factor won't be applied until you sign out". So, click the link "Sign out now" that is just below the red writing;
Now you can restart SciTE, AutoIt help, and a smaller version of AutoIt3 AU3Info window.  Run your script, and the mouse moves as expected.  Well, it did for me.

ahha Universalist

ahha

Posted
  • Author
Posted (edited)

Malkey,

Thanks for confirming that I'm not imagining the weird MouseMove behaviour.  That's a big vote for my sanity :)

Your proposed approach indeed does work and I noticed that the $scale = 1.  Which means that somehow the OS (I'm also running Win 10) is mucking with _GetScale() (the DLL call) to fake out the true resolution of the monitors.  This is just plain strange.

The problem I have with your approach is the users are not going to change to a custom resolution.  It seems to me that regardless of the scaling that MouseMove should not be behaving like this.  BTW it's not just AutoIt windows it acts weird in.  It was actually another program of mine - I just used an AutoIt window to demonstrate the weirdness.

*Edit* This whole issue comes up because on a 4K UHD monitor, which are becoming more common, the native resolution is 3840 x 2160, however Win 10 is set at 150% scaling otherwise most aps are too tiny to read.  AND the MouseMove weird behaviour occurs.  I used the 1920x1080 and 125% example just to illustrate the problem as not everyone has a UHD monitor.  I'm making an assumption that once I figure out how to fix it on the 1920x1080 125% example, I'll be able to fix it on the UHD 150% case.

Is this a bug that should be reported?  I can't believe moving into a widow should cause such weirdness.

Thanks again for the sanity check.

ahha

Edited by ahha
Added UHD info.
ahha Universalist

ahha

Posted
  • Author
Posted

Okay, I've added a function _DynamicallyCorrectMouseMove for the UP movement only at this point (I'll work on the others tomorrow) and it works but this whole compensating is a kludge.   The Beeps are to give an indication of what's going on.  

;
;#AutoIt3Wrapper_run_debug_mode=Y   ; use this to debug in console window <--- LOOK


;Trying to figure out how to scale properly for MouseMove - moving mouse in Up, Down, Left, Right moves diagonally on some windows if scale <> 1 (100%).
;Test on main monitor and use 1920x1080 then scale to 125% as simple example.
;To change display scale in Windows 10, right click on desktop "Display Settings" --> Scale and layout

;arrow keys on arrow keypad
HotKeySet("{UP}", "__UpMouse")
HotKeySet("{DOWN}", "__DownMouse")
HotKeySet("{LEFT}", "__LeftMouse")
HotKeySet("{RIGHT}", "__RightMouse")

;v2g - cutting out superfluous code to show issue, added scaling
;v2h - added scaling
;v2i - trying dynamic correction for when MouseMove is erratic


Opt("MouseCoordMode",1)         ;1 = absolute screen coordinates (default) - use this because otherwise active window mucks it up

$scale = _GetScale()
MsgBox(0,"DEBUG", "$scale = " & $scale)


While 1
    ;watch the console - the mouse is not moving correctly
    Sleep(10)   ;this works well and does not appear to use the CPU
WEnd


;================= Functions ==================

Func __UpMouse()
    Local $mousepos, $mouseposafter
    $mousepos = MouseGetPos()
    MouseMove($mousepos[0]+0, $mousepos[1]-(1*$scale))
    ;MouseMove($mousepos[0]-1, $mousepos[1]-(1*$scale)) ;working some of the time
    ;MouseMove($mousepos[0]+0, $mousepos[1]-(1/$scale))
    $mouseposafter = MouseGetPos()
    ConsoleWrite("in  __UpMouse()" & @CRLF & "Before UP (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & " After UP (x,y) = " & $mouseposafter[0] & "," & $mouseposafter[1] & @CRLF & @CRLF)
    ;v2i
    If $mousepos[0] <> $mouseposafter[0] Then _DynamicallyCorrectMouseMove("UP", $mousepos[0], $mousepos[1])    ;check that no X-axis movement, pass original information
EndFunc

Func __DownMouse()
    Local $mousepos, $mouseposafter
    $mousepos = MouseGetPos()
    MouseMove($mousepos[0]+0, $mousepos[1]+(1*$scale))
    $mouseposafter = MouseGetPos()
    ConsoleWrite("in  __DownMouse()" & @CRLF & "Before DOWN (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & " After DOWN (x,y) = " & $mouseposafter[0] & "," & $mouseposafter[1] & @CRLF & @CRLF)
EndFunc

Func __LeftMouse()
    Local $mousepos, $mouseposafter
    $mousepos = MouseGetPos()
    MouseMove($mousepos[0]-(1*$scale), $mousepos[1]+0)
    $mouseposafter = MouseGetPos()
    ConsoleWrite("in  __LeftMouse()" & @CRLF & "Before LEFT (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & " After LEFT (x,y) = " & $mouseposafter[0] & "," & $mouseposafter[1] & @CRLF & @CRLF)
EndFunc

Func __RightMouse()
    Local $mousepos, $mouseposafter
    $mousepos = MouseGetPos()
    MouseMove($mousepos[0]+(1*$scale), $mousepos[1]+0)
    $mouseposafter = MouseGetPos()
    ConsoleWrite("in  __RightMouse()" & @CRLF & "Before RIGHT (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & " After RIGHT (x,y) = " & $mouseposafter[0] & "," & $mouseposafter[1] & @CRLF & @CRLF)
EndFunc


Func _GetScale()    ;returns a scale factor for PixelGetColor
    ;from: https://www.autoitscript.com/forum/topic/156251-mouse-coordinates/?tab=comments#comment-1130002
   $tDEVMODE = DllStructCreate('byte[32];dword[10];byte[32];dword[6]')
   $pDEVMODE = DllStructGetPtr($tDEVMODE)
   $i = 0

   while 1
      $ret = DllCall('user32.dll', 'int', 'EnumDisplaySettings', 'ptr', 0, 'dword', $i, 'ptr', $pDEVMODE)
      if ($ret[0] = 0) then exitloop
      $width = DllStructGetData($tDEVMODE, 4, 2) ; native width
      $i += 1
   wend

   $scale = $width / @DesktopWidth
   Return($scale)
EndFunc



Func _DynamicallyCorrectMouseMove($sDirection, $iXorignal, $iYoriginal)     ;check that no X-axis movement, pass original information

    Select
        Case StringLower($sDirection) = "up"
            ;okay we have the original coordinates and it messed up on the move up and also went left or right
            ;so try dynamically to adjust
            ;try a brute force search pattern - limit it to 10 pixels

            ;try this first
            MouseMove($iXorignal-1, $iYoriginal-2)
            $mousepos = MouseGetPos()
            If (($mousepos[0] = $iXorignal) AND ($mousepos[1] = $iYoriginal-1)) Then
                Beep(1500, 25) ;debug to show we're still searching
                ConsoleWrite("in  _DynamicallyCorrectMouseMove" & @CRLF & "After correction (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & @CRLF)

            Else

                $iCount = 0
                For $x = -5 to +5
                    For $y = -5 to +5
                        $iCount = $iCount +1    ;get an idea of how many tries we needed
                        Beep(500, 25) ;debug to show we're still searching
                        MouseMove($iXorignal+$x, $iYoriginal+$y)
                        $mousepos = MouseGetPos()
                        If (($mousepos[0] = $iXorignal) AND ($mousepos[1] = $iYoriginal-1)) Then
                            ConsoleWrite("in  _DynamicallyCorrectMouseMove   $x,$y offset = " & $x  & "," & $y & "  $iCount = " & $iCount & @CRLF & "After correction (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & @CRLF)
                            ;from testing most common is $x,$y offset = -1,-2
                            ;so try this first before dropping into this
                            ;-1 makes sense as moving 1 X pixel to the right, -2 makes little sense as it moved correcly in Y direction but it works
                            ExitLoop 2  ;we found it
                        EndIf
                    Next
                Next
            EndIf

        Case StringLower($sDirection) = "down"

        Case StringLower($sDirection) = "left"

        Case StringLower($sDirection) = "right"

        Case Else
            ;really messed up so do nothing
    EndSelect


EndFunc ;_DynamicallyCorrectMouseMove

Can someone point me to a lower level or different approach to moving the mouse?

ahha Universalist

ahha

Posted
  • Author
Posted

Bert - thanks will dive into it.  However, I still need to MouseMove because I need to watch a pixel to know when done (much like progress bar).

ahha Universalist

ahha

Posted
  • Author
Posted

All - latest version v2j with UP, DOWN, LEFT, and RIGHT MouseMove dynamically corrected for.  Tested at 125%. 

;
;#AutoIt3Wrapper_run_debug_mode=Y   ; use this to debug in console window <--- LOOK


;Trying to figure out how to scale properly for MouseMove - moving mouse in Up, Down, Left, Right moves diagonally on some windows if scale <> 1 (100%).
;Test on main monitor and use 1920x1080 then scale to 125% as simple example.
;To change display scale in Windows 10, right click on desktop "Display Settings" --> Scale and layout

;arrow keys on arrow keypad
HotKeySet("{UP}", "__UpMouse")
HotKeySet("{DOWN}", "__DownMouse")
HotKeySet("{LEFT}", "__LeftMouse")
HotKeySet("{RIGHT}", "__RightMouse")

HotKeySet("^!x", "Terminate") ;Press Ctrl+Alt+x to exit the script

;v2g - cutting out superfluous code to show issue, added scaling
;v2h - added scaling
;v2i - trying dynamic correction for when MouseMove is erratic - added UP only it works
;v2j - added dynamic DOWN, LEFT, RIGHT, and Ctrl+Alt+x to exit the script, $iMouseSpeed, $debug for sound   ;may still need to add screen boundary checks

Opt("MouseCoordMode",1)         ;1 = absolute screen coordinates (default) - use this because otherwise active window mucks it up

$scale = _GetScale()
MsgBox(0,"DEBUG", "$scale = " & $scale)

;scaling just messes up the MouseMove - so force to 1
$scale = 1 ;debug force to a value

While 1
    ;watch the console - the mouse is not moving correctly
    Sleep(10)   ;this works well and does not appear to use the CPU
WEnd


;================= Functions ==================

Func __UpMouse()
    Local $mousepos, $mouseposafter
    $mousepos = MouseGetPos()
    MouseMove($mousepos[0]+0, $mousepos[1]-(1*$scale))
    ;MouseMove($mousepos[0]-1, $mousepos[1]-(1*$scale)) ;working some of the time
    ;MouseMove($mousepos[0]+0, $mousepos[1]-(1/$scale))
    $mouseposafter = MouseGetPos()
    ConsoleWrite("in  __UpMouse()" & @CRLF & "Before UP (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & " After UP (x,y) = " & $mouseposafter[0] & "," & $mouseposafter[1] & @CRLF & @CRLF)
    ;v2i
    If $mousepos[0] <> $mouseposafter[0] Then _DynamicallyCorrectMouseMove("UP", $mousepos[0], $mousepos[1])    ;check that no X-axis movement, pass original information
EndFunc

Func __DownMouse()
    Local $mousepos, $mouseposafter
    $mousepos = MouseGetPos()
    MouseMove($mousepos[0]+0, $mousepos[1]+(1*$scale))
    $mouseposafter = MouseGetPos()
    ConsoleWrite("in  __DownMouse()" & @CRLF & "Before DOWN (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & " After DOWN (x,y) = " & $mouseposafter[0] & "," & $mouseposafter[1] & @CRLF & @CRLF)
    ;v2j
    If $mousepos[0] <> $mouseposafter[0] Then _DynamicallyCorrectMouseMove("DOWN", $mousepos[0], $mousepos[1])  ;check that no X-axis movement, pass original information
EndFunc

Func __LeftMouse()
    Local $mousepos, $mouseposafter
    $mousepos = MouseGetPos()
    MouseMove($mousepos[0]-(1*$scale), $mousepos[1]+0)
    $mouseposafter = MouseGetPos()
    ConsoleWrite("in  __LeftMouse()" & @CRLF & "Before LEFT (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & " After LEFT (x,y) = " & $mouseposafter[0] & "," & $mouseposafter[1] & @CRLF & @CRLF)
    ;v2j
    If $mousepos[1] <> $mouseposafter[1] Then _DynamicallyCorrectMouseMove("LEFT", $mousepos[0], $mousepos[1])  ;check that no Y-axis movement, pass original information
EndFunc

Func __RightMouse()
    Local $mousepos, $mouseposafter
    $mousepos = MouseGetPos()
    MouseMove($mousepos[0]+(1*$scale), $mousepos[1]+0)
    $mouseposafter = MouseGetPos()
    ConsoleWrite("in  __RightMouse()" & @CRLF & "Before RIGHT (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & " After RIGHT (x,y) = " & $mouseposafter[0] & "," & $mouseposafter[1] & @CRLF & @CRLF)
    ;v2j
    If $mousepos[1] <> $mouseposafter[1] Then _DynamicallyCorrectMouseMove("RIGHT", $mousepos[0], $mousepos[1])     ;check that no Y-axis movement, pass original information
EndFunc


Func _GetScale()    ;returns a scale factor for PixelGetColor
    ;from: https://www.autoitscript.com/forum/topic/156251-mouse-coordinates/?tab=comments#comment-1130002
   $tDEVMODE = DllStructCreate('byte[32];dword[10];byte[32];dword[6]')
   $pDEVMODE = DllStructGetPtr($tDEVMODE)
   $i = 0

   while 1
      $ret = DllCall('user32.dll', 'int', 'EnumDisplaySettings', 'ptr', 0, 'dword', $i, 'ptr', $pDEVMODE)
      if ($ret[0] = 0) then exitloop
      $width = DllStructGetData($tDEVMODE, 4, 2) ; native width
      $i += 1
   wend

   $scale = $width / @DesktopWidth
   Return($scale)
EndFunc



Func _DynamicallyCorrectMouseMove($sDirection, $iXorignal, $iYoriginal)     ;check that no X-axis movement, pass original information
    $iCaseExit = 0  ;effectively an ExitSelect.
                    ;just position Case $iCaseExit = 1  as the *next* Case statement as ContinueCase drops from the code in current Case into the code of the next Case
                    ;it DOES NOT evaluate the next Case statement, it just executes the next code which here is just reset the flag (i.e. $iCaseExit = 0)

    $iMouseSpeed = 0
    $debug = 0  ;0=off, 1=on

    Select
        Case StringUpper($sDirection) = "UP"
            ;okay we have the original coordinates and it messed up on the move up and also went left or right
            ;so try dynamically to adjust
            ;try a brute force search pattern - limit it to 10 pixels

            ;try this first
            MouseMove($iXorignal-1, $iYoriginal-2, $iMouseSpeed)
            $mousepos = MouseGetPos()
            If (($mousepos[0] = $iXorignal) AND ($mousepos[1] = $iYoriginal-1)) Then
                If $debug = 1 Then Beep(1500, 25) ;debug to show we're still searching
                ConsoleWrite("in  _DynamicallyCorrectMouseMove using -1,-2" & @CRLF & "After correction (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & @CRLF)
            Else
                $iCount = 0
                For $x = -5 to +5
                    For $y = -5 to +5
                        $iCount = $iCount +1    ;get an idea of how many tries we needed
                        If $debug = 1 Then Beep(500, 25) ;debug to show we're still searching
                        MouseMove($iXorignal+$x, $iYoriginal+$y, $iMouseSpeed)
                        $mousepos = MouseGetPos()
                        If (($mousepos[0] = $iXorignal) AND ($mousepos[1] = $iYoriginal-1)) Then
                            ConsoleWrite("in  _DynamicallyCorrectMouseMove   $x,$y offset = " & $x  & "," & $y & "  $iCount = " & $iCount & @CRLF & "After correction (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & @CRLF)
                            ;from testing most common is $x,$y offset = -1,-2
                            ;so try this first before dropping into this
                            ;-1 makes sense as moving 1 X pixel to the right, -2 makes little sense as it moved correcly in Y direction but it works
                            ExitLoop 2  ;we found it
                        EndIf
                    Next
                Next
            EndIf


        Case StringUpper($sDirection) = "DOWN"
            ;okay we have the original coordinates and it messed up on the move down and also went left or right
            ;so try dynamically to adjust
            ;try a brute force search pattern - limit it to 10 pixels

            ;try this first
            MouseMove($iXorignal-1, $iYoriginal-0, $iMouseSpeed)
            $mousepos = MouseGetPos()
            If (($mousepos[0] = $iXorignal) AND ($mousepos[1] = $iYoriginal+1)) Then
                If $debug = 1 Then Beep(1500, 25) ;debug to show we're still searching
                ConsoleWrite("in  _DynamicallyCorrectMouseMove using -1,0" & @CRLF & "After correction (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & @CRLF)
            Else
                $iCount = 0
                For $x = -5 to +5
                    For $y = -5 to +5
                        $iCount = $iCount +1    ;get an idea of how many tries we needed
                        If $debug = 1 Then Beep(500, 25) ;debug to show we're still searching
                        MouseMove($iXorignal+$x, $iYoriginal+$y, $iMouseSpeed)
                        $mousepos = MouseGetPos()
                        If (($mousepos[0] = $iXorignal) AND ($mousepos[1] = $iYoriginal+1)) Then
                            ConsoleWrite("in  _DynamicallyCorrectMouseMove   $x,$y offset = " & $x  & "," & $y & "  $iCount = " & $iCount & @CRLF & "After correction (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & @CRLF)
                            ;from testing most common is $x,$y offset = -1,0
                            ;so try this first before dropping into this
                            ;-1 makes sense as moving 1 X pixel to the right, 0 makes little sense as it moved correcly in Y direction but it works
                            ExitLoop 2  ;we found it
                        EndIf
                    Next
                Next
            EndIf


        Case StringUpper($sDirection) = "LEFT"
            ;okay we have the original coordinates and it messed up on the move left and also went up or down
            ;so try dynamically to adjust
            ;try a brute force search pattern - limit it to 10 pixels

            ;try this first
            MouseMove($iXorignal-1, $iYoriginal-0, $iMouseSpeed)
            $mousepos = MouseGetPos()
            If (($mousepos[0] = $iXorignal-1) AND ($mousepos[1] = $iYoriginal)) Then
                If $debug = 1 Then Beep(1500, 25) ;debug to show we're still searching
                ConsoleWrite("in  _DynamicallyCorrectMouseMove using -1,0" & @CRLF & "After correction (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & @CRLF)
                $iCaseExit = 1  ;exit Case
                ContinueCase
            EndIf

            ;try this second
            MouseMove($iXorignal-2, $iYoriginal-1, $iMouseSpeed)
            $mousepos = MouseGetPos()
            If (($mousepos[0] = $iXorignal-1) AND ($mousepos[1] = $iYoriginal)) Then
                If $debug = 1 Then Beep(1500, 25) ;debug to show we're still searching
                ConsoleWrite("in  _DynamicallyCorrectMouseMove using -2,-1" & @CRLF & "After correction (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & @CRLF)
                $iCaseExit = 1  ;exit Case
                ContinueCase
            EndIf

            ;else let's do the hunt
;~          MouseMove($iXorignal-1, $iYoriginal-0, $iMouseSpeed)
;~          $mousepos = MouseGetPos()
;~          If (($mousepos[0] = $iXorignal-1) AND ($mousepos[1] = $iYoriginal)) Then
;~              If $debug = 1 Then Beep(1500, 25) ;debug to show we're still searching
;~              ConsoleWrite("in  _DynamicallyCorrectMouseMove" & @CRLF & "After correction (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & @CRLF)
;~          EndIf

            $iCount = 0
            For $x = -5 to +5
                For $y = -5 to +5
                    $iCount = $iCount +1    ;get an idea of how many tries we needed
                    If $debug = 1 Then Beep(500, 25) ;debug to show we're still searching
                    MouseMove($iXorignal+$x, $iYoriginal+$y, $iMouseSpeed)
                    $mousepos = MouseGetPos()
                    If (($mousepos[0] = $iXorignal-1) AND ($mousepos[1] = $iYoriginal)) Then    ;moved correctly
                        ConsoleWrite("in  _DynamicallyCorrectMouseMove   $x,$y offset = " & $x  & "," & $y & "  $iCount = " & $iCount & @CRLF & "After correction (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & @CRLF)
                        ;from testing most common is strange as there are 2 most common
                        ;one is $x,$y offset = -1,0 and the second is -2,-1
                        ;perhaps try -1,0 first then -2,-1
                        ;so try this first before dropping into this
                        ;-1 and -2 make no sense as moving 1 X pixel to the left, 0 and -1 make little sense
                        ExitLoop 2  ;we found it
                    EndIf
                Next
            Next
;~      EndIf


        Case $iCaseExit = 1 ;exit Case statement directly above
            If $debug = 1 Then Beep(2000, 100)      ;debug
            $iCaseExit = 0  ;reset it and drop to EndSelect


        Case StringUpper($sDirection) = "RIGHT"

            ;try this first
            MouseMove($iXorignal-0, $iYoriginal-1, $iMouseSpeed)
            $mousepos = MouseGetPos()
            If (($mousepos[0] = $iXorignal+1) AND ($mousepos[1] = $iYoriginal)) Then
                If $debug = 1 Then Beep(1500, 25) ;debug to show we're still searching
                ConsoleWrite("in  _DynamicallyCorrectMouseMove using 0,-1" & @CRLF & "After correction (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & @CRLF)
            Else
                $iCount = 0
                For $x = -5 to +5
                    For $y = -5 to +5
                        $iCount = $iCount +1    ;get an idea of how many tries we needed
                        If $debug = 1 Then Beep(500, 25) ;debug to show we're still searching
                        MouseMove($iXorignal+$x, $iYoriginal+$y, $iMouseSpeed)
                        $mousepos = MouseGetPos()
                        If (($mousepos[0] = $iXorignal+1) AND ($mousepos[1] = $iYoriginal)) Then    ;moved correctly
                            ConsoleWrite("in  _DynamicallyCorrectMouseMove   $x,$y offset = " & $x  & "," & $y & "  $iCount = " & $iCount & @CRLF & "After correction (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & @CRLF)
                            ;from testing most common is $x,$y offset = 0,-1
                            ;so try this first before dropping into this
                            ;0 makes no sense as moving 1 X pixel to the right
                            ExitLoop 2  ;we found it
                        EndIf
                    Next
                Next
            EndIf

        Case Else
            ;really messed up so do nothing

    EndSelect


EndFunc ;_DynamicallyCorrectMouseMove


Func Terminate()
    Beep(200, 200)
    Exit 0
EndFunc

 

ahha Universalist

ahha

Posted
  • Author
Posted

Version v2n - works for 1920x1080 display at 100%, 125%, 150% - NOT at 175% it appears erratic.

Next I'll try and check on UDH monitor as Win 10 defaults to 150%. 

;
;#AutoIt3Wrapper_run_debug_mode=Y   ; use this to debug in console window <--- LOOK


;Trying to figure out how to scale properly for MouseMove - moving mouse in Up, Down, Left, Right moves diagonally on some windows if scale <> 1 (100%).
;Test on main monitor and use 1920x1080 then scale to 125% as simple example.
;To change display scale in Windows 10, right click on desktop "Display Settings" --> Scale and layout

;arrow keys on arrow keypad
HotKeySet("{UP}", "__UpMouse")
HotKeySet("{DOWN}", "__DownMouse")
HotKeySet("{LEFT}", "__LeftMouse")
HotKeySet("{RIGHT}", "__RightMouse")

HotKeySet("^!x", "Terminate") ;Press Ctrl+Alt+x to exit the script

;v2g - cutting out superfluous code to show issue, added scaling
;v2h - added scaling
;v2i - trying dynamic correction for when MouseMove is erratic - added UP only it works
;v2j - added dynamic DOWN, LEFT, RIGHT, and Ctrl+Alt+x to exit the script, $iMouseSpeed, $debug for sound   ;may still need to add screen boundary checks
;v2k - check both X and Y coordinates after MouseMove not just if one coordinate changed that shouldn't.
;       We are using $scale to make a best guess at a first or second coordinate correction before using a brute force hunt
;v2l - skip worthless
;v2m - checking at other scale factors, works at 100%, 125%, 150%,
;       175% is struggling and not really moving the mouse up and left and is *NOT* dropping into the dynamic adjustment - SO MouseGetPos() is AFU!!!
;v2n - AND logic is wrong we want to dynamically correct if either coordinate is off so use OR for checking
;       works for 1920x1080 display at 100%, 125%, 150% - NOT at 175% it appears erratic

Opt("MouseCoordMode",1)         ;1 = absolute screen coordinates (default) - use this because otherwise active window mucks it up

$scale = _GetScale()
MsgBox(0,"DEBUG", "$scale = " & $scale)

;scaling just messes up the MouseMove - so force to 1
$scale = 1 ;debug force to a value

While 1
    ;watch the console - the mouse is not moving correctly
    Sleep(10)   ;this works well and does not appear to use the CPU
WEnd


;================= Functions ==================

Func __UpMouse()
    Local $mousepos, $mouseposafter
    $mousepos = MouseGetPos()
    MouseMove($mousepos[0]+0, $mousepos[1]-(1*$scale))
    ;MouseMove($mousepos[0]-1, $mousepos[1]-(1*$scale)) ;working some of the time
    ;MouseMove($mousepos[0]+0, $mousepos[1]-(1/$scale))
    $mouseposafter = MouseGetPos()
    ConsoleWrite("in  __UpMouse()" & @CRLF & "Before UP (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & " After UP (x,y) = " & $mouseposafter[0] & "," & $mouseposafter[1] & @CRLF & @CRLF)
    ;v2k
    ;ConsoleWrite("$mouseposafter[0] = " & $mouseposafter[0] & @CRLF & "$mousepos[0]+0 = " & $mousepos[0]+0 & @CRLF & @CRLF)
    ;If ( ($mouseposafter[0] <> $mousepos[0]+0) AND ($mouseposafter[1] <> $mousepos[1]-(1*$scale)) ) Then _DynamicallyCorrectMouseMove("UP", $mousepos[0], $mousepos[1])    ;check both coordinates, pass original information
    ;AND is WRONG it's OR
    If ( ($mouseposafter[0] <> $mousepos[0]+0) OR ($mouseposafter[1] <> $mousepos[1]-(1*$scale)) ) Then _DynamicallyCorrectMouseMove("UP", $mousepos[0], $mousepos[1])  ;check both coordinates, pass original information
EndFunc

Func __DownMouse()
    Local $mousepos, $mouseposafter
    $mousepos = MouseGetPos()
    MouseMove($mousepos[0]+0, $mousepos[1]+(1*$scale))
    $mouseposafter = MouseGetPos()
    ConsoleWrite("in  __DownMouse()" & @CRLF & "Before DOWN (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & " After DOWN (x,y) = " & $mouseposafter[0] & "," & $mouseposafter[1] & @CRLF & @CRLF)
    ;v2k
    If ( ($mouseposafter[0] <> $mousepos[0]+0) OR ($mouseposafter[1] <> $mousepos[1]+(1*$scale)) ) Then _DynamicallyCorrectMouseMove("DOWN", $mousepos[0], $mousepos[1])    ;check both coordinates, pass original information
EndFunc

Func __LeftMouse()
    Local $mousepos, $mouseposafter
    $mousepos = MouseGetPos()
    MouseMove($mousepos[0]-(1*$scale), $mousepos[1]+0)
    $mouseposafter = MouseGetPos()
    ConsoleWrite("in  __LeftMouse()" & @CRLF & "Before LEFT (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & " After LEFT (x,y) = " & $mouseposafter[0] & "," & $mouseposafter[1] & @CRLF & @CRLF)
    ;v2k
    If ( ($mouseposafter[0] <> $mousepos[0]-(1*$scale)) OR ($mouseposafter[1] <> $mousepos[1]) ) Then _DynamicallyCorrectMouseMove("LEFT", $mousepos[0], $mousepos[1])  ;check both coordinates, pass original information
EndFunc

Func __RightMouse()
    Local $mousepos, $mouseposafter
    $mousepos = MouseGetPos()
    MouseMove($mousepos[0]+(1*$scale), $mousepos[1]+0)
    $mouseposafter = MouseGetPos()
    ConsoleWrite("in  __RightMouse()" & @CRLF & "Before RIGHT (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & " After RIGHT (x,y) = " & $mouseposafter[0] & "," & $mouseposafter[1] & @CRLF & @CRLF)
    ;v2k
    If ( ($mouseposafter[0] <> $mousepos[0]+(1*$scale)) OR ($mouseposafter[1] <> $mousepos[1]) ) Then _DynamicallyCorrectMouseMove("RIGHT", $mousepos[0], $mousepos[1])     ;check both coordinates, pass original information
EndFunc


Func _GetScale()    ;returns a scale factor for PixelGetColor
    ;from: https://www.autoitscript.com/forum/topic/156251-mouse-coordinates/?tab=comments#comment-1130002
   $tDEVMODE = DllStructCreate('byte[32];dword[10];byte[32];dword[6]')
   $pDEVMODE = DllStructGetPtr($tDEVMODE)
   $i = 0

   while 1
      $ret = DllCall('user32.dll', 'int', 'EnumDisplaySettings', 'ptr', 0, 'dword', $i, 'ptr', $pDEVMODE)
      if ($ret[0] = 0) then exitloop
      $width = DllStructGetData($tDEVMODE, 4, 2) ; native width
      $i += 1
   wend

   $scale = $width / @DesktopWidth
   Return($scale)
EndFunc



Func _DynamicallyCorrectMouseMove($sDirection, $iXorignal, $iYoriginal)     ;check that no X-axis movement, pass original information
    $iCaseExit = 0  ;effectively an ExitSelect.
                    ;just position Case $iCaseExit = 1  as the *next* Case statement as ContinueCase drops from the code in current Case into the code of the next Case
                    ;it DOES NOT evaluate the next Case statement, it just executes the next code which here is just reset the flag (i.e. $iCaseExit = 0)

    $iMouseSpeed = 0
    $debug = 0  ;0=off, 1=on

    Select
        Case StringUpper($sDirection) = "UP"
            ;okay we have the original coordinates and it messed up on the move up and also went left or right or too far up
            ;so try dynamically to adjust
            ;try a brute force search pattern - limit it to 10 pixels

            ;try this first - learned from running brute force and observing
            MouseMove($iXorignal-1, $iYoriginal-2, $iMouseSpeed)
            $mousepos = MouseGetPos()
            If (($mousepos[0] = $iXorignal) AND ($mousepos[1] = $iYoriginal-1)) Then    ;we moved the correct amount
                If $debug = 1 Then Beep(1500, 25) ;debug to show we're still searching
                ConsoleWrite("in  _DynamicallyCorrectMouseMove using -1,-2" & @CRLF & "After correction (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & @CRLF)
            Else    ;do the brute search +/-5 X and Y pixels from the original location
                $iCount = 0
                For $x = -5 to +5
                    For $y = -5 to +5
                        $iCount = $iCount + 1   ;get an idea of how many tries we needed
                        If $debug = 1 Then Beep(500, 25) ;debug to show we're still searching
                        MouseMove($iXorignal+$x, $iYoriginal+$y, $iMouseSpeed)
                        $mousepos = MouseGetPos()
                        If (($mousepos[0] = $iXorignal) AND ($mousepos[1] = $iYoriginal-1)) Then    ;we moved the correct amount
                            ConsoleWrite("in  _DynamicallyCorrectMouseMove   $x,$y offset = " & $x  & "," & $y & "  $iCount = " & $iCount & @CRLF & "After correction (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & @CRLF)
                            ;from testing most common is $x,$y offset = -1,-2
                            ;so try this first before dropping into this
                            ;-1 makes sense as moving 1 X pixel to the right, -2 makes little sense as it moved correcly in Y direction but it works
                            ExitLoop 2  ;we found it
                        EndIf
                    Next
                Next
            EndIf


        Case StringUpper($sDirection) = "DOWN"
            ;okay we have the original coordinates and it messed up on the move down and also went left or right
            ;so try dynamically to adjust
            ;try a brute force search pattern - limit it to 10 pixels

            ;try this first
            MouseMove($iXorignal-1, $iYoriginal-0, $iMouseSpeed)
            $mousepos = MouseGetPos()
            If (($mousepos[0] = $iXorignal) AND ($mousepos[1] = $iYoriginal+1)) Then    ;we moved the correct amount
                If $debug = 1 Then Beep(1500, 25) ;debug to show we're still searching
                ConsoleWrite("in  _DynamicallyCorrectMouseMove using -1,0" & @CRLF & "After correction (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & @CRLF)
            Else
                $iCount = 0
                For $x = -5 to +5
                    For $y = -5 to +5
                        $iCount = $iCount +1    ;get an idea of how many tries we needed
                        If $debug = 1 Then Beep(500, 25) ;debug to show we're still searching
                        MouseMove($iXorignal+$x, $iYoriginal+$y, $iMouseSpeed)
                        $mousepos = MouseGetPos()
                        If (($mousepos[0] = $iXorignal) AND ($mousepos[1] = $iYoriginal+1)) Then    ;we moved the correct amount
                            ConsoleWrite("in  _DynamicallyCorrectMouseMove   $x,$y offset = " & $x  & "," & $y & "  $iCount = " & $iCount & @CRLF & "After correction (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & @CRLF)
                            ;from testing most common is $x,$y offset = -1,0
                            ;so try this first before dropping into this
                            ;-1 makes sense as moving 1 X pixel to the right, 0 makes little sense as it moved correcly in Y direction but it works
                            ExitLoop 2  ;we found it
                        EndIf
                    Next
                Next
            EndIf


        Case StringUpper($sDirection) = "LEFT"
            ;okay we have the original coordinates and it messed up on the move left and also went up or down
            ;so try dynamically to adjust
            ;try a brute force search pattern - limit it to 10 pixels

            ;try this first
            MouseMove($iXorignal-1, $iYoriginal-0, $iMouseSpeed)
            $mousepos = MouseGetPos()
            If (($mousepos[0] = $iXorignal-1) AND ($mousepos[1] = $iYoriginal)) Then    ;we moved the correct amount
                If $debug = 1 Then Beep(1500, 25) ;debug to show we're still searching
                ConsoleWrite("in  _DynamicallyCorrectMouseMove using -1,0" & @CRLF & "After correction (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & @CRLF)
                $iCaseExit = 1  ;exit Case
                ContinueCase
            EndIf

            ;try this second
            MouseMove($iXorignal-2, $iYoriginal-1, $iMouseSpeed)
            $mousepos = MouseGetPos()
            If (($mousepos[0] = $iXorignal-1) AND ($mousepos[1] = $iYoriginal)) Then    ;we moved the correct amount
                If $debug = 1 Then Beep(1500, 25) ;debug to show we're still searching
                ConsoleWrite("in  _DynamicallyCorrectMouseMove using -2,-1" & @CRLF & "After correction (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & @CRLF)
                $iCaseExit = 1  ;exit Case
                ContinueCase
            EndIf

            ;else let's do the hunt
            $iCount = 0
            For $x = -5 to +5
                For $y = -5 to +5
                    $iCount = $iCount +1    ;get an idea of how many tries we needed
                    If $debug = 1 Then Beep(500, 25) ;debug to show we're still searching
                    MouseMove($iXorignal+$x, $iYoriginal+$y, $iMouseSpeed)
                    $mousepos = MouseGetPos()
                    If (($mousepos[0] = $iXorignal-1) AND ($mousepos[1] = $iYoriginal)) Then    ;we moved the correct amount
                        ConsoleWrite("in  _DynamicallyCorrectMouseMove   $x,$y offset = " & $x  & "," & $y & "  $iCount = " & $iCount & @CRLF & "After correction (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & @CRLF)
                        ;from testing most common is strange as there are 2 most common
                        ;one is $x,$y offset = -1,0 and the second is -2,-1
                        ;perhaps try -1,0 first then -2,-1
                        ;so try this first before dropping into this
                        ;-1 and -2 make no sense as moving 1 X pixel to the left, 0 and -1 make little sense
                        ExitLoop 2  ;we found it
                    EndIf
                Next
            Next


        Case $iCaseExit = 1 ;exit Case statement directly above
            If $debug = 1 Then Beep(2000, 100)      ;debug
            $iCaseExit = 0  ;reset it and drop to EndSelect


        Case StringUpper($sDirection) = "RIGHT"

            ;try this first
            MouseMove($iXorignal-0, $iYoriginal-1, $iMouseSpeed)
            $mousepos = MouseGetPos()
            If (($mousepos[0] = $iXorignal+1) AND ($mousepos[1] = $iYoriginal)) Then    ;we moved the correct amount
                If $debug = 1 Then Beep(1500, 25) ;debug to show we're still searching
                ConsoleWrite("in  _DynamicallyCorrectMouseMove using 0,-1" & @CRLF & "After correction (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & @CRLF)
            Else
                $iCount = 0
                For $x = -5 to +5
                    For $y = -5 to +5
                        $iCount = $iCount +1    ;get an idea of how many tries we needed
                        If $debug = 1 Then Beep(500, 25) ;debug to show we're still searching
                        MouseMove($iXorignal+$x, $iYoriginal+$y, $iMouseSpeed)
                        $mousepos = MouseGetPos()
                        If (($mousepos[0] = $iXorignal+1) AND ($mousepos[1] = $iYoriginal)) Then    ;we moved the correct amount
                            ConsoleWrite("in  _DynamicallyCorrectMouseMove   $x,$y offset = " & $x  & "," & $y & "  $iCount = " & $iCount & @CRLF & "After correction (x,y) = " & $mousepos[0] & "," & $mousepos[1] & @CRLF & @CRLF)
                            ;from testing most common is $x,$y offset = 0,-1
                            ;so try this first before dropping into this
                            ;0 makes no sense as moving 1 X pixel to the right
                            ExitLoop 2  ;we found it
                        EndIf
                    Next
                Next
            EndIf

        Case Else
            ;really messed up so do nothing

    EndSelect


EndFunc ;_DynamicallyCorrectMouseMove


Func Terminate()
    Beep(200, 200)
    Exit 0
EndFunc

 

  • 9 months later...
spindel Seeker

spindel

Posted
Posted

ahha

Cool exercise your code is helping me figure out how to best tackle debuging my own mouse move issue.

I am curious though if it is Windows scaling problem why not package ur AutoIT script with a powershell script that sets scaling to 100 or 125 (this will enable the custom scaling factor mentioned eariler) follow it up with a runonce regkey restart the unit on restart runs ur AutoIT which launches at the end a powershell script to then reset scaling to defaults and restart one last time. In other words automate your way around the issue not sure but sounds like that would work with windows set at any scaling factor.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing
×
×
  • Create New...