Sending keys and mouse clicks inside a VMWare VM

[griefman]

Hi everyone,

i am writing to you after a very long struggle i had while trying to figure out how to send a simple click inside a virtual machine running in vmware workstation 14.

i have an autoit script running on my host machine watching for the UAC prompt to be displayed in a running vm. Both the host and the guest OS are Windows 10. This script worked perfectly with virtual box. It recognized the UAC prompt and clicked inside and the UAC was accepted. Since i switched to VMware Workstation 14, the script no longer clicks inside the VM successfully. It acts as if it clicks, but it doesn't. 

I tried sending key combinations instead of a click, so that the VM can grab the input, but it also did not work. Every attempt that i made to send clicks or keys from the host inside the VM did not work. I tried using:

MouseClick

ControlClick

MouseMove

_WinAPI_Mouse_Event

_WinAPI_Keybd_Event

 

I also noticed that while the cursor moves to the target which has to be cilcked when my vmware worstation window is not focused, it even doesn't do that when i WinActivate the vmware workstation window first.

 

Did anyone experience such an issue, or maybe could give me a hint, what else i could use to send a key combination or a mouse click in a vmware workstation 14 pro guest window?

 

here is my code, which works with virtualbox:

 

expandcollapse popup

#AutoIt3Wrapper_Icon=".\uac.ico"
#include <ImageSearchSubrogated.au3>

FileInstall(".\ImageSearchDLL.dll", ".\ImageSearchDLL.dll", 0)
FileInstall(".\UAC_ginloSetup.bmp", ".\UAC_ginloSetup.bmp", 0)
FileInstall(".\UAC_Yes.bmp", ".\UAC_Yes.bmp", 0)

; set global variables for the coordinates, which should be delivered
global $x1 = 0, $y1 = 0
global $x2 = 0, $y2 = 0

global $counter1 = 0
global $counter2 = 0

global $sleep = 10000
global $smallSleep = 5000

; execute the script in a loop, so that it will hopefully recover from some unexpected errors
While $counter1 < 1
  checkForImage()
WEnd

#cs ------------

    Functions

#ce ------------

Func checkForImage()
  While $counter2 < 1
    ; search for the UAC in the entire screen - 2 screens supported
    local $searchUac = _ImageSearchArea('UAC_ginloSetup.bmp', 1, -2568, -8, 5136, 1440, $x1, $y1, 0)
    If $searchUac = 1 Then
      ; if the UAC was found search for the Yes button in a an area 200 x 200 from the middle of the found UAC image
      local $searchYes = _ImageSearchArea('UAC_Yes.bmp', 1, $x1, $y1, $x1 + 200, $y1 + 200, $x2, $y2, 0)
      If $searchYes = 1 Then
        ; if the Yes button was found click it and pause the script for $sleep seconds
        MouseClick("left", $x2, $y2, 1,0)
        Sleep($sleep)
      Else
        ; if the Yes button was not found retry from the beginning in $smallSleep seconds
        MsgBox(0, "UAC found error", "UAC was found but the 'Yes' button was not found. Script will retry in " & $smallSleep & " seconds.", $smallSleep)
      EndIf
        ; another way to accept the UAC - via shortcut
        ;Send("{TAB}{TAB}{TAB}{TAB}{TAB}{TAB}")
        ;Send("!y")
    Else
      ; if UAC was not found try again in $sleep seconds
      Sleep($sleep)
    EndIf
  WEnd
  ; if some error occured which expired the loop, pause the script for $sleep seconds
  MsgBox(0, "Error", "Some Error expired the timer and the script could not recover. The script will restart in " & $sleep & " seconds.", $sleep)
EndFunc

 

[JLogan3o13]

@griefman we're not big on helping to automate a way around the UAC, as it is a security feature there for a reason. As this sounds like your own machine running on VMware Workstation, why not temporarily disable UAC? Perhaps if you explain what you are trying to accomplish, we can offer more suggestions (looks like a software install of some sort?)

[griefman]

Hi @JLogan3o13,

Yes, it is a software install. I am a QA Engineer and am testing our installation. For what i think are understandable reasons, we want to make sure that in our tests we cover scenarios as close to the reality as possible and since nobody really deactivates UAC and it does matter to the installer, if it is deactivated or not, we decided to work around it in our tests. However in this particular use case, i think, it is not really about automating the UAC, it is just an example. It is about the sudden impossibility to automate any click or key send from the host to a guest system in vmware workstation 14 pro, which works perfectly on virtualbox. I really hope that somebody else gas experienced such an issue and has a workaround for it. 

Hope i was able to give you the answer, you were looking for

[Earthshine]

Run autoit script inside the vm. Why are you trying to send commands to a vm 

[griefman]

Hey @Earthshine,

There is already automation in the vm. The problem is that not every scenario in our test can be covered with tests inside the vm, so cases like uac or, clicking things, when windows is not runing, or similar scenarios which you can't automate in the guest, have to be automated ot the host. Usually with image recognition.

 

So running automation for these cases is not only not a valid solution, but it us simply impossible. It is a very unusual approach, i know. People would usually disable UAC, or use auto-login or similar things, but there tests, where you just can't do that, otherwise you are not covering everything or you will not be testing the product or the system which you want to support. It makes a lot of sense, when you consider certification, auditing, etc reasons.

Edited March 23, 2018 by griefman

[Bilgus]

It might that VMware blocks simulated mouse inputs two ideas,

1. run vnc software on the vmware and try interaction through that

2. create 2 small small scripts one on the host that presses a hotkey when mouse clicks and one on the guest the that when it gets a hotkey clicks the mouse at current position assuming your vm accepts keyboard input

[JLogan3o13]

VMware does indeed make it difficult to do mouse manipulation. If you absolutely must use mouseclicks, I would suggest something like this, which I have had to resort to in the past:

;pseudocode

WinActivate(<VMware Workstation Window>)
Send("^g") ;CTRL+g = Grab Input
<Do whatever else>

 

[Earthshine]

never mind

Edited March 23, 2018 by Earthshine

[JLogan3o13]

There have been a couple. But to my knowledge they both deal with the underlying registry to lower/deactivate it. Not sending keystrokes to bypass it.

[Earthshine]

i just read M23 say it can't be done. that's good enough for me.

however, with .net you can do it apparently

 

Edited March 23, 2018 by Earthshine

[griefman]

Hey everyone, 

 

thanks for the responses! 

@JLogan3o13: this is one of the first thing, which i tried, however as soon as the focus is on the vmware application, sending these keys does not work. i was only able to get the focus in the VM, when i selected this particular control and send the hotkeys for every menu item there. but afterwards i can't send anything in the VM. 

@Earthshine: i know about this solution, and since it is very severely changing the registry of the OS, i would prefer not to do it, unless there is no other way. Thanks!

I tried today to use VMware Workstation 12(the previous version) and it works like a charm. I thought that it might be the enhanced keyboard driver which vmware installs, but it is not the reason. I will stick with VMware Workstation 12 for now in the hope, that somebody will maybe shed some light what is going on and i will also post on the vmware forum.

 

Still keep it coming! I could not have exhausted every idea

[JLogan3o13]

Apologies, @griefman I missed that you were on v. 14. Yes, that version has given me a number of problems; I ended up backing down to 12.5.8

[Earthshine]

well, after all  UAC prompt doesn't open on the current desktop, but rather in a Secure Desktop, which suspends any currently open desktops. It's this way intentionally. However, Trusted processes running as the SYSTEM user can still interact with the Secure Desktop

Edited March 23, 2018 by Earthshine

[Earthshine]

there is a workaround I have read about. Using C# project using MSTest framework you can make agents and install those to your VMs and they listen on the network for instructions. You can make it so they run at SYSTEM user so it can talk to the Secure Desktop

Edited March 23, 2018 by Earthshine

[JLogan3o13]

That is exactly what you're doing when you run something through PSExec.

[Earthshine]

i was just thinking about using PSExec for this too... lol, thanks. I get his pain. I feel it too. I'm not going back to VMware 12, sorry. Onward and forward. I like to test all our stuff with it on as well so we don't get those gotcha's

the ONLY time our apps have trouble by our customers is 99.9% their machine is hosed or misconfigured. We never get any support tickets that things did not install. I mean they always blame me and when I look at all the logs, I see it's usually their system. One rare case a customer had to chcdsk /f and reboot because his drive was messed up. then repaired our install and it worked like a champ. An MSI log informed me of the drive error. It pays to look at ALL the logs in %TEMP% if they are willing to share them. My MSI installers also copy the MSI logs to a product folder under our %ProgramData% area for ease of access when they call us for support. I was going to copy all %temp% *.log and *.txt but I guess that would be illegal, even though I usually need it for the rare circumstances where our stuff does not run after install.

Edited March 23, 2018 by Earthshine

[griefman]

Hi everyone,

 

thanks a lot for your input! The solution with the agents is not a bad one, but too much of an overhead for something that simple. I hope that VMware will fix the issue soon and until then i am sticking with workstation 12 on the machine i need this or if there is a definitive need of 14, i can go over VNC(thanks @Bilgus). 

 

i started a discussion on the vmware forums and the only response i got so far is from somebody, who says that this is a security feature. I just don't agree with him. 

https://communities.vmware.com/message/2757661