Android Visual Control by USB. NO SDK Needed, NO SCREEN or TOUCH Screen Needed.

[alien4u]

This is an small script to Control Android Devices by USB using adb, you can use it with phones with a Cracked Screen or not working Touch Screen.

What it Does:
- Show your Android Screen on your Desktop.
- Allow you to Swipe or Tap on the screen.
- Allow you to send BACK button event.
- Allow you to send WakeUp event.
- Allow you to move in List Menu using Swipe gestures or Mouse Wheel Forward or Backward.
- Refresh Screen with Wheel Click(Wheel Down)(To update your device screen on your desktop view).
- Allow you to unlock your Device with Pattern only 3 points pattern for now.

What it does not:
- Complex Unlock Pattern not supported yet I'm working on that.
- Complex Swipe gesture not supported yet, only straight lines.
- PC Keyboard to write SMS or Email not Supported yet, I'm working on that.
- Volume UP or Down not supported yet I will add it soon.
- No configuration interface or file yet, keys and actions are pre set on script code. I will change that.

What it Requires:
- In order to Compile the Script you need 3 files:
adb.exe, AdbWinApi.dll, AdbWinUsbApi.dll
- In order to use the script without these files or without compile it you need to copy that files to user Temp Dir, or windows Temp dir I mean @TempDir.
 

Keys Used and Notes:
- Left Click will tap on the screen.
- Keep Left Click down, move the mouse and release it with perform a Swipe operation.(Mouse Drag = Swipe).
- Right Click will go back.
- Enter on the keyboard will send WakeUp event(Power Button)
- Wheel Down will Refresh the screen on your desktop, also F5.(Take last pictures of your android device screen)
 

IMPORTANT: There is an $scale variable on the script that scale down your Phone screenshot image to a good resolution to show on your Desktop, I use 1.5 of scale factor because my phone resolution is 480x800 and my Desktop Screen is 1366x768 and that scale factor is good enough for me.(I will calculate that in the future to make it automatic).

IMPORTANT 2: If you feel is very slow feel free to decrease Sleep() values from 400 to 200, that will be faster but you will need to Refresh with Wheel Click after you perform certain actions.

Thanks and Credits:
UDFs\MouseControl.au3 Base on Low Level Mouse Hook by @_Kurt all credits to him.
UDFs\GUICtrlPicPNG.au3 Based on PNG work around by @UEZ from the help file, All credits to him.

Kind Regards
Alien.
 

AndroidControl.zip

Edited March 29, 2016 by alien4u
Adding Swipe Operation Instruction.

[soloprt]

cool , how can i integrat it inside an application am developing to help ime in my word?

[WormIT]

Using screen capture to show Android's screens and control will make your Android system lag.

[alien4u]

On 10/10/2016 at 5:22 AM, WormIT said:

Using screen capture to show Android's screens and control will make your Android system lag.

One of the purposes of this code is to learn and also access to an Android Device with a broken screen, is not really about efficiency.

[Lakes]

Use the TeamViewer UDF instead of screen cap?

[TheDcoder]

Bad idea to use Team Viewer as an intermediator, not only does it not work while the phone is not connected to the internet, you have to totally trust them with your device. I think there are better ways to do this... something like a VNC Server, I only know droidVNC but it does not work on my phone... But screen cap does work for now

[alien4u]

8 hours ago, Lakes said:

Use the TeamViewer UDF instead of screen cap?

@Lakes
The idea here is you will not need any software installed on your Android Device with the broken screen.
You only need USB Debugging Enable that all.

As an example, a family:
Man, Woman, Kid they all enable USB debugging and connect at least once to this computer or any computer they can run the Script.
6 months or a year from that moment, if the computer operating system was not re-installed or the phone system, if they get a not working touch-screen or display they will be able to connect to their phones and save data or do some things... regardless what they have installed on their phones.

Kind Regards
Alien. 

[Qwerty212]

Can wait to give it a try. Thanks for sharing it!!

[Parsix]

Hi, Thanks

screen capture is very slow,

have any solution for speedup?