Help with sending keyboard strokes

[orion]

Let me state first off that I am new to coding and auto edit. The last time I wrote a bit of code I was 7 and it was under the direction of my Dad and written in pascal for his Apple 2 e. Before asking what I am sure is a stupid question to you guys, let me say that I have invested around 16 hours or so seeking a solution to my problem. I have been looking around this forum for most of that time, at this point my eyes are starting to bleed and I am feeling rather desperate. What I am trying to do is to create an interactive display with a 42inch LCD ELO touchscreen and an Eee box. The software I am using is a program called "Xlobby". What I need to do is create a web browser page with a virtual keyboard as obviously there will be no physical keyboard in this application. I can create a keyboard in Xlobby, (or any other skinnable HTPC program) however this wouldn't be a true virtual keyboard as it will only interact with elements within Xlobby... say search a music database created within Xlobby. What I can do however is to create a page with individual images for each and every individual key on a keyboard, and then link each and every image to execute a specific file within windows. Hence I need to create an individual script for every key on a keyboard. I have tried various virtual keyboards, but these aren't delivering the results I am after. What I have discovered so far is that "winactivate" does nothing for me as I need to input data into text fields in internet explorer. I know that I have to somehow find the focus within a window where the cursor is and then prevent that formlosing focus, and then sending a keystroke to it. I have tried looking at the coding for the various virtual keyboards here, (as basically what I am after is a virtual keyboard that just sends one character)and I can't make heads or tails of them. Any help would be appreciated.

Thanks

This is what I have so far.... after three days of searching and trying to learn this stuff... yep I know... sad.

WinActivate("Untitled -") where " " is the handle of the application

Send ("A")

[lordicast]

Before you go and search the forum you should really search the help file! It's the best.

Search: ControlSetText,Send,Hotkey

Run("notepad.exe")
WinWait("Untitled -")
ControlSetText("Untitled -", "", "Edit1", "New Text Here" )

[orion]

Before you go and search the forum you should really search the help file! It's the best.

Search: ControlSetText,Send,Hotkey

Run("notepad.exe")
WinWait("Untitled -")
ControlSetText("Untitled -", "", "Edit1", "New Text Here" )

Yeah, I've looked in the help file, and went though a few of the tutorials that I downloaded. I really wouldn't have asked if I wasn't totally stumped. From what I can gather none of those are going to do it for me (of course I could be completely daft... not unlike me) Basically I'm trying to emulate a keyboard, that just sends one key. It's sending the key to where the cursor is on the screen that's got me at this point. Is your avitar, the bad guy from "Three Amigos" ? Loved that movie.

[evilertoaster]

Basically I'm trying to emulate a keyboard, that just sends one key. It's sending the key to where the cursor is on the screen that's got me at this point. Is your avitar, the bad guy from "Three Amigos" ? Loved that movie.

I may be having a hard time understanding what you're really going after...

Are you SURE you've seen these two posts (found by searching for 'virtual keyboard' in the forums) and that they do not do something like what you wanted?

http://www.autoitscript.com/forum/index.php?showtopic=98007

and the original- http://www.autoitscript.com/forum/index.php?showtopic=97867&st=0&p=704421&#entry704421

Other than maybe removing certain keys from being functional, can you maybe re-describe what you're going after? (maybe a picture would help )

[orion]

I may be having a hard time understanding what you're really going after...

Are you SURE you've seen these two posts (found by searching for 'virtual keyboard' in the forums) and that they do not do something like what you wanted?

http://www.autoitscript.com/forum/index.php?showtopic=98007

and the original- http://www.autoitscript.com/forum/index.php?showtopic=97867&st=0&p=704421&#entry704421

Other than maybe removing certain keys from being functional, can you maybe re-describe what you're going after? (maybe a picture would help )

Thank you all for your replies. Yes I did see those posts. I was looking at deconstructing them, but I do believe that there is no way of getting rid of the GUI and still have it all work. Basically what I want to do is mouse click on a field or within a program like I'm going to type, and then launch an EXE that will send a letter to where I'm trying to type to. All this is necessary due to limitations in the Xlobby program I am using. I'm beginning to suspect that this approach might just not be possible. Another route I could take is to somehow figure out how to "skin" a virtual keyboard. Is it possible with these virtual keyboards written here to use a PNG or JPEG as the images for the keys on the keypad? Again thanks for all the replies!

[evilertoaster]

...and then launch an EXE that will send a letter to where I'm trying to type to.

So the flow is :

1. User opens program.

2. User clicks field in program.

3. Xlobby automatically realizes that you clicked a field, and launches a designated .exe (a compiled autoit script).

4. The Autoit exe sends a key (so you'd need 26 sperate .exe's?) and closes.

Is that correct?

Is it possible with these virtual keyboards written here to use a PNG or JPEG as the images for the keys on the keypad?

Having not used them personally, I can't say for sure, but I would think that would quite easy. Edited November 3, 2009 by evilertoaster

[orion]

So the flow is :

1. User opens program.

2. User clicks field in program.

3. Xlobby automatically realizes that you clicked a field, and launches a designated .exe (a compiled autoit script).

4. The Autoit exe sends a key (so you'd need 26 sperate .exe's?) and closes.

Is that correct?

Having not used them personally, I can't say for sure, but I would think that would quite easy.

1. User opens program. (Yes)

2. User clicks field in program.(Yes)

3. Xlobby automatically realizes that you clicked a field, and launches a designated .exe (a compiled autoit script). (Xlobby automatically realizes that you clicked a field,(actually Xlobby creates a windowed version of IE and that's what your clicking on) however, you would then have to click on a picture of a key within Xlobby that is linked to launch a designated .exe)

4. The Autoit exe sends a key (so you'd need 26 sperate .exe's?) and closes. (Yes)

Skinning a virtual keyboard was my first option, but I found none that gives you that ability. If I can figure enough of this out and determine that is indeed possible, that would be the simplest thing to do.

Again, thanks for your replies. Even if I don't figure this out I appreciate the help!

[evilertoaster]

Before you put too much work into this... have you tried the built-in windows on screen keyboard (osk.exe)?

I'm having trouble discerning what the use case is for a keyboard that you DON'T want to use in xloppy but give functionality that the many standard on-screen keyboards don't already...

Best of luck!

[orion]

Before you put too much work into this... have you tried the built-in windows on screen keyboard (osk.exe)?

I'm having trouble discerning what the use case is for a keyboard that you DON'T want to use in xloppy but give functionality that the many standard on-screen keyboards don't already...

Best of luck!

Thank you again for all your help. Before I try an explain things any further, perhaps it would help things if you had a look here. The windows on screen keyboard won't work because A)It doesn't match the look of the existing keys in the skin and B ) I want something simpler and something that users can't move around. I found a script here that is getting me in the direction I need to go, but I can't seem to figure out how to get the shift key functions to work or and more importantly recreate the keyboard look I already have. Now all this is for a new project, that I'm modifying my existing skin for. Here is a pic of what I have so far in the web screen I am making. Link

Here is the code I have so far. It took me all day yesterday just to figure out how to add a Spacebar and a Backspace key.

expandcollapse popup

#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <WinAPI.au3>
#include <ButtonConstants.au3>

$WS_EX_NOACTIVATE = 0x08000000
$MA_NOACTIVATE = 3
$MA_NOACTIVATEANDEAT = 4

HotKeySet("{ESC}", "On_Exit")

Global $aKeys[48] = ["1", "2", "3", "4", "5", "6", "7", "8", "9", "0", "-", "=", _
                     "Q", "W", "E", "R", "T", "Y", "U", "I", "O", "P", "[", "]", _
                     "A", "S", "D", "F", "G", "H", "J", "K", "L", ";", "'", '"', _
                     "Z", "X", "C", "V", "B", "N", "M", ",", ".", "/", "back", "enter"]

; Create "keyboard" GUI
$hGUI = GUICreate("Test", 360, 150, 660, 615, $WS_POPUPWINDOW, BitOR($WS_EX_TOOLWINDOW, $WS_EX_TOPMOST, $WS_EX_NOACTIVATE))

$dummy1 = GUICtrlCreateDummy()
$iCount = 0
For $j = 0 To 3
    For $i = 0 To 11
        GUICtrlCreateButton("", $i * 30, $j * 30, 30, 30)
        GUICtrlSetData(-1, $aKeys[$iCount])
        $iCount += 1
;~         GUICtrlSetFont(-1, 10)
;~         GUICtrlSetBkColor(-1, 30000 + 2000 * ($i + 1) + 2000 * ($j + 1))
$Space = GUICtrlCreateButton("space", 60, 120, 240, 33, $WS_GROUP)
    Next
Next
$dummy2 = GUICtrlCreateDummy()
GUISetState()

GUIRegisterMsg($WM_MOUSEACTIVATE, 'WM_EVENTS')



While 1
    $msg = GUIGetMsg()
    Switch $msg
        Case $GUI_EVENT_CLOSE
            Exit
        Case $dummy1 To $dummy2
            Local $sText = ControlGetText($hGUI, "", $msg)
            ; Write key
            If $sText = "space" Then
                Send("{SPACE}")
            ElseIf $sText = "back" Then
                Send("{Backspace}")
            ElseIf $sText = "enter" Then
                Send("{ENTER}")
            Else
                Send($sText)
            EndIf
    EndSwitch
WEnd

Func WM_EVENTS($hWndGUI, $MsgID, $WParam, $LParam)
    Switch $hWndGUI
        Case $hGUI
            Switch $MsgID
                Case $WM_MOUSEACTIVATE
                    ; Check mouse position
                    Local $aMouse_Pos = GUIGetCursorInfo($hGUI)
                    If $aMouse_Pos[4] <> 0 Then
                        Local $word = _WinAPI_MakeLong($aMouse_Pos[4], $BN_CLICKED)
                        _SendMessage($hGUI, $WM_COMMAND, $word, GUICtrlGetHandle($aMouse_Pos[4]))
                    EndIf
                    Return $MA_NOACTIVATEANDEAT
            EndSwitch
    EndSwitch
    Return $GUI_RUNDEFMSG
EndFunc

Func On_Exit()
    Exit
EndFunc

Again, thanks for your help.

[evilertoaster]

I attached an example on skinning the buttons. I have no skill in actual skinning so the buttons in this case are a exact rip off (copied via MS Paint) of the screen shot I saw in your post. This means that you'd have to skin each button with it's own backround image. I'm not sure if it's the best way to go about it, and you may want to play around with GUICtrlSetBkColor(),GUICtrlSetFont or for advanced skinning, GUICtrlSetGraphic(). But anyways, it may be something to go on.

I only skinned a few buttons to show concept and I had to resize them to match the size of the button images (if you scale on you'd want to scale the other).

Skinner.zip

Edited November 7, 2009 by evilertoaster

[orion]

I attached an example on skinning the buttons. I have no skill in actual skinning so the buttons in this case are a exact rip off (copied via MS Paint) of the screen shot I saw in your post. This means that you'd have to skin each button with it's own backround image. I'm not sure if it's the best way to go about it, and you may want to play around with GUICtrlSetBkColor(),GUICtrlSetFont or for advanced skinning, GUICtrlSetGraphic(). But anyways, it may be something to go on.

I only skinned a few buttons to show concept and I had to resize them to match the size of the button images (if you scale on you'd want to scale the other).

Thanks! This would be exactly the kind of info I need! I don't see the attachment though, where do you find those on these posts? I take it that the GUICtrlSetGraphic()is the command that gets the image. I was trying to play around with GUICtrlCreatePic()but having no luck with that.What is the difference between the two anyway? Do the imagaes have to be bitmaps, jpegs or pngs? Anyway I will do a search for that and see if I can figure it out!

Thanks again!

[orion]

I attached an example on skinning the buttons. I have no skill in actual skinning so the buttons in this case are a exact rip off (copied via MS Paint) of the screen shot I saw in your post. This means that you'd have to skin each button with it's own backround image. I'm not sure if it's the best way to go about it, and you may want to play around with GUICtrlSetBkColor(),GUICtrlSetFont or for advanced skinning, GUICtrlSetGraphic(). But anyways, it may be something to go on.

I only skinned a few buttons to show concept and I had to resize them to match the size of the button images (if you scale on you'd want to scale the other).

OK I would really like to see your "skinning" method as I think that might be a lot simpler than what I just figured out. Apparently I can set icon images to buttons doing something like this

$Space = GUICtrlCreateButton("space", 60, 120, 240, 33, $BS_ICON,$WS_GROUP)

GUICtrlSetImage(-1, "shell32.dll", 22)

However this requires creating a .dill with the icon files in it that I would have to use. I have no idea how to go about creating such a thing, but I did find that Autoit can't do that. Is there a free program to create a .dll of icon files????

Getting closer!

expandcollapse popup

#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <WinAPI.au3>
#include <ButtonConstants.au3>

$WS_EX_NOACTIVATE = 0x08000000
$MA_NOACTIVATE = 3
$MA_NOACTIVATEANDEAT = 4

HotKeySet("{ESC}", "On_Exit")

Global $aKeys[48] = ["1", "2", "3", "4", "5", "6", "7", "8", "9", "0", "-", "=", _
                     "Q", "W", "E", "R", "T", "Y", "U", "I", "O", "P", "[", "]", _
                     "A", "S", "D", "F", "G", "H", "J", "K", "L", ";", "'", '"', _
                     "Z", "X", "C", "V", "B", "N", "M", ",", ".", "/", "back", "enter"]

; Create "keyboard" GUI
$hGUI = GUICreate("Test", 360, 150, 660, 615, $WS_POPUPWINDOW, BitOR($WS_EX_TOOLWINDOW, $WS_EX_TOPMOST, $WS_EX_NOACTIVATE))

$dummy1 = GUICtrlCreateDummy()
$iCount = 0
For $j = 0 To 3
    For $i = 0 To 11
        GUICtrlCreateButton("", $i * 30, $j * 30, 30, 30)
        GUICtrlSetData(-1, $aKeys[$iCount])
        $iCount += 1
;~         GUICtrlSetFont(-1, 10)
;~         GUICtrlSetBkColor(-1, 30000 + 2000 * ($i + 1) + 2000 * ($j + 1))
$Space = GUICtrlCreateButton("space", 60, 120, 240, 33, $BS_ICON,$WS_GROUP)
         GUICtrlSetImage(-1, "shell32.dll", 22)
    Next
Next
$dummy2 = GUICtrlCreateDummy()
GUISetState()

GUIRegisterMsg($WM_MOUSEACTIVATE, 'WM_EVENTS')



While 1
    $msg = GUIGetMsg()
    Switch $msg
        Case $GUI_EVENT_CLOSE
            Exit
        Case $dummy1 To $dummy2
            Local $sText = ControlGetText($hGUI, "", $msg)
            ; Write key
            If $sText = "space" Then
                Send("{SPACE}")
            ElseIf $sText = "back" Then
                Send("{Backspace}")
            ElseIf $sText = "enter" Then
                Send("{ENTER}")
            Else
                Send($sText)
            EndIf
    EndSwitch
WEnd

Func WM_EVENTS($hWndGUI, $MsgID, $WParam, $LParam)
    Switch $hWndGUI
        Case $hGUI
            Switch $MsgID
                Case $WM_MOUSEACTIVATE
                    ; Check mouse position
                    Local $aMouse_Pos = GUIGetCursorInfo($hGUI)
                    If $aMouse_Pos[4] <> 0 Then
                        Local $word = _WinAPI_MakeLong($aMouse_Pos[4], $BN_CLICKED)
                        _SendMessage($hGUI, $WM_COMMAND, $word, GUICtrlGetHandle($aMouse_Pos[4]))
                    EndIf
                    Return $MA_NOACTIVATEANDEAT
            EndSwitch
    EndSwitch
    Return $GUI_RUNDEFMSG
EndFunc

Func On_Exit()
    Exit
EndFunc

[evilertoaster]

Sorry I failed to use the new forum UI correctly and didn't actually 'commit' my attachment selection. I should be available now in the last post.

[orion]

Sorry I failed to use the new forum UI correctly and didn't actually 'commit' my attachment selection. I should be available now in the last post.

Thanks! I will take a look at that and see how it's done!

[orion]

Sorry I failed to use the new forum UI correctly and didn't actually 'commit' my attachment selection. I should be available now in the last post.

Using your example, I managed to get a lot done! The only thing I would like to figure out now is how to get the shift buttons to work. I would like to create another button, with everything in lower case and then somehow have the shift buttons toggle between the two. Does anyone know how to go about doing this? I looked at the keyboard mentioned earlier, but I honestly can't make heads or tails of it. Thanks for all your help evilertoaster, I couldn't have figured this out without your insight, and I really appreciate it!

Orion Keyboard.7z

Edited November 15, 2009 by orion

[orion]

I managed to make my keyboard and I got it running how I wanted. I'm sure there are better ways of doing certain things in it, but for my first attempt at writing a program, I don't think I did so bad! Thanks again for your help,evilertoaster!

Orion Keyboard

Edited November 20, 2009 by orion

[Bert]

Nice work!