SAPIListBox (Speech Recognition) UDF

[seangriffin]

This UDF allows users to create and control a Microsoft Speech API ListBox (SAPIListBox) within an AutoIT GUI. It requires the Microsoft Speech SDK 5.1 (a free download from Microsoft).

The SAPIListBox is a simple and effective way to add voice recognition / speech recogition to your AutoIT script. I personally have found this method of speech recognition to be even more accurate than via the other tools and methods within the SAPI SDK.

The SAPIListBox looks like any other traditional ListBox (ie. AutoIT's "GUIListBox"). However it is voice sensitive. When the listbox is active, it listens to your microphone, and if a spoken word matches an item within the listbox (via Microsoft's Speech API), then that item is automatically selected. My UDF also includes the ability to detect this change in the listbox, so a script can then perform a particular action once the word is spoken.

I'd used the SAPIListBox with great success in a MS Access application previously. I thought I'd port it to AutoIT just for the fun of it

If you find that the speech recognition capability of this UDF is not that accurate, then go to your Windows Control Panel and open the "Speech" applet. A "Speech Properties" window will be displayed. Click the "Train Profile..." button to train the Speech API to understand your own voice better. Once you have finished this, retry this UDF. You should notice an improvement in speech recognition.

REQUIREMENTS:

AutoIt3 3.2 or higher

the Microsoft Speech SDK 5.1 (a 68 MB free download)

LIST OF FUNCTIONS:

_GUICtrlSAPIListBox_Create($left, $top, $width, $height, $enable_speech = True, $enable_listbox = True, $hide_listbox = False, $items = False)

_GUICtrlSAPIListBox_EnableSpeech($SAPIListBox, $toggle)

_GUICtrlSAPIListBox_Enable($SAPIListBox, $toggle)

_GUICtrlSAPIListBox_Hide($SAPIListBox, $toggle)

_GUICtrlSAPIListBox_AddString($SAPIListBox, $text)

_GUICtrlSAPIListBox_DeleteString($SAPIListBox, $index)

_GUICtrlSAPIListBox_GetSel($SAPIListBox, $index)

_GUICtrlSAPIListBox_InsertString($SAPIListBox, $text, $index)

_GUICtrlSAPIListBox_SetToolTip($SAPIListBox, $text)

_GUICtrlSAPIListBox_AddString($SAPIListBox, $array)

_GUICtrlSAPIListBox_GetCount($SAPIListBox)

_GUICtrlSAPIListBox_GetText($SAPIListBox, $index = -1)

_GUICtrlSAPIListBox_GetNewIndex($SAPIListBox)

_GUICtrlSAPIListBox_Refresh($SAPIListBox)

_GUICtrlSAPIListBox_ResetContent($SAPIListBox)

_GUICtrlSAPIListBox_CurSelChanged($SAPIListBox)

SAPI_SpeechProperties()

EXAMPLES:

You must have downloaded and installed the Microsoft SAPI 5.1 SDK (in the Requirements section above) prior to running these examples, or using the UDF.

The following example demonstrates the full set of functions in the UDF. When the script is run, a GUI is displayed. The user should be able to immediately begin talking into the microphone, reciting any of the words listed in the SAPIListBox. If a word is spoken correctly, that word should get automatically selected in the listbox. The bottom half of the GUI will display changes (as the user speaks the correct words), and also allow the user to add and remove words, and adjust other listbox properties. Clicking the "Speech Properties" button will display the settings for SAPI, and allow users to train the speech engine to better recognise their own voice.

SAPIListBox control Test.au3

The following example is of a voice activated application launcher. When the script is run, a GUI is displayed. The user should be able to immediately begin talking into the microphone, reciting any of the words listed in the SAPIListBox. If a word is spoken correctly, the corresponding application should start. Clicking the "Speech Properties" button will display the settings for SAPI, and allow users to train the speech engine to better recognise their own voice. Saying the word "close" should close the active window (as launched with a previous voice command).

voice activate app launcher.au3

DOWNLOAD:

Latest Version - v0.1 (21/05/10)

SAPIListBox.au3

Edited May 21, 2010 by seangriffin

[gr8BigGeek]

I downloaded and installed Microsoft Speech SDK 5.1. I also downloaded your "voice activated app launcher.au3" file. Whenever I go to open/run the program a GUI "SAPIListBox control Test - Error!" Message pops up?

Boy I was really hoping to find something that would allow me to run programs on voice command. Is there anything I can do to fix this problem?

I am running Win7 64 bit (so all files are installed to C:\Program Files (x86) does that make a difference?

anyway, please respond to me if you can that would be wonderful if I could get this running.

Matt~

[JRowe]

You could try my simple SAPI demo for a more generic setup. Setting the grammar (word list) is a tad more complicated, but the essence is there, as well as the links to docs to help you on your way.

http://www.autoitscript.com/forum/index.php?showtopic=115203

Edited August 12, 2010 by JRowe

[gr8BigGeek]

Thank you very much for responding. I did follow that link. I copied the code that popped up in the window there and ran the program only it didn't do anything. It just sat there down by the clock. I took out the sleep (5000) command to see if that is what was taking so long but it didn't work then either. I'm totally lost as to what I'm doing wrong with your code.

I know you are so much smarter than I am; I'm just missing something somewhere. I've been looking for a program that will allow me to speak a command and run a given program. I have dragon 10.1 (for dictation, but it doesn't have the option for opening programs, I need to pay them more for an upgrade). I've also tried Tazti, from Tazti.com and that has a limit of 35 commands so I was real bummed out about that. I need something that will allow me to keep building and building a large database of commands (probably in the thousands).

Anyway, if I'm not smart enough to get this program of yours running I can always just try something else.

Again, you have been very kind for responding. Thank you so much for everything.

Matt

[Spiff59]

Our Dragon Naturally Speaking 9.51 SP1 Medical allows for creating fairly powerful xml scripts.

Am pretty sure that applies to all versions. What are you running? Preferred? Pro?

Googling a phrase like "dragon naturally launch application from script" gets a lot of hits...

[gr8BigGeek]

I have nuanced Dragon 10.1. I believe that this program would be what I would need to open and run programs. however, I am finding now that nuance Dragon 10.1 has many limitations. One of which is that it does not allow you to open numerous applications if not just one application by using VoiceCommands. I believe the software was designed to do nothing more than to dictate. I have been seeking a product much like Tazti because it allows for custom commands. The bad part about that is that Tazti's custom commands limitation is 35 maximum. I am looking for a product that allows me to build an endless database of custom commands. Do you know of any products that don't cost $300 (free preferably)?

Funny enough this entire entry was entered into this form using Dragon 10.1. I am not looking for primarily dictation software as I am for software to run certain programs based on certain VoiceCommands.

I look forward to hearing back from you.

Matt

[JRowe]

Ok, run my script from inside Scite, and speak into your microphone. It will output everything you say into the lower pane (the console.) To do what you want, you're going to have to learn how to use DllCall and how to use MSDN to convert functions to AutoIt. I have an inkling that you're also going to need to spend time learning some AutoIt basics, to get yourself a foundation to start from.

You're trying to pole vault before you can walk, here. Spend the time learning and figuring out how to use this script (SAPI ListBox) or my own. Hit up the general help and support forum if you run into issues, and we'll be more than happy to help you on your way.

[gr8BigGeek]

I've written a lot of coding using Autoit before. I've just never attempted anything like voice activation. I understand most of your code but just not all of it. I downloaded the three .au3 files (SAPIListBox.au3, voice activation app launcher.au3 and SAPIListBox control Test.au3). When I click on "voice activation app launcher I get the error "SAPIListBox control Test - Error! Could not create a SAPI LISTBox control. Ensure you have installed the Microsoft Speech SDK5.1 (Speech SDK51.exe) and try again." The only thing is; I have downloaded and installed that.

What's the deali~o?

ps - I attached screenshots of the error message and of my program listing indicating that I have installed SDK5.1 already. What else am I needing here?

[JRowe]

The object isn't on your system. It's dependent on your OS version, and I think Windows 7 just doesnt have it.

I ran into the same problem, so I went and figured out how to do it on my own, thus the simple speech recognition demo - what you're going to have to do is read up on how the Microsoft SAPI works, following the MSDN links, and then basically do this:

Create a grammar, a listener, and a recognizer - the grammar is the list of words, the listener is the audio input stream, the recognizer is the output. You can use those to create your own trainer (recording, then processing the input stream,) which would be good for an application launcher, because you'll want the user to say the name of a program a few times in order to recognize it. At any rate, you'll need to do a lot of reading - I've done a lot of reading and still haven't figured out how to easily do what you're attempting.

I'd recommend SAPI over Dragon Naturally Speaking any day, simply because it's just as good, and it's free for you to use. DNS will cost you money, and gain you basically nothing - the two technologies are pretty much the same.

[playlet]

---

Edited August 18, 2016 by playlet

[gr8BigGeek]

Hi, seangriffin

Thank you for this script, it's the best thing I tried since Vista came out.

Now I can finally use Speech Recognition on my XP Home. Everything I used so far was either bad at recognizing words or something I wasn't looking for.

Is there any way to make it work like on Vista or 7 (say what you see)?

If anyone finds the code that will allow this thing to work on windows 7 please let me know. I am using e-speaking and it's like asking a grandpa to do things for you. Half the time he's just grumpy and not listening and the other half of the time he does the wrong thing (opens the wrong program out of spite) probably because he was busy soiling himself. e-speaking you may mean well, so sorry about body-slamming ya'll like e-speaking is a Belvedere but come on folks. e-speaking piggy backs off of Win7's built in voice recognition technology so you have two programs you are juggling. You have to manually mute them both, and then they seem they are almost fighting you sometimes (which is where the grumpy old man butler analogy comes in).

You can't even speak a command to mute the mic, and then speak another to unmute it? My two year-old comes in hollering and I run scrambling to the keyboard and mouse to stop it from listening so it won't start opening and closing a bunch of windows and programs. This is so War Games with Matthew Broderick! Come on people what happened to Minority Report with Tom Cruise? That flick came out in 2002 (8 friggin' years ago) and we are still suffering from Carpal Tunnel Syndrome because we are STILL using the same keyboard that was designed in 1866 by Christopher Sholes!!

Computers are tools that are here to help us not hurt us. They should be our assistants and we should have WAY better interaction with them than this mouse/(1866)keyboard business.

"Computer open my jazz music", then your smooth contemporary jazz begins playing softly in the background. "Computer, dim lights to 45%" the lights dim...how hard is this???? I feel like Fred Flintstone dreaming of a cell phone so I can quickly shoot Wilma a text message back at the cave that I'll be running late because Mr. Slate is making me stay overtime at the Slate Rock and Gravel Company.

I went through the trouble of getting Nuance preferred Dragon Naturally Speaking, English, Version 10.10.000.078; and now ($799.00 dollars later) I receive and email back from Nuance help stating that, The majority of commands to actually control the computer are only included in the Professional Edition of Dragon". Wow, that would have been nice to know 800 bucks ago. So now I'm seeking a way of doing this for free. Are all of the options of having an (unlimited amount of custom voice commands) sung to the tune of $569.00 (for the Pro version of Dragon)?

Has anyone been able to hack Tazti? It has a new release coming out at the 1st of next month. They will allow 75 commands, but I'm happy using the old version (with the limit of 35 custom commands) so long as a hack would allow it to have unlimited custom commands instead of just 35 because it's truly the best thing out there.

[dan6789]

I downloaded and installed Microsoft Speech SDK 5.1. I also downloaded your "voice activated app launcher.au3" file. Whenever I go to open/run the program a GUI "SAPIListBox control Test - Error!" Message pops up?

Boy I was really hoping to find something that would allow me to run programs on voice command. Is there anything I can do to fix this problem?

I am running Win7 64 bit (so all files are installed to C:\Program Files (x86) does that make a difference?

anyway, please respond to me if you can that would be wonderful if I could get this running.

Matt~

You likely need to 'register' the 'SAPI51ListBox.ocx' active X control on your machine.

Start->Run->regsvr32 C:\Program Files\Microsoft Speech SDK 5.1\Bin\SAPI51ListBox.ocx

It will ask you 'open with', just pick notepad, doesn't matter, then close it.

Try the autoit script again.

[manuel6000]

i read a tutorial for this udf and i get the error

>"C:\Programme\AutoIt3\SciTE\..\autoit3.exe" /ErrorStdOut "C:\ord\445.au3"

C:\ord\SAPIListBox.au3 (96) : ==> Variable must be of type "Object".:

$SAPIListBox.SpeechEnabled = $toggle

$SAPIListBox^ ERROR

>Exit code: 1 Time: 0.464

#include <GuiConstantsEx.au3>
#include <SAPIListBox.au3>
dim $msg, $wortbox_array[5] = ["Up","Down","Left","Right","Wait"]
$fester = GUICreate("Beispiel zur Spracherkennung",400,300)
$wortbox = _GUICtrlSAPIListBox_Create(10, 10, 380, 280)
GUISetState()
_GUICtrlSAPIListBox_EnableSpeech($wortbox, 1)
_GUICtrlSAPIListBox_AddArray($wortbox, $wortbox_array)
While 1
$msg = GUIGetMsg()
$mausposition = MouseGetPos()

if _GUICtrlSAPIListBox_CurSelChanged($wortbox) = True Then
 
  if StringCompare(_GUICtrlSAPIListBox_GetText($wortbox), "Up") = 0 Then
   MouseMove($mausposition[0],$mausposition[1]-50)
  EndIf
 
  if StringCompare(_GUICtrlSAPIListBox_GetText($wortbox), "Down") = 0 Then
   MouseMove($mausposition[0],$mausposition[1]+50)
  EndIf
 
  if StringCompare(_GUICtrlSAPIListBox_GetText($wortbox), "Left") = 0 Then
   MouseMove($mausposition[0]-50,$mausposition[1])
  EndIf
 
  if StringCompare(_GUICtrlSAPIListBox_GetText($wortbox), "Right") = 0 Then
   MouseMove($mausposition[0]+50,$mausposition[1])
  EndIf
EndIf

switch $msg
case $GUI_Event_Close
  ExitLoop
EndSwitch
WEnd

Edited September 13, 2011 by manuel6000

[Damein]

For those who are having trouble with this here is what you need to do:

(This has only been proven to work on Vista, I do not own Windows 7 to test)

• Download the SDK provided in the first link.
• Install the SDK
• Start Menu > Accessories > Run Command Prompt in Elevated mode (If you have UAC on)
• Type in regsvr32 /u SAPI15ListBox.ocx (This un registers the OCX)
• Type in regsvr32 SAPI15ListBox.ocx (This re-registers the OCX)

Now you should have no problem running the scripts provided. This is a little one I made myself to do what I was looking for

#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <SAPIListBox.au3>
HotKeySet("!{x}", "_Close")
$iTunesApp = ObjCreate("iTunes.Application")
Const $num_apps = 4
Dim $msg
Dim $SpeechListBox1_item[$num_apps] = ["play", "next", "previous", "Default"]
Dim $SpeechListBox1_cmd[$num_apps] = ["Play", "Next", "Previous", "Default"]
; Setup the GUI
GUICreate("Voice Controlled iTunes", 100, 200)
$SpeechListBox1 = _GUICtrlSAPIListBox_Create(10, 40, 80, 150, True, False, False, $SpeechListBox1_item)
If $SpeechListBox1 = False Then
MsgBox((262144 + 16), "SAPIListBox control Test - Error!", "Could not create a SAPI ListBox control." & @CRLF & "Ensure you have installed the" & @CRLF & "Microsoft Speech SDK 5.1 (SpeechSDK51.exe) and try again")
Exit
EndIf
; Display the GUI
GUISetState(@SW_HIDE, -1)
Func _Close()
Exit
EndFunc
; Main loop
While 1
If _GUICtrlSAPIListBox_CurSelChanged($SpeechListBox1) = True Then
   If $SpeechListBox1_cmd[_GUICtrlSAPIListBox_GetCurSel($SpeechListBox1)] = "play" Then
    $iTunesApp.PlayPause
    ElseIf $SpeechListBox1_cmd[_GUICtrlSAPIListBox_GetCurSel($SpeechListBox1)] = "next" Then
    $iTunesApp.NextTrack
   ElseIf $SpeechListBox1_cmd[_GUICtrlSAPIListBox_GetCurSel($SpeechListBox1)] = "previous" Then
    $iTunesApp.PreviousTrackC
    EndIf
  EndIf
WEnd

You need the SAPIListBox.au3 provided in the first post to run it.

This will allow you to play/skip/go back on iTunes. If you say "play" and then want to pause before skipping or going back you have to say default first. I couldn't figure out how to allow it to select the same item twice.

Well, let me know if you guys have any other problems.

[TrailFox]

Is there any chance to add pronounciation to the words in the list box. i have german language pack and it just wont recognize any engish words in the list box.

[TrailFox]

Okay, here is the most important issue on the listbox.

How can we disable the integrated WSR Commands. They are so annoying.

Plz HELP

ty

[KabianLeng]

Hello there,

I've been using the Microsoft SAPI Listbox for my Final Year Project at the moment. The problem that I have been experiencing is that I would like to have the selected commands in the listbox displayed on a label. By the way I have also changed the GUI of the default listbox. But I have managed to use If Then Else Function but the problem I am facing is that I assign them based on index numbers but only the first index on the list appears the caption the rest does not. Do you have any idea on how to do it?