System.Speech

MSDN July: Text Services Framework

Jeff Certain — Wed, 13 Jun 2007 18:42:27 GMT

Most of what I do with speech recognition centers around very limited grammars, designed to improve recognition accuracy in noisy environments. We also have a requirement to support XP, since most of our users are on Tablet PCs running XP SP2.

The latest issue of MSDN Magazine (not online yet) has an interesting article by Eric Brown on using the Text Services Framework to provide full-featured dictation in apps running on Vista. Among other features, TFS looks like an interesting approach to the problem of allowing your users to train the recognition engine as they dictate -- a feature which is of great interest to many developers, but which I have very little experience with at the point.

Make sure the speech API is installed

Jeff Certain — Tue, 12 Jun 2007 15:47:44 GMT

This should go without saying, but I've had recent problems with people not understanding this concept.

If you're using System.Speech in Windows XP, you need to make sure that the speech recognition engine from SAPI 5.1 is installed.

If you're deploying an application, make sure you include the SAPI 5.1 merge module in your installer.

[Update] June issue of VIsual Studio Magazine

Jeff Certain — Tue, 12 Jun 2007 04:09:07 GMT

Well, it's official -- the June issue of Visual Studio Magazine is out. I have the cover article in this issue, in which I discuss System.Speech, showing the use of dynamically-generated custom grammers for data collection applications.

Front Range Code Camp Post Mortem

Jeff Certain — Tue, 22 May 2007 00:50:26 GMT

Well, the Front Range Code Camp was this past Saturday. I gave two talks -- threading and speech recognition.

The threading talk (code samples and slide deck here) was pretty well attended -- about 12 people, which was a good third of the attendees. The strangest part was that there were only two people in the room that were .NET developers, and one of those was still using 1.1 so my BackgroundWorker examples weren't all that much use to him. The rest were Ruby and Java developers, plus a small cadre of profs from Metro College.

That only set the stage for my speech recognition talk (code and slide deck here), which only had three people in the audience... one dev each for Ruby, Java and C++. Considering that the talk was entirely focused on System.Speech, and writeen using VB .NET, the code samples were again somewhat less than relevant.

"Is Speech Recognition Finally Good Enough?"

Jeff Certain — Mon, 21 May 2007 23:25:01 GMT

An interesting article from Computer World.

Among other issues, the article explores some of the reasons why most people don't use SR. The biggest reason is the amount of training -- both of the user and the engine -- to get to 98% recognition accuracy, which is the threshold at which users feel that SR is actually useful.

"Without help on those issues, about three-fourths of the people who attempt to use SR eventually put it aside and go back to keyboarding, Parks said, and even among those with training the rate is about 20%."

Dynamically Building Custom Grammars

Jeff Certain — Thu, 17 May 2007 15:34:28 GMT

One of my favorite features of System.Speech is the ability to generate custom grammars dynamically. As I've mentioned earler, this took about 50 lines of code in SAPI 5.1. The System.Speech team has included some very nice coarse-grained methods to accomplish this same task in just four lines of code:

Protected WithEvents reco As SpeechRecognizer
Public Sub LoadGrammar(ByVal options As List(Of String))
    Dim choices As New Choices(options.ToArray)
    Dim gb As New GrammarBuilder(choices)
    Dim g As New Grammar(gb)
    reco.LoadGrammar(g)
End Sub

System.Speech -- What is it?

Jeff Certain — Sun, 13 May 2007 18:37:19 GMT

I've been lurking on the fringes of speech recognition for quite some time. My first exposure required installation of a dedicated ISA sound card. Yes, it was that long ago.

As you can imagine, that experience was somewhat less than exhilarating.

My next go round was with Dragon in 1999 (I think they were owned by Corel at that point). Although the engine took a while to train, and didn't do very well on some of the words I tried to feed it ("Kosovo" comes to mind). However, once you corrected what it thought you had said, the engine worked splendidly. At the time, I was still in the Canadian Forces (Army), so much of what I was dictating invovled acronyms and jargon.

Fast-forward to the summer of 2005. The company I work for (which shall remain nameless for the time being) proposed a project to speech-enable an application in a very noisy environment. I spent a few days researching the various technologies available. Most were either intended for embedded systems, required a lot of specialized knowledge, were expensive, had a C API with no .NET wrapper... or some combination of the above.

Then I found SAPI 5.1 -- the speech recognition API for the engine that ships as part of XP. Talk about an awesome tool!

No specialized knowledge -- just give it a list of English phrases in an XML file and it would recognize them all. I even tried it with such esoterica as "Boba Fett" and "Obi-Wan Kenobi". Someone on the SAPI team must be a serious Star Wars geek, because those worked the first time.

There's lots of other great features in SAPI 5.1 -- but the one my boss liked best was the price... free. I won't dwell on the other advantages, since this is deprecated technology. And, who'd want to use it, now that System.Speech is released?

Of course, nothing is perfect. The documentation was pretty much useless to me. And, not even Google pulled very much from the web.

So let's talk about System.Speech...

For those of you who haven't found out about this namespace, it's actually buried in WPF. One namespace each for recognition and text-to-speech. System.Speech sits on top of the SAPI shipped with your OS -- either 5.1 for XP or 5.3 for Vista.

The team at Microsoft has done a superb job of making speech recognition easy. There's a SpeechRecognizer class that exposes that functionality you need to do basic speech recognition. If you're feeling really masochistic, there's also the RecognitionEngine class. I like to describe this class a SpeechRecognizer's ugly bigger brother -- who will gladly give you more than enough rope to hang yourself.

As a developer, the part I appreciate most is the inclusion of methods that make my life easy. One of the most graphic examples is in the creation of a custom grammar at run-time. With SAPI 5.1, this took about 50 lines of code -- generating an XML file on the fly, saving it to disk, and loading it into the engine. (Okay, maybe there are better ways to do this -- this was the one I could make work.)

In System.Speech, this takes four lines of code. With no disk IO operations. This is the sort of improvement that makes my life as a developer sooo much easier.

If you've ever been interested in speech-enabling your applications, you've really got to look into System.Speech. If you want a hand getting started, you'll just have to wait until the June issue of Visual Studio Magazine (hint, hint).