Voice command JS issue


#1

Hi guys.

What I’m trying to do
So, I’m trying to create a simple Voice enabled prototype in Axure - yes!!
I’m using Annyang JS (https://www.talater.com/annyang/) which leverages the SpeechRecongition API in all modern webkits.

How far I got
I can get the browser mic listening, so the annyang JS is kicking in ok. But I cannot seem to pass back a (some kind of GlobalVar?) into Axure, in order to get it to Show a dynamic panel (this serves as my success trace based on the desired voice command - by saying “homebot search” > axure DP shows itself).

Just for clarity, the desired behaviour is:
01. User says “homebot search” (with Annyang JS processing the voice data)
02. Axure prototype shows success DP

I have tried a few approaches - such as referencing the dynamic panel (to be shown) via it’s data-label in the JS - but it didn’t work (maybe my syntax is wack? probably … )

I’ve attached the test prototype RP file, and I setup the Annyang JS inside of Axshare - but both the RP and a text version of the annyang JS are right here:

annyang-voice-test.rp (45.6 KB)

annyang-JS-axure.txt (523 Bytes)

Could anyone help me out a bit, please?
(I’m no JS ninjar :slight_smile: )

Thank you.
/d.


Voice control in Axure
Axure Plugins: Using JS to show a Dynamic Panel as a Lightbox
#2

@DaireM,

OMG! This is so cool. Last year I needed to test some voice interactions and got text-to-speech javascript working (more here: Instant text to speech ) but couldn’t get voice recognition working in Axure–I’ve been wanting to do this ever since.

And now… it works! IT’S ALIVE !

See this demo and .rp file for a demonstration:
https://l0q5r1.axshare.com
annyang-voice-test.rp (78.0 KB)

To better demonstrate, edit and debug the javascript code, I used a method from @Gregor outlined in this thread: Library for JavaScript, CSS and some little helps. This also allows it to work locally as well as from AxShare. You can take the javascript code from this and paste it in your AxShare plugin or separate .js file if you wish. You might also be interested in this method from @sam.hepworth : Improved Javascript Injection


Yes, I think so… Take a look at my .rp file and how I reference the “success” dynamic panel:
$axure('@success').show({});

I used this excellent resource to find out how to do tons of stuff in Axure via javasript, including how to show/hide widgets: Axure Javascript API

I can’t wait to dig in to the annyang stuff, play around with it and combine it with ResponsiveVoice.JS for a true voice interface prototype. Thanks!


Artificial Intelligence, Voice Recognition, Text-to-Speech and Speech-to-Text
#3

@mbc66
Yay! Thank you so much!
I ended up doing this, when it worked : )

FYI
I noticed a quirk - on the annyang JS side - non-standard phrases/frags (such as “homebot”) won’t be recognized/learnt (well, not with the simple code we have been using) - but I used “search my files” and it triggers it nicely.

Hey, and that Axure/JS API is the sheit!
Limitless.

If you ever in London, hit me up - beers on me.
Axure’s most dynamic component? The community.

Origato!
/d.


#4

Try saying “homebot” in a slow, Yankee drawl… maybe your best Trump impersonation. :rolling_eyes:
It works when I say it (and I was pretty surprised it did.)

I certainly hope I get the opportunity to go to London again soon. I can taste it already!


#5

This is great on desktop browsers, however on a mobile and axshare iOS and Android app, it does not. Is there a way to make this work in axhare iOS/
android?


#6

Hi Dan.

True!
Annyang JS doesn’t really work on mobile browsers just yet, although Chrome on Android seems to have lim support.

This is a crap workaround suggest, but you could show your mobile prototype on a desktop, placed inside a device? (Invision style)

all the best,
/d.


closed #7

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.