Hi Christopher, Now it works in my ESL app though I am just able to do one dialogue ( I need to add the event catching for furthur dialgoues). I have a couple of questions here: 1. In the first try, my Nuance server was able to be accessed somehow (FS says the MRCP is not responding in 5000ms, something like that), then FS says: [WARNING] rtsp_client.c:386 () Failed to Connect to RTSP Server 99.185.85.31:554, later FS says: [ERR] mod_unimrcp.c:1860 (TTS-6) SYNTHESIZER channel error! [ERR] switch_ivr_play_say.c:2439 Invalid TTS module! The SYNTHESIZER channel error and Invalid TTS module error are obvious. What I don't understand is why it went to this stange address: 99.185.85.31:554? 2. I specified TTS engine in play_and_detect_speech as "say:unimrcp:nuance5-mrcp1-1: the text to speak" It works though I didn't specify the TTS voice. How do I specify the TTS voice? In the mrcp profile (how?)? or something like: "say:unimrcp:nuance5-mrcp1-1:Serena: the text to speak" (this seems not right.) 3. The barge-in works well, thanks!. Is the barge-in configurable? In some scenarios, we might not allow barge-in. 4. How could I get the text which has spoken to the user when barge-in occurs? Or Could I get the time when barge-in occurs? If I know the barge-in time and rough totale time for the whole text to be spoken I can figure out the spoken text by manually checking the recorded audio file later, which would be painful. 5. when I use "speak" and "detect_speech" apps in ESL, I can catch event: DETECTED_SPEECH and speech-type: begin-speaking and "detected-speech", then I do the recognition results processing. The new app play_and_detect_speech seems not generate these events any more. The way that I can think of to get the results is to catch event:CHANNEL_EXECUTE_COMPLETE then check if variable_current_application=play_and_detect_speech, then get the results from variable_detect_speech_result. Is this the proper way to get the results in ESL app? Or will play_and_detect_speech later on be consistent with detect_speech in term of ASR events? 6. I'd like to set start-input-timers=false in the initial request then start the recognition timers (start-input-timers=true) after the TTS finishes. How possibly could I do this? Apologies for so many questions! Cheers, Xing On 16/11/11 13:10, Christopher Rienzo wrote:
You are missing the tts_engine. Set the tts_engine channel variable to "unimrcp:nuance5-mrcp1-1" or set the default tts profile to nuance5-mrcp1-1 in the unimrcp.conf.xml file and use "say:unimrcp:text to speak"
On Wed, Nov 16, 2011 at 7:36 AM, Liu, Xingkun <X.Liu@hw.ac.uk <mailto:X.Liu@hw.ac.uk>> wrote:
Great work, thanks Christopher!
I've quick tested it from the dialplan based on the example. It works on my setup.
Now I am trying to put it into my Java ESL app, FS complains:
switch_ivr_play_say.c:1167 Invalid Args
A piece of codes I am using:
SendMsg msg = new SendMsg(); msg.addCallCommand("execute"); msg.addExecuteAppName("play_and_detect_speech"); String arg = "say:" + utt + " detect:unimrcp:nuance5-mrcp1-1 {start-input-timers=false,no-input-timeout=5000," + "recognition-timeout=5000}" +this.crntGrammar; msg.addExecuteAppArg(arg); EslMessage response = sendSyncMultiLineCommand(channel, msg.getMsgLines());
Is there something I am missing?
Cheers,
-----Original Message----- From: freeswitch-users-bounces@lists.freeswitch.org <mailto:freeswitch-users-bounces@lists.freeswitch.org> on behalf of Christopher Rienzo Sent: Tue 11/15/2011 20:05 To: FreeSWITCH Users Help Subject: Re: [Freeswitch-users] How to implement TTS barge-in using FS ESL
Latest version of FreeSWITCH now has http://wiki.freeswitch.org/wiki/Misc._Dialplan_Tools_play_and_detect_speech Give it a try and let us know if there is anything that can be improved.
On Mon, Nov 14, 2011 at 12:06 PM, xl127 <x.liu@hw.ac.uk <mailto:x.liu@hw.ac.uk>> wrote:
> The way you did and described here is similar to what I wanted. > I want to set start-input-timers=false in the initial setup, then start > the recognition timer once the TTS finishes playing the prompt. > I do not use input callback as I am using ESL event (Java ESL Client > 0.9.2). > > I've not yet figured out how to know the prompt playing has finished, how > to stop the TTS and how to start the input timer. > I didn't get many events when I did barge-in: only got two CHANNEL_EXECUTE > and CAHNNEL_EXECUTE_COMPLETE events. > so didn't get the DETECTED_SPEECH at this point. > > In principle it shouldn't stop FS generating events when barging in, so > there must be somethings wrong in my setup. > > From Examples_directory_lua_asr_tts in the wiki, it mentions the session > needs to sleep for a few seconds before getting the results. > I tried but it didn't help. > > > > > On 14/11/11 14:48, Christopher Rienzo wrote: > > I'm not sure why it locks up for you. It works for me and I know of at > least one other developer that got it all to work. > > The way I do it in my custom APP is to set {start-input-timers=false} in > the speech recognition request and register an input callback function to > deal with the recognition and DTMF events. The input callback reacts to > the start of speech and recognition complete events by stopping the > prompt. At start of speech, the input timers are started and silence is > played until the final result arrives from the recognizer (no match, match, > etc). > > An alternative way to do this is to watch for the recognition events over > ESL and react to them. > > > > On Mon, Nov 14, 2011 at 9:19 AM, xl127 <x.liu@hw.ac.uk <mailto:x.liu@hw.ac.uk>> wrote: > >> Hi Christopher, >> >> That will be great! Thanks and looking forward to your app! >> >> By the way, do you have a quick thoughts about the reasons the FS stucks >> when garge-in occurs? >> Otherwise if it is not straightforward to explain it, just leave it >> alone until your new app comes out. >> >> Cheers, >> Xing >> >> >> >> >> On 14/11/11 13:23, Christopher Rienzo wrote: >> >> I'll write up a dialplan app this week to deal with this. Getting tired >> of being asked the same question over and over since it's too complicated >> as is currently designed :) >> >> >> >> >> On Mon, Nov 14, 2011 at 8:07 AM, xl127 <x.liu@hw.ac.uk <mailto:x.liu@hw.ac.uk>> wrote: >> >>> Hi, >>> >>> The first problem I need to solve is that FS stucks when I speak during >>> the prompt is playing. >>> It looks like the recognizer is not listening any more. It works fine if >>> I speak after the playing prompt finishes. >>> >>> I tried Nuance and PocketSphinx recognizers and got same problem. >>> >>> Any idea about what the possible causes are? >>> >>> Thanks! >>> Xing >>> >>> >>> On 11/11/11 18:59, xl127 wrote: >>> > Hello, >>> > >>> > I found this is a question that was asked before by others, but I >>> didn't >>> > find the answer. >>> > >>> > Anyway, I am using FS ESL outbound mode connecting to my IVR app, >>> using >>> > FS's "speak" >>> > and "detect_speech" to access Nuance MRCP V1 server. >>> > >>> > I want the user to be able to barge-in during the system's speaking. >>> > How could I implement it? >>> > >>> > I tried to specify "kill-on-barge-in=true" in the mrcp config profile. >>> > The barge-in doesn't work. >>> > >>> > With or without setting kill-on-barge-in, FS stops responding to my >>> > phone call and >>> > eventually it hangs up my call if I speak somthing (do the barge-in) >>> > during the system's speaking. >>> > >>> > I made a turn-by-turn loop in my app, the ASR/TTS works fine if I do >>> not >>> > do barge-in ( I wait until the TTS finishes then I start to speak) >>> > >>> > Any advices please? >>> > >>> > Thanks! >>> > Xing >>> > >>> > >>> > >>> >>> >>> >>> -- >>> Heriot-Watt University is a Scottish charity >>> registered under charity number SC000278. >>> >>> Heriot-Watt University is the Sunday Times >>> Scottish University of the Year 2011-2012 >>> >>> >>> >>> _________________________________________________________________________ >>> Professional FreeSWITCH Consulting Services: >>> consulting@freeswitch.org <mailto:consulting@freeswitch.org> >>> http://www.freeswitchsolutions.com >>> >>> FreeSWITCH-powered IP PBX: The CudaTel Communication Server >>> http://www.cudatel.com >>> >>> Official FreeSWITCH Sites >>> http://www.freeswitch.org >>> http://wiki.freeswitch.org >>> http://www.cluecon.com >>> >>> FreeSWITCH-users mailing list >>> FreeSWITCH-users@lists.freeswitch.org <mailto:FreeSWITCH-users@lists.freeswitch.org> >>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users >>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users >>> http://www.freeswitch.org >>> >> >> >> >> _________________________________________________________________________ >> Professional FreeSWITCH Consulting Services:consulting@freeswitch.orghttp://www.freeswitchsolutions.com <http://www.freeswitchsolutions.com>
>> >> FreeSWITCH-powered IP PBX: The CudaTel Communication Serverhttp://www.cudatel.com <http://www.cudatel.com> >> >> Official FreeSWITCH Siteshttp://www.freeswitch.orghttp://wiki.freeswitch.orghttp://www.cluecon.com <http://www.cluecon.com> >> >> FreeSWITCH-users mailing listFreeSWITCH-users@lists.freeswitch.orghttp://lists.freeswitch.org/mailman/listinfo/freeswitch-users <http://lists.freeswitch.org/mailman/listinfo/freeswitch-users> >> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-usershttp://www.frees... >> >> >> >> ------------------------------ >> >> [image: Scottish University of the Year 2011-12] *Heriot-Watt
>> University is the Sunday Times >> >> Scottish University of the Year 2011-2012 >> * >> >> Heriot-Watt University is a Scottish charity >> registered under charity number SC000278. >> >> _________________________________________________________________________ >> Professional FreeSWITCH Consulting Services: >> consulting@freeswitch.org <mailto:consulting@freeswitch.org> >> http://www.freeswitchsolutions.com >> >> FreeSWITCH-powered IP PBX: The CudaTel Communication Server >> http://www.cudatel.com >> >> Official FreeSWITCH Sites >> http://www.freeswitch.org >> http://wiki.freeswitch.org >> http://www.cluecon.com >> >> FreeSWITCH-users mailing list >> FreeSWITCH-users@lists.freeswitch.org <mailto:FreeSWITCH-users@lists.freeswitch.org> >> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users >> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users >> http://www.freeswitch.org >> >> > > > _________________________________________________________________________ > Professional FreeSWITCH Consulting Services:consulting@freeswitch.orghttp://www.freeswitchsolutions.com <http://www.freeswitchsolutions.com>
> > FreeSWITCH-powered IP PBX: The CudaTel Communication Serverhttp://www.cudatel.com <http://www.cudatel.com> > > Official FreeSWITCH Siteshttp://www.freeswitch.orghttp://wiki.freeswitch.orghttp://www.cluecon.com <http://www.cluecon.com> > > FreeSWITCH-users mailing listFreeSWITCH-users@lists.freeswitch.orghttp://lists.freeswitch.org/mailman/listinfo/freeswitch-users <http://lists.freeswitch.org/mailman/listinfo/freeswitch-users> > UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-usershttp://www.frees... > > > > ------------------------------ > > [image: Scottish University of the Year 2011-12] *Heriot-Watt
> University is the Sunday Times > Scottish University of the Year 2011-2012*
> > Heriot-Watt University is a Scottish charity > registered under charity number SC000278. > > _________________________________________________________________________ > Professional FreeSWITCH Consulting Services: > consulting@freeswitch.org <mailto:consulting@freeswitch.org> > http://www.freeswitchsolutions.com > > FreeSWITCH-powered IP PBX: The CudaTel Communication Server > http://www.cudatel.com > > Official FreeSWITCH Sites > http://www.freeswitch.org > http://wiki.freeswitch.org > http://www.cluecon.com > > FreeSWITCH-users mailing list > FreeSWITCH-users@lists.freeswitch.org <mailto:FreeSWITCH-users@lists.freeswitch.org> > http://lists.freeswitch.org/mailman/listinfo/freeswitch-users > UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users > http://www.freeswitch.org > >
------------------------------------------------------------------------
Scottish University of the Year 2011-12 *Heriot-Watt University is the Sunday Times Scottish University of the Year 2011-2012*
Heriot-Watt University is a Scottish charity registered under charity number SC000278.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org <mailto:consulting@freeswitch.org> http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org <mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Heriot-Watt University is a Scottish charity registered under charity number SC000278. Heriot-Watt University is the Sunday Times Scottish University of the Year 2011-2012
