[Freeswitch-users] How to implement TTS barge-in using FS ESL

Wed Nov 16 16:10:59 MSK 2011

You are missing the tts_engine.  Set the tts_engine channel variable to
"unimrcp:nuance5-mrcp1-1" or set the default tts profile to nuance5-mrcp1-1
in the unimrcp.conf.xml file and use "say:unimrcp:text to speak"

On Wed, Nov 16, 2011 at 7:36 AM, Liu, Xingkun <X.Liu at hw.ac.uk> wrote:

> **
>
> Great work, thanks Christopher!
>
> I've quick tested it from the dialplan based on the example. It works on
> my setup.
>
> Now I am trying to put it into my Java ESL app, FS complains:
>
>  switch_ivr_play_say.c:1167 Invalid Args
>
> A piece of codes I am using:
>
>         SendMsg msg = new SendMsg();
>         msg.addCallCommand("execute");
>         msg.addExecuteAppName("play_and_detect_speech");
>         String arg = "say:" + utt + " detect:unimrcp:nuance5-mrcp1-1
> {start-input-timers=false,no-input-timeout=5000,"
>                   + "recognition-timeout=5000}" +this.crntGrammar;
>         msg.addExecuteAppArg(arg);
>         EslMessage response = sendSyncMultiLineCommand(channel,
> msg.getMsgLines());
>
> Is there something I am missing?
>
> Cheers,
>
> Xing
>
>
>
>
> -----Original Message-----
> From: freeswitch-users-bounces at lists.freeswitch.org on behalf of
> Christopher Rienzo
> Sent: Tue 11/15/2011 20:05
> To: FreeSWITCH Users Help
> Subject: Re: [Freeswitch-users] How to implement TTS barge-in using FS ESL
>
> Latest version of FreeSWITCH now has
> http://wiki.freeswitch.org/wiki/Misc._Dialplan_Tools_play_and_detect_speech
> Give it a try and let us know if there is anything that can be improved.
>
>
>
> On Mon, Nov 14, 2011 at 12:06 PM, xl127 <x.liu at hw.ac.uk> wrote:
>
> >  The way you did and described here is similar to what I wanted.
> > I want to set start-input-timers=false in the initial setup, then start
> > the recognition timer once the TTS finishes playing the prompt.
> > I do not use input callback as I am using ESL event (Java ESL Client
> > 0.9.2).
> >
> > I've not yet figured out how to know the prompt playing has finished, how
> > to stop the TTS and how to start the input timer.
> > I didn't get many events when I did barge-in: only got two
> CHANNEL_EXECUTE
> > and CAHNNEL_EXECUTE_COMPLETE events.
> > so didn't get the DETECTED_SPEECH at this point.
> >
> > In principle it shouldn't stop FS generating events when barging in, so
> > there must be somethings wrong in my setup.
> >
> > From Examples_directory_lua_asr_tts in the wiki, it mentions the session
> > needs to sleep for a few seconds before getting the results.
> > I tried but it didn't help.
> >
> >
> >
> >
> > On 14/11/11 14:48, Christopher Rienzo wrote:
> >
> > I'm not sure why it locks up for you.  It works for me and I know of at
> > least one other developer that got it all to work.
> >
> > The way I do it in my custom APP is to set {start-input-timers=false} in
> > the speech recognition request and register an input callback function to
> > deal with the recognition and DTMF events.  The input callback reacts to
> > the start of speech and recognition complete events by stopping the
> > prompt.  At start of speech, the input timers are started and silence is
> > played until the final result arrives from the recognizer (no match,
> match,
> > etc).
> >
> > An alternative way to do this is to watch for the recognition events over
> > ESL and react to them.
> >
> >
> >
> > On Mon, Nov 14, 2011 at 9:19 AM, xl127 <x.liu at hw.ac.uk> wrote:
> >
> >>  Hi Christopher,
> >>
> >> That will be great! Thanks and looking forward to your app!
> >>
> >> By the way, do you have a quick thoughts about the reasons the FS stucks
> >> when garge-in occurs?
> >> Otherwise if it is not straightforward to explain it,  just leave it
> >> alone until your new app comes out.
> >>
> >> Cheers,
> >> Xing
> >>
> >>
> >>
> >>
> >> On 14/11/11 13:23, Christopher Rienzo wrote:
> >>
> >> I'll write up a dialplan app this week to deal with this.  Getting tired
> >> of being asked the same question over and over since it's too
> complicated
> >> as is currently designed :)
> >>
> >>
> >>
> >>
> >>  On Mon, Nov 14, 2011 at 8:07 AM, xl127 <x.liu at hw.ac.uk> wrote:
> >>
> >>> Hi,
> >>>
> >>> The first problem I need to solve is that FS stucks when I speak during
> >>> the prompt is playing.
> >>> It looks like the recognizer is not listening any more. It works fine
> if
> >>> I speak after the playing prompt finishes.
> >>>
> >>> I tried Nuance and PocketSphinx recognizers and got same problem.
> >>>
> >>> Any idea about what the possible causes are?
> >>>
> >>> Thanks!
> >>> Xing
> >>>
> >>>
> >>> On 11/11/11 18:59, xl127 wrote:
> >>> > Hello,
> >>> >
> >>> > I found this is a question that was asked before by others, but I
> >>> didn't
> >>> > find the answer.
> >>> >
> >>> > Anyway, I am using FS ESL outbound mode connecting to my IVR app,
> >>>  using
> >>> > FS's "speak"
> >>> > and "detect_speech" to access Nuance MRCP V1 server.
> >>> >
> >>> > I want the user to be able to barge-in during the system's speaking.
> >>> > How could I implement it?
> >>> >
> >>> > I tried to specify "kill-on-barge-in=true" in the mrcp config
> profile.
> >>> > The barge-in doesn't work.
> >>> >
> >>> > With or without setting kill-on-barge-in, FS stops responding to my
> >>> > phone call and
> >>> > eventually it hangs up my call if I speak somthing (do the barge-in)
> >>> > during the system's speaking.
> >>> >
> >>> > I made a turn-by-turn loop in my app, the ASR/TTS works fine if I do
> >>> not
> >>> > do barge-in ( I wait until the TTS finishes then I start to speak)
> >>> >
> >>> > Any advices please?
> >>> >
> >>> > Thanks!
> >>> > Xing
> >>> >
> >>> >
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> Heriot-Watt University is a Scottish charity
> >>> registered under charity number SC000278.
> >>>
> >>> Heriot-Watt University is the Sunday Times
> >>> Scottish University of the Year 2011-2012
> >>>
> >>>
> >>>
> >>>
> _________________________________________________________________________
> >>> Professional FreeSWITCH Consulting Services:
> >>> consulting at freeswitch.org
> >>> http://www.freeswitchsolutions.com
> >>>
> >>> 
> >>> 
> >>>
> >>> Official FreeSWITCH Sites
> >>> http://www.freeswitch.org
> >>> http://wiki.freeswitch.org
> >>> http://www.cluecon.com
> >>>
> >>> FreeSWITCH-users mailing list
> >>> FreeSWITCH-users at lists.freeswitch.org
> >>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> >>> UNSUBSCRIBE:
> http://lists.freeswitch.org/mailman/options/freeswitch-users
> >>> http://www.freeswitch.org
> >>>
> >>
> >>
> >>
> >>
> _________________________________________________________________________
> >> Professional FreeSWITCH Consulting Services:consulting
> @freeswitch.orghttp://www.freeswitchsolutions.com
>
> >>
> >> http://
> www.cudatel.com
> >>
> >> Official FreeSWITCH
> Siteshttp://www.freeswitch.orghttp://wiki.freeswitch.orghttp://
> www.cluecon.com
> >>
> >> FreeSWITCH-users mailing listFreeSWITCH-users
> @lists.freeswitch.orghttp://
> lists.freeswitch.org/mailman/listinfo/freeswitch-users
> >> UNSUBSCRIBE:
> http://lists.freeswitch.org/mailman/options/freeswitch-usershttp://www.freeswitch.org
> >>
> >>
> >>
> >>  ------------------------------
> >>
> >> [image: Scottish University of the Year 2011-12]   *Heriot-Watt
>
> >> University is the Sunday Times
> >>
> >>   Scottish University of the Year 2011-2012
> >> *
> >>
> >>    Heriot-Watt University is a Scottish charity
> >>   registered under charity number SC000278.
> >>
> >>
> _________________________________________________________________________
> >> Professional FreeSWITCH Consulting Services:
> >> consulting at freeswitch.org
> >> http://www.freeswitchsolutions.com
> >>
> >> 
> >> 
> >>
> >> Official FreeSWITCH Sites
> >> http://www.freeswitch.org
> >> http://wiki.freeswitch.org
> >> http://www.cluecon.com
> >>
> >> FreeSWITCH-users mailing list
> >> FreeSWITCH-users at lists.freeswitch.org
> >> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> >> UNSUBSCRIBE:
> http://lists.freeswitch.org/mailman/options/freeswitch-users
> >> http://www.freeswitch.org
> >>
> >>
> >
> >
> > _________________________________________________________________________
> > Professional FreeSWITCH Consulting Services:consulting
> @freeswitch.orghttp://www.freeswitchsolutions.com
>
> >
> > http://
> www.cudatel.com
> >
> > Official FreeSWITCH
> Siteshttp://www.freeswitch.orghttp://wiki.freeswitch.orghttp://
> www.cluecon.com
> >
> > FreeSWITCH-users mailing listFreeSWITCH-users
> @lists.freeswitch.orghttp://
> lists.freeswitch.org/mailman/listinfo/freeswitch-users
> > UNSUBSCRIBE:
> http://lists.freeswitch.org/mailman/options/freeswitch-usershttp://www.freeswitch.org
> >
> >
> >
> > ------------------------------
> >
> > [image: Scottish University of the Year 2011-12]   *Heriot-Watt
>
> > University is the Sunday Times
> >   Scottish University of the Year 2011-2012*
>
> >
> >    Heriot-Watt University is a Scottish charity
> >   registered under charity number SC000278.
> >
> > _________________________________________________________________________
> > Professional FreeSWITCH Consulting Services:
> > consulting at freeswitch.org
> > http://www.freeswitchsolutions.com
> >
> > 
> > 
> >
> > Official FreeSWITCH Sites
> > http://www.freeswitch.org
> > http://wiki.freeswitch.org
> > http://www.cluecon.com
> >
> > FreeSWITCH-users mailing list
> > FreeSWITCH-users at lists.freeswitch.org
> > http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> > UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> > http://www.freeswitch.org
> >
> >
>
>
> ------------------------------
>
> [image: Scottish University of the Year 2011-12]   *Heriot-Watt
> University is the Sunday Times
>   Scottish University of the Year 2011-2012*
>
>    Heriot-Watt University is a Scottish charity
>   registered under charity number SC000278.
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> 
> 
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111116/6b8f14e9/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 4803 bytes
Desc: not available
Url : http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111116/6b8f14e9/attachment-0001.jpe 