[Freeswitch-users] How to implement TTS barge-in using FS ESL

Wed Nov 16 15:36:49 MSK 2011

Great work, thanks Christopher!

I've quick tested it from the dialplan based on the example. It works on my setup.

Now I am trying to put it into my Java ESL app, FS complains:

 switch_ivr_play_say.c:1167 Invalid Args

A piece of codes I am using:

        SendMsg msg = new SendMsg();
        msg.addCallCommand("execute");
        msg.addExecuteAppName("play_and_detect_speech");
        String arg = "say:" + utt + " detect:unimrcp:nuance5-mrcp1-1 {start-input-timers=false,no-input-timeout=5000,"
                  + "recognition-timeout=5000}" +this.crntGrammar;
        msg.addExecuteAppArg(arg);
        EslMessage response = sendSyncMultiLineCommand(channel, msg.getMsgLines());

Is there something I am missing?

Cheers,

Xing

-----Original Message-----
From: freeswitch-users-bounces at lists.freeswitch.org on behalf of Christopher Rienzo
Sent: Tue 11/15/2011 20:05
To: FreeSWITCH Users Help
Subject: Re: [Freeswitch-users] How to implement TTS barge-in using FS ESL

Latest version of FreeSWITCH now has
http://wiki.freeswitch.org/wiki/Misc._Dialplan_Tools_play_and_detect_speech
Give it a try and let us know if there is anything that can be improved.

On Mon, Nov 14, 2011 at 12:06 PM, xl127 <x.liu at hw.ac.uk> wrote:

>  The way you did and described here is similar to what I wanted.
> I want to set start-input-timers=false in the initial setup, then start
> the recognition timer once the TTS finishes playing the prompt.
> I do not use input callback as I am using ESL event (Java ESL Client
> 0.9.2).
>
> I've not yet figured out how to know the prompt playing has finished, how
> to stop the TTS and how to start the input timer.
> I didn't get many events when I did barge-in: only got two CHANNEL_EXECUTE
> and CAHNNEL_EXECUTE_COMPLETE events.
> so didn't get the DETECTED_SPEECH at this point.
>
> In principle it shouldn't stop FS generating events when barging in, so
> there must be somethings wrong in my setup.
>
> From Examples_directory_lua_asr_tts in the wiki, it mentions the session
> needs to sleep for a few seconds before getting the results.
> I tried but it didn't help.
>
>
>
>
> On 14/11/11 14:48, Christopher Rienzo wrote:
>
> I'm not sure why it locks up for you.  It works for me and I know of at
> least one other developer that got it all to work.
>
> The way I do it in my custom APP is to set {start-input-timers=false} in
> the speech recognition request and register an input callback function to
> deal with the recognition and DTMF events.  The input callback reacts to
> the start of speech and recognition complete events by stopping the
> prompt.  At start of speech, the input timers are started and silence is
> played until the final result arrives from the recognizer (no match, match,
> etc).
>
> An alternative way to do this is to watch for the recognition events over
> ESL and react to them.
>
>
>
> On Mon, Nov 14, 2011 at 9:19 AM, xl127 <x.liu at hw.ac.uk> wrote:
>
>>  Hi Christopher,
>>
>> That will be great! Thanks and looking forward to your app!
>>
>> By the way, do you have a quick thoughts about the reasons the FS stucks
>> when garge-in occurs?
>> Otherwise if it is not straightforward to explain it,  just leave it
>> alone until your new app comes out.
>>
>> Cheers,
>> Xing
>>
>>
>>
>>
>> On 14/11/11 13:23, Christopher Rienzo wrote:
>>
>> I'll write up a dialplan app this week to deal with this.  Getting tired
>> of being asked the same question over and over since it's too complicated
>> as is currently designed :)
>>
>>
>>
>>
>>  On Mon, Nov 14, 2011 at 8:07 AM, xl127 <x.liu at hw.ac.uk> wrote:
>>
>>> Hi,
>>>
>>> The first problem I need to solve is that FS stucks when I speak during
>>> the prompt is playing.
>>> It looks like the recognizer is not listening any more. It works fine if
>>> I speak after the playing prompt finishes.
>>>
>>> I tried Nuance and PocketSphinx recognizers and got same problem.
>>>
>>> Any idea about what the possible causes are?
>>>
>>> Thanks!
>>> Xing
>>>
>>>
>>> On 11/11/11 18:59, xl127 wrote:
>>> > Hello,
>>> >
>>> > I found this is a question that was asked before by others, but I
>>> didn't
>>> > find the answer.
>>> >
>>> > Anyway, I am using FS ESL outbound mode connecting to my IVR app,
>>>  using
>>> > FS's "speak"
>>> > and "detect_speech" to access Nuance MRCP V1 server.
>>> >
>>> > I want the user to be able to barge-in during the system's speaking.
>>> > How could I implement it?
>>> >
>>> > I tried to specify "kill-on-barge-in=true" in the mrcp config profile.
>>> > The barge-in doesn't work.
>>> >
>>> > With or without setting kill-on-barge-in, FS stops responding to my
>>> > phone call and
>>> > eventually it hangs up my call if I speak somthing (do the barge-in)
>>> > during the system's speaking.
>>> >
>>> > I made a turn-by-turn loop in my app, the ASR/TTS works fine if I do
>>> not
>>> > do barge-in ( I wait until the TTS finishes then I start to speak)
>>> >
>>> > Any advices please?
>>> >
>>> > Thanks!
>>> > Xing
>>> >
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> Heriot-Watt University is a Scottish charity
>>> registered under charity number SC000278.
>>>
>>> Heriot-Watt University is the Sunday Times
>>> Scottish University of the Year 2011-2012
>>>
>>>
>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org
>>> http://www.freeswitchsolutions.com
>>>
>>> 
>>> 
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://wiki.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>
>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:consulting at freeswitch.orghttp://www.freeswitchsolutions.com
>>
>> FreeSWITCH-powered IP PBX: The CudaTel Communication Server
>>
>> Official FreeSWITCH Siteshttp://www.freeswitch.orghttp://wiki.freeswitch.orghttp://www.cluecon.com
>>
>> FreeSWITCH-users mailing listFreeSWITCH-users at lists.freeswitch.orghttp://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-usershttp://www.freeswitch.org
>>
>>
>>
>>  ------------------------------
>>
>> [image: Scottish University of the Year 2011-12]   *Heriot-Watt
>> University is the Sunday Times
>>
>>   Scottish University of the Year 2011-2012
>> *
>>
>>    Heriot-Watt University is a Scottish charity
>>   registered under charity number SC000278.
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> 
>> 
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>>
>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:consulting at freeswitch.orghttp://www.freeswitchsolutions.com
>
> FreeSWITCH-powered IP PBX: The CudaTel Communication Server
>
> Official FreeSWITCH Siteshttp://www.freeswitch.orghttp://wiki.freeswitch.orghttp://www.cluecon.com
>
> FreeSWITCH-users mailing listFreeSWITCH-users at lists.freeswitch.orghttp://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-usershttp://www.freeswitch.org
>
>
>
> ------------------------------
>
> [image: Scottish University of the Year 2011-12]   *Heriot-Watt
> University is the Sunday Times
>   Scottish University of the Year 2011-2012*
>
>    Heriot-Watt University is a Scottish charity
>   registered under charity number SC000278.
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> 
> 
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>

-- 
Heriot-Watt University is a Scottish charity
registered under charity number SC000278.

Heriot-Watt University is the Sunday Times
Scottish University of the Year 2011-2012

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111116/2ee31abe/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: hw_uni_of_year.jpg
Type: image/jpeg
Size: 4803 bytes
Desc: not available
Url : http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111116/2ee31abe/attachment-0003.jpg 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: hw_uni_of_year.jpg
Type: image/jpeg
Size: 4803 bytes
Desc: hw_uni_of_year.jpg
Url : http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111116/2ee31abe/attachment-0004.jpg 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ATT16976631.jpg
Type: image/jpeg
Size: 4803 bytes
Desc: ATT16976631.jpg
Url : http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111116/2ee31abe/attachment-0005.jpg 