[Freeswitch-users] How to implement TTS barge-in using FS ESL
xl127
x.liu at hw.ac.uk
Thu Nov 17 19:21:30 MSK 2011
Yeah, I see, it should be set to true there. Thanks for reminding me this.
On 17/11/11 15:48, Christopher Rienzo wrote:
> Actually in that silence prompt suggestion I gave, use
> {start-input-timers=true}
>
>
> On Thu, Nov 17, 2011 at 8:37 AM, Christopher Rienzo
> <cmrienzo at gmail.com <mailto:cmrienzo at gmail.com>> wrote:
>
>
>
>
> After some more tests, I found followings:
>
> 1. Testing from dialplan, the log output is the string of
> "CRIT ${speech_detect_result}" rather than the recognition
> results.
>
>
> oops
>
>
> 2. Regarding to non barge in, I first send "speak" command
> then send play_and_detect_speech with parameter:
> detect:unimrcp:nuance5-mrcp1-1
> {start-input-timers=false,no-input-timeout=25000,recognition-timeout=25000}dudeYNNC_Nuance
>
> in which I removed the "say:" part.
>
> But very soon I received event CHANNEL_EXECUTE_COMPLETE
> for play_and_detect_speech before the text playing finishes,
> and speech_detect_result is null (actually the event
> header does not contain this variable)
>
> I tried with "say:" part with empty text like
> "say:unimrcp:en-GB: " but it doesn't work, see issue 3 below.
>
>
> Try with silence as I originally suggested:
>
> silence_stream://1000 detect:unimrcp:nuance5-mrcp1-1
> {start-input-timers=false,no-input-timeout=25000,
> recognition-timeout=25000}dudeYNNC_Nuance
>
>
> 3. It seems that I can not send "speak" command twice before
> the first one finishes. In my case I run one port TTS server
> on one machine.
>
> If I send the command twice FS will give me Synthesizer
> Error/Invalid TTS Module.
> I thought the TTS request would be queued rather than it
> immediately looks for the TTS resource.
>
> If I send the second command to another TTS machine, no
> error occurs but I can only hear one utterance being spoken,
> it looks like one utteraance was dropped somehow.
>
>
> Wait for speak to finish before starting a new one.
>
>
> On 16/11/11 16:51, xl127 wrote:
>> Hi Christopher,
>>
>> The questions are cleared to me now. Many thanks for your
>> explanations!
>>
>> Best regards,
>>
>> Xing
>>
>>
>> On 16/11/11 15:52, Christopher Rienzo wrote:
>>>
>>> Responses inline
>>>
>>>
>>> Now it works in my ESL app though I am just able to do
>>> one dialogue ( I need to add the event catching for
>>> furthur dialgoues).
>>>
>>> I have a couple of questions here:
>>>
>>> 1. In the first try, my Nuance server was able to be
>>> accessed somehow (FS says the MRCP is not responding in
>>> 5000ms,
>>> something like that), then FS says: [WARNING]
>>> rtsp_client.c:386 () Failed to Connect to RTSP Server
>>> *MailScanner has detected a possible fraud attempt from
>>> "99.185.85.31:554" claiming to be* *MailScanner warning:
>>> numerical links are often malicious:* 99.185.85.31:554
>>> <http://99.185.85.31:554>,
>>>
>>> later FS says:
>>> [ERR] mod_unimrcp.c:1860 (TTS-6) SYNTHESIZER
>>> channel error!
>>> [ERR] switch_ivr_play_say.c:2439 Invalid TTS module!
>>>
>>> The SYNTHESIZER channel error and Invalid TTS module
>>> error are obvious.
>>>
>>> What I don't understand is why it went to this
>>> stange address: *MailScanner has detected a possible
>>> fraud attempt from "99.185.85.31:554" claiming to be*
>>> *MailScanner warning: numerical links are often
>>> malicious:* 99.185.85.31:554 <http://99.185.85.31:554>?
>>>
>>>
>>> check your unimrcp configuration. Make sure the default TTS
>>> and ASR profiles are set to actual servers.
>>>
>>> 2. I specified TTS engine in play_and_detect_speech as
>>> "say:unimrcp:nuance5-mrcp1-1: the text to speak"
>>> It works though I didn't specify the TTS voice.
>>>
>>> How do I specify the TTS voice? In the mrcp profile
>>> (how?)? or something like:
>>> "say:unimrcp:nuance5-mrcp1-1:Serena: the text
>>> to speak" (this seems not right.)
>>>
>>>
>>> That won't work. Set the tts_engine variable as I explained
>>> previously, or use say:unimrcp:voice:text to speak with the
>>> desired voice and the correct default TTS profile defined in
>>> unimrcp.conf.xml. This is a limitation of the say:
>>> notation. Alternatively, the voice can be defined with the
>>> tts_voice channel variable.
>>>
>>> 3. The barge-in works well, thanks!. Is the barge-in
>>> configurable? In some scenarios, we might not allow
>>> barge-in.
>>>
>>>
>>> If you don't want to barge in, just do "playback (or speak)"
>>> first, then "play_and_detect_speech" with a silence prompt.
>>>
>>>
>>> 4. How could I get the text which has spoken to the
>>> user when barge-in occurs?
>>> Or Could I get the time when barge-in occurs? If I
>>> know the barge-in time and rough totale time for the
>>> whole text
>>> to be spoken I can figure out the spoken text by
>>> manually checking the recorded audio file later, which
>>> would be painful.
>>>
>>>
>>> If this is necessary, you might want to use the lower-level
>>> functions instead to watch for the begin-speaking event.
>>>
>>>
>>> 5. when I use "speak" and "detect_speech" apps in ESL,
>>> I can catch event: DETECTED_SPEECH and speech-type:
>>> begin-speaking
>>> and "detected-speech", then I do the recognition
>>> results processing.
>>>
>>> The new app play_and_detect_speech seems not
>>> generate these events any more. The way that I can think
>>> of to get the results
>>> is to catch event:CHANNEL_EXECUTE_COMPLETE then
>>> check if
>>> variable_current_application=play_and_detect_speech,
>>> then get
>>> the results from variable_detect_speech_result.
>>>
>>> Is this the proper way to get the results in ESL
>>> app? Or will play_and_detect_speech later on be
>>> consistent with detect_speech
>>> in term of ASR events?
>>>
>>>
>>> play_and_detect_speech is a higher level abstraction to
>>> simplify things. If you want to have more control, go back
>>> to using the ESL events. Reading the code in mod_dptools
>>> and switch_ivr_async will give you hints about how to do it
>>> correctly.
>>>
>>>
>>> 6. I'd like to set start-input-timers=false in the
>>> initial request then start the recognition timers
>>> (start-input-timers=true)
>>> after the TTS finishes.
>>> How possibly could I do this?
>>>
>>>
>>> This is automatically done in the
>>> switch_ivr_play_and_detect_speech() function. You just need
>>> to specify start-input-timers=false in the beginning.
>>>
>>>
>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org <mailto:consulting at freeswitch.org>
>>> http://www.freeswitchsolutions.com
>>>
>>>
>>>
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://wiki.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org <mailto:FreeSWITCH-users at lists.freeswitch.org>
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>
>>
>> ------------------------------------------------------------------------
>>
>> Scottish University of the Year 2011-12 *Heriot-Watt
>> University is the Sunday Times
>> Scottish University of the Year 2011-2012*
>>
>> Heriot-Watt University is a Scottish charity
>> registered under charity number SC000278.
>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org <mailto:consulting at freeswitch.org>
>> http://www.freeswitchsolutions.com
>>
>>
>>
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org <mailto:FreeSWITCH-users at lists.freeswitch.org>
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>
>
> ------------------------------------------------------------------------
>
> MailScanner Signature HW *Heriot-Watt University is the Sunday
> Times
>
> Scottish University of the Year 2011-2012
> *
>
> Heriot-Watt University is a Scottish charity
> registered under charity number SC000278.
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org <mailto:consulting at freeswitch.org>
> http://www.freeswitchsolutions.com
>
>
>
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> <mailto:FreeSWITCH-users at lists.freeswitch.org>
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>
>
>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
>
>
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
--
Heriot-Watt University is a Scottish charity
registered under charity number SC000278.
Heriot-Watt University is the Sunday Times
Scottish University of the Year 2011-2012
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111117/f7888131/attachment-0001.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 4803 bytes
Desc: not available
Url : http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111117/f7888131/attachment-0002.jpe
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 4803 bytes
Desc: not available
Url : http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111117/f7888131/attachment-0003.jpe
-------------- next part --------------
A non-text attachment was scrubbed...
Name: hw_uni_of_year.jpg
Type: image/jpeg
Size: 4803 bytes
Desc: not available
Url : http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111117/f7888131/attachment-0001.jpg
Join us at ClueCon 2011 Aug 9-11, 2011
More information about the FreeSWITCH-users
mailing list