[Freeswitch-users] How to implement TTS barge-in using FS ESL
Christopher Rienzo
cmrienzo at gmail.com
Thu Nov 17 18:48:02 MSK 2011
Actually in that silence prompt suggestion I gave, use
{start-input-timers=true}
On Thu, Nov 17, 2011 at 8:37 AM, Christopher Rienzo <cmrienzo at gmail.com>wrote:
>
>
>
>> After some more tests, I found followings:
>>
>> 1. Testing from dialplan, the log output is the string of "CRIT
>> ${speech_detect_result}" rather than the recognition results.
>>
>
> oops
>
>
>>
>> 2. Regarding to non barge in, I first send "speak" command then send
>> play_and_detect_speech with parameter:
>> detect:unimrcp:nuance5-mrcp1-1
>> {start-input-timers=false,no-input-timeout=25000,recognition-timeout=25000}dudeYNNC_Nuance
>>
>> in which I removed the "say:" part.
>>
>> But very soon I received event CHANNEL_EXECUTE_COMPLETE for
>> play_and_detect_speech before the text playing finishes,
>> and speech_detect_result is null (actually the event header does
>> not contain this variable)
>>
>> I tried with "say:" part with empty text like "say:unimrcp:en-GB: "
>> but it doesn't work, see issue 3 below.
>>
>
> Try with silence as I originally suggested:
>
> silence_stream://1000 detect:unimrcp:nuance5-mrcp1-1
> {start-input-timers=false,no-input-timeout=25000,
> recognition-timeout=25000}dudeYNNC_Nuance
>
>>
>> 3. It seems that I can not send "speak" command twice before the first
>> one finishes. In my case I run one port TTS server on one machine.
>>
>> If I send the command twice FS will give me Synthesizer
>> Error/Invalid TTS Module.
>> I thought the TTS request would be queued rather than it immediately
>> looks for the TTS resource.
>>
>> If I send the second command to another TTS machine, no error occurs
>> but I can only hear one utterance being spoken,
>> it looks like one utteraance was dropped somehow.
>>
>
> Wait for speak to finish before starting a new one.
>
>
>>
>> On 16/11/11 16:51, xl127 wrote:
>>
>> Hi Christopher,
>>
>> The questions are cleared to me now. Many thanks for your explanations!
>>
>> Best regards,
>>
>> Xing
>>
>>
>> On 16/11/11 15:52, Christopher Rienzo wrote:
>>
>>
>> Responses inline
>>
>>
>> Now it works in my ESL app though I am just able to do one dialogue (
>>> I need to add the event catching for furthur dialgoues).
>>>
>>> I have a couple of questions here:
>>>
>>> 1. In the first try, my Nuance server was able to be accessed somehow
>>> (FS says the MRCP is not responding in 5000ms,
>>> something like that), then FS says: [WARNING] rtsp_client.c:386 ()
>>> Failed to Connect to RTSP Server *MailScanner has detected a possible
>>> fraud attempt from "99.185.85.31:554" claiming to be* *MailScanner
>>> warning: numerical links are often malicious:* 99.185.85.31:554<http://99.185.85.31:554>
>>> ,
>>>
>>> later FS says:
>>> [ERR] mod_unimrcp.c:1860 (TTS-6) SYNTHESIZER channel error!
>>> [ERR] switch_ivr_play_say.c:2439 Invalid TTS module!
>>>
>>> The SYNTHESIZER channel error and Invalid TTS module error are
>>> obvious.
>>>
>>> What I don't understand is why it went to this stange address: *MailScanner
>>> has detected a possible fraud attempt from "99.185.85.31:554" claiming to be
>>> * *MailScanner warning: numerical links are often malicious:*99.185.85.31:554<http://99.185.85.31:554>
>>> ?
>>>
>>
>> check your unimrcp configuration. Make sure the default TTS and ASR
>> profiles are set to actual servers.
>>
>>
>>> 2. I specified TTS engine in play_and_detect_speech as
>>> "say:unimrcp:nuance5-mrcp1-1: the text to speak"
>>> It works though I didn't specify the TTS voice.
>>>
>>> How do I specify the TTS voice? In the mrcp profile (how?)? or
>>> something like:
>>> "say:unimrcp:nuance5-mrcp1-1:Serena: the text to speak" (this
>>> seems not right.)
>>>
>>
>> That won't work. Set the tts_engine variable as I explained previously,
>> or use say:unimrcp:voice:text to speak with the desired voice and the
>> correct default TTS profile defined in unimrcp.conf.xml. This is a
>> limitation of the say: notation. Alternatively, the voice can be defined
>> with the tts_voice channel variable.
>>
>>
>>
>>> 3. The barge-in works well, thanks!. Is the barge-in configurable? In
>>> some scenarios, we might not allow barge-in.
>>>
>>
>> If you don't want to barge in, just do "playback (or speak)" first, then
>> "play_and_detect_speech" with a silence prompt.
>>
>>
>>>
>>> 4. How could I get the text which has spoken to the user when barge-in
>>> occurs?
>>> Or Could I get the time when barge-in occurs? If I know the
>>> barge-in time and rough totale time for the whole text
>>> to be spoken I can figure out the spoken text by manually checking
>>> the recorded audio file later, which would be painful.
>>>
>>
>> If this is necessary, you might want to use the lower-level functions
>> instead to watch for the begin-speaking event.
>>
>>
>>>
>>> 5. when I use "speak" and "detect_speech" apps in ESL, I can catch
>>> event: DETECTED_SPEECH and speech-type: begin-speaking
>>> and "detected-speech", then I do the recognition results processing.
>>>
>>> The new app play_and_detect_speech seems not generate these events
>>> any more. The way that I can think of to get the results
>>> is to catch event:CHANNEL_EXECUTE_COMPLETE then check if
>>> variable_current_application=play_and_detect_speech, then get
>>> the results from variable_detect_speech_result.
>>>
>>> Is this the proper way to get the results in ESL app? Or will
>>> play_and_detect_speech later on be consistent with detect_speech
>>> in term of ASR events?
>>>
>>
>> play_and_detect_speech is a higher level abstraction to simplify things.
>> If you want to have more control, go back to using the ESL events. Reading
>> the code in mod_dptools and switch_ivr_async will give you hints about how
>> to do it correctly.
>>
>>
>>>
>>> 6. I'd like to set start-input-timers=false in the initial request
>>> then start the recognition timers (start-input-timers=true)
>>> after the TTS finishes.
>>> How possibly could I do this?
>>>
>>
>> This is automatically done in the switch_ivr_play_and_detect_speech()
>> function. You just need to specify start-input-timers=false in the
>> beginning.
>>
>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:consulting at freeswitch.orghttp://www.freeswitchsolutions.com
>>
>> FreeSWITCH-powered IP PBX: The CudaTel Communication Server
>>
>> Official FreeSWITCH Siteshttp://www.freeswitch.orghttp://wiki.freeswitch.orghttp://www.cluecon.com
>>
>> FreeSWITCH-users mailing listFreeSWITCH-users at lists.freeswitch.orghttp://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-usershttp://www.freeswitch.org
>>
>>
>>
>> ------------------------------
>>
>> [image: Scottish University of the Year 2011-12] *Heriot-Watt
>> University is the Sunday Times
>> Scottish University of the Year 2011-2012*
>>
>> Heriot-Watt University is a Scottish charity
>> registered under charity number SC000278.
>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:consulting at freeswitch.orghttp://www.freeswitchsolutions.com
>>
>> FreeSWITCH-powered IP PBX: The CudaTel Communication Server
>>
>> Official FreeSWITCH Siteshttp://www.freeswitch.orghttp://wiki.freeswitch.orghttp://www.cluecon.com
>>
>> FreeSWITCH-users mailing listFreeSWITCH-users at lists.freeswitch.orghttp://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-usershttp://www.freeswitch.org
>>
>>
>>
>> ------------------------------
>>
>> [image: MailScanner Signature HW] *Heriot-Watt University is the
>> Sunday Times
>>
>> Scottish University of the Year 2011-2012
>> *
>>
>> Heriot-Watt University is a Scottish charity
>> registered under charity number SC000278.
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>>
>>
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111117/031b09bd/attachment-0001.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 4803 bytes
Desc: not available
Url : http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111117/031b09bd/attachment-0002.jpe
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 4803 bytes
Desc: not available
Url : http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111117/031b09bd/attachment-0003.jpe
Join us at ClueCon 2011 Aug 9-11, 2011
More information about the FreeSWITCH-users
mailing list