[Freeswitch-users] How to implement TTS barge-in using FS ESL
xl127
x.liu at hw.ac.uk
Fri Nov 18 20:33:20 MSK 2011
I want to specify <grammar> for play_and_detect_speech using URI format.
I am running a Tomcat server that hosts the grammars and I am sure it
can be accessed from local machine or from a different machine.
If I use http://localhost:8084/grammars/FS_Nuance/myGrammar.gram
or http://127.0.0.1:8084/grammars/FS_Nuance/myGrammar.gram
FS got grammar loading failure (MRCP/1.0 1 407 COMPLETE)
If I change localhost to my concrete external IP, it works for Nuance
server, but not for Voxeo Prophecy server.
I also have difficulties to specify URI grammars for detect_speech.
Any suggestions please?
Thanks,
Xing
On 17/11/11 16:21, xl127 wrote:
> Yeah, I see, it should be set to true there. Thanks for reminding me this.
>
>
> On 17/11/11 15:48, Christopher Rienzo wrote:
>> Actually in that silence prompt suggestion I gave, use
>> {start-input-timers=true}
>>
>>
>> On Thu, Nov 17, 2011 at 8:37 AM, Christopher Rienzo
>> <cmrienzo at gmail.com <mailto:cmrienzo at gmail.com>> wrote:
>>
>>
>>
>>
>> After some more tests, I found followings:
>>
>> 1. Testing from dialplan, the log output is the string of
>> "CRIT ${speech_detect_result}" rather than the recognition
>> results.
>>
>>
>> oops
>>
>>
>> 2. Regarding to non barge in, I first send "speak" command
>> then send play_and_detect_speech with parameter:
>> detect:unimrcp:nuance5-mrcp1-1
>> {start-input-timers=false,no-input-timeout=25000,recognition-timeout=25000}dudeYNNC_Nuance
>>
>> in which I removed the "say:" part.
>>
>> But very soon I received event CHANNEL_EXECUTE_COMPLETE
>> for play_and_detect_speech before the text playing finishes,
>> and speech_detect_result is null (actually the event
>> header does not contain this variable)
>>
>> I tried with "say:" part with empty text like
>> "say:unimrcp:en-GB: " but it doesn't work, see issue 3 below.
>>
>>
>> Try with silence as I originally suggested:
>>
>> silence_stream://1000 detect:unimrcp:nuance5-mrcp1-1
>> {start-input-timers=false,no-input-timeout=25000,
>> recognition-timeout=25000}dudeYNNC_Nuance
>>
>>
>> 3. It seems that I can not send "speak" command twice
>> before the first one finishes. In my case I run one port TTS
>> server on one machine.
>>
>> If I send the command twice FS will give me Synthesizer
>> Error/Invalid TTS Module.
>> I thought the TTS request would be queued rather than it
>> immediately looks for the TTS resource.
>>
>> If I send the second command to another TTS machine, no
>> error occurs but I can only hear one utterance being spoken,
>> it looks like one utteraance was dropped somehow.
>>
>>
>> Wait for speak to finish before starting a new one.
>>
>>
>> On 16/11/11 16:51, xl127 wrote:
>>> Hi Christopher,
>>>
>>> The questions are cleared to me now. Many thanks for your
>>> explanations!
>>>
>>> Best regards,
>>>
>>> Xing
>>>
>>>
>>> On 16/11/11 15:52, Christopher Rienzo wrote:
>>>>
>>>> Responses inline
>>>>
>>>>
>>>> Now it works in my ESL app though I am just able to do
>>>> one dialogue ( I need to add the event catching for
>>>> furthur dialgoues).
>>>>
>>>> I have a couple of questions here:
>>>>
>>>> 1. In the first try, my Nuance server was able to be
>>>> accessed somehow (FS says the MRCP is not responding in
>>>> 5000ms,
>>>> something like that), then FS says: [WARNING]
>>>> rtsp_client.c:386 () Failed to Connect to RTSP Server
>>>> *MailScanner has detected a possible fraud attempt from
>>>> "99.185.85.31:554" claiming to be* *MailScanner
>>>> warning: numerical links are often malicious:*
>>>> 99.185.85.31:554 <http://99.185.85.31:554>,
>>>>
>>>> later FS says:
>>>> [ERR] mod_unimrcp.c:1860 (TTS-6) SYNTHESIZER
>>>> channel error!
>>>> [ERR] switch_ivr_play_say.c:2439 Invalid TTS module!
>>>>
>>>> The SYNTHESIZER channel error and Invalid TTS module
>>>> error are obvious.
>>>>
>>>> What I don't understand is why it went to this
>>>> stange address: *MailScanner has detected a possible
>>>> fraud attempt from "99.185.85.31:554" claiming to be*
>>>> *MailScanner warning: numerical links are often
>>>> malicious:* 99.185.85.31:554 <http://99.185.85.31:554>?
>>>>
>>>>
>>>> check your unimrcp configuration. Make sure the default
>>>> TTS and ASR profiles are set to actual servers.
>>>>
>>>> 2. I specified TTS engine in play_and_detect_speech as
>>>> "say:unimrcp:nuance5-mrcp1-1: the text to speak"
>>>> It works though I didn't specify the TTS voice.
>>>>
>>>> How do I specify the TTS voice? In the mrcp
>>>> profile (how?)? or something like:
>>>> "say:unimrcp:nuance5-mrcp1-1:Serena: the text
>>>> to speak" (this seems not right.)
>>>>
>>>>
>>>> That won't work. Set the tts_engine variable as I
>>>> explained previously, or use say:unimrcp:voice:text to
>>>> speak with the desired voice and the correct default TTS
>>>> profile defined in unimrcp.conf.xml. This is a limitation
>>>> of the say: notation. Alternatively, the voice can be
>>>> defined with the tts_voice channel variable.
>>>>
>>>> 3. The barge-in works well, thanks!. Is the barge-in
>>>> configurable? In some scenarios, we might not allow
>>>> barge-in.
>>>>
>>>>
>>>> If you don't want to barge in, just do "playback (or
>>>> speak)" first, then "play_and_detect_speech" with a silence
>>>> prompt.
>>>>
>>>>
>>>> 4. How could I get the text which has spoken to the
>>>> user when barge-in occurs?
>>>> Or Could I get the time when barge-in occurs? If I
>>>> know the barge-in time and rough totale time for the
>>>> whole text
>>>> to be spoken I can figure out the spoken text by
>>>> manually checking the recorded audio file later, which
>>>> would be painful.
>>>>
>>>>
>>>> If this is necessary, you might want to use the lower-level
>>>> functions instead to watch for the begin-speaking event.
>>>>
>>>>
>>>> 5. when I use "speak" and "detect_speech" apps in
>>>> ESL, I can catch event: DETECTED_SPEECH and
>>>> speech-type: begin-speaking
>>>> and "detected-speech", then I do the recognition
>>>> results processing.
>>>>
>>>> The new app play_and_detect_speech seems not
>>>> generate these events any more. The way that I can
>>>> think of to get the results
>>>> is to catch event:CHANNEL_EXECUTE_COMPLETE then
>>>> check if
>>>> variable_current_application=play_and_detect_speech,
>>>> then get
>>>> the results from variable_detect_speech_result.
>>>>
>>>> Is this the proper way to get the results in ESL
>>>> app? Or will play_and_detect_speech later on be
>>>> consistent with detect_speech
>>>> in term of ASR events?
>>>>
>>>>
>>>> play_and_detect_speech is a higher level abstraction to
>>>> simplify things. If you want to have more control, go back
>>>> to using the ESL events. Reading the code in mod_dptools
>>>> and switch_ivr_async will give you hints about how to do it
>>>> correctly.
>>>>
>>>>
>>>> 6. I'd like to set start-input-timers=false in the
>>>> initial request then start the recognition timers
>>>> (start-input-timers=true)
>>>> after the TTS finishes.
>>>> How possibly could I do this?
>>>>
>>>>
>>>> This is automatically done in the
>>>> switch_ivr_play_and_detect_speech() function. You just
>>>> need to specify start-input-timers=false in the beginning.
>>>>
>>>>
>>>>
>>>> _________________________________________________________________________
>>>> Professional FreeSWITCH Consulting Services:
>>>> consulting at freeswitch.org <mailto:consulting at freeswitch.org>
>>>> http://www.freeswitchsolutions.com
>>>>
>>>>
>>>>
>>>>
>>>> Official FreeSWITCH Sites
>>>> http://www.freeswitch.org
>>>> http://wiki.freeswitch.org
>>>> http://www.cluecon.com
>>>>
>>>> FreeSWITCH-users mailing list
>>>> FreeSWITCH-users at lists.freeswitch.org <mailto:FreeSWITCH-users at lists.freeswitch.org>
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> http://www.freeswitch.org
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> Scottish University of the Year 2011-12 *Heriot-Watt
>>> University is the Sunday Times
>>> Scottish University of the Year 2011-2012*
>>>
>>> Heriot-Watt University is a Scottish charity
>>> registered under charity number SC000278.
>>>
>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org <mailto:consulting at freeswitch.org>
>>> http://www.freeswitchsolutions.com
>>>
>>>
>>>
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://wiki.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org <mailto:FreeSWITCH-users at lists.freeswitch.org>
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>
>>
>> ------------------------------------------------------------------------
>>
>> MailScanner Signature HW *Heriot-Watt University is the
>> Sunday Times
>>
>> Scottish University of the Year 2011-2012
>> *
>>
>> Heriot-Watt University is a Scottish charity
>> registered under charity number SC000278.
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org <mailto:consulting at freeswitch.org>
>> http://www.freeswitchsolutions.com
>>
>>
>>
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> <mailto:FreeSWITCH-users at lists.freeswitch.org>
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>>
>>
>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>>
>>
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>
>
> ------------------------------------------------------------------------
>
> MailScanner Signature HW *Heriot-Watt University is the Sunday Times
> Scottish University of the Year 2011-2012*
>
> Heriot-Watt University is a Scottish charity
> registered under charity number SC000278.
>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
>
>
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
--
Heriot-Watt University is a Scottish charity
registered under charity number SC000278.
Heriot-Watt University is the Sunday Times
Scottish University of the Year 2011-2012
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111118/0ea4952d/attachment-0001.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 4803 bytes
Desc: not available
Url : http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111118/0ea4952d/attachment-0003.jpe
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 4803 bytes
Desc: not available
Url : http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111118/0ea4952d/attachment-0004.jpe
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 4803 bytes
Desc: not available
Url : http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111118/0ea4952d/attachment-0005.jpe
-------------- next part --------------
A non-text attachment was scrubbed...
Name: hw_uni_of_year.jpg
Type: image/jpeg
Size: 4803 bytes
Desc: not available
Url : http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111118/0ea4952d/attachment-0001.jpg
Join us at ClueCon 2011 Aug 9-11, 2011
More information about the FreeSWITCH-users
mailing list