[Freeswitch-users] How to implement TTS barge-in using FS ESL

xl127 x.liu at hw.ac.uk
Fri Nov 18 20:33:20 MSK 2011


I want to specify <grammar> for play_and_detect_speech using URI format.

I am running a Tomcat server that hosts the grammars and I am sure it 
can be accessed from local machine or from a different machine.

If I use http://localhost:8084/grammars/FS_Nuance/myGrammar.gram
  or http://127.0.0.1:8084/grammars/FS_Nuance/myGrammar.gram

FS got grammar loading failure (MRCP/1.0 1 407 COMPLETE)

If I change localhost to my concrete external IP, it works for Nuance 
server, but not for Voxeo Prophecy server.

I also have difficulties to specify URI grammars for detect_speech.

Any suggestions please?

Thanks,
Xing


On 17/11/11 16:21, xl127 wrote:
> Yeah, I see, it should be set to true there. Thanks for reminding me this.
>
>
> On 17/11/11 15:48, Christopher Rienzo wrote:
>> Actually in that silence prompt suggestion I gave, use 
>> {start-input-timers=true}
>>
>>
>> On Thu, Nov 17, 2011 at 8:37 AM, Christopher Rienzo 
>> <cmrienzo at gmail.com <mailto:cmrienzo at gmail.com>> wrote:
>>
>>
>>
>>
>>           After some more tests, I found followings:
>>
>>           1. Testing from dialplan, the log output is the string of
>>         "CRIT ${speech_detect_result}" rather than the recognition
>>         results.
>>
>>
>>       oops
>>
>>
>>           2. Regarding to non barge in, I first send "speak" command
>>         then send play_and_detect_speech with parameter:
>>               detect:unimrcp:nuance5-mrcp1-1
>>         {start-input-timers=false,no-input-timeout=25000,recognition-timeout=25000}dudeYNNC_Nuance
>>
>>               in which I removed the "say:" part.
>>
>>               But very soon I received event CHANNEL_EXECUTE_COMPLETE
>>         for play_and_detect_speech before the text playing finishes,
>>               and speech_detect_result is null (actually the event
>>         header does not contain this variable)
>>
>>               I tried with "say:" part with empty text like
>>         "say:unimrcp:en-GB: " but it doesn't work, see issue 3 below.
>>
>>
>>     Try with silence as I originally suggested:
>>
>>     silence_stream://1000 detect:unimrcp:nuance5-mrcp1-1
>>     {start-input-timers=false,no-input-timeout=25000,
>>     recognition-timeout=25000}dudeYNNC_Nuance
>>
>>
>>           3. It seems that I can not send "speak" command twice
>>         before the first one finishes. In my case I run one port TTS
>>         server on one machine.
>>
>>              If I send the command twice FS will give me Synthesizer
>>         Error/Invalid TTS Module.
>>              I thought the TTS request would be queued rather than it
>>         immediately looks for the TTS resource.
>>
>>              If I send the second command to another TTS machine, no
>>         error occurs but I can only hear one utterance being spoken,
>>              it looks like one utteraance was dropped somehow.
>>
>>
>>     Wait for speak to finish before starting a new one.
>>
>>
>>         On 16/11/11 16:51, xl127 wrote:
>>>         Hi Christopher,
>>>
>>>         The questions are cleared to me now. Many thanks for your
>>>         explanations!
>>>
>>>         Best regards,
>>>
>>>         Xing
>>>
>>>
>>>         On 16/11/11 15:52, Christopher Rienzo wrote:
>>>>
>>>>         Responses inline
>>>>
>>>>
>>>>             Now it works in my ESL app though I am just able to do
>>>>             one dialogue ( I need to add the event catching for
>>>>             furthur dialgoues).
>>>>
>>>>             I have a couple of questions here:
>>>>
>>>>               1. In the first try, my Nuance server was able to be
>>>>             accessed somehow (FS says the MRCP is not responding in
>>>>             5000ms,
>>>>                 something like that), then FS says: [WARNING]
>>>>             rtsp_client.c:386 () Failed to Connect to RTSP Server
>>>>             *MailScanner has detected a possible fraud attempt from
>>>>             "99.185.85.31:554" claiming to be* *MailScanner
>>>>             warning: numerical links are often malicious:*
>>>>             99.185.85.31:554 <http://99.185.85.31:554>,
>>>>
>>>>                 later FS says:
>>>>                  [ERR] mod_unimrcp.c:1860 (TTS-6) SYNTHESIZER
>>>>             channel error!
>>>>                  [ERR] switch_ivr_play_say.c:2439 Invalid TTS module!
>>>>
>>>>                The SYNTHESIZER channel error and Invalid TTS module
>>>>             error are obvious.
>>>>
>>>>                 What I don't understand is why it went to this
>>>>             stange address: *MailScanner has detected a possible
>>>>             fraud attempt from "99.185.85.31:554" claiming to be*
>>>>             *MailScanner warning: numerical links are often
>>>>             malicious:* 99.185.85.31:554 <http://99.185.85.31:554>?
>>>>
>>>>
>>>>         check your unimrcp configuration.  Make sure the default
>>>>         TTS and ASR profiles are set to actual servers.
>>>>
>>>>               2. I specified TTS engine in play_and_detect_speech as
>>>>                      "say:unimrcp:nuance5-mrcp1-1: the text to speak"
>>>>                  It works though I didn't specify the TTS voice.
>>>>
>>>>                  How do I specify the TTS voice? In the mrcp
>>>>             profile (how?)? or something like:
>>>>                      "say:unimrcp:nuance5-mrcp1-1:Serena: the text
>>>>             to speak" (this seems not right.)
>>>>
>>>>
>>>>         That won't work.  Set the tts_engine variable as I
>>>>         explained previously, or use say:unimrcp:voice:text to
>>>>         speak with the desired voice and the correct default TTS
>>>>         profile defined in unimrcp.conf.xml.  This is a limitation
>>>>         of the say: notation.  Alternatively, the voice can be
>>>>         defined with the tts_voice channel variable.
>>>>
>>>>               3. The barge-in works well, thanks!. Is the barge-in
>>>>             configurable? In some scenarios, we might not allow
>>>>             barge-in.
>>>>
>>>>
>>>>         If you don't want to barge in, just do "playback (or
>>>>         speak)" first, then "play_and_detect_speech" with a silence
>>>>         prompt.
>>>>
>>>>
>>>>               4. How could I get the text which has spoken to the
>>>>             user when barge-in occurs?
>>>>                  Or Could I get the time when barge-in occurs? If I
>>>>             know the barge-in time and rough totale time for the
>>>>             whole text
>>>>                  to be spoken I can figure out the spoken text by
>>>>             manually checking the recorded audio file later, which
>>>>             would be painful.
>>>>
>>>>
>>>>         If this is necessary, you might want to use the lower-level
>>>>         functions instead to watch for the begin-speaking event.
>>>>
>>>>
>>>>               5. when I use "speak" and "detect_speech" apps in
>>>>             ESL, I can catch event: DETECTED_SPEECH and
>>>>             speech-type: begin-speaking
>>>>                  and "detected-speech", then I do the recognition
>>>>             results processing.
>>>>
>>>>                 The new app play_and_detect_speech seems not
>>>>             generate these events any more. The way that I can
>>>>             think of to get the results
>>>>                 is to catch event:CHANNEL_EXECUTE_COMPLETE then
>>>>             check if
>>>>             variable_current_application=play_and_detect_speech,
>>>>             then get
>>>>                 the results from variable_detect_speech_result.
>>>>
>>>>                 Is this the proper way to get the results in ESL
>>>>             app? Or will play_and_detect_speech later on be
>>>>             consistent with detect_speech
>>>>                 in term of ASR events?
>>>>
>>>>
>>>>         play_and_detect_speech is a higher level abstraction to
>>>>         simplify things.  If you want to have more control, go back
>>>>         to using the ESL events.  Reading the code in mod_dptools
>>>>         and switch_ivr_async will give you hints about how to do it
>>>>         correctly.
>>>>
>>>>
>>>>               6. I'd like to set start-input-timers=false in the
>>>>             initial request then start the recognition timers
>>>>             (start-input-timers=true)
>>>>                  after the TTS finishes.
>>>>                  How possibly could I do this?
>>>>
>>>>
>>>>         This is automatically done in the 
>>>>         switch_ivr_play_and_detect_speech() function.  You just
>>>>         need to specify start-input-timers=false in the beginning.
>>>>
>>>>
>>>>
>>>>         _________________________________________________________________________
>>>>         Professional FreeSWITCH Consulting Services:
>>>>         consulting at freeswitch.org  <mailto:consulting at freeswitch.org>
>>>>         http://www.freeswitchsolutions.com
>>>>
>>>>         
>>>>         
>>>>
>>>>         Official FreeSWITCH Sites
>>>>         http://www.freeswitch.org
>>>>         http://wiki.freeswitch.org
>>>>         http://www.cluecon.com
>>>>
>>>>         FreeSWITCH-users mailing list
>>>>         FreeSWITCH-users at lists.freeswitch.org  <mailto:FreeSWITCH-users at lists.freeswitch.org>
>>>>         http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>         UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>         http://www.freeswitch.org
>>>
>>>
>>>         ------------------------------------------------------------------------
>>>
>>>         Scottish University of the Year 2011-12 *Heriot-Watt
>>>         University is the Sunday Times
>>>           Scottish University of the Year 2011-2012*
>>>
>>>           Heriot-Watt University is a Scottish charity
>>>           registered under charity number SC000278.
>>>
>>>
>>>         _________________________________________________________________________
>>>         Professional FreeSWITCH Consulting Services:
>>>         consulting at freeswitch.org  <mailto:consulting at freeswitch.org>
>>>         http://www.freeswitchsolutions.com
>>>
>>>         
>>>         
>>>
>>>         Official FreeSWITCH Sites
>>>         http://www.freeswitch.org
>>>         http://wiki.freeswitch.org
>>>         http://www.cluecon.com
>>>
>>>         FreeSWITCH-users mailing list
>>>         FreeSWITCH-users at lists.freeswitch.org  <mailto:FreeSWITCH-users at lists.freeswitch.org>
>>>         http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>         UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>         http://www.freeswitch.org
>>
>>
>>         ------------------------------------------------------------------------
>>
>>         MailScanner Signature HW *Heriot-Watt University is the
>>         Sunday Times
>>
>>           Scottish University of the Year 2011-2012
>>         *
>>
>>           Heriot-Watt University is a Scottish charity
>>           registered under charity number SC000278.
>>
>>         _________________________________________________________________________
>>         Professional FreeSWITCH Consulting Services:
>>         consulting at freeswitch.org <mailto:consulting at freeswitch.org>
>>         http://www.freeswitchsolutions.com
>>
>>         
>>         
>>
>>         Official FreeSWITCH Sites
>>         http://www.freeswitch.org
>>         http://wiki.freeswitch.org
>>         http://www.cluecon.com
>>
>>         FreeSWITCH-users mailing list
>>         FreeSWITCH-users at lists.freeswitch.org
>>         <mailto:FreeSWITCH-users at lists.freeswitch.org>
>>         http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>         UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>         http://www.freeswitch.org
>>
>>
>>
>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> 
>> 
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>
>
> ------------------------------------------------------------------------
>
> MailScanner Signature HW *Heriot-Watt University is the Sunday Times
>   Scottish University of the Year 2011-2012*
>
>   Heriot-Watt University is a Scottish charity
>   registered under charity number SC000278.
>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> 
> 
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org



-- 
Heriot-Watt University is a Scottish charity
registered under charity number SC000278.

Heriot-Watt University is the Sunday Times
Scottish University of the Year 2011-2012


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111118/0ea4952d/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 4803 bytes
Desc: not available
Url : http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111118/0ea4952d/attachment-0003.jpe 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 4803 bytes
Desc: not available
Url : http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111118/0ea4952d/attachment-0004.jpe 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 4803 bytes
Desc: not available
Url : http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111118/0ea4952d/attachment-0005.jpe 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: hw_uni_of_year.jpg
Type: image/jpeg
Size: 4803 bytes
Desc: not available
Url : http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111118/0ea4952d/attachment-0001.jpg 


Join us at ClueCon 2011 Aug 9-11, 2011
More information about the FreeSWITCH-users mailing list