[Freeswitch-users] How to implement TTS barge-in using FS ESL
xl127
x.liu at hw.ac.uk
Fri Nov 18 22:09:44 MSK 2011
Now I see the reason for localhost not working with MRCP server is that
the URI address is sent to the MRCP server rather than used to fetch the
grammars in FS side.
(if just using grammar name, it seems the grammar is fetched from FS
side and sent to MRCP server)
so the localhost means the MRCP server, not my FS (with Tomcat server
running) machine.
The problem for Voxeo Prophecy:
From the Prophecy log, it has
http:\\x.x.x.x:8084\Iced_FS_AsrSluTts\grammars\FS_Nuance\pizza_order_voxeo.gram
MRCPServerAPP/Couldn't process new grammar - Content-Id header is missing.
MRCPServerAPP/Failed to start speech recognition. Could not process the
grammar specified.
It looks like the http URI was translated into a format of a file directory.
Is this because of FS or because of Voxeo Prophecy MRCP Server?
On 18/11/11 17:33, xl127 wrote:
> I want to specify <grammar> for play_and_detect_speech using URI format.
>
> I am running a Tomcat server that hosts the grammars and I am sure it
> can be accessed from local machine or from a different machine.
>
> If I use http://localhost:8084/grammars/FS_Nuance/myGrammar.gram
> or http://127.0.0.1:8084/grammars/FS_Nuance/myGrammar.gram
>
> FS got grammar loading failure (MRCP/1.0 1 407 COMPLETE)
>
> If I change localhost to my concrete external IP, it works for Nuance
> server, but not for Voxeo Prophecy server.
>
> I also have difficulties to specify URI grammars for detect_speech.
>
> Any suggestions please?
>
> Thanks,
> Xing
>
>
> On 17/11/11 16:21, xl127 wrote:
>> Yeah, I see, it should be set to true there. Thanks for reminding me
>> this.
>>
>>
>> On 17/11/11 15:48, Christopher Rienzo wrote:
>>> Actually in that silence prompt suggestion I gave, use
>>> {start-input-timers=true}
>>>
>>>
>>> On Thu, Nov 17, 2011 at 8:37 AM, Christopher Rienzo
>>> <cmrienzo at gmail.com <mailto:cmrienzo at gmail.com>> wrote:
>>>
>>>
>>>
>>>
>>> After some more tests, I found followings:
>>>
>>> 1. Testing from dialplan, the log output is the string of
>>> "CRIT ${speech_detect_result}" rather than the recognition
>>> results.
>>>
>>>
>>> oops
>>>
>>>
>>> 2. Regarding to non barge in, I first send "speak" command
>>> then send play_and_detect_speech with parameter:
>>> detect:unimrcp:nuance5-mrcp1-1
>>> {start-input-timers=false,no-input-timeout=25000,recognition-timeout=25000}dudeYNNC_Nuance
>>>
>>> in which I removed the "say:" part.
>>>
>>> But very soon I received event
>>> CHANNEL_EXECUTE_COMPLETE for play_and_detect_speech before
>>> the text playing finishes,
>>> and speech_detect_result is null (actually the event
>>> header does not contain this variable)
>>>
>>> I tried with "say:" part with empty text like
>>> "say:unimrcp:en-GB: " but it doesn't work, see issue 3 below.
>>>
>>>
>>> Try with silence as I originally suggested:
>>>
>>> silence_stream://1000 detect:unimrcp:nuance5-mrcp1-1
>>> {start-input-timers=false,no-input-timeout=25000,
>>> recognition-timeout=25000}dudeYNNC_Nuance
>>>
>>>
>>> 3. It seems that I can not send "speak" command twice
>>> before the first one finishes. In my case I run one port TTS
>>> server on one machine.
>>>
>>> If I send the command twice FS will give me Synthesizer
>>> Error/Invalid TTS Module.
>>> I thought the TTS request would be queued rather than
>>> it immediately looks for the TTS resource.
>>>
>>> If I send the second command to another TTS machine, no
>>> error occurs but I can only hear one utterance being spoken,
>>> it looks like one utteraance was dropped somehow.
>>>
>>>
>>> Wait for speak to finish before starting a new one.
>>>
>>>
>>> On 16/11/11 16:51, xl127 wrote:
>>>> Hi Christopher,
>>>>
>>>> The questions are cleared to me now. Many thanks for your
>>>> explanations!
>>>>
>>>> Best regards,
>>>>
>>>> Xing
>>>>
>>>>
>>>> On 16/11/11 15:52, Christopher Rienzo wrote:
>>>>>
>>>>> Responses inline
>>>>>
>>>>>
>>>>> Now it works in my ESL app though I am just able to do
>>>>> one dialogue ( I need to add the event catching for
>>>>> furthur dialgoues).
>>>>>
>>>>> I have a couple of questions here:
>>>>>
>>>>> 1. In the first try, my Nuance server was able to be
>>>>> accessed somehow (FS says the MRCP is not responding
>>>>> in 5000ms,
>>>>> something like that), then FS says: [WARNING]
>>>>> rtsp_client.c:386 () Failed to Connect to RTSP Server
>>>>> *MailScanner has detected a possible fraud attempt
>>>>> from "99.185.85.31:554" claiming to be* *MailScanner
>>>>> warning: numerical links are often malicious:*
>>>>> 99.185.85.31:554 <http://99.185.85.31:554>,
>>>>>
>>>>> later FS says:
>>>>> [ERR] mod_unimrcp.c:1860 (TTS-6) SYNTHESIZER
>>>>> channel error!
>>>>> [ERR] switch_ivr_play_say.c:2439 Invalid TTS module!
>>>>>
>>>>> The SYNTHESIZER channel error and Invalid TTS
>>>>> module error are obvious.
>>>>>
>>>>> What I don't understand is why it went to this
>>>>> stange address: *MailScanner has detected a possible
>>>>> fraud attempt from "99.185.85.31:554" claiming to be*
>>>>> *MailScanner warning: numerical links are often
>>>>> malicious:* 99.185.85.31:554 <http://99.185.85.31:554>?
>>>>>
>>>>>
>>>>> check your unimrcp configuration. Make sure the default
>>>>> TTS and ASR profiles are set to actual servers.
>>>>>
>>>>> 2. I specified TTS engine in play_and_detect_speech as
>>>>> "say:unimrcp:nuance5-mrcp1-1: the text to speak"
>>>>> It works though I didn't specify the TTS voice.
>>>>>
>>>>> How do I specify the TTS voice? In the mrcp
>>>>> profile (how?)? or something like:
>>>>> "say:unimrcp:nuance5-mrcp1-1:Serena: the text
>>>>> to speak" (this seems not right.)
>>>>>
>>>>>
>>>>> That won't work. Set the tts_engine variable as I
>>>>> explained previously, or use say:unimrcp:voice:text to
>>>>> speak with the desired voice and the correct default TTS
>>>>> profile defined in unimrcp.conf.xml. This is a limitation
>>>>> of the say: notation. Alternatively, the voice can be
>>>>> defined with the tts_voice channel variable.
>>>>>
>>>>> 3. The barge-in works well, thanks!. Is the barge-in
>>>>> configurable? In some scenarios, we might not allow
>>>>> barge-in.
>>>>>
>>>>>
>>>>> If you don't want to barge in, just do "playback (or
>>>>> speak)" first, then "play_and_detect_speech" with a
>>>>> silence prompt.
>>>>>
>>>>>
>>>>> 4. How could I get the text which has spoken to the
>>>>> user when barge-in occurs?
>>>>> Or Could I get the time when barge-in occurs? If
>>>>> I know the barge-in time and rough totale time for the
>>>>> whole text
>>>>> to be spoken I can figure out the spoken text by
>>>>> manually checking the recorded audio file later, which
>>>>> would be painful.
>>>>>
>>>>>
>>>>> If this is necessary, you might want to use the
>>>>> lower-level functions instead to watch for the
>>>>> begin-speaking event.
>>>>>
>>>>>
>>>>> 5. when I use "speak" and "detect_speech" apps in
>>>>> ESL, I can catch event: DETECTED_SPEECH and
>>>>> speech-type: begin-speaking
>>>>> and "detected-speech", then I do the recognition
>>>>> results processing.
>>>>>
>>>>> The new app play_and_detect_speech seems not
>>>>> generate these events any more. The way that I can
>>>>> think of to get the results
>>>>> is to catch event:CHANNEL_EXECUTE_COMPLETE then
>>>>> check if
>>>>> variable_current_application=play_and_detect_speech,
>>>>> then get
>>>>> the results from variable_detect_speech_result.
>>>>>
>>>>> Is this the proper way to get the results in ESL
>>>>> app? Or will play_and_detect_speech later on be
>>>>> consistent with detect_speech
>>>>> in term of ASR events?
>>>>>
>>>>>
>>>>> play_and_detect_speech is a higher level abstraction to
>>>>> simplify things. If you want to have more control, go
>>>>> back to using the ESL events. Reading the code in
>>>>> mod_dptools and switch_ivr_async will give you hints about
>>>>> how to do it correctly.
>>>>>
>>>>>
>>>>> 6. I'd like to set start-input-timers=false in the
>>>>> initial request then start the recognition timers
>>>>> (start-input-timers=true)
>>>>> after the TTS finishes.
>>>>> How possibly could I do this?
>>>>>
>>>>>
>>>>> This is automatically done in the
>>>>> switch_ivr_play_and_detect_speech() function. You just
>>>>> need to specify start-input-timers=false in the beginning.
>>>>>
>>>>>
>>>>>
>>>>> _________________________________________________________________________
>>>>> Professional FreeSWITCH Consulting Services:
>>>>> consulting at freeswitch.org <mailto:consulting at freeswitch.org>
>>>>> http://www.freeswitchsolutions.com
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Official FreeSWITCH Sites
>>>>> http://www.freeswitch.org
>>>>> http://wiki.freeswitch.org
>>>>> http://www.cluecon.com
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org <mailto:FreeSWITCH-users at lists.freeswitch.org>
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> http://www.freeswitch.org
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>> Scottish University of the Year 2011-12 *Heriot-Watt
>>>> University is the Sunday Times
>>>> Scottish University of the Year 2011-2012*
>>>>
>>>> Heriot-Watt University is a Scottish charity
>>>> registered under charity number SC000278.
>>>>
>>>>
>>>> _________________________________________________________________________
>>>> Professional FreeSWITCH Consulting Services:
>>>> consulting at freeswitch.org <mailto:consulting at freeswitch.org>
>>>> http://www.freeswitchsolutions.com
>>>>
>>>>
>>>>
>>>>
>>>> Official FreeSWITCH Sites
>>>> http://www.freeswitch.org
>>>> http://wiki.freeswitch.org
>>>> http://www.cluecon.com
>>>>
>>>> FreeSWITCH-users mailing list
>>>> FreeSWITCH-users at lists.freeswitch.org <mailto:FreeSWITCH-users at lists.freeswitch.org>
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> http://www.freeswitch.org
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> MailScanner Signature HW *Heriot-Watt University is the
>>> Sunday Times
>>>
>>> Scottish University of the Year 2011-2012
>>> *
>>>
>>> Heriot-Watt University is a Scottish charity
>>> registered under charity number SC000278.
>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org <mailto:consulting at freeswitch.org>
>>> http://www.freeswitchsolutions.com
>>>
>>>
>>>
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://wiki.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> <mailto:FreeSWITCH-users at lists.freeswitch.org>
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>>
>>>
>>>
>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org
>>> http://www.freeswitchsolutions.com
>>>
>>>
>>>
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://wiki.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>
>>
>> ------------------------------------------------------------------------
>>
>> MailScanner Signature HW *Heriot-Watt University is the Sunday Times
>> Scottish University of the Year 2011-2012*
>>
>> Heriot-Watt University is a Scottish charity
>> registered under charity number SC000278.
>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>>
>>
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>
>
> ------------------------------------------------------------------------
>
> MailScanner Signature HW *Heriot-Watt University is the Sunday Times
> Scottish University of the Year 2011-2012*
>
> Heriot-Watt University is a Scottish charity
> registered under charity number SC000278.
>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
>
>
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
--
Heriot-Watt University is a Scottish charity
registered under charity number SC000278.
Heriot-Watt University is the Sunday Times
Scottish University of the Year 2011-2012
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111118/8dfa726c/attachment-0001.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 4803 bytes
Desc: not available
Url : http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111118/8dfa726c/attachment-0004.jpe
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 4803 bytes
Desc: not available
Url : http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111118/8dfa726c/attachment-0005.jpe
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 4803 bytes
Desc: not available
Url : http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111118/8dfa726c/attachment-0006.jpe
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 4803 bytes
Desc: not available
Url : http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111118/8dfa726c/attachment-0007.jpe
-------------- next part --------------
A non-text attachment was scrubbed...
Name: hw_uni_of_year.jpg
Type: image/jpeg
Size: 4803 bytes
Desc: not available
Url : http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111118/8dfa726c/attachment-0001.jpg
Join us at ClueCon 2011 Aug 9-11, 2011
More information about the FreeSWITCH-users
mailing list