[Freeswitch-users] Record and live stream WAV to HTTP server

Anthony Minessale anthony.minessale at gmail.com
Wed Apr 6 20:10:39 MSD 2016 

Firstly,

You cannot stream WAV.  Its not possible.  WAV is not a streaming format,
its a container.  Its basically a raw audio data stream with a header
explaining the characteristics of the stream.


Next,

You need to compare your specific needs to the goal of the software to be
flexible and usable to all.

The reason the speech interface was suggested is that the point of the
speech interface is precisely to tap into live audio streams and perform
asynchronous ASR operations while the original call does whatever it wants
like talk to another caller or listen to a file playing etc.

What you envision is more of a synchronous single instruction to record a
file in realtime to the remote server over httapi.
The considerations to implement this would be what format to send the audio
in. The source audio could be any combination of sample rate, channel count
and encoding format.
To record we would translate the encoded format into signed linear at the
original input sample rate and channels.  These variances must be
communicated before you start sending the data so it would be a matter of
figuring out an existing standard or implement a proprietary one.





On Wed, Apr 6, 2016 at 10:42 AM, Vincent Gire <vincent.gire at gmail.com>
wrote:

> We already handle the multiple providers on the HTTP server.
> So you would recommend to implement a module against the speech interface
> streaming to the HTTP server ?
> Wouldn't it be easier (and less redundant) to add streaming capabilities
> to mod_httpapi ?
>
> On Wed, Apr 6, 2016 at 12:22 AM, Michael Jerris <mike at jerris.com> wrote:
>
>> The best way to handle this is probably to write a custom module against
>> the speech interface that sends the streams like you are describing, and
>> supports multiple providers.  How exactly do you go about sending to
>> multiple at the same time and combining the results, as this might actually
>> be the trickiest part as youll need to integrate that into a module that
>> uses the speech interface to be able to at all sanely handle that in
>> freeswitch
>>
>> On Apr 5, 2016, at 1:24 PM, Vincent Gire <vincent.gire at gmail.com> wrote:
>>
>> We use multiple ones to compare confidence.
>> They all support chunk transfer and provide a HTTP transaction similar to
>> :
>> http://developer.att.com/apis/speech/docs/v3
>> Look at 3/ Make API Calls
>>
>> On Tue, Apr 5, 2016 at 7:14 PM, Michael Jerris <mike at jerris.com> wrote:
>>
>>> Unimrcp isn't going to provide the interface he's talking about.  Can I
>>> ask which engine this is?
>>>
>>>
>>> On Tuesday, April 5, 2016, Sergey Safarov <s.safarov at gmail.com> wrote:
>>>
>>>> Look at http://www.unimrcp.org/ and
>>>> https://wiki.freeswitch.org/wiki/Mod_unimrcp
>>>>
>>>> On Tue, Apr 5, 2016, 19:38 Vincent Gire <vincent.gire at gmail.com> wrote:
>>>>
>>>>> We are building an IVR completely driven by ASR.
>>>>> ASR is performed in a distant location by a HTTP service (supporting
>>>>> chunked transfer) and adds an incompressible latency. We would like to
>>>>> stream the record to the ASR service as soon as it starts to reduce the
>>>>> overall latency before response.
>>>>> Does it make sense ?
>>>>>
>>>>>
>>>>> On Tue, Apr 5, 2016 at 5:39 PM, Michael Jerris <mike at jerris.com>
>>>>> wrote:
>>>>>
>>>>>> I would stay away from mod_vlc.  Its audio portions with recording
>>>>>> have known issues.  We do use the rtmp streaming in mod_av heavily but
>>>>>> thats obviously not wav.  Can you explain a bit more why you have this
>>>>>> requirement?
>>>>>>
>>>>>> On Apr 5, 2016, at 5:36 AM, Vincent Gire <vincent.gire at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Ok thanks.
>>>>>> It looks promising !
>>>>>> I'll dig into mod_vlc.
>>>>>>
>>>>>> Best
>>>>>>
>>>>>> Vincent
>>>>>>
>>>>>> On Tue, Apr 5, 2016 at 10:04 AM, Sergey Safarov <s.safarov at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Think is requred streaming feature of freeswitch.
>>>>>>> Look at mod_esf
>>>>>>> <https://freeswitch.org/confluence/display/FREESWITCH/mod_esf> and
>>>>>>> mod_vlc <https://wiki.freeswitch.org/wiki/Mod_vlc>
>>>>>>> Instruction to compile mod_vlc on provided link is to old but
>>>>>>> helpfull to undestand how to stream media to http server.
>>>>>>>
>>>>>>> For compiling mod_vlc please use vlc repo
>>>>>>> <https://freeswitch.org/stash/projects/SD/repos/vlc/browse> and centos
>>>>>>> instruction
>>>>>>> <https://freeswitch.org/confluence/display/FREESWITCH/Creating+RPM+repositary>
>>>>>>> .
>>>>>>> After you intall vlc, then you can enable mod_vlc module in
>>>>>>> freeswitch sources(SPEC file) and compile freeswitch.
>>>>>>>
>>>>>>> Sergey
>>>>>>>
>>>>>>>
>>>>>>> вт, 5 апр. 2016 г. в 10:31, Vincent Gire <vincent.gire at gmail.com>:
>>>>>>>
>>>>>>>> webdav, mod_http_cache or mod_httapi all results in sending the
>>>>>>>> recording only *after* it is complete.
>>>>>>>> They all write the recording to a file, wait for the recording to
>>>>>>>> complete and the file to close and then send it over HTTP.
>>>>>>>>
>>>>>>>> I would like to start sending the recording to the remove server as
>>>>>>>> soon as it starts (max 1 sec latency).
>>>>>>>> mod_http_cache or mod_httapi would be perfect if they were
>>>>>>>> streaming the recording like mod_shout.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Apr 4, 2016 at 8:50 PM, Sergey Safarov <s.safarov at gmail.com
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>> Input/output latency is not problem.  I use Kazoo on my servers
>>>>>>>>> and call recording is stored to database during 5 seconds after hangup.
>>>>>>>>> What is broken in your case if save file using webdav or
>>>>>>>>> http_cache?
>>>>>>>>>
>>>>>>>>> On Mon, Apr 4, 2016, 21:10 Vincent Gire <vincent.gire at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hello Sergey,
>>>>>>>>>>
>>>>>>>>>> Thank you for your answer.
>>>>>>>>>> I've looked into webdav mounted filesystem.
>>>>>>>>>>
>>>>>>>>>> Unfortunately, most WebDav clients (especially davfs2 on debian)
>>>>>>>>>> do a lot of buffering, caching and even lock-null requests (lock a non
>>>>>>>>>> existent resource before writing to it). I also suspect that they wait for
>>>>>>>>>> the end of the write operation.
>>>>>>>>>> The result is a latency of a few seconds witch is not much better
>>>>>>>>>> than what I achieve with mod_shout if I transcode the MP3.
>>>>>>>>>>
>>>>>>>>>> Any other idea ?
>>>>>>>>>>
>>>>>>>>>> Thank you !
>>>>>>>>>>
>>>>>>>>>> Best regards
>>>>>>>>>>
>>>>>>>>>> Vincent
>>>>>>>>>>
>>>>>>>>>> On Sun, Apr 3, 2016 at 7:30 PM, Sergey Safarov <
>>>>>>>>>> s.safarov at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Please look at webdav mounted filesystem.
>>>>>>>>>>>
>>>>>>>>>>> On Sun, Apr 3, 2016, 19:17 Vincent Gire <vincent.gire at gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you to all contributing to FreeSWITCH !
>>>>>>>>>>>>
>>>>>>>>>>>> I'm working on a IVR project where logic is implemented on a
>>>>>>>>>>>> HTTP server.
>>>>>>>>>>>> We are leaving Twilio because we now need to record and live
>>>>>>>>>>>> stream the session to the HTTP server in WAV format (chunked transfer
>>>>>>>>>>>> encoding).
>>>>>>>>>>>>
>>>>>>>>>>>> *mod_httapi* looks great (HT TAPI very similar to Twilio's)
>>>>>>>>>>>> but it seems that the records are first saved to disk before there are sent
>>>>>>>>>>>> to the server as chunked data.
>>>>>>>>>>>> We need the transfer to start as soon as the recording starts.
>>>>>>>>>>>>
>>>>>>>>>>>> *mod_shout* does start the request almost as the records
>>>>>>>>>>>> starts but it does not support WAV file and shout:// is not exactly a HTTP
>>>>>>>>>>>> request (SOURCE method instead of PUT).
>>>>>>>>>>>>
>>>>>>>>>>>> Is there a way to use these modules to achieve our goal ?
>>>>>>>>>>>>
>>>>>>>>>>>> If not, we are willing to author a specific module or rather
>>>>>>>>>>>> contribute to the existing ones.
>>>>>>>>>>>>
>>>>>>>>>>>> We've identified two approaches:
>>>>>>>>>>>>
>>>>>>>>>>>>    1. From *mod_httapi* Modify mod_httapi to directly stream
>>>>>>>>>>>>    the record instead of completely saving it to disk before the HTTP chunked
>>>>>>>>>>>>    transfer starts.
>>>>>>>>>>>>    This seems the most logical but with more than 3000 lines,
>>>>>>>>>>>>    mod_httapi does not seem to be the easiest module to build upon for
>>>>>>>>>>>>    newcomers!
>>>>>>>>>>>>
>>>>>>>>>>>>    2. From *mod_shout*
>>>>>>>>>>>>       1. Modify libshoot to replace the custom SOURCE method
>>>>>>>>>>>>       with standard HTTP PUT method
>>>>>>>>>>>>       2. Modify mod_shout to support wav files
>>>>>>>>>>>>       3. Implement our IVR in script (javascript/lua)
>>>>>>>>>>>>
>>>>>>>>>>>> What do you think ?
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you for your help.
>>>>>>>>>>>>
>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>
>>>>>> _________________________________________________________________________
>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>> consulting at freeswitch.org
>>>>>> http://www.freeswitchsolutions.com
>>>>>>
>>>>>> Official FreeSWITCH Sites
>>>>>> http://www.freeswitch.org
>>>>>> http://confluence.freeswitch.org
>>>>>> http://www.cluecon.com
>>>>>>
>>>>>> FreeSWITCH-users mailing list
>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>> UNSUBSCRIBE:
>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>> http://www.freeswitch.org
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Vincent Gire
>>>>>
>>>>> _________________________________________________________________________
>>>>> Professional FreeSWITCH Consulting Services:
>>>>> consulting at freeswitch.org
>>>>> http://www.freeswitchsolutions.com
>>>>>
>>>>> Official FreeSWITCH Sites
>>>>> http://www.freeswitch.org
>>>>> http://confluence.freeswitch.org
>>>>> http://www.cluecon.com
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:
>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> http://www.freeswitch.org
>>>>
>>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org
>>> http://www.freeswitchsolutions.com
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://confluence.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>
>>
>>
>> --
>> Vincent Gire
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://confluence.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://confluence.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>
>
>
> --
> Vincent Gire
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://confluence.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>



-- 
Anthony Minessale II       ♬ @anthmfs  ♬ @FreeSWITCH  ♬

☞ http://freeswitch.org/  ☞ http://cluecon.com/  ☞
http://twitter.com/FreeSWITCH
☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+
<http://freeswitch.org/g+>*

ClueCon Weekly Development Call
☎ sip:888 at conference.freeswitch.org  ☎ +19193869900

https://www.youtube.com/watch?v=9XXgW34t40s
https://www.youtube.com/watch?v=NLaDpGQuZDA
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20160406/7aabdb4d/attachment-0001.html

Join us at ClueCon 2016 Aug 8-12, 2016
More information about the FreeSWITCH-users mailing list