[Freeswitch-users] Record and live stream WAV to HTTP server

Anthony Minessale anthony.minessale at gmail.com
Wed Apr 6 20:59:53 MSD 2016 

On Wed, Apr 6, 2016 at 11:47 AM, Vincent Gire <vincent.gire at gmail.com>
wrote:

>
>
> On Wed, Apr 6, 2016 at 6:10 PM, Anthony Minessale <
> anthony.minessale at gmail.com> wrote:
>
>> Firstly,
>>
>> You cannot stream WAV.  Its not possible.  WAV is not a streaming format,
>> its a container.  Its basically a raw audio data stream with a header
>> explaining the characteristics of the stream.
>>
>
> Yes. Sorry for the language abuse and thank you for clearing this out.
>
>
>>
>>
>> Next,
>>
>> You need to compare your specific needs to the goal of the software to be
>> flexible and usable to all.
>>
>> The reason the speech interface was suggested is that the point of the
>> speech interface is precisely to tap into live audio streams and perform
>> asynchronous ASR operations while the original call does whatever it wants
>> like talk to another caller or listen to a file playing etc.
>>
>> What you envision is more of a synchronous single instruction to record a
>> file in realtime to the remote server over httapi.
>>
>
> Exactly.
>
>
>> The considerations to implement this would be what format to send the
>> audio in. The source audio could be any combination of sample rate, channel
>> count and encoding format.
>> To record we would translate the encoded format into signed linear at the
>> original input sample rate and channels.  These variances must be
>> communicated before you start sending the data so it would be a matter of
>> figuring out an existing standard or implement a proprietary one.
>>
>
> I am not sure I am following here.
> I might again have misused the word streaming.
> I was thinking that when we specify a http target for a record in
> mod_httapi, it could start sending the corresponding file (WAV or other) to
> the HTTP server as chunked data as soon as the record starts.
> In the case of a WAV target, the WAV header would be sent first as part of
> the first HTTP data chunks and the raw audio data stream would follow in
> the next data chunks.
> Does it make sense ?
>


I agree that you are not following as I gave a relatively lengthy and
specific response.  I know exactly what you are asking for so there is no
need to re-explain.
The way you ask for it is very specific to your own needs so it would not
make a very useful feature in FS.

Are you asking for someone to make this feature for you or are you asking
for advice on how you can implement it?
I think you are underestimating the complexity and potential scalability of
the overall project you are doing (not just the FS portion) and you should
review your entire architecture.




>
>
>>
>>
>>
>>
>> On Wed, Apr 6, 2016 at 10:42 AM, Vincent Gire <vincent.gire at gmail.com>
>> wrote:
>>
>>> We already handle the multiple providers on the HTTP server.
>>> So you would recommend to implement a module against the speech
>>> interface streaming to the HTTP server ?
>>> Wouldn't it be easier (and less redundant) to add streaming capabilities
>>> to mod_httpapi ?
>>>
>>> On Wed, Apr 6, 2016 at 12:22 AM, Michael Jerris <mike at jerris.com> wrote:
>>>
>>>> The best way to handle this is probably to write a custom module
>>>> against the speech interface that sends the streams like you are
>>>> describing, and supports multiple providers.  How exactly do you go about
>>>> sending to multiple at the same time and combining the results, as this
>>>> might actually be the trickiest part as youll need to integrate that into a
>>>> module that uses the speech interface to be able to at all sanely handle
>>>> that in freeswitch
>>>>
>>>> On Apr 5, 2016, at 1:24 PM, Vincent Gire <vincent.gire at gmail.com>
>>>> wrote:
>>>>
>>>> We use multiple ones to compare confidence.
>>>> They all support chunk transfer and provide a HTTP transaction similar
>>>> to :
>>>> http://developer.att.com/apis/speech/docs/v3
>>>> Look at 3/ Make API Calls
>>>>
>>>> On Tue, Apr 5, 2016 at 7:14 PM, Michael Jerris <mike at jerris.com> wrote:
>>>>
>>>>> Unimrcp isn't going to provide the interface he's talking about.  Can
>>>>> I ask which engine this is?
>>>>>
>>>>>
>>>>> On Tuesday, April 5, 2016, Sergey Safarov <s.safarov at gmail.com> wrote:
>>>>>
>>>>>> Look at http://www.unimrcp.org/ and
>>>>>> https://wiki.freeswitch.org/wiki/Mod_unimrcp
>>>>>>
>>>>>> On Tue, Apr 5, 2016, 19:38 Vincent Gire <vincent.gire at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> We are building an IVR completely driven by ASR.
>>>>>>> ASR is performed in a distant location by a HTTP service (supporting
>>>>>>> chunked transfer) and adds an incompressible latency. We would like to
>>>>>>> stream the record to the ASR service as soon as it starts to reduce the
>>>>>>> overall latency before response.
>>>>>>> Does it make sense ?
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Apr 5, 2016 at 5:39 PM, Michael Jerris <mike at jerris.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I would stay away from mod_vlc.  Its audio portions with recording
>>>>>>>> have known issues.  We do use the rtmp streaming in mod_av heavily but
>>>>>>>> thats obviously not wav.  Can you explain a bit more why you have this
>>>>>>>> requirement?
>>>>>>>>
>>>>>>>> On Apr 5, 2016, at 5:36 AM, Vincent Gire <vincent.gire at gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Ok thanks.
>>>>>>>> It looks promising !
>>>>>>>> I'll dig into mod_vlc.
>>>>>>>>
>>>>>>>> Best
>>>>>>>>
>>>>>>>> Vincent
>>>>>>>>
>>>>>>>> On Tue, Apr 5, 2016 at 10:04 AM, Sergey Safarov <
>>>>>>>> s.safarov at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Think is requred streaming feature of freeswitch.
>>>>>>>>> Look at mod_esf
>>>>>>>>> <https://freeswitch.org/confluence/display/FREESWITCH/mod_esf>
>>>>>>>>> and mod_vlc <https://wiki.freeswitch.org/wiki/Mod_vlc>
>>>>>>>>> Instruction to compile mod_vlc on provided link is to old but
>>>>>>>>> helpfull to undestand how to stream media to http server.
>>>>>>>>>
>>>>>>>>> For compiling mod_vlc please use vlc repo
>>>>>>>>> <https://freeswitch.org/stash/projects/SD/repos/vlc/browse> and centos
>>>>>>>>> instruction
>>>>>>>>> <https://freeswitch.org/confluence/display/FREESWITCH/Creating+RPM+repositary>
>>>>>>>>> .
>>>>>>>>> After you intall vlc, then you can enable mod_vlc module in
>>>>>>>>> freeswitch sources(SPEC file) and compile freeswitch.
>>>>>>>>>
>>>>>>>>> Sergey
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> вт, 5 апр. 2016 г. в 10:31, Vincent Gire <vincent.gire at gmail.com>:
>>>>>>>>>
>>>>>>>>>> webdav, mod_http_cache or mod_httapi all results in sending the
>>>>>>>>>> recording only *after* it is complete.
>>>>>>>>>> They all write the recording to a file, wait for the recording
>>>>>>>>>> to complete and the file to close and then send it over HTTP.
>>>>>>>>>>
>>>>>>>>>> I would like to start sending the recording to the remove server
>>>>>>>>>> as soon as it starts (max 1 sec latency).
>>>>>>>>>> mod_http_cache or mod_httapi would be perfect if they were
>>>>>>>>>> streaming the recording like mod_shout.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Apr 4, 2016 at 8:50 PM, Sergey Safarov <
>>>>>>>>>> s.safarov at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Input/output latency is not problem.  I use Kazoo on my servers
>>>>>>>>>>> and call recording is stored to database during 5 seconds after hangup.
>>>>>>>>>>> What is broken in your case if save file using webdav or
>>>>>>>>>>> http_cache?
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Apr 4, 2016, 21:10 Vincent Gire <vincent.gire at gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hello Sergey,
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you for your answer.
>>>>>>>>>>>> I've looked into webdav mounted filesystem.
>>>>>>>>>>>>
>>>>>>>>>>>> Unfortunately, most WebDav clients (especially davfs2 on
>>>>>>>>>>>> debian) do a lot of buffering, caching and even lock-null requests (lock a
>>>>>>>>>>>> non existent resource before writing to it). I also suspect that they wait
>>>>>>>>>>>> for the end of the write operation.
>>>>>>>>>>>> The result is a latency of a few seconds witch is not much
>>>>>>>>>>>> better than what I achieve with mod_shout if I transcode the MP3.
>>>>>>>>>>>>
>>>>>>>>>>>> Any other idea ?
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you !
>>>>>>>>>>>>
>>>>>>>>>>>> Best regards
>>>>>>>>>>>>
>>>>>>>>>>>> Vincent
>>>>>>>>>>>>
>>>>>>>>>>>> On Sun, Apr 3, 2016 at 7:30 PM, Sergey Safarov <
>>>>>>>>>>>> s.safarov at gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Please look at webdav mounted filesystem.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sun, Apr 3, 2016, 19:17 Vincent Gire <
>>>>>>>>>>>>> vincent.gire at gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thank you to all contributing to FreeSWITCH !
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm working on a IVR project where logic is implemented on a
>>>>>>>>>>>>>> HTTP server.
>>>>>>>>>>>>>> We are leaving Twilio because we now need to record and live
>>>>>>>>>>>>>> stream the session to the HTTP server in WAV format (chunked transfer
>>>>>>>>>>>>>> encoding).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *mod_httapi* looks great (HT TAPI very similar to Twilio's)
>>>>>>>>>>>>>> but it seems that the records are first saved to disk before there are sent
>>>>>>>>>>>>>> to the server as chunked data.
>>>>>>>>>>>>>> We need the transfer to start as soon as the recording starts.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> *mod_shout* does start the request almost as the records
>>>>>>>>>>>>>> starts but it does not support WAV file and shout:// is not exactly a HTTP
>>>>>>>>>>>>>> request (SOURCE method instead of PUT).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Is there a way to use these modules to achieve our goal ?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If not, we are willing to author a specific module or rather
>>>>>>>>>>>>>> contribute to the existing ones.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We've identified two approaches:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>    1. From *mod_httapi* Modify mod_httapi to directly stream
>>>>>>>>>>>>>>    the record instead of completely saving it to disk before the HTTP chunked
>>>>>>>>>>>>>>    transfer starts.
>>>>>>>>>>>>>>    This seems the most logical but with more than 3000
>>>>>>>>>>>>>>    lines, mod_httapi does not seem to be the easiest module to build upon for
>>>>>>>>>>>>>>    newcomers!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>    2. From *mod_shout*
>>>>>>>>>>>>>>       1. Modify libshoot to replace the custom SOURCE method
>>>>>>>>>>>>>>       with standard HTTP PUT method
>>>>>>>>>>>>>>       2. Modify mod_shout to support wav files
>>>>>>>>>>>>>>       3. Implement our IVR in script (javascript/lua)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> What do you think ?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thank you for your help.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>
>>>>>>>> _________________________________________________________________________
>>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>>> consulting at freeswitch.org
>>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>>
>>>>>>>> Official FreeSWITCH Sites
>>>>>>>> http://www.freeswitch.org
>>>>>>>> http://confluence.freeswitch.org
>>>>>>>> http://www.cluecon.com
>>>>>>>>
>>>>>>>> FreeSWITCH-users mailing list
>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>> UNSUBSCRIBE:
>>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>> http://www.freeswitch.org
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Vincent Gire
>>>>>>>
>>>>>>> _________________________________________________________________________
>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>> consulting at freeswitch.org
>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>
>>>>>>> Official FreeSWITCH Sites
>>>>>>> http://www.freeswitch.org
>>>>>>> http://confluence.freeswitch.org
>>>>>>> http://www.cluecon.com
>>>>>>>
>>>>>>> FreeSWITCH-users mailing list
>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>> UNSUBSCRIBE:
>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>> http://www.freeswitch.org
>>>>>>
>>>>>>
>>>>>
>>>>> _________________________________________________________________________
>>>>> Professional FreeSWITCH Consulting Services:
>>>>> consulting at freeswitch.org
>>>>> http://www.freeswitchsolutions.com
>>>>>
>>>>> Official FreeSWITCH Sites
>>>>> http://www.freeswitch.org
>>>>> http://confluence.freeswitch.org
>>>>> http://www.cluecon.com
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:
>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> http://www.freeswitch.org
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Vincent Gire
>>>>
>>>> _________________________________________________________________________
>>>> Professional FreeSWITCH Consulting Services:
>>>> consulting at freeswitch.org
>>>> http://www.freeswitchsolutions.com
>>>>
>>>> Official FreeSWITCH Sites
>>>> http://www.freeswitch.org
>>>> http://confluence.freeswitch.org
>>>> http://www.cluecon.com
>>>>
>>>> FreeSWITCH-users mailing list
>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> http://www.freeswitch.org
>>>>
>>>>
>>>>
>>>>
>>>> _________________________________________________________________________
>>>> Professional FreeSWITCH Consulting Services:
>>>> consulting at freeswitch.org
>>>> http://www.freeswitchsolutions.com
>>>>
>>>> Official FreeSWITCH Sites
>>>> http://www.freeswitch.org
>>>> http://confluence.freeswitch.org
>>>> http://www.cluecon.com
>>>>
>>>> FreeSWITCH-users mailing list
>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> http://www.freeswitch.org
>>>>
>>>
>>>
>>>
>>> --
>>> Vincent Gire
>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org
>>> http://www.freeswitchsolutions.com
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://confluence.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>
>>
>>
>> --
>> Anthony Minessale II       ♬ @anthmfs  ♬ @FreeSWITCH  ♬
>>
>> ☞ http://freeswitch.org/  ☞ http://cluecon.com/  ☞
>> http://twitter.com/FreeSWITCH
>> ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+
>> <http://freeswitch.org/g+>*
>>
>> ClueCon Weekly Development Call
>> ☎ sip:888 at conference.freeswitch.org  ☎ +19193869900
>>
>> https://www.youtube.com/watch?v=9XXgW34t40s
>> https://www.youtube.com/watch?v=NLaDpGQuZDA
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://confluence.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>
>
>
> --
> Vincent Gire
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://confluence.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>



-- 
Anthony Minessale II       ♬ @anthmfs  ♬ @FreeSWITCH  ♬

☞ http://freeswitch.org/  ☞ http://cluecon.com/  ☞
http://twitter.com/FreeSWITCH
☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+
<http://freeswitch.org/g+>*

ClueCon Weekly Development Call
☎ sip:888 at conference.freeswitch.org  ☎ +19193869900

https://www.youtube.com/watch?v=9XXgW34t40s
https://www.youtube.com/watch?v=NLaDpGQuZDA
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20160406/0a7876da/attachment-0001.html

Join us at ClueCon 2016 Aug 8-12, 2016
More information about the FreeSWITCH-users mailing list