Freeswitch support for text messages/commands between Verto/webRTC to/from conferenced backend SIP/RTP systems
We are wondering if FreeSWITCH supports sending and receiving text messages between Verto/webRTC to/from backend SIP/RTP systems that conferenced together? 1) We want to use this to send commands from the web browser to the backend systems, for example: a. "hey backend system, please turn on the ASR to start listening." 2) We also wonder if FreeSWITCH supports sending commands from the backend to the frontend (web browser), for example: a. "hey avatar, here are some visemes to tell you how you should move your lips, arms, ... at the same time as playing this audio" If both Verto/SIP or webRTC/RTP are supported, is either more ideal? For instance, if the text/commands are over RTC then maybe we can use one clock for all RTP streams and/or RTP sessions and use the RTP timestamps to synchronize the commands with audio/video. Or maybe Verto/SIP is better for commands. I see there are various RFCs for text over RTP and text over RTCDataChannel in webRTC, but not sure if FreeSWITCH supports any of these. I also see FreeSWITCH mod_verto support text messages, but not sure if FreeSWITCH sends these to/from SIP (e.g. maybe via SIMPLE SIP) or via RTCDataChannel? Robert ________________________________ This e-mail and any files transmitted with it may contain privileged or confidential information. It is solely for use by the individual for whom it is intended, even if addressed incorrectly. If you received this e-mail in error, please notify the sender; do not disclose, copy, distribute, or take any action in reliance on the contents of this information; and delete it from your system. Any other use of this e-mail is prohibited. Thank you for your compliance. ________________________________
I think the verto demo has text . On Wed, Feb 7, 2018 at 5:23 AM, Mundkowsky, Robert <rmundkowsky@ets.org> wrote:
We are wondering if FreeSWITCH supports sending and receiving text messages between Verto/webRTC to/from backend SIP/RTP systems that conferenced together?
1) We want to use this to send commands from the web browser to the backend systems, for example:
a. “hey backend system, please turn on the ASR to start listening.”
2) We also wonder if FreeSWITCH supports sending commands from the backend to the frontend (web browser), for example:
a. “hey avatar, here are some visemes to tell you how you should move your lips, arms, … at the same time as playing this audio”
If both Verto/SIP or webRTC/RTP are supported, is either more ideal?
For instance, if the text/commands are over RTC then maybe we can use one clock for all RTP streams and/or RTP sessions and use the RTP timestamps to synchronize the commands with audio/video. Or maybe Verto/SIP is better for commands.
I see there are various RFCs for text over RTP and text over RTCDataChannel in webRTC, but not sure if FreeSWITCH supports any of these.
I also see FreeSWITCH mod_verto support text messages, but not sure if FreeSWITCH sends these to/from SIP (e.g. maybe via SIMPLE SIP) or via RTCDataChannel?
Robert
------------------------------
This e-mail and any files transmitted with it may contain privileged or confidential information. It is solely for use by the individual for whom it is intended, even if addressed incorrectly. If you received this e-mail in error, please notify the sender; do not disclose, copy, distribute, or take any action in reliance on the contents of this information; and delete it from your system. Any other use of this e-mail is prohibited.
Thank you for your compliance. ------------------------------
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- About: http://about.me/dujinfang Blog: http://www.dujinfang.com Proj: http://www.freeswitch.org.cn
Well, I read up on this in the FreeSWITCH 1.8 book, seems VERTO webRTC clients can send text messages to SIP/RTP clients via messages out of band from the main audio/video stream. For SIP/RTP clients seems they get messages via SIP SIMPLE hence they are over SIP. And FreeSWITCH modules that support the FreeSWITCH Chat API can inter-communicate (mod_dptools: chat supports sip, verto, and maybe jingles protocol, mod_sms supports sms,…). I did not see anything about Message Session Relay Protocol, text over RTP, nor RTCDataChannel, so I assume the text messages likely need to be small. Robert Mundkowsky From: Mundkowsky, Robert Sent: Monday, February 12, 2018 11:02 PM To: FreeSWITCH Users Help <freeswitch-users@lists.freeswitch.org> Subject: RE: [Freeswitch-users] Freeswitch support for text messages/commands between Verto/webRTC to/from conferenced backend SIP/RTP systems Yes, but where does the text go for SIP/RTP users? Robert Mundkowsky From: FreeSWITCH-users [mailto:freeswitch-users-bounces@lists.freeswitch.org] On Behalf Of Seven Du Sent: Monday, February 12, 2018 8:07 PM To: FreeSWITCH Users Help <freeswitch-users@lists.freeswitch.org<mailto:freeswitch-users@lists.freeswitch.org>> Subject: Re: [Freeswitch-users] Freeswitch support for text messages/commands between Verto/webRTC to/from conferenced backend SIP/RTP systems I think the verto demo has text . On Wed, Feb 7, 2018 at 5:23 AM, Mundkowsky, Robert <rmundkowsky@ets.org<mailto:rmundkowsky@ets.org>> wrote: We are wondering if FreeSWITCH supports sending and receiving text messages between Verto/webRTC to/from backend SIP/RTP systems that conferenced together? 1) We want to use this to send commands from the web browser to the backend systems, for example: a. “hey backend system, please turn on the ASR to start listening.” 2) We also wonder if FreeSWITCH supports sending commands from the backend to the frontend (web browser), for example: a. “hey avatar, here are some visemes to tell you how you should move your lips, arms, … at the same time as playing this audio” If both Verto/SIP or webRTC/RTP are supported, is either more ideal? For instance, if the text/commands are over RTC then maybe we can use one clock for all RTP streams and/or RTP sessions and use the RTP timestamps to synchronize the commands with audio/video. Or maybe Verto/SIP is better for commands. I see there are various RFCs for text over RTP and text over RTCDataChannel in webRTC, but not sure if FreeSWITCH supports any of these. I also see FreeSWITCH mod_verto support text messages, but not sure if FreeSWITCH sends these to/from SIP (e.g. maybe via SIMPLE SIP) or via RTCDataChannel? Robert ________________________________ This e-mail and any files transmitted with it may contain privileged or confidential information. It is solely for use by the individual for whom it is intended, even if addressed incorrectly. If you received this e-mail in error, please notify the sender; do not disclose, copy, distribute, or take any action in reliance on the contents of this information; and delete it from your system. Any other use of this e-mail is prohibited. Thank you for your compliance. ________________________________ _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org<mailto:consulting@freeswitch.org> http://www.freeswitchsolutions.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.freeswitchsolutions.com&data=02%7C01%7Crmundkowsky%40ets.org%7C4ee0793cda15486ebacc08d5727e7fe3%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636540810006480750&sdata=c7hDLCM0HfQPpmdrvQJPJlJkGoBUJbnTCK89%2FwZZ9DQ%3D&reserved=0> Official FreeSWITCH Sites http://www.freeswitch.org<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.freeswitch.org&data=02%7C01%7Crmundkowsky%40ets.org%7C4ee0793cda15486ebacc08d5727e7fe3%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636540810006480750&sdata=ZfTTFQbetqjHZbYHfHFeBNurQSUH50RVcJ3HiIYPEhU%3D&reserved=0> http://confluence.freeswitch.org<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fconfluence.freeswitch.org&data=02%7C01%7Crmundkowsky%40ets.org%7C4ee0793cda15486ebacc08d5727e7fe3%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636540810006480750&sdata=LtabaZ03tH9ea6Zc0yxoOkUk8MYti2%2Bl9HbNuxHdEJY%3D&reserved=0> http://www.cluecon.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.cluecon.com&data=02%7C01%7Crmundkowsky%40ets.org%7C4ee0793cda15486ebacc08d5727e7fe3%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636540810006480750&sdata=9SaU1M8gc4k9DkUQr1SRG0PmEyqN5wWtDt3bsz%2BfxHQ%3D&reserved=0> FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org<mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.freeswitch.org%2Fmailman%2Flistinfo%2Ffreeswitch-users&data=02%7C01%7Crmundkowsky%40ets.org%7C4ee0793cda15486ebacc08d5727e7fe3%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636540810006480750&sdata=Fsqbd9Mr4Y%2BivVT8nLrRLP9TF7lgP1YWjr8UXHc6Oq0%3D&reserved=0> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.freeswitch.org%2Fmailman%2Foptions%2Ffreeswitch-users&data=02%7C01%7Crmundkowsky%40ets.org%7C4ee0793cda15486ebacc08d5727e7fe3%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636540810006480750&sdata=x16EqvZvMHfXx%2FzVCjWb2Pen3ZrXAbzwXDbkul4BErE%3D&reserved=0> http://www.freeswitch.org<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.freeswitch.org&data=02%7C01%7Crmundkowsky%40ets.org%7C4ee0793cda15486ebacc08d5727e7fe3%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636540810006480750&sdata=ZfTTFQbetqjHZbYHfHFeBNurQSUH50RVcJ3HiIYPEhU%3D&reserved=0> -- About: http://about.me/dujinfang<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fabout.me%2Fdujinfang&data=02%7C01%7Crmundkowsky%40ets.org%7C4ee0793cda15486ebacc08d5727e7fe3%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636540810006480750&sdata=%2F2T5mwvGT%2Fz%2BwDfBV2P4WnO%2B1YjF9wg1lcFLPbRYIeM%3D&reserved=0> Blog: http://www.dujinfang.com<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.dujinfang.com&data=02%7C01%7Crmundkowsky%40ets.org%7C4ee0793cda15486ebacc08d5727e7fe3%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636540810006480750&sdata=DgISgM20AhRgTGCaaSjtBkW217MrXxGK%2B0YVDAcTmpw%3D&reserved=0> Proj: http://www.freeswitch.org.cn<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.freeswitch.org.cn&data=02%7C01%7Crmundkowsky%40ets.org%7C4ee0793cda15486ebacc08d5727e7fe3%7C0ba6e9b760b34fae92f37e6ddd9e9b65%7C0%7C0%7C636540810006480750&sdata=qICqXLCccr9PLm1UgzZA1wjuUWF3BdOcBxq3h0kKH08%3D&reserved=0> ________________________________ This e-mail and any files transmitted with it may contain privileged or confidential information. It is solely for use by the individual for whom it is intended, even if addressed incorrectly. If you received this e-mail in error, please notify the sender; do not disclose, copy, distribute, or take any action in reliance on the contents of this information; and delete it from your system. Any other use of this e-mail is prohibited. Thank you for your compliance. ________________________________
participants (2)
-
Mundkowsky, Robert
-
Seven Du