[Freeswitch-dev] Stereo support?
hey guys, Have anyone try to implement stereo support in freeswitch to streaming the music? I tried, but only 1 channel is hardcoded inside freeswitch code everywhere, i can't make it work properly, just wonder if any1 tried before? Can we share some experiences? What I tried: File IO needs to support 2 channels shout module needs to support 2 channels L16 needs to support 2 channels it should auto convert between 1 channel codec and 2 channel codec for encoding/decoding. The problems I had are: * some random RTP sending error. (using PCMA codec, before encode, i did convert audio from stereo into mono). * music voice becomes noise. ( I tried to use MPG123_FORCE_STEREO to convert 1 channel music to 2 channel music). Cheers Xijing
bugger…. sounds complicated On 13 Dec 2012, at 09:44, Xijing Dai wrote:
hey guys,
Have anyone try to implement stereo support in freeswitch to streaming the music?
I tried, but only 1 channel is hardcoded inside freeswitch code everywhere, i can't make it work properly, just wonder if any1 tried before?
Can we share some experiences?
What I tried:
File IO needs to support 2 channels shout module needs to support 2 channels L16 needs to support 2 channels it should auto convert between 1 channel codec and 2 channel codec for encoding/decoding.
The problems I had are:
* some random RTP sending error. (using PCMA codec, before encode, i did convert audio from stereo into mono).
* music voice becomes noise. ( I tried to use MPG123_FORCE_STEREO to convert 1 channel music to 2 channel music).
Cheers Xijing _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-dev mailing list FreeSWITCH-dev@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev http://www.freeswitch.org
Richard Screene Senior Developer Drum Web Meetings +44 1273 936125 www.thisisdrum.com Drum is the collaboration solution by NetDev Ltd. Registered in England and Wales Company Number 04741258
Hey,
Have anyone try to implement stereo support in freeswitch to streaming the music?
It'd also be a great asset to conference calls if we could hear the people "in the room" each talk from their position. Add stereo microphones and it is even possible to "look at people" and direct speech more loudly their way. This is more sensational than you can imagine :- I once ate in a restaurant where the situation of being blind was simulated through absolute darkness. I was struck by how instinctively the mind registers if someone is looking at you while speaking, almost as if it has once been useful for our defense or so. I think this sensation could make an incredible impact on the concept of a conference call, and make it come *much* closer to a live conversation.
I tried, but only 1 channel is hardcoded inside freeswitch code everywhere, i can't make it work properly, just wonder if any1 tried before?
Can't answer that, just motivate it as a more generally useful idea. Ciao, -Rick
On Thu, Dec 13, 2012 at 6:01 PM, Rick van Rein <rick@openfortress.nl> wrote:
Hey,
Have anyone try to implement stereo support in freeswitch to streaming the music?
It'd also be a great asset to conference calls if we could hear the people "in the room" each talk from their position. Add stereo microphones and it is even possible to "look at people" and direct speech more loudly their way.
This is more sensational than you can imagine :- I once ate in a restaurant where the situation of being blind was simulated through absolute darkness. I was struck by how instinctively the mind registers if someone is looking at you while speaking, almost as if it has once been useful for our defense or so.
I think this sensation could make an incredible impact on the concept of a conference call, and make it come *much* closer to a live conversation.
Great idea. I did have a look at conference algorithm. It's first added up all mic voice wave and than minus listener's voice from it. By this mean, we could also increase volume of voice of whom looked at listener and added up before sent the wav to listeners. However, the difficult part is how to figure out if someone is looked at someone?
I tried, but only 1 channel is hardcoded inside freeswitch code everywhere, i can't make it work properly, just wonder if any1 tried before?
Can't answer that, just motivate it as a more generally useful idea.
Ciao, -Rick
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-dev mailing list FreeSWITCH-dev@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev http://www.freeswitch.org
We have the plumbing for it but we did not currently attempt to support it yet. The codecs and file handles all support a number of channels. Basically all the places where we detect sample rate differences and engage transcoding we would need to mux/demux stereo/mono. For the conference, my idea was to make it so you can create many channel conferences and either call in with stereo RTP on opus or some mono codec into one specific channel like you could call 2 mono speakerphones into 2 separate channels of the conference on the same table or something. On Thu, Dec 13, 2012 at 9:18 AM, Xijing Dai <dxj19831029@gmail.com> wrote:
On Thu, Dec 13, 2012 at 6:01 PM, Rick van Rein <rick@openfortress.nl>wrote:
Hey,
Have anyone try to implement stereo support in freeswitch to streaming the music?
It'd also be a great asset to conference calls if we could hear the people "in the room" each talk from their position. Add stereo microphones and it is even possible to "look at people" and direct speech more loudly their way.
This is more sensational than you can imagine :- I once ate in a restaurant where the situation of being blind was simulated through absolute darkness. I was struck by how instinctively the mind registers if someone is looking at you while speaking, almost as if it has once been useful for our defense or so.
I think this sensation could make an incredible impact on the concept of a conference call, and make it come *much* closer to a live conversation.
Great idea. I did have a look at conference algorithm. It's first added up all mic voice wave and than minus listener's voice from it. By this mean, we could also increase volume of voice of whom looked at listener and added up before sent the wav to listeners.
However, the difficult part is how to figure out if someone is looked at someone?
I tried, but only 1 channel is hardcoded inside freeswitch code everywhere, i can't make it work properly, just wonder if any1 tried before?
Can't answer that, just motivate it as a more generally useful idea.
Ciao, -Rick
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-dev mailing list FreeSWITCH-dev@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-dev mailing list FreeSWITCH-dev@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev http://www.freeswitch.org
-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900
Hello,
the difficult part is how to figure out if someone is looked at someone?
Assuming that they delivered stereo microphone sound, which is a stretch in today's setups (a chicken-egg problem) it would be trivial I think. You would basically do a calculation like tan (alpha) = (R - L) / 0.5 * (R + L) and find in alpha the angle right from straight-forward. Of course it is useful to dampen movements with a low-pass filter on momentary calculations, and/or to make these calculations over periods rather than for each individual audio sample pair. If the switch is aware what person is put in what position it could figure out which other is addressed. That might call for a switch that does this work separately for each participant. I suppose it would be practical to imagine conference partners sitting in a circle. N persons would be spread with 360 degrees / N apart. If they spoke out under an angle alpha, than their i'th neighbour would hear the sound at an angle (alpha + i*360/N degrees). And that angle would be used in the opposite way as the tangent-formula above. Sorry about the maths... I think it helps to clarify the approach though. If you'd like me to, I could try to find an elegant and *practical* way of modelling this with a simple matrix operation. Let me know if you'd like me to contribute that. It'd mean that mixing inputs and forming outputs from them is a nice operation with 2N inputs and 2N outputs with all the above isolated into a series of multiplications and additions. Of the sort that would easily be met in hardware optimisations, if present. I'm sure that hardware vendors won't mind adding a layer of luxoury models for telephony... certainly not for business applications where they money is available... but AFAIK there are no stereo switches yet. Imagine a good-quality codec (screwing ISDN, embracing OPUS) and the advantages of stereo communication, we could have a good laugh at all those people who still think 3000 Hz, audio-only and IPv4 define the VoIP arena. Cheers, -Rick
If everyone develops an interest in it and we can finde some equipment that actually will take advantage of it then I am all for it. I've just been waiting for that to happen. It does seem like that's pretty soon..... On Thu, Dec 13, 2012 at 5:07 PM, Rick van Rein <rick@openfortress.nl> wrote:
Hello,
the difficult part is how to figure out if someone is looked at someone?
Assuming that they delivered stereo microphone sound, which is a stretch in today's setups (a chicken-egg problem) it would be trivial I think. You would basically do a calculation like
tan (alpha) = (R - L) / 0.5 * (R + L)
and find in alpha the angle right from straight-forward. Of course it is useful to dampen movements with a low-pass filter on momentary calculations, and/or to make these calculations over periods rather than for each individual audio sample pair.
If the switch is aware what person is put in what position it could figure out which other is addressed. That might call for a switch that does this work separately for each participant.
I suppose it would be practical to imagine conference partners sitting in a circle. N persons would be spread with 360 degrees / N apart. If they spoke out under an angle alpha, than their i'th neighbour would hear the sound at an angle (alpha + i*360/N degrees). And that angle would be used in the opposite way as the tangent-formula above.
Sorry about the maths... I think it helps to clarify the approach though. If you'd like me to, I could try to find an elegant and *practical* way of modelling this with a simple matrix operation. Let me know if you'd like me to contribute that. It'd mean that mixing inputs and forming outputs from them is a nice operation with 2N inputs and 2N outputs with all the above isolated into a series of multiplications and additions. Of the sort that would easily be met in hardware optimisations, if present.
I'm sure that hardware vendors won't mind adding a layer of luxoury models for telephony... certainly not for business applications where they money is available... but AFAIK there are no stereo switches yet. Imagine a good-quality codec (screwing ISDN, embracing OPUS) and the advantages of stereo communication, we could have a good laugh at all those people who still think 3000 Hz, audio-only and IPv4 define the VoIP arena.
Cheers, -Rick
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-dev mailing list FreeSWITCH-dev@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev http://www.freeswitch.org
-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900
Hey,
If everyone develops an interest in it and we can finde some equipment that actually will take advantage of it then I am all for it. I've just been waiting for that to happen. It does seem like that's pretty soon.....
Just like me! I'm working on open source SIP firmware, http://0cpm.org/firmerware/ and have also included this as a remark in its documentation, to tickle hardware manufacturers, https://github.com/vanrein/0cpm-Firmerware/blob/master/doc/hardware-design.r... I very much liked your idea of using two speakerphones BTW! To fill a circle, we'd probably need three channels, but it might work out fine with two and spreading people over half a circle. Coming back to my experience of someone talking directly to you when vision is blocked -- not sure what causes that exactlye. The stereo represents just the angle from which the sound comes, but the info that makes you feel that the sound is not bouncing indirectly to you might be something else. Not sure here, probably blind people know more about it. Cheers, -Rick
Bear in mind that doing this, you'll need to do the calculation for every listening member. So instead of just muxing the audio for every speaker once and broadcasting it to all members, you'll have to do that for every single user. So much more work on the CPU, especially as your conferences grow. So you'd want to keep the traditional mod_conference working the way it is now, and have any stereo support like this in something like mod_conference_spatial (I'm guessing the code would diverge too much to make it a single config option). It'd be a cool feature, but you'd want it opt-in. -Steve On 13 December 2012 23:07, Rick van Rein <rick@openfortress.nl> wrote:
Hello,
the difficult part is how to figure out if someone is looked at someone?
Assuming that they delivered stereo microphone sound, which is a stretch in today's setups (a chicken-egg problem) it would be trivial I think. You would basically do a calculation like
tan (alpha) = (R - L) / 0.5 * (R + L)
and find in alpha the angle right from straight-forward. Of course it is useful to dampen movements with a low-pass filter on momentary calculations, and/or to make these calculations over periods rather than for each individual audio sample pair.
If the switch is aware what person is put in what position it could figure out which other is addressed. That might call for a switch that does this work separately for each participant.
I suppose it would be practical to imagine conference partners sitting in a circle. N persons would be spread with 360 degrees / N apart. If they spoke out under an angle alpha, than their i'th neighbour would hear the sound at an angle (alpha + i*360/N degrees). And that angle would be used in the opposite way as the tangent-formula above.
Sorry about the maths... I think it helps to clarify the approach though. If you'd like me to, I could try to find an elegant and *practical* way of modelling this with a simple matrix operation. Let me know if you'd like me to contribute that. It'd mean that mixing inputs and forming outputs from them is a nice operation with 2N inputs and 2N outputs with all the above isolated into a series of multiplications and additions. Of the sort that would easily be met in hardware optimisations, if present.
I'm sure that hardware vendors won't mind adding a layer of luxoury models for telephony... certainly not for business applications where they money is available... but AFAIK there are no stereo switches yet. Imagine a good-quality codec (screwing ISDN, embracing OPUS) and the advantages of stereo communication, we could have a good laugh at all those people who still think 3000 Hz, audio-only and IPv4 define the VoIP arena.
Cheers, -Rick
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-dev mailing list FreeSWITCH-dev@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev http://www.freeswitch.org
Maybe use 2 conference bridges and a SIP UA modified to make 2 calls 1000L@conference.bridge 1000R@conference.bridge Left channel @ 1000L, right channel @ 1000R, using other conference magic you could even steer people around the room by playing with their L and R gains K On 12/13/12 4:01 AM, "Rick van Rein" <rick@openfortress.nl> wrote:
Hey,
Have anyone try to implement stereo support in freeswitch to streaming the music?
It'd also be a great asset to conference calls if we could hear the people "in the room" each talk from their position. Add stereo microphones and it is even possible to "look at people" and direct speech more loudly their way.
This is more sensational than you can imagine :- I once ate in a restaurant where the situation of being blind was simulated through absolute darkness. I was struck by how instinctively the mind registers if someone is looking at you while speaking, almost as if it has once been useful for our defense or so.
I think this sensation could make an incredible impact on the concept of a conference call, and make it come *much* closer to a live conversation.
I tried, but only 1 channel is hardcoded inside freeswitch code everywhere, i can't make it work properly, just wonder if any1 tried before?
Can't answer that, just motivate it as a more generally useful idea.
Ciao, -Rick
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-dev mailing list FreeSWITCH-dev@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev http://www.freeswitch.org
-- Ken http://www.FreeSWITCH.org http://www.ClueCon.com http://www.OSTAG.org irc.freenode.net #freeswitch
participants (6)
-
Anthony Minessale
-
Ken Rice
-
Richard Screene
-
Rick van Rein
-
Steven Ayre
-
Xijing Dai