[Freeswitch-dev] mod_portaudio multiple endpoints support
Hello, I've been doing some changes to mod_portaudio sponsored by Comrex Corporation. The changes involve adding support for using multiple audio devices simultaneously, including devices with multiple channels (note that is not exactly the same as stereo). The changes are available at git:// github.com/moises-silva/mod_portaudio-endpoints.git Complete documentation on the new functionality in portaudio can be found in the sample configuration in the form of comments (conf/autoload_configs/portaudio.conf.xml). The portaudio changes are totally backwards compatible, existing configurations should keep working as expected. In addition to the new portaudio functionality there are a few other changes that were necessary to use the new functionality. All the changes described below: 1. mod_portaudio.c and pablio.c modifications to allow usage of multiple streams, channels and devices. 2. uuid_outgoing_answer API added to mod_commands.c, this API allows to answer outgoing channels. This may only be useful for portaudio endpoints since most, if not all, endpoints receive the answer notification using some sort of signaling message (200 OK for example). 3. Changes to switch_generate_sln_silence() in switch_resample.c to allow specifying -1 as a special divisor value meaning to zero-out the buffer completely. This option was required by Comrex too since the equipment they have seems to be amplifying the random low-level noise that switch_generate_sln_silence() currently generates by default and they don't like the "hissing noise" on the background while the call is answered and there is no ringback tone. I'd appreciate anyone who can review and comment. I just merged a few moments ago with latest git therefore reviewing should be easy with git diff adding the github repo as a remote. I'd like to get approval from the core FreeSWITCH team to merge this into mainstream. Needless to say I'll be available in the case any bugs are introduced in the portaudio code, virtually impossible, but just in case :-) Thanks, - Moy
On Wed, May 25, 2011 at 11:20 PM, Moises Silva <moises.silva@gmail.com> wrote:
I've been doing some changes to mod_portaudio sponsored by Comrex Corporation. The changes involve adding support for using multiple audio devices simultaneously, including devices with multiple channels (note that is not exactly the same as stereo).
All these changes sound *very* cool. I can't wait to try them out. Thanks Comrex for your willingness to get them into FS :) Gabe
Hi Moy, First of all I wanted to thank you and Comrex for the new improvements and code you have done I think a lot of people will find these features very useful! Thanks for clarifying some things on IRC. The changes look good my main concerns with the current code are how well it reacts to dynamic changes (usb devices plugged or unplugged) and the fact all streams and endpoints have to be specified in the xml config itself. I am not sure if there is a large reason to have endpoints that are defined separately from streams. An endpoint is just a name, a the indev, the outdev, and the channel for each. Streams already define an indev and outdev which makes things a bit confusing. If channel selection was moved to the stream definition it could all be defined there. The only downside to this that I see is when you have an audio device with multiple channels you are going to end up duplicating the stream section partially for each but I am not sure if that is enough to warrant to require the double configuration of an audio device before you can start using it. I would say that if the device selection was expanded so that rather than just a device ID /name to also include the channel I think it would be an improved interface. Short of that I think before it is merged into trunk the changes should be merged in a bit better with the existing commands and the existing stream code should be merged with the shared stream code as right now some functionality is being duplicated. While the way the changes were made looks like relatively few new code paths are followed if you don't use the new shared streams/endpoints, it makes for a confusing interface for someone trying to figure out how things work. My suggested list of changes would be: -Merge the existing stream code with the new stream code, this should eliminate some of the redundancies and allow the new code to be used everywhere, streams would be expanded to have the naming, channels, rates, and other features that shared streams allow -Ability to create streams through the API (with the merge of the above code its just expanding the API to take the additional params) -Remove endpoints and allow bridging directly to a stream by using the stream's name -Do not validate streams until you go to use them, right now if a device is not ready when the xml config is read you can't use it -pa answer, pa call, pa play are expanded to optionally take the streamname to act on -pa switchstream takes the call to switch the stream on -pa switch this is a tougher one but would need to figure something else out -Checks to make sure a specific device input is not already in use prior to use it for input or output by a stream, this needs to go a step further than just making sure a specific stream isn't used multiple times as multiple streams could use the same device and I have found this rarely works out Many of the changes are just to make portaudio a bit more aware and capable of multi-active call multi-device support and allowing anything to be specified dynamically. It will be important to also do it in such a way that someone can still use the functions without inadvertently enabling multiple active calls when they are only looking to use one device. I think it will probably be a decent bit of work to do these things but should allow for the great new features you have added without causing confusion for other users. By all means I am not asking you do this work any time someone contributes code back to FS its great and something much appreciated and certainly others can work to fully integrate it into freeswitch. I figured I would just give some other feedback about it as you asked for comments. I will try and test it on windows sometime shortly also for compatibility there but again thanks to you and Comrex for the new feature and I am sure it took some real work to ensure full backwards compatibility. My 2c. ~Mitch On Wed, May 25, 2011 at 10:20 PM, Moises Silva <moises.silva@gmail.com> wrote:
Hello, I've been doing some changes to mod_portaudio sponsored by Comrex Corporation. The changes involve adding support for using multiple audio devices simultaneously, including devices with multiple channels (note that is not exactly the same as stereo).
Hello Mitch, Thanks for taking your time to review this feature and share your thoughts. On Thu, May 26, 2011 at 1:37 PM, Mitch Capper <mitch.capper@gmail.com>wrote:
Hi Moy, First of all I wanted to thank you and Comrex for the new improvements and code you have done I think a lot of people will find these features very useful!
You're certainly welcomed, it is nice to work with companies like Comrex that realize is better to contribute and be part of the community.
Thanks for clarifying some things on IRC. The changes look good my main concerns with the current code are how well it reacts to dynamic changes (usb devices plugged or unplugged) and the fact all streams and endpoints have to be specified in the xml config itself.
Where else should be specified if not in the XML config? doing it dynamically would have created extra complexity and I did not feel added much value for the current user.
I am not sure if there is a large reason to have endpoints that are defined separately from streams. An endpoint is just a name, a the indev, the outdev, and the channel for each. Streams already define an indev and outdev which makes things a bit confusing. If channel selection was moved to the stream definition it could all be defined there. The only downside to this that I see is when you have an audio device with multiple channels you are going to end up duplicating the stream section partially for each but I am not sure if that is enough to warrant to require the double configuration of an audio device before you can start using it. I would say that if the device selection was expanded so that rather than just a device ID /name to also include the channel I think it would be an improved interface.
Well, you have also the sampling rate and IO sizes (codec-ms) which belong to the stream. The main reason for the separation is that they are meant to be used separate. You can define 10 different endpoints using the same stream. As long as they don't conflict in the channel they use and the direction. You can define an endpoint that only uses the input of a given stream and channel, and does not use the output, and another endpoint doing the opposite, both at the same time having a call using the same stream.
Short of that I think before it is merged into trunk the changes should be merged in a bit better with the existing commands and the existing stream code should be merged with the shared stream code as right now some functionality is being duplicated. While the way the changes were made looks like relatively few new code paths are followed if you don't use the new shared streams/endpoints, it makes for a confusing interface for someone trying to figure out how things work.
The decision here had to do mostly with backwards compatibility and the amount of things I'd most likely break if I touched the original code paths and the amount of testing required to make sure the previous functionality wasn't affected. I'd rather slowly start moving chunks from the old code compatible with the new one as the need arises.
My suggested list of changes would be: -Merge the existing stream code with the new stream code, this should eliminate some of the redundancies and allow the new code to be used everywhere, streams would be expanded to have the naming, channels, rates, and other features that shared streams allow
I don't think I'd be able to do that at this point.
-Ability to create streams through the API (with the merge of the above code its just expanding the API to take the additional params) -Remove endpoints and allow bridging directly to a stream by using the stream's name
For the reasons I explained before (the same stream being used by multiple endpoints), I can't see how this would work.
-Do not validate streams until you go to use them, right now if a device is not ready when the xml config is read you can't use it
I like failing early, is easier to spot mistakes in the configuration. I understand this might be a problem if you're planing to plug/unplug USB devices, but that is not our case yet :-)
-pa answer, pa call, pa play are expanded to optionally take the streamname to act on -pa switchstream takes the call to switch the stream on -pa switch this is a tougher one but would need to figure something else out
pa answer, pa call are replaced by uuid_outgoing_answer, originate from the command line respectively.
I think it will probably be a decent bit of work to do these things but should allow for the great new features you have added without causing confusion for other users.
The current state of things is exactly to avoid confusing existing users. They can go ahead and keep being happy with their current monodevice configuration and usage. If, and only if they want to move to multi devices they will have to learn something new. I found too complicated expanding the current API's to support the new architecture without changing the usage of the commands.
By all means I am not asking you do this work any time someone contributes code back to FS its great and something much appreciated and certainly others can work to fully integrate it into freeswitch. I figured I would just give some other feedback about it as you asked for comments.
Your input is very much appreciated. Although I can certainly tell you, given that my own needs and Comrex needs are completely fulfilled with the current feature set, I am looking mostly for bugs or small improvements. Most the things you described make the interface nicer and consistent, but will require a reasonable amount of work and a high chance of breaking existing users (either monodevice users or Comrex themselves). Once the code is merged is easier to start improving one thing at the time. - Moy
Where else should be specified if not in the XML config? doing it dynamically would have created extra complexity and I did not feel added much value for the current user. Sorry I was not saying not in the xml config, just the ability to specify it through the API is useful. Freeswitch can be interfaced through things in a lot of ways and many users configure portaudio through the command line or web interface so it makes sense just to do it both ways I figure.
Well, you have also the sampling rate and IO sizes (codec-ms) which belong to the stream. The main reason for the separation is that they are meant to be used separate. You can define 10 different endpoints using the same stream. As long as they don't conflict in the channel they use and the direction. You can define an endpoint that only uses the input of a given stream and channel, and does not use the output, and another endpoint doing the opposite, both at the same time having a call using the same stream. Sure, but you could just as easily define multiple <stream> entries that use the same device but different channels.
Your input is very much appreciated. Although I can certainly tell you, given that my own needs and Comrex needs are completely fulfilled with the current feature set, I am looking mostly for bugs or small improvements. Completely understandable my suggestions were a bit less of what you should do and more of what should be done for integration into trunk. Your code works well, obviously accomplishes what you need, and doesn't break existing stuff all which are good. The downside is anyone trying to understand portaudio and use portaudio now has two very different interfaces for doing so depending on what they want to do. In addition any code that wants to take advantage of multi-channel portaudio has to be rewritten.
~Mitch On Thu, May 26, 2011 at 6:47 PM, Moises Silva <moises.silva@gmail.com> wrote:
Hello Mitch, Thanks for taking your time to review this feature and share your thoughts.
On Thu, May 26, 2011 at 1:37 PM, Mitch Capper <mitch.capper@gmail.com> wrote:
Hi Moy, First of all I wanted to thank you and Comrex for the new improvements and code you have done I think a lot of people will find these features very useful!
You're certainly welcomed, it is nice to work with companies like Comrex that realize is better to contribute and be part of the community.
Thanks for clarifying some things on IRC. The changes look good my main concerns with the current code are how well it reacts to dynamic changes (usb devices plugged or unplugged) and the fact all streams and endpoints have to be specified in the xml config itself.
Where else should be specified if not in the XML config? doing it dynamically would have created extra complexity and I did not feel added much value for the current user.
I am not sure if there is a large reason to have endpoints that are defined separately from streams. An endpoint is just a name, a the indev, the outdev, and the channel for each. Streams already define an indev and outdev which makes things a bit confusing. If channel selection was moved to the stream definition it could all be defined there. The only downside to this that I see is when you have an audio device with multiple channels you are going to end up duplicating the stream section partially for each but I am not sure if that is enough to warrant to require the double configuration of an audio device before you can start using it. I would say that if the device selection was expanded so that rather than just a device ID /name to also include the channel I think it would be an improved interface.
Well, you have also the sampling rate and IO sizes (codec-ms) which belong to the stream. The main reason for the separation is that they are meant to be used separate. You can define 10 different endpoints using the same stream. As long as they don't conflict in the channel they use and the direction. You can define an endpoint that only uses the input of a given stream and channel, and does not use the output, and another endpoint doing the opposite, both at the same time having a call using the same stream.
Short of that I think before it is merged into trunk the changes should be merged in a bit better with the existing commands and the existing stream code should be merged with the shared stream code as right now some functionality is being duplicated. While the way the changes were made looks like relatively few new code paths are followed if you don't use the new shared streams/endpoints, it makes for a confusing interface for someone trying to figure out how things work.
The decision here had to do mostly with backwards compatibility and the amount of things I'd most likely break if I touched the original code paths and the amount of testing required to make sure the previous functionality wasn't affected. I'd rather slowly start moving chunks from the old code compatible with the new one as the need arises.
My suggested list of changes would be: -Merge the existing stream code with the new stream code, this should eliminate some of the redundancies and allow the new code to be used everywhere, streams would be expanded to have the naming, channels, rates, and other features that shared streams allow
I don't think I'd be able to do that at this point.
-Ability to create streams through the API (with the merge of the above code its just expanding the API to take the additional params) -Remove endpoints and allow bridging directly to a stream by using the stream's name
For the reasons I explained before (the same stream being used by multiple endpoints), I can't see how this would work.
-Do not validate streams until you go to use them, right now if a device is not ready when the xml config is read you can't use it
I like failing early, is easier to spot mistakes in the configuration. I understand this might be a problem if you're planing to plug/unplug USB devices, but that is not our case yet :-)
-pa answer, pa call, pa play are expanded to optionally take the streamname to act on -pa switchstream takes the call to switch the stream on -pa switch this is a tougher one but would need to figure something else out
pa answer, pa call are replaced by uuid_outgoing_answer, originate from the command line respectively.
I think it will probably be a decent bit of work to do these things but should allow for the great new features you have added without causing confusion for other users.
The current state of things is exactly to avoid confusing existing users. They can go ahead and keep being happy with their current monodevice configuration and usage. If, and only if they want to move to multi devices they will have to learn something new. I found too complicated expanding the current API's to support the new architecture without changing the usage of the commands.
By all means I am not asking you do this work any time someone contributes code back to FS its great and something much appreciated and certainly others can work to fully integrate it into freeswitch. I figured I would just give some other feedback about it as you asked for comments.
Your input is very much appreciated. Although I can certainly tell you, given that my own needs and Comrex needs are completely fulfilled with the current feature set, I am looking mostly for bugs or small improvements. Most the things you described make the interface nicer and consistent, but will require a reasonable amount of work and a high chance of breaking existing users (either monodevice users or Comrex themselves). Once the code is merged is easier to start improving one thing at the time. - Moy
_______________________________________________ FreeSWITCH-dev mailing list FreeSWITCH-dev@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev http://www.freeswitch.org
Completely understandable my suggestions were a bit less of what you should do and more of what should be done for integration into trunk. Your code works well, obviously accomplishes what you need, and doesn't break existing stuff all which are good. The downside is anyone trying to understand portaudio and use portaudio now has two very different interfaces for doing so depending on what they want to do. In addition any code that wants to take advantage of multi-channel portaudio has to be rewritten.
And may be that's the reason writing a mod_portaudio2 would make more sense where we would not be worried about breaking current code. There was a guy back in January that mentioned having something ready and about to release, but I could not find anything in the tree. - Moy
Wondering if I should log this to Jira? Looks like a recent code addition - so not sure protocol here - also never having logged a Jira before. In switch_core_sqldb.c there are some PRAGMA calls. I'm finding that in my MYSQL unixODBC environment - these PRAGMA calls are causing syntax errors. I'm not familiar with the syntax - I've seen equivalents on MS-SQL and Oracle etc - but never seen such on MYSQL. http://fisheye.freeswitch.org/changelog/freeswitch.git/?showid=11690aff4c4b8... Any ideas? switch_cache_db_execute_sql(sql_manager.event_db, "PRAGMA synchronous=OFF;", NULL); switch_cache_db_execute_sql(sql_manager.event_db, "PRAGMA count_changes=OFF;", NULL); switch_cache_db_execute_sql(sql_manager.event_db, "PRAGMA temp_store=MEMORY;", NULL); switch_cache_db_execute_sql(sql_manager.event_db, "PRAGMA journal_mode=OFF;", NULL);
If I'm correct these are just run on startup, and those statements would just harmlessly fail on mysql. Is this a problem? On Jul 17, 2011, at 3:39 PM, Robert Huddleston wrote:
Wondering if I should log this to Jira?
Looks like a recent code addition - so not sure protocol here - also never having logged a Jira before.
In switch_core_sqldb.c there are some PRAGMA calls.
I'm finding that in my MYSQL unixODBC environment - these PRAGMA calls are causing syntax errors.
I'm not familiar with the syntax - I've seen equivalents on MS-SQL and Oracle etc - but never seen such on MYSQL.
http://fisheye.freeswitch.org/changelog/freeswitch.git/?showid=11690aff4c4b8...
Any ideas?
switch_cache_db_execute_sql(sql_manager.event_db, "PRAGMA synchronous=OFF;", NULL); switch_cache_db_execute_sql(sql_manager.event_db, "PRAGMA count_changes=OFF;", NULL); switch_cache_db_execute_sql(sql_manager.event_db, "PRAGMA temp_store=MEMORY;", NULL); switch_cache_db_execute_sql(sql_manager.event_db, "PRAGMA journal_mode=OFF;", NULL);
_______________________________________________ Join us at ClueCon 2011, Aug 9-11, Chicago http://www.cluecon.com 877-7-4ACLUE
FreeSWITCH-dev mailing list FreeSWITCH-dev@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev http://www.freeswitch.org
I think that was already fixed. It always pays to update to GIT HEAD when in doubt. On Sun, Jul 17, 2011 at 2:39 PM, Robert Huddleston <rhuddleston@gmail.com>wrote:
Wondering if I should log this to Jira?
Looks like a recent code addition - so not sure protocol here - also never having logged a Jira before.
In switch_core_sqldb.c there are some PRAGMA calls.
I'm finding that in my MYSQL unixODBC environment - these PRAGMA calls are causing syntax errors.
I'm not familiar with the syntax - I've seen equivalents on MS-SQL and Oracle etc - but never seen such on MYSQL.
http://fisheye.freeswitch.org/changelog/freeswitch.git/?showid=11690aff4c4b8...
Any ideas?
switch_cache_db_execute_sql(sql_manager.event_db, "PRAGMA synchronous=OFF;", NULL); switch_cache_db_execute_sql(sql_manager.event_db, "PRAGMA count_changes=OFF;", NULL); switch_cache_db_execute_sql(sql_manager.event_db, "PRAGMA temp_store=MEMORY;", NULL); switch_cache_db_execute_sql(sql_manager.event_db, "PRAGMA journal_mode=OFF;", NULL);
_______________________________________________ Join us at ClueCon 2011, Aug 9-11, Chicago http://www.cluecon.com 877-7-4ACLUE
FreeSWITCH-dev mailing list FreeSWITCH-dev@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev http://www.freeswitch.org
-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900
participants (6)
-
Anthony Minessale
-
Gabriel Gunderson
-
Michael Jerris
-
Mitch Capper
-
Moises Silva
-
Robert Huddleston