[Freeswitch-users] Portaudio: Call For Input

[Andrew Thompson]

On Sun, May 30, 2010 at 03:09:33PM -0400, Mitch Capper wrote:
> Recently I have started attempting to replace an existing sip client with
> freeswitch.  In doing so I have started looking more towards portaudio.   My
> goal is to add some robustness to portaudio to make it better suited to
> being used by sip clients.    Some of the features I have currently worked
> to implement:
> 
> *Event generation for call holding and resuming
> *Ability to switch input/output devices during calls with live audio going
> on (useful for say speakerphone support)

These are both good ideas. I've definitely wanted to switch the devices
during a call and not been able to.

> *Ability to switch both the indev and outdev at once rather than separately
> (time savings)

I'm not convinced this is really worth it, but it probably won't hurt
anything.

> *Ability to keep the audio stream initialized rather than each time a call
> is made/ device is rang (time savings)
> *Only init codecs during load / config reload rather than on audio stream
> init (time savings)

I think these should be done with a lot of caution. I believe the first
one used to be the cause but it caused some weird issues..?

> *Ability to call play (for sounds) with active calls going on

Another good idea.
> 
> Most of the improvements so far have been to make portaudio a bit faster at
> doing things and less restricted than previously coded.  Most changes have
> been made through additional configuration variables so out of the box
> portaudio will not function any differently than currently.   My request for
> input is on next steps and warnings.
> 
> Right now portaudio keeps track of two streams, the call audio stream and
> the ring audio stream (and both are inited on demand only accept with my
> always active stream change).
> 
> I was thinking of taking the always active streams to a slightly higher
> level:
>   I am thinking about moving to have it keep track of an arbitrary number of
> streams,  a linked list of streams it is keeping track of with no more than
> one stream per input/output pair.    This would allow for initing a device
> prior to use to allow for near instant use of that stream.  This would
> remove the small delay that still exists (under 1 second currently I would
> estimate) when say switching on the speaker phone.  Or allow for very quick
> playing of audio on a specific device.
> Aside from the speed increases in stream switching this would allow for you
> to play audio on a device that isnt currently the primary audio device.
> While I do not plan to take it this far currently, it would actually allow
> for a much easier time of handling multiple calls on different devices at
> the same time.
> 
> The other way to go is to look towards trying to speed up initing streams
> more, there are some yields in the code that look like safety things that
> may be possible to remove without negative affects.

The problem is, a lot of them are there for a reason. Dig into the
commit logs (svn blame or git blame) and see *why* and *when* they were
added. Don't assume that they're just there because of 'cargo cult
programming', they were added for a reason.
> 
> 
> Also I looked at the last merge of portaudio from upstream into trunk (end
> of 2007) and the changes that were made back then.  The good news is there
> were not actually a lot of base code changes and most of them have actually
> been merged into upstream now so updating to the latest may not be a very
> hard thing to do.   This is in part to see if it helps with the audio
> quality issues that people of portaudio seem to report as it certainly is
> not something you want in your client.

I believe I was behind the last sync with upstream (fix for FreeBSD). If
you want to try running it with a newer portaudio, by all means try it,
but TEST IT THOROUGHLY - this means linux/mac/windows and maybe even
solaris. Making it better on windows but worse everywhere else will not
make you popular.

> 
> If anyone has any input into the current state of portaudio or purposed
> changes please let me know.   In addition if anyone has input into getting
> better call quality out of PA that would be extremely advantageous, as other
> than updating to trunk I doubt my changes will result in much of a quality
> improvement.   I am specifically also interested if anyone knows of any of
> the reasons some of the safeguards that are there are in place, some of
> which I don't see a technical reason for them to be there unless there is
> funnyness in libportaudio itself  (which certainly could be and may be
> partially resolved with updating to the changes from the last 3 years).   If
> anyone knows any stress testing etc that presented issues for PA previously
> that would be helpful.  I am working with PA in windows, do have a linux box
> I will do some limited testing on, but overall if you just can test various
> changes let me know as that will be helpful too as I believe removing some
> of the safety steps may not present as an issue right away. I could also add
> a "faster" option to portaudio config that would result in the optimizations
> rather than just removing them completely.
> 

I would definitely like to see some work on portaudio, but be aware that
the module works as well as it does because we tested it in all sorts of
crazy situations and make it work in the majority of them and that you
don't want to add regressions just to pursue some mythical
performance/quality goal.

Just do some investigating on why things are the way they are before you
get to hacking, so you know what to test for.

Good luck,

Andrew