[Freeswitch-users] Portaudio: Call For Input

Tue Jun 1 15:42:26 PDT 2010

I can try to help as much as I can with the testing, specially because
that's what I use on FSComm, so...

I would love to add to the wishlist integrating SpeexDSP's AEC, VAD, and
other features available directly out of the PA stream so that it is more
efficient. I failed to use what is in the core, maybe due to latency?

Thanks,
João Mesquita

On Tue, Jun 1, 2010 at 7:22 PM, Mitch Capper <mitch.capper at gmail.com> wrote:

> Thanks, I will definitely review the full commit history on the portaudio
> files.  The regression is my worst fear, I don't have the boxes, test cases,
> or time to really test the aggressive changeset as best as properly should
> be.  I think I will concentrate on ensuring as minimal as possible changes
> out of the box and add configuration options to enable some of the
> optimizations which should allow users looking for the boosts or additional
> functionality to enable it without the possible negative affects to existing
> users.  Syncing with upstream however is something that probably should be
> aggressively tested and there isn't a good way to just make that optional.
> The good news is mod_portaudio and libportaudio are pretty separate so I
> believe that at worst there will be a patchset to bring libportaudio up to
> trunk if people desire to do that but trunk could remain out of date.
>
> Thanks for the input!
>
> ~Mitch
> PS. the reason for switching both indev and outdev at once was for
> speakerphone (where you switch both the input and output at the same time
> sequentially during an active call), originally I was doing blocking calls
> and the delay there was a bit annoying, seeing how most of the time taken
> was on opening the stream itself (which opens both input and output at once)
> having an api command to set them both effectively halved that time.
>
>
> On Tue, Jun 1, 2010 at 2:43 PM, Andrew Thompson <andrew at hijacked.us>wrote:
>
>> On Sun, May 30, 2010 at 03:09:33PM -0400, Mitch Capper wrote:
>> > Recently I have started attempting to replace an existing sip client
>> with
>> > freeswitch.  In doing so I have started looking more towards portaudio.
>>   My
>> > goal is to add some robustness to portaudio to make it better suited to
>> > being used by sip clients.    Some of the features I have currently
>> worked
>> > to implement:
>> >
>> > *Event generation for call holding and resuming
>> > *Ability to switch input/output devices during calls with live audio
>> going
>> > on (useful for say speakerphone support)
>>
>> These are both good ideas. I've definitely wanted to switch the devices
>> during a call and not been able to.
>>
>> > *Ability to switch both the indev and outdev at once rather than
>> separately
>> > (time savings)
>>
>> I'm not convinced this is really worth it, but it probably won't hurt
>> anything.
>>
>> > *Ability to keep the audio stream initialized rather than each time a
>> call
>> > is made/ device is rang (time savings)
>> > *Only init codecs during load / config reload rather than on audio
>> stream
>> > init (time savings)
>>
>> I think these should be done with a lot of caution. I believe the first
>> one used to be the cause but it caused some weird issues..?
>>
>> > *Ability to call play (for sounds) with active calls going on
>>
>> Another good idea.
>> >
>> > Most of the improvements so far have been to make portaudio a bit faster
>> at
>> > doing things and less restricted than previously coded.  Most changes
>> have
>> > been made through additional configuration variables so out of the box
>> > portaudio will not function any differently than currently.   My request
>> for
>> > input is on next steps and warnings.
>> >
>> > Right now portaudio keeps track of two streams, the call audio stream
>> and
>> > the ring audio stream (and both are inited on demand only accept with my
>> > always active stream change).
>> >
>> > I was thinking of taking the always active streams to a slightly higher
>> > level:
>> >   I am thinking about moving to have it keep track of an arbitrary
>> number of
>> > streams,  a linked list of streams it is keeping track of with no more
>> than
>> > one stream per input/output pair.    This would allow for initing a
>> device
>> > prior to use to allow for near instant use of that stream.  This would
>> > remove the small delay that still exists (under 1 second currently I
>> would
>> > estimate) when say switching on the speaker phone.  Or allow for very
>> quick
>> > playing of audio on a specific device.
>> > Aside from the speed increases in stream switching this would allow for
>> you
>> > to play audio on a device that isnt currently the primary audio device.
>> > While I do not plan to take it this far currently, it would actually
>> allow
>> > for a much easier time of handling multiple calls on different devices
>> at
>> > the same time.
>> >
>> > The other way to go is to look towards trying to speed up initing
>> streams
>> > more, there are some yields in the code that look like safety things
>> that
>> > may be possible to remove without negative affects.
>>
>> The problem is, a lot of them are there for a reason. Dig into the
>> commit logs (svn blame or git blame) and see *why* and *when* they were
>> added. Don't assume that they're just there because of 'cargo cult
>> programming', they were added for a reason.
>> >
>> >
>> > Also I looked at the last merge of portaudio from upstream into trunk
>> (end
>> > of 2007) and the changes that were made back then.  The good news is
>> there
>> > were not actually a lot of base code changes and most of them have
>> actually
>> > been merged into upstream now so updating to the latest may not be a
>> very
>> > hard thing to do.   This is in part to see if it helps with the audio
>> > quality issues that people of portaudio seem to report as it certainly
>> is
>> > not something you want in your client.
>>
>> I believe I was behind the last sync with upstream (fix for FreeBSD). If
>> you want to try running it with a newer portaudio, by all means try it,
>> but TEST IT THOROUGHLY - this means linux/mac/windows and maybe even
>> solaris. Making it better on windows but worse everywhere else will not
>> make you popular.
>>
>> >
>> > If anyone has any input into the current state of portaudio or purposed
>> > changes please let me know.   In addition if anyone has input into
>> getting
>> > better call quality out of PA that would be extremely advantageous, as
>> other
>> > than updating to trunk I doubt my changes will result in much of a
>> quality
>> > improvement.   I am specifically also interested if anyone knows of any
>> of
>> > the reasons some of the safeguards that are there are in place, some of
>> > which I don't see a technical reason for them to be there unless there
>> is
>> > funnyness in libportaudio itself  (which certainly could be and may be
>> > partially resolved with updating to the changes from the last 3 years).
>>   If
>> > anyone knows any stress testing etc that presented issues for PA
>> previously
>> > that would be helpful.  I am working with PA in windows, do have a linux
>> box
>> > I will do some limited testing on, but overall if you just can test
>> various
>> > changes let me know as that will be helpful too as I believe removing
>> some
>> > of the safety steps may not present as an issue right away. I could also
>> add
>> > a "faster" option to portaudio config that would result in the
>> optimizations
>> > rather than just removing them completely.
>> >
>>
>> I would definitely like to see some work on portaudio, but be aware that
>> the module works as well as it does because we tested it in all sorts of
>> crazy situations and make it work in the majority of them and that you
>> don't want to add regressions just to pursue some mythical
>> performance/quality goal.
>>
>> Just do some investigating on why things are the way they are before you
>> get to hacking, so you know what to test for.
>>
>> Good luck,
>>
>> Andrew
>>
>>
>> _______________________________________________
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>
>
> _______________________________________________
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20100601/5c8c6365/attachment-0001.html 