[Freeswitch-users] mod_portaudio send 3 rtp packet/60msinsteadof1 packet/20ms

Csaba Zelei csaba.zelei at gmail.com
Tue May 6 11:59:11 EDT 2008


Hi Anthony,

I tried your suggestions (the PCMU at 30i  60i and omitting line 158 in 
pablio.c) but none solve the problem.
I also tried the latest trunk but its the same.
I reached the best performance with the latest trunk and the patch 
Thomas wrote.(~20ms rtp packet delta, rarely 100+ms)
The only problem with it thats FS ate up 95% cpu but I am not sure if it 
has to do anything with mod_portaudio or the windows timer.

If I am correct we have 2 problem here.
The first is windows timer isnt very accurate.
The second is that mod_portaudio dont have any delay when it can return 
audio data immadiately .
Let me explain:

1. call for channel_read_frame
2. waiting for audio data (~60ms)
3. got 60ms audio data
4. get 20ms (remaining 40ms) audio data, return (and send rtp frame)
5. call for channel_read_frame
6. get 20ms, return(and send rtp frame) within ~1ms
7. call for channel_read_frame
8. get 20ms, return(and send rtp frame) within ~1ms
go back to 1 and start again

That's what I tried to solve with my patch, but because of the windows 
timer it failed.
I hope I could help, and dont misunderstand something.

Csaba

Anthony Minessale írta:
> I added a small patch to turn up the resolution of Sleep to 1ms.
> Can you see if that helps?
>
> On Tue, May 6, 2008 at 8:38 AM, Sluschny, Thomas 
> <Thomas.Sluschny at siemens.com <mailto:Thomas.Sluschny at siemens.com>> wrote:
>
>     i already tried to implement such a routine, but as i remember i
>     had problems to compile,
>     anyway this may be help you:
>      
>     // sleep with smaller tick time, wait in µs
>     void betterSleep(long wait)
>     {
>      LARGE_INTEGER lElapse;
>      BOOL succ;
>      HANDLE timerHandle = 0;
>      DWORD dwWaitResult;
>      
>      __try {
>      
>       // return on 0
>       if (wait < 1)
>        return;
>      
>       // create the timer
>       timerHandle = (HANDLE) CreateWaitableTimerA(NULL, FALSE, NULL);
>      
>       // set the timer
>       lElapse.QuadPart = 2500LL - (wait * 10000LL);
>       succ = SetWaitableTimer(timerHandle, &lElapse, 0, NULL, NULL,
>     FALSE);
>      
>       if (!succ) {
>        printf( "set timer not successful!\n" );
>       }
>      
>       dwWaitResult = WaitForSingleObject(
>        timerHandle,  
>        wait   // fall back timeout with 15ms granularity
>       );
>      
>       switch (dwWaitResult)
>       {
>        // The thread got mutex ownership.
>       case WAIT_OBJECT_0:
>        //printf( "WAIT_OBJECT_0!\n" );
>        break;
>      
>        // Cannot get mutex ownership due to time-out.
>       case WAIT_TIMEOUT:
>        printf( "WAIT_TIMEOUT!\n" );
>        break;
>      
>        // Got ownership of the abandoned mutex object.
>       case WAIT_ABANDONED:
>        printf( "WAIT_ABANDONED!\n" );
>        break;
>       }
>      }
>      __finally {
>       if (timerHandle) CloseHandle(timerHandle);
>      }
>     }
>      
>     It may be hard work but: Its cool to be cross platform - you reach
>     all people around
>     we count on you ;)
>      
>     Thomas
>
>     ------------------------------------------------------------------------
>     *Von:* freeswitch-users-bounces at lists.freeswitch.org
>     <mailto:freeswitch-users-bounces at lists.freeswitch.org>
>     [mailto:freeswitch-users-bounces at lists.freeswitch.org
>     <mailto:freeswitch-users-bounces at lists.freeswitch.org>] *Im
>     Auftrag von *Anthony Minessale
>     *Gesendet:* Dienstag, 6. Mai 2008 15:08
>
>     *An:* freeswitch-users at lists.freeswitch.org
>     <mailto:freeswitch-users at lists.freeswitch.org>
>     *Betreff:* Re: [Freeswitch-users] mod_portaudio send 3 rtp
>     packet/60msinsteadof1 packet/20ms
>
>     if switch_yield for windows is not working properly maybe if we
>     fix that the code will work as planned.
>
>     switch_timer_check tells you if the timer has ticked or not and
>     it's based on a single timer thread that also has the necessity to
>     do 1ms sleeps to broadcast the time changes to the rest of the
>     system.  so let's try using performance counters to implement
>     proper switch_yield and see if the code begins to work as
>     planned.  I'll try to come up with a patch because there are
>     several places including the RTP code where semi-accurate 1ms
>     sleeps are absolutely necessary.
>
>     This is one of the many joys of being cross platform. =D
>
>
>
>
>     On Tue, May 6, 2008 at 7:37 AM, Sluschny, Thomas
>     <Thomas.Sluschny at siemens.com <mailto:Thomas.Sluschny at siemens.com>>
>     wrote:
>
>         Anthony seams to be right.
>         I tested around a little bit and i see:
>         - the main problem is the soundcard (-driver), it gives 480
>         frames all 60 ms
>         - if i chose 160 (256) frames ringbuffer size i get only 160
>         frames all 60 ms, and it sounds really croppy (of course,
>         there are not enough samples ...)
>         - i get really good results if i store the last timestamp when
>         ReadAudioStream() returns, and if current timestamp is less
>         than 20ms from last i wait 20 ms,
>         so the packet difference went from 60-0-0-60-0-0 ms to
>         20-20-20-20-20-20
>         - i use HighPerformanceCounter to measure the time in WinXP
>          
>         I thought we could use switch_core_timer_check() to do that
>         timestamp work, but it doesn't work as expected,
>         seams we have to synchronize to the 60ms clock from soundcard,
>         at least at the beginning, which means the delay increase.
>          
>         Thomas
>          
>         PS: i attach a patch for better understanding 
>
>         ------------------------------------------------------------------------
>         *Von:* freeswitch-users-bounces at lists.freeswitch.org
>         <mailto:freeswitch-users-bounces at lists.freeswitch.org>
>         [mailto:freeswitch-users-bounces at lists.freeswitch.org
>         <mailto:freeswitch-users-bounces at lists.freeswitch.org>] *Im
>         Auftrag von *Anthony Minessale
>         *Gesendet:* Dienstag, 6. Mai 2008 00:38
>
>         *An:* freeswitch-users at lists.freeswitch.org
>         <mailto:freeswitch-users at lists.freeswitch.org>
>         *Betreff:* Re: [Freeswitch-users] mod_portaudio send 3 rtp
>         packet/60ms insteadof1 packet/20ms
>
>         This is not normal sleep it's microsecond sleep.  Its done
>         because we are doing nonblocking read on the ringbuffer that's
>         tied to the hardware.  Since this is voip, we must drop audio
>         frames when they are late and in order to do that we must have
>         a high resolution loop.  This is only a problem when the audio
>         device is not sending audio at the interval we asked it to. 
>         Some cheap hardware cannot reliably deliver audio at 20ms
>         intervals which is why i suggested higher value intervals in
>         the config.  The request to remove the sleep is to confirm the
>         proposition that sleep 1ms was really taking 15ms.
>
>         I appreciate the suggestion and I understand you are not
>         trying to be a smart ass.
>
>
>
>         On Mon, May 5, 2008 at 4:35 PM, Michael Jerris
>         <mike at jerris.com <mailto:mike at jerris.com>> wrote:
>
>             The place this is trickiest is when you are in a loop
>             where you want
>             to wait for audio, AND do something every x ms or so.  You
>             can't do a
>             blocking read, and a read with timeout tends to be quite
>             expensive.
>             There are some ways around this, but sometimes its just
>             the most
>             efficient, even if not preferred method.
>
>             Mike
>
>
>             On May 5, 2008, at 5:26 PM, ?ukasz Zwierko wrote:
>
>             > Hi,
>             >
>             > Just a thought here: using calls like delay(), sleep()
>             etc. for very
>             > short amount of time (like in this case a couple of
>             miliseconds)
>             > should in my opinion be really discouraged. It is often
>             a symptom of
>             > bad programming even in an embedded enviroment, not
>             mentioning
>             > platforms like Windows or Linux where you can't really
>             tell what
>             > priority does your task have, and will it not be starved
>             for a long
>             > time by other tasks. I should be avoided whenever possible.
>             > Don't want to sound like a smart ass here, but isn't
>             there any other
>             > way? From what I understood you wait until some  amount
>             of voice
>             > samples is collected? If that's the case than perhaps
>             you can measure
>             > an amount of data collected not the time... If these are
>             PCM samples
>             > than the correlation is straightforward.
>             > Again, sorry if I'm being a smart ass here but I've seen
>             some really
>             > bad code with sleep() calls and such like, and I can
>             tell you that it
>             > only worked fine in specific conditions, and had a
>             tendency to work
>             > very poorly when for example CPU was under heavy load.
>             >
>             > Luaksz
>             >
>             > 2008/5/5 Anthony Minessale <anthony.minessale at gmail.com
>             <mailto:anthony.minessale at gmail.com>>:
>             >> did you try setting the ptime on the rtp to 30 or 60ms
>             >>
>             >> when you choose a codec in your sip settings on FS in
>             vars.xml,
>             >> instead of
>             >> PCMU try PCMU at 30i or PCMU at 60i
>             >> it may be that the other side is doing 30 or 60 ms and
>             not telling
>             >> us.
>             >>
>             >> also in pablio.c in the portaudio_mod directory in
>             ReadAudioStream
>             >> func,
>             >> there is a sleep 1 ms too
>             >> if the windows is really sleeping a lot longer than
>             that, try
>             >> omitting line
>             >> 158.
>             >> This probably will consume the whole cpu but if it
>             fixes your
>             >> problem it
>             >> will support the theory that the sleep on windows in
>             inaccurate.
>             >>
>             >>
>             >>
>             >>
>             >>
>             >> On Mon, May 5, 2008 at 11:38 AM, Csaba Zelei
>             >> <csaba.zelei at gmail.com <mailto:csaba.zelei at gmail.com>>
>             wrote:
>             >>>
>             >>>
>             >>>
>             >>>
>             >>> With a little hack I can make mod_portaudio to send
>             rtp packets with
>             >> ~24ms, ~16ms delay alternately on linux. This result in
>             a constant
>             >> 4ms
>             >> jitter but its better than the original. (diff attached)
>             >>> However on windows the delta between rtp packets is
>             15-32 ms
>             >>> randomly,
>             >> with occasionally high 70-100ms delta.
>             >>> I also tried to tweak the windows timer without success.
>             >>> Does anybody has any idea how to make windows xp more
>             accurate?
>             >>>
>             >>> Sluschny, Thomas wrote:
>             >>>
>             >>>
>             >>>
>             >>>
>             >>> as you can see here:
>             >>>
>             >>>
>             >>> http://jira.freeswitch.org/browse/MODENDP-40
>             >>>
>             >>> i have this problem all the time (the error mentioned
>             in this
>             >>> issue was
>             >> only related with this).
>             >>>
>             >>> It has to do with windows handle sleep() method, you
>             has say
>             >>> sleep(1) for
>             >> 1ms but on my
>             >>>
>             >>> machine it waits 15ms (it depends on your hardware,
>             other PCs behave
>             >> different!). So i tested around with high performance
>             counters.
>             >>>
>             >>> For now i ignore that problem an set jitterbuffers on
>             other device
>             >>> big
>             >> enough.
>             >>>
>             >>> Thomas
>             >>>
>             >>> ________________________________
>             >> Von: freeswitch-users-bounces at lists.freeswitch.org
>             <mailto:freeswitch-users-bounces at lists.freeswitch.org>
>             >> [mailto:freeswitch-users-bounces at lists.freeswitch.org
>             <mailto:freeswitch-users-bounces at lists.freeswitch.org>] Im
>             Auftrag
>             >> von Zelei
>             >> Csaba
>             >>> Gesendet: Donnerstag, 24. April 2008 19:05
>             >>> An: freeswitch-users at lists.freeswitch.org
>             <mailto:freeswitch-users at lists.freeswitch.org>
>             >>> Betreff: [Freeswitch-users] mod_portaudio send 3 rtp
>             packet/60ms
>             >>> instead
>             >> of1 packet/20ms
>             >>>
>             >>> Dear all,
>             >>>
>             >>> I tried to use FS in client mode, starting calls with
>             >>> mod_portaudio to our
>             >> providers gateway ( a Cirpack softswitch )
>             >>> I experienced that there is 2-3 sec delay in the call,
>             its choppy
>             >>> and
>             >> robot like.
>             >>> I tested it with a softphone, and an ip phone and
>             everything was
>             >>> fine. I
>             >> traced back the problem to mod_portaudio sending 3 rtp
>             packet in 60ms
>             >> instead of 1 packet/20ms.
>             >>>
>             >>> Here is an rtp statistic from a call: (see
>             >> http://pastebin.freeswitch.org/4307 for the complete
>             list and sip
>             >> trace)
>             >>>
>             >>> Packet    Sequence    Delta (ms)
>             >>> 42  26138    0.00
>             >>> 43    26139    0.02
>             >>> 46    26140    45.69
>             >>> 47    26141    0.02
>             >>> 48    26142    2.96
>             >>> 52    26143    56.31
>             >>> 53    26144    5.75
>             >>> 54    26145    0.02
>             >>> 58    26146    51.99
>             >>> 59    26147    0.03
>             >>> 60    26148    2.96
>             >>> 63    26149    42.95
>             >>> 65    26150    17.06
>             >>> 66    26151    0.02
>             >>> 67    26152    2.90
>             >>> 71    26153    56.99
>             >>> 72    26154    0.03
>             >>> 73    26155    0.02
>             >>>
>             >>> Did anyone else experience similar problems?
>             >>> Is this the desired behaviour, because portaudio get
>             data in 60ms
>             >>> interval
>             >> or can I set it to 20ms somehow?
>             >>>
>             >>> Thanks,
>             >>>
>             >>> Csaba Zelei
>             >>>
>             >>>
>             >>> ________________________________
>             >>
>             >>> _______________________________________________
>             >>> Freeswitch-users mailing list
>             >>> Freeswitch-users at lists.freeswitch.org
>             <mailto:Freeswitch-users at lists.freeswitch.org>
>             >>>
>             http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>             >>>
>             UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>             >>> http://www.freeswitch.org
>             >>>
>             >>>
>             >>>
>             >>> Index: mod_portaudio.c
>             >>>
>             ===================================================================
>             >>> --- mod_portaudio.c     (revision 8260)
>             >>> +++ mod_portaudio.c     (working copy)
>             >>> @@ -121,6 +121,7 @@
>             >>>       int ring_interval;
>             >>>       GFLAGS flags;
>             >>>       switch_timer_t timer;
>             >>> +       switch_timer_t sync_timer;
>             >>> } globals;
>             >>>
>             >>>
>             >>> @@ -282,7 +283,15 @@
>             >>>       }
>             >>>
>             >>>       switch_set_flag_locked(tech_pvt, TFLAG_IO);
>             >>> +
>             >>> +       /* Start Synchronization Timer */
>             >>> +       //Is it ok to always use 20ms? What about the
>             160 sample????
>             >>> +       if (
>             >> switch_core_timer_init(&globals.sync_timer,"soft",
>             >> 20,160,switch_core_session_get_pool(session))
>             >> != SWITCH_STATUS_SUCCESS)
>             >>> +       {
>             >>> +               switch_log_printf(SWITCH_CHANNEL_LOG,
>             >>> SWITCH_LOG_DEBUG,
>             >> "Sync Timer failed!!\n");
>             >>> +       }
>             >>>
>             >>> +
>             >>>       /* Move Channel's State Machine to RING */
>             >>>       switch_channel_set_state(channel, CS_RING);
>             >>>
>             >>> @@ -412,6 +421,8 @@
>             >>>       }
>             >>>
>             >>>       switch_log_printf(SWITCH_CHANNEL_LOG,
>             SWITCH_LOG_DEBUG, "%s
>             >>> CHANNEL
>             >> HANGUP\n",
>             >>
>             switch_channel_get_name(switch_core_session_get_channel(session)));
>             >>> +       /* Destroy timer */
>             >>> +       switch_core_timer_destroy(&globals.sync_timer);
>             >>>
>             >>>       return SWITCH_STATUS_SUCCESS;
>             >>> }
>             >>> @@ -542,12 +553,17 @@
>             >>>       switch_mutex_lock(globals.device_lock);
>             >>>
>             >>> get_samples:
>             >>> -
>             >>> +
>             >>>       if ((samples = ReadAudioStream(globals.audio_stream,
>             >> globals.read_frame.data,
>             >>>
>             >> globals.read_codec.implementation->samples_per_frame,
>             >>>
>             >> &globals.timer)) == 0) {
>             >>> +
>             >>> +       //switch_log_printf(SWITCH_CHANNEL_LOG,
>             SWITCH_LOG_DEBUG,
>             >>> "No data
>             >> reset timer\n");
>             >>> +       switch_core_timer_sync(&globals.sync_timer);
>             >>>        switch_yield(1000);
>             >>> +
>             >>>        goto get_samples;
>             >>> +
>             >>>       } else {
>             >>>               globals.read_frame.datalen = samples * 2;
>             >>>               globals.read_frame.samples = samples;
>             >>> @@ -562,7 +578,9 @@
>             >>>               status = SWITCH_STATUS_SUCCESS;
>             >>>       }
>             >>>       switch_mutex_unlock(globals.device_lock);
>             >>> -
>             >>> +
>             >>> +       switch_core_timer_next(&globals.sync_timer);
>             >>> +
>             >>>       return status;
>             >>>
>             >>> }
>             >>>
>             >>>
>             >>> _______________________________________________
>             >>> Freeswitch-users mailing list
>             >>> Freeswitch-users at lists.freeswitch.org
>             <mailto:Freeswitch-users at lists.freeswitch.org>
>             >>>
>             http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>             >>>
>             UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>             >>> http://www.freeswitch.org
>             >>>
>             >>>
>             >>
>             >>
>             >>
>             >> --
>             >> Anthony Minessale II
>             >>
>             >> FreeSWITCH http://www.freeswitch.org/
>             >> ClueCon http://www.cluecon.com/
>             >>
>             >> AIM: anthm
>             >> MSN:anthony_minessale at hotmail.com
>             <mailto:MSN%3Aanthony_minessale at hotmail.com>
>             >> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>             <mailto:PAYPAL%3Aanthony.minessale at gmail.com>
>             >> IRC: irc.freenode.net <http://irc.freenode.net> #freeswitch
>             >>
>             >> FreeSWITCH Developer Conference
>             >> sip:888 at conference.freeswitch.org
>             <mailto:sip%3A888 at conference.freeswitch.org>
>             >> iax:guest at conference.freeswitch.org/888
>             <http://iax:guest@conference.freeswitch.org/888>
>             >> googletalk:conf+888 at conference.freeswitch.org
>             <mailto:googletalk%3Aconf%2B888 at conference.freeswitch.org>
>             >> pstn:213-799-1400
>             >> _______________________________________________
>             >> Freeswitch-users mailing list
>             >> Freeswitch-users at lists.freeswitch.org
>             <mailto:Freeswitch-users at lists.freeswitch.org>
>             >>
>             http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>             >>
>             UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>             >> http://www.freeswitch.org
>             >>
>             >>
>             >
>             > _______________________________________________
>             > Freeswitch-users mailing list
>             > Freeswitch-users at lists.freeswitch.org
>             <mailto:Freeswitch-users at lists.freeswitch.org>
>             >
>             http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>             >
>             UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>             > http://www.freeswitch.org
>
>
>             _______________________________________________
>             Freeswitch-users mailing list
>             Freeswitch-users at lists.freeswitch.org
>             <mailto:Freeswitch-users at lists.freeswitch.org>
>             http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>             UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>             http://www.freeswitch.org
>
>
>
>
>         -- 
>         Anthony Minessale II
>
>         FreeSWITCH http://www.freeswitch.org/
>         ClueCon http://www.cluecon.com/
>
>         AIM: anthm
>         MSN:anthony_minessale at hotmail.com
>         <mailto:MSN%3Aanthony_minessale at hotmail.com>
>         GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>         <mailto:PAYPAL%3Aanthony.minessale at gmail.com>
>         IRC: irc.freenode.net <http://irc.freenode.net> #freeswitch
>
>         FreeSWITCH Developer Conference
>         sip:888 at conference.freeswitch.org
>         <mailto:sip%3A888 at conference.freeswitch.org>
>         iax:guest at conference.freeswitch.org/888
>         <http://iax:guest@conference.freeswitch.org/888>
>         googletalk:conf+888 at conference.freeswitch.org
>         <mailto:googletalk%3Aconf%2B888 at conference.freeswitch.org>
>         pstn:213-799-1400
>
>         _______________________________________________
>         Freeswitch-users mailing list
>         Freeswitch-users at lists.freeswitch.org
>         <mailto:Freeswitch-users at lists.freeswitch.org>
>         http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>         UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>         http://www.freeswitch.org
>
>
>
>
>     -- 
>     Anthony Minessale II
>
>     FreeSWITCH http://www.freeswitch.org/
>     ClueCon http://www.cluecon.com/
>
>     AIM: anthm
>     MSN:anthony_minessale at hotmail.com
>     <mailto:MSN%3Aanthony_minessale at hotmail.com>
>     GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>     <mailto:PAYPAL%3Aanthony.minessale at gmail.com>
>     IRC: irc.freenode.net <http://irc.freenode.net> #freeswitch
>
>     FreeSWITCH Developer Conference
>     sip:888 at conference.freeswitch.org
>     <mailto:sip%3A888 at conference.freeswitch.org>
>     iax:guest at conference.freeswitch.org/888
>     <http://iax:guest@conference.freeswitch.org/888>
>     googletalk:conf+888 at conference.freeswitch.org
>     <mailto:googletalk%3Aconf%2B888 at conference.freeswitch.org>
>     pstn:213-799-1400
>
>     _______________________________________________
>     Freeswitch-users mailing list
>     Freeswitch-users at lists.freeswitch.org
>     <mailto:Freeswitch-users at lists.freeswitch.org>
>     http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>     UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>     http://www.freeswitch.org
>
>
>
>
> -- 
> Anthony Minessale II
>
> FreeSWITCH http://www.freeswitch.org/
> ClueCon http://www.cluecon.com/
>
> AIM: anthm
> MSN:anthony_minessale at hotmail.com 
> <mailto:MSN%3Aanthony_minessale at hotmail.com>
> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com 
> <mailto:PAYPAL%3Aanthony.minessale at gmail.com>
> IRC: irc.freenode.net <http://irc.freenode.net> #freeswitch
>
> FreeSWITCH Developer Conference
> sip:888 at conference.freeswitch.org 
> <mailto:sip%3A888 at conference.freeswitch.org>
> iax:guest at conference.freeswitch.org/888 
> <http://iax:guest@conference.freeswitch.org/888>
> googletalk:conf+888 at conference.freeswitch.org 
> <mailto:googletalk%3Aconf%2B888 at conference.freeswitch.org>
> pstn:213-799-1400
> ------------------------------------------------------------------------
>
> _______________________________________________
> Freeswitch-users mailing list
> Freeswitch-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>   

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20080506/064f16d5/attachment-0001.html 


More information about the Freeswitch-users mailing list