[Freeswitch-users] B-Leg / A-Leg Race Condition

Mike Burlingame mike.burlingame at me.com
Sat Sep 8 20:36:00 MSD 2012


Ok got another crash oddly enough it's kinda the same as yesterday's 

Yesterday the last message I see in the log file is a 401 Unauthorized before the log file stopped recording data and fs_cli exited to prompt today same thing happened but this time it looks like it did a core dump.

Opening JIRA case now


On Sep 7, 2012, at 10:23 AM, Mike Burlingame <mike.burlingame at me.com> wrote:

> I figured I would update to the latest git due to in the past I have seen "make sure your running the latest git" the odd thing is no core dump was created on the last crash.
> 
> I will post the backtrace if a core dump is generated after the next crash
> 
> 
> On Sep 7, 2012, at 10:16 AM, Anthony Minessale <anthony.minessale at gmail.com> wrote:
> 
>> FYI: You need to get the backtrace before you update or the core file becomes useless.
>> It dumps core by default so you should have one but if you updated, its tool late unless you can checkout the exact previous rev.
>> 
>> 
>> 
>> On Fri, Sep 7, 2012 at 12:00 PM, Mike Burlingame <mike.burlingame at me.com> wrote:
>> after putting the server in production and  loading it up it just randomly crashed I am updating to the latest git will try to capture a core dump if one is created on exit nothing in the logs point to anything it was handling calls and just exited.
>> 
>> However on a positive note the switch sip_wait_for_aleg_ack=true seems to do as expected with all my tests.
>> 
>> 
>> 
>> On Sep 6, 2012, at 12:50 PM, Mike Burlingame <mike.burlingame at me.com> wrote:
>> 
>> > After about 20K test calls this seems to have addressed the issue - I will keep running my test's for today and put this box in a production environment tomorrow to validate it still holds up with load. I will report back after that is completed.
>> >
>> > Thanks
>> >
>> > On Sep 5, 2012, at 12:28 PM, Mike Burlingame <mike.burlingame at me.com> wrote:
>> >
>> >> Looks much much better Thank you -- Now to conduct more testing
>> >>
>> >> 0.000000 A-Leg -> FreeSwitch SIP/SDP Request: INVITE sip:+13605551212 at A-Leg:5060, with session description
>> >> 0.000639 FreeSwitch -> A-Leg SIP Status: 100 Trying
>> >> 0.051096 FreeSwitch -> B-Leg SIP/SDP Request: INVITE sip:13605551212 at B-Leg, with session description
>> >> 0.051351 B-Leg -> FreeSwitch SIP Status: 100 Giving a try
>> >> 0.286387 B-Leg -> FreeSwitch SIP Status: 180 Ringing
>> >> 0.288216 FreeSwitch -> A-Leg SIP Status: 180 Ringing
>> >> 0.475452 B-Leg -> FreeSwitch SIP Status: 180 Ringing
>> >> 5.311144 B-Leg -> FreeSwitch SIP/SDP Status: 200 OK, with session description
>> >> 5.313775 FreeSwitch -> A-Leg SIP/SDP Status: 200 OK, with session description
>> >> 5.385087 A-Leg -> FreeSwitch SIP Request: ACK sip:+13605551212 at FreeSwitch:5070;transport=udp
>> >> 5.385796 FreeSwitch -> B-Leg SIP Request: ACK sip:13605551212 at B-Leg
>> >> 12.027026 A-Leg -> FreeSwitch SIP Request: BYE sip:+13605551212 at FreeSwitch:5070;transport=udp
>> >> 12.029232 FreeSwitch -> A-Leg SIP Status: 200 OK
>> >> 12.030707 FreeSwitch -> B-Leg SIP Request: BYE sip:13605551212 at B-Leg
>> >> 12.033888 B-Leg -> FreeSwitch SIP Status: 200 OK
>> >>
>> >>
>> >>
>> >> On Sep 5, 2012, at 12:01 PM, Anthony Minessale <anthony.minessale at gmail.com> wrote:
>> >>
>> >>> ok,
>> >>>
>> >>> update one more time, if it still does not work just go right to jira
>> >>> with the latest (not before today's changes)
>> >>>
>> >>>
>> >>> On Wed, Sep 5, 2012 at 1:37 PM, Mike Burlingame <mike.burlingame at me.com> wrote:
>> >>>> The change seems to have broken the ability for the call to connect - would you like me to open a jira up with the current log files or before the change was made today?
>> >>>>
>> >>>> commit d45db898339e1b2212f5befff1af714abcec034f
>> >>>> Author: Anthony Minessale <anthm at freeswitch.org>
>> >>>> Date:   Wed Sep 5 13:11:32 2012 -0500
>> >>>>
>> >>>> 0.000000 A-LEG -> FreeSwitch SIP/SDP Request: INVITE sip:+13605551212 at A-LEG:5060, with session description
>> >>>> 0.002715 FreeSwitch -> A-LEG SIP Status: 100 Trying
>> >>>> 0.062694 FreeSwitch -> B-Leg SIP/SDP Request: INVITE sip:13605551212 at B-Leg, with session description
>> >>>> 0.062976 B-Leg -> FreeSwitch SIP Status: 100 Giving a try
>> >>>> 0.238255 B-Leg -> FreeSwitch SIP Status: 180 Ringing
>> >>>> 0.239830 FreeSwitch -> A-LEG SIP Status: 180 Ringing
>> >>>> 0.456828 B-Leg -> FreeSwitch SIP Status: 180 Ringing
>> >>>> 4.642993 B-Leg -> FreeSwitch SIP/SDP Status: 200 OK, with session description
>> >>>> 5.637738 B-Leg -> FreeSwitch SIP/SDP Status: 200 OK, with session description
>> >>>> 5.638091 B-Leg -> FreeSwitch SIP/SDP Status: 200 OK, with session description
>> >>>> 6.037804 B-Leg -> FreeSwitch SIP/SDP Status: 200 OK, with session description
>> >>>> 6.839818 B-Leg -> FreeSwitch SIP/SDP Status: 200 OK, with session description
>> >>>> 8.438750 B-Leg -> FreeSwitch SIP/SDP Status: 200 OK, with session description
>> >>>> 11.638797 B-Leg -> FreeSwitch SIP/SDP Status: 200 OK, with session description
>> >>>> 24.925970 FreeSwitch -> A-LEG SIP Status: 480 Temporarily Unavailable
>> >>>> 24.926257 A-LEG -> FreeSwitch SIP Request: ACK sip:+13605551212 at A-LEG:5060
>> >>>> 24.926321 FreeSwitch -> B-Leg SIP Request: ACK sip:13605551212 at B-Leg
>> >>>> 24.926580 FreeSwitch -> B-Leg SIP Request: BYE sip:13605551212 at B-Leg
>> >>>> 24.927029 B-Leg -> FreeSwitch SIP Status: 481 Call leg/transaction does not exist
>> >>>> 27.078016 A-LEG -> FreeSwitch SIP/SDP Request: INVITE sip:+13605551212 at A-LEG:5060, with session description
>> >>>> 27.078467 FreeSwitch -> A-LEG SIP Status: 100 Trying
>> >>>> 27.123233 FreeSwitch -> B-Leg SIP/SDP Request: INVITE sip:13605551212 at B-Leg, with session description
>> >>>> 27.123445 B-Leg -> FreeSwitch SIP Status: 100 Giving a try
>> >>>> 27.315802 B-Leg -> FreeSwitch SIP Status: 180 Ringing
>> >>>> 27.317391 FreeSwitch -> A-LEG SIP Status: 180 Ringing
>> >>>> 27.529142 B-Leg -> FreeSwitch SIP Status: 180 Ringing
>> >>>> 31.520118 B-Leg -> FreeSwitch SIP/SDP Status: 200 OK, with session description
>> >>>> 31.769831 B-Leg -> FreeSwitch SIP/SDP Status: 200 OK, with session description
>> >>>> 31.920832 B-Leg -> FreeSwitch SIP/SDP Status: 200 OK, with session description
>> >>>> 32.319816 B-Leg -> FreeSwitch SIP/SDP Status: 200 OK, with session description
>> >>>> 33.120808 B-Leg -> FreeSwitch SIP/SDP Status: 200 OK, with session description
>> >>>> 34.720813 B-Leg -> FreeSwitch SIP/SDP Status: 200 OK, with session description
>> >>>> 37.920852 B-Leg -> FreeSwitch SIP/SDP Status: 200 OK, with session description
>> >>>> 49.362647 A-LEG -> FreeSwitch SIP Request: CANCEL sip:+13605551212 at A-LEG:5060
>> >>>> 49.362952 FreeSwitch -> A-LEG SIP Status: 200 OK
>> >>>> 49.363196 FreeSwitch -> A-LEG SIP Status: 487 Request Terminated
>> >>>> 49.363307 A-LEG -> FreeSwitch SIP Request: ACK sip:+13605551212 at A-LEG:5060
>> >>>> 49.365975 FreeSwitch -> B-Leg SIP Request: ACK sip:13605551212 at B-Leg
>> >>>> 49.366171 FreeSwitch -> B-Leg SIP Request: BYE sip:13605551212 at B-Leg
>> >>>> 49.366522 B-Leg -> FreeSwitch SIP Status: 481 Call leg/transaction does not exist
>> >>>>
>> >>>>
>> >>>> On Sep 5, 2012, at 11:10 AM, Anthony Minessale <anthony.minessale at gmail.com> wrote:
>> >>>>
>> >>>>> update and try again, if it still doesn't work open a jira
>> >>>>>
>> >>>>>
>> >>>>> On Wed, Sep 5, 2012 at 12:18 PM, Mike Burlingame <mike.burlingame at me.com> wrote:
>> >>>>>> as promised here is the update testing and enabling
>> >>>>>> {sip_wait_for_aleg_ack=true} causes a log delay before the audio starts to
>> >>>>>> flow to the B-Leg endpoint with multi 200 OK's being sent to FS from the
>> >>>>>> B-Leg as well as an abnormally long delay in getting an ACK from the A-Leg
>> >>>>>>
>> >>>>>> I can open a JIRA case on this and provide the console log file / PCAP's ect
>> >>>>>> if that would help
>> >>>>>>
>> >>>>>>
>> >>>>>> Call Flow with out {sip_wait_for_aleg_ack=true}
>> >>>>>> 0.000000 A-LEG -> FreeSwitch SIP/SDP Request: INVITE
>> >>>>>> sip:+13605551212 at A-LEG:5060, with session description
>> >>>>>> 0.000652 FreeSwitch -> A-LEG SIP Status: 100 Trying
>> >>>>>> 0.042837 FreeSwitch -> B-Leg SIP/SDP Request: INVITE
>> >>>>>> sip:13605551212 at B-Leg, with session description
>> >>>>>> 0.043059 B-Leg -> FreeSwitch SIP Status: 100 Giving a try
>> >>>>>> 0.290947 B-Leg -> FreeSwitch SIP Status: 180 Ringing
>> >>>>>> 0.292890 FreeSwitch -> A-LEG SIP Status: 180 Ringing
>> >>>>>> 0.490220 B-Leg -> FreeSwitch SIP Status: 180 Ringing
>> >>>>>> 4.481038 B-Leg -> FreeSwitch SIP/SDP Status: 200 OK, with session
>> >>>>>> description
>> >>>>>> 4.482310 FreeSwitch -> B-Leg SIP Request: ACK sip:13605551212 at B-Leg
>> >>>>>> 4.483474 FreeSwitch -> A-LEG SIP/SDP Status: 200 OK, with session
>> >>>>>> description
>> >>>>>> 4.533691 A-LEG -> FreeSwitch SIP Request: ACK
>> >>>>>> sip:+13605551212 at FreeSwitch:5070;transport=udp
>> >>>>>> 19.953061 A-LEG -> FreeSwitch SIP Request: BYE
>> >>>>>> sip:+13605551212 at FreeSwitch:5070;transport=udp
>> >>>>>> 19.954592 FreeSwitch -> A-LEG SIP Status: 200 OK
>> >>>>>> 19.955454 FreeSwitch -> B-Leg SIP Request: BYE sip:13605551212 at B-Leg
>> >>>>>> 19.956014 B-Leg -> FreeSwitch SIP Status: 200 OK
>> >>>>>>
>> >>>>>> Call Flow with {sip_wait_for_aleg_ack=true} enabled
>> >>>>>> 0.000000 A-LEG -> FreeSwitch SIP/SDP Request: INVITE
>> >>>>>> sip:+13605551212 at A-LEG:5060, with session description
>> >>>>>> 0.000651 FreeSwitch -> A-LEG SIP Status: 100 Trying
>> >>>>>> 0.039455 FreeSwitch -> B-Leg SIP/SDP Request: INVITE
>> >>>>>> sip:13605551212 at B-Leg, with session description
>> >>>>>> 0.039709 B-Leg -> FreeSwitch SIP Status: 100 Giving a try
>> >>>>>> 0.244269 B-Leg -> FreeSwitch SIP Status: 180 Ringing
>> >>>>>> 0.245607 FreeSwitch -> A-LEG SIP Status: 180 Ringing
>> >>>>>> 0.363325 B-Leg -> FreeSwitch SIP Status: 180 Ringing
>> >>>>>> 4.718173 B-Leg -> FreeSwitch SIP/SDP Status: 200 OK, with session
>> >>>>>> description
>> >>>>>> 4.918915 B-Leg -> FreeSwitch SIP/SDP Status: 200 OK, with session
>> >>>>>> description
>> >>>>>> 5.117917 B-Leg -> FreeSwitch SIP/SDP Status: 200 OK, with session
>> >>>>>> description
>> >>>>>> 5.518902 B-Leg -> FreeSwitch SIP/SDP Status: 200 OK, with session
>> >>>>>> description
>> >>>>>> 6.318053 B-Leg -> FreeSwitch SIP/SDP Status: 200 OK, with session
>> >>>>>> description
>> >>>>>> 7.917921 B-Leg -> FreeSwitch SIP/SDP Status: 200 OK, with session
>> >>>>>> description
>> >>>>>> 11.118917 B-Leg -> FreeSwitch SIP/SDP Status: 200 OK, with session
>> >>>>>> description
>> >>>>>> 15.102917 FreeSwitch -> B-Leg SIP Request: ACK sip:13605551212 at B-Leg
>> >>>>>> 15.104488 FreeSwitch -> A-LEG SIP/SDP Status: 200 OK, with session
>> >>>>>> description
>> >>>>>> 15.173265 A-LEG -> FreeSwitch SIP Request: ACK
>> >>>>>> sip:+13605551212 at FreeSwitch:5070;transport=udp
>> >>>>>> 29.566504 A-LEG -> FreeSwitch SIP Request: BYE
>> >>>>>> sip:+13605551212 at FreeSwitch:5070;transport=udp
>> >>>>>> 29.568902 FreeSwitch -> A-LEG SIP Status: 200 OK
>> >>>>>> 29.570603 FreeSwitch -> B-Leg SIP Request: BYE sip:13605551212 at B-Leg
>> >>>>>> 29.571240 B-Leg -> FreeSwitch SIP Status: 200 OK
>> >>>>>>
>> >>>>>> On Aug 31, 2012, at 1:52 PM, Mike Burlingame <mike.burlingame at me.com> wrote:
>> >>>>>>
>> >>>>>> No worries I will be out this weekend for the long weekend I will work on
>> >>>>>> getting the test box upgraded and a test case setup on Tuesday I will report
>> >>>>>> back the results mid to late next week and provided everything works as I
>> >>>>>> hope it will I will be happy to pay the Wiki tax :)
>> >>>>>>
>> >>>>>>
>> >>>>>> On Aug 31, 2012, at 1:42 PM, Michael Collins <msc at freeswitch.org> wrote:
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> On Fri, Aug 31, 2012 at 12:59 PM, Mike Burlingame <mike.burlingame at me.com>
>> >>>>>> wrote:
>> >>>>>>>
>> >>>>>>> Cool I will nail that up on my test box and see if that works
>> >>>>>>>
>> >>>>>> Please report back on whether it works or not and then be prepared to pay
>> >>>>>> the wiki tax. :) I'll be glad to assist with getting this documented
>> >>>>>> although I think you're in the best position to give that documentation some
>> >>>>>> real-world context.
>> >>>>>>
>> >>>>>> -MC
>> >>>>>> _________________________________________________________________________
>> >>>>>> Professional FreeSWITCH Consulting Services:
>> >>>>>> consulting at freeswitch.org
>> >>>>>> http://www.freeswitchsolutions.com
>> >>>>>>
>> >>>>>> 
>> >>>>>> 
>> >>>>>>
>> >>>>>> Official FreeSWITCH Sites
>> >>>>>> http://www.freeswitch.org
>> >>>>>> http://wiki.freeswitch.org
>> >>>>>> http://www.cluecon.com
>> >>>>>>
>> >>>>>> FreeSWITCH-users mailing list
>> >>>>>> FreeSWITCH-users at lists.freeswitch.org
>> >>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> >>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> >>>>>> http://www.freeswitch.org
>> >>>>>>
>> >>>>>>
>> >>>>>> _________________________________________________________________________
>> >>>>>> Professional FreeSWITCH Consulting Services:
>> >>>>>> consulting at freeswitch.org
>> >>>>>> http://www.freeswitchsolutions.com
>> >>>>>>
>> >>>>>> 
>> >>>>>> 
>> >>>>>>
>> >>>>>> Official FreeSWITCH Sites
>> >>>>>> http://www.freeswitch.org
>> >>>>>> http://wiki.freeswitch.org
>> >>>>>> http://www.cluecon.com
>> >>>>>>
>> >>>>>> FreeSWITCH-users mailing list
>> >>>>>> FreeSWITCH-users at lists.freeswitch.org
>> >>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> >>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> >>>>>> http://www.freeswitch.org
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> _________________________________________________________________________
>> >>>>>> Professional FreeSWITCH Consulting Services:
>> >>>>>> consulting at freeswitch.org
>> >>>>>> http://www.freeswitchsolutions.com
>> >>>>>>
>> >>>>>> 
>> >>>>>> 
>> >>>>>>
>> >>>>>> Official FreeSWITCH Sites
>> >>>>>> http://www.freeswitch.org
>> >>>>>> http://wiki.freeswitch.org
>> >>>>>> http://www.cluecon.com
>> >>>>>>
>> >>>>>> FreeSWITCH-users mailing list
>> >>>>>> FreeSWITCH-users at lists.freeswitch.org
>> >>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> >>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> >>>>>> http://www.freeswitch.org
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> Anthony Minessale II
>> >>>>>
>> >>>>> FreeSWITCH http://www.freeswitch.org/
>> >>>>> ClueCon http://www.cluecon.com/
>> >>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>> >>>>>
>> >>>>> AIM: anthm
>> >>>>> MSN:anthony_minessale at hotmail.com
>> >>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>> >>>>> IRC: irc.freenode.net #freeswitch
>> >>>>>
>> >>>>> FreeSWITCH Developer Conference
>> >>>>> sip:888 at conference.freeswitch.org
>> >>>>> googletalk:conf+888 at conference.freeswitch.org
>> >>>>> pstn:+19193869900
>> >>>>>
>> >>>>> _________________________________________________________________________
>> >>>>> Professional FreeSWITCH Consulting Services:
>> >>>>> consulting at freeswitch.org
>> >>>>> http://www.freeswitchsolutions.com
>> >>>>>
>> >>>>> 
>> >>>>> 
>> >>>>>
>> >>>>> Official FreeSWITCH Sites
>> >>>>> http://www.freeswitch.org
>> >>>>> http://wiki.freeswitch.org
>> >>>>> http://www.cluecon.com
>> >>>>>
>> >>>>> FreeSWITCH-users mailing list
>> >>>>> FreeSWITCH-users at lists.freeswitch.org
>> >>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> >>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> >>>>> http://www.freeswitch.org
>> >>>>
>> >>>>
>> >>>> _________________________________________________________________________
>> >>>> Professional FreeSWITCH Consulting Services:
>> >>>> consulting at freeswitch.org
>> >>>> http://www.freeswitchsolutions.com
>> >>>>
>> >>>> 
>> >>>> 
>> >>>>
>> >>>> Official FreeSWITCH Sites
>> >>>> http://www.freeswitch.org
>> >>>> http://wiki.freeswitch.org
>> >>>> http://www.cluecon.com
>> >>>>
>> >>>> FreeSWITCH-users mailing list
>> >>>> FreeSWITCH-users at lists.freeswitch.org
>> >>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> >>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> >>>> http://www.freeswitch.org
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Anthony Minessale II
>> >>>
>> >>> FreeSWITCH http://www.freeswitch.org/
>> >>> ClueCon http://www.cluecon.com/
>> >>> Twitter: http://twitter.com/FreeSWITCH_wire
>> >>>
>> >>> AIM: anthm
>> >>> MSN:anthony_minessale at hotmail.com
>> >>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>> >>> IRC: irc.freenode.net #freeswitch
>> >>>
>> >>> FreeSWITCH Developer Conference
>> >>> sip:888 at conference.freeswitch.org
>> >>> googletalk:conf+888 at conference.freeswitch.org
>> >>> pstn:+19193869900
>> >>>
>> >>> _________________________________________________________________________
>> >>> Professional FreeSWITCH Consulting Services:
>> >>> consulting at freeswitch.org
>> >>> http://www.freeswitchsolutions.com
>> >>>
>> >>> 
>> >>> 
>> >>>
>> >>> Official FreeSWITCH Sites
>> >>> http://www.freeswitch.org
>> >>> http://wiki.freeswitch.org
>> >>> http://www.cluecon.com
>> >>>
>> >>> FreeSWITCH-users mailing list
>> >>> FreeSWITCH-users at lists.freeswitch.org
>> >>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> >>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> >>> http://www.freeswitch.org
>> >>
>> >>
>> >> _________________________________________________________________________
>> >> Professional FreeSWITCH Consulting Services:
>> >> consulting at freeswitch.org
>> >> http://www.freeswitchsolutions.com
>> >>
>> >> 
>> >> 
>> >>
>> >> Official FreeSWITCH Sites
>> >> http://www.freeswitch.org
>> >> http://wiki.freeswitch.org
>> >> http://www.cluecon.com
>> >>
>> >> FreeSWITCH-users mailing list
>> >> FreeSWITCH-users at lists.freeswitch.org
>> >> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> >> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> >> http://www.freeswitch.org
>> >
>> >
>> > _________________________________________________________________________
>> > Professional FreeSWITCH Consulting Services:
>> > consulting at freeswitch.org
>> > http://www.freeswitchsolutions.com
>> >
>> > 
>> > 
>> >
>> > Official FreeSWITCH Sites
>> > http://www.freeswitch.org
>> > http://wiki.freeswitch.org
>> > http://www.cluecon.com
>> >
>> > FreeSWITCH-users mailing list
>> > FreeSWITCH-users at lists.freeswitch.org
>> > http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> > UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> > http://www.freeswitch.org
>> 
>> 
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>> 
>> 
>> 
>> 
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>> 
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>> 
>> 
>> 
>> -- 
>> Anthony Minessale II
>> 
>> FreeSWITCH http://www.freeswitch.org/
>> ClueCon http://www.cluecon.com/
>> Twitter: http://twitter.com/FreeSWITCH_wire
>> 
>> AIM: anthm
>> MSN:anthony_minessale at hotmail.com
>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>> IRC: irc.freenode.net #freeswitch
>> 
>> FreeSWITCH Developer Conference
>> sip:888 at conference.freeswitch.org
>> googletalk:conf+888 at conference.freeswitch.org
>> pstn:+19193869900
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>> 
>> 
>> 
>> 
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>> 
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
> 
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
> 
> 
> 
> 
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
> 
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20120908/09a631eb/attachment-0001.html 


Join us at ClueCon 2011 Aug 9-11, 2011
More information about the FreeSWITCH-users mailing list