[Freeswitch-users] registration fails after several hours - FS problem?
Mario G
mario_fs at mgtech.com
Sat Oct 30 12:55:09 PDT 2010
No I was wrong, the problem was Terminal not FS, it keeps it's log in memory. See my previous post with trace info. No memory problem now.
On Oct 30, 2010, at 12:34 PM, Anthony Minessale wrote:
> ok so there must be a problem in MAC, have you compared the results on
> win32 or linux?
> some other people are reporting mem issues on mac lets find it.
>
> http://valgrind.org/downloads/valgrind-3.6.0.tar.bz2
>
> build and install valgrind
>
> run fs in vg mode
> valgrind --tool=memcheck --log-file=vg.log --leak-check=full
> --leak-resolution=high --show-reachable=yes /path/to/lt-freeswitch -vg
>
> do something that eats mem
>
> exit and get me the vg.log
>
>
>
> On Sat, Oct 30, 2010 at 11:37 AM, Mario G <mario_fs at mgtech.com> wrote:
>> Setting sofia loglevel to 9 made Terminal app memory go from 20meg to eventually using all memory and locking the machine up. I spent all night trying to get a good trace. I need to get one when the failure occurs early before the lockup. Will be working all w/e on this. Thanks for you help.
>>
>> I got this with only 1 gateway defined but did not catch it early. Notice how there is no SIP trace activity:
>> 2010-10-30 09:30:15.046900 [WARNING] sofia_reg.c:387 mvvyl Failed Registration, setting retry to 15 seconds.
>> nta: timer K fired, terminate NOTIFY (3892042)
>> outgoing_reclaim_all(0x0, 0x0, 0x102309cb0)
>> nta_outgoing_timer: 0/0 resent, 0/0 tout, 1/1 term, 1/1 free
>> nta: timer not set
>> 2010-10-30 09:30:31.926382 [NOTICE] sofia_reg.c:342 Registering mvvyl
>> nua: nua_register: entering
>> nua(0x1003c3750): sent signal r_register
>> nua(0x1003c3750): recv signal r_register
>> 2010-10-30 09:30:47.685212 [WARNING] sofia_reg.c:387 mvvyl Failed Registration, setting retry to 15 seconds.
>> 2010-10-30 09:31:03.478941 [NOTICE] sofia_reg.c:342 Registering mvvyl
>> nua: nua_register: entering
>> nua(0x1003c3750): sent signal r_register
>> nua(0x1003c3750): recv signal r_register
>> 2010-10-30 09:31:19.219346 [WARNING] sofia_reg.c:387 mvvyl Failed Registration, setting retry to 15 seconds.
>> 2010-10-30 09:31:36.208448 [NOTICE] sofia_reg.c:342 Registering mvvyl
>> nua: nua_register: entering
>> nua(0x1003c3750): sent signal r_register
>> nua(0x1003c3750): recv signal r_register
>>
>> Mario
>>
>>>
>>> On Oct 29, 2010, at 5:43 PM, Anthony Minessale wrote:
>>>
>>>> also raise loglevel to debug
>>>> console loglevel debug
>>>>
>>>>
>>>> On Fri, Oct 29, 2010 at 7:29 PM, Mario G <mario_fs at mgtech.com> wrote:
>>>>> Will so, BTW, I mentioned below one is dns and another uses IP, I tested that theory using an IP does not help. I also updated to todays git version and no help there. Will post when the trace is done.
>>>>>
>>>>> On Oct 29, 2010, at 5:17 PM, Anthony Minessale wrote:
>>>>>
>>>>>> can you repeat that trace with sofia debug on
>>>>>> sofia loglevel all 9
>>>>>>
>>>>>> Are you doing DNS by any chance in the gateway "proxy" param?
>>>>>> you could try filling in the register-proxy param in your gateway to
>>>>>> sip:<ip> <-- not dns but ip that dns resolves to
>>>>>> <param name="register-proxy" value="sip:1.2.3.4"/>
>>>>>>
>>>>>> I'm just guessing but Its possible some bad dns query could be throwing FS off.
>>>>>> so this test would force all the packets to the exact IP of your host
>>>>>> instead of looking it up.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Oct 29, 2010 at 6:30 PM, Mario G <mario_fs at mgtech.com> wrote:
>>>>>>> Oh Boy: Not only is everything set to the static route but I turned
>>>>>>> off/disconnected the dynamic DSL line so I only had 1 static line to the
>>>>>>> router. The second wan is set in the router off. Even turned off router uPnP
>>>>>>> even though I am using -nonat. Guess what.... I still have the problem. Look
>>>>>>> like when FS says is going to retry it actually does not.
>>>>>>> Here is a short one: http://pastebin.freeswitch.org/14359 - I caught one
>>>>>>> right after I started FS, failure occurred in minutes (lucky). Look at the
>>>>>>> bottom of the trace, you see SIP trace activity and then when it fails no
>>>>>>> SIP trace activity. Could this possibly be a FS bug? (I am a mainframe
>>>>>>> assembler systems programmer and I might think so if there were error retry
>>>>>>> messages but nothing showing in one of my traces...)
>>>>>>> Notes:
>>>>>>> idone is gateway 1
>>>>>>> idtwo is gateway 2 I had to trace both because it was impossible to figure
>>>>>>> out which one would fail first. Ran several times but kept missing the right
>>>>>>> one.
>>>>>>> I use a url for one gateway and ip for another but it makes no difference
>>>>>>> since both eventually fail.
>>>>>>> 10. is local lan
>>>>>>> 210. is external ip
>>>>>>> 216. is itsp
>>>>>>>
>>>>>>> Here is a longer one from earlier http://pastebin.freeswitch.org/14357
>>>>>>> Notes:
>>>>>>> A call was received and hung up for idtwo - beginning of trace
>>>>>>> 11 minutes later idtwo failed - see last line of trace
>>>>>>>
>>>>>>> Thank you very much!
>>>>>>> Mario
>>>>>>>
>>>>>>> On Oct 29, 2010, at 12:11 PM, Anthony Minessale wrote:
>>>>>>>
>>>>>>> stun-enabled must be true in your profile XML to see what you pasted.
>>>>>>>
>>>>>>> Get me a sip trace of this from when it works until when it fails
>>>>>>>
>>>>>>> only enable the sip trace on the profile with the gateway to reduce traffic
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Oct 29, 2010 at 1:56 PM, Mario G <mario_fs at mgtech.com> wrote:
>>>>>>>
>>>>>>> Oh my... looks it was not NAT after all? Please help! I changed to all
>>>>>>> profiles to static per instructions below and still have the problem:
>>>>>>>
>>>>>>> 2010-10-29 11:15:18.536446 [NOTICE] sofia_reg.c:342 Registering uuid1
>>>>>>>
>>>>>>> 2010-10-29 11:15:34.313150 [WARNING] sofia_reg.c:387 uuid1 Failed
>>>>>>> Registration, setting retry to 15 seconds.
>>>>>>>
>>>>>>> sofia global siptrace on did not show any activity for this gateway in or
>>>>>>> out, others were fine but eventually fail. I setup static:
>>>>>>>
>>>>>>> 1. set the params ext-sip-ip and ext-rtp-ip to my external static IP
>>>>>>>
>>>>>>> 2. map the sip ports (5060-5080) and all of the rtp ports (16384-32767) to
>>>>>>> FS lan addr.
>>>>>>>
>>>>>>> 3. set sip-ip and rtp-ip to the lan addr of FS
>>>>>>>
>>>>>>> 4. start FS with -nonat
>>>>>>>
>>>>>>> I don't know what to try next. BTW, the sofia status for the profiles shows
>>>>>>> stun enabled but I did not set it up anywhere:
>>>>>>>
>>>>>>> Name uuid1
>>>>>>>
>>>>>>> Domain Name N/A
>>>>>>>
>>>>>>> Auto-NAT false
>>>>>>>
>>>>>>> DBName sofia_reg_mvvyl
>>>>>>>
>>>>>>> Pres Hosts
>>>>>>>
>>>>>>> Dialplan XML
>>>>>>>
>>>>>>> Context public
>>>>>>>
>>>>>>> Challenge Realm auto_to
>>>>>>>
>>>>>>> RTP-IP 10.x.x.20
>>>>>>>
>>>>>>> Ext-RTP-IP 210.x.x.100
>>>>>>>
>>>>>>> SIP-IP 10.x.x.20
>>>>>>>
>>>>>>> Ext-SIP-IP 210.x.x.100
>>>>>>>
>>>>>>> URL sip:mod_sofia at 210.x.x.100:5068
>>>>>>>
>>>>>>> BIND-URL sip:mod_sofia at 210.x.x.100:5068;maddr=10.x.x.20
>>>>>>>
>>>>>>> HOLD-MUSIC local_stream://moh
>>>>>>>
>>>>>>> OUTBOUND-PROXY N/A
>>>>>>>
>>>>>>> CODECS IN G7221 at 32000h,G7221 at 16000h,G722,PCMU,PCMA,GSM
>>>>>>>
>>>>>>> CODECS OUT PCMU,PCMA,GSM
>>>>>>>
>>>>>>> TEL-EVENT 101
>>>>>>>
>>>>>>> DTMF-MODE rfc2833
>>>>>>>
>>>>>>> CNG 13
>>>>>>>
>>>>>>> SESSION-TO 0
>>>>>>>
>>>>>>> MAX-DIALOG 0
>>>>>>>
>>>>>>> NOMEDIA false
>>>>>>>
>>>>>>> LATE-NEG false
>>>>>>>
>>>>>>> PROXY-MEDIA false
>>>>>>>
>>>>>>> AGGRESSIVENAT false
>>>>>>>
>>>>>>> STUN-ENABLED true
>>>>>>>
>>>>>>> STUN-AUTO-DISABLE false
>>>>>>>
>>>>>>> CALLS-IN 0
>>>>>>>
>>>>>>> FAILED-CALLS-IN 0
>>>>>>>
>>>>>>> CALLS-OUT 0
>>>>>>>
>>>>>>> FAILED-CALLS-OUT 0
>>>>>>>
>>>>>>>
>>>>>>> On Oct 27, 2010, at 10:04 AM, Anthony Minessale wrote:
>>>>>>>
>>>>>>> if you map it or not, a scanner would penetrate it.
>>>>>>>
>>>>>>> There are lot of sip scanners out there now, you just need to beware of
>>>>>>> them.
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Oct 27, 2010 at 11:50 AM, Mario G <mario_fs at mgtech.com> wrote:
>>>>>>>
>>>>>>> Thanks so much! I am sure many others will find this info invaluable. I will
>>>>>>> try the static route again but have one question: When I started with FS I
>>>>>>> found a "sip scanner" in FS and someone on this group said not to use port
>>>>>>> mapping since it was a security risk. Is that true?
>>>>>>>
>>>>>>> On Oct 27, 2010, at 9:10 AM, Anthony Minessale wrote:
>>>>>>>
>>>>>>> you are completely guessing at things.
>>>>>>>
>>>>>>> I want you to understand that the only reason you are having problems
>>>>>>>
>>>>>>> with this is because you don't understand how it works enough to know
>>>>>>>
>>>>>>> what you are doing 100%
>>>>>>>
>>>>>>> Its a given that the pnp stuff is only for your dynamic IP.
>>>>>>>
>>>>>>> aggressive-nat-detection and sip-force-expires are all related to
>>>>>>>
>>>>>>> inbound calls when the things who are registering to you may be behind
>>>>>>>
>>>>>>> nat.
>>>>>>>
>>>>>>> You need to learn the difference between which nat tools are
>>>>>>>
>>>>>>> *) designed for your FS to run behind nat
>>>>>>>
>>>>>>> *) designed for FS to run public and accept connections from devices behind
>>>>>>> nat.
>>>>>>>
>>>>>>> If you have a static IP, you don't need the pnp stuff so -nonat is fine
>>>>>>>
>>>>>>> What you need to do is set
>>>>>>>
>>>>>>> 1) set the params ext-sip-ip and ext-rtp-ip to your external static IP
>>>>>>>
>>>>>>> 2) map the sip ports and all of the rtp ports from your static IP to FS lan
>>>>>>> addr
>>>>>>>
>>>>>>> 3) set sip-ip and rtp-ip to the lan addr you forwarded through.
>>>>>>>
>>>>>>>
>>>>>>> If you don't do this: your outbound registration will use NAT to your
>>>>>>>
>>>>>>> provider and if there is no activity for the expire time on your NAT
>>>>>>>
>>>>>>> mapping the reverse port mapping from your provider back to you is
>>>>>>>
>>>>>>> lost. This is why you set your register expires to a very low number,
>>>>>>>
>>>>>>> (you need to make sure the provider does not turn the expires back up
>>>>>>>
>>>>>>> in the reply because it will beat your choice *see sip trace) if this
>>>>>>>
>>>>>>> is the case then you need the "ping" option set to 30, to continuously
>>>>>>>
>>>>>>> send an options to your provider.
>>>>>>>
>>>>>>> The static mapping is obviously the better, easier and more reliable
>>>>>>> solution.
>>>>>>>
>>>>>>> So I want you to understand that the only way to keep a nat mapped
>>>>>>>
>>>>>>> port alive is to continuously send traffic, all the other methods that
>>>>>>>
>>>>>>> you are mentioning are to detect that phones registered to your are
>>>>>>>
>>>>>>> behind nat, I gave you that force-expires option before because your
>>>>>>>
>>>>>>> trace was full of inbound reg so I thought that is what you wanted
>>>>>>>
>>>>>>> help with.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Oct 27, 2010 at 10:43 AM, Mario G <mario_fs at mgtech.com> wrote:
>>>>>>>
>>>>>>> I should mention that I did not have this problem with an SPA9000 PBX
>>>>>>>
>>>>>>> (asterisk based) for over two years so FS may be pickier about upnp and/or
>>>>>>>
>>>>>>> nat, or just better at it exposing a problem in the router.
>>>>>>>
>>>>>>> I made different changes to the gateways to test different things. One
>>>>>>>
>>>>>>> failed after 17 hours, the other two stayed up. What did not work:
>>>>>>>
>>>>>>> added <variable name="sip-force-expires" value="30"/> to the directory
>>>>>>>
>>>>>>> entries as suggested.
>>>>>>>
>>>>>>> set the gateway expire times to 30 seconds.
>>>>>>>
>>>>>>> What worked (could be coincidental) for the two gateways that stayed up:
>>>>>>>
>>>>>>> I Added <param name="aggressive-nat-detection" value="true"/>
>>>>>>>
>>>>>>> I originally setup FS to use the static ip by setting external sip/rtp to
>>>>>>>
>>>>>>> just the static ip (no autonat:) and ran with -nonat but I could not get
>>>>>>>
>>>>>>> incoming calls. The only way it worked was to use autonat:1.2.3.4. The
>>>>>>>
>>>>>>> router has 1 static public address and 1 dynamic external IP, this is the
>>>>>>>
>>>>>>> root of the problem, upnp only tells FS about the dynamic ip Will keep this
>>>>>>>
>>>>>>> thread up-to-date for anyone who may be in the same boat someday. Thanks
>>>>>>>
>>>>>>> again for looking at the trace.
>>>>>>>
>>>>>>> Mario
>>>>>>>
>>>>>>> You should be setting the req freq to a low number on the outbound gateways
>>>>>>>
>>>>>>> The examples you showed had a series of inbound reg
>>>>>>>
>>>>>>> also set expire-seconds to 30 in your gateway xml
>>>>>>>
>>>>>>>
>>>>>>> The problem is if you are not constantly sending traffic to the box
>>>>>>>
>>>>>>> the nat mapping will go away.
>>>>>>>
>>>>>>> If you are in production you should be using a static ip with a static
>>>>>>>
>>>>>>> mapping, any trouble you are having is your own fault for playing with
>>>>>>>
>>>>>>> fire. The best we can do is tell you how to keep it contained.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Oct 26, 2010 at 12:34 PM, Mario G <mario_fs at mgtech.com> wrote:
>>>>>>>
>>>>>>> I made the change. I had no idea the settings for the inside phones effected
>>>>>>>
>>>>>>> nat for the outside sip accounts. I was looking into aggressive-nat-
>>>>>>>
>>>>>>> detection since the internal profile status always shows the right external
>>>>>>>
>>>>>>> static IP but the nat_ap status always shows the dynamic ip. Crossing
>>>>>>>
>>>>>>> fingers/etc since this problem is 85% of time (weeks!) into FS changeover.
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>> Mario
>>>>>>>
>>>>>>> On Oct 26, 2010, at 10:15 AM, Anthony Minessale wrote:
>>>>>>>
>>>>>>> add
>>>>>>>
>>>>>>> <variable name="sip-force-expires" value="30"/>
>>>>>>>
>>>>>>> to the <variables> section of your <user>
>>>>>>>
>>>>>>> you have it at 600 and the nat mapping is timing out while the 600
>>>>>>>
>>>>>>> seconds is ticking away
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Oct 26, 2010 at 12:01 PM, Mario G <mario_fs at mgtech.com> wrote:
>>>>>>>
>>>>>>> From the TSP:
>>>>>>>
>>>>>>> "I have enabled the SIP trace on your account. We are not currently seeing
>>>>>>>
>>>>>>> any registration attempts to your account within the last 15 minutes. Please
>>>>>>>
>>>>>>> restart FreeSwitch so that registration attempts begin again. Thank you. ".
>>>>>>>
>>>>>>> So FS is not getting past router.
>>>>>>>
>>>>>>> On Oct 26, 2010, at 9:09 AM, Mario G wrote:
>>>>>>>
>>>>>>> I ran the global trace during the problem and it is
>>>>>>>
>>>>>>> at http://pastebin.freeswitch.org/14324 . You can find "rnktel", "acctone",
>>>>>>>
>>>>>>> "accttwo", "acct3". The trace includes phones since it was global. I am
>>>>>>>
>>>>>>> using:
>>>>>>>
>>>>>>> <param name="ext-rtp-ip" value="autonat:my-static.ip"/>
>>>>>>>
>>>>>>> <param name="ext-sip-ip" value="autonat:my-static.ip"/>
>>>>>>>
>>>>>>> I tried dumping nat and removing the autonat: above and using -nonat but
>>>>>>>
>>>>>>> that did not work, registration proceeded but no calls inbound.
>>>>>>>
>>>>>>> On Oct 25, 2010, at 4:11 PM, Mario G wrote:
>>>>>>>
>>>>>>> Whoops, I am using an IP address for at least one gateway so that is not the
>>>>>>>
>>>>>>> problem:
>>>>>>>
>>>>>>> They look outbound to me and I am using dns for 2 and an IP for one so that
>>>>>>>
>>>>>>> is not the issue. I was able to get FS to clear this up by doing "nat_map
>>>>>>>
>>>>>>> reinit" which is why I think this is a nat problem. I will do the trace you
>>>>>>>
>>>>>>> mentioned. I will plug an ip address into one of the gateways to see what
>>>>>>>
>>>>>>> happens, they all fail at once. Thanks for responding!
>>>>>>>
>>>>>>> Mario
>>>>>>>
>>>>>>> On Oct 25, 2010, at 3:26 PM, Mario wrote:
>>>>>>>
>>>>>>> I really need help on this as I have weeks into this problem. I thought I
>>>>>>>
>>>>>>> had it nailed but I guess not. After 5.5 hours I get:
>>>>>>>
>>>>>>> 2010-10-25 15:05:43.407272 [WARNING] sofia_reg.c:387 mguuid Failed
>>>>>>>
>>>>>>> Registration, setting retry to 15 seconds.
>>>>>>>
>>>>>>> 2010-10-25 15:05:49.557478 [NOTICE] sofia_reg.c:342 Registering mvuuid
>>>>>>>
>>>>>>> 2010-10-25 15:05:59.206273 [NOTICE] sofia_reg.c:342 Registering mguuid
>>>>>>>
>>>>>>> 2010-10-25 15:06:04.923157 [WARNING] sofia_reg.c:387 mynum777 Failed
>>>>>>>
>>>>>>> Registration, setting retry to 30 seconds.
>>>>>>>
>>>>>>> 2010-10-25 15:06:05.358321 [WARNING] sofia_reg.c:387 mvuuid Failed
>>>>>>>
>>>>>>> Registration, setting retry to 15 seconds.
>>>>>>>
>>>>>>> 2010-10-25 15:06:16.125060 [WARNING] sofia_reg.c:387 mguuid Failed
>>>>>>>
>>>>>>> Registration, setting retry to 15 seconds.
>>>>>>>
>>>>>>> 2010-10-25 15:06:21.151240 [NOTICE] sofia_reg.c:342 Registering mvuuid
>>>>>>>
>>>>>>> 2010-10-25 15:06:33.060421 [NOTICE] sofia_reg.c:342 Registering mguuid
>>>>>>>
>>>>>>> 2010-10-25 15:06:35.392655 [NOTICE] sofia_reg.c:342 Registering mynum777
>>>>>>>
>>>>>>> and no way to make/get calls until I restart FS. I did this:
>>>>>>>
>>>>>>> 1. log 7
>>>>>>>
>>>>>>> 2. sofia profile xxxx siptrace on for each profile/gateway
>>>>>>>
>>>>>>> 3. restarted router
>>>>>>>
>>>>>>> All three did not solve the problem. The trace and log produced no
>>>>>>>
>>>>>>> additional lines which is why I am wondering if FS has a problem since the
>>>>>>>
>>>>>>> trace shows no SIP activity.
>>>>>>>
>>>>>>> 3 gateways with 2 ITSPs
>>>>>>>
>>>>>>> 2 DSL/WAN lines, 1 static and 1 dynamic
>>>>>>>
>>>>>>> I am using autonat:1.2.3.4 in internal and external profiles. 1.2.3.4 is the
>>>>>>>
>>>>>>> external static ip.
>>>>>>>
>>>>>>> sofia status profile ... has the right ext ip
>>>>>>>
>>>>>>> nat_map status shows the dynamic (wrong) IP
>>>>>>>
>>>>>>> I tried starting with -nonat but that was worse
>>>>>>>
>>>>>>> the only way to fix is restart FS.
>>>>>>>
>>>>>>> I read the wiki on external nat, auto_nat and everything else many times.
>>>>>>>
>>>>>>> Thanks Mario
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>>
>>>>>>> FreeSWITCH-users mailing list
>>>>>>>
>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>
>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>
>>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>
>>>>>>> http://www.freeswitch.org
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>>
>>>>>>> FreeSWITCH-users mailing list
>>>>>>>
>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>
>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>
>>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>
>>>>>>> http://www.freeswitch.org
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>>
>>>>>>> FreeSWITCH-users mailing list
>>>>>>>
>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>
>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>
>>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>
>>>>>>> http://www.freeswitch.org
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>>
>>>>>>> FreeSWITCH-users mailing list
>>>>>>>
>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>
>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>
>>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>
>>>>>>> http://www.freeswitch.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Anthony Minessale II
>>>>>>>
>>>>>>> FreeSWITCH http://www.freeswitch.org/
>>>>>>>
>>>>>>> ClueCon http://www.cluecon.com/
>>>>>>>
>>>>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>>>>>
>>>>>>> AIM: anthm
>>>>>>>
>>>>>>> MSN:anthony_minessale at hotmail.com
>>>>>>>
>>>>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>>>>>
>>>>>>> IRC: irc.freenode.net #freeswitch
>>>>>>>
>>>>>>> FreeSWITCH Developer Conference
>>>>>>>
>>>>>>> sip:888 at conference.freeswitch.org
>>>>>>>
>>>>>>> googletalk:conf+888 at conference.freeswitch.org
>>>>>>>
>>>>>>> pstn:+19193869900
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>>
>>>>>>> FreeSWITCH-users mailing list
>>>>>>>
>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>
>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>
>>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>
>>>>>>> http://www.freeswitch.org
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>>
>>>>>>> FreeSWITCH-users mailing list
>>>>>>>
>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>
>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>
>>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>
>>>>>>> http://www.freeswitch.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Anthony Minessale II
>>>>>>>
>>>>>>> FreeSWITCH http://www.freeswitch.org/
>>>>>>>
>>>>>>> ClueCon http://www.cluecon.com/
>>>>>>>
>>>>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>>>>>
>>>>>>> AIM: anthm
>>>>>>>
>>>>>>> MSN:anthony_minessale at hotmail.com
>>>>>>>
>>>>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>>>>>
>>>>>>> IRC: irc.freenode.net #freeswitch
>>>>>>>
>>>>>>> FreeSWITCH Developer Conference
>>>>>>>
>>>>>>> sip:888 at conference.freeswitch.org
>>>>>>>
>>>>>>> googletalk:conf+888 at conference.freeswitch.org
>>>>>>>
>>>>>>> pstn:+19193869900
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>>
>>>>>>> FreeSWITCH-users mailing list
>>>>>>>
>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>
>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>
>>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>
>>>>>>> http://www.freeswitch.org
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>>
>>>>>>> FreeSWITCH-users mailing list
>>>>>>>
>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>
>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>
>>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>
>>>>>>> http://www.freeswitch.org
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>>
>>>>>>> FreeSWITCH-users mailing list
>>>>>>>
>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>
>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>
>>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>
>>>>>>> http://www.freeswitch.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Anthony Minessale II
>>>>>>>
>>>>>>> FreeSWITCH http://www.freeswitch.org/
>>>>>>>
>>>>>>> ClueCon http://www.cluecon.com/
>>>>>>>
>>>>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>>>>>
>>>>>>> AIM: anthm
>>>>>>>
>>>>>>> MSN:anthony_minessale at hotmail.com
>>>>>>>
>>>>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>>>>>
>>>>>>> IRC: irc.freenode.net #freeswitch
>>>>>>>
>>>>>>> FreeSWITCH Developer Conference
>>>>>>>
>>>>>>> sip:888 at conference.freeswitch.org
>>>>>>>
>>>>>>> googletalk:conf+888 at conference.freeswitch.org
>>>>>>>
>>>>>>> pstn:+19193869900
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>>
>>>>>>> FreeSWITCH-users mailing list
>>>>>>>
>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>
>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>
>>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>
>>>>>>> http://www.freeswitch.org
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>>
>>>>>>> FreeSWITCH-users mailing list
>>>>>>>
>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>
>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>
>>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>
>>>>>>> http://www.freeswitch.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Anthony Minessale II
>>>>>>>
>>>>>>> FreeSWITCH http://www.freeswitch.org/
>>>>>>>
>>>>>>> ClueCon http://www.cluecon.com/
>>>>>>>
>>>>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>>>>>
>>>>>>> AIM: anthm
>>>>>>>
>>>>>>> MSN:anthony_minessale at hotmail.com
>>>>>>>
>>>>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>>>>>
>>>>>>> IRC: irc.freenode.net #freeswitch
>>>>>>>
>>>>>>> FreeSWITCH Developer Conference
>>>>>>>
>>>>>>> sip:888 at conference.freeswitch.org
>>>>>>>
>>>>>>> googletalk:conf+888 at conference.freeswitch.org
>>>>>>>
>>>>>>> pstn:+19193869900
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>>
>>>>>>> FreeSWITCH-users mailing list
>>>>>>>
>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>
>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>
>>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>
>>>>>>> http://www.freeswitch.org
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>>
>>>>>>> FreeSWITCH-users mailing list
>>>>>>>
>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>
>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>
>>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>
>>>>>>> http://www.freeswitch.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Anthony Minessale II
>>>>>>>
>>>>>>> FreeSWITCH http://www.freeswitch.org/
>>>>>>>
>>>>>>> ClueCon http://www.cluecon.com/
>>>>>>>
>>>>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>>>>>
>>>>>>> AIM: anthm
>>>>>>>
>>>>>>> MSN:anthony_minessale at hotmail.com
>>>>>>>
>>>>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>>>>>
>>>>>>> IRC: irc.freenode.net #freeswitch
>>>>>>>
>>>>>>> FreeSWITCH Developer Conference
>>>>>>>
>>>>>>> sip:888 at conference.freeswitch.org
>>>>>>>
>>>>>>> googletalk:conf+888 at conference.freeswitch.org
>>>>>>>
>>>>>>> pstn:+19193869900
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>>
>>>>>>> FreeSWITCH-users mailing list
>>>>>>>
>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>
>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>
>>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>
>>>>>>> http://www.freeswitch.org
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> FreeSWITCH-users mailing list
>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>> http://www.freeswitch.org
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> FreeSWITCH-users mailing list
>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>> http://www.freeswitch.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Anthony Minessale II
>>>>>>
>>>>>> FreeSWITCH http://www.freeswitch.org/
>>>>>> ClueCon http://www.cluecon.com/
>>>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>>>>
>>>>>> AIM: anthm
>>>>>> MSN:anthony_minessale at hotmail.com
>>>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>>>> IRC: irc.freenode.net #freeswitch
>>>>>>
>>>>>> FreeSWITCH Developer Conference
>>>>>> sip:888 at conference.freeswitch.org
>>>>>> googletalk:conf+888 at conference.freeswitch.org
>>>>>> pstn:+19193869900
>>>>>>
>>>>>> _______________________________________________
>>>>>> FreeSWITCH-users mailing list
>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> http://www.freeswitch.org
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Anthony Minessale II
>>>>
>>>> FreeSWITCH http://www.freeswitch.org/
>>>> ClueCon http://www.cluecon.com/
>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>>
>>>> AIM: anthm
>>>> MSN:anthony_minessale at hotmail.com
>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>> IRC: irc.freenode.net #freeswitch
>>>>
>>>> FreeSWITCH Developer Conference
>>>> sip:888 at conference.freeswitch.org
>>>> googletalk:conf+888 at conference.freeswitch.org
>>>> pstn:+19193869900
>>>>
>>>> _______________________________________________
>>>> FreeSWITCH-users mailing list
>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> http://www.freeswitch.org
>>>
>>>
>>> _______________________________________________
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>
>>
>> _______________________________________________
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>
>
>
> --
> Anthony Minessale II
>
> FreeSWITCH http://www.freeswitch.org/
> ClueCon http://www.cluecon.com/
> Twitter: http://twitter.com/FreeSWITCH_wire
>
> AIM: anthm
> MSN:anthony_minessale at hotmail.com
> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
> IRC: irc.freenode.net #freeswitch
>
> FreeSWITCH Developer Conference
> sip:888 at conference.freeswitch.org
> googletalk:conf+888 at conference.freeswitch.org
> pstn:+19193869900
>
> _______________________________________________
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
More information about the FreeSWITCH-users
mailing list