[Freeswitch-users] registration fails after several hours - FS problem?
Mario G
mario_fs at mgtech.com
Sat Oct 30 09:37:17 PDT 2010
Setting sofia loglevel to 9 made Terminal app memory go from 20meg to eventually using all memory and locking the machine up. I spent all night trying to get a good trace. I need to get one when the failure occurs early before the lockup. Will be working all w/e on this. Thanks for you help.
I got this with only 1 gateway defined but did not catch it early. Notice how there is no SIP trace activity:
2010-10-30 09:30:15.046900 [WARNING] sofia_reg.c:387 mvvyl Failed Registration, setting retry to 15 seconds.
nta: timer K fired, terminate NOTIFY (3892042)
outgoing_reclaim_all(0x0, 0x0, 0x102309cb0)
nta_outgoing_timer: 0/0 resent, 0/0 tout, 1/1 term, 1/1 free
nta: timer not set
2010-10-30 09:30:31.926382 [NOTICE] sofia_reg.c:342 Registering mvvyl
nua: nua_register: entering
nua(0x1003c3750): sent signal r_register
nua(0x1003c3750): recv signal r_register
2010-10-30 09:30:47.685212 [WARNING] sofia_reg.c:387 mvvyl Failed Registration, setting retry to 15 seconds.
2010-10-30 09:31:03.478941 [NOTICE] sofia_reg.c:342 Registering mvvyl
nua: nua_register: entering
nua(0x1003c3750): sent signal r_register
nua(0x1003c3750): recv signal r_register
2010-10-30 09:31:19.219346 [WARNING] sofia_reg.c:387 mvvyl Failed Registration, setting retry to 15 seconds.
2010-10-30 09:31:36.208448 [NOTICE] sofia_reg.c:342 Registering mvvyl
nua: nua_register: entering
nua(0x1003c3750): sent signal r_register
nua(0x1003c3750): recv signal r_register
Mario
>
> On Oct 29, 2010, at 5:43 PM, Anthony Minessale wrote:
>
>> also raise loglevel to debug
>> console loglevel debug
>>
>>
>> On Fri, Oct 29, 2010 at 7:29 PM, Mario G <mario_fs at mgtech.com> wrote:
>>> Will so, BTW, I mentioned below one is dns and another uses IP, I tested that theory using an IP does not help. I also updated to todays git version and no help there. Will post when the trace is done.
>>>
>>> On Oct 29, 2010, at 5:17 PM, Anthony Minessale wrote:
>>>
>>>> can you repeat that trace with sofia debug on
>>>> sofia loglevel all 9
>>>>
>>>> Are you doing DNS by any chance in the gateway "proxy" param?
>>>> you could try filling in the register-proxy param in your gateway to
>>>> sip:<ip> <-- not dns but ip that dns resolves to
>>>> <param name="register-proxy" value="sip:1.2.3.4"/>
>>>>
>>>> I'm just guessing but Its possible some bad dns query could be throwing FS off.
>>>> so this test would force all the packets to the exact IP of your host
>>>> instead of looking it up.
>>>>
>>>>
>>>>
>>>> On Fri, Oct 29, 2010 at 6:30 PM, Mario G <mario_fs at mgtech.com> wrote:
>>>>> Oh Boy: Not only is everything set to the static route but I turned
>>>>> off/disconnected the dynamic DSL line so I only had 1 static line to the
>>>>> router. The second wan is set in the router off. Even turned off router uPnP
>>>>> even though I am using -nonat. Guess what.... I still have the problem. Look
>>>>> like when FS says is going to retry it actually does not.
>>>>> Here is a short one: http://pastebin.freeswitch.org/14359 - I caught one
>>>>> right after I started FS, failure occurred in minutes (lucky). Look at the
>>>>> bottom of the trace, you see SIP trace activity and then when it fails no
>>>>> SIP trace activity. Could this possibly be a FS bug? (I am a mainframe
>>>>> assembler systems programmer and I might think so if there were error retry
>>>>> messages but nothing showing in one of my traces...)
>>>>> Notes:
>>>>> idone is gateway 1
>>>>> idtwo is gateway 2 I had to trace both because it was impossible to figure
>>>>> out which one would fail first. Ran several times but kept missing the right
>>>>> one.
>>>>> I use a url for one gateway and ip for another but it makes no difference
>>>>> since both eventually fail.
>>>>> 10. is local lan
>>>>> 210. is external ip
>>>>> 216. is itsp
>>>>>
>>>>> Here is a longer one from earlier http://pastebin.freeswitch.org/14357
>>>>> Notes:
>>>>> A call was received and hung up for idtwo - beginning of trace
>>>>> 11 minutes later idtwo failed - see last line of trace
>>>>>
>>>>> Thank you very much!
>>>>> Mario
>>>>>
>>>>> On Oct 29, 2010, at 12:11 PM, Anthony Minessale wrote:
>>>>>
>>>>> stun-enabled must be true in your profile XML to see what you pasted.
>>>>>
>>>>> Get me a sip trace of this from when it works until when it fails
>>>>>
>>>>> only enable the sip trace on the profile with the gateway to reduce traffic
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Oct 29, 2010 at 1:56 PM, Mario G <mario_fs at mgtech.com> wrote:
>>>>>
>>>>> Oh my... looks it was not NAT after all? Please help! I changed to all
>>>>> profiles to static per instructions below and still have the problem:
>>>>>
>>>>> 2010-10-29 11:15:18.536446 [NOTICE] sofia_reg.c:342 Registering uuid1
>>>>>
>>>>> 2010-10-29 11:15:34.313150 [WARNING] sofia_reg.c:387 uuid1 Failed
>>>>> Registration, setting retry to 15 seconds.
>>>>>
>>>>> sofia global siptrace on did not show any activity for this gateway in or
>>>>> out, others were fine but eventually fail. I setup static:
>>>>>
>>>>> 1. set the params ext-sip-ip and ext-rtp-ip to my external static IP
>>>>>
>>>>> 2. map the sip ports (5060-5080) and all of the rtp ports (16384-32767) to
>>>>> FS lan addr.
>>>>>
>>>>> 3. set sip-ip and rtp-ip to the lan addr of FS
>>>>>
>>>>> 4. start FS with -nonat
>>>>>
>>>>> I don't know what to try next. BTW, the sofia status for the profiles shows
>>>>> stun enabled but I did not set it up anywhere:
>>>>>
>>>>> Name uuid1
>>>>>
>>>>> Domain Name N/A
>>>>>
>>>>> Auto-NAT false
>>>>>
>>>>> DBName sofia_reg_mvvyl
>>>>>
>>>>> Pres Hosts
>>>>>
>>>>> Dialplan XML
>>>>>
>>>>> Context public
>>>>>
>>>>> Challenge Realm auto_to
>>>>>
>>>>> RTP-IP 10.x.x.20
>>>>>
>>>>> Ext-RTP-IP 210.x.x.100
>>>>>
>>>>> SIP-IP 10.x.x.20
>>>>>
>>>>> Ext-SIP-IP 210.x.x.100
>>>>>
>>>>> URL sip:mod_sofia at 210.x.x.100:5068
>>>>>
>>>>> BIND-URL sip:mod_sofia at 210.x.x.100:5068;maddr=10.x.x.20
>>>>>
>>>>> HOLD-MUSIC local_stream://moh
>>>>>
>>>>> OUTBOUND-PROXY N/A
>>>>>
>>>>> CODECS IN G7221 at 32000h,G7221 at 16000h,G722,PCMU,PCMA,GSM
>>>>>
>>>>> CODECS OUT PCMU,PCMA,GSM
>>>>>
>>>>> TEL-EVENT 101
>>>>>
>>>>> DTMF-MODE rfc2833
>>>>>
>>>>> CNG 13
>>>>>
>>>>> SESSION-TO 0
>>>>>
>>>>> MAX-DIALOG 0
>>>>>
>>>>> NOMEDIA false
>>>>>
>>>>> LATE-NEG false
>>>>>
>>>>> PROXY-MEDIA false
>>>>>
>>>>> AGGRESSIVENAT false
>>>>>
>>>>> STUN-ENABLED true
>>>>>
>>>>> STUN-AUTO-DISABLE false
>>>>>
>>>>> CALLS-IN 0
>>>>>
>>>>> FAILED-CALLS-IN 0
>>>>>
>>>>> CALLS-OUT 0
>>>>>
>>>>> FAILED-CALLS-OUT 0
>>>>>
>>>>>
>>>>> On Oct 27, 2010, at 10:04 AM, Anthony Minessale wrote:
>>>>>
>>>>> if you map it or not, a scanner would penetrate it.
>>>>>
>>>>> There are lot of sip scanners out there now, you just need to beware of
>>>>> them.
>>>>>
>>>>>
>>>>> On Wed, Oct 27, 2010 at 11:50 AM, Mario G <mario_fs at mgtech.com> wrote:
>>>>>
>>>>> Thanks so much! I am sure many others will find this info invaluable. I will
>>>>> try the static route again but have one question: When I started with FS I
>>>>> found a "sip scanner" in FS and someone on this group said not to use port
>>>>> mapping since it was a security risk. Is that true?
>>>>>
>>>>> On Oct 27, 2010, at 9:10 AM, Anthony Minessale wrote:
>>>>>
>>>>> you are completely guessing at things.
>>>>>
>>>>> I want you to understand that the only reason you are having problems
>>>>>
>>>>> with this is because you don't understand how it works enough to know
>>>>>
>>>>> what you are doing 100%
>>>>>
>>>>> Its a given that the pnp stuff is only for your dynamic IP.
>>>>>
>>>>> aggressive-nat-detection and sip-force-expires are all related to
>>>>>
>>>>> inbound calls when the things who are registering to you may be behind
>>>>>
>>>>> nat.
>>>>>
>>>>> You need to learn the difference between which nat tools are
>>>>>
>>>>> *) designed for your FS to run behind nat
>>>>>
>>>>> *) designed for FS to run public and accept connections from devices behind
>>>>> nat.
>>>>>
>>>>> If you have a static IP, you don't need the pnp stuff so -nonat is fine
>>>>>
>>>>> What you need to do is set
>>>>>
>>>>> 1) set the params ext-sip-ip and ext-rtp-ip to your external static IP
>>>>>
>>>>> 2) map the sip ports and all of the rtp ports from your static IP to FS lan
>>>>> addr
>>>>>
>>>>> 3) set sip-ip and rtp-ip to the lan addr you forwarded through.
>>>>>
>>>>>
>>>>> If you don't do this: your outbound registration will use NAT to your
>>>>>
>>>>> provider and if there is no activity for the expire time on your NAT
>>>>>
>>>>> mapping the reverse port mapping from your provider back to you is
>>>>>
>>>>> lost. This is why you set your register expires to a very low number,
>>>>>
>>>>> (you need to make sure the provider does not turn the expires back up
>>>>>
>>>>> in the reply because it will beat your choice *see sip trace) if this
>>>>>
>>>>> is the case then you need the "ping" option set to 30, to continuously
>>>>>
>>>>> send an options to your provider.
>>>>>
>>>>> The static mapping is obviously the better, easier and more reliable
>>>>> solution.
>>>>>
>>>>> So I want you to understand that the only way to keep a nat mapped
>>>>>
>>>>> port alive is to continuously send traffic, all the other methods that
>>>>>
>>>>> you are mentioning are to detect that phones registered to your are
>>>>>
>>>>> behind nat, I gave you that force-expires option before because your
>>>>>
>>>>> trace was full of inbound reg so I thought that is what you wanted
>>>>>
>>>>> help with.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Oct 27, 2010 at 10:43 AM, Mario G <mario_fs at mgtech.com> wrote:
>>>>>
>>>>> I should mention that I did not have this problem with an SPA9000 PBX
>>>>>
>>>>> (asterisk based) for over two years so FS may be pickier about upnp and/or
>>>>>
>>>>> nat, or just better at it exposing a problem in the router.
>>>>>
>>>>> I made different changes to the gateways to test different things. One
>>>>>
>>>>> failed after 17 hours, the other two stayed up. What did not work:
>>>>>
>>>>> added <variable name="sip-force-expires" value="30"/> to the directory
>>>>>
>>>>> entries as suggested.
>>>>>
>>>>> set the gateway expire times to 30 seconds.
>>>>>
>>>>> What worked (could be coincidental) for the two gateways that stayed up:
>>>>>
>>>>> I Added <param name="aggressive-nat-detection" value="true"/>
>>>>>
>>>>> I originally setup FS to use the static ip by setting external sip/rtp to
>>>>>
>>>>> just the static ip (no autonat:) and ran with -nonat but I could not get
>>>>>
>>>>> incoming calls. The only way it worked was to use autonat:1.2.3.4. The
>>>>>
>>>>> router has 1 static public address and 1 dynamic external IP, this is the
>>>>>
>>>>> root of the problem, upnp only tells FS about the dynamic ip Will keep this
>>>>>
>>>>> thread up-to-date for anyone who may be in the same boat someday. Thanks
>>>>>
>>>>> again for looking at the trace.
>>>>>
>>>>> Mario
>>>>>
>>>>> You should be setting the req freq to a low number on the outbound gateways
>>>>>
>>>>> The examples you showed had a series of inbound reg
>>>>>
>>>>> also set expire-seconds to 30 in your gateway xml
>>>>>
>>>>>
>>>>> The problem is if you are not constantly sending traffic to the box
>>>>>
>>>>> the nat mapping will go away.
>>>>>
>>>>> If you are in production you should be using a static ip with a static
>>>>>
>>>>> mapping, any trouble you are having is your own fault for playing with
>>>>>
>>>>> fire. The best we can do is tell you how to keep it contained.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Oct 26, 2010 at 12:34 PM, Mario G <mario_fs at mgtech.com> wrote:
>>>>>
>>>>> I made the change. I had no idea the settings for the inside phones effected
>>>>>
>>>>> nat for the outside sip accounts. I was looking into aggressive-nat-
>>>>>
>>>>> detection since the internal profile status always shows the right external
>>>>>
>>>>> static IP but the nat_ap status always shows the dynamic ip. Crossing
>>>>>
>>>>> fingers/etc since this problem is 85% of time (weeks!) into FS changeover.
>>>>>
>>>>> Thanks!
>>>>>
>>>>> Mario
>>>>>
>>>>> On Oct 26, 2010, at 10:15 AM, Anthony Minessale wrote:
>>>>>
>>>>> add
>>>>>
>>>>> <variable name="sip-force-expires" value="30"/>
>>>>>
>>>>> to the <variables> section of your <user>
>>>>>
>>>>> you have it at 600 and the nat mapping is timing out while the 600
>>>>>
>>>>> seconds is ticking away
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Oct 26, 2010 at 12:01 PM, Mario G <mario_fs at mgtech.com> wrote:
>>>>>
>>>>> From the TSP:
>>>>>
>>>>> "I have enabled the SIP trace on your account. We are not currently seeing
>>>>>
>>>>> any registration attempts to your account within the last 15 minutes. Please
>>>>>
>>>>> restart FreeSwitch so that registration attempts begin again. Thank you. ".
>>>>>
>>>>> So FS is not getting past router.
>>>>>
>>>>> On Oct 26, 2010, at 9:09 AM, Mario G wrote:
>>>>>
>>>>> I ran the global trace during the problem and it is
>>>>>
>>>>> at http://pastebin.freeswitch.org/14324 . You can find "rnktel", "acctone",
>>>>>
>>>>> "accttwo", "acct3". The trace includes phones since it was global. I am
>>>>>
>>>>> using:
>>>>>
>>>>> <param name="ext-rtp-ip" value="autonat:my-static.ip"/>
>>>>>
>>>>> <param name="ext-sip-ip" value="autonat:my-static.ip"/>
>>>>>
>>>>> I tried dumping nat and removing the autonat: above and using -nonat but
>>>>>
>>>>> that did not work, registration proceeded but no calls inbound.
>>>>>
>>>>> On Oct 25, 2010, at 4:11 PM, Mario G wrote:
>>>>>
>>>>> Whoops, I am using an IP address for at least one gateway so that is not the
>>>>>
>>>>> problem:
>>>>>
>>>>> They look outbound to me and I am using dns for 2 and an IP for one so that
>>>>>
>>>>> is not the issue. I was able to get FS to clear this up by doing "nat_map
>>>>>
>>>>> reinit" which is why I think this is a nat problem. I will do the trace you
>>>>>
>>>>> mentioned. I will plug an ip address into one of the gateways to see what
>>>>>
>>>>> happens, they all fail at once. Thanks for responding!
>>>>>
>>>>> Mario
>>>>>
>>>>> On Oct 25, 2010, at 3:26 PM, Mario wrote:
>>>>>
>>>>> I really need help on this as I have weeks into this problem. I thought I
>>>>>
>>>>> had it nailed but I guess not. After 5.5 hours I get:
>>>>>
>>>>> 2010-10-25 15:05:43.407272 [WARNING] sofia_reg.c:387 mguuid Failed
>>>>>
>>>>> Registration, setting retry to 15 seconds.
>>>>>
>>>>> 2010-10-25 15:05:49.557478 [NOTICE] sofia_reg.c:342 Registering mvuuid
>>>>>
>>>>> 2010-10-25 15:05:59.206273 [NOTICE] sofia_reg.c:342 Registering mguuid
>>>>>
>>>>> 2010-10-25 15:06:04.923157 [WARNING] sofia_reg.c:387 mynum777 Failed
>>>>>
>>>>> Registration, setting retry to 30 seconds.
>>>>>
>>>>> 2010-10-25 15:06:05.358321 [WARNING] sofia_reg.c:387 mvuuid Failed
>>>>>
>>>>> Registration, setting retry to 15 seconds.
>>>>>
>>>>> 2010-10-25 15:06:16.125060 [WARNING] sofia_reg.c:387 mguuid Failed
>>>>>
>>>>> Registration, setting retry to 15 seconds.
>>>>>
>>>>> 2010-10-25 15:06:21.151240 [NOTICE] sofia_reg.c:342 Registering mvuuid
>>>>>
>>>>> 2010-10-25 15:06:33.060421 [NOTICE] sofia_reg.c:342 Registering mguuid
>>>>>
>>>>> 2010-10-25 15:06:35.392655 [NOTICE] sofia_reg.c:342 Registering mynum777
>>>>>
>>>>> and no way to make/get calls until I restart FS. I did this:
>>>>>
>>>>> 1. log 7
>>>>>
>>>>> 2. sofia profile xxxx siptrace on for each profile/gateway
>>>>>
>>>>> 3. restarted router
>>>>>
>>>>> All three did not solve the problem. The trace and log produced no
>>>>>
>>>>> additional lines which is why I am wondering if FS has a problem since the
>>>>>
>>>>> trace shows no SIP activity.
>>>>>
>>>>> 3 gateways with 2 ITSPs
>>>>>
>>>>> 2 DSL/WAN lines, 1 static and 1 dynamic
>>>>>
>>>>> I am using autonat:1.2.3.4 in internal and external profiles. 1.2.3.4 is the
>>>>>
>>>>> external static ip.
>>>>>
>>>>> sofia status profile ... has the right ext ip
>>>>>
>>>>> nat_map status shows the dynamic (wrong) IP
>>>>>
>>>>> I tried starting with -nonat but that was worse
>>>>>
>>>>> the only way to fix is restart FS.
>>>>>
>>>>> I read the wiki on external nat, auto_nat and everything else many times.
>>>>>
>>>>> Thanks Mario
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>>
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>>
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>
>>>>> http://www.freeswitch.org
>>>>>
>>>>> _______________________________________________
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>>
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>>
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Anthony Minessale II
>>>>>
>>>>> FreeSWITCH http://www.freeswitch.org/
>>>>>
>>>>> ClueCon http://www.cluecon.com/
>>>>>
>>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>>>
>>>>> AIM: anthm
>>>>>
>>>>> MSN:anthony_minessale at hotmail.com
>>>>>
>>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>>>
>>>>> IRC: irc.freenode.net #freeswitch
>>>>>
>>>>> FreeSWITCH Developer Conference
>>>>>
>>>>> sip:888 at conference.freeswitch.org
>>>>>
>>>>> googletalk:conf+888 at conference.freeswitch.org
>>>>>
>>>>> pstn:+19193869900
>>>>>
>>>>> _______________________________________________
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>>
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>>
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Anthony Minessale II
>>>>>
>>>>> FreeSWITCH http://www.freeswitch.org/
>>>>>
>>>>> ClueCon http://www.cluecon.com/
>>>>>
>>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>>>
>>>>> AIM: anthm
>>>>>
>>>>> MSN:anthony_minessale at hotmail.com
>>>>>
>>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>>>
>>>>> IRC: irc.freenode.net #freeswitch
>>>>>
>>>>> FreeSWITCH Developer Conference
>>>>>
>>>>> sip:888 at conference.freeswitch.org
>>>>>
>>>>> googletalk:conf+888 at conference.freeswitch.org
>>>>>
>>>>> pstn:+19193869900
>>>>>
>>>>> _______________________________________________
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>>
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>>
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>>
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Anthony Minessale II
>>>>>
>>>>> FreeSWITCH http://www.freeswitch.org/
>>>>>
>>>>> ClueCon http://www.cluecon.com/
>>>>>
>>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>>>
>>>>> AIM: anthm
>>>>>
>>>>> MSN:anthony_minessale at hotmail.com
>>>>>
>>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>>>
>>>>> IRC: irc.freenode.net #freeswitch
>>>>>
>>>>> FreeSWITCH Developer Conference
>>>>>
>>>>> sip:888 at conference.freeswitch.org
>>>>>
>>>>> googletalk:conf+888 at conference.freeswitch.org
>>>>>
>>>>> pstn:+19193869900
>>>>>
>>>>> _______________________________________________
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>>
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>>
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Anthony Minessale II
>>>>>
>>>>> FreeSWITCH http://www.freeswitch.org/
>>>>>
>>>>> ClueCon http://www.cluecon.com/
>>>>>
>>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>>>
>>>>> AIM: anthm
>>>>>
>>>>> MSN:anthony_minessale at hotmail.com
>>>>>
>>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>>>
>>>>> IRC: irc.freenode.net #freeswitch
>>>>>
>>>>> FreeSWITCH Developer Conference
>>>>>
>>>>> sip:888 at conference.freeswitch.org
>>>>>
>>>>> googletalk:conf+888 at conference.freeswitch.org
>>>>>
>>>>> pstn:+19193869900
>>>>>
>>>>> _______________________________________________
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>>
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>>
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Anthony Minessale II
>>>>>
>>>>> FreeSWITCH http://www.freeswitch.org/
>>>>>
>>>>> ClueCon http://www.cluecon.com/
>>>>>
>>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>>>
>>>>> AIM: anthm
>>>>>
>>>>> MSN:anthony_minessale at hotmail.com
>>>>>
>>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>>>
>>>>> IRC: irc.freenode.net #freeswitch
>>>>>
>>>>> FreeSWITCH Developer Conference
>>>>>
>>>>> sip:888 at conference.freeswitch.org
>>>>>
>>>>> googletalk:conf+888 at conference.freeswitch.org
>>>>>
>>>>> pstn:+19193869900
>>>>>
>>>>> _______________________________________________
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>>
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Anthony Minessale II
>>>>
>>>> FreeSWITCH http://www.freeswitch.org/
>>>> ClueCon http://www.cluecon.com/
>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>>
>>>> AIM: anthm
>>>> MSN:anthony_minessale at hotmail.com
>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>> IRC: irc.freenode.net #freeswitch
>>>>
>>>> FreeSWITCH Developer Conference
>>>> sip:888 at conference.freeswitch.org
>>>> googletalk:conf+888 at conference.freeswitch.org
>>>> pstn:+19193869900
>>>>
>>>> _______________________________________________
>>>> FreeSWITCH-users mailing list
>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> http://www.freeswitch.org
>>>
>>>
>>> _______________________________________________
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>
>>
>>
>> --
>> Anthony Minessale II
>>
>> FreeSWITCH http://www.freeswitch.org/
>> ClueCon http://www.cluecon.com/
>> Twitter: http://twitter.com/FreeSWITCH_wire
>>
>> AIM: anthm
>> MSN:anthony_minessale at hotmail.com
>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>> IRC: irc.freenode.net #freeswitch
>>
>> FreeSWITCH Developer Conference
>> sip:888 at conference.freeswitch.org
>> googletalk:conf+888 at conference.freeswitch.org
>> pstn:+19193869900
>>
>> _______________________________________________
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>
>
> _______________________________________________
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
More information about the FreeSWITCH-users
mailing list