[Freeswitch-users] FreeSWITCH freezes when upgrading to 1.2 branch - SOLVED

[Anthony Minessale]

yes


On Thu, Sep 12, 2013 at 9:50 PM, Ashley Breeden <ash at url.net.au> wrote:

> Hi Anthony,
>
> Thanks for doing that so quickly, greatly appreciated.
>
> Is it possible to have this patch added to the next 1.2 stable release?
>
> Ash.
>
> On 13/09/2013, at 1:23 AM, Anthony Minessale <anthony.minessale at gmail.com>
> wrote:
>
> I checked the code and found the problem.  Zoiper was not happy to get a
> NOTIFY with Expires: 0 after sending a SUBSCRIBE with Expires: 0
> I added a patch to the code to skip responding with such a NOTIFY when
> zoiper is present.
>
> This is not a real fix but its better than having everyone's box die.
>
>
>
> On Thu, Sep 12, 2013 at 9:40 AM, Ashley Breeden <ash at url.net.au> wrote:
>
>> Hi All,
>>
>> I wanted to share my findings with everyone on the list in case anyone
>> runs into the same issue I had as it what a nightmare to find.  A special
>> thanks must be given to FreeSWITCH Consulting, especially Eliot who
>> assisted in debugging and giving me some fresh ideas to look at.
>>
>> For some time now I have struggled to upgrade my servers to 1.2 stable
>> from 1.0.head (git-e3698c2 2012-03-23 01-21-55 +0100).  The issue I always
>> faced was is that after I upgraded the servers it would run happily for the
>> next 5-10 hours but for no reason FreeSWITCH would become unresponsive and
>> stop accepting registrations.  If you ran command like show channels,
>> status etc the command would just freeze and there would be no response.
>>
>> Typically the freeze would occur when the servers were practically idle,
>> looking at the logs you would see:
>>
>> [WARNING] sofia_reg.c:1524 SIP auth challenge (REGISTER) on sofia profile
>> 'internal' for [1050 at domain.com] from ip x.x.x.x
>> [WARNING] sofia.c:1639 MSG Thread 2 Started
>> [WARNING] sofia.c:1639 MSG Thread 1 Started
>>
>> In attempt to resolve it I tried new hardware, changed from Centos to
>> Debian, MySQL to Postgres, nothing I seemed to do fixed the issue.
>>
>> I started to run some network analysis on the server and what I noticed
>> is about 20 minutes before the server freeze the traffic would spike from
>> 100kbps to around 400kbps, on closer inspection I noticed that the 300kbps
>> spike was actually between the server and a single IP on port 5060.
>>  Analysing the packets (tcpdump -s0 -A port 5060)  I could see that one
>> registration was sending SUBSCRIBE events at the rate of about 300-400 per
>> second and FreeSWITCH was sending a NOTIFY back for each SUBSCRIBE, once
>> the server froze the traffic would stop which is what made it hard to find
>> as you had to be looking at the network traffic before the freeze.  Once I
>> knew the user causing the increased traffic flow I was able to work out
>> that the user agent causing this was Zoiper (rev. 11137).  It would appear
>> Zoiper would get into a loop with the server and keep sending SUBSCRIBE
>> packets for the message waiting indicator.
>>
>> I have spoken to Zoiper who have taken the issue on board and informed me
>> it does not happen on Zoiper 3, but does happen on Zoiper 2, however they
>> plan to fix it in the next release of Zoiper 2.  They suggested for the
>> moment to disable the MWI Subscribe option by doing "Go to the preferences
>> -> select your sip account -> enable "show advanced options" andchange the
>> mwi subscribe drop down to "disabled"
>>
>> The strange thing with all this is that the same Zoiper client never had
>> a problem on version 1.0.head (git-e3698c2 2012-03-23 01-21-55 +0100) but
>> as soon as you upgraded to 1.2 or even the current git versions you would
>> find that FreeSWITCH and Zoiper would get into a SUBSCRIBE/NOTIFY Loop.
>>
>> I hope this information is of some help to someone else in the future and
>> is able to ensure they do not have to go through the same nightmare I did.
>>
>> If anyone has any suggestions on ways to limit the SUBSCRIBE/NOTIFY
>> responses so that you can avoid this happening in the future I would be
>> very interested to hear, even if you have any other feedback that might
>> have assisted in finding this issue quicker I would be very interested.
>>
>>
>>   Regards,
>>
>> *Ashley Breeden*
>> URL Networks Pty Ltd
>> *WWW:* http://www.url.net.au
>> *Twitter: *@urlnetworks
>>
>> *
>> *
>>
>>
>>
>>
>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> 
>> 
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>>
>
>
> --
> Anthony Minessale II
>
> FreeSWITCH http://www.freeswitch.org/
> ClueCon http://www.cluecon.com/
> Twitter: http://twitter.com/FreeSWITCH_wire
>
> AIM: anthm
> MSN:anthony_minessale at hotmail.com
> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
> IRC: irc.freenode.net #freeswitch
>
> FreeSWITCH Developer Conference
> sip:888 at conference.freeswitch.org
> googletalk:conf+888 at conference.freeswitch.org
> pstn:+19193869900
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> 
> 
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> 
> 
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>


-- 
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org
googletalk:conf+888 at conference.freeswitch.org
pstn:+19193869900
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20130912/7c569d51/attachment.html