[Freeswitch-users] FreeSWITCH freezes when upgrading to 1.2 branch - SOLVED

[Ashley Breeden]

Hi Anthony,

Thanks for doing that so quickly, greatly appreciated.

Is it possible to have this patch added to the next 1.2 stable release?

Ash. 

On 13/09/2013, at 1:23 AM, Anthony Minessale <anthony.minessale at gmail.com> wrote:

> I checked the code and found the problem.  Zoiper was not happy to get a NOTIFY with Expires: 0 after sending a SUBSCRIBE with Expires: 0
> I added a patch to the code to skip responding with such a NOTIFY when zoiper is present.
> 
> This is not a real fix but its better than having everyone's box die.
> 
> 
> 
> On Thu, Sep 12, 2013 at 9:40 AM, Ashley Breeden <ash at url.net.au> wrote:
> Hi All,
> 
> I wanted to share my findings with everyone on the list in case anyone runs into the same issue I had as it what a nightmare to find.  A special thanks must be given to FreeSWITCH Consulting, especially Eliot who assisted in debugging and giving me some fresh ideas to look at.
> 
> For some time now I have struggled to upgrade my servers to 1.2 stable from 1.0.head (git-e3698c2 2012-03-23 01-21-55 +0100).  The issue I always faced was is that after I upgraded the servers it would run happily for the next 5-10 hours but for no reason FreeSWITCH would become unresponsive and stop accepting registrations.  If you ran command like show channels, status etc the command would just freeze and there would be no response.
> 
> Typically the freeze would occur when the servers were practically idle, looking at the logs you would see:
> 
> [WARNING] sofia_reg.c:1524 SIP auth challenge (REGISTER) on sofia profile 'internal' for [1050 at domain.com] from ip x.x.x.x
> [WARNING] sofia.c:1639 MSG Thread 2 Started
> [WARNING] sofia.c:1639 MSG Thread 1 Started
> 
> In attempt to resolve it I tried new hardware, changed from Centos to Debian, MySQL to Postgres, nothing I seemed to do fixed the issue.
> 
> I started to run some network analysis on the server and what I noticed is about 20 minutes before the server freeze the traffic would spike from 100kbps to around 400kbps, on closer inspection I noticed that the 300kbps spike was actually between the server and a single IP on port 5060.  Analysing the packets (tcpdump -s0 -A port 5060)  I could see that one registration was sending SUBSCRIBE events at the rate of about 300-400 per second and FreeSWITCH was sending a NOTIFY back for each SUBSCRIBE, once the server froze the traffic would stop which is what made it hard to find as you had to be looking at the network traffic before the freeze.  Once I knew the user causing the increased traffic flow I was able to work out that the user agent causing this was Zoiper (rev. 11137).  It would appear Zoiper would get into a loop with the server and keep sending SUBSCRIBE packets for the message waiting indicator.  
> 
> I have spoken to Zoiper who have taken the issue on board and informed me it does not happen on Zoiper 3, but does happen on Zoiper 2, however they plan to fix it in the next release of Zoiper 2.  They suggested for the moment to disable the MWI Subscribe option by doing "Go to the preferences -> select your sip account -> enable "show advanced options" andchange the mwi subscribe drop down to "disabled"
> 
> The strange thing with all this is that the same Zoiper client never had a problem on version 1.0.head (git-e3698c2 2012-03-23 01-21-55 +0100) but as soon as you upgraded to 1.2 or even the current git versions you would find that FreeSWITCH and Zoiper would get into a SUBSCRIBE/NOTIFY Loop.
> 
> I hope this information is of some help to someone else in the future and is able to ensure they do not have to go through the same nightmare I did.  
> 
> If anyone has any suggestions on ways to limit the SUBSCRIBE/NOTIFY responses so that you can avoid this happening in the future I would be very interested to hear, even if you have any other feedback that might have assisted in finding this issue quicker I would be very interested.
> 
> 
> Regards,
> 
> Ashley Breeden
> URL Networks Pty Ltd
> WWW: http://www.url.net.au
> Twitter: @urlnetworks
> 
> 
> 
> 
> 
> 
> 
> 
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
> 
> 
> 
> 
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
> 
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
> 
> 
> 
> 
> -- 
> Anthony Minessale II
> 
> FreeSWITCH http://www.freeswitch.org/
> ClueCon http://www.cluecon.com/
> Twitter: http://twitter.com/FreeSWITCH_wire
> 
> AIM: anthm
> MSN:anthony_minessale at hotmail.com
> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
> IRC: irc.freenode.net #freeswitch
> 
> FreeSWITCH Developer Conference
> sip:888 at conference.freeswitch.org
> googletalk:conf+888 at conference.freeswitch.org
> pstn:+19193869900
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
> 
> 
> 
> 
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
> 
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20130913/d9a095d0/attachment-0001.html