[Freeswitch-users] FreeSWITCH freezes when upgrading to 1.2 branch - SOLVED

Fri Sep 13 17:24:05 MSD 2013

Tony, Thank you for the fix. This will help a lot.

Ashley and Regis, I would like to share some information about this bug. I
reported in july http://jira.freeswitch.org/browse/FS-5582 and Tony find
the reason why this is happening.

Then I went to ClueCon. This is my first time in ClueCon. Before the
conference night in the Pizza party, I met Tony and introduce myself. He
then introduce the Zoper developer and explain this problem.

---

I went with so many questions in my mind before ClueCon. I got all the
answers by end of ClueCon.

Tony and Team - Thank you very much for such a Great Conference and help
all the time.

Thank you
Lloyd

On Fri, Sep 13, 2013 at 2:46 AM, Regis M <regis.freeswitch.org at tornad.net>wrote:

> Hi,
> Thanks a lot for this bug reporting.
> We need more bug/problems solved report like this. It is very interesting
> and important to find help when we try to solve similar bugs and find
> people who report a clear and factual solution to a problem and not only
> bug or asking help with big thread with a lot a suggestion...
>
> Thanks a lot and a nice work to solve this..!! :).
>
> Regards,
>
>
>
>
>
>
> 2013/9/13 Anthony Minessale <anthony.minessale at gmail.com>
>
>> yes
>>
>>
>> On Thu, Sep 12, 2013 at 9:50 PM, Ashley Breeden <ash at url.net.au> wrote:
>>
>>> Hi Anthony,
>>>
>>> Thanks for doing that so quickly, greatly appreciated.
>>>
>>> Is it possible to have this patch added to the next 1.2 stable release?
>>>
>>> Ash.
>>>
>>> On 13/09/2013, at 1:23 AM, Anthony Minessale <
>>> anthony.minessale at gmail.com> wrote:
>>>
>>> I checked the code and found the problem.  Zoiper was not happy to get a
>>> NOTIFY with Expires: 0 after sending a SUBSCRIBE with Expires: 0
>>> I added a patch to the code to skip responding with such a NOTIFY when
>>> zoiper is present.
>>>
>>> This is not a real fix but its better than having everyone's box die.
>>>
>>>
>>>
>>> On Thu, Sep 12, 2013 at 9:40 AM, Ashley Breeden <ash at url.net.au> wrote:
>>>
>>>> Hi All,
>>>>
>>>> I wanted to share my findings with everyone on the list in case anyone
>>>> runs into the same issue I had as it what a nightmare to find.  A special
>>>> thanks must be given to FreeSWITCH Consulting, especially Eliot who
>>>> assisted in debugging and giving me some fresh ideas to look at.
>>>>
>>>> For some time now I have struggled to upgrade my servers to 1.2 stable
>>>> from 1.0.head (git-e3698c2 2012-03-23 01-21-55 +0100).  The issue I always
>>>> faced was is that after I upgraded the servers it would run happily for the
>>>> next 5-10 hours but for no reason FreeSWITCH would become unresponsive and
>>>> stop accepting registrations.  If you ran command like show channels,
>>>> status etc the command would just freeze and there would be no response.
>>>>
>>>> Typically the freeze would occur when the servers were practically
>>>> idle, looking at the logs you would see:
>>>>
>>>> [WARNING] sofia_reg.c:1524 SIP auth challenge (REGISTER) on sofia
>>>> profile 'internal' for [1050 at domain.com] from ip x.x.x.x
>>>> [WARNING] sofia.c:1639 MSG Thread 2 Started
>>>> [WARNING] sofia.c:1639 MSG Thread 1 Started
>>>>
>>>> In attempt to resolve it I tried new hardware, changed from Centos to
>>>> Debian, MySQL to Postgres, nothing I seemed to do fixed the issue.
>>>>
>>>> I started to run some network analysis on the server and what I noticed
>>>> is about 20 minutes before the server freeze the traffic would spike from
>>>> 100kbps to around 400kbps, on closer inspection I noticed that the 300kbps
>>>> spike was actually between the server and a single IP on port 5060.
>>>>  Analysing the packets (tcpdump -s0 -A port 5060)  I could see that one
>>>> registration was sending SUBSCRIBE events at the rate of about 300-400 per
>>>> second and FreeSWITCH was sending a NOTIFY back for each SUBSCRIBE, once
>>>> the server froze the traffic would stop which is what made it hard to find
>>>> as you had to be looking at the network traffic before the freeze.  Once I
>>>> knew the user causing the increased traffic flow I was able to work out
>>>> that the user agent causing this was Zoiper (rev. 11137).  It would appear
>>>> Zoiper would get into a loop with the server and keep sending SUBSCRIBE
>>>> packets for the message waiting indicator.
>>>>
>>>> I have spoken to Zoiper who have taken the issue on board and informed
>>>> me it does not happen on Zoiper 3, but does happen on Zoiper 2, however
>>>> they plan to fix it in the next release of Zoiper 2.  They suggested for
>>>> the moment to disable the MWI Subscribe option by doing "Go to the
>>>> preferences -> select your sip account -> enable "show advanced options"
>>>> andchange the mwi subscribe drop down to "disabled"
>>>>
>>>> The strange thing with all this is that the same Zoiper client never
>>>> had a problem on version 1.0.head (git-e3698c2 2012-03-23 01-21-55 +0100)
>>>> but as soon as you upgraded to 1.2 or even the current git versions you
>>>> would find that FreeSWITCH and Zoiper would get into a SUBSCRIBE/NOTIFY
>>>> Loop.
>>>>
>>>> I hope this information is of some help to someone else in the future
>>>> and is able to ensure they do not have to go through the same nightmare I
>>>> did.
>>>>
>>>> If anyone has any suggestions on ways to limit the SUBSCRIBE/NOTIFY
>>>> responses so that you can avoid this happening in the future I would be
>>>> very interested to hear, even if you have any other feedback that might
>>>> have assisted in finding this issue quicker I would be very interested.
>>>>
>>>>
>>>>   Regards,
>>>>
>>>> *Ashley Breeden*
>>>> URL Networks Pty Ltd
>>>> *WWW:* http://www.url.net.au
>>>> *Twitter: *@urlnetworks
>>>>
>>>> *
>>>> *
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _________________________________________________________________________
>>>> Professional FreeSWITCH Consulting Services:
>>>> consulting at freeswitch.org
>>>> http://www.freeswitchsolutions.com
>>>>
>>>> 
>>>> 
>>>>
>>>> Official FreeSWITCH Sites
>>>> http://www.freeswitch.org
>>>> http://wiki.freeswitch.org
>>>> http://www.cluecon.com
>>>>
>>>> FreeSWITCH-users mailing list
>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> http://www.freeswitch.org
>>>>
>>>>
>>>
>>>
>>> --
>>> Anthony Minessale II
>>>
>>> FreeSWITCH http://www.freeswitch.org/
>>> ClueCon http://www.cluecon.com/
>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>
>>> AIM: anthm
>>> MSN:anthony_minessale at hotmail.com
>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>> IRC: irc.freenode.net #freeswitch
>>>
>>> FreeSWITCH Developer Conference
>>> sip:888 at conference.freeswitch.org
>>> googletalk:conf+888 at conference.freeswitch.org
>>> pstn:+19193869900
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org
>>> http://www.freeswitchsolutions.com
>>>
>>> 
>>> 
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://wiki.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>>
>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org
>>> http://www.freeswitchsolutions.com
>>>
>>> 
>>> 
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://wiki.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>>
>>
>>
>> --
>> Anthony Minessale II
>>
>> FreeSWITCH http://www.freeswitch.org/
>> ClueCon http://www.cluecon.com/
>> Twitter: http://twitter.com/FreeSWITCH_wire
>>
>> AIM: anthm
>> MSN:anthony_minessale at hotmail.com
>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>> IRC: irc.freenode.net #freeswitch
>>
>> FreeSWITCH Developer Conference
>> sip:888 at conference.freeswitch.org
>> googletalk:conf+888 at conference.freeswitch.org
>> pstn:+19193869900
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> 
>> 
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> 
> 
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20130913/79c95223/attachment-0001.html 