[Freeswitch-users] Freeswitch 1.4.15 and a sudden jump in CPU usage

Wed Feb 11 23:54:39 MSK 2015

Thanks Tristan.

I have had rtp-timer-name set to none for a long time but I still see the
sudden and random jumps in CPU.

I haven't been able to reproduce it in our Dev/QA environment as well. If I
can reproduce it, I will definitely do the logs as well.

On Wed, Feb 11, 2015 at 3:31 PM, Tristan Mahé <t.mahe at b-and-c.net> wrote:

>  Hi Ali,
>
> What I see is kinda the same thing, I still need to provide Anthony with
> the debug logs ( still had no chance to do the requested tests on the
> production, could be related to some of our users UA/callflow and cannot
> reproduce the issue in a lab ). The interesting part as discussed in
> FS-7066 is the timer change is correcting the trouble, and without the
> timer change, we eventually experience blocking of sofia profile ( seems
> related to media playback ).
>
> I hope to have a chance soon to be able to furnish the debug logs, if you
> can, do the same, Anthony gave the steps to provide him some usefull logs
> on the bug comments.
>
>
> Le 11/02/2015 10:58, Ali Pey a écrit :
>
> Hi Tristan,
>
>  Do you see a sudden jump or just more CPU usage?
>
>  In my case, everything is good for a few days with less than %10 CPU
> usage, then there is a sudden jump to ~%98 CPU usage and it doesn't come
> down until a restart. After restart, it's good again with the same load.
>
>  Thanks,
> Ali Pey
>
>
> On Wed, Feb 11, 2015 at 1:40 PM, Tristan Mahé <t.mahe at b-and-c.net> wrote:
>
>> Maybe related to https://freeswitch.org/jira/browse/FS-7066 ?
>>
>> Le 11/02/2015 08:00, Giovanni Maruzzelli a écrit :
>>  > somebody has tested if it's a regression, eg: is 1.4.14 having
>> > same symptoms?
>> >
>> > -giovanni
>> >
>> > On Wed, Feb 11, 2015 at 4:57 PM, Moishe Grunstein <max at nysolutions.com>
>> wrote:
>> >> Why not a Jira?
>> >>
>> >> Thanks,
>> >>
>> >> Moishe Grunstein
>> >> Tornado Computer Systems, Inc.
>> >> 212.400.7650 888.IPPBX.US
>> >> Service Request Email: support at nysolutions.com
>> >>
>> >> Computer Networking * Managed Services * IP Video Surveillance *
>> Network Assessments * Web Solutions * Voice over IP * Disaster Recovery *
>> Network Security * Site Surveys * CMS
>> >>
>> >> -----Original Message-----
>> >> From: freeswitch-users-bounces at lists.freeswitch.org [mailto:
>> freeswitch-users-bounces at lists.freeswitch.org] On Behalf Of ??
>> >> Sent: Wednesday, February 11, 2015 10:49 AM
>> >> To: FreeSWITCH Users Help
>> >> Subject: Re: [Freeswitch-users] Freeswitch 1.4.15 and a sudden jump in
>> CPU usage
>> >>
>> >> i think it's bug of freeswitch:
>> >>
>> >> -- the Freeswitch using more than 1000% CPU, and there is no other
>> process eating CPU
>> >> -- we move all SIP traffic from the server away, the cpu is low. and
>> after sevral hours, if we move the SIP traffic back again, the FS eating
>> all CPU again. the only way to recovery is to restart the FS.
>> >> -- there is no DB such as mysql on the server, and we have add "nosql"
>> >> option, no help
>> >>
>> >> On Wed, Feb 11, 2015 at 10:03 PM, John Nash <john.nash778 at gmail.com>
>> wrote:
>> >>> Dear Muhammad,
>> >>>
>> >>> I think your issue is different than Ali Pey because CPU rises at a
>> >>> specific time (approx) while running fine most of the time. In my
>> >>> experience you should check following ..
>> >>> - Check using top commands if its indeed freeswitch taking up CPU. It
>> >>> may be logrotate or something.
>> >>> - If you are using DB on same server, also check DB connections, May
>> >>> be some long running query causing it (Like someone fires some billing
>> >>> report at approx same time)
>> >>>
>> >>> On the other hand when you say "it runs fine all day". You may want to
>> >>> check that using top command at different time of day, may be CPU
>> >>> usage keep on increasing and in your peak time it increases so much
>> that you see symptoms.
>> >>> (If this is true may be you and Ali pey have same issue perhaps some
>> >>> CPU leak).
>> >>>
>> >>> P.S. All points I mentioned are from my VOIP experiences in general. I
>> >>> am NOT freeswitch expert. I have recently started experiments with
>> >>> freeswitch so people who are using freeswitch or experts will give
>> >>> more meaningful insight.
>> >>>
>> >>>
>> >>>
>> >>> On Wed, Feb 11, 2015 at 7:16 PM, 彭勇 <ppyy at pubyun.com> wrote:
>> >>>> we have add this option to grub and reboot one months ago. no help.
>> >>>>
>> >>>> we have media-pass-through  option, and have more than 10 thousands
>> >>>> sessions. it works fine all day and the CPU will rise suddenly about
>> >>>> 9:00 am every day.  i should restart the FS to recovery.
>> >>>>
>> >>>> On Wed, Feb 11, 2015 at 9:25 PM, Muhammad Naseer Bhatti
>> >>>> <nbhatti at gmail.com> wrote:
>> >>>>> Try nohz=off reboot and see if that helps.
>> >>>>>
>> >>>>> Sent from my iPad
>> >>>>>
>> >>>>> On Feb 11, 2015, at 16:18, Aqs Younas <aqsyounas at gmail.com> wrote:
>> >>>>>
>> >>>>> I am facing same issue. Freeswitch taking more than 100% cpu usage
>> >>>>> in htop even on 200 concurrent calls. For 1000 concurrent calls i
>> >>>>> have to install 3 freeswitch instances due to this much cpu usage
>> >>>>> taking by a single freeswitch node. My Dist is Debian 7.
>> >>>>>
>> >>>>> On 11 February 2015 at 17:35, Muhammad Naseer Bhatti
>> >>>>> <nbhatti at gmail.com>
>> >>>>> wrote:
>> >>>>>>
>> >>>>>> It looks like this is the same tickless kernel issue. I am using
>> >>>>>> Debian
>> >>>>>> 7
>> >>>>>> and the CPU usage shows very high. Adding nohz=off to grub options
>> >>>>>> does makes it lower. I am gathering more information in order to
>> >>>>>> file a Jira.
>> >>>>>> Debian 6 on the other hand works very quietly on the same
>> >>>>>> workload. We are talking about 200+% CPU utilization on Debian 7
>> >>>>>> and around 50% on Debian 6 showing in htop. I know so many other
>> >>>>>> factors also involved but I am still digging out more.
>> >>>>>>
>> >>>>>> —
>> >>>>>> Thanks,
>> >>>>>> Muhammad Naseer Bhatti
>> >>>>>>
>> >>>>>> From: 彭勇 <ppyy at pubyun.com>
>> >>>>>> Reply: FreeSWITCH Users Help
>> >>>>>> <freeswitch-users at lists.freeswitch.org>>
>> >>>>>> Date: February 11, 2015 at 3:20:46 PM
>> >>>>>> To: FreeSWITCH Users Help <freeswitch-users at lists.freeswitch.org>>
>> >>>>>> Subject:  Re: [Freeswitch-users] Freeswitch 1.4.15 and a sudden
>> >>>>>> jump in CPU usage
>> >>>>>>
>> >>>>>> same issue.
>> >>>>>>
>> >>>>>> I have several FS 1.4.15 running on CentOS 6.5 x86_64, the CPU
>> >>>>>> will be very high about 9:00 am every morning. there is no crontab
>> >>>>>> at this time.
>> >>>>>>
>> >>>>>> i will downgrade to 1.4.14 to see if there is such issue.
>> >>>>>>
>> >>>>>> On Wed, Feb 11, 2015 at 1:14 AM, Ali Pey <alipey at gmail.com> wrote:
>> >>>>>>> Hi Anthony,
>> >>>>>>>
>> >>>>>>> I do not know if I am hitting a bug and if this is a bug, I
>> >>>>>>> don't have much info. I'm using this mailing list to see if
>> >>>>>>> anyone else is experiencing this problem and if there is a
>> >>>>>>> configuration change on Linux or Freeswitch that can fix this
>> >>>>>>> problem. Jira would be to report bugs when you know there is a
>> >>>>>>> bug. My problem could be (very likely) an environment related
>> issue.
>> >>>>>>>
>> >>>>>>> Thanks.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Tue, Feb 10, 2015 at 11:55 AM, Anthony Minessale
>> >>>>>>> <anthony.minessale at gmail.com> wrote:
>> >>>>>>>> Also you will have better luck using JIRA. The mailing list is
>> >>>>>>>> expressly NOT for reporting issues.
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On Tue, Feb 10, 2015 at 10:51 AM, Michael Jerris
>> >>>>>>>> <mike at jerris.com>
>> >>>>>>>> wrote:
>> >>>>>>>>> you might want to use gcore to look at the threads and see
>> >>>>>>>>> what is different when it in this state and not.
>> >>>>>>>>>
>> >>>>>>>>>> On Feb 10, 2015, at 11:28 AM, Ali Pey <alipey at gmail.com>
>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>> Hello,
>> >>>>>>>>>>
>> >>>>>>>>>> Normally our CPU usage during peak hours are less than %10;
>> >>>>>>>>>> however, time to time, freeswitch uses more than %95 of the
>> >>>>>>>>>> CPU and it stays there forever. The only way to get it back
>> >>>>>>>>>> to normal is to restart freeswitch service. During the high
>> >>>>>>>>>> CPU usage, freeswitch continues to operate but very very
>> >>>>>>>>>> slow.
>> >>>>>>>>>>
>> >>>>>>>>>> If I lower the traffic quite a bit, the CPU comes down, but
>> >>>>>>>>>> adding back the traffic, the CPU usage goes right back up.
>> >>>>>>>>>> After freeswitch restart, it becomes normal again with the
>> >>>>>>>>>> same traffic.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> Any ideas or suggestions?
>> >>>>>>>>>>
>> >>>>>>>>>> Much appreciated.
>> >>>>>>>>>>
>> >>>>>>>>>> Thanks,
>> >>>>>>>>>> Ali Pey
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> ______________________________________________________________
>> >>>>>>>>> ___________ Professional FreeSWITCH Consulting Services:
>> >>>>>>>>> consulting at freeswitch.org
>> >>>>>>>>> http://www.freeswitchsolutions.com
>> >>>>>>>>>
>> >>>>>>>>> Official FreeSWITCH Sites
>> >>>>>>>>> http://www.freeswitch.org
>> >>>>>>>>> http://confluence.freeswitch.org http://www.cluecon.com
>> >>>>>>>>>
>> >>>>>>>>> FreeSWITCH-users mailing list
>> >>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>> >>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freesw
>> >>>>>>>>> itch-users
>> >>>>>>>>> http://www.freeswitch.org
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> --
>> >>>>>>>> Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
>> >>>>>>>>
>> >>>>>>>> ☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞
>> >>>>>>>> http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞
>> >>>>>>>> http://freeswitch.org/g+
>> >>>>>>>>
>> >>>>>>>> ClueCon Weekly Development Call ☎
>> >>>>>>>> sip:888 at conference.freeswitch.org ☎ +19193869900
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> _______________________________________________________________
>> >>>>>>>> __________ Professional FreeSWITCH Consulting Services:
>> >>>>>>>> consulting at freeswitch.org
>> >>>>>>>> http://www.freeswitchsolutions.com
>> >>>>>>>>
>> >>>>>>>> Official FreeSWITCH Sites
>> >>>>>>>> http://www.freeswitch.org
>> >>>>>>>> http://confluence.freeswitch.org http://www.cluecon.com
>> >>>>>>>>
>> >>>>>>>> FreeSWITCH-users mailing list
>> >>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>> >>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswi
>> >>>>>>>> tch-users
>> >>>>>>>> http://www.freeswitch.org
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> ________________________________________________________________
>> >>>>>>> _________ Professional FreeSWITCH Consulting Services:
>> >>>>>>> consulting at freeswitch.org
>> >>>>>>> http://www.freeswitchsolutions.com
>> >>>>>>>
>> >>>>>>> Official FreeSWITCH Sites
>> >>>>>>> http://www.freeswitch.org
>> >>>>>>> http://confluence.freeswitch.org http://www.cluecon.com
>> >>>>>>>
>> >>>>>>> FreeSWITCH-users mailing list
>> >>>>>>> FreeSWITCH-users at lists.freeswitch.org
>> >>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> >>>>>>>
>> >>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswit
>> >>>>>>> ch-users
>> >>>>>>> http://www.freeswitch.org
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> Peng Yong
>> >>>>>>
>> >>>>>>
>> >>>>>> __________________________________________________________________
>> >>>>>> _______ Professional FreeSWITCH Consulting Services:
>> >>>>>> consulting at freeswitch.org
>> >>>>>> http://www.freeswitchsolutions.com
>> >>>>>>
>> >>>>>> Official FreeSWITCH Sites
>> >>>>>> http://www.freeswitch.org
>> >>>>>> http://confluence.freeswitch.org
>> >>>>>> http://www.cluecon.com
>> >>>>>>
>> >>>>>> FreeSWITCH-users mailing list
>> >>>>>> FreeSWITCH-users at lists.freeswitch.org
>> >>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> >>>>>>
>> >>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch
>> >>>>>> -users
>> >>>>>> http://www.freeswitch.org
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> __________________________________________________________________
>> >>>>>> _______ Professional FreeSWITCH Consulting Services:
>> >>>>>> consulting at freeswitch.org
>> >>>>>> http://www.freeswitchsolutions.com
>> >>>>>>
>> >>>>>> Official FreeSWITCH Sites
>> >>>>>> http://www.freeswitch.org
>> >>>>>> http://confluence.freeswitch.org
>> >>>>>> http://www.cluecon.com
>> >>>>>>
>> >>>>>> FreeSWITCH-users mailing list
>> >>>>>> FreeSWITCH-users at lists.freeswitch.org
>> >>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> >>>>>>
>> >>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch
>> >>>>>> -users
>> >>>>>> http://www.freeswitch.org
>> >>>>>
>> >>>>>
>> >>>>> ___________________________________________________________________
>> >>>>> ______ Professional FreeSWITCH Consulting Services:
>> >>>>> consulting at freeswitch.org
>> >>>>> http://www.freeswitchsolutions.com
>> >>>>>
>> >>>>> Official FreeSWITCH Sites
>> >>>>> http://www.freeswitch.org
>> >>>>> http://confluence.freeswitch.org
>> >>>>> http://www.cluecon.com
>> >>>>>
>> >>>>> FreeSWITCH-users mailing list
>> >>>>> FreeSWITCH-users at lists.freeswitch.org
>> >>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> >>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-
>> >>>>> users
>> >>>>> http://www.freeswitch.org
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> ___________________________________________________________________
>> >>>>> ______ Professional FreeSWITCH Consulting Services:
>> >>>>> consulting at freeswitch.org
>> >>>>> http://www.freeswitchsolutions.com
>> >>>>>
>> >>>>> Official FreeSWITCH Sites
>> >>>>> http://www.freeswitch.org
>> >>>>> http://confluence.freeswitch.org
>> >>>>> http://www.cluecon.com
>> >>>>>
>> >>>>> FreeSWITCH-users mailing list
>> >>>>> FreeSWITCH-users at lists.freeswitch.org
>> >>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> >>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-
>> >>>>> users
>> >>>>> http://www.freeswitch.org
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Peng Yong
>> >>>>
>> >>>> _____________________________________________________________________
>> >>>> ____ Professional FreeSWITCH Consulting Services:
>> >>>> consulting at freeswitch.org
>> >>>> http://www.freeswitchsolutions.com
>> >>>>
>> >>>> Official FreeSWITCH Sites
>> >>>> http://www.freeswitch.org
>> >>>> http://confluence.freeswitch.org
>> >>>> http://www.cluecon.com
>> >>>>
>> >>>> FreeSWITCH-users mailing list
>> >>>> FreeSWITCH-users at lists.freeswitch.org
>> >>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> >>>> UNSUBSCRIBE:
>> http://lists.freeswitch.org/mailman/options/freeswitch-us
>> >>>> ers
>> >>>> http://www.freeswitch.org
>> >>>
>> >>>
>> >>> ______________________________________________________________________
>> >>> ___ Professional FreeSWITCH Consulting Services:
>> >>> consulting at freeswitch.org
>> >>> http://www.freeswitchsolutions.com
>> >>>
>> >>> Official FreeSWITCH Sites
>> >>> http://www.freeswitch.org
>> >>> http://confluence.freeswitch.org
>> >>> http://www.cluecon.com
>> >>>
>> >>> FreeSWITCH-users mailing list
>> >>> FreeSWITCH-users at lists.freeswitch.org
>> >>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> >>> UNSUBSCRIBE:
>> http://lists.freeswitch.org/mailman/options/freeswitch-use
>> >>> rs
>> >>> http://www.freeswitch.org
>> >>
>> >>
>> >> --
>> >> Peng Yong
>> >>
>> >>
>> _________________________________________________________________________
>> >> Professional FreeSWITCH Consulting Services:
>> >> consulting at freeswitch.org
>> >> http://www.freeswitchsolutions.com
>> >>
>> >> Official FreeSWITCH Sites
>> >> http://www.freeswitch.org
>> >> http://confluence.freeswitch.org
>> >> http://www.cluecon.com
>> >>
>> >> FreeSWITCH-users mailing list
>> >> FreeSWITCH-users at lists.freeswitch.org
>> >> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> >> UNSUBSCRIBE:
>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>> >> http://www.freeswitch.org
>> >>
>> _________________________________________________________________________
>> >> Professional FreeSWITCH Consulting Services:
>> >> consulting at freeswitch.org
>> >> http://www.freeswitchsolutions.com
>> >>
>> >> Official FreeSWITCH Sites
>> >> http://www.freeswitch.org
>> >> http://confluence.freeswitch.org
>> >> http://www.cluecon.com
>> >>
>> >> FreeSWITCH-users mailing list
>> >> FreeSWITCH-users at lists.freeswitch.org
>> >> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> >> UNSUBSCRIBE:
>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>> >> http://www.freeswitch.org
>> >
>> >
>>
>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://confluence.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>
>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services: consulting at freeswitch.orghttp://www.freeswitchsolutions.com
>
> Official FreeSWITCH Siteshttp://www.freeswitch.orghttp://confluence.freeswitch.orghttp://www.cluecon.com
>
> FreeSWITCH-users mailing listFreeSWITCH-users at lists.freeswitch.orghttp://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-usershttp://www.freeswitch.org
>
>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://confluence.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20150211/d3837aa6/attachment-0001.html 