[Freeswitch-users] FreeSWITCH 100% CPU utilization

Örn Arnarson orn at arnarson.net
Tue Apr 11 18:29:01 MSD 2017


Hello again,

It was on the old wiki that I saw it, but the same info is present on the
new wiki:
https://freeswitch.org/confluence/display/FREESWITCH/Performance+Testing+and+Configurations
Specifically, this part:

o libsofia only handles 1 thread per profile, so if that is your bottle
neck use more profiles

When one is not familiar with the internals of FreeSWITCH, as I'm not, one
might incorrectly reach the same conclusion as I did. Not sure how to
prevent that, though.

Just above that, it does mention the multi-threaded handling of FS where
each call leg has its own thread, so if one reads the whole page (which I
clearly didn't), this confusion is probably avoided.

Regards,
Örn

On Sun, Apr 9, 2017 at 1:50 AM, Anthony Minessale <
anthony.minessale at gmail.com> wrote:

> What can happen is that the incorrect info can also get archived and its
> hard to undo so there is a similar duty to debunk anything that may lead to
> confusion for those who read it in years to come.  Maybe supply the wiki
> page you saw that on and we can correct it.  Hopefully its on the old wiki.
>
> We can't do much to debug 1.2 so you are correct.  We will have to update
> to latest to get any clean idea of what you problem is.  (If it persists)
> Rest assured that a properly functioning FS will do its best to distribute
> the load across all CPU when it can.
>
>
> On Sat, Apr 8, 2017 at 8:03 PM, Örn Arnarson <orn at arnarson.net> wrote:
>
>> Hi Anthony,
>>
>> Thanks for your response.
>>
>> You make a fair point. I was making some assumptions.
>>
>> I wasn't really reporting an issue, though. At the time of the post I was
>> asking questions about ways to leverage more of the CPU cores due to a
>> misconception I had on how that was being done. Partially, the wording on
>> the wiki is what led me to that incorrect assumption.
>>
>> Subsequently, FreeSWITCH stopped running, for one reason or another, and
>> upon restarting, the issue was gone. I felt a duty to report that for the
>> archives, should someone else encounter a similar problem in the future.
>>
>> For the record/archives, both the OS and the FreeSWITCH versions are
>> ancient; Ubuntu 12.04.5 LTS, with FreeSWITCH version 1.2.14. They are both
>> earmarked for upgrades later this year.
>>
>> Sorry for wasting your guys' time.
>>
>> Regards,
>> Örn
>>
>> On Tue, Apr 4, 2017 at 12:02 AM, Anthony Minessale <
>> anthony.minessale at gmail.com> wrote:
>>
>>> You are making guesses and assumptions about the architecture and
>>> enabling FUD.
>>> You should report issues to JIRA and not here and you should supply
>>> pertinent info such as version of the OS and cpu etc.
>>>
>>>
>>>
>>> On Mon, Apr 3, 2017 at 5:21 PM, Brian West <brian at freeswitch.org> wrote:
>>>
>>>> Sure it's not getting attached? Or your DB?
>>>>
>>>> On Mon, Apr 3, 2017 at 12:02 PM Örn Arnarson <orn at arnarson.net> wrote:
>>>>
>>>>> Thanks again.
>>>>>
>>>>> I did another strace, which for some reason crashed my FreeSWITCH
>>>>> (whoops), but after the restart, I'm seeing around 10% CPU utilization on
>>>>> that thread.
>>>>>
>>>>> It seems that there was something odd going on with the main
>>>>> signalling thread.
>>>>>
>>>>> Regards,
>>>>> Örn
>>>>>
>>>>> On Mon, Apr 3, 2017 at 4:57 PM, Brian West <brian at freeswitch.org>
>>>>> wrote:
>>>>>
>>>>> RTP is in the Session thread and each session gets its own thread.
>>>>> Two per bridged call.
>>>>>
>>>>> On Mon, Apr 3, 2017 at 11:55 AM, Örn Arnarson <orn at arnarson.net>
>>>>> wrote:
>>>>>
>>>>> Thanks for your reply Brian.
>>>>>
>>>>> Does RTP only have a single thread? Is it identifiable somehow? The
>>>>> 100 (120%, really) thread is likely the RTP thread then. In fact, after an
>>>>> strace on the thread and its forks it seems to be the case.
>>>>>
>>>>> Presence is disabled and I'm receiving about 20 SIP messages per
>>>>> second combined on all profiles. Not quite getting the extensive processing
>>>>> time.
>>>>>
>>>>> Regards,
>>>>> Örn
>>>>>
>>>>> On Mon, Apr 3, 2017 at 4:49 PM, Brian West <brian at freeswitch.org>
>>>>> wrote:
>>>>>
>>>>> RTP already has its own thread,  The signaling thread in Sofia is the
>>>>> only bottle neck, Sounds like your presence or registration traffic is
>>>>> clogging up your system.
>>>>>
>>>>> /b
>>>>>
>>>>>
>>>>> On Mon, Apr 3, 2017 at 11:42 AM, Örn Arnarson <orn at arnarson.net>
>>>>> wrote:
>>>>>
>>>>> Hello FS users,
>>>>>
>>>>> I'm running a FS server with 8 CPU cores and around 3000 users, ~50
>>>>> concurrent calls.
>>>>> FS is proxying the media.
>>>>>
>>>>> I'm running into about a 1-1.5 second delay when forwarding SIP
>>>>> responses back to the UAC (200 OK for example).
>>>>>
>>>>> It seems that almost all of the processing is being done by one CPU
>>>>> core, while the other 7 sit idle.
>>>>>
>>>>> I found this on the old wiki:
>>>>> libsofia only handles 1 thread per profile, so if that is your bottle
>>>>> neck use more profiles
>>>>>
>>>>> There's no mention of this on the new one. Has this changed at all?
>>>>>
>>>>> In any case, I'm assuming that SIP and RTP co-exist on the thread,
>>>>> seeing as I don't have that many SIP requests per second, and yet it is
>>>>> taking more than 1 second to forward the requests.
>>>>>
>>>>> I guess my question is; Is it possible to offload the RTP handling off
>>>>> to different cores somehow? Or force FreeSWITCH to use more than one thread
>>>>> per sofia profile?
>>>>>
>>>>> Regards,
>>>>> Örn
>>>>>
>>>>> ____________________________________________________________
>>>>> _____________
>>>>> Professional FreeSWITCH Consulting Services:
>>>>> consulting at freeswitch.org
>>>>> http://www.freeswitchsolutions.com
>>>>>
>>>>> Official FreeSWITCH Sites
>>>>> http://www.freeswitch.org
>>>>> http://confluence.freeswitch.org
>>>>> http://www.cluecon.com
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/free
>>>>> switch-users
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> *Brian West*
>>>>> brian at freeswitch.org
>>>>>
>>>>> *Twitter: @FreeSWITCH , @briankwest*
>>>>>
>>>>> http://www.freeswitchbook.com
>>>>> http://www.freeswitchcookbook.com
>>>>>
>>>>> Allison prompts for FreeSWITCH:
>>>>>
>>>>> *https://www.gofundme.com/allison-prompts-for-freeswitch*
>>>>> <https://www.gofundme.com/allison-prompts-for-freeswitch>
>>>>>
>>>>> Got Bugs? Report them here <https://freeswitch.org/jira>! | Reddit:
>>>>> /r/freeswitch <https://www.reddit.com/r/freeswitch>
>>>>>
>>>>> *T:*+19184209001 <+1%20918-420-9001> | *F:*+19184209002
>>>>> <+1%20918-420-9002> | *M:*+1918424WEST (9378)
>>>>> *Skype:*briankwest
>>>>>
>>>>> ____________________________________________________________
>>>>> _____________
>>>>> Professional FreeSWITCH Consulting Services:
>>>>> consulting at freeswitch.org
>>>>> http://www.freeswitchsolutions.com
>>>>>
>>>>> Official FreeSWITCH Sites
>>>>> http://www.freeswitch.org
>>>>> http://confluence.freeswitch.org
>>>>> http://www.cluecon.com
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/free
>>>>> switch-users
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>>
>>>>> ____________________________________________________________
>>>>> _____________
>>>>> Professional FreeSWITCH Consulting Services:
>>>>> consulting at freeswitch.org
>>>>> http://www.freeswitchsolutions.com
>>>>>
>>>>> Official FreeSWITCH Sites
>>>>> http://www.freeswitch.org
>>>>> http://confluence.freeswitch.org
>>>>> http://www.cluecon.com
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/free
>>>>> switch-users
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> *Brian West*
>>>>> brian at freeswitch.org
>>>>>
>>>>> *Twitter: @FreeSWITCH , @briankwest*
>>>>>
>>>>> http://www.freeswitchbook.com
>>>>> http://www.freeswitchcookbook.com
>>>>>
>>>>> Allison prompts for FreeSWITCH:
>>>>>
>>>>> *https://www.gofundme.com/allison-prompts-for-freeswitch*
>>>>> <https://www.gofundme.com/allison-prompts-for-freeswitch>
>>>>>
>>>>> Got Bugs? Report them here <https://freeswitch.org/jira>! | Reddit:
>>>>> /r/freeswitch <https://www.reddit.com/r/freeswitch>
>>>>>
>>>>> *T:*+19184209001 <+1%20918-420-9001> | *F:*+19184209002
>>>>> <+1%20918-420-9002> | *M:*+1918424WEST (9378)
>>>>> *Skype:*briankwest
>>>>>
>>>>> ____________________________________________________________
>>>>> _____________
>>>>> Professional FreeSWITCH Consulting Services:
>>>>> consulting at freeswitch.org
>>>>> http://www.freeswitchsolutions.com
>>>>>
>>>>> Official FreeSWITCH Sites
>>>>> http://www.freeswitch.org
>>>>> http://confluence.freeswitch.org
>>>>> http://www.cluecon.com
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/free
>>>>> switch-users
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>> ____________________________________________________________
>>>>> _____________
>>>>> Professional FreeSWITCH Consulting Services:
>>>>> consulting at freeswitch.org
>>>>> http://www.freeswitchsolutions.com
>>>>>
>>>>> Official FreeSWITCH Sites
>>>>> http://www.freeswitch.org
>>>>> http://confluence.freeswitch.org
>>>>> http://www.cluecon.com
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/free
>>>>> switch-users
>>>>> http://www.freeswitch.org
>>>>
>>>> --
>>>>
>>>> *Brian West*
>>>> brian at freeswitch.org
>>>>
>>>> *Twitter: @FreeSWITCH , @briankwest*
>>>>
>>>> http://www.freeswitchbook.com
>>>> http://www.freeswitchcookbook.com
>>>>
>>>> Allison prompts for FreeSWITCH:
>>>>
>>>> *https://www.gofundme.com/allison-prompts-for-freeswitch*
>>>> <https://www.gofundme.com/allison-prompts-for-freeswitch>
>>>>
>>>> Got Bugs? Report them here <https://freeswitch.org/jira>! | Reddit:
>>>> /r/freeswitch <https://www.reddit.com/r/freeswitch>
>>>>
>>>> *T:*+19184209001 <(918)%20420-9001> | *F:*+19184209002
>>>> <(918)%20420-9002> | *M:*+1918424WEST (9378)
>>>> *Skype:*briankwest
>>>>
>>>> ____________________________________________________________
>>>> _____________
>>>> Professional FreeSWITCH Consulting Services:
>>>> consulting at freeswitch.org
>>>> http://www.freeswitchsolutions.com
>>>>
>>>> Official FreeSWITCH Sites
>>>> http://www.freeswitch.org
>>>> http://confluence.freeswitch.org
>>>> http://www.cluecon.com
>>>>
>>>> FreeSWITCH-users mailing list
>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/free
>>>> switch-users
>>>> http://www.freeswitch.org
>>>>
>>>
>>>
>>>
>>> --
>>> Anthony Minessale II       ♬ @anthmfs  ♬ @FreeSWITCH  ♬
>>>
>>>http://freeswitch.org/http://cluecon.com/>>> http://twitter.com/FreeSWITCH
>>> ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+
>>> <http://freeswitch.org/g+>*
>>>
>>> ClueCon Weekly Development Call
>>> ☎ sip:888 at conference.freeswitch.org  ☎ +19193869900 <+1%20919-386-9900>
>>>
>>> https://www.youtube.com/watch?v=9XXgW34t40s
>>> https://www.youtube.com/watch?v=NLaDpGQuZDA
>>>
>>> ____________________________________________________________
>>> _____________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org
>>> http://www.freeswitchsolutions.com
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://confluence.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://confluence.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>
>
>
> --
> Anthony Minessale II       ♬ @anthmfs  ♬ @FreeSWITCH  ♬
>
>http://freeswitch.org/http://cluecon.com/> http://twitter.com/FreeSWITCH
> ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+
> <http://freeswitch.org/g+>*
>
> ClueCon Weekly Development Call
> ☎ sip:888 at conference.freeswitch.org  ☎ +19193869900 <+1%20919-386-9900>
>
> https://www.youtube.com/watch?v=9XXgW34t40s
> https://www.youtube.com/watch?v=NLaDpGQuZDA
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://confluence.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20170411/230c7244/attachment-0001.html 


Join us at ClueCon 2016 Aug 8-12, 2016
More information about the FreeSWITCH-users mailing list