High load with low cpu utilization
Hello We see a high load on server with two 10 core HT processors. cat /proc/cpuinfo shows 40 processors. 1 minute load average is often grater than 70 when processing ~120 concurrent calls. CPU utilization shows 97%idle and 0.1%wa I checked also io stats and everything looks OK. Operating system Centos 7 uname -a Linux pbx-fs-drc.ib 3.10.0-229.7.2.el7.x86_64 #1 SMP Tue Jun 23 22:06:11 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux Below link to pastebin with gdb output gdb /usr/local/freeswitch/bin/freeswitch /usr/local/freeswitch/log/freeswitch.gcore.2699 set pagination off thread apply all bt https://pastebin.freeswitch.org/24487 We are planning to run much more concurrent calls but do not understand these stats. Why processes are waiting in queue and we do not see any bottleneck ? Please help
Load avg is often skewed in many CPU env from the scheduler and RT threads used in FS. Its better to trust the results of top -H `cat /usr/local/freeswitch/run/freeswitch.pid` and make audible calls to listen to the audio and make sure its working. Also, we do not really touch Centos7 other than to make packages for it, its only community-supported and we don't know the details of how its kernel is configured etc. On Thu, Dec 10, 2015 at 11:36 AM, Adam Kuśmirek <amkusmirek@gmail.com> wrote:
Hello
We see a high load on server with two 10 core HT processors. cat /proc/cpuinfo shows 40 processors.
1 minute load average is often grater than 70 when processing ~120 concurrent calls.
CPU utilization shows 97%idle and 0.1%wa I checked also io stats and everything looks OK.
Operating system Centos 7 uname -a Linux pbx-fs-drc.ib 3.10.0-229.7.2.el7.x86_64 #1 SMP Tue Jun 23 22:06:11 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
Below link to pastebin with gdb output
gdb /usr/local/freeswitch/bin/freeswitch /usr/local/freeswitch/log/freeswitch.gcore.2699 set pagination off thread apply all bt
https://pastebin.freeswitch.org/24487
We are planning to run much more concurrent calls but do not understand these stats. Why processes are waiting in queue and we do not see any bottleneck ?
Please help
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬ ☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>* ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA
participants (2)
-
Adam Kuśmirek
-
Anthony Minessale