Also keep in mind that since FS is highly threaded its designed to take advantage of more cores... ie: a 8 core 2.3ghz box should easily out perform a 2core 3ghz box... This isnt the case with all software... Most software due to legacy coding restrictions more cores help a fair bit, but not as much as increasing clock speeds due certain parts of the software requiring single threaded (or atleast 1 thread at a time) accessing key linked lists which drags down over all system performance.... K On 2/13/12 10:52 AM, "Tim St. Pierre" <fs-list@communicatefreely.net> wrote:
Thanks Ken,
I did wonder if it was due to the thread count. If I can safely ignore it, then maybe I shouldn't worry until the CPU usage is higher. We had some spotty service earlier this week that got me scared - phones taking a long time to respond, etc. I think that was on account of the database being choked though.
Thanks!
-Tim
Ken Rice wrote:
How is that working hard? That thing is only using about 4% of the cpu
Most programs arent as heavily threaded as FS so the load avg never really gets that high... You can safely ignore it... The real thing to pay attention to is how much idle time you have on the cpu... And w0w its 93.6% on both cores.... That should tell you something
On 2/13/12 10:32 AM, "Tim St. Pierre" <fs-list@communicatefreely.net> wrote:
Hello,
I'm having some scaling problems with our Freeswitch system. The machine running Freeswitch seems to be working very hard during the day, and we don't have that big a load. I need to be able to double our subscriber base, and I don't really want to go to a cluster just yet. Here's what top -CP displays during busy hour:
tim@stefan%top -PC last pid: 75968; load averages: 4.64, 2.92, 1.74 up 10+08:54:14 11:17:25 129 processes: 1 running, 128 sleeping CPU 0: 4.5% user, 0.0% nice, 1.9% system, 0.0% interrupt, 93.6% idle CPU 1: 3.8% user, 0.0% nice, 2.6% system, 0.0% interrupt, 93.6% idle Mem: 632M Active, 2093M Inact, 322M Wired, 83M Cache, 384M Buf, 785M Free Swap: 4096M Total, 36K Used, 4096M Free
PID USERNAME THR PRI NICE SIZE RES STATE C TIME CPU COMMAND 58521 freeswitch 98 74 r30 749M 648M ucond 0 528:54 0.10% freeswitch 983 root 1 44 0 10824K 2272K select 1 0:40 0.00% ntpd
As you can see, the load average is pretty high, at least for something not that big. FS often has up to about 110 threads going. At night, this will settle right down to a 0.05 load average. CPU load never gets that high, but the machine still seems busy.
Here's some other details:
40 channels, nearly all of them in CS_EXECUTE or CS_EXCHANGE_MEDIA We see only a few CPS, maybe 10 if it's really busy, but most calls are longer. Nearly all our customers are regular office users.
There are 340 phones registered. About 300 of those are behind NAT getting an options-ping every 30s. They re-register every 5 minutes, so we see around 1 registration attempt per second, although it can be a little bursty.
Each phone has at least 5 BLF subscriptions, There are 1156 rows in the sip_subscriptions table.
All our trunks are SIP. Trunk side is g.711u and endpoints can do either g.711u or g.722
We use ODBC in the core - as well as for sofia, voicemail, and anything else that can use it. Directory and dialplan are done using CURL and handled by another server. The database is also on another server.
Our MySQL system sits at about 100 Queries per second during the day , down to about 80 qps at night. This system started to choke until I changed a lot of the tables to the MEMORY engine and installed SSDs as the primary storage devices. Now the database is pretty happy - about 0.5 load average at the moment. It was up to 2 with 60% CPU going to mysqld before I went to memory tables.
I keep hearing about FS systems handling thousands of users on one box. I'm nowhere near that and it seems to be maxed out.
This system has a dual core Xeon 5100 at 2.6 GHz and 4GB of ram. Memory doesn't seem to be the issue, as there is still some unused. Operating system is FreeBSD 8.0-RELEASE (amd64) Freeswitch is Version 1.0.head (git-7531fed 2011-08-17 11-27-20 -0500) I'm trying to update, but newer version break functionality, so I have to find a release that work before I can upgrade.
What should I look for to remedy this, or is this considered normal and I just need more power? I could easily upgrade this box and install a pair of quad core chips.
Any other information that would be helpful?
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org