[Freeswitch-users] FreeSwitch - Performance issues

[Michael Jerris]

There are multiple known issues with Ubuntu 12.04.  We reccomend Debian 8
Jessie.

On Friday, August 14, 2015, Shaun Stokes <shaun.stokes at itec-support.co.uk>
wrote:

> Hi,
>
>
>
> We’re experiencing performance issues with FreeSwitch, our target is 500
> concurrent sessions, but at the moment this starts to bottleneck around 30.
>
>
>
> Host system:
>
> Windows Server 2012 Hyper-V Host
>
> AMD Opteron 4386 (2 processors) – 16 cores total
>
> 128GB DDR3
>
> 2TB RAID 5 (700MB/s tested read and write throughput)
>
>
>
> FreeSwitch Virtual Machine:
>
> FreeSwitch 1.4.15
>
> Ubuntu 12.04 LTS
>
> 16 Virtual cores (high priority)
>
> 2GB RAM (would assign more but FreeSwitch never seems to use much)
>
> 500GB HD (on VHDX)
>
>
>
> After around 30 concurrent sessions we begin to see CPU spikes almost
> every time a new call comes in, as the sessions increase the size and
> frequency of the CPU spikes also increase. The system seems to be able to
> sit comfortably with over 100 concurrent sessions and 80% idle CPU,
> providing we don’t have any new calls hitting the platform. The spikes are
> causing audio (RTP) to stutter or in some cases drop completely for a few
> seconds.
>
>
>
> The FreeSwitch spikes are occurring on all 16 cores, we have been
> monitoring the system using htop and mpstat.
>
>
>
> This is an example of when we receive an inbound call while we have 34
> concurrent sessions:
>
> 14:26:04     CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal
> %guest   %idle
>
> 14:26:05     all    1.67    0.00   27.34    0.00    0.25    0.00
> 0.00    0.00   70.74
>
> 14:26:05       0    1.00    0.00   28.00    0.00    0.00    0.00
> 0.00    0.00   71.00
>
> 14:26:05       1    0.99    0.00   26.73    0.00    0.00    0.00
> 0.00    0.00   72.28
>
> 14:26:05       2    0.99    0.00   26.73    0.00    0.00    0.00
> 0.00    0.00   72.28
>
> 14:26:05       3    3.92    0.00   26.47    0.00    0.00    0.00
> 0.00    0.00   69.61
>
> 14:26:05       4    0.99    0.00   25.74    0.00    1.98    0.00
> 0.00    0.00   71.29
>
> 14:26:05       5    0.99    0.00   26.73    0.00    0.99    0.00
> 0.00    0.00   71.29
>
> 14:26:05       6    0.00    0.00   26.47    0.00    0.98    0.00
> 0.00    0.00   72.55
>
> 14:26:05       7    0.00    0.00   26.73    0.00    0.00    0.00
>  0.00    0.00   73.27
>
> 14:26:05       8    3.96    0.00   27.72    0.00    0.00    0.00
> 0.00    0.00   68.32
>
> 14:26:05       9    2.00    0.00   27.00    0.00    0.00    0.00
> 0.00    0.00   71.00
>
> 14:26:05      10    6.00    0.00   32.00    0.00    0.00    0.00
> 0.00    0.00   62.00
>
> 14:26:05      11    1.96    0.00   27.45    0.00    0.00    0.00
> 0.00    0.00   70.59
>
> 14:26:05      12    0.99    0.00   26.73    0.00    0.00    0.00
> 0.00    0.00   72.28
>
> 14:26:05      13    1.00    0.00   28.00    0.00    0.00    0.00
> 0.00    0.00   71.00
>
> 14:26:05      14    0.00    0.00   27.00    0.00    0.00    0.00
> 0.00    0.00   73.00
>
>
>
> This is when the system is not receiving an inbound call but is sitting
> comfortably at 34 concurrent sessions:
>
> 14:25:57     CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal
> %guest   %idle
>
> 14:25:58     all    0.87    0.00    0.62    0.00    0.12    0.00
> 0.00    0.00   98.39
>
> 14:25:58       0    0.99    0.00    0.99    0.00    0.00    0.00
> 0.00    0.00   98.02
>
> 14:25:58       1    0.99    0.00    0.99    0.00    0.00    0.00
> 0.00    0.00   98.02
>
> 14:25:58       2    0.98    0.00    0.98    0.00    0.00    0.00
> 0.00    0.00   98.04
>
> 14:25:58       3    0.00    0.00    0.99    0.00    0.00    0.00
> 0.00    0.00   99.01
>
> 14:25:58       4    0.00    0.00    0.00    0.00    1.98    0.00
> 0.00    0.00   98.02
>
> 14:25:58       5    0.00    0.00    0.00    0.00    0.99    0.00
> 0.00    0.00   99.01
>
> 14:25:58       6    0.00    0.00    0.99    0.00    0.00    0.00
> 0.00    0.00   99.01
>
> 14:25:58       7    0.00    0.00    0.99    0.00    0.00    0.00
> 0.00    0.00   99.01
>
> 14:25:58       8    5.00    0.00    1.00    0.00    0.00    0.00
> 0.00    0.00   94.00
>
> 14:25:58       9    1.00    0.00    0.00    0.00    0.00    0.00
> 0.00    0.00   99.00
>
> 14:25:58      10    1.00    0.00    0.00    0.00    0.00    0.00
> 0.00    0.00   99.00
>
> 14:25:58      11    0.99    0.00    0.99    0.00    0.00    0.00
> 0.00    0.00   98.02
>
> 14:25:58      12    0.00    0.00    0.00    0.00    0.00    0.00
> 0.00    0.00  100.00
>
> 14:25:58      13    1.98    0.00    0.99    0.00    0.00    0.00
> 0.00    0.00   97.03
>
> 14:25:58      14    0.00    0.00    0.00    0.00    0.00    0.00
> 0.00    0.00  100.00
>
> 14:25:58      15    0.00    0.00    0.00    0.00    0.00    0.00
> 0.00    0.00  100.00
>
>
>
> We’re using FreeSwitch in multi-tenant mode, we have tried dedicating a
> single VM for one tenancy but still experience the issue. My assumption has
> been that this will be due to the dialplan, I’ve optimized this slightly by
> writing LUA scripts to handle some of the inbound calls this appears to
> take away some of the load but we’re still using the internal dialplans for
> each tenancy (tenancy dialplans have an average of 300 entries).
>
>
>
> We are using the following arguments when running FreeSwitch:
>
> -rp –nc –nonat
>
>
>
> I’ve seen another post which says we should operate FreeSwitch in High
> Performance mode using –hp but can’t find anything information about this,
> is this still a valid argument to use with FreeSwitch?
>
>
>
> Has anyone experienced similar performance issues before or have any
> advice?
>
>
>
> Hope someone may be able help.
>
>
>
> Thanks,
>
> Shaun
>
> ______________________________________________________________________
> This message has been checked for all known viruses by MessageLabs Virus
> Scanning Service.
> ______________________________________________________________________
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20150815/9e81098f/attachment.html