<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Hi Jay,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Thanks for the suggestion.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">We’re actually using PostgreSQL, the file system is a 2TB RAID5 with 10x 278GB SAS 10K RPM disks, read is 1GB/s (23,000 IOPS) write
is 500MB/s (15,000 IOPS) but we’re only using a fraction of this.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Shaun<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> freeswitch-users-bounces@lists.freeswitch.org [mailto:freeswitch-users-bounces@lists.freeswitch.org]
<b>On Behalf Of </b>jay binks<br>
<b>Sent:</b> 02 September 2015 11:37<br>
<b>To:</b> FreeSWITCH Users Help <freeswitch-users@lists.freeswitch.org><br>
<b>Subject:</b> Re: [Freeswitch-users] FreeSwitch - Performance issues<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">I may have missed it in your previous emails, but im assuming you have the core DB on a tempfs<br>
or somewhere other than the default location on spinning rust ?<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Jay<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On 2 September 2015 at 18:53, Shaun Stokes <<a href="mailto:shaun.stokes@itec-support.co.uk" target="_blank">shaun.stokes@itec-support.co.uk</a>> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hi Brian,<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I’m a huge fan of virtualization, but in our environment it simply doesn’t provide the performance we’re looking for. The tests carried out were 10 second calls which were routed
to a Call Centre Queue with one registered SIP extension as a member.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">There was a significant improvement when we tested on an Amazon VM in comparison to Hyper-V, but we’d like a system which can handle large volumes of call traffic, based on our
tests this will need to be physical hardware to achieve the best performance.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Further to this, we have managed to resolve our initial problem with CPU spikes on Hyper-V following these steps:<o:p></o:p></p>
<p><span style="font-family:Symbol">·</span><span style="font-size:7.0pt">
</span>Disable VMQ on all physical and virtual adapters on the Hyper-V host which are linked to the FreeSwitch VM<o:p></o:p></p>
<p><span style="font-family:Symbol">·</span><span style="font-size:7.0pt">
</span>Reduce the virtual cores on all VMs to at least half of the available physical cores<o:p></o:p></p>
<p><span style="font-family:Symbol">·</span><span style="font-size:7.0pt">
</span>Increase the Processor Relative weight on the FreeSwitch VMs to 200 (default 100)<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thanks,<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Shaun<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">
<a href="mailto:freeswitch-users-bounces@lists.freeswitch.org" target="_blank">freeswitch-users-bounces@lists.freeswitch.org</a> [mailto:<a href="mailto:freeswitch-users-bounces@lists.freeswitch.org" target="_blank">freeswitch-users-bounces@lists.freeswitch.org</a>]
<b>On Behalf Of </b>Brian West<br>
<b>Sent:</b> 01 September 2015 21:21</span><o:p></o:p></p>
<div>
<div>
<p class="MsoNormal"><br>
<b>To:</b> FreeSWITCH Users Help <<a href="mailto:freeswitch-users@lists.freeswitch.org" target="_blank">freeswitch-users@lists.freeswitch.org</a>><br>
<b>Subject:</b> Re: [Freeswitch-users] FreeSwitch - Performance issues<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Shaun,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">800 concurrent with 400cps, expectations are a little out of touch with reality and your test is no where near real world. Your call duration is ~2000ms, little to no time to actually
do anything useful. You're burning the bulk of your CPU time on signaling and doing no real work with the session.<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">So yes, you'll always get the best results out of bare metal, but you can also get good performance on a VM (*)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">(*) This is in no way an endorsement of running FreeSWITCH in a VM, Your mileage may vary depending on your specific application and load requirements.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thanks,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Tue, Sep 1, 2015 at 3:08 PM, Shaun Stokes <<a href="mailto:shaun.stokes@itec-support.co.uk" target="_blank">shaun.stokes@itec-support.co.uk</a>> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Update<br>
<br>
We've performed various tests and have come to the conclusion as advised by others, that visualization is causing bottlenecks.<br>
<br>
Using Amazon EC2 on an 8 core high performance virtual server this maxed out at around 800 concurrent sessions, 400 sessions per second.<br>
<br>
Using a Dell 1950 8 core server this was able to handle 1000 concurrent sessions, 400 sessions per second and still have plenty of resource available. Average CPU was fluctuating between 30-40% under peak load.<br>
<br>
Conclusion, if you want to run a high performance FreeSwitch server in a production you should stick with physical hardware and avoid visualization where possible.<br>
________________________________________<br>
From: <a href="mailto:freeswitch-users-bounces@lists.freeswitch.org" target="_blank">
freeswitch-users-bounces@lists.freeswitch.org</a> [<a href="mailto:freeswitch-users-bounces@lists.freeswitch.org" target="_blank">freeswitch-users-bounces@lists.freeswitch.org</a>] on behalf of Shaun Stokes [<a href="mailto:shaun.stokes@itec-support.co.uk" target="_blank">shaun.stokes@itec-support.co.uk</a>]<br>
Sent: 28 August 2015 15:37<br>
To: 'FreeSWITCH Users Help'<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Subject: Re: [Freeswitch-users] FreeSwitch - Performance issues<br>
<br>
Hi Nathan,<br>
<br>
Thanks for the suggestion, we still see the CPU spikes when is under load after reducing to 8 cores but I think there may actually be an improvement in the audio quality.<br>
<br>
It's still not what we're expecting, so we will have to test our build on physical hardware too.<br>
<br>
Thanks,<br>
Shaun<br>
<br>
-----Original Message-----<br>
From: <a href="mailto:freeswitch-users-bounces@lists.freeswitch.org" target="_blank">
freeswitch-users-bounces@lists.freeswitch.org</a> [mailto:<a href="mailto:freeswitch-users-bounces@lists.freeswitch.org" target="_blank">freeswitch-users-bounces@lists.freeswitch.org</a>] On Behalf Of Nathan Neulinger<br>
Sent: 28 August 2015 15:04<br>
To: FreeSWITCH Users Help <<a href="mailto:freeswitch-users@lists.freeswitch.org" target="_blank">freeswitch-users@lists.freeswitch.org</a>><br>
Subject: Re: [Freeswitch-users] FreeSwitch - Performance issues<br>
<br>
Yes, I know you said Hyper-V, I don't know if it has the same sort of scheduler behavior or not, just mentioned the vmware for context.<br>
<br>
On 08/28/2015 08:46 AM, Nathan Neulinger wrote:<br>
> One thing that I see jump out - you've probably got too many virtual<br>
> cores defined on that VM, particularly since you've assigned # vcpu the same as the number of real cores on the host system.<br>
><br>
> Try your test again dropping that to 8 virtual CPUs and I think you<br>
> might see better results. (Or at least drop to less than 100% of the<br>
> total cores on the hardware.)<br>
><br>
> Particularly if this host system has other vms running on it, you're<br>
> hurting the vm scheduler's ability to hand out resources. With vmware,<br>
> I believe a simple rule of thumb is that scheduler requires N-1 CPUs<br>
> to be locked to schedule execution of an N cpu virtual machine - so even if it doesn't need all of the CPUs, it has to reserve all of them any time you have more than a 2 vCPU vm.<br>
><br>
> -- Nathan<br>
><br>
> On 08/28/2015 08:18 AM, Michael Jerris wrote:<br>
>> The numbers your posting are fairly low, I'm guessing this is a<br>
>> grossly underpowered vm that is causing your issues<br>
>><br>
>> On Friday, August 28, 2015, Shaun Stokes <<a href="mailto:shaun.stokes@itec-support.co.uk" target="_blank">shaun.stokes@itec-support.co.uk</a> <mailto:<a href="mailto:shaun.stokes@itec-support.co.uk" target="_blank">shaun.stokes@itec-support.co.uk</a>>>
wrote:<br>
>><br>
>> Hi Peter,____<br>
>><br>
>> __ __<br>
>><br>
>> Thanks for the advice, this is something we're already looking into but we don't have the new hardware available<br>
>> yet.____<br>
>><br>
>> __ __<br>
>><br>
>> Debian 8 has great integration with Hyper-V on 2012R2 and operates as a Generation 2 VM, obviously it's never going<br>
>> to be as good as running directly on the hardware but we were<br>
>> hoping for better performance.____<br>
>><br>
>> __ __<br>
>><br>
>> Thanks,____<br>
>><br>
>> Shaun____<br>
>><br>
>> __ __<br>
>><br>
>> *From:*<a href="mailto:freeswitch-users-bounces@lists.freeswitch.org" target="_blank">freeswitch-users-bounces@lists.freeswitch.org</a><br>
>> <javascript:_e(%7B%7D,'cvml','<a href="mailto:freeswitch-users-bounces@lists.freeswitch.org" target="_blank">freeswitch-users-bounces@lists.freeswitch.org</a>');><br>
>> [mailto:<a href="mailto:freeswitch-users-bounces@lists.freeswitch.org" target="_blank">freeswitch-users-bounces@lists.freeswitch.org</a><br>
>> <javascript:_e(%7B%7D,'cvml','<a href="mailto:freeswitch-users-bounces@lists.freeswitch.org" target="_blank">freeswitch-users-bounces@lists.freeswitch.org</a>');>] *On Behalf Of *Peter Olsson<br>
>> *Sent:* 28 August 2015 10:43<br>
>> *To:* FreeSWITCH Users Help <<a href="mailto:freeswitch-users@lists.freeswitch.org" target="_blank">freeswitch-users@lists.freeswitch.org</a><br>
>> <javascript:_e(%7B%7D,'cvml','<a href="mailto:freeswitch-users@lists.freeswitch.org" target="_blank">freeswitch-users@lists.freeswitch.org</a>');>><br>
>> *Subject:* Re: [Freeswitch-users] FreeSwitch - Performance<br>
>> issues____<br>
>><br>
>> __ __<br>
>><br>
>> If you want high performance, I recommend using real hardware, not a virtual machine setup - it will cause you<br>
>> issues.____<br>
>><br>
>> __ __<br>
>><br>
>> /Peter____<br>
>><br>
>> __ __<br>
>><br>
>> 2015-08-28 9:44 GMT+02:00 Shaun Stokes <<a href="mailto:shaun.stokes@itec-support.co.uk" target="_blank">shaun.stokes@itec-support.co.uk</a><br>
>><br>
>> <javascript:_e(%7B%7D,'cvml','<a href="mailto:shaun.stokes@itec-support.co.uk" target="_blank">shaun.stokes@itec-support.co.uk</a>');>>:__<br>
>> __<br>
>><br>
>> Hi Michael,____<br>
>><br>
>> ____<br>
>><br>
>> Thanks for the response and recommendation.____<br>
>><br>
>> ____<br>
>><br>
>> Our new build is now on Debian 8 Jessie, performance on the system is noticeably better and any audio problems<br>
>> while the system is under load are significantly reduced but we do still experience spikes along with reduced<br>
>> audio quality when the system is processing 20 calls per<br>
>> second while already maintaining 80 existing calls.____<br>
>><br>
>> ____<br>
>><br>
>> We're still hitting some kind of bottleneck, what should we expect to be able to support on a single piece of<br>
>> hardware? We've kept our dialplans as small as possible and we're using Memcache, but something tells me it's<br>
>> the call setup process which is triggering the spikes, is this possibly a limitation on the SIP profiles which<br>
>> are single threaded?____<br>
>><br>
>> ____<br>
>><br>
>> I also see FreeSWITCH supports clustering but it's in relation to high availability, is it possible to cluster a<br>
>> single FreeSWITCH instance across multiple hardware or should we be looking elsewhere to solve these<br>
>> bottlenecks?____<br>
>><br>
>> ____<br>
>><br>
>> We switched to TCP Vegas on our new build as recommended on the FreeSWITCH Wiki, this has provided a noticeable<br>
>> reduction in audio latency but doesn't solve the issue with<br>
>> CPU spikes.____<br>
>><br>
>> ____<br>
>><br>
>> Many Thanks,____<br>
>><br>
>> Shaun____<br>
>><br>
>> ____<br>
>><br>
>> *From:*<a href="mailto:freeswitch-users-bounces@lists.freeswitch.org" target="_blank">freeswitch-users-bounces@lists.freeswitch.org</a><br>
>> <javascript:_e(%7B%7D,'cvml','<a href="mailto:freeswitch-users-bounces@lists.freeswitch.org" target="_blank">freeswitch-users-bounces@lists.freeswitch.org</a>');><br>
>> [mailto:<a href="mailto:freeswitch-users-bounces@lists.freeswitch.org" target="_blank">freeswitch-users-bounces@lists.freeswitch.org</a><br>
>> <javascript:_e(%7B%7D,'cvml','<a href="mailto:freeswitch-users-bounces@lists.freeswitch.org" target="_blank">freeswitch-users-bounces@lists.freeswitch.org</a>');>] *On Behalf Of *Michael Jerris<br>
>> *Sent:* 15 August 2015 23:00<br>
>> *To:* FreeSWITCH Users Help <<a href="mailto:freeswitch-users@lists.freeswitch.org" target="_blank">freeswitch-users@lists.freeswitch.org</a><br>
>> <javascript:_e(%7B%7D,'cvml','<a href="mailto:freeswitch-users@lists.freeswitch.org" target="_blank">freeswitch-users@lists.freeswitch.org</a>');>><br>
>> *Subject:* Re: [Freeswitch-users] FreeSwitch - Performance<br>
>> issues____<br>
>><br>
>> ____<br>
>><br>
>> There are multiple known issues with Ubuntu 12.04. We reccomend Debian 8 Jessie.<br>
>><br>
>> On Friday, August 14, 2015, Shaun Stokes <<a href="mailto:shaun.stokes@itec-support.co.uk" target="_blank">shaun.stokes@itec-support.co.uk</a><br>
>><br>
>> <javascript:_e(%7B%7D,'cvml','<a href="mailto:shaun.stokes@itec-support.co.uk" target="_blank">shaun.stokes@itec-support.co.uk</a>');>><br>
>> wrote:____<br>
>><br>
>> Hi,____<br>
>><br>
>> ____<br>
>><br>
>> We're experiencing performance issues with FreeSwitch, our target is 500 concurrent sessions, but at the<br>
>> moment this starts to bottleneck around 30.____<br>
>><br>
>> ____<br>
>><br>
>> Host system:____<br>
>><br>
>> Windows Server 2012 Hyper-V Host____<br>
>><br>
>> AMD Opteron 4386 (2 processors) - 16 cores total____<br>
>><br>
>> 128GB DDR3____<br>
>><br>
>> 2TB RAID 5 (700MB/s tested read and write<br>
>> throughput)____<br>
>><br>
>> ____<br>
>><br>
>> FreeSwitch Virtual Machine:____<br>
>><br>
>> FreeSwitch 1.4.15____<br>
>><br>
>> Ubuntu 12.04 LTS____<br>
>><br>
>> 16 Virtual cores (high priority)____<br>
>><br>
>> 2GB RAM (would assign more but FreeSwitch never seems to<br>
>> use much)____<br>
>><br>
>> 500GB HD (on VHDX)____<br>
>><br>
>> ____<br>
>><br>
>> After around 30 concurrent sessions we begin to see CPU spikes almost every time a new call comes in, as the<br>
>> sessions increase the size and frequency of the CPU spikes also increase. The system seems to be able to sit<br>
>> comfortably with over 100 concurrent sessions and 80% idle CPU, providing we don't have any new calls<br>
>> hitting the platform. The spikes are causing audio (RTP) to stutter or in some cases drop completely for a<br>
>> few seconds.____<br>
>><br>
>> ____<br>
>><br>
>> The FreeSwitch spikes are occurring on all 16 cores, we have been monitoring the system using htop and<br>
>> mpstat.____<br>
>><br>
>> ____<br>
>><br>
>> This is an example of when we receive an inbound call<br>
>> while we have 34 concurrent sessions:____<br>
>><br>
>> 14:26:04 CPU %usr %nice %sys %iowait %irq %soft %steal %guest %idle____<br>
>><br>
>> 14:26:05 all 1.67 0.00 27.34 0.00 0.25 0.00 0.00 0.00 70.74____<br>
>><br>
>> 14:26:05 0 1.00 0.00 28.00 0.00 0.00 0.00 0.00 0.00 71.00____<br>
>><br>
>> 14:26:05 1 0.99 0.00 26.73 0.00 0.00 0.00 0.00 0.00 72.28____<br>
>><br>
>> 14:26:05 2 0.99 0.00 26.73 0.00 0.00 0.00 0.00 0.00 72.28____<br>
>><br>
>> 14:26:05 3 3.92 0.00 26.47 0.00 0.00 0.00 0.00 0.00 69.61____<br>
>><br>
>> 14:26:05 4 0.99 0.00 25.74 0.00 1.98 0.00 0.00 0.00 71.29____<br>
>><br>
>> 14:26:05 5 0.99 0.00 26.73 0.00 0.99 0.00 0.00 0.00 71.29____<br>
>><br>
>> 14:26:05 6 0.00 0.00 26.47 0.00 0.98 0.00 0.00 0.00 72.55____<br>
>><br>
>> 14:26:05 7 0.00 0.00 26.73 0.00 0.00 0.00 0.00 0.00 73.27____<br>
>><br>
>> 14:26:05 8 3.96 0.00 27.72 0.00 0.00 0.00 0.00 0.00 68.32____<br>
>><br>
>> 14:26:05 9 2.00 0.00 27.00 0.00 0.00 0.00 0.00 0.00 71.00____<br>
>><br>
>> 14:26:05 10 6.00 0.00 32.00 0.00 0.00 0.00 0.00 0.00 62.00____<br>
>><br>
>> 14:26:05 11 1.96 0.00 27.45 0.00 0.00 0.00 0.00 0.00 70.59____<br>
>><br>
>> 14:26:05 12 0.99 0.00 26.73 0.00 0.00 0.00 0.00 0.00 72.28____<br>
>><br>
>> 14:26:05 13 1.00 0.00 28.00 0.00 0.00 0.00 0.00 0.00 71.00____<br>
>><br>
>> 14:26:05 14 0.00 0.00 27.00 0.00 0.00 0.00 0.00 0.00 73.00____<br>
>><br>
>> ____<br>
>><br>
>> This is when the system is not receiving an inbound call but is sitting comfortably at 34 concurrent<br>
>> sessions:____<br>
>><br>
>> 14:25:57 CPU %usr %nice %sys %iowait %irq %soft %steal %guest %idle____<br>
>><br>
>> 14:25:58 all 0.87 0.00 0.62 0.00 0.12 0.00 0.00 0.00 98.39____<br>
>><br>
>> 14:25:58 0 0.99 0.00 0.99 0.00 0.00 0.00 0.00 0.00 98.02____<br>
>><br>
>> 14:25:58 1 0.99 0.00 0.99 0.00 0.00 0.00 0.00 0.00 98.02____<br>
>><br>
>> 14:25:58 2 0.98 0.00 0.98 0.00 0.00 0.00 0.00 0.00 98.04____<br>
>><br>
>> 14:25:58 3 0.00 0.00 0.99 0.00 0.00 0.00 0.00 0.00 99.01____<br>
>><br>
>> 14:25:58 4 0.00 0.00 0.00 0.00 1.98 0.00 0.00 0.00 98.02____<br>
>><br>
>> 14:25:58 5 0.00 0.00 0.00 0.00 0.99 0.00 0.00 0.00 99.01____<br>
>><br>
>> 14:25:58 6 0.00 0.00 0.99 0.00 0.00 0.00 0.00 0.00 99.01____<br>
>><br>
>> 14:25:58 7 0.00 0.00 0.99 0.00 0.00 0.00 0.00 0.00 99.01____<br>
>><br>
>> 14:25:58 8 5.00 0.00 1.00 0.00 0.00 0.00 0.00 0.00 94.00____<br>
>><br>
>> 14:25:58 9 1.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 99.00____<br>
>><br>
>> 14:25:58 10 1.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 99.00____<br>
>><br>
>> 14:25:58 11 0.99 0.00 0.99 0.00 0.00 0.00 0.00 0.00 98.02____<br>
>><br>
>> 14:25:58 12 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00____<br>
>><br>
>> 14:25:58 13 1.98 0.00 0.99 0.00 0.00 0.00 0.00 0.00 97.03____<br>
>><br>
>> 14:25:58 14 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00____<br>
>><br>
>> 14:25:58 15 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 100.00____<br>
>><br>
>> ____<br>
>><br>
>> We're using FreeSwitch in multi-tenant mode, we have tried dedicating a single VM for one tenancy but still<br>
>> experience the issue. My assumption has been that this will be due to the dialplan, I've optimized this<br>
>> slightly by writing LUA scripts to handle some of the inbound calls this appears to take away some of the<br>
>> load but we're still using the internal dialplans for each tenancy (tenancy dialplans have an average of 300<br>
>> entries).____<br>
>><br>
>> ____<br>
>><br>
>> We are using the following arguments when running<br>
>> FreeSwitch:____<br>
>><br>
>> -rp -nc -nonat____<br>
>><br>
>> ____<br>
>><br>
>> I've seen another post which says we should operate FreeSwitch in High Performance mode using -hp but can't<br>
>> find anything information about this, is this still a<br>
>> valid argument to use with FreeSwitch?____<br>
>><br>
>> ____<br>
>><br>
>> Has anyone experienced similar performance issues before<br>
>> or have any advice?____<br>
>><br>
>> ____<br>
>><br>
>> Hope someone may be able help.____<br>
>><br>
>> ____<br>
>><br>
>> Thanks,____<br>
>><br>
>> Shaun____<br>
>><br>
>><br>
>> ______________________________________________________________________<br>
>> This message has been checked for all known viruses by MessageLabs Virus Scanning Service.<br>
>><br>
>> _____________________________________________________________________<br>
>> _____<br>
>><br>
>><br>
>> ______________________________________________________________________<br>
>> This message has been checked for all known viruses by MessageLabs Virus Scanning Service.<br>
>><br>
>> _____________________________________________________________________<br>
>> _____<br>
>><br>
>><br>
>> ______________________________________________________________________<br>
>> This message has been checked for all known viruses by MessageLabs Virus Scanning Service.<br>
>><br>
>> _____________________________________________________________________<br>
>> _____<br>
>><br>
>><br>
>> _________________________________________________________________________<br>
>> Professional FreeSWITCH Consulting Services:<br>
>> <a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a> <javascript:_e(%7B%7D,'cvml','<a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a>');><br>
>> <a href="http://www.freeswitchsolutions.com" target="_blank">http://www.freeswitchsolutions.com</a><br>
>><br>
>> Official FreeSWITCH Sites<br>
>> <a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
>> <a href="http://confluence.freeswitch.org" target="_blank">http://confluence.freeswitch.org</a><br>
>> <a href="http://www.cluecon.com" target="_blank">http://www.cluecon.com</a><br>
>><br>
>> FreeSWITCH-users mailing list<br>
>> <a href="mailto:FreeSWITCH-users@lists.freeswitch.org" target="_blank">
FreeSWITCH-users@lists.freeswitch.org</a> <javascript:_e(%7B%7D,'cvml','<a href="mailto:FreeSWITCH-users@lists.freeswitch.org" target="_blank">FreeSWITCH-users@lists.freeswitch.org</a>');><br>
>> <a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
>> UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br>
>> <a href="http://www.freeswitch.org____" target="_blank">http://www.freeswitch.org____</a><br>
>><br>
>> __ __<br>
>><br>
>><br>
>> ______________________________________________________________________<br>
>> This message has been checked for all known viruses by MessageLabs Virus Scanning Service.<br>
>><br>
>> _____________________________________________________________________<br>
>> _____<br>
>><br>
>><br>
>> ______________________________________________________________________<br>
>> This message has been checked for all known viruses by MessageLabs Virus Scanning Service.<br>
>><br>
>> _____________________________________________________________________<br>
>> _<br>
>><br>
>><br>
>><br>
>> _____________________________________________________________________<br>
>> ____ Professional FreeSWITCH Consulting Services:<br>
>> <a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a><br>
>> <a href="http://www.freeswitchsolutions.com" target="_blank">http://www.freeswitchsolutions.com</a><br>
>><br>
>> Official FreeSWITCH Sites<br>
>> <a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
>> <a href="http://confluence.freeswitch.org" target="_blank">http://confluence.freeswitch.org</a><br>
>> <a href="http://www.cluecon.com" target="_blank">http://www.cluecon.com</a><br>
>><br>
>> FreeSWITCH-users mailing list<br>
>> <a href="mailto:FreeSWITCH-users@lists.freeswitch.org" target="_blank">FreeSWITCH-users@lists.freeswitch.org</a><br>
>> <a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
>> UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-us" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-us</a><br>
>> ers<br>
>> <a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
>><br>
><br>
<br>
--<br>
------------------------------------------------------------<br>
Nathan Neulinger <a href="mailto:nneul@mst.edu" target="_blank">nneul@mst.edu</a><br>
Missouri S&T Information Technology <a href="tel:%28573%29%20612-1412" target="_blank">
(573) 612-1412</a><br>
System Administrator - Architect<br>
<br>
_________________________________________________________________________<br>
Professional FreeSWITCH Consulting Services:<br>
<a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a><br>
<a href="http://www.freeswitchsolutions.com" target="_blank">http://www.freeswitchsolutions.com</a><br>
<br>
Official FreeSWITCH Sites<br>
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
<a href="http://confluence.freeswitch.org" target="_blank">http://confluence.freeswitch.org</a><br>
<a href="http://www.cluecon.com" target="_blank">http://www.cluecon.com</a><br>
<br>
FreeSWITCH-users mailing list<br>
<a href="mailto:FreeSWITCH-users@lists.freeswitch.org" target="_blank">FreeSWITCH-users@lists.freeswitch.org</a><br>
<a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br>
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
<br>
______________________________________________________________________<br>
This message has been checked for all known viruses by MessageLabs Virus Scanning Service.<br>
______________________________________________________________________<br>
<br>
______________________________________________________________________<br>
This message has been checked for all known viruses by MessageLabs Virus Scanning Service.<br>
______________________________________________________________________<br>
<br>
_________________________________________________________________________<br>
Professional FreeSWITCH Consulting Services:<br>
<a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a><br>
<a href="http://www.freeswitchsolutions.com" target="_blank">http://www.freeswitchsolutions.com</a><br>
<br>
Official FreeSWITCH Sites<br>
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
<a href="http://confluence.freeswitch.org" target="_blank">http://confluence.freeswitch.org</a><br>
<a href="http://www.cluecon.com" target="_blank">http://www.cluecon.com</a><br>
<br>
FreeSWITCH-users mailing list<br>
<a href="mailto:FreeSWITCH-users@lists.freeswitch.org" target="_blank">FreeSWITCH-users@lists.freeswitch.org</a><br>
<a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br>
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
<br>
______________________________________________________________________<br>
This message has been checked for all known viruses by MessageLabs Virus Scanning Service.<br>
______________________________________________________________________<br>
<br>
______________________________________________________________________<br>
This message has been checked for all known viruses by MessageLabs Virus Scanning Service.<br>
______________________________________________________________________<br>
<br>
_________________________________________________________________________<br>
Professional FreeSWITCH Consulting Services:<br>
<a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a><br>
<a href="http://www.freeswitchsolutions.com" target="_blank">http://www.freeswitchsolutions.com</a><br>
<br>
Official FreeSWITCH Sites<br>
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
<a href="http://confluence.freeswitch.org" target="_blank">http://confluence.freeswitch.org</a><br>
<a href="http://www.cluecon.com" target="_blank">http://www.cluecon.com</a><br>
<br>
FreeSWITCH-users mailing list<br>
<a href="mailto:FreeSWITCH-users@lists.freeswitch.org" target="_blank">FreeSWITCH-users@lists.freeswitch.org</a><br>
<a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br>
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><o:p></o:p></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><br>
<br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">--
<o:p></o:p></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p><b><i><span style="font-size:13.5pt;font-family:"Courier New"">Brian West</span></i></b><span style="font-family:"Courier New""><br>
</span><span style="font-size:10.0pt;font-family:"Courier New""><a href="mailto:brian@freeswitch.org" target="_blank">brian@freeswitch.org</a></span><o:p></o:p></p>
<p><span style="font-size:7.5pt;font-family:"Courier New""><img border="0" id="_x0000_i1025" src="http://billing.freeswitch.org/templates/default/img/whmcslogo.png"></span><o:p></o:p></p>
<p><b><i><span style="font-size:10.0pt;font-family:"Courier New"">Twitter: @FreeSWITCH , @briankwest</span></i></b><span style="font-size:10.0pt;font-family:"Courier New""><br>
<a href="http://www.freeswitchbook.com" target="_blank">http://www.freeswitchbook.com</a><br>
<a href="http://www.freeswitchcookbook.com" target="_blank">http://www.freeswitchcookbook.com</a></span><o:p></o:p></p>
<p><span style="font-family:"Courier New"">Got Bugs? Report them <a href="https://freeswitch.org/jira" target="_blank">
here</a>! | Reddit: <a href="https://www.reddit.com/r/freeswitch" target="_blank">/r/freeswitch</a></span><o:p></o:p></p>
<p><b><span style="font-size:10.0pt;font-family:"Courier New"">T:</span></b><span style="font-size:10.0pt;font-family:"Courier New""><a href="tel:%2B19184209001" target="_blank">+19184209001</a> |
<b>F:</b><a href="tel:%2B19184209002" target="_blank">+19184209002</a> | <b>M:</b>+1918424WEST (9378)<br>
<b>iNUM:</b>+883 5100 1420 9001 | <b>ISN:</b>410*543 | <b>Skype:</b>briankwest</span><o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><br>
______________________________________________________________________<br>
This message has been checked for all known viruses by MessageLabs Virus Scanning Service.<br>
______________________________________________________________________<o:p></o:p></p>
</div>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><br>
______________________________________________________________________<br>
This message has been checked for all known viruses by MessageLabs Virus Scanning Service.<br>
______________________________________________________________________<o:p></o:p></p>
</div>
</div>
</div>
<p class="MsoNormal"><br>
_________________________________________________________________________<br>
Professional FreeSWITCH Consulting Services:<br>
<a href="mailto:consulting@freeswitch.org">consulting@freeswitch.org</a><br>
<a href="http://www.freeswitchsolutions.com" target="_blank">http://www.freeswitchsolutions.com</a><br>
<br>
Official FreeSWITCH Sites<br>
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
<a href="http://confluence.freeswitch.org" target="_blank">http://confluence.freeswitch.org</a><br>
<a href="http://www.cluecon.com" target="_blank">http://www.cluecon.com</a><br>
<br>
FreeSWITCH-users mailing list<br>
<a href="mailto:FreeSWITCH-users@lists.freeswitch.org">FreeSWITCH-users@lists.freeswitch.org</a><br>
<a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br>
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><o:p></o:p></p>
</blockquote>
</div>
<p class="MsoNormal"><br>
<br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- <o:p></o:p></p>
<div>
<p class="MsoNormal">Sincerely<br>
<br>
Jay<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><br>
______________________________________________________________________<br>
This message has been checked for all known viruses by MessageLabs Virus Scanning Service.<br>
______________________________________________________________________<o:p></o:p></p>
</div>
<br clear="both">
______________________________________________________________________<BR>
This message has been checked for all known viruses by MessageLabs Virus Scanning Service.<BR>
______________________________________________________________________<BR>
</body>
</html>