[Freeswitch-users] Running FS on a heavy loaded VMWare host

[Grant Bagdasarian]

The host's CPU is being completely maxed out when SIPp generates 1000+ concurrent calls.
500 concurrent calls takes about 60% of the host's CPU.

SBC 1 and SBC 2 both have two call legs, App Server has just one call leg.
Because there are 3 virtual machines where RTP traffic is flowing through, this is a total of TotalCallLegsPerCall * NumberOfConcurrentCalls.
So for 500 concurrent calls generated by SIPp: 5 * 500 = 2500 call legs
And for 1000 concurrent calls generated by SIPp: 5 * 1000 = 5000 call legs

I'll also do a test where only one virtual gets all the resources and the other two are shut down.

From: freeswitch-users-bounces at lists.freeswitch.org [mailto:freeswitch-users-bounces at lists.freeswitch.org] On Behalf Of Anthony Minessale
Sent: Wednesday, September 4, 2013 2:56 PM
To: Freeswitch-users
Subject: Re: [Freeswitch-users] Running FS on a heavy loaded VMWare host


You should compare that to bare metal on the same box. And look at the stats of the host during the test to make sure its not maxed out.
On Sep 4, 2013 4:52 AM, "Grant Bagdasarian" <GB at cm.nl<mailto:GB at cm.nl>> wrote:
I'll look for those options, thanks.

I did some random tests with Sipp 3.2. (more detailed results will follow a future post)

The flow is as following:

Sipp ----> FreeSWITCH SBC 1 ----> FreeSWITCH SBC 2 ----> FreeSWITCH App Server

The SBC's just bridge the incoming call with an outgoing one.
The App Server plays a file which is about 2 minutes in length. G711 Alaw codec.

The host has a 6 Core Intel Xeon(12 Threads) CPU and 8 GB of RAM. Running ESXi 5.1
Each virtual has 4 Virtual cores and 2 GB of memory.

I tweaked the ulimit settings on all machines according to the information in : http://wiki.freeswitch.org/wiki/Performance_testing_and_configurations

I generated about 2000 calls using Sipp.
The audio quality was good until it hit 800+ concurrent calls.
I let the test run and called in myself using X-Lite.
SBC 1 and SBC 2 had each about ~1600 channels active (incoming and outgoing leg) and the App Server had 800.
At 1000 concurrent calls the audio was still fair, but did have some audio drops. At around 1400 the audio was completely dropped, no sound from that point on.

SBC 1 and SBC 2 were at 100% CPU usage, on all 4 cores. The App Server was at a constant 50% on all 4 cores.

I'm still wondering the following, the audio file which was played is stored on the App Server in /freeswitch/sounds/. Does FreeSWITCH read the file for every new call? Or is the file cached in memory and accessed from there?
If it does read it for every new call, maybe I can get even more performance out of this if we can cache it somehow.

From: freeswitch-users-bounces at lists.freeswitch.org<mailto:freeswitch-users-bounces at lists.freeswitch.org> [mailto:freeswitch-users-bounces at lists.freeswitch.org<mailto:freeswitch-users-bounces at lists.freeswitch.org>] On Behalf Of Anthony Minessale
Sent: Tuesday, September 3, 2013 10:13 PM
To: FreeSWITCH Users Help
Subject: Re: [Freeswitch-users] Running FS on a heavy loaded VMWare host

The key setting in vmware is one regarding passing the timer reqs down to the real hardware.
I don't remember the name but if its set to emulate the timer source rather than share the real one, it will break for sure.
The other thing was the mode the network is in, there is on that is like a hub with the host box and that one duplicates the RTP traffic over a bridge vs the virtual interface mode that is more performant.

That said we don't officially guarantee any results with virtual hosts but the community is encouraged to document their experiences here and on the wiki.



On Tue, Sep 3, 2013 at 7:40 AM, Grant Bagdasarian <GB at cm.nl<mailto:GB at cm.nl>> wrote:
Hello,

Thank you all for your comments.

I've decided to use a separate VM host dedicated to Voice services. Our current physical box running FreeSWITCH will be re-installed with ESXi and will host 3 virtuals where FreeSWITCH will be installed on. Each having dedicated cores.
I'm planning to do exactly what Michael Jerris suggested, but with SIPp generating load on the virtuals and calling in using X-Lite to measure the audio quality.
This way I can simulate a (heavy) loaded host.

I'll post the results once I have completed the tests.

Grant

From: freeswitch-users-bounces at lists.freeswitch.org<mailto:freeswitch-users-bounces at lists.freeswitch.org> [mailto:freeswitch-users-bounces at lists.freeswitch.org<mailto:freeswitch-users-bounces at lists.freeswitch.org>] On Behalf Of Michael Jerris
Sent: Tuesday, September 3, 2013 2:15 PM
To: FreeSWITCH Users Help
Subject: Re: [Freeswitch-users] Running FS on a heavy loaded VMWare host

Any voip running on a very heavily loaded VM host runs the risk of having audio issues.  The question is, is it so heavily loaded that freeswitch will be starved of clock cycles or not.  Testing it is the only way to see.  Even if it is to some extent, things that don't need timing like regular bridged calls will be much more tolerant of some load than things that require more timing like playing sound files or conferences.  I would test spinning up a bunch of channels and calling in and listening to a milliwatt tone to see if its clean or not.

Mike

On Sep 2, 2013, at 2:34 AM, Grant Bagdasarian <GB at cm.nl<mailto:GB at cm.nl>> wrote:

Hello,

I've been going through some threads about virtualization, some say it works for them, others say it doesn't. Multiple platforms are used like MS Hyper-V, VM Ware, etc.
We currently run FS on a physical box, which works great.

We have our own virtual environment, VMWare vSphere 5.0. Our 5.0 environment is running under heavy load and I was wondering would FS (configured properly for correct timing) still function properly and able to handle hundreds of concurrent calls on a heavy loaded VMWare host? The FS virtual will also process RTP in default mode. I was hoping if someone had any experience using this setup and could give me some insights on this matter. The FS virtuals will function only as SBC's, so no complex dial plans, no call recording, no conferencing. Just bridging the incoming call leg with an outgoing one.

VMWare has a tech paper about running VoIP on their virtual machine, and they claim it works. (http://www.vmware.com/files/pdf/techpaper/voip-perf-vsphere5.pdf). In the paper they used Mitel vMCD, but I'm not sure if the software has been optimized to run on VMWare so the tests would come out better than running any other telephony software like FreeSwitch or Asterisk.

Grant
_________________________________________________________________________
Professional FreeSWITCH Consulting Services:
consulting at freeswitch.org<mailto:consulting at freeswitch.org>
http://www.freeswitchsolutions.com




Official FreeSWITCH Sites
http://www.freeswitch.org
http://wiki.freeswitch.org
http://www.cluecon.com

FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org<mailto:FreeSWITCH-users at lists.freeswitch.org>
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users<http://lists.freeswitch.org/mailman/options/freeswitch-users>
http://www.freeswitch.org


_________________________________________________________________________
Professional FreeSWITCH Consulting Services:
consulting at freeswitch.org<mailto:consulting at freeswitch.org>
http://www.freeswitchsolutions.com




Official FreeSWITCH Sites
http://www.freeswitch.org
http://wiki.freeswitch.org
http://www.cluecon.com

FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org<mailto:FreeSWITCH-users at lists.freeswitch.org>
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org



--
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com<mailto:MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<mailto:PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net<http://irc.freenode.net> #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org<mailto:sip%3A888 at conference.freeswitch.org>
googletalk:conf+888 at conference.freeswitch.org<mailto:googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:+19193869900<tel:%2B19193869900>

_________________________________________________________________________
Professional FreeSWITCH Consulting Services:
consulting at freeswitch.org<mailto:consulting at freeswitch.org>
http://www.freeswitchsolutions.com




Official FreeSWITCH Sites
http://www.freeswitch.org
http://wiki.freeswitch.org
http://www.cluecon.com

FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org<mailto:FreeSWITCH-users at lists.freeswitch.org>
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20130904/3312c6e4/attachment-0001.html