Settings for maximum performance
Hi all, We are evaluating FreeSWITCH as a replacement for our current commercial SBCs - before anyone points it out, we need a B2BUA for sane CDR generation (aka billing) and there is some ideas for future functionality that would leverage more of the FreeSWITCH platform. That being said we are currently forcing static IPs, disabling registration and presence and setting inbound-late-negotiation for the sip profiles. "Client" gateways are being authenticated using remote IP ACLs and a dial string prefix. The dialplan looks like this... http://dpaste.com/196561/ The box in question has two Intel Xeon 5130 dual core processors w/ 4 GB Ram (being upgraded, trying to figure out how much ram we should have). Disks are 4x 15k RPM SCSI in a Raid 10. Operating system is CentOS 5.4 x86_64. FreeSWITCH is being started with "-nc -nonat -waste". What things should we be looking at to squeeze as much performance as possible out of these boxes? Thanks, Max [root@fs-sbc02 freeswitch]# vmstat 3 procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------ r b swpd free buff cache si so bi bo in cs us sy id wa st 4 0 112 25220 237172 2829060 0 0 0 10 2 2 0 0 99 0 0 0 1 112 48412 236400 2800596 0 0 0 2408 13298 212312 20 14 63 3 0 25 0 112 46024 236444 2803632 0 0 0 1172 13245 219590 15 13 69 4 0 32 0 112 45784 236512 2808200 0 0 0 2839 13153 164973 21 14 61 3 0 63 1 112 37880 236544 2811808 0 0 0 2884 13048 199455 21 16 60 4 0 1 0 112 37488 236588 2815612 0 0 0 4000 13201 142378 16 17 50 17 0 0 0 112 42648 236616 2818376 0 0 0 2125 13171 107533 13 13 70 3 0 1 0 112 47904 236636 2820704 0 0 0 1989 13400 86644 10 11 72 6 0 6 0 112 50372 236652 2822200 0 0 0 1585 13339 69289 8 9 78 5 0
On Wed, May 19, 2010 at 10:55 AM, Max Clark <max.clark@gmail.com> wrote:
Hi all,
We are evaluating FreeSWITCH as a replacement for our current commercial SBCs - before anyone points it out, we need a B2BUA for sane CDR generation (aka billing) and there is some ideas for future functionality that would leverage more of the FreeSWITCH platform.
That being said we are currently forcing static IPs, disabling registration and presence and setting inbound-late-negotiation for the sip profiles. "Client" gateways are being authenticated using remote IP ACLs and a dial string prefix. The dialplan looks like this...
The box in question has two Intel Xeon 5130 dual core processors w/ 4 GB Ram (being upgraded, trying to figure out how much ram we should have). Disks are 4x 15k RPM SCSI in a Raid 10. Operating system is CentOS 5.4 x86_64.
FreeSWITCH is being started with "-nc -nonat -waste".
What things should we be looking at to squeeze as much performance as possible out of these boxes?
Thanks, Max
Instead of using -waste on the CLI you should be setting the correct ulimit parameters in your init script: ulimit -c unlimited ulimit -d unlimited ulimit -f unlimited ulimit -i unlimited ulimit -n 999999 ulimit -q unlimited ulimit -u unlimited ulimit -v unlimited ulimit -x unlimited ulimit -s 240 ulimit -l unlimited ulimit -a Something like that, at least. It sounds like you've already done most of this, but: - Disable all NAT/STUN/etc handling in your profiles. Your FreeSWITCH instance is not behind NAT (you can still detect NAT from your endpoints) - Check, check, double check your SIP profile settings. Session timers? Comfort noise? TCP? Do you need these? - Increase the maximum number of sessions and sessions per second in switch.conf. Make sure the RTP port ranges "jive" with what your network expects. - Look at the timer test results when FreeSWITCH starts up. Make sure its not working too hard just to maintain a timer. - Disable any unneeded modules in modules.conf. - Don't use proxy_media. You probably don't really want it. - Use as many SIP profiles as you can (I believe each SIP profile runs in its own thread). - Make sure APIC, MSI/MSI-X, etc are all working. Make sure you've got decent NICs and drivers loaded. - Email consulting@freeswitch.org :) -- Kristian Kielhofner http://www.astlinux.org http://blog.krisk.org http://www.star2star.com http://www.submityoursip.com http://www.voalte.com
On Wed, May 19, 2010 at 8:45 AM, Kristian Kielhofner <kris@kriskinc.com> wrote:
ulimit -c unlimited ulimit -d unlimited ulimit -f unlimited ulimit -i unlimited ulimit -n 999999 ulimit -q unlimited ulimit -u unlimited ulimit -v unlimited ulimit -x unlimited ulimit -s 240 ulimit -l unlimited ulimit -a
Okay, set and removed -waste from init script.
- Disable all NAT/STUN/etc handling in your profiles. Your FreeSWITCH instance is not behind NAT (you can still detect NAT from your endpoints)
So my sip profile now looks like this: <param name="rtp-ip" value="$${local_ip_v4}"/> <param name="sip-ip" value="$${local_ip_v4}"/> <param name="ext-rtp-ip" value="$${local_ip_v4}"/> <param name="ext-sip-ip" value="$${local_ip_v4}"/> <param name="stun-enabled" value="false"/> <param name="stun-auto-disable" value="true"/> I didn't find anything in the sofia configuration for disabling nat which is why I have the -nonat on startup now. Is there another option I should be setting?
- Check, check, double check your SIP profile settings. Session timers? Comfort noise? TCP? Do you need these?
Looking into these settings - what's sane for my usage?
- Increase the maximum number of sessions and sessions per second in switch.conf. Make sure the RTP port ranges "jive" with what your network expects.
How do I size sessions & sessions per second based on the machine cpu/ram?
- Look at the timer test results when FreeSWITCH starts up. Make sure its not working too hard just to maintain a timer.
Looking through the freeswitch.log file I do not see any output on timers - should I be looking somewhere else?
- Disable any unneeded modules in modules.conf.
Check
- Don't use proxy_media. You probably don't really want it.
I have proxy_media because we want to keep the RTP streams through our boxes (topology hiding), we were looking for better t.38 handling, we don't want to transcode, and it seemed okay to let the endpoints negotiate g.711 or g.729 on their own.
- Use as many SIP profiles as you can (I believe each SIP profile runs in its own thread).
This one is tough, we'll take another look at it.
- Make sure APIC, MSI/MSI-X, etc are all working. Make sure you've got decent NICs and drivers loaded.
I'm assuming that Dell PowerEdge servers aren't going to be an issue.
- Email consulting@freeswitch.org :)
Trying to knock out all the "easy" stuff first. Thank you very much for the pointers. -Max
- Check, check, double check your SIP profile settings. Session timers? Comfort noise? TCP? Do you need these?
Looking into these settings - what's sane for my usage?
Session timers are useful if you have issues with ghost calls. If you think you're "ghost-free", you may disable it.
- Increase the maximum number of sessions and sessions per second in switch.conf. Make sure the RTP port ranges "jive" with what your network expects.
How do I size sessions & sessions per second based on the machine cpu/ram?
That's a good question. I think you'll probably need to discover that yourself from experience. For a box like the one you described, I would say 2000 calls and 100 cps is a good start.
- Look at the timer test results when FreeSWITCH starts up. Make sure its not working too hard just to maintain a timer.
Looking through the freeswitch.log file I do not see any output on timers - should I be looking somewhere else?
This shows up during startup. David Ponzone Direction Technique email: david.ponzone@ipeva.fr tel: 01 74 03 18 97 gsm: 06 66 98 76 34 Service Client IPeva tel: 0811 46 26 26 www.ipeva.fr - www.ipeva-studio.com Ce message et toutes les pièces jointes sont confidentiels et établis à l'intention exclusive de ses destinataires. Toute utilisation ou diffusion non autorisée est interdite. Tout message électronique est susceptible d'altération. IPeva décline toute responsabilité au titre de ce message s'il a été altéré, déformé ou falsifié. Si vous n'êtes pas destinataire de ce message, merci de le détruire immédiatement et d'avertir l'expéditeur.
On Wed, May 19, 2010 at 4:06 PM, Max Clark <max.clark@gmail.com> wrote:
So my sip profile now looks like this:
<param name="rtp-ip" value="$${local_ip_v4}"/> <param name="sip-ip" value="$${local_ip_v4}"/> <param name="ext-rtp-ip" value="$${local_ip_v4}"/> <param name="ext-sip-ip" value="$${local_ip_v4}"/> <param name="stun-enabled" value="false"/> <param name="stun-auto-disable" value="true"/>
I didn't find anything in the sofia configuration for disabling nat which is why I have the -nonat on startup now. Is there another option I should be setting?
Not really. Remove the ext-*-ip params and you should be fine.
Looking into these settings - what's sane for my usage?
You probably don't need any of them. Disable timers, CNG, and TCP.
How do I size sessions & sessions per second based on the machine cpu/ram?
You'll have to find out from testing with your config but I'm assuming you'll want to go beyond the defaults. You could probably start by doubling or tripling the default values.
Looking through the freeswitch.log file I do not see any output on timers - should I be looking somewhere else?
Start FreeSWITCH with the console (-c). Look at the output in the first ~20 lines. You'll see things about timers and tests. Make sure it doesn't issue a warning about audio quality and that the offset isn't too large.
I have proxy_media because we want to keep the RTP streams through our boxes (topology hiding), we were looking for better t.38 handling, we don't want to transcode, and it seemed okay to let the endpoints negotiate g.711 or g.729 on their own.
T.38 requires proxy_media but other than that you may be better off with the default bridging behavior. It will proxy media and hide your topology. You can disable transcoding in the profile and/or have all kinds of control over codec parameters in the dialplan in combination with late-negotiation.
I'm assuming that Dell PowerEdge servers aren't going to be an issue.
If they're relatively new they should be ok but check /proc/interrupts to make sure (look for MSI/MSI-X/APIC next to eth0, eth1, etc). -- Kristian Kielhofner http://www.astlinux.org http://blog.krisk.org http://www.star2star.com http://www.submityoursip.com http://www.voalte.com
If they're relatively new they should be ok but check /proc/interrupts to make sure (look for MSI/MSI-X/APIC next to eth0, eth1, etc).
You said earlier to make sure that MSI was working properly - how do I confirm this? Or is this something I should disable? What is best for FreeSWITCH? Thanks, Max [mclark@fs-sbc02 ~]$ cat /proc/interrupts CPU0 CPU1 CPU2 CPU3 0: 3969295082 0 0 0 IO-APIC-edge timer 1: 3 0 0 0 IO-APIC-edge i8042 8: 1 0 0 0 IO-APIC-edge rtc 9: 0 0 0 0 IO-APIC-level acpi 12: 4 0 0 0 IO-APIC-edge i8042 14: 25 0 0 0 IO-APIC-edge ide0 66: 57 0 0 0 IO-APIC-level ehci_hcd:usb1, uhci_hcd:usb2, uhci_hcd:usb4 74: 0 0 0 0 IO-APIC-level uhci_hcd:usb3 82: 11263182 4932428 71397 54669 IO-APIC-level megasas 98: 1738518148 0 0 0 PCI-MSI eth0 NMI: 91918 46871 42712 41086 LOC: 3969292329 3969292268 3969292197 3969292121 ERR: 0 MIS: 0
participants (3)
-
David Ponzone
-
Kristian Kielhofner
-
Max Clark