Hi folks, We are currently running FreeSWITCH as a (sip/rtp) conference endpoint in our system and recently completed some latency testing. We found that audio passing through FreeSWITCH was taking approx 40-60ms to be mixed and sent out the other side. I did some looking around and as far as I can tell, we don't have FreeSWITCH configured with a jitter-buffer. The only other possible performance issue I could find was the 'tickless' issue ( http://wiki.freeswitch.org/wiki/Installation_Guide#Release.28es.29_6_and_Lat...) when running FreeSWITCH on a RedHat 6+ installation. However, its not clear to me if the 'tickless' setting could cause this type of latency when there are only two clients connected to FreeSWITCH. I was wondering if anybody else had experienced this type of problem or had any suggestions? For reference, we are using a RTP/PCMU payload with 10ms samples. Thanks, Daniel
EL6/Centos 6 has been showing us some rather bad performance issues... if you can run your config under oprofile and give us some of the outputs related to freeswitch we would love to see this data and try to figure out where the issue is and if its something we can work around or if its something we need to look at getting theupstream to fix.... see http://oprofile.sourceforge.net/examples/ for how to use oprofile On Wed, Aug 29, 2012 at 3:18 PM, Daniel Eiland <daniel.eiland@gmail.com>wrote:
Hi folks,
We are currently running FreeSWITCH as a (sip/rtp) conference endpoint in our system and recently completed some latency testing. We found that audio passing through FreeSWITCH was taking approx 40-60ms to be mixed and sent out the other side.
I did some looking around and as far as I can tell, we don't have FreeSWITCH configured with a jitter-buffer. The only other possible performance issue I could find was the 'tickless' issue ( http://wiki.freeswitch.org/wiki/Installation_Guide#Release.28es.29_6_and_Lat...) when running FreeSWITCH on a RedHat 6+ installation. However, its not clear to me if the 'tickless' setting could cause this type of latency when there are only two clients connected to FreeSWITCH. I was wondering if anybody else had experienced this type of problem or had any suggestions?
For reference, we are using a RTP/PCMU payload with 10ms samples.
Thanks, Daniel
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
What IS the recommended linux distro to use for optimum performance? I've heard debian, can someone be more specific though, like what version, kernel, etc... ? On Wed, Aug 29, 2012 at 4:50 PM, Ken Rice <krice@freeswitch.org> wrote:
EL6/Centos 6 has been showing us some rather bad performance issues... if you can run your config under oprofile and give us some of the outputs related to freeswitch we would love to see this data and try to figure out where the issue is and if its something we can work around or if its something we need to look at getting theupstream to fix....
see http://oprofile.sourceforge.net/examples/ for how to use oprofile
On Wed, Aug 29, 2012 at 3:18 PM, Daniel Eiland <daniel.eiland@gmail.com> wrote:
Hi folks,
We are currently running FreeSWITCH as a (sip/rtp) conference endpoint in our system and recently completed some latency testing. We found that audio passing through FreeSWITCH was taking approx 40-60ms to be mixed and sent out the other side.
I did some looking around and as far as I can tell, we don't have FreeSWITCH configured with a jitter-buffer. The only other possible performance issue I could find was the 'tickless' issue (http://wiki.freeswitch.org/wiki/Installation_Guide#Release.28es.29_6_and_Lat...) when running FreeSWITCH on a RedHat 6+ installation. However, its not clear to me if the 'tickless' setting could cause this type of latency when there are only two clients connected to FreeSWITCH. I was wondering if anybody else had experienced this type of problem or had any suggestions?
For reference, we are using a RTP/PCMU payload with 10ms samples.
Thanks, Daniel
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
On Wed, Aug 29, 2012 at 1:56 PM, Vik Killa <vipkilla@gmail.com> wrote:
What IS the recommended linux distro to use for optimum performance? I've heard debian, can someone be more specific though, like what version, kernel, etc... ?
We try to stay out of the OS Holy Wars, so we don't have an "official" recommended distro. However, I know more than a few people (disclosure: my self included) who have been using Debian Squeeze 32 and 64 bit with zero drama, zero issues, and solid performance. -- Michael S Collins Twitter: @mercutioviz http://www.FreeSWITCH.org http://www.ClueCon.com http://www.OSTAG.org
Centos5 worked well for me, Centos6 worked for crap... I went to debian 6 (squeeze) and problems went a way... using the latest stable on the debian6 line installing from the netinstall (non-free variant as I have hardware thats not supported under the main netinstall iso) and then installing freeswitch via the instructions on the wiki and no problems so far (the centos6 issues were bad enough i moved my dev box to debian6) On Wed, Aug 29, 2012 at 4:01 PM, Michael Collins <msc@freeswitch.org> wrote:
On Wed, Aug 29, 2012 at 1:56 PM, Vik Killa <vipkilla@gmail.com> wrote:
What IS the recommended linux distro to use for optimum performance? I've heard debian, can someone be more specific though, like what version, kernel, etc... ?
We try to stay out of the OS Holy Wars, so we don't have an "official" recommended distro. However, I know more than a few people (disclosure: my self included) who have been using Debian Squeeze 32 and 64 bit with zero drama, zero issues, and solid performance.
-- Michael S Collins Twitter: @mercutioviz http://www.FreeSWITCH.org http://www.ClueCon.com http://www.OSTAG.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Woe is me. I'm using centos 6 and I figured that performance issues was due to my cheap box and was actually planning to buy a more powerful computer. What specific issues did you guys encounter and has anyone tried the new 6.3 version. I would like to keep an eye out for these problems on my distro. On Aug 29, 2012 5:10 PM, "Ken Rice" <krice@freeswitch.org> wrote:
Centos5 worked well for me, Centos6 worked for crap... I went to debian 6 (squeeze) and problems went a way... using the latest stable on the debian6 line installing from the netinstall (non-free variant as I have hardware thats not supported under the main netinstall iso) and then installing freeswitch via the instructions on the wiki and no problems so far
(the centos6 issues were bad enough i moved my dev box to debian6)
On Wed, Aug 29, 2012 at 4:01 PM, Michael Collins <msc@freeswitch.org>wrote:
On Wed, Aug 29, 2012 at 1:56 PM, Vik Killa <vipkilla@gmail.com> wrote:
What IS the recommended linux distro to use for optimum performance? I've heard debian, can someone be more specific though, like what version, kernel, etc... ?
We try to stay out of the OS Holy Wars, so we don't have an "official" recommended distro. However, I know more than a few people (disclosure: my self included) who have been using Debian Squeeze 32 and 64 bit with zero drama, zero issues, and solid performance.
-- Michael S Collins Twitter: @mercutioviz http://www.FreeSWITCH.org http://www.ClueCon.com http://www.OSTAG.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Reports are that when the load starts to increase there's an exponential increase in kernel calls, causing all kinds of havoc. If you can run it in oprofile and get the logs to Ken Rice that would be helpful. -MC On Wed, Aug 29, 2012 at 3:21 PM, BookBag <asaad2@gmail.com> wrote:
Woe is me. I'm using centos 6 and I figured that performance issues was due to my cheap box and was actually planning to buy a more powerful computer.
What specific issues did you guys encounter and has anyone tried the new 6.3 version. I would like to keep an eye out for these problems on my distro. On Aug 29, 2012 5:10 PM, "Ken Rice" <krice@freeswitch.org> wrote:
Centos5 worked well for me, Centos6 worked for crap... I went to debian 6 (squeeze) and problems went a way... using the latest stable on the debian6 line installing from the netinstall (non-free variant as I have hardware thats not supported under the main netinstall iso) and then installing freeswitch via the instructions on the wiki and no problems so far
(the centos6 issues were bad enough i moved my dev box to debian6)
On Wed, Aug 29, 2012 at 4:01 PM, Michael Collins <msc@freeswitch.org>wrote:
On Wed, Aug 29, 2012 at 1:56 PM, Vik Killa <vipkilla@gmail.com> wrote:
What IS the recommended linux distro to use for optimum performance? I've heard debian, can someone be more specific though, like what version, kernel, etc... ?
We try to stay out of the OS Holy Wars, so we don't have an "official" recommended distro. However, I know more than a few people (disclosure: my self included) who have been using Debian Squeeze 32 and 64 bit with zero drama, zero issues, and solid performance.
-- Michael S Collins Twitter: @mercutioviz http://www.FreeSWITCH.org http://www.ClueCon.com http://www.OSTAG.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Michael S Collins Twitter: @mercutioviz http://www.FreeSWITCH.org http://www.ClueCon.com http://www.OSTAG.org
Centos5 worked well for me, Centos6 worked for crap... I went to debian 6 (squeeze) and problems went a way... using the latest stable on the debian6 line installing from the netinstall (non-free variant as I have hardware thats not supported under the main netinstall iso) and then installing freeswitch via the instructions on the wiki and no problems so far
Same here. We're CentOS 5 here and Debian Squeezy for new boxes. -- http://www.suretecsystems.com/services/openldap/ http://www.surevoip.co.uk
Is anyone having the same problems with RedHat or is this CentOS specific? On Thu, Aug 30, 2012 at 6:23 PM, Gavin Henry <gavin.henry@gmail.com> wrote:
Centos5 worked well for me, Centos6 worked for crap... I went to debian 6 (squeeze) and problems went a way... using the latest stable on the debian6 line installing from the netinstall (non-free variant as I have hardware thats not supported under the main netinstall iso) and then installing freeswitch via the instructions on the wiki and no problems so far
Same here. We're CentOS 5 here and Debian Squeezy for new boxes.
-- http://www.suretecsystems.com/services/openldap/ http://www.surevoip.co.uk
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Guillermo Ruiz Camauer
On Thu, Aug 30, 2012 at 3:12 PM, Guillermo Ruiz Camauer <grcamauer@gmail.com
wrote:
Is anyone having the same problems with RedHat or is this CentOS specific?
An excellent question. I'd really like to know. Same with those running Scientific Linux (SL) - it's RHEL based also. -MC
What exactly is the problem? From: freeswitch-users-bounces@lists.freeswitch.org [mailto:freeswitch-users-bounces@lists.freeswitch.org] On Behalf Of Guillermo Ruiz Camauer Sent: Thursday, August 30, 2012 6:13 PM To: FreeSWITCH Users Help Subject: Re: [Freeswitch-users] RedHat 6.X Performance Is anyone having the same problems with RedHat or is this CentOS specific? On Thu, Aug 30, 2012 at 6:23 PM, Gavin Henry <gavin.henry@gmail.com<mailto:gavin.henry@gmail.com>> wrote:
Centos5 worked well for me, Centos6 worked for crap... I went to debian 6 (squeeze) and problems went a way... using the latest stable on the debian6 line installing from the netinstall (non-free variant as I have hardware thats not supported under the main netinstall iso) and then installing freeswitch via the instructions on the wiki and no problems so far Same here. We're CentOS 5 here and Debian Squeezy for new boxes.
-- http://www.suretecsystems.com/services/openldap/ http://www.surevoip.co.uk _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org<mailto:consulting@freeswitch.org> http://www.freeswitchsolutions.com FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org<mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org -- Guillermo Ruiz Camauer
Performance problems as load increases which are not found on other distros such as Debian. On Thu, Aug 30, 2012 at 7:17 PM, Colin Mason <cmason@frontiernetworks.ca>wrote:
What exactly is the problem?****
** **
*From:* freeswitch-users-bounces@lists.freeswitch.org [mailto: freeswitch-users-bounces@lists.freeswitch.org] *On Behalf Of *Guillermo Ruiz Camauer *Sent:* Thursday, August 30, 2012 6:13 PM *To:* FreeSWITCH Users Help *Subject:* Re: [Freeswitch-users] RedHat 6.X Performance****
** **
Is anyone having the same problems with RedHat or is this CentOS specific? ****
** **
On Thu, Aug 30, 2012 at 6:23 PM, Gavin Henry <gavin.henry@gmail.com> wrote:****
Centos5 worked well for me, Centos6 worked for crap... I went to debian 6 (squeeze) and problems went a way... using the latest stable on the debian6 line installing from the netinstall (non-free variant as I have hardware thats not supported under the main netinstall iso) and then installing freeswitch via the instructions on the wiki and no problems so far****
Same here. We're CentOS 5 here and Debian Squeezy for new boxes.
-- http://www.suretecsystems.com/services/openldap/ http://www.surevoip.co.uk****
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org****
****
** **
-- Guillermo Ruiz Camauer****
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Guillermo Ruiz Camauer
Never had any performance issues on CentOS 6 under load. Can you be more specific? From: freeswitch-users-bounces@lists.freeswitch.org [mailto:freeswitch-users-bounces@lists.freeswitch.org] On Behalf Of Guillermo Ruiz Camauer Sent: Thursday, August 30, 2012 6:24 PM To: FreeSWITCH Users Help Subject: Re: [Freeswitch-users] RedHat 6.X Performance Performance problems as load increases which are not found on other distros such as Debian. On Thu, Aug 30, 2012 at 7:17 PM, Colin Mason <cmason@frontiernetworks.ca<mailto:cmason@frontiernetworks.ca>> wrote: What exactly is the problem? From: freeswitch-users-bounces@lists.freeswitch.org<mailto:freeswitch-users-bounces@lists.freeswitch.org> [mailto:freeswitch-users-bounces@lists.freeswitch.org<mailto:freeswitch-users-bounces@lists.freeswitch.org>] On Behalf Of Guillermo Ruiz Camauer Sent: Thursday, August 30, 2012 6:13 PM To: FreeSWITCH Users Help Subject: Re: [Freeswitch-users] RedHat 6.X Performance Is anyone having the same problems with RedHat or is this CentOS specific? On Thu, Aug 30, 2012 at 6:23 PM, Gavin Henry <gavin.henry@gmail.com<mailto:gavin.henry@gmail.com>> wrote:
Centos5 worked well for me, Centos6 worked for crap... I went to debian 6 (squeeze) and problems went a way... using the latest stable on the debian6 line installing from the netinstall (non-free variant as I have hardware thats not supported under the main netinstall iso) and then installing freeswitch via the instructions on the wiki and no problems so far Same here. We're CentOS 5 here and Debian Squeezy for new boxes.
-- http://www.suretecsystems.com/services/openldap/ http://www.surevoip.co.uk _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org<mailto:consulting@freeswitch.org> http://www.freeswitchsolutions.com FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org<mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org -- Guillermo Ruiz Camauer _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org<mailto:consulting@freeswitch.org> http://www.freeswitchsolutions.com FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org<mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org -- Guillermo Ruiz Camauer
We haven't had any issues either with Centos 6.3 kernel 2.6.32-279.el6.x86_64 Do you have a specific benchmark? On Thu, Aug 30, 2012 at 3:31 PM, Colin Mason <cmason@frontiernetworks.ca>wrote:
Never had any performance issues on CentOS 6 under load. Can you be more specific?****
** **
*From:* freeswitch-users-bounces@lists.freeswitch.org [mailto: freeswitch-users-bounces@lists.freeswitch.org] *On Behalf Of *Guillermo Ruiz Camauer *Sent:* Thursday, August 30, 2012 6:24 PM *To:* FreeSWITCH Users Help *Subject:* Re: [Freeswitch-users] RedHat 6.X Performance****
** **
Performance problems as load increases which are not found on other distros such as Debian.****
On Thu, Aug 30, 2012 at 7:17 PM, Colin Mason <cmason@frontiernetworks.ca> wrote:****
What exactly is the problem?****
****
*From:* freeswitch-users-bounces@lists.freeswitch.org [mailto: freeswitch-users-bounces@lists.freeswitch.org] *On Behalf Of *Guillermo Ruiz Camauer *Sent:* Thursday, August 30, 2012 6:13 PM *To:* FreeSWITCH Users Help *Subject:* Re: [Freeswitch-users] RedHat 6.X Performance****
****
Is anyone having the same problems with RedHat or is this CentOS specific? ****
****
On Thu, Aug 30, 2012 at 6:23 PM, Gavin Henry <gavin.henry@gmail.com> wrote:****
Centos5 worked well for me, Centos6 worked for crap... I went to debian 6 (squeeze) and problems went a way... using the latest stable on the debian6 line installing from the netinstall (non-free variant as I have hardware thats not supported under the main netinstall iso) and then installing freeswitch via the instructions on the wiki and no problems so far****
Same here. We're CentOS 5 here and Debian Squeezy for new boxes.
-- http://www.suretecsystems.com/services/openldap/ http://www.surevoip.co.uk****
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org****
****
****
-- Guillermo Ruiz Camauer****
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org****
****
** **
-- Guillermo Ruiz Camauer****
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
At least on one of our boxes, running on CentOS 6.2 ends up with one FS thread stuck at full CPU. We haven't noticed anything getting worse as more load gets added. My feeling is that perhaps there's some sleep or something that is now much more granular with a tickless kernel so now it's waking up all the time, instead of 1000Hz. But several folks have told me tickles ain't the issue. At any rate, machines are big and profitable, so losing a single core to a runaway thread isn't a huge priority for me :\. -Michael From: freeswitch-users-bounces@lists.freeswitch.org [mailto:freeswitch-users-bounces@lists.freeswitch.org] On Behalf Of Colin Mason Sent: Thursday, August 30, 2012 4:17 PM To: FreeSWITCH Users Help Subject: Re: [Freeswitch-users] RedHat 6.X Performance What exactly is the problem? From: freeswitch-users-bounces@lists.freeswitch.org<mailto:freeswitch-users-bounces@lists.freeswitch.org> [mailto:freeswitch-users-bounces@lists.freeswitch.org]<mailto:[mailto:freeswitch-users-bounces@lists.freeswitch.org]> On Behalf Of Guillermo Ruiz Camauer Sent: Thursday, August 30, 2012 6:13 PM To: FreeSWITCH Users Help Subject: Re: [Freeswitch-users] RedHat 6.X Performance Is anyone having the same problems with RedHat or is this CentOS specific? On Thu, Aug 30, 2012 at 6:23 PM, Gavin Henry <gavin.henry@gmail.com<mailto:gavin.henry@gmail.com>> wrote:
Centos5 worked well for me, Centos6 worked for crap... I went to debian 6 (squeeze) and problems went a way... using the latest stable on the debian6 line installing from the netinstall (non-free variant as I have hardware thats not supported under the main netinstall iso) and then installing freeswitch via the instructions on the wiki and no problems so far Same here. We're CentOS 5 here and Debian Squeezy for new boxes.
-- http://www.suretecsystems.com/services/openldap/ http://www.surevoip.co.uk _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org<mailto:consulting@freeswitch.org> http://www.freeswitchsolutions.com FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org<mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org -- Guillermo Ruiz Camauer
I'm running probably one of the most stripped down FreeSWITCH configs you can run ... Sofia only, bypass media, with a custom C routing module that uses libpq directly... the problem happens at somewhere around 50 to 100 CPS, system % goes thru the roof, and its not IO.. loglevel 0... etc... now its also worth mentioning that on 6.2 the number of context switches for similar amounts of calls on the same physical hardware seems to be double vs something like cent5 or deb6.... we're currently trying to get people to submit reports from oprofile so we can isolate the issue... if it were just me, I would chalk it up to something specific on the installation, but , the number of reporters with similar observations is something to take notice of... On Thu, Aug 30, 2012 at 8:19 PM, Michael Giagnocavo <mgg@giagnocavo.net>wrote:
At least on one of our boxes, running on CentOS 6.2 ends up with one FS thread stuck at full CPU. We haven’t noticed anything getting worse as more load gets added. My feeling is that perhaps there’s some sleep or something that is now much more granular with a tickless kernel so now it’s waking up all the time, instead of 1000Hz. But several folks have told me tickles ain’t the issue. ****
** **
At any rate, machines are big and profitable, so losing a single core to a runaway thread isn’t a huge priority for me :\.****
** **
-Michael****
** **
*From:* freeswitch-users-bounces@lists.freeswitch.org [mailto: freeswitch-users-bounces@lists.freeswitch.org] *On Behalf Of *Colin Mason *Sent:* Thursday, August 30, 2012 4:17 PM
*To:* FreeSWITCH Users Help *Subject:* Re: [Freeswitch-users] RedHat 6.X Performance****
** **
What exactly is the problem?****
** **
*From:* freeswitch-users-bounces@lists.freeswitch.org [mailto:freeswitch-users-bounces@lists.freeswitch.org] *On Behalf Of *Guillermo Ruiz Camauer *Sent:* Thursday, August 30, 2012 6:13 PM *To:* FreeSWITCH Users Help *Subject:* Re: [Freeswitch-users] RedHat 6.X Performance****
** **
Is anyone having the same problems with RedHat or is this CentOS specific? ****
** **
On Thu, Aug 30, 2012 at 6:23 PM, Gavin Henry <gavin.henry@gmail.com> wrote:****
Centos5 worked well for me, Centos6 worked for crap... I went to debian 6 (squeeze) and problems went a way... using the latest stable on the debian6 line installing from the netinstall (non-free variant as I have hardware thats not supported under the main netinstall iso) and then installing freeswitch via the instructions on the wiki and no problems so far****
Same here. We're CentOS 5 here and Debian Squeezy for new boxes.
-- http://www.suretecsystems.com/services/openldap/ http://www.surevoip.co.uk****
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org****
****
** **
-- Guillermo Ruiz Camauer****
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
I do 250 cps and somewhere around 100000 sessions per day on a cent6 + fs1.2 install. The box has 8 cores and I never see it using more than 1 core. Is this worth oprofiling? If so, where/who should I send the oprofile to? Colin From: Ken Rice [mailto:krice@freeswitch.org] Sent: Thursday, August 30, 2012 11:45 PM To: FreeSWITCH Users Help <freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] RedHat 6.X Performance I'm running probably one of the most stripped down FreeSWITCH configs you can run ... Sofia only, bypass media, with a custom C routing module that uses libpq directly... the problem happens at somewhere around 50 to 100 CPS, system % goes thru the roof, and its not IO.. loglevel 0... etc... now its also worth mentioning that on 6.2 the number of context switches for similar amounts of calls on the same physical hardware seems to be double vs something like cent5 or deb6.... we're currently trying to get people to submit reports from oprofile so we can isolate the issue... if it were just me, I would chalk it up to something specific on the installation, but , the number of reporters with similar observations is something to take notice of... On Thu, Aug 30, 2012 at 8:19 PM, Michael Giagnocavo <mgg@giagnocavo.net<mailto:mgg@giagnocavo.net>> wrote: At least on one of our boxes, running on CentOS 6.2 ends up with one FS thread stuck at full CPU. We haven’t noticed anything getting worse as more load gets added. My feeling is that perhaps there’s some sleep or something that is now much more granular with a tickless kernel so now it’s waking up all the time, instead of 1000Hz. But several folks have told me tickles ain’t the issue. At any rate, machines are big and profitable, so losing a single core to a runaway thread isn’t a huge priority for me :\. -Michael From: freeswitch-users-bounces@lists.freeswitch.org<mailto:freeswitch-users-bounces@lists.freeswitch.org> [mailto:freeswitch-users-bounces@lists.freeswitch.org<mailto:freeswitch-users-bounces@lists.freeswitch.org>] On Behalf Of Colin Mason Sent: Thursday, August 30, 2012 4:17 PM To: FreeSWITCH Users Help Subject: Re: [Freeswitch-users] RedHat 6.X Performance What exactly is the problem? From: freeswitch-users-bounces@lists.freeswitch.org<mailto:freeswitch-users-bounces@lists.freeswitch.org> [mailto:freeswitch-users-bounces@lists.freeswitch.org]<mailto:[mailto:freeswitch-users-bounces@lists.freeswitch.org]> On Behalf Of Guillermo Ruiz Camauer Sent: Thursday, August 30, 2012 6:13 PM To: FreeSWITCH Users Help Subject: Re: [Freeswitch-users] RedHat 6.X Performance Is anyone having the same problems with RedHat or is this CentOS specific? On Thu, Aug 30, 2012 at 6:23 PM, Gavin Henry <gavin.henry@gmail.com<mailto:gavin.henry@gmail.com>> wrote:
Centos5 worked well for me, Centos6 worked for crap... I went to debian 6 (squeeze) and problems went a way... using the latest stable on the debian6 line installing from the netinstall (non-free variant as I have hardware thats not supported under the main netinstall iso) and then installing freeswitch via the instructions on the wiki and no problems so far Same here. We're CentOS 5 here and Debian Squeezy for new boxes.
-- http://www.suretecsystems.com/services/openldap/ http://www.surevoip.co.uk _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org<mailto:consulting@freeswitch.org> http://www.freeswitchsolutions.com FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org<mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org -- Guillermo Ruiz Camauer _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org<mailto:consulting@freeswitch.org> http://www.freeswitchsolutions.com FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org<mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
On 31-08-12 05:45, Ken Rice wrote:
the problem happens at somewhere around 50 to 100 CPS, system % goes thru the roof, and its not IO.. loglevel 0... etc... now its also worth mentioning that on 6.2 the number of context switches for similar amounts of calls on the same physical hardware seems to be double vs something like cent5 or deb6....
Is this also the case for 6.3 with all post-6.3 updates applied? Regards, Patrick
this was the case with all the updates applied at the last time i tried it ie: yum update -y On Fri, Aug 31, 2012 at 5:13 AM, Patrick Lists < freeswitch-list@puzzled.xs4all.nl> wrote:
On 31-08-12 05:45, Ken Rice wrote:
the problem happens at somewhere around 50 to 100 CPS, system % goes thru the roof, and its not IO.. loglevel 0... etc... now its also worth mentioning that on 6.2 the number of context switches for similar amounts of calls on the same physical hardware seems to be double vs something like cent5 or deb6....
Is this also the case for 6.3 with all post-6.3 updates applied?
Regards, Patrick
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Anymore info on specific traffic patterns that cause this? I've been beating up FS on 6.3 for several days now and haven't seen anything unusual yet... On Thu, Aug 30, 2012 at 11:45 PM, Ken Rice <krice@freeswitch.org> wrote:
I'm running probably one of the most stripped down FreeSWITCH configs you can run ... Sofia only, bypass media, with a custom C routing module that uses libpq directly...
the problem happens at somewhere around 50 to 100 CPS, system % goes thru the roof, and its not IO.. loglevel 0... etc... now its also worth mentioning that on 6.2 the number of context switches for similar amounts of calls on the same physical hardware seems to be double vs something like cent5 or deb6....
we're currently trying to get people to submit reports from oprofile so we can isolate the issue... if it were just me, I would chalk it up to something specific on the installation, but , the number of reporters with similar observations is something to take notice of...
-- Kristian Kielhofner
Hey Kris, Can you grab the oprofile reports from that box and send them to me offlist? K On 9/3/12 1:41 PM, "Kristian Kielhofner" <kris@kriskinc.com> wrote:
Anymore info on specific traffic patterns that cause this? I've been beating up FS on 6.3 for several days now and haven't seen anything unusual yet...
On Thu, Aug 30, 2012 at 11:45 PM, Ken Rice <krice@freeswitch.org> wrote:
I'm running probably one of the most stripped down FreeSWITCH configs you can run ... Sofia only, bypass media, with a custom C routing module that uses libpq directly...
the problem happens at somewhere around 50 to 100 CPS, system % goes thru the roof, and its not IO.. loglevel 0... etc... now its also worth mentioning that on 6.2 the number of context switches for similar amounts of calls on the same physical hardware seems to be double vs something like cent5 or deb6....
we're currently trying to get people to submit reports from oprofile so we can isolate the issue... if it were just me, I would chalk it up to something specific on the installation, but , the number of reporters with similar observations is something to take notice of...
-- Ken http://www.FreeSWITCH.org http://www.ClueCon.com http://www.OSTAG.org irc.freenode.net #freeswitch
I found someone with more info on scheduling problems, especially with sleep/wait, of which FS does a lot. There's a parameter called sched_migration_cost which might be worth investigating. http://pastebin.freeswitch.com/19903 On a benchmark for sleeps, he says RHEL6 is taking up to 45% more time. With the huge number of threads and sleeps FS does, maybe those little differences add up: http://unix.stackexchange.com/questions/37391/high-cpu-usage-with-cfs Separately, to everyone that hasn't rebooted their server in the last few months, you may still have the leap-second set which will cause lots of CPU spinning due to a kernel bug. Running date -s "`date`" should fix that. I am pretty sure that freed up at least one core on one of our boxes that was still spinning since Jul 1. We've now upgraded some systems to CentOS 6.3 (with all post release updates) and left others at 6.2. Over the next week we'll see if they seem to be much different. At the moment, the general CPU usage seems to be similar; we'll have to pound some dialer on it and see what happens. -Michael -----Original Message----- From: freeswitch-users-bounces@lists.freeswitch.org [mailto:freeswitch-users-bounces@lists.freeswitch.org] On Behalf Of Kristian Kielhofner Sent: Monday, September 03, 2012 12:42 PM To: FreeSWITCH Users Help Subject: Re: [Freeswitch-users] RedHat 6.X Performance Anymore info on specific traffic patterns that cause this? I've been beating up FS on 6.3 for several days now and haven't seen anything unusual yet... On Thu, Aug 30, 2012 at 11:45 PM, Ken Rice <krice@freeswitch.org> wrote:
I'm running probably one of the most stripped down FreeSWITCH configs you can run ... Sofia only, bypass media, with a custom C routing module that uses libpq directly...
the problem happens at somewhere around 50 to 100 CPS, system % goes thru the roof, and its not IO.. loglevel 0... etc... now its also worth mentioning that on 6.2 the number of context switches for similar amounts of calls on the same physical hardware seems to be double vs something like cent5 or deb6....
we're currently trying to get people to submit reports from oprofile so we can isolate the issue... if it were just me, I would chalk it up to something specific on the installation, but , the number of reporters with similar observations is something to take notice of...
-- Kristian Kielhofner _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Hi All: Does this performance problem still exists in CentOS 6.3? Thank you in advance. Thanks Lloyd * * * <http://www.sunteltech.ca/blog/>* On Fri, Sep 14, 2012 at 6:48 PM, Michael Giagnocavo <mgg@giagnocavo.net>wrote:
I found someone with more info on scheduling problems, especially with sleep/wait, of which FS does a lot. There's a parameter called sched_migration_cost which might be worth investigating.
http://pastebin.freeswitch.com/19903
On a benchmark for sleeps, he says RHEL6 is taking up to 45% more time. With the huge number of threads and sleeps FS does, maybe those little differences add up: http://unix.stackexchange.com/questions/37391/high-cpu-usage-with-cfs
Separately, to everyone that hasn't rebooted their server in the last few months, you may still have the leap-second set which will cause lots of CPU spinning due to a kernel bug. Running date -s "`date`" should fix that. I am pretty sure that freed up at least one core on one of our boxes that was still spinning since Jul 1.
We've now upgraded some systems to CentOS 6.3 (with all post release updates) and left others at 6.2. Over the next week we'll see if they seem to be much different. At the moment, the general CPU usage seems to be similar; we'll have to pound some dialer on it and see what happens.
-Michael
-----Original Message----- From: freeswitch-users-bounces@lists.freeswitch.org [mailto: freeswitch-users-bounces@lists.freeswitch.org] On Behalf Of Kristian Kielhofner Sent: Monday, September 03, 2012 12:42 PM To: FreeSWITCH Users Help Subject: Re: [Freeswitch-users] RedHat 6.X Performance
Anymore info on specific traffic patterns that cause this? I've been beating up FS on 6.3 for several days now and haven't seen anything unusual yet...
On Thu, Aug 30, 2012 at 11:45 PM, Ken Rice <krice@freeswitch.org> wrote:
I'm running probably one of the most stripped down FreeSWITCH configs you can run ... Sofia only, bypass media, with a custom C routing module that uses libpq directly...
the problem happens at somewhere around 50 to 100 CPS, system % goes thru the roof, and its not IO.. loglevel 0... etc... now its also worth mentioning that on 6.2 the number of context switches for similar amounts of calls on the same physical hardware seems to be double vs something like cent5 or deb6....
we're currently trying to get people to submit reports from oprofile so we can isolate the issue... if it were just me, I would chalk it up to something specific on the installation, but , the number of reporters with similar observations is something to take notice of...
-- Kristian Kielhofner
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
participants (13)
-
BookBag
-
Colin Mason
-
Daniel Eiland
-
Ed Kochanowski
-
Gavin Henry
-
Guillermo Ruiz Camauer
-
Ken Rice
-
Kristian Kielhofner
-
Lloyd Aloysius
-
Michael Collins
-
Michael Giagnocavo
-
Patrick Lists
-
Vik Killa