[Freeswitch-users] mod_conference scalability

Fri Dec 18 05:04:51 PST 2009

Brian,

You haven't said what codecs are being used yet. Are the listeners
using a different codec to the speaker? If so, you're potentially
doing transcoding on every single channel, which would make CPU usage
skyrocket.

-Steve

2009/12/17 Anthony Minessale <anthony.minessale at gmail.com>:
> What exactly is your test process?
>
> you should try increasing the interval in the conference profile to a bigger
> time slice maybe 30 40 or 60ms
> you could also increase the ptime to match as well.
>
>
> like brian said you could use mod_shout to broadcast the single speaker to
> icecast and let people listen with itunes/winamp
>
>
> On Thu, Dec 17, 2009 at 3:41 PM, Brian <brian at proximosystems.com> wrote:
>>
>> I did a test with the trunk version for the one conference case, and it is
>> the same results as for 1.0.4. The audio failed at around 300 listeners.
>> Oddly though, it consumed less %CPU (240% instead of 300%), and yet the
>> audio still failed at the same number of listeners.
>>
>>
>>
>> Brian.
>>
>>
>>
>> From: Anthony Minessale [mailto:anthony.minessale at gmail.com]
>> Sent: Thursday, December 17, 2009 3:49 PM
>> To: freeswitch-users at lists.freeswitch.org
>> Subject: Re: [Freeswitch-users] mod_conference scalability
>>
>>
>>
>> We didn't post it anywhere but we just get overwhelmed with them and many
>> of them are unfounded and take up a lot of time to track down.  That does
>> not mean you have not found a real problem but the first step is trying
>> trunk.
>>
>>
>> On Thu, Dec 17, 2009 at 2:32 PM, Brian <brian at proximosystems.com> wrote:
>>
>> I didn’t realize there was a policy about load testing questions. What
>> forum should I have used for this?
>>
>>
>>
>> I didn’t get the chance to test on FS trunk yet, but when I do I will
>> provide you with the feedback when I do. Just let me know what forum to use
>> for this topic from now on.
>>
>>
>>
>> Thanks,
>>
>>
>>
>> Brian.
>>
>>
>>
>> From: Anthony Minessale [mailto:anthony.minessale at gmail.com]
>> Sent: Thursday, December 17, 2009 2:42 PM
>>
>> To: freeswitch-users at lists.freeswitch.org
>> Subject: Re: [Freeswitch-users] mod_conference scalability
>>
>>
>>
>> One man's stable release is another man's 6 month old release with
>> hundreds of known fixed bugs.
>> If one of the core developers tells you to try it, you may as well take
>> the time to try it now that you have opened a forum questioning the
>> scalability.
>>
>> When you tested asterisk did you actually use 600 phones and verify that
>> each one can hear the audio perfectly and in time with what the speaker was
>> saying?  Did you try same on FS?
>>
>> Did you optimize your dialplan on FS to deal with a load test or follow
>> any of the recommended performance tuning page.
>>
>> All of the answers to these questions are really moot because we have a
>> policy against entertaining load testing questions but if you like asterisk,
>> by all means, use it, and good luck to you if those numbers you are testing
>> at are what you plan to put in real production.........
>>
>> On Thu, Dec 17, 2009 at 1:29 PM, Brian <brian at proximosystems.com> wrote:
>>
>> Hi Mike,
>>
>>
>>
>> I didn’t get around to testing on the FreeSWITCH trunk yet. Are there
>> substantial fixes to mod_conference in the FreeSWITCH trunk that might
>> increase capacity for my scenario of one speaker and many listeners? If I
>> want to put this into a production environment, I would need a stable
>> version, which as far as I know is the 1.0.4 version.
>>
>>
>>
>> However, I did test on Asterisk 1.4 using app_conference, and doing the
>> same scenario was able to get 1 speaker and 600 listeners on a single
>> conference with no audio issues. The CPU at that point was just over 300%,
>> same as where the single conference scenario failed on FreeSWITCH with 300
>> listeners.  I was able to push it to over 700 listeners before I reached
>> 400% CPU usage (I guess maxing out my quad-core processors), and asterisk
>> finally crashed. But up until that point, there were no audio problems.
>>
>>
>>
>> I’ve read a lot about how FreeSWITCH is supposed to be more scalable than
>> Asterisk, but unless there is something wrong with my FreeSWITCH setup,
>> Asterisk was clearly the winner in this test – more than doubling FreeSWITCH
>> capacity in this case. Again, maybe there is something on the FreeSWITCH
>> side that I’m doing wrong, but I don’t see what it could be.
>>
>>
>>
>> Brian.
>>
>>
>>
>>
>>
>> From: Michael Jerris [mailto:mike at jerris.com]
>> Sent: Thursday, December 17, 2009 10:18 AM
>> To: freeswitch-users at lists.freeswitch.org
>> Subject: Re: [Freeswitch-users] mod_conference scalability
>>
>>
>>
>> I would be curious what the same tests produce with svn trunk of
>> FreeSWITCH.
>>
>>
>>
>> Mike
>>
>>
>>
>> On Dec 16, 2009, at 4:49 PM, Brian wrote:
>>
>>
>>
>> Hi,
>>
>>
>>
>> I’m new to FreeSWITCH and I’m testing the scalability of mod_conference to
>> see if it will scale better that other solutions. My scenario is to have one
>> speaker, and many listeners (mute). Since I have only one speaker, I was
>> expecting this to scale well because there is no audio mixing required, just
>> send each frame of the single speaker to each listener. Unfortunately, my
>> testing was disappointing, and it didn’t scale nearly as well as I’d hoped
>> (based on what I’ve read on how FreeSWITCH is supposed to be generally very
>> scalable).
>>
>>
>>
>> Here’s my server setup is this:
>>
>>
>>
>> FreeSWITCH 1.0.4, 64 bit CentOS 5.3, on a quad-core Xeon server, 4 Gig of
>> RAM. I’ve set file logging to “notice” level. My conference profile is
>> configured to suppress several events, hoping that it would improve
>> performance.
>>
>>
>>
>> Here are a few scenarios I tested, and roughly where I reached the point
>> of audio failure on the conferences:
>>
>>
>>
>> Scenario 1:
>>
>> 1 conference, 1 speaker, audio failed at approx 300 listeners (mute)
>>
>>
>>
>> Scenario 2:
>>
>> 4 conferences, 1 speaker per conference, audio failed approx 110 listeners
>> per conference (so just over 400 total channels on the system).
>>
>>
>>
>> Scenario 3:
>>
>> 16 conferences, 1 speaker per conference, audio failed at 32 listeners per
>> conference (so just over 500 total channels on the system).
>>
>>
>>
>>
>>
>> Looking at the output from “top”, it seems that in all 3 scenarios, the
>> audio quality failed when the % CPU for the FreeSWITCH process exceeded
>> 300%.
>>
>>
>>
>> I was hoping maybe someone else might have done similar testing, or maybe
>> has suggestions on how to improve the performance. Or perhaps an alternate
>> solution to the one speaker, many listener case?
>>
>>
>>
>> Thanks,
>>
>>
>>
>> Brian.
>>
>>
>>
>> _______________________________________________
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>>
>>
>> _______________________________________________
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>>
>> --
>> Anthony Minessale II
>>
>> FreeSWITCH http://www.freeswitch.org/
>> ClueCon http://www.cluecon.com/
>> Twitter: http://twitter.com/FreeSWITCH_wire
>>
>> AIM: anthm
>> MSN:anthony_minessale at hotmail.com
>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>> IRC: irc.freenode.net #freeswitch
>>
>> FreeSWITCH Developer Conference
>> sip:888 at conference.freeswitch.org
>> iax:guest at conference.freeswitch.org/888
>> googletalk:conf+888 at conference.freeswitch.org
>> pstn:+19193869900
>>
>> _______________________________________________
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>>
>> --
>> Anthony Minessale II
>>
>> FreeSWITCH http://www.freeswitch.org/
>> ClueCon http://www.cluecon.com/
>> Twitter: http://twitter.com/FreeSWITCH_wire
>>
>> AIM: anthm
>> MSN:anthony_minessale at hotmail.com
>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>> IRC: irc.freenode.net #freeswitch
>>
>> FreeSWITCH Developer Conference
>> sip:888 at conference.freeswitch.org
>> iax:guest at conference.freeswitch.org/888
>> googletalk:conf+888 at conference.freeswitch.org
>> pstn:+19193869900
>>
>> _______________________________________________
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>
>
>
> --
> Anthony Minessale II
>
> FreeSWITCH http://www.freeswitch.org/
> ClueCon http://www.cluecon.com/
> Twitter: http://twitter.com/FreeSWITCH_wire
>
> AIM: anthm
> MSN:anthony_minessale at hotmail.com
> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
> IRC: irc.freenode.net #freeswitch
>
> FreeSWITCH Developer Conference
> sip:888 at conference.freeswitch.org
> iax:guest at conference.freeswitch.org/888
> googletalk:conf+888 at conference.freeswitch.org
> pstn:+19193869900
>
> _______________________________________________
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>