[Freeswitch-users] mod_conference scalability

Fri Dec 18 11:14:33 PST 2009

I was evaluating the technologies available, and I thought you would be
interested in my results. However, almost every other reply I get from you
to my posts, rather than being helpful, has been hostile and insulting.

My scenario is not a hypothetical one of “having robots call the conference
in a way that probably does not match reality”. In fact, this will very much
reflect the reality of the application I’m building. Only instead of 300
listeners, I need to scale to over 2000 listeners minimum – per event, with
possibly more than one concurrent event. I want to pack as many listeners on
one server as I can. I’m trying to find a real solution to a real problem.

I work with other open source projects and fund enhancements or fixes I
need. FreeSWITCH would be no different. 

Brian.

From: Anthony Minessale [mailto:anthony.minessale at gmail.com] 
Sent: Friday, December 18, 2009 11:34 AM
To: freeswitch-users at lists.freeswitch.org
Subject: Re: [Freeswitch-users] mod_conference scalability

Conferencing is hardly the best place to judge performance.
Quality is a far more important goal to me in conferencing.

Lets compare who can do 48khz conferences with several 32k siren callers on
a polycom 6000, several more using G722 at 16khz and another handful of
people on g711 ulaw all at different rates and ptimes talking in near-real
time with low delay and low echo.  The fact that you can broadcast the
conferences to icecast, control it from an external application and play
files etc, and oh yeah, it can stream video.

Frankly, considering this is a free software project and so many people
benefit, i would rather focus on quality than what numbers i can get from
having robots call the conference in some way that probably does not match
reality.  I would love for someone to sponsor the effort to add features to
the conference module, but of course, I do not hold my breath, instead I
continue to improve it for free when I find time.  This is one of many
reasons I do not enjoy performance discussions unless I am talking to an
engineer who understands the code or a banker ready to pay for improvements.
That is not my way of saying pay me or forget it as you can clearly see the
conference module has made it to where it is today with no financial support
at all.  Just the efforts of myself and several brave volunteers over the
years who have contributed to it.

BTW,

We have a weekly call, there is one today in 30 minutes.
Drop by sip:888 at conference.freeswitch.org
<mailto:sip%3A888 at conference.freeswitch.org>  This is just an openVZ
instance mind you running at 48khz waiting for anyone to call in and say hi.

On Fri, Dec 18, 2009 at 10:12 AM, François Delawarde
<fdelawarde at wirelessmundi.com> wrote:

Hearing that Asterisk (1.4) scales 2x like FS is not common, sounds like
a configuration error.

If not, I already see the title of the next Digium blog entry:
"FreeSwitch scalability myth finally ends: The worst Asterisk version
ever (1.4) beating the crap of the best and latest FS."

Anyway, you should compare FS trunk to Asterisk 1.6.2 to see who wins
the final conference battle! :-)

François.

On Thu, 2009-12-17 at 16:41 -0500, Brian wrote:
> I did a test with the trunk version for the one conference case, and
> it is the same results as for 1.0.4. The audio failed at around 300
> listeners. Oddly though, it consumed less %CPU (240% instead of 300%),
> and yet the audio still failed at the same number of listeners.
>
>
>
> Brian.
>
>
>
> From: Anthony Minessale [mailto:anthony.minessale at gmail.com]
> Sent: Thursday, December 17, 2009 3:49 PM
> To: freeswitch-users at lists.freeswitch.org
> Subject: Re: [Freeswitch-users] mod_conference scalability
>
>
>
>
> We didn't post it anywhere but we just get overwhelmed with them and
> many of them are unfounded and take up a lot of time to track down.
> That does not mean you have not found a real problem but the first
> step is trying trunk.
>
>
>
>
> On Thu, Dec 17, 2009 at 2:32 PM, Brian <brian at proximosystems.com>
> wrote:
>
> I didn’t realize there was a policy about load testing questions. What
> forum should I have used for this?
>
>
>
> I didn’t get the chance to test on FS trunk yet, but when I do I will
> provide you with the feedback when I do. Just let me know what forum
> to use for this topic from now on.
>
>
>
> Thanks,
>
>
>
> Brian.
>
>
>
> From: Anthony Minessale [mailto:anthony.minessale at gmail.com]
> Sent: Thursday, December 17, 2009 2:42 PM
>
>
> To: freeswitch-users at lists.freeswitch.org
> Subject: Re: [Freeswitch-users] mod_conference scalability
>
>
>
>
> One man's stable release is another man's 6 month old release with
> hundreds of known fixed bugs.
> If one of the core developers tells you to try it, you may as well
> take the time to try it now that you have opened a forum questioning
> the scalability.
>
> When you tested asterisk did you actually use 600 phones and verify
> that each one can hear the audio perfectly and in time with what the
> speaker was saying?  Did you try same on FS?
>
> Did you optimize your dialplan on FS to deal with a load test or
> follow any of the recommended performance tuning page.
>
> All of the answers to these questions are really moot because we have
> a policy against entertaining load testing questions but if you like
> asterisk, by all means, use it, and good luck to you if those numbers
> you are testing at are what you plan to put in real
> production.........
>
> On Thu, Dec 17, 2009 at 1:29 PM, Brian <brian at proximosystems.com>
> wrote:
>
> Hi Mike,
>
>
>
> I didn’t get around to testing on the FreeSWITCH trunk yet. Are there
> substantial fixes to mod_conference in the FreeSWITCH trunk that might
> increase capacity for my scenario of one speaker and many listeners?
> If I want to put this into a production environment, I would need a
> stable version, which as far as I know is the 1.0.4 version.
>
>
>
> However, I did test on Asterisk 1.4 using app_conference, and doing
> the same scenario was able to get 1 speaker and 600 listeners on a
> single conference with no audio issues. The CPU at that point was just
> over 300%, same as where the single conference scenario failed on
> FreeSWITCH with 300 listeners.  I was able to push it to over 700
> listeners before I reached 400% CPU usage (I guess maxing out my
> quad-core processors), and asterisk finally crashed. But up until that
> point, there were no audio problems.
>
>
>
> I’ve read a lot about how FreeSWITCH is supposed to be more scalable
> than Asterisk, but unless there is something wrong with my FreeSWITCH
> setup, Asterisk was clearly the winner in this test – more than
> doubling FreeSWITCH capacity in this case. Again, maybe there is
> something on the FreeSWITCH side that I’m doing wrong, but I don’t see
> what it could be.
>
>
>
> Brian.
>
>
>
>
>
> From: Michael Jerris [mailto:mike at jerris.com]
> Sent: Thursday, December 17, 2009 10:18 AM
> To: freeswitch-users at lists.freeswitch.org
> Subject: Re: [Freeswitch-users] mod_conference scalability
>
>
>
>
> I would be curious what the same tests produce with svn trunk of
> FreeSWITCH.
>
>
>
>
> Mike
>
>
>
>
> On Dec 16, 2009, at 4:49 PM, Brian wrote:
>
>
>
>
> Hi,
>
>
>
>
>
> I’m new to FreeSWITCH and I’m testing the scalability of
> mod_conference to see if it will scale better that other solutions. My
> scenario is to have one speaker, and many listeners (mute). Since I
> have only one speaker, I was expecting this to scale well because
> there is no audio mixing required, just send each frame of the single
> speaker to each listener. Unfortunately, my testing was disappointing,
> and it didn’t scale nearly as well as I’d hoped (based on what I’ve
> read on how FreeSWITCH is supposed to be generally very scalable).
>
>
>
>
>
> Here’s my server setup is this:
>
>
>
>
>
> FreeSWITCH 1.0.4, 64 bit CentOS 5.3, on a quad-core Xeon server, 4 Gig
> of RAM. I’ve set file logging to “notice” level. My conference profile
> is configured to suppress several events, hoping that it would improve
> performance.
>
>
>
>
>
> Here are a few scenarios I tested, and roughly where I reached the
> point of audio failure on the conferences:
>
>
>
>
>
> Scenario 1:
>
>
> 1 conference, 1 speaker, audio failed at approx 300 listeners (mute)
>
>
>
>
>
> Scenario 2:
>
>
> 4 conferences, 1 speaker per conference, audio failed approx 110
> listeners per conference (so just over 400 total channels on the
> system).
>
>
>
>
>
> Scenario 3:
>
>
> 16 conferences, 1 speaker per conference, audio failed at 32 listeners
> per conference (so just over 500 total channels on the system).
>
>
>
>
>
>
>
>
> Looking at the output from “top”, it seems that in all 3 scenarios,
> the audio quality failed when the % CPU for the FreeSWITCH process
> exceeded 300%.
>
>
>
>
>
> I was hoping maybe someone else might have done similar testing, or
> maybe has suggestions on how to improve the performance. Or perhaps an
> alternate solution to the one speaker, many listener case?
>
>
>
>
>
> Thanks,
>
>
>
>
>
> Brian.
>
>
>
>
>
> _______________________________________________
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>
>
>
>
>
> _______________________________________________
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>
>
>
>
> --
> Anthony Minessale II
>
> FreeSWITCH http://www.freeswitch.org/
> ClueCon http://www.cluecon.com/
> Twitter: http://twitter.com/FreeSWITCH_wire
>
> AIM: anthm
> MSN:anthony_minessale at hotmail.com
<mailto:MSN%3Aanthony_minessale at hotmail.com> 
> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
<mailto:PAYPAL%3Aanthony.minessale at gmail.com> 
> IRC: irc.freenode.net #freeswitch
>
> FreeSWITCH Developer Conference
> sip:888 at conference.freeswitch.org
<mailto:sip%3A888 at conference.freeswitch.org> 
> iax:guest at conference.freeswitch.org/888
> googletalk:conf+888 at conference.freeswitch.org
<mailto:googletalk%3Aconf%2B888 at conference.freeswitch.org> 
> pstn:+19193869900
>
>
>
> _______________________________________________
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>
>
>
>
> --
> Anthony Minessale II
>
> FreeSWITCH http://www.freeswitch.org/
> ClueCon http://www.cluecon.com/
> Twitter: http://twitter.com/FreeSWITCH_wire
>
> AIM: anthm
> MSN:anthony_minessale at hotmail.com
<mailto:MSN%3Aanthony_minessale at hotmail.com> 
> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
<mailto:PAYPAL%3Aanthony.minessale at gmail.com> 
> IRC: irc.freenode.net #freeswitch
>
> FreeSWITCH Developer Conference
> sip:888 at conference.freeswitch.org
<mailto:sip%3A888 at conference.freeswitch.org> 
> iax:guest at conference.freeswitch.org/888
> googletalk:conf+888 at conference.freeswitch.org
<mailto:googletalk%3Aconf%2B888 at conference.freeswitch.org> 
> pstn:+19193869900
>
>
> _______________________________________________
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org

_______________________________________________
FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org

-- 
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com
<mailto:MSN%3Aanthony_minessale at hotmail.com> 
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
<mailto:PAYPAL%3Aanthony.minessale at gmail.com> 
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org
<mailto:sip%3A888 at conference.freeswitch.org> 
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org
<mailto:googletalk%3Aconf%2B888 at conference.freeswitch.org> 
pstn:+19193869900

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20091218/a880f3d8/attachment-0002.html 