On Oct Thu 09 2008 16:36, Jon Bruel wrote:
I have been asked by Mark Crane and others to contribute more positively by joining the community, and I would be glad to do so. As with anyone I have my own agenda being to create a virtual PBX suite - using FS or what ever I find most suitable. At this stage, I have added a load testing page to the wiki, a modest first step to my contribution.
Thanks a lot for your contribution :)
A few words about my findings regarding load tests: On the surface the FS compares with the Asterisk: +/- 20% in terms of CPU load. The test may not credit the FS for possible better resilience. Knowing Asterisk I also know its instability under load, and with the architecture of the FS, this may be resolved by design, and I'm looking forward to verifying that. So I'll - in between the other tasks - continue with load tests which attempt to simulate real life situations, with complex dial plans and with erroneous internets. Ideas are welcome!
I am not a Load Test expert, but from an engineering point of view, I rather first go for a "baseline suite" that would perform load testing under the best conditions possible (that is, an average dial plan, and certainly not an "erroneous internet" --I suppose you mean a lossy connection :-?). This will give a reference result of the capabilities of FS (and any other soft switch/pbx that can run the tests). Then, you can begin to push FS to the limits, which will certainly test its resilience, architecture, etc. If you go first for the real world, worst-case scenario, you might not get an objective result. Imagine a test scenario such as the following: a web browser that is incredibly fast rendering well-formed XML pages (such as XHTML) but barks when rendering a malformed page. On the other hand, a crappy browser, say with respect to speed or standards, but that has a best-effort approach and manages to render, say 90% of a malformed page. The first is a really good, speedy, browser, while the other is a crappy but certainly resilient, best-effort, one. Broad, different results that have to be viewed in their own frame of reference. I would also try to approach the tests with an "atomic" point of view. That is, if you go for a worst-case, real life, dense-in-features, scenario you might find a multi-variable bug or whatever that might break FS and will be hard to track because the test is itself multivariable, etc. If you build tests by small, "atomic", steps, up to your real-life scenario, where you to hit a nasty performance bug (or any other bug by this matter) it will surely be much easier to track the reason triggering. Finally, it would be wonderful to end up having the foundations for a load test framework that would be able to test different switches / pbx capabilities, much like the existing testing frameworks. Perhaps a nice starting point would be http://en.wikipedia.org/wiki/Load_testing http://en.wikipedia.org/wiki/Performance_testing http://en.wikipedia.org/wiki/Reliability_testing http://en.wikipedia.org/wiki/Volume_testing http://en.wikipedia.org/wiki/Category:Load_testing_tools So these are my 2 cents :D and again, take all this with a grain of salt since IANALTE 8) Cheers, -- J. Javier Maestro <jjmaestro@nosys.es> Socio Consultor - Nosys AJjV S.L.