What I would like to see is full and complete information that would allow someone to repeat the test and get similar enough results to say its the same. Specifically I would like to see: what operating system (include kernel version and libs resolved via ldd) what hardware (motherboard, cpu, if disk matters disk type, network card - and driver versions if seperate from the kernel for that stuff) sipp configs (uac.xml etc) FS configs (dialplan, sofia.conf, etc) FS version (svn tag is fine) FS modules loaded (even if they arent used, just in case) Basically enough so that it can not only be duplicated but the test itself be questioned by anyone to say its either a valid fair test or its invalid and *why* so that it can be repeated with those corrections. Benchmarking often has some criticism of the test method used, and the more we are open about the test conditions, methods, and procedures the more we can correct or defend against attacks on the testing itself, and then start to get to a way to test different platforms to see which actually work better and maybe isolate why some dont. I think these can go a long way towards unbiased benchmarks. On 10/6/07, Ken Rice <krice@suspicious.org> wrote:
Currently I am working out a test plan to determine several performance metrics independent of Tony and his team. If anyone has any suggestions on performace metrics and test tools you would like to see here drop me a note and let me know.
-- Trixter http://www.0xdecafbad.com Bret McDanel Belfast +44 28 9099 6461 US +1 516 687 5200 http://www.trxtel.com the phone company that pays you!