http://en.wikipedia.org/wiki/Software_testing#Input_combinations_and_precond... "A very fundamental problem with software testing is that testing under all combinations of inputs and preconditions (initial state) is not feasible, even with a simple product. This means that the number of defects in a software product can be very large and defects that occur infrequently are difficult to find in testing. More significantly, non-functional dimensions of quality (how it is supposed to be versus what it is supposed to do)—usability, scalability, performance, compatibility, reliability—can be highly subjective; something that constitutes sufficient value to one person may be intolerable to another. Software testing is a very difficult thing. Testing every possible combination of actions and specifically covering the edge cases most likely to cause problems is pretty much impossible - to run through every possible action for something as large as FS would literally take longer than the age of the universe. As a result a script to test for all possible errors is pretty much impossible, especially for new functionality that'll be added before the script's updated to test them. There will always be some bugs, users testing and reporting is the best way to detect ones that the developers have missed. Generally FS is pretty good at avoiding them. Testing yourself is always a good idea before rolling out to a production system. That'll show any obvious ones up that can be fixed quickly. -Steve On 4 February 2011 15:13, Madovsky <infos@madovsky.org> wrote:
FS team needs user as users need FS team... maybe think to a script (developped by all possible users) that test every new functionality...
----- Original Message ----- From: "Anthony Minessale" <anthony.minessale@gmail.com> To: "FreeSWITCH Users Help" <freeswitch-users@lists.freeswitch.org> Sent: Friday, February 04, 2011 9:58 AM Subject: Re: [Freeswitch-users] Auto test FS install - now necessary?
I've been doing this a long time and it boils down to:
50% superstition/paranoia. 50% regressions from commits.
Stop and think. Without people updating everyday and telling us when something is not right, we may never find out. Engaging in the community and this mailing list and irc is a community effort and its sometimes a BETA program too. Just like if this was a cool video game, you get to play it before everyone else. It's hard to follow every possible execution path and get into every nook and cranny of 3 million lines of code. Even with the best unit testing int the world, we can't do it without real people testing. If you stop updating or persuade others to stop, now we are in real trouble.
Also, pay attention to the GIT commit messages:
This particular problem (which is fixed now btw) was committed with a warning to please test. There was a bigger problem earlier this week that resulted in some minority invasive changes and its easy to catch the problems and in the end the software is now more stable.
commit f60fdf653dd2d7f8d3eaa6a9086e1f68bd993c59 Author: Anthony Minessale <anthm@freeswitch.org> Date: Wed Feb 2 16:22:43 2011 -0600
fix possible bad pointer in global vars (please test)
On Thu, Feb 3, 2011 at 11:46 PM, Avi Marcus <avi@avimarcus.net> wrote:
Maybe I'm just being paranoid, but I've been seeing a lot of "I just updated to latest git and X broke" And that x is always something else... I'm not a seasoned coder on this magnitude, but perhaps we can develop a test suite (at least part of one) to test that a fresh install is functioning? A FULL test would probably be a huge undertaking and involve recqual for testing media, too... Also, each mod would need to have it's own test suite. Due to the rapid updating of FreeSWITCH this not be what you want to put your effort into, but... Thoughts?
On Feb 4, 2011 4:04 AM, "Anirudha Shimpi" <anu@familytv.com> wrote:
Just downloaded (from git) and installed FS 1.0.7, however, every time I do originate user/1001 &park, FS crashes. Running Centos 5.5 64 bit.
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm MSN:anthony_minessale@hotmail.com <MSN%3Aanthony_minessale@hotmail.com> GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com<PAYPAL%3Aanthony.minessale@gmail.com> IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference sip:888@conference.freeswitch.org <sip%3A888@conference.freeswitch.org> googletalk:conf+888@conference.freeswitch.org<googletalk%3Aconf%2B888@conference.freeswitch.org> pstn:+19193869900
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org