[Freeswitch-users] Auto test FS install - now necessary?

Fri Feb 4 19:03:56 MSK 2011

I understand well,
but I think everything is possible, even if it seems huge work
by the right method the huge work can be a reasonable work.
It needs only to reflect deeply of how to do this....
  ----- Original Message ----- 
  From: Steven Ayre 
  To: FreeSWITCH Users Help 
  Sent: Friday, February 04, 2011 10:45 AM
  Subject: Re: [Freeswitch-users] Auto test FS install - now necessary?

  http://en.wikipedia.org/wiki/Software_testing#Input_combinations_and_preconditions
  "A very fundamental problem with software testing is that testing under all combinations of inputs and preconditions (initial state) is not feasible, even with a simple product. This means that the number of defects in a software product can be very large and defects that occur infrequently are difficult to find in testing. More significantly, non-functional dimensions of quality (how it is supposed to be versus what it is supposed to do)—usability, scalability, performance, compatibility, reliability—can be highly subjective; something that constitutes sufficient value to one person may be intolerable to another.

  Software testing is a very difficult thing. Testing every possible combination of actions and specifically covering the edge cases most likely to cause problems is pretty much impossible - to run through every possible action for something as large as FS would literally take longer than the age of the universe. As a result a script to test for all possible errors is pretty much impossible, especially for new functionality that'll be added before the script's updated to test them. There will always be some bugs, users testing and reporting is the best way to detect ones that the developers have missed. Generally FS is pretty good at avoiding them.

  Testing yourself is always a good idea before rolling out to a production system. That'll show any obvious ones up that can be fixed quickly.

  -Steve

  On 4 February 2011 15:13, Madovsky <infos at madovsky.org> wrote:

    FS team needs user as users need FS team...
    maybe think to a script (developped by all possible users)
    that test every new functionality...

    ----- Original Message -----
    From: "Anthony Minessale" <anthony.minessale at gmail.com>
    To: "FreeSWITCH Users Help" <freeswitch-users at lists.freeswitch.org>
    Sent: Friday, February 04, 2011 9:58 AM
    Subject: Re: [Freeswitch-users] Auto test FS install - now necessary?

    > I've been doing this a long time and it boils down to:
    >
    > 50% superstition/paranoia.
    > 50% regressions from commits.
    >
    > Stop and think. Without people updating everyday and telling us when
    > something is not right, we may never find out.
    > Engaging in the community and this mailing list and irc is a community
    > effort and its sometimes a BETA program too.
    > Just like if this was a cool video game, you get to play it before
    > everyone else.
    > It's hard to follow every possible execution path and get into every
    > nook and cranny of 3 million lines of code.
    > Even with the best unit testing int the world, we can't do it without
    > real people testing.
    > If you stop updating or persuade others to stop, now we are in real
    > trouble.
    >
    > Also, pay attention to the GIT commit messages:
    >
    > This particular problem (which is fixed now btw) was committed with a
    > warning to please test.
    > There was a bigger problem earlier this week that resulted in some
    > minority invasive changes and its easy to catch the problems and in
    > the end the software is now more stable.
    >
    > commit f60fdf653dd2d7f8d3eaa6a9086e1f68bd993c59
    > Author: Anthony Minessale <anthm at freeswitch.org>
    > Date:   Wed Feb 2 16:22:43 2011 -0600
    >
    >    fix possible bad pointer in global vars (please test)
    >
    >
    >
    > On Thu, Feb 3, 2011 at 11:46 PM, Avi Marcus <avi at avimarcus.net> wrote:
    >> Maybe I'm just being paranoid, but I've been seeing a lot of "I just
    >> updated
    >> to latest git and X broke"
    >> And that x is always something else...
    >> I'm not a seasoned coder on this magnitude, but perhaps we can develop a
    >> test suite (at least part of one) to test that a fresh install is
    >> functioning?
    >> A FULL test would probably be a huge undertaking and involve recqual for
    >> testing media, too... Also, each mod would need to have it's own test
    >> suite.
    >> Due to the rapid updating of FreeSWITCH this not be what you want to put
    >> your effort into, but...
    >> Thoughts?
    >>
    >> On Feb 4, 2011 4:04 AM, "Anirudha Shimpi" <anu at familytv.com> wrote:
    >>> Just downloaded (from git) and installed FS 1.0.7, however, every time I
    >>> do
    >>> originate user/1001 &park, FS crashes. Running Centos 5.5 64 bit.
    >>>
    >>
    >> _______________________________________________
    >> FreeSWITCH-users mailing list
    >> FreeSWITCH-users at lists.freeswitch.org
    >> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
    >> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
    >> http://www.freeswitch.org
    >>
    >>
    >
    >
    >
    > --
    > Anthony Minessale II
    >
    > FreeSWITCH http://www.freeswitch.org/
    > ClueCon http://www.cluecon.com/
    > Twitter: http://twitter.com/FreeSWITCH_wire
    >
    > AIM: anthm
    > MSN:anthony_minessale at hotmail.com
    > GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
    > IRC: irc.freenode.net #freeswitch
    >
    > FreeSWITCH Developer Conference
    > sip:888 at conference.freeswitch.org
    > googletalk:conf+888 at conference.freeswitch.org
    > pstn:+19193869900
    >
    > _______________________________________________
    > FreeSWITCH-users mailing list
    > FreeSWITCH-users at lists.freeswitch.org
    > http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
    > UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
    > http://www.freeswitch.org

    _______________________________________________
    FreeSWITCH-users mailing list
    FreeSWITCH-users at lists.freeswitch.org
    http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
    UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
    http://www.freeswitch.org

------------------------------------------------------------------------------

  _______________________________________________
  FreeSWITCH-users mailing list
  FreeSWITCH-users at lists.freeswitch.org
  http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
  UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
  http://www.freeswitch.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20110204/a6155753/attachment.html 