[Freeswitch-users] How to test Freeswitch before moving to production?

Avi Marcus avi at avimarcus.net
Wed Jan 23 22:14:35 MSK 2013 

Taking a step back -- much of this can be monitored simply by watching FS
events, especially if we have dummy/cooperative endpoints for testing.

E.g. when you run "originate $long_bridge_with_vars" does a new channel get
started? etc.

Individual phone compliance is a different issue.

-Avi

On Wed, Jan 23, 2013 at 8:59 PM, Michael Collins <msc at freeswitch.org> wrote:

> Well, there's a difference between testing to certify as compliant and
> testing to see if the latest version of FS breaks something. In any event,
> I think Avi's suggestion about a script (or whatever) that runs while a
> human sits and watches a phone. It could even be interactive:
>
> What extension number is the test phone?
> >2000
>
> Sending call to 2000. Is the phone ringing? (Y/N)
> >Y
>
> Is the line key blinking? (Y/N)
> >Y
>
> Interactivity is not a requirement, just an option. It could even just
> spit out the name of the test that it's running and offer some information
> about what it thinks should be happening on the phone at the moment.
>
> -MC
>
>
> On Wed, Jan 23, 2013 at 10:32 AM, João Mesquita <jmesquita at freeswitch.org>wrote:
>
>> Mc, agree to disagree. That is what is so wonderful in this brave new
>> world of Ngn telephony of ours. If you limit the number of devices you
>> declare compatible with your platform and have the money to test each and
>> every phone of that list on every and each firmware it has, your thorough
>> test would be feasible and ONLY If the list of devices is small.
>>
>> But if you have a long list of devices you're screwed and instead of
>> developing the product itself you'll spend your time testing phones or even
>> worse, making patches for each model that works differently.
>>
>> I guess that all there is to do is stick to standard compliance and
>> report to manufacturers when thy are not so they can fix their crap. Even
>> that is not enough, I know, because there is always a customer saying: "
>> but it works with Asterisk...."
>>
>> Anyhow, a sip compliance test would be awesome but really hard to
>> accomplish correctly because it implies almost implementing another stack
>> (kinda).
>>
>> Sent from my iPhone
>>
>> On Jan 23, 2013, at 2:25 PM, Michael Collins <msc at freeswitch.org> wrote:
>>
>> After 4 years of working on the CudaTel team and hearing about SLA and
>> BLF failures I can tell you that you need to see what's actually displayed
>> on the phone itself, so checking the SIP traffic is not sufficient for a
>> thorough test.
>>
>> If anyone is willing to crack open a phone and do some wiring they might
>> be able to use a raspi or arduino and some components to monitor the phone.
>> It's not the easiest thing in the world to do. That's why having a phone
>> with an API would be so helpful. Autotesting would save enormous amounts of
>> time in the long run.
>>
>> -MC
>>
>> On Wed, Jan 23, 2013 at 4:18 AM, Regis M <regis.freeswitch.org at tornad.net
>> > wrote:
>>
>>> Maybe It's could be done with some sip session record that we can
>>> playback to freeswitch ?
>>>
>>> In this mode, sofia will ignore or bypass time announce and sequence
>>> number to anwer the correct/same thing as the first "real" recording
>>> sequence.
>>>
>>> A sort of mod_sofia_test, one FS can read pcap or sip sequence and throw
>>> mod_sofia_test "simulate" the endpoint to another or itself.
>>> It could allow to record all phone session (register, makecall,
>>> holdcall, transfer,...) and check FS still answer same things or corrects
>>> thing.
>>>
>>> A little hard maybe ? :)
>>> I'm not a specialist of "sequence" or SIP change that occurs in a
>>> session dialog between 2 calls from exactly the same device.
>>>
>>>
>>>
>>>
>>>
>>> 2013/1/23 Marcin Gozdalik <gozdal at gmail.com>
>>>
>>>> I'd doubt there is any VoIP phone with such support.
>>>>
>>>> But I'd guess writing tests querying actual phones wouldn't be much
>>>> more work than:
>>>> a) testing what kind of messages for BLF/MWI actually work on given
>>>> phone
>>>> b) testing (in ESL?) if FS sends good messages when certain actions are
>>>> done (leaving voicemail or making a call)
>>>>
>>>>
>>>>
>>>>
>>>> 2013/1/22 Michael Collins <msc at freeswitch.org>
>>>>
>>>>> That begs the question: does anyone know of any SIP phones that have
>>>>> an API where you can poll the status of the lamps as well as what's on the
>>>>> display, whether MWI is on, etc.? That would be epic.
>>>>>
>>>>> -MC
>>>>>
>>>>> On Mon, Jan 21, 2013 at 8:40 PM, Tim St. Pierre <
>>>>> fs-list at communicatefreely.net> wrote:
>>>>>
>>>>>> Hmm.  Good idea.
>>>>>>
>>>>>> I know that you can do this with Aastra phones, and possibly others.
>>>>>>
>>>>>> With the Aastra's, there is an XML API that you can use to push
>>>>>> notifications, apps, etc.
>>>>>>   You can push XML to the phone on an HTTP interface (there is an ACL
>>>>>> to tell it where to
>>>>>> allow it from), and you can initiate phone calls, as well as hang up,
>>>>>> transfer, etc.
>>>>>> There is also an action URI, so you can have the phone report what is
>>>>>> coming in on caller
>>>>>> ID, when it is ringing, hung up, etc.
>>>>>>
>>>>>> I don't know if you can get lamp status from this or not, but perhaps
>>>>>> you could have a
>>>>>> system where a pattern of calls is generated, and after many
>>>>>> repetitions, the system calls
>>>>>> or a human to check and see that the lights match an expected state.
>>>>>>
>>>>>> Just an idea anyway.
>>>>>>
>>>>>> -Tim
>>>>>>
>>>>>> Michael Collins wrote:
>>>>>> > This test suite does not presently exist. I would love to see some
>>>>>> work
>>>>>> > done in this area.
>>>>>> >
>>>>>> > One challenge is that there are numerous kinds of devices that can
>>>>>> > interop with FS so the test suite will need to have some way to
>>>>>> handle
>>>>>> > the various phone makes and models. Also, I don't know which hard
>>>>>> > phones, if any, support automated testing. I would *love* to see
>>>>>> some
>>>>>> > sort of web/network API on various phones where you could poll the
>>>>>> phone
>>>>>> > to see the state of each SLA/BLF key as well as what's in the
>>>>>> display.
>>>>>> > That would make automated testing much cleaner. As it stands now,
>>>>>> > testing BLFs is a tedious process of watching phones and waiting to
>>>>>> see
>>>>>> > if the lights do what they should be doing.
>>>>>> >
>>>>>> > If anyone has practical suggestions (nothing pie-in-the-sky, if you
>>>>>> get
>>>>>> > my meaning) then by all means share with the group.
>>>>>> >
>>>>>> > -MC
>>>>>> >
>>>>>> > On Mon, Jan 21, 2013 at 10:03 AM, Yehavi Bourvine
>>>>>> > <yehavi.bourvine at gmail.com <mailto:yehavi.bourvine at gmail.com>>
>>>>>> wrote:
>>>>>> >
>>>>>> >     Hello,
>>>>>> >
>>>>>> >       We are running version 1.0.6; we did not upgrade for a long
>>>>>> time
>>>>>> >     due to some problems (mainly stuck BLFs) with the newer versions
>>>>>> >     which have been corrected a few months ago.
>>>>>> >
>>>>>> >     A few weeks ago I pulled the latest GIT and did functional
>>>>>> testings
>>>>>> >     on our development system; all was OK.
>>>>>> >
>>>>>> >     Yesterday I moved our production system to that version. After
>>>>>> a few
>>>>>> >     hours the problems started: stuck BLFs, hangups, LUA scripts
>>>>>> >     sometimes do not run. I had to revert back to the old version.
>>>>>> >
>>>>>> >     My question: Is there some simple test suite which I can test
>>>>>> >     various scnarios on my development system? like, "call XX, while
>>>>>> >     this call is active call YY which is monitored by XX and
>>>>>> hangup",
>>>>>> >     etc. It is hard to cover all cases manually, so I would like
>>>>>> >     something automated.
>>>>>> >
>>>>>> >                        Thanks, __Yehavi:
>>>>>> >
>>>>>> >
>>>>>> _________________________________________________________________________
>>>>>> >     Professional FreeSWITCH Consulting Services:
>>>>>> >     consulting at freeswitch.org <mailto:consulting at freeswitch.org>
>>>>>> >     http://www.freeswitchsolutions.com
>>>>>> >
>>>>>> >     
>>>>>> >     
>>>>>> >
>>>>>> >     Official FreeSWITCH Sites
>>>>>> >     http://www.freeswitch.org
>>>>>> >     http://wiki.freeswitch.org
>>>>>> >     http://www.cluecon.com
>>>>>> >
>>>>>> >     FreeSWITCH-users mailing list
>>>>>> >     FreeSWITCH-users at lists.freeswitch.org
>>>>>> >     <mailto:FreeSWITCH-users at lists.freeswitch.org>
>>>>>> >     http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>> >     UNSUBSCRIBE:
>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>> >     http://www.freeswitch.org
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > --
>>>>>> > Michael S Collins
>>>>>> > Twitter: @mercutioviz
>>>>>> > http://www.FreeSWITCH.org
>>>>>> > http://www.ClueCon.com
>>>>>> > http://www.OSTAG.org
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> ------------------------------------------------------------------------
>>>>>> >
>>>>>> >
>>>>>> _________________________________________________________________________
>>>>>> > Professional FreeSWITCH Consulting Services:
>>>>>> > consulting at freeswitch.org
>>>>>> > http://www.freeswitchsolutions.com
>>>>>> >
>>>>>> > 
>>>>>> > 
>>>>>> >
>>>>>> > Official FreeSWITCH Sites
>>>>>> > http://www.freeswitch.org
>>>>>> > http://wiki.freeswitch.org
>>>>>> > http://www.cluecon.com
>>>>>> >
>>>>>> > FreeSWITCH-users mailing list
>>>>>> > FreeSWITCH-users at lists.freeswitch.org
>>>>>> > http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>> > UNSUBSCRIBE:
>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>> > http://www.freeswitch.org
>>>>>>
>>>>>>
>>>>>>
>>>>>> _________________________________________________________________________
>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>> consulting at freeswitch.org
>>>>>> http://www.freeswitchsolutions.com
>>>>>>
>>>>>> 
>>>>>> 
>>>>>>
>>>>>> Official FreeSWITCH Sites
>>>>>> http://www.freeswitch.org
>>>>>> http://wiki.freeswitch.org
>>>>>> http://www.cluecon.com
>>>>>>
>>>>>> FreeSWITCH-users mailing list
>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>> UNSUBSCRIBE:
>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>> http://www.freeswitch.org
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Michael S Collins
>>>>> Twitter: @mercutioviz
>>>>> http://www.FreeSWITCH.org
>>>>> http://www.ClueCon.com
>>>>> http://www.OSTAG.org
>>>>>
>>>>>
>>>>>
>>>>> _________________________________________________________________________
>>>>> Professional FreeSWITCH Consulting Services:
>>>>> consulting at freeswitch.org
>>>>> http://www.freeswitchsolutions.com
>>>>>
>>>>> 
>>>>> 
>>>>>
>>>>> Official FreeSWITCH Sites
>>>>> http://www.freeswitch.org
>>>>> http://wiki.freeswitch.org
>>>>> http://www.cluecon.com
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:
>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Marcin Gozdalik
>>>>
>>>>
>>>> _________________________________________________________________________
>>>> Professional FreeSWITCH Consulting Services:
>>>> consulting at freeswitch.org
>>>> http://www.freeswitchsolutions.com
>>>>
>>>> 
>>>> 
>>>>
>>>> Official FreeSWITCH Sites
>>>> http://www.freeswitch.org
>>>> http://wiki.freeswitch.org
>>>> http://www.cluecon.com
>>>>
>>>> FreeSWITCH-users mailing list
>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> http://www.freeswitch.org
>>>>
>>>>
>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org
>>> http://www.freeswitchsolutions.com
>>>
>>> 
>>> 
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://wiki.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>>
>>
>>
>> --
>> Michael S Collins
>> Twitter: @mercutioviz
>> http://www.FreeSWITCH.org
>> http://www.ClueCon.com
>> http://www.OSTAG.org
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> 
>> 
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> 
>> 
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>>
>
>
> --
> Michael S Collins
> Twitter: @mercutioviz
> http://www.FreeSWITCH.org
> http://www.ClueCon.com
> http://www.OSTAG.org
>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> 
> 
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20130123/9b0ba995/attachment-0001.html

Join us at ClueCon 2011 Aug 9-11, 2011
More information about the FreeSWITCH-users mailing list