[Freeswitch-users] How to test Freeswitch before moving to production?

Michael Collins msc at freeswitch.org
Wed Jan 23 21:59:51 MSK 2013


Well, there's a difference between testing to certify as compliant and
testing to see if the latest version of FS breaks something. In any event,
I think Avi's suggestion about a script (or whatever) that runs while a
human sits and watches a phone. It could even be interactive:

What extension number is the test phone?
>2000

Sending call to 2000. Is the phone ringing? (Y/N)
>Y

Is the line key blinking? (Y/N)
>Y

Interactivity is not a requirement, just an option. It could even just spit
out the name of the test that it's running and offer some information about
what it thinks should be happening on the phone at the moment.

-MC

On Wed, Jan 23, 2013 at 10:32 AM, João Mesquita <jmesquita at freeswitch.org>wrote:

> Mc, agree to disagree. That is what is so wonderful in this brave new
> world of Ngn telephony of ours. If you limit the number of devices you
> declare compatible with your platform and have the money to test each and
> every phone of that list on every and each firmware it has, your thorough
> test would be feasible and ONLY If the list of devices is small.
>
> But if you have a long list of devices you're screwed and instead of
> developing the product itself you'll spend your time testing phones or even
> worse, making patches for each model that works differently.
>
> I guess that all there is to do is stick to standard compliance and report
> to manufacturers when thy are not so they can fix their crap. Even that is
> not enough, I know, because there is always a customer saying: " but it
> works with Asterisk...."
>
> Anyhow, a sip compliance test would be awesome but really hard to
> accomplish correctly because it implies almost implementing another stack
> (kinda).
>
> Sent from my iPhone
>
> On Jan 23, 2013, at 2:25 PM, Michael Collins <msc at freeswitch.org> wrote:
>
> After 4 years of working on the CudaTel team and hearing about SLA and BLF
> failures I can tell you that you need to see what's actually displayed on
> the phone itself, so checking the SIP traffic is not sufficient for a
> thorough test.
>
> If anyone is willing to crack open a phone and do some wiring they might
> be able to use a raspi or arduino and some components to monitor the phone.
> It's not the easiest thing in the world to do. That's why having a phone
> with an API would be so helpful. Autotesting would save enormous amounts of
> time in the long run.
>
> -MC
>
> On Wed, Jan 23, 2013 at 4:18 AM, Regis M <regis.freeswitch.org at tornad.net>wrote:
>
>> Maybe It's could be done with some sip session record that we can
>> playback to freeswitch ?
>>
>> In this mode, sofia will ignore or bypass time announce and sequence
>> number to anwer the correct/same thing as the first "real" recording
>> sequence.
>>
>> A sort of mod_sofia_test, one FS can read pcap or sip sequence and throw
>> mod_sofia_test "simulate" the endpoint to another or itself.
>> It could allow to record all phone session (register, makecall, holdcall,
>> transfer,...) and check FS still answer same things or corrects thing.
>>
>> A little hard maybe ? :)
>> I'm not a specialist of "sequence" or SIP change that occurs in a session
>> dialog between 2 calls from exactly the same device.
>>
>>
>>
>>
>>
>> 2013/1/23 Marcin Gozdalik <gozdal at gmail.com>
>>
>>> I'd doubt there is any VoIP phone with such support.
>>>
>>> But I'd guess writing tests querying actual phones wouldn't be much more
>>> work than:
>>> a) testing what kind of messages for BLF/MWI actually work on given phone
>>> b) testing (in ESL?) if FS sends good messages when certain actions are
>>> done (leaving voicemail or making a call)
>>>
>>>
>>>
>>>
>>> 2013/1/22 Michael Collins <msc at freeswitch.org>
>>>
>>>> That begs the question: does anyone know of any SIP phones that have an
>>>> API where you can poll the status of the lamps as well as what's on the
>>>> display, whether MWI is on, etc.? That would be epic.
>>>>
>>>> -MC
>>>>
>>>> On Mon, Jan 21, 2013 at 8:40 PM, Tim St. Pierre <
>>>> fs-list at communicatefreely.net> wrote:
>>>>
>>>>> Hmm.  Good idea.
>>>>>
>>>>> I know that you can do this with Aastra phones, and possibly others.
>>>>>
>>>>> With the Aastra's, there is an XML API that you can use to push
>>>>> notifications, apps, etc.
>>>>>   You can push XML to the phone on an HTTP interface (there is an ACL
>>>>> to tell it where to
>>>>> allow it from), and you can initiate phone calls, as well as hang up,
>>>>> transfer, etc.
>>>>> There is also an action URI, so you can have the phone report what is
>>>>> coming in on caller
>>>>> ID, when it is ringing, hung up, etc.
>>>>>
>>>>> I don't know if you can get lamp status from this or not, but perhaps
>>>>> you could have a
>>>>> system where a pattern of calls is generated, and after many
>>>>> repetitions, the system calls
>>>>> or a human to check and see that the lights match an expected state.
>>>>>
>>>>> Just an idea anyway.
>>>>>
>>>>> -Tim
>>>>>
>>>>> Michael Collins wrote:
>>>>> > This test suite does not presently exist. I would love to see some
>>>>> work
>>>>> > done in this area.
>>>>> >
>>>>> > One challenge is that there are numerous kinds of devices that can
>>>>> > interop with FS so the test suite will need to have some way to
>>>>> handle
>>>>> > the various phone makes and models. Also, I don't know which hard
>>>>> > phones, if any, support automated testing. I would *love* to see some
>>>>> > sort of web/network API on various phones where you could poll the
>>>>> phone
>>>>> > to see the state of each SLA/BLF key as well as what's in the
>>>>> display.
>>>>> > That would make automated testing much cleaner. As it stands now,
>>>>> > testing BLFs is a tedious process of watching phones and waiting to
>>>>> see
>>>>> > if the lights do what they should be doing.
>>>>> >
>>>>> > If anyone has practical suggestions (nothing pie-in-the-sky, if you
>>>>> get
>>>>> > my meaning) then by all means share with the group.
>>>>> >
>>>>> > -MC
>>>>> >
>>>>> > On Mon, Jan 21, 2013 at 10:03 AM, Yehavi Bourvine
>>>>> > <yehavi.bourvine at gmail.com <mailto:yehavi.bourvine at gmail.com>>
>>>>> wrote:
>>>>> >
>>>>> >     Hello,
>>>>> >
>>>>> >       We are running version 1.0.6; we did not upgrade for a long
>>>>> time
>>>>> >     due to some problems (mainly stuck BLFs) with the newer versions
>>>>> >     which have been corrected a few months ago.
>>>>> >
>>>>> >     A few weeks ago I pulled the latest GIT and did functional
>>>>> testings
>>>>> >     on our development system; all was OK.
>>>>> >
>>>>> >     Yesterday I moved our production system to that version. After a
>>>>> few
>>>>> >     hours the problems started: stuck BLFs, hangups, LUA scripts
>>>>> >     sometimes do not run. I had to revert back to the old version.
>>>>> >
>>>>> >     My question: Is there some simple test suite which I can test
>>>>> >     various scnarios on my development system? like, "call XX, while
>>>>> >     this call is active call YY which is monitored by XX and hangup",
>>>>> >     etc. It is hard to cover all cases manually, so I would like
>>>>> >     something automated.
>>>>> >
>>>>> >                        Thanks, __Yehavi:
>>>>> >
>>>>> >
>>>>> _________________________________________________________________________
>>>>> >     Professional FreeSWITCH Consulting Services:
>>>>> >     consulting at freeswitch.org <mailto:consulting at freeswitch.org>
>>>>> >     http://www.freeswitchsolutions.com
>>>>> >
>>>>> >     
>>>>> >     
>>>>> >
>>>>> >     Official FreeSWITCH Sites
>>>>> >     http://www.freeswitch.org
>>>>> >     http://wiki.freeswitch.org
>>>>> >     http://www.cluecon.com
>>>>> >
>>>>> >     FreeSWITCH-users mailing list
>>>>> >     FreeSWITCH-users at lists.freeswitch.org
>>>>> >     <mailto:FreeSWITCH-users at lists.freeswitch.org>
>>>>> >     http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> >     UNSUBSCRIBE:
>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> >     http://www.freeswitch.org
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> > --
>>>>> > Michael S Collins
>>>>> > Twitter: @mercutioviz
>>>>> > http://www.FreeSWITCH.org
>>>>> > http://www.ClueCon.com
>>>>> > http://www.OSTAG.org
>>>>> >
>>>>> >
>>>>> >
>>>>> ------------------------------------------------------------------------
>>>>> >
>>>>> >
>>>>> _________________________________________________________________________
>>>>> > Professional FreeSWITCH Consulting Services:
>>>>> > consulting at freeswitch.org
>>>>> > http://www.freeswitchsolutions.com
>>>>> >
>>>>> > 
>>>>> > 
>>>>> >
>>>>> > Official FreeSWITCH Sites
>>>>> > http://www.freeswitch.org
>>>>> > http://wiki.freeswitch.org
>>>>> > http://www.cluecon.com
>>>>> >
>>>>> > FreeSWITCH-users mailing list
>>>>> > FreeSWITCH-users at lists.freeswitch.org
>>>>> > http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> > UNSUBSCRIBE:
>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> > http://www.freeswitch.org
>>>>>
>>>>>
>>>>>
>>>>> _________________________________________________________________________
>>>>> Professional FreeSWITCH Consulting Services:
>>>>> consulting at freeswitch.org
>>>>> http://www.freeswitchsolutions.com
>>>>>
>>>>> 
>>>>> 
>>>>>
>>>>> Official FreeSWITCH Sites
>>>>> http://www.freeswitch.org
>>>>> http://wiki.freeswitch.org
>>>>> http://www.cluecon.com
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:
>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> http://www.freeswitch.org
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Michael S Collins
>>>> Twitter: @mercutioviz
>>>> http://www.FreeSWITCH.org
>>>> http://www.ClueCon.com
>>>> http://www.OSTAG.org
>>>>
>>>>
>>>>
>>>> _________________________________________________________________________
>>>> Professional FreeSWITCH Consulting Services:
>>>> consulting at freeswitch.org
>>>> http://www.freeswitchsolutions.com
>>>>
>>>> 
>>>> 
>>>>
>>>> Official FreeSWITCH Sites
>>>> http://www.freeswitch.org
>>>> http://wiki.freeswitch.org
>>>> http://www.cluecon.com
>>>>
>>>> FreeSWITCH-users mailing list
>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> http://www.freeswitch.org
>>>>
>>>>
>>>
>>>
>>> --
>>> Marcin Gozdalik
>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org
>>> http://www.freeswitchsolutions.com
>>>
>>> 
>>> 
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://wiki.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> 
>> 
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>>
>
>
> --
> Michael S Collins
> Twitter: @mercutioviz
> http://www.FreeSWITCH.org
> http://www.ClueCon.com
> http://www.OSTAG.org
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> 
> 
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> 
> 
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>


-- 
Michael S Collins
Twitter: @mercutioviz
http://www.FreeSWITCH.org
http://www.ClueCon.com
http://www.OSTAG.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20130123/0732331c/attachment-0001.html 


Join us at ClueCon 2011 Aug 9-11, 2011
More information about the FreeSWITCH-users mailing list