[Freeswitch-users] How to test Freeswitch before moving to production?

Wed Jan 23 21:09:28 MSK 2013

In general I think it's true, but maybe at lest some of the cases may be
covered by some automated scripts? Like knowing that version X of firmware
for Y brand phones requires certain header to be present or shorter than X
bytes or whatever could provide automated test case for all future versions.
Of course it won't cover new firmwares or new phones or some obscure cases
but maybe at least most common cases of regressions in code could be caught
using automated testing without phones.

Personally I don't believe in "hook some Arduino to VoIP phone and check if
some LED lights up" to be supported by open-source community. It might be
possible inside commercial company that would keep the stack of VoIP phones
and custom-made hardware tailored for those phones, but it won't leverage
community folks.

2013/1/23 Michael Collins <msc at freeswitch.org>

> After 4 years of working on the CudaTel team and hearing about SLA and BLF
> failures I can tell you that you need to see what's actually displayed on
> the phone itself, so checking the SIP traffic is not sufficient for a
> thorough test.
>
> If anyone is willing to crack open a phone and do some wiring they might
> be able to use a raspi or arduino and some components to monitor the phone.
> It's not the easiest thing in the world to do. That's why having a phone
> with an API would be so helpful. Autotesting would save enormous amounts of
> time in the long run.
>
> -MC
>
> On Wed, Jan 23, 2013 at 4:18 AM, Regis M <regis.freeswitch.org at tornad.net>wrote:
>
>> Maybe It's could be done with some sip session record that we can
>> playback to freeswitch ?
>>
>> In this mode, sofia will ignore or bypass time announce and sequence
>> number to anwer the correct/same thing as the first "real" recording
>> sequence.
>>
>> A sort of mod_sofia_test, one FS can read pcap or sip sequence and throw
>> mod_sofia_test "simulate" the endpoint to another or itself.
>> It could allow to record all phone session (register, makecall, holdcall,
>> transfer,...) and check FS still answer same things or corrects thing.
>>
>> A little hard maybe ? :)
>> I'm not a specialist of "sequence" or SIP change that occurs in a session
>> dialog between 2 calls from exactly the same device.
>>
>>
>>
>>
>>
>> 2013/1/23 Marcin Gozdalik <gozdal at gmail.com>
>>
>>> I'd doubt there is any VoIP phone with such support.
>>>
>>> But I'd guess writing tests querying actual phones wouldn't be much more
>>> work than:
>>> a) testing what kind of messages for BLF/MWI actually work on given phone
>>> b) testing (in ESL?) if FS sends good messages when certain actions are
>>> done (leaving voicemail or making a call)
>>>
>>>
>>>
>>>
>>> 2013/1/22 Michael Collins <msc at freeswitch.org>
>>>
>>>> That begs the question: does anyone know of any SIP phones that have an
>>>> API where you can poll the status of the lamps as well as what's on the
>>>> display, whether MWI is on, etc.? That would be epic.
>>>>
>>>> -MC
>>>>
>>>> On Mon, Jan 21, 2013 at 8:40 PM, Tim St. Pierre <
>>>> fs-list at communicatefreely.net> wrote:
>>>>
>>>>> Hmm.  Good idea.
>>>>>
>>>>> I know that you can do this with Aastra phones, and possibly others.
>>>>>
>>>>> With the Aastra's, there is an XML API that you can use to push
>>>>> notifications, apps, etc.
>>>>>   You can push XML to the phone on an HTTP interface (there is an ACL
>>>>> to tell it where to
>>>>> allow it from), and you can initiate phone calls, as well as hang up,
>>>>> transfer, etc.
>>>>> There is also an action URI, so you can have the phone report what is
>>>>> coming in on caller
>>>>> ID, when it is ringing, hung up, etc.
>>>>>
>>>>> I don't know if you can get lamp status from this or not, but perhaps
>>>>> you could have a
>>>>> system where a pattern of calls is generated, and after many
>>>>> repetitions, the system calls
>>>>> or a human to check and see that the lights match an expected state.
>>>>>
>>>>> Just an idea anyway.
>>>>>
>>>>> -Tim
>>>>>
>>>>> Michael Collins wrote:
>>>>> > This test suite does not presently exist. I would love to see some
>>>>> work
>>>>> > done in this area.
>>>>> >
>>>>> > One challenge is that there are numerous kinds of devices that can
>>>>> > interop with FS so the test suite will need to have some way to
>>>>> handle
>>>>> > the various phone makes and models. Also, I don't know which hard
>>>>> > phones, if any, support automated testing. I would *love* to see some
>>>>> > sort of web/network API on various phones where you could poll the
>>>>> phone
>>>>> > to see the state of each SLA/BLF key as well as what's in the
>>>>> display.
>>>>> > That would make automated testing much cleaner. As it stands now,
>>>>> > testing BLFs is a tedious process of watching phones and waiting to
>>>>> see
>>>>> > if the lights do what they should be doing.
>>>>> >
>>>>> > If anyone has practical suggestions (nothing pie-in-the-sky, if you
>>>>> get
>>>>> > my meaning) then by all means share with the group.
>>>>> >
>>>>> > -MC
>>>>> >
>>>>> > On Mon, Jan 21, 2013 at 10:03 AM, Yehavi Bourvine
>>>>> > <yehavi.bourvine at gmail.com <mailto:yehavi.bourvine at gmail.com>>
>>>>> wrote:
>>>>> >
>>>>> >     Hello,
>>>>> >
>>>>> >       We are running version 1.0.6; we did not upgrade for a long
>>>>> time
>>>>> >     due to some problems (mainly stuck BLFs) with the newer versions
>>>>> >     which have been corrected a few months ago.
>>>>> >
>>>>> >     A few weeks ago I pulled the latest GIT and did functional
>>>>> testings
>>>>> >     on our development system; all was OK.
>>>>> >
>>>>> >     Yesterday I moved our production system to that version. After a
>>>>> few
>>>>> >     hours the problems started: stuck BLFs, hangups, LUA scripts
>>>>> >     sometimes do not run. I had to revert back to the old version.
>>>>> >
>>>>> >     My question: Is there some simple test suite which I can test
>>>>> >     various scnarios on my development system? like, "call XX, while
>>>>> >     this call is active call YY which is monitored by XX and hangup",
>>>>> >     etc. It is hard to cover all cases manually, so I would like
>>>>> >     something automated.
>>>>> >
>>>>> >                        Thanks, __Yehavi:
>>>>> >
>>>>> >
>>>>> _________________________________________________________________________
>>>>> >     Professional FreeSWITCH Consulting Services:
>>>>> >     consulting at freeswitch.org <mailto:consulting at freeswitch.org>
>>>>> >     http://www.freeswitchsolutions.com
>>>>> >
>>>>> >     
>>>>> >     
>>>>> >
>>>>> >     Official FreeSWITCH Sites
>>>>> >     http://www.freeswitch.org
>>>>> >     http://wiki.freeswitch.org
>>>>> >     http://www.cluecon.com
>>>>> >
>>>>> >     FreeSWITCH-users mailing list
>>>>> >     FreeSWITCH-users at lists.freeswitch.org
>>>>> >     <mailto:FreeSWITCH-users at lists.freeswitch.org>
>>>>> >     http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> >     UNSUBSCRIBE:
>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> >     http://www.freeswitch.org
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> > --
>>>>> > Michael S Collins
>>>>> > Twitter: @mercutioviz
>>>>> > http://www.FreeSWITCH.org
>>>>> > http://www.ClueCon.com
>>>>> > http://www.OSTAG.org
>>>>> >
>>>>> >
>>>>> >
>>>>> ------------------------------------------------------------------------
>>>>> >
>>>>> >
>>>>> _________________________________________________________________________
>>>>> > Professional FreeSWITCH Consulting Services:
>>>>> > consulting at freeswitch.org
>>>>> > http://www.freeswitchsolutions.com
>>>>> >
>>>>> > 
>>>>> > 
>>>>> >
>>>>> > Official FreeSWITCH Sites
>>>>> > http://www.freeswitch.org
>>>>> > http://wiki.freeswitch.org
>>>>> > http://www.cluecon.com
>>>>> >
>>>>> > FreeSWITCH-users mailing list
>>>>> > FreeSWITCH-users at lists.freeswitch.org
>>>>> > http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> > UNSUBSCRIBE:
>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> > http://www.freeswitch.org
>>>>>
>>>>>
>>>>>
>>>>> _________________________________________________________________________
>>>>> Professional FreeSWITCH Consulting Services:
>>>>> consulting at freeswitch.org
>>>>> http://www.freeswitchsolutions.com
>>>>>
>>>>> 
>>>>> 
>>>>>
>>>>> Official FreeSWITCH Sites
>>>>> http://www.freeswitch.org
>>>>> http://wiki.freeswitch.org
>>>>> http://www.cluecon.com
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:
>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> http://www.freeswitch.org
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Michael S Collins
>>>> Twitter: @mercutioviz
>>>> http://www.FreeSWITCH.org
>>>> http://www.ClueCon.com
>>>> http://www.OSTAG.org
>>>>
>>>>
>>>>
>>>> _________________________________________________________________________
>>>> Professional FreeSWITCH Consulting Services:
>>>> consulting at freeswitch.org
>>>> http://www.freeswitchsolutions.com
>>>>
>>>> 
>>>> 
>>>>
>>>> Official FreeSWITCH Sites
>>>> http://www.freeswitch.org
>>>> http://wiki.freeswitch.org
>>>> http://www.cluecon.com
>>>>
>>>> FreeSWITCH-users mailing list
>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> http://www.freeswitch.org
>>>>
>>>>
>>>
>>>
>>> --
>>> Marcin Gozdalik
>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org
>>> http://www.freeswitchsolutions.com
>>>
>>> 
>>> 
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://wiki.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> 
>> 
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>>
>
>
> --
> Michael S Collins
> Twitter: @mercutioviz
> http://www.FreeSWITCH.org
> http://www.ClueCon.com
> http://www.OSTAG.org
>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> 
> 
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>

-- 
Marcin Gozdalik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20130123/cd284c39/attachment-0001.html 