[Freeswitch-users] How to test Freeswitch before moving to production?

Wed Jan 23 21:32:37 MSK 2013

Mc, agree to disagree. That is what is so wonderful in this brave new world of Ngn telephony of ours. If you limit the number of devices you declare compatible with your platform and have the money to test each and every phone of that list on every and each firmware it has, your thorough test would be feasible and ONLY If the list of devices is small. 

But if you have a long list of devices you're screwed and instead of developing the product itself you'll spend your time testing phones or even worse, making patches for each model that works differently. 

I guess that all there is to do is stick to standard compliance and report to manufacturers when thy are not so they can fix their crap. Even that is not enough, I know, because there is always a customer saying: " but it works with Asterisk...."

Anyhow, a sip compliance test would be awesome but really hard to accomplish correctly because it implies almost implementing another stack (kinda). 

Sent from my iPhone

On Jan 23, 2013, at 2:25 PM, Michael Collins <msc at freeswitch.org> wrote:

> After 4 years of working on the CudaTel team and hearing about SLA and BLF failures I can tell you that you need to see what's actually displayed on the phone itself, so checking the SIP traffic is not sufficient for a thorough test. 
> 
> If anyone is willing to crack open a phone and do some wiring they might be able to use a raspi or arduino and some components to monitor the phone. It's not the easiest thing in the world to do. That's why having a phone with an API would be so helpful. Autotesting would save enormous amounts of time in the long run. 
> 
> -MC
> 
> On Wed, Jan 23, 2013 at 4:18 AM, Regis M <regis.freeswitch.org at tornad.net> wrote:
>> Maybe It's could be done with some sip session record that we can playback to freeswitch ?
>> 
>> In this mode, sofia will ignore or bypass time announce and sequence number to anwer the correct/same thing as the first "real" recording sequence.
>> 
>> A sort of mod_sofia_test, one FS can read pcap or sip sequence and throw mod_sofia_test "simulate" the endpoint to another or itself.
>> It could allow to record all phone session (register, makecall, holdcall, transfer,...) and check FS still answer same things or corrects thing.
>> 
>> A little hard maybe ? :)
>> I'm not a specialist of "sequence" or SIP change that occurs in a session dialog between 2 calls from exactly the same device.
>> 
>> 
>> 
>> 
>> 
>> 2013/1/23 Marcin Gozdalik <gozdal at gmail.com>
>>> I'd doubt there is any VoIP phone with such support.
>>> 
>>> But I'd guess writing tests querying actual phones wouldn't be much more work than:
>>> a) testing what kind of messages for BLF/MWI actually work on given phone
>>> b) testing (in ESL?) if FS sends good messages when certain actions are done (leaving voicemail or making a call)
>>> 
>>> 
>>> 
>>> 
>>> 2013/1/22 Michael Collins <msc at freeswitch.org>
>>>> That begs the question: does anyone know of any SIP phones that have an API where you can poll the status of the lamps as well as what's on the display, whether MWI is on, etc.? That would be epic.
>>>> 
>>>> -MC
>>>> 
>>>> On Mon, Jan 21, 2013 at 8:40 PM, Tim St. Pierre <fs-list at communicatefreely.net> wrote:
>>>>> Hmm.  Good idea.
>>>>> 
>>>>> I know that you can do this with Aastra phones, and possibly others.
>>>>> 
>>>>> With the Aastra's, there is an XML API that you can use to push notifications, apps, etc.
>>>>>   You can push XML to the phone on an HTTP interface (there is an ACL to tell it where to
>>>>> allow it from), and you can initiate phone calls, as well as hang up, transfer, etc.
>>>>> There is also an action URI, so you can have the phone report what is coming in on caller
>>>>> ID, when it is ringing, hung up, etc.
>>>>> 
>>>>> I don't know if you can get lamp status from this or not, but perhaps you could have a
>>>>> system where a pattern of calls is generated, and after many repetitions, the system calls
>>>>> or a human to check and see that the lights match an expected state.
>>>>> 
>>>>> Just an idea anyway.
>>>>> 
>>>>> -Tim
>>>>> 
>>>>> Michael Collins wrote:
>>>>> > This test suite does not presently exist. I would love to see some work
>>>>> > done in this area.
>>>>> >
>>>>> > One challenge is that there are numerous kinds of devices that can
>>>>> > interop with FS so the test suite will need to have some way to handle
>>>>> > the various phone makes and models. Also, I don't know which hard
>>>>> > phones, if any, support automated testing. I would *love* to see some
>>>>> > sort of web/network API on various phones where you could poll the phone
>>>>> > to see the state of each SLA/BLF key as well as what's in the display.
>>>>> > That would make automated testing much cleaner. As it stands now,
>>>>> > testing BLFs is a tedious process of watching phones and waiting to see
>>>>> > if the lights do what they should be doing.
>>>>> >
>>>>> > If anyone has practical suggestions (nothing pie-in-the-sky, if you get
>>>>> > my meaning) then by all means share with the group.
>>>>> >
>>>>> > -MC
>>>>> >
>>>>> > On Mon, Jan 21, 2013 at 10:03 AM, Yehavi Bourvine
>>>>> > <yehavi.bourvine at gmail.com <mailto:yehavi.bourvine at gmail.com>> wrote:
>>>>> >
>>>>> >     Hello,
>>>>> >
>>>>> >       We are running version 1.0.6; we did not upgrade for a long time
>>>>> >     due to some problems (mainly stuck BLFs) with the newer versions
>>>>> >     which have been corrected a few months ago.
>>>>> >
>>>>> >     A few weeks ago I pulled the latest GIT and did functional testings
>>>>> >     on our development system; all was OK.
>>>>> >
>>>>> >     Yesterday I moved our production system to that version. After a few
>>>>> >     hours the problems started: stuck BLFs, hangups, LUA scripts
>>>>> >     sometimes do not run. I had to revert back to the old version.
>>>>> >
>>>>> >     My question: Is there some simple test suite which I can test
>>>>> >     various scnarios on my development system? like, "call XX, while
>>>>> >     this call is active call YY which is monitored by XX and hangup",
>>>>> >     etc. It is hard to cover all cases manually, so I would like
>>>>> >     something automated.
>>>>> >
>>>>> >                        Thanks, __Yehavi:
>>>>> >
>>>>> >     _________________________________________________________________________
>>>>> >     Professional FreeSWITCH Consulting Services:
>>>>> >     consulting at freeswitch.org <mailto:consulting at freeswitch.org>
>>>>> >     http://www.freeswitchsolutions.com
>>>>> >
>>>>> >     
>>>>> >     
>>>>> >
>>>>> >     Official FreeSWITCH Sites
>>>>> >     http://www.freeswitch.org
>>>>> >     http://wiki.freeswitch.org
>>>>> >     http://www.cluecon.com
>>>>> >
>>>>> >     FreeSWITCH-users mailing list
>>>>> >     FreeSWITCH-users at lists.freeswitch.org
>>>>> >     <mailto:FreeSWITCH-users at lists.freeswitch.org>
>>>>> >     http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> >     UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> >     http://www.freeswitch.org
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> > --
>>>>> > Michael S Collins
>>>>> > Twitter: @mercutioviz
>>>>> > http://www.FreeSWITCH.org
>>>>> > http://www.ClueCon.com
>>>>> > http://www.OSTAG.org
>>>>> >
>>>>> >
>>>>> > ------------------------------------------------------------------------
>>>>> >
>>>>> > _________________________________________________________________________
>>>>> > Professional FreeSWITCH Consulting Services:
>>>>> > consulting at freeswitch.org
>>>>> > http://www.freeswitchsolutions.com
>>>>> >
>>>>> > 
>>>>> > 
>>>>> >
>>>>> > Official FreeSWITCH Sites
>>>>> > http://www.freeswitch.org
>>>>> > http://wiki.freeswitch.org
>>>>> > http://www.cluecon.com
>>>>> >
>>>>> > FreeSWITCH-users mailing list
>>>>> > FreeSWITCH-users at lists.freeswitch.org
>>>>> > http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> > UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> > http://www.freeswitch.org
>>>>> 
>>>>> 
>>>>> _________________________________________________________________________
>>>>> Professional FreeSWITCH Consulting Services:
>>>>> consulting at freeswitch.org
>>>>> http://www.freeswitchsolutions.com
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> Official FreeSWITCH Sites
>>>>> http://www.freeswitch.org
>>>>> http://wiki.freeswitch.org
>>>>> http://www.cluecon.com
>>>>> 
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> http://www.freeswitch.org
>>>> 
>>>> 
>>>> 
>>>> -- 
>>>> Michael S Collins
>>>> Twitter: @mercutioviz
>>>> http://www.FreeSWITCH.org
>>>> http://www.ClueCon.com
>>>> http://www.OSTAG.org
>>>> 
>>>> 
>>>> _________________________________________________________________________
>>>> Professional FreeSWITCH Consulting Services:
>>>> consulting at freeswitch.org
>>>> http://www.freeswitchsolutions.com
>>>> 
>>>> 
>>>> 
>>>> 
>>>> Official FreeSWITCH Sites
>>>> http://www.freeswitch.org
>>>> http://wiki.freeswitch.org
>>>> http://www.cluecon.com
>>>> 
>>>> FreeSWITCH-users mailing list
>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> http://www.freeswitch.org
>>> 
>>> 
>>> 
>>> -- 
>>> Marcin Gozdalik
>>> 
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org
>>> http://www.freeswitchsolutions.com
>>> 
>>> 
>>> 
>>> 
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://wiki.freeswitch.org
>>> http://www.cluecon.com
>>> 
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>> 
>> 
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>> 
>> 
>> 
>> 
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>> 
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
> 
> 
> 
> -- 
> Michael S Collins
> Twitter: @mercutioviz
> http://www.FreeSWITCH.org
> http://www.ClueCon.com
> http://www.OSTAG.org
> 
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
> 
> 
> 
> 
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
> 
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20130123/b6f96796/attachment-0001.html 