[Freeswitch-users] Freeswitch 1.4.15 and a sudden jump in CPU usage

Thu Feb 12 03:53:17 MSK 2015

Hi Dmitry,

Can you elaborate a bit please? Are you saying that it won't happen with
-np? Or it's just a suggestion?

What's the relationship?

Thanks,
Ali Pey

On Wed, Feb 11, 2015 at 6:30 PM, Dmitry Lysenko <dvl36.ripe.nick at gmail.com>
wrote:

> Hello guys,
>
> Try to reproduce this bug with normal priority ("-np") command line switch.
>
> Best Wishes,
>  Dmitry.
>
> 2015-02-11 22:54 GMT+02:00 Ali Pey <alipey at gmail.com>:
>
>> Thanks Tristan.
>>
>> I have had rtp-timer-name set to none for a long time but I still see the
>> sudden and random jumps in CPU.
>>
>> I haven't been able to reproduce it in our Dev/QA environment as well. If
>> I can reproduce it, I will definitely do the logs as well.
>>
>>
>>
>>
>>
>> On Wed, Feb 11, 2015 at 3:31 PM, Tristan Mahé <t.mahe at b-and-c.net> wrote:
>>
>>>  Hi Ali,
>>>
>>> What I see is kinda the same thing, I still need to provide Anthony with
>>> the debug logs ( still had no chance to do the requested tests on the
>>> production, could be related to some of our users UA/callflow and cannot
>>> reproduce the issue in a lab ). The interesting part as discussed in
>>> FS-7066 is the timer change is correcting the trouble, and without the
>>> timer change, we eventually experience blocking of sofia profile ( seems
>>> related to media playback ).
>>>
>>> I hope to have a chance soon to be able to furnish the debug logs, if
>>> you can, do the same, Anthony gave the steps to provide him some usefull
>>> logs on the bug comments.
>>>
>>>
>>> Le 11/02/2015 10:58, Ali Pey a écrit :
>>>
>>> Hi Tristan,
>>>
>>>  Do you see a sudden jump or just more CPU usage?
>>>
>>>  In my case, everything is good for a few days with less than %10 CPU
>>> usage, then there is a sudden jump to ~%98 CPU usage and it doesn't come
>>> down until a restart. After restart, it's good again with the same load.
>>>
>>>  Thanks,
>>> Ali Pey
>>>
>>>
>>> On Wed, Feb 11, 2015 at 1:40 PM, Tristan Mahé <t.mahe at b-and-c.net>
>>> wrote:
>>>
>>>> Maybe related to https://freeswitch.org/jira/browse/FS-7066 ?
>>>>
>>>> Le 11/02/2015 08:00, Giovanni Maruzzelli a écrit :
>>>>  > somebody has tested if it's a regression, eg: is 1.4.14 having
>>>> > same symptoms?
>>>> >
>>>> > -giovanni
>>>> >
>>>> > On Wed, Feb 11, 2015 at 4:57 PM, Moishe Grunstein <
>>>> max at nysolutions.com> wrote:
>>>> >> Why not a Jira?
>>>> >>
>>>> >> Thanks,
>>>> >>
>>>> >> Moishe Grunstein
>>>> >> Tornado Computer Systems, Inc.
>>>> >> 212.400.7650 888.IPPBX.US
>>>> >> Service Request Email: support at nysolutions.com
>>>> >>
>>>> >> Computer Networking * Managed Services * IP Video Surveillance *
>>>> Network Assessments * Web Solutions * Voice over IP * Disaster Recovery *
>>>> Network Security * Site Surveys * CMS
>>>> >>
>>>> >> -----Original Message-----
>>>> >> From: freeswitch-users-bounces at lists.freeswitch.org [mailto:
>>>> freeswitch-users-bounces at lists.freeswitch.org] On Behalf Of ??
>>>> >> Sent: Wednesday, February 11, 2015 10:49 AM
>>>> >> To: FreeSWITCH Users Help
>>>> >> Subject: Re: [Freeswitch-users] Freeswitch 1.4.15 and a sudden jump
>>>> in CPU usage
>>>> >>
>>>> >> i think it's bug of freeswitch:
>>>> >>
>>>> >> -- the Freeswitch using more than 1000% CPU, and there is no other
>>>> process eating CPU
>>>> >> -- we move all SIP traffic from the server away, the cpu is low. and
>>>> after sevral hours, if we move the SIP traffic back again, the FS eating
>>>> all CPU again. the only way to recovery is to restart the FS.
>>>> >> -- there is no DB such as mysql on the server, and we have add
>>>> "nosql"
>>>> >> option, no help
>>>> >>
>>>> >> On Wed, Feb 11, 2015 at 10:03 PM, John Nash <john.nash778 at gmail.com>
>>>> wrote:
>>>> >>> Dear Muhammad,
>>>> >>>
>>>> >>> I think your issue is different than Ali Pey because CPU rises at a
>>>> >>> specific time (approx) while running fine most of the time. In my
>>>> >>> experience you should check following ..
>>>> >>> - Check using top commands if its indeed freeswitch taking up CPU.
>>>> It
>>>> >>> may be logrotate or something.
>>>> >>> - If you are using DB on same server, also check DB connections, May
>>>> >>> be some long running query causing it (Like someone fires some
>>>> billing
>>>> >>> report at approx same time)
>>>> >>>
>>>> >>> On the other hand when you say "it runs fine all day". You may want
>>>> to
>>>> >>> check that using top command at different time of day, may be CPU
>>>> >>> usage keep on increasing and in your peak time it increases so much
>>>> that you see symptoms.
>>>> >>> (If this is true may be you and Ali pey have same issue perhaps some
>>>> >>> CPU leak).
>>>> >>>
>>>> >>> P.S. All points I mentioned are from my VOIP experiences in
>>>> general. I
>>>> >>> am NOT freeswitch expert. I have recently started experiments with
>>>> >>> freeswitch so people who are using freeswitch or experts will give
>>>> >>> more meaningful insight.
>>>> >>>
>>>> >>>
>>>> >>>
>>>> >>> On Wed, Feb 11, 2015 at 7:16 PM, 彭勇 <ppyy at pubyun.com> wrote:
>>>> >>>> we have add this option to grub and reboot one months ago. no help.
>>>> >>>>
>>>> >>>> we have media-pass-through  option, and have more than 10 thousands
>>>> >>>> sessions. it works fine all day and the CPU will rise suddenly
>>>> about
>>>> >>>> 9:00 am every day.  i should restart the FS to recovery.
>>>> >>>>
>>>> >>>> On Wed, Feb 11, 2015 at 9:25 PM, Muhammad Naseer Bhatti
>>>> >>>> <nbhatti at gmail.com> wrote:
>>>> >>>>> Try nohz=off reboot and see if that helps.
>>>> >>>>>
>>>> >>>>> Sent from my iPad
>>>> >>>>>
>>>> >>>>> On Feb 11, 2015, at 16:18, Aqs Younas <aqsyounas at gmail.com>
>>>> wrote:
>>>> >>>>>
>>>> >>>>> I am facing same issue. Freeswitch taking more than 100% cpu usage
>>>> >>>>> in htop even on 200 concurrent calls. For 1000 concurrent calls i
>>>> >>>>> have to install 3 freeswitch instances due to this much cpu usage
>>>> >>>>> taking by a single freeswitch node. My Dist is Debian 7.
>>>> >>>>>
>>>> >>>>> On 11 February 2015 at 17:35, Muhammad Naseer Bhatti
>>>> >>>>> <nbhatti at gmail.com>
>>>> >>>>> wrote:
>>>> >>>>>>
>>>> >>>>>> It looks like this is the same tickless kernel issue. I am using
>>>> >>>>>> Debian
>>>> >>>>>> 7
>>>> >>>>>> and the CPU usage shows very high. Adding nohz=off to grub
>>>> options
>>>> >>>>>> does makes it lower. I am gathering more information in order to
>>>> >>>>>> file a Jira.
>>>> >>>>>> Debian 6 on the other hand works very quietly on the same
>>>> >>>>>> workload. We are talking about 200+% CPU utilization on Debian 7
>>>> >>>>>> and around 50% on Debian 6 showing in htop. I know so many other
>>>> >>>>>> factors also involved but I am still digging out more.
>>>> >>>>>>
>>>> >>>>>> —
>>>> >>>>>> Thanks,
>>>> >>>>>> Muhammad Naseer Bhatti
>>>> >>>>>>
>>>> >>>>>> From: 彭勇 <ppyy at pubyun.com>
>>>> >>>>>> Reply: FreeSWITCH Users Help
>>>> >>>>>> <freeswitch-users at lists.freeswitch.org>>
>>>> >>>>>> Date: February 11, 2015 at 3:20:46 PM
>>>> >>>>>> To: FreeSWITCH Users Help <freeswitch-users at lists.freeswitch.org
>>>> >>
>>>> >>>>>> Subject:  Re: [Freeswitch-users] Freeswitch 1.4.15 and a sudden
>>>> >>>>>> jump in CPU usage
>>>> >>>>>>
>>>> >>>>>> same issue.
>>>> >>>>>>
>>>> >>>>>> I have several FS 1.4.15 running on CentOS 6.5 x86_64, the CPU
>>>> >>>>>> will be very high about 9:00 am every morning. there is no
>>>> crontab
>>>> >>>>>> at this time.
>>>> >>>>>>
>>>> >>>>>> i will downgrade to 1.4.14 to see if there is such issue.
>>>> >>>>>>
>>>> >>>>>> On Wed, Feb 11, 2015 at 1:14 AM, Ali Pey <alipey at gmail.com>
>>>> wrote:
>>>> >>>>>>> Hi Anthony,
>>>> >>>>>>>
>>>> >>>>>>> I do not know if I am hitting a bug and if this is a bug, I
>>>> >>>>>>> don't have much info. I'm using this mailing list to see if
>>>> >>>>>>> anyone else is experiencing this problem and if there is a
>>>> >>>>>>> configuration change on Linux or Freeswitch that can fix this
>>>> >>>>>>> problem. Jira would be to report bugs when you know there is a
>>>> >>>>>>> bug. My problem could be (very likely) an environment related
>>>> issue.
>>>> >>>>>>>
>>>> >>>>>>> Thanks.
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>> On Tue, Feb 10, 2015 at 11:55 AM, Anthony Minessale
>>>> >>>>>>> <anthony.minessale at gmail.com> wrote:
>>>> >>>>>>>> Also you will have better luck using JIRA. The mailing list is
>>>> >>>>>>>> expressly NOT for reporting issues.
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>> On Tue, Feb 10, 2015 at 10:51 AM, Michael Jerris
>>>> >>>>>>>> <mike at jerris.com>
>>>> >>>>>>>> wrote:
>>>> >>>>>>>>> you might want to use gcore to look at the threads and see
>>>> >>>>>>>>> what is different when it in this state and not.
>>>> >>>>>>>>>
>>>> >>>>>>>>>> On Feb 10, 2015, at 11:28 AM, Ali Pey <alipey at gmail.com>
>>>> wrote:
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> Hello,
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> Normally our CPU usage during peak hours are less than %10;
>>>> >>>>>>>>>> however, time to time, freeswitch uses more than %95 of the
>>>> >>>>>>>>>> CPU and it stays there forever. The only way to get it back
>>>> >>>>>>>>>> to normal is to restart freeswitch service. During the high
>>>> >>>>>>>>>> CPU usage, freeswitch continues to operate but very very
>>>> >>>>>>>>>> slow.
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> If I lower the traffic quite a bit, the CPU comes down, but
>>>> >>>>>>>>>> adding back the traffic, the CPU usage goes right back up.
>>>> >>>>>>>>>> After freeswitch restart, it becomes normal again with the
>>>> >>>>>>>>>> same traffic.
>>>> >>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> Any ideas or suggestions?
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> Much appreciated.
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> Thanks,
>>>> >>>>>>>>>> Ali Pey
>>>> >>>>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>>>>> ______________________________________________________________
>>>> >>>>>>>>> ___________ Professional FreeSWITCH Consulting Services:
>>>> >>>>>>>>> consulting at freeswitch.org
>>>> >>>>>>>>> http://www.freeswitchsolutions.com
>>>> >>>>>>>>>
>>>> >>>>>>>>> Official FreeSWITCH Sites
>>>> >>>>>>>>> http://www.freeswitch.org
>>>> >>>>>>>>> http://confluence.freeswitch.org http://www.cluecon.com
>>>> >>>>>>>>>
>>>> >>>>>>>>> FreeSWITCH-users mailing list
>>>> >>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> >>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> >>>>>>>>>
>>>> >>>>>>>>>
>>>> >>>>>>>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freesw
>>>> >>>>>>>>> itch-users
>>>> >>>>>>>>> http://www.freeswitch.org
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>> --
>>>> >>>>>>>> Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
>>>> >>>>>>>>
>>>> >>>>>>>> ☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞
>>>> >>>>>>>> http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞
>>>> >>>>>>>> http://freeswitch.org/g+
>>>> >>>>>>>>
>>>> >>>>>>>> ClueCon Weekly Development Call ☎
>>>> >>>>>>>> sip:888 at conference.freeswitch.org ☎ +19193869900
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>> _______________________________________________________________
>>>> >>>>>>>> __________ Professional FreeSWITCH Consulting Services:
>>>> >>>>>>>> consulting at freeswitch.org
>>>> >>>>>>>> http://www.freeswitchsolutions.com
>>>> >>>>>>>>
>>>> >>>>>>>> Official FreeSWITCH Sites
>>>> >>>>>>>> http://www.freeswitch.org
>>>> >>>>>>>> http://confluence.freeswitch.org http://www.cluecon.com
>>>> >>>>>>>>
>>>> >>>>>>>> FreeSWITCH-users mailing list
>>>> >>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> >>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswi
>>>> >>>>>>>> tch-users
>>>> >>>>>>>> http://www.freeswitch.org
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>>
>>>> >>>>>>> ________________________________________________________________
>>>> >>>>>>> _________ Professional FreeSWITCH Consulting Services:
>>>> >>>>>>> consulting at freeswitch.org
>>>> >>>>>>> http://www.freeswitchsolutions.com
>>>> >>>>>>>
>>>> >>>>>>> Official FreeSWITCH Sites
>>>> >>>>>>> http://www.freeswitch.org
>>>> >>>>>>> http://confluence.freeswitch.org http://www.cluecon.com
>>>> >>>>>>>
>>>> >>>>>>> FreeSWITCH-users mailing list
>>>> >>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> >>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> >>>>>>>
>>>> >>>>>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswit
>>>> >>>>>>> ch-users
>>>> >>>>>>> http://www.freeswitch.org
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>> --
>>>> >>>>>> Peng Yong
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> __________________________________________________________________
>>>> >>>>>> _______ Professional FreeSWITCH Consulting Services:
>>>> >>>>>> consulting at freeswitch.org
>>>> >>>>>> http://www.freeswitchsolutions.com
>>>> >>>>>>
>>>> >>>>>> Official FreeSWITCH Sites
>>>> >>>>>> http://www.freeswitch.org
>>>> >>>>>> http://confluence.freeswitch.org
>>>> >>>>>> http://www.cluecon.com
>>>> >>>>>>
>>>> >>>>>> FreeSWITCH-users mailing list
>>>> >>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> >>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> >>>>>>
>>>> >>>>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch
>>>> >>>>>> -users
>>>> >>>>>> http://www.freeswitch.org
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> __________________________________________________________________
>>>> >>>>>> _______ Professional FreeSWITCH Consulting Services:
>>>> >>>>>> consulting at freeswitch.org
>>>> >>>>>> http://www.freeswitchsolutions.com
>>>> >>>>>>
>>>> >>>>>> Official FreeSWITCH Sites
>>>> >>>>>> http://www.freeswitch.org
>>>> >>>>>> http://confluence.freeswitch.org
>>>> >>>>>> http://www.cluecon.com
>>>> >>>>>>
>>>> >>>>>> FreeSWITCH-users mailing list
>>>> >>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> >>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> >>>>>>
>>>> >>>>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch
>>>> >>>>>> -users
>>>> >>>>>> http://www.freeswitch.org
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>
>>>> ___________________________________________________________________
>>>> >>>>> ______ Professional FreeSWITCH Consulting Services:
>>>> >>>>> consulting at freeswitch.org
>>>> >>>>> http://www.freeswitchsolutions.com
>>>> >>>>>
>>>> >>>>> Official FreeSWITCH Sites
>>>> >>>>> http://www.freeswitch.org
>>>> >>>>> http://confluence.freeswitch.org
>>>> >>>>> http://www.cluecon.com
>>>> >>>>>
>>>> >>>>> FreeSWITCH-users mailing list
>>>> >>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> >>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> >>>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch-
>>>> >>>>> users
>>>> >>>>> http://www.freeswitch.org
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>
>>>> >>>>>
>>>> ___________________________________________________________________
>>>> >>>>> ______ Professional FreeSWITCH Consulting Services:
>>>> >>>>> consulting at freeswitch.org
>>>> >>>>> http://www.freeswitchsolutions.com
>>>> >>>>>
>>>> >>>>> Official FreeSWITCH Sites
>>>> >>>>> http://www.freeswitch.org
>>>> >>>>> http://confluence.freeswitch.org
>>>> >>>>> http://www.cluecon.com
>>>> >>>>>
>>>> >>>>> FreeSWITCH-users mailing list
>>>> >>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> >>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> >>>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch-
>>>> >>>>> users
>>>> >>>>> http://www.freeswitch.org
>>>> >>>>
>>>> >>>>
>>>> >>>> --
>>>> >>>> Peng Yong
>>>> >>>>
>>>> >>>>
>>>> _____________________________________________________________________
>>>> >>>> ____ Professional FreeSWITCH Consulting Services:
>>>> >>>> consulting at freeswitch.org
>>>> >>>> http://www.freeswitchsolutions.com
>>>> >>>>
>>>> >>>> Official FreeSWITCH Sites
>>>> >>>> http://www.freeswitch.org
>>>> >>>> http://confluence.freeswitch.org
>>>> >>>> http://www.cluecon.com
>>>> >>>>
>>>> >>>> FreeSWITCH-users mailing list
>>>> >>>> FreeSWITCH-users at lists.freeswitch.org
>>>> >>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> >>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch-us
>>>> >>>> ers
>>>> >>>> http://www.freeswitch.org
>>>> >>>
>>>> >>>
>>>> >>>
>>>> ______________________________________________________________________
>>>> >>> ___ Professional FreeSWITCH Consulting Services:
>>>> >>> consulting at freeswitch.org
>>>> >>> http://www.freeswitchsolutions.com
>>>> >>>
>>>> >>> Official FreeSWITCH Sites
>>>> >>> http://www.freeswitch.org
>>>> >>> http://confluence.freeswitch.org
>>>> >>> http://www.cluecon.com
>>>> >>>
>>>> >>> FreeSWITCH-users mailing list
>>>> >>> FreeSWITCH-users at lists.freeswitch.org
>>>> >>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> >>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch-use
>>>> >>> rs
>>>> >>> http://www.freeswitch.org
>>>> >>
>>>> >>
>>>> >> --
>>>> >> Peng Yong
>>>> >>
>>>> >>
>>>> _________________________________________________________________________
>>>> >> Professional FreeSWITCH Consulting Services:
>>>> >> consulting at freeswitch.org
>>>> >> http://www.freeswitchsolutions.com
>>>> >>
>>>> >> Official FreeSWITCH Sites
>>>> >> http://www.freeswitch.org
>>>> >> http://confluence.freeswitch.org
>>>> >> http://www.cluecon.com
>>>> >>
>>>> >> FreeSWITCH-users mailing list
>>>> >> FreeSWITCH-users at lists.freeswitch.org
>>>> >> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> >> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> >> http://www.freeswitch.org
>>>> >>
>>>> _________________________________________________________________________
>>>> >> Professional FreeSWITCH Consulting Services:
>>>> >> consulting at freeswitch.org
>>>> >> http://www.freeswitchsolutions.com
>>>> >>
>>>> >> Official FreeSWITCH Sites
>>>> >> http://www.freeswitch.org
>>>> >> http://confluence.freeswitch.org
>>>> >> http://www.cluecon.com
>>>> >>
>>>> >> FreeSWITCH-users mailing list
>>>> >> FreeSWITCH-users at lists.freeswitch.org
>>>> >> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> >> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> >> http://www.freeswitch.org
>>>> >
>>>> >
>>>>
>>>>
>>>>
>>>>
>>>> _________________________________________________________________________
>>>> Professional FreeSWITCH Consulting Services:
>>>> consulting at freeswitch.org
>>>> http://www.freeswitchsolutions.com
>>>>
>>>> Official FreeSWITCH Sites
>>>> http://www.freeswitch.org
>>>> http://confluence.freeswitch.org
>>>> http://www.cluecon.com
>>>>
>>>> FreeSWITCH-users mailing list
>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> http://www.freeswitch.org
>>>>
>>>
>>>
>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services: consulting at freeswitch.orghttp://www.freeswitchsolutions.com
>>>
>>> Official FreeSWITCH Siteshttp://www.freeswitch.orghttp://confluence.freeswitch.orghttp://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing listFreeSWITCH-users at lists.freeswitch.orghttp://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-usershttp://www.freeswitch.org
>>>
>>>
>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org
>>> http://www.freeswitchsolutions.com
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://confluence.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://confluence.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://confluence.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20150211/4a87f70d/attachment-0001.html 