[Freeswitch-users] Freeswitch 1.4.15 and a sudden jump in CPU usage

Dmitry Lysenko dvl36.ripe.nick at gmail.com
Thu Feb 12 02:30:34 MSK 2015


Hello guys,

Try to reproduce this bug with normal priority ("-np") command line switch.

Best Wishes,
 Dmitry.

2015-02-11 22:54 GMT+02:00 Ali Pey <alipey at gmail.com>:

> Thanks Tristan.
>
> I have had rtp-timer-name set to none for a long time but I still see the
> sudden and random jumps in CPU.
>
> I haven't been able to reproduce it in our Dev/QA environment as well. If
> I can reproduce it, I will definitely do the logs as well.
>
>
>
>
>
> On Wed, Feb 11, 2015 at 3:31 PM, Tristan Mahé <t.mahe at b-and-c.net> wrote:
>
>>  Hi Ali,
>>
>> What I see is kinda the same thing, I still need to provide Anthony with
>> the debug logs ( still had no chance to do the requested tests on the
>> production, could be related to some of our users UA/callflow and cannot
>> reproduce the issue in a lab ). The interesting part as discussed in
>> FS-7066 is the timer change is correcting the trouble, and without the
>> timer change, we eventually experience blocking of sofia profile ( seems
>> related to media playback ).
>>
>> I hope to have a chance soon to be able to furnish the debug logs, if you
>> can, do the same, Anthony gave the steps to provide him some usefull logs
>> on the bug comments.
>>
>>
>> Le 11/02/2015 10:58, Ali Pey a écrit :
>>
>> Hi Tristan,
>>
>>  Do you see a sudden jump or just more CPU usage?
>>
>>  In my case, everything is good for a few days with less than %10 CPU
>> usage, then there is a sudden jump to ~%98 CPU usage and it doesn't come
>> down until a restart. After restart, it's good again with the same load.
>>
>>  Thanks,
>> Ali Pey
>>
>>
>> On Wed, Feb 11, 2015 at 1:40 PM, Tristan Mahé <t.mahe at b-and-c.net> wrote:
>>
>>> Maybe related to https://freeswitch.org/jira/browse/FS-7066 ?
>>>
>>> Le 11/02/2015 08:00, Giovanni Maruzzelli a écrit :
>>>  > somebody has tested if it's a regression, eg: is 1.4.14 having
>>> > same symptoms?
>>> >
>>> > -giovanni
>>> >
>>> > On Wed, Feb 11, 2015 at 4:57 PM, Moishe Grunstein <max at nysolutions.com>
>>> wrote:
>>> >> Why not a Jira?
>>> >>
>>> >> Thanks,
>>> >>
>>> >> Moishe Grunstein
>>> >> Tornado Computer Systems, Inc.
>>> >> 212.400.7650 888.IPPBX.US
>>> >> Service Request Email: support at nysolutions.com
>>> >>
>>> >> Computer Networking * Managed Services * IP Video Surveillance *
>>> Network Assessments * Web Solutions * Voice over IP * Disaster Recovery *
>>> Network Security * Site Surveys * CMS
>>> >>
>>> >> -----Original Message-----
>>> >> From: freeswitch-users-bounces at lists.freeswitch.org [mailto:
>>> freeswitch-users-bounces at lists.freeswitch.org] On Behalf Of ??
>>> >> Sent: Wednesday, February 11, 2015 10:49 AM
>>> >> To: FreeSWITCH Users Help
>>> >> Subject: Re: [Freeswitch-users] Freeswitch 1.4.15 and a sudden jump
>>> in CPU usage
>>> >>
>>> >> i think it's bug of freeswitch:
>>> >>
>>> >> -- the Freeswitch using more than 1000% CPU, and there is no other
>>> process eating CPU
>>> >> -- we move all SIP traffic from the server away, the cpu is low. and
>>> after sevral hours, if we move the SIP traffic back again, the FS eating
>>> all CPU again. the only way to recovery is to restart the FS.
>>> >> -- there is no DB such as mysql on the server, and we have add "nosql"
>>> >> option, no help
>>> >>
>>> >> On Wed, Feb 11, 2015 at 10:03 PM, John Nash <john.nash778 at gmail.com>
>>> wrote:
>>> >>> Dear Muhammad,
>>> >>>
>>> >>> I think your issue is different than Ali Pey because CPU rises at a
>>> >>> specific time (approx) while running fine most of the time. In my
>>> >>> experience you should check following ..
>>> >>> - Check using top commands if its indeed freeswitch taking up CPU. It
>>> >>> may be logrotate or something.
>>> >>> - If you are using DB on same server, also check DB connections, May
>>> >>> be some long running query causing it (Like someone fires some
>>> billing
>>> >>> report at approx same time)
>>> >>>
>>> >>> On the other hand when you say "it runs fine all day". You may want
>>> to
>>> >>> check that using top command at different time of day, may be CPU
>>> >>> usage keep on increasing and in your peak time it increases so much
>>> that you see symptoms.
>>> >>> (If this is true may be you and Ali pey have same issue perhaps some
>>> >>> CPU leak).
>>> >>>
>>> >>> P.S. All points I mentioned are from my VOIP experiences in general.
>>> I
>>> >>> am NOT freeswitch expert. I have recently started experiments with
>>> >>> freeswitch so people who are using freeswitch or experts will give
>>> >>> more meaningful insight.
>>> >>>
>>> >>>
>>> >>>
>>> >>> On Wed, Feb 11, 2015 at 7:16 PM, 彭勇 <ppyy at pubyun.com> wrote:
>>> >>>> we have add this option to grub and reboot one months ago. no help.
>>> >>>>
>>> >>>> we have media-pass-through  option, and have more than 10 thousands
>>> >>>> sessions. it works fine all day and the CPU will rise suddenly about
>>> >>>> 9:00 am every day.  i should restart the FS to recovery.
>>> >>>>
>>> >>>> On Wed, Feb 11, 2015 at 9:25 PM, Muhammad Naseer Bhatti
>>> >>>> <nbhatti at gmail.com> wrote:
>>> >>>>> Try nohz=off reboot and see if that helps.
>>> >>>>>
>>> >>>>> Sent from my iPad
>>> >>>>>
>>> >>>>> On Feb 11, 2015, at 16:18, Aqs Younas <aqsyounas at gmail.com> wrote:
>>> >>>>>
>>> >>>>> I am facing same issue. Freeswitch taking more than 100% cpu usage
>>> >>>>> in htop even on 200 concurrent calls. For 1000 concurrent calls i
>>> >>>>> have to install 3 freeswitch instances due to this much cpu usage
>>> >>>>> taking by a single freeswitch node. My Dist is Debian 7.
>>> >>>>>
>>> >>>>> On 11 February 2015 at 17:35, Muhammad Naseer Bhatti
>>> >>>>> <nbhatti at gmail.com>
>>> >>>>> wrote:
>>> >>>>>>
>>> >>>>>> It looks like this is the same tickless kernel issue. I am using
>>> >>>>>> Debian
>>> >>>>>> 7
>>> >>>>>> and the CPU usage shows very high. Adding nohz=off to grub options
>>> >>>>>> does makes it lower. I am gathering more information in order to
>>> >>>>>> file a Jira.
>>> >>>>>> Debian 6 on the other hand works very quietly on the same
>>> >>>>>> workload. We are talking about 200+% CPU utilization on Debian 7
>>> >>>>>> and around 50% on Debian 6 showing in htop. I know so many other
>>> >>>>>> factors also involved but I am still digging out more.
>>> >>>>>>
>>> >>>>>> —
>>> >>>>>> Thanks,
>>> >>>>>> Muhammad Naseer Bhatti
>>> >>>>>>
>>> >>>>>> From: 彭勇 <ppyy at pubyun.com>
>>> >>>>>> Reply: FreeSWITCH Users Help
>>> >>>>>> <freeswitch-users at lists.freeswitch.org>>
>>> >>>>>> Date: February 11, 2015 at 3:20:46 PM
>>> >>>>>> To: FreeSWITCH Users Help <freeswitch-users at lists.freeswitch.org
>>> >>
>>> >>>>>> Subject:  Re: [Freeswitch-users] Freeswitch 1.4.15 and a sudden
>>> >>>>>> jump in CPU usage
>>> >>>>>>
>>> >>>>>> same issue.
>>> >>>>>>
>>> >>>>>> I have several FS 1.4.15 running on CentOS 6.5 x86_64, the CPU
>>> >>>>>> will be very high about 9:00 am every morning. there is no crontab
>>> >>>>>> at this time.
>>> >>>>>>
>>> >>>>>> i will downgrade to 1.4.14 to see if there is such issue.
>>> >>>>>>
>>> >>>>>> On Wed, Feb 11, 2015 at 1:14 AM, Ali Pey <alipey at gmail.com>
>>> wrote:
>>> >>>>>>> Hi Anthony,
>>> >>>>>>>
>>> >>>>>>> I do not know if I am hitting a bug and if this is a bug, I
>>> >>>>>>> don't have much info. I'm using this mailing list to see if
>>> >>>>>>> anyone else is experiencing this problem and if there is a
>>> >>>>>>> configuration change on Linux or Freeswitch that can fix this
>>> >>>>>>> problem. Jira would be to report bugs when you know there is a
>>> >>>>>>> bug. My problem could be (very likely) an environment related
>>> issue.
>>> >>>>>>>
>>> >>>>>>> Thanks.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> On Tue, Feb 10, 2015 at 11:55 AM, Anthony Minessale
>>> >>>>>>> <anthony.minessale at gmail.com> wrote:
>>> >>>>>>>> Also you will have better luck using JIRA. The mailing list is
>>> >>>>>>>> expressly NOT for reporting issues.
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> On Tue, Feb 10, 2015 at 10:51 AM, Michael Jerris
>>> >>>>>>>> <mike at jerris.com>
>>> >>>>>>>> wrote:
>>> >>>>>>>>> you might want to use gcore to look at the threads and see
>>> >>>>>>>>> what is different when it in this state and not.
>>> >>>>>>>>>
>>> >>>>>>>>>> On Feb 10, 2015, at 11:28 AM, Ali Pey <alipey at gmail.com>
>>> wrote:
>>> >>>>>>>>>>
>>> >>>>>>>>>> Hello,
>>> >>>>>>>>>>
>>> >>>>>>>>>> Normally our CPU usage during peak hours are less than %10;
>>> >>>>>>>>>> however, time to time, freeswitch uses more than %95 of the
>>> >>>>>>>>>> CPU and it stays there forever. The only way to get it back
>>> >>>>>>>>>> to normal is to restart freeswitch service. During the high
>>> >>>>>>>>>> CPU usage, freeswitch continues to operate but very very
>>> >>>>>>>>>> slow.
>>> >>>>>>>>>>
>>> >>>>>>>>>> If I lower the traffic quite a bit, the CPU comes down, but
>>> >>>>>>>>>> adding back the traffic, the CPU usage goes right back up.
>>> >>>>>>>>>> After freeswitch restart, it becomes normal again with the
>>> >>>>>>>>>> same traffic.
>>> >>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>>> Any ideas or suggestions?
>>> >>>>>>>>>>
>>> >>>>>>>>>> Much appreciated.
>>> >>>>>>>>>>
>>> >>>>>>>>>> Thanks,
>>> >>>>>>>>>> Ali Pey
>>> >>>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> ______________________________________________________________
>>> >>>>>>>>> ___________ Professional FreeSWITCH Consulting Services:
>>> >>>>>>>>> consulting at freeswitch.org
>>> >>>>>>>>> http://www.freeswitchsolutions.com
>>> >>>>>>>>>
>>> >>>>>>>>> Official FreeSWITCH Sites
>>> >>>>>>>>> http://www.freeswitch.org
>>> >>>>>>>>> http://confluence.freeswitch.org http://www.cluecon.com
>>> >>>>>>>>>
>>> >>>>>>>>> FreeSWITCH-users mailing list
>>> >>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>> >>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> >>>>>>>>>
>>> >>>>>>>>>
>>> >>>>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freesw
>>> >>>>>>>>> itch-users
>>> >>>>>>>>> http://www.freeswitch.org
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> --
>>> >>>>>>>> Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
>>> >>>>>>>>
>>> >>>>>>>> ☞ http://freeswitch.org/http://cluecon.com/>>> >>>>>>>> http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞
>>> >>>>>>>> http://freeswitch.org/g+
>>> >>>>>>>>
>>> >>>>>>>> ClueCon Weekly Development Call ☎
>>> >>>>>>>> sip:888 at conference.freeswitch.org ☎ +19193869900
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> _______________________________________________________________
>>> >>>>>>>> __________ Professional FreeSWITCH Consulting Services:
>>> >>>>>>>> consulting at freeswitch.org
>>> >>>>>>>> http://www.freeswitchsolutions.com
>>> >>>>>>>>
>>> >>>>>>>> Official FreeSWITCH Sites
>>> >>>>>>>> http://www.freeswitch.org
>>> >>>>>>>> http://confluence.freeswitch.org http://www.cluecon.com
>>> >>>>>>>>
>>> >>>>>>>> FreeSWITCH-users mailing list
>>> >>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>> >>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswi
>>> >>>>>>>> tch-users
>>> >>>>>>>> http://www.freeswitch.org
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> ________________________________________________________________
>>> >>>>>>> _________ Professional FreeSWITCH Consulting Services:
>>> >>>>>>> consulting at freeswitch.org
>>> >>>>>>> http://www.freeswitchsolutions.com
>>> >>>>>>>
>>> >>>>>>> Official FreeSWITCH Sites
>>> >>>>>>> http://www.freeswitch.org
>>> >>>>>>> http://confluence.freeswitch.org http://www.cluecon.com
>>> >>>>>>>
>>> >>>>>>> FreeSWITCH-users mailing list
>>> >>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>> >>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> >>>>>>>
>>> >>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswit
>>> >>>>>>> ch-users
>>> >>>>>>> http://www.freeswitch.org
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> --
>>> >>>>>> Peng Yong
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> __________________________________________________________________
>>> >>>>>> _______ Professional FreeSWITCH Consulting Services:
>>> >>>>>> consulting at freeswitch.org
>>> >>>>>> http://www.freeswitchsolutions.com
>>> >>>>>>
>>> >>>>>> Official FreeSWITCH Sites
>>> >>>>>> http://www.freeswitch.org
>>> >>>>>> http://confluence.freeswitch.org
>>> >>>>>> http://www.cluecon.com
>>> >>>>>>
>>> >>>>>> FreeSWITCH-users mailing list
>>> >>>>>> FreeSWITCH-users at lists.freeswitch.org
>>> >>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> >>>>>>
>>> >>>>>> UNSUBSCRIBE:
>>> http://lists.freeswitch.org/mailman/options/freeswitch
>>> >>>>>> -users
>>> >>>>>> http://www.freeswitch.org
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> __________________________________________________________________
>>> >>>>>> _______ Professional FreeSWITCH Consulting Services:
>>> >>>>>> consulting at freeswitch.org
>>> >>>>>> http://www.freeswitchsolutions.com
>>> >>>>>>
>>> >>>>>> Official FreeSWITCH Sites
>>> >>>>>> http://www.freeswitch.org
>>> >>>>>> http://confluence.freeswitch.org
>>> >>>>>> http://www.cluecon.com
>>> >>>>>>
>>> >>>>>> FreeSWITCH-users mailing list
>>> >>>>>> FreeSWITCH-users at lists.freeswitch.org
>>> >>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> >>>>>>
>>> >>>>>> UNSUBSCRIBE:
>>> http://lists.freeswitch.org/mailman/options/freeswitch
>>> >>>>>> -users
>>> >>>>>> http://www.freeswitch.org
>>> >>>>>
>>> >>>>>
>>> >>>>> ___________________________________________________________________
>>> >>>>> ______ Professional FreeSWITCH Consulting Services:
>>> >>>>> consulting at freeswitch.org
>>> >>>>> http://www.freeswitchsolutions.com
>>> >>>>>
>>> >>>>> Official FreeSWITCH Sites
>>> >>>>> http://www.freeswitch.org
>>> >>>>> http://confluence.freeswitch.org
>>> >>>>> http://www.cluecon.com
>>> >>>>>
>>> >>>>> FreeSWITCH-users mailing list
>>> >>>>> FreeSWITCH-users at lists.freeswitch.org
>>> >>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> >>>>> UNSUBSCRIBE:
>>> http://lists.freeswitch.org/mailman/options/freeswitch-
>>> >>>>> users
>>> >>>>> http://www.freeswitch.org
>>> >>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> ___________________________________________________________________
>>> >>>>> ______ Professional FreeSWITCH Consulting Services:
>>> >>>>> consulting at freeswitch.org
>>> >>>>> http://www.freeswitchsolutions.com
>>> >>>>>
>>> >>>>> Official FreeSWITCH Sites
>>> >>>>> http://www.freeswitch.org
>>> >>>>> http://confluence.freeswitch.org
>>> >>>>> http://www.cluecon.com
>>> >>>>>
>>> >>>>> FreeSWITCH-users mailing list
>>> >>>>> FreeSWITCH-users at lists.freeswitch.org
>>> >>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> >>>>> UNSUBSCRIBE:
>>> http://lists.freeswitch.org/mailman/options/freeswitch-
>>> >>>>> users
>>> >>>>> http://www.freeswitch.org
>>> >>>>
>>> >>>>
>>> >>>> --
>>> >>>> Peng Yong
>>> >>>>
>>> >>>>
>>> _____________________________________________________________________
>>> >>>> ____ Professional FreeSWITCH Consulting Services:
>>> >>>> consulting at freeswitch.org
>>> >>>> http://www.freeswitchsolutions.com
>>> >>>>
>>> >>>> Official FreeSWITCH Sites
>>> >>>> http://www.freeswitch.org
>>> >>>> http://confluence.freeswitch.org
>>> >>>> http://www.cluecon.com
>>> >>>>
>>> >>>> FreeSWITCH-users mailing list
>>> >>>> FreeSWITCH-users at lists.freeswitch.org
>>> >>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> >>>> UNSUBSCRIBE:
>>> http://lists.freeswitch.org/mailman/options/freeswitch-us
>>> >>>> ers
>>> >>>> http://www.freeswitch.org
>>> >>>
>>> >>>
>>> >>>
>>> ______________________________________________________________________
>>> >>> ___ Professional FreeSWITCH Consulting Services:
>>> >>> consulting at freeswitch.org
>>> >>> http://www.freeswitchsolutions.com
>>> >>>
>>> >>> Official FreeSWITCH Sites
>>> >>> http://www.freeswitch.org
>>> >>> http://confluence.freeswitch.org
>>> >>> http://www.cluecon.com
>>> >>>
>>> >>> FreeSWITCH-users mailing list
>>> >>> FreeSWITCH-users at lists.freeswitch.org
>>> >>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> >>> UNSUBSCRIBE:
>>> http://lists.freeswitch.org/mailman/options/freeswitch-use
>>> >>> rs
>>> >>> http://www.freeswitch.org
>>> >>
>>> >>
>>> >> --
>>> >> Peng Yong
>>> >>
>>> >>
>>> _________________________________________________________________________
>>> >> Professional FreeSWITCH Consulting Services:
>>> >> consulting at freeswitch.org
>>> >> http://www.freeswitchsolutions.com
>>> >>
>>> >> Official FreeSWITCH Sites
>>> >> http://www.freeswitch.org
>>> >> http://confluence.freeswitch.org
>>> >> http://www.cluecon.com
>>> >>
>>> >> FreeSWITCH-users mailing list
>>> >> FreeSWITCH-users at lists.freeswitch.org
>>> >> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> >> UNSUBSCRIBE:
>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> >> http://www.freeswitch.org
>>> >>
>>> _________________________________________________________________________
>>> >> Professional FreeSWITCH Consulting Services:
>>> >> consulting at freeswitch.org
>>> >> http://www.freeswitchsolutions.com
>>> >>
>>> >> Official FreeSWITCH Sites
>>> >> http://www.freeswitch.org
>>> >> http://confluence.freeswitch.org
>>> >> http://www.cluecon.com
>>> >>
>>> >> FreeSWITCH-users mailing list
>>> >> FreeSWITCH-users at lists.freeswitch.org
>>> >> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> >> UNSUBSCRIBE:
>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> >> http://www.freeswitch.org
>>> >
>>> >
>>>
>>>
>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org
>>> http://www.freeswitchsolutions.com
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://confluence.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>
>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services: consulting at freeswitch.orghttp://www.freeswitchsolutions.com
>>
>> Official FreeSWITCH Siteshttp://www.freeswitch.orghttp://confluence.freeswitch.orghttp://www.cluecon.com
>>
>> FreeSWITCH-users mailing listFreeSWITCH-users at lists.freeswitch.orghttp://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-usershttp://www.freeswitch.org
>>
>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://confluence.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://confluence.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20150212/a7e177a1/attachment-0001.html 


Join us at ClueCon 2016 Aug 8-12, 2016
More information about the FreeSWITCH-users mailing list