[Freeswitch-users] Freeswitch 1.4.15 and a sudden jump in CPU usage

Thu Feb 12 18:43:59 MSK 2015

Do we have a JIRA on this yet?

On Thu, Feb 12, 2015 at 4:31 AM, Dmitry Lysenko <dvl36.ripe.nick at gmail.com>
wrote:

> Hi Ali,
>
> 2 years ago here, in mail-list, was a discussion
> <http://lists.freeswitch.org/pipermail/freeswitch-users/2013-March/093257.html>.
> (memleak can be omitted)
> Not sure, but It seems the symptoms was similar.
>
> Best Wishes,
>  Dmitry.
>
> 2015-02-12 2:53 GMT+02:00 Ali Pey <alipey at gmail.com>:
>
>> Hi Dmitry,
>>
>> Can you elaborate a bit please? Are you saying that it won't happen with
>> -np? Or it's just a suggestion?
>>
>> What's the relationship?
>>
>> Thanks,
>> Ali Pey
>>
>>
>> On Wed, Feb 11, 2015 at 6:30 PM, Dmitry Lysenko <
>> dvl36.ripe.nick at gmail.com> wrote:
>>
>>> Hello guys,
>>>
>>> Try to reproduce this bug with normal priority ("-np") command line
>>> switch.
>>>
>>> Best Wishes,
>>>  Dmitry.
>>>
>>> 2015-02-11 22:54 GMT+02:00 Ali Pey <alipey at gmail.com>:
>>>
>>>> Thanks Tristan.
>>>>
>>>> I have had rtp-timer-name set to none for a long time but I still see
>>>> the sudden and random jumps in CPU.
>>>>
>>>> I haven't been able to reproduce it in our Dev/QA environment as well.
>>>> If I can reproduce it, I will definitely do the logs as well.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, Feb 11, 2015 at 3:31 PM, Tristan Mahé <t.mahe at b-and-c.net>
>>>> wrote:
>>>>
>>>>>  Hi Ali,
>>>>>
>>>>> What I see is kinda the same thing, I still need to provide Anthony
>>>>> with the debug logs ( still had no chance to do the requested tests on the
>>>>> production, could be related to some of our users UA/callflow and cannot
>>>>> reproduce the issue in a lab ). The interesting part as discussed in
>>>>> FS-7066 is the timer change is correcting the trouble, and without the
>>>>> timer change, we eventually experience blocking of sofia profile ( seems
>>>>> related to media playback ).
>>>>>
>>>>> I hope to have a chance soon to be able to furnish the debug logs, if
>>>>> you can, do the same, Anthony gave the steps to provide him some usefull
>>>>> logs on the bug comments.
>>>>>
>>>>>
>>>>> Le 11/02/2015 10:58, Ali Pey a écrit :
>>>>>
>>>>> Hi Tristan,
>>>>>
>>>>>  Do you see a sudden jump or just more CPU usage?
>>>>>
>>>>>  In my case, everything is good for a few days with less than %10 CPU
>>>>> usage, then there is a sudden jump to ~%98 CPU usage and it doesn't come
>>>>> down until a restart. After restart, it's good again with the same load.
>>>>>
>>>>>  Thanks,
>>>>> Ali Pey
>>>>>
>>>>>
>>>>> On Wed, Feb 11, 2015 at 1:40 PM, Tristan Mahé <t.mahe at b-and-c.net>
>>>>> wrote:
>>>>>
>>>>>> Maybe related to https://freeswitch.org/jira/browse/FS-7066 ?
>>>>>>
>>>>>> Le 11/02/2015 08:00, Giovanni Maruzzelli a écrit :
>>>>>>  > somebody has tested if it's a regression, eg: is 1.4.14 having
>>>>>> > same symptoms?
>>>>>> >
>>>>>> > -giovanni
>>>>>> >
>>>>>> > On Wed, Feb 11, 2015 at 4:57 PM, Moishe Grunstein <
>>>>>> max at nysolutions.com> wrote:
>>>>>> >> Why not a Jira?
>>>>>> >>
>>>>>> >> Thanks,
>>>>>> >>
>>>>>> >> Moishe Grunstein
>>>>>> >> Tornado Computer Systems, Inc.
>>>>>> >> 212.400.7650 888.IPPBX.US
>>>>>> >> Service Request Email: support at nysolutions.com
>>>>>> >>
>>>>>> >> Computer Networking * Managed Services * IP Video Surveillance *
>>>>>> Network Assessments * Web Solutions * Voice over IP * Disaster Recovery *
>>>>>> Network Security * Site Surveys * CMS
>>>>>> >>
>>>>>> >> -----Original Message-----
>>>>>> >> From: freeswitch-users-bounces at lists.freeswitch.org [mailto:
>>>>>> freeswitch-users-bounces at lists.freeswitch.org] On Behalf Of ??
>>>>>> >> Sent: Wednesday, February 11, 2015 10:49 AM
>>>>>> >> To: FreeSWITCH Users Help
>>>>>> >> Subject: Re: [Freeswitch-users] Freeswitch 1.4.15 and a sudden
>>>>>> jump in CPU usage
>>>>>> >>
>>>>>> >> i think it's bug of freeswitch:
>>>>>> >>
>>>>>> >> -- the Freeswitch using more than 1000% CPU, and there is no other
>>>>>> process eating CPU
>>>>>> >> -- we move all SIP traffic from the server away, the cpu is low.
>>>>>> and after sevral hours, if we move the SIP traffic back again, the FS
>>>>>> eating all CPU again. the only way to recovery is to restart the FS.
>>>>>> >> -- there is no DB such as mysql on the server, and we have add
>>>>>> "nosql"
>>>>>> >> option, no help
>>>>>> >>
>>>>>> >> On Wed, Feb 11, 2015 at 10:03 PM, John Nash <
>>>>>> john.nash778 at gmail.com> wrote:
>>>>>> >>> Dear Muhammad,
>>>>>> >>>
>>>>>> >>> I think your issue is different than Ali Pey because CPU rises at
>>>>>> a
>>>>>> >>> specific time (approx) while running fine most of the time. In my
>>>>>> >>> experience you should check following ..
>>>>>> >>> - Check using top commands if its indeed freeswitch taking up
>>>>>> CPU. It
>>>>>> >>> may be logrotate or something.
>>>>>> >>> - If you are using DB on same server, also check DB connections,
>>>>>> May
>>>>>> >>> be some long running query causing it (Like someone fires some
>>>>>> billing
>>>>>> >>> report at approx same time)
>>>>>> >>>
>>>>>> >>> On the other hand when you say "it runs fine all day". You may
>>>>>> want to
>>>>>> >>> check that using top command at different time of day, may be CPU
>>>>>> >>> usage keep on increasing and in your peak time it increases so
>>>>>> much that you see symptoms.
>>>>>> >>> (If this is true may be you and Ali pey have same issue perhaps
>>>>>> some
>>>>>> >>> CPU leak).
>>>>>> >>>
>>>>>> >>> P.S. All points I mentioned are from my VOIP experiences in
>>>>>> general. I
>>>>>> >>> am NOT freeswitch expert. I have recently started experiments with
>>>>>> >>> freeswitch so people who are using freeswitch or experts will give
>>>>>> >>> more meaningful insight.
>>>>>> >>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> On Wed, Feb 11, 2015 at 7:16 PM, 彭勇 <ppyy at pubyun.com> wrote:
>>>>>> >>>> we have add this option to grub and reboot one months ago. no
>>>>>> help.
>>>>>> >>>>
>>>>>> >>>> we have media-pass-through  option, and have more than 10
>>>>>> thousands
>>>>>> >>>> sessions. it works fine all day and the CPU will rise suddenly
>>>>>> about
>>>>>> >>>> 9:00 am every day.  i should restart the FS to recovery.
>>>>>> >>>>
>>>>>> >>>> On Wed, Feb 11, 2015 at 9:25 PM, Muhammad Naseer Bhatti
>>>>>> >>>> <nbhatti at gmail.com> wrote:
>>>>>> >>>>> Try nohz=off reboot and see if that helps.
>>>>>> >>>>>
>>>>>> >>>>> Sent from my iPad
>>>>>> >>>>>
>>>>>> >>>>> On Feb 11, 2015, at 16:18, Aqs Younas <aqsyounas at gmail.com>
>>>>>> wrote:
>>>>>> >>>>>
>>>>>> >>>>> I am facing same issue. Freeswitch taking more than 100% cpu
>>>>>> usage
>>>>>> >>>>> in htop even on 200 concurrent calls. For 1000 concurrent calls
>>>>>> i
>>>>>> >>>>> have to install 3 freeswitch instances due to this much cpu
>>>>>> usage
>>>>>> >>>>> taking by a single freeswitch node. My Dist is Debian 7.
>>>>>> >>>>>
>>>>>> >>>>> On 11 February 2015 at 17:35, Muhammad Naseer Bhatti
>>>>>> >>>>> <nbhatti at gmail.com>
>>>>>> >>>>> wrote:
>>>>>> >>>>>>
>>>>>> >>>>>> It looks like this is the same tickless kernel issue. I am
>>>>>> using
>>>>>> >>>>>> Debian
>>>>>> >>>>>> 7
>>>>>> >>>>>> and the CPU usage shows very high. Adding nohz=off to grub
>>>>>> options
>>>>>> >>>>>> does makes it lower. I am gathering more information in order
>>>>>> to
>>>>>> >>>>>> file a Jira.
>>>>>> >>>>>> Debian 6 on the other hand works very quietly on the same
>>>>>> >>>>>> workload. We are talking about 200+% CPU utilization on Debian
>>>>>> 7
>>>>>> >>>>>> and around 50% on Debian 6 showing in htop. I know so many
>>>>>> other
>>>>>> >>>>>> factors also involved but I am still digging out more.
>>>>>> >>>>>>
>>>>>> >>>>>> —
>>>>>> >>>>>> Thanks,
>>>>>> >>>>>> Muhammad Naseer Bhatti
>>>>>> >>>>>>
>>>>>> >>>>>> From: 彭勇 <ppyy at pubyun.com>
>>>>>> >>>>>> Reply: FreeSWITCH Users Help
>>>>>> >>>>>> <freeswitch-users at lists.freeswitch.org>>
>>>>>> >>>>>> Date: February 11, 2015 at 3:20:46 PM
>>>>>> >>>>>> To: FreeSWITCH Users Help <
>>>>>> freeswitch-users at lists.freeswitch.org>>
>>>>>> >>>>>> Subject:  Re: [Freeswitch-users] Freeswitch 1.4.15 and a sudden
>>>>>> >>>>>> jump in CPU usage
>>>>>> >>>>>>
>>>>>> >>>>>> same issue.
>>>>>> >>>>>>
>>>>>> >>>>>> I have several FS 1.4.15 running on CentOS 6.5 x86_64, the CPU
>>>>>> >>>>>> will be very high about 9:00 am every morning. there is no
>>>>>> crontab
>>>>>> >>>>>> at this time.
>>>>>> >>>>>>
>>>>>> >>>>>> i will downgrade to 1.4.14 to see if there is such issue.
>>>>>> >>>>>>
>>>>>> >>>>>> On Wed, Feb 11, 2015 at 1:14 AM, Ali Pey <alipey at gmail.com>
>>>>>> wrote:
>>>>>> >>>>>>> Hi Anthony,
>>>>>> >>>>>>>
>>>>>> >>>>>>> I do not know if I am hitting a bug and if this is a bug, I
>>>>>> >>>>>>> don't have much info. I'm using this mailing list to see if
>>>>>> >>>>>>> anyone else is experiencing this problem and if there is a
>>>>>> >>>>>>> configuration change on Linux or Freeswitch that can fix this
>>>>>> >>>>>>> problem. Jira would be to report bugs when you know there is a
>>>>>> >>>>>>> bug. My problem could be (very likely) an environment related
>>>>>> issue.
>>>>>> >>>>>>>
>>>>>> >>>>>>> Thanks.
>>>>>> >>>>>>>
>>>>>> >>>>>>>
>>>>>> >>>>>>> On Tue, Feb 10, 2015 at 11:55 AM, Anthony Minessale
>>>>>> >>>>>>> <anthony.minessale at gmail.com> wrote:
>>>>>> >>>>>>>> Also you will have better luck using JIRA. The mailing list
>>>>>> is
>>>>>> >>>>>>>> expressly NOT for reporting issues.
>>>>>> >>>>>>>>
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> On Tue, Feb 10, 2015 at 10:51 AM, Michael Jerris
>>>>>> >>>>>>>> <mike at jerris.com>
>>>>>> >>>>>>>> wrote:
>>>>>> >>>>>>>>> you might want to use gcore to look at the threads and see
>>>>>> >>>>>>>>> what is different when it in this state and not.
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>>> On Feb 10, 2015, at 11:28 AM, Ali Pey <alipey at gmail.com>
>>>>>> wrote:
>>>>>> >>>>>>>>>>
>>>>>> >>>>>>>>>> Hello,
>>>>>> >>>>>>>>>>
>>>>>> >>>>>>>>>> Normally our CPU usage during peak hours are less than %10;
>>>>>> >>>>>>>>>> however, time to time, freeswitch uses more than %95 of the
>>>>>> >>>>>>>>>> CPU and it stays there forever. The only way to get it back
>>>>>> >>>>>>>>>> to normal is to restart freeswitch service. During the high
>>>>>> >>>>>>>>>> CPU usage, freeswitch continues to operate but very very
>>>>>> >>>>>>>>>> slow.
>>>>>> >>>>>>>>>>
>>>>>> >>>>>>>>>> If I lower the traffic quite a bit, the CPU comes down, but
>>>>>> >>>>>>>>>> adding back the traffic, the CPU usage goes right back up.
>>>>>> >>>>>>>>>> After freeswitch restart, it becomes normal again with the
>>>>>> >>>>>>>>>> same traffic.
>>>>>> >>>>>>>>>>
>>>>>> >>>>>>>>>>
>>>>>> >>>>>>>>>> Any ideas or suggestions?
>>>>>> >>>>>>>>>>
>>>>>> >>>>>>>>>> Much appreciated.
>>>>>> >>>>>>>>>>
>>>>>> >>>>>>>>>> Thanks,
>>>>>> >>>>>>>>>> Ali Pey
>>>>>> >>>>>>>>>>
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>>
>>>>>> ______________________________________________________________
>>>>>> >>>>>>>>> ___________ Professional FreeSWITCH Consulting Services:
>>>>>> >>>>>>>>> consulting at freeswitch.org
>>>>>> >>>>>>>>> http://www.freeswitchsolutions.com
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>> Official FreeSWITCH Sites
>>>>>> >>>>>>>>> http://www.freeswitch.org
>>>>>> >>>>>>>>> http://confluence.freeswitch.org http://www.cluecon.com
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>> FreeSWITCH-users mailing list
>>>>>> >>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>> >>>>>>>>>
>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>>
>>>>>> >>>>>>>>> UNSUBSCRIBE:
>>>>>> http://lists.freeswitch.org/mailman/options/freesw
>>>>>> >>>>>>>>> itch-users
>>>>>> >>>>>>>>> http://www.freeswitch.org
>>>>>> >>>>>>>>
>>>>>> >>>>>>>>
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> --
>>>>>> >>>>>>>> Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> ☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞
>>>>>> >>>>>>>> http://twitter.com/FreeSWITCH ☞ irc.freenode.net
>>>>>> #freeswitch ☞
>>>>>> >>>>>>>> http://freeswitch.org/g+
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> ClueCon Weekly Development Call ☎
>>>>>> >>>>>>>> sip:888 at conference.freeswitch.org ☎ +19193869900
>>>>>> >>>>>>>>
>>>>>> >>>>>>>>
>>>>>> >>>>>>>>
>>>>>> >>>>>>>>
>>>>>> >>>>>>>>
>>>>>> _______________________________________________________________
>>>>>> >>>>>>>> __________ Professional FreeSWITCH Consulting Services:
>>>>>> >>>>>>>> consulting at freeswitch.org
>>>>>> >>>>>>>> http://www.freeswitchsolutions.com
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> Official FreeSWITCH Sites
>>>>>> >>>>>>>> http://www.freeswitch.org
>>>>>> >>>>>>>> http://confluence.freeswitch.org http://www.cluecon.com
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> FreeSWITCH-users mailing list
>>>>>> >>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>> >>>>>>>>
>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>> >>>>>>>>
>>>>>> >>>>>>>>
>>>>>> >>>>>>>> UNSUBSCRIBE:
>>>>>> http://lists.freeswitch.org/mailman/options/freeswi
>>>>>> >>>>>>>> tch-users
>>>>>> >>>>>>>> http://www.freeswitch.org
>>>>>> >>>>>>>
>>>>>> >>>>>>>
>>>>>> >>>>>>>
>>>>>> >>>>>>>
>>>>>> >>>>>>>
>>>>>> ________________________________________________________________
>>>>>> >>>>>>> _________ Professional FreeSWITCH Consulting Services:
>>>>>> >>>>>>> consulting at freeswitch.org
>>>>>> >>>>>>> http://www.freeswitchsolutions.com
>>>>>> >>>>>>>
>>>>>> >>>>>>> Official FreeSWITCH Sites
>>>>>> >>>>>>> http://www.freeswitch.org
>>>>>> >>>>>>> http://confluence.freeswitch.org http://www.cluecon.com
>>>>>> >>>>>>>
>>>>>> >>>>>>> FreeSWITCH-users mailing list
>>>>>> >>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>> >>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>> >>>>>>>
>>>>>> >>>>>>> UNSUBSCRIBE:
>>>>>> http://lists.freeswitch.org/mailman/options/freeswit
>>>>>> >>>>>>> ch-users
>>>>>> >>>>>>> http://www.freeswitch.org
>>>>>> >>>>>>
>>>>>> >>>>>>
>>>>>> >>>>>> --
>>>>>> >>>>>> Peng Yong
>>>>>> >>>>>>
>>>>>> >>>>>>
>>>>>> >>>>>>
>>>>>> __________________________________________________________________
>>>>>> >>>>>> _______ Professional FreeSWITCH Consulting Services:
>>>>>> >>>>>> consulting at freeswitch.org
>>>>>> >>>>>> http://www.freeswitchsolutions.com
>>>>>> >>>>>>
>>>>>> >>>>>> Official FreeSWITCH Sites
>>>>>> >>>>>> http://www.freeswitch.org
>>>>>> >>>>>> http://confluence.freeswitch.org
>>>>>> >>>>>> http://www.cluecon.com
>>>>>> >>>>>>
>>>>>> >>>>>> FreeSWITCH-users mailing list
>>>>>> >>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>> >>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>> >>>>>>
>>>>>> >>>>>> UNSUBSCRIBE:
>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch
>>>>>> >>>>>> -users
>>>>>> >>>>>> http://www.freeswitch.org
>>>>>> >>>>>>
>>>>>> >>>>>>
>>>>>> >>>>>>
>>>>>> >>>>>>
>>>>>> __________________________________________________________________
>>>>>> >>>>>> _______ Professional FreeSWITCH Consulting Services:
>>>>>> >>>>>> consulting at freeswitch.org
>>>>>> >>>>>> http://www.freeswitchsolutions.com
>>>>>> >>>>>>
>>>>>> >>>>>> Official FreeSWITCH Sites
>>>>>> >>>>>> http://www.freeswitch.org
>>>>>> >>>>>> http://confluence.freeswitch.org
>>>>>> >>>>>> http://www.cluecon.com
>>>>>> >>>>>>
>>>>>> >>>>>> FreeSWITCH-users mailing list
>>>>>> >>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>> >>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>> >>>>>>
>>>>>> >>>>>> UNSUBSCRIBE:
>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch
>>>>>> >>>>>> -users
>>>>>> >>>>>> http://www.freeswitch.org
>>>>>> >>>>>
>>>>>> >>>>>
>>>>>> >>>>>
>>>>>> ___________________________________________________________________
>>>>>> >>>>> ______ Professional FreeSWITCH Consulting Services:
>>>>>> >>>>> consulting at freeswitch.org
>>>>>> >>>>> http://www.freeswitchsolutions.com
>>>>>> >>>>>
>>>>>> >>>>> Official FreeSWITCH Sites
>>>>>> >>>>> http://www.freeswitch.org
>>>>>> >>>>> http://confluence.freeswitch.org
>>>>>> >>>>> http://www.cluecon.com
>>>>>> >>>>>
>>>>>> >>>>> FreeSWITCH-users mailing list
>>>>>> >>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>> >>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>> >>>>> UNSUBSCRIBE:
>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-
>>>>>> >>>>> users
>>>>>> >>>>> http://www.freeswitch.org
>>>>>> >>>>>
>>>>>> >>>>>
>>>>>> >>>>>
>>>>>> >>>>>
>>>>>> ___________________________________________________________________
>>>>>> >>>>> ______ Professional FreeSWITCH Consulting Services:
>>>>>> >>>>> consulting at freeswitch.org
>>>>>> >>>>> http://www.freeswitchsolutions.com
>>>>>> >>>>>
>>>>>> >>>>> Official FreeSWITCH Sites
>>>>>> >>>>> http://www.freeswitch.org
>>>>>> >>>>> http://confluence.freeswitch.org
>>>>>> >>>>> http://www.cluecon.com
>>>>>> >>>>>
>>>>>> >>>>> FreeSWITCH-users mailing list
>>>>>> >>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>> >>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>> >>>>> UNSUBSCRIBE:
>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-
>>>>>> >>>>> users
>>>>>> >>>>> http://www.freeswitch.org
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>> --
>>>>>> >>>> Peng Yong
>>>>>> >>>>
>>>>>> >>>>
>>>>>> _____________________________________________________________________
>>>>>> >>>> ____ Professional FreeSWITCH Consulting Services:
>>>>>> >>>> consulting at freeswitch.org
>>>>>> >>>> http://www.freeswitchsolutions.com
>>>>>> >>>>
>>>>>> >>>> Official FreeSWITCH Sites
>>>>>> >>>> http://www.freeswitch.org
>>>>>> >>>> http://confluence.freeswitch.org
>>>>>> >>>> http://www.cluecon.com
>>>>>> >>>>
>>>>>> >>>> FreeSWITCH-users mailing list
>>>>>> >>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>> >>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>> >>>> UNSUBSCRIBE:
>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-us
>>>>>> >>>> ers
>>>>>> >>>> http://www.freeswitch.org
>>>>>> >>>
>>>>>> >>>
>>>>>> >>>
>>>>>> ______________________________________________________________________
>>>>>> >>> ___ Professional FreeSWITCH Consulting Services:
>>>>>> >>> consulting at freeswitch.org
>>>>>> >>> http://www.freeswitchsolutions.com
>>>>>> >>>
>>>>>> >>> Official FreeSWITCH Sites
>>>>>> >>> http://www.freeswitch.org
>>>>>> >>> http://confluence.freeswitch.org
>>>>>> >>> http://www.cluecon.com
>>>>>> >>>
>>>>>> >>> FreeSWITCH-users mailing list
>>>>>> >>> FreeSWITCH-users at lists.freeswitch.org
>>>>>> >>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>> >>> UNSUBSCRIBE:
>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-use
>>>>>> >>> rs
>>>>>> >>> http://www.freeswitch.org
>>>>>> >>
>>>>>> >>
>>>>>> >> --
>>>>>> >> Peng Yong
>>>>>> >>
>>>>>> >>
>>>>>> _________________________________________________________________________
>>>>>> >> Professional FreeSWITCH Consulting Services:
>>>>>> >> consulting at freeswitch.org
>>>>>> >> http://www.freeswitchsolutions.com
>>>>>> >>
>>>>>> >> Official FreeSWITCH Sites
>>>>>> >> http://www.freeswitch.org
>>>>>> >> http://confluence.freeswitch.org
>>>>>> >> http://www.cluecon.com
>>>>>> >>
>>>>>> >> FreeSWITCH-users mailing list
>>>>>> >> FreeSWITCH-users at lists.freeswitch.org
>>>>>> >> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>> >> UNSUBSCRIBE:
>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>> >> http://www.freeswitch.org
>>>>>> >>
>>>>>> _________________________________________________________________________
>>>>>> >> Professional FreeSWITCH Consulting Services:
>>>>>> >> consulting at freeswitch.org
>>>>>> >> http://www.freeswitchsolutions.com
>>>>>> >>
>>>>>> >> Official FreeSWITCH Sites
>>>>>> >> http://www.freeswitch.org
>>>>>> >> http://confluence.freeswitch.org
>>>>>> >> http://www.cluecon.com
>>>>>> >>
>>>>>> >> FreeSWITCH-users mailing list
>>>>>> >> FreeSWITCH-users at lists.freeswitch.org
>>>>>> >> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>> >> UNSUBSCRIBE:
>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>> >> http://www.freeswitch.org
>>>>>> >
>>>>>> >
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _________________________________________________________________________
>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>> consulting at freeswitch.org
>>>>>> http://www.freeswitchsolutions.com
>>>>>>
>>>>>> Official FreeSWITCH Sites
>>>>>> http://www.freeswitch.org
>>>>>> http://confluence.freeswitch.org
>>>>>> http://www.cluecon.com
>>>>>>
>>>>>> FreeSWITCH-users mailing list
>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>> UNSUBSCRIBE:
>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>> http://www.freeswitch.org
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _________________________________________________________________________
>>>>> Professional FreeSWITCH Consulting Services: consulting at freeswitch.orghttp://www.freeswitchsolutions.com
>>>>>
>>>>> Official FreeSWITCH Siteshttp://www.freeswitch.orghttp://confluence.freeswitch.orghttp://www.cluecon.com
>>>>>
>>>>> FreeSWITCH-users mailing listFreeSWITCH-users at lists.freeswitch.orghttp://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-usershttp://www.freeswitch.org
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _________________________________________________________________________
>>>>> Professional FreeSWITCH Consulting Services:
>>>>> consulting at freeswitch.org
>>>>> http://www.freeswitchsolutions.com
>>>>>
>>>>> Official FreeSWITCH Sites
>>>>> http://www.freeswitch.org
>>>>> http://confluence.freeswitch.org
>>>>> http://www.cluecon.com
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:
>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> http://www.freeswitch.org
>>>>>
>>>>
>>>>
>>>>
>>>> _________________________________________________________________________
>>>> Professional FreeSWITCH Consulting Services:
>>>> consulting at freeswitch.org
>>>> http://www.freeswitchsolutions.com
>>>>
>>>> Official FreeSWITCH Sites
>>>> http://www.freeswitch.org
>>>> http://confluence.freeswitch.org
>>>> http://www.cluecon.com
>>>>
>>>> FreeSWITCH-users mailing list
>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> http://www.freeswitch.org
>>>>
>>>
>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org
>>> http://www.freeswitchsolutions.com
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://confluence.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://confluence.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://confluence.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>

-- 

*Brian West*
brian at freeswitch.org

*Twitter: @FreeSWITCH , @briankwest*
http://www.freeswitchbook.com
http://www.freeswitchcookbook.com

*T:*+19184209001 | *F:*+19184209002 | *M:*+1918424WEST (9378)
*iNUM:*+883 5100 1420 9001 | *ISN:*410*543 | *Skype:*briankwest
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20150212/8d37c950/attachment-0001.html 