[Freeswitch-users] FS priority

Sergey Safarov s.safarov at gmail.com
Wed Oct 7 17:16:06 MSD 2015 

Created PR
https://freeswitch.org/stash/projects/FS/repos/freeswitch/pull-requests/548/overview
it allow run FS daemon with correct sheduler and priority


On Wed, Oct 7, 2015 at 2:28 PM, Sergey Safarov <s.safarov at gmail.com> wrote:

> It can be made via "sudo" or may be configured via permission.
>
> On Wed, Oct 7, 2015 at 2:23 PM, Stanislav Sinyagin <ssinyagin at gmail.com>
> wrote:
>
>> yes, because it's not only about the priority. FreeSWITCH also tries to
>> set a real-time scheduler, and that requires root privileges.
>>
>>
>>
>> On Wed, Oct 7, 2015 at 1:08 PM, Sergey Safarov <s.safarov at gmail.com>
>> wrote:
>>
>>> It is PR give FS root priviledges at startup.
>>> I will create PR where FS start in unpriveledged mode
>>>
>>> https://freeswitch.org/stash/projects/FS/repos/freeswitch/pull-requests/542/overview
>>>
>>> On Wed, Oct 7, 2015 at 2:01 PM, Stanislav Sinyagin <ssinyagin at gmail.com>
>>> wrote:
>>>
>>>> there is already a pull request that sets already the proper priority
>>>>
>>>> On Wed, Oct 7, 2015 at 9:37 AM, Sergey Safarov <s.safarov at gmail.com>
>>>> wrote:
>>>>
>>>>> Nice level can be setted in
>>>>> - systemd unit file (Nice directive
>>>>> http://www.freedesktop.org/software/systemd/man/systemd.exec.html)
>>>>> - init.d file (nice command)
>>>>>
>>>>> I can create pull request for this changes. Can any body say used by
>>>>> FS nice level and other priority parameters?
>>>>>
>>>>> Sergey
>>>>>
>>>>> On Tue, Oct 6, 2015 at 6:53 AM, Sergey Safarov <s.safarov at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> What is proliority must be used?
>>>>>>
>>>>>> On Mon, Oct 5, 2015, 21:13 Stanislav Sinyagin <ssinyagin at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> in the current freeswitch.service, freewitch is started under
>>>>>>> freeswitch UID, and as a result, it fails to set itself to the proper
>>>>>>> priority.
>>>>>>>
>>>>>>> On Mon, Oct 5, 2015 at 7:13 PM, Brian West <brian at freeswitch.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> "Now the daemon starts as root and switches to freewitch UID." is
>>>>>>>> how its always done it.
>>>>>>>>
>>>>>>>> On Mon, Oct 5, 2015 at 10:27 AM, Stanislav Sinyagin <
>>>>>>>> ssinyagin at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> see
>>>>>>>>> https://freeswitch.org/stash/projects/FS/repos/freeswitch/pull-requests/542/overview
>>>>>>>>>
>>>>>>>>> it was a bit nontrivial because freeswitch.service was installed
>>>>>>>>> in an unusual way. Now the daemon starts as root and switches to freewitch
>>>>>>>>> UID.
>>>>>>>>>
>>>>>>>>> Still unresolved is https://freeswitch.org/jira/browse/FS-7937
>>>>>>>>> The package installer starts the daemon, but does not enable the
>>>>>>>>> service for starting at boot.
>>>>>>>>> I will dig into that after the merge.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Sun, Sep 6, 2015 at 6:24 PM, Bote Man <bote_radio at botecomm.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I agree IFF my assumptions and results are applicable to package
>>>>>>>>>> installations. If you have a package installation I would prefer that you
>>>>>>>>>> verify these results on such an installation since I mostly let my Master
>>>>>>>>>> build do what it wants. I always build from Master, never use the packages.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Do we have to specify the –run runtime directory on the FS
>>>>>>>>>> command line?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Do we have to specify the –temp files directory?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Right now the unit file for the package specifies none of those
>>>>>>>>>> so I don’t know where FS would put its runtime and temp files.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> By the way, while testing the location of runtime directory for
>>>>>>>>>> the PID file I noted that FS will create the ./run directory with the
>>>>>>>>>> proper permissions and owner, then write the PID file in it on its own
>>>>>>>>>> without systemd doing it. This happened without specifying anything about
>>>>>>>>>> that on the command line and without the tmpfiles.d entry.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> But since the .deb package places files in the FHS locations this
>>>>>>>>>> would be necessary, so your recent ticket that adds that applies. I omitted
>>>>>>>>>> that from my Confluence instructions built from Master, FYI.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Please let me know how to proceed.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Bote
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> *From:* Stanislav Sinyagin
>>>>>>>>>> *Sent:* Sunday, 06 September, 2015 06:00
>>>>>>>>>>
>>>>>>>>>> *Subject:* Re: [Freeswitch-users] FS priority
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Looks like another jira ticket is needed for Debian packaging.
>>>>>>>>>>
>>>>>>>>>> On Sep 6, 2015 6:16 AM, "Anthony Minessale" <
>>>>>>>>>> anthony.minessale at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>> It's because FS changes the scheduler and enables some realtime
>>>>>>>>>> threads when it can.  If you have multiple cpu np and rp are the same.  FS
>>>>>>>>>> always needs root privs to change the platform parameters and nice level
>>>>>>>>>> etc.  The scheduler change is not possible if the shell is an unpriveledged
>>>>>>>>>> user nor are a bunch of other things unless you carefully allowed them
>>>>>>>>>> somehow as root before the shell started.  So basically that is all
>>>>>>>>>> expected behavior.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Saturday, September 5, 2015, Bote Man <bote_radio at botecomm.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Sorry, I did not use the utility named ‘runas’ I simply labeled
>>>>>>>>>> the column that way and was trying to conserve character space in the
>>>>>>>>>> header to get it to fit in a reasonable space.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Anyway, thanks to your post and some research I just changed my
>>>>>>>>>>  FS unit file to start FS as user root, but specified –u freeswitch –g
>>>>>>>>>> freeswitch on the command line to FS, and changed the
>>>>>>>>>> WorkingDirectory=/usr/local/freeswitch/bin (it had been set to ‘run’) and
>>>>>>>>>> it’s doing the Right Thing, so that is what I will go with. I vaguely
>>>>>>>>>> remember that FS can (should) start as root, then drops privileges to what
>>>>>>>>>> is specified on the command line, so it looks like it is doing exactly that.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ‘top’ shows FS running as real and effective user ‘freeswitch’
>>>>>>>>>> with Priority=-2 and Nice=-10 so I am a happy camper.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> If nobody on the FS core development team has any objection to
>>>>>>>>>> this approach I will update the Confluence page for the systemd unit file
>>>>>>>>>> for building from MASTER. The Debian packages have their own file locations.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> https://freeswitch.org/confluence/display/FREESWITCH/FreeSWITCH+1.6+Video#FreeSWITCH1.6Video-systemd
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Any security concerns doing this?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks!
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Bote
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> *From:* Shaun Stokes
>>>>>>>>>> *Sent:* Saturday, 05 September, 2015 03:18
>>>>>>>>>> *Subject:* Re: [Freeswitch-users] FS priority
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Are you using FreeSwitch to specify the user to runas or is this
>>>>>>>>>> being done by systemd?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> In FreeSwitch you use the -u argument to specify the user and the
>>>>>>>>>> -g argument to specify the group, if you do this then I assume running the
>>>>>>>>>> service as root should be ok providing you've given FreeSwitch an
>>>>>>>>>> alternative user and group (in our environment we use the same for user and
>>>>>>>>>> group).
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>
>>>>>>>>>> Shaun
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ------------------------------
>>>>>>>>>>
>>>>>>>>>> *From:* Bote Man
>>>>>>>>>> *Sent:* 05 September 2015 04:28
>>>>>>>>>> *Subject:* Re: [Freeswitch-users] FS priority
>>>>>>>>>>
>>>>>>>>>> I'm not sure how much nice level matters compared to scheduler
>>>>>>>>>> priority. I ran a series of tests to find out what Priority and Nice level
>>>>>>>>>> are reported by the 'top' utility.
>>>>>>>>>>
>>>>>>>>>> I ran the first 6 tests by using systemd to start FreeSWITCH, 3
>>>>>>>>>> times as user root with each of the FS priority flags, then 3 times as user
>>>>>>>>>> freeswitch with each of the FS flags. Then I repeated that block of tests
>>>>>>>>>> from the command line, 3 flags as root, 3 flags as freeswitch. You won't
>>>>>>>>>> believe what happened next!
>>>>>>>>>>
>>>>>>>>>> systemd starting FreeSWITCH as 'RUNAS' user with 'FLAG' command
>>>>>>>>>> line priority flags to FS results in top showing priority 'PRI', nice level
>>>>>>>>>> 'NICE' on a month-old install of Debian 8 on a bare metal Dell R320 server.
>>>>>>>>>>
>>>>>>>>>> RUNAS    FLAG    PRI    NICE
>>>>>>>>>> root         -rp         -2      -10
>>>>>>>>>> root        -np         39      19
>>>>>>>>>> root         -lp         39      19
>>>>>>>>>>
>>>>>>>>>> fs            -rp         -2       19
>>>>>>>>>> fs            -np        39      19
>>>>>>>>>> fs            -lp         39      19
>>>>>>>>>>
>>>>>>>>>> Run as root from command line
>>>>>>>>>> root         -rp         -2     -10
>>>>>>>>>> root         -np        20      0
>>>>>>>>>> root         -lp         39     19
>>>>>>>>>>
>>>>>>>>>> Run as su=freeswitch from command line
>>>>>>>>>> fs            -rp        20      0
>>>>>>>>>> fs            -np       20      0
>>>>>>>>>> fs            -lp        39     19
>>>>>>>>>>
>>>>>>>>>> Most processes show Priority of 20 so I assume that is considered
>>>>>>>>>> "normal".
>>>>>>>>>>
>>>>>>>>>> So it looks like the only way to get truly higher priority for a
>>>>>>>>>> process is to run it as root, which I expected. Once the scheduler priority
>>>>>>>>>> is at -2 (higher priority) I don't know whether the nice level even matters.
>>>>>>>>>>
>>>>>>>>>> For now, the systemd unit file that I posted on Confluence runs
>>>>>>>>>> as the freeswitch user so even with the -rp flag to FreeSWITCH it gets
>>>>>>>>>> niced down to 19 which is the lowest level available for nice. Does this
>>>>>>>>>> matter?
>>>>>>>>>>
>>>>>>>>>> Is there a serious security concern running FreeSWITCH as root?
>>>>>>>>>>
>>>>>>>>>> Thanks.
>>>>>>>>>>
>>>>>>>>>> Bote
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Sep 4, 2015 at 3:38 PM, Bote Man <bote_radio at botecomm.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Thanks for that. I was under the impression that systemd was
>>>>>>>>>> throwing FreeSWITCH into the generic scheduling group and starving it of
>>>>>>>>>> resources as a result, but when I manually ran ./freeswitch as root it
>>>>>>>>>> still showed the same values.
>>>>>>>>>>
>>>>>>>>>> Running FS manually with -np yielded pri=20 nice=0 and System
>>>>>>>>>> Monitor reports priority "normal"
>>>>>>>>>>
>>>>>>>>>> Running FS manually with -rp yielded pri=-2 nice=-10 and System
>>>>>>>>>> Monitor reports priority "very high", same results as when FS was started
>>>>>>>>>> without any priority switch on the command line.
>>>>>>>>>>
>>>>>>>>>> BUT! When I start FS with systemd it maintains priority=-2 but
>>>>>>>>>> nice all the way down to 19 which is why System Monitor reports "very low".
>>>>>>>>>> This happens even with the -rp switch specified in the unit file.
>>>>>>>>>>
>>>>>>>>>> I don't know how scheduling priority and nice level interact on
>>>>>>>>>> Debian, but it looks like I have a new research project for this weekend,
>>>>>>>>>> assuming this is truly something to be concerned about. Or is it?
>>>>>>>>>>
>>>>>>>>>> Thanks for the tips. I will report my findings to the list if I
>>>>>>>>>> discover anything substantive.
>>>>>>>>>>
>>>>>>>>>> Bote
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Sep 4, 2015 at 2:02 PM, Shaun Stokes <
>>>>>>>>>> shaun.stokes at itec-support.co.uk> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Bote,
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I believe priority works in a similar way to metric (i.e. lower
>>>>>>>>>> comes first), so -20 (most favorable scheduling) to +19 (least favorable
>>>>>>>>>> scheduling).
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> -rp                    -- enable high(realtime) priority settings
>>>>>>>>>>
>>>>>>>>>> -lp                    -- enable low priority settings
>>>>>>>>>>
>>>>>>>>>> -np                    -- enable normal priority settings (system default)
>>>>>>>>>>
>>>>>>>>>> Source: https://wiki.freeswitch.org/wiki/Command_line
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Hope this helps.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>
>>>>>>>>>> Shaun
>>>>>>>>>> ------------------------------
>>>>>>>>>>
>>>>>>>>>> *From:* freeswitch-users-bounces at lists.freeswitch.org [
>>>>>>>>>> freeswitch-users-bounces at lists.freeswitch.org] on behalf of Bote
>>>>>>>>>> Man [bote_radio at botecomm.com]
>>>>>>>>>> *Sent:* 04 September 2015 15:54
>>>>>>>>>> *To:* FreeSWITCH Users Help
>>>>>>>>>> *Subject:* [Freeswitch-users] FS priority
>>>>>>>>>>
>>>>>>>>>> I’m trying to set the priority on a new FreeSWITCH installation
>>>>>>>>>> built from master on Debian 8 running on bare metal. It is currently
>>>>>>>>>> running at “very low” priority according to Resource Monitor in the GUI and
>>>>>>>>>> ‘top’ reports FS is running at priority = -2 (that’s negative two) and nice
>>>>>>>>>> = 19
>>>>>>>>>>
>>>>>>>>>> So with the way FreeSWITCH is now launched by systemd is it
>>>>>>>>>> considered a service or a user application that is simply run in the
>>>>>>>>>> background?
>>>>>>>>>>
>>>>>>>>>> This affects how systemd treats its control groups and priority
>>>>>>>>>> and how I will go about troubleshooting this.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Bote
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>>
>>>>>>>>>> Anthony Minessale II       ♬ @anthmfs  ♬ @FreeSWITCH  ♬
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ☞ http://freeswitch.org/  ☞ http://cluecon.com/  ☞
>>>>>>>>>> http://twitter.com/FreeSWITCH
>>>>>>>>>>
>>>>>>>>>> ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+
>>>>>>>>>> <http://freeswitch.org/g+>*
>>>>>>>>>>
>>>>>>>>>> ClueCon Weekly Development Call
>>>>>>>>>>
>>>>>>>>>> ☎ sip:888 at conference.freeswitch.org  ☎ +19193869900
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> https://www.youtube.com/watch?v=9XXgW34t40s
>>>>>>>>>>
>>>>>>>>>> https://www.youtube.com/watch?v=NLaDpGQuZDA
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _________________________________________________________________________
>>>>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>>>>> consulting at freeswitch.org
>>>>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>>>>
>>>>>>>>>> Official FreeSWITCH Sites
>>>>>>>>>> http://www.freeswitch.org
>>>>>>>>>> http://confluence.freeswitch.org
>>>>>>>>>> http://www.cluecon.com
>>>>>>>>>>
>>>>>>>>>> FreeSWITCH-users mailing list
>>>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>>>> UNSUBSCRIBE:
>>>>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>>>> http://www.freeswitch.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _________________________________________________________________________
>>>>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>>>>> consulting at freeswitch.org
>>>>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>>>>
>>>>>>>>>> Official FreeSWITCH Sites
>>>>>>>>>> http://www.freeswitch.org
>>>>>>>>>> http://confluence.freeswitch.org
>>>>>>>>>> http://www.cluecon.com
>>>>>>>>>>
>>>>>>>>>> FreeSWITCH-users mailing list
>>>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>>>> UNSUBSCRIBE:
>>>>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>>>> http://www.freeswitch.org
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _________________________________________________________________________
>>>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>>>> consulting at freeswitch.org
>>>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>>>
>>>>>>>>> Official FreeSWITCH Sites
>>>>>>>>> http://www.freeswitch.org
>>>>>>>>> http://confluence.freeswitch.org
>>>>>>>>> http://www.cluecon.com
>>>>>>>>>
>>>>>>>>> FreeSWITCH-users mailing list
>>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>>> UNSUBSCRIBE:
>>>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>>> http://www.freeswitch.org
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> *Brian West*
>>>>>>>> brian at freeswitch.org
>>>>>>>>
>>>>>>>>
>>>>>>>> *Twitter: @FreeSWITCH , @briankwest*
>>>>>>>> http://www.freeswitchbook.com
>>>>>>>> http://www.freeswitchcookbook.com
>>>>>>>>
>>>>>>>> Got Bugs? Report them here <https://freeswitch.org/jira>! |
>>>>>>>> Reddit: /r/freeswitch <https://www.reddit.com/r/freeswitch>
>>>>>>>>
>>>>>>>> *T:*+19184209001 | *F:*+19184209002 | *M:*+1918424WEST (9378)
>>>>>>>> *iNUM:*+883 5100 1420 9001 | *ISN:*410*543 | *Skype:*briankwest
>>>>>>>>
>>>>>>>>
>>>>>>>> _________________________________________________________________________
>>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>>> consulting at freeswitch.org
>>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>>
>>>>>>>> Official FreeSWITCH Sites
>>>>>>>> http://www.freeswitch.org
>>>>>>>> http://confluence.freeswitch.org
>>>>>>>> http://www.cluecon.com
>>>>>>>>
>>>>>>>> FreeSWITCH-users mailing list
>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>> UNSUBSCRIBE:
>>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>> http://www.freeswitch.org
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _________________________________________________________________________
>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>> consulting at freeswitch.org
>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>
>>>>>>> Official FreeSWITCH Sites
>>>>>>> http://www.freeswitch.org
>>>>>>> http://confluence.freeswitch.org
>>>>>>> http://www.cluecon.com
>>>>>>>
>>>>>>> FreeSWITCH-users mailing list
>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>> UNSUBSCRIBE:
>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>> http://www.freeswitch.org
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> _________________________________________________________________________
>>>>> Professional FreeSWITCH Consulting Services:
>>>>> consulting at freeswitch.org
>>>>> http://www.freeswitchsolutions.com
>>>>>
>>>>> Official FreeSWITCH Sites
>>>>> http://www.freeswitch.org
>>>>> http://confluence.freeswitch.org
>>>>> http://www.cluecon.com
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:
>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> http://www.freeswitch.org
>>>>>
>>>>
>>>>
>>>>
>>>> _________________________________________________________________________
>>>> Professional FreeSWITCH Consulting Services:
>>>> consulting at freeswitch.org
>>>> http://www.freeswitchsolutions.com
>>>>
>>>> Official FreeSWITCH Sites
>>>> http://www.freeswitch.org
>>>> http://confluence.freeswitch.org
>>>> http://www.cluecon.com
>>>>
>>>> FreeSWITCH-users mailing list
>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> http://www.freeswitch.org
>>>>
>>>
>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org
>>> http://www.freeswitchsolutions.com
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://confluence.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://confluence.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20151007/775d07f8/attachment-0001.html

Join us at ClueCon 2016 Aug 8-12, 2016
More information about the FreeSWITCH-users mailing list