<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Segoe UI Symbol";
panose-1:2 11 5 2 4 2 4 2 2 3;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.hoenzb
{mso-style-name:hoenzb;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Perhaps write a patch to detect these limits and then CRIT on start if they conflict with settings.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> freeswitch-users-bounces@lists.freeswitch.org [mailto:freeswitch-users-bounces@lists.freeswitch.org]
<b>On Behalf Of </b>Vladislav Ivanov<br>
<b>Sent:</b> Tuesday, 15 December, 2015 2:15<br>
<b>To:</b> FreeSWITCH Users Help <freeswitch-users@lists.freeswitch.org><br>
<b>Subject:</b> Re: [Freeswitch-users] Thread failure with low CPU utilization?<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Hey guys,<br>
<br>
Anthony was correct in this case, I ran freeswitch not from root, for some reason my /etc/security/limits.conf file line:<o:p></o:p></p>
<div>
<p class="MsoNormal">* soft nproc 32768<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">* hard nproc 32768<br>
<br>
was not propagated on freeswitch user... <br>
<br>
Added new line:<o:p></o:p></p>
<div>
<p class="MsoNormal">freeswitch soft nproc 32768<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">freeswitch hard nproc 32768<br>
<br>
and it worked fine from that.<br>
Thanks to you all!<o:p></o:p></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">2015-12-14 23:06 GMT+02:00 Anthony Minessale <<a href="mailto:anthony.minessale@gmail.com" target="_blank">anthony.minessale@gmail.com</a>>:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<p class="MsoNormal">Are you running FS as root or a regular user, you need to check the ulimits of concurrent threads and some boxes have a hard limit of threads, its not all tied to CPU usage.<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Mon, Dec 14, 2015 at 1:30 PM, Kevin Wormington <<a href="mailto:kworm@sofnet.com" target="_blank">kworm@sofnet.com</a>> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">This post<br>
<a href="http://unix.stackexchange.com/questions/47595/linux-max-threads-count" target="_blank">http://unix.stackexchange.com/questions/47595/linux-max-threads-count</a><br>
has a simple program to check out how many threads can be created. You<br>
can run the test program as root and then as FS user and see if it's a<br>
ulimit type of problem.<br>
<br>
On 12/14/2015 01:08 PM, Michael Giagnocavo wrote:<br>
> Is there a reason to think that CentOS 6 would cause threads to fail to<br>
> spawn? This is basically just calling libapr, which is code from like 8<br>
> years ago.<br>
><br>
> Vladislav: This means it failed to create a thread (switch_thread_create<br>
> in switch_apr.c). FS generates a ton of threads. The only time I’ve seen<br>
> this was a long time ago on 32-bit Windows, since default stack of 1MB<br>
> means you hit address space limits pretty quickly. But on 64-bit, you<br>
> should be able to go all out.<br>
><br>
> I’d start by adding some if’d printf’s to check the various things that<br>
> switch_thread_create does and try to pinpoint what is failing. If the<br>
> libapr call is failing, then you should be able to create a minimal<br>
> program that just spawns threads and see if that one is able to get<br>
> 1000+. From there I’d guess troubleshooting is gonna be the same as<br>
> figuring out why pthread_create fails and that should be generic to your<br>
> system. (Please post if you figure out why.)<br>
><br>
> If other apps are able to spawn and keep thousands of threads and just<br>
> FS isn’t then I guess you’re unlucky?<br>
><br>
> -Michael<br>
><br>
> *From:*<a href="mailto:freeswitch-users-bounces@lists.freeswitch.org" target="_blank">freeswitch-users-bounces@lists.freeswitch.org</a><br>
> [mailto:<a href="mailto:freeswitch-users-bounces@lists.freeswitch.org" target="_blank">freeswitch-users-bounces@lists.freeswitch.org</a>] *On Behalf Of<br>
> *Giovanni Maruzzelli<br>
> *Sent:* Monday, 14 December, 2015 10:57<br>
> *To:* FreeSWITCH Users Help <<a href="mailto:freeswitch-users@lists.freeswitch.org" target="_blank">freeswitch-users@lists.freeswitch.org</a>><br>
> *Subject:* Re: [Freeswitch-users] Thread failure with low CPU utilization?<br>
><br>
> centos 6 is not supported by FreeSWITCH 1.6 !<br>
><br>
> -giovanni<br>
><br>
> On Mon, Dec 14, 2015 at 5:47 PM, Vladislav Ivanov <<a href="mailto:deforceczt@gmail.com" target="_blank">deforceczt@gmail.com</a><br>
> <mailto:<a href="mailto:deforceczt@gmail.com" target="_blank">deforceczt@gmail.com</a>>> wrote:<br>
><br>
> Centos 6,latest update. Via init script<br>
><br>
> On 14 Dec 2015 6:25 p.m., "Ken Rice" <<a href="mailto:krice@freeswitch.org" target="_blank">krice@freeswitch.org</a><br>
> <mailto:<a href="mailto:krice@freeswitch.org" target="_blank">krice@freeswitch.org</a>>> wrote:<br>
><br>
> A)What platform are you running FreeSWITCH on? (eg: Debian 8 etc)<br>
><br>
> b)How are you starting FreeSWITCH? (eg: in the foreground or<br>
> background with an init script)<br>
><br>
> *From:*<a href="mailto:freeswitch-users-bounces@lists.freeswitch.org" target="_blank">freeswitch-users-bounces@lists.freeswitch.org</a><br>
> <mailto:<a href="mailto:freeswitch-users-bounces@lists.freeswitch.org" target="_blank">freeswitch-users-bounces@lists.freeswitch.org</a>><br>
> [mailto:<a href="mailto:freeswitch-users-bounces@lists.freeswitch.org" target="_blank">freeswitch-users-bounces@lists.freeswitch.org</a><br>
> <mailto:<a href="mailto:freeswitch-users-bounces@lists.freeswitch.org" target="_blank">freeswitch-users-bounces@lists.freeswitch.org</a>>] *On<br>
> Behalf Of *Vladislav Ivanov<br>
> *Sent:* Monday, December 14, 2015 9:47 AM<br>
> *To:* <a href="mailto:freeswitch-users@lists.freeswitch.org" target="_blank">
freeswitch-users@lists.freeswitch.org</a><br>
> <mailto:<a href="mailto:freeswitch-users@lists.freeswitch.org" target="_blank">freeswitch-users@lists.freeswitch.org</a>><br>
> *Subject:* [Freeswitch-users] Thread failure with low CPU<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal">> utilization?<br>
><br>
> Hello,<br>
><br>
> I have 1.6.5 Freeswitch and I'm trying to test how much<br>
> lines/cps it can handle on my test machine:<br>
><br>
> CPU: Intel(R) Xeon(R) CPU E5450 @ 3.00GHz x2 (8 cores, 3 GHz)<br>
> RAM: 32GB<br>
> SSD: 500 GB<br>
><br>
> I'm using call generator to check flood the freeswitch, but when<br>
> FS gets 1k sessions it sends<br>
> [CRIT] switch_core_session.c:1781 Thread Failure!<br>
> and the Star Wars quote... But the CPU load is less than 10%<br>
><br>
> Ulimit is as follows:<br>
><br>
> core file size (blocks, -c) 0<br>
><br>
> data seg size (kbytes, -d) unlimited<br>
><br>
> scheduling priority (-e) 0<br>
><br>
> file size (blocks, -f) unlimited<br>
><br>
> pending signals (-i) 128330<br>
><br>
> max locked memory (kbytes, -l) 64<br>
><br>
> max memory size (kbytes, -m) unlimited<br>
><br>
> open files (-n) 32768<br>
><br>
> pipe size (512 bytes, -p) 8<br>
><br>
> POSIX message queues (bytes, -q) 819200<br>
><br>
> real-time priority (-r) 0<br>
><br>
> stack size (kbytes, -s) 2048<br>
><br>
> cpu time (seconds, -t) unlimited<br>
><br>
> max user processes (-u) 32768<br>
><br>
> virtual memory (kbytes, -v) unlimited<br>
><br>
> file locks (-x) unlimited<br>
><br>
><br>
> Server is physical, not virtual, but I dont have the same issue<br>
> with virtual freeswitch.<br>
><br>
> Here is the "status" before the session limit:<br>
><br>
> freeswitch@internal> status<br>
><br>
> UP 0 years, 0 days, 0 hours, 0 minutes, 56 seconds, 137<br>
> milliseconds, 603 microseconds<br>
><br>
> FreeSWITCH (Version 1.6.5 git 70b8c17 2015-11-20 20:57:50Z<br>
> 64bit) is ready<br>
><br>
> 994 session(s) since startup<br>
><br>
> 994 session(s) - peak 994, last 5min 994<br>
><br>
> 40 session(s) per Sec out of max 200, peak 41, last 5min 41<br>
><br>
> 10000 session(s) max<br>
><br>
> min idle cpu 10.00/96.77<br>
><br>
> Current Stack Size/Max 2048K/2048K<br>
><br>
> freeswitch@internal> status<br>
><br>
> UP 0 years, 0 days, 0 hours, 0 minutes, 56 seconds, 370<br>
> milliseconds, 877 microseconds<br>
><br>
> FreeSWITCH (Version 1.6.5 git 70b8c17 2015-11-20 20:57:50Z<br>
> 64bit) is ready<br>
><br>
> 1003 session(s) since startup<br>
><br>
> 1003 session(s) - peak 1003, last 5min 1003<br>
><br>
> 40 session(s) per Sec out of max 200, peak 41, last 5min 41<br>
><br>
> 993 session(s) max<br>
><br>
> min idle cpu 10.00/96.77<br>
><br>
> Current Stack Size/Max 2048K/2048K<br>
><br>
> freeswitch@internal> 2015-12-14 15:37:37.436968 [CRIT]<br>
> switch_core_session.c:1781 Thread Failure!<br>
><br>
> 2015-12-14 15:37:37.436968 [CRIT] switch_core_session.c:1737<br>
> LUKE: I'm hit, but not bad.<br>
><br>
> 2015-12-14 15:37:37.436968 [CRIT] switch_core_session.c:1738<br>
> LUKE'S VOICE: Artoo, see what you can do with it. Hang on back<br>
> there....<br>
><br>
> Green laserfire moves past the beeping little robot as his head<br>
> turns. After a few beeps and a twist of his mechanical arm,<br>
><br>
> Artoo reduces the max sessions to 993 thus, saving the switch<br>
> from certain doom.<br>
><br>
> Any help with that? Really strange error...<br>
><br>
> _________________________________________________________________________<br>
> Professional FreeSWITCH Consulting Services:<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal">> <a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a> <mailto:<a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a>><br>
> <a href="http://www.freeswitchsolutions.com" target="_blank">http://www.freeswitchsolutions.com</a><br>
><br>
> Official FreeSWITCH Sites<br>
> <a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
> <a href="http://confluence.freeswitch.org" target="_blank">http://confluence.freeswitch.org</a><br>
> <a href="http://www.cluecon.com" target="_blank">http://www.cluecon.com</a><br>
><br>
> FreeSWITCH-users mailing list<br>
> <a href="mailto:FreeSWITCH-users@lists.freeswitch.org" target="_blank">FreeSWITCH-users@lists.freeswitch.org</a><br>
> <mailto:<a href="mailto:FreeSWITCH-users@lists.freeswitch.org" target="_blank">FreeSWITCH-users@lists.freeswitch.org</a>><br>
> <a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
> UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br>
> <a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
><br>
><br>
> _________________________________________________________________________<br>
> Professional FreeSWITCH Consulting Services:<br>
> <a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a> <mailto:<a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a>><br>
> <a href="http://www.freeswitchsolutions.com" target="_blank">http://www.freeswitchsolutions.com</a><br>
><br>
> Official FreeSWITCH Sites<br>
> <a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
> <a href="http://confluence.freeswitch.org" target="_blank">http://confluence.freeswitch.org</a><br>
> <a href="http://www.cluecon.com" target="_blank">http://www.cluecon.com</a><br>
><br>
> FreeSWITCH-users mailing list<br>
> <a href="mailto:FreeSWITCH-users@lists.freeswitch.org" target="_blank">FreeSWITCH-users@lists.freeswitch.org</a><br>
> <mailto:<a href="mailto:FreeSWITCH-users@lists.freeswitch.org" target="_blank">FreeSWITCH-users@lists.freeswitch.org</a>><o:p></o:p></p>
<div>
<div>
<p class="MsoNormal">> <a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
> UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br>
> <a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
><br>
><br>
><br>
><br>
> --<br>
><br>
> Sincerely,<br>
><br>
> Giovanni Maruzzelli<br>
> Cell : <a href="tel:%2B39-347-2665618" target="_blank">+39-347-2665618</a><br>
><br>
><br>
><br>
> _________________________________________________________________________<br>
> Professional FreeSWITCH Consulting Services:<br>
> <a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a><br>
> <a href="http://www.freeswitchsolutions.com" target="_blank">http://www.freeswitchsolutions.com</a><br>
><br>
> Official FreeSWITCH Sites<br>
> <a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
> <a href="http://confluence.freeswitch.org" target="_blank">http://confluence.freeswitch.org</a><br>
> <a href="http://www.cluecon.com" target="_blank">http://www.cluecon.com</a><br>
><br>
> FreeSWITCH-users mailing list<br>
> <a href="mailto:FreeSWITCH-users@lists.freeswitch.org" target="_blank">FreeSWITCH-users@lists.freeswitch.org</a><br>
> <a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
> UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br>
> <a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
><br>
<br>
_________________________________________________________________________<br>
Professional FreeSWITCH Consulting Services:<br>
<a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a><br>
<a href="http://www.freeswitchsolutions.com" target="_blank">http://www.freeswitchsolutions.com</a><br>
<br>
Official FreeSWITCH Sites<br>
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
<a href="http://confluence.freeswitch.org" target="_blank">http://confluence.freeswitch.org</a><br>
<a href="http://www.cluecon.com" target="_blank">http://www.cluecon.com</a><br>
<br>
FreeSWITCH-users mailing list<br>
<a href="mailto:FreeSWITCH-users@lists.freeswitch.org" target="_blank">FreeSWITCH-users@lists.freeswitch.org</a><br>
<a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br>
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><o:p></o:p></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><br>
<br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
<p class="MsoNormal"><span class="hoenzb"><span style="color:#888888">-- <o:p></o:p></span></span></p>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="color:#888888">Anthony Minessale II </span>
<span style="font-family:"Segoe UI Symbol",sans-serif;color:#888888">♬</span><span style="color:#888888"> @anthmfs </span><span style="font-family:"Segoe UI Symbol",sans-serif;color:#888888">♬</span><span style="color:#888888"> @FreeSWITCH </span><span style="font-family:"Segoe UI Symbol",sans-serif;color:#888888">♬</span><o:p></o:p></p>
<div>
<p class="MsoNormal"><span style="color:#888888"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-family:"Segoe UI Symbol",sans-serif;color:#888888">☞</span><span style="color:#888888">
<a href="http://freeswitch.org/" target="_blank">http://freeswitch.org/</a> </span><span style="font-family:"Segoe UI Symbol",sans-serif;color:#888888">☞</span><span style="color:#888888">
<a href="http://cluecon.com/" target="_blank">http://cluecon.com/</a> </span><span style="font-family:"Segoe UI Symbol",sans-serif;color:#888888">☞</span><span style="color:#888888">
<a href="http://twitter.com/FreeSWITCH" target="_blank">http://twitter.com/FreeSWITCH</a><o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Segoe UI Symbol",sans-serif;color:#888888">☞</span><span style="color:#888888">
<a href="http://irc.freenode.net" target="_blank">irc.freenode.net</a> #freeswitch </span><span style="font-family:"Segoe UI Symbol",sans-serif;color:#888888">☞</span><span style="color:#888888">
<u><a href="http://freeswitch.org/g+" target="_blank">http://freeswitch.org/g+</a></u><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#888888">ClueCon Weekly Development Call <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Segoe UI Symbol",sans-serif;color:#888888">☎</span><span style="color:#888888"> <a href="mailto:sip%3A888@conference.freeswitch.org" target="_blank">sip:888@conference.freeswitch.org</a> </span><span style="font-family:"Segoe UI Symbol",sans-serif;color:#888888">☎</span><span style="color:#888888">
<a href="tel:%2B19193869900" target="_blank">+19193869900</a> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#888888"><o:p> </o:p></span></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal"><span style="color:#888888"><a href="https://www.youtube.com/watch?v=9XXgW34t40s" target="_blank"><span style="font-size:9.5pt;color:#1155CC">https://www.youtube.com/watch?v=9XXgW34t40s</span></a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#888888"><a href="https://www.youtube.com/watch?v=NLaDpGQuZDA" target="_blank">https://www.youtube.com/watch?v=NLaDpGQuZDA</a><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><br>
_________________________________________________________________________<br>
Professional FreeSWITCH Consulting Services:<br>
<a href="mailto:consulting@freeswitch.org">consulting@freeswitch.org</a><br>
<a href="http://www.freeswitchsolutions.com" target="_blank">http://www.freeswitchsolutions.com</a><br>
<br>
Official FreeSWITCH Sites<br>
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
<a href="http://confluence.freeswitch.org" target="_blank">http://confluence.freeswitch.org</a><br>
<a href="http://www.cluecon.com" target="_blank">http://www.cluecon.com</a><br>
<br>
FreeSWITCH-users mailing list<br>
<a href="mailto:FreeSWITCH-users@lists.freeswitch.org">FreeSWITCH-users@lists.freeswitch.org</a><br>
<a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br>
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><o:p></o:p></p>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</body>
</html>