<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>After all day of testing yesterday (max 100 cps) it appears that odbc for windows won’t work. It must be a bug because if I set the core db, profiles etc. to use odbc to MSSQL it crashes after 1 or 2 minutes of cps of 50. If I remove that odbc connection and use the built in databases it works like a champ. Yes I do still see a small memory leak still so that will need to be discovered and fixed.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> I think it’s what you said about too many transitions and it can’t keep up.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Andre<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>From:</span></b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'> freeswitch-users-bounces@lists.freeswitch.org [mailto:freeswitch-users-bounces@lists.freeswitch.org] <b>On Behalf Of </b>Steven Ayre<br><b>Sent:</b> Friday, November 22, 2013 9:13 AM<br><b>To:</b> FreeSWITCH Users Help<br><b>Subject:</b> Re: [Freeswitch-users] Freeswitch Performance<o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>It didn't explictly say 'pause'... what i noticed grep-ing the log for the uuid of a call's that had hung was that it was going through a sequence of log messages that are usually <1ms apart and instead were ~30s apart.<o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>-Steve<o:p></o:p></p></div></div><div><p class=MsoNormal style='margin-bottom:12.0pt'><o:p> </o:p></p><div><p class=MsoNormal>On 20 November 2013 21:56, Andre Demattia <<a href="mailto:andretodd@verizon.net" target="_blank">andretodd@verizon.net</a>> wrote:<o:p></o:p></p><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in'><div><div><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>I had debug turned off but I'll enable it to see if anything interesting happens.<br>I didn't see anything that said pause.<br><br>I'll also turn off odbc for the core db and use the default one to see if that helps. Un testing it didn't help but I'll try it in again. <o:p></o:p></span></p></div></div><div><div class=MsoNormal align=center style='text-align:center'><hr size=3 width="100%" align=center></div><p class=MsoNormal><b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>From: </span></b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'><a href="mailto:steveayre@gmail.com" target="_blank">Steven Ayre</a></span><br><b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>Sent: </span></b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>11/20/2013 4:36 PM</span><br><b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>To: </span></b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'><a href="mailto:freeswitch-users@lists.freeswitch.org" target="_blank">FreeSWITCH Users Help</a></span><o:p></o:p></p><div><p class=MsoNormal style='margin-bottom:12.0pt'><br><b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>Subject: </span></b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>Re: [Freeswitch-users] Freeswitch Performance</span><o:p></o:p></p></div></div><p class=MsoNormal>><o:p> </o:p></p><div><p class=MsoNormal><br>> show nothing other than FreeSWITCH was receiving the new messages but no<br>> bridge or cancel actions were taking place<br><br><br>Do you have your logs at debug level and are there any log messages that<br>indicate a call reaching a point then pausing?<br><br>I saw something recently similar under load, and it turned out to be the<br>limit subsystem was waiting on DB queries that were severely backing up<br>because there were too many commits happening per second and InnoDB was<br>configured to flush to disk on every transaction commit which can only<br>happen ~200 times per second. (Switching to the memory engine fixed the<br>error although isn't transactional).<br><br>-Steve<br><br>-Steve<br><br><br>On 20 November 2013 20:38, Andre <<a href="mailto:andretodd@verizon.net" target="_blank">andretodd@verizon.net</a>> wrote:<br><br>> I tested again and when I use 25 cps for 2 hours everything worked fine.<br>> When I sent 50 cps I noticed FreeSWITCH worked for about 5 minutes then I<br>> saw the memory jump from 122 mb to to 1.6 gb in seconds. I think FreeSWITCH<br>> crashed but the logs show nothing other than FreeSWITCH was receiving the<br>> new messages but no bridge or cancel actions were taking place.<br>><br>><br>><br>> How do I tell what happened that made freeswitch just stop working? The<br>> GUI didn’t show anything like a throw exception and when I type version or<br>> sofia status in the FreeSWITCH gui it doesn’t respond.<br>><br>><br>><br>> Andre<br>><br>><br>><o:p></o:p></p></div><p class=MsoNormal>> *From:* <a href="mailto:freeswitch-users-bounces@lists.freeswitch.org" target="_blank">freeswitch-users-bounces@lists.freeswitch.org</a> [mailto:<br>> <a href="mailto:freeswitch-users-bounces@lists.freeswitch.org" target="_blank">freeswitch-users-bounces@lists.freeswitch.org</a>] *On Behalf Of *Anthony<br>> Minessale<br>> *Sent:* Wednesday, November 20, 2013 12:33 AM<br>> *To:* Freeswitch-users<br>><br>> *Subject:* Re: [Freeswitch-users] Freeswitch Performance<o:p></o:p></p><div><div><p class=MsoNormal style='margin-bottom:12.0pt'><br>><br>><br>><br>> Get HEAD of stable branch or HEAD of master for latest build files.<br>><br>> There is no promise windows can do 500cps the industry standard is 50 and<br>> you'll have probably more luck on linux but there is only on way to fimd<br>> out.<br>><br>> On Nov 19, 2013 11:27 PM, "I put the Who? in Mishehu" <<br>> <a href="mailto:mishehu@freeswitch.org" target="_blank">mishehu@freeswitch.org</a>> wrote:<br>><br>> The amusing thing about the recurring load test threads that pop up every<br>> now and then is that the most important tips are already documented on the<br>> wiki in one single page. Beyond that, you probably spend more time load<br>> testing than you do just tossing up another server and using something like<br>> kamailio or opensips to load balance...<br>><br>><br>> --<br>><br>> Yossi Neiman<br>><br>><br>><br>><br>><br>> On 11/19/2013 06:48 PM, Antonio Teixeira wrote:<br>><br>> Hello Andre.<br>><br>> Solving performance problems is always hard , but a few points to keep in<br>> mind :<br>><br>> To my knowledge FS is pretty much a linux software , it works on windows<br>> by that i doesn't means it performs on windows ( please correct me if i'm<br>> wrong)<br>> The native DB for FS is Postgresql ( this should make you oracle admin<br>> more happy :) ).<br>><br>> Try this :<br>> Linux Server Realtime Kernel + Last FS Stable + Postgresql ( tune it well).<br>> Take care with ulimit on linux , its needed for the extra punch of<br>> performance<br>> Check if your "performance test" is real if you are pointing to 1500CPS<br>> for a two minute call you will have 180k calls before the first call gets<br>> disconnected, if you are also testing RTP that means 180k * 2 ports needed<br>> , The Kernel will start to complaining and you will run out of ports you<br>> will also run out of FD probably before you run out of ports on a not tuned<br>> linux install.<br>> If the DB is on the same server it will also stall since the FD will not<br>> be present for new connections to the back end.<br>><br>> In terms of BW you will exceed 230000 kb/s add a few more kb/s for the DB<br>> ( Not a problem here i think)<br>><br>> Another thing , no sip server ( please include all the nice names here<br>> like B2BUA , etc ) should ever queue anything.<br>> I have never read the code that handles this on FS but the way i would do<br>> it should be something like : can we sustain the load ? No in that case<br>> send 503 else proceed.<br>> This is done in freeswitch by this var<br>> <a href="https://wiki.freeswitch.org/wiki/XML_Switch_Configuration#max-sessions" target="_blank">https://wiki.freeswitch.org/wiki/XML_Switch_Configuration#max-sessions</a><br>><br>> A more general approach:<br>> In SIp Servers you will generally ( if not running cpu intensive tasks,<br>> trancoding , tone detection , etc ), consume all the ram , Network IO ,than<br>> the cpu.<br>> For the guys that plan something big you should take a look at this :<br>> <a href="http://www.solarflare.com/High-Frequency-Trading" target="_blank">http://www.solarflare.com/High-Frequency-Trading</a><br>> Not by The NIC itself but because of this :<br>> <a href="http://www.openonload.org" target="_blank">http://www.openonload.org</a><br>><br>> Without more data i don't think we can help , above all performance<br>> testing is a dark art you just need to keep on trying different things and<br>> above all have some type of monitor tool to understand what exactly is<br>> holding you up , maybe its no FS...<br>><br>> Also 1500CPS is something seen in carrier operations generally with a<br>> named SBC in front , keep in mind that FS is a great software but it<br>> doesn't hold against SBC's that use special processors like ASIC to perform<br>> transcoding , sip parsing , tone detection ,etc etc<br>><br>><br>> Disclaimer : post written deep into the night...<br>><br>> On 20/11/13 00:09, Andre wrote:<br>><br>> Hi, I am testing out my FreeSWITCH 1.2.14 64 bit windows 8.1 on my<br>> Alienware Laptop with a very fast SSD and I7 processor with 8 GB DDR3 Ram.<br>><br>><br>><br>> I have been using a pjsip test tool to test FreeSWITCH to see how it<br>> performs. Tell me if I’m wrong here, I believe if I sent “too many” calls<br>> to FreeSWITCH that it queues up the calls and sooner or later uses all the<br>> memory on my PC then just stops working. I do have the CPS sent to 1500<br>> for testing but I never hit that high, I hit about 200 CPS.<br>><br>><br>><br>> The CDRs that save to my database show it takes about 8 seconds from start<br>> time to end time. It does start out only taking milliseconds then it<br>> progresses to the 8 second mark.<br>><br>><br>><br>> The calls are just going to FreeSWITCH then I’m sending back a 503.<br>><br>><br>><br>> I have every mod turned off, logging set to 0, Core db going to a SQL<br>> Server and I even set FreeSWITCH to real time.<br>><br>><br>><br>> Can anyone give me advise on what I can do to get better performance?<br>> I’ve also tested it out on my real server but in about 3 minutes of 50 cps<br>> FreeSWITCH uses up 3gb of memory and stops responding.<br>><br>><br>><br>> I’m at a loss here on what else to do.<br>><br>> Andre<br>><br>><br>><br>> _________________________________________________________________________<br>><br>> Professional FreeSWITCH Consulting Services:<br>><br>> <a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a><br>><br>> <a href="http://www.freeswitchsolutions.com" target="_blank">http://www.freeswitchsolutions.com</a><br>><br>><br>><br>> FreeSWITCH-powered IP PBX: The CudaTel Communication Server<br>><br>> <a href="http://www.cudatel.com" target="_blank">http://www.cudatel.com</a><br>><br>><br>><br>> Official FreeSWITCH Sites<br>><br>> <a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>><br>> <a href="http://wiki.freeswitch.org" target="_blank">http://wiki.freeswitch.org</a><br>><br>> <a href="http://www.cluecon.com" target="_blank">http://www.cluecon.com</a><br>><br>><br>><br>> FreeSWITCH-users mailing list<br>><br>> <a href="mailto:FreeSWITCH-users@lists.freeswitch.org" target="_blank">FreeSWITCH-users@lists.freeswitch.org</a><br>><br>> <a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>><br>> UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br>><br>> <a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>><br>><br>><br>> _________________________________________________________________________<br>><br>> Professional FreeSWITCH Consulting Services:<br>><br>> <a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a><br>><br>> <a href="http://www.freeswitchsolutions.com" target="_blank">http://www.freeswitchsolutions.com</a><br>><br>><br>><br>> FreeSWITCH-powered IP PBX: The CudaTel Communication Server<br>><br>> <a href="http://www.cudatel.com" target="_blank">http://www.cudatel.com</a><br>><br>><br>><br>> Official FreeSWITCH Sites<br>><br>> <a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>><br>> <a href="http://wiki.freeswitch.org" target="_blank">http://wiki.freeswitch.org</a><br>><br>> <a href="http://www.cluecon.com" target="_blank">http://www.cluecon.com</a><br>><br>><br>><br>> FreeSWITCH-users mailing list<br>><br>> <a href="mailto:FreeSWITCH-users@lists.freeswitch.org" target="_blank">FreeSWITCH-users@lists.freeswitch.org</a><br>><br>> <a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><o:p></o:p></p></div></div><div><p class=MsoNormal>[The entire original message is not included.]<o:p></o:p></p></div></div><p class=MsoNormal style='margin-bottom:12.0pt'><br>_________________________________________________________________________<br>Professional FreeSWITCH Consulting Services:<br><a href="mailto:consulting@freeswitch.org">consulting@freeswitch.org</a><br><a href="http://www.freeswitchsolutions.com" target="_blank">http://www.freeswitchsolutions.com</a><br><br>FreeSWITCH-powered IP PBX: The CudaTel Communication Server<br><a href="http://www.cudatel.com" target="_blank">http://www.cudatel.com</a><br><br>Official FreeSWITCH Sites<br><a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br><a href="http://wiki.freeswitch.org" target="_blank">http://wiki.freeswitch.org</a><br><a href="http://www.cluecon.com" target="_blank">http://www.cluecon.com</a><br><br>FreeSWITCH-users mailing list<br><a href="mailto:FreeSWITCH-users@lists.freeswitch.org">FreeSWITCH-users@lists.freeswitch.org</a><br><a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br><a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><o:p></o:p></p></blockquote></div><p class=MsoNormal><o:p> </o:p></p></div></div></body></html>