<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Tony and Andre,<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Rather than going back and forth on this matter, here are the instructions on how you can find out exactly what is going on. <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>On your target system, make sure you have the sysinternals tools installed, and in your path. This will enable you to use procdump from the command line. When FS gets into the state with an assload of hung threads, use procdump to take a full dump. This will be a very large file.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Copy the .dmp file from the target box, to the build box, if they’re the same then forgot about copying it. Now on your build box, open the FS solution in VS. Then in VS from the File->Open menu, open the .DMP file. This will put you in the debugger, just as if you were running FS under a debugger.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Check the Threads windows and see if a buttload of threads are all hung in the same place. Double clicking on the thread will push that thread into the call stack window. Now you can see where the thread is hung. If it’s in a kernel call go up the strack till you get to code in FS. Double click on that stack frame, that will open you to the line of source code at the point it got hung. In the source window it’s just like in the debugger. Hover over any variable to see it’s value. <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>This is the same process for doing it under linux, but rather than using GDB you have the full VS GUI debugger experience. Tony, you can quote him the same if he gives you access to the build box, as you would for someone that gave you access to a dump and GDB. <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Because you most likely will have a lot of threads all hung in the same place, and threads call stack, and source code are in different windows, this should take about 10 minutes to find out what’s going on.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>--Dave<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>From:</span></b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'> freeswitch-users-bounces@lists.freeswitch.org [mailto:freeswitch-users-bounces@lists.freeswitch.org] <b>On Behalf Of </b>Anthony Minessale<br><b>Sent:</b> Sunday, August 17, 2014 2:53 PM<br><b>To:</b> Freeswitch-users<br><b>Subject:</b> Re: [Freeswitch-users] high pdd<o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p><p>When you refer to more getting performance as getting it fixed, it strikes a nerve.<o:p></o:p></p><p>Start with what I said and test for same results on debian 7 to have clarity if you have a windows specific performance limitation. There might be up to 1 million lines of code difference between windows and *nix. We can't even tell you how much to pay unless we know there is something to change. Also there is no indication you are using realistic traffic to test. All of these details are critical.<o:p></o:p></p><div><p class=MsoNormal>On Aug 17, 2014 1:43 PM, "Andre Demattia" <<a href="mailto:andretodd@verizon.net">andretodd@verizon.net</a>> wrote:<o:p></o:p></p><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in'><div><div><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>Thanks, I'm using the newest master and the last stable 1.2.23<br>We are willing to pay to get this fixed.<br>Andre <o:p></o:p></span></p></div></div><div><div class=MsoNormal align=center style='text-align:center'><hr size=2 width="100%" align=center></div><p class=MsoNormal style='margin-bottom:12.0pt'><b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>From: </span></b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'><a href="mailto:anthony.minessale@gmail.com" target="_blank">Anthony Minessale</a></span><br><b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>Sent: </span></b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>8/17/2014 12:30 PM</span><br><b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>To: </span></b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'><a href="mailto:freeswitch-users@lists.freeswitch.org" target="_blank">Freeswitch-users</a></span><br><b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>Subject: </span></b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>Re: [Freeswitch-users] high pdd</span><o:p></o:p></p></div><p>We dont have a lot of tools to troubleshoot windows. You should mention windows in your subject to have a better chance to get the attention of our windows users. <o:p></o:p></p><p>You should consider installing debian 7 on the same hardware and perform similar tests to help determine if your issue is o/s specific or not.<o:p></o:p></p><p>You also have not indicated the revision of FS. If you have not worked with latest master, you should be also to rule out improvements. <o:p></o:p></p><div><p class=MsoNormal>On Aug 17, 2014 9:54 AM, "Andre Demattia" <<a href="mailto:andretodd@verizon.net" target="_blank">andretodd@verizon.net</a>> wrote:<o:p></o:p></p><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in'><div><div><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>Yes that's off and I believe all the databases are off too. No authentication or registration is enabled. The sizes of the database are 0 until I call the shutdown command then its about 90k. I have the DBS in Ra drive too.<br><br>Windows seems pretty good until the 3000 sessions per min mark. Then I just get long pdd and the threads get backed up then no more sessions can be created. <br><br>Funny thing is I has 10 profiles and the same results. I also tried two consoles with the same results.<br><br>FreeSWITCH is great.<br><br>I'd be grateful for help. I've worked on this for 2 years and hate to give up. <br><br>If any one from <a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a> is willing to help I'd appreciate that.<br><br>Thanks again,<br>Andre <o:p></o:p></span></p></div></div><div><div class=MsoNormal align=center style='text-align:center'><hr size=2 width="100%" align=center></div><p class=MsoNormal style='margin-bottom:12.0pt'><b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>From: </span></b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'><a href="mailto:anthony.minessale@gmail.com" target="_blank">Anthony Minessale</a></span><br><b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>Sent: </span></b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>8/17/2014 10:13 AM</span><br><b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>To: </span></b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'><a href="mailto:freeswitch-users@lists.freeswitch.org" target="_blank">Freeswitch-users</a></span><br><b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>Subject: </span></b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>Re: [Freeswitch-users] high pdd</span><o:p></o:p></p></div><p>Did you comment out manage-presence ? Also we have no real idea how well FS performs in windows under extreme load.<o:p></o:p></p><div><p class=MsoNormal>On Aug 17, 2014 5:42 AM, "Andre Demattia" <<a href="mailto:andretodd@verizon.net" target="_blank">andretodd@verizon.net</a>> wrote:<o:p></o:p></p><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in'><div><div><div><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>Thanks Ken, I'll try the consulting again.<br>Andre <o:p></o:p></span></p></div></div><div><div class=MsoNormal align=center style='text-align:center'><hr size=2 width="100%" align=center></div><p class=MsoNormal style='margin-bottom:12.0pt'><b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>From: </span></b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'><a href="mailto:krice@freeswitch.org" target="_blank">Ken Rice</a></span><br><b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>Sent: </span></b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>8/17/2014 1:19 AM</span><br><b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>To: </span></b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'><a href="mailto:freeswitch-users@lists.freeswitch.org" target="_blank">FreeSWITCH Users Help</a></span><br><b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>Subject: </span></b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>Re: [Freeswitch-users] high pdd</span><o:p></o:p></p></div><div><p class=MsoNormal style='margin-bottom:12.0pt'>if the last post does not work contact <a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a> for pro help from one of the developers... thats about as far as the list will take you<o:p></o:p></p><div><p class=MsoNormal>Ken<o:p></o:p></p></div><p class=MsoNormal>Sent from my iPad<o:p></o:p></p></div><div><p class=MsoNormal style='margin-bottom:12.0pt'><br>On Aug 16, 2014, at 15:23, "Andre" <<a href="mailto:andretodd@verizon.net" target="_blank">andretodd@verizon.net</a>> wrote:<o:p></o:p></p></div><blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Going with the idea it’s IO related I did a db_cache status. I just disabled limit in my code by removing it but limit should be HASH not a database however I stills how a database.</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Below is the results of the db cache status. I’m not real sure what its saying. I should have all registration and limit disabled and should not be using any sql database from my understanding.</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Anyone know what the results mean? I’ve only tested 2 calls on this status.</span><o:p></o:p></p></div></div></blockquote><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>[The entire original message is not included.]<o:p></o:p></p></div></div><p class=MsoNormal><br>_________________________________________________________________________<br>Professional FreeSWITCH Consulting Services:<br><a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a><br><a href="http://www.freeswitchsolutions.com" target="_blank">http://www.freeswitchsolutions.com</a><br><br>Official FreeSWITCH Sites<br><a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br><a href="http://confluence.freeswitch.org" target="_blank">http://confluence.freeswitch.org</a><br><a href="http://www.cluecon.com" target="_blank">http://www.cluecon.com</a><br><br>FreeSWITCH-powered IP PBX: The CudaTel Communication Server<br><a href="http://www.cudatel.com" target="_blank">http://www.cudatel.com</a><br><br>FreeSWITCH-users mailing list<br><a href="mailto:FreeSWITCH-users@lists.freeswitch.org" target="_blank">FreeSWITCH-users@lists.freeswitch.org</a><br><a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><o:p></o:p></p></blockquote></div></div></blockquote></div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>[The entire original message is not included.]<o:p></o:p></p></div></div><p class=MsoNormal><br>_________________________________________________________________________<br>Professional FreeSWITCH Consulting Services:<br><a href="mailto:consulting@freeswitch.org">consulting@freeswitch.org</a><br><a href="http://www.freeswitchsolutions.com" target="_blank">http://www.freeswitchsolutions.com</a><br><br>Official FreeSWITCH Sites<br><a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br><a href="http://confluence.freeswitch.org" target="_blank">http://confluence.freeswitch.org</a><br><a href="http://www.cluecon.com" target="_blank">http://www.cluecon.com</a><br><br>FreeSWITCH-powered IP PBX: The CudaTel Communication Server<br><a href="http://www.cudatel.com" target="_blank">http://www.cudatel.com</a><br><br>FreeSWITCH-users mailing list<br><a href="mailto:FreeSWITCH-users@lists.freeswitch.org">FreeSWITCH-users@lists.freeswitch.org</a><br><a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br><a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><o:p></o:p></p></blockquote></div></div></body></html>