FreeSWITCH changing max_sessions automatically
Just have noticed something new today. Testing a local instance on CentOS 6.0 with FreeSWITCH Version 1.2.23~64bit. FreeSWITCH seems to be changing the max_sessions param automatically. This happens when typically I bridge around 1000 sessions and whatever value I set to sps and max_sessions, it comes down. UP 0 years, 0 days, 0 hours, 37 minutes, 54 seconds, 362 milliseconds, 294 microseconds FreeSWITCH (Version 1.2.23 64bit) is ready 74716 session(s) since startup 976 session(s) - peak 1022, last 5min 985 112 session(s) per Sec out of max 400, peak 9053, last 5min 317 <- BTW This peak was never tried, not sure why it says 9053 5000 session(s) max min idle cpu 0.00/55.00 Current Stack Size/Max 240K/8192K ======= followed by ======= switch_core_session.c:2288 Over Session Limit! 1013 mod_sofia.c:4884 Error Creating Session ======= And then ======= freeswitch@internal> status UP 0 years, 0 days, 0 hours, 38 minutes, 35 seconds, 372 milliseconds, 831 microseconds FreeSWITCH (Version 1.2.23 64bit) is ready 76231 session(s) since startup 560 session(s) - peak 1023, last 5min 1023 0 session(s) per Sec out of max 400, peak 9053, last 5min 317 1013 session(s) max min idle cpu 0.00/99.00 Current Stack Size/Max 240K/8192K Is this something changed recently? Or some sort of server meltdown protection mechanism being forked :) -- Thanks, Muhammad Naseer Bhatti
You¹re hitting a melt down protection... Once it reaches a limit and cant allocate the additional resources for creating anew session it will back down the max sessions limit to help save itself... The PEAK SPS there was a value seen at some point the SPS buffer... Depending on you beating the machine to death it is possible that number got skewed due to a timer miss On 3/27/14 10:57 AM, "Muhammad Bhatti" <nbhatti@gmail.com> wrote:
Just have noticed something new today. Testing a local instance on CentOS 6.0 with FreeSWITCH Version 1.2.23~64bit. FreeSWITCH seems to be changing the max_sessions param automatically. This happens when typically I bridge around 1000 sessions and whatever value I set to sps and max_sessions, it comes down.
UP 0 years, 0 days, 0 hours, 37 minutes, 54 seconds, 362 milliseconds, 294 microseconds FreeSWITCH (Version 1.2.23 64bit) is ready 74716 session(s) since startup 976 session(s) - peak 1022, last 5min 985 112 session(s) per Sec out of max 400, peak 9053, last 5min 317 <- BTW This peak was never tried, not sure why it says 9053 5000 session(s) max min idle cpu 0.00/55.00 Current Stack Size/Max 240K/8192K
======= followed by =======
switch_core_session.c:2288 Over Session Limit! 1013 mod_sofia.c:4884 Error Creating Session
======= And then =======
freeswitch@internal> status UP 0 years, 0 days, 0 hours, 38 minutes, 35 seconds, 372 milliseconds, 831 microseconds FreeSWITCH (Version 1.2.23 64bit) is ready 76231 session(s) since startup 560 session(s) - peak 1023, last 5min 1023 0 session(s) per Sec out of max 400, peak 9053, last 5min 317 1013 session(s) max min idle cpu 0.00/99.00 Current Stack Size/Max 240K/8192K
Is this something changed recently? Or some sort of server meltdown protection mechanism being forked :)
-- Ken http://www.FreeSWITCH.org http://www.ClueCon.com http://www.OSTAG.org irc.freenode.net #freeswitch Twitter: @FreeSWITCH
No i am definitely not going to that SPS limit. The peak was set to 400 so maybe this is just a skewed value. 1000 session does not look to be high load. The call is bridged and set to play silence and disconnects at random time under 1 minute. The resources on the server also clearly seem to be available. I am not gearing this towards a how do i .... Performance kind of a thread but could there be something unusual here? Sent from my iPad
On Mar 27, 2014, at 9:07 PM, Ken Rice <krice@freeswitch.org> wrote:
You’re hitting a melt down protection... Once it reaches a limit and cant allocate the additional resources for creating anew session it will back down the max sessions limit to help save itself...
The PEAK SPS there was a value seen at some point the SPS buffer... Depending on you beating the machine to death it is possible that number got skewed due to a timer miss
On 3/27/14 10:57 AM, "Muhammad Bhatti" <nbhatti@gmail.com> wrote:
Just have noticed something new today. Testing a local instance on CentOS 6.0 with FreeSWITCH Version 1.2.23~64bit. FreeSWITCH seems to be changing the max_sessions param automatically. This happens when typically I bridge around 1000 sessions and whatever value I set to sps and max_sessions, it comes down.
UP 0 years, 0 days, 0 hours, 37 minutes, 54 seconds, 362 milliseconds, 294 microseconds FreeSWITCH (Version 1.2.23 64bit) is ready 74716 session(s) since startup 976 session(s) - peak 1022, last 5min 985 112 session(s) per Sec out of max 400, peak 9053, last 5min 317 <- BTW This peak was never tried, not sure why it says 9053 5000 session(s) max min idle cpu 0.00/55.00 Current Stack Size/Max 240K/8192K
======= followed by =======
switch_core_session.c:2288 Over Session Limit! 1013 mod_sofia.c:4884 Error Creating Session
======= And then =======
freeswitch@internal> status UP 0 years, 0 days, 0 hours, 38 minutes, 35 seconds, 372 milliseconds, 831 microseconds FreeSWITCH (Version 1.2.23 64bit) is ready 76231 session(s) since startup 560 session(s) - peak 1023, last 5min 1023 0 session(s) per Sec out of max 400, peak 9053, last 5min 317 1013 session(s) max min idle cpu 0.00/99.00 Current Stack Size/Max 240K/8192K
Is this something changed recently? Or some sort of server meltdown protection mechanism being forked :)
-- Ken http://www.FreeSWITCH.org http://www.ClueCon.com http://www.OSTAG.org irc.freenode.net #freeswitch Twitter: @FreeSWITCH
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Is your FS instance running in a virtual environment? William King Senior Engineer Quentus Technologies, INC 1037 NE 65th St Suite 273 Seattle, WA 98115 Main: (877) 211-9337 Office: (206) 388-4772 Cell: (253) 686-5518 william.king@quentustech.com On 03/27/2014 11:07 AM, Muhammad Naseer Bhatti wrote:
No i am definitely not going to that SPS limit. The peak was set to 400 so maybe this is just a skewed value. 1000 session does not look to be high load. The call is bridged and set to play silence and disconnects at random time under 1 minute. The resources on the server also clearly seem to be available. I am not gearing this towards a how do i .... Performance kind of a thread but could there be something unusual here?
Sent from my iPad
On Mar 27, 2014, at 9:07 PM, Ken Rice <krice@freeswitch.org <mailto:krice@freeswitch.org>> wrote:
Re: [Freeswitch-users] FreeSWITCH changing max_sessions automatically You’re hitting a melt down protection... Once it reaches a limit and cant allocate the additional resources for creating anew session it will back down the max sessions limit to help save itself...
The PEAK SPS there was a value seen at some point the SPS buffer... Depending on you beating the machine to death it is possible that number got skewed due to a timer miss
On 3/27/14 10:57 AM, "Muhammad Bhatti" <nbhatti@gmail.com> wrote:
Just have noticed something new today. Testing a local instance on CentOS 6.0 with FreeSWITCH Version 1.2.23~64bit. FreeSWITCH seems to be changing the max_sessions param automatically. This happens when typically I bridge around 1000 sessions and whatever value I set to sps and max_sessions, it comes down.
UP 0 years, 0 days, 0 hours, 37 minutes, 54 seconds, 362 milliseconds, 294 microseconds FreeSWITCH (Version 1.2.23 64bit) is ready 74716 session(s) since startup *976 session(s) - peak 1022, last 5min 985 112 session(s) per Sec out of max 400, peak 9053, last 5min 317 <- BTW This peak was never tried, not sure why it says 9053 5000 session(s) max min idle cpu 0.00/55.00 Current Stack Size/Max 240K/8192K
======= followed by =======
switch_core_session.c:2288 Over Session Limit! 1013 mod_sofia.c:4884 Error Creating Session
======= And then =======
freeswitch@internal> status UP 0 years, 0 days, 0 hours, 38 minutes, 35 seconds, 372 milliseconds, 831 microseconds FreeSWITCH (Version 1.2.23 64bit) is ready 76231 session(s) since startup 560 session(s) - peak 1023, last 5min 1023 0 session(s) per Sec out of max 400, peak 9053, last 5min 317 1013 session(s) max min idle cpu 0.00/99.00 Current Stack Size/Max 240K/8192K
Is this something changed recently? Or some sort of server meltdown protection mechanism being forked :)
*
* *-- Ken _http://www.FreeSWITCH.org http://www.ClueCon.com http://www.OSTAG.org _irc.freenode.net <http://irc.freenode.net> #freeswitch Twitter: @FreeSWITCH
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org <mailto:consulting@freeswitch.org> http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org <mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users <http://lists.freeswitch.org/mailman/options/freeswitch-users> http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Yes, i was thinking on same lines. Its running over OpenVZ kernel. I am now going to try replicate this in a physical env. Sent from my iPad
On Mar 27, 2014, at 9:11 PM, William King <william.king@quentustech.com> wrote:
Is your FS instance running in a virtual environment?
William King Senior Engineer Quentus Technologies, INC 1037 NE 65th St Suite 273 Seattle, WA 98115 Main: (877) 211-9337 Office: (206) 388-4772 Cell: (253) 686-5518 william.king@quentustech.com
On 03/27/2014 11:07 AM, Muhammad Naseer Bhatti wrote: No i am definitely not going to that SPS limit. The peak was set to 400 so maybe this is just a skewed value. 1000 session does not look to be high load. The call is bridged and set to play silence and disconnects at random time under 1 minute. The resources on the server also clearly seem to be available. I am not gearing this towards a how do i .... Performance kind of a thread but could there be something unusual here?
Sent from my iPad
On Mar 27, 2014, at 9:07 PM, Ken Rice <krice@freeswitch.org <mailto:krice@freeswitch.org>> wrote:
Re: [Freeswitch-users] FreeSWITCH changing max_sessions automatically You’re hitting a melt down protection... Once it reaches a limit and cant allocate the additional resources for creating anew session it will back down the max sessions limit to help save itself...
The PEAK SPS there was a value seen at some point the SPS buffer... Depending on you beating the machine to death it is possible that number got skewed due to a timer miss
On 3/27/14 10:57 AM, "Muhammad Bhatti" <nbhatti@gmail.com> wrote:
Just have noticed something new today. Testing a local instance on CentOS 6.0 with FreeSWITCH Version 1.2.23~64bit. FreeSWITCH seems to be changing the max_sessions param automatically. This happens when typically I bridge around 1000 sessions and whatever value I set to sps and max_sessions, it comes down.
UP 0 years, 0 days, 0 hours, 37 minutes, 54 seconds, 362 milliseconds, 294 microseconds FreeSWITCH (Version 1.2.23 64bit) is ready 74716 session(s) since startup *976 session(s) - peak 1022, last 5min 985 112 session(s) per Sec out of max 400, peak 9053, last 5min 317 <- BTW This peak was never tried, not sure why it says 9053 5000 session(s) max min idle cpu 0.00/55.00 Current Stack Size/Max 240K/8192K
======= followed by =======
switch_core_session.c:2288 Over Session Limit! 1013 mod_sofia.c:4884 Error Creating Session
======= And then =======
freeswitch@internal> status UP 0 years, 0 days, 0 hours, 38 minutes, 35 seconds, 372 milliseconds, 831 microseconds FreeSWITCH (Version 1.2.23 64bit) is ready 76231 session(s) since startup 560 session(s) - peak 1023, last 5min 1023 0 session(s) per Sec out of max 400, peak 9053, last 5min 317 1013 session(s) max min idle cpu 0.00/99.00 Current Stack Size/Max 240K/8192K
Is this something changed recently? Or some sort of server meltdown protection mechanism being forked :)
*
* *-- Ken _http://www.FreeSWITCH.org http://www.ClueCon.com http://www.OSTAG.org _irc.freenode.net <http://irc.freenode.net> #freeswitch Twitter: @FreeSWITCH
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org <mailto:consulting@freeswitch.org> http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org <mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users <http://lists.freeswitch.org/mailman/options/freeswitch-users> http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Take a look in your FS logs.. Im pretty sure you will see the whimsical & Awesome log message : *LUKE: I'm hit, but not bad.LUKE'S VOICE: Artoo, see what you can do with it. Hang on back there....* this is in thread_launch_failure in switch_core_session.c ( around line 1714 ) and around this code you can easily see the following sess_count = switch_core_session_count(); if (sess_count > 110) { switch_core_session_limit(sess_count - 10); as you can see, it bumps 10 off your SPS, every time thread_launch_failure is called. thread_launch_failure is called whenever switch_thread_create fails to create a thread. which basically means apr_thread_create I have seen this myself with MUCH lower SPS on openVZ machines, but since your seeing it also im now hoping we can nut this out :) im going to dig a little more and see what I can find. what openVZ kernel are you using out of interest ? ive seen this on a fairly old, but stable 2.6.32-19-pve let me know what you find ! On 28 March 2014 04:30, Muhammad Naseer Bhatti <nbhatti@gmail.com> wrote:
Yes, i was thinking on same lines. Its running over OpenVZ kernel. I am now going to try replicate this in a physical env.
Sent from my iPad
On Mar 27, 2014, at 9:11 PM, William King <william.king@quentustech.com> wrote:
Is your FS instance running in a virtual environment?
William King Senior Engineer Quentus Technologies, INC 1037 NE 65th St Suite 273 Seattle, WA 98115 Main: (877) 211-9337 Office: (206) 388-4772 Cell: (253) 686-5518 william.king@quentustech.com
On 03/27/2014 11:07 AM, Muhammad Naseer Bhatti wrote: No i am definitely not going to that SPS limit. The peak was set to 400 so maybe this is just a skewed value. 1000 session does not look to be high load. The call is bridged and set to play silence and disconnects at random time under 1 minute. The resources on the server also clearly seem to be available. I am not gearing this towards a how do i .... Performance kind of a thread but could there be something unusual here?
Sent from my iPad
On Mar 27, 2014, at 9:07 PM, Ken Rice <krice@freeswitch.org <mailto:krice@freeswitch.org>> wrote:
Re: [Freeswitch-users] FreeSWITCH changing max_sessions automatically You're hitting a melt down protection... Once it reaches a limit and cant allocate the additional resources for creating anew session it will back down the max sessions limit to help save itself...
The PEAK SPS there was a value seen at some point the SPS buffer... Depending on you beating the machine to death it is possible that number got skewed due to a timer miss
On 3/27/14 10:57 AM, "Muhammad Bhatti" <nbhatti@gmail.com> wrote:
Just have noticed something new today. Testing a local instance on CentOS 6.0 with FreeSWITCH Version 1.2.23~64bit. FreeSWITCH seems to be changing the max_sessions param automatically. This happens when typically I bridge around 1000 sessions and whatever value I set to sps and max_sessions, it comes down.
UP 0 years, 0 days, 0 hours, 37 minutes, 54 seconds, 362 milliseconds, 294 microseconds FreeSWITCH (Version 1.2.23 64bit) is ready 74716 session(s) since startup *976 session(s) - peak 1022, last 5min 985 112 session(s) per Sec out of max 400, peak 9053, last 5min 317 <- BTW This peak was never tried, not sure why it says 9053 5000 session(s) max min idle cpu 0.00/55.00 Current Stack Size/Max 240K/8192K
======= followed by =======
switch_core_session.c:2288 Over Session Limit! 1013 mod_sofia.c:4884 Error Creating Session
======= And then =======
freeswitch@internal> status UP 0 years, 0 days, 0 hours, 38 minutes, 35 seconds, 372 milliseconds, 831 microseconds FreeSWITCH (Version 1.2.23 64bit) is ready 76231 session(s) since startup 560 session(s) - peak 1023, last 5min 1023 0 session(s) per Sec out of max 400, peak 9053, last 5min 317 1013 session(s) max min idle cpu 0.00/99.00 Current Stack Size/Max 240K/8192K
Is this something changed recently? Or some sort of server meltdown protection mechanism being forked :)
*
* *-- Ken _http://www.FreeSWITCH.org http://www.ClueCon.com http://www.OSTAG.org _irc.freenode.net <http://irc.freenode.net> #freeswitch Twitter: @FreeSWITCH
Professional FreeSWITCH Consulting Services: consulting@freeswitch.org <mailto:consulting@freeswitch.org> http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org <mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users <http://lists.freeswitch.org/mailman/options/freeswitch-users> http://www.freeswitch.org
Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Sincerely Jay
Check this out that I just found ! http://forum.openvz.org/index.php?t=msg&th=607& so my beancounters are not showing any failcnt for privvmpages, but I do have failcnt on numproc which is interesting. - numproc - The maximal number of processes and threads the VE may create. increase this limit and see if you can make it happen again. vzctl set <VZUID> --numproc=6000 --save On 29 March 2014 19:09, jay binks <jaybinks@gmail.com> wrote:
Take a look in your FS logs.. Im pretty sure you will see the whimsical & Awesome log message :
*LUKE: I'm hit, but not bad.LUKE'S VOICE: Artoo, see what you can do with it. Hang on back there....* this is in thread_launch_failure in switch_core_session.c ( around line 1714 ) and around this code you can easily see the following
sess_count = switch_core_session_count(); if (sess_count > 110) { switch_core_session_limit(sess_count - 10); as you can see, it bumps 10 off your SPS, every time thread_launch_failure is called.
thread_launch_failure is called whenever switch_thread_create fails to create a thread. which basically means apr_thread_create
I have seen this myself with MUCH lower SPS on openVZ machines, but since your seeing it also im now hoping we can nut this out :) im going to dig a little more and see what I can find.
what openVZ kernel are you using out of interest ? ive seen this on a fairly old, but stable 2.6.32-19-pve
let me know what you find !
On 28 March 2014 04:30, Muhammad Naseer Bhatti <nbhatti@gmail.com> wrote:
Yes, i was thinking on same lines. Its running over OpenVZ kernel. I am now going to try replicate this in a physical env.
Sent from my iPad
On Mar 27, 2014, at 9:11 PM, William King <william.king@quentustech.com> wrote:
Is your FS instance running in a virtual environment?
William King Senior Engineer Quentus Technologies, INC 1037 NE 65th St Suite 273 Seattle, WA 98115 Main: (877) 211-9337 Office: (206) 388-4772 Cell: (253) 686-5518 william.king@quentustech.com
On 03/27/2014 11:07 AM, Muhammad Naseer Bhatti wrote: No i am definitely not going to that SPS limit. The peak was set to 400 so maybe this is just a skewed value. 1000 session does not look to be high load. The call is bridged and set to play silence and disconnects at random time under 1 minute. The resources on the server also clearly seem to be available. I am not gearing this towards a how do i .... Performance kind of a thread but could there be something unusual here?
Sent from my iPad
On Mar 27, 2014, at 9:07 PM, Ken Rice <krice@freeswitch.org <mailto:krice@freeswitch.org>> wrote:
Re: [Freeswitch-users] FreeSWITCH changing max_sessions automatically You're hitting a melt down protection... Once it reaches a limit and cant allocate the additional resources for creating anew session it will back down the max sessions limit to help save itself...
The PEAK SPS there was a value seen at some point the SPS buffer... Depending on you beating the machine to death it is possible that number got skewed due to a timer miss
On 3/27/14 10:57 AM, "Muhammad Bhatti" <nbhatti@gmail.com> wrote:
Just have noticed something new today. Testing a local instance on CentOS 6.0 with FreeSWITCH Version 1.2.23~64bit. FreeSWITCH seems to be changing the max_sessions param automatically. This happens when typically I bridge around 1000 sessions and whatever value I set to sps and max_sessions, it comes down.
UP 0 years, 0 days, 0 hours, 37 minutes, 54 seconds, 362 milliseconds, 294 microseconds FreeSWITCH (Version 1.2.23 64bit) is ready 74716 session(s) since startup *976 session(s) - peak 1022, last 5min 985 112 session(s) per Sec out of max 400, peak 9053, last 5min 317 <- BTW This peak was never tried, not sure why it says 9053 5000 session(s) max min idle cpu 0.00/55.00 Current Stack Size/Max 240K/8192K
======= followed by =======
switch_core_session.c:2288 Over Session Limit! 1013 mod_sofia.c:4884 Error Creating Session
======= And then =======
freeswitch@internal> status UP 0 years, 0 days, 0 hours, 38 minutes, 35 seconds, 372 milliseconds, 831 microseconds FreeSWITCH (Version 1.2.23 64bit) is ready 76231 session(s) since startup 560 session(s) - peak 1023, last 5min 1023 0 session(s) per Sec out of max 400, peak 9053, last 5min 317 1013 session(s) max min idle cpu 0.00/99.00 Current Stack Size/Max 240K/8192K
Is this something changed recently? Or some sort of server meltdown protection mechanism being forked :)
*
* *-- Ken _http://www.FreeSWITCH.org http://www.ClueCon.com http://www.OSTAG.org _irc.freenode.net <http://irc.freenode.net> #freeswitch Twitter: @FreeSWITCH
Professional FreeSWITCH Consulting Services: consulting@freeswitch.org <mailto:consulting@freeswitch.org> http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org <mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users <http://lists.freeswitch.org/mailman/options/freeswitch-users> http://www.freeswitch.org
Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________
Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Sincerely
Jay
-- Sincerely Jay
Well look at you Nancy Drew, You solved the case. ;) -- Brian West brian@freeswitch.org FreeSWITCH Solutions, LLC PO BOX 2531 Brookfield, WI 53008-2531 Twitter: @FreeSWITCH , @briankwest http://www.freeswitchbook.com http://www.freeswitchcookbook.com T: +1.918.420.9001 | F: +1.918.420.9002 | M: +1.918.424.WEST iNUM: +883 5100 1420 9001 ISN: 410*543 Skype:briankwest PGP Key: http://www.bkw.org/key.txt (AB93356707C76CED) On Mar 29, 2014, at 4:17 AM, jay binks <jaybinks@gmail.com> wrote:
Check this out that I just found ! http://forum.openvz.org/index.php?t=msg&th=607&
so my beancounters are not showing any failcnt for privvmpages, but I do have failcnt on numproc which is interesting. • numproc - The maximal number of processes and threads the VE may create.
increase this limit and see if you can make it happen again. vzctl set <VZUID> --numproc=6000 --save
hopefully.... would be great if Muhammad Naseer Bhatti can confirm for me :) I moved some load around to avoid it :( ( cant easily put it back ) but its great to know whats going on, coz I didnt REALLY think I was hitting the edges of my servers :P Jay On 31 March 2014 22:48, Brian West <brian@freeswitch.org> wrote:
Well look at you Nancy Drew, You solved the case. ;) -- Brian West brian@freeswitch.org FreeSWITCH Solutions, LLC PO BOX 2531 Brookfield, WI 53008-2531 Twitter: @FreeSWITCH , @briankwest http://www.freeswitchbook.com http://www.freeswitchcookbook.com
T: +1.918.420.9001 | F: +1.918.420.9002 | M: +1.918.424.WEST iNUM: +883 5100 1420 9001 ISN: 410*543 Skype:briankwest PGP Key: http://www.bkw.org/key.txt (AB93356707C76CED)
On Mar 29, 2014, at 4:17 AM, jay binks <jaybinks@gmail.com> wrote:
Check this out that I just found ! http://forum.openvz.org/index.php?t=msg&th=607&
so my beancounters are not showing any failcnt for privvmpages, but I do have failcnt on numproc which is interesting. * numproc - The maximal number of processes and threads the VE may create.
increase this limit and see if you can make it happen again. vzctl set <VZUID> --numproc=6000 --save
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Sincerely Jay
participants (5)
-
Brian West
-
jay binks
-
Ken Rice
-
Muhammad Naseer Bhatti
-
William King