<div dir="ltr">Its a large number of handles to have open all in sofia if you are not doing anything<div><br></div><div>db_cache status</div><div><br></div><div>from cli can show where those handles come from and what db they are using.</div>
<div>Clearly the box is being overloaded past its limits.</div><div><br></div><div>This is why we don't endorse virtual servers.</div><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Jul 22, 2013 at 2:49 PM, Victor Chukalovskiy <span dir="ltr"><<a href="mailto:victor.chukalovskiy@gmail.com" target="_blank">victor.chukalovskiy@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div>Disabled in all profiles:<br>
<param name="manage-presence" value="false"/><div><div class="h5"><br>
<br>
On 13-07-22 03:40 PM, Anthony Minessale wrote:<br>
</div></div></div><div><div class="h5">
<blockquote type="cite">
<div dir="ltr">Did you turn of the presence in the sofia profile,
that is enabled by default and very costly. comment out
manage-presence</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Mon, Jul 22, 2013 at 2:32 PM, Victor
Chukalovskiy <span dir="ltr"><<a href="mailto:victor.chukalovskiy@gmail.com" target="_blank">victor.chukalovskiy@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div>Michael, Ken, <br>
<br>
Thank you for the response. Firstly, let me clarify
setup for each of DBs in use:<br>
<br>
<b>core_db:</b> sqlite, /dev/shm/core.db (RAM) - using
default 50 max handles<br>
<b>mod_sofia db</b><b>:</b> sqlite, /opt/fs/db (HDD) <br>
<b>mod_lcr db:</b> Postgres<br>
<b>mod_limit:</b> hash backend<br>
<br>
Note: I believe we rarely use mod_sofia DB's...this is
because we don't use any registrations, acl only. When I
list sofia per-profile db files on production box, most
of them has not been edited for a couple days. Are my
conclusions correct here, or do I miss something? Maybe
they are accessed a lot but only for reading?<br>
<br>
I understand recommendations from Ken and from Michaels
are along the lines of increasing core db and sofia db
throughput. However here is what puzzles me:<br>
-the db used most extensively is core_db. It' already in
the ram.<br>
-we don't have much load on sofia dbs (or am I mistaken
in my logic?)<br>
-error log started with:
<div><br>
<br>
[CRIT] switch_time.c:967 Virtual Migration Detected!
Syncing Clock<br>
</div>
<div> [WARNING] switch_scheduler.c:115 Task
was executed late by 19 seconds 1 heartbeat (core)<br>
<br>
</div>
Does it mean that there were abnormal events prior
hitting 50 max handles limit?<br>
<br>
Thank you,<br>
-Victor
<div>
<div><br>
<br>
On 13-07-22 02:52 PM, Anthony Minessale wrote:<br>
</div>
</div>
</div>
<div>
<div>
<blockquote type="cite">
<div dir="ltr">Check the odbcinst.ini settings from
the wiki or use native pg mode.
<div>increase the max number of pg connections on
your server and turn up FS to match
in max-db-handles in switch.conf.xml</div>
<div><br>
</div>
<div>You are already beating it up pretty good esp
for a fake server though. 2500 is a lot.</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Mon, Jul 22, 2013 at
1:44 PM, Ken Rice <span dir="ltr"><<a href="mailto:krice@freeswitch.org" target="_blank">krice@freeswitch.org</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Mount the entire
freeswitch/db dir as tmpfs... most of these
errors are for<br>
the sofia DB getting backed up...<br>
<br>
The coredb is a completely different sqlite db
from the sofia db<br>
<br>
On 7/22/13 12:54 PM, "Victor Chukalovskiy"
<<a href="mailto:victor.chukalovskiy@gmail.com" target="_blank">victor.chukalovskiy@gmail.com</a>><br>
wrote:<br>
<div>
<div><br>
> Hello,<br>
><br>
> Need some help finding the bottleneck
here. Running 1.2.stable on a<br>
> decent multi-core vmware platform. No
RTP, SIP signaling only. Around<br>
> 2500 sessions and 30 average CPS
load. Essentially, it's a multi-profile<br>
> .xml config that uses mod_lcr
extensively (postgres DB). Yesterday it<br>
> stopped processing calls with the
following log output:<br>
><br>
> 2013-07-21 06:00:37.642678 [CRIT]
switch_time.c:967 Virtual Migration<br>
> Detected! Syncing Clock<br>
> 2013-07-21 06:01:07.141734 [WARNING]
switch_scheduler.c:115 Task was<br>
> executed late by 19 seconds 1
heartbeat (core)<br>
> 2013-07-21 06:02:32.402668 [CRIT]
switch_time.c:967 Virtual Migration<br>
> Detected! Syncing Clock<br>
> 2013-07-21 06:04:42.499744 [WARNING]
switch_scheduler.c:115 Task was<br>
> executed late by 115 seconds 1
heartbeat (core)<br>
> 2013-07-21 06:04:42.499744 [WARNING]
switch_scheduler.c:115 Task was<br>
> executed late by 74 seconds 2
check_ip (core)<br>
> 2013-07-21 06:05:15.102683 [CRIT]
switch_time.c:967 Virtual Migration<br>
> Detected! Syncing Clock<br>
> 2013-07-21 06:05:44.961004 [WARNING]
switch_scheduler.c:115 Task was<br>
> executed late by 22 seconds 1
heartbeat (core)<br>
> 2013-07-21 06:05:44.961004 [WARNING]
switch_scheduler.c:115 Task was<br>
> executed late by 2 seconds 2 check_ip
(core)<br>
> 2013-07-21 06:05:49.322672 [WARNING]
mod_lcr.c:448 Max handles 50<br>
> exceeded, blocking....<br>
> 2013-07-21 06:05:50.962987 [WARNING]
sofia_glue.c:6605 Max handles 50<br>
> exceeded, blocking....<br>
> 2013-07-21 06:05:50.962987 [WARNING]
sofia_glue.c:6605 Max handles 50<br>
> exceeded, blocking....<br>
> 2013-07-21 06:05:50.962987 [WARNING]
sofia_glue.c:6605 Max handles 50<br>
> exceeded, blocking....<br>
> ................................<br>
> 2013-07-21 06:05:53.962672 [WARNING]
sofia_glue.c:6605 Max handles 50<br>
> exceeded, blocking....<br>
> 2013-07-21 06:05:54.422671 [ERR]
mod_lcr.c:448 Error connecting<br>
> 2013-07-21 06:05:54.422671 [WARNING]
mod_lcr.c:1779 LCR lookup failed<br>
> for 14389955082<br>
> 2013-07-21 06:05:54.962671 [WARNING]
sofia_glue.c:6605 Max handles 50<br>
> exceeded, blocking....<br>
> 2013-07-21 06:05:54.962671 [WARNING]
sofia_glue.c:6605 Max handles 50<br>
> exceeded, blocking....<br>
> ................................<br>
> 2013-07-21 06:05:55.962758 [WARNING]
sofia_glue.c:6605 Max handles 50<br>
> exceeded, blocking....<br>
> 2013-07-21 06:05:56.062671 [ERR]
sofia_glue.c:6605 Error connecting<br>
> 2013-07-21 06:05:56.062671 [ERR]
sofia_glue.c:6606 Error Opening DB<br>
> 2013-07-21 06:05:56.062671 [WARNING]
sofia_glue.c:6605 Max handles 50<br>
> exceeded, blocking....<br>
> 2013-07-21 06:05:56.062671 [ERR]
sofia_glue.c:6605 Error connecting<br>
> 2013-07-21 06:05:56.062671 [ERR]
sofia_glue.c:6606 Error Opening DB<br>
><br>
> FreeSWITCH stayed unusable since then
until it was restarted.<br>
><br>
> FYI core DB is already in the RAM:<br>
> <param name="core-db-name"
value="/dev/shm/core.db" /><br>
><br>
><br>
> Thank you,<br>
> Victor<br>
><br>
>
_________________________________________________________________________<br>
> Professional FreeSWITCH Consulting
Services:<br>
> <a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a><br>
> <a href="http://www.freeswitchsolutions.com" target="_blank">http://www.freeswitchsolutions.com</a><br>
><br>
> FreeSWITCH-powered IP PBX: The
CudaTel Communication Server<br>
> <a href="http://www.cudatel.com" target="_blank">http://www.cudatel.com</a><br>
><br>
> Official FreeSWITCH Sites<br>
> <a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
> <a href="http://wiki.freeswitch.org" target="_blank">http://wiki.freeswitch.org</a><br>
> <a href="http://www.cluecon.com" target="_blank">http://www.cluecon.com</a><br>
><br>
> FreeSWITCH-users mailing list<br>
> <a href="mailto:FreeSWITCH-users@lists.freeswitch.org" target="_blank">FreeSWITCH-users@lists.freeswitch.org</a><br>
> <a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
> UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br>
> <a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
<br>
</div>
</div>
<span><font color="#888888">--<br>
Ken<br>
<a href="http://www.FreeSWITCH.org" target="_blank">http://www.FreeSWITCH.org</a><br>
<a href="http://www.ClueCon.com" target="_blank">http://www.ClueCon.com</a><br>
<a href="http://www.OSTAG.org" target="_blank">http://www.OSTAG.org</a><br>
<a href="http://irc.freenode.net" target="_blank">irc.freenode.net</a>
#freeswitch<br>
</font></span>
<div>
<div><br>
<br>
<br>
_________________________________________________________________________<br>
Professional FreeSWITCH Consulting
Services:<br>
<a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a><br>
<a href="http://www.freeswitchsolutions.com" target="_blank">http://www.freeswitchsolutions.com</a><br>
<br>
FreeSWITCH-powered IP PBX: The CudaTel
Communication Server<br>
<a href="http://www.cudatel.com" target="_blank">http://www.cudatel.com</a><br>
<br>
Official FreeSWITCH Sites<br>
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
<a href="http://wiki.freeswitch.org" target="_blank">http://wiki.freeswitch.org</a><br>
<a href="http://www.cluecon.com" target="_blank">http://www.cluecon.com</a><br>
<br>
FreeSWITCH-users mailing list<br>
<a href="mailto:FreeSWITCH-users@lists.freeswitch.org" target="_blank">FreeSWITCH-users@lists.freeswitch.org</a><br>
<a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br>
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
Anthony Minessale II<br>
<br>
FreeSWITCH <a href="http://www.freeswitch.org/" target="_blank">http://www.freeswitch.org/</a><br>
ClueCon <a href="http://www.cluecon.com/" target="_blank">http://www.cluecon.com/</a><br>
Twitter: <a href="http://twitter.com/FreeSWITCH_wire" target="_blank">http://twitter.com/FreeSWITCH_wire</a><br>
<br>
AIM: anthm<br>
<a href="mailto:MSN%3Aanthony_minessale@hotmail.com" target="_blank">MSN:anthony_minessale@hotmail.com</a><br>
GTALK/JABBER/<a href="mailto:PAYPAL%3Aanthony.minessale@gmail.com" target="_blank">PAYPAL:anthony.minessale@gmail.com</a><br>
IRC: <a href="http://irc.freenode.net" target="_blank">irc.freenode.net</a>
#freeswitch<br>
<br>
FreeSWITCH Developer Conference<br>
<a href="mailto:sip%3A888@conference.freeswitch.org" target="_blank">sip:888@conference.freeswitch.org</a><br>
<a href="mailto:googletalk%3Aconf%2B888@conference.freeswitch.org" target="_blank">googletalk:conf+888@conference.freeswitch.org</a><br>
pstn:+19193869900 </div>
<br>
<fieldset></fieldset>
<br>
<pre>_________________________________________________________________________
Professional FreeSWITCH Consulting Services:
<a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a>
<a href="http://www.freeswitchsolutions.com" target="_blank">http://www.freeswitchsolutions.com</a>
FreeSWITCH-powered IP PBX: The CudaTel Communication Server
<a href="http://www.cudatel.com" target="_blank">http://www.cudatel.com</a>
Official FreeSWITCH Sites
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a>
<a href="http://wiki.freeswitch.org" target="_blank">http://wiki.freeswitch.org</a>
<a href="http://www.cluecon.com" target="_blank">http://www.cluecon.com</a>
FreeSWITCH-users mailing list
<a href="mailto:FreeSWITCH-users@lists.freeswitch.org" target="_blank">FreeSWITCH-users@lists.freeswitch.org</a>
<a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a>
UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a>
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a>
</pre>
</blockquote>
<br>
</div>
</div>
</div>
<br>
_________________________________________________________________________<br>
Professional FreeSWITCH Consulting Services:<br>
<a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a><br>
<a href="http://www.freeswitchsolutions.com" target="_blank">http://www.freeswitchsolutions.com</a><br>
<br>
FreeSWITCH-powered IP PBX: The CudaTel Communication Server<br>
<a href="http://www.cudatel.com" target="_blank">http://www.cudatel.com</a><br>
<br>
Official FreeSWITCH Sites<br>
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
<a href="http://wiki.freeswitch.org" target="_blank">http://wiki.freeswitch.org</a><br>
<a href="http://www.cluecon.com" target="_blank">http://www.cluecon.com</a><br>
<br>
FreeSWITCH-users mailing list<br>
<a href="mailto:FreeSWITCH-users@lists.freeswitch.org" target="_blank">FreeSWITCH-users@lists.freeswitch.org</a><br>
<a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br>
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
<br>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
-- <br>
Anthony Minessale II<br>
<br>
FreeSWITCH <a href="http://www.freeswitch.org/" target="_blank">http://www.freeswitch.org/</a><br>
ClueCon <a href="http://www.cluecon.com/" target="_blank">http://www.cluecon.com/</a><br>
Twitter: <a href="http://twitter.com/FreeSWITCH_wire" target="_blank">http://twitter.com/FreeSWITCH_wire</a><br>
<br>
AIM: anthm<br>
<a href="mailto:MSN%3Aanthony_minessale@hotmail.com" target="_blank">MSN:anthony_minessale@hotmail.com</a><br>
GTALK/JABBER/<a href="mailto:PAYPAL%3Aanthony.minessale@gmail.com" target="_blank">PAYPAL:anthony.minessale@gmail.com</a><br>
IRC: <a href="http://irc.freenode.net" target="_blank">irc.freenode.net</a>
#freeswitch<br>
<br>
FreeSWITCH Developer Conference<br>
<a href="mailto:sip%3A888@conference.freeswitch.org" target="_blank">sip:888@conference.freeswitch.org</a><br>
<a href="mailto:googletalk%3Aconf%2B888@conference.freeswitch.org" target="_blank">googletalk:conf+888@conference.freeswitch.org</a><br>
pstn:+19193869900
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_________________________________________________________________________
Professional FreeSWITCH Consulting Services:
<a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a>
<a href="http://www.freeswitchsolutions.com" target="_blank">http://www.freeswitchsolutions.com</a>
FreeSWITCH-powered IP PBX: The CudaTel Communication Server
<a href="http://www.cudatel.com" target="_blank">http://www.cudatel.com</a>
Official FreeSWITCH Sites
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a>
<a href="http://wiki.freeswitch.org" target="_blank">http://wiki.freeswitch.org</a>
<a href="http://www.cluecon.com" target="_blank">http://www.cluecon.com</a>
FreeSWITCH-users mailing list
<a href="mailto:FreeSWITCH-users@lists.freeswitch.org" target="_blank">FreeSWITCH-users@lists.freeswitch.org</a>
<a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a>
UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a>
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a>
</pre>
</blockquote>
<br>
</div></div></div>
<br>_________________________________________________________________________<br>
Professional FreeSWITCH Consulting Services:<br>
<a href="mailto:consulting@freeswitch.org">consulting@freeswitch.org</a><br>
<a href="http://www.freeswitchsolutions.com" target="_blank">http://www.freeswitchsolutions.com</a><br>
<br>
FreeSWITCH-powered IP PBX: The CudaTel Communication Server<br>
<a href="http://www.cudatel.com" target="_blank">http://www.cudatel.com</a><br>
<br>
Official FreeSWITCH Sites<br>
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
<a href="http://wiki.freeswitch.org" target="_blank">http://wiki.freeswitch.org</a><br>
<a href="http://www.cluecon.com" target="_blank">http://www.cluecon.com</a><br>
<br>
FreeSWITCH-users mailing list<br>
<a href="mailto:FreeSWITCH-users@lists.freeswitch.org">FreeSWITCH-users@lists.freeswitch.org</a><br>
<a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br>
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br>Anthony Minessale II<br><br>FreeSWITCH <a href="http://www.freeswitch.org/">http://www.freeswitch.org/</a><br>ClueCon <a href="http://www.cluecon.com/">http://www.cluecon.com/</a><br>
Twitter: <a href="http://twitter.com/FreeSWITCH_wire">http://twitter.com/FreeSWITCH_wire</a><br><br>AIM: anthm<br><a href="mailto:MSN%3Aanthony_minessale@hotmail.com">MSN:anthony_minessale@hotmail.com</a><br>GTALK/JABBER/<a href="mailto:PAYPAL%3Aanthony.minessale@gmail.com">PAYPAL:anthony.minessale@gmail.com</a><br>
IRC: <a href="http://irc.freenode.net">irc.freenode.net</a> #freeswitch<br><br>FreeSWITCH Developer Conference<br><a href="mailto:sip%3A888@conference.freeswitch.org">sip:888@conference.freeswitch.org</a><br><a href="mailto:googletalk%3Aconf%2B888@conference.freeswitch.org">googletalk:conf+888@conference.freeswitch.org</a><br>
pstn:+19193869900
</div>