last_used is seconds typically unattached handles unused for more than 30 sec are released. On Tue, Jul 23, 2013 at 3:23 PM, Victor Chukalovskiy < victor.chukalovskiy@gmail.com> wrote:
Yes, these are the only ones I see printed on our sandbox. Will try if I can get any other ones. I understand it's time consuming for you to respond, thanks again for your help.
Today it got stuck again on production server and I ran db_cache status:
db="sofia_reg_<profile_name>" Type: CORE_DB Last used: 150861 Flags: Locked, Attached(1) Creator: sofia_glue.c:6380 Last User: src/switch_core_sqldb.c:1816
..... <same as above for each profile, we got a few dozens> ..... db="/dev/shm/core.db" Type: CORE_DB Last used: 150862 Flags: Locked, Attached(1) Creator: src/switch_core_sqldb.c:1816 Last User:
In what units is "last used"?
VM runs on iSCSI SAN Storage.
On 13-07-23 03:01 PM, Anthony Minessale wrote:
Is that the only ones? how frequent are they? Those are only for expiration and run very infrequently. You may wish to study it longer because that should not be enough to slow everything down. Your server is not on a fake HDD too is it?
I generally do not help on load testing or virtual server setups because it can consume a lot of time.
On Tue, Jul 23, 2013 at 12:05 PM, Victor Chukalovskiy < victor.chukalovskiy@gmail.com> wrote:
Anthony, thanks a lot. Here is what I got:
[ERR] sofia_glue.c:6614 SQL: [select call_id,sip_user,sip_host,contact,status,rpid,expires,user_agent,server_user,server_host,profile_name,network_ip,0 from sip_registrations where expires > 0 and expires <= 1374597711] [ERR] sofia_glue.c:6614 SQL: [select call_id from sip_shared_appearance_dialogs where hostname='test' and profile_name='test' and expires <= 1374597711]
Is there a way to disable those two selects to save a few DB calls? Keep in mind that manage-shared-appearance and manage-presence are already off.
As for the performance tweaking, I'm going to: -move ALL sqlite DBs to RAM -bump max handles. Does it sound reasonable and safe to go from 50 to 100 handles?
@Cal, saw your msg, thanks. I was indeed looking at master while Anthony meant stable.
.Victor
On 13-07-22 04:42 PM, Anthony Minessale wrote:
do you have manage-shared-appearance off too cos that re-enables presence?
edit sofia_glue.c on line 6614 and add a print to see what the sql is that is being executed.
switch_log_printf(SWITCH_CHANNEL_LOG, SWITCH_LOG_ERROR, "SQL: [%s]\n", sql);
then make mod_sofia-install
and relaunch and try to catch a sample of what it's doing.
On Mon, Jul 22, 2013 at 3:27 PM, Victor Chukalovskiy < victor.chukalovskiy@gmail.com> wrote:
Anthony,
Thank you, this is a very useful command. I see a bunch of records like:
db="sofia_reg_<profile_name>" Type: CORE_DB Last used: 10 Flags: Unlocked, Detached(0) Creator: *sofia_glue.c:6605* Last User: *sofia_glue.c:6605*
Appears that these DBs are in use and user/creator is sofia_glue.c:6605. Same as in error log:
[WARNING] *sofia_glue.c:6605* Max handles 50
And this is not a cored.db... it's sitting on the disk drive still. Is there a way to setup sip_profiles not to create any sofia_reg_...DBs?
Re: vmware setup. It's good to know it's not optimal, however it gives rack space, administrator's time, and $ savings compared to dedicated servers.
Should it be Ok to stick with VM setup provided enough CPU and RAM is allocated to each FS guest? Are there reasons not to run FS on VM other than CPU and RAM allocation?
Thank you, Victor
Its a large number of handles to have open all in sofia if you are not doing anything
db_cache status
from cli can show where those handles come from and what db they are using. Clearly the box is being overloaded past its limits.
This is why we don't endorse virtual servers.
On Mon, Jul 22, 2013 at 2:49 PM, Victor Chukalovskiy < victor.chukalovskiy@gmail.com> wrote:
Disabled in all profiles: <param name="manage-presence" value="false"/>
On 13-07-22 03:40 PM, Anthony Minessale wrote:
Did you turn of the presence in the sofia profile, that is enabled by default and very costly. comment out manage-presence
On Mon, Jul 22, 2013 at 2:32 PM, Victor Chukalovskiy < victor.chukalovskiy@gmail.com> wrote:
Michael, Ken,
Thank you for the response. Firstly, let me clarify setup for each of DBs in use:
*core_db:* sqlite, /dev/shm/core.db (RAM) - using default 50 max handles *mod_sofia db**:* sqlite, /opt/fs/db (HDD) *mod_lcr db:* Postgres *mod_limit:* hash backend
Note: I believe we rarely use mod_sofia DB's...this is because we don't use any registrations, acl only. When I list sofia per-profile db files on production box, most of them has not been edited for a couple days. Are my conclusions correct here, or do I miss something? Maybe they are accessed a lot but only for reading?
I understand recommendations from Ken and from Michaels are along the lines of increasing core db and sofia db throughput. However here is what puzzles me: -the db used most extensively is core_db. It' already in the ram. -we don't have much load on sofia dbs (or am I mistaken in my logic?) -error log started with:
[CRIT] switch_time.c:967 Virtual Migration Detected! Syncing Clock [WARNING] switch_scheduler.c:115 Task was executed late by 19 seconds 1 heartbeat (core)
Does it mean that there were abnormal events prior hitting 50 max handles limit?
Thank you, -Victor
On 13-07-22 02:52 PM, Anthony Minessale wrote:
Check the odbcinst.ini settings from the wiki or use native pg mode. increase the max number of pg connections on your server and turn up FS to match in max-db-handles in switch.conf.xml
You are already beating it up pretty good esp for a fake server though. 2500 is a lot.
On Mon, Jul 22, 2013 at 1:44 PM, Ken Rice <krice@freeswitch.org>wrote:
Mount the entire freeswitch/db dir as tmpfs... most of these errors are for the sofia DB getting backed up...
The coredb is a completely different sqlite db from the sofia db
On 7/22/13 12:54 PM, "Victor Chukalovskiy" < victor.chukalovskiy@gmail.com> wrote:
> Hello, > > Need some help finding the bottleneck here. Running 1.2.stable on a > decent multi-core vmware platform. No RTP, SIP signaling only. Around > 2500 sessions and 30 average CPS load. Essentially, it's a multi-profile > .xml config that uses mod_lcr extensively (postgres DB). Yesterday it > stopped processing calls with the following log output: > > 2013-07-21 06:00:37.642678 [CRIT] switch_time.c:967 Virtual Migration > Detected! Syncing Clock > 2013-07-21 06:01:07.141734 [WARNING] switch_scheduler.c:115 Task was > executed late by 19 seconds 1 heartbeat (core) > 2013-07-21 06:02:32.402668 [CRIT] switch_time.c:967 Virtual Migration > Detected! Syncing Clock > 2013-07-21 06:04:42.499744 [WARNING] switch_scheduler.c:115 Task was > executed late by 115 seconds 1 heartbeat (core) > 2013-07-21 06:04:42.499744 [WARNING] switch_scheduler.c:115 Task was > executed late by 74 seconds 2 check_ip (core) > 2013-07-21 06:05:15.102683 [CRIT] switch_time.c:967 Virtual Migration > Detected! Syncing Clock > 2013-07-21 06:05:44.961004 [WARNING] switch_scheduler.c:115 Task was > executed late by 22 seconds 1 heartbeat (core) > 2013-07-21 06:05:44.961004 [WARNING] switch_scheduler.c:115 Task was > executed late by 2 seconds 2 check_ip (core) > 2013-07-21 06:05:49.322672 [WARNING] mod_lcr.c:448 Max handles 50 > exceeded, blocking.... > 2013-07-21 06:05:50.962987 [WARNING] sofia_glue.c:6605 Max handles 50 > exceeded, blocking.... > 2013-07-21 06:05:50.962987 [WARNING] sofia_glue.c:6605 Max handles 50 > exceeded, blocking.... > 2013-07-21 06:05:50.962987 [WARNING] sofia_glue.c:6605 Max handles 50 > exceeded, blocking.... > ................................ > 2013-07-21 06:05:53.962672 [WARNING] sofia_glue.c:6605 Max handles 50 > exceeded, blocking.... > 2013-07-21 06:05:54.422671 [ERR] mod_lcr.c:448 Error connecting > 2013-07-21 06:05:54.422671 [WARNING] mod_lcr.c:1779 LCR lookup failed > for 14389955082 > 2013-07-21 06:05:54.962671 [WARNING] sofia_glue.c:6605 Max handles 50 > exceeded, blocking.... > 2013-07-21 06:05:54.962671 [WARNING] sofia_glue.c:6605 Max handles 50 > exceeded, blocking.... > ................................ > 2013-07-21 06:05:55.962758 [WARNING] sofia_glue.c:6605 Max handles 50 > exceeded, blocking.... > 2013-07-21 06:05:56.062671 [ERR] sofia_glue.c:6605 Error connecting > 2013-07-21 06:05:56.062671 [ERR] sofia_glue.c:6606 Error Opening DB > 2013-07-21 06:05:56.062671 [WARNING] sofia_glue.c:6605 Max handles 50 > exceeded, blocking.... > 2013-07-21 06:05:56.062671 [ERR] sofia_glue.c:6605 Error connecting > 2013-07-21 06:05:56.062671 [ERR] sofia_glue.c:6606 Error Opening DB > > FreeSWITCH stayed unusable since then until it was restarted. > > FYI core DB is already in the RAM: > <param name="core-db-name" value="/dev/shm/core.db" /> > > > Thank you, > Victor > > _________________________________________________________________________ > Professional FreeSWITCH Consulting Services: > consulting@freeswitch.org > http://www.freeswitchsolutions.com > > FreeSWITCH-powered IP PBX: The CudaTel Communication Server > http://www.cudatel.com > > Official FreeSWITCH Sites > http://www.freeswitch.org > http://wiki.freeswitch.org > http://www.cluecon.com > > FreeSWITCH-users mailing list > FreeSWITCH-users@lists.freeswitch.org > http://lists.freeswitch.org/mailman/listinfo/freeswitch-users > UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users > http://www.freeswitch.org
-- Ken http://www.FreeSWITCH.org http://www.ClueCon.com http://www.OSTAG.org irc.freenode.net #freeswitch
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900
_________________________________________________________________________ Professional FreeSWITCH Consulting Services:consulting@freeswitch.orghttp://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Serverhttp://www.cudatel.com
Official FreeSWITCH Siteshttp://www.freeswitch.orghttp://wiki.freeswitch.orghttp://www.cluecon.com
FreeSWITCH-users mailing listFreeSWITCH-users@lists.freeswitch.orghttp://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-usershttp://www.frees...
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900
_________________________________________________________________________ Professional FreeSWITCH Consulting Services:consulting@freeswitch.orghttp://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Serverhttp://www.cudatel.com
Official FreeSWITCH Siteshttp://www.freeswitch.orghttp://wiki.freeswitch.orghttp://www.cluecon.com
FreeSWITCH-users mailing listFreeSWITCH-users@lists.freeswitch.orghttp://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-usershttp://www.frees...
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900
_________________________________________________________________________ Professional FreeSWITCH Consulting Services:consulting@freeswitch.orghttp://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Serverhttp://www.cudatel.com
Official FreeSWITCH Siteshttp://www.freeswitch.orghttp://wiki.freeswitch.orghttp://www.cluecon.com
FreeSWITCH-users mailing listFreeSWITCH-users@lists.freeswitch.orghttp://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-usershttp://www.frees...
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900
_________________________________________________________________________ Professional FreeSWITCH Consulting Services:consulting@freeswitch.orghttp://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Serverhttp://www.cudatel.com
Official FreeSWITCH Siteshttp://www.freeswitch.orghttp://wiki.freeswitch.orghttp://www.cluecon.com
FreeSWITCH-users mailing listFreeSWITCH-users@lists.freeswitch.orghttp://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-usershttp://www.frees...
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900
_________________________________________________________________________ Professional FreeSWITCH Consulting Services:consulting@freeswitch.orghttp://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Serverhttp://www.cudatel.com
Official FreeSWITCH Siteshttp://www.freeswitch.orghttp://wiki.freeswitch.orghttp://www.cluecon.com
FreeSWITCH-users mailing listFreeSWITCH-users@lists.freeswitch.orghttp://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-usershttp://www.frees...
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900