[Freeswitch-users] CPU Spikes every 20 minutes

Guillermo Ruiz Camauer grcamauer at gmail.com
Tue Oct 23 00:25:53 MSD 2012


I am on CentOS 6.3.

in /etc/fstab I have:
       tmpfs                    /dev/shm             tmpfs         defaults
      0  0
 in freeswitch/conf/autoload_configs/switch.conf.xml I have: <param
name="core-db-name" value="/dev/shm/core.db" />

Guillermo

On Mon, Oct 22, 2012 at 4:40 PM, Ricardo Martinez <rmartinez at redvoiss.net>wrote:

> HI Guillero,
>
> I will try that and let you know the results.
>
> One final question, the link you gave me says :
>
>
>
> 1) In /etc/fstab, add a mountpoint and save.
>
> tmpfs    /usr/local/freeswitch/ramdb     tmpfs   defaults        0 0
>
> 2) Make sure you also create the directory.
>
> mkdir /usr/local/freeswitch/ramdisk
>
> mount -a
>
> 3) Then, in freeswitch/conf/autoload_configs/switch.conf.xml uncomment
> core-db-name and set it to the path you created in fstab.
>
> <param name="core-db-name" value="/usr/local/freeswitch/ramdisk/core.db" />
>
> 4) Restart FreeSWITCH
>
>
>
> The directory for the fstab file is :
>
> /usr/local/freeswitch/ramdb
>
>
>
> The directory to mount is :
>
> /usr/local/freeswitch/ramdisk
>
>
>
> Is this a typo error ? or it is ok?
>
>
>
> Thanks
>
> Ricardo.-
>
>
>
>
>
> *De:* freeswitch-users-bounces at lists.freeswitch.org [mailto:
> freeswitch-users-bounces at lists.freeswitch.org] *En nombre de *Guillermo
> Ruiz Camauer
> *Enviado el:* lunes, 22 de octubre de 2012 16:35
>
> *Para:* FreeSWITCH Users Help
> *Asunto:* Re: [Freeswitch-users] CPU Spikes every 20 minutes
>
>
>
> Try this out if is not already configured this way:
> http://wiki.freeswitch.org/index.php?title=FreeSWITCH_DB_In_RAMdrive&redirect=no
>
>
>
> Guillermo
>
> On Mon, Oct 22, 2012 at 4:30 PM, Ricardo Martinez <rmartinez at redvoiss.net>
> wrote:
>
> Hi Ken.
>
> Maybe that is the problem.  I was sure that the “load average” can’t go
> more than #cores in your system.  In my case I have 8 cores, so a “load
> average” of 12 is high from my point of view.  Where did you get the number
> 100 for this parameter?
>
> Anyway, what is running a sqlite in freeswitch.  Does this have to do with
> the core.db file? Or with the cdr_sqlite module?.
>
>
>
> Thanks
>
> Ricardo.-
>
>
>
> *De:* freeswitch-users-bounces at lists.freeswitch.org [mailto:
> freeswitch-users-bounces at lists.freeswitch.org] *En nombre de *Ken Rice
> *Enviado el:* lunes, 22 de octubre de 2012 16:18
>
>
> *Para:* FreeSWITCH Users Help
> *Asunto:* Re: [Freeswitch-users] CPU Spikes every 20 minutes
>
>
>
> Could it be just a normal sqlite running a vacuum?
>
> Sqlite does this every so often or if its calling a sync....
>
> Also freeswitch with a load avg of 12 for a short period of time isnt
> anything to really worry about. Due to the number of threads that’s
> possible with FS (ie: 1 per call leg) its not unusual to see a box humming
> along and the load avg well above 100 in my experience... This is due to
> the number of threads waiting in the run queue when it was sampled... Now,
> that’s not saying that this is not specifically indicative or a problem but
> something else to think about
>
> On 10/22/12 1:52 PM, "Ricardo Martinez" <rmartinez at redvoiss.net> wrote:
>
> This is the gdb from the proccess consuming more cpu :
>
> (gdb) thread 7
> [Switching to thread 7 (Thread 31781)]#0  0x0000003c03c0e047 in fsync ()
> from /lib64/libpthread.so.0
> (gdb) bt
> #0  0x0000003c03c0e047 in fsync () from /lib64/libpthread.so.0
> #1  0x00002aede19276de in unixSync (id=0x2aaaac726380, dataOnly=<value
> optimized out>) at ./src/os_unix.c:1229
> #2  0x00002aede1951178 in syncJournal (pPager=0x1a1c200) at
> ./src/pager.c:2229
> #3  0x00002aede1953b65 in sqlite3pager_sync (pPager=0x1a1c200,
> zMaster=0x0, nTrunc=0) at ./src/pager.c:3770
> #4  0x00002aede193d07b in sqlite3BtreeSync (p=<value optimized out>,
> zMaster=0x0) at ./src/btree.c:6520
> #5  0x00002aede1931f15 in sqlite3VdbeHalt (p=0x2aaaad17ac70) at
> ./src/vdbeaux.c:1062
> #6  0x00002aede1962613 in sqlite3VdbeExec (p=0x2aaaad17ac70) at
> ./src/vdbe.c:2360
> #7  0x00002aede193036c in sqlite3Step (pStmt=0x2aaaad17ac70) at
> ./src/vdbeapi.c:236
> #8  sqlite3_step (pStmt=0x2aaaad17ac70) at ./src/vdbeapi.c:289
> #9  0x00002aede1938978 in sqlite3_exec (db=0x1a1f070, zSql=0x2aede199e717
> "COMMIT", xCallback=0, pArg=0x0, pzErrMsg=0x4178bed0)
>     at ./src/legacy.c:78
> #10 0x00002aede18ae0c1 in switch_core_db_exec (db=0x1a1f070,
> sql=0x2aede199e717 "COMMIT", callback=0, data=0x0, errmsg=0x4178bf38)
>     at src/switch_core_db.c:93
> #11 0x00002aede1897c18 in switch_cache_db_execute_sql_real (dbh=0x1a2dce0,
> sql=0x2aede199e717 "COMMIT", err=0x0)
>     at src/switch_core_sqldb.c:445
> #12 0x00002aede18985c3 in switch_cache_db_persistant_execute_trans
> (dbh=0x1a2dce0,
>     sql=0x5f03bd0 "delete from calls where
> (caller_uuid='e64319c5-7f91-47a1-9561-42da291e499a' or
> callee_uuid='e64319c5-7f91-47a1-9561-42da291e499a') and hostname='
> transcoder.red.net <http://transcoder.red.net> ';\ndelete from calls
> where (call"..., retries=1)
>     at src/switch_core_sqldb.c:787
> #13 0x00002aede189bf1c in switch_core_sql_thread (thread=<value optimized
> out>, obj=<value optimized out>)
>     at src/switch_core_sqldb.c:1026
> #14 0x0000003c03c0673d in start_thread () from /lib64/libpthread.so.0
> #15 0x0000003c030d3f6d in clone () from /lib64/libc.so.6
>
>
> *De:* Ricardo Martinez [mailto:rmartinez at redvoiss.net<rmartinez at redvoiss.net>]
>
> *Enviado el:* lunes, 22 de octubre de 2012 15:50
> *Para:* 'FreeSWITCH Users Help'
> *Asunto:* RE: [Freeswitch-users] CPU Spikes every 20 minutes
>
> AS far as i can see there is no java running.
> This is the top at the moment of the spike :
>
> top - 15:48:29 up 17 days, 14:55,  3 users,  load average: 14.07, 8.87,
> 6.48
> Tasks: 158 total,   1 running, 157 sleeping,   0 stopped,   0 zombie
> Cpu(s): 12.6%us,  9.0%sy,  0.0%ni, 66.1%id,  2.7%wa,  0.3%hi,  9.2%si,
> 0.0%st
> Mem:   8171500k total,  8041556k used,   129944k free,   177688k buffers
> Swap: 10223608k total,        0k used, 10223608k free,  6951988k cached
>
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+
> COMMAND
> 31772 root      15   0  870m 481m 6880 S 212.4  6.0   9523:51
> freeswitch
>  4210 root      19   0 36344 2940 1444 S  1.0  0.0  56:21.08
> sngtc_server
> 29586 root      15   0 12740 1132  808 R  0.3  0.0   0:00.02
> top
>     1 root      15   0 10348  704  588 S  0.0  0.0   0:01.13
> init
>     2 root      RT  -5     0    0    0 S  0.0  0.0   0:00.41
> migration/0
>     3 root      34  19     0    0    0 S  0.0  0.0   0:00.00
> ksoftirqd/0
>     4 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00
> watchdog/0
>     5 root      RT  -5     0    0    0 S  0.0  0.0   0:00.83
> migration/1
>     6 root      34  19     0    0    0 S  0.0  0.0   0:00.00
> ksoftirqd/1
>     7 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 watchdog/1
>
>     8 root      RT  -5     0    0    0 S  0.0  0.0   0:00.14
> migration/2
>     9 root      34  19     0    0    0 S  0.0  0.0   0:00.00
> ksoftirqd/2
>    10 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00
> watchdog/2
>    11 root      RT  -5     0    0    0 S  0.0  0.0   0:00.07
> migration/3
>    12 root      34  19     0    0    0 S  0.0  0.0   0:00.00
> ksoftirqd/3
>    13 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00
> watchdog/3
>    14 root      RT  -5     0    0    0 S  0.0  0.0   0:00.06
> migration/4
>    15 root      34  19     0    0    0 S  0.0  0.0   0:00.00
> ksoftirqd/4
>    16 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00
> watchdog/4
>    17 root      RT  -5     0    0    0 S  0.0  0.0   0:00.06
> migration/5
>    18 root      34  19     0    0    0 S  0.0  0.0   0:00.00
> ksoftirqd/5
>    19 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00
> watchdog/5
>    20 root      RT  -5     0    0    0 S  0.0  0.0   0:00.07
> migration/6
>
>
> Regards,
> Ricardo.-
>
> *De:* freeswitch-users-bounces at lists.freeswitch.org [
> mailto:freeswitch-users-bounces at lists.freeswitch.org<freeswitch-users-bounces at lists.freeswitch.org>]
> *En nombre de *Guillermo Ruiz Camauer
> *Enviado el:* lunes, 22 de octubre de 2012 15:38
> *Para:* FreeSWITCH Users Help
> *Asunto:* Re: [Freeswitch-users] CPU Spikes every 20 minutes
>
> Do you have anything like Java running that could be doing garbage
> collection?
>
>
>
> What other processes are running on the machine?  Does TOP show which
> process is causing the spike?
>
>
>
> Guillermo
>
> On Mon, Oct 22, 2012 at 3:29 PM, Ricardo Martinez <rmartinez at redvoiss.net>
> wrote:
> Hi Gabe.
> I'm mean between the spikes there is nothing different as regarding to the
> calls... the cps maintains the same.. there are no scripts running or
> crontabs running, the simultaneous calls are the same also.  So, for some
> reason the "load average" start to grow (every 20 or 25 minutes) without
> any clear cause.
> I have debug with gdb the moment the server is with 12.5 of "load average"
> here : http://pastebin.freeswitch.org/20112
>
> Can we detect a problem from here?
>
> Regards,
> Ricardo.
>
>
>
> -----Mensaje original-----
> De: freeswitch-users-bounces at lists.freeswitch.org
> [mailto:freeswitch-users-bounces at lists.freeswitch.org<freeswitch-users-bounces at lists.freeswitch.org>]
> En nombre de
> Gabriel Gunderson
> Enviado el: lunes, 22 de octubre de 2012 13:31
> Para: FreeSWITCH Users Help
> Asunto: Re: [Freeswitch-users] CPU Spikes every 20 minutes
>
>
> On Mon, Oct 22, 2012 at 8:37 AM, Ricardo Martinez <rmartinez at redvoiss.net>
> wrote:
> > The problem is that I'm having weird spikes of  "load average" cpu
> > every 20 minutes (aprox), without any reason, reaching near of 12.5 of
> > load average for this machine.
>
> What have you done to verify that it happens 'without any reason'?
>
>
> Gabe
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> 
> 
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> 
> 
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>
>
>
>
> --
> Ken
> *http://www.FreeSWITCH.org
> http://www.ClueCon.com
> http://www.OSTAG.org
> *irc.freenode.net #freeswitch
>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> 
> 
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>
>
>
>
> --
> Guillermo Ruiz Camauer
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> 
> 
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>


-- 
Guillermo Ruiz Camauer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20121022/187237f7/attachment-0001.html 


Join us at ClueCon 2011 Aug 9-11, 2011
More information about the FreeSWITCH-users mailing list