[Freeswitch-users] CPU Spikes every 20 minutes

Ricardo Martinez rmartinez at redvoiss.net
Mon Oct 22 22:52:28 MSD 2012 

This is the gdb from the proccess consuming more cpu :



(gdb) thread 7

[Switching to thread 7 (Thread 31781)]#0  0x0000003c03c0e047 in fsync ()
from /lib64/libpthread.so.0

(gdb) bt

#0  0x0000003c03c0e047 in fsync () from /lib64/libpthread.so.0

#1  0x00002aede19276de in unixSync (id=0x2aaaac726380, dataOnly=<value
optimized out>) at ./src/os_unix.c:1229

#2  0x00002aede1951178 in syncJournal (pPager=0x1a1c200) at
./src/pager.c:2229

#3  0x00002aede1953b65 in sqlite3pager_sync (pPager=0x1a1c200, zMaster=0x0,
nTrunc=0) at ./src/pager.c:3770

#4  0x00002aede193d07b in sqlite3BtreeSync (p=<value optimized out>,
zMaster=0x0) at ./src/btree.c:6520

#5  0x00002aede1931f15 in sqlite3VdbeHalt (p=0x2aaaad17ac70) at
./src/vdbeaux.c:1062

#6  0x00002aede1962613 in sqlite3VdbeExec (p=0x2aaaad17ac70) at
./src/vdbe.c:2360

#7  0x00002aede193036c in sqlite3Step (pStmt=0x2aaaad17ac70) at
./src/vdbeapi.c:236

#8  sqlite3_step (pStmt=0x2aaaad17ac70) at ./src/vdbeapi.c:289

#9  0x00002aede1938978 in sqlite3_exec (db=0x1a1f070, zSql=0x2aede199e717
"COMMIT", xCallback=0, pArg=0x0, pzErrMsg=0x4178bed0)

    at ./src/legacy.c:78

#10 0x00002aede18ae0c1 in switch_core_db_exec (db=0x1a1f070,
sql=0x2aede199e717 "COMMIT", callback=0, data=0x0, errmsg=0x4178bf38)

    at src/switch_core_db.c:93

#11 0x00002aede1897c18 in switch_cache_db_execute_sql_real (dbh=0x1a2dce0,
sql=0x2aede199e717 "COMMIT", err=0x0)

    at src/switch_core_sqldb.c:445

#12 0x00002aede18985c3 in switch_cache_db_persistant_execute_trans
(dbh=0x1a2dce0,

    sql=0x5f03bd0 "delete from calls where
(caller_uuid='e64319c5-7f91-47a1-9561-42da291e499a' or
callee_uuid='e64319c5-7f91-47a1-9561-42da291e499a') and hostname='
transcoder.red.net';\ndelete from calls where (call"..., retries=1)

    at src/switch_core_sqldb.c:787

#13 0x00002aede189bf1c in switch_core_sql_thread (thread=<value optimized
out>, obj=<value optimized out>)

    at src/switch_core_sqldb.c:1026

#14 0x0000003c03c0673d in start_thread () from /lib64/libpthread.so.0

#15 0x0000003c030d3f6d in clone () from /lib64/libc.so.6



*De:* Ricardo Martinez [mailto:rmartinez at redvoiss.net]
*Enviado el:* lunes, 22 de octubre de 2012 15:50
*Para:* 'FreeSWITCH Users Help'
*Asunto:* RE: [Freeswitch-users] CPU Spikes every 20 minutes



AS far as i can see there is no java running.

This is the top at the moment of the spike :



top - 15:48:29 up 17 days, 14:55,  3 users,  load average: 14.07, 8.87, 6.48

Tasks: 158 total,   1 running, 157 sleeping,   0 stopped,   0 zombie

Cpu(s): 12.6%us,  9.0%sy,  0.0%ni, 66.1%id,  2.7%wa,  0.3%hi,  9.2%si,
0.0%st

Mem:   8171500k total,  8041556k used,   129944k free,   177688k buffers

Swap: 10223608k total,        0k used, 10223608k free,  6951988k cached



  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+
COMMAND

31772 root      15   0  870m 481m 6880 S 212.4  6.0   9523:51
freeswitch

 4210 root      19   0 36344 2940 1444 S  1.0  0.0  56:21.08
sngtc_server

29586 root      15   0 12740 1132  808 R  0.3  0.0   0:00.02
top

    1 root      15   0 10348  704  588 S  0.0  0.0   0:01.13
init

    2 root      RT  -5     0    0    0 S  0.0  0.0   0:00.41
migration/0

    3 root      34  19     0    0    0 S  0.0  0.0   0:00.00
ksoftirqd/0

    4 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00
watchdog/0

    5 root      RT  -5     0    0    0 S  0.0  0.0   0:00.83
migration/1

    6 root      34  19     0    0    0 S  0.0  0.0   0:00.00
ksoftirqd/1

    7 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 watchdog/1


    8 root      RT  -5     0    0    0 S  0.0  0.0   0:00.14
migration/2

    9 root      34  19     0    0    0 S  0.0  0.0   0:00.00
ksoftirqd/2

   10 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00
watchdog/2

   11 root      RT  -5     0    0    0 S  0.0  0.0   0:00.07
migration/3

   12 root      34  19     0    0    0 S  0.0  0.0   0:00.00
ksoftirqd/3

   13 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00
watchdog/3

   14 root      RT  -5     0    0    0 S  0.0  0.0   0:00.06
migration/4

   15 root      34  19     0    0    0 S  0.0  0.0   0:00.00
ksoftirqd/4

   16 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00
watchdog/4

   17 root      RT  -5     0    0    0 S  0.0  0.0   0:00.06
migration/5

   18 root      34  19     0    0    0 S  0.0  0.0   0:00.00
ksoftirqd/5

   19 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00
watchdog/5

   20 root      RT  -5     0    0    0 S  0.0  0.0   0:00.07
migration/6





Regards,

Ricardo.-



*De:* freeswitch-users-bounces at lists.freeswitch.org [
mailto:freeswitch-users-bounces at lists.freeswitch.org<freeswitch-users-bounces at lists.freeswitch.org>]
*En nombre de *Guillermo Ruiz Camauer
*Enviado el:* lunes, 22 de octubre de 2012 15:38
*Para:* FreeSWITCH Users Help
*Asunto:* Re: [Freeswitch-users] CPU Spikes every 20 minutes



Do you have anything like Java running that could be doing garbage
collection?



What other processes are running on the machine?  Does TOP show which
process is causing the spike?



Guillermo

On Mon, Oct 22, 2012 at 3:29 PM, Ricardo Martinez <rmartinez at redvoiss.net>
wrote:

Hi Gabe.
I'm mean between the spikes there is nothing different as regarding to the
calls... the cps maintains the same.. there are no scripts running or
crontabs running, the simultaneous calls are the same also.  So, for some
reason the "load average" start to grow (every 20 or 25 minutes) without
any clear cause.
I have debug with gdb the moment the server is with 12.5 of "load average"
here : http://pastebin.freeswitch.org/20112

Can we detect a problem from here?

Regards,
Ricardo.



-----Mensaje original-----
De: freeswitch-users-bounces at lists.freeswitch.org
[mailto:freeswitch-users-bounces at lists.freeswitch.org] En nombre de
Gabriel Gunderson
Enviado el: lunes, 22 de octubre de 2012 13:31
Para: FreeSWITCH Users Help
Asunto: Re: [Freeswitch-users] CPU Spikes every 20 minutes


On Mon, Oct 22, 2012 at 8:37 AM, Ricardo Martinez <rmartinez at redvoiss.net>
wrote:
> The problem is that I'm having weird spikes of  "load average" cpu
> every 20 minutes (aprox), without any reason, reaching near of 12.5 of
> load average for this machine.

What have you done to verify that it happens 'without any reason'?


Gabe

_________________________________________________________________________
Professional FreeSWITCH Consulting Services:
consulting at freeswitch.org
http://www.freeswitchsolutions.com




Official FreeSWITCH Sites
http://www.freeswitch.org
http://wiki.freeswitch.org
http://www.cluecon.com

FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org

_________________________________________________________________________
Professional FreeSWITCH Consulting Services:
consulting at freeswitch.org
http://www.freeswitchsolutions.com




Official FreeSWITCH Sites
http://www.freeswitch.org
http://wiki.freeswitch.org
http://www.cluecon.com

FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org





-- 
Guillermo Ruiz Camauer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20121022/e5de8b41/attachment-0001.html

Join us at ClueCon 2011 Aug 9-11, 2011
More information about the FreeSWITCH-users mailing list