I have been using Freeswitch (git updated regularly) for about 9 months on a very low load ubuntu server. I have noticed that the after a period of time (sometimes only a few days) I am unable to make calls from attached SIP phones. Logging into the server I find freeswitch is taking upto 70% of the memory (measured by top), and I can hear the disk being thrashed (swapping, I assume). Restarting Freeswitch fixes all the problems & it then uses only 5% of them memory. How can I collect more debuging information to help fix the problem?
Look at the *.db files. Are any of them getting really large? -MC On Sun, Oct 24, 2010 at 4:27 PM, Babstar <babstar99@gmail.com> wrote:
I have been using Freeswitch (git updated regularly) for about 9 months on a very low load ubuntu server. I have noticed that the after a period of time (sometimes only a few days) I am unable to make calls from attached SIP phones. Logging into the server I find freeswitch is taking upto 70% of the memory (measured by top), and I can hear the disk being thrashed (swapping, I assume). Restarting Freeswitch fixes all the problems & it then uses only 5% of them memory.
How can I collect more debuging information to help fix the problem?
On Tue, Oct 26, 2010 at 6:23 AM, Michael Collins <msc@freeswitch.org> wrote:
Look at the *.db files. Are any of them getting really large? -MC
Michael, just had the latest failure, nothing unusually large with the *.db files: 16K /usr/local/freeswitch/db/call_limit.db 148K /usr/local/freeswitch/db/core.db 8.0K /usr/local/freeswitch/db/fifo.db 76K /usr/local/freeswitch/db/sofia_reg_external.db 504K /usr/local/freeswitch/db/sofia_reg_internal.db 8.0K /usr/local/freeswitch/db/sofia_reg_internal.db-journal 76K /usr/local/freeswitch/db/sofia_reg_internal-ipv6.db 16K /usr/local/freeswitch/db/voicemail_default.db
When I restarted the daemon, the usage over a few minutes jump up to 24%, only a reboot could fix it. -- Babstar
Is it consistently driving the CPU usage up after a restart? You might want to do top -H to see if you can narrow down the exact process that is sucking up all the resources. Also, make sure that you are not suffering a SIP attack. You may want to consider setting up fail2ban: http://wiki.freeswitch.org/wiki/Fail2ban Make sure you update to the latest as soon as possible as Tony has fixed some recent bugs. -MC On Wed, Oct 27, 2010 at 10:04 PM, Babstar <babstar99@gmail.com> wrote:
On Tue, Oct 26, 2010 at 6:23 AM, Michael Collins <msc@freeswitch.org>wrote:
Look at the *.db files. Are any of them getting really large? -MC
Michael, just had the latest failure, nothing unusually large with the *.db files: 16K /usr/local/freeswitch/db/call_limit.db 148K /usr/local/freeswitch/db/core.db 8.0K /usr/local/freeswitch/db/fifo.db 76K /usr/local/freeswitch/db/sofia_reg_external.db 504K /usr/local/freeswitch/db/sofia_reg_internal.db 8.0K /usr/local/freeswitch/db/sofia_reg_internal.db-journal 76K /usr/local/freeswitch/db/sofia_reg_internal-ipv6.db 16K /usr/local/freeswitch/db/voicemail_default.db
When I restarted the daemon, the usage over a few minutes jump up to 24%, only a reboot could fix it.
-- Babstar
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
AFAIK the original poster wrote about memory usage, but not CPU. I have exactly the same problem - after few days of run FS RSS grows to more than 1GB, after FS restart RSS is about 50M. Version git-fe638ad 2010-10-13. Boxes running older FS version git-828960a 2010-09-25 have RSS in 60-100M range for weeks. On Thursday 28 October 2010, Michael Collins wrote:
Is it consistently driving the CPU usage up after a restart? You might want to do top -H to see if you can narrow down the exact process that is sucking up all the resources. Also, make sure that you are not suffering a SIP attack. You may want to consider setting up fail2ban: http://wiki.freeswitch.org/wiki/Fail2ban
Make sure you update to the latest as soon as possible as Tony has fixed some recent bugs.
-MC
On Wed, Oct 27, 2010 at 10:04 PM, Babstar <babstar99@gmail.com> wrote:
On Tue, Oct 26, 2010 at 6:23 AM, Michael Collins <msc@freeswitch.org>wrote:
Look at the *.db files. Are any of them getting really large? -MC
Michael,
just had the latest failure, nothing unusually large with the *.db files: 16K /usr/local/freeswitch/db/call_limit.db 148K /usr/local/freeswitch/db/core.db 8.0K /usr/local/freeswitch/db/fifo.db 76K /usr/local/freeswitch/db/sofia_reg_external.db 504K /usr/local/freeswitch/db/sofia_reg_internal.db 8.0K /usr/local/freeswitch/db/sofia_reg_internal.db-journal 76K /usr/local/freeswitch/db/sofia_reg_internal-ipv6.db 16K /usr/local/freeswitch/db/voicemail_default.db
When I restarted the daemon, the usage over a few minutes jump up to 24%, only a reboot could fix it.
-- Babstar
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
install valgrind valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes .libs/lt-freeswitch -vg make several minutes of typical calls shutdown FS cleanly and send me vg.log Also you could consider posting a jira in the future.... On Thu, Oct 28, 2010 at 12:26 PM, Sergey Okhapkin <sos@sokhapkin.dyndns.org> wrote:
AFAIK the original poster wrote about memory usage, but not CPU. I have exactly the same problem - after few days of run FS RSS grows to more than 1GB, after FS restart RSS is about 50M. Version git-fe638ad 2010-10-13. Boxes running older FS version git-828960a 2010-09-25 have RSS in 60-100M range for weeks.
On Thursday 28 October 2010, Michael Collins wrote:
Is it consistently driving the CPU usage up after a restart? You might want to do top -H to see if you can narrow down the exact process that is sucking up all the resources. Also, make sure that you are not suffering a SIP attack. You may want to consider setting up fail2ban: http://wiki.freeswitch.org/wiki/Fail2ban
Make sure you update to the latest as soon as possible as Tony has fixed some recent bugs.
-MC
On Wed, Oct 27, 2010 at 10:04 PM, Babstar <babstar99@gmail.com> wrote:
On Tue, Oct 26, 2010 at 6:23 AM, Michael Collins <msc@freeswitch.org>wrote:
Look at the *.db files. Are any of them getting really large? -MC
Michael,
just had the latest failure, nothing unusually large with the *.db files: 16K /usr/local/freeswitch/db/call_limit.db 148K /usr/local/freeswitch/db/core.db 8.0K /usr/local/freeswitch/db/fifo.db 76K /usr/local/freeswitch/db/sofia_reg_external.db 504K /usr/local/freeswitch/db/sofia_reg_internal.db 8.0K /usr/local/freeswitch/db/sofia_reg_internal.db-journal 76K /usr/local/freeswitch/db/sofia_reg_internal-ipv6.db 16K /usr/local/freeswitch/db/voicemail_default.db
When I restarted the daemon, the usage over a few minutes jump up to 24%, only a reboot could fix it.
-- Babstar
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900
oh, and update to current first git pull make update-clean make modwipe make install On Thu, Oct 28, 2010 at 3:38 PM, Anthony Minessale <anthony.minessale@gmail.com> wrote:
install valgrind
valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes .libs/lt-freeswitch -vg
make several minutes of typical calls shutdown FS cleanly and send me vg.log
Also you could consider posting a jira in the future....
On Thu, Oct 28, 2010 at 12:26 PM, Sergey Okhapkin <sos@sokhapkin.dyndns.org> wrote:
AFAIK the original poster wrote about memory usage, but not CPU. I have exactly the same problem - after few days of run FS RSS grows to more than 1GB, after FS restart RSS is about 50M. Version git-fe638ad 2010-10-13. Boxes running older FS version git-828960a 2010-09-25 have RSS in 60-100M range for weeks.
On Thursday 28 October 2010, Michael Collins wrote:
Is it consistently driving the CPU usage up after a restart? You might want to do top -H to see if you can narrow down the exact process that is sucking up all the resources. Also, make sure that you are not suffering a SIP attack. You may want to consider setting up fail2ban: http://wiki.freeswitch.org/wiki/Fail2ban
Make sure you update to the latest as soon as possible as Tony has fixed some recent bugs.
-MC
On Wed, Oct 27, 2010 at 10:04 PM, Babstar <babstar99@gmail.com> wrote:
On Tue, Oct 26, 2010 at 6:23 AM, Michael Collins <msc@freeswitch.org>wrote:
Look at the *.db files. Are any of them getting really large? -MC
Michael,
just had the latest failure, nothing unusually large with the *.db files: 16K /usr/local/freeswitch/db/call_limit.db 148K /usr/local/freeswitch/db/core.db 8.0K /usr/local/freeswitch/db/fifo.db 76K /usr/local/freeswitch/db/sofia_reg_external.db 504K /usr/local/freeswitch/db/sofia_reg_internal.db 8.0K /usr/local/freeswitch/db/sofia_reg_internal.db-journal 76K /usr/local/freeswitch/db/sofia_reg_internal-ipv6.db 16K /usr/local/freeswitch/db/voicemail_default.db
When I restarted the daemon, the usage over a few minutes jump up to 24%, only a reboot could fix it.
-- Babstar
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900
-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900
I can't run valgrind on the production server (you know why)... On Thursday 28 October 2010, Anthony Minessale wrote:
install valgrind
valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes .libs/lt-freeswitch -vg
make several minutes of typical calls shutdown FS cleanly and send me vg.log
Also you could consider posting a jira in the future....
On Thu, Oct 28, 2010 at 12:26 PM, Sergey Okhapkin
<sos@sokhapkin.dyndns.org> wrote:
AFAIK the original poster wrote about memory usage, but not CPU. I have exactly the same problem - after few days of run FS RSS grows to more than 1GB, after FS restart RSS is about 50M. Version git-fe638ad 2010-10-13. Boxes running older FS version git-828960a 2010-09-25 have RSS in 60-100M range for weeks.
On Thursday 28 October 2010, Michael Collins wrote:
Is it consistently driving the CPU usage up after a restart? You might want to do top -H to see if you can narrow down the exact process that is sucking up all the resources. Also, make sure that you are not suffering a SIP attack. You may want to consider setting up fail2ban: http://wiki.freeswitch.org/wiki/Fail2ban
Make sure you update to the latest as soon as possible as Tony has fixed some recent bugs.
-MC
On Wed, Oct 27, 2010 at 10:04 PM, Babstar <babstar99@gmail.com> wrote:
On Tue, Oct 26, 2010 at 6:23 AM, Michael Collins
<msc@freeswitch.org>wrote:
Look at the *.db files. Are any of them getting really large? -MC
Michael,
just had the latest failure, nothing unusually large with the *.db files: 16K /usr/local/freeswitch/db/call_limit.db 148K /usr/local/freeswitch/db/core.db 8.0K /usr/local/freeswitch/db/fifo.db 76K /usr/local/freeswitch/db/sofia_reg_external.db 504K /usr/local/freeswitch/db/sofia_reg_internal.db 8.0K /usr/local/freeswitch/db/sofia_reg_internal.db-journal 76K /usr/local/freeswitch/db/sofia_reg_internal-ipv6.db 16K /usr/local/freeswitch/db/voicemail_default.db
When I restarted the daemon, the usage over a few minutes jump up to 24%, only a reboot could fix it.
-- Babstar
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-use rs http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Many month ago we had similar discussion about memory leak in mod_nibblebill. Valgring didn't find the cause of the leak. On Thursday 28 October 2010, Sergey Okhapkin wrote:
I can't run valgrind on the production server (you know why)...
On Thursday 28 October 2010, Anthony Minessale wrote:
install valgrind
valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes .libs/lt-freeswitch -vg
make several minutes of typical calls shutdown FS cleanly and send me vg.log
Also you could consider posting a jira in the future....
On Thu, Oct 28, 2010 at 12:26 PM, Sergey Okhapkin
<sos@sokhapkin.dyndns.org> wrote:
AFAIK the original poster wrote about memory usage, but not CPU. I have exactly the same problem - after few days of run FS RSS grows to more than 1GB, after FS restart RSS is about 50M. Version git-fe638ad 2010-10-13. Boxes running older FS version git-828960a 2010-09-25 have RSS in 60-100M range for weeks.
On Thursday 28 October 2010, Michael Collins wrote:
Is it consistently driving the CPU usage up after a restart? You might want to do top -H to see if you can narrow down the exact process that is sucking up all the resources. Also, make sure that you are not suffering a SIP attack. You may want to consider setting up fail2ban: http://wiki.freeswitch.org/wiki/Fail2ban
Make sure you update to the latest as soon as possible as Tony has fixed some recent bugs.
-MC
On Wed, Oct 27, 2010 at 10:04 PM, Babstar <babstar99@gmail.com> wrote:
On Tue, Oct 26, 2010 at 6:23 AM, Michael Collins
<msc@freeswitch.org>wrote:
Look at the *.db files. Are any of them getting really large? -MC
Michael,
just had the latest failure, nothing unusually large with the *.db files: 16K /usr/local/freeswitch/db/call_limit.db 148K /usr/local/freeswitch/db/core.db 8.0K /usr/local/freeswitch/db/fifo.db 76K /usr/local/freeswitch/db/sofia_reg_external.db 504K /usr/local/freeswitch/db/sofia_reg_internal.db 8.0K /usr/local/freeswitch/db/sofia_reg_internal.db-journal 76K /usr/local/freeswitch/db/sofia_reg_internal-ipv6.db 16K /usr/local/freeswitch/db/voicemail_default.db
When I restarted the daemon, the usage over a few minutes jump up to 24%, only a reboot could fix it.
-- Babstar
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-u se rs http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-user s http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
participants (4)
-
Anthony Minessale
-
Babstar
-
Michael Collins
-
Sergey Okhapkin