[Freeswitch-dev] Memory Leak? - FreeSWITCH Version 1.0.4pre5
I'm wondering about a possible memory leak in the latest versions. For each FS process, the resident size seems to be incrementing by 8k at a slow, but fairly steady rate. Sometimes there are a few quick "bursts" in the increase and sometimes it seems to take longer to see a "bump" in usage. There are no calls being processed. I just started FS and have been watching it. After upgrading from 1.0.1 to 1.0.4pre4, my FS process died after a few hours. I'm thinking this was due to a memory leak and the service running out of memory. If it makes a difference, I upgraded from 1.0.1 to 1.0.4pre4 (saw memory leak there too) and then on to 1.0.4pre5 without wiping the instadirectory and starting over. Please let me know what other information I can provide that will help find the reason for the ever increasing memory usage. Thanks, Charlie +OK log level [7] freeswitch@internal> version FreeSWITCH Version 1.0.4pre5 (hacked) <-- not sure what "hacked" is about. :) Linux conference1 2.6.18-128.1.6.el5 #1 SMP Wed Apr 1 09:19:18 EDT 2009 i686 i686 i386 GNU/Linux CentOS release 5.3 (Final) Output from htop: VIRT RES SHR S CPU% MEM% TIME+ Command 27104 12544 5104 S 0.0 1.2 0:01.00 /usr/local/freeswitch/bin/freeswitch -nc 27104 12544 5104 S 0.0 1.2 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 27104 12544 5104 S 0.0 1.2 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 27104 12544 5104 S 0.0 1.2 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 27104 12544 5104 S 0.0 1.2 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 27104 12544 5104 S 0.0 1.2 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 27104 12544 5104 S 0.0 1.2 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 27104 12544 5104 S 0.0 1.2 0:00.04 /usr/local/freeswitch/bin/freeswitch -nc 27104 12544 5104 S 0.0 1.2 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 27104 12544 5104 S 0.0 1.2 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 27104 12544 5104 S 0.0 1.2 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 27104 12544 5104 S 0.0 1.2 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 27104 12544 5104 S 0.0 1.2 0:00.01 /usr/local/freeswitch/bin/freeswitch -nc 27104 12544 5104 S 0.0 1.2 0:00.09 /usr/local/freeswitch/bin/freeswitch -nc 27104 12544 5104 S 0.0 1.2 0:00.08 /usr/local/freeswitch/bin/freeswitch -nc 27104 12544 5104 S 0.0 1.2 0:00.02 /usr/local/freeswitch/bin/freeswitch -nc 27104 12544 5104 S 0.0 1.2 0:00.22 /usr/local/freeswitch/bin/freeswitch -nc 27104 12544 5104 S 0.0 1.2 0:00.24 /usr/local/freeswitch/bin/freeswitch -nc 27104 12544 5104 S 0.0 1.2 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 27104 12544 5104 S 0.0 1.2 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 27104 12544 5104 S 0.0 1.2 0:00.04 /usr/local/freeswitch/bin/freeswitch -nc 27104 12544 5104 S 0.0 1.2 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 27104 12544 5104 S 0.0 1.2 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc VIRT RES SHR S CPU% MEM% TIME+ Command 28188 13620 5104 S 0.0 1.3 0:01.00 /usr/local/freeswitch/bin/freeswitch -nc 28188 13620 5104 S 0.0 1.3 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 28188 13620 5104 S 0.0 1.3 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 28188 13620 5104 S 0.0 1.3 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 28188 13620 5104 S 0.0 1.3 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 28188 13620 5104 S 0.0 1.3 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 28188 13620 5104 S 0.0 1.3 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 28188 13620 5104 S 0.0 1.3 0:00.06 /usr/local/freeswitch/bin/freeswitch -nc 28188 13620 5104 S 0.0 1.3 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 28188 13620 5104 S 0.0 1.3 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 28188 13620 5104 S 0.0 1.3 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 28188 13620 5104 S 0.0 1.3 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 28188 13620 5104 S 0.0 1.3 0:00.04 /usr/local/freeswitch/bin/freeswitch -nc 28188 13620 5104 S 0.0 1.3 0:00.14 /usr/local/freeswitch/bin/freeswitch -nc 28188 13620 5104 R 0.0 1.3 0:00.12 /usr/local/freeswitch/bin/freeswitch -nc 28188 13620 5104 S 0.0 1.3 0:00.04 /usr/local/freeswitch/bin/freeswitch -nc 28188 13620 5104 S 0.0 1.3 0:00.31 /usr/local/freeswitch/bin/freeswitch -nc 28188 13620 5104 S 0.0 1.3 0:00.28 /usr/local/freeswitch/bin/freeswitch -nc 28188 13620 5104 S 0.0 1.3 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 28188 13620 5104 S 0.0 1.3 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 28188 13620 5104 S 0.0 1.3 0:00.07 /usr/local/freeswitch/bin/freeswitch -nc 28188 13620 5104 S 0.0 1.3 0:00.01 /usr/local/freeswitch/bin/freeswitch -nc 28188 13620 5104 S 0.0 1.3 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc VIRT RES SHR S CPU% MEM% TIME+ Command 30092 15436 5120 S 0.0 1.5 0:01.00 /usr/local/freeswitch/bin/freeswitch -nc 30092 15436 5120 S 0.0 1.5 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 30092 15436 5120 S 0.0 1.5 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 30092 15436 5120 S 0.0 1.5 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 30092 15436 5120 S 0.0 1.5 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 30092 15436 5120 S 0.0 1.5 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 30092 15436 5120 S 0.0 1.5 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 30092 15436 5120 S 0.0 1.5 0:03.30 /usr/local/freeswitch/bin/freeswitch -nc 30092 15436 5120 S 0.0 1.5 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 30092 15436 5120 S 0.0 1.5 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 30092 15436 5120 S 0.0 1.5 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 30092 15436 5120 S 0.0 1.5 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 30092 15436 5120 S 0.0 1.5 0:00.11 /usr/local/freeswitch/bin/freeswitch -nc 30092 15436 5120 S 0.0 1.5 0:00.21 /usr/local/freeswitch/bin/freeswitch -nc 30092 15436 5120 S 0.0 1.5 0:06.80 /usr/local/freeswitch/bin/freeswitch -nc 30092 15436 5120 S 0.0 1.5 0:00.19 /usr/local/freeswitch/bin/freeswitch -nc 30092 15436 5120 S 0.0 1.5 0:07.05 /usr/local/freeswitch/bin/freeswitch -nc 30092 15436 5120 S 0.0 1.5 0:08.72 /usr/local/freeswitch/bin/freeswitch -nc 30092 15436 5120 S 0.0 1.5 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 30092 15436 5120 S 0.0 1.5 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc 30092 15436 5120 S 0.0 1.5 0:00.14 /usr/local/freeswitch/bin/freeswitch -nc 30092 15436 5120 S 0.0 1.5 0:00.01 /usr/local/freeswitch/bin/freeswitch -nc 30092 15436 5120 S 0.0 1.5 0:00.00 /usr/local/freeswitch/bin/freeswitch -nc
The numbers you posted are very low... can you elaborate on what exactly you're doing? If you have gigs and gigs of ram used then you have a leak... I suspect you do not. /b On Apr 24, 2009, at 5:40 PM, Charles Boening wrote:
I'm wondering about a possible memory leak in the latest versions.
For each FS process, the resident size seems to be incrementing by 8k at a slow, but fairly steady rate. Sometimes there are a few quick "bursts" in the increase and sometimes it seems to take longer to see a "bump" in usage. There are no calls being processed. I just started FS and have been watching it.
After upgrading from 1.0.1 to 1.0.4pre4, my FS process died after a few hours. I'm thinking this was due to a memory leak and the service running out of memory.
If it makes a difference, I upgraded from 1.0.1 to 1.0.4pre4 (saw memory leak there too) and then on to 1.0.4pre5 without wiping the instadirectory and starting over.
Please let me know what other information I can provide that will help find the reason for the ever increasing memory usage.
Thanks, Charlie
Brian West brian@freeswitch.org -- Meet us at ClueCon! http://www.cluecon.com
Brian, Thanks for the quick reply. I'm using FS for a conference server. We have just one conference room for internal use. It is connected via a SIP registration to our MetaSwitch. That registration allows multiple calls on the same registration. I could do a SIP trunk but this was quick and easy to setup. I'm concerned that FS quit unexpectedly a few time (pre4 not pre5) and that's how I got to poking around. Sounds like maybe I'm being a little alarmist. It's just harsh when the boss asks why the conference room isn't working. :) I'll see if I can get something setup to monitor memory usage over time. I'm just concerned that over time memory will keep increasing and increasing and that's what caused the process to crash. What would you suggest to try and track down why the FS process would suddenly stop working (not in process list even)? Charlie From: freeswitch-dev-bounces@lists.freeswitch.org [mailto:freeswitch-dev-bounces@lists.freeswitch.org] On Behalf Of Brian West Sent: Friday, April 24, 2009 4:26 PM To: freeswitch-dev@lists.freeswitch.org Subject: Re: [Freeswitch-dev] Memory Leak? - FreeSWITCH Version 1.0.4pre5 The numbers you posted are very low... can you elaborate on what exactly you're doing? If you have gigs and gigs of ram used then you have a leak... I suspect you do not. /b On Apr 24, 2009, at 5:40 PM, Charles Boening wrote: I'm wondering about a possible memory leak in the latest versions. For each FS process, the resident size seems to be incrementing by 8k at a slow, but fairly steady rate. Sometimes there are a few quick "bursts" in the increase and sometimes it seems to take longer to see a "bump" in usage. There are no calls being processed. I just started FS and have been watching it. After upgrading from 1.0.1 to 1.0.4pre4, my FS process died after a few hours. I'm thinking this was due to a memory leak and the service running out of memory. If it makes a difference, I upgraded from 1.0.1 to 1.0.4pre4 (saw memory leak there too) and then on to 1.0.4pre5 without wiping the instadirectory and starting over. Please let me know what other information I can provide that will help find the reason for the ever increasing memory usage. Thanks, Charlie Brian West brian@freeswitch.org<mailto:brian@freeswitch.org> -- Meet us at ClueCon! http://www.cluecon.com<http://www.cluecon.com/>
If you can please refer to the wiki for debugging procedures... I also recommend you join us on IRC. We run conference.freeswitch.org with many participants in a conference all day long without any problems and we usually keep it up to date every few days when we remember to do so. We do not see any memory leaks. http://wiki.freeswitch.org/wiki/Reporting_Bugs If freeswitch exits like you say it'll have a core file which you can run gdb on and it will tell us why it crashed usually. /b On Apr 24, 2009, at 6:36 PM, Charles Boening wrote:
Brian,
Thanks for the quick reply.
I’m using FS for a conference server. We have just one conference room for internal use. It is connected via a SIP registration to our MetaSwitch. That registration allows multiple calls on the same registration. I could do a SIP trunk but this was quick and easy to setup.
I’m concerned that FS quit unexpectedly a few time (pre4 not pre5) and that’s how I got to poking around. Sounds like maybe I’m being a little alarmist. It’s just harsh when the boss asks why the conference room isn’t working. :)
I’ll see if I can get something setup to monitor memory usage over time. I’m just concerned that over time memory will keep increasing and increasing and that’s what caused the process to crash.
What would you suggest to try and track down why the FS process would suddenly stop working (not in process list even)?
Charlie
Brian West brian@freeswitch.org -- Meet us at ClueCon! http://www.cluecon.com
Brian, Sounds good. I'll keep an eye on it. I appreciate the tips. I'll see about joining the IRC channel. :) Charlie -----Original Message----- From: freeswitch-dev-bounces@lists.freeswitch.org [mailto:freeswitch-dev-bounces@lists.freeswitch.org] On Behalf Of Brian West Sent: Friday, April 24, 2009 4:45 PM To: freeswitch-dev@lists.freeswitch.org Subject: Re: [Freeswitch-dev] Memory Leak? - FreeSWITCH Version 1.0.4pre5 If you can please refer to the wiki for debugging procedures... I also recommend you join us on IRC. We run conference.freeswitch.org with many participants in a conference all day long without any problems and we usually keep it up to date every few days when we remember to do so. We do not see any memory leaks. http://wiki.freeswitch.org/wiki/Reporting_Bugs If freeswitch exits like you say it'll have a core file which you can run gdb on and it will tell us why it crashed usually. /b On Apr 24, 2009, at 6:36 PM, Charles Boening wrote:
Brian,
Thanks for the quick reply.
I'm using FS for a conference server. We have just one conference room for internal use. It is connected via a SIP registration to our MetaSwitch. That registration allows multiple calls on the same registration. I could do a SIP trunk but this was quick and easy to setup.
I'm concerned that FS quit unexpectedly a few time (pre4 not pre5) and that's how I got to poking around. Sounds like maybe I'm being a little alarmist. It's just harsh when the boss asks why the conference room isn't working. :)
I'll see if I can get something setup to monitor memory usage over time. I'm just concerned that over time memory will keep increasing and increasing and that's what caused the process to crash.
What would you suggest to try and track down why the FS process would suddenly stop working (not in process list even)?
Charlie
Brian West brian@freeswitch.org -- Meet us at ClueCon! http://www.cluecon.com _______________________________________________ Freeswitch-dev mailing list Freeswitch-dev@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev http://www.freeswitch.org
if it already crashed, you would already have core files now you could trace. did you witness any crash or find any core files? mention to the boss if he emails consulting@freeswitch.org he could purchase a support contract from FreeSWITCH Solutions and we can make sure his box is stable. On Fri, Apr 24, 2009 at 7:16 PM, Charles Boening <charlieb@cot.net> wrote:
Brian,
Sounds good. I'll keep an eye on it.
I appreciate the tips. I'll see about joining the IRC channel. :)
Charlie
-----Original Message----- From: freeswitch-dev-bounces@lists.freeswitch.org [mailto: freeswitch-dev-bounces@lists.freeswitch.org] On Behalf Of Brian West Sent: Friday, April 24, 2009 4:45 PM To: freeswitch-dev@lists.freeswitch.org Subject: Re: [Freeswitch-dev] Memory Leak? - FreeSWITCH Version 1.0.4pre5
If you can please refer to the wiki for debugging procedures... I also recommend you join us on IRC. We run conference.freeswitch.org with many participants in a conference all day long without any problems and we usually keep it up to date every few days when we remember to do so. We do not see any memory leaks.
http://wiki.freeswitch.org/wiki/Reporting_Bugs
If freeswitch exits like you say it'll have a core file which you can run gdb on and it will tell us why it crashed usually.
/b
On Apr 24, 2009, at 6:36 PM, Charles Boening wrote:
Brian,
Thanks for the quick reply.
I'm using FS for a conference server. We have just one conference room for internal use. It is connected via a SIP registration to our MetaSwitch. That registration allows multiple calls on the same registration. I could do a SIP trunk but this was quick and easy to setup.
I'm concerned that FS quit unexpectedly a few time (pre4 not pre5) and that's how I got to poking around. Sounds like maybe I'm being a little alarmist. It's just harsh when the boss asks why the conference room isn't working. :)
I'll see if I can get something setup to monitor memory usage over time. I'm just concerned that over time memory will keep increasing and increasing and that's what caused the process to crash.
What would you suggest to try and track down why the FS process would suddenly stop working (not in process list even)?
Charlie
Brian West brian@freeswitch.org
-- Meet us at ClueCon! http://www.cluecon.com
_______________________________________________ Freeswitch-dev mailing list Freeswitch-dev@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev http://www.freeswitch.org
_______________________________________________ Freeswitch-dev mailing list Freeswitch-dev@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev http://www.freeswitch.org
-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ AIM: anthm MSN:anthony_minessale@hotmail.com <MSN%3Aanthony_minessale@hotmail.com> GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com<PAYPAL%3Aanthony.minessale@gmail.com> IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org <sip%3A888@conference.freeswitch.org> iax:guest@conference.freeswitch.org/888 googletalk:conf+888@conference.freeswitch.org<googletalk%3Aconf%2B888@conference.freeswitch.org> pstn:213-799-1400
participants (3)
-
Anthony Minessale
-
Brian West
-
Charles Boening