Re: [Freeswitch-users] High CPU load but only few sessions
It could be due to registrations. I am currently trying to troubleshoot this problem. I used a sipp scenario to authenticate with fs and register about 2000 different accounts (absolutely no calls made on the test setup). Memory usage increases continuously and does not decrease at all and crosses more than 1 GB in a few hours. On the other hand, there is another fs setup with bypass media turned on and no registrations and is up for almost 45 days without restart and has consumed only about 95 MB of memory and twice as much virtual memory. B Karthik
If you guys are not on rev 12914 then you'll need to update. /b On Apr 6, 2009, at 9:04 AM, B Karthik wrote:
It could be due to registrations. I am currently trying to troubleshoot this problem. I used a sipp scenario to authenticate with fs and register about 2000 different accounts (absolutely no calls made on the test setup). Memory usage increases continuously and does not decrease at all and crosses more than 1 GB in a few hours. On the other hand, there is another fs setup with bypass media turned on and no registrations and is up for almost 45 days without restart and has consumed only about 95 MB of memory and twice as much virtual memory.
B Karthik
Brian West brian@freeswitch.org -- Meet us a ClueCon! http://www.cluecon.com
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello, in my scenario I have a reregistration interval of 60 seconds and 32 sip phones connected. So I have a good amount of registrations. Additionally each phone subscribes to itself for MWI and some phone subscribes to others for BLF. Registrar database looks fine. No unused entries there. First I will upgrade to recent svn trunk. If that doesn't help, I will run valgrind on my production system and hope that my machine is strong enough to deliver its service even with valrgind. regards Helmut On 06.04.2009 16:04, B Karthik wrote:
It could be due to registrations. I am currently trying to troubleshoot this problem. I used a sipp scenario to authenticate with fs and register about 2000 different accounts (absolutely no calls made on the test setup). Memory usage increases continuously and does not decrease at all and crosses more than 1 GB in a few hours. On the other hand, there is another fs setup with bypass media turned on and no registrations and is up for almost 45 days without restart and has consumed only about 95 MB of memory and twice as much virtual memory.
B Karthik
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) iD8DBQFJ2hJl4tZeNddg3dwRAmEUAKCTt1aBPlp1pgs3RHw2AVEuH8ixqgCfWkm8 6vsgh6Ha34/gdg6iDEEEOR0= =2H4m -----END PGP SIGNATURE-----
I updated to the latest revision. No Luck -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello, in my scenario I have a reregistration interval of 60 seconds and 32 sip phones connected. So I have a good amount of registrations. Additionally each phone subscribes to itself for MWI and some phone subscribes to others for BLF. Registrar database looks fine. No unused entries there. First I will upgrade to recent svn trunk. If that doesn't help, I will run valgrind on my production system and hope that my machine is strong enough to deliver its service even with valrgind. regards Helmut On 06.04.2009 16:04, B Karthik wrote:
It could be due to registrations. I am currently trying to troubleshoot this problem. I used a sipp scenario to authenticate with fs and register about 2000 different accounts (absolutely no calls made on the test setup). Memory usage increases continuously and does not decrease at all and crosses more than 1 GB in a few hours. On the other hand, there is another fs setup with bypass media turned on and no registrations and is up for almost 45 days without restart and has consumed only about 95 MB of memory and twice as much virtual memory.
B Karthik
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) iD8DBQFJ2hJl4tZeNddg3dwRAmEUAKCTt1aBPlp1pgs3RHw2AVEuH8ixqgCfWkm8 6vsgh6Ha34/gdg6iDEEEOR0= =2H4m -----END PGP SIGNATURE----- _______________________________________________ Freeswitch-users mailing list Freeswitch-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org -- View this message in context: http://n2.nabble.com/High-CPU-load-but-only-few-sessions-tp2538703p2593558.h... Sent from the freeswitch-users mailing list archive at Nabble.com.
did you both follow the policy to upgrade? stop fs type make current restart fs if you do not rebuild sofia too (only happens in make current) I just fixed all the problems with these symptoms, 38 million registrations in a 2 day span using 62mb btw, did we not make the policy clear enough about not reporting bugs on the mailing list? On Mon, Apr 6, 2009 at 10:02 AM, B Karthik <carthick84@gmail.com> wrote:
I updated to the latest revision. No Luck
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hello,
in my scenario I have a reregistration interval of 60 seconds and 32 sip phones connected. So I have a good amount of registrations. Additionally each phone subscribes to itself for MWI and some phone subscribes to others for BLF.
Registrar database looks fine. No unused entries there.
First I will upgrade to recent svn trunk. If that doesn't help, I will run valgrind on my production system and hope that my machine is strong enough to deliver its service even with valrgind.
regards Helmut
On 06.04.2009 16:04, B Karthik wrote:
It could be due to registrations. I am currently trying to troubleshoot this problem. I used a sipp scenario to authenticate with fs and register about 2000 different accounts (absolutely no calls made on the test setup). Memory usage increases continuously and does not decrease at all and crosses more than 1 GB in a few hours. On the other hand, there is another fs setup with bypass media turned on and no registrations and is up for almost 45 days without restart and has consumed only about 95 MB of memory and twice as much virtual memory.
B Karthik
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32)
iD8DBQFJ2hJl4tZeNddg3dwRAmEUAKCTt1aBPlp1pgs3RHw2AVEuH8ixqgCfWkm8 6vsgh6Ha34/gdg6iDEEEOR0= =2H4m -----END PGP SIGNATURE-----
_______________________________________________ Freeswitch-users mailing list Freeswitch-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- View this message in context: http://n2.nabble.com/High-CPU-load-but-only-few-sessions-tp2538703p2593558.h... Sent from the freeswitch-users mailing list archive at Nabble.com.
_______________________________________________ Freeswitch-users mailing list Freeswitch-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ AIM: anthm MSN:anthony_minessale@hotmail.com <MSN%3Aanthony_minessale@hotmail.com> GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com<PAYPAL%3Aanthony.minessale@gmail.com> IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org <sip%3A888@conference.freeswitch.org> iax:guest@conference.freeswitch.org/888 googletalk:conf+888@conference.freeswitch.org<googletalk%3Aconf%2B888@conference.freeswitch.org> pstn:213-799-1400
yes, i did exactly as you mentioned. I will try building again from a fresh checkout. I am sorry about not following the policy, I didn't intend to report it as a bug since i was still unsure that it could be a problem in Freeswitch. did you both follow the policy to upgrade? stop fs type make current restart fs if you do not rebuild sofia too (only happens in make current) I just fixed all the problems with these symptoms, 38 million registrations in a 2 day span using 62mb btw, did we not make the policy clear enough about not reporting bugs on the mailing list? On Mon, Apr 6, 2009 at 10:02 AM, B Karthik <carthick84@gmail.com> wrote:
I updated to the latest revision. No Luck
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hello,
in my scenario I have a reregistration interval of 60 seconds and 32 sip phones connected. So I have a good amount of registrations. Additionally each phone subscribes to itself for MWI and some phone subscribes to others for BLF.
Registrar database looks fine. No unused entries there.
First I will upgrade to recent svn trunk. If that doesn't help, I will run valgrind on my production system and hope that my machine is strong enough to deliver its service even with valrgind.
regards Helmut
On 06.04.2009 16:04, B Karthik wrote:
It could be due to registrations. I am currently trying to troubleshoot this problem. I used a sipp scenario to authenticate with fs and register about 2000 different accounts (absolutely no calls made on the test setup). Memory usage increases continuously and does not decrease at all and crosses more than 1 GB in a few hours. On the other hand, there is another fs setup with bypass media turned on and no registrations and is up for almost 45 days without restart and has consumed only about 95 MB of memory and twice as much virtual memory.
B Karthik
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32)
iD8DBQFJ2hJl4tZeNddg3dwRAmEUAKCTt1aBPlp1pgs3RHw2AVEuH8ixqgCfWkm8 6vsgh6Ha34/gdg6iDEEEOR0= =2H4m -----END PGP SIGNATURE-----
_______________________________________________ Freeswitch-users mailing list Freeswitch-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- View this message in context: http://n2.nabble.com/High-CPU-load-but-only-few-sessions-tp2538703p2593558.h... Sent from the freeswitch-users mailing list archive at Nabble.com.
_______________________________________________ Freeswitch-users mailing list Freeswitch-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ AIM: anthm MSN:anthony_minessale@hotmail.com <MSN%3Aanthony_minessale@hotmail.com> GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com<PAYPAL%3Aanthony.minessale@gmail.com> IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org <sip%3A888@conference.freeswitch.org> iax:guest@conference.freeswitch.org/888 googletalk:conf+888@conference.freeswitch.org<googletalk%3Aconf%2B888@conference.freeswitch.org> pstn:213-799-1400 _______________________________________________ Freeswitch-users mailing list Freeswitch-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org -- View this message in context: http://n2.nabble.com/High-CPU-load-but-only-few-sessions-tp2538703p2593704.h... Sent from the freeswitch-users mailing list archive at Nabble.com.
Great work. Memory usage is constant now. Memory is now Res :162M Virt: 483M for more than 10 mins without increasing. Call rate was set to 100 in sipp. However "top" usage is very high - 137% - 200%. MySQL usage is about 3% constant. I will also try overriding the XML bind function with a function which can lookup an in memory cached hash table for directory entries. I will also try by disabling mysql and post the results shortly. yes, i did exactly as you mentioned. I will try building again from a fresh checkout. I am sorry about not following the policy, I didn't intend to report it as a bug since i was still unsure that it could be a problem in Freeswitch. did you both follow the policy to upgrade? stop fs type make current restart fs if you do not rebuild sofia too (only happens in make current) I just fixed all the problems with these symptoms, 38 million registrations in a 2 day span using 62mb btw, did we not make the policy clear enough about not reporting bugs on the mailing list? On Mon, Apr 6, 2009 at 10:02 AM, B Karthik <carthick84@gmail.com> wrote:
I updated to the latest revision. No Luck
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hello,
in my scenario I have a reregistration interval of 60 seconds and 32 sip phones connected. So I have a good amount of registrations. Additionally each phone subscribes to itself for MWI and some phone subscribes to others for BLF.
Registrar database looks fine. No unused entries there.
First I will upgrade to recent svn trunk. If that doesn't help, I will run valgrind on my production system and hope that my machine is strong enough to deliver its service even with valrgind.
regards Helmut
On 06.04.2009 16:04, B Karthik wrote:
It could be due to registrations. I am currently trying to troubleshoot this problem. I used a sipp scenario to authenticate with fs and register about 2000 different accounts (absolutely no calls made on the test setup). Memory usage increases continuously and does not decrease at all and crosses more than 1 GB in a few hours. On the other hand, there is another fs setup with bypass media turned on and no registrations and is up for almost 45 days without restart and has consumed only about 95 MB of memory and twice as much virtual memory.
B Karthik
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32)
iD8DBQFJ2hJl4tZeNddg3dwRAmEUAKCTt1aBPlp1pgs3RHw2AVEuH8ixqgCfWkm8 6vsgh6Ha34/gdg6iDEEEOR0= =2H4m -----END PGP SIGNATURE-----
_______________________________________________ Freeswitch-users mailing list Freeswitch-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- View this message in context: http://n2.nabble.com/High-CPU-load-but-only-few-sessions-tp2538703p2593558.h... Sent from the freeswitch-users mailing list archive at Nabble.com.
_______________________________________________ Freeswitch-users mailing list Freeswitch-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ AIM: anthm MSN:anthony_minessale@hotmail.com <MSN%3Aanthony_minessale@hotmail.com> GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com<PAYPAL%3Aanthony.minessale@gmail.com> IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org <sip%3A888@conference.freeswitch.org> iax:guest@conference.freeswitch.org/888 googletalk:conf+888@conference.freeswitch.org<googletalk%3Aconf%2B888@conference.freeswitch.org> pstn:213-799-1400 _______________________________________________ Freeswitch-users mailing list Freeswitch-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org -- View this message in context: http://n2.nabble.com/High-CPU-load-but-only-few-sessions-tp2538703p2593907.h... Sent from the freeswitch-users mailing list archive at Nabble.com.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello Anthony, I did a fresh checkout, compiled it, installed it into a clean directory and will switch over to it tomorrow morning. I hope I can reuse this existing directories: db/ conf/ storage/ sounds/ On 06.04.2009 17:21, Anthony Minessale wrote:
btw, did we not make the policy clear enough about not reporting bugs on the mailing list?
Yes you did ... I apologize! regards Helmut -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) iD8DBQFJ2iko4tZeNddg3dwRAnfWAKCHz7MeJZscWPLMkKQV6lflp8Wi+gCePRQm Cz3J66fWtZEMK+n7D8GXAM8= =mFi1 -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello Anthony, after one day running the actual trunk things looks much better than before. FS started 24h ago with 129MB VRAM and grows to 136MB VRAM by now. CPU is around 1.3% Thanks for your work! regards helmut On 06.04.2009 18:09, Helmut Kuper wrote:
Hello Anthony,
I did a fresh checkout, compiled it, installed it into a clean directory and will switch over to it tomorrow morning. I hope I can reuse this existing directories: -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32)
iD8DBQFJ3G4H4tZeNddg3dwRAjU7AJ0T9Fl230VfOS00Wbot3A1DTZtBUwCghFHw /CrpYYhdSGmFy+C6RxaIK1A= =yeof -----END PGP SIGNATURE-----
participants (4)
-
Anthony Minessale
-
B Karthik
-
Brian West
-
Helmut Kuper