Memory leak when using ESL
Hello Using ESL found a memory leak problem valgrind --tool=memcheck --leak-check=full ./myprog output as follows ==6872== 29,986 (88 direct, 29,898 indirect) bytes in 1 blocks are definitely lost in loss record 401 of 408 ==6872== at 0x4A06AAA: malloc (vg_replace_malloc.c:291) ==6872== by 0x413941: esl_event_create_subclass (esl_event.c:179) ==6872== by 0x410DC3: esl_recv_event (esl.c:1245) ==6872== by 0x411675: esl_send_recv_timed (esl.c:1517) ==6872== by 0x411C1B: esl_execute (esl.c:555) ==6872== by 0x408758: CRBT::Route() (CRBT.cpp:42) ==6872== by 0x40A02F: DPL::route() (DPL.cpp:234) ==6872== by 0x40FCA8: mycallback(int, int, sockaddr_in*) (listener.cpp:47) ==6872== by 0x41032F: client_thread (esl.c:654) ==6872== by 0x413FC7: thread_launch (esl_threadmutex.c:74) ==6872== by 0x34CAA07850: start_thread (in /lib64/libpthread-2.12.so) ==6872== by 0x34CA6E890C: clone (in /lib64/libc-2.12.so) ==6872== LEAK SUMMARY: ==6872== definitely lost: 88 bytes in 1 blocks ==6872== indirectly lost: 29,898 bytes in 76 blocks ==6872== possibly lost: 550,082 bytes in 51 blocks ==6872== still reachable: 832,937 bytes in 840 blocks ==6872== suppressed: 0 bytes in 0 blocks ==6872== Reachable blocks (those to which a pointer was found) are not shown. ==6872== To see them, rerun with: --leak-check=full --show-leak-kinds=all Part of my code is as follows bool CRBT::Route() { string data = "ringback=file_string://"; data.append(FileName); esl_execute(handle, "set", data.c_str(), NULL); return true; } The OS version is Centos x86_64 6.3 The Freeswitch version is 1.2.13 Thanks for your help.
I can confirm significant Freeswitch process memory growth over time when ESL is used intensively, from 100MB in normal case to 700-900MB after few hours of high esl load, though I have no valgrind check results, as I can't do tests at production server. With less than 300 channels in peak we have 900MB FS process size, never shrinking back :( 2013/11/12 gggyniidt <gggyniidt@gmail.com>
Hello Using ESL found a memory leak problem valgrind --tool=memcheck --leak-check=full ./myprog output as follows
==6872== 29,986 (88 direct, 29,898 indirect) bytes in 1 blocks are definitely lost in loss record 401 of 408 ==6872== at 0x4A06AAA: malloc (vg_replace_malloc.c:291) ==6872== by 0x413941: esl_event_create_subclass (esl_event.c:179)
==6872== by 0x410DC3: esl_recv_event (esl.c:1245)
==6872== by 0x411675: esl_send_recv_timed (esl.c:1517)
==6872== by 0x411C1B: esl_execute (esl.c:555)
==6872== by 0x408758: CRBT::Route() (CRBT.cpp:42)
==6872== by 0x40A02F: DPL::route() (DPL.cpp:234)
==6872== by 0x40FCA8: mycallback(int, int, sockaddr_in*) (listener.cpp:47) ==6872== by 0x41032F: client_thread (esl.c:654)
==6872== by 0x413FC7: thread_launch (esl_threadmutex.c:74)
==6872== by 0x34CAA07850: start_thread (in /lib64/libpthread-2.12.so)
==6872== by 0x34CA6E890C: clone (in /lib64/libc-2.12.so)
==6872== LEAK SUMMARY: ==6872== definitely lost: 88 bytes in 1 blocks ==6872== indirectly lost: 29,898 bytes in 76 blocks ==6872== possibly lost: 550,082 bytes in 51 blocks ==6872== still reachable: 832,937 bytes in 840 blocks ==6872== suppressed: 0 bytes in 0 blocks ==6872== Reachable blocks (those to which a pointer was found) are not shown. ==6872== To see them, rerun with: --leak-check=full --show-leak-kinds=all
Part of my code is as follows
bool CRBT::Route() { string data = "ringback=file_string://"; data.append(FileName);
esl_execute(handle, "set", data.c_str(), NULL); return true; }
The OS version is Centos x86_64 6.3 The Freeswitch version is 1.2.13
Thanks for your help.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Best regards, Dmitry Sytchev, IT Engineer
You are showing valgrind traces of ESL client code but now you are saying your FS process has memory growth not the ESL app? You should be doing this on jira and you should have more details about the env. This mailing list is not for bug reports. I did push a patch for what I think might be the problem you showed on esl lib. FS can use up to a gig of memory without giving it back. To make sure you have a real memory leak you need to have it never peak but continue to grow. On Tue, Nov 12, 2013 at 11:17 AM, Dmitry Sytchev <kbdfck@gmail.com> wrote:
I can confirm significant Freeswitch process memory growth over time when ESL is used intensively, from 100MB in normal case to 700-900MB after few hours of high esl load, though I have no valgrind check results, as I can't do tests at production server. With less than 300 channels in peak we have 900MB FS process size, never shrinking back :(
2013/11/12 gggyniidt <gggyniidt@gmail.com>
Hello Using ESL found a memory leak problem valgrind --tool=memcheck --leak-check=full ./myprog output as follows
==6872== 29,986 (88 direct, 29,898 indirect) bytes in 1 blocks are definitely lost in loss record 401 of 408 ==6872== at 0x4A06AAA: malloc (vg_replace_malloc.c:291) ==6872== by 0x413941: esl_event_create_subclass (esl_event.c:179)
==6872== by 0x410DC3: esl_recv_event (esl.c:1245)
==6872== by 0x411675: esl_send_recv_timed (esl.c:1517)
==6872== by 0x411C1B: esl_execute (esl.c:555)
==6872== by 0x408758: CRBT::Route() (CRBT.cpp:42)
==6872== by 0x40A02F: DPL::route() (DPL.cpp:234)
==6872== by 0x40FCA8: mycallback(int, int, sockaddr_in*) (listener.cpp:47) ==6872== by 0x41032F: client_thread (esl.c:654)
==6872== by 0x413FC7: thread_launch (esl_threadmutex.c:74)
==6872== by 0x34CAA07850: start_thread (in /lib64/libpthread-2.12.so)
==6872== by 0x34CA6E890C: clone (in /lib64/libc-2.12.so)
==6872== LEAK SUMMARY: ==6872== definitely lost: 88 bytes in 1 blocks ==6872== indirectly lost: 29,898 bytes in 76 blocks ==6872== possibly lost: 550,082 bytes in 51 blocks ==6872== still reachable: 832,937 bytes in 840 blocks ==6872== suppressed: 0 bytes in 0 blocks ==6872== Reachable blocks (those to which a pointer was found) are not shown. ==6872== To see them, rerun with: --leak-check=full --show-leak-kinds=all
Part of my code is as follows
bool CRBT::Route() { string data = "ringback=file_string://"; data.append(FileName);
esl_execute(handle, "set", data.c_str(), NULL); return true; }
The OS version is Centos x86_64 6.3 The Freeswitch version is 1.2.13
Thanks for your help.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Best regards,
Dmitry Sytchev, IT Engineer
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900
Thanks for Dmitry and Anthony's help, But my problem is not like Dmitry said. My program is a esl client. The leak is from esl client, not freeswitch. I add the Anthony's patch code into esl.c,but it doesn't work. Each call will cause the 20-40KB of memory growth. I make 3 calls, the valgrind outputs like this ==14459== 90,642 (264 direct, 90,378 indirect) bytes in 3 blocks are definitely lost in loss record 403 of 408 ==14459== at 0x4A06AAA: malloc (vg_replace_malloc.c:291) ==14459== by 0x414524: esl_event_create_subclass (esl_event.c:179) ==14459== by 0x412054: esl_recv_event (esl.c:1245) ==14459== by 0x411C38: esl_recv_event_timed (esl.c:1137) ==14459== by 0x412E22: esl_send_recv_timed (esl.c:1521) ==14459== by 0x410C06: esl_execute (esl.c:555) ==14459== by 0x4087A8: CRBT::Route() (CRBT.cpp:42) ==14459== by 0x409FE6: DPL::route() (DPL.cpp:286) ==14459== by 0x40FBE0: mycallback(int, int, sockaddr_in*) (listener.cpp:47) ==14459== by 0x410FED: client_thread (esl.c:654) ==14459== by 0x41A334: thread_launch (esl_threadmutex.c:74) ==14459== by 0x34CAA07850: start_thread (in /lib64/libpthread-2.12.so) ==14459== ==14459== LEAK SUMMARY: ==14459== definitely lost: 264 bytes in 3 blocks ==14459== indirectly lost: 90,378 bytes in 228 blocks ==14459== possibly lost: 550,082 bytes in 51 blocks ==14459== still reachable: 832,938 bytes in 840 blocks ==14459== suppressed: 0 bytes in 0 blocks ==14459== Reachable blocks (those to which a pointer was found) are not shown. ==14459== To see them, rerun with: --leak-check=full --show-leak-kinds=all I checked my code and no leak found. The leak may be in esl.c. 2013/11/13 Anthony Minessale <anthony.minessale@gmail.com>
You are showing valgrind traces of ESL client code but now you are saying your FS process has memory growth not the ESL app? You should be doing this on jira and you should have more details about the env. This mailing list is not for bug reports.
I did push a patch for what I think might be the problem you showed on esl lib.
FS can use up to a gig of memory without giving it back. To make sure you have a real memory leak you need to have it never peak but continue to grow.
On Tue, Nov 12, 2013 at 11:17 AM, Dmitry Sytchev <kbdfck@gmail.com> wrote:
I can confirm significant Freeswitch process memory growth over time when ESL is used intensively, from 100MB in normal case to 700-900MB after few hours of high esl load, though I have no valgrind check results, as I can't do tests at production server. With less than 300 channels in peak we have 900MB FS process size, never shrinking back :(
2013/11/12 gggyniidt <gggyniidt@gmail.com>
Hello Using ESL found a memory leak problem valgrind --tool=memcheck --leak-check=full ./myprog output as follows
==6872== 29,986 (88 direct, 29,898 indirect) bytes in 1 blocks are definitely lost in loss record 401 of 408 ==6872== at 0x4A06AAA: malloc (vg_replace_malloc.c:291) ==6872== by 0x413941: esl_event_create_subclass (esl_event.c:179)
==6872== by 0x410DC3: esl_recv_event (esl.c:1245)
==6872== by 0x411675: esl_send_recv_timed (esl.c:1517)
==6872== by 0x411C1B: esl_execute (esl.c:555)
==6872== by 0x408758: CRBT::Route() (CRBT.cpp:42)
==6872== by 0x40A02F: DPL::route() (DPL.cpp:234)
==6872== by 0x40FCA8: mycallback(int, int, sockaddr_in*) (listener.cpp:47) ==6872== by 0x41032F: client_thread (esl.c:654)
==6872== by 0x413FC7: thread_launch (esl_threadmutex.c:74)
==6872== by 0x34CAA07850: start_thread (in /lib64/libpthread-2.12.so)
==6872== by 0x34CA6E890C: clone (in /lib64/libc-2.12.so)
==6872== LEAK SUMMARY: ==6872== definitely lost: 88 bytes in 1 blocks ==6872== indirectly lost: 29,898 bytes in 76 blocks ==6872== possibly lost: 550,082 bytes in 51 blocks ==6872== still reachable: 832,937 bytes in 840 blocks ==6872== suppressed: 0 bytes in 0 blocks ==6872== Reachable blocks (those to which a pointer was found) are not shown. ==6872== To see them, rerun with: --leak-check=full --show-leak-kinds=all
Part of my code is as follows
bool CRBT::Route() { string data = "ringback=file_string://"; data.append(FileName);
esl_execute(handle, "set", data.c_str(), NULL); return true; }
The OS version is Centos x86_64 6.3 The Freeswitch version is 1.2.13
Thanks for your help.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Best regards,
Dmitry Sytchev, IT Engineer
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Please create a JIRA ticket with your information! 2013/11/13 gggyniidt <gggyniidt@gmail.com>
Thanks for Dmitry and Anthony's help, But my problem is not like Dmitry said.
My program is a esl client. The leak is from esl client, not freeswitch.
I add the Anthony's patch code into esl.c,but it doesn't work. Each call will cause the 20-40KB of memory growth.
I make 3 calls, the valgrind outputs like this
==14459== 90,642 (264 direct, 90,378 indirect) bytes in 3 blocks are definitely lost in loss record 403 of 408 ==14459== at 0x4A06AAA: malloc (vg_replace_malloc.c:291)
==14459== by 0x414524: esl_event_create_subclass (esl_event.c:179)
==14459== by 0x412054: esl_recv_event (esl.c:1245)
==14459== by 0x411C38: esl_recv_event_timed (esl.c:1137)
==14459== by 0x412E22: esl_send_recv_timed (esl.c:1521)
==14459== by 0x410C06: esl_execute (esl.c:555)
==14459== by 0x4087A8: CRBT::Route() (CRBT.cpp:42)
==14459== by 0x409FE6: DPL::route() (DPL.cpp:286)
==14459== by 0x40FBE0: mycallback(int, int, sockaddr_in*) (listener.cpp:47) ==14459== by 0x410FED: client_thread (esl.c:654)
==14459== by 0x41A334: thread_launch (esl_threadmutex.c:74)
==14459== by 0x34CAA07850: start_thread (in /lib64/libpthread-2.12.so)
==14459==
==14459== LEAK SUMMARY: ==14459== definitely lost: 264 bytes in 3 blocks ==14459== indirectly lost: 90,378 bytes in 228 blocks ==14459== possibly lost: 550,082 bytes in 51 blocks ==14459== still reachable: 832,938 bytes in 840 blocks ==14459== suppressed: 0 bytes in 0 blocks ==14459== Reachable blocks (those to which a pointer was found) are not shown. ==14459== To see them, rerun with: --leak-check=full --show-leak-kinds=all
I checked my code and no leak found. The leak may be in esl.c.
2013/11/13 Anthony Minessale <anthony.minessale@gmail.com>
You are showing valgrind traces of ESL client code but now you are saying your FS process has memory growth not the ESL app? You should be doing this on jira and you should have more details about the env. This mailing list is not for bug reports.
I did push a patch for what I think might be the problem you showed on esl lib.
FS can use up to a gig of memory without giving it back. To make sure you have a real memory leak you need to have it never peak but continue to grow.
On Tue, Nov 12, 2013 at 11:17 AM, Dmitry Sytchev <kbdfck@gmail.com>wrote:
I can confirm significant Freeswitch process memory growth over time when ESL is used intensively, from 100MB in normal case to 700-900MB after few hours of high esl load, though I have no valgrind check results, as I can't do tests at production server. With less than 300 channels in peak we have 900MB FS process size, never shrinking back :(
2013/11/12 gggyniidt <gggyniidt@gmail.com>
Hello Using ESL found a memory leak problem valgrind --tool=memcheck --leak-check=full ./myprog output as follows
==6872== 29,986 (88 direct, 29,898 indirect) bytes in 1 blocks are definitely lost in loss record 401 of 408 ==6872== at 0x4A06AAA: malloc (vg_replace_malloc.c:291) ==6872== by 0x413941: esl_event_create_subclass (esl_event.c:179)
==6872== by 0x410DC3: esl_recv_event (esl.c:1245)
==6872== by 0x411675: esl_send_recv_timed (esl.c:1517)
==6872== by 0x411C1B: esl_execute (esl.c:555)
==6872== by 0x408758: CRBT::Route() (CRBT.cpp:42)
==6872== by 0x40A02F: DPL::route() (DPL.cpp:234)
==6872== by 0x40FCA8: mycallback(int, int, sockaddr_in*) (listener.cpp:47) ==6872== by 0x41032F: client_thread (esl.c:654)
==6872== by 0x413FC7: thread_launch (esl_threadmutex.c:74)
==6872== by 0x34CAA07850: start_thread (in /lib64/libpthread-2.12.so)
==6872== by 0x34CA6E890C: clone (in /lib64/libc-2.12.so)
==6872== LEAK SUMMARY: ==6872== definitely lost: 88 bytes in 1 blocks ==6872== indirectly lost: 29,898 bytes in 76 blocks ==6872== possibly lost: 550,082 bytes in 51 blocks ==6872== still reachable: 832,937 bytes in 840 blocks ==6872== suppressed: 0 bytes in 0 blocks ==6872== Reachable blocks (those to which a pointer was found) are not shown. ==6872== To see them, rerun with: --leak-check=full --show-leak-kinds=all
Part of my code is as follows
bool CRBT::Route() { string data = "ringback=file_string://"; data.append(FileName);
esl_execute(handle, "set", data.c_str(), NULL); return true; }
The OS version is Centos x86_64 6.3 The Freeswitch version is 1.2.13
Thanks for your help.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Best regards,
Dmitry Sytchev, IT Engineer
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Best regards, Dmitry Sytchev, IT Engineer
I dug into ESL for a while. I ran into some issues that I corrected but the only thing that I can see that would show up as a memory leak looking like what you show there is if you are doing a forking server and you are calling fork after you create or attach the handle. Make sure you fork at the beginning or you will duplicate the handle and all the memory its using and it will never be destroyed. If you do "make" in esl dir you will have a default app "testserver_fork" you can run and point to it with socket app to port 8040. Running it on valgrind with full leak detection gives me. ==11807== Memcheck, a memory error detector ==11807== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al. ==11807== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info ==11807== Command: ./testserver ==11807== Parent PID: 28177 ==11807== ==11820== ==11820== HEAP SUMMARY: ==11820== in use at exit: 0 bytes in 0 blocks ==11820== total heap usage: 5,555 allocs, 5,555 frees, 7,293,931 bytes allocated ==11820== ==11820== All heap blocks were freed -- no leaks are possible ==11820== ==11820== For counts of detected and suppressed errors, rerun with: -v ==11820== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 4 from 4) ==11807== ==11807== HEAP SUMMARY: ==11807== in use at exit: 0 bytes in 0 blocks ==11807== total heap usage: 0 allocs, 0 frees, 0 bytes allocated ==11807== ==11807== All heap blocks were freed -- no leaks are possible ==11807== ==11807== For counts of detected and suppressed errors, rerun with: -v ==11807== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 4 from 4) On Wed, Nov 13, 2013 at 7:21 AM, Dmitry Sytchev <kbdfck@gmail.com> wrote:
Please create a JIRA ticket with your information!
2013/11/13 gggyniidt <gggyniidt@gmail.com>
Thanks for Dmitry and Anthony's help, But my problem is not like Dmitry said.
My program is a esl client. The leak is from esl client, not freeswitch.
I add the Anthony's patch code into esl.c,but it doesn't work. Each call will cause the 20-40KB of memory growth.
I make 3 calls, the valgrind outputs like this
==14459== 90,642 (264 direct, 90,378 indirect) bytes in 3 blocks are definitely lost in loss record 403 of 408 ==14459== at 0x4A06AAA: malloc (vg_replace_malloc.c:291)
==14459== by 0x414524: esl_event_create_subclass (esl_event.c:179)
==14459== by 0x412054: esl_recv_event (esl.c:1245)
==14459== by 0x411C38: esl_recv_event_timed (esl.c:1137)
==14459== by 0x412E22: esl_send_recv_timed (esl.c:1521)
==14459== by 0x410C06: esl_execute (esl.c:555)
==14459== by 0x4087A8: CRBT::Route() (CRBT.cpp:42)
==14459== by 0x409FE6: DPL::route() (DPL.cpp:286)
==14459== by 0x40FBE0: mycallback(int, int, sockaddr_in*) (listener.cpp:47) ==14459== by 0x410FED: client_thread (esl.c:654)
==14459== by 0x41A334: thread_launch (esl_threadmutex.c:74)
==14459== by 0x34CAA07850: start_thread (in /lib64/libpthread-2.12.so)
==14459==
==14459== LEAK SUMMARY: ==14459== definitely lost: 264 bytes in 3 blocks ==14459== indirectly lost: 90,378 bytes in 228 blocks ==14459== possibly lost: 550,082 bytes in 51 blocks ==14459== still reachable: 832,938 bytes in 840 blocks ==14459== suppressed: 0 bytes in 0 blocks ==14459== Reachable blocks (those to which a pointer was found) are not shown. ==14459== To see them, rerun with: --leak-check=full --show-leak-kinds=all
I checked my code and no leak found. The leak may be in esl.c.
2013/11/13 Anthony Minessale <anthony.minessale@gmail.com>
You are showing valgrind traces of ESL client code but now you are saying your FS process has memory growth not the ESL app? You should be doing this on jira and you should have more details about the env. This mailing list is not for bug reports.
I did push a patch for what I think might be the problem you showed on esl lib.
FS can use up to a gig of memory without giving it back. To make sure you have a real memory leak you need to have it never peak but continue to grow.
On Tue, Nov 12, 2013 at 11:17 AM, Dmitry Sytchev <kbdfck@gmail.com>wrote:
I can confirm significant Freeswitch process memory growth over time when ESL is used intensively, from 100MB in normal case to 700-900MB after few hours of high esl load, though I have no valgrind check results, as I can't do tests at production server. With less than 300 channels in peak we have 900MB FS process size, never shrinking back :(
2013/11/12 gggyniidt <gggyniidt@gmail.com>
Hello Using ESL found a memory leak problem valgrind --tool=memcheck --leak-check=full ./myprog output as follows
==6872== 29,986 (88 direct, 29,898 indirect) bytes in 1 blocks are definitely lost in loss record 401 of 408 ==6872== at 0x4A06AAA: malloc (vg_replace_malloc.c:291) ==6872== by 0x413941: esl_event_create_subclass (esl_event.c:179)
==6872== by 0x410DC3: esl_recv_event (esl.c:1245)
==6872== by 0x411675: esl_send_recv_timed (esl.c:1517)
==6872== by 0x411C1B: esl_execute (esl.c:555)
==6872== by 0x408758: CRBT::Route() (CRBT.cpp:42)
==6872== by 0x40A02F: DPL::route() (DPL.cpp:234)
==6872== by 0x40FCA8: mycallback(int, int, sockaddr_in*) (listener.cpp:47) ==6872== by 0x41032F: client_thread (esl.c:654)
==6872== by 0x413FC7: thread_launch (esl_threadmutex.c:74)
==6872== by 0x34CAA07850: start_thread (in /lib64/ libpthread-2.12.so)
==6872== by 0x34CA6E890C: clone (in /lib64/libc-2.12.so)
==6872== LEAK SUMMARY: ==6872== definitely lost: 88 bytes in 1 blocks ==6872== indirectly lost: 29,898 bytes in 76 blocks ==6872== possibly lost: 550,082 bytes in 51 blocks ==6872== still reachable: 832,937 bytes in 840 blocks ==6872== suppressed: 0 bytes in 0 blocks ==6872== Reachable blocks (those to which a pointer was found) are not shown. ==6872== To see them, rerun with: --leak-check=full --show-leak-kinds=all
Part of my code is as follows
bool CRBT::Route() { string data = "ringback=file_string://"; data.append(FileName);
esl_execute(handle, "set", data.c_str(), NULL); return true; }
The OS version is Centos x86_64 6.3 The Freeswitch version is 1.2.13
Thanks for your help.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Best regards,
Dmitry Sytchev, IT Engineer
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Best regards,
Dmitry Sytchev, IT Engineer
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900
Sorry,I reply this late. In these days I did some test for esl client app.My code like this: #include <stdio.h> #include <stdlib.h> #include <esl.h> static void mycallback(esl_socket_t server_sock, esl_socket_t client_sock, struct sockaddr_in *addr) { esl_handle_t handle = {{0}}; int done = 0; esl_status_t status; time_t exp = 0; esl_attach_handle(&handle, client_sock, addr); esl_log(ESL_LOG_INFO, "Connected! %d\n", handle.sock); esl_filter(&handle, "unique-id", esl_event_get_header(handle.info_event, "caller-unique-id")); esl_events(&handle, ESL_EVENT_TYPE_PLAIN, "SESSION_HEARTBEAT CHANNEL_ANSWER CHANNEL_ORIGINATE CHANNEL_PROGRESS CHANNEL_HANGUP " "CHANNEL_BRIDGE CHANNEL_UNBRIDGE CHANNEL_OUTGOING CHANNEL_EXECUTE CHANNEL_EXECUTE_COMPLETE DTMF CUSTOM conference::maintenance"); esl_send_recv(&handle, "linger"); esl_execute(&handle, "set", "recfile=/usr/local/rec/test.gsm", NULL); esl_send_recv(&handle,"api sofia status profile internal reg 6100010030100\n\n"); char *b=esl_event_get_body(handle.last_sr_event); printf("%s\r\n", b); esl_send_recv(&handle,"api sofia status profile internal reg 6100010030200\n\n"); b=esl_event_get_body(handle.last_sr_event); printf("%s\r\n", b); while((status = esl_recv_timed(&handle, 1000)) != ESL_FAIL) { if (done) { if (time(NULL) >= exp) { break; } } else if (status == ESL_SUCCESS) { const char *type = esl_event_get_header(handle.last_event, "content-type"); if (type && !strcasecmp(type, "text/disconnect-notice")) { const char *dispo = esl_event_get_header(handle.last_event, "content-disposition"); esl_log(ESL_LOG_INFO, "Got a disconnection notice dispostion: [%s]\n", dispo ? dispo : ""); if (dispo && !strcmp(dispo, "linger")) { done = 1; esl_log(ESL_LOG_INFO, "Waiting 5 seconds for any remaining events.\n"); exp = time(NULL) + 5; } } } } esl_log(ESL_LOG_INFO, "Disconnected! %d\n", handle.sock); esl_disconnect(&handle); } int main(){ esl_global_set_default_logger(0); esl_listen_threaded("localhost", 8084, mycallback, 100000); return 0; } I made 10 calls from a rtmp client to a eyeBeam client. sometimes there was no lake, but sometimes there was still a leak . The valgrind full log as follows valgrind --tool=memcheck --leak-check=full ./wtdpl ==11071== Memcheck, a memory error detector ==11071== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al. ==11071== Using Valgrind-3.9.0 and LibVEX; rerun with -h for copyright info ==11071== Command: ./wtdpl ==11071== Registrations: ================================================================================================= Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU. User: 6100010030100@192.168.1.200 Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0> Agent: eyeBeam release 1011d stamp 40820 Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2321) Host: test.u1u2u3.com IP: 192.168.1.6 Port: 2964 Auth-User: 6100010030100 Auth-Realm: 192.168.1.200 MWI-Account: 6100010030100@192.168.1.200 Total items returned: 1 ================================================================================================= Registrations: ================================================================================================= Total items returned: 0 ================================================================================================= Registrations: ================================================================================================= Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU. User: 6100010030100@192.168.1.200 Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0> Agent: eyeBeam release 1011d stamp 40820 Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2320) Host: test.u1u2u3.com IP: 192.168.1.6 Port: 2964 Auth-User: 6100010030100 Auth-Realm: 192.168.1.200 MWI-Account: 6100010030100@192.168.1.200 Total items returned: 1 ================================================================================================= Registrations: ================================================================================================= Total items returned: 0 ================================================================================================= Registrations: ================================================================================================= Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU. User: 6100010030100@192.168.1.200 Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0> Agent: eyeBeam release 1011d stamp 40820 Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2319) Host: test.u1u2u3.com IP: 192.168.1.6 Port: 2964 Auth-User: 6100010030100 Auth-Realm: 192.168.1.200 MWI-Account: 6100010030100@192.168.1.200 Total items returned: 1 ================================================================================================= Registrations: ================================================================================================= Total items returned: 0 ================================================================================================= Registrations: ================================================================================================= Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU. User: 6100010030100@192.168.1.200 Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0> Agent: eyeBeam release 1011d stamp 40820 Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2318) Host: test.u1u2u3.com IP: 192.168.1.6 Port: 2964 Auth-User: 6100010030100 Auth-Realm: 192.168.1.200 MWI-Account: 6100010030100@192.168.1.200 Total items returned: 1 ================================================================================================= Registrations: ================================================================================================= Total items returned: 0 ================================================================================================= Registrations: ================================================================================================= Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU. User: 6100010030100@192.168.1.200 Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0> Agent: eyeBeam release 1011d stamp 40820 Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2316) Host: test.u1u2u3.com IP: 192.168.1.6 Port: 2964 Auth-User: 6100010030100 Auth-Realm: 192.168.1.200 MWI-Account: 6100010030100@192.168.1.200 Total items returned: 1 ================================================================================================= Registrations: ================================================================================================= Total items returned: 0 ================================================================================================= Registrations: ================================================================================================= Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU. User: 6100010030100@192.168.1.200 Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0> Agent: eyeBeam release 1011d stamp 40820 Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2315) Host: test.u1u2u3.com IP: 192.168.1.6 Port: 2964 Auth-User: 6100010030100 Auth-Realm: 192.168.1.200 MWI-Account: 6100010030100@192.168.1.200 Total items returned: 1 ================================================================================================= Registrations: ================================================================================================= Total items returned: 0 ================================================================================================= Registrations: ================================================================================================= Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU. User: 6100010030100@192.168.1.200 Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0> Agent: eyeBeam release 1011d stamp 40820 Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2314) Host: test.u1u2u3.com IP: 192.168.1.6 Port: 2964 Auth-User: 6100010030100 Auth-Realm: 192.168.1.200 MWI-Account: 6100010030100@192.168.1.200 Total items returned: 1 ================================================================================================= Registrations: ================================================================================================= Total items returned: 0 ================================================================================================= Registrations: ================================================================================================= Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU. User: 6100010030100@192.168.1.200 Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0> Agent: eyeBeam release 1011d stamp 40820 Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2313) Host: test.u1u2u3.com IP: 192.168.1.6 Port: 2964 Auth-User: 6100010030100 Auth-Realm: 192.168.1.200 MWI-Account: 6100010030100@192.168.1.200 Total items returned: 1 ================================================================================================= Registrations: ================================================================================================= Total items returned: 0 ================================================================================================= Registrations: ================================================================================================= Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU. User: 6100010030100@192.168.1.200 Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0> Agent: eyeBeam release 1011d stamp 40820 Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2312) Host: test.u1u2u3.com IP: 192.168.1.6 Port: 2964 Auth-User: 6100010030100 Auth-Realm: 192.168.1.200 MWI-Account: 6100010030100@192.168.1.200 Total items returned: 1 ================================================================================================= Registrations: ================================================================================================= Total items returned: 0 ================================================================================================= Registrations: ================================================================================================= Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU. User: 6100010030100@192.168.1.200 Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0> Agent: eyeBeam release 1011d stamp 40820 Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2310) Host: test.u1u2u3.com IP: 192.168.1.6 Port: 2964 Auth-User: 6100010030100 Auth-Realm: 192.168.1.200 MWI-Account: 6100010030100@192.168.1.200 Total items returned: 1 ================================================================================================= Registrations: ================================================================================================= Total items returned: 0 ================================================================================================= ^C==11071== ==11071== HEAP SUMMARY: ==11071== in use at exit: 26,309,844 bytes in 2,639 blocks ==11071== total heap usage: 14,394 allocs, 11,755 frees, 66,216,694 bytes allocated ==11071== ==11071== 1,152 bytes in 4 blocks are possibly lost in loss record 41 of 51 ==11071== at 0x4A0580F: calloc (vg_replace_malloc.c:618) ==11071== by 0x34C9E11892: _dl_allocate_tls (in /lib64/ld-2.12.so) ==11071== by 0x34CAA07068: pthread_create@@GLIBC_2.2.5 (in /lib64/ libpthread-2.12.so) ==11071== by 0x405CFC: esl_thread_create_detached_ex (esl_threadmutex.c:118) ==11071== by 0x402410: esl_listen_threaded (esl.c:771) ==11071== by 0x401CD6: main (wtdpl.cpp:58) ==11071== ==11071== 4,008 (88 direct, 3,920 indirect) bytes in 1 blocks are definitely lost in loss record 43 of 51 ==11071== at 0x4A06AAA: malloc (vg_replace_malloc.c:291) ==11071== by 0x4056B1: esl_event_create_subclass (esl_event.c:179) ==11071== by 0x402B33: esl_recv_event (esl.c:1245) ==11071== by 0x4033E5: esl_send_recv_timed (esl.c:1517) ==11071== by 0x401AE1: mycallback(int, int, sockaddr_in*) (wtdpl.cpp:28) ==11071== by 0x40209F: client_thread (esl.c:654) ==11071== by 0x405D37: thread_launch (esl_threadmutex.c:74) ==11071== by 0x34CAA07850: start_thread (in /lib64/libpthread-2.12.so) ==11071== by 0x34CA6E890C: clone (in /lib64/libc-2.12.so) ==11071== ==11071== LEAK SUMMARY: ==11071== definitely lost: 88 bytes in 1 blocks ==11071== indirectly lost: 3,920 bytes in 10 blocks ==11071== possibly lost: 1,152 bytes in 4 blocks ==11071== still reachable: 26,304,684 bytes in 2,624 blocks ==11071== suppressed: 0 bytes in 0 blocks ==11071== Reachable blocks (those to which a pointer was found) are not shown. ==11071== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==11071== ==11071== For counts of detected and suppressed errors, rerun with: -v ==11071== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 4 from 4) Looking forward to your help 2013/11/14 Anthony Minessale <anthony.minessale@gmail.com>
I dug into ESL for a while.
I ran into some issues that I corrected but the only thing that I can see that would show up as a memory leak looking like what you show there is if you are doing a forking server and you are calling fork after you create or attach the handle.
Make sure you fork at the beginning or you will duplicate the handle and all the memory its using and it will never be destroyed.
If you do "make" in esl dir you will have a default app "testserver_fork" you can run and point to it with socket app to port 8040. Running it on valgrind with full leak detection gives me.
==11807== Memcheck, a memory error detector ==11807== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al. ==11807== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info ==11807== Command: ./testserver ==11807== Parent PID: 28177 ==11807== ==11820== ==11820== HEAP SUMMARY: ==11820== in use at exit: 0 bytes in 0 blocks ==11820== total heap usage: 5,555 allocs, 5,555 frees, 7,293,931 bytes allocated ==11820== ==11820== All heap blocks were freed -- no leaks are possible ==11820== ==11820== For counts of detected and suppressed errors, rerun with: -v ==11820== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 4 from 4) ==11807== ==11807== HEAP SUMMARY: ==11807== in use at exit: 0 bytes in 0 blocks ==11807== total heap usage: 0 allocs, 0 frees, 0 bytes allocated ==11807== ==11807== All heap blocks were freed -- no leaks are possible ==11807== ==11807== For counts of detected and suppressed errors, rerun with: -v ==11807== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 4 from 4)
On Wed, Nov 13, 2013 at 7:21 AM, Dmitry Sytchev <kbdfck@gmail.com> wrote:
Please create a JIRA ticket with your information!
2013/11/13 gggyniidt <gggyniidt@gmail.com>
Thanks for Dmitry and Anthony's help, But my problem is not like Dmitry said.
My program is a esl client. The leak is from esl client, not freeswitch.
I add the Anthony's patch code into esl.c,but it doesn't work. Each call will cause the 20-40KB of memory growth.
I make 3 calls, the valgrind outputs like this
==14459== 90,642 (264 direct, 90,378 indirect) bytes in 3 blocks are definitely lost in loss record 403 of 408 ==14459== at 0x4A06AAA: malloc (vg_replace_malloc.c:291)
==14459== by 0x414524: esl_event_create_subclass (esl_event.c:179)
==14459== by 0x412054: esl_recv_event (esl.c:1245)
==14459== by 0x411C38: esl_recv_event_timed (esl.c:1137)
==14459== by 0x412E22: esl_send_recv_timed (esl.c:1521)
==14459== by 0x410C06: esl_execute (esl.c:555)
==14459== by 0x4087A8: CRBT::Route() (CRBT.cpp:42)
==14459== by 0x409FE6: DPL::route() (DPL.cpp:286)
==14459== by 0x40FBE0: mycallback(int, int, sockaddr_in*) (listener.cpp:47) ==14459== by 0x410FED: client_thread (esl.c:654)
==14459== by 0x41A334: thread_launch (esl_threadmutex.c:74)
==14459== by 0x34CAA07850: start_thread (in /lib64/libpthread-2.12.so)
==14459==
==14459== LEAK SUMMARY: ==14459== definitely lost: 264 bytes in 3 blocks ==14459== indirectly lost: 90,378 bytes in 228 blocks ==14459== possibly lost: 550,082 bytes in 51 blocks ==14459== still reachable: 832,938 bytes in 840 blocks ==14459== suppressed: 0 bytes in 0 blocks ==14459== Reachable blocks (those to which a pointer was found) are not shown. ==14459== To see them, rerun with: --leak-check=full --show-leak-kinds=all
I checked my code and no leak found. The leak may be in esl.c.
2013/11/13 Anthony Minessale <anthony.minessale@gmail.com>
You are showing valgrind traces of ESL client code but now you are saying your FS process has memory growth not the ESL app? You should be doing this on jira and you should have more details about the env. This mailing list is not for bug reports.
I did push a patch for what I think might be the problem you showed on esl lib.
FS can use up to a gig of memory without giving it back. To make sure you have a real memory leak you need to have it never peak but continue to grow.
On Tue, Nov 12, 2013 at 11:17 AM, Dmitry Sytchev <kbdfck@gmail.com>wrote:
I can confirm significant Freeswitch process memory growth over time when ESL is used intensively, from 100MB in normal case to 700-900MB after few hours of high esl load, though I have no valgrind check results, as I can't do tests at production server. With less than 300 channels in peak we have 900MB FS process size, never shrinking back :(
2013/11/12 gggyniidt <gggyniidt@gmail.com>
Hello Using ESL found a memory leak problem valgrind --tool=memcheck --leak-check=full ./myprog output as follows
==6872== 29,986 (88 direct, 29,898 indirect) bytes in 1 blocks are definitely lost in loss record 401 of 408 ==6872== at 0x4A06AAA: malloc (vg_replace_malloc.c:291) ==6872== by 0x413941: esl_event_create_subclass (esl_event.c:179)
==6872== by 0x410DC3: esl_recv_event (esl.c:1245)
==6872== by 0x411675: esl_send_recv_timed (esl.c:1517)
==6872== by 0x411C1B: esl_execute (esl.c:555)
==6872== by 0x408758: CRBT::Route() (CRBT.cpp:42)
==6872== by 0x40A02F: DPL::route() (DPL.cpp:234)
==6872== by 0x40FCA8: mycallback(int, int, sockaddr_in*) (listener.cpp:47) ==6872== by 0x41032F: client_thread (esl.c:654)
==6872== by 0x413FC7: thread_launch (esl_threadmutex.c:74)
==6872== by 0x34CAA07850: start_thread (in /lib64/ libpthread-2.12.so)
==6872== by 0x34CA6E890C: clone (in /lib64/libc-2.12.so)
==6872== LEAK SUMMARY: ==6872== definitely lost: 88 bytes in 1 blocks ==6872== indirectly lost: 29,898 bytes in 76 blocks ==6872== possibly lost: 550,082 bytes in 51 blocks ==6872== still reachable: 832,937 bytes in 840 blocks ==6872== suppressed: 0 bytes in 0 blocks ==6872== Reachable blocks (those to which a pointer was found) are not shown. ==6872== To see them, rerun with: --leak-check=full --show-leak-kinds=all
Part of my code is as follows
bool CRBT::Route() { string data = "ringback=file_string://"; data.append(FileName);
esl_execute(handle, "set", data.c_str(), NULL); return true; }
The OS version is Centos x86_64 6.3 The Freeswitch version is 1.2.13
Thanks for your help.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Best regards,
Dmitry Sytchev, IT Engineer
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Best regards,
Dmitry Sytchev, IT Engineer
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
I've explained it all already. Your app needs to shutdown cleanly and close everything to know for sure. Please stop reporting bugs here. http://jira.freeswitch.org is specifically for issues. On Nov 15, 2013 10:13 PM, "gggyniidt" <gggyniidt@gmail.com> wrote:
Sorry,I reply this late. In these days I did some test for esl client app.My code like this:
#include <stdio.h> #include <stdlib.h> #include <esl.h>
static void mycallback(esl_socket_t server_sock, esl_socket_t client_sock, struct sockaddr_in *addr) { esl_handle_t handle = {{0}}; int done = 0; esl_status_t status; time_t exp = 0;
esl_attach_handle(&handle, client_sock, addr);
esl_log(ESL_LOG_INFO, "Connected! %d\n", handle.sock);
esl_filter(&handle, "unique-id", esl_event_get_header(handle.info_event, "caller-unique-id")); esl_events(&handle, ESL_EVENT_TYPE_PLAIN, "SESSION_HEARTBEAT CHANNEL_ANSWER CHANNEL_ORIGINATE CHANNEL_PROGRESS CHANNEL_HANGUP " "CHANNEL_BRIDGE CHANNEL_UNBRIDGE CHANNEL_OUTGOING CHANNEL_EXECUTE CHANNEL_EXECUTE_COMPLETE DTMF CUSTOM conference::maintenance");
esl_send_recv(&handle, "linger");
esl_execute(&handle, "set", "recfile=/usr/local/rec/test.gsm", NULL); esl_send_recv(&handle,"api sofia status profile internal reg 6100010030100\n\n"); char *b=esl_event_get_body(handle.last_sr_event); printf("%s\r\n", b);
esl_send_recv(&handle,"api sofia status profile internal reg 6100010030200\n\n"); b=esl_event_get_body(handle.last_sr_event); printf("%s\r\n", b);
while((status = esl_recv_timed(&handle, 1000)) != ESL_FAIL) { if (done) { if (time(NULL) >= exp) { break; } } else if (status == ESL_SUCCESS) { const char *type = esl_event_get_header(handle.last_event, "content-type"); if (type && !strcasecmp(type, "text/disconnect-notice")) { const char *dispo = esl_event_get_header(handle.last_event, "content-disposition"); esl_log(ESL_LOG_INFO, "Got a disconnection notice dispostion: [%s]\n", dispo ? dispo : ""); if (dispo && !strcmp(dispo, "linger")) { done = 1; esl_log(ESL_LOG_INFO, "Waiting 5 seconds for any remaining events.\n"); exp = time(NULL) + 5; } } } }
esl_log(ESL_LOG_INFO, "Disconnected! %d\n", handle.sock); esl_disconnect(&handle); }
int main(){ esl_global_set_default_logger(0); esl_listen_threaded("localhost", 8084, mycallback, 100000); return 0; }
I made 10 calls from a rtmp client to a eyeBeam client. sometimes there was no lake, but sometimes there was still a leak . The valgrind full log as follows
valgrind --tool=memcheck --leak-check=full ./wtdpl ==11071== Memcheck, a memory error detector
==11071== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==11071== Using Valgrind-3.9.0 and LibVEX; rerun with -h for copyright info ==11071== Command: ./wtdpl
==11071==
Registrations:
================================================================================================= Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU.
User: 6100010030100@192.168.1.200
Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0>
Agent: eyeBeam release 1011d stamp 40820
Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2321) Host: test.u1u2u3.com
IP: 192.168.1.6
Port: 2964
Auth-User: 6100010030100
Auth-Realm: 192.168.1.200
MWI-Account: 6100010030100@192.168.1.200
Total items returned: 1
=================================================================================================
Registrations:
================================================================================================= Total items returned: 0
=================================================================================================
Registrations:
================================================================================================= Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU.
User: 6100010030100@192.168.1.200
Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0>
Agent: eyeBeam release 1011d stamp 40820
Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2320) Host: test.u1u2u3.com
IP: 192.168.1.6
Port: 2964
Auth-User: 6100010030100
Auth-Realm: 192.168.1.200
MWI-Account: 6100010030100@192.168.1.200
Total items returned: 1
=================================================================================================
Registrations:
================================================================================================= Total items returned: 0
=================================================================================================
Registrations:
================================================================================================= Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU.
User: 6100010030100@192.168.1.200
Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0>
Agent: eyeBeam release 1011d stamp 40820
Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2319) Host: test.u1u2u3.com
IP: 192.168.1.6
Port: 2964
Auth-User: 6100010030100
Auth-Realm: 192.168.1.200
MWI-Account: 6100010030100@192.168.1.200
Total items returned: 1
=================================================================================================
Registrations:
================================================================================================= Total items returned: 0
=================================================================================================
Registrations:
================================================================================================= Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU.
User: 6100010030100@192.168.1.200
Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0>
Agent: eyeBeam release 1011d stamp 40820
Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2318) Host: test.u1u2u3.com
IP: 192.168.1.6
Port: 2964
Auth-User: 6100010030100
Auth-Realm: 192.168.1.200
MWI-Account: 6100010030100@192.168.1.200
Total items returned: 1
=================================================================================================
Registrations:
================================================================================================= Total items returned: 0
=================================================================================================
Registrations:
================================================================================================= Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU.
User: 6100010030100@192.168.1.200
Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0>
Agent: eyeBeam release 1011d stamp 40820
Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2316) Host: test.u1u2u3.com
IP: 192.168.1.6
Port: 2964
Auth-User: 6100010030100
Auth-Realm: 192.168.1.200
MWI-Account: 6100010030100@192.168.1.200
Total items returned: 1
=================================================================================================
Registrations:
================================================================================================= Total items returned: 0
=================================================================================================
Registrations:
================================================================================================= Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU.
User: 6100010030100@192.168.1.200
Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0>
Agent: eyeBeam release 1011d stamp 40820
Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2315) Host: test.u1u2u3.com
IP: 192.168.1.6
Port: 2964
Auth-User: 6100010030100
Auth-Realm: 192.168.1.200
MWI-Account: 6100010030100@192.168.1.200
Total items returned: 1
=================================================================================================
Registrations:
================================================================================================= Total items returned: 0
=================================================================================================
Registrations:
================================================================================================= Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU.
User: 6100010030100@192.168.1.200
Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0>
Agent: eyeBeam release 1011d stamp 40820
Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2314) Host: test.u1u2u3.com
IP: 192.168.1.6
Port: 2964
Auth-User: 6100010030100
Auth-Realm: 192.168.1.200
MWI-Account: 6100010030100@192.168.1.200
Total items returned: 1
=================================================================================================
Registrations:
================================================================================================= Total items returned: 0
=================================================================================================
Registrations:
================================================================================================= Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU.
User: 6100010030100@192.168.1.200
Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0>
Agent: eyeBeam release 1011d stamp 40820
Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2313) Host: test.u1u2u3.com
IP: 192.168.1.6
Port: 2964
Auth-User: 6100010030100
Auth-Realm: 192.168.1.200
MWI-Account: 6100010030100@192.168.1.200
Total items returned: 1
=================================================================================================
Registrations:
================================================================================================= Total items returned: 0
=================================================================================================
Registrations:
================================================================================================= Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU.
User: 6100010030100@192.168.1.200
Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0>
Agent: eyeBeam release 1011d stamp 40820
Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2312) Host: test.u1u2u3.com
IP: 192.168.1.6
Port: 2964
Auth-User: 6100010030100
Auth-Realm: 192.168.1.200
MWI-Account: 6100010030100@192.168.1.200
Total items returned: 1
=================================================================================================
Registrations:
================================================================================================= Total items returned: 0
=================================================================================================
Registrations:
================================================================================================= Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU.
User: 6100010030100@192.168.1.200
Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0>
Agent: eyeBeam release 1011d stamp 40820
Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2310) Host: test.u1u2u3.com
IP: 192.168.1.6
Port: 2964
Auth-User: 6100010030100
Auth-Realm: 192.168.1.200
MWI-Account: 6100010030100@192.168.1.200
Total items returned: 1
=================================================================================================
Registrations:
================================================================================================= Total items returned: 0
=================================================================================================
^C==11071== ==11071== HEAP SUMMARY: ==11071== in use at exit: 26,309,844 bytes in 2,639 blocks ==11071== total heap usage: 14,394 allocs, 11,755 frees, 66,216,694 bytes allocated ==11071== ==11071== 1,152 bytes in 4 blocks are possibly lost in loss record 41 of 51 ==11071== at 0x4A0580F: calloc (vg_replace_malloc.c:618) ==11071== by 0x34C9E11892: _dl_allocate_tls (in /lib64/ld-2.12.so) ==11071== by 0x34CAA07068: pthread_create@@GLIBC_2.2.5 (in /lib64/ libpthread-2.12.so) ==11071== by 0x405CFC: esl_thread_create_detached_ex (esl_threadmutex.c:118) ==11071== by 0x402410: esl_listen_threaded (esl.c:771) ==11071== by 0x401CD6: main (wtdpl.cpp:58) ==11071== ==11071== 4,008 (88 direct, 3,920 indirect) bytes in 1 blocks are definitely lost in loss record 43 of 51 ==11071== at 0x4A06AAA: malloc (vg_replace_malloc.c:291) ==11071== by 0x4056B1: esl_event_create_subclass (esl_event.c:179) ==11071== by 0x402B33: esl_recv_event (esl.c:1245) ==11071== by 0x4033E5: esl_send_recv_timed (esl.c:1517) ==11071== by 0x401AE1: mycallback(int, int, sockaddr_in*) (wtdpl.cpp:28) ==11071== by 0x40209F: client_thread (esl.c:654) ==11071== by 0x405D37: thread_launch (esl_threadmutex.c:74) ==11071== by 0x34CAA07850: start_thread (in /lib64/libpthread-2.12.so) ==11071== by 0x34CA6E890C: clone (in /lib64/libc-2.12.so) ==11071== ==11071== LEAK SUMMARY: ==11071== definitely lost: 88 bytes in 1 blocks ==11071== indirectly lost: 3,920 bytes in 10 blocks ==11071== possibly lost: 1,152 bytes in 4 blocks ==11071== still reachable: 26,304,684 bytes in 2,624 blocks ==11071== suppressed: 0 bytes in 0 blocks ==11071== Reachable blocks (those to which a pointer was found) are not shown. ==11071== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==11071== ==11071== For counts of detected and suppressed errors, rerun with: -v ==11071== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 4 from 4)
Looking forward to your help
2013/11/14 Anthony Minessale <anthony.minessale@gmail.com>
I dug into ESL for a while.
I ran into some issues that I corrected but the only thing that I can see that would show up as a memory leak looking like what you show there is if you are doing a forking server and you are calling fork after you create or attach the handle.
Make sure you fork at the beginning or you will duplicate the handle and all the memory its using and it will never be destroyed.
If you do "make" in esl dir you will have a default app "testserver_fork" you can run and point to it with socket app to port 8040. Running it on valgrind with full leak detection gives me.
==11807== Memcheck, a memory error detector ==11807== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al. ==11807== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info ==11807== Command: ./testserver ==11807== Parent PID: 28177 ==11807== ==11820== ==11820== HEAP SUMMARY: ==11820== in use at exit: 0 bytes in 0 blocks ==11820== total heap usage: 5,555 allocs, 5,555 frees, 7,293,931 bytes allocated ==11820== ==11820== All heap blocks were freed -- no leaks are possible ==11820== ==11820== For counts of detected and suppressed errors, rerun with: -v ==11820== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 4 from 4) ==11807== ==11807== HEAP SUMMARY: ==11807== in use at exit: 0 bytes in 0 blocks ==11807== total heap usage: 0 allocs, 0 frees, 0 bytes allocated ==11807== ==11807== All heap blocks were freed -- no leaks are possible ==11807== ==11807== For counts of detected and suppressed errors, rerun with: -v ==11807== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 4 from 4)
On Wed, Nov 13, 2013 at 7:21 AM, Dmitry Sytchev <kbdfck@gmail.com> wrote:
Please create a JIRA ticket with your information!
2013/11/13 gggyniidt <gggyniidt@gmail.com>
Thanks for Dmitry and Anthony's help, But my problem is not like Dmitry said.
My program is a esl client. The leak is from esl client, not freeswitch.
I add the Anthony's patch code into esl.c,but it doesn't work. Each call will cause the 20-40KB of memory growth.
I make 3 calls, the valgrind outputs like this
==14459== 90,642 (264 direct, 90,378 indirect) bytes in 3 blocks are definitely lost in loss record 403 of 408 ==14459== at 0x4A06AAA: malloc (vg_replace_malloc.c:291)
==14459== by 0x414524: esl_event_create_subclass (esl_event.c:179)
==14459== by 0x412054: esl_recv_event (esl.c:1245)
==14459== by 0x411C38: esl_recv_event_timed (esl.c:1137)
==14459== by 0x412E22: esl_send_recv_timed (esl.c:1521)
==14459== by 0x410C06: esl_execute (esl.c:555)
==14459== by 0x4087A8: CRBT::Route() (CRBT.cpp:42)
==14459== by 0x409FE6: DPL::route() (DPL.cpp:286)
==14459== by 0x40FBE0: mycallback(int, int, sockaddr_in*) (listener.cpp:47) ==14459== by 0x410FED: client_thread (esl.c:654)
==14459== by 0x41A334: thread_launch (esl_threadmutex.c:74)
==14459== by 0x34CAA07850: start_thread (in /lib64/ libpthread-2.12.so) ==14459==
==14459== LEAK SUMMARY: ==14459== definitely lost: 264 bytes in 3 blocks ==14459== indirectly lost: 90,378 bytes in 228 blocks ==14459== possibly lost: 550,082 bytes in 51 blocks ==14459== still reachable: 832,938 bytes in 840 blocks ==14459== suppressed: 0 bytes in 0 blocks ==14459== Reachable blocks (those to which a pointer was found) are not shown. ==14459== To see them, rerun with: --leak-check=full --show-leak-kinds=all
I checked my code and no leak found. The leak may be in esl.c.
2013/11/13 Anthony Minessale <anthony.minessale@gmail.com>
You are showing valgrind traces of ESL client code but now you are saying your FS process has memory growth not the ESL app? You should be doing this on jira and you should have more details about the env. This mailing list is not for bug reports.
I did push a patch for what I think might be the problem you showed on esl lib.
FS can use up to a gig of memory without giving it back. To make sure you have a real memory leak you need to have it never peak but continue to grow.
On Tue, Nov 12, 2013 at 11:17 AM, Dmitry Sytchev <kbdfck@gmail.com>wrote:
I can confirm significant Freeswitch process memory growth over time when ESL is used intensively, from 100MB in normal case to 700-900MB after few hours of high esl load, though I have no valgrind check results, as I can't do tests at production server. With less than 300 channels in peak we have 900MB FS process size, never shrinking back :(
2013/11/12 gggyniidt <gggyniidt@gmail.com>
> Hello > Using ESL found a memory leak problem > valgrind --tool=memcheck --leak-check=full ./myprog > output as follows > > ==6872== 29,986 (88 direct, 29,898 indirect) bytes in 1 blocks are > definitely lost in loss record 401 of 408 > ==6872== at 0x4A06AAA: malloc (vg_replace_malloc.c:291) > ==6872== by 0x413941: esl_event_create_subclass (esl_event.c:179) > > ==6872== by 0x410DC3: esl_recv_event (esl.c:1245) > > ==6872== by 0x411675: esl_send_recv_timed (esl.c:1517) > > ==6872== by 0x411C1B: esl_execute (esl.c:555) > > ==6872== by 0x408758: CRBT::Route() (CRBT.cpp:42) > > ==6872== by 0x40A02F: DPL::route() (DPL.cpp:234) > > ==6872== by 0x40FCA8: mycallback(int, int, sockaddr_in*) > (listener.cpp:47) > ==6872== by 0x41032F: client_thread (esl.c:654) > > ==6872== by 0x413FC7: thread_launch (esl_threadmutex.c:74) > > ==6872== by 0x34CAA07850: start_thread (in /lib64/ > libpthread-2.12.so) > > ==6872== by 0x34CA6E890C: clone (in /lib64/libc-2.12.so) > > > ==6872== LEAK SUMMARY: > ==6872== definitely lost: 88 bytes in 1 blocks > ==6872== indirectly lost: 29,898 bytes in 76 blocks > ==6872== possibly lost: 550,082 bytes in 51 blocks > ==6872== still reachable: 832,937 bytes in 840 blocks > ==6872== suppressed: 0 bytes in 0 blocks > ==6872== Reachable blocks (those to which a pointer was found) are > not shown. > ==6872== To see them, rerun with: --leak-check=full > --show-leak-kinds=all > > > > Part of my code is as follows > > bool CRBT::Route() { > string data = "ringback=file_string://"; > data.append(FileName); > > esl_execute(handle, "set", data.c_str(), NULL); > return true; > } > > The OS version is Centos x86_64 6.3 > The Freeswitch version is 1.2.13 > > Thanks for your help. > > > _________________________________________________________________________ > Professional FreeSWITCH Consulting Services: > consulting@freeswitch.org > http://www.freeswitchsolutions.com > > FreeSWITCH-powered IP PBX: The CudaTel Communication Server > http://www.cudatel.com > > Official FreeSWITCH Sites > http://www.freeswitch.org > http://wiki.freeswitch.org > http://www.cluecon.com > > FreeSWITCH-users mailing list > FreeSWITCH-users@lists.freeswitch.org > http://lists.freeswitch.org/mailman/listinfo/freeswitch-users > UNSUBSCRIBE: > http://lists.freeswitch.org/mailman/options/freeswitch-users > http://www.freeswitch.org > >
-- Best regards,
Dmitry Sytchev, IT Engineer
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Best regards,
Dmitry Sytchev, IT Engineer
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
participants (3)
-
Anthony Minessale
-
Dmitry Sytchev
-
gggyniidt