I've explained it all already.
Your app needs to shutdown cleanly and close everything to know for sure.
Please stop reporting bugs here.
http://jira.freeswitch.org is specifically for issues.
Sorry,I reply this late.In these days I did some test for esl client app.My code like this:#include <stdio.h>#include <stdlib.h>#include <esl.h>static void mycallback(esl_socket_t server_sock, esl_socket_t client_sock, struct sockaddr_in *addr){esl_handle_t handle = {{0}};int done = 0;esl_status_t status;time_t exp = 0;esl_attach_handle(&handle, client_sock, addr);esl_log(ESL_LOG_INFO, "Connected! %d\n", handle.sock);esl_filter(&handle, "unique-id", esl_event_get_header(handle.info_event, "caller-unique-id"));esl_events(&handle, ESL_EVENT_TYPE_PLAIN, "SESSION_HEARTBEAT CHANNEL_ANSWER CHANNEL_ORIGINATE CHANNEL_PROGRESS CHANNEL_HANGUP ""CHANNEL_BRIDGE CHANNEL_UNBRIDGE CHANNEL_OUTGOING CHANNEL_EXECUTE CHANNEL_EXECUTE_COMPLETE DTMF CUSTOM conference::maintenance");esl_send_recv(&handle, "linger");esl_execute(&handle, "set", "recfile=/usr/local/rec/test.gsm", NULL);esl_send_recv(&handle,"api sofia status profile internal reg 6100010030100\n\n");char *b=esl_event_get_body(handle.last_sr_event);printf("%s\r\n", b);esl_send_recv(&handle,"api sofia status profile internal reg 6100010030200\n\n");b=esl_event_get_body(handle.last_sr_event);printf("%s\r\n", b);while((status = esl_recv_timed(&handle, 1000)) != ESL_FAIL) {if (done) {if (time(NULL) >= exp) {break;}} else if (status == ESL_SUCCESS) {const char *type = esl_event_get_header(handle.last_event, "content-type");if (type && !strcasecmp(type, "text/disconnect-notice")) {const char *dispo = esl_event_get_header(handle.last_event, "content-disposition");esl_log(ESL_LOG_INFO, "Got a disconnection notice dispostion: [%s]\n", dispo ? dispo : "");if (dispo && !strcmp(dispo, "linger")) {done = 1;esl_log(ESL_LOG_INFO, "Waiting 5 seconds for any remaining events.\n");exp = time(NULL) + 5;}}}}esl_log(ESL_LOG_INFO, "Disconnected! %d\n", handle.sock);esl_disconnect(&handle);}int main(){esl_global_set_default_logger(0);esl_listen_threaded("localhost", 8084, mycallback, 100000);return 0;}I made 10 calls from a rtmp client to a eyeBeam client. sometimes there was no lake, but sometimes there was still a leak .The valgrind full log as followsvalgrind --tool=memcheck --leak-check=full ./wtdpl==11071== Memcheck, a memory error detector==11071== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.==11071== Using Valgrind-3.9.0 and LibVEX; rerun with -h for copyright info==11071== Command: ./wtdpl==11071==Registrations:=================================================================================================Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU.Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0>Agent: eyeBeam release 1011d stamp 40820Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2321)Host: test.u1u2u3.comIP: 192.168.1.6Port: 2964Auth-User: 6100010030100Auth-Realm: 192.168.1.200MWI-Account: 6100010030100@192.168.1.200Total items returned: 1=================================================================================================Registrations:=================================================================================================Total items returned: 0=================================================================================================Registrations:=================================================================================================Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU.Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0>Agent: eyeBeam release 1011d stamp 40820Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2320)Host: test.u1u2u3.comIP: 192.168.1.6Port: 2964Auth-User: 6100010030100Auth-Realm: 192.168.1.200MWI-Account: 6100010030100@192.168.1.200Total items returned: 1=================================================================================================Registrations:=================================================================================================Total items returned: 0=================================================================================================Registrations:=================================================================================================Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU.Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0>Agent: eyeBeam release 1011d stamp 40820Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2319)Host: test.u1u2u3.comIP: 192.168.1.6Port: 2964Auth-User: 6100010030100Auth-Realm: 192.168.1.200MWI-Account: 6100010030100@192.168.1.200Total items returned: 1=================================================================================================Registrations:=================================================================================================Total items returned: 0=================================================================================================Registrations:=================================================================================================Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU.Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0>Agent: eyeBeam release 1011d stamp 40820Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2318)Host: test.u1u2u3.comIP: 192.168.1.6Port: 2964Auth-User: 6100010030100Auth-Realm: 192.168.1.200MWI-Account: 6100010030100@192.168.1.200Total items returned: 1=================================================================================================Registrations:=================================================================================================Total items returned: 0=================================================================================================Registrations:=================================================================================================Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU.Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0>Agent: eyeBeam release 1011d stamp 40820Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2316)Host: test.u1u2u3.comIP: 192.168.1.6Port: 2964Auth-User: 6100010030100Auth-Realm: 192.168.1.200MWI-Account: 6100010030100@192.168.1.200Total items returned: 1=================================================================================================Registrations:=================================================================================================Total items returned: 0=================================================================================================Registrations:=================================================================================================Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU.Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0>Agent: eyeBeam release 1011d stamp 40820Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2315)Host: test.u1u2u3.comIP: 192.168.1.6Port: 2964Auth-User: 6100010030100Auth-Realm: 192.168.1.200MWI-Account: 6100010030100@192.168.1.200Total items returned: 1=================================================================================================Registrations:=================================================================================================Total items returned: 0=================================================================================================Registrations:=================================================================================================Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU.Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0>Agent: eyeBeam release 1011d stamp 40820Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2314)Host: test.u1u2u3.comIP: 192.168.1.6Port: 2964Auth-User: 6100010030100Auth-Realm: 192.168.1.200MWI-Account: 6100010030100@192.168.1.200Total items returned: 1=================================================================================================Registrations:=================================================================================================Total items returned: 0=================================================================================================Registrations:=================================================================================================Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU.Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0>Agent: eyeBeam release 1011d stamp 40820Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2313)Host: test.u1u2u3.comIP: 192.168.1.6Port: 2964Auth-User: 6100010030100Auth-Realm: 192.168.1.200MWI-Account: 6100010030100@192.168.1.200Total items returned: 1=================================================================================================Registrations:=================================================================================================Total items returned: 0=================================================================================================Registrations:=================================================================================================Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU.Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0>Agent: eyeBeam release 1011d stamp 40820Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2312)Host: test.u1u2u3.comIP: 192.168.1.6Port: 2964Auth-User: 6100010030100Auth-Realm: 192.168.1.200MWI-Account: 6100010030100@192.168.1.200Total items returned: 1=================================================================================================Registrations:=================================================================================================Total items returned: 0=================================================================================================Registrations:=================================================================================================Call-ID: Y2VkZjY4MzY4OTFmZDMxZjJkY2FkNmQ1OWZkNjg5YTU.Contact: "abc" <sip:6100010030100@192.168.1.6:2964;rinstance=239a1e88dd886ca0>Agent: eyeBeam release 1011d stamp 40820Status: Registered(UDP)(unknown) EXP(2013-11-16 12:31:53) EXPSECS(2310)Host: test.u1u2u3.comIP: 192.168.1.6Port: 2964Auth-User: 6100010030100Auth-Realm: 192.168.1.200MWI-Account: 6100010030100@192.168.1.200Total items returned: 1=================================================================================================Registrations:=================================================================================================Total items returned: 0=================================================================================================^C==11071====11071== HEAP SUMMARY:==11071== in use at exit: 26,309,844 bytes in 2,639 blocks==11071== total heap usage: 14,394 allocs, 11,755 frees, 66,216,694 bytes allocated==11071====11071== 1,152 bytes in 4 blocks are possibly lost in loss record 41 of 51==11071== at 0x4A0580F: calloc (vg_replace_malloc.c:618)==11071== by 0x34C9E11892: _dl_allocate_tls (in /lib64/ld-2.12.so)==11071== by 0x34CAA07068: pthread_create@@GLIBC_2.2.5 (in /lib64/libpthread-2.12.so)==11071== by 0x405CFC: esl_thread_create_detached_ex (esl_threadmutex.c:118)==11071== by 0x402410: esl_listen_threaded (esl.c:771)==11071== by 0x401CD6: main (wtdpl.cpp:58)==11071====11071== 4,008 (88 direct, 3,920 indirect) bytes in 1 blocks are definitely lost in loss record 43 of 51==11071== at 0x4A06AAA: malloc (vg_replace_malloc.c:291)==11071== by 0x4056B1: esl_event_create_subclass (esl_event.c:179)==11071== by 0x402B33: esl_recv_event (esl.c:1245)==11071== by 0x4033E5: esl_send_recv_timed (esl.c:1517)==11071== by 0x401AE1: mycallback(int, int, sockaddr_in*) (wtdpl.cpp:28)==11071== by 0x40209F: client_thread (esl.c:654)==11071== by 0x405D37: thread_launch (esl_threadmutex.c:74)==11071== by 0x34CAA07850: start_thread (in /lib64/libpthread-2.12.so)==11071== by 0x34CA6E890C: clone (in /lib64/libc-2.12.so)==11071====11071== LEAK SUMMARY:==11071== definitely lost: 88 bytes in 1 blocks==11071== indirectly lost: 3,920 bytes in 10 blocks==11071== possibly lost: 1,152 bytes in 4 blocks==11071== still reachable: 26,304,684 bytes in 2,624 blocks==11071== suppressed: 0 bytes in 0 blocks==11071== Reachable blocks (those to which a pointer was found) are not shown.==11071== To see them, rerun with: --leak-check=full --show-leak-kinds=all==11071====11071== For counts of detected and suppressed errors, rerun with: -v==11071== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 4 from 4)Looking forward to your help2013/11/14 Anthony Minessale <anthony.minessale@gmail.com>
I dug into ESL for a while.I ran into some issues that I corrected but the only thing that I can see that would show up as a memory leak looking like what you show there is if you are doing a forking server and you are calling fork after you create or attach the handle.Make sure you fork at the beginning or you will duplicate the handle and all the memory its using and it will never be destroyed.If you do "make" in esl dir you will have a default app "testserver_fork" you can run and point to it with socket app to port 8040.Running it on valgrind with full leak detection gives me.==11807== Memcheck, a memory error detector==11807== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.==11807== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info==11807== Command: ./testserver==11807== Parent PID: 28177==11807====11820====11820== HEAP SUMMARY:==11820== in use at exit: 0 bytes in 0 blocks==11820== total heap usage: 5,555 allocs, 5,555 frees, 7,293,931 bytes allocated==11820====11820== All heap blocks were freed -- no leaks are possible==11820====11820== For counts of detected and suppressed errors, rerun with: -v==11820== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 4 from 4)==11807====11807== HEAP SUMMARY:==11807== in use at exit: 0 bytes in 0 blocks==11807== total heap usage: 0 allocs, 0 frees, 0 bytes allocated==11807====11807== All heap blocks were freed -- no leaks are possible==11807====11807== For counts of detected and suppressed errors, rerun with: -v==11807== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 4 from 4)On Wed, Nov 13, 2013 at 7:21 AM, Dmitry Sytchev <kbdfck@gmail.com> wrote:
Please create a JIRA ticket with your information!2013/11/13 gggyniidt <gggyniidt@gmail.com>
Thanks for Dmitry and Anthony's help, But my problem is not like Dmitry said.
My program is a esl client. The leak is from esl client, not freeswitch.I add the Anthony's patch code into esl.c,but it doesn't work. Each call will cause the 20-40KB of memory growth.I make 3 calls, the valgrind outputs like this
==14459== 90,642 (264 direct, 90,378 indirect) bytes in 3 blocks are definitely lost in loss record 403 of 408==14459== at 0x4A06AAA: malloc (vg_replace_malloc.c:291)==14459== by 0x414524: esl_event_create_subclass (esl_event.c:179)==14459== by 0x412054: esl_recv_event (esl.c:1245)==14459== by 0x411C38: esl_recv_event_timed (esl.c:1137)==14459== by 0x412E22: esl_send_recv_timed (esl.c:1521)==14459== by 0x410C06: esl_execute (esl.c:555)==14459== by 0x4087A8: CRBT::Route() (CRBT.cpp:42)==14459== by 0x409FE6: DPL::route() (DPL.cpp:286)==14459== by 0x40FBE0: mycallback(int, int, sockaddr_in*) (listener.cpp:47)==14459== by 0x410FED: client_thread (esl.c:654)==14459== by 0x41A334: thread_launch (esl_threadmutex.c:74)==14459== by 0x34CAA07850: start_thread (in /lib64/libpthread-2.12.so)==14459====14459== LEAK SUMMARY:==14459== definitely lost: 264 bytes in 3 blocks==14459== indirectly lost: 90,378 bytes in 228 blocks==14459== possibly lost: 550,082 bytes in 51 blocks==14459== still reachable: 832,938 bytes in 840 blocks==14459== suppressed: 0 bytes in 0 blocks==14459== Reachable blocks (those to which a pointer was found) are not shown.==14459== To see them, rerun with: --leak-check=full --show-leak-kinds=allI checked my code and no leak found. The leak may be in esl.c.
2013/11/13 Anthony Minessale <anthony.minessale@gmail.com>
You are showing valgrind traces of ESL client code but now you are saying your FS process has memory growth not the ESL app?You should be doing this on jira and you should have more details about the env.This mailing list is not for bug reports.I did push a patch for what I think might be the problem you showed on esl lib.FS can use up to a gig of memory without giving it back. To make sure you have a real memory leak you need to have it never peak but continue to grow.Anthony Minessale IIOn Tue, Nov 12, 2013 at 11:17 AM, Dmitry Sytchev <kbdfck@gmail.com> wrote:
I can confirm significant Freeswitch process memory growth over time when ESL is used intensively, from 100MB in normal case to 700-900MB after few hours of high esl load, though I have no valgrind check results, as I can't do tests at production server.With less than 300 channels in peak we have 900MB FS process size, never shrinking back :(
2013/11/12 gggyniidt <gggyniidt@gmail.com>
_________________________________________________________________________HelloUsing ESL found a memory leak problemvalgrind --tool=memcheck --leak-check=full ./myprog
output as follows==6872== 29,986 (88 direct, 29,898 indirect) bytes in 1 blocks are definitely lost in loss record 401 of 408==6872== at 0x4A06AAA: malloc (vg_replace_malloc.c:291)==6872== by 0x413941: esl_event_create_subclass (esl_event.c:179)==6872== by 0x410DC3: esl_recv_event (esl.c:1245)==6872== by 0x411675: esl_send_recv_timed (esl.c:1517)==6872== by 0x411C1B: esl_execute (esl.c:555)==6872== by 0x408758: CRBT::Route() (CRBT.cpp:42)==6872== by 0x40A02F: DPL::route() (DPL.cpp:234)==6872== by 0x40FCA8: mycallback(int, int, sockaddr_in*) (listener.cpp:47)==6872== by 0x41032F: client_thread (esl.c:654)==6872== by 0x413FC7: thread_launch (esl_threadmutex.c:74)==6872== by 0x34CAA07850: start_thread (in /lib64/libpthread-2.12.so)==6872== by 0x34CA6E890C: clone (in /lib64/libc-2.12.so)==6872== LEAK SUMMARY:==6872== definitely lost: 88 bytes in 1 blocks==6872== indirectly lost: 29,898 bytes in 76 blocks==6872== possibly lost: 550,082 bytes in 51 blocks==6872== still reachable: 832,937 bytes in 840 blocks==6872== suppressed: 0 bytes in 0 blocks==6872== Reachable blocks (those to which a pointer was found) are not shown.==6872== To see them, rerun with: --leak-check=full --show-leak-kinds=allPart of my code is as followsbool CRBT::Route() {string data = "ringback=file_string://";data.append(FileName);
esl_execute(handle, "set", data.c_str(), NULL);return true;}The OS version is Centos x86_64 6.3The Freeswitch version is 1.2.13Thanks for your help.
Professional FreeSWITCH Consulting Services:
consulting@freeswitch.org
http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server
http://www.cudatel.com
Official FreeSWITCH Sites
http://www.freeswitch.org
http://wiki.freeswitch.org
http://www.cluecon.com
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Best regards,
Dmitry Sytchev,
IT Engineer
_________________________________________________________________________
Professional FreeSWITCH Consulting Services:
consulting@freeswitch.org
http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server
http://www.cudatel.com
Official FreeSWITCH Sites
http://www.freeswitch.org
http://wiki.freeswitch.org
http://www.cluecon.com
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale@hotmail.com
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org
googletalk:conf+888@conference.freeswitch.org
pstn:+19193869900
_________________________________________________________________________
Professional FreeSWITCH Consulting Services:
consulting@freeswitch.org
http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server
http://www.cudatel.com
Official FreeSWITCH Sites
http://www.freeswitch.org
http://wiki.freeswitch.org
http://www.cluecon.com
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_________________________________________________________________________
Professional FreeSWITCH Consulting Services:
consulting@freeswitch.org
http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server
http://www.cudatel.com
Official FreeSWITCH Sites
http://www.freeswitch.org
http://wiki.freeswitch.org
http://www.cluecon.com
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Best regards,
Dmitry Sytchev,
IT Engineer
_________________________________________________________________________
Professional FreeSWITCH Consulting Services:
consulting@freeswitch.org
http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server
http://www.cudatel.com
Official FreeSWITCH Sites
http://www.freeswitch.org
http://wiki.freeswitch.org
http://www.cluecon.com
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
--
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm
MSN:anthony_minessale@hotmail.com
GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com
IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference
sip:888@conference.freeswitch.org
googletalk:conf+888@conference.freeswitch.org
pstn:+19193869900
_________________________________________________________________________
Professional FreeSWITCH Consulting Services:
consulting@freeswitch.org
http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server
http://www.cudatel.com
Official FreeSWITCH Sites
http://www.freeswitch.org
http://wiki.freeswitch.org
http://www.cluecon.com
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org
_________________________________________________________________________
Professional FreeSWITCH Consulting Services:
consulting@freeswitch.org
http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server
http://www.cudatel.com
Official FreeSWITCH Sites
http://www.freeswitch.org
http://wiki.freeswitch.org
http://www.cluecon.com
FreeSWITCH-users mailing list
FreeSWITCH-users@lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org