I use monitoring tool (zabbix) to check health of FreeSwitch servers. I has found that after 60 days of uptime FS host has less free memory than after server reboot. This memory is returned back after FS daemon restart. On FS host is installed 66fe6e5. How to correctly detect memory leak in FS?
Does it climb up to a point and then plateau, or keep climbing indefinitely? The former would be normal behaviour as you handle calls while the latter would suggest a memory leak. Memory allocated within FreeSWITCH is not necessarily returned to the OS but should be reused within the process. -Steve On 19 August 2016 at 09:35, Sergey Safarov <s.safarov@gmail.com> wrote:
I use monitoring tool (zabbix) to check health of FreeSwitch servers. I has found that after 60 days of uptime FS host has less free memory than after server reboot. This memory is returned back after FS daemon restart. On FS host is installed 66fe6e5.
How to correctly detect memory leak in FS?
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
As example available memory chart from jan to mar [image: Inline image 1] and from last FS daemon restart [image: Inline image 2] Also I can configure track FS daemon memory usage via command "ps -o size $(cat /var/run/freeswitch/freeswitch.pid) | tail -n 1" On Fri, Aug 19, 2016 at 1:44 PM, Steven Ayre <steveayre@gmail.com> wrote:
Does it climb up to a point and then plateau, or keep climbing indefinitely? The former would be normal behaviour as you handle calls while the latter would suggest a memory leak. Memory allocated within FreeSWITCH is not necessarily returned to the OS but should be reused within the process.
-Steve
On 19 August 2016 at 09:35, Sergey Safarov <s.safarov@gmail.com> wrote:
I use monitoring tool (zabbix) to check health of FreeSwitch servers. I has found that after 60 days of uptime FS host has less free memory than after server reboot. This memory is returned back after FS daemon restart. On FS host is installed 66fe6e5.
How to correctly detect memory leak in FS?
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
I has configured FreeSwitch process memory usage graph. According this graph durring 24 days size of used memory ingrezed about two times. [image: FS-memory-chart.png] Pastebin of FreeSwitch process memory map is placed at https://pastebin.freeswitch.org/view/9e66572a Are you have any suggestion how to find memory leak.
That looks normal to me. FreeSWITCH needs a minimum of 2 GB dedicated ram for prolonged use. If you chart goes past 2 to 2.5 gigs, you may have a problem then. you can run valgrind but you can only run 1 call at a time testing typical callflow. valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /path/to/freeswitch -vg Also when those months are over, hopefully you update cos you can keep running the same FS for many months anyway ;) On Tue, Sep 13, 2016 at 9:29 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
I has configured FreeSwitch process memory usage graph. According this graph durring 24 days size of used memory ingrezed about two times. [image: FS-memory-chart.png] Pastebin of FreeSwitch process memory map is placed at https://pastebin.freeswitch.org/view/9e66572a Are you have any suggestion how to find memory leak.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬ ☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>* ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA
Hello, We experience the same behaviour here. UP 0 years, 47 days, 19 hours, 27 minutes, 31 seconds, 208 milliseconds, 341 microseconds 2172632 session(s) since startup cat /proc/11818/status shows: VmPeak: 6829064 kB VmSize: 6778648 kB VmLck: 0 kB VmPin: 0 kB VmHWM: 442120 kB VmRSS: 428764 kB VmData: 6561024 kB VmStk: 136 kB VmExe: 24 kB VmLib: 29644 kB VmPTE: 1828 kB VmSwap: 0 kB Threads: 507 And free says: total used free shared buffers cached Mem: 15G 15G 181M 188M 170M 14G -/+ buffers/cache: 1.2G 14G Swap: 15G 28M 15G Dunno if it helps, but we don't appear to meet any issue with calls (no crash or nothing, on other servers we have it has been running for months or even years with full memory if I'm not mistaken). Regards Florent 2016-09-13 18:11 GMT+02:00 Anthony Minessale <anthony.minessale@gmail.com>:
That looks normal to me. FreeSWITCH needs a minimum of 2 GB dedicated ram for prolonged use. If you chart goes past 2 to 2.5 gigs, you may have a problem then.
you can run valgrind but you can only run 1 call at a time testing typical callflow.
valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /path/to/freeswitch -vg
Also when those months are over, hopefully you update cos you can keep running the same FS for many months anyway ;)
On Tue, Sep 13, 2016 at 9:29 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
I has configured FreeSwitch process memory usage graph. According this graph durring 24 days size of used memory ingrezed about two times. [image: FS-memory-chart.png] Pastebin of FreeSwitch process memory map is placed at https://pastebin.freeswitch.org/view/9e66572a Are you have any suggestion how to find memory leak.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Your prize for 47 days uptime is probably a new version! That is the best way to check for memory leaks, update whenever you can. On Tue, Sep 13, 2016 at 11:41 AM, Florent Krieg <flokrrr@gmail.com> wrote:
Hello,
We experience the same behaviour here.
UP 0 years, 47 days, 19 hours, 27 minutes, 31 seconds, 208 milliseconds, 341 microseconds 2172632 session(s) since startup
cat /proc/11818/status shows: VmPeak: 6829064 kB VmSize: 6778648 kB VmLck: 0 kB VmPin: 0 kB VmHWM: 442120 kB VmRSS: 428764 kB VmData: 6561024 kB VmStk: 136 kB VmExe: 24 kB VmLib: 29644 kB VmPTE: 1828 kB VmSwap: 0 kB Threads: 507
And free says: total used free shared buffers cached Mem: 15G 15G 181M 188M 170M 14G -/+ buffers/cache: 1.2G 14G Swap: 15G 28M 15G
Dunno if it helps, but we don't appear to meet any issue with calls (no crash or nothing, on other servers we have it has been running for months or even years with full memory if I'm not mistaken).
Regards Florent
2016-09-13 18:11 GMT+02:00 Anthony Minessale <anthony.minessale@gmail.com> :
That looks normal to me. FreeSWITCH needs a minimum of 2 GB dedicated ram for prolonged use. If you chart goes past 2 to 2.5 gigs, you may have a problem then.
you can run valgrind but you can only run 1 call at a time testing typical callflow.
valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /path/to/freeswitch -vg
Also when those months are over, hopefully you update cos you can keep running the same FS for many months anyway ;)
On Tue, Sep 13, 2016 at 9:29 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
I has configured FreeSwitch process memory usage graph. According this graph durring 24 days size of used memory ingrezed about two times. [image: FS-memory-chart.png] Pastebin of FreeSwitch process memory map is placed at https://pastebin.freeswitch.org/view/9e66572a Are you have any suggestion how to find memory leak.
____________________________________________________________ _____________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬ ☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>* ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA
I don't have the data with me now, but i have servers that wave been running fs for literally months, last time i checked it had processed like 700k+ calls On Tue, Sep 13, 2016 at 7:04 PM Anthony Minessale < anthony.minessale@gmail.com> wrote:
Your prize for 47 days uptime is probably a new version! That is the best way to check for memory leaks, update whenever you can.
On Tue, Sep 13, 2016 at 11:41 AM, Florent Krieg <flokrrr@gmail.com> wrote:
Hello,
We experience the same behaviour here.
UP 0 years, 47 days, 19 hours, 27 minutes, 31 seconds, 208 milliseconds, 341 microseconds 2172632 session(s) since startup
cat /proc/11818/status shows: VmPeak: 6829064 kB VmSize: 6778648 kB VmLck: 0 kB VmPin: 0 kB VmHWM: 442120 kB VmRSS: 428764 kB VmData: 6561024 kB VmStk: 136 kB VmExe: 24 kB VmLib: 29644 kB VmPTE: 1828 kB VmSwap: 0 kB Threads: 507
And free says: total used free shared buffers cached Mem: 15G 15G 181M 188M 170M 14G -/+ buffers/cache: 1.2G 14G Swap: 15G 28M 15G
Dunno if it helps, but we don't appear to meet any issue with calls (no crash or nothing, on other servers we have it has been running for months or even years with full memory if I'm not mistaken).
Regards Florent
2016-09-13 18:11 GMT+02:00 Anthony Minessale <anthony.minessale@gmail.com
:
That looks normal to me. FreeSWITCH needs a minimum of 2 GB dedicated ram for prolonged use. If you chart goes past 2 to 2.5 gigs, you may have a problem then.
you can run valgrind but you can only run 1 call at a time testing typical callflow.
valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /path/to/freeswitch -vg
Also when those months are over, hopefully you update cos you can keep running the same FS for many months anyway ;)
On Tue, Sep 13, 2016 at 9:29 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
I has configured FreeSwitch process memory usage graph. According this graph durring 24 days size of used memory ingrezed about two times. [image: FS-memory-chart.png] Pastebin of FreeSwitch process memory map is placed at https://pastebin.freeswitch.org/view/9e66572a Are you have any suggestion how to find memory leak.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
These are just a couple of them: Note the sessions and uptime On Tue, Sep 13, 2016 at 8:07 PM David Villasmil < david.villasmil.work@gmail.com> wrote:
I don't have the data with me now, but i have servers that wave been running fs for literally months, last time i checked it had processed like 700k+ calls
On Tue, Sep 13, 2016 at 7:04 PM Anthony Minessale < anthony.minessale@gmail.com> wrote:
Your prize for 47 days uptime is probably a new version! That is the best way to check for memory leaks, update whenever you can.
On Tue, Sep 13, 2016 at 11:41 AM, Florent Krieg <flokrrr@gmail.com> wrote:
Hello,
We experience the same behaviour here.
UP 0 years, 47 days, 19 hours, 27 minutes, 31 seconds, 208 milliseconds, 341 microseconds 2172632 session(s) since startup
cat /proc/11818/status shows: VmPeak: 6829064 kB VmSize: 6778648 kB VmLck: 0 kB VmPin: 0 kB VmHWM: 442120 kB VmRSS: 428764 kB VmData: 6561024 kB VmStk: 136 kB VmExe: 24 kB VmLib: 29644 kB VmPTE: 1828 kB VmSwap: 0 kB Threads: 507
And free says: total used free shared buffers cached Mem: 15G 15G 181M 188M 170M 14G -/+ buffers/cache: 1.2G 14G Swap: 15G 28M 15G
Dunno if it helps, but we don't appear to meet any issue with calls (no crash or nothing, on other servers we have it has been running for months or even years with full memory if I'm not mistaken).
Regards Florent
2016-09-13 18:11 GMT+02:00 Anthony Minessale < anthony.minessale@gmail.com>:
That looks normal to me. FreeSWITCH needs a minimum of 2 GB dedicated ram for prolonged use. If you chart goes past 2 to 2.5 gigs, you may have a problem then.
you can run valgrind but you can only run 1 call at a time testing typical callflow.
valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /path/to/freeswitch -vg
Also when those months are over, hopefully you update cos you can keep running the same FS for many months anyway ;)
On Tue, Sep 13, 2016 at 9:29 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
I has configured FreeSwitch process memory usage graph. According this graph durring 24 days size of used memory ingrezed about two times. [image: FS-memory-chart.png] Pastebin of FreeSwitch process memory map is placed at https://pastebin.freeswitch.org/view/9e66572a Are you have any suggestion how to find memory leak.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
I’ve fixed a few memory leaks in the last 60 days (now in the 1.6.10 release). If you can capture ones with a specific set of call flows in valgrind, that would be the path to try to fix them if there is indeed a leak. Just a graph showing memory use growth isn’t a particularly useful metric to find a specific issue, or often to even know if there is an issue. If you suspect a problem, capturing the exact allocation that is not being freed using valgrind or clang memory tools, and coming up with a reproducible scenario to create the issue, is the only way we can address probes like these. Mike
On Sep 13, 2016, at 2:12 PM, David Villasmil <david.villasmil.work@gmail.com> wrote:
These are just a couple of them: Note the sessions and uptime <IMG_9202.PNG><IMG_9200.PNG>
On Tue, Sep 13, 2016 at 8:07 PM David Villasmil <david.villasmil.work@gmail.com <mailto:david.villasmil.work@gmail.com>> wrote: I don't have the data with me now, but i have servers that wave been running fs for literally months, last time i checked it had processed like 700k+ calls
On Tue, Sep 13, 2016 at 7:04 PM Anthony Minessale <anthony.minessale@gmail.com <mailto:anthony.minessale@gmail.com>> wrote: Your prize for 47 days uptime is probably a new version! That is the best way to check for memory leaks, update whenever you can.
On Tue, Sep 13, 2016 at 11:41 AM, Florent Krieg <flokrrr@gmail.com <mailto:flokrrr@gmail.com>> wrote: Hello,
We experience the same behaviour here.
UP 0 years, 47 days, 19 hours, 27 minutes, 31 seconds, 208 milliseconds, 341 microseconds 2172632 session(s) since startup
cat /proc/11818/status shows: VmPeak: 6829064 kB VmSize: 6778648 kB VmLck: 0 kB VmPin: 0 kB VmHWM: 442120 kB VmRSS: 428764 kB VmData: 6561024 kB VmStk: 136 kB VmExe: 24 kB VmLib: 29644 kB VmPTE: 1828 kB VmSwap: 0 kB Threads: 507
And free says: total used free shared buffers cached Mem: 15G 15G 181M 188M 170M 14G -/+ buffers/cache: 1.2G 14G Swap: 15G 28M 15G
Dunno if it helps, but we don't appear to meet any issue with calls (no crash or nothing, on other servers we have it has been running for months or even years with full memory if I'm not mistaken).
Regards Florent
2016-09-13 18:11 GMT+02:00 Anthony Minessale <anthony.minessale@gmail.com <mailto:anthony.minessale@gmail.com>>: That looks normal to me. FreeSWITCH needs a minimum of 2 GB dedicated ram for prolonged use. If you chart goes past 2 to 2.5 gigs, you may have a problem then.
you can run valgrind but you can only run 1 call at a time testing typical callflow. valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /path/to/freeswitch -vg
Also when those months are over, hopefully you update cos you can keep running the same FS for many months anyway ;)
On Tue, Sep 13, 2016 at 9:29 AM, Sergey Safarov <s.safarov@gmail.com <mailto:s.safarov@gmail.com>> wrote: I has configured FreeSwitch process memory usage graph. According this graph durring 24 days size of used memory ingrezed about two times. <FS-memory-chart.png> Pastebin of FreeSwitch process memory map is placed at https://pastebin.freeswitch.org/view/9e66572a <https://pastebin.freeswitch.org/view/9e66572a> Are you have any suggestion how to find memory leak.
Thank you Michael I will update FS to current master on one of hosts. And compare memory usage graphs with old versions. Also as Anthony has suggested i will wait for memory usage is grows (or stop) up to 2,5Gb. Thank you Anthony for valgrind usage instructions. ср, 14 сент. 2016 г. в 20:15, Michael Jerris <mike@jerris.com>:
I’ve fixed a few memory leaks in the last 60 days (now in the 1.6.10 release). If you can capture ones with a specific set of call flows in valgrind, that would be the path to try to fix them if there is indeed a leak. Just a graph showing memory use growth isn’t a particularly useful metric to find a specific issue, or often to even know if there is an issue. If you suspect a problem, capturing the exact allocation that is not being freed using valgrind or clang memory tools, and coming up with a reproducible scenario to create the issue, is the only way we can address probes like these.
Mike
On Sep 13, 2016, at 2:12 PM, David Villasmil < david.villasmil.work@gmail.com> wrote:
These are just a couple of them: Note the sessions and uptime
<IMG_9202.PNG><IMG_9200.PNG>
On Tue, Sep 13, 2016 at 8:07 PM David Villasmil < david.villasmil.work@gmail.com> wrote:
I don't have the data with me now, but i have servers that wave been
running fs for literally months, last time i checked it had processed like 700k+ calls
On Tue, Sep 13, 2016 at 7:04 PM Anthony Minessale < anthony.minessale@gmail.com> wrote:
Your prize for 47 days uptime is probably a new version!
That is the best way to check for memory leaks, update whenever you can.
On Tue, Sep 13, 2016 at 11:41 AM, Florent Krieg <flokrrr@gmail.com> wrote:
Hello,
We experience the same behaviour here.
UP 0 years, 47 days, 19 hours, 27 minutes, 31 seconds, 208 milliseconds, 341 microseconds 2172632 session(s) since startup
cat /proc/11818/status shows: VmPeak: 6829064 kB VmSize: 6778648 kB VmLck: 0 kB VmPin: 0 kB VmHWM: 442120 kB VmRSS: 428764 kB VmData: 6561024 kB VmStk: 136 kB VmExe: 24 kB VmLib: 29644 kB VmPTE: 1828 kB VmSwap: 0 kB Threads: 507
And free says: total used free shared buffers cached Mem: 15G 15G 181M 188M 170M 14G -/+ buffers/cache: 1.2G 14G Swap: 15G 28M 15G
Dunno if it helps, but we don't appear to meet any issue with calls (no crash or nothing, on other servers we have it has been running for months or even years with full memory if I'm not mistaken).
Regards Florent
2016-09-13 18:11 GMT+02:00 Anthony Minessale <
anthony.minessale@gmail.com>:
That looks normal to me.
FreeSWITCH needs a minimum of 2 GB dedicated ram for prolonged use. If you chart goes past 2 to 2.5 gigs, you may have a problem then.
you can run valgrind but you can only run 1 call at a time testing typical callflow.
valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /path/to/freeswitch -vg
Also when those months are over, hopefully you update cos you can keep running the same FS for many months anyway ;)
On Tue, Sep 13, 2016 at 9:29 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
I has configured FreeSwitch process memory usage graph.
According this graph durring 24 days size of used memory ingrezed about two times.
<FS-memory-chart.png>
Pastebin of FreeSwitch process memory map is placed at https://pastebin.freeswitch.org/view/9e66572a Are you have any suggestion how to find memory leak.
Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Hello Anthony On old FS version memory usage graph is attached. I think memory usage grows is stoped near 3Gb but last week memory usage is increased about 200Mb later i will report result on new version and with valgrind results. [image: pasted1] вт, 13 сент. 2016 г. в 19:12, Anthony Minessale <anthony.minessale@gmail.com
:
That looks normal to me. FreeSWITCH needs a minimum of 2 GB dedicated ram for prolonged use. If you chart goes past 2 to 2.5 gigs, you may have a problem then.
you can run valgrind but you can only run 1 call at a time testing typical callflow.
valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /path/to/freeswitch -vg
Also when those months are over, hopefully you update cos you can keep running the same FS for many months anyway ;)
On Tue, Sep 13, 2016 at 9:29 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
I has configured FreeSwitch process memory usage graph. According this graph durring 24 days size of used memory ingrezed about two times. [image: FS-memory-chart.png] Pastebin of FreeSwitch process memory map is placed at https://pastebin.freeswitch.org/view/9e66572a Are you have any suggestion how to find memory leak.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 <+1%20919-386-9900>
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Good news is 2 releases came out during that test run (and another one in a week or less) so now you can start over testing newest version again. On Tue, Nov 15, 2016 at 3:23 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
Hello Anthony On old FS version memory usage graph is attached. I think memory usage grows is stoped near 3Gb but last week memory usage is increased about 200Mb later i will report result on new version and with valgrind results. [image: pasted1]
вт, 13 сент. 2016 г. в 19:12, Anthony Minessale < anthony.minessale@gmail.com>:
That looks normal to me. FreeSWITCH needs a minimum of 2 GB dedicated ram for prolonged use. If you chart goes past 2 to 2.5 gigs, you may have a problem then.
you can run valgrind but you can only run 1 call at a time testing typical callflow.
valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /path/to/freeswitch -vg
Also when those months are over, hopefully you update cos you can keep running the same FS for many months anyway ;)
On Tue, Sep 13, 2016 at 9:29 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
I has configured FreeSwitch process memory usage graph. According this graph durring 24 days size of used memory ingrezed about two times. [image: FS-memory-chart.png] Pastebin of FreeSwitch process memory map is placed at https://pastebin.freeswitch.org/view/9e66572a Are you have any suggestion how to find memory leak.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 <+1%20919-386-9900>
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬ ☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>* ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA
Hello Anthony I make tests on current master 594b36022e0e5bb72f9072ac0a11d98e7bbd5a7c. Results is available at http://node1.pbx.rcsnet.ru/vg.log FreeSwitch runned by command valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /usr/local/freeswitch/bin/freeswitch -nf -nc -vg On server installed vanilla config and maked one call. вт, 15 нояб. 2016 г. в 19:30, Anthony Minessale <anthony.minessale@gmail.com
:
Good news is 2 releases came out during that test run (and another one in a week or less) so now you can start over testing newest version again.
On Tue, Nov 15, 2016 at 3:23 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
Hello Anthony On old FS version memory usage graph is attached. I think memory usage grows is stoped near 3Gb but last week memory usage is increased about 200Mb later i will report result on new version and with valgrind results. [image: pasted1]
вт, 13 сент. 2016 г. в 19:12, Anthony Minessale < anthony.minessale@gmail.com>:
That looks normal to me. FreeSWITCH needs a minimum of 2 GB dedicated ram for prolonged use. If you chart goes past 2 to 2.5 gigs, you may have a problem then.
you can run valgrind but you can only run 1 call at a time testing typical callflow.
valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /path/to/freeswitch -vg
Also when those months are over, hopefully you update cos you can keep running the same FS for many months anyway ;)
On Tue, Sep 13, 2016 at 9:29 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
I has configured FreeSwitch process memory usage graph. According this graph durring 24 days size of used memory ingrezed about two times. [image: FS-memory-chart.png] Pastebin of FreeSwitch process memory map is placed at https://pastebin.freeswitch.org/view/9e66572a Are you have any suggestion how to find memory leak.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 <+1%20919-386-9900>
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 <(919)%20386-9900>
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Anthony i want remove messages like "blocks are still reachable in loss record" in valgring log. To do this i defined GLIBCXX_FORCE_NEW=1 enviroment variable at compilation state and execution state but this not take effect. Is it related to memory caching in APR library? пт, 30 дек. 2016 г. в 14:01, Sergey Safarov <s.safarov@gmail.com>:
Hello Anthony I make tests on current master 594b36022e0e5bb72f9072ac0a11d98e7bbd5a7c. Results is available at http://node1.pbx.rcsnet.ru/vg.log
FreeSwitch runned by command valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /usr/local/freeswitch/bin/freeswitch -nf -nc -vg
On server installed vanilla config and maked one call.
вт, 15 нояб. 2016 г. в 19:30, Anthony Minessale < anthony.minessale@gmail.com>:
Good news is 2 releases came out during that test run (and another one in a week or less) so now you can start over testing newest version again.
On Tue, Nov 15, 2016 at 3:23 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
Hello Anthony On old FS version memory usage graph is attached. I think memory usage grows is stoped near 3Gb but last week memory usage is increased about 200Mb later i will report result on new version and with valgrind results. [image: pasted1]
вт, 13 сент. 2016 г. в 19:12, Anthony Minessale < anthony.minessale@gmail.com>:
That looks normal to me. FreeSWITCH needs a minimum of 2 GB dedicated ram for prolonged use. If you chart goes past 2 to 2.5 gigs, you may have a problem then.
you can run valgrind but you can only run 1 call at a time testing typical callflow.
valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /path/to/freeswitch -vg
Also when those months are over, hopefully you update cos you can keep running the same FS for many months anyway ;)
On Tue, Sep 13, 2016 at 9:29 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
I has configured FreeSwitch process memory usage graph. According this graph durring 24 days size of used memory ingrezed about two times. [image: FS-memory-chart.png] Pastebin of FreeSwitch process memory map is placed at https://pastebin.freeswitch.org/view/9e66572a Are you have any suggestion how to find memory leak.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 <+1%20919-386-9900>
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 <(919)%20386-9900>
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
It looks like the ones you referenced are one time allocations from loading modules. To do a real test you should make calls that match production calls but only one at a time very slowly and get a few hundred calls while under vg. Also its very important to have a 100% clean shutdown or you can get false positives. 2gb over 24 days is not outside the realm of normal for FS. You could do a stress test and get up that high in minutes. For non-video usage, you would want to start pushing into 3gb before getting worried. I would set 4gb as min ram to run FS but not in all cases, 8gb is much safer. On Fri, Dec 30, 2016 at 6:15 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
Anthony i want remove messages like "blocks are still reachable in loss record" in valgring log. To do this i defined GLIBCXX_FORCE_NEW=1 enviroment variable at compilation state and execution state but this not take effect.
Is it related to memory caching in APR library?
пт, 30 дек. 2016 г. в 14:01, Sergey Safarov <s.safarov@gmail.com>:
Hello Anthony I make tests on current master 594b36022e0e5bb72f9072ac0a11d98e7bbd5a7c. Results is available at http://node1.pbx.rcsnet.ru/vg.log
FreeSwitch runned by command valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /usr/local/freeswitch/bin/freeswitch -nf -nc -vg
On server installed vanilla config and maked one call.
вт, 15 нояб. 2016 г. в 19:30, Anthony Minessale < anthony.minessale@gmail.com>:
Good news is 2 releases came out during that test run (and another one in a week or less) so now you can start over testing newest version again.
On Tue, Nov 15, 2016 at 3:23 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
Hello Anthony On old FS version memory usage graph is attached. I think memory usage grows is stoped near 3Gb but last week memory usage is increased about 200Mb later i will report result on new version and with valgrind results. [image: pasted1]
вт, 13 сент. 2016 г. в 19:12, Anthony Minessale < anthony.minessale@gmail.com>:
That looks normal to me. FreeSWITCH needs a minimum of 2 GB dedicated ram for prolonged use. If you chart goes past 2 to 2.5 gigs, you may have a problem then.
you can run valgrind but you can only run 1 call at a time testing typical callflow.
valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /path/to/freeswitch -vg
Also when those months are over, hopefully you update cos you can keep running the same FS for many months anyway ;)
On Tue, Sep 13, 2016 at 9:29 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
I has configured FreeSwitch process memory usage graph. According this graph durring 24 days size of used memory ingrezed about two times. [image: FS-memory-chart.png] Pastebin of FreeSwitch process memory map is placed at https://pastebin.freeswitch.org/view/9e66572a Are you have any suggestion how to find memory leak.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 <+1%20919-386-9900>
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 <(919)%20386-9900>
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬ ☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>* ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA
If I do call test using SIPP script (10 sec each call, PCMA codec) and about 25 000 calls (3 days) under valgrind - is it will be enough? P.S. On my servers about 10000 calls per day and about 300 000 per month. Sergey пт, 30 дек. 2016 г. в 21:13, Anthony Minessale <anthony.minessale@gmail.com
:
It looks like the ones you referenced are one time allocations from loading modules. To do a real test you should make calls that match production calls but only one at a time very slowly and get a few hundred calls while under vg. Also its very important to have a 100% clean shutdown or you can get false positives.
2gb over 24 days is not outside the realm of normal for FS. You could do a stress test and get up that high in minutes. For non-video usage, you would want to start pushing into 3gb before getting worried. I would set 4gb as min ram to run FS but not in all cases, 8gb is much safer.
On Fri, Dec 30, 2016 at 6:15 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
Anthony i want remove messages like "blocks are still reachable in loss record" in valgring log. To do this i defined GLIBCXX_FORCE_NEW=1 enviroment variable at compilation state and execution state but this not take effect.
Is it related to memory caching in APR library?
пт, 30 дек. 2016 г. в 14:01, Sergey Safarov <s.safarov@gmail.com>:
Hello Anthony I make tests on current master 594b36022e0e5bb72f9072ac0a11d98e7bbd5a7c. Results is available at http://node1.pbx.rcsnet.ru/vg.log
FreeSwitch runned by command valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /usr/local/freeswitch/bin/freeswitch -nf -nc -vg
On server installed vanilla config and maked one call.
вт, 15 нояб. 2016 г. в 19:30, Anthony Minessale < anthony.minessale@gmail.com>:
Good news is 2 releases came out during that test run (and another one in a week or less) so now you can start over testing newest version again.
On Tue, Nov 15, 2016 at 3:23 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
Hello Anthony On old FS version memory usage graph is attached. I think memory usage grows is stoped near 3Gb but last week memory usage is increased about 200Mb later i will report result on new version and with valgrind results. [image: pasted1]
вт, 13 сент. 2016 г. в 19:12, Anthony Minessale < anthony.minessale@gmail.com>:
That looks normal to me. FreeSWITCH needs a minimum of 2 GB dedicated ram for prolonged use. If you chart goes past 2 to 2.5 gigs, you may have a problem then.
you can run valgrind but you can only run 1 call at a time testing typical callflow.
valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /path/to/freeswitch -vg
Also when those months are over, hopefully you update cos you can keep running the same FS for many months anyway ;)
On Tue, Sep 13, 2016 at 9:29 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
I has configured FreeSwitch process memory usage graph. According this graph durring 24 days size of used memory ingrezed about two times. [image: FS-memory-chart.png] Pastebin of FreeSwitch process memory map is placed at https://pastebin.freeswitch.org/view/9e66572a Are you have any suggestion how to find memory leak.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 <+1%20919-386-9900>
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 <(919)%20386-9900>
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 <(919)%20386-9900>
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
It will surely give you a better picture if there is anything of concern. Like I said before you are not outside normal memory footprints in what you demonstrated. On Fri, Dec 30, 2016 at 2:12 PM, Sergey Safarov <s.safarov@gmail.com> wrote:
If I do call test using SIPP script (10 sec each call, PCMA codec) and about 25 000 calls (3 days) under valgrind - is it will be enough?
P.S. On my servers about 10000 calls per day and about 300 000 per month.
Sergey
пт, 30 дек. 2016 г. в 21:13, Anthony Minessale < anthony.minessale@gmail.com>:
It looks like the ones you referenced are one time allocations from loading modules. To do a real test you should make calls that match production calls but only one at a time very slowly and get a few hundred calls while under vg. Also its very important to have a 100% clean shutdown or you can get false positives.
2gb over 24 days is not outside the realm of normal for FS. You could do a stress test and get up that high in minutes. For non-video usage, you would want to start pushing into 3gb before getting worried. I would set 4gb as min ram to run FS but not in all cases, 8gb is much safer.
On Fri, Dec 30, 2016 at 6:15 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
Anthony i want remove messages like "blocks are still reachable in loss record" in valgring log. To do this i defined GLIBCXX_FORCE_NEW=1 enviroment variable at compilation state and execution state but this not take effect.
Is it related to memory caching in APR library?
пт, 30 дек. 2016 г. в 14:01, Sergey Safarov <s.safarov@gmail.com>:
Hello Anthony I make tests on current master 594b36022e0e5bb72f9072ac0a11d98e7bbd5a7c. Results is available at http://node1.pbx.rcsnet.ru/vg.log
FreeSwitch runned by command valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /usr/local/freeswitch/bin/freeswitch -nf -nc -vg
On server installed vanilla config and maked one call.
вт, 15 нояб. 2016 г. в 19:30, Anthony Minessale < anthony.minessale@gmail.com>:
Good news is 2 releases came out during that test run (and another one in a week or less) so now you can start over testing newest version again.
On Tue, Nov 15, 2016 at 3:23 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
Hello Anthony On old FS version memory usage graph is attached. I think memory usage grows is stoped near 3Gb but last week memory usage is increased about 200Mb later i will report result on new version and with valgrind results. [image: pasted1]
вт, 13 сент. 2016 г. в 19:12, Anthony Minessale < anthony.minessale@gmail.com>:
That looks normal to me. FreeSWITCH needs a minimum of 2 GB dedicated ram for prolonged use. If you chart goes past 2 to 2.5 gigs, you may have a problem then.
you can run valgrind but you can only run 1 call at a time testing typical callflow.
valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /path/to/freeswitch -vg
Also when those months are over, hopefully you update cos you can keep running the same FS for many months anyway ;)
On Tue, Sep 13, 2016 at 9:29 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
I has configured FreeSwitch process memory usage graph. According this graph durring 24 days size of used memory ingrezed about two times. [image: FS-memory-chart.png] Pastebin of FreeSwitch process memory map is placed at https://pastebin.freeswitch.org/view/9e66572a Are you have any suggestion how to find memory leak.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 <+1%20919-386-9900>
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 <(919)%20386-9900>
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 <(919)%20386-9900>
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬ ☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>* ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA
Then is required do tests until memory usage is will be more 4gb? пт, 30 дек. 2016, 23:51 Anthony Minessale <anthony.minessale@gmail.com>:
It will surely give you a better picture if there is anything of concern. Like I said before you are not outside normal memory footprints in what you demonstrated.
On Fri, Dec 30, 2016 at 2:12 PM, Sergey Safarov <s.safarov@gmail.com> wrote:
If I do call test using SIPP script (10 sec each call, PCMA codec) and about 25 000 calls (3 days) under valgrind - is it will be enough?
P.S. On my servers about 10000 calls per day and about 300 000 per month.
Sergey
пт, 30 дек. 2016 г. в 21:13, Anthony Minessale < anthony.minessale@gmail.com>:
It looks like the ones you referenced are one time allocations from loading modules. To do a real test you should make calls that match production calls but only one at a time very slowly and get a few hundred calls while under vg. Also its very important to have a 100% clean shutdown or you can get false positives.
2gb over 24 days is not outside the realm of normal for FS. You could do a stress test and get up that high in minutes. For non-video usage, you would want to start pushing into 3gb before getting worried. I would set 4gb as min ram to run FS but not in all cases, 8gb is much safer.
On Fri, Dec 30, 2016 at 6:15 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
Anthony i want remove messages like "blocks are still reachable in loss record" in valgring log. To do this i defined GLIBCXX_FORCE_NEW=1 enviroment variable at compilation state and execution state but this not take effect.
Is it related to memory caching in APR library?
пт, 30 дек. 2016 г. в 14:01, Sergey Safarov <s.safarov@gmail.com>:
Hello Anthony I make tests on current master 594b36022e0e5bb72f9072ac0a11d98e7bbd5a7c. Results is available at http://node1.pbx.rcsnet.ru/vg.log
FreeSwitch runned by command valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /usr/local/freeswitch/bin/freeswitch -nf -nc -vg
On server installed vanilla config and maked one call.
вт, 15 нояб. 2016 г. в 19:30, Anthony Minessale < anthony.minessale@gmail.com>:
Good news is 2 releases came out during that test run (and another one in a week or less) so now you can start over testing newest version again.
On Tue, Nov 15, 2016 at 3:23 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
Hello Anthony On old FS version memory usage graph is attached. I think memory usage grows is stoped near 3Gb but last week memory usage is increased about 200Mb later i will report result on new version and with valgrind results. [image: pasted1]
вт, 13 сент. 2016 г. в 19:12, Anthony Minessale < anthony.minessale@gmail.com>:
That looks normal to me. FreeSWITCH needs a minimum of 2 GB dedicated ram for prolonged use. If you chart goes past 2 to 2.5 gigs, you may have a problem then.
you can run valgrind but you can only run 1 call at a time testing typical callflow.
valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /path/to/freeswitch -vg
Also when those months are over, hopefully you update cos you can keep running the same FS for many months anyway ;)
On Tue, Sep 13, 2016 at 9:29 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
I has configured FreeSwitch process memory usage graph. According this graph durring 24 days size of used memory ingrezed about two times. [image: FS-memory-chart.png] Pastebin of FreeSwitch process memory map is placed at https://pastebin.freeswitch.org/view/9e66572a Are you have any suggestion how to find memory leak.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 <+1%20919-386-9900>
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 <(919)%20386-9900>
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 <(919)%20386-9900>
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Not for valgrind. you just need a good sampling of many calls of all types of typical traffic. You cannot exceed 1 or 2 cps usually or it will begin to fail. On Fri, Dec 30, 2016 at 2:55 PM, Sergey Safarov <s.safarov@gmail.com> wrote:
Then is required do tests until memory usage is will be more 4gb?
пт, 30 дек. 2016, 23:51 Anthony Minessale <anthony.minessale@gmail.com>:
It will surely give you a better picture if there is anything of concern. Like I said before you are not outside normal memory footprints in what you demonstrated.
On Fri, Dec 30, 2016 at 2:12 PM, Sergey Safarov <s.safarov@gmail.com> wrote:
If I do call test using SIPP script (10 sec each call, PCMA codec) and about 25 000 calls (3 days) under valgrind - is it will be enough?
P.S. On my servers about 10000 calls per day and about 300 000 per month.
Sergey
пт, 30 дек. 2016 г. в 21:13, Anthony Minessale < anthony.minessale@gmail.com>:
It looks like the ones you referenced are one time allocations from loading modules. To do a real test you should make calls that match production calls but only one at a time very slowly and get a few hundred calls while under vg. Also its very important to have a 100% clean shutdown or you can get false positives.
2gb over 24 days is not outside the realm of normal for FS. You could do a stress test and get up that high in minutes. For non-video usage, you would want to start pushing into 3gb before getting worried. I would set 4gb as min ram to run FS but not in all cases, 8gb is much safer.
On Fri, Dec 30, 2016 at 6:15 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
Anthony i want remove messages like "blocks are still reachable in loss record" in valgring log. To do this i defined GLIBCXX_FORCE_NEW=1 enviroment variable at compilation state and execution state but this not take effect.
Is it related to memory caching in APR library?
пт, 30 дек. 2016 г. в 14:01, Sergey Safarov <s.safarov@gmail.com>:
Hello Anthony I make tests on current master 594b36022e0e5bb72f9072ac0a11d98e7bbd5a7c. Results is available at http://node1.pbx.rcsnet.ru/vg.log
FreeSwitch runned by command valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /usr/local/freeswitch/bin/freeswitch -nf -nc -vg
On server installed vanilla config and maked one call.
вт, 15 нояб. 2016 г. в 19:30, Anthony Minessale < anthony.minessale@gmail.com>:
Good news is 2 releases came out during that test run (and another one in a week or less) so now you can start over testing newest version again.
On Tue, Nov 15, 2016 at 3:23 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
Hello Anthony On old FS version memory usage graph is attached. I think memory usage grows is stoped near 3Gb but last week memory usage is increased about 200Mb later i will report result on new version and with valgrind results. [image: pasted1]
вт, 13 сент. 2016 г. в 19:12, Anthony Minessale < anthony.minessale@gmail.com>:
That looks normal to me. FreeSWITCH needs a minimum of 2 GB dedicated ram for prolonged use. If you chart goes past 2 to 2.5 gigs, you may have a problem then.
you can run valgrind but you can only run 1 call at a time testing typical callflow.
valgrind --tool=memcheck --log-file=vg.log --leak-check=full --leak-resolution=high --show-reachable=yes /path/to/freeswitch -vg
Also when those months are over, hopefully you update cos you can keep running the same FS for many months anyway ;)
On Tue, Sep 13, 2016 at 9:29 AM, Sergey Safarov <s.safarov@gmail.com> wrote:
I has configured FreeSwitch process memory usage graph. According this graph durring 24 days size of used memory ingrezed about two times. [image: FS-memory-chart.png] Pastebin of FreeSwitch process memory map is placed at https://pastebin.freeswitch.org/view/9e66572a Are you have any suggestion how to find memory leak.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 <+1%20919-386-9900>
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 <(919)%20386-9900>
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 <(919)%20386-9900>
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬
☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>*
ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 <(919)%20386-9900>
https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬ ☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>* ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA
participants (6)
-
Anthony Minessale
-
David Villasmil
-
Florent Krieg
-
Michael Jerris
-
Sergey Safarov
-
Steven Ayre