Reliability of HEARTBEAT events every 20 seconds? (in mod_event_socket)
Hello everyone, I have a question about the reliability of heartbeat events, it should be sent from FreeSWITCH every 20 seconds - right? If you look below in my log sample (from my own process), you can see that it comes 12:07:01, then 21, then 41, but after this it suddenly takes 31 seconds for it to show up (12:08:12). I've also done a Wireshark trace, and it shows the same thing - so it is really delayed from FS. The load at the time was zero calls, and the server itself was using maybe 5% CPU. After the delayed event it continues to show up every 20 seconds. [2010-12-09 12:07:01] Received HEARTBEAT event - we're alive. [2010-12-09 12:07:21] Received HEARTBEAT event - we're alive. [2010-12-09 12:07:41] Received HEARTBEAT event - we're alive. [2010-12-09 12:08:12] Received HEARTBEAT event - we're alive. [2010-12-09 12:08:32] Received HEARTBEAT event - we're alive. [2010-12-09 12:08:52] Received HEARTBEAT event - we're alive. [2010-12-09 12:09:12] Received HEARTBEAT event - we're alive. Should the HEARTBEAT event be 100% reliable, or could something cause a delay like this? This server is not on latest GIT, but it's not more then one month old. I can debug this further myself, but I just want to know if it always should be triggered every 20 seconds, or if these events for some reasons might be delayed. By the way - no clock skew or anything is detected on the machine, so that's not causing it. Thanks, Peter
What does the heartbeat event show as the uptime? That should tell you whether the delay is occuring in the event being generated, or whether the delay is when the event fires. -Steve On 9 December 2010 11:27, Peter Olsson <peter.olsson@visionutveckling.se> wrote:
Hello everyone,
I have a question about the reliability of heartbeat events, it should be sent from FreeSWITCH every 20 seconds – right? If you look below in my log sample (from my own process), you can see that it comes 12:07:01, then 21, then 41, but after this it suddenly takes 31 seconds for it to show up (12:08:12). I’ve also done a Wireshark trace, and it shows the same thing – so it is really delayed from FS. The load at the time was zero calls, and the server itself was using maybe 5% CPU. After the delayed event it continues to show up every 20 seconds.
[2010-12-09 12:07:01] Received HEARTBEAT event - we're alive.
[2010-12-09 12:07:21] Received HEARTBEAT event - we're alive.
[2010-12-09 12:07:41] Received HEARTBEAT event - we're alive.
[2010-12-09 12:08:12] Received HEARTBEAT event - we're alive.
[2010-12-09 12:08:32] Received HEARTBEAT event - we're alive.
[2010-12-09 12:08:52] Received HEARTBEAT event - we're alive.
[2010-12-09 12:09:12] Received HEARTBEAT event - we're alive.
Should the HEARTBEAT event be 100% reliable, or could something cause a delay like this? This server is not on latest GIT, but it’s not more then one month old. I can debug this further myself, but I just want to know if it always should be triggered every 20 seconds, or if these events for some reasons might be delayed.
By the way – no clock skew or anything is detected on the machine, so that’s not causing it.
Thanks,
Peter
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Also do you have any warning messages like this in the fs log? Task was executed late by %d seconds (That should show up if the delay is in the scheduler, if the task runs >1s late). -Steve On 9 December 2010 11:27, Peter Olsson <peter.olsson@visionutveckling.se> wrote:
Hello everyone,
I have a question about the reliability of heartbeat events, it should be sent from FreeSWITCH every 20 seconds – right? If you look below in my log sample (from my own process), you can see that it comes 12:07:01, then 21, then 41, but after this it suddenly takes 31 seconds for it to show up (12:08:12). I’ve also done a Wireshark trace, and it shows the same thing – so it is really delayed from FS. The load at the time was zero calls, and the server itself was using maybe 5% CPU. After the delayed event it continues to show up every 20 seconds.
[2010-12-09 12:07:01] Received HEARTBEAT event - we're alive.
[2010-12-09 12:07:21] Received HEARTBEAT event - we're alive.
[2010-12-09 12:07:41] Received HEARTBEAT event - we're alive.
[2010-12-09 12:08:12] Received HEARTBEAT event - we're alive.
[2010-12-09 12:08:32] Received HEARTBEAT event - we're alive.
[2010-12-09 12:08:52] Received HEARTBEAT event - we're alive.
[2010-12-09 12:09:12] Received HEARTBEAT event - we're alive.
Should the HEARTBEAT event be 100% reliable, or could something cause a delay like this? This server is not on latest GIT, but it’s not more then one month old. I can debug this further myself, but I just want to know if it always should be triggered every 20 seconds, or if these events for some reasons might be delayed.
By the way – no clock skew or anything is detected on the machine, so that’s not causing it.
Thanks,
Peter
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Nope, no warning message. Thanks for the tip of uptime, I will grab a log for that and wait for it to occur again. It seems to occur once every 30-45 minutes, so not very often - but still very strange :) /Peter -----Ursprungligt meddelande----- Från: freeswitch-users-bounces@lists.freeswitch.org [mailto:freeswitch-users-bounces@lists.freeswitch.org] För Steven Ayre Skickat: den 9 december 2010 12:42 Till: FreeSWITCH Users Help Ämne: Re: [Freeswitch-users] Reliability of HEARTBEAT events every 20 seconds? (in mod_event_socket) Also do you have any warning messages like this in the fs log? Task was executed late by %d seconds (That should show up if the delay is in the scheduler, if the task runs >1s late). -Steve On 9 December 2010 11:27, Peter Olsson <peter.olsson@visionutveckling.se> wrote:
Hello everyone,
I have a question about the reliability of heartbeat events, it should be sent from FreeSWITCH every 20 seconds - right? If you look below in my log sample (from my own process), you can see that it comes 12:07:01, then 21, then 41, but after this it suddenly takes 31 seconds for it to show up (12:08:12). I've also done a Wireshark trace, and it shows the same thing - so it is really delayed from FS. The load at the time was zero calls, and the server itself was using maybe 5% CPU. After the delayed event it continues to show up every 20 seconds.
[2010-12-09 12:07:01] Received HEARTBEAT event - we're alive.
[2010-12-09 12:07:21] Received HEARTBEAT event - we're alive.
[2010-12-09 12:07:41] Received HEARTBEAT event - we're alive.
[2010-12-09 12:08:12] Received HEARTBEAT event - we're alive.
[2010-12-09 12:08:32] Received HEARTBEAT event - we're alive.
[2010-12-09 12:08:52] Received HEARTBEAT event - we're alive.
[2010-12-09 12:09:12] Received HEARTBEAT event - we're alive.
Should the HEARTBEAT event be 100% reliable, or could something cause a delay like this? This server is not on latest GIT, but it's not more then one month old. I can debug this further myself, but I just want to know if it always should be triggered every 20 seconds, or if these events for some reasons might be delayed.
By the way - no clock skew or anything is detected on the machine, so that's not causing it.
Thanks,
Peter
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org !DSPAM:4d00c1a432762107714629!
Ok - I trapped the problem with full event info. The up-time also reports a difference, in this case 32 seconds. As you can see there is no load on the server at all by this time, no calls etc. Hmm, this is really strange I must say. [2010-12-09 13:53:49] Received HEARTBEAT event - we're alive content-length == 629 content-type == text/event-plain core-uuid == 1550f368-b577-4304-bacd-2681d4067036 event-calling-file == switch_core.c event-calling-function == send_heartbeat event-calling-line-number == 65 event-date-gmt == Thu, 09 Dec 2010 12:53:53 GMT event-date-local == 2010-12-09 13:53:53 event-date-timestamp == 1291899233256303 event-info == System Ready event-name == HEARTBEAT freeswitch-hostname == 79w3sipt4 freeswitch-ipv4 == 10.66.195.14 freeswitch-ipv6 == ::1 idle-cpu == 98.437500 session-count == 0 session-per-sec == 30 session-since-startup == 277 up-time == 0 years, 7 days, 1 hour, 1 minute, 1 second, 680 milliseconds, 750 microseconds [2010-12-09 13:54:21] Received HEARTBEAT event - we're alive content-length == 631 content-type == text/event-plain core-uuid == 1550f368-b577-4304-bacd-2681d4067036 event-calling-file == switch_core.c event-calling-function == send_heartbeat event-calling-line-number == 65 event-date-gmt == Thu, 09 Dec 2010 12:54:25 GMT event-date-local == 2010-12-09 13:54:25 event-date-timestamp == 1291899265259303 event-info == System Ready event-name == HEARTBEAT freeswitch-hostname == 79w3sipt4 freeswitch-ipv4 == 10.66.195.14 freeswitch-ipv6 == ::1 idle-cpu == 97.656250 session-count == 0 session-per-sec == 30 session-since-startup == 277 up-time == 0 years, 7 days, 1 hour, 1 minute, 33 seconds, 683 milliseconds, 750 microseconds Mvh Peter Olsson -----Ursprungligt meddelande----- Från: freeswitch-users-bounces@lists.freeswitch.org [mailto:freeswitch-users-bounces@lists.freeswitch.org] För Steven Ayre Skickat: den 9 december 2010 12:42 Till: FreeSWITCH Users Help Ämne: Re: [Freeswitch-users] Reliability of HEARTBEAT events every 20 seconds? (in mod_event_socket) Also do you have any warning messages like this in the fs log? Task was executed late by %d seconds (That should show up if the delay is in the scheduler, if the task runs >1s late). -Steve On 9 December 2010 11:27, Peter Olsson <peter.olsson@visionutveckling.se> wrote:
Hello everyone,
I have a question about the reliability of heartbeat events, it should be sent from FreeSWITCH every 20 seconds - right? If you look below in my log sample (from my own process), you can see that it comes 12:07:01, then 21, then 41, but after this it suddenly takes 31 seconds for it to show up (12:08:12). I've also done a Wireshark trace, and it shows the same thing - so it is really delayed from FS. The load at the time was zero calls, and the server itself was using maybe 5% CPU. After the delayed event it continues to show up every 20 seconds.
[2010-12-09 12:07:01] Received HEARTBEAT event - we're alive.
[2010-12-09 12:07:21] Received HEARTBEAT event - we're alive.
[2010-12-09 12:07:41] Received HEARTBEAT event - we're alive.
[2010-12-09 12:08:12] Received HEARTBEAT event - we're alive.
[2010-12-09 12:08:32] Received HEARTBEAT event - we're alive.
[2010-12-09 12:08:52] Received HEARTBEAT event - we're alive.
[2010-12-09 12:09:12] Received HEARTBEAT event - we're alive.
Should the HEARTBEAT event be 100% reliable, or could something cause a delay like this? This server is not on latest GIT, but it's not more then one month old. I can debug this further myself, but I just want to know if it always should be triggered every 20 seconds, or if these events for some reasons might be delayed.
By the way - no clock skew or anything is detected on the machine, so that's not causing it.
Thanks,
Peter
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org !DSPAM:4d00c1a432762107714629!
Are you running FS with -nocal? -djbinter On Thu, Dec 9, 2010 at 5:12 AM, Peter Olsson < peter.olsson@visionutveckling.se> wrote:
Ok - I trapped the problem with full event info. The up-time also reports a difference, in this case 32 seconds. As you can see there is no load on the server at all by this time, no calls etc. Hmm, this is really strange I must say.
[2010-12-09 13:53:49] Received HEARTBEAT event - we're alive content-length == 629 content-type == text/event-plain core-uuid == 1550f368-b577-4304-bacd-2681d4067036 event-calling-file == switch_core.c event-calling-function == send_heartbeat event-calling-line-number == 65 event-date-gmt == Thu, 09 Dec 2010 12:53:53 GMT event-date-local == 2010-12-09 13:53:53 event-date-timestamp == 1291899233256303 event-info == System Ready event-name == HEARTBEAT freeswitch-hostname == 79w3sipt4 freeswitch-ipv4 == 10.66.195.14 freeswitch-ipv6 == ::1 idle-cpu == 98.437500 session-count == 0 session-per-sec == 30 session-since-startup == 277 up-time == 0 years, 7 days, 1 hour, 1 minute, 1 second, 680 milliseconds, 750 microseconds
[2010-12-09 13:54:21] Received HEARTBEAT event - we're alive content-length == 631 content-type == text/event-plain core-uuid == 1550f368-b577-4304-bacd-2681d4067036 event-calling-file == switch_core.c event-calling-function == send_heartbeat event-calling-line-number == 65 event-date-gmt == Thu, 09 Dec 2010 12:54:25 GMT event-date-local == 2010-12-09 13:54:25 event-date-timestamp == 1291899265259303 event-info == System Ready event-name == HEARTBEAT freeswitch-hostname == 79w3sipt4 freeswitch-ipv4 == 10.66.195.14 freeswitch-ipv6 == ::1 idle-cpu == 97.656250 session-count == 0 session-per-sec == 30 session-since-startup == 277 up-time == 0 years, 7 days, 1 hour, 1 minute, 33 seconds, 683 milliseconds, 750 microseconds
Mvh Peter Olsson
-----Ursprungligt meddelande----- Från: freeswitch-users-bounces@lists.freeswitch.org [mailto: freeswitch-users-bounces@lists.freeswitch.org] För Steven Ayre Skickat: den 9 december 2010 12:42 Till: FreeSWITCH Users Help Ämne: Re: [Freeswitch-users] Reliability of HEARTBEAT events every 20 seconds? (in mod_event_socket)
Also do you have any warning messages like this in the fs log? Task was executed late by %d seconds
(That should show up if the delay is in the scheduler, if the task runs >1s late).
-Steve
Hello everyone,
I have a question about the reliability of heartbeat events, it should be sent from FreeSWITCH every 20 seconds - right? If you look below in my log sample (from my own process), you can see that it comes 12:07:01, then 21, then 41, but after this it suddenly takes 31 seconds for it to show up (12:08:12). I've also done a Wireshark trace, and it shows the same thing
On 9 December 2010 11:27, Peter Olsson <peter.olsson@visionutveckling.se> wrote: -
so it is really delayed from FS. The load at the time was zero calls, and the server itself was using maybe 5% CPU. After the delayed event it continues to show up every 20 seconds.
[2010-12-09 12:07:01] Received HEARTBEAT event - we're alive.
[2010-12-09 12:07:21] Received HEARTBEAT event - we're alive.
[2010-12-09 12:07:41] Received HEARTBEAT event - we're alive.
[2010-12-09 12:08:12] Received HEARTBEAT event - we're alive.
[2010-12-09 12:08:32] Received HEARTBEAT event - we're alive.
[2010-12-09 12:08:52] Received HEARTBEAT event - we're alive.
[2010-12-09 12:09:12] Received HEARTBEAT event - we're alive.
Should the HEARTBEAT event be 100% reliable, or could something cause a delay like this? This server is not on latest GIT, but it's not more then one month old. I can debug this further myself, but I just want to know if it always should be triggered every 20 seconds, or if these events for some reasons might be delayed.
By the way - no clock skew or anything is detected on the machine, so that's not causing it.
Thanks,
Peter
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
!DSPAM:4d00c1a432762107714629!
_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
participants (3)
-
DJB International
-
Peter Olsson
-
Steven Ayre