Module Loading Timeout Exit on Start-up
Hi all, Just a quick question before I start searching through the source. On FS start-up, there seems to be a 30 second timeout FS allows for modules to load, after which FS will exit. I appreciate that modules should take less than this to load, but I'm running into this timeout mainly during development, for various reasons, including when debugging the loading logic of the module (hence debugging has become a bit of a race!). Is there any way to disable or extend this module load timeout? Thanks in advance, Shane Mitchell
I can’t think of what that might be. Do you get any logs when that fails to point me towards what you are hitting? The modules loading is done in the main thread from the executable and we don’t have anything I can think of that times out there.
On Feb 7, 2017, at 4:34 PM, Shane Mitchell <Shane.Mitchell@fonedynamics.com> wrote:
Hi all,
Just a quick question before I start searching through the source.
On FS start-up, there seems to be a 30 second timeout FS allows for modules to load, after which FS will exit. I appreciate that modules should take less than this to load, but I'm running into this timeout mainly during development, for various reasons, including when debugging the loading logic of the module (hence debugging has become a bit of a race!).
Is there any way to disable or extend this module load timeout?
Thanks in advance,
Shane Mitchell
Hi Michael, The module is written in C# using mod_managed. I have created a barebones CSX file to illustrate the issue, available here: https://pastebin.freeswitch.org/view/a9e03c6e The last part of the console log is shown here: https://pastebin.freeswitch.org/view/0f2a230f The test should count up to 60, but only counts to 30 before the process crashes. In Windows Event viewer, the following information is displayed: Faulting application name: FreeSwitchConsole.exe, version: 1.5.15.0, time stamp: 0x54ace818 Faulting module name: MSVCR100.dll, version: 10.0.40219.325, time stamp: 0x4df2bcac Exception code: 0x40000015 Fault offset: 0x00000000000761c9 Faulting process id: 0xcea8 Faulting application start time: 0x01d281a878a2c8c2 Faulting application path: C:\Program Files\FreeSWITCH\FreeSwitchConsole.exe Faulting module path: C:\Windows\system32\MSVCR100.dll I'm not sure if this is isolated to mod_managed / CLR, or Windows, or if this occurs more broadly. Any ideas? Cheers, Shane Mitchell -----Original Message----- From: freeswitch-users-bounces@lists.freeswitch.org [mailto:freeswitch-users-bounces@lists.freeswitch.org] On Behalf Of Michael Jerris Sent: Wednesday, 8 February 2017 7:42 AM To: FreeSWITCH Users Help <freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] Module Loading Timeout Exit on Start-up I can’t think of what that might be. Do you get any logs when that fails to point me towards what you are hitting? The modules loading is done in the main thread from the executable and we don’t have anything I can think of that times out there.
On Feb 7, 2017, at 4:34 PM, Shane Mitchell wrote:
Hi all,
Just a quick question before I start searching through the source.
On FS start-up, there seems to be a 30 second timeout FS allows for modules to load, after which FS will exit. I appreciate that modules should take less than this to load, but I'm running into this timeout mainly during development, for various reasons, including when debugging the loading logic of the module (hence debugging has become a bit of a race!).
Is there any way to disable or extend this module load timeout?
Thanks in advance,
Shane Mitchell
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Can you catch the crash in the debugger and see why it's crashing? On Tue, Feb 7, 2017 at 8:28 PM Shane Mitchell < shane.mitchell@fonedynamics.com.au> wrote:
Hi Michael,
The module is written in C# using mod_managed. I have created a barebones CSX file to illustrate the issue, available here: https://pastebin.freeswitch.org/view/a9e03c6e
The last part of the console log is shown here: https://pastebin.freeswitch.org/view/0f2a230f
The test should count up to 60, but only counts to 30 before the process crashes.
In Windows Event viewer, the following information is displayed:
Faulting application name: FreeSwitchConsole.exe, version: 1.5.15.0, time stamp: 0x54ace818 Faulting module name: MSVCR100.dll, version: 10.0.40219.325, time stamp: 0x4df2bcac Exception code: 0x40000015 Fault offset: 0x00000000000761c9 Faulting process id: 0xcea8 Faulting application start time: 0x01d281a878a2c8c2 Faulting application path: C:\Program Files\FreeSWITCH\FreeSwitchConsole.exe Faulting module path: C:\Windows\system32\MSVCR100.dll
I'm not sure if this is isolated to mod_managed / CLR, or Windows, or if this occurs more broadly.
Any ideas?
Cheers,
Shane Mitchell
-----Original Message----- From: freeswitch-users-bounces@lists.freeswitch.org [mailto: freeswitch-users-bounces@lists.freeswitch.org] On Behalf Of Michael Jerris Sent: Wednesday, 8 February 2017 7:42 AM To: FreeSWITCH Users Help <freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] Module Loading Timeout Exit on Start-up
I can’t think of what that might be. Do you get any logs when that fails to point me towards what you are hitting? The modules loading is done in the main thread from the executable and we don’t have anything I can think of that times out there.
On Feb 7, 2017, at 4:34 PM, Shane Mitchell wrote:
Hi all,
Just a quick question before I start searching through the source.
On FS start-up, there seems to be a 30 second timeout FS allows for modules to load, after which FS will exit. I appreciate that modules should take less than this to load, but I'm running into this timeout mainly during development, for various reasons, including when debugging the loading logic of the module (hence debugging has become a bit of a race!).
Is there any way to disable or extend this module load timeout?
Thanks in advance,
Shane Mitchell
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Hi Michael, Here we go - latest master switch_time.c:703 has a delay while it waits for globals.STARTED, which times out after 30 seconds with an abort. while (globals.STARTED == 0) { do_sleep(100000); if (++sanity == 300) { abort(); } } Does it make sense to abort? If a module did get "stuck" loading that would cause FS to never start, whereas with the timeout FS would abort and the service manager would probably attempt a restart in the hope that it would work next time. Perhaps the sanity check could be a pre-processor option? Kind regards, Shane Mitchell. From: freeswitch-users-bounces@lists.freeswitch.org [mailto:freeswitch-users-bounces@lists.freeswitch.org] On Behalf Of Michael Jerris Sent: Wednesday, 8 February 2017 2:22 PM To: FreeSWITCH Users Help <freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] Module Loading Timeout Exit on Start-up Can you catch the crash in the debugger and see why it's crashing? On Tue, Feb 7, 2017 at 8:28 PM Shane Mitchellwrote: Hi Michael, The module is written in C# using mod_managed. I have created a barebones CSX file to illustrate the issue, available here: https://pastebin.freeswitch.org/view/a9e03c6e The last part of the console log is shown here: https://pastebin.freeswitch.org/view/0f2a230f The test should count up to 60, but only counts to 30 before the process crashes. In Windows Event viewer, the following information is displayed: Faulting application name: FreeSwitchConsole.exe, version: 1.5.15.0, time stamp: 0x54ace818 Faulting module name: MSVCR100.dll, version: 10.0.40219.325, time stamp: 0x4df2bcac Exception code: 0x40000015 Fault offset: 0x00000000000761c9 Faulting process id: 0xcea8 Faulting application start time: 0x01d281a878a2c8c2 Faulting application path: C:\Program Files\FreeSWITCH\FreeSwitchConsole.exe Faulting module path: C:\Windows\system32\MSVCR100.dll I'm not sure if this is isolated to mod_managed / CLR, or Windows, or if this occurs more broadly. Any ideas? Cheers, Shane Mitchell -----Original Message----- From: mailto:freeswitch-users-bounces@lists.freeswitch.org [mailto:mailto:freeswitch-users-bounces@lists.freeswitch.org] On Behalf Of Michael Jerris Sent: Wednesday, 8 February 2017 7:42 AM To: FreeSWITCH Users Help <mailto:freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] Module Loading Timeout Exit on Start-up I can’t think of what that might be. Do you get any logs when that fails to point me towards what you are hitting? The modules loading is done in the main thread from the executable and we don’t have anything I can think of that times out there.
On Feb 7, 2017, at 4:34 PM, Shane Mitchell wrote:
Hi all,
Just a quick question before I start searching through the source.
On FS start-up, there seems to be a 30 second timeout FS allows for modules to load, after which FS will exit. I appreciate that modules should take less than this to load, but I'm running into this timeout mainly during development, for various reasons, including when debugging the loading logic of the module (hence debugging has become a bit of a race!).
Is there any way to disable or extend this module load timeout?
Thanks in advance,
Shane Mitchell
FreeSWITCH-users mailing list mailto:FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org _________________________________________________________________________ Professional FreeSWITCH Consulting Services: mailto:consulting@freeswitch.org http://www.freeswitchsolutions.com Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com FreeSWITCH-users mailing list mailto:FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
given this is the first time I’ve ever seen an issue with it, I’m not particularly concerned with this behavior. If your trying to debug something and you are hitting it you could just comment it out.
On Feb 8, 2017, at 10:37 AM, Shane Mitchell <shane.mitchell@fonedynamics.com.au> wrote:
Hi Michael,
Here we go - latest master switch_time.c:703 has a delay while it waits for globals.STARTED, which times out after 30 seconds with an abort.
while (globals.STARTED == 0) { do_sleep(100000); if (++sanity == 300) { abort(); } }
Does it make sense to abort? If a module did get "stuck" loading that would cause FS to never start, whereas with the timeout FS would abort and the service manager would probably attempt a restart in the hope that it would work next time.
Perhaps the sanity check could be a pre-processor option?
Kind regards, Shane Mitchell.
From: freeswitch-users-bounces@lists.freeswitch.org [mailto:freeswitch-users-bounces@lists.freeswitch.org] On Behalf Of Michael Jerris Sent: Wednesday, 8 February 2017 2:22 PM To: FreeSWITCH Users Help <freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] Module Loading Timeout Exit on Start-up
Can you catch the crash in the debugger and see why it's crashing?
On Tue, Feb 7, 2017 at 8:28 PM Shane Mitchellwrote: Hi Michael,
The module is written in C# using mod_managed. I have created a barebones CSX file to illustrate the issue, available here: https://pastebin.freeswitch.org/view/a9e03c6e
The last part of the console log is shown here: https://pastebin.freeswitch.org/view/0f2a230f
The test should count up to 60, but only counts to 30 before the process crashes.
In Windows Event viewer, the following information is displayed:
Faulting application name: FreeSwitchConsole.exe, version: 1.5.15.0, time stamp: 0x54ace818 Faulting module name: MSVCR100.dll, version: 10.0.40219.325, time stamp: 0x4df2bcac Exception code: 0x40000015 Fault offset: 0x00000000000761c9 Faulting process id: 0xcea8 Faulting application start time: 0x01d281a878a2c8c2 Faulting application path: C:\Program Files\FreeSWITCH\FreeSwitchConsole.exe Faulting module path: C:\Windows\system32\MSVCR100.dll
I'm not sure if this is isolated to mod_managed / CLR, or Windows, or if this occurs more broadly.
Any ideas?
Cheers,
Shane Mitchell
-----Original Message----- From: mailto:freeswitch-users-bounces@lists.freeswitch.org [mailto:mailto:freeswitch-users-bounces@lists.freeswitch.org] On Behalf Of Michael Jerris Sent: Wednesday, 8 February 2017 7:42 AM To: FreeSWITCH Users Help <mailto:freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] Module Loading Timeout Exit on Start-up
I can’t think of what that might be. Do you get any logs when that fails to point me towards what you are hitting? The modules loading is done in the main thread from the executable and we don’t have anything I can think of that times out there.
On Feb 7, 2017, at 4:34 PM, Shane Mitchell wrote:
Hi all,
Just a quick question before I start searching through the source.
On FS start-up, there seems to be a 30 second timeout FS allows for modules to load, after which FS will exit. I appreciate that modules should take less than this to load, but I'm running into this timeout mainly during development, for various reasons, including when debugging the loading logic of the module (hence debugging has become a bit of a race!).
Is there any way to disable or extend this module load timeout?
Thanks in advance,
Shane Mitchell
FreeSWITCH-users mailing list mailto:FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org _________________________________________________________________________ Professional FreeSWITCH Consulting Services: mailto:consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list mailto:FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
participants (3)
-
Michael Jerris
-
Shane Mitchell
-
Shane Mitchell