Frustrating and inconsistent log format !
Dear FreeSWITCH Developers, Permit me a modest rant if you may. Why do log lines start with UUID ? Its format is hard to define and subject to the developer's whims ....why not start the line with a date, like almost any other sane program, and put the UUID elsewhere ? Why on earth is the log format so inconsistent ? e.g. Sometimes developers feel like giving us a date .... 7d28af7d-XXXX-XXXX-XXXX-2e20e2bXXXX 2016-01-13 12:00:47.198394 [DEBUG] switch_core_state_machine.c:636 Sometimes they even manage the date in a sensible location .... 2016-01-13 11:58:14.838392 [DEBUG] sofia.c:9124 Other times they can't be bothered to provide such a critical piece of information..... 4891efd5-XXXX-XXXX-XXXX-58d153b8XXXX Dialplan: sofia/internal/ Please devs... spend a little time cleaning up the logging ... it helps debugging and troubleshooting so much if we have some consistency there !!!!! Thanks Tim
Hello Tim, I love having the UUID in the logs. This helps a lot during debugging a system with high call volume. Thus I can simply grep the logs by UUID and get (almost) all data for this call. Best regards Peter On 01/13/16 17:06, Tim Smith wrote:
Dear FreeSWITCH Developers,
Permit me a modest rant if you may.
Why do log lines start with UUID ? Its format is hard to define and subject to the developer's whims ....why not start the line with a date, like almost any other sane program, and put the UUID elsewhere ?
Why on earth is the log format so inconsistent ? e.g.
Sometimes developers feel like giving us a date ....
7d28af7d-XXXX-XXXX-XXXX-2e20e2bXXXX 2016-01-13 12:00:47.198394 [DEBUG] switch_core_state_machine.c:636
Sometimes they even manage the date in a sensible location ....
2016-01-13 11:58:14.838392 [DEBUG] sofia.c:9124
Other times they can't be bothered to provide such a critical piece of information.....
4891efd5-XXXX-XXXX-XXXX-58d153b8XXXX Dialplan: sofia/internal/
Please devs... spend a little time cleaning up the logging ... it helps debugging and troubleshooting so much if we have some consistency there !!!!!
Thanks
Tim
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- With kind regards Peter Steinbach Telefaks Services GmbH mailto:lists (att) telefaks.de Internet: www.telefaks.de
Hello Peter, You've got the wrong end of the stick. Remove the UUIDs is not what I'm saying .... at all. ;-) Tim On 13 January 2016 at 16:22, Peter Steinbach <lists@telefaks.de> wrote:
Hello Tim,
I love having the UUID in the logs. This helps a lot during debugging a system with high call volume. Thus I can simply grep the logs by UUID and get (almost) all data for this call.
Best regards Peter
On 01/13/16 17:06, Tim Smith wrote:
Dear FreeSWITCH Developers,
Permit me a modest rant if you may.
Why do log lines start with UUID ? Its format is hard to define and subject to the developer's whims ....why not start the line with a date, like almost any other sane program, and put the UUID elsewhere ?
Why on earth is the log format so inconsistent ? e.g.
Sometimes developers feel like giving us a date ....
7d28af7d-XXXX-XXXX-XXXX-2e20e2bXXXX 2016-01-13 12:00:47.198394 [DEBUG] switch_core_state_machine.c:636
Sometimes they even manage the date in a sensible location ....
2016-01-13 11:58:14.838392 [DEBUG] sofia.c:9124
Other times they can't be bothered to provide such a critical piece of information.....
4891efd5-XXXX-XXXX-XXXX-58d153b8XXXX Dialplan: sofia/internal/
Please devs... spend a little time cleaning up the logging ... it helps debugging and troubleshooting so much if we have some consistency there !!!!!
Thanks
Tim
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- With kind regards Peter Steinbach
Telefaks Services GmbH mailto:lists (att) telefaks.de Internet: www.telefaks.de
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Hi, Its not very nice to make demands to "the devs" like they are little elves that work for you. Have you no tact? Aren't there 100 more polite ways to get your point across? You knew you were ranting because you said so. That was your brain telling you not to press send.... If you don't like uuids, you just open logfile.conf.xml and comment out the uuid param. Anything else you don't like, submit a pull request.... On Wed, Jan 13, 2016 at 10:28 AM, Tim Smith <randomdev4@gmail.com> wrote:
Hello Peter,
You've got the wrong end of the stick.
Remove the UUIDs is not what I'm saying .... at all. ;-)
Tim
On 13 January 2016 at 16:22, Peter Steinbach <lists@telefaks.de> wrote:
Hello Tim,
I love having the UUID in the logs. This helps a lot during debugging a system with high call volume. Thus I can simply grep the logs by UUID and get (almost) all data for this call.
Best regards Peter
On 01/13/16 17:06, Tim Smith wrote:
Dear FreeSWITCH Developers,
Permit me a modest rant if you may.
Why do log lines start with UUID ? Its format is hard to define and subject to the developer's whims ....why not start the line with a date, like almost any other sane program, and put the UUID elsewhere ?
Why on earth is the log format so inconsistent ? e.g.
Sometimes developers feel like giving us a date ....
7d28af7d-XXXX-XXXX-XXXX-2e20e2bXXXX 2016-01-13 12:00:47.198394 [DEBUG] switch_core_state_machine.c:636
Sometimes they even manage the date in a sensible location ....
2016-01-13 11:58:14.838392 [DEBUG] sofia.c:9124
Other times they can't be bothered to provide such a critical piece of information.....
4891efd5-XXXX-XXXX-XXXX-58d153b8XXXX Dialplan: sofia/internal/
Please devs... spend a little time cleaning up the logging ... it helps debugging and troubleshooting so much if we have some consistency there !!!!!
Thanks
Tim
Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- With kind regards Peter Steinbach
Telefaks Services GmbH mailto:lists (att) telefaks.de Internet: www.telefaks.de
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬ ☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>* ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA
There is mod_graylog2 which allows you to emit json formatted structured logs to either logstash or graylog server. It would be nice if some more support could be pushed to core, but it's not a priority for me. Chris
On Jan 13, 2016, at 11:28, Tim Smith <randomdev4@gmail.com> wrote:
Hello Peter,
You've got the wrong end of the stick.
Remove the UUIDs is not what I'm saying .... at all. ;-)
Tim
On 13 January 2016 at 16:22, Peter Steinbach <lists@telefaks.de> wrote: Hello Tim,
I love having the UUID in the logs. This helps a lot during debugging a system with high call volume. Thus I can simply grep the logs by UUID and get (almost) all data for this call.
Best regards Peter
On 01/13/16 17:06, Tim Smith wrote: Dear FreeSWITCH Developers,
Permit me a modest rant if you may.
Why do log lines start with UUID ? Its format is hard to define and subject to the developer's whims ....why not start the line with a date, like almost any other sane program, and put the UUID elsewhere ?
Why on earth is the log format so inconsistent ? e.g.
Sometimes developers feel like giving us a date ....
7d28af7d-XXXX-XXXX-XXXX-2e20e2bXXXX 2016-01-13 12:00:47.198394 [DEBUG] switch_core_state_machine.c:636
Sometimes they even manage the date in a sensible location ....
2016-01-13 11:58:14.838392 [DEBUG] sofia.c:9124
Other times they can't be bothered to provide such a critical piece of information.....
4891efd5-XXXX-XXXX-XXXX-58d153b8XXXX Dialplan: sofia/internal/
Please devs... spend a little time cleaning up the logging ... it helps debugging and troubleshooting so much if we have some consistency there !!!!!
Thanks
Tim
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- With kind regards Peter Steinbach
Telefaks Services GmbH mailto:lists (att) telefaks.de Internet: www.telefaks.de
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Ha! That is a good point. I think that also not all logs have uuids - serious errors tend not to.
Why do log lines start with UUID ? Its format is hard to define and subject to the developer's whims ....why not start the line with a date, like almost any other sane program, and put the UUID elsewhere ?
If you don't like the uuid at the front, just remove it from the logfile.conf.xml configuration <param name="uuid" value="false" /> You've clearly not attempted to debug complex scenarios, where the uuid at the front is more useful, because when you have hundreds or thousands of calls and you are trying to figure out what happened with a particular call leg, you search for the uuid first, date second. The format you despise, is terribly useful, so try to understand it rather than just hate it because is different to what you're used to.
Why on earth is the log format so inconsistent ? e.g.
Sometimes developers feel like giving us a date ....
7d28af7d-XXXX-XXXX-XXXX-2e20e2bXXXX 2016-01-13 12:00:47.198394 [DEBUG] switch_core_state_machine.c:636
Sometimes they even manage the date in a sensible location ....
2016-01-13 11:58:14.838392 [DEBUG] sofia.c:9124
Other times they can't be bothered to provide such a critical piece of information.....
4891efd5-XXXX-XXXX-XXXX-58d153b8XXXX Dialplan: sofia/internal/
This is by design, not a whim, and many of us like it, makes routing logs easy to spot. Some people instead of calling these design decisions whims, actually provided a solution so it fits what they prefer: https://freeswitch.org/jira/browse/FS-6805 https://freeswitch.org/stash/projects/FS/repos/freeswitch/pull-requests/44/o... So you can add this to switch.conf.xml: <param name="dialplan-timestamps" value="true"/> And you get the behavior you want. Please spend the time learning the software rather than blaming the devs because you can't understand it or because they have preferences different than yours. Please devs... spend a little time cleaning up the logging ... it
helps debugging and troubleshooting so much if we have some consistency there !!!!!
Consistency is good, but certain functionality cannot be provided if every single line has exactly the same format. Some log lines are not associated to a session, so they can't have a uuid at all. The FreeSWITCH logs are as consistent as is possible in a given context.
Moises, Thank you for your reply, you're the only person who read my post and didn't somehow think I was saying "remove the UUIDs" ! ;-) Regarding : "Consistency is good, but certain functionality cannot be provided if every single line has exactly the same format." Yes, but the inconsistency makes it painful, verging on impossible, to correctly integrate with various log parsing systems, whether Splunk, Logstash or otherwise. If the devs have some personal preference for one format here, another format there, and something else over there, I would unfortunatley still call that a whim, especially as there is no clear documentation on the matter either !!!! Even the Freeswitch docs don't know what's what ... " The log file will contain log lines in this general format: YYYY-MM-DD hh:mm:ss [LEVEL] source_file_name:line_number function_name() <log data>" , that statement is not true. That's not the general format. its not even the dominant format, infact in my logs, lines that start YYYY-MM-DD are a rarity ! I have not come accross any other product (either closed or open source) that has the quite same amount of randomness in the way it logs. That is what I find so frustrating. On 14 January 2016 at 07:05, Moises Silva <moises.silva@gmail.com> wrote:
Why do log lines start with UUID ? Its format is hard to define and subject to the developer's whims ....why not start the line with a date, like almost any other sane program, and put the UUID elsewhere ?
If you don't like the uuid at the front, just remove it from the logfile.conf.xml configuration
<param name="uuid" value="false" />
You've clearly not attempted to debug complex scenarios, where the uuid at the front is more useful, because when you have hundreds or thousands of calls and you are trying to figure out what happened with a particular call leg, you search for the uuid first, date second.
The format you despise, is terribly useful, so try to understand it rather than just hate it because is different to what you're used to.
Why on earth is the log format so inconsistent ? e.g.
Sometimes developers feel like giving us a date ....
7d28af7d-XXXX-XXXX-XXXX-2e20e2bXXXX 2016-01-13 12:00:47.198394 [DEBUG] switch_core_state_machine.c:636
Sometimes they even manage the date in a sensible location ....
2016-01-13 11:58:14.838392 [DEBUG] sofia.c:9124
Other times they can't be bothered to provide such a critical piece of information.....
4891efd5-XXXX-XXXX-XXXX-58d153b8XXXX Dialplan: sofia/internal/
This is by design, not a whim, and many of us like it, makes routing logs easy to spot. Some people instead of calling these design decisions whims, actually provided a solution so it fits what they prefer:
https://freeswitch.org/jira/browse/FS-6805 https://freeswitch.org/stash/projects/FS/repos/freeswitch/pull-requests/44/o...
So you can add this to switch.conf.xml:
<param name="dialplan-timestamps" value="true"/>
And you get the behavior you want. Please spend the time learning the software rather than blaming the devs because you can't understand it or because they have preferences different than yours.
Please devs... spend a little time cleaning up the logging ... it helps debugging and troubleshooting so much if we have some consistency there !!!!!
Consistency is good, but certain functionality cannot be provided if every single line has exactly the same format. Some log lines are not associated to a session, so they can't have a uuid at all. The FreeSWITCH logs are as consistent as is possible in a given context.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
If you don't like the way the things are made, just change it. You are always welcome to help instead of whining. Gustavo Sent from my iPhone
On Jan 14, 2016, at 7:12 AM, Tim Smith <randomdev4@gmail.com> wrote:
Moises,
Thank you for your reply, you're the only person who read my post and didn't somehow think I was saying "remove the UUIDs" ! ;-)
Regarding : "Consistency is good, but certain functionality cannot be provided if every single line has exactly the same format."
Yes, but the inconsistency makes it painful, verging on impossible, to correctly integrate with various log parsing systems, whether Splunk, Logstash or otherwise.
If the devs have some personal preference for one format here, another format there, and something else over there, I would unfortunatley still call that a whim, especially as there is no clear documentation on the matter either !!!!
Even the Freeswitch docs don't know what's what ... " The log file will contain log lines in this general format: YYYY-MM-DD hh:mm:ss [LEVEL] source_file_name:line_number function_name() <log data>" , that statement is not true. That's not the general format. its not even the dominant format, infact in my logs, lines that start YYYY-MM-DD are a rarity !
I have not come accross any other product (either closed or open source) that has the quite same amount of randomness in the way it logs. That is what I find so frustrating.
On 14 January 2016 at 07:05, Moises Silva <moises.silva@gmail.com> wrote:
Why do log lines start with UUID ? Its format is hard to define and subject to the developer's whims ....why not start the line with a date, like almost any other sane program, and put the UUID elsewhere ?
If you don't like the uuid at the front, just remove it from the logfile.conf.xml configuration
<param name="uuid" value="false" />
You've clearly not attempted to debug complex scenarios, where the uuid at the front is more useful, because when you have hundreds or thousands of calls and you are trying to figure out what happened with a particular call leg, you search for the uuid first, date second.
The format you despise, is terribly useful, so try to understand it rather than just hate it because is different to what you're used to.
Why on earth is the log format so inconsistent ? e.g.
Sometimes developers feel like giving us a date ....
7d28af7d-XXXX-XXXX-XXXX-2e20e2bXXXX 2016-01-13 12:00:47.198394 [DEBUG] switch_core_state_machine.c:636
Sometimes they even manage the date in a sensible location ....
2016-01-13 11:58:14.838392 [DEBUG] sofia.c:9124
Other times they can't be bothered to provide such a critical piece of information.....
4891efd5-XXXX-XXXX-XXXX-58d153b8XXXX Dialplan: sofia/internal/
This is by design, not a whim, and many of us like it, makes routing logs easy to spot. Some people instead of calling these design decisions whims, actually provided a solution so it fits what they prefer:
https://freeswitch.org/jira/browse/FS-6805 https://freeswitch.org/stash/projects/FS/repos/freeswitch/pull-requests/44/o...
So you can add this to switch.conf.xml:
<param name="dialplan-timestamps" value="true"/>
And you get the behavior you want. Please spend the time learning the software rather than blaming the devs because you can't understand it or because they have preferences different than yours.
Please devs... spend a little time cleaning up the logging ... it helps debugging and troubleshooting so much if we have some consistency there !!!!!
Consistency is good, but certain functionality cannot be provided if every single line has exactly the same format. Some log lines are not associated to a session, so they can't have a uuid at all. The FreeSWITCH logs are as consistent as is possible in a given context.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
One good suggestion that I read is to use a perl (or similar) script to intercept and parse the log lines, then write them in a format of your liking. This gives you ultimate control over how the log lines appear. Bote
-----Original Message----- From: Tim Smith Sent: Thursday, 14 January, 2016 05:12 Subject: Re: [Freeswitch-users] Frustrating and inconsistent log format !
Moises,
Thank you for your reply, you're the only person who read my post and didn't somehow think I was saying "remove the UUIDs" ! ;-)
Regarding : "Consistency is good, but certain functionality cannot be provided if every single line has exactly the same format."
Yes, but the inconsistency makes it painful, verging on impossible, to correctly integrate with various log parsing systems, whether Splunk, Logstash or otherwise.
The Logs from FreeSWITCH are quite consistent... UUID, TimeDate Stamp, loglevel, file:line_num message. If the log line is not directly associated with a session the UUID is omitted. I'm not sure how you can call this random and inconsistent. Now you may be confused by additional logging that may be enabled such as the logging from maybe libsofia, or one of the other many libs that FreeSWITCH uses. FreeSWITCH has no control over underlying library logging formats. An example of this is if you have sofia global siptrace enabled. The formatting there is directly from the library and we have no control over it other than to enabled or disable it. This is common with libraries not just for FreeSWITCH but many other open source projects... As with pretty much everything in FreeSWITCH it's a module, don't like the logging modules that are there now you are welcome and encouraged to create your own, and patches are accepted on a regular basis. At this time I will consider this thread closed. -----Original Message----- From: freeswitch-users-bounces@lists.freeswitch.org [mailto:freeswitch-users-bounces@lists.freeswitch.org] On Behalf Of Tim Smith Sent: Thursday, January 14, 2016 4:12 AM To: FreeSWITCH Users Help <freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] Frustrating and inconsistent log format ! Moises, Thank you for your reply, you're the only person who read my post and didn't somehow think I was saying "remove the UUIDs" ! ;-) Regarding : "Consistency is good, but certain functionality cannot be provided if every single line has exactly the same format." Yes, but the inconsistency makes it painful, verging on impossible, to correctly integrate with various log parsing systems, whether Splunk, Logstash or otherwise. If the devs have some personal preference for one format here, another format there, and something else over there, I would unfortunatley still call that a whim, especially as there is no clear documentation on the matter either !!!! Even the Freeswitch docs don't know what's what ... " The log file will contain log lines in this general format: YYYY-MM-DD hh:mm:ss [LEVEL] source_file_name:line_number function_name() <log data>" , that statement is not true. That's not the general format. its not even the dominant format, infact in my logs, lines that start YYYY-MM-DD are a rarity ! I have not come accross any other product (either closed or open source) that has the quite same amount of randomness in the way it logs. That is what I find so frustrating. On 14 January 2016 at 07:05, Moises Silva <moises.silva@gmail.com> wrote:
Why do log lines start with UUID ? Its format is hard to define and subject to the developer's whims ....why not start the line with a date, like almost any other sane program, and put the UUID elsewhere ?
If you don't like the uuid at the front, just remove it from the logfile.conf.xml configuration
<param name="uuid" value="false" />
You've clearly not attempted to debug complex scenarios, where the uuid at the front is more useful, because when you have hundreds or thousands of calls and you are trying to figure out what happened with a particular call leg, you search for the uuid first, date second.
The format you despise, is terribly useful, so try to understand it rather than just hate it because is different to what you're used to.
Why on earth is the log format so inconsistent ? e.g.
Sometimes developers feel like giving us a date ....
7d28af7d-XXXX-XXXX-XXXX-2e20e2bXXXX 2016-01-13 12:00:47.198394 [DEBUG] switch_core_state_machine.c:636
Sometimes they even manage the date in a sensible location ....
2016-01-13 11:58:14.838392 [DEBUG] sofia.c:9124
Other times they can't be bothered to provide such a critical piece of information.....
4891efd5-XXXX-XXXX-XXXX-58d153b8XXXX Dialplan: sofia/internal/
This is by design, not a whim, and many of us like it, makes routing logs easy to spot. Some people instead of calling these design decisions whims, actually provided a solution so it fits what they prefer:
https://freeswitch.org/jira/browse/FS-6805 https://freeswitch.org/stash/projects/FS/repos/freeswitch/pull-request s/44/overview
So you can add this to switch.conf.xml:
<param name="dialplan-timestamps" value="true"/>
And you get the behavior you want. Please spend the time learning the software rather than blaming the devs because you can't understand it or because they have preferences different than yours.
Please devs... spend a little time cleaning up the logging ... it helps debugging and troubleshooting so much if we have some consistency there !!!!!
Consistency is good, but certain functionality cannot be provided if every single line has exactly the same format. Some log lines are not associated to a session, so they can't have a uuid at all. The FreeSWITCH logs are as consistent as is possible in a given context.
______________________________________________________________________ ___ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-use rs http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
One thing that might also puzzle Tim the 'clean' logging format, eg SWITCH_CHANNEL_ID_LOG_CLEAN vs SWITCH_CHANNEL_ID_LOG which has the uuid but lacks the datetime. @Tim should be possible to make both UUID and date/time optional easily with a regex. On 17 January 2016 at 16:43, Ken Rice <krice@freeswitch.org> wrote:
The Logs from FreeSWITCH are quite consistent...
UUID, TimeDate Stamp, loglevel, file:line_num message.
If the log line is not directly associated with a session the UUID is omitted.
I'm not sure how you can call this random and inconsistent.
Now you may be confused by additional logging that may be enabled such as the logging from maybe libsofia, or one of the other many libs that FreeSWITCH uses. FreeSWITCH has no control over underlying library logging formats. An example of this is if you have sofia global siptrace enabled. The formatting there is directly from the library and we have no control over it other than to enabled or disable it.
This is common with libraries not just for FreeSWITCH but many other open source projects...
As with pretty much everything in FreeSWITCH it's a module, don't like the logging modules that are there now you are welcome and encouraged to create your own, and patches are accepted on a regular basis.
At this time I will consider this thread closed.
-----Original Message----- From: freeswitch-users-bounces@lists.freeswitch.org [mailto:freeswitch-users-bounces@lists.freeswitch.org] On Behalf Of Tim Smith Sent: Thursday, January 14, 2016 4:12 AM To: FreeSWITCH Users Help <freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] Frustrating and inconsistent log format !
Moises,
Thank you for your reply, you're the only person who read my post and didn't somehow think I was saying "remove the UUIDs" ! ;-)
Regarding : "Consistency is good, but certain functionality cannot be provided if every single line has exactly the same format."
Yes, but the inconsistency makes it painful, verging on impossible, to correctly integrate with various log parsing systems, whether Splunk, Logstash or otherwise.
If the devs have some personal preference for one format here, another format there, and something else over there, I would unfortunatley still call that a whim, especially as there is no clear documentation on the matter either !!!!
Even the Freeswitch docs don't know what's what ... " The log file will contain log lines in this general format: YYYY-MM-DD hh:mm:ss [LEVEL] source_file_name:line_number function_name() <log data>" , that statement is not true. That's not the general format. its not even the dominant format, infact in my logs, lines that start YYYY-MM-DD are a rarity !
I have not come accross any other product (either closed or open source) that has the quite same amount of randomness in the way it logs. That is what I find so frustrating.
On 14 January 2016 at 07:05, Moises Silva <moises.silva@gmail.com> wrote:
Why do log lines start with UUID ? Its format is hard to define and subject to the developer's whims ....why not start the line with a date, like almost any other sane program, and put the UUID elsewhere ?
If you don't like the uuid at the front, just remove it from the logfile.conf.xml configuration
<param name="uuid" value="false" />
You've clearly not attempted to debug complex scenarios, where the uuid at the front is more useful, because when you have hundreds or thousands of calls and you are trying to figure out what happened with a particular call leg, you search for the uuid first, date second.
The format you despise, is terribly useful, so try to understand it rather than just hate it because is different to what you're used to.
Why on earth is the log format so inconsistent ? e.g.
Sometimes developers feel like giving us a date ....
7d28af7d-XXXX-XXXX-XXXX-2e20e2bXXXX 2016-01-13 12:00:47.198394 [DEBUG] switch_core_state_machine.c:636
Sometimes they even manage the date in a sensible location ....
2016-01-13 11:58:14.838392 [DEBUG] sofia.c:9124
Other times they can't be bothered to provide such a critical piece of information.....
4891efd5-XXXX-XXXX-XXXX-58d153b8XXXX Dialplan: sofia/internal/
This is by design, not a whim, and many of us like it, makes routing logs easy to spot. Some people instead of calling these design decisions whims, actually provided a solution so it fits what they prefer:
https://freeswitch.org/jira/browse/FS-6805 https://freeswitch.org/stash/projects/FS/repos/freeswitch/pull-request s/44/overview
So you can add this to switch.conf.xml:
<param name="dialplan-timestamps" value="true"/>
And you get the behavior you want. Please spend the time learning the software rather than blaming the devs because you can't understand it or because they have preferences different than yours.
Please devs... spend a little time cleaning up the logging ... it helps debugging and troubleshooting so much if we have some consistency there !!!!!
Consistency is good, but certain functionality cannot be provided if every single line has exactly the same format. Some log lines are not associated to a session, so they can't have a uuid at all. The FreeSWITCH logs are as consistent as is possible in a given context.
______________________________________________________________________ ___ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-use rs http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Even the Freeswitch docs don't know what's what ... " The log file will contain log lines in this general format: YYYY-MM-DD hh:mm:ss [LEVEL] source_file_name:line_number function_name() <log data>" , that statement is not true. That's not the general format. its not even the dominant format, infact in my logs, lines that start
YYYY-MM-DD are a rarity ! The dominant logging format is going to be the session log as most messages will relate to a call. So the majority of log lines will start with the UUID, if that is enabled. The aren't that many logging formats. There is logging against a session (has UUID) or a global log (no UUID). Both have clean variants (no datetime information). So both UUID and datetime are optional. Other than that it's consistent. Anything else is likely to come from libraries used by FreeSWITCH that don't go through that logging system. On 14 January 2016 at 10:12, Tim Smith <randomdev4@gmail.com> wrote:
Moises,
Thank you for your reply, you're the only person who read my post and didn't somehow think I was saying "remove the UUIDs" ! ;-)
Regarding : "Consistency is good, but certain functionality cannot be provided if every single line has exactly the same format."
Yes, but the inconsistency makes it painful, verging on impossible, to correctly integrate with various log parsing systems, whether Splunk, Logstash or otherwise.
If the devs have some personal preference for one format here, another format there, and something else over there, I would unfortunatley still call that a whim, especially as there is no clear documentation on the matter either !!!!
Even the Freeswitch docs don't know what's what ... " The log file will contain log lines in this general format: YYYY-MM-DD hh:mm:ss [LEVEL] source_file_name:line_number function_name() <log data>" , that statement is not true. That's not the general format. its not even the dominant format, infact in my logs, lines that start YYYY-MM-DD are a rarity !
I have not come accross any other product (either closed or open source) that has the quite same amount of randomness in the way it logs. That is what I find so frustrating.
On 14 January 2016 at 07:05, Moises Silva <moises.silva@gmail.com> wrote:
Why do log lines start with UUID ? Its format is hard to define and subject to the developer's whims ....why not start the line with a date, like almost any other sane program, and put the UUID elsewhere ?
If you don't like the uuid at the front, just remove it from the logfile.conf.xml configuration
<param name="uuid" value="false" />
You've clearly not attempted to debug complex scenarios, where the uuid at the front is more useful, because when you have hundreds or thousands of calls and you are trying to figure out what happened with a particular call leg, you search for the uuid first, date second.
The format you despise, is terribly useful, so try to understand it rather than just hate it because is different to what you're used to.
Why on earth is the log format so inconsistent ? e.g.
Sometimes developers feel like giving us a date ....
7d28af7d-XXXX-XXXX-XXXX-2e20e2bXXXX 2016-01-13 12:00:47.198394 [DEBUG] switch_core_state_machine.c:636
Sometimes they even manage the date in a sensible location ....
2016-01-13 11:58:14.838392 [DEBUG] sofia.c:9124
Other times they can't be bothered to provide such a critical piece of information.....
4891efd5-XXXX-XXXX-XXXX-58d153b8XXXX Dialplan: sofia/internal/
This is by design, not a whim, and many of us like it, makes routing logs easy to spot. Some people instead of calling these design decisions whims, actually provided a solution so it fits what they prefer:
https://freeswitch.org/stash/projects/FS/repos/freeswitch/pull-requests/44/o...
So you can add this to switch.conf.xml:
<param name="dialplan-timestamps" value="true"/>
And you get the behavior you want. Please spend the time learning the software rather than blaming the devs because you can't understand it or because they have preferences different than yours.
Please devs... spend a little time cleaning up the logging ... it helps debugging and troubleshooting so much if we have some consistency there !!!!!
Consistency is good, but certain functionality cannot be provided if
every
single line has exactly the same format. Some log lines are not associated to a session, so they can't have a uuid at all. The FreeSWITCH logs are as consistent as is possible in a given context.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Steven, Spot on. Also some pro tips on logging. 1) You can use ESL or a C module to do logging as you see fit. You get a callback with all the pieces of the data and you can write any format you want. fs_cli does this via ESL and mod_logfile does it via C. 2) There is another module called mod_syslog that will write to syslog also prepending the date to everything. 3) The mod_graylog that Chris mentioned sounds pretty interesting and is also taking advantage of the raw data. On Mon, Jan 18, 2016 at 7:09 AM, Steven Ayre <steveayre@gmail.com> wrote:
Even the Freeswitch docs don't know what's what ... " The log file
will contain log lines in this general format: YYYY-MM-DD hh:mm:ss [LEVEL] source_file_name:line_number function_name() <log data>" , that statement is not true. That's not the general format. its not even the dominant format, infact in my logs, lines that start
YYYY-MM-DD are a rarity !
The dominant logging format is going to be the session log as most messages will relate to a call. So the majority of log lines will start with the UUID, if that is enabled.
The aren't that many logging formats. There is logging against a session (has UUID) or a global log (no UUID). Both have clean variants (no datetime information). So both UUID and datetime are optional. Other than that it's consistent.
Anything else is likely to come from libraries used by FreeSWITCH that don't go through that logging system.
On 14 January 2016 at 10:12, Tim Smith <randomdev4@gmail.com> wrote:
Moises,
Thank you for your reply, you're the only person who read my post and didn't somehow think I was saying "remove the UUIDs" ! ;-)
Regarding : "Consistency is good, but certain functionality cannot be provided if every single line has exactly the same format."
Yes, but the inconsistency makes it painful, verging on impossible, to correctly integrate with various log parsing systems, whether Splunk, Logstash or otherwise.
If the devs have some personal preference for one format here, another format there, and something else over there, I would unfortunatley still call that a whim, especially as there is no clear documentation on the matter either !!!!
Even the Freeswitch docs don't know what's what ... " The log file will contain log lines in this general format: YYYY-MM-DD hh:mm:ss [LEVEL] source_file_name:line_number function_name() <log data>" , that statement is not true. That's not the general format. its not even the dominant format, infact in my logs, lines that start YYYY-MM-DD are a rarity !
I have not come accross any other product (either closed or open source) that has the quite same amount of randomness in the way it logs. That is what I find so frustrating.
On 14 January 2016 at 07:05, Moises Silva <moises.silva@gmail.com> wrote:
Why do log lines start with UUID ? Its format is hard to define and subject to the developer's whims ....why not start the line with a date, like almost any other sane program, and put the UUID elsewhere ?
If you don't like the uuid at the front, just remove it from the logfile.conf.xml configuration
<param name="uuid" value="false" />
You've clearly not attempted to debug complex scenarios, where the uuid at the front is more useful, because when you have hundreds or thousands of calls and you are trying to figure out what happened with a particular call leg, you search for the uuid first, date second.
The format you despise, is terribly useful, so try to understand it rather than just hate it because is different to what you're used to.
Why on earth is the log format so inconsistent ? e.g.
Sometimes developers feel like giving us a date ....
7d28af7d-XXXX-XXXX-XXXX-2e20e2bXXXX 2016-01-13 12:00:47.198394 [DEBUG] switch_core_state_machine.c:636
Sometimes they even manage the date in a sensible location ....
2016-01-13 11:58:14.838392 [DEBUG] sofia.c:9124
Other times they can't be bothered to provide such a critical piece of information.....
4891efd5-XXXX-XXXX-XXXX-58d153b8XXXX Dialplan: sofia/internal/
This is by design, not a whim, and many of us like it, makes routing logs easy to spot. Some people instead of calling these design decisions whims, actually provided a solution so it fits what they prefer:
https://freeswitch.org/stash/projects/FS/repos/freeswitch/pull-requests/44/o...
So you can add this to switch.conf.xml:
<param name="dialplan-timestamps" value="true"/>
And you get the behavior you want. Please spend the time learning the software rather than blaming the devs because you can't understand it or because they have preferences different than yours.
Please devs... spend a little time cleaning up the logging ... it helps debugging and troubleshooting so much if we have some consistency there !!!!!
Consistency is good, but certain functionality cannot be provided if
every
single line has exactly the same format. Some log lines are not associated to a session, so they can't have a uuid at all. The FreeSWITCH logs are as consistent as is possible in a given context.
_________________________________________________________________________
Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II ♬ @anthmfs ♬ @FreeSWITCH ♬ ☞ http://freeswitch.org/ ☞ http://cluecon.com/ ☞ http://twitter.com/FreeSWITCH ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+ <http://freeswitch.org/g+>* ClueCon Weekly Development Call ☎ sip:888@conference.freeswitch.org ☎ +19193869900 https://www.youtube.com/watch?v=9XXgW34t40s https://www.youtube.com/watch?v=NLaDpGQuZDA
participants (10)
-
Alex Lake
-
Anthony Minessale
-
Bote Man
-
cmrienzo＠gmail.com
-
Ken Rice
-
Moises Silva
-
Peter Steinbach
-
servtelar＠gmail.com
-
Steven Ayre
-
Tim Smith