You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by "John Vines (Created) (JIRA)" <ji...@apache.org> on 2012/02/15 22:51:59 UTC
[jira] [Created] (ACCUMULO-407) Look into on the fly log4j
configuration
Look into on the fly log4j configuration
----------------------------------------
Key: ACCUMULO-407
URL: https://issues.apache.org/jira/browse/ACCUMULO-407
Project: Accumulo
Issue Type: Improvement
Reporter: John Vines
Assignee: Josh Elser
For long running systems, logs may not want to be kept at the debug level 24/7. But there may be times where a single long query may be want to looked into without cycling the entire systems. We think it may be possible to make log4j configurable on fly, so lets start by looking into how difficult it will be.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Commented] (ACCUMULO-407) Look into on the fly log4j configuration
Posted by John Vines <jo...@ugov.gov>.
Loading out of hdfs would be easier in 1.5 with the plans for an hdfs class
loader. Similarly, the monitor configuration would also be seamless with
the plans to improve command and control in the monitor, but that may not
even make 1.5.
Sent from my phone, so pardon the typos and brevity.
On Feb 18, 2012 10:54 PM, "Josh Elser (Commented) (JIRA)" <ji...@apache.org>
wrote:
>
>
> [
https://issues.apache.org/jira/browse/ACCUMULO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211197#comment-13211197]
>
> Josh Elser commented on ACCUMULO-407:
> -------------------------------------
>
> All nodes already need to have a valid local Accumulo installation (which
includes the conf directory), so no, I don't see a point to having it read
from HDFS currently. There could be some benefit from exposing something in
the monitor.
>
> I'd like to see this rolled into the upcoming 1.4.0 release, so I didn't
want to make a big change. I would assume that you have some sort of tool
to copy a new log4j file out already installed on a cluster (e.g. pscp,
pdcp). Same goes for making a read-only conf directory writable to do the
copy.
>
> All that being said, I don't think anyone would object to you writing
something to procedurally alter the log level either. Please have at it.
>
> > Look into on the fly log4j configuration
> > ----------------------------------------
> >
> > Key: ACCUMULO-407
> > URL: https://issues.apache.org/jira/browse/ACCUMULO-407
> > Project: Accumulo
> > Issue Type: Improvement
> > Affects Versions: 1.4.0
> > Reporter: John Vines
> > Assignee: Josh Elser
> > Fix For: 1.4.0
> >
> > Attachments: ACCUMULO-407-auto-reload-log4j.patch
> >
> >
> > For long running systems, logs may not want to be kept at the debug
level 24/7. But there may be times where a single long query may be want
to looked into without cycling the entire systems. We think it may be
possible to make log4j configurable on fly, so lets start by looking into
how difficult it will be.
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
[jira] [Updated] (ACCUMULO-407) Look into on the fly log4j
configuration
Posted by "Keith Turner (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Keith Turner updated ACCUMULO-407:
----------------------------------
Fix Version/s: (was: 1.4.1)
1.4.0
> Look into on the fly log4j configuration
> ----------------------------------------
>
> Key: ACCUMULO-407
> URL: https://issues.apache.org/jira/browse/ACCUMULO-407
> Project: Accumulo
> Issue Type: Improvement
> Reporter: John Vines
> Assignee: Josh Elser
> Fix For: 1.4.0
>
> Attachments: ACCUMULO-407-auto-reload-log4j.patch, ACCUMULO-407-auto-reload-log4j.patch, ACCUMULO-407-auto-reload-log4j.patch
>
>
> For long running systems, logs may not want to be kept at the debug level 24/7. But there may be times where a single long query may be want to looked into without cycling the entire systems. We think it may be possible to make log4j configurable on fly, so lets start by looking into how difficult it will be.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ACCUMULO-407) Look into on the fly log4j
configuration
Posted by "Keith Turner (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Keith Turner updated ACCUMULO-407:
----------------------------------
Affects Version/s: (was: 1.4.0)
> Look into on the fly log4j configuration
> ----------------------------------------
>
> Key: ACCUMULO-407
> URL: https://issues.apache.org/jira/browse/ACCUMULO-407
> Project: Accumulo
> Issue Type: Improvement
> Reporter: John Vines
> Assignee: Josh Elser
> Fix For: 1.4.0
>
> Attachments: ACCUMULO-407-auto-reload-log4j.patch, ACCUMULO-407-auto-reload-log4j.patch, ACCUMULO-407-auto-reload-log4j.patch
>
>
> For long running systems, logs may not want to be kept at the debug level 24/7. But there may be times where a single long query may be want to looked into without cycling the entire systems. We think it may be possible to make log4j configurable on fly, so lets start by looking into how difficult it will be.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-407) Look into on the fly log4j
configuration
Posted by "Drew Farris (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222056#comment-13222056 ]
Drew Farris commented on ACCUMULO-407:
--------------------------------------
{quote}
1. Do we want to be watching both the *_logger.xml AND the log4j.properties?
{quote}
Yes, because log4j.properties provides a mechanism for influencing the logging configuration for all applications regardless of whether they have a specific *_logger.xml or not.
{quote}
2. Do we want to provide a mechanism to pick up the introduction of an application specific logger.xml file after start up?
{quote}
Would this involve picking up the new application specific configuration file and disabling the watch on the generic configuration file? This would involve implementing a custom config file watcher because log4j doesn't seem to have hooks to cancel a watch initiated by configureAndWatch. An alternative would be to watch both a generic and application specific logger, but I suspect that could quickly get confusing.
{quote}
3 . Do we want to continue to have, by default, 2 different logs for each application, or can we just have 1?
{quote}
How about this: by default the A2 (debug) appender is commented out and default log level is set to INFO. This would produce 1 log per application. If someone wants to re-enable DEBUG logging they can uncomment the appender and set the levels to DEBUG to get the current behavior with 2 files per application OR they do whatever they they want.
> Look into on the fly log4j configuration
> ----------------------------------------
>
> Key: ACCUMULO-407
> URL: https://issues.apache.org/jira/browse/ACCUMULO-407
> Project: Accumulo
> Issue Type: Improvement
> Affects Versions: 1.4.0
> Reporter: John Vines
> Assignee: Josh Elser
> Fix For: 1.4.0
>
> Attachments: ACCUMULO-407-auto-reload-log4j.patch
>
>
> For long running systems, logs may not want to be kept at the debug level 24/7. But there may be times where a single long query may be want to looked into without cycling the entire systems. We think it may be possible to make log4j configurable on fly, so lets start by looking into how difficult it will be.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ACCUMULO-407) Look into on the fly log4j
configuration
Posted by "Drew Farris (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Drew Farris updated ACCUMULO-407:
---------------------------------
Attachment: ACCUMULO-407-auto-reload-log4j.patch
This patch removes the {{PropertyConfigurator}} for {{log4j.properties}} which was present in prior patches. {{Accumulo}} now initializes logging solely via {{DOMConfigurator.configureAndWatch()}}
> Look into on the fly log4j configuration
> ----------------------------------------
>
> Key: ACCUMULO-407
> URL: https://issues.apache.org/jira/browse/ACCUMULO-407
> Project: Accumulo
> Issue Type: Improvement
> Affects Versions: 1.4.0
> Reporter: John Vines
> Assignee: Josh Elser
> Fix For: 1.4.0
>
> Attachments: ACCUMULO-407-auto-reload-log4j.patch, ACCUMULO-407-auto-reload-log4j.patch, ACCUMULO-407-auto-reload-log4j.patch
>
>
> For long running systems, logs may not want to be kept at the debug level 24/7. But there may be times where a single long query may be want to looked into without cycling the entire systems. We think it may be possible to make log4j configurable on fly, so lets start by looking into how difficult it will be.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (ACCUMULO-407) Look into on the fly
log4j configuration
Posted by "Eric Newton (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230112#comment-13230112 ]
Eric Newton edited comment on ACCUMULO-407 at 3/15/12 12:43 PM:
----------------------------------------------------------------
As someone who spends much of his life reading the debug logs, it makes me a little worried that it is turned off by default. We can't avoid having the log4j.properties affect the accumulo processes, so I think we should watch it, too. Besides, it's a lot easier to edit.
I'll mark this for 1.4.1 and test it for a while.
was (Author: ecn):
As someone who spends much of his life reading the debug logs, it makes me a little worried that it is turned off by default. We can't avoid having the log4j.properties affect the accumulo processes, so I think we should watch it, too. Besides, it's a lot easier to edit. I'm in favor of watching it, too.
I'll mark this for 1.4.1 and test it for a while.
> Look into on the fly log4j configuration
> ----------------------------------------
>
> Key: ACCUMULO-407
> URL: https://issues.apache.org/jira/browse/ACCUMULO-407
> Project: Accumulo
> Issue Type: Improvement
> Reporter: John Vines
> Assignee: Josh Elser
> Fix For: 1.4.1
>
> Attachments: ACCUMULO-407-auto-reload-log4j.patch, ACCUMULO-407-auto-reload-log4j.patch, ACCUMULO-407-auto-reload-log4j.patch
>
>
> For long running systems, logs may not want to be kept at the debug level 24/7. But there may be times where a single long query may be want to looked into without cycling the entire systems. We think it may be possible to make log4j configurable on fly, so lets start by looking into how difficult it will be.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-407) Look into on the fly log4j
configuration
Posted by "Drew Farris (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223432#comment-13223432 ]
Drew Farris commented on ACCUMULO-407:
--------------------------------------
{quote}
Drew, thanks for the patch. I've added you to our contributor's role in JIRA.
{quote}
Thanks Billie.
{quote}
Drew- my concern with 1 is that we have a property which is initialized in the properties file, but then overwritten by the xml. If we have reloading on both, we then run the potential of it periodically change back and forth, which would cause logging inconsistencies. If there was a way to set a bunch of files to reload with an order of inheritance, that would be awesome. But I don't think that's supported.
{quote}
John, I can understand your concern, agree and believe that log4j doesn't really do inheritance of config files. Based on my understanding of what's going on here it seems that {{log4j.properties}} is used as the default logging configuration at startup prior to calling {{Accumulo.init()}} which uses the {{DOMConfigurator}} to configure logging. This approach continues to maintain {{log4j.properties}} as a bootstrap logging configuration and nothing else -- I like that.
The implication here is that the patch should no longer reference the {{PropertyConfigurator}} for {{log4j.properties}} from the {{Accumulo.init()}} method.
> Look into on the fly log4j configuration
> ----------------------------------------
>
> Key: ACCUMULO-407
> URL: https://issues.apache.org/jira/browse/ACCUMULO-407
> Project: Accumulo
> Issue Type: Improvement
> Affects Versions: 1.4.0
> Reporter: John Vines
> Assignee: Josh Elser
> Fix For: 1.4.0
>
> Attachments: ACCUMULO-407-auto-reload-log4j.patch, ACCUMULO-407-auto-reload-log4j.patch
>
>
> For long running systems, logs may not want to be kept at the debug level 24/7. But there may be times where a single long query may be want to looked into without cycling the entire systems. We think it may be possible to make log4j configurable on fly, so lets start by looking into how difficult it will be.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-407) Look into on the fly log4j
configuration
Posted by "Josh Elser (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211197#comment-13211197 ]
Josh Elser commented on ACCUMULO-407:
-------------------------------------
All nodes already need to have a valid local Accumulo installation (which includes the conf directory), so no, I don't see a point to having it read from HDFS currently. There could be some benefit from exposing something in the monitor.
I'd like to see this rolled into the upcoming 1.4.0 release, so I didn't want to make a big change. I would assume that you have some sort of tool to copy a new log4j file out already installed on a cluster (e.g. pscp, pdcp). Same goes for making a read-only conf directory writable to do the copy.
All that being said, I don't think anyone would object to you writing something to procedurally alter the log level either. Please have at it.
> Look into on the fly log4j configuration
> ----------------------------------------
>
> Key: ACCUMULO-407
> URL: https://issues.apache.org/jira/browse/ACCUMULO-407
> Project: Accumulo
> Issue Type: Improvement
> Affects Versions: 1.4.0
> Reporter: John Vines
> Assignee: Josh Elser
> Fix For: 1.4.0
>
> Attachments: ACCUMULO-407-auto-reload-log4j.patch
>
>
> For long running systems, logs may not want to be kept at the debug level 24/7. But there may be times where a single long query may be want to looked into without cycling the entire systems. We think it may be possible to make log4j configurable on fly, so lets start by looking into how difficult it will be.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ACCUMULO-407) Look into on the fly log4j
configuration
Posted by "Drew Farris (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Drew Farris updated ACCUMULO-407:
---------------------------------
Attachment: ACCUMULO-407-auto-reload-log4j.patch
Updated patch that does the following:
* Cleans up the foibles
* Watches log4j.properties in the ACCUMULO_HOME/conf directory regardless of whether it exists at startup.
* Comments out the A2 (debug) appender and sets the log level to INFO by default.
> Look into on the fly log4j configuration
> ----------------------------------------
>
> Key: ACCUMULO-407
> URL: https://issues.apache.org/jira/browse/ACCUMULO-407
> Project: Accumulo
> Issue Type: Improvement
> Affects Versions: 1.4.0
> Reporter: John Vines
> Assignee: Josh Elser
> Fix For: 1.4.0
>
> Attachments: ACCUMULO-407-auto-reload-log4j.patch, ACCUMULO-407-auto-reload-log4j.patch
>
>
> For long running systems, logs may not want to be kept at the debug level 24/7. But there may be times where a single long query may be want to looked into without cycling the entire systems. We think it may be possible to make log4j configurable on fly, so lets start by looking into how difficult it will be.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ACCUMULO-407) Look into on the fly log4j
configuration
Posted by "Eric Newton (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Newton updated ACCUMULO-407:
---------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Look into on the fly log4j configuration
> ----------------------------------------
>
> Key: ACCUMULO-407
> URL: https://issues.apache.org/jira/browse/ACCUMULO-407
> Project: Accumulo
> Issue Type: Improvement
> Reporter: John Vines
> Assignee: Josh Elser
> Fix For: 1.4.1
>
> Attachments: ACCUMULO-407-auto-reload-log4j.patch, ACCUMULO-407-auto-reload-log4j.patch, ACCUMULO-407-auto-reload-log4j.patch
>
>
> For long running systems, logs may not want to be kept at the debug level 24/7. But there may be times where a single long query may be want to looked into without cycling the entire systems. We think it may be possible to make log4j configurable on fly, so lets start by looking into how difficult it will be.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ACCUMULO-407) Look into on the fly log4j
configuration
Posted by "Eric Newton (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Newton updated ACCUMULO-407:
---------------------------------
Fix Version/s: (was: 1.4.0)
1.4.1
> Look into on the fly log4j configuration
> ----------------------------------------
>
> Key: ACCUMULO-407
> URL: https://issues.apache.org/jira/browse/ACCUMULO-407
> Project: Accumulo
> Issue Type: Improvement
> Reporter: John Vines
> Assignee: Josh Elser
> Fix For: 1.4.1
>
> Attachments: ACCUMULO-407-auto-reload-log4j.patch, ACCUMULO-407-auto-reload-log4j.patch, ACCUMULO-407-auto-reload-log4j.patch
>
>
> For long running systems, logs may not want to be kept at the debug level 24/7. But there may be times where a single long query may be want to looked into without cycling the entire systems. We think it may be possible to make log4j configurable on fly, so lets start by looking into how difficult it will be.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-407) Look into on the fly log4j
configuration
Posted by "Eric Newton (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230112#comment-13230112 ]
Eric Newton commented on ACCUMULO-407:
--------------------------------------
As someone who spends much of his life reading the debug logs, it makes me a little worried that it is turned off by default. We can't avoid having the log4j.properties affect the accumulo processes, so I think we should watch it, too. Besides, it's a lot easier to edit. I'm in favor of watching it, too.
I'll mark this for 1.4.1 and test it for a while.
> Look into on the fly log4j configuration
> ----------------------------------------
>
> Key: ACCUMULO-407
> URL: https://issues.apache.org/jira/browse/ACCUMULO-407
> Project: Accumulo
> Issue Type: Improvement
> Reporter: John Vines
> Assignee: Josh Elser
> Fix For: 1.4.1
>
> Attachments: ACCUMULO-407-auto-reload-log4j.patch, ACCUMULO-407-auto-reload-log4j.patch, ACCUMULO-407-auto-reload-log4j.patch
>
>
> For long running systems, logs may not want to be kept at the debug level 24/7. But there may be times where a single long query may be want to looked into without cycling the entire systems. We think it may be possible to make log4j configurable on fly, so lets start by looking into how difficult it will be.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ACCUMULO-407) Look into on the fly log4j
configuration
Posted by "Josh Elser (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Josh Elser updated ACCUMULO-407:
--------------------------------
Attachment: ACCUMULO-407-auto-reload-log4j.patch
> Look into on the fly log4j configuration
> ----------------------------------------
>
> Key: ACCUMULO-407
> URL: https://issues.apache.org/jira/browse/ACCUMULO-407
> Project: Accumulo
> Issue Type: Improvement
> Affects Versions: 1.4.0
> Reporter: John Vines
> Assignee: Josh Elser
> Fix For: 1.4.0
>
> Attachments: ACCUMULO-407-auto-reload-log4j.patch
>
>
> For long running systems, logs may not want to be kept at the debug level 24/7. But there may be times where a single long query may be want to looked into without cycling the entire systems. We think it may be possible to make log4j configurable on fly, so lets start by looking into how difficult it will be.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-407) Look into on the fly log4j
configuration
Posted by "John Vines (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221325#comment-13221325 ]
John Vines commented on ACCUMULO-407:
-------------------------------------
Found some issues-
+ String propertyLogConfig = String.format("%s/conf/log4j.properties", System.getenv("ACCUMULO_HOME"), application);
+ if (new File(logConfig).exists()) {
+ PropertyConfigurator.configureAndWatch(propertyLogConfig, 5000);
line 1 should not have the application argument
line 2 should be new File propertyLogConfig
To put aside these minor foibles, there are a few questions we need to ask ourselves in this implementation.
1. Do we want to be watching both the *_logger.xml AND the log4j.properties?
2. Do we want to provide a mechanism to pick up the introduction of an application specific logger.xml file after start up?
3. Do we want to continue to have, by default, 2 different logs for each application, or can we just have 1?
As for me,
1. Not if we don't have to. I believe the log4j.properties is only used as a default until we start watching the *_logger.xml.
2. Not for 1.4, but definitely in the future.
3. Unsure, but not for 1.4
> Look into on the fly log4j configuration
> ----------------------------------------
>
> Key: ACCUMULO-407
> URL: https://issues.apache.org/jira/browse/ACCUMULO-407
> Project: Accumulo
> Issue Type: Improvement
> Affects Versions: 1.4.0
> Reporter: John Vines
> Assignee: Josh Elser
> Fix For: 1.4.0
>
> Attachments: ACCUMULO-407-auto-reload-log4j.patch
>
>
> For long running systems, logs may not want to be kept at the debug level 24/7. But there may be times where a single long query may be want to looked into without cycling the entire systems. We think it may be possible to make log4j configurable on fly, so lets start by looking into how difficult it will be.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-407) Look into on the fly log4j
configuration
Posted by "Billie Rinaldi (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222107#comment-13222107 ]
Billie Rinaldi commented on ACCUMULO-407:
-----------------------------------------
Drew, thanks for the patch. I've added you to our contributor's role in JIRA.
> Look into on the fly log4j configuration
> ----------------------------------------
>
> Key: ACCUMULO-407
> URL: https://issues.apache.org/jira/browse/ACCUMULO-407
> Project: Accumulo
> Issue Type: Improvement
> Affects Versions: 1.4.0
> Reporter: John Vines
> Assignee: Josh Elser
> Fix For: 1.4.0
>
> Attachments: ACCUMULO-407-auto-reload-log4j.patch, ACCUMULO-407-auto-reload-log4j.patch
>
>
> For long running systems, logs may not want to be kept at the debug level 24/7. But there may be times where a single long query may be want to looked into without cycling the entire systems. We think it may be possible to make log4j configurable on fly, so lets start by looking into how difficult it will be.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-407) Look into on the fly log4j
configuration
Posted by "Josh Elser (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221449#comment-13221449 ]
Josh Elser commented on ACCUMULO-407:
-------------------------------------
0. Sorry, John. I saw I had borked the patch earlier this week but hadn't taken the time to push a new one.
1-3. I don't think we know what exactly the best logging management solution is given the amount of chatter I see above. So, I concur with you John: a complete solution isn't ready for 1.4.0. However, I do believe that the $ACCUMULO_HOME/conf/generic_logger.xml should be auto-reloaded with the configureAndWatch. It is highly undesirable to leave verbose iterator logging enabled on a multi-user system, but at the same time, it's also a non-starter to have to bounce Accumulo to get those verbose logging statements when trying to debug some unknown issue.
IMO, the best short term solution would be to merge the DOMConfigurator.configureAndWatch(logConfig, 5000) change into the 1.4 (and thus 1.4.0), and then decide what the best action to take would be for 1.4.1+/1.5.
> Look into on the fly log4j configuration
> ----------------------------------------
>
> Key: ACCUMULO-407
> URL: https://issues.apache.org/jira/browse/ACCUMULO-407
> Project: Accumulo
> Issue Type: Improvement
> Affects Versions: 1.4.0
> Reporter: John Vines
> Assignee: Josh Elser
> Fix For: 1.4.0
>
> Attachments: ACCUMULO-407-auto-reload-log4j.patch
>
>
> For long running systems, logs may not want to be kept at the debug level 24/7. But there may be times where a single long query may be want to looked into without cycling the entire systems. We think it may be possible to make log4j configurable on fly, so lets start by looking into how difficult it will be.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ACCUMULO-407) Look into on the fly log4j
configuration
Posted by "Josh Elser (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Josh Elser updated ACCUMULO-407:
--------------------------------
Fix Version/s: 1.4.0
Affects Version/s: 1.4.0
Status: Patch Available (was: Open)
Simple patch that appears to enable dynamic reloading of conf/*_logging.xml and conf/log4j.properties.
> Look into on the fly log4j configuration
> ----------------------------------------
>
> Key: ACCUMULO-407
> URL: https://issues.apache.org/jira/browse/ACCUMULO-407
> Project: Accumulo
> Issue Type: Improvement
> Affects Versions: 1.4.0
> Reporter: John Vines
> Assignee: Josh Elser
> Fix For: 1.4.0
>
> Attachments: ACCUMULO-407-auto-reload-log4j.patch
>
>
> For long running systems, logs may not want to be kept at the debug level 24/7. But there may be times where a single long query may be want to looked into without cycling the entire systems. We think it may be possible to make log4j configurable on fly, so lets start by looking into how difficult it will be.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-407) Look into on the fly log4j
configuration
Posted by "John Vines (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222416#comment-13222416 ]
John Vines commented on ACCUMULO-407:
-------------------------------------
Drew- my concern with 1 is that we have a property which is initialized in the properties file, but then overwritten by the xml. If we have reloading on both, we then run the potential of it periodically change back and forth, which would cause logging inconsistencies. If there was a way to set a bunch of files to reload with an order of inheritance, that would be awesome. But I don't think that's supported.
> Look into on the fly log4j configuration
> ----------------------------------------
>
> Key: ACCUMULO-407
> URL: https://issues.apache.org/jira/browse/ACCUMULO-407
> Project: Accumulo
> Issue Type: Improvement
> Affects Versions: 1.4.0
> Reporter: John Vines
> Assignee: Josh Elser
> Fix For: 1.4.0
>
> Attachments: ACCUMULO-407-auto-reload-log4j.patch, ACCUMULO-407-auto-reload-log4j.patch
>
>
> For long running systems, logs may not want to be kept at the debug level 24/7. But there may be times where a single long query may be want to looked into without cycling the entire systems. We think it may be possible to make log4j configurable on fly, so lets start by looking into how difficult it will be.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-407) Look into on the fly log4j
configuration
Posted by "Keith Turner (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221328#comment-13221328 ]
Keith Turner commented on ACCUMULO-407:
---------------------------------------
The reason for having .log and .debug.log file was that sometimes debug messages would fill the logs files and important errors would be lost. There are other ways to solve this issue. Just wanted to chime in on the reason we have two logs.
> Look into on the fly log4j configuration
> ----------------------------------------
>
> Key: ACCUMULO-407
> URL: https://issues.apache.org/jira/browse/ACCUMULO-407
> Project: Accumulo
> Issue Type: Improvement
> Affects Versions: 1.4.0
> Reporter: John Vines
> Assignee: Josh Elser
> Fix For: 1.4.0
>
> Attachments: ACCUMULO-407-auto-reload-log4j.patch
>
>
> For long running systems, logs may not want to be kept at the debug level 24/7. But there may be times where a single long query may be want to looked into without cycling the entire systems. We think it may be possible to make log4j configurable on fly, so lets start by looking into how difficult it will be.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-407) Look into on the fly log4j
configuration
Posted by "Dave Marion (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209046#comment-13209046 ]
Dave Marion commented on ACCUMULO-407:
--------------------------------------
I have done this before using org.apache.log4j.extras.DOMConfigurator.configureAndWatch(). According to the faq[1] it looks like they have support for it for properties files also.
[1] http://logging.apache.org/log4j/1.2/faq.html#a3.6
> Look into on the fly log4j configuration
> ----------------------------------------
>
> Key: ACCUMULO-407
> URL: https://issues.apache.org/jira/browse/ACCUMULO-407
> Project: Accumulo
> Issue Type: Improvement
> Reporter: John Vines
> Assignee: Josh Elser
>
> For long running systems, logs may not want to be kept at the debug level 24/7. But there may be times where a single long query may be want to looked into without cycling the entire systems. We think it may be possible to make log4j configurable on fly, so lets start by looking into how difficult it will be.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-407) Look into on the fly log4j
configuration
Posted by "David Medinets (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211194#comment-13211194 ]
David Medinets commented on ACCUMULO-407:
-----------------------------------------
Does this property file change need to be done on all nodes in the cluster? Do you think it would be hard to have log4j read from an hdfs file? Also note that Accumulo might be running in an environment in which the files are all 'locked down' (i.e. unchangeable). In the past, I have written a small web interface to change the log levels. The web application writes the log4j property file.
> Look into on the fly log4j configuration
> ----------------------------------------
>
> Key: ACCUMULO-407
> URL: https://issues.apache.org/jira/browse/ACCUMULO-407
> Project: Accumulo
> Issue Type: Improvement
> Affects Versions: 1.4.0
> Reporter: John Vines
> Assignee: Josh Elser
> Fix For: 1.4.0
>
> Attachments: ACCUMULO-407-auto-reload-log4j.patch
>
>
> For long running systems, logs may not want to be kept at the debug level 24/7. But there may be times where a single long query may be want to looked into without cycling the entire systems. We think it may be possible to make log4j configurable on fly, so lets start by looking into how difficult it will be.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ACCUMULO-407) Look into on the fly log4j
configuration
Posted by "Drew Farris (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ACCUMULO-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211968#comment-13211968 ]
Drew Farris commented on ACCUMULO-407:
--------------------------------------
Would it make sense to set the default log level for org.apache.accumulo to INFO in generic_logger.xml and monitor_logger.xml as well? I suspect there's no need to generate these logs by default any longer if DEBUG level can be turned on dynamically.
> Look into on the fly log4j configuration
> ----------------------------------------
>
> Key: ACCUMULO-407
> URL: https://issues.apache.org/jira/browse/ACCUMULO-407
> Project: Accumulo
> Issue Type: Improvement
> Affects Versions: 1.4.0
> Reporter: John Vines
> Assignee: Josh Elser
> Fix For: 1.4.0
>
> Attachments: ACCUMULO-407-auto-reload-log4j.patch
>
>
> For long running systems, logs may not want to be kept at the debug level 24/7. But there may be times where a single long query may be want to looked into without cycling the entire systems. We think it may be possible to make log4j configurable on fly, so lets start by looking into how difficult it will be.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira