You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Jerome Boulon (JIRA)" <ji...@apache.org> on 2009/01/14 23:18:59 UTC
[jira] Created: (HADOOP-5042) Add expiration handling to the
chukwa log4j appender
Add expiration handling to the chukwa log4j appender
-----------------------------------------------------
Key: HADOOP-5042
URL: https://issues.apache.org/jira/browse/HADOOP-5042
Project: Hadoop Core
Issue Type: New Feature
Components: contrib/chukwa
Reporter: Jerome Boulon
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5042) Add expiration handling to the
chukwa log4j appender
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated HADOOP-5042:
----------------------------------
Status: Open (was: Patch Available)
The new patch still adds @author tags...
> Add expiration handling to the chukwa log4j appender
> -----------------------------------------------------
>
> Key: HADOOP-5042
> URL: https://issues.apache.org/jira/browse/HADOOP-5042
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/chukwa
> Reporter: Jerome Boulon
> Assignee: Jerome Boulon
> Attachments: HADOOP-5042.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5042) Add expiration handling to the
chukwa log4j appender
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12675976#action_12675976 ]
Jerome Boulon commented on HADOOP-5042:
---------------------------------------
-1 core tests. The patch failed core unit tests. is not related to that patch
> Add expiration handling to the chukwa log4j appender
> -----------------------------------------------------
>
> Key: HADOOP-5042
> URL: https://issues.apache.org/jira/browse/HADOOP-5042
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/chukwa
> Reporter: Jerome Boulon
> Assignee: Jerome Boulon
> Attachments: HADOOP-5042-2.patch, HADOOP-5042.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5042) Add expiration handling to the
chukwa log4j appender
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12677026#action_12677026 ]
Hudson commented on HADOOP-5042:
--------------------------------
Integrated in Hadoop-trunk #766 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/766/])
> Add expiration handling to the chukwa log4j appender
> -----------------------------------------------------
>
> Key: HADOOP-5042
> URL: https://issues.apache.org/jira/browse/HADOOP-5042
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/chukwa
> Reporter: Jerome Boulon
> Assignee: Jerome Boulon
> Fix For: 0.21.0
>
> Attachments: HADOOP-5042-2.patch, HADOOP-5042.patch
>
>
> Chukwa log4j appender is not doing any sort of cleanup.
> The idea here is to keep only n rotate files and delete the older ones.
> This way we don't have to worry about manually cleaning old files
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5042) Add expiration handling to the
chukwa log4j appender
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated HADOOP-5042:
------------------------------
Attachment: (was: HADOOP-5042.patch)
> Add expiration handling to the chukwa log4j appender
> -----------------------------------------------------
>
> Key: HADOOP-5042
> URL: https://issues.apache.org/jira/browse/HADOOP-5042
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/chukwa
> Reporter: Jerome Boulon
> Assignee: Jerome Boulon
> Attachments: HADOOP-5042.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5042) Add expiration handling to the
chukwa log4j appender
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12675632#action_12675632 ]
Hadoop QA commented on HADOOP-5042:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12400634/HADOOP-5042-2.patch
against trunk revision 746340.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3898/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3898/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3898/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3898/console
This message is automatically generated.
> Add expiration handling to the chukwa log4j appender
> -----------------------------------------------------
>
> Key: HADOOP-5042
> URL: https://issues.apache.org/jira/browse/HADOOP-5042
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/chukwa
> Reporter: Jerome Boulon
> Assignee: Jerome Boulon
> Attachments: HADOOP-5042-2.patch, HADOOP-5042.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5042) Add expiration handling to the
chukwa log4j appender
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jerome Boulon updated HADOOP-5042:
----------------------------------
Comment: was deleted
> Add expiration handling to the chukwa log4j appender
> -----------------------------------------------------
>
> Key: HADOOP-5042
> URL: https://issues.apache.org/jira/browse/HADOOP-5042
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/chukwa
> Reporter: Jerome Boulon
> Assignee: Jerome Boulon
> Attachments: HADOOP-5042-2.patch, HADOOP-5042.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5042) Add expiration handling to the
chukwa log4j appender
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12664050#action_12664050 ]
Eric Yang commented on HADOOP-5042:
-----------------------------------
It may be better to match files starting with the same name, and stat them and delete oldest file(s) up the maxBackupIndex specified number. It's cheaper to do file stat than regex filename matching.
> Add expiration handling to the chukwa log4j appender
> -----------------------------------------------------
>
> Key: HADOOP-5042
> URL: https://issues.apache.org/jira/browse/HADOOP-5042
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/chukwa
> Reporter: Jerome Boulon
> Assignee: Jerome Boulon
> Attachments: HADOOP-5042.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5042) Add expiration handling to the
chukwa log4j appender
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12664195#action_12664195 ]
Jerome Boulon commented on HADOOP-5042:
---------------------------------------
- the @Author tags were already there in my version (Revision 685353 ), I will remove them
- the sort is out of any loop and it's done only once per rotate period so it should not cause any performance issue
- I will put a check on $filename before applying the regex to take into account Ari's feedback regarding the security issue
> Add expiration handling to the chukwa log4j appender
> -----------------------------------------------------
>
> Key: HADOOP-5042
> URL: https://issues.apache.org/jira/browse/HADOOP-5042
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/chukwa
> Reporter: Jerome Boulon
> Assignee: Jerome Boulon
> Attachments: HADOOP-5042.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5042) Add expiration handling to the
chukwa log4j appender
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jerome Boulon updated HADOOP-5042:
----------------------------------
Status: Patch Available (was: In Progress)
> Add expiration handling to the chukwa log4j appender
> -----------------------------------------------------
>
> Key: HADOOP-5042
> URL: https://issues.apache.org/jira/browse/HADOOP-5042
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/chukwa
> Reporter: Jerome Boulon
> Assignee: Jerome Boulon
> Attachments: HADOOP-5042.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5042) Add expiration handling to the
chukwa log4j appender
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12664045#action_12664045 ]
Ari Rabkin commented on HADOOP-5042:
------------------------------------
Couple things I'm nervous about.
1) This patch introduces @author tags. This conflicts with Hadoop coding standards. Is there a reason for adding them?
2) You do a lexical sort to figure out which rotated files are old. Is this safe in context?
3) It's sort of icky to repeatedly remove the first element from an array -- the remove is O(N) each time. That''s probably okay here, since the array never gets big and we don't run this very often. But it might be nice to leave a comment explaining why this is okay.
4) This might not bother us, but there's a little bit of a security/correctness issue here: The pattern can easily match files that don't contain $fileName -- just do something like "$filename | .*". And we don't check for that. A fix would be to first match against .*$filename.* before applying the user's regex.
> Add expiration handling to the chukwa log4j appender
> -----------------------------------------------------
>
> Key: HADOOP-5042
> URL: https://issues.apache.org/jira/browse/HADOOP-5042
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/chukwa
> Reporter: Jerome Boulon
> Assignee: Jerome Boulon
> Attachments: HADOOP-5042.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5042) Add expiration handling to the
chukwa log4j appender
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jerome Boulon updated HADOOP-5042:
----------------------------------
Attachment: HADOOP-5042-2.patch
new patch
> Add expiration handling to the chukwa log4j appender
> -----------------------------------------------------
>
> Key: HADOOP-5042
> URL: https://issues.apache.org/jira/browse/HADOOP-5042
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/chukwa
> Reporter: Jerome Boulon
> Assignee: Jerome Boulon
> Attachments: HADOOP-5042-2.patch, HADOOP-5042.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5042) Add expiration handling to the
chukwa log4j appender
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jerome Boulon updated HADOOP-5042:
----------------------------------
Attachment: HADOOP-5042-2.patch
new patch
> Add expiration handling to the chukwa log4j appender
> -----------------------------------------------------
>
> Key: HADOOP-5042
> URL: https://issues.apache.org/jira/browse/HADOOP-5042
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/chukwa
> Reporter: Jerome Boulon
> Assignee: Jerome Boulon
> Attachments: HADOOP-5042-2.patch, HADOOP-5042.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5042) Add expiration handling to the
chukwa log4j appender
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jerome Boulon updated HADOOP-5042:
----------------------------------
Attachment: HADOOP-5042.patch
Revise to take feedback into account + add one more check to exclude current log file from the list
> Add expiration handling to the chukwa log4j appender
> -----------------------------------------------------
>
> Key: HADOOP-5042
> URL: https://issues.apache.org/jira/browse/HADOOP-5042
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/chukwa
> Reporter: Jerome Boulon
> Assignee: Jerome Boulon
> Attachments: HADOOP-5042.patch, HADOOP-5042.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-5042) Add expiration handling to the
chukwa log4j appender
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jerome Boulon reassigned HADOOP-5042:
-------------------------------------
Assignee: Jerome Boulon
> Add expiration handling to the chukwa log4j appender
> -----------------------------------------------------
>
> Key: HADOOP-5042
> URL: https://issues.apache.org/jira/browse/HADOOP-5042
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/chukwa
> Reporter: Jerome Boulon
> Assignee: Jerome Boulon
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5042) Add expiration handling to the
chukwa log4j appender
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jerome Boulon updated HADOOP-5042:
----------------------------------
Description:
Chukwa log4j appender is not doing any sort of cleanup.
The idea here is to keep only n rotate files and delete the older ones.
This way we don't have to worry about manually cleaning old files
> Add expiration handling to the chukwa log4j appender
> -----------------------------------------------------
>
> Key: HADOOP-5042
> URL: https://issues.apache.org/jira/browse/HADOOP-5042
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/chukwa
> Reporter: Jerome Boulon
> Assignee: Jerome Boulon
> Fix For: 0.21.0
>
> Attachments: HADOOP-5042-2.patch, HADOOP-5042.patch
>
>
> Chukwa log4j appender is not doing any sort of cleanup.
> The idea here is to keep only n rotate files and delete the older ones.
> This way we don't have to worry about manually cleaning old files
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Work started: (HADOOP-5042) Add expiration handling to the
chukwa log4j appender
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on HADOOP-5042 started by Jerome Boulon.
> Add expiration handling to the chukwa log4j appender
> -----------------------------------------------------
>
> Key: HADOOP-5042
> URL: https://issues.apache.org/jira/browse/HADOOP-5042
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/chukwa
> Reporter: Jerome Boulon
> Assignee: Jerome Boulon
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5042) Add expiration handling to the
chukwa log4j appender
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jerome Boulon updated HADOOP-5042:
----------------------------------
Attachment: (was: HADOOP-5042-2.patch)
> Add expiration handling to the chukwa log4j appender
> -----------------------------------------------------
>
> Key: HADOOP-5042
> URL: https://issues.apache.org/jira/browse/HADOOP-5042
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/chukwa
> Reporter: Jerome Boulon
> Assignee: Jerome Boulon
> Attachments: HADOOP-5042.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5042) Add expiration handling to the
chukwa log4j appender
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jerome Boulon updated HADOOP-5042:
----------------------------------
Attachment: HADOOP-5042.patch
Adding these 2 parameters to the log4j properties file, will allow the Chukwalog4JAppender to automatically do some cleanup on the log file
log4j.appender.R.maxBackupIndex=3
log4j.appender.R.cleanUpRegex=$fileName.20*
> Add expiration handling to the chukwa log4j appender
> -----------------------------------------------------
>
> Key: HADOOP-5042
> URL: https://issues.apache.org/jira/browse/HADOOP-5042
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/chukwa
> Reporter: Jerome Boulon
> Assignee: Jerome Boulon
> Attachments: HADOOP-5042.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5042) Add expiration handling to the
chukwa log4j appender
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12676021#action_12676021 ]
Ari Rabkin commented on HADOOP-5042:
------------------------------------
Patch looks fine. +1
> Add expiration handling to the chukwa log4j appender
> -----------------------------------------------------
>
> Key: HADOOP-5042
> URL: https://issues.apache.org/jira/browse/HADOOP-5042
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/chukwa
> Reporter: Jerome Boulon
> Assignee: Jerome Boulon
> Attachments: HADOOP-5042-2.patch, HADOOP-5042.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5042) Add expiration handling to the
chukwa log4j appender
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jerome Boulon updated HADOOP-5042:
----------------------------------
Status: Patch Available (was: Open)
> Add expiration handling to the chukwa log4j appender
> -----------------------------------------------------
>
> Key: HADOOP-5042
> URL: https://issues.apache.org/jira/browse/HADOOP-5042
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/chukwa
> Reporter: Jerome Boulon
> Assignee: Jerome Boulon
> Attachments: HADOOP-5042-2.patch, HADOOP-5042.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5042) Add expiration handling to the
chukwa log4j appender
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated HADOOP-5042:
----------------------------------
Resolution: Fixed
Fix Version/s: 0.21.0
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I committed this. Thanks, Jerome
> Add expiration handling to the chukwa log4j appender
> -----------------------------------------------------
>
> Key: HADOOP-5042
> URL: https://issues.apache.org/jira/browse/HADOOP-5042
> Project: Hadoop Core
> Issue Type: New Feature
> Components: contrib/chukwa
> Reporter: Jerome Boulon
> Assignee: Jerome Boulon
> Fix For: 0.21.0
>
> Attachments: HADOOP-5042-2.patch, HADOOP-5042.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.