You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Sanjeev Jaiswal (JIRA)" <ji...@apache.org> on 2009/03/10 05:08:50 UTC
[jira] Created: (HADOOP-5447) Logging turned on by default while
using distcp
Logging turned on by default while using distcp
-----------------------------------------------
Key: HADOOP-5447
URL: https://issues.apache.org/jira/browse/HADOOP-5447
Project: Hadoop Core
Issue Type: Bug
Components: tools/distcp
Affects Versions: 0.18.3, 0.18.2
Environment: Linux RHEL54, Linux RHEL5
Reporter: Sanjeev Jaiswal
Fix For: 0.20.0, 0.21.0
Distcp should have an option to disable logging during a distcp
eg: hadoop distcp --nolog <source dir> <destination dir>
By default logging is enabled or turned on while using distcp.This generates logs in DFS, which need to be cleaned up periodically. During this time, critical applications sometimes fail, when they see distcp logs in the DFS.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5447) Logging turned on by default while
using distcp
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Gummadi updated HADOOP-5447:
---------------------------------
Attachment: d_log_5447.patch
Without -log option and with relative path specified to -log, the logs dir distcp_logs_randomID was created in copy destination dir.
Attaching patch that makes distcp to create distcp_logs_randomID dir in mapred.system.dir and not in the copy destination dir. Also displays a message giving the path of this log dir for users to look into its contents, if needed. Since the log dir name contains randomID as part of it, multiple distcp jobs shouldn't get the same log dir name, thus avoiding clashing.
Please review and provide your comments.
> Logging turned on by default while using distcp
> -----------------------------------------------
>
> Key: HADOOP-5447
> URL: https://issues.apache.org/jira/browse/HADOOP-5447
> Project: Hadoop Core
> Issue Type: New Feature
> Components: tools/distcp
> Affects Versions: 0.18.2, 0.18.3
> Environment: Linux RHEL54, Linux RHEL5
> Reporter: Sanjeev Jaiswal
> Assignee: Ravi Gummadi
> Fix For: 0.20.1
>
> Attachments: d_log_5447.patch
>
>
> Distcp should have an option to disable logging during a distcp
> eg: hadoop distcp --nolog <source dir> <destination dir>
> By default logging is enabled or turned on while using distcp.This generates logs in DFS, which need to be cleaned up periodically. During this time, critical applications sometimes fail, when they see distcp logs in the DFS.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5447) Logging turned on by default while
using distcp
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700318#action_12700318 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-5447:
------------------------------------------------
> For disable logging, you may change the values set in log4j.properties.
Oops, I mis-understood the problem.
In such case, you may use -log <logdir> to specify a logging directory, say /tmp/foo, which is outside the critical applications' directories. Of course, it is still good to have the suggested new feature.
> Logging turned on by default while using distcp
> -----------------------------------------------
>
> Key: HADOOP-5447
> URL: https://issues.apache.org/jira/browse/HADOOP-5447
> Project: Hadoop Core
> Issue Type: New Feature
> Components: tools/distcp
> Affects Versions: 0.18.2, 0.18.3
> Environment: Linux RHEL54, Linux RHEL5
> Reporter: Sanjeev Jaiswal
> Fix For: 0.20.0, 0.21.0
>
>
> Distcp should have an option to disable logging during a distcp
> eg: hadoop distcp --nolog <source dir> <destination dir>
> By default logging is enabled or turned on while using distcp.This generates logs in DFS, which need to be cleaned up periodically. During this time, critical applications sometimes fail, when they see distcp logs in the DFS.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5447) Logging turned on by default while
using distcp
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-5447:
-------------------------------------------
Issue Type: New Feature (was: Bug)
For disable logging, you may change the values set in log4j.properties.
> Logging turned on by default while using distcp
> -----------------------------------------------
>
> Key: HADOOP-5447
> URL: https://issues.apache.org/jira/browse/HADOOP-5447
> Project: Hadoop Core
> Issue Type: New Feature
> Components: tools/distcp
> Affects Versions: 0.18.2, 0.18.3
> Environment: Linux RHEL54, Linux RHEL5
> Reporter: Sanjeev Jaiswal
> Fix For: 0.20.0, 0.21.0
>
>
> Distcp should have an option to disable logging during a distcp
> eg: hadoop distcp --nolog <source dir> <destination dir>
> By default logging is enabled or turned on while using distcp.This generates logs in DFS, which need to be cleaned up periodically. During this time, critical applications sometimes fail, when they see distcp logs in the DFS.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-5447) Logging turned on by default while
using distcp
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravi Gummadi reassigned HADOOP-5447:
------------------------------------
Assignee: Ravi Gummadi
> Logging turned on by default while using distcp
> -----------------------------------------------
>
> Key: HADOOP-5447
> URL: https://issues.apache.org/jira/browse/HADOOP-5447
> Project: Hadoop Core
> Issue Type: New Feature
> Components: tools/distcp
> Affects Versions: 0.18.2, 0.18.3
> Environment: Linux RHEL54, Linux RHEL5
> Reporter: Sanjeev Jaiswal
> Assignee: Ravi Gummadi
> Fix For: 0.20.1
>
>
> Distcp should have an option to disable logging during a distcp
> eg: hadoop distcp --nolog <source dir> <destination dir>
> By default logging is enabled or turned on while using distcp.This generates logs in DFS, which need to be cleaned up periodically. During this time, critical applications sometimes fail, when they see distcp logs in the DFS.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.