You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Owen O'Malley (JIRA)" <ji...@apache.org> on 2007/07/01 08:08:04 UTC
[jira] Updated: (HADOOP-1533) Distcp should log to specified
location
[ https://issues.apache.org/jira/browse/HADOOP-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-1533:
----------------------------------
Status: Open (was: Patch Available)
This patch seems confused whether the "log path" is a Path, URI, or a String. Since you are implementing it as the output of the Map (with reduces = 0), it pretty clearly must be a Path. It should be coded as such.
Instead of requiring a parameter for the log path, I think it should be an option like:
... - log hdfs://server:1234/my/log
and I'd propose that the default be: "$output_dir/_distcp_logs"
That will work well in most cases and keep the output/logs together.
The chunk of code that looks like:
+ String logURL = "hdfs://" + fsConfig.get("fs.default.name", "local") +
+ archiveDirectory + "distcpLogDir." + System.currentTimeMillis();
is problematic, since it won't construct useful urls in a wide variety of values for fs.default.name.
> Distcp should log to specified location
> ---------------------------------------
>
> Key: HADOOP-1533
> URL: https://issues.apache.org/jira/browse/HADOOP-1533
> Project: Hadoop
> Issue Type: Improvement
> Components: util
> Reporter: Senthil Subramanian
> Fix For: 0.14.0
>
> Attachments: HADOOP-1533_1.patch
>
>
> Distcp errors are now not logged anywhere (other that reporter.setStatus which gets overwritten). Add support to mention a log URI where the log files should go.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.