You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Owen O'Malley (JIRA)" <ji...@apache.org> on 2008/02/22 00:52:20 UTC
[jira] Created: (HADOOP-2872) Default value for
hadoop.job.history.location is broken
Default value for hadoop.job.history.location is broken
-------------------------------------------------------
Key: HADOOP-2872
URL: https://issues.apache.org/jira/browse/HADOOP-2872
Project: Hadoop Core
Issue Type: Bug
Components: mapred
Affects Versions: 0.17.0
Reporter: Owen O'Malley
Assignee: Amareshwari Sri Ramadasu
Fix For: 0.17.0
The default value for hadoop.job.history.location is currently:
file://${hadoop.log.dir}/history
which assumes that hadoop.log.dir is an absolute path. If the path is relative, the job tracker dies with:
08/02/21 23:43:04 FATAL mapred.JobTracker: java.lang.IllegalArgumentException: Wrong FS: file://./logs/history, expected: file:///
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:268)
at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:54)
at org.apache.hadoop.fs.RawLocalFileSystem.exists(RawLocalFileSystem.java:223)
at org.apache.hadoop.fs.FilterFileSystem.exists(FilterFileSystem.java:147)
at org.apache.hadoop.mapred.JobHistory.init(JobHistory.java:124)
at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:706)
at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:124)
at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:2146)
I think that the right fix is to make the default value:
"history"
and when building, use the Path code:
{code}
new Path(get("hadoop.log.dir", conf), get("hadoop.job.history.location"));
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2872) Default value for
hadoop.job.history.location is broken
Posted by "Amareshwari Sri Ramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12573209#action_12573209 ]
Amareshwari Sri Ramadasu commented on HADOOP-2872:
--------------------------------------------------
Owen, Could you confirm whether the issue is fixed with the patch for HADOOP-2871?
> Default value for hadoop.job.history.location is broken
> -------------------------------------------------------
>
> Key: HADOOP-2872
> URL: https://issues.apache.org/jira/browse/HADOOP-2872
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sri Ramadasu
> Fix For: 0.17.0
>
>
> The default value for hadoop.job.history.location is currently:
> file://${hadoop.log.dir}/history
> which assumes that hadoop.log.dir is an absolute path. If the path is relative, the job tracker dies with:
> 08/02/21 23:43:04 FATAL mapred.JobTracker: java.lang.IllegalArgumentException: Wrong FS: file://./logs/history, expected: file:///
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:268)
> at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:54)
> at org.apache.hadoop.fs.RawLocalFileSystem.exists(RawLocalFileSystem.java:223)
> at org.apache.hadoop.fs.FilterFileSystem.exists(FilterFileSystem.java:147)
> at org.apache.hadoop.mapred.JobHistory.init(JobHistory.java:124)
> at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:706)
> at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:124)
> at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:2146)
> I think that the right fix is to make the default value:
> "history"
> and when building, use the Path code:
> {code}
> new Path(get("hadoop.log.dir", conf), get("hadoop.job.history.location"));
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2872) Default value for
hadoop.job.history.location is broken
Posted by "Amareshwari Sri Ramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12571304#action_12571304 ]
Amareshwari Sri Ramadasu commented on HADOOP-2872:
--------------------------------------------------
bq. use the Path code:{code}
new Path(get("hadoop.log.dir", conf), get("hadoop.job.history.location"));
{code}
This wouldnt work, because job history location need not be in hadoop.log.dir when it is on dfs. By default it will be on local fs and in 'hadoop.log.dir'. So, its location has to be specified as file:/// uri.
I uploaded patch which fixes the case relative path in hadoop.log.dir at HADOOP-2871. There the location is built as
{code}
LOG_DIR = conf.get("hadoop.job.history.location" , "file:///" + new File ( System.getProperty("hadoop.log.dir")).getAbsolutePath() + File.separator + "history");
{code}
> Default value for hadoop.job.history.location is broken
> -------------------------------------------------------
>
> Key: HADOOP-2872
> URL: https://issues.apache.org/jira/browse/HADOOP-2872
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sri Ramadasu
> Fix For: 0.17.0
>
>
> The default value for hadoop.job.history.location is currently:
> file://${hadoop.log.dir}/history
> which assumes that hadoop.log.dir is an absolute path. If the path is relative, the job tracker dies with:
> 08/02/21 23:43:04 FATAL mapred.JobTracker: java.lang.IllegalArgumentException: Wrong FS: file://./logs/history, expected: file:///
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:268)
> at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:54)
> at org.apache.hadoop.fs.RawLocalFileSystem.exists(RawLocalFileSystem.java:223)
> at org.apache.hadoop.fs.FilterFileSystem.exists(FilterFileSystem.java:147)
> at org.apache.hadoop.mapred.JobHistory.init(JobHistory.java:124)
> at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:706)
> at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:124)
> at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:2146)
> I think that the right fix is to make the default value:
> "history"
> and when building, use the Path code:
> {code}
> new Path(get("hadoop.log.dir", conf), get("hadoop.job.history.location"));
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-2872) Default value for
hadoop.job.history.location is broken
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu resolved HADOOP-2872.
---------------------------------------------
Resolution: Duplicate
Fixed as part of HADOOP-2871
> Default value for hadoop.job.history.location is broken
> -------------------------------------------------------
>
> Key: HADOOP-2872
> URL: https://issues.apache.org/jira/browse/HADOOP-2872
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.17.0
>
>
> The default value for hadoop.job.history.location is currently:
> file://${hadoop.log.dir}/history
> which assumes that hadoop.log.dir is an absolute path. If the path is relative, the job tracker dies with:
> 08/02/21 23:43:04 FATAL mapred.JobTracker: java.lang.IllegalArgumentException: Wrong FS: file://./logs/history, expected: file:///
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:268)
> at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:54)
> at org.apache.hadoop.fs.RawLocalFileSystem.exists(RawLocalFileSystem.java:223)
> at org.apache.hadoop.fs.FilterFileSystem.exists(FilterFileSystem.java:147)
> at org.apache.hadoop.mapred.JobHistory.init(JobHistory.java:124)
> at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:706)
> at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:124)
> at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:2146)
> I think that the right fix is to make the default value:
> "history"
> and when building, use the Path code:
> {code}
> new Path(get("hadoop.log.dir", conf), get("hadoop.job.history.location"));
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.