You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Di Li <di...@ca.ibm.com> on 2017/03/16 12:47:12 UTC
Review Request 57686: AMBARI-20472 MapRed History server start does
not create directories in HDFS when WEBHDFS is disabled.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57686/
-----------------------------------------------------------
Review request for Ambari and Alejandro Fernandez.
Bugs: AMBARI-20472
https://issues.apache.org/jira/browse/AMBARI-20472
Repository: ambari
Description
-------
MapRed HistoryServer start logic (implemented in yarn.py) creates various dirs in HDFS. The dir creation only runs thru when WEBHDFS is enabled. When WEBHDFS is disabled, HdfsResource calls bailed with NPE as the following one.
Creating: Resource [source=null, target=/app-logs, type=directory, action=create, owner=yarn, group=hadoop, mode=777, recursiveChown=false, recursiveChmod=true, changePermissionforParents=false, manageIfExists=false]
Exception occurred, Reason: null
java.lang.NullPointerException
at org.apache.hadoop.fs.FileSystem.fixRelativePart(FileSystem.java:2211)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1300)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1426)
at org.apache.ambari.fast_hdfs_resource.Runner.main(Runner.java:80)
Diffs
-----
contrib/fast-hdfs-resource/src/main/java/org/apache/ambari/fast_hdfs_resource/Runner.java e210876
Diff: https://reviews.apache.org/r/57686/diff/1/
Testing
-------
build fast-hdfs-resource jar, patch a trunk cluster with the new jar, verify the dirs , such as /mapred, are created when WEBHDFS is disabled.
Thanks,
Di Li
Re: Review Request 57686: AMBARI-20472 MapRed History server start
does not create directories in HDFS when WEBHDFS is disabled.
Posted by Di Li <di...@ca.ibm.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57686/
-----------------------------------------------------------
(Updated March 16, 2017, 3:55 p.m.)
Review request for Ambari and Alejandro Fernandez.
Bugs: AMBARI-20472
https://issues.apache.org/jira/browse/AMBARI-20472
Repository: ambari
Description
-------
MapRed HistoryServer start logic (implemented in yarn.py) creates various dirs in HDFS. The dir creation only runs thru when WEBHDFS is enabled. When WEBHDFS is disabled, HdfsResource calls bailed with NPE as the following one.
Creating: Resource [source=null, target=/app-logs, type=directory, action=create, owner=yarn, group=hadoop, mode=777, recursiveChown=false, recursiveChmod=true, changePermissionforParents=false, manageIfExists=false]
Exception occurred, Reason: null
java.lang.NullPointerException
at org.apache.hadoop.fs.FileSystem.fixRelativePart(FileSystem.java:2211)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1300)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1426)
at org.apache.ambari.fast_hdfs_resource.Runner.main(Runner.java:80)
Diffs (updated)
-----
contrib/fast-hdfs-resource/src/main/java/org/apache/ambari/fast_hdfs_resource/Runner.java e210876
Diff: https://reviews.apache.org/r/57686/diff/2/
Changes: https://reviews.apache.org/r/57686/diff/1-2/
Testing
-------
build fast-hdfs-resource jar, patch a trunk cluster with the new jar, verify the dirs , such as /mapred, are created when WEBHDFS is disabled.
Thanks,
Di Li
Re: Review Request 57686: AMBARI-20472 MapRed History server start
does not create directories in HDFS when WEBHDFS is disabled.
Posted by Di Li <di...@ca.ibm.com>.
> On March 16, 2017, 1:25 p.m., Attila Doroszlai wrote:
> > The bug is also present in [branch-2.5](http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=ae75c871e497ee4a8de23579b3fbc175f8d073d5), shouldn't the fix be committed there, too?
yes, sure, I will push the fix to both branches.
> On March 16, 2017, 1:25 p.m., Attila Doroszlai wrote:
> > contrib/fast-hdfs-resource/src/main/java/org/apache/ambari/fast_hdfs_resource/Runner.java
> > Line 81 (original), 82 (patched)
> > <https://reviews.apache.org/r/57686/diff/1/?file=1666070#file1666070line82>
> >
> > Can you please also replace this occurrence of `resource.getTarget()`?
yes, fixed.
- Di
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57686/#review169144
-----------------------------------------------------------
On March 16, 2017, 12:47 p.m., Di Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57686/
> -----------------------------------------------------------
>
> (Updated March 16, 2017, 12:47 p.m.)
>
>
> Review request for Ambari and Alejandro Fernandez.
>
>
> Bugs: AMBARI-20472
> https://issues.apache.org/jira/browse/AMBARI-20472
>
>
> Repository: ambari
>
>
> Description
> -------
>
> MapRed HistoryServer start logic (implemented in yarn.py) creates various dirs in HDFS. The dir creation only runs thru when WEBHDFS is enabled. When WEBHDFS is disabled, HdfsResource calls bailed with NPE as the following one.
> Creating: Resource [source=null, target=/app-logs, type=directory, action=create, owner=yarn, group=hadoop, mode=777, recursiveChown=false, recursiveChmod=true, changePermissionforParents=false, manageIfExists=false]
> Exception occurred, Reason: null
> java.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.fixRelativePart(FileSystem.java:2211)
> at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1300)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1426)
> at org.apache.ambari.fast_hdfs_resource.Runner.main(Runner.java:80)
>
>
> Diffs
> -----
>
> contrib/fast-hdfs-resource/src/main/java/org/apache/ambari/fast_hdfs_resource/Runner.java e210876
>
>
> Diff: https://reviews.apache.org/r/57686/diff/1/
>
>
> Testing
> -------
>
> build fast-hdfs-resource jar, patch a trunk cluster with the new jar, verify the dirs , such as /mapred, are created when WEBHDFS is disabled.
>
>
> Thanks,
>
> Di Li
>
>
Re: Review Request 57686: AMBARI-20472 MapRed History server start
does not create directories in HDFS when WEBHDFS is disabled.
Posted by Attila Doroszlai <ad...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57686/#review169144
-----------------------------------------------------------
Ship it!
The bug is also present in [branch-2.5](http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=ae75c871e497ee4a8de23579b3fbc175f8d073d5), shouldn't the fix be committed there, too?
contrib/fast-hdfs-resource/src/main/java/org/apache/ambari/fast_hdfs_resource/Runner.java
Line 81 (original), 82 (patched)
<https://reviews.apache.org/r/57686/#comment241474>
Can you please also replace this occurrence of `resource.getTarget()`?
- Attila Doroszlai
On March 16, 2017, 1:47 p.m., Di Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57686/
> -----------------------------------------------------------
>
> (Updated March 16, 2017, 1:47 p.m.)
>
>
> Review request for Ambari and Alejandro Fernandez.
>
>
> Bugs: AMBARI-20472
> https://issues.apache.org/jira/browse/AMBARI-20472
>
>
> Repository: ambari
>
>
> Description
> -------
>
> MapRed HistoryServer start logic (implemented in yarn.py) creates various dirs in HDFS. The dir creation only runs thru when WEBHDFS is enabled. When WEBHDFS is disabled, HdfsResource calls bailed with NPE as the following one.
> Creating: Resource [source=null, target=/app-logs, type=directory, action=create, owner=yarn, group=hadoop, mode=777, recursiveChown=false, recursiveChmod=true, changePermissionforParents=false, manageIfExists=false]
> Exception occurred, Reason: null
> java.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.fixRelativePart(FileSystem.java:2211)
> at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1300)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1426)
> at org.apache.ambari.fast_hdfs_resource.Runner.main(Runner.java:80)
>
>
> Diffs
> -----
>
> contrib/fast-hdfs-resource/src/main/java/org/apache/ambari/fast_hdfs_resource/Runner.java e210876
>
>
> Diff: https://reviews.apache.org/r/57686/diff/1/
>
>
> Testing
> -------
>
> build fast-hdfs-resource jar, patch a trunk cluster with the new jar, verify the dirs , such as /mapred, are created when WEBHDFS is disabled.
>
>
> Thanks,
>
> Di Li
>
>