You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Devaraj K (JIRA)" <ji...@apache.org> on 2011/02/10 13:10:57 UTC
[jira] Created: (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
----------------------------------------------------------------------------------------
Key: MAPREDUCE-2317
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: harchive
Affects Versions: 0.20.1, 0.23.0
Environment: windows
Reporter: Devaraj K
Priority: Minor
While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
{code:xml}
java.lang.NullPointerException
at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
{code}
I see the code flow to handle this feature in windows also,
{code:title=Path.java|borderStyle=solid}
/** Returns the parent of a path or null if at root. */
public Path getParent() {
String path = uri.getPath();
int lastSlash = path.lastIndexOf('/');
int start = hasWindowsDrive(path, true) ? 3 : 0;
if ((path.length() == start) || // empty path
(lastSlash == start && path.length() == start+1)) { // at root
return null;
}
String parent;
if (lastSlash==-1) {
parent = CUR_DIR;
} else {
int end = hasWindowsDrive(path, true) ? 3 : 0;
parent = path.substring(0, lastSlash==end?end+1:lastSlash);
}
return new Path(uri.getScheme(), uri.getAuthority(), parent);
}
{code}
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017806#comment-13017806 ]
Devaraj K commented on MAPREDUCE-2317:
--------------------------------------
Hi Nicholas, I couldnot find the jira id, but the problem is rectified in the trunk.
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.1, 0.23.0
> Environment: windows
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Minor
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K reassigned MAPREDUCE-2317:
------------------------------------
Assignee: Devaraj K
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.1, 0.23.0
> Environment: windows
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Minor
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015761#comment-13015761 ]
Devaraj K commented on MAPREDUCE-2317:
--------------------------------------
This problem is not there in the latest trunk. It is coming in 0.20.3 branch. Patch is provided for that.
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.1, 0.23.0
> Environment: windows
> Reporter: Devaraj K
> Priority: Minor
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13019035#comment-13019035 ]
Tsz Wo (Nicholas), SZE commented on MAPREDUCE-2317:
---------------------------------------------------
For 0.20, is it the same that when {{parent == null}}, it should not be added to parents list?
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.1, 0.23.0
> Environment: windows
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Minor
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13020629#comment-13020629 ]
Devaraj K commented on MAPREDUCE-2317:
--------------------------------------
Sorry for late reply.
parents is assigning to justDirs and justDirs used in the for loop only. When null gets added to parents loop ends because i reaches to deepest.depth().
If we move parents.add(parent); inside null check or if we keep outside also doesnot give any functionality difference.
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.3
> Environment: windows
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Minor
> Fix For: 0.20.3
>
> Attachments: MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K updated MAPREDUCE-2317:
---------------------------------
Attachment: MAPREDUCE-2317-trunk.patch
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.3
> Environment: windows
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Minor
> Fix For: 0.20.3
>
> Attachments: MAPREDUCE-2317-0.20.patch, MAPREDUCE-2317-trunk.patch, MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025209#comment-13025209 ]
Hudson commented on MAPREDUCE-2317:
-----------------------------------
Integrated in Hadoop-Mapreduce-22-branch #42 (See [https://builds.apache.org/hudson/job/Hadoop-Mapreduce-22-branch/42/])
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.3
> Environment: windows
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Minor
> Fix For: 0.20.4, 0.21.1, 0.22.0, 0.23.0
>
> Attachments: MAPREDUCE-2317-0.20.patch, MAPREDUCE-2317-trunk.patch, MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K updated MAPREDUCE-2317:
---------------------------------
Attachment: MAPREDUCE-2317.patch
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.1, 0.23.0
> Environment: windows
> Reporter: Devaraj K
> Priority: Minor
> Attachments: MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13023250#comment-13023250 ]
Tsz Wo (Nicholas), SZE commented on MAPREDUCE-2317:
---------------------------------------------------
+1 patch look good. Thanks a lot!
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.3
> Environment: windows
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Minor
> Fix For: 0.20.3, 0.23.0
>
> Attachments: MAPREDUCE-2317-0.20.patch, MAPREDUCE-2317-trunk.patch, MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K updated MAPREDUCE-2317:
---------------------------------
Affects Version/s: (was: 0.23.0)
(was: 0.20.1)
0.20.3
Fix Version/s: (was: 0.23.0)
0.20.3
I have mistakenly selected the patch version as 0.23, but it is intended for 0.20.3. It would be better if we commit these changes in 0.20.3.
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.3
> Environment: windows
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Minor
> Fix For: 0.20.3
>
> Attachments: MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated MAPREDUCE-2317:
----------------------------------------------
Resolution: Fixed
Fix Version/s: (was: 0.20.3)
0.22.0
0.21.1
0.20.4
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
The failed tests are not related; see MAPREDUCE-2448.
This is a simple change. Devaraj has tested it manually. No new tests needed.
I have committed this. Thanks, Devaraj!
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.3
> Environment: windows
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Minor
> Fix For: 0.20.4, 0.21.1, 0.22.0, 0.23.0
>
> Attachments: MAPREDUCE-2317-0.20.patch, MAPREDUCE-2317-trunk.patch, MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13024861#comment-13024861 ]
Hudson commented on MAPREDUCE-2317:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #645 (See [https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/645/])
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.3
> Environment: windows
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Minor
> Fix For: 0.20.4, 0.21.1, 0.22.0, 0.23.0
>
> Attachments: MAPREDUCE-2317-0.20.patch, MAPREDUCE-2317-trunk.patch, MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015518#comment-13015518 ]
Tsz Wo (Nicholas), SZE commented on MAPREDUCE-2317:
---------------------------------------------------
- When {{parent == null}}, should it be not added to {{parents}} list?
- It seems that you are not using the latest trunk.
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.1, 0.23.0
> Environment: windows
> Reporter: Devaraj K
> Priority: Minor
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13019461#comment-13019461 ]
Tsz Wo (Nicholas), SZE commented on MAPREDUCE-2317:
---------------------------------------------------
Let me clarify my question: after the patch, the codes look like
{code}
Path parent = p.getParent();
if(null != parent) {
...
}
parents.add(parent);
{code}
If {{parent == null}}, we add it to {{parents}}. Should it not be added? Otherwise, I think we will get NPE later.
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.3
> Environment: windows
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Minor
> Fix For: 0.20.3
>
> Attachments: MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13019045#comment-13019045 ]
Mahadev konar commented on MAPREDUCE-2317:
------------------------------------------
devaraj,
the jira says that the bug affects 0.23 and the fix is for 0.23. But you mentioned above that the trunk doesnt have this issue? Do you intend to close the jira as resolved?
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.1, 0.23.0
> Environment: windows
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Minor
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13023234#comment-13023234 ]
Devaraj K commented on MAPREDUCE-2317:
--------------------------------------
Hi Nicholas, I have updated the patch and also provided patch as per above comments.
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.3
> Environment: windows
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Minor
> Fix For: 0.20.3, 0.23.0
>
> Attachments: MAPREDUCE-2317-0.20.patch, MAPREDUCE-2317-trunk.patch, MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13023331#comment-13023331 ]
Hadoop QA commented on MAPREDUCE-2317:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12477114/MAPREDUCE-2317-trunk.patch
against trunk revision 1094093.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/179//testReport/
Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/179//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/179//console
This message is automatically generated.
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.3
> Environment: windows
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Minor
> Fix For: 0.20.3, 0.23.0
>
> Attachments: MAPREDUCE-2317-0.20.patch, MAPREDUCE-2317-trunk.patch, MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K updated MAPREDUCE-2317:
---------------------------------
Attachment: MAPREDUCE-2317-0.20.patch
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.3
> Environment: windows
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Minor
> Fix For: 0.20.3
>
> Attachments: MAPREDUCE-2317-0.20.patch, MAPREDUCE-2317-trunk.patch, MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13028772#comment-13028772 ]
Hudson commented on MAPREDUCE-2317:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk #669 (See [https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk/669/])
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.3
> Environment: windows
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Minor
> Fix For: 0.20.4, 0.21.1, 0.22.0, 0.23.0
>
> Attachments: MAPREDUCE-2317-0.20.patch, MAPREDUCE-2317-trunk.patch, MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13015996#comment-13015996 ]
Tsz Wo (Nicholas), SZE commented on MAPREDUCE-2317:
---------------------------------------------------
Hi Devaraj, if the problem is not in the latest trunk, is it already fixed by some other JIRAs? Do you know which JIRAs have fixed it?
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.1, 0.23.0
> Environment: windows
> Reporter: Devaraj K
> Priority: Minor
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K updated MAPREDUCE-2317:
---------------------------------
Fix Version/s: 0.23.0
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.3
> Environment: windows
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Minor
> Fix For: 0.20.3, 0.23.0
>
> Attachments: MAPREDUCE-2317-0.20.patch, MAPREDUCE-2317-trunk.patch, MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K updated MAPREDUCE-2317:
---------------------------------
Fix Version/s: 0.23.0
Status: Patch Available (was: Open)
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.1, 0.23.0
> Environment: windows
> Reporter: Devaraj K
> Priority: Minor
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13021385#comment-13021385 ]
Tsz Wo (Nicholas), SZE commented on MAPREDUCE-2317:
---------------------------------------------------
Hi Devaraj, thanks for the explanation of the codes. I really like the following comment in the codes.
{code}
// this is tricky
{code}
Still, how about moving {{parents.add(parent)}} inside null check for easing future maintenance?
Since the 0.23 have this part of codes. let's also commit it.
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.3
> Environment: windows
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Minor
> Fix For: 0.20.3
>
> Attachments: MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2317) HadoopArchives throwing
NullPointerException while creating hadoop archives (.har files)
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13019208#comment-13019208 ]
Devaraj K commented on MAPREDUCE-2317:
--------------------------------------
Thanks for reviewing.
@Nicholas:
{quote}
is it the same that when parent == null, it should not be added to parents list?
{quote}
Yes.
If it is unix system, for root dir it will go here and do nothing.
{code:xml}
Path root = new Path(Path.SEPARATOR);
for (int i = 0; i < deepest.depth(); i++) {
List<Path> parents = new ArrayList<Path>();
for (Path p: justDirs) {
if (p.compareTo(root) == 0){
//don nothing
}
{code}
For windows system, when p is windows root dir (i.e. like C: ), it will go in else and parent will come as null, and causes NullPointerException.
{code:xml}
else {
Path parent = p.getParent();
if (allpaths.containsKey(parent.toString())) {
HashSet<String> children = allpaths.get(parent.toString());
children.add(p.getName());
}
{code}
> HadoopArchives throwing NullPointerException while creating hadoop archives (.har files)
> ----------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2317
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2317
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: harchive
> Affects Versions: 0.20.3
> Environment: windows
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Minor
> Fix For: 0.20.3
>
> Attachments: MAPREDUCE-2317.patch
>
>
> While we are trying to run hadoop archive tool in widows using this way, it is giving the below exception.
> java org.apache.hadoop.tools.HadoopArchives -archiveName temp.har D:/test/in E:/temp
> {code:xml}
> java.lang.NullPointerException
> at org.apache.hadoop.tools.HadoopArchives.writeTopLevelDirs(HadoopArchives.java:320)
> at org.apache.hadoop.tools.HadoopArchives.archive(HadoopArchives.java:386)
> at org.apache.hadoop.tools.HadoopArchives.run(HadoopArchives.java:725)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.HadoopArchives.main(HadoopArchives.java:739)
> {code}
> I see the code flow to handle this feature in windows also,
> {code:title=Path.java|borderStyle=solid}
> /** Returns the parent of a path or null if at root. */
> public Path getParent() {
> String path = uri.getPath();
> int lastSlash = path.lastIndexOf('/');
> int start = hasWindowsDrive(path, true) ? 3 : 0;
> if ((path.length() == start) || // empty path
> (lastSlash == start && path.length() == start+1)) { // at root
> return null;
> }
> String parent;
> if (lastSlash==-1) {
> parent = CUR_DIR;
> } else {
> int end = hasWindowsDrive(path, true) ? 3 : 0;
> parent = path.substring(0, lastSlash==end?end+1:lastSlash);
> }
> return new Path(uri.getScheme(), uri.getAuthority(), parent);
> }
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira