You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Aaron Kimball (JIRA)" <ji...@apache.org> on 2010/06/22 22:20:55 UTC
[jira] Created: (MAPREDUCE-1887) MRAsyncDiskService does not
properly absolutize volume root paths
MRAsyncDiskService does not properly absolutize volume root paths
-----------------------------------------------------------------
Key: MAPREDUCE-1887
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1887
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Aaron Kimball
Assignee: Aaron Kimball
In MRAsyncDiskService, volume names are sometimes specified as relative paths, which are not converted to absolute paths. This can cause errors of the form "cannot delete </full/path/to/foo> since it is outside of <relative/volume/root>" even though the actual path is inside the root.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1887) MRAsyncDiskService does not
properly absolutize volume root paths
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated MAPREDUCE-1887:
-------------------------------------
Attachment: MAPREDUCE-1887.2.patch
new patch to address Todd's comments.
> MRAsyncDiskService does not properly absolutize volume root paths
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1887
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1887
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: MAPREDUCE-1887.2.patch, MAPREDUCE-1887.patch
>
>
> In MRAsyncDiskService, volume names are sometimes specified as relative paths, which are not converted to absolute paths. This can cause errors of the form "cannot delete </full/path/to/foo> since it is outside of <relative/volume/root>" even though the actual path is inside the root.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1887) MRAsyncDiskService does not
properly absolutize volume root paths
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881369#action_12881369 ]
Todd Lipcon commented on MAPREDUCE-1887:
----------------------------------------
lgtm. +1
> MRAsyncDiskService does not properly absolutize volume root paths
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1887
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1887
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: MAPREDUCE-1887.2.patch, MAPREDUCE-1887.patch
>
>
> In MRAsyncDiskService, volume names are sometimes specified as relative paths, which are not converted to absolute paths. This can cause errors of the form "cannot delete </full/path/to/foo> since it is outside of <relative/volume/root>" even though the actual path is inside the root.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1887) MRAsyncDiskService does not
properly absolutize volume root paths
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12882412#action_12882412 ]
Aaron Kimball commented on MAPREDUCE-1887:
------------------------------------------
The only failing test has failed for the last 37 builds. Unrelated to this patch. I think we're good.
> MRAsyncDiskService does not properly absolutize volume root paths
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1887
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1887
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: MAPREDUCE-1887.2.patch, MAPREDUCE-1887.3.patch, MAPREDUCE-1887.patch
>
>
> In MRAsyncDiskService, volume names are sometimes specified as relative paths, which are not converted to absolute paths. This can cause errors of the form "cannot delete </full/path/to/foo> since it is outside of <relative/volume/root>" even though the actual path is inside the root.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1887) MRAsyncDiskService does not
properly absolutize volume root paths
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12882132#action_12882132 ]
Hadoop QA commented on MAPREDUCE-1887:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12447778/MAPREDUCE-1887.3.patch
against trunk revision 957437.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/585/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/585/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/585/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/585/console
This message is automatically generated.
> MRAsyncDiskService does not properly absolutize volume root paths
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1887
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1887
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: MAPREDUCE-1887.2.patch, MAPREDUCE-1887.3.patch, MAPREDUCE-1887.patch
>
>
> In MRAsyncDiskService, volume names are sometimes specified as relative paths, which are not converted to absolute paths. This can cause errors of the form "cannot delete </full/path/to/foo> since it is outside of <relative/volume/root>" even though the actual path is inside the root.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1887) MRAsyncDiskService does not
properly absolutize volume root paths
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated MAPREDUCE-1887:
----------------------------------
Status: Open (was: Patch Available)
> MRAsyncDiskService does not properly absolutize volume root paths
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1887
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1887
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: MAPREDUCE-1887.2.patch, MAPREDUCE-1887.patch
>
>
> In MRAsyncDiskService, volume names are sometimes specified as relative paths, which are not converted to absolute paths. This can cause errors of the form "cannot delete </full/path/to/foo> since it is outside of <relative/volume/root>" even though the actual path is inside the root.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1887) MRAsyncDiskService does not
properly absolutize volume root paths
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881596#action_12881596 ]
Ravi Gummadi commented on MAPREDUCE-1887:
-----------------------------------------
Yes.
Fine with me even if it is not modified as it is not a big perf issue.
> MRAsyncDiskService does not properly absolutize volume root paths
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1887
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1887
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: MAPREDUCE-1887.2.patch, MAPREDUCE-1887.3.patch, MAPREDUCE-1887.patch
>
>
> In MRAsyncDiskService, volume names are sometimes specified as relative paths, which are not converted to absolute paths. This can cause errors of the form "cannot delete </full/path/to/foo> since it is outside of <relative/volume/root>" even though the actual path is inside the root.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1887) MRAsyncDiskService does not
properly absolutize volume root paths
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881347#action_12881347 ]
Todd Lipcon commented on MAPREDUCE-1887:
----------------------------------------
Couple notes:
- Rather than changing all the instances of "volumes" to "this.volumes" can you rename one of the variables? like rename the constructor argument to unnormalizedVolumes, and then you can just use "volumes" throughout? That should reduce changes that someone changing this code later makes the mistake again.
- Missing @Test annotation on testRelativeToWorking()
- Might be worth adding a comment to testVolumeNormalization stating it's a regression test for this JIRA number.
- Either remove or change the log message "Volume: foo -> bar" - eg "Normalized MRAsyncDiskService volume foo -> bar" or just get rid of it.
> MRAsyncDiskService does not properly absolutize volume root paths
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1887
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1887
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: MAPREDUCE-1887.patch
>
>
> In MRAsyncDiskService, volume names are sometimes specified as relative paths, which are not converted to absolute paths. This can cause errors of the form "cannot delete </full/path/to/foo> since it is outside of <relative/volume/root>" even though the actual path is inside the root.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1887) MRAsyncDiskService does not
properly absolutize volume root paths
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated MAPREDUCE-1887:
-------------------------------------
Status: Open (was: Patch Available)
> MRAsyncDiskService does not properly absolutize volume root paths
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1887
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1887
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: MAPREDUCE-1887.2.patch, MAPREDUCE-1887.3.patch, MAPREDUCE-1887.patch
>
>
> In MRAsyncDiskService, volume names are sometimes specified as relative paths, which are not converted to absolute paths. This can cause errors of the form "cannot delete </full/path/to/foo> since it is outside of <relative/volume/root>" even though the actual path is inside the root.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1887) MRAsyncDiskService does not
properly absolutize volume root paths
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881879#action_12881879 ]
Aaron Kimball commented on MAPREDUCE-1887:
------------------------------------------
Unfortunately, Hudson's web server is not responding to me, so not at present. I've been trying every couple hours but no luck yet.
> MRAsyncDiskService does not properly absolutize volume root paths
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1887
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1887
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: MAPREDUCE-1887.2.patch, MAPREDUCE-1887.3.patch, MAPREDUCE-1887.patch
>
>
> In MRAsyncDiskService, volume names are sometimes specified as relative paths, which are not converted to absolute paths. This can cause errors of the form "cannot delete </full/path/to/foo> since it is outside of <relative/volume/root>" even though the actual path is inside the root.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1887) MRAsyncDiskService does not
properly absolutize volume root paths
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated MAPREDUCE-1887:
-------------------------------------
Status: Patch Available (was: Open)
> MRAsyncDiskService does not properly absolutize volume root paths
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1887
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1887
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: MAPREDUCE-1887.2.patch, MAPREDUCE-1887.3.patch, MAPREDUCE-1887.patch
>
>
> In MRAsyncDiskService, volume names are sometimes specified as relative paths, which are not converted to absolute paths. This can cause errors of the form "cannot delete </full/path/to/foo> since it is outside of <relative/volume/root>" even though the actual path is inside the root.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1887) MRAsyncDiskService does not
properly absolutize volume root paths
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated MAPREDUCE-1887:
-------------------------------------
Status: Patch Available (was: Open)
> MRAsyncDiskService does not properly absolutize volume root paths
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1887
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1887
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: MAPREDUCE-1887.patch
>
>
> In MRAsyncDiskService, volume names are sometimes specified as relative paths, which are not converted to absolute paths. This can cause errors of the form "cannot delete </full/path/to/foo> since it is outside of <relative/volume/root>" even though the actual path is inside the root.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1887) MRAsyncDiskService does not
properly absolutize volume root paths
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881560#action_12881560 ]
Hadoop QA commented on MAPREDUCE-1887:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12447778/MAPREDUCE-1887.3.patch
against trunk revision 957086.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/260/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/260/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/260/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/260/console
This message is automatically generated.
> MRAsyncDiskService does not properly absolutize volume root paths
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1887
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1887
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: MAPREDUCE-1887.2.patch, MAPREDUCE-1887.3.patch, MAPREDUCE-1887.patch
>
>
> In MRAsyncDiskService, volume names are sometimes specified as relative paths, which are not converted to absolute paths. This can cause errors of the form "cannot delete </full/path/to/foo> since it is outside of <relative/volume/root>" even though the actual path is inside the root.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1887) MRAsyncDiskService does not
properly absolutize volume root paths
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated MAPREDUCE-1887:
-------------------------------------
Status: Patch Available (was: Open)
Trying again with Hudson, to get this result more accessible.
> MRAsyncDiskService does not properly absolutize volume root paths
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1887
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1887
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: MAPREDUCE-1887.2.patch, MAPREDUCE-1887.3.patch, MAPREDUCE-1887.patch
>
>
> In MRAsyncDiskService, volume names are sometimes specified as relative paths, which are not converted to absolute paths. This can cause errors of the form "cannot delete </full/path/to/foo> since it is outside of <relative/volume/root>" even though the actual path is inside the root.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1887) MRAsyncDiskService does not
properly absolutize volume root paths
Posted by "Ravi Gummadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881571#action_12881571 ]
Ravi Gummadi commented on MAPREDUCE-1887:
-----------------------------------------
LOG.debug() statement is not in LOG.isDebugEnabled() check and causes unnecessary string concatenations in the for loop. Would you please add if(LOG.isDebugEnabled()) { } around that ?
> MRAsyncDiskService does not properly absolutize volume root paths
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1887
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1887
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: MAPREDUCE-1887.2.patch, MAPREDUCE-1887.3.patch, MAPREDUCE-1887.patch
>
>
> In MRAsyncDiskService, volume names are sometimes specified as relative paths, which are not converted to absolute paths. This can cause errors of the form "cannot delete </full/path/to/foo> since it is outside of <relative/volume/root>" even though the actual path is inside the root.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1887) MRAsyncDiskService does not
properly absolutize volume root paths
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated MAPREDUCE-1887:
-------------------------------------
Attachment: MAPREDUCE-1887.3.patch
Agree that the javadoc comment was confusing. Updated. Thanks for the fast review!
> MRAsyncDiskService does not properly absolutize volume root paths
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1887
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1887
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: MAPREDUCE-1887.2.patch, MAPREDUCE-1887.3.patch, MAPREDUCE-1887.patch
>
>
> In MRAsyncDiskService, volume names are sometimes specified as relative paths, which are not converted to absolute paths. This can cause errors of the form "cannot delete </full/path/to/foo> since it is outside of <relative/volume/root>" even though the actual path is inside the root.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1887) MRAsyncDiskService does not
properly absolutize volume root paths
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881875#action_12881875 ]
Zheng Shao commented on MAPREDUCE-1887:
---------------------------------------
Aaron, can you take a look at the unit test failures?
> MRAsyncDiskService does not properly absolutize volume root paths
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1887
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1887
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: MAPREDUCE-1887.2.patch, MAPREDUCE-1887.3.patch, MAPREDUCE-1887.patch
>
>
> In MRAsyncDiskService, volume names are sometimes specified as relative paths, which are not converted to absolute paths. This can cause errors of the form "cannot delete </full/path/to/foo> since it is outside of <relative/volume/root>" even though the actual path is inside the root.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1887) MRAsyncDiskService does not
properly absolutize volume root paths
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated MAPREDUCE-1887:
-------------------------------------
Attachment: MAPREDUCE-1887.patch
This has caused intermittent failure of TestMRServerPorts on our hudson instance. I am attaching a patch with a test case that reliably reproduces this error as well as the fix.
> MRAsyncDiskService does not properly absolutize volume root paths
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1887
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1887
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: MAPREDUCE-1887.patch
>
>
> In MRAsyncDiskService, volume names are sometimes specified as relative paths, which are not converted to absolute paths. This can cause errors of the form "cannot delete </full/path/to/foo> since it is outside of <relative/volume/root>" even though the actual path is inside the root.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1887) MRAsyncDiskService does not
properly absolutize volume root paths
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881583#action_12881583 ]
Todd Lipcon commented on MAPREDUCE-1887:
----------------------------------------
Ravi: this log message only runs once at tasktracker startup, and the number of volumes is usually <12. Not worth guarding in isDebugEnabled since it's not going to ever be a bottleneck.
> MRAsyncDiskService does not properly absolutize volume root paths
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1887
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1887
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: MAPREDUCE-1887.2.patch, MAPREDUCE-1887.3.patch, MAPREDUCE-1887.patch
>
>
> In MRAsyncDiskService, volume names are sometimes specified as relative paths, which are not converted to absolute paths. This can cause errors of the form "cannot delete </full/path/to/foo> since it is outside of <relative/volume/root>" even though the actual path is inside the root.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1887) MRAsyncDiskService does not
properly absolutize volume root paths
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12882372#action_12882372 ]
Zheng Shao commented on MAPREDUCE-1887:
---------------------------------------
Can you take a look at the failed contrib tests?
> MRAsyncDiskService does not properly absolutize volume root paths
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1887
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1887
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: MAPREDUCE-1887.2.patch, MAPREDUCE-1887.3.patch, MAPREDUCE-1887.patch
>
>
> In MRAsyncDiskService, volume names are sometimes specified as relative paths, which are not converted to absolute paths. This can cause errors of the form "cannot delete </full/path/to/foo> since it is outside of <relative/volume/root>" even though the actual path is inside the root.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1887) MRAsyncDiskService does not
properly absolutize volume root paths
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881397#action_12881397 ]
Zheng Shao commented on MAPREDUCE-1887:
---------------------------------------
Code looks good.
Can we change
{code}
+ * @param nonCanonicalVols The roots of the file system volumes, which may not
+ * be canonical paths.
{code}
to
{code}
+ * @param nonCanonicalVols The roots of the file system volumes, which can be
+ * absolute paths from root or relative path from cwd.
{code}
?
I think the second one is easier to understand.
> MRAsyncDiskService does not properly absolutize volume root paths
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1887
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1887
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: MAPREDUCE-1887.2.patch, MAPREDUCE-1887.patch
>
>
> In MRAsyncDiskService, volume names are sometimes specified as relative paths, which are not converted to absolute paths. This can cause errors of the form "cannot delete </full/path/to/foo> since it is outside of <relative/volume/root>" even though the actual path is inside the root.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1887) MRAsyncDiskService does not
properly absolutize volume root paths
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated MAPREDUCE-1887:
----------------------------------
Status: Resolved (was: Patch Available)
Hadoop Flags: [Reviewed]
Release Note: MAPREDUCE-1887. MRAsyncDiskService now properly absolutizes volume root paths. (Aaron Kimball via zshao)
Fix Version/s: 0.22.0
Resolution: Fixed
Committed revision 957772. Thanks Aaron!
> MRAsyncDiskService does not properly absolutize volume root paths
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1887
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1887
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-1887.2.patch, MAPREDUCE-1887.3.patch, MAPREDUCE-1887.patch
>
>
> In MRAsyncDiskService, volume names are sometimes specified as relative paths, which are not converted to absolute paths. This can cause errors of the form "cannot delete </full/path/to/foo> since it is outside of <relative/volume/root>" even though the actual path is inside the root.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1887) MRAsyncDiskService does not
properly absolutize volume root paths
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-1887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12926076#action_12926076 ]
Hudson commented on MAPREDUCE-1887:
-----------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/])
> MRAsyncDiskService does not properly absolutize volume root paths
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-1887
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1887
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-1887.2.patch, MAPREDUCE-1887.3.patch, MAPREDUCE-1887.patch
>
>
> In MRAsyncDiskService, volume names are sometimes specified as relative paths, which are not converted to absolute paths. This can cause errors of the form "cannot delete </full/path/to/foo> since it is outside of <relative/volume/root>" even though the actual path is inside the root.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.