You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Ivan Mitic (JIRA)" <ji...@apache.org> on 2012/06/06 23:35:23 UTC
[jira] [Created] (HADOOP-8487) Address test failures related to
Windows paths not being valid DFS paths
Ivan Mitic created HADOOP-8487:
----------------------------------
Summary: Address test failures related to Windows paths not being valid DFS paths
Key: HADOOP-8487
URL: https://issues.apache.org/jira/browse/HADOOP-8487
Project: Hadoop Common
Issue Type: Bug
Components: test
Reporter: Ivan Mitic
There is a number of tests that fail on Windows because Hadoop's distributed file system does not allow colon character in DFS paths. Specifically, passing in the following path to DFS:
{code}/c:/some/path{code}
would fail DFSUtil#isValidName check and cause the current operation to fail. Any test that is using local absolute path in the context of the DFS will fail because of this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8487) Address test failures related to
Windows paths not being valid DFS paths
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Mitic updated HADOOP-8487:
-------------------------------
Attachment: HADOOP-8487-branch-1-win(3).patch
Thanks for reviewing Sanjay. Attaching updated patch that addresses your comment.
> Address test failures related to Windows paths not being valid DFS paths
> ------------------------------------------------------------------------
>
> Key: HADOOP-8487
> URL: https://issues.apache.org/jira/browse/HADOOP-8487
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
> Reporter: Ivan Mitic
> Attachments: HADOOP-8487-branch-1-win(2).patch, HADOOP-8487-branch-1-win(3).patch, HADOOP-8487-branch-1-win.alternate.patch, HADOOP-8487-branch-1-win.patch
>
>
> There is a number of tests that fail on Windows because Hadoop's distributed file system does not allow colon character in DFS paths. Specifically, passing in the following path to DFS:
> {code}/c:/some/path{code}
> would fail DFSUtil#isValidName check and cause the current operation to fail. Any test that is using local absolute path in the context of the DFS will fail because of this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8487) Many HDFS tests use a test path
intended for local file system tests
Posted by "Sanjay Radia (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406639#comment-13406639 ]
Sanjay Radia commented on HADOOP-8487:
--------------------------------------
I have committed this to Hadoop-1 windows branch. Thanks Ivan.
Ivan, please provide a patch for trunk and I will commit this to trunk.
> Many HDFS tests use a test path intended for local file system tests
> --------------------------------------------------------------------
>
> Key: HADOOP-8487
> URL: https://issues.apache.org/jira/browse/HADOOP-8487
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8487-branch-1-win(2).patch, HADOOP-8487-branch-1-win(3).patch, HADOOP-8487-branch-1-win.alternate.patch, HADOOP-8487-branch-1-win.patch
>
>
> Many tests use a test path intended for local tests setup by build environment. In some cases the tests fails on platforms such as windows because the path contains a c:
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8487) Address test failures related to
Windows paths not being valid DFS paths
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Mitic updated HADOOP-8487:
-------------------------------
Attachment: HADOOP-8487-branch-1-win.alternate.patch
@Daryn: Another idea I was thinking about is to allow DFS paths with a colon (for example, let {{/c:/some/path}} be a valid DFS path). I attached an alternate patch that showcases this approach. Ideally we would scope this down to test only, but it is not obvious how.
> Address test failures related to Windows paths not being valid DFS paths
> ------------------------------------------------------------------------
>
> Key: HADOOP-8487
> URL: https://issues.apache.org/jira/browse/HADOOP-8487
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
> Reporter: Ivan Mitic
> Attachments: HADOOP-8487-branch-1-win.alternate.patch, HADOOP-8487-branch-1-win.patch
>
>
> There is a number of tests that fail on Windows because Hadoop's distributed file system does not allow colon character in DFS paths. Specifically, passing in the following path to DFS:
> {code}/c:/some/path{code}
> would fail DFSUtil#isValidName check and cause the current operation to fail. Any test that is using local absolute path in the context of the DFS will fail because of this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-8487) Many HDFS tests use a test path
intended for local file system tests
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Mitic resolved HADOOP-8487.
--------------------------------
Resolution: Fixed
Resolving as this is committed to branch-1-win. This way, the state of active Jiras is up to date under HADOOP-8645. Will reference this Jira once we get to fixing this in trunk.
> Many HDFS tests use a test path intended for local file system tests
> --------------------------------------------------------------------
>
> Key: HADOOP-8487
> URL: https://issues.apache.org/jira/browse/HADOOP-8487
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8487-branch-1-win(2).patch, HADOOP-8487-branch-1-win(3).patch, HADOOP-8487-branch-1-win(3).update.patch, HADOOP-8487-branch-1-win.alternate.patch, HADOOP-8487-branch-1-win.patch
>
>
> Many tests use a test path intended for local tests setup by build environment. In some cases the tests fails on platforms such as windows because the path contains a c:
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8487) Address test failures related to
Windows paths not being valid DFS paths
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291030#comment-13291030 ]
Daryn Sharp commented on HADOOP-8487:
-------------------------------------
Quick question, why is {{WebHdfsFileSystem}} special cased in a few instances?
> Address test failures related to Windows paths not being valid DFS paths
> ------------------------------------------------------------------------
>
> Key: HADOOP-8487
> URL: https://issues.apache.org/jira/browse/HADOOP-8487
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
> Reporter: Ivan Mitic
> Attachments: HADOOP-8487-branch-1-win.patch
>
>
> There is a number of tests that fail on Windows because Hadoop's distributed file system does not allow colon character in DFS paths. Specifically, passing in the following path to DFS:
> {code}/c:/some/path{code}
> would fail DFSUtil#isValidName check and cause the current operation to fail. Any test that is using local absolute path in the context of the DFS will fail because of this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8487) Many HDFS tests use a test path
intended for local file system tests
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409445#comment-13409445 ]
Daryn Sharp commented on HADOOP-8487:
-------------------------------------
A few small comments:
# In {{FileSystemTestHelper}}, does the {{final}} keyword need to be removed?
# In {{TestFSMainOperationsLocalFileSystem}}, are any changes actually needed? Ie. why override just to call {{super}}?
# In all of test class changes, please default to build/test/data. Hardcoding /tmp may cause multiple test runs to collide.
> Many HDFS tests use a test path intended for local file system tests
> --------------------------------------------------------------------
>
> Key: HADOOP-8487
> URL: https://issues.apache.org/jira/browse/HADOOP-8487
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8487-branch-1-win(2).patch, HADOOP-8487-branch-1-win(3).patch, HADOOP-8487-branch-1-win.alternate.patch, HADOOP-8487-branch-1-win.patch
>
>
> Many tests use a test path intended for local tests setup by build environment. In some cases the tests fails on platforms such as windows because the path contains a c:
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8487) Address test failures related to
Windows paths not being valid DFS paths
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291342#comment-13291342 ]
Ivan Mitic commented on HADOOP-8487:
------------------------------------
bq. why is WebHdfsFileSystem special cased in a few instances?
Test utility class {{FileSystemTestHelper}} is used by two tests, {{TestFSMainOperationsLocalFileSystem}} and {{TestFSMainOperationsWebHdfs}}. The former test uses DFS and the latter uses LocalFileSystem. I could change {{FileSystemTestHelper}} to always use {{DFS_TEST_ROOT_DIR}} but in that case, local files would be created under a path different than {{System.getProperty("test.build.data")}} what I was trying to avoid in my changes.
> Address test failures related to Windows paths not being valid DFS paths
> ------------------------------------------------------------------------
>
> Key: HADOOP-8487
> URL: https://issues.apache.org/jira/browse/HADOOP-8487
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
> Reporter: Ivan Mitic
> Attachments: HADOOP-8487-branch-1-win.patch
>
>
> There is a number of tests that fail on Windows because Hadoop's distributed file system does not allow colon character in DFS paths. Specifically, passing in the following path to DFS:
> {code}/c:/some/path{code}
> would fail DFSUtil#isValidName check and cause the current operation to fail. Any test that is using local absolute path in the context of the DFS will fail because of this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8487) Address test failures related to
Windows paths not being valid DFS paths
Posted by "Sanjay Radia (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406103#comment-13406103 ]
Sanjay Radia commented on HADOOP-8487:
--------------------------------------
For a few places, if you leave the the variable name to TEST_DIR, rather than change it to DFS_TEST_PATH, the patch becomes a lot smaller. TEST_DIR is a good enough variable name.
> Address test failures related to Windows paths not being valid DFS paths
> ------------------------------------------------------------------------
>
> Key: HADOOP-8487
> URL: https://issues.apache.org/jira/browse/HADOOP-8487
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
> Reporter: Ivan Mitic
> Attachments: HADOOP-8487-branch-1-win(2).patch, HADOOP-8487-branch-1-win.alternate.patch, HADOOP-8487-branch-1-win.patch
>
>
> There is a number of tests that fail on Windows because Hadoop's distributed file system does not allow colon character in DFS paths. Specifically, passing in the following path to DFS:
> {code}/c:/some/path{code}
> would fail DFSUtil#isValidName check and cause the current operation to fail. Any test that is using local absolute path in the context of the DFS will fail because of this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8487) Many HDFS tests use a test path
intended for local file system tests
Posted by "Sanjay Radia (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sanjay Radia updated HADOOP-8487:
---------------------------------
Description:
Many tests use a test path intended for local tests setup by build environment. In some cases the tests fails on platforms such as windows because the path contains a c:
was:
There is a number of tests that fail on Windows because Hadoop's distributed file system does not allow colon character in DFS paths. Specifically, passing in the following path to DFS:
{code}/c:/some/path{code}
would fail DFSUtil#isValidName check and cause the current operation to fail. Any test that is using local absolute path in the context of the DFS will fail because of this.
Assignee: Ivan Mitic
Summary: Many HDFS tests use a test path intended for local file system tests (was: Address test failures related to Windows paths not being valid DFS paths)
> Many HDFS tests use a test path intended for local file system tests
> --------------------------------------------------------------------
>
> Key: HADOOP-8487
> URL: https://issues.apache.org/jira/browse/HADOOP-8487
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8487-branch-1-win(2).patch, HADOOP-8487-branch-1-win(3).patch, HADOOP-8487-branch-1-win.alternate.patch, HADOOP-8487-branch-1-win.patch
>
>
> Many tests use a test path intended for local tests setup by build environment. In some cases the tests fails on platforms such as windows because the path contains a c:
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8487) Address test failures related to
Windows paths not being valid DFS paths
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Mitic updated HADOOP-8487:
-------------------------------
Attachment: HADOOP-8487-branch-1-win.patch
Attaching patch with the fix proposal. This fix modifies problematic tests to use an "abstract" hardcode path instead of System.getProperty("test.build.data") path, as the later one resolves to a local path that fails the DFS check.
This was initially part of the patch for HADOOP-8409 but was spit as it generally represents an issue on its own.
> Address test failures related to Windows paths not being valid DFS paths
> ------------------------------------------------------------------------
>
> Key: HADOOP-8487
> URL: https://issues.apache.org/jira/browse/HADOOP-8487
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
> Reporter: Ivan Mitic
> Attachments: HADOOP-8487-branch-1-win.patch
>
>
> There is a number of tests that fail on Windows because Hadoop's distributed file system does not allow colon character in DFS paths. Specifically, passing in the following path to DFS:
> {code}/c:/some/path{code}
> would fail DFSUtil#isValidName check and cause the current operation to fail. Any test that is using local absolute path in the context of the DFS will fail because of this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8487) Many HDFS tests use a test path
intended for local file system tests
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Mitic updated HADOOP-8487:
-------------------------------
Attachment: HADOOP-8487-branch-1-win(3).update.patch
Apologies for the late response Daryn. To follow up on your commend #2 from above, I verified and the super call is not needed. Given that this is already committed and the actual change is small/irrelevant, it might be an overhead to fix this. I attached the patch that reverts this part anyways, your call on whether you want to commit this :)
Thanks for catching this!
> Many HDFS tests use a test path intended for local file system tests
> --------------------------------------------------------------------
>
> Key: HADOOP-8487
> URL: https://issues.apache.org/jira/browse/HADOOP-8487
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8487-branch-1-win(2).patch, HADOOP-8487-branch-1-win(3).patch, HADOOP-8487-branch-1-win(3).update.patch, HADOOP-8487-branch-1-win.alternate.patch, HADOOP-8487-branch-1-win.patch
>
>
> Many tests use a test path intended for local tests setup by build environment. In some cases the tests fails on platforms such as windows because the path contains a c:
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8487) Many HDFS tests use a test path
intended for local file system tests
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13442467#comment-13442467 ]
Daryn Sharp commented on HADOOP-8487:
-------------------------------------
Thanks for the followup, but it's not a big deal since it's a test.
> Many HDFS tests use a test path intended for local file system tests
> --------------------------------------------------------------------
>
> Key: HADOOP-8487
> URL: https://issues.apache.org/jira/browse/HADOOP-8487
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8487-branch-1-win(2).patch, HADOOP-8487-branch-1-win(3).patch, HADOOP-8487-branch-1-win(3).update.patch, HADOOP-8487-branch-1-win.alternate.patch, HADOOP-8487-branch-1-win.patch
>
>
> Many tests use a test path intended for local tests setup by build environment. In some cases the tests fails on platforms such as windows because the path contains a c:
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8487) Many HDFS tests use a test path
intended for local file system tests
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13442473#comment-13442473 ]
Ivan Mitic commented on HADOOP-8487:
------------------------------------
Great, thanks!
> Many HDFS tests use a test path intended for local file system tests
> --------------------------------------------------------------------
>
> Key: HADOOP-8487
> URL: https://issues.apache.org/jira/browse/HADOOP-8487
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8487-branch-1-win(2).patch, HADOOP-8487-branch-1-win(3).patch, HADOOP-8487-branch-1-win(3).update.patch, HADOOP-8487-branch-1-win.alternate.patch, HADOOP-8487-branch-1-win.patch
>
>
> Many tests use a test path intended for local tests setup by build environment. In some cases the tests fails on platforms such as windows because the path contains a c:
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8487) Many HDFS tests use a test path
intended for local file system tests
Posted by "Daryn Sharp (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413729#comment-13413729 ]
Daryn Sharp commented on HADOOP-8487:
-------------------------------------
Makes perfect sense. I'm fine with /tmp as long as it's always a path in dfs and not the local fs.
> Many HDFS tests use a test path intended for local file system tests
> --------------------------------------------------------------------
>
> Key: HADOOP-8487
> URL: https://issues.apache.org/jira/browse/HADOOP-8487
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8487-branch-1-win(2).patch, HADOOP-8487-branch-1-win(3).patch, HADOOP-8487-branch-1-win.alternate.patch, HADOOP-8487-branch-1-win.patch
>
>
> Many tests use a test path intended for local tests setup by build environment. In some cases the tests fails on platforms such as windows because the path contains a c:
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8487) Many HDFS tests use a test path
intended for local file system tests
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13413443#comment-13413443 ]
Ivan Mitic commented on HADOOP-8487:
------------------------------------
Thanks for reviewing Daryn.
bq. In FileSystemTestHelper, does the final keyword need to be removed?
We need this because we want to be able to override the value by tests ({{TestFSMainOperationsWebHdfs}} is one example). This path is not always used in the context of the local path, and this causes problems on Windows, as paths like {{c:/some/path/build/test/data}} are not valid DFS paths (because of the colon).
bq. In TestFSMainOperationsLocalFileSystem, are any changes actually needed? Ie. why override just to call super?
I think I saw this test failing on Windows because the super {{tearDown()}} was not called, causing subsequent tests to fail (could it be related to junit version?). Will try to repro the problem and report back.
bq. In all of test class changes, please default to build/test/data. Hardcoding /tmp may cause multiple test runs to collide.
This is actually the test fix. Similar comment to #1, {{build.test.data}} is a local path and given that it is used in HDFS tests, it fails the valid DFS path check. IOW, these tests should not write to the local file system, and if I understood your worry correctly, should not collide with other tests. We also have the test name embedded in the path, so it should be easy to spot such cases if they exists. Does it make sense?
> Many HDFS tests use a test path intended for local file system tests
> --------------------------------------------------------------------
>
> Key: HADOOP-8487
> URL: https://issues.apache.org/jira/browse/HADOOP-8487
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
> Reporter: Ivan Mitic
> Assignee: Ivan Mitic
> Attachments: HADOOP-8487-branch-1-win(2).patch, HADOOP-8487-branch-1-win(3).patch, HADOOP-8487-branch-1-win.alternate.patch, HADOOP-8487-branch-1-win.patch
>
>
> Many tests use a test path intended for local tests setup by build environment. In some cases the tests fails on platforms such as windows because the path contains a c:
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8487) Address test failures related to
Windows paths not being valid DFS paths
Posted by "Ivan Mitic (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Mitic updated HADOOP-8487:
-------------------------------
Attachment: HADOOP-8487-branch-1-win(2).patch
Attaching updated patch.
New patch fixes the following tests on Windows:
TestBBWBlockReport
TestNameNodeMetrics
TestJobQueueInformation
TestSubmitJob
TestFSMainOperationsWebHdfs
TestMultipleCachefiles
TestStreamingBadRecords
TestSymLink
TestDistCacheEmulation
TestSleepJob
> Address test failures related to Windows paths not being valid DFS paths
> ------------------------------------------------------------------------
>
> Key: HADOOP-8487
> URL: https://issues.apache.org/jira/browse/HADOOP-8487
> Project: Hadoop Common
> Issue Type: Bug
> Components: test
> Reporter: Ivan Mitic
> Attachments: HADOOP-8487-branch-1-win(2).patch, HADOOP-8487-branch-1-win.alternate.patch, HADOOP-8487-branch-1-win.patch
>
>
> There is a number of tests that fail on Windows because Hadoop's distributed file system does not allow colon character in DFS paths. Specifically, passing in the following path to DFS:
> {code}/c:/some/path{code}
> would fail DFSUtil#isValidName check and cause the current operation to fail. Any test that is using local absolute path in the context of the DFS will fail because of this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira