You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Nigel Daley (JIRA)" <ji...@apache.org> on 2007/02/14 21:16:05 UTC
[jira] Created: (HADOOP-1020) Path class on Windows seems broken
Path class on Windows seems broken
----------------------------------
Key: HADOOP-1020
URL: https://issues.apache.org/jira/browse/HADOOP-1020
Project: Hadoop
Issue Type: Bug
Components: dfs
Affects Versions: 0.11.1
Reporter: Nigel Daley
Executing this code:
Path file = new Path("file:///", "C:/trunk/build/test/data"); or Path file = new Path("C:/trunk/build/test/data");
FileSystem fs = file.getFileSystem(conf);
fs.mkdirs(file))
produces this exception. It looks like it's defaulting to the DistributedFileSystem.
2007-02-14 11:36:31,700 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(296)) - Error from task_0001_m_000000_0: java.lang.IllegalArgumentException: Pathname /C:/trunk/build/test/data from C:/trunk/build/test/data is not a valid DFS filename.
at org.apache.hadoop.dfs.DistributedFileSystem.getPath(DistributedFileSystem.java:111)
at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:211)
at org.apache.hadoop.mapred.MRCaching$MapClass.configure(MRCaching.java:68)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:178)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1396)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Created: (HADOOP-1020) Path class on Windows seems broken
Posted by Nigel Daley <nd...@yahoo-inc.com>.
This is blocking me from fixing TestMiniMRDFSCaching, which is
currently broken on Windows.
On Feb 14, 2007, at 12:16 PM, Nigel Daley (JIRA) wrote:
> Path class on Windows seems broken
> ----------------------------------
>
> Key: HADOOP-1020
> URL: https://issues.apache.org/jira/browse/
> HADOOP-1020
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.11.1
> Reporter: Nigel Daley
>
>
> Executing this code:
>
> Path file = new Path("file:///", "C:/trunk/build/test/data");
> or Path file = new Path("C:/trunk/build/test/data");
> FileSystem fs = file.getFileSystem(conf);
> fs.mkdirs(file))
>
> produces this exception. It looks like it's defaulting to the
> DistributedFileSystem.
>
> 2007-02-14 11:36:31,700 INFO mapred.TaskInProgress
> (TaskInProgress.java:updateStatus(296)) - Error from
> task_0001_m_000000_0: java.lang.IllegalArgumentException: Pathname /
> C:/trunk/build/test/data from C:/trunk/build/test/data is not a
> valid DFS filename.
> at org.apache.hadoop.dfs.DistributedFileSystem.getPath
> (DistributedFileSystem.java:111)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs
> (DistributedFileSystem.java:211)
> at org.apache.hadoop.mapred.MRCaching$MapClass.configure
> (MRCaching.java:68)
> at org.apache.hadoop.util.ReflectionUtils.setConf
> (ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance
> (ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapRunner.configure
> (MapRunner.java:34)
> at org.apache.hadoop.util.ReflectionUtils.setConf
> (ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance
> (ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:178)
> at org.apache.hadoop.mapred.TaskTracker$Child.main
> (TaskTracker.java:1396)
>
>
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
[jira] Commented: (HADOOP-1020) Path class on Windows seems broken
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12476628 ]
Hadoop QA commented on HADOOP-1020:
-----------------------------------
+1, because http://issues.apache.org/jira/secure/attachment/12352189/HADOOP-1020-2.patch applied and successfully tested against trunk revision http://svn.apache.org/repos/asf/lucene/hadoop/trunk/512499. Results are at http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch
> Path class on Windows seems broken
> ----------------------------------
>
> Key: HADOOP-1020
> URL: https://issues.apache.org/jira/browse/HADOOP-1020
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.11.1
> Reporter: Nigel Daley
> Assigned To: Doug Cutting
> Fix For: 0.12.0
>
> Attachments: HADOOP-1020-2.patch, HADOOP-1020.patch, TestPath.java.patch
>
>
> Executing this code:
> Path file = new Path("file:///", "C:/trunk/build/test/data"); or Path file = new Path("C:/trunk/build/test/data");
> FileSystem fs = file.getFileSystem(conf);
> fs.mkdirs(file))
> produces this exception. It looks like it's defaulting to the DistributedFileSystem.
> 2007-02-14 11:36:31,700 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(296)) - Error from task_0001_m_000000_0: java.lang.IllegalArgumentException: Pathname /C:/trunk/build/test/data from C:/trunk/build/test/data is not a valid DFS filename.
> at org.apache.hadoop.dfs.DistributedFileSystem.getPath(DistributedFileSystem.java:111)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:211)
> at org.apache.hadoop.mapred.MRCaching$MapClass.configure(MRCaching.java:68)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:178)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1396)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1020) Path class on Windows seems broken
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12476088 ]
Hadoop QA commented on HADOOP-1020:
-----------------------------------
-1, because 2 attempts failed to build and test the latest attachment (http://issues.apache.org/jira/secure/attachment/12352066/HADOOP-1020.patch</a>) against trunk revision <a href=) against trunk revision http://svn.apache.org/repos/asf/lucene/hadoop/trunk/512006. Please note that this message is automatically generated and may represent a problem with the automation system and not the patch. Results are at http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch
> Path class on Windows seems broken
> ----------------------------------
>
> Key: HADOOP-1020
> URL: https://issues.apache.org/jira/browse/HADOOP-1020
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.11.1
> Reporter: Nigel Daley
> Assigned To: Doug Cutting
> Fix For: 0.12.0
>
> Attachments: HADOOP-1020.patch, TestPath.java.patch
>
>
> Executing this code:
> Path file = new Path("file:///", "C:/trunk/build/test/data"); or Path file = new Path("C:/trunk/build/test/data");
> FileSystem fs = file.getFileSystem(conf);
> fs.mkdirs(file))
> produces this exception. It looks like it's defaulting to the DistributedFileSystem.
> 2007-02-14 11:36:31,700 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(296)) - Error from task_0001_m_000000_0: java.lang.IllegalArgumentException: Pathname /C:/trunk/build/test/data from C:/trunk/build/test/data is not a valid DFS filename.
> at org.apache.hadoop.dfs.DistributedFileSystem.getPath(DistributedFileSystem.java:111)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:211)
> at org.apache.hadoop.mapred.MRCaching$MapClass.configure(MRCaching.java:68)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:178)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1396)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1020) Path class on Windows seems broken
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-1020:
---------------------------------
Attachment: HADOOP-1020-2.patch
The problem with TestSymLink was that hadoop.tmp.dir had c:/... in it, and this is by default used as the basis for hadoop.system.dir, but it's not a valid HDFS path, since it contains a colon, and the system dir is located in the default filesystem, HDFS in this case.
Here's a new version of that patch that fixes that by explicitly specifying a relative path for hadoop.system.dir in contrib tests.
> Path class on Windows seems broken
> ----------------------------------
>
> Key: HADOOP-1020
> URL: https://issues.apache.org/jira/browse/HADOOP-1020
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.11.1
> Reporter: Nigel Daley
> Assigned To: Doug Cutting
> Fix For: 0.12.0
>
> Attachments: HADOOP-1020-2.patch, HADOOP-1020.patch, TestPath.java.patch
>
>
> Executing this code:
> Path file = new Path("file:///", "C:/trunk/build/test/data"); or Path file = new Path("C:/trunk/build/test/data");
> FileSystem fs = file.getFileSystem(conf);
> fs.mkdirs(file))
> produces this exception. It looks like it's defaulting to the DistributedFileSystem.
> 2007-02-14 11:36:31,700 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(296)) - Error from task_0001_m_000000_0: java.lang.IllegalArgumentException: Pathname /C:/trunk/build/test/data from C:/trunk/build/test/data is not a valid DFS filename.
> at org.apache.hadoop.dfs.DistributedFileSystem.getPath(DistributedFileSystem.java:111)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:211)
> at org.apache.hadoop.mapred.MRCaching$MapClass.configure(MRCaching.java:68)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:178)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1396)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1020) Path class on Windows seems broken
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-1020:
---------------------------------
Status: Open (was: Patch Available)
> Path class on Windows seems broken
> ----------------------------------
>
> Key: HADOOP-1020
> URL: https://issues.apache.org/jira/browse/HADOOP-1020
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.11.1
> Reporter: Nigel Daley
> Assigned To: Doug Cutting
> Fix For: 0.12.0
>
> Attachments: HADOOP-1020.patch, TestPath.java.patch
>
>
> Executing this code:
> Path file = new Path("file:///", "C:/trunk/build/test/data"); or Path file = new Path("C:/trunk/build/test/data");
> FileSystem fs = file.getFileSystem(conf);
> fs.mkdirs(file))
> produces this exception. It looks like it's defaulting to the DistributedFileSystem.
> 2007-02-14 11:36:31,700 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(296)) - Error from task_0001_m_000000_0: java.lang.IllegalArgumentException: Pathname /C:/trunk/build/test/data from C:/trunk/build/test/data is not a valid DFS filename.
> at org.apache.hadoop.dfs.DistributedFileSystem.getPath(DistributedFileSystem.java:111)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:211)
> at org.apache.hadoop.mapred.MRCaching$MapClass.configure(MRCaching.java:68)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:178)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1396)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1020) Path class on Windows seems broken
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-1020:
---------------------------------
Status: Patch Available (was: Open)
> Path class on Windows seems broken
> ----------------------------------
>
> Key: HADOOP-1020
> URL: https://issues.apache.org/jira/browse/HADOOP-1020
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.11.1
> Reporter: Nigel Daley
> Assigned To: Doug Cutting
> Fix For: 0.12.0
>
> Attachments: HADOOP-1020-2.patch, HADOOP-1020.patch, TestPath.java.patch
>
>
> Executing this code:
> Path file = new Path("file:///", "C:/trunk/build/test/data"); or Path file = new Path("C:/trunk/build/test/data");
> FileSystem fs = file.getFileSystem(conf);
> fs.mkdirs(file))
> produces this exception. It looks like it's defaulting to the DistributedFileSystem.
> 2007-02-14 11:36:31,700 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(296)) - Error from task_0001_m_000000_0: java.lang.IllegalArgumentException: Pathname /C:/trunk/build/test/data from C:/trunk/build/test/data is not a valid DFS filename.
> at org.apache.hadoop.dfs.DistributedFileSystem.getPath(DistributedFileSystem.java:111)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:211)
> at org.apache.hadoop.mapred.MRCaching$MapClass.configure(MRCaching.java:68)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:178)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1396)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-1020) Path class on Windows seems broken
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting reassigned HADOOP-1020:
------------------------------------
Assignee: Doug Cutting
> Path class on Windows seems broken
> ----------------------------------
>
> Key: HADOOP-1020
> URL: https://issues.apache.org/jira/browse/HADOOP-1020
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.11.1
> Reporter: Nigel Daley
> Assigned To: Doug Cutting
> Attachments: TestPath.java.patch
>
>
> Executing this code:
> Path file = new Path("file:///", "C:/trunk/build/test/data"); or Path file = new Path("C:/trunk/build/test/data");
> FileSystem fs = file.getFileSystem(conf);
> fs.mkdirs(file))
> produces this exception. It looks like it's defaulting to the DistributedFileSystem.
> 2007-02-14 11:36:31,700 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(296)) - Error from task_0001_m_000000_0: java.lang.IllegalArgumentException: Pathname /C:/trunk/build/test/data from C:/trunk/build/test/data is not a valid DFS filename.
> at org.apache.hadoop.dfs.DistributedFileSystem.getPath(DistributedFileSystem.java:111)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:211)
> at org.apache.hadoop.mapred.MRCaching$MapClass.configure(MRCaching.java:68)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:178)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1396)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1020) Path class on Windows seems broken
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-1020:
---------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this.
> Path class on Windows seems broken
> ----------------------------------
>
> Key: HADOOP-1020
> URL: https://issues.apache.org/jira/browse/HADOOP-1020
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.11.1
> Reporter: Nigel Daley
> Assigned To: Doug Cutting
> Fix For: 0.12.0
>
> Attachments: HADOOP-1020-2.patch, HADOOP-1020.patch, TestPath.java.patch
>
>
> Executing this code:
> Path file = new Path("file:///", "C:/trunk/build/test/data"); or Path file = new Path("C:/trunk/build/test/data");
> FileSystem fs = file.getFileSystem(conf);
> fs.mkdirs(file))
> produces this exception. It looks like it's defaulting to the DistributedFileSystem.
> 2007-02-14 11:36:31,700 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(296)) - Error from task_0001_m_000000_0: java.lang.IllegalArgumentException: Pathname /C:/trunk/build/test/data from C:/trunk/build/test/data is not a valid DFS filename.
> at org.apache.hadoop.dfs.DistributedFileSystem.getPath(DistributedFileSystem.java:111)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:211)
> at org.apache.hadoop.mapred.MRCaching$MapClass.configure(MRCaching.java:68)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:178)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1396)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1020) Path class on Windows seems broken
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12476035 ]
Hadoop QA commented on HADOOP-1020:
-----------------------------------
-1, because 2 attempts failed to build and test the latest attachment (http://issues.apache.org/jira/secure/attachment/12352066/HADOOP-1020.patch) against trunk revision http://svn.apache.org/repos/asf/lucene/hadoop/trunk/512006. Please note that this message is automatically generated and may represent a problem with the automation system and not the patch. Results are at http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch
> Path class on Windows seems broken
> ----------------------------------
>
> Key: HADOOP-1020
> URL: https://issues.apache.org/jira/browse/HADOOP-1020
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.11.1
> Reporter: Nigel Daley
> Assigned To: Doug Cutting
> Fix For: 0.12.0
>
> Attachments: HADOOP-1020.patch, TestPath.java.patch
>
>
> Executing this code:
> Path file = new Path("file:///", "C:/trunk/build/test/data"); or Path file = new Path("C:/trunk/build/test/data");
> FileSystem fs = file.getFileSystem(conf);
> fs.mkdirs(file))
> produces this exception. It looks like it's defaulting to the DistributedFileSystem.
> 2007-02-14 11:36:31,700 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(296)) - Error from task_0001_m_000000_0: java.lang.IllegalArgumentException: Pathname /C:/trunk/build/test/data from C:/trunk/build/test/data is not a valid DFS filename.
> at org.apache.hadoop.dfs.DistributedFileSystem.getPath(DistributedFileSystem.java:111)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:211)
> at org.apache.hadoop.mapred.MRCaching$MapClass.configure(MRCaching.java:68)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:178)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1396)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1020) Path class on Windows seems broken
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nigel Daley updated HADOOP-1020:
--------------------------------
Attachment: TestPath.java.patch
The TestPath.java.patch attachment adds the suggested test case to TestPath. The last 2 assertions in TestPath.testGetFileSystem fail on both Windows and Linux:
assertEquals(new Path("file:///","C:/foo/bar").getFileSystem(conf), FileSystem.getLocal(conf));
assertEquals(new Path("file:///","/foo/bar").getFileSystem(conf), FileSystem.getLocal(conf));
where the conf object has an "fs.default.name" set to a dfs filesytem.
The second assertion on Linux and both on Windows fails with:
expected:<DFS[org.apache.hadoop.dfs.DFSClient@1cd66ea]> but was:<LocalFS>
junit.framework.AssertionFailedError: expected:<DFS[org.apache.hadoop.dfs.DFSClient@1cd66ea]> but was:<LocalFS>
at org.apache.hadoop.fs.TestPath.testGetFileSystem(TestPath.java:145)
The first assertion on Linux fails with:
No FileSystem for scheme: C
java.io.IOException: No FileSystem for scheme: C
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:157)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:163)
at org.apache.hadoop.fs.TestPath.testGetFileSystem(TestPath.java:145)
Given the lack of specification in Path, the expected behaviour is not clear.
> Path class on Windows seems broken
> ----------------------------------
>
> Key: HADOOP-1020
> URL: https://issues.apache.org/jira/browse/HADOOP-1020
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.11.1
> Reporter: Nigel Daley
> Attachments: TestPath.java.patch
>
>
> Executing this code:
> Path file = new Path("file:///", "C:/trunk/build/test/data"); or Path file = new Path("C:/trunk/build/test/data");
> FileSystem fs = file.getFileSystem(conf);
> fs.mkdirs(file))
> produces this exception. It looks like it's defaulting to the DistributedFileSystem.
> 2007-02-14 11:36:31,700 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(296)) - Error from task_0001_m_000000_0: java.lang.IllegalArgumentException: Pathname /C:/trunk/build/test/data from C:/trunk/build/test/data is not a valid DFS filename.
> at org.apache.hadoop.dfs.DistributedFileSystem.getPath(DistributedFileSystem.java:111)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:211)
> at org.apache.hadoop.mapred.MRCaching$MapClass.configure(MRCaching.java:68)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:178)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1396)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Commented: (HADOOP-1020) Path class on Windows seems broken
Posted by Nigel Daley <nd...@yahoo-inc.com>.
Argh! These -1 are false negatives. I changed the way I determine
the max acceptable warnings, by doing a quick build of trunk (ant
tar) and compare this to the number of warnings from patched build
and test (ant test tar). The problem is that the test target causes
compile-core-test to be executed twice, and thus the total number of
warnings are different. A fix is now in place for this and I'll test
it out tonight.
On Feb 27, 2007, at 4:19 PM, Hadoop QA (JIRA) wrote:
>
> [ https://issues.apache.org/jira/browse/HADOOP-1020?
> page=com.atlassian.jira.plugin.system.issuetabpanels:comment-
> tabpanel#action_12476462 ]
>
> Hadoop QA commented on HADOOP-1020:
> -----------------------------------
>
> -1, because javac generated 768 warnings (more than the acceptable
> 766 warnings) when testing the latest attachment (http://
> issues.apache.org/jira/secure/attachment/12352189/
> HADOOP-1020-2.patch) against trunk revision http://svn.apache.org/
> repos/asf/lucene/hadoop/trunk/512461. Please note that this message
> is automatically generated and may represent a problem with the
> automation system and not the patch. Results are at http://
> lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch
>
>> Path class on Windows seems broken
>> ----------------------------------
>>
>> Key: HADOOP-1020
>> URL: https://issues.apache.org/jira/browse/
>> HADOOP-1020
>> Project: Hadoop
>> Issue Type: Bug
>> Components: dfs
>> Affects Versions: 0.11.1
>> Reporter: Nigel Daley
>> Assigned To: Doug Cutting
>> Fix For: 0.12.0
>>
>> Attachments: HADOOP-1020-2.patch, HADOOP-1020.patch,
>> TestPath.java.patch
>>
>>
>> Executing this code:
>> Path file = new Path("file:///", "C:/trunk/build/test/data");
>> or Path file = new Path("C:/trunk/build/test/data");
>> FileSystem fs = file.getFileSystem(conf);
>> fs.mkdirs(file))
>> produces this exception. It looks like it's defaulting to the
>> DistributedFileSystem.
>> 2007-02-14 11:36:31,700 INFO mapred.TaskInProgress
>> (TaskInProgress.java:updateStatus(296)) - Error from
>> task_0001_m_000000_0: java.lang.IllegalArgumentException:
>> Pathname /C:/trunk/build/test/data from C:/trunk/build/test/data
>> is not a valid DFS filename.
>> at org.apache.hadoop.dfs.DistributedFileSystem.getPath
>> (DistributedFileSystem.java:111)
>> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs
>> (DistributedFileSystem.java:211)
>> at org.apache.hadoop.mapred.MRCaching$MapClass.configure
>> (MRCaching.java:68)
>> at org.apache.hadoop.util.ReflectionUtils.setConf
>> (ReflectionUtils.java:50)
>> at org.apache.hadoop.util.ReflectionUtils.newInstance
>> (ReflectionUtils.java:70)
>> at org.apache.hadoop.mapred.MapRunner.configure
>> (MapRunner.java:34)
>> at org.apache.hadoop.util.ReflectionUtils.setConf
>> (ReflectionUtils.java:50)
>> at org.apache.hadoop.util.ReflectionUtils.newInstance
>> (ReflectionUtils.java:70)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:178)
>> at org.apache.hadoop.mapred.TaskTracker$Child.main
>> (TaskTracker.java:1396)
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
[jira] Commented: (HADOOP-1020) Path class on Windows seems broken
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12476462 ]
Hadoop QA commented on HADOOP-1020:
-----------------------------------
-1, because javac generated 768 warnings (more than the acceptable 766 warnings) when testing the latest attachment (http://issues.apache.org/jira/secure/attachment/12352189/HADOOP-1020-2.patch) against trunk revision http://svn.apache.org/repos/asf/lucene/hadoop/trunk/512461. Please note that this message is automatically generated and may represent a problem with the automation system and not the patch. Results are at http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch
> Path class on Windows seems broken
> ----------------------------------
>
> Key: HADOOP-1020
> URL: https://issues.apache.org/jira/browse/HADOOP-1020
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.11.1
> Reporter: Nigel Daley
> Assigned To: Doug Cutting
> Fix For: 0.12.0
>
> Attachments: HADOOP-1020-2.patch, HADOOP-1020.patch, TestPath.java.patch
>
>
> Executing this code:
> Path file = new Path("file:///", "C:/trunk/build/test/data"); or Path file = new Path("C:/trunk/build/test/data");
> FileSystem fs = file.getFileSystem(conf);
> fs.mkdirs(file))
> produces this exception. It looks like it's defaulting to the DistributedFileSystem.
> 2007-02-14 11:36:31,700 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(296)) - Error from task_0001_m_000000_0: java.lang.IllegalArgumentException: Pathname /C:/trunk/build/test/data from C:/trunk/build/test/data is not a valid DFS filename.
> at org.apache.hadoop.dfs.DistributedFileSystem.getPath(DistributedFileSystem.java:111)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:211)
> at org.apache.hadoop.mapred.MRCaching$MapClass.configure(MRCaching.java:68)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:178)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1396)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1020) Path class on Windows seems broken
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473278 ]
Doug Cutting commented on HADOOP-1020:
--------------------------------------
Jira mangled the second occurrence of <letter><colon> in the previous comment.
> Path class on Windows seems broken
> ----------------------------------
>
> Key: HADOOP-1020
> URL: https://issues.apache.org/jira/browse/HADOOP-1020
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.11.1
> Reporter: Nigel Daley
> Attachments: TestPath.java.patch
>
>
> Executing this code:
> Path file = new Path("file:///", "C:/trunk/build/test/data"); or Path file = new Path("C:/trunk/build/test/data");
> FileSystem fs = file.getFileSystem(conf);
> fs.mkdirs(file))
> produces this exception. It looks like it's defaulting to the DistributedFileSystem.
> 2007-02-14 11:36:31,700 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(296)) - Error from task_0001_m_000000_0: java.lang.IllegalArgumentException: Pathname /C:/trunk/build/test/data from C:/trunk/build/test/data is not a valid DFS filename.
> at org.apache.hadoop.dfs.DistributedFileSystem.getPath(DistributedFileSystem.java:111)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:211)
> at org.apache.hadoop.mapred.MRCaching$MapClass.configure(MRCaching.java:68)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:178)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1396)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1020) Path class on Windows seems broken
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-1020:
---------------------------------
Fix Version/s: 0.12.0
Status: Patch Available (was: Open)
> Path class on Windows seems broken
> ----------------------------------
>
> Key: HADOOP-1020
> URL: https://issues.apache.org/jira/browse/HADOOP-1020
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.11.1
> Reporter: Nigel Daley
> Assigned To: Doug Cutting
> Fix For: 0.12.0
>
> Attachments: HADOOP-1020.patch, TestPath.java.patch
>
>
> Executing this code:
> Path file = new Path("file:///", "C:/trunk/build/test/data"); or Path file = new Path("C:/trunk/build/test/data");
> FileSystem fs = file.getFileSystem(conf);
> fs.mkdirs(file))
> produces this exception. It looks like it's defaulting to the DistributedFileSystem.
> 2007-02-14 11:36:31,700 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(296)) - Error from task_0001_m_000000_0: java.lang.IllegalArgumentException: Pathname /C:/trunk/build/test/data from C:/trunk/build/test/data is not a valid DFS filename.
> at org.apache.hadoop.dfs.DistributedFileSystem.getPath(DistributedFileSystem.java:111)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:211)
> at org.apache.hadoop.mapred.MRCaching$MapClass.configure(MRCaching.java:68)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:178)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1396)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1020) Path class on Windows seems broken
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473202 ]
Doug Cutting commented on HADOOP-1020:
--------------------------------------
Is this on Windows? The Path code behaves differently on Windows.
Can you provide a standalone test? Adding something to TestPath might be appropriate.
> Path class on Windows seems broken
> ----------------------------------
>
> Key: HADOOP-1020
> URL: https://issues.apache.org/jira/browse/HADOOP-1020
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.11.1
> Reporter: Nigel Daley
>
> Executing this code:
> Path file = new Path("file:///", "C:/trunk/build/test/data"); or Path file = new Path("C:/trunk/build/test/data");
> FileSystem fs = file.getFileSystem(conf);
> fs.mkdirs(file))
> produces this exception. It looks like it's defaulting to the DistributedFileSystem.
> 2007-02-14 11:36:31,700 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(296)) - Error from task_0001_m_000000_0: java.lang.IllegalArgumentException: Pathname /C:/trunk/build/test/data from C:/trunk/build/test/data is not a valid DFS filename.
> at org.apache.hadoop.dfs.DistributedFileSystem.getPath(DistributedFileSystem.java:111)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:211)
> at org.apache.hadoop.mapred.MRCaching$MapClass.configure(MRCaching.java:68)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:178)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1396)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1020) Path class on Windows seems broken
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473203 ]
Doug Cutting commented on HADOOP-1020:
--------------------------------------
A unit test might simply check that, e.g.: fs == FileSystem.getLocal(conf).
> Path class on Windows seems broken
> ----------------------------------
>
> Key: HADOOP-1020
> URL: https://issues.apache.org/jira/browse/HADOOP-1020
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.11.1
> Reporter: Nigel Daley
>
> Executing this code:
> Path file = new Path("file:///", "C:/trunk/build/test/data"); or Path file = new Path("C:/trunk/build/test/data");
> FileSystem fs = file.getFileSystem(conf);
> fs.mkdirs(file))
> produces this exception. It looks like it's defaulting to the DistributedFileSystem.
> 2007-02-14 11:36:31,700 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(296)) - Error from task_0001_m_000000_0: java.lang.IllegalArgumentException: Pathname /C:/trunk/build/test/data from C:/trunk/build/test/data is not a valid DFS filename.
> at org.apache.hadoop.dfs.DistributedFileSystem.getPath(DistributedFileSystem.java:111)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:211)
> at org.apache.hadoop.mapred.MRCaching$MapClass.configure(MRCaching.java:68)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:178)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1396)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Updated: (HADOOP-1020) Path class on Windows seems broken
Posted by Nigel Daley <nd...@yahoo-inc.com>.
Still broken. On Windows I still get:
[junit] Job tracker crashed:
[junit] java.lang.IllegalArgumentException: Pathname /C:/trunk/
build/contrib/streaming/test/mapred/system from C:/trunk/build/
contrib/streaming/test/mapred/system is not a valid DFS filename.
[junit] at
org.apache.hadoop.dfs.DistributedFileSystem.getPath
(DistributedFileSystem.java:111)
[junit] at
org.apache.hadoop.dfs.DistributedFileSystem.isDirectory
(DistributedFileSystem.java:167)
[junit] at org.apache.hadoop.fs.FileSystem.delete
(FileSystem.java:507)
[junit] at org.apache.hadoop.mapred.JobTracker.<init>
(JobTracker.java:582)
[junit] at org.apache.hadoop.mapred.JobTracker.startTracker
(JobTracker.java:92)
[junit] at org.apache.hadoop.mapred.MiniMRCluster
$JobTrackerRunner.run(MiniMRCluster.java:61)
[junit] at java.lang.Thread.run(Thread.java:595)
from org.apache.hadoop.streaming.TestSymLink
On Feb 26, 2007, at 1:38 PM, Doug Cutting (JIRA) wrote:
>
> [ https://issues.apache.org/jira/browse/HADOOP-1020?
> page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>
> Doug Cutting updated HADOOP-1020:
> ---------------------------------
>
> Attachment: HADOOP-1020.patch
>
> The bug was an erroneous optimization in the Path(Path,Path)
> constructor. I removed that and added a test case in the attached
> patch.
>
> Nigel, does this fix things for you on Windows?
>
>> Path class on Windows seems broken
>> ----------------------------------
>>
>> Key: HADOOP-1020
>> URL: https://issues.apache.org/jira/browse/
>> HADOOP-1020
>> Project: Hadoop
>> Issue Type: Bug
>> Components: dfs
>> Affects Versions: 0.11.1
>> Reporter: Nigel Daley
>> Assigned To: Doug Cutting
>> Attachments: HADOOP-1020.patch, TestPath.java.patch
>>
>>
>> Executing this code:
>> Path file = new Path("file:///", "C:/trunk/build/test/data");
>> or Path file = new Path("C:/trunk/build/test/data");
>> FileSystem fs = file.getFileSystem(conf);
>> fs.mkdirs(file))
>> produces this exception. It looks like it's defaulting to the
>> DistributedFileSystem.
>> 2007-02-14 11:36:31,700 INFO mapred.TaskInProgress
>> (TaskInProgress.java:updateStatus(296)) - Error from
>> task_0001_m_000000_0: java.lang.IllegalArgumentException:
>> Pathname /C:/trunk/build/test/data from C:/trunk/build/test/data
>> is not a valid DFS filename.
>> at org.apache.hadoop.dfs.DistributedFileSystem.getPath
>> (DistributedFileSystem.java:111)
>> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs
>> (DistributedFileSystem.java:211)
>> at org.apache.hadoop.mapred.MRCaching$MapClass.configure
>> (MRCaching.java:68)
>> at org.apache.hadoop.util.ReflectionUtils.setConf
>> (ReflectionUtils.java:50)
>> at org.apache.hadoop.util.ReflectionUtils.newInstance
>> (ReflectionUtils.java:70)
>> at org.apache.hadoop.mapred.MapRunner.configure
>> (MapRunner.java:34)
>> at org.apache.hadoop.util.ReflectionUtils.setConf
>> (ReflectionUtils.java:50)
>> at org.apache.hadoop.util.ReflectionUtils.newInstance
>> (ReflectionUtils.java:70)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:178)
>> at org.apache.hadoop.mapred.TaskTracker$Child.main
>> (TaskTracker.java:1396)
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
[jira] Updated: (HADOOP-1020) Path class on Windows seems broken
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-1020:
---------------------------------
Attachment: HADOOP-1020.patch
The bug was an erroneous optimization in the Path(Path,Path) constructor. I removed that and added a test case in the attached patch.
Nigel, does this fix things for you on Windows?
> Path class on Windows seems broken
> ----------------------------------
>
> Key: HADOOP-1020
> URL: https://issues.apache.org/jira/browse/HADOOP-1020
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.11.1
> Reporter: Nigel Daley
> Assigned To: Doug Cutting
> Attachments: HADOOP-1020.patch, TestPath.java.patch
>
>
> Executing this code:
> Path file = new Path("file:///", "C:/trunk/build/test/data"); or Path file = new Path("C:/trunk/build/test/data");
> FileSystem fs = file.getFileSystem(conf);
> fs.mkdirs(file))
> produces this exception. It looks like it's defaulting to the DistributedFileSystem.
> 2007-02-14 11:36:31,700 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(296)) - Error from task_0001_m_000000_0: java.lang.IllegalArgumentException: Pathname /C:/trunk/build/test/data from C:/trunk/build/test/data is not a valid DFS filename.
> at org.apache.hadoop.dfs.DistributedFileSystem.getPath(DistributedFileSystem.java:111)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:211)
> at org.apache.hadoop.mapred.MRCaching$MapClass.configure(MRCaching.java:68)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:178)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1396)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1020) Path class on Windows seems broken
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473248 ]
Doug Cutting commented on HADOOP-1020:
--------------------------------------
Windows pathnames are problematic as URIs. For back-compatibility, I special-cased things on Windows so that a path that begins with <letter><colon> is treated as though it begins file:///<letter><colon>. This handling is disabled on other operating systems. Perhaps we should disable it altogether, but then I fear lots of things would break on Windows.
That explains the first assertion failing on Linux. The failure of the second assertion looks like a bug to me.
> Path class on Windows seems broken
> ----------------------------------
>
> Key: HADOOP-1020
> URL: https://issues.apache.org/jira/browse/HADOOP-1020
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.11.1
> Reporter: Nigel Daley
> Attachments: TestPath.java.patch
>
>
> Executing this code:
> Path file = new Path("file:///", "C:/trunk/build/test/data"); or Path file = new Path("C:/trunk/build/test/data");
> FileSystem fs = file.getFileSystem(conf);
> fs.mkdirs(file))
> produces this exception. It looks like it's defaulting to the DistributedFileSystem.
> 2007-02-14 11:36:31,700 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(296)) - Error from task_0001_m_000000_0: java.lang.IllegalArgumentException: Pathname /C:/trunk/build/test/data from C:/trunk/build/test/data is not a valid DFS filename.
> at org.apache.hadoop.dfs.DistributedFileSystem.getPath(DistributedFileSystem.java:111)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:211)
> at org.apache.hadoop.mapred.MRCaching$MapClass.configure(MRCaching.java:68)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:178)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1396)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1020) Path class on Windows seems broken
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12476535 ]
Hadoop QA commented on HADOOP-1020:
-----------------------------------
-1, because 3 attempts failed to build and test the latest attachment http://issues.apache.org/jira/secure/attachment/12352189/HADOOP-1020-2.patch</a>) against trunk revision <a href= against trunk revision http://svn.apache.org/repos/asf/lucene/hadoop/trunk/512499. Please note that this message is automatically generated and may represent a problem with the automation system and not the patch. Results are at http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch
> Path class on Windows seems broken
> ----------------------------------
>
> Key: HADOOP-1020
> URL: https://issues.apache.org/jira/browse/HADOOP-1020
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.11.1
> Reporter: Nigel Daley
> Assigned To: Doug Cutting
> Fix For: 0.12.0
>
> Attachments: HADOOP-1020-2.patch, HADOOP-1020.patch, TestPath.java.patch
>
>
> Executing this code:
> Path file = new Path("file:///", "C:/trunk/build/test/data"); or Path file = new Path("C:/trunk/build/test/data");
> FileSystem fs = file.getFileSystem(conf);
> fs.mkdirs(file))
> produces this exception. It looks like it's defaulting to the DistributedFileSystem.
> 2007-02-14 11:36:31,700 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(296)) - Error from task_0001_m_000000_0: java.lang.IllegalArgumentException: Pathname /C:/trunk/build/test/data from C:/trunk/build/test/data is not a valid DFS filename.
> at org.apache.hadoop.dfs.DistributedFileSystem.getPath(DistributedFileSystem.java:111)
> at org.apache.hadoop.dfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:211)
> at org.apache.hadoop.mapred.MRCaching$MapClass.configure(MRCaching.java:68)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:50)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:70)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:178)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1396)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.