You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "NOMURA Yoshihide (JIRA)" <ji...@apache.org> on 2008/08/25 08:37:44 UTC
[jira] Created: (HADOOP-4014) DFS upgrade fails on Windows
DFS upgrade fails on Windows
----------------------------
Key: HADOOP-4014
URL: https://issues.apache.org/jira/browse/HADOOP-4014
Project: Hadoop Core
Issue Type: Bug
Components: fs
Affects Versions: 0.18.0
Environment: Windows XP, Windows 2003 Server
Reporter: NOMURA Yoshihide
FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
The windows command 'fsutil' requires the arguments link name first as follows,
> fsutil hardlink create <link name> <target>
But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12628236#action_12628236 ]
Konstantin Shvachko commented on HADOOP-4014:
---------------------------------------------
I'd rather introduce a static member {{OSType osType;}} and use it to verify in which order the arguments should be set in the command line.
{{osType}} should replace {{osName}} with the latter becoming a local variable of {{getOSType()}}.
You effectively do the same by comparing command name is "fsutil". Checking osType seems to be cleaner.
"XpP" - is a good catch.
Did you really try it on Vista?
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Attachments: HADOOP-4014.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12636789#action_12636789 ]
Konstantin Shvachko commented on HADOOP-4014:
---------------------------------------------
Yes, FileNotFoundException is thrown instead of the custom IOException, which changes the description returned. E.g.
{code}
08/10/03 22:32:17 INFO cli.TestCLI: Expected output: [copyFromLocal: /user/wrongdata: No such file or directory]
08/10/03 22:32:17 INFO cli.TestCLI: Actual output: [copyFromLocal: File /user/wrongdata does not exist.
{code}
It is really a bad idea to verify correctness based on an exception description.
So now the question is whether these descriptions are a part of the CLI api and it is going to be an incompatible change if I modify the "Expected output"?
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Attachments: HADOOP-4014.patch, HardLinkWin.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HADOOP-4014:
----------------------------------------
Attachment: HardLinkWin.patch
I modified the "Expected output" in testConf.xml to the new values.
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4014.patch, HardLinkWin.patch, HardLinkWin.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "NOMURA Yoshihide (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
NOMURA Yoshihide updated HADOOP-4014:
-------------------------------------
Attachment: HADOOP-4014.patch
This is a patch recommendation for this issue.
If the hard link command is 'fsutil', the arguments of command will be opposite order to 'ln' command.
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Attachments: HADOOP-4014.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HADOOP-4014:
----------------------------------------
Fix Version/s: 0.19.0
Status: Patch Available (was: Open)
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4014.patch, HardLinkWin.patch, HardLinkWin.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12636794#action_12636794 ]
Konstantin Shvachko commented on HADOOP-4014:
---------------------------------------------
About other tests:
- TestFileCreation and TestDatanodeDeath are not failing on my Win machine. But I see different tests fail sporadically. I believe this is related to some not cleaning up issues which should be investigated separately.
- TestLeaseRecovery2 seem to be related to HADOOP-4257
- TestMiniMRMapRedDebugScript is reported among other test in HADOOP-4172.
I'll create a new patch fixing TestCLI by changing the "Expected output".
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Attachments: HADOOP-4014.patch, HardLinkWin.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12636474#action_12636474 ]
Raghu Angadi commented on HADOOP-4014:
--------------------------------------
TestDFSUpgradeFromImage fails on XP about 60-80% of the time with following trace. This might be because of some flakiness with Cygwin's 'ln'. This patch seems to fix it. Can we mark this a blocker for 0.19? Konstantin, do you have an improved patch?
{noformat}
java.io.IOException:
at org.apache.hadoop.fs.FileUtil$HardLink.createHardLink(FileUtil.java:585)
at org.apache.hadoop.hdfs.server.datanode.DataStorage.linkBlocks(DataStorage.java:377)
at org.apache.hadoop.hdfs.server.datanode.DataStorage.linkBlocks(DataStorage.java:393)
at org.apache.hadoop.hdfs.server.datanode.DataStorage.doUpgrade(DataStorage.java:282)
at org.apache.hadoop.hdfs.server.datanode.DataStorage.doTransition(DataStorage.java:244)
at org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:148)
at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:287)
at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:205)
{noformat}
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Attachments: HADOOP-4014.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639849#action_12639849 ]
Hudson commented on HADOOP-4014:
--------------------------------
Integrated in Hadoop-trunk #634 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/634/])
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4014.patch, HardLinkWin.patch, HardLinkWin.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12638733#action_12638733 ]
Hadoop QA commented on HADOOP-4014:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12391601/HardLinkWin.patch
against trunk revision 703609.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 24 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3444/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3444/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3444/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3444/console
This message is automatically generated.
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4014.patch, HardLinkWin.patch, HardLinkWin.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Steve Loughran (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12628606#action_12628606 ]
Steve Loughran commented on HADOOP-4014:
----------------------------------------
The quoted OS in in a JVM comes from the JRE; different JVMs detect different values, and tend to lag OS releases. OpenJDK even reports different things from a closed JDK. Which makes it very hard to do OS family detection reliably. Its probalby only java6+ that detects Vista, for example.
The most broadly tested code for this in apache is Ant's OS condition
http://svn.apache.org/viewvc/ant/core/trunk/src/main/org/apache/tools/ant/taskdefs/condition/Os.java?view=markup
This
-lowercases in the US locale the jvm properties
-assumes the presence of "windows" in the OS => windows, assumes further than only a few limited versions are win9x; the rest is NT-based
-looks for various strings for macos
-guesses that systems with / and : as seperators are unix-like
-looks for various obscure operating systems
-or bails out
I'd recommend taking this code and adapting it; strip out netware and OS/360 support if need be. This routine is the first one to get bugreps when a new OS or JDK ships, so is maintained.
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Attachments: HADOOP-4014.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HADOOP-4014:
----------------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this.
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4014.patch, HardLinkWin.patch, HardLinkWin.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639183#action_12639183 ]
Konstantin Shvachko commented on HADOOP-4014:
---------------------------------------------
Thank you NOMURA Yoshihide for reporting the issue and the initial patch.
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4014.patch, HardLinkWin.patch, HardLinkWin.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12636665#action_12636665 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-4014:
------------------------------------------------
{code}
- if (srcFS.isDirectory(src)) {
+ if (srcFS.getFileStatus(src).isDir()) {
{code}
It seems to me that isDirectory(src) cannot be replaced by getFileStatus(src).isDir() since FileNotFoundException may be thrown.
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Attachments: HADOOP-4014.patch, HardLinkWin.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12638690#action_12638690 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-4014:
------------------------------------------------
The term "earliest version of windows" may be ill-defined. Win2003 probably was released after WinXP but Win2003 is not a updated version/replacement of WinXP. They are for different customers.
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4014.patch, HardLinkWin.patch, HardLinkWin.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639151#action_12639151 ]
Konstantin Shvachko commented on HADOOP-4014:
---------------------------------------------
Nicholas, I don't care about windows customers. I care about fsutil. WinXP and all later versions support fsutil, while earlier versions do not. Without fsutil there is no way to create hard links in Windows. And this is why the name contain XP in it. Probably more correct name would be OS_TYPE_WIN_SUPPOTING_FSUTIL, but it is confusing.
TestJobQueueInformation fails because of HADOOP-4378.
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4014.patch, HardLinkWin.patch, HardLinkWin.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Raghu Angadi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12636467#action_12636467 ]
Raghu Angadi commented on HADOOP-4014:
--------------------------------------
Nomura,
How did you notice this error? because of the "XpP" typo, Hadoop ends up using cygwin's 'ln' and it works. Thats why the unit tests have been passing.
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Attachments: HADOOP-4014.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12638687#action_12638687 ]
Konstantin Shvachko commented on HADOOP-4014:
---------------------------------------------
I think we should keep OS_TYPE_WINXP, which indicates the earliest version of windows which supports that.
WinNT and Win98 do not have fsutil. OS_TYPE_WINDOWS will be too general.
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4014.patch, HardLinkWin.patch, HardLinkWin.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "NOMURA Yoshihide (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12636556#action_12636556 ]
NOMURA Yoshihide commented on HADOOP-4014:
------------------------------------------
The patch looks fine.
I found this issue in Windows 2003 Server first.
And the 'XP' typo was found when I review the source code.
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Attachments: HADOOP-4014.patch, HardLinkWin.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HADOOP-4014:
----------------------------------------
Attachment: HardLinkWin.patch
Updating the patch.
I ran TestDFSUpgradeFromImage several times - no failures.
I also fixed 4 warnings in FileUtil.
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Attachments: HADOOP-4014.patch, HardLinkWin.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HADOOP-4014:
----------------------------------------
Priority: Blocker (was: Major)
Assignee: Konstantin Shvachko
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Attachments: HADOOP-4014.patch, HardLinkWin.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HADOOP-4014:
----------------------------------------
Status: Open (was: Patch Available)
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Attachments: HADOOP-4014.patch, HardLinkWin.patch, HardLinkWin.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HADOOP-4014:
----------------------------------------
Status: Patch Available (was: Open)
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4014.patch, HardLinkWin.patch, HardLinkWin.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12636579#action_12636579 ]
Hadoop QA commented on HADOOP-4014:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12391401/HardLinkWin.patch
against trunk revision 701307.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
-1 core tests. The patch failed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3427/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3427/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3427/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3427/console
This message is automatically generated.
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Attachments: HADOOP-4014.patch, HardLinkWin.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HADOOP-4014:
----------------------------------------
Status: Patch Available (was: Open)
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Attachments: HADOOP-4014.patch, HardLinkWin.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12637589#action_12637589 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-4014:
------------------------------------------------
How about rename OSType.OS_TYPE_WINXP to OSType.OS_TYPE_WINDOWS since it represents not only WinXP but also Win2003 and Vista?
Otherwise, the patch looks good.
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4014.patch, HardLinkWin.patch, HardLinkWin.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-4014) DFS upgrade fails on Windows
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HADOOP-4014:
----------------------------------------
Status: Open (was: Patch Available)
> DFS upgrade fails on Windows
> ----------------------------
>
> Key: HADOOP-4014
> URL: https://issues.apache.org/jira/browse/HADOOP-4014
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0
> Environment: Windows XP, Windows 2003 Server
> Reporter: NOMURA Yoshihide
> Assignee: Konstantin Shvachko
> Priority: Blocker
> Fix For: 0.19.0
>
> Attachments: HADOOP-4014.patch, HardLinkWin.patch, HardLinkWin.patch
>
>
> FileUtil.HardLink#createHardLink() didn't work on Windows, and DFS upgrade of Datanode fails.
> The windows command 'fsutil' requires the arguments link name first as follows,
> > fsutil hardlink create <link name> <target>
> But the current FileUtil passes the target first.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.