You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Arpit Agarwal (JIRA)" <ji...@apache.org> on 2012/11/17 00:45:11 UTC
[jira] [Created] (HADOOP-9056) Build native library on Windows
Arpit Agarwal created HADOOP-9056:
-------------------------------------
Summary: Build native library on Windows
Key: HADOOP-9056
URL: https://issues.apache.org/jira/browse/HADOOP-9056
Project: Hadoop Common
Issue Type: Improvement
Components: native
Affects Versions: trunk-win
Reporter: Arpit Agarwal
Assignee: Arpit Agarwal
Fix For: trunk-win
The native library (hadoop.dll) must be compiled on Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9056) Build native library on Windows
Posted by "Arpit Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-9056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arpit Agarwal updated HADOOP-9056:
----------------------------------
Attachment: HADOOP-9056.6.patch
> Build native library on Windows
> -------------------------------
>
> Key: HADOOP-9056
> URL: https://issues.apache.org/jira/browse/HADOOP-9056
> Project: Hadoop Common
> Issue Type: Improvement
> Components: native
> Affects Versions: trunk-win
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Fix For: trunk-win
>
> Attachments: HADOOP-9056.1.patch, HADOOP-9056.3.patch, HADOOP-9056.5.patch, HADOOP-9056.6.patch, HADOOP-9056.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> The native library (hadoop.dll) must be compiled on Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9056) Build native library on Windows
Posted by "Arpit Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-9056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arpit Agarwal updated HADOOP-9056:
----------------------------------
Attachment: HADOOP-9056.3.patch
> Build native library on Windows
> -------------------------------
>
> Key: HADOOP-9056
> URL: https://issues.apache.org/jira/browse/HADOOP-9056
> Project: Hadoop Common
> Issue Type: Improvement
> Components: native
> Affects Versions: trunk-win
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Fix For: trunk-win
>
> Attachments: HADOOP-9056.1.patch, HADOOP-9056.3.patch, HADOOP-9056.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> The native library (hadoop.dll) must be compiled on Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9056) Build native library on Windows
Posted by "Arpit Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-9056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arpit Agarwal updated HADOOP-9056:
----------------------------------
Attachment: HADOOP-9056.5.patch
> Build native library on Windows
> -------------------------------
>
> Key: HADOOP-9056
> URL: https://issues.apache.org/jira/browse/HADOOP-9056
> Project: Hadoop Common
> Issue Type: Improvement
> Components: native
> Affects Versions: trunk-win
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Fix For: trunk-win
>
> Attachments: HADOOP-9056.1.patch, HADOOP-9056.3.patch, HADOOP-9056.5.patch, HADOOP-9056.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> The native library (hadoop.dll) must be compiled on Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9056) Build native library on Windows
Posted by "Arpit Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-9056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arpit Agarwal updated HADOOP-9056:
----------------------------------
Attachment: HADOOP-9056.patch
> Build native library on Windows
> -------------------------------
>
> Key: HADOOP-9056
> URL: https://issues.apache.org/jira/browse/HADOOP-9056
> Project: Hadoop Common
> Issue Type: Improvement
> Components: native
> Affects Versions: trunk-win
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Fix For: trunk-win
>
> Attachments: HADOOP-9056.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> The native library (hadoop.dll) must be compiled on Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9056) Build native library on Windows
Posted by "Arpit Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-9056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506943#comment-13506943 ]
Arpit Agarwal commented on HADOOP-9056:
---------------------------------------
Thanks for the feedback Chuan. I have addressed most of your comments. Most of the SecureIOUtils changes don't seem to be applicable in trunk.
There is no equivalent to posix_fadvise in Win32 but we may be able to achieve a similar effect by passing flags to CreateFile, we can address that in a separate patch.
> Build native library on Windows
> -------------------------------
>
> Key: HADOOP-9056
> URL: https://issues.apache.org/jira/browse/HADOOP-9056
> Project: Hadoop Common
> Issue Type: Improvement
> Components: native
> Affects Versions: trunk-win
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Fix For: trunk-win
>
> Attachments: HADOOP-9056.1.patch, HADOOP-9056.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> The native library (hadoop.dll) must be compiled on Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9056) Build native library on Windows
Posted by "Arpit Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-9056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501440#comment-13501440 ]
Arpit Agarwal commented on HADOOP-9056:
---------------------------------------
Attached a patch to port Chuan Liu's changes to branch-trunk-win. Some additional changes to account for branch divergence.
> Build native library on Windows
> -------------------------------
>
> Key: HADOOP-9056
> URL: https://issues.apache.org/jira/browse/HADOOP-9056
> Project: Hadoop Common
> Issue Type: Improvement
> Components: native
> Affects Versions: trunk-win
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Fix For: trunk-win
>
> Attachments: HADOOP-9056.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> The native library (hadoop.dll) must be compiled on Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9056) Build native library on Windows
Posted by "Chuan Liu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-9056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13504872#comment-13504872 ]
Chuan Liu commented on HADOOP-9056:
-----------------------------------
Looks good in general! I have the following questions and comments.
# It seems this patch only contains changes that made compiling ‘hadoop.dll’ available on Windows. Some functionality was missing. For example, ‘POSIX.chmod’ was not ported over from branch-1-win; so were some other IO methods. I also notice some new functions were introduced in trunk compared with branch-1. For example, posixFadviseIfPossible() and POSIX.posix_fadvis(). What is the plan for those missing functions? Do we plan to port them over in other JIRAs?
# “native.vcxproj.user” file is not needed.
# Snappy does not work on Windows right now. It may be better to exclude snappy related files from Windows build.
# I think it is better to use ‘WINDOWS’ instead of ‘ _WIN32’ macro in some places because we explicitly defined the ‘UNIX’ and ‘WINDOWS’ macros in the beginning of “org_apache_hadoop.h”. It will make it easier in the future if we want to change the definition of the macros.
# Some changes in SecureIOUtils and Datanode are not ported over. Again I think this is related to point 1 above.
> Build native library on Windows
> -------------------------------
>
> Key: HADOOP-9056
> URL: https://issues.apache.org/jira/browse/HADOOP-9056
> Project: Hadoop Common
> Issue Type: Improvement
> Components: native
> Affects Versions: trunk-win
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Fix For: trunk-win
>
> Attachments: HADOOP-9056.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> The native library (hadoop.dll) must be compiled on Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9056) Build native library on Windows
Posted by "Chuan Liu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-9056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510113#comment-13510113 ]
Chuan Liu commented on HADOOP-9056:
-----------------------------------
[~arpitagarwal], why the SecureIOUtils and Datanode changes not apply to trunk?
Could you explain a little bit? Because I still see the old methods presenting in the trunk code base.
> Build native library on Windows
> -------------------------------
>
> Key: HADOOP-9056
> URL: https://issues.apache.org/jira/browse/HADOOP-9056
> Project: Hadoop Common
> Issue Type: Improvement
> Components: native
> Affects Versions: trunk-win
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Fix For: trunk-win
>
> Attachments: HADOOP-9056.1.patch, HADOOP-9056.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> The native library (hadoop.dll) must be compiled on Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9056) Build native library on Windows
Posted by "Arpit Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-9056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13511032#comment-13511032 ]
Arpit Agarwal commented on HADOOP-9056:
---------------------------------------
[~chuanliu], I had missed porting one change to SecureIOUtils.CheckStat so thanks for pointing that out.
Fixed that and updated the patch to merge HADOOP-7115 which was pulled down from trunk on 9/30. Also added NativeCrc32 into hadoop.dll (this is new to trunk).
The function SecureIOUtils.checkStatFileSystem appears to have been deprecated in trunk. Please let me know if you see anything else missing.
Thanks!
Arpit
> Build native library on Windows
> -------------------------------
>
> Key: HADOOP-9056
> URL: https://issues.apache.org/jira/browse/HADOOP-9056
> Project: Hadoop Common
> Issue Type: Improvement
> Components: native
> Affects Versions: trunk-win
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Fix For: trunk-win
>
> Attachments: HADOOP-9056.1.patch, HADOOP-9056.3.patch, HADOOP-9056.5.patch, HADOOP-9056.6.patch, HADOOP-9056.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> The native library (hadoop.dll) must be compiled on Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9056) Build native library on Windows
Posted by "Arpit Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-9056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arpit Agarwal updated HADOOP-9056:
----------------------------------
Attachment: HADOOP-9056.1.patch
> Build native library on Windows
> -------------------------------
>
> Key: HADOOP-9056
> URL: https://issues.apache.org/jira/browse/HADOOP-9056
> Project: Hadoop Common
> Issue Type: Improvement
> Components: native
> Affects Versions: trunk-win
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Fix For: trunk-win
>
> Attachments: HADOOP-9056.1.patch, HADOOP-9056.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> The native library (hadoop.dll) must be compiled on Windows.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira