You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2011/02/17 07:47:24 UTC
[jira] Created: (HADOOP-7147) Native code does not build on BSD/OSX
Native code does not build on BSD/OSX
-------------------------------------
Key: HADOOP-7147
URL: https://issues.apache.org/jira/browse/HADOOP-7147
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 0.23.0
Reporter: Todd Lipcon
HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HADOOP-7147) setnetgrent in native code is not
portable
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Colin Patrick McCabe reopened HADOOP-7147:
------------------------------------------
Assignee: (was: Allen Wittenauer)
"Won't fix" makes it sound like this is not a valid bug, which it is.
> setnetgrent in native code is not portable
> ------------------------------------------
>
> Key: HADOOP-7147
> URL: https://issues.apache.org/jira/browse/HADOOP-7147
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Todd Lipcon
> Attachments: hadoop-7147.patch, HADOOP-7147.patch
>
>
> HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7147) Native code does not build on
BSD/OSX
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045630#comment-13045630 ]
Allen Wittenauer commented on HADOOP-7147:
------------------------------------------
Actually, we probably want to ignore the call anyway. Looks like setnetgrent suffers from some really bizarro portability issues. On Linux, setnetgrent returns 1 on success. On Solaris, it returns 0.
There is a side issue as well, at least on Solaris, where setnetgrent() will actually reset for *all* threads that are in the midst of enumerating netgroups. I haven't verified if this is true on Linux or other platforms.
> Native code does not build on BSD/OSX
> -------------------------------------
>
> Key: HADOOP-7147
> URL: https://issues.apache.org/jira/browse/HADOOP-7147
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
>
> HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7147) setnetgrent in native code is not
portable
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Allen Wittenauer updated HADOOP-7147:
-------------------------------------
Component/s: native
> setnetgrent in native code is not portable
> ------------------------------------------
>
> Key: HADOOP-7147
> URL: https://issues.apache.org/jira/browse/HADOOP-7147
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Todd Lipcon
> Assignee: Allen Wittenauer
> Attachments: hadoop-7147.patch
>
>
> HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7147) Native code does not build on
BSD/OSX
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Allen Wittenauer updated HADOOP-7147:
-------------------------------------
Attachment: hadoop-7147.patch
This patch cleans up the variable re-use as well as changes setnetgrent by ignoring the return code.
> Native code does not build on BSD/OSX
> -------------------------------------
>
> Key: HADOOP-7147
> URL: https://issues.apache.org/jira/browse/HADOOP-7147
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Todd Lipcon
> Attachments: hadoop-7147.patch
>
>
> HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-7147) Native code does not build on
BSD/OSX
Posted by "Erik Steffl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12995984#comment-12995984 ]
Erik Steffl commented on HADOOP-7147:
-------------------------------------
Proposed solution: just make a call to setnetgrent, don't make it a condition, i.e. execute the if block (call to getnetgrent) unconditionally.
Rationale:
As far as I can tell in BSD there is no need to check the setnetgrent for errors, there doesn't seem to be a way to check (based on what BSD man page says) and they don't do it:
http://lists.freebsd.org/pipermail/freebsd-bugs/2009-March/034497.html
I assume that it's the same way Linux API would work (if setnetgrent fails then getnetgrent doesn't return any users) but it needs to be tested.
> Native code does not build on BSD/OSX
> -------------------------------------
>
> Key: HADOOP-7147
> URL: https://issues.apache.org/jira/browse/HADOOP-7147
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
>
> HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7147) Native code does not build on
BSD/OSX
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Allen Wittenauer updated HADOOP-7147:
-------------------------------------
Assignee: Allen Wittenauer
Affects Version/s: 0.22.0
Status: Patch Available (was: Open)
> Native code does not build on BSD/OSX
> -------------------------------------
>
> Key: HADOOP-7147
> URL: https://issues.apache.org/jira/browse/HADOOP-7147
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Todd Lipcon
> Assignee: Allen Wittenauer
> Attachments: hadoop-7147.patch
>
>
> HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7147) setnetgrent in native code is not
portable
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13403147#comment-13403147 ]
Allen Wittenauer commented on HADOOP-7147:
------------------------------------------
I'm thinking about compiler flags, so CHECK_FUNCTION_EXISTS won't work here. (i.e., -xCC, -xc99, -xstrconst, -xparallel, etc). Also need to pop all the GNU specific bits into protected areas so as not to infect everything else.
> setnetgrent in native code is not portable
> ------------------------------------------
>
> Key: HADOOP-7147
> URL: https://issues.apache.org/jira/browse/HADOOP-7147
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Todd Lipcon
> Assignee: Allen Wittenauer
> Attachments: HADOOP-7147.patch, hadoop-7147.patch
>
>
> HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7147) setnetgrent in native code is not
portable
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13403347#comment-13403347 ]
Colin Patrick McCabe commented on HADOOP-7147:
----------------------------------------------
Yeah, using CMAKE_SYSTEM_NAME to determine what platform flags to use seems totally reasonable to me.
> setnetgrent in native code is not portable
> ------------------------------------------
>
> Key: HADOOP-7147
> URL: https://issues.apache.org/jira/browse/HADOOP-7147
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Todd Lipcon
> Assignee: Allen Wittenauer
> Attachments: HADOOP-7147.patch, hadoop-7147.patch
>
>
> HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7147) setnetgrent in native code is not
portable
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13402872#comment-13402872 ]
Allen Wittenauer commented on HADOOP-7147:
------------------------------------------
BTW, Colin, see my note in the other bug about using get*_r routines on Solaris. Now that we're in cmake-ville, I haven't figured out how to do OS or compiler detection to do the correct thing on non-gcc/non-Linux. (e.g., on Darwin, we should *really* be passing -framework JavaVM amongst other flags when building libhadoop.dylib so we get linked properly, can build a fat binary, etc.).
> setnetgrent in native code is not portable
> ------------------------------------------
>
> Key: HADOOP-7147
> URL: https://issues.apache.org/jira/browse/HADOOP-7147
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Todd Lipcon
> Assignee: Allen Wittenauer
> Attachments: HADOOP-7147.patch, hadoop-7147.patch
>
>
> HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7147) setnetgrent in native code is not
portable
Posted by "Robert Joseph Evans (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans updated HADOOP-7147:
----------------------------------------
Status: Open (was: Patch Available)
Canceling the patch as it no longer applies to trunk or 0.23. The patch looks like it could easily be upmerged if it is still an issue.
> setnetgrent in native code is not portable
> ------------------------------------------
>
> Key: HADOOP-7147
> URL: https://issues.apache.org/jira/browse/HADOOP-7147
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 0.23.0, 0.22.0
> Reporter: Todd Lipcon
> Assignee: Allen Wittenauer
> Attachments: hadoop-7147.patch
>
>
> HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7147) setnetgrent in native code is not
portable
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Allen Wittenauer updated HADOOP-7147:
-------------------------------------
Summary: setnetgrent in native code is not portable (was: Native code does not build on BSD/OSX)
> setnetgrent in native code is not portable
> ------------------------------------------
>
> Key: HADOOP-7147
> URL: https://issues.apache.org/jira/browse/HADOOP-7147
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Todd Lipcon
> Assignee: Allen Wittenauer
> Attachments: hadoop-7147.patch
>
>
> HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7147) setnetgrent in native code is not
portable
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045665#comment-13045665 ]
Hadoop QA commented on HADOOP-7147:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12481747/hadoop-7147.patch
against trunk revision 1133125.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/589//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/589//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/589//console
This message is automatically generated.
> setnetgrent in native code is not portable
> ------------------------------------------
>
> Key: HADOOP-7147
> URL: https://issues.apache.org/jira/browse/HADOOP-7147
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Todd Lipcon
> Assignee: Allen Wittenauer
> Attachments: hadoop-7147.patch
>
>
> HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7147) setnetgrent in native code is not
portable
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13402848#comment-13402848 ]
Colin Patrick McCabe commented on HADOOP-7147:
----------------------------------------------
{code}
> int maxposix=sysconf(_SC_NGROUPS_MAX);
> gid_t *gps = (gid_t *) malloc(maxposix * sizeof (gid_t));
> if (!gps) {
> *ngroups = 0;
> free(pwbuf);
> return ENOMEM;
> }
...
{code}
Allocating an array of size _SC_NGROUPS_MAX seems a little excessive. I think it would be better to use realloc() to expand an array of gid_t as needed.
{code}
// set the name of the group for subsequent calls to getnetgrent
// note that we want to end group lokup regardless whether setnetgrent
// was successful or not (as long as it was called we need to call
// endnetgrent)
setnetgrentCalledFlag = 1;
setnetgrent(cgroup); // some platforms return void here
UserList *current = NULL;
// three pointers are for host, user, domain, we only care
// about user now
char *p[3];
while(getnetgrent(p, p + 1, p + 2)) {
if(p[1]) {
current = (UserList *)malloc(sizeof(UserList));
current->string = malloc(strlen(p[1]) + 1);
strcpy(current->string, p[1]);
current->next = userListHead;
userListHead = current;
userListSize++;
}
}
// shortcut null, nonexistent groups
if (userListSize == 0)
goto END;
{code}
It seems like this should be extracted into a function which would just return a linked list of UserList. There's no reason why the JNI function itself needs to reference setgrent(), endnetgrent(), etc.
I'm also disturbed by the non-re-entrancy of the getnetgrent() function. It seems like if we're going to use this function, we need to put a mutex around it. Alternately, getnetgrent_r() is re-entrant, and available on Linux (but not on some other platforms). You could do exactly the same thing you did here and add this to the CMakeLists.txt:
{code}
CHECK_FUNCTION_EXISTS(getnetgrent_r HAVE_NETGRENT_R)
{code}
The uses of getnetgrent() need to be surrounded by a mutex, probably similar to this:
{code}
static pthread_mutex_t g_getnetgrent_lock = PTHREAD_MUTEX_INITIALIZER;
UserList *get_net_groups(const char *name) {
pthread_mutex_lock(&lock);
... do non-thread-safe stuff ...
pthread_mutex_unlock(&lock);
}
{code}
> setnetgrent in native code is not portable
> ------------------------------------------
>
> Key: HADOOP-7147
> URL: https://issues.apache.org/jira/browse/HADOOP-7147
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Todd Lipcon
> Assignee: Allen Wittenauer
> Attachments: HADOOP-7147.patch, hadoop-7147.patch
>
>
> HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7147) setnetgrent in native code is not
portable
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13402856#comment-13402856 ]
Allen Wittenauer commented on HADOOP-7147:
------------------------------------------
It just occurred to me that this latest patch is incomplete anyway. getgrouplist() has weird behavior on OS X. I just forgot to forward port it from what I'm running in 1.x.
In any case, this patch is here for those that need it. I'm not really planning on working on it to get it commit-ready. (note this bug is marked as won't fix).
> setnetgrent in native code is not portable
> ------------------------------------------
>
> Key: HADOOP-7147
> URL: https://issues.apache.org/jira/browse/HADOOP-7147
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Todd Lipcon
> Assignee: Allen Wittenauer
> Attachments: HADOOP-7147.patch, hadoop-7147.patch
>
>
> HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7147) setnetgrent in native code is not
portable
Posted by "Alejandro Abdelnur (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13198934#comment-13198934 ]
Alejandro Abdelnur commented on HADOOP-7147:
--------------------------------------------
If it still an issue, but not a big priority, shouldn't we leave it open, with low priority?
If the patch gets rebased to trunk I'd be happy to tested it review it in Linux and OSX (don't have Solaris box around)
> setnetgrent in native code is not portable
> ------------------------------------------
>
> Key: HADOOP-7147
> URL: https://issues.apache.org/jira/browse/HADOOP-7147
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Todd Lipcon
> Assignee: Allen Wittenauer
> Attachments: hadoop-7147.patch
>
>
> HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-7147) setnetgrent in native code is not
portable
Posted by "Robert Joseph Evans (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans resolved HADOOP-7147.
-----------------------------------------
Resolution: Won't Fix
This is still an issue, but it is not a big priority right now. If support for OS's becomes important we can reopen this issue.
> setnetgrent in native code is not portable
> ------------------------------------------
>
> Key: HADOOP-7147
> URL: https://issues.apache.org/jira/browse/HADOOP-7147
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Todd Lipcon
> Assignee: Allen Wittenauer
> Attachments: hadoop-7147.patch
>
>
> HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7147) setnetgrent in native code is not
portable
Posted by "Allen Wittenauer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13198939#comment-13198939 ]
Allen Wittenauer commented on HADOOP-7147:
------------------------------------------
There is so much broken in the native code that ships with Hadoop with regards to portability that it isn't worth spending the time to fix. A major attitude shift really needs to happen first. So while I had a working libhadoop.so for OS X, Solaris, and other OSes for hadoop 0.20.20x (and I've been too busy fixing bugs 1.x to bring them up to date to 1.x), I don't feel particularly compelled to contribute the patches because they are just going to rot like this one did.
> setnetgrent in native code is not portable
> ------------------------------------------
>
> Key: HADOOP-7147
> URL: https://issues.apache.org/jira/browse/HADOOP-7147
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Todd Lipcon
> Assignee: Allen Wittenauer
> Attachments: hadoop-7147.patch
>
>
> HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7147) setnetgrent in native code is not
portable
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Allen Wittenauer updated HADOOP-7147:
-------------------------------------
Attachment: HADOOP-7147.patch
> setnetgrent in native code is not portable
> ------------------------------------------
>
> Key: HADOOP-7147
> URL: https://issues.apache.org/jira/browse/HADOOP-7147
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Todd Lipcon
> Assignee: Allen Wittenauer
> Attachments: HADOOP-7147.patch, hadoop-7147.patch
>
>
> HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7147) setnetgrent in native code is not
portable
Posted by "Allen Wittenauer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13198538#comment-13198538 ]
Allen Wittenauer commented on HADOOP-7147:
------------------------------------------
It's still an issue in the code sense, but not an issue in the priority sense. Probably better to just close as won't fix since portability isn't a concern for the project.
> setnetgrent in native code is not portable
> ------------------------------------------
>
> Key: HADOOP-7147
> URL: https://issues.apache.org/jira/browse/HADOOP-7147
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Todd Lipcon
> Assignee: Allen Wittenauer
> Attachments: hadoop-7147.patch
>
>
> HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7147) setnetgrent in native code is not
portable
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13403005#comment-13403005 ]
Colin Patrick McCabe commented on HADOOP-7147:
----------------------------------------------
Oops, meant to say "you can check to see if APPLE is set, but that covers Mac OS classic too."
> setnetgrent in native code is not portable
> ------------------------------------------
>
> Key: HADOOP-7147
> URL: https://issues.apache.org/jira/browse/HADOOP-7147
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Todd Lipcon
> Assignee: Allen Wittenauer
> Attachments: HADOOP-7147.patch, hadoop-7147.patch
>
>
> HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7147) setnetgrent in native code is not
portable
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13403002#comment-13403002 ]
Colin Patrick McCabe commented on HADOOP-7147:
----------------------------------------------
bq. I haven't figured out how to do OS or compiler detection to do the correct thing on non-gcc/non-Linux. (e.g., on Darwin, we should really be passing -framework JavaVM amongst other flags when building libhadoop.dylib so we get linked properly, can build a fat binary, etc.).
You want something like this:
{code}
IF(${CMAKE_SYSTEM_NAME} MATCHES "Darwin")
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -framework JavaVM")
ENDIF()
{code}
There are a bunch of CMake variables that are set based on the platform and environment. See http://www.cmake.org/Wiki/CMake_Useful_Variables. Of course, it's usually better to check for the specific feature you need if that is at all possible-- CHECK_FUNCTION_EXISTS is a good example of that.
You can also match against CMAKE_SYSTEM_NAME "Apple", but I think that covers Mac OS classic too. I *hope* that nobody is trying to compile Hadoop on that, although of course you can never be sure :)
> setnetgrent in native code is not portable
> ------------------------------------------
>
> Key: HADOOP-7147
> URL: https://issues.apache.org/jira/browse/HADOOP-7147
> Project: Hadoop Common
> Issue Type: Bug
> Components: native
> Affects Versions: 0.22.0, 0.23.0
> Reporter: Todd Lipcon
> Assignee: Allen Wittenauer
> Attachments: HADOOP-7147.patch, hadoop-7147.patch
>
>
> HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira