[jira] [Commented] (HADOOP-11127) Improve versioning and compatibility support in native library for downstream hadoop-common users.

["Chris Nauroth (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-11127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15061083#comment-15061083 ] 

Chris Nauroth commented on HADOOP-11127:
----------------------------------------

I've done some more thinking about this, and I'm now on board with version-stamping the library names.  I'd like us to proceed with that as the scope for this JIRA.

Version-stamping the libraries alone is an incomplete solution because of the issues I raised in my earlier comments.  However, I believe version-stamping the libraries is a pre-requisite for any more complete solution, such as bundling the native code into the jar.

There are a few remaining requirements though that were not addressed by the earlier proposals and patches:

# Fallback logic should make a best effort attempt to load a possibly compatible library if an exact match is not available.  For example, assuming hadoop-common-2.8.3.jar, it should attempt to load, in order, libhadoop-2.8.3.so, libhadoop-2.8.2.so, libhadoop-2.8.1.so, libhadoop-2.8.0.so, and finally libhadoop.so as the ultimate fallback.  This would help mitigate the deficiencies I described earlier when server-side deployment falls behind client-side upgrades.  A client who decides to upgrade form hadoop-common-2.8.2.jar to hadoop-common-2.8.3.jar should not suddenly see their application break because the library loading logic went all the way back to the old libhadoop.so.
# The solution must cover both *nix (libhadoop.so) and Windows (hadoop.dll and winutils.exe) completely.
# Let's review with someone from BigTop to make sure the change wouldn't have unintended consequences for rpm/deb packaging.

> Improve versioning and compatibility support in native library for downstream hadoop-common users.
> --------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-11127
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11127
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: native
>            Reporter: Chris Nauroth
>            Assignee: Alan Burlison
>         Attachments: HADOOP-11064.003.patch, proposal.01.txt
>
>
> There is no compatibility policy enforced on the JNI function signatures implemented in the native library.  This library typically is deployed to all nodes in a cluster, built from a specific source code version.  However, downstream applications that want to run in that cluster might choose to bundle a hadoop-common jar at a different version.  Since there is no compatibility policy, this can cause link errors at runtime when the native function signatures expected by hadoop-common.jar do not exist in libhadoop.so/hadoop.dll.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)