You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by "James Page (Created) (JIRA)" <ji...@apache.org> on 2011/11/29 12:41:39 UTC
[jira] [Created] (BIGTOP-280) Hadoop native fails to compile with
default linker option of --as-needed
Hadoop native fails to compile with default linker option of --as-needed
------------------------------------------------------------------------
Key: BIGTOP-280
URL: https://issues.apache.org/jira/browse/BIGTOP-280
Project: Bigtop
Issue Type: Bug
Components: Debian, General
Affects Versions: 0.2.0
Environment: Ubuntu Precise
Reporter: James Page
Priority: Minor
>From Ubuntu Oneiric onwards the default linker option for linked shared libraries is --as-needed.
As a result the AC_COMPUTE_NEEDED_DSO fails to locate the -lz shared library name and the build of hadoop native fails.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Commented] (BIGTOP-280) Hadoop native fails to compile
with default linker option of --as-needed
Posted by Roman Shaposhnik <rv...@apache.org>.
On Fri, Dec 9, 2011 at 12:50 PM, Anand L Ranganathan
<an...@gmail.com> wrote:
> I am facing similar issue. Till the patch is applied do you have
> workaround for this issue ?
You can always apply the patch locally or see the latest and greatest
state of things
in James's github repo.
Thanks,
Roman.
Re: [jira] [Commented] (BIGTOP-280) Hadoop native fails to compile
with default linker option of --as-needed
Posted by Anand L Ranganathan <an...@gmail.com>.
I am facing similar issue. Till the patch is applied do you have
workaround for this issue ?
Thanks
Anand
On Wed, Nov 30, 2011 at 5:51 AM, James Page (Commented) (JIRA) <
jira@apache.org> wrote:
>
> [
> https://issues.apache.org/jira/browse/BIGTOP-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160053#comment-13160053]
>
> James Page commented on BIGTOP-280:
> -----------------------------------
>
> So its possible to work around this in the bigtop packaging by including
> -Wl,--no-as-needed in the LDFLAGS setup in do-component-build for hadoop.
>
> It pushed the linker back to linking blindly rather than efficiently but
> it would work as a temp fix until stuff is accepted and released upstream.
>
> Its in-efficient but does also resolve the issue in c++ pipes (related to
> MAPREDUCE-2127 and the -lssl to -lcrypto switch).
>
> > Hadoop native fails to compile with default linker option of --as-needed
> > ------------------------------------------------------------------------
> >
> > Key: BIGTOP-280
> > URL: https://issues.apache.org/jira/browse/BIGTOP-280
> > Project: Bigtop
> > Issue Type: Bug
> > Components: Debian, General
> > Affects Versions: 0.2.0
> > Environment: Ubuntu Precise
> > Reporter: James Page
> > Priority: Minor
> >
> > From Ubuntu Oneiric onwards the default linker option for linked shared
> libraries is --as-needed.
> > As a result the AC_COMPUTE_NEEDED_DSO fails to locate the -lz shared
> library name and the build of hadoop native fails.
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> administrators:
> https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
>
[jira] [Commented] (BIGTOP-280) Hadoop native fails to compile with
default linker option of --as-needed
Posted by "Sridhar Iyer (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13166477#comment-13166477 ]
Sridhar Iyer commented on BIGTOP-280:
-------------------------------------
Can't pkg-config be used to find these libraries, instead of compiling a dummy file and running ld on it? Most pc files have a "libdir" variable in them that can be used to get the path of the library. We need to verify if .pc files are packaged uniformly across all the distros for this solution though.
> Hadoop native fails to compile with default linker option of --as-needed
> ------------------------------------------------------------------------
>
> Key: BIGTOP-280
> URL: https://issues.apache.org/jira/browse/BIGTOP-280
> Project: Bigtop
> Issue Type: Bug
> Components: Debian, General
> Affects Versions: 0.2.0
> Environment: Ubuntu Precise
> Reporter: James Page
> Priority: Minor
>
> From Ubuntu Oneiric onwards the default linker option for linked shared libraries is --as-needed.
> As a result the AC_COMPUTE_NEEDED_DSO fails to locate the -lz shared library name and the build of hadoop native fails.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-280) Hadoop native fails to compile with
default linker option of --as-needed
Posted by "James Page (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159204#comment-13159204 ]
James Page commented on BIGTOP-280:
-----------------------------------
Bug reported against hadoop-common with proposed patch.
> Hadoop native fails to compile with default linker option of --as-needed
> ------------------------------------------------------------------------
>
> Key: BIGTOP-280
> URL: https://issues.apache.org/jira/browse/BIGTOP-280
> Project: Bigtop
> Issue Type: Bug
> Components: Debian, General
> Affects Versions: 0.2.0
> Environment: Ubuntu Precise
> Reporter: James Page
> Priority: Minor
>
> From Ubuntu Oneiric onwards the default linker option for linked shared libraries is --as-needed.
> As a result the AC_COMPUTE_NEEDED_DSO fails to locate the -lz shared library name and the build of hadoop native fails.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-280) Hadoop native fails to compile with
default linker option of --as-needed
Posted by "Sridhar Iyer (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sridhar Iyer updated BIGTOP-280:
--------------------------------
Attachment: hadoopconfig.diff
James, this patch can be used to workaround the compilation failure for libz on ubuntu 11.10. This is not a complete patch (which is why I just attached and not submitted it).. If you think that this would be a way to go, I could make similar changes to other makefiles.
> Hadoop native fails to compile with default linker option of --as-needed
> ------------------------------------------------------------------------
>
> Key: BIGTOP-280
> URL: https://issues.apache.org/jira/browse/BIGTOP-280
> Project: Bigtop
> Issue Type: Bug
> Components: Debian, General
> Affects Versions: 0.2.0
> Environment: Ubuntu Precise
> Reporter: James Page
> Priority: Minor
> Attachments: hadoopconfig.diff
>
>
> From Ubuntu Oneiric onwards the default linker option for linked shared libraries is --as-needed.
> As a result the AC_COMPUTE_NEEDED_DSO fails to locate the -lz shared library name and the build of hadoop native fails.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-280) Hadoop native fails to compile with
default linker option of --as-needed
Posted by "James Page (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
James Page updated BIGTOP-280:
------------------------------
Attachment: BIGTOP-280.patch
> Hadoop native fails to compile with default linker option of --as-needed
> ------------------------------------------------------------------------
>
> Key: BIGTOP-280
> URL: https://issues.apache.org/jira/browse/BIGTOP-280
> Project: Bigtop
> Issue Type: Bug
> Components: Debian, General
> Affects Versions: 0.2.0
> Environment: Ubuntu Precise
> Reporter: James Page
> Priority: Minor
> Attachments: BIGTOP-280.patch, hadoopconfig.diff
>
>
> From Ubuntu Oneiric onwards the default linker option for linked shared libraries is --as-needed.
> As a result the AC_COMPUTE_NEEDED_DSO fails to locate the -lz shared library name and the build of hadoop native fails.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-280) Hadoop native fails to compile with
default linker option of --as-needed
Posted by "James Page (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176823#comment-13176823 ]
James Page commented on BIGTOP-280:
-----------------------------------
Sridhar, I'm reluctant to deviate from the way that hadoop native resolves library names as I think its done this way to ensure cross platform build functionality; as stated above its possible to work around this issue by ensuring that all libraries are linked irrespective or not as to whether they are actually required - I will produce a patch for this in the next couple of days for bigtop.
> Hadoop native fails to compile with default linker option of --as-needed
> ------------------------------------------------------------------------
>
> Key: BIGTOP-280
> URL: https://issues.apache.org/jira/browse/BIGTOP-280
> Project: Bigtop
> Issue Type: Bug
> Components: Debian, General
> Affects Versions: 0.2.0
> Environment: Ubuntu Precise
> Reporter: James Page
> Priority: Minor
> Attachments: hadoopconfig.diff
>
>
> From Ubuntu Oneiric onwards the default linker option for linked shared libraries is --as-needed.
> As a result the AC_COMPUTE_NEEDED_DSO fails to locate the -lz shared library name and the build of hadoop native fails.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-280) Hadoop native fails to compile with
default linker option of --as-needed
Posted by "Roman Shaposhnik (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160428#comment-13160428 ]
Roman Shaposhnik commented on BIGTOP-280:
-----------------------------------------
Typically we're fine with hosting workaround in Bigtop for as long as it take for components to have a release with the patch applied. Please, send a Bigtop patch.
> Hadoop native fails to compile with default linker option of --as-needed
> ------------------------------------------------------------------------
>
> Key: BIGTOP-280
> URL: https://issues.apache.org/jira/browse/BIGTOP-280
> Project: Bigtop
> Issue Type: Bug
> Components: Debian, General
> Affects Versions: 0.2.0
> Environment: Ubuntu Precise
> Reporter: James Page
> Priority: Minor
>
> From Ubuntu Oneiric onwards the default linker option for linked shared libraries is --as-needed.
> As a result the AC_COMPUTE_NEEDED_DSO fails to locate the -lz shared library name and the build of hadoop native fails.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-280) Hadoop native fails to compile with
default linker option of --as-needed
Posted by "James Page (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160053#comment-13160053 ]
James Page commented on BIGTOP-280:
-----------------------------------
So its possible to work around this in the bigtop packaging by including -Wl,--no-as-needed in the LDFLAGS setup in do-component-build for hadoop.
It pushed the linker back to linking blindly rather than efficiently but it would work as a temp fix until stuff is accepted and released upstream.
Its in-efficient but does also resolve the issue in c++ pipes (related to MAPREDUCE-2127 and the -lssl to -lcrypto switch).
> Hadoop native fails to compile with default linker option of --as-needed
> ------------------------------------------------------------------------
>
> Key: BIGTOP-280
> URL: https://issues.apache.org/jira/browse/BIGTOP-280
> Project: Bigtop
> Issue Type: Bug
> Components: Debian, General
> Affects Versions: 0.2.0
> Environment: Ubuntu Precise
> Reporter: James Page
> Priority: Minor
>
> From Ubuntu Oneiric onwards the default linker option for linked shared libraries is --as-needed.
> As a result the AC_COMPUTE_NEEDED_DSO fails to locate the -lz shared library name and the build of hadoop native fails.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-280) Hadoop native fails to compile with
default linker option of --as-needed
Posted by "Mark Butler (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13192199#comment-13192199 ]
Mark Butler commented on BIGTOP-280:
------------------------------------
This is related to https://issues.apache.org/jira/browse/HADOOP-7979
> Hadoop native fails to compile with default linker option of --as-needed
> ------------------------------------------------------------------------
>
> Key: BIGTOP-280
> URL: https://issues.apache.org/jira/browse/BIGTOP-280
> Project: Bigtop
> Issue Type: Bug
> Components: Debian, General
> Affects Versions: 0.2.0
> Environment: Ubuntu Precise
> Reporter: James Page
> Priority: Minor
> Attachments: BIGTOP-280.patch, hadoopconfig.diff
>
>
> From Ubuntu Oneiric onwards the default linker option for linked shared libraries is --as-needed.
> As a result the AC_COMPUTE_NEEDED_DSO fails to locate the -lz shared library name and the build of hadoop native fails.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-280) Hadoop native fails to compile with
default linker option of --as-needed
Posted by "Bruno Mahé (Commented JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13160447#comment-13160447 ]
Bruno Mahé commented on BIGTOP-280:
-----------------------------------
If you provide such patch to Bigtop, please, add also a reference to the upstream jira in the comments, so that workaround can be tracked and removed when it becomes unnecessary.
> Hadoop native fails to compile with default linker option of --as-needed
> ------------------------------------------------------------------------
>
> Key: BIGTOP-280
> URL: https://issues.apache.org/jira/browse/BIGTOP-280
> Project: Bigtop
> Issue Type: Bug
> Components: Debian, General
> Affects Versions: 0.2.0
> Environment: Ubuntu Precise
> Reporter: James Page
> Priority: Minor
>
> From Ubuntu Oneiric onwards the default linker option for linked shared libraries is --as-needed.
> As a result the AC_COMPUTE_NEEDED_DSO fails to locate the -lz shared library name and the build of hadoop native fails.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-280) Hadoop native fails to compile with
default linker option of --as-needed
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Roman Shaposhnik updated BIGTOP-280:
------------------------------------
Fix Version/s: 0.3.1
Assignee: James Page
> Hadoop native fails to compile with default linker option of --as-needed
> ------------------------------------------------------------------------
>
> Key: BIGTOP-280
> URL: https://issues.apache.org/jira/browse/BIGTOP-280
> Project: Bigtop
> Issue Type: Bug
> Components: Debian, General
> Affects Versions: 0.2.0
> Environment: Ubuntu Precise
> Reporter: James Page
> Assignee: James Page
> Priority: Minor
> Fix For: 0.3.1
>
> Attachments: BIGTOP-280.patch, hadoopconfig.diff
>
>
> From Ubuntu Oneiric onwards the default linker option for linked shared libraries is --as-needed.
> As a result the AC_COMPUTE_NEEDED_DSO fails to locate the -lz shared library name and the build of hadoop native fails.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-280) Hadoop native fails to compile with
default linker option of --as-needed
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397769#comment-13397769 ]
Roman Shaposhnik commented on BIGTOP-280:
-----------------------------------------
This issue only affects Hadoop 1.X code line -- hence pushing to the 0.3.1 potential release.
> Hadoop native fails to compile with default linker option of --as-needed
> ------------------------------------------------------------------------
>
> Key: BIGTOP-280
> URL: https://issues.apache.org/jira/browse/BIGTOP-280
> Project: Bigtop
> Issue Type: Bug
> Components: Debian, General
> Affects Versions: 0.2.0
> Environment: Ubuntu Precise
> Reporter: James Page
> Assignee: James Page
> Priority: Minor
> Fix For: 0.3.1
>
> Attachments: BIGTOP-280.patch, hadoopconfig.diff
>
>
> From Ubuntu Oneiric onwards the default linker option for linked shared libraries is --as-needed.
> As a result the AC_COMPUTE_NEEDED_DSO fails to locate the -lz shared library name and the build of hadoop native fails.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-280) Hadoop native fails to compile with
default linker option of --as-needed
Posted by "Eli Collins (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159351#comment-13159351 ]
Eli Collins commented on BIGTOP-280:
------------------------------------
Thanks James, does the rest of the build work with this patch? Seems like this should work with the old RHEL5 toolchain but I didn't look closely.
> Hadoop native fails to compile with default linker option of --as-needed
> ------------------------------------------------------------------------
>
> Key: BIGTOP-280
> URL: https://issues.apache.org/jira/browse/BIGTOP-280
> Project: Bigtop
> Issue Type: Bug
> Components: Debian, General
> Affects Versions: 0.2.0
> Environment: Ubuntu Precise
> Reporter: James Page
> Priority: Minor
>
> From Ubuntu Oneiric onwards the default linker option for linked shared libraries is --as-needed.
> As a result the AC_COMPUTE_NEEDED_DSO fails to locate the -lz shared library name and the build of hadoop native fails.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-280) Hadoop native fails to compile with
default linker option of --as-needed
Posted by "James Page (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159949#comment-13159949 ]
James Page commented on BIGTOP-280:
-----------------------------------
Not quite; I'm also looking at build failures in the c++ pipes and fuse-dfs modules - similar issues.
> Hadoop native fails to compile with default linker option of --as-needed
> ------------------------------------------------------------------------
>
> Key: BIGTOP-280
> URL: https://issues.apache.org/jira/browse/BIGTOP-280
> Project: Bigtop
> Issue Type: Bug
> Components: Debian, General
> Affects Versions: 0.2.0
> Environment: Ubuntu Precise
> Reporter: James Page
> Priority: Minor
>
> From Ubuntu Oneiric onwards the default linker option for linked shared libraries is --as-needed.
> As a result the AC_COMPUTE_NEEDED_DSO fails to locate the -lz shared library name and the build of hadoop native fails.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira