You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@impala.apache.org by Jim Apple <jb...@cloudera.com> on 2016/08/11 20:12:12 UTC

New (to me) linker failures

When linking impalad I get errors like

be/build/debug/util/libUtil.so: error: undefined reference to
'BZ2_bzBuffToBuffCompress'

This includes with versions of Impala that I have compiled as recently
as a couple of hours ago. It is now affecting all recent git hashes.
The full list of libraries not found is:

BZ2_bzBuffToBuffCompress
snappy::MaxCompressedLength(unsigned long)
snappy::RawCompress(char const*, unsigned long, char*, unsigned long*)
LZ4_compress
BZ2_bzDecompressEnd
BZ2_bzDecompressInit
BZ2_bzBuffToBuffDecompress
BZ2_bzDecompress
snappy::GetUncompressedLength(char const*, unsigned long, unsigned long*)
snappy::RawUncompress(char const*, unsigned long, char*)
LZ4_uncompress
my_strlen
google_breakpad::ExceptionHandler::WriteMinidump(std::string const&,
bool (*)(google_breakpad::MinidumpDescriptor const&, void*, bool),
void*)
google_breakpad::ExceptionHandler::ExceptionHandler(google_breakpad::MinidumpDescriptor
const&, bool (*)(void*), bool (*)(google_breakpad::MinidumpDescriptor
const&, void*, bool), void*, bool, int)
ProfilerStart
ProfilerStop
boost::basic_regex<char, boost::regex_traits<char,
boost::cpp_regex_traits<char> > >::do_assign(char const*, char const*,
unsigned int)
boost::re_detail::perl_matcher<__gnu_cxx::__normal_iterator<char
const*, std::string>,
std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char
const*, std::string> > >, boost::regex_traits<char,
boost::cpp_regex_traits<char> > >::find()
boost::re_detail::perl_matcher<__gnu_cxx::__normal_iterator<char
const*, std::string>,
std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char
const*, std::string> > >, boost::regex_traits<char,
boost::cpp_regex_traits<char> >
>::construct_init(boost::basic_regex<char, boost::regex_traits<char,
boost::cpp_regex_traits<char> > > const&,
boost::regex_constants::_match_flags)

I tried a clean build, I tried

rm $IMPALA_HOME/CMakeCache.txt

I tried

cd $IMPALA_HOME && find -iname '*cmake*' -not -name CMakeLists.txt |
grep -v -e cmake_module | grep -v -e thirdparty | xargs rm -Rf

(both listed here:
https://groups.google.com/a/cloudera.org/forum/#!topic/impala-dev/V5K7u7hs56w)

I tried

unset LD_LIBRARY_PATH && source bin/impala-config.sh

I tried unsetting the use of the gold linker, by setting
USE_GOLD_LINKER to "false" (and by unsetting it)

Has anyone seen and fixed something like this?

Re: New (to me) linker failures

Posted by Taras Bobrovytsky <tb...@cloudera.com>.
I haven't seen that. As a last resort, you could try running something like
the following in the IMPALA_HOME directory: This will delete everything in
the directory except the checked in files. Then it will reload data
(because it will be deleted).
./testdata/bin/kill-all.sh && git clean -dfx &&
$IMPALA_AUX_TEST_HOME/jenkins/copy-test-data.sh && ./buildall.sh -format
-testdata -notests

On Thu, Aug 11, 2016 at 8:12 PM, Jim Apple <jb...@cloudera.com> wrote:

> When linking impalad I get errors like
>
> be/build/debug/util/libUtil.so: error: undefined reference to
> 'BZ2_bzBuffToBuffCompress'
>
> This includes with versions of Impala that I have compiled as recently
> as a couple of hours ago. It is now affecting all recent git hashes.
> The full list of libraries not found is:
>
> BZ2_bzBuffToBuffCompress
> snappy::MaxCompressedLength(unsigned long)
> snappy::RawCompress(char const*, unsigned long, char*, unsigned long*)
> LZ4_compress
> BZ2_bzDecompressEnd
> BZ2_bzDecompressInit
> BZ2_bzBuffToBuffDecompress
> BZ2_bzDecompress
> snappy::GetUncompressedLength(char const*, unsigned long, unsigned long*)
> snappy::RawUncompress(char const*, unsigned long, char*)
> LZ4_uncompress
> my_strlen
> google_breakpad::ExceptionHandler::WriteMinidump(std::string const&,
> bool (*)(google_breakpad::MinidumpDescriptor const&, void*, bool),
> void*)
> google_breakpad::ExceptionHandler::ExceptionHandler(google_
> breakpad::MinidumpDescriptor
> const&, bool (*)(void*), bool (*)(google_breakpad::MinidumpDescriptor
> const&, void*, bool), void*, bool, int)
> ProfilerStart
> ProfilerStop
> boost::basic_regex<char, boost::regex_traits<char,
> boost::cpp_regex_traits<char> > >::do_assign(char const*, char const*,
> unsigned int)
> boost::re_detail::perl_matcher<__gnu_cxx::__normal_iterator<char
> const*, std::string>,
> std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char
> const*, std::string> > >, boost::regex_traits<char,
> boost::cpp_regex_traits<char> > >::find()
> boost::re_detail::perl_matcher<__gnu_cxx::__normal_iterator<char
> const*, std::string>,
> std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char
> const*, std::string> > >, boost::regex_traits<char,
> boost::cpp_regex_traits<char> >
> >::construct_init(boost::basic_regex<char, boost::regex_traits<char,
> boost::cpp_regex_traits<char> > > const&,
> boost::regex_constants::_match_flags)
>
> I tried a clean build, I tried
>
> rm $IMPALA_HOME/CMakeCache.txt
>
> I tried
>
> cd $IMPALA_HOME && find -iname '*cmake*' -not -name CMakeLists.txt |
> grep -v -e cmake_module | grep -v -e thirdparty | xargs rm -Rf
>
> (both listed here:
> https://groups.google.com/a/cloudera.org/forum/#!topic/
> impala-dev/V5K7u7hs56w)
>
> I tried
>
> unset LD_LIBRARY_PATH && source bin/impala-config.sh
>
> I tried unsetting the use of the gold linker, by setting
> USE_GOLD_LINKER to "false" (and by unsetting it)
>
> Has anyone seen and fixed something like this?
>

Re: New (to me) linker failures

Posted by Dimitris Tsirogiannis <dt...@cloudera.com>.
Thanks Jim.

On Wed, Aug 17, 2016 at 10:09 AM, Jim Apple <jb...@cloudera.com> wrote:

> David Knupp and Tim Armstrong saw this, too:
>
> https://issues.cloudera.org/browse/IMPALA-3979
>
> On Thu, Aug 11, 2016 at 2:08 PM, Jim Apple <jb...@cloudera.com> wrote:
> > It works if I don't pass the '-so' flag to buildall.
> >
> > On Thu, Aug 11, 2016 at 1:36 PM, Jim Apple <jb...@cloudera.com> wrote:
> >> Oh, that reminds me: I also tried bin/bootstrap_toolchain.py
> >>
> >> On Thu, Aug 11, 2016 at 1:34 PM, Matthew Jacobs <mj...@cloudera.com>
> wrote:
> >>> Thanks. I didn't see anything wrong there (assuming the toolchain
> >>> directories exist), so I'm not sure, sorry :/
> >>>
> >>> On Thu, Aug 11, 2016 at 1:28 PM, Jim Apple <jb...@cloudera.com>
> wrote:
> >>>>> Just to be clear: did clear your cmake files _after_ resetting and
> >>>>> resourcing the environment variables?
> >>>>
> >>>> yes
> >>>>
> >>>>>
> >>>>> What's your LD_LIBRARY_PATH and IMPALA_TOOLCHAIN ? (Maybe just post
> >>>>> the output of env.)
> >>>>
> >>>> Attached.
>

Re: New (to me) linker failures

Posted by Jim Apple <jb...@cloudera.com>.
David Knupp and Tim Armstrong saw this, too:

https://issues.cloudera.org/browse/IMPALA-3979

On Thu, Aug 11, 2016 at 2:08 PM, Jim Apple <jb...@cloudera.com> wrote:
> It works if I don't pass the '-so' flag to buildall.
>
> On Thu, Aug 11, 2016 at 1:36 PM, Jim Apple <jb...@cloudera.com> wrote:
>> Oh, that reminds me: I also tried bin/bootstrap_toolchain.py
>>
>> On Thu, Aug 11, 2016 at 1:34 PM, Matthew Jacobs <mj...@cloudera.com> wrote:
>>> Thanks. I didn't see anything wrong there (assuming the toolchain
>>> directories exist), so I'm not sure, sorry :/
>>>
>>> On Thu, Aug 11, 2016 at 1:28 PM, Jim Apple <jb...@cloudera.com> wrote:
>>>>> Just to be clear: did clear your cmake files _after_ resetting and
>>>>> resourcing the environment variables?
>>>>
>>>> yes
>>>>
>>>>>
>>>>> What's your LD_LIBRARY_PATH and IMPALA_TOOLCHAIN ? (Maybe just post
>>>>> the output of env.)
>>>>
>>>> Attached.

Re: New (to me) linker failures

Posted by Jim Apple <jb...@cloudera.com>.
It works if I don't pass the '-so' flag to buildall.

On Thu, Aug 11, 2016 at 1:36 PM, Jim Apple <jb...@cloudera.com> wrote:
> Oh, that reminds me: I also tried bin/bootstrap_toolchain.py
>
> On Thu, Aug 11, 2016 at 1:34 PM, Matthew Jacobs <mj...@cloudera.com> wrote:
>> Thanks. I didn't see anything wrong there (assuming the toolchain
>> directories exist), so I'm not sure, sorry :/
>>
>> On Thu, Aug 11, 2016 at 1:28 PM, Jim Apple <jb...@cloudera.com> wrote:
>>>> Just to be clear: did clear your cmake files _after_ resetting and
>>>> resourcing the environment variables?
>>>
>>> yes
>>>
>>>>
>>>> What's your LD_LIBRARY_PATH and IMPALA_TOOLCHAIN ? (Maybe just post
>>>> the output of env.)
>>>
>>> Attached.

Re: New (to me) linker failures

Posted by Jim Apple <jb...@cloudera.com>.
Oh, that reminds me: I also tried bin/bootstrap_toolchain.py

On Thu, Aug 11, 2016 at 1:34 PM, Matthew Jacobs <mj...@cloudera.com> wrote:
> Thanks. I didn't see anything wrong there (assuming the toolchain
> directories exist), so I'm not sure, sorry :/
>
> On Thu, Aug 11, 2016 at 1:28 PM, Jim Apple <jb...@cloudera.com> wrote:
>>> Just to be clear: did clear your cmake files _after_ resetting and
>>> resourcing the environment variables?
>>
>> yes
>>
>>>
>>> What's your LD_LIBRARY_PATH and IMPALA_TOOLCHAIN ? (Maybe just post
>>> the output of env.)
>>
>> Attached.

Re: New (to me) linker failures

Posted by Matthew Jacobs <mj...@cloudera.com>.
Thanks. I didn't see anything wrong there (assuming the toolchain
directories exist), so I'm not sure, sorry :/

On Thu, Aug 11, 2016 at 1:28 PM, Jim Apple <jb...@cloudera.com> wrote:
>> Just to be clear: did clear your cmake files _after_ resetting and
>> resourcing the environment variables?
>
> yes
>
>>
>> What's your LD_LIBRARY_PATH and IMPALA_TOOLCHAIN ? (Maybe just post
>> the output of env.)
>
> Attached.

Re: New (to me) linker failures

Posted by Jim Apple <jb...@cloudera.com>.
> Just to be clear: did clear your cmake files _after_ resetting and
> resourcing the environment variables?

yes

>
> What's your LD_LIBRARY_PATH and IMPALA_TOOLCHAIN ? (Maybe just post
> the output of env.)

Attached.

Re: New (to me) linker failures

Posted by Matthew Jacobs <mj...@cloudera.com>.
Just to be clear: did clear your cmake files _after_ resetting and
resourcing the environment variables?

What's your LD_LIBRARY_PATH and IMPALA_TOOLCHAIN ? (Maybe just post
the output of env.)

On Thu, Aug 11, 2016 at 1:14 PM, Jim Apple <jb...@cloudera.com> wrote:
> Oh, and I'm on Ubuntu 14.04. I did not upgrade any software on my
> machine between when this did build and when it did not build.
>
> On Thu, Aug 11, 2016 at 1:12 PM, Jim Apple <jb...@cloudera.com> wrote:
>> When linking impalad I get errors like
>>
>> be/build/debug/util/libUtil.so: error: undefined reference to
>> 'BZ2_bzBuffToBuffCompress'
>>
>> This includes with versions of Impala that I have compiled as recently
>> as a couple of hours ago. It is now affecting all recent git hashes.
>> The full list of libraries not found is:
>>
>> BZ2_bzBuffToBuffCompress
>> snappy::MaxCompressedLength(unsigned long)
>> snappy::RawCompress(char const*, unsigned long, char*, unsigned long*)
>> LZ4_compress
>> BZ2_bzDecompressEnd
>> BZ2_bzDecompressInit
>> BZ2_bzBuffToBuffDecompress
>> BZ2_bzDecompress
>> snappy::GetUncompressedLength(char const*, unsigned long, unsigned long*)
>> snappy::RawUncompress(char const*, unsigned long, char*)
>> LZ4_uncompress
>> my_strlen
>> google_breakpad::ExceptionHandler::WriteMinidump(std::string const&,
>> bool (*)(google_breakpad::MinidumpDescriptor const&, void*, bool),
>> void*)
>> google_breakpad::ExceptionHandler::ExceptionHandler(google_breakpad::MinidumpDescriptor
>> const&, bool (*)(void*), bool (*)(google_breakpad::MinidumpDescriptor
>> const&, void*, bool), void*, bool, int)
>> ProfilerStart
>> ProfilerStop
>> boost::basic_regex<char, boost::regex_traits<char,
>> boost::cpp_regex_traits<char> > >::do_assign(char const*, char const*,
>> unsigned int)
>> boost::re_detail::perl_matcher<__gnu_cxx::__normal_iterator<char
>> const*, std::string>,
>> std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char
>> const*, std::string> > >, boost::regex_traits<char,
>> boost::cpp_regex_traits<char> > >::find()
>> boost::re_detail::perl_matcher<__gnu_cxx::__normal_iterator<char
>> const*, std::string>,
>> std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char
>> const*, std::string> > >, boost::regex_traits<char,
>> boost::cpp_regex_traits<char> >
>>>::construct_init(boost::basic_regex<char, boost::regex_traits<char,
>> boost::cpp_regex_traits<char> > > const&,
>> boost::regex_constants::_match_flags)
>>
>> I tried a clean build, I tried
>>
>> rm $IMPALA_HOME/CMakeCache.txt
>>
>> I tried
>>
>> cd $IMPALA_HOME && find -iname '*cmake*' -not -name CMakeLists.txt |
>> grep -v -e cmake_module | grep -v -e thirdparty | xargs rm -Rf
>>
>> (both listed here:
>> https://groups.google.com/a/cloudera.org/forum/#!topic/impala-dev/V5K7u7hs56w)
>>
>> I tried
>>
>> unset LD_LIBRARY_PATH && source bin/impala-config.sh
>>
>> I tried unsetting the use of the gold linker, by setting
>> USE_GOLD_LINKER to "false" (and by unsetting it)
>>
>> Has anyone seen and fixed something like this?

Re: New (to me) linker failures

Posted by Jim Apple <jb...@cloudera.com>.
Oh, and I'm on Ubuntu 14.04. I did not upgrade any software on my
machine between when this did build and when it did not build.

On Thu, Aug 11, 2016 at 1:12 PM, Jim Apple <jb...@cloudera.com> wrote:
> When linking impalad I get errors like
>
> be/build/debug/util/libUtil.so: error: undefined reference to
> 'BZ2_bzBuffToBuffCompress'
>
> This includes with versions of Impala that I have compiled as recently
> as a couple of hours ago. It is now affecting all recent git hashes.
> The full list of libraries not found is:
>
> BZ2_bzBuffToBuffCompress
> snappy::MaxCompressedLength(unsigned long)
> snappy::RawCompress(char const*, unsigned long, char*, unsigned long*)
> LZ4_compress
> BZ2_bzDecompressEnd
> BZ2_bzDecompressInit
> BZ2_bzBuffToBuffDecompress
> BZ2_bzDecompress
> snappy::GetUncompressedLength(char const*, unsigned long, unsigned long*)
> snappy::RawUncompress(char const*, unsigned long, char*)
> LZ4_uncompress
> my_strlen
> google_breakpad::ExceptionHandler::WriteMinidump(std::string const&,
> bool (*)(google_breakpad::MinidumpDescriptor const&, void*, bool),
> void*)
> google_breakpad::ExceptionHandler::ExceptionHandler(google_breakpad::MinidumpDescriptor
> const&, bool (*)(void*), bool (*)(google_breakpad::MinidumpDescriptor
> const&, void*, bool), void*, bool, int)
> ProfilerStart
> ProfilerStop
> boost::basic_regex<char, boost::regex_traits<char,
> boost::cpp_regex_traits<char> > >::do_assign(char const*, char const*,
> unsigned int)
> boost::re_detail::perl_matcher<__gnu_cxx::__normal_iterator<char
> const*, std::string>,
> std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char
> const*, std::string> > >, boost::regex_traits<char,
> boost::cpp_regex_traits<char> > >::find()
> boost::re_detail::perl_matcher<__gnu_cxx::__normal_iterator<char
> const*, std::string>,
> std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char
> const*, std::string> > >, boost::regex_traits<char,
> boost::cpp_regex_traits<char> >
>>::construct_init(boost::basic_regex<char, boost::regex_traits<char,
> boost::cpp_regex_traits<char> > > const&,
> boost::regex_constants::_match_flags)
>
> I tried a clean build, I tried
>
> rm $IMPALA_HOME/CMakeCache.txt
>
> I tried
>
> cd $IMPALA_HOME && find -iname '*cmake*' -not -name CMakeLists.txt |
> grep -v -e cmake_module | grep -v -e thirdparty | xargs rm -Rf
>
> (both listed here:
> https://groups.google.com/a/cloudera.org/forum/#!topic/impala-dev/V5K7u7hs56w)
>
> I tried
>
> unset LD_LIBRARY_PATH && source bin/impala-config.sh
>
> I tried unsetting the use of the gold linker, by setting
> USE_GOLD_LINKER to "false" (and by unsetting it)
>
> Has anyone seen and fixed something like this?