You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hadoop.apache.org by Bulldog20630405 <bu...@gmail.com> on 2022/03/10 20:54:42 UTC

hadoop 3.3 native libs missing lz4 and snappy

when building hadoop native on the same system:
3.2.3 builds with lz4 and snappy
3.3.2 builds without lz4 and snappy

how to get lz4 and snappy native libs with 3.3.2 ?


> ~/software/hadoop/hadoop-3.2.3/bin/hadoop checknative -a
Native library checking:
hadoop:  true ~/software/hadoop/*hadoop-3.2.3*/lib/native/libhadoop.so.1.0.0
zlib:    true /lib64/libz.so.1
zstd  :  true /lib64/libzstd.so.1

*snappy:  true /lib64/libsnappy.so.1lz4:     true revision:10301*
bzip2:   true /lib64/libbz2.so.1
openssl: true /lib64/libcrypto.so
ISA-L:   false libhadoop was built without ISA-L support
PMDK:    false The native code was built without PMDK support.


> ~/software/hadoop/*hadoop-3.3.2*/bin/hadoop checknative -a
Native library checking:
hadoop:  true ~/software/hadoop/hadoop-3.3.2/lib/native/libhadoop.so.1.0.0
zlib:    true /lib64/libz.so.1
zstd  :  true /lib64/libzstd.so.1
bzip2:   true /lib64/libbz2.so.1
openssl: true /lib64/libcrypto.so
ISA-L:   false libhadoop was built without ISA-L support
PMDK:    false The native code was built without PMDK support.
>

Re: hadoop 3.3 native libs missing lz4 and snappy

Posted by Chao Sun <su...@apache.org>.

Hi, starting from 3.3.1, Hadoop has switched to use lz4-java and
snappy-java and no longer depending on the native libraries.
See HADOOP-17292 and HADOOP-17125 for details.

Chao

On Thu, Mar 10, 2022 at 12:55 PM Bulldog20630405 <bu...@gmail.com>
wrote:

> when building hadoop native on the same system:
> 3.2.3 builds with lz4 and snappy
> 3.3.2 builds without lz4 and snappy
>
> how to get lz4 and snappy native libs with 3.3.2 ?
>
>
> > ~/software/hadoop/hadoop-3.2.3/bin/hadoop checknative -a
> Native library checking:
> hadoop:  true ~/software/hadoop/*hadoop-3.2.3*
> /lib/native/libhadoop.so.1.0.0
> zlib:    true /lib64/libz.so.1
> zstd  :  true /lib64/libzstd.so.1
>
> *snappy:  true /lib64/libsnappy.so.1lz4:     true revision:10301*
> bzip2:   true /lib64/libbz2.so.1
> openssl: true /lib64/libcrypto.so
> ISA-L:   false libhadoop was built without ISA-L support
> PMDK:    false The native code was built without PMDK support.
>
>
> > ~/software/hadoop/*hadoop-3.3.2*/bin/hadoop checknative -a
> Native library checking:
> hadoop:  true ~/software/hadoop/hadoop-3.3.2/lib/native/libhadoop.so.1.0.0
> zlib:    true /lib64/libz.so.1
> zstd  :  true /lib64/libzstd.so.1
> bzip2:   true /lib64/libbz2.so.1
> openssl: true /lib64/libcrypto.so
> ISA-L:   false libhadoop was built without ISA-L support
> PMDK:    false The native code was built without PMDK support.
> >
>
>
>
>
>
>
>
>