You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by "Robert Metzger (Jira)" <ji...@apache.org> on 2022/09/06 12:24:00 UTC
[jira] [Created] (FLINK-29212) Properly load Hadoop native libraries in Flink docker iamges
Robert Metzger created FLINK-29212:
--------------------------------------
Summary: Properly load Hadoop native libraries in Flink docker iamges
Key: FLINK-29212
URL: https://issues.apache.org/jira/browse/FLINK-29212
Project: Flink
Issue Type: Bug
Components: flink-docker
Affects Versions: 1.17.0
Reporter: Robert Metzger
On startup, Flink logs:
{code:java}
2022-09-04 12:36:03.559 [main] WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
{code}
Hadoop native libraries are used for:
- Compression Codecs (bzip2, lz4, zlib)
- Native IO utilities for HDFS Short-Circuit Local Reads and Centralized Cache Management in HDFS
- CRC32 checksum implementation
(Source: https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/NativeLibraries.html)
Resolving this for the docker images we are providing should be easy, remove one unnecessary WARNing and provide performance benefits for some users.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)