You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Ryan Skraba (Jira)" <ji...@apache.org> on 2024/04/19 13:22:00 UTC
[jira] [Commented] (FLINK-35175) HadoopDataInputStream can't compile with Hadoop 3.2.3
[ https://issues.apache.org/jira/browse/FLINK-35175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17838999#comment-17838999 ]
Ryan Skraba commented on FLINK-35175:
-------------------------------------
[~hangxiang]: what do you think? Is it better to bump the supported version of Hadoop or fix the class that relies on this capability?
> HadoopDataInputStream can't compile with Hadoop 3.2.3
> -----------------------------------------------------
>
> Key: FLINK-35175
> URL: https://issues.apache.org/jira/browse/FLINK-35175
> Project: Flink
> Issue Type: Bug
> Affects Versions: 1.20.0
> Reporter: Ryan Skraba
> Priority: Critical
>
> Unfortunately, introduced in FLINK-35045: [PREADWRITEBUFFER|https://github.com/apache/flink/commit/a312a3bdd258e0ff7d6f94e979b32e2bc762b82f#diff-3ed57be01895ba0f792110e40f4283427c55528f11a5105b4bf34ebd4e6fef0dR182] was added in Hadoop releases [3.3.0|https://github.com/apache/hadoop/blob/rel/release-3.3.0/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/StreamCapabilities.java#L72] and [2.10.0|https://github.com/apache/hadoop/blob/rel/release-2.10.0/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/StreamCapabilities.java#L72].
> It doesn't exist in flink.hadoop.version [3.2.3|https://github.com/apache/hadoop/blob/rel/release-3.2.3/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/StreamCapabilities.java], which we are using in end-to-end tests.
> {code:java}
> 00:23:55.093 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile) on project flink-hadoop-fs: Compilation failure: Compilation failure:
> 00:23:55.093 [ERROR] /home/vsts/work/1/s/flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopDataInputStream.java:[151,63] cannot find symbol
> 00:23:55.094 [ERROR] symbol: variable READBYTEBUFFER
> 00:23:55.094 [ERROR] location: interface org.apache.hadoop.fs.StreamCapabilities
> 00:23:55.094 [ERROR] /home/vsts/work/1/s/flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopDataInputStream.java:[182,63] cannot find symbol
> 00:23:55.094 [ERROR] symbol: variable PREADBYTEBUFFER
> 00:23:55.094 [ERROR] location: interface org.apache.hadoop.fs.StreamCapabilities
> 00:23:55.094 [ERROR] /home/vsts/work/1/s/flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopDataInputStream.java:[183,43] incompatible types: long cannot be converted to org.apache.hadoop.io.ByteBufferPool
> 00:23:55.094 [ERROR] -> [Help 1] {code}
> * 1.20 compile_cron_hadoop313 https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=59012&view=logs&j=87489130-75dc-54e4-1f45-80c30aa367a3&t=73da6d75-f30d-5d5a-acbe-487a9dcff678&l=3630
--
This message was sent by Atlassian Jira
(v8.20.10#820010)