You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Sergey Nuyanzin (Jira)" <ji...@apache.org> on 2023/11/05 14:37:00 UTC
[jira] [Reopened] (FLINK-33331) Upgrade netty to 4.1.93+
[ https://issues.apache.org/jira/browse/FLINK-33331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sergey Nuyanzin reopened FLINK-33331:
-------------------------------------
Sorry, probably I wasn't clear enough
Need to reopen this because to fix we need to update Netty in Flink main repo as well (for some reason flink-python doesn't use the version from flink-shaded). Also since flink-python uses arrow depending on netty, arrow also should be updated.
Without this updates the error in description still appears
I will submit a PR for Flink main repo
> Upgrade netty to 4.1.93+
> ------------------------
>
> Key: FLINK-33331
> URL: https://issues.apache.org/jira/browse/FLINK-33331
> Project: Flink
> Issue Type: Sub-task
> Components: API / Python
> Reporter: Sergey Nuyanzin
> Assignee: Martijn Visser
> Priority: Major
> Labels: pull-request-available
> Fix For: shaded-18.0
>
>
> A number of tests fails like e.g.for
> {{org.apache.flink.table.runtime.operators.python.scalar.arrow.ArrowPythonScalarFunctionOpe}}
> {noformat}
> [ERROR] ratorTest.testFinishBundleTriggeredByTime Time elapsed: 0.031 s <<< ERROR!
> java.lang.NoClassDefFoundError: Could not initialize class org.apache.flink.table.runtime.arrow.serializers.ArrowSerializer
> at org.apache.flink.table.runtime.operators.python.scalar.arrow.ArrowPythonScalarFunctionOperator.open(ArrowPythonScalarFunctionOperator.java:72)
> at org.apache.flink.streaming.util.AbstractStreamOperatorTestHarness.open(AbstractStreamOperatorTestHarness.java:681)
> at org.apache.flink.table.runtime.operators.python.scalar.PythonScalarFunctionOperatorTestBase.testFinishBundleTriggeredByTime(PythonScalarFunctionOperatorTestBase.java:156)
> at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
> ...
> Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.RuntimeException: Arrow depends on DirectByteBuffer.<init>(long, int) which is not available. Please set the system property 'io.netty.tryReflectionSetAccessible' to 'true'. [in thread "ForkJoinPool-3-worker-1"]
> at org.apache.flink.table.runtime.arrow.ArrowUtils.checkArrowUsable(ArrowUtils.java:184)
> at org.apache.flink.table.runtime.arrow.serializers.ArrowSerializer.<clinit>(ArrowSerializer.java:44)
> at org.apache.flink.table.runtime.utils.PassThroughPythonAggregateFunctionRunner.<init>(PassThroughPythonAggregateFunctionRunner.java:96)
> at org.apache.flink.table.runtime.operators.python.aggregate.arrow.batch.BatchArrowPythonGroupWindowAggregateFunctionOperatorTest$PassThroughBatchArrowPythonGroupWindowAggregateFunctionOperator.createPythonFunctionRunner(BatchArrowPythonGroupWindowAggregateFunctionOperatorTest.java:414)
> at org.apache.flink.streaming.api.operators.python.process.AbstractExternalPythonFunctionOperator.open(AbstractExternalPythonFunctionOperator.java:56)
> at org.apache.flink.table.runtime.operators.python.AbstractStatelessFunctionOperator.open(AbstractStatelessFunctionOperator.java:92)
> at org.apache.flink.table.runtime.operators.python.aggregate.arrow.AbstractArrowPythonAggregateFunctionOperator.open(AbstractArrowPythonAggregateFunctionOperator.java:89)
> at org.apache.flink.table.runtime.operators.python.aggregate.arrow.batch.AbstractBatchArrowPythonAggregateFunctionOperator.open(AbstractBatchArrowPythonAggregateFunctionOperator.java:82)
> at org.apache.flink.table.runtime.operators.python.aggregate.arrow.batch.BatchArrowPythonGroupWindowAggregateFunctionOperator.open(BatchArrowPythonGroupWindowAggregateFunctionOperator.java:119)
> at org.apache.flink.streaming.util.AbstractStreamOperatorTestHarness.open(AbstractStreamOperatorTestHarness.java:681)
> at org.apache.flink.table.runtime.operators.python.aggregate.arrow.batch.BatchArrowPythonGroupWindowAggregateFunctionOperatorTest.testFinishBundleTriggeredByCount(BatchArrowPythonGroupWindowAggregateFunctionOperatorTest.java:140)
> ... 57 more
> {noformat}
> UPDATE
> The reason is that since JDK21 there was removed this constructor within https://bugs.openjdk.org/browse/JDK-8303083
> and corresponding changes in Netty are done at https://github.com/netty/netty/pull/13366 which is a part of 4.1.93.Final
--
This message was sent by Atlassian Jira
(v8.20.10#820010)