You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Henning Schmiedehausen (Jira)" <ji...@apache.org> on 2024/03/07 01:24:00 UTC
[jira] [Comment Edited] (FLINK-34581) streaming code throws java.lang.reflect.InaccessibleObjectException
[ https://issues.apache.org/jira/browse/FLINK-34581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824225#comment-17824225 ]
Henning Schmiedehausen edited comment on FLINK-34581 at 3/7/24 1:23 AM:
------------------------------------------------------------------------
The code basically looks like this:
{{public class FlinkTester {}}
{{ @RegisterExtension}}
{{ public static MiniClusterExtension flinkCluster = new MiniClusterExtension(}}
{{ new MiniClusterResourceConfiguration.Builder()}}
{{ .setNumberSlotsPerTaskManager(1)}}
{{ .setNumberTaskManagers(1)}}
{{ .build());}}
{{ @Test}}
{{ public void runLocalTest() throws Exception {}}
{{ String[] args = new String[] {}}
{{ ... some args here ...}}
{{ };}}
{{{} FlinkJobMain job = new }}\{{{}FlinkJobMain{}}}{{{}(args);{}}}
{{ job.run();}}
}
{{}}}
and I run this in the IDE with "run unit test (which uses the JUnit 5 test runner).
was (Author: henning):
The code basically looks like this:
{{public class FlinkTester {}}
{{ @RegisterExtension}}
{{ public static MiniClusterExtension flinkCluster = new MiniClusterExtension(}}
{{ new MiniClusterResourceConfiguration.Builder()}}
{{ .setNumberSlotsPerTaskManager(1)}}
{{ .setNumberTaskManagers(1)}}
{{ .build());}}
{{ @Test}}
{{ public void runLocalTest() throws Exception {}}
{{ String[] args = new String[] {}}
{{ ... some args here ...}}
{{ };}}
{{{} FlinkJobMain job = new }}\{{{}FlinkJobMain{}}}{{{}(args);{}}}
{{ job.run();}}
{\{ }}}
{{}}}
and I run this in the IDE with "run unit test (which uses the JUnit 5 test runner).
> streaming code throws java.lang.reflect.InaccessibleObjectException
> -------------------------------------------------------------------
>
> Key: FLINK-34581
> URL: https://issues.apache.org/jira/browse/FLINK-34581
> Project: Flink
> Issue Type: Bug
> Components: API / DataStream
> Affects Versions: 1.18.1
> Reporter: Henning Schmiedehausen
> Priority: Blocker
>
> I have a pretty simple test pipeline (read a bunch of tables from Apache Kafka, join and project them, then write to Apache Iceberg) that I run locally with Junit5 and the MiniClusterExtension. This works ok with Java 11 and Flink 1.17.2 and Flink 1.18.1.
> With Java 17, I see
> {{[WARN ] IcebergStreamWriter (1/1)#1 (78d9251dbbab3aae84bf303dfc080d23_626c1e687bcaad0c13c507629a5894a8_0_1) switched from RUNNING to FAILED with failure cause:}}
> {{java.io.IOException: Could not perform checkpoint 2 for operator IcebergStreamWriter (1/1)#1.}}
> {{at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointOnBarrier(StreamTask.java:1275)}}
> {{at org.apache.flink.streaming.runtime.io.checkpointing.CheckpointBarrierHandler.notifyCheckpoint(CheckpointBarrierHandler.java:147)}}
> {{at org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.triggerCheckpoint(SingleCheckpointBarrierHandler.java:287)}}
> {{at org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.access$100(SingleCheckpointBarrierHandler.java:64)}}
> {{at org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler$ControllerImpl.triggerGlobalCheckpoint(SingleCheckpointBarrierHandler.java:488)}}
> {{at org.apache.flink.streaming.runtime.io.checkpointing.AbstractAlignedBarrierHandlerState.triggerGlobalCheckpoint(AbstractAlignedBarrierHandlerState.java:74)}}
> {{at org.apache.flink.streaming.runtime.io.checkpointing.AbstractAlignedBarrierHandlerState.barrierReceived(AbstractAlignedBarrierHandlerState.java:66)}}
> {{at org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.lambda$processBarrier$2(SingleCheckpointBarrierHandler.java:234)}}
> {{at org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.markCheckpointAlignedAndTransformState(SingleCheckpointBarrierHandler.java:262)}}
> {{at org.apache.flink.streaming.runtime.io.checkpointing.SingleCheckpointBarrierHandler.processBarrier(SingleCheckpointBarrierHandler.java:231)}}
> {{at org.apache.flink.streaming.runtime.io.checkpointing.CheckpointedInputGate.handleEvent(CheckpointedInputGate.java:181)}}
> {{at org.apache.flink.streaming.runtime.io.checkpointing.CheckpointedInputGate.pollNext(CheckpointedInputGate.java:159)}}
> {{at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:118)}}
> {{at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)}}
> {{at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:562)}}
> {{at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)}}
> {{at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:858)}}
> {{at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:807)}}
> {{at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:953)}}
> {{at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:932)}}
> {{at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:746)}}
> {{at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562)}}
> {{at java.base/java.lang.Thread.run(Thread.java:840)}}
> {{Caused by: java.lang.RuntimeException: java.lang.reflect.InaccessibleObjectException: Unable to make field private final java.lang.Object[] java.util.Arrays$ArrayList.a accessible: module java.base does not "opens java.util" to unnamed module @6af93788}}
> {{at com.twitter.chill.java.ArraysAsListSerializer.<init>(ArraysAsListSerializer.java:69)}}
> {{at org.apache.flink.api.java.typeutils.runtime.kryo.FlinkChillPackageRegistrar.registerSerializers(FlinkChillPackageRegistrar.java:67)}}
> {{at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.getKryoInstance(KryoSerializer.java:513)}}
> {{at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.checkKryoInitialized(KryoSerializer.java:522)}}
> {{at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.serialize(KryoSerializer.java:348)}}
> {{at org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.serialize(StreamElementSerializer.java:165)}}
> {{at org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.serialize(StreamElementSerializer.java:43)}}
> {{at org.apache.flink.runtime.plugable.SerializationDelegate.write(SerializationDelegate.java:54)}}
> {{at org.apache.flink.runtime.io.network.api.writer.RecordWriter.serializeRecord(RecordWriter.java:141)}}
> {{at org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:107)}}
> {{at org.apache.flink.runtime.io.network.api.writer.ChannelSelectorRecordWriter.emit(ChannelSelectorRecordWriter.java:55)}}
> {{at org.apache.flink.streaming.runtime.io.RecordWriterOutput.pushToRecordWriter(RecordWriterOutput.java:134)}}
> {{at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collectAndCheckIfChained(RecordWriterOutput.java:114)}}
> {{at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:95)}}
> {{at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:48)}}
> {{at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:59)}}
> {{at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:31)}}
> {{at org.apache.iceberg.flink.sink.IcebergStreamWriter.flush(IcebergStreamWriter.java:113)}}
> {{at org.apache.iceberg.flink.sink.IcebergStreamWriter.prepareSnapshotPreBarrier(IcebergStreamWriter.java:66)}}
> {{at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.prepareSnapshotPreBarrier(RegularOperatorChain.java:89)}}
> {{at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointState(SubtaskCheckpointCoordinatorImpl.java:322)}}
> {{at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$performCheckpoint$15(StreamTask.java:1318)}}
> {{at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)}}
> {{at org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:1306)}}
> {{at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointOnBarrier(StreamTask.java:1263)}}
> {{... 22 more}}
> {{Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field private final java.lang.Object[] java.util.Arrays$ArrayList.a accessible: module java.base does not "opens java.util" to unnamed module @6af93788}}
> {{at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)}}
> {{at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)}}
> {{at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178)}}
> {{at java.base/java.lang.reflect.Field.setAccessible(Field.java:172)}}
> {{at com.twitter.chill.java.ArraysAsListSerializer.<init>(ArraysAsListSerializer.java:67)}}
> {{... 46 more}}
> This is blocking us right now from upgrading to Java 17.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)