You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "Aljoscha Krettek (JIRA)" <ji...@apache.org> on 2018/08/10 14:41:00 UTC
[jira] [Updated] (BEAM-4798) IndexOutOfBoundsException when Flink
parallelism > 1
[ https://issues.apache.org/jira/browse/BEAM-4798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aljoscha Krettek updated BEAM-4798:
-----------------------------------
Fix Version/s: 2.7.0
> IndexOutOfBoundsException when Flink parallelism > 1
> ----------------------------------------------------
>
> Key: BEAM-4798
> URL: https://issues.apache.org/jira/browse/BEAM-4798
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Affects Versions: 2.4.0, 2.5.0
> Reporter: Alexey Romanenko
> Assignee: Alexey Romanenko
> Priority: Major
> Fix For: 2.7.0
>
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> Running job on Flink in streaming mode and get data from a Kafka topic with parallelism > 1 causes an exception:
> {noformat}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
> at java.util.ArrayList.rangeCheck(ArrayList.java:657)
> at java.util.ArrayList.get(ArrayList.java:433)
> at org.apache.beam.runners.flink.translation.wrappers.streaming.io.UnboundedSourceWrapper.run(UnboundedSourceWrapper.java:277)
> at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:87)
> at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:56)
> at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:99)
> at org.apache.flink.streaming.runtime.tasks.StoppableSourceStreamTask.run(StoppableSourceStreamTask.java:45)
> at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:306)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> It happens when number of Kafka topic partitions is less than value of parallelism (number of task slots).
> So, workaround for now can be to set parallelism <= number of topic partitions, thus if parallelism=2 then number_partitions >= 2
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)