You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Ori Popowski (Jira)" <ji...@apache.org> on 2020/07/19 09:04:00 UTC
[jira] [Updated] (FLINK-18637) Key group is not in KeyGroupRange
[ https://issues.apache.org/jira/browse/FLINK-18637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ori Popowski updated FLINK-18637:
---------------------------------
Description:
I'm getting this error when creating a savepoint. I've read in https://issues.apache.org/jira/browse/FLINK-16193 that it's caused by unstable hashcode or equals on the key, or improper use of reinterpretAsKeyedStream.
My key is a string and I don't use reinterpretAsKeyedStream.
{code:java}
senv
.addSource(source)
.flatMap(…)
.filterWith { case (metadata, _, _) => … }
.assignTimestampsAndWatermarks(new BoundedOutOfOrdernessTimestampExtractor(…))
.keyingBy { case (meta, _) => meta.toPath.toString }
.flatMap(new TruncateLargeSessions(config.sessionSizeLimit))
.keyingBy { case (meta, _) => meta.toPath.toString }
.window(EventTimeSessionWindows.withGap(Time.of(…)))
.process(new ProcessSession(sessionPlayback, config))
.addSink(sink){code}
was:
I'm getting this error when creating a savepoint. I've read in https://issues.apache.org/jira/browse/FLINK-16193 that it's caused by unstable hashcode or equals on the key, or improper use of reinterpretAsKeyedStream.
My key is a string and I don't use reinterpretAsKeyedStream.
{code:java}
senv
.addSource(source)
.flatMap(…)
.filterWith { case (metadata, _, _) => … }
.assignTimestampsAndWatermarks(new BoundedOutOfOrdernessTimestampExtractor(…))
.keyingBy { case (meta, _) => meta.toPath.toString }
.flatMap(new TruncateLargeSessions(config.sessionSizeLimit))
.keyingBy { case (meta, _) => meta.toPath.toString }
.window(EventTimeSessionWindows.withGap(Time.of(…)))
.process(new ProcessSession(sessionPlayback, config))
.addSink(sink){code}
> Key group is not in KeyGroupRange
> ---------------------------------
>
> Key: FLINK-18637
> URL: https://issues.apache.org/jira/browse/FLINK-18637
> Project: Flink
> Issue Type: Bug
> Environment: OS current user: yarn
> Current Hadoop/Kerberos user: hadoop
> JVM: Java HotSpot(TM) 64-Bit Server VM - Oracle Corporation - 1.8/25.141-b15
> Maximum heap size: 28960 MiBytes
> JAVA_HOME: /usr/java/jdk1.8.0_141/jre
> Hadoop version: 2.8.5-amzn-6
> JVM Options:
> -Xmx30360049728
> -Xms30360049728
> -XX:MaxDirectMemorySize=4429185024
> -XX:MaxMetaspaceSize=1073741824
> -XX:+UseG1GC
> -XX:+UnlockDiagnosticVMOptions
> -XX:+G1SummarizeConcMark
> -verbose:gc
> -XX:+PrintGCDetails
> -XX:+PrintGCDateStamps
> -XX:+UnlockCommercialFeatures
> -XX:+FlightRecorder
> -XX:+DebugNonSafepoints
> -XX:FlightRecorderOptions=defaultrecording=true,settings=/home/hadoop/heap.jfc,dumponexit=true,dumponexitpath=/var/lib/hadoop-yarn/recording.jfr,loglevel=info
> -Dlog.file=/var/log/hadoop-yarn/containers/application_1593935560662_0002/container_1593935560662_0002_01_000002/taskmanager.log
> -Dlog4j.configuration=file:./log4j.properties
> Program Arguments:
> -Dtaskmanager.memory.framework.off-heap.size=134217728b
> -Dtaskmanager.memory.network.max=1073741824b
> -Dtaskmanager.memory.network.min=1073741824b
> -Dtaskmanager.memory.framework.heap.size=134217728b
> -Dtaskmanager.memory.managed.size=23192823744b
> -Dtaskmanager.cpu.cores=7.0
> -Dtaskmanager.memory.task.heap.size=30225832000b
> -Dtaskmanager.memory.task.off-heap.size=3221225472b
> --configDir.
> -Djobmanager.rpc.address=ip-10-180-30-250.us-west-2.compute.internal-Dweb.port=0
> -Dweb.tmpdir=/tmp/flink-web-64f613cf-bf04-4a09-8c14-75c31b619574
> -Djobmanager.rpc.port=33739
> -Drest.address=ip-10-180-30-250.us-west-2.compute.internal
> Reporter: Ori Popowski
> Priority: Major
>
> I'm getting this error when creating a savepoint. I've read in https://issues.apache.org/jira/browse/FLINK-16193 that it's caused by unstable hashcode or equals on the key, or improper use of reinterpretAsKeyedStream.
>
> My key is a string and I don't use reinterpretAsKeyedStream.
>
> {code:java}
> senv
> .addSource(source)
> .flatMap(…)
> .filterWith { case (metadata, _, _) => … }
> .assignTimestampsAndWatermarks(new BoundedOutOfOrdernessTimestampExtractor(…))
> .keyingBy { case (meta, _) => meta.toPath.toString }
> .flatMap(new TruncateLargeSessions(config.sessionSizeLimit))
> .keyingBy { case (meta, _) => meta.toPath.toString }
> .window(EventTimeSessionWindows.withGap(Time.of(…)))
> .process(new ProcessSession(sessionPlayback, config))
> .addSink(sink){code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)