[jira] [Commented] (BEAM-10375) HBaseIO.Read not deterministic coder exception when executed with flink 1.10.1

["Beam JIRA Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-10375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169011#comment-17169011 ] 

Beam JIRA Bot commented on BEAM-10375:
--------------------------------------

This issue is assigned but has not received an update in 30 days so it has been labeled "stale-assigned". If you are still working on the issue, please give an update and remove the label. If you are no longer working on the issue, please unassign so someone else may work on it. In 7 days the issue will be automatically unassigned.

> HBaseIO.Read not deterministic coder exception when executed with flink 1.10.1
> ------------------------------------------------------------------------------
>
>                 Key: BEAM-10375
>                 URL: https://issues.apache.org/jira/browse/BEAM-10375
>             Project: Beam
>          Issue Type: Bug
>          Components: io-java-hbase
>    Affects Versions: 2.21.0
>            Reporter: Richard Moorhead
>            Assignee: Ismaël Mejía
>            Priority: P2
>              Labels: stale-assigned
>
> Given that HBaseIO.readAll() is applied to a PCollection<HBaseIO.Read>, the following exception is thrown when readAll is executed by the flink runner:
> {code:java}
>  The program finished with the following exception:
> org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: the keyCoder of a GroupByKey must be deterministic
>         at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335)
>         at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
>         at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:138)
>         at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:662)
>         at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:210)
>         at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:893)
>         at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:966)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:422)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
>         at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
>         at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:966)
> Caused by: java.lang.IllegalStateException: the keyCoder of a GroupByKey must be deterministic
>         at org.apache.beam.sdk.transforms.GroupByKey.expand(GroupByKey.java:234)
>         at org.apache.beam.sdk.transforms.GroupByKey.expand(GroupByKey.java:110)
>         at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:542)
>         at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:476)
>         at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:355)
>         at org.apache.beam.sdk.transforms.Reshuffle.expand(Reshuffle.java:87)
>         at org.apache.beam.sdk.transforms.Reshuffle.expand(Reshuffle.java:49)
>         at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:542)
>         at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
>         at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:368)
>         at org.apache.beam.runners.core.construction.SplittableParDoNaiveBounded$SplittableProcessNaive.expand(SplittableParDoNaiveBounded.java:
> 110)
>         at org.apache.beam.runners.core.construction.SplittableParDoNaiveBounded$SplittableProcessNaive.expand(SplittableParDoNaiveBounded.java:
> 95)
>         at org.apache.beam.sdk.Pipeline.applyReplacement(Pipeline.java:569)
>         at org.apache.beam.sdk.Pipeline.replace(Pipeline.java:294)
>         at org.apache.beam.sdk.Pipeline.replaceAll(Pipeline.java:212)
>         at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.translate(FlinkPipelineExecutionEnvironment.java:114)
>         at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:107)
>         at org.apache.beam.sdk.Pipeline.run(Pipeline.java:317)
>         at org.apache.beam.sdk.Pipeline.run(Pipeline.java:303)
>         at com.cerner.pophealth.mpm.processing.beam.pipeline.bounded.EmpiPersonConversionPipeline.main(EmpiPersonConversionPipeline.java:96)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
>         ... 11 more
> Caused by: org.apache.beam.sdk.coders.Coder$NonDeterministicException: SerializableCoder(org.apache.beam.sdk.io.hbase.HBaseIO$Read) is not deter
> ministic because:
>         Java Serialization may be non-deterministic.
>         at org.apache.beam.sdk.coders.SerializableCoder.verifyDeterministic(SerializableCoder.java:205)
>         at org.apache.beam.sdk.transforms.GroupByKey.expand(GroupByKey.java:232) {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)