You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@beam.apache.org by Alexey Romanenko <ar...@gmail.com> on 2023/04/18 15:20:20 UTC
Re: ClassCastException when I have bytes type in row
Hi Jeff,
Sorry for delay with an answer.
Could you give more details (e.g. your pipeline and sql query code snippets) about how to reproduce this issue?
Which Beam version do you use?
—
Alexey
> On 30 Mar 2023, at 03:21, Jeff Zhang <zj...@gmail.com> wrote:
>
>
> Hi, folks,
>
> I asked this question in the beam slack channel, but there seems to be no comments, so I would like to ask it in the user mail list again. Here's the problem I hit, I use SqlTransform to process my data, this following ClassCastException happens when my data has bytes type. It seems like a bug to me. Can anyone help confirm that? Thanks
>
>
> Caused by: java.lang.ClassCastException: org.apache.beam.vendor.calcite.v1_28_0.org <http://v1_28_0.org/>.apache.calcite.avatica.util.ByteString cannot be cast to [B
> at SC.eval0(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.codehaus.janino.ScriptEvaluator.evaluate(ScriptEvaluator.java:798)
> at org.codehaus.janino.ScriptEvaluator.evaluate(ScriptEvaluator.java:790)
> at org.apache.beam.sdk.extensions.sql.impl.rel.BeamCalcRel$CalcFn.processElement(BeamCalcRel.java:316)
> at org.apache.beam.sdk.extensions.sql.impl.rel.BeamCalcRel$CalcFn$DoFnInvoker.invokeProcessElement(Unknown Source)
> at org.apache.beam.repackaged.direct_java.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:211)
> at org.apache.beam.repackaged.direct_java.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188)
> at org.apache.beam.repackaged.direct_java.runners.core.SimplePushbackSideInputDoFnRunner.processElementInReadyWindows(SimplePushbackSideInputDoFnRunner.java:79)
> at org.apache.beam.runners.direct.ParDoEvaluator.processElement(ParDoEvaluator.java:244)
> at org.apache.beam.runners.direct.DoFnLifecycleManagerRemovingTransformEvaluator.processElement(DoFnLifecycleManagerRemovingTransformEvaluator.java:54)
> at org.apache.beam.runners.direct.DirectTransformExecutor.processElements(DirectTransformExecutor.java:165)
> at org.apache.beam.runners.direct.DirectTransformExecutor.run(DirectTransformExecutor.java:129) (edited)
>
>
>
> --
> Best Regards
>
> Jeff Zhang
Re: ClassCastException when I have bytes type in row
Posted by Jeff Zhang <zj...@gmail.com>.
Hi Alexey,
I am using Beam 2.46.0, and this issue happens in both direct-runner and
flink-runner (haven't tried other runners).
Here's the gist to reproduce this issue. (I can not share the full piece of
code, but the gist should contain enough information).
This issue only happens when I join this kafka source with one side input
source (created from memory). No such issue when I only query the kafka
source. And also no such issue when I join another table with this side
input source.
So I suspect it is highly related to my kafka topic data, but I am sorry I
can not share too much about our data. What I can share is that the topic
schema is a nested structure, 2 sub rows inside another row.
https://gist.github.com/zjffdu/76bf394b4eae9b29eb2352dd068a1feb
On Tue, Apr 18, 2023 at 11:20 PM Alexey Romanenko <ar...@gmail.com>
wrote:
> Hi Jeff,
>
> Sorry for delay with an answer.
>
> Could you give more details (e.g. your pipeline and sql query code
> snippets) about how to reproduce this issue?
> Which Beam version do you use?
>
> —
> Alexey
>
> On 30 Mar 2023, at 03:21, Jeff Zhang <zj...@gmail.com> wrote:
>
>
> Hi, folks,
>
> I asked this question in the beam slack channel, but there seems to be no
> comments, so I would like to ask it in the user mail list again. Here's the
> problem I hit, I use SqlTransform to process my data, this following
> ClassCastException happens when my data has bytes type. It seems like a bug
> to me. Can anyone help confirm that? Thanks
>
>
> Caused by: java.lang.ClassCastException: org.apache.beam.vendor.calcite.
> v1_28_0.org.apache.calcite.avatica.util.ByteString cannot be cast to [B
> at SC.eval0(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.codehaus.janino.ScriptEvaluator.evaluate(ScriptEvaluator.java:798)
> at org.codehaus.janino.ScriptEvaluator.evaluate(ScriptEvaluator.java:790)
> at
> org.apache.beam.sdk.extensions.sql.impl.rel.BeamCalcRel$CalcFn.processElement(BeamCalcRel.java:316)
> at
> org.apache.beam.sdk.extensions.sql.impl.rel.BeamCalcRel$CalcFn$DoFnInvoker.invokeProcessElement(Unknown
> Source)
> at
> org.apache.beam.repackaged.direct_java.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:211)
> at
> org.apache.beam.repackaged.direct_java.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188)
> at
> org.apache.beam.repackaged.direct_java.runners.core.SimplePushbackSideInputDoFnRunner.processElementInReadyWindows(SimplePushbackSideInputDoFnRunner.java:79)
> at
> org.apache.beam.runners.direct.ParDoEvaluator.processElement(ParDoEvaluator.java:244)
> at
> org.apache.beam.runners.direct.DoFnLifecycleManagerRemovingTransformEvaluator.processElement(DoFnLifecycleManagerRemovingTransformEvaluator.java:54)
> at
> org.apache.beam.runners.direct.DirectTransformExecutor.processElements(DirectTransformExecutor.java:165)
> at
> org.apache.beam.runners.direct.DirectTransformExecutor.run(DirectTransformExecutor.java:129)
> (edited)
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>
>
>
--
Best Regards
Jeff Zhang