You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@beam.apache.org by George Dekermenjian <ge...@gmail.com> on 2024/02/24 08:24:10 UTC
Cross Language Runtime error python-Java
I have a python pipeline that uses the bigquery storage write method (cross
language with Java). I’m building launcher and worker docker images and
then launching the flex template. The launcher fails due to the following
runtime error in dataflow.
I’m using runner v2 and it is a streaming pipeline using the streaming
engine.
Any ideas of what is causing this?
RuntimeError:
org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.util.concurrent.UncheckedExecutionException:
org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.util.concurrent.UncheckedExecutionException:
java.lang.ClassCastException: class
org.apache.beam.sdk.transforms.windowing.Repeatedly cannot be cast to class
org.apache.beam.sdk.transforms.windowing.Trigger$OnceTrigger
(org.apache.beam.sdk.transforms.windowing.Repeatedly and
org.apache.beam.sdk.transforms.windowing.Trigger$OnceTrigger are in unnamed
module of loader 'app')"
line: "exec.go:66"
Re: Cross Language Runtime error python-Java
Posted by XQ Hu via user <us...@beam.apache.org>.
Great, you figured it out. Thanks for posting this back to the list.
On Sat, Feb 24, 2024 at 5:23 PM George Dekermenjian <ge...@gmail.com>
wrote:
> Adding the following to both Dockerfile.launcher and Dockerfile.worker did
> the trick for me.
>
>
>
> COPY --from=apache/beam_java11_sdk:latest /opt/apache/beam/jars
> /opt/apache/beam/jars COPY --from=apache/beam_java11_sdk:latest
> /opt/java/openjdk /opt/java/openjdk ENV JAVA_HOME=/opt/java/openjdk ENV
> PATH="${JAVA_HOME}/bin:${PATH}"
>
>
>
> On Sat, Feb 24, 2024 at 21:55 XQ Hu via user <us...@beam.apache.org> wrote:
>
>> Does your code work without the launcher? Better check this step by step
>> to figure out which part causes this error.
>>
>> On Sat, Feb 24, 2024 at 3:25 AM George Dekermenjian <ge...@gmail.com>
>> wrote:
>>
>>> I have a python pipeline that uses the bigquery storage write method
>>> (cross language with Java). I’m building launcher and worker docker images
>>> and then launching the flex template. The launcher fails due to the
>>> following runtime error in dataflow.
>>>
>>> I’m using runner v2 and it is a streaming pipeline using the streaming
>>> engine.
>>>
>>> Any ideas of what is causing this?
>>>
>>> RuntimeError:
>>> org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.util.concurrent.UncheckedExecutionException:
>>> org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.util.concurrent.UncheckedExecutionException:
>>> java.lang.ClassCastException: class
>>> org.apache.beam.sdk.transforms.windowing.Repeatedly cannot be cast to class
>>> org.apache.beam.sdk.transforms.windowing.Trigger$OnceTrigger
>>> (org.apache.beam.sdk.transforms.windowing.Repeatedly and
>>> org.apache.beam.sdk.transforms.windowing.Trigger$OnceTrigger are in unnamed
>>> module of loader 'app')"
>>> line: "exec.go:66"
>>>
>>>
Re: Cross Language Runtime error python-Java
Posted by George Dekermenjian <ge...@gmail.com>.
Adding the following to both Dockerfile.launcher and Dockerfile.worker did
the trick for me.
COPY --from=apache/beam_java11_sdk:latest /opt/apache/beam/jars
/opt/apache/beam/jars COPY --from=apache/beam_java11_sdk:latest
/opt/java/openjdk /opt/java/openjdk ENV JAVA_HOME=/opt/java/openjdk ENV
PATH="${JAVA_HOME}/bin:${PATH}"
On Sat, Feb 24, 2024 at 21:55 XQ Hu via user <us...@beam.apache.org> wrote:
> Does your code work without the launcher? Better check this step by step
> to figure out which part causes this error.
>
> On Sat, Feb 24, 2024 at 3:25 AM George Dekermenjian <ge...@gmail.com>
> wrote:
>
>> I have a python pipeline that uses the bigquery storage write method
>> (cross language with Java). I’m building launcher and worker docker images
>> and then launching the flex template. The launcher fails due to the
>> following runtime error in dataflow.
>>
>> I’m using runner v2 and it is a streaming pipeline using the streaming
>> engine.
>>
>> Any ideas of what is causing this?
>>
>> RuntimeError:
>> org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.util.concurrent.UncheckedExecutionException:
>> org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.util.concurrent.UncheckedExecutionException:
>> java.lang.ClassCastException: class
>> org.apache.beam.sdk.transforms.windowing.Repeatedly cannot be cast to class
>> org.apache.beam.sdk.transforms.windowing.Trigger$OnceTrigger
>> (org.apache.beam.sdk.transforms.windowing.Repeatedly and
>> org.apache.beam.sdk.transforms.windowing.Trigger$OnceTrigger are in unnamed
>> module of loader 'app')"
>> line: "exec.go:66"
>>
>>
Re: Cross Language Runtime error python-Java
Posted by XQ Hu via user <us...@beam.apache.org>.
Does your code work without the launcher? Better check this step by step to
figure out which part causes this error.
On Sat, Feb 24, 2024 at 3:25 AM George Dekermenjian <ge...@gmail.com>
wrote:
> I have a python pipeline that uses the bigquery storage write method
> (cross language with Java). I’m building launcher and worker docker images
> and then launching the flex template. The launcher fails due to the
> following runtime error in dataflow.
>
> I’m using runner v2 and it is a streaming pipeline using the streaming
> engine.
>
> Any ideas of what is causing this?
>
> RuntimeError:
> org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.util.concurrent.UncheckedExecutionException:
> org.apache.beam.vendor.guava.v32_1_2_jre.com.google.common.util.concurrent.UncheckedExecutionException:
> java.lang.ClassCastException: class
> org.apache.beam.sdk.transforms.windowing.Repeatedly cannot be cast to class
> org.apache.beam.sdk.transforms.windowing.Trigger$OnceTrigger
> (org.apache.beam.sdk.transforms.windowing.Repeatedly and
> org.apache.beam.sdk.transforms.windowing.Trigger$OnceTrigger are in unnamed
> module of loader 'app')"
> line: "exec.go:66"
>
>