You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@beam.apache.org by "Kumbhare, Yogesh" <Yo...@intuit.com> on 2020/03/20 09:33:41 UTC

Issue in testing doing DirectRunner Apache Beam Local Machine

Hi Team ,

I am facing issue to run the simple project on Direct Runner Apache beam , to test the example locally in our machine .

I added in pom.xml dependency ,

<dependency>
   <groupId>org.apache.beam</groupId>
   <artifactId>beam-runners-direct-java</artifactId>
   <version>2.19.0</version>
   <scope>runtime</scope>
</dependency>



Please find the below error , let me know any update .


Exception in thread "main" java.lang.IncompatibleClassChangeError: Class org.apache.beam.model.pipeline.v1.RunnerApi$StandardPTransforms$Primitives does not implement the requested interface org.apache.beam.vendor.grpc.v1p21p0.com.google.protobuf.ProtocolMessageEnum

                at org.apache.beam.repackaged.direct_java.runners.core.construction.BeamUrns.getUrn(BeamUrns.java:27)

                at org.apache.beam.repackaged.direct_java.runners.core.construction.PTransformTranslation.<clinit>(PTransformTranslation.java:129)

                at org.apache.beam.repackaged.direct_java.runners.core.construction.PTransformMatchers.lambda$writeWithRunnerDeterminedSharding$1(PTransformMatchers.java:483)

                at org.apache.beam.sdk.Pipeline$2.enterCompositeTransform(Pipeline.java:268)

                at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:645)

                at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)

                at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:311)

                at org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:245)

                at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:458)

                at org.apache.beam.sdk.Pipeline.replace(Pipeline.java:258)

                at org.apache.beam.sdk.Pipeline.replaceAll(Pipeline.java:208)

                at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:170)

                at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:67)

                at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)

                at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)

                at com.intuit.dedupe.beam.poc.StreamingDedupe.main(StreamingDedupe.java:144)

Re: Issue in testing doing DirectRunner Apache Beam Local Machine

Posted by Tomo Suzuki <su...@google.com>.

Yi Yogesh,

It seems you have incompatible dependencies. Would you share your pom.xml?
I'm interested in "dependencies" and "dependencyManagement" sections.

Regards,
Tomo

On Fri, Mar 20, 2020 at 12:43 PM Austin Bennett <wh...@gmail.com>
wrote:

> Hi Yogesh,
>
> Many of the examples, use the direct runner:
>
> https://github.com/apache/beam/tree/master/examples/java/src/main/java/org/apache/beam/examples
>
> As does the word count example:
> https://beam.apache.org/get-started/quickstart-java/#get-the-wordcount-code
>
> I'm not well versed enough with our Java SDK (and dependencies for it), to
> recognize the error you shared.
>
> Good luck!
> Austin
>
>
> On Fri, Mar 20, 2020 at 9:25 AM Kumbhare, Yogesh <
> Yogesh_Kumbhare@intuit.com> wrote:
>
>> Hi Team,
>>
>>
>>
>> Please find the below command to build .
>>
>>
>>
>> mvn compile exec:java
>> -Dexec.mainClass=com.intuit.dedupe.beam.poc.StreamingDedupe -Pdirect-runner
>>
>>
>>
>> is there any DirectRunner example you have please do share with us.
>>
>>
>>
>>
>>
>> Thanks ,
>>
>> Yogesh K
>>
>> 8050397434
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *From: *"Kumbhare, Yogesh" <Yo...@intuit.com>
>> *Date: *Friday, 20 March 2020 at 3:03 PM
>> *To: *"dev@beam.apache.org" <de...@beam.apache.org>
>> *Subject: *Issue in testing doing DirectRunner Apache Beam Local Machine
>>
>>
>>
>> Hi Team ,
>>
>>
>>
>> I am facing issue to run the simple project on Direct Runner Apache beam
>> , to test the example locally in our machine .
>>
>>
>>
>> I added in pom.xml dependency ,
>>
>>
>>
>> <dependency>
>>
>>    <groupId>org.apache.beam</groupId>
>>
>>    <artifactId>beam-runners-direct-java</artifactId>
>>
>>    <version>2.19.0</version>
>>
>>    <scope>runtime</scope>
>>
>> </dependency>
>>
>>
>>
>>
>>
>>
>>
>> Please find the below error , let me know any update .
>>
>>
>>
>> Exception in thread "main" java.lang.IncompatibleClassChangeError: Class
>> org.apache.beam.model.pipeline.v1.RunnerApi$StandardPTransforms$Primitives
>> does not implement the requested interface
>> org.apache.beam.vendor.grpc.v1p21p0.com.google.protobuf.ProtocolMessageEnum
>>
>>                 at
>> org.apache.beam.repackaged.direct_java.runners.core.construction.BeamUrns.getUrn(
>> BeamUrns.java:27)
>>
>>                 at
>> org.apache.beam.repackaged.direct_java.runners.core.construction.PTransformTranslation.<clinit>(
>> PTransformTranslation.java:129)
>>
>>                 at
>> org.apache.beam.repackaged.direct_java.runners.core.construction.PTransformMatchers.lambda$writeWithRunnerDeterminedSharding$1(
>> PTransformMatchers.java:483)
>>
>>                 at
>> org.apache.beam.sdk.Pipeline$2.enterCompositeTransform(Pipeline.java:268)
>>
>>                 at
>> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(
>> TransformHierarchy.java:645)
>>
>>                 at
>> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(
>> TransformHierarchy.java:649)
>>
>>                 at
>> org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(
>> TransformHierarchy.java:311)
>>
>>                 at org.apache.beam.sdk.runners.TransformHierarchy.visit(
>> TransformHierarchy.java:245)
>>
>>                 at org.apache.beam.sdk.Pipeline.traverseTopologically(
>> Pipeline.java:458)
>>
>>                 at org.apache.beam.sdk.Pipeline.replace(Pipeline.java:258
>> )
>>
>>                 at org.apache.beam.sdk.Pipeline.replaceAll(
>> Pipeline.java:208)
>>
>>                 at org.apache.beam.runners.direct.DirectRunner.run(
>> DirectRunner.java:170)
>>
>>                 at org.apache.beam.runners.direct.DirectRunner.run(
>> DirectRunner.java:67)
>>
>>                 at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)
>>
>>                 at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)
>>
>>                 at com.intuit.dedupe.beam.poc.StreamingDedupe.main(
>> StreamingDedupe.java:144)
>>
>>
>>
>>
>>
>>
>>
>

-- 
Regards,
Tomo

Re: Issue in testing doing DirectRunner Apache Beam Local Machine

Posted by Austin Bennett <wh...@gmail.com>.

Hi Yogesh,

Many of the examples, use the direct runner:
https://github.com/apache/beam/tree/master/examples/java/src/main/java/org/apache/beam/examples

As does the word count example:
https://beam.apache.org/get-started/quickstart-java/#get-the-wordcount-code

I'm not well versed enough with our Java SDK (and dependencies for it), to
recognize the error you shared.

Good luck!
Austin


On Fri, Mar 20, 2020 at 9:25 AM Kumbhare, Yogesh <Yo...@intuit.com>
wrote:

> Hi Team,
>
>
>
> Please find the below command to build .
>
>
>
> mvn compile exec:java
> -Dexec.mainClass=com.intuit.dedupe.beam.poc.StreamingDedupe -Pdirect-runner
>
>
>
> is there any DirectRunner example you have please do share with us.
>
>
>
>
>
> Thanks ,
>
> Yogesh K
>
> 8050397434
>
>
>
>
>
>
>
>
>
>
>
> *From: *"Kumbhare, Yogesh" <Yo...@intuit.com>
> *Date: *Friday, 20 March 2020 at 3:03 PM
> *To: *"dev@beam.apache.org" <de...@beam.apache.org>
> *Subject: *Issue in testing doing DirectRunner Apache Beam Local Machine
>
>
>
> Hi Team ,
>
>
>
> I am facing issue to run the simple project on Direct Runner Apache beam ,
> to test the example locally in our machine .
>
>
>
> I added in pom.xml dependency ,
>
>
>
> <dependency>
>
>    <groupId>org.apache.beam</groupId>
>
>    <artifactId>beam-runners-direct-java</artifactId>
>
>    <version>2.19.0</version>
>
>    <scope>runtime</scope>
>
> </dependency>
>
>
>
>
>
>
>
> Please find the below error , let me know any update .
>
>
>
> Exception in thread "main" java.lang.IncompatibleClassChangeError: Class
> org.apache.beam.model.pipeline.v1.RunnerApi$StandardPTransforms$Primitives
> does not implement the requested interface
> org.apache.beam.vendor.grpc.v1p21p0.com.google.protobuf.ProtocolMessageEnum
>
>                 at
> org.apache.beam.repackaged.direct_java.runners.core.construction.BeamUrns.getUrn(
> BeamUrns.java:27)
>
>                 at
> org.apache.beam.repackaged.direct_java.runners.core.construction.PTransformTranslation.<clinit>(
> PTransformTranslation.java:129)
>
>                 at
> org.apache.beam.repackaged.direct_java.runners.core.construction.PTransformMatchers.lambda$writeWithRunnerDeterminedSharding$1(
> PTransformMatchers.java:483)
>
>                 at org.apache.beam.sdk.Pipeline$2.enterCompositeTransform(
> Pipeline.java:268)
>
>                 at
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(
> TransformHierarchy.java:645)
>
>                 at
> org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(
> TransformHierarchy.java:649)
>
>                 at
> org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(
> TransformHierarchy.java:311)
>
>                 at org.apache.beam.sdk.runners.TransformHierarchy.visit(
> TransformHierarchy.java:245)
>
>                 at org.apache.beam.sdk.Pipeline.traverseTopologically(
> Pipeline.java:458)
>
>                 at org.apache.beam.sdk.Pipeline.replace(Pipeline.java:258)
>
>                 at org.apache.beam.sdk.Pipeline.replaceAll(
> Pipeline.java:208)
>
>                 at org.apache.beam.runners.direct.DirectRunner.run(
> DirectRunner.java:170)
>
>                 at org.apache.beam.runners.direct.DirectRunner.run(
> DirectRunner.java:67)
>
>                 at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)
>
>                 at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)
>
>                 at com.intuit.dedupe.beam.poc.StreamingDedupe.main(
> StreamingDedupe.java:144)
>
>
>
>
>
>
>

Re: Issue in testing doing DirectRunner Apache Beam Local Machine

Posted by "Kumbhare, Yogesh" <Yo...@intuit.com>.

Hi Team,

Please find the below command to build .


mvn compile exec:java -Dexec.mainClass=com.intuit.dedupe.beam.poc.StreamingDedupe -Pdirect-runner



is there any DirectRunner example you have please do share with us.





Thanks ,

Yogesh K

8050397434






From: "Kumbhare, Yogesh" <Yo...@intuit.com>
Date: Friday, 20 March 2020 at 3:03 PM
To: "dev@beam.apache.org" <de...@beam.apache.org>
Subject: Issue in testing doing DirectRunner Apache Beam Local Machine

Hi Team ,

I am facing issue to run the simple project on Direct Runner Apache beam , to test the example locally in our machine .

I added in pom.xml dependency ,

<dependency>
   <groupId>org.apache.beam</groupId>
   <artifactId>beam-runners-direct-java</artifactId>
   <version>2.19.0</version>
   <scope>runtime</scope>
</dependency>



Please find the below error , let me know any update .


Exception in thread "main" java.lang.IncompatibleClassChangeError: Class org.apache.beam.model.pipeline.v1.RunnerApi$StandardPTransforms$Primitives does not implement the requested interface org.apache.beam.vendor.grpc.v1p21p0.com.google.protobuf.ProtocolMessageEnum

                at org.apache.beam.repackaged.direct_java.runners.core.construction.BeamUrns.getUrn(BeamUrns.java:27)

                at org.apache.beam.repackaged.direct_java.runners.core.construction.PTransformTranslation.<clinit>(PTransformTranslation.java:129)

                at org.apache.beam.repackaged.direct_java.runners.core.construction.PTransformMatchers.lambda$writeWithRunnerDeterminedSharding$1(PTransformMatchers.java:483)

                at org.apache.beam.sdk.Pipeline$2.enterCompositeTransform(Pipeline.java:268)

                at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:645)

                at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:649)

                at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:311)

                at org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:245)

                at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:458)

                at org.apache.beam.sdk.Pipeline.replace(Pipeline.java:258)

                at org.apache.beam.sdk.Pipeline.replaceAll(Pipeline.java:208)

                at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:170)

                at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:67)

                at org.apache.beam.sdk.Pipeline.run(Pipeline.java:313)

                at org.apache.beam.sdk.Pipeline.run(Pipeline.java:299)

                at com.intuit.dedupe.beam.poc.StreamingDedupe.main(StreamingDedupe.java:144)