You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@beam.apache.org by Talat Uyarer <tu...@paloaltonetworks.com> on 2020/06/12 23:29:38 UTC

Building Dataflow Worker

Hi,

I want to build the dataflow worker on apache beram 2.19. However I faced a
grpc issue. I did not change anything. Just checked release-2.19.0 branch
and run build command. Could you help me understand why it does not build.
[1]

Additional information, Based on my limited knowledge Looks like it is
looking for a class which is coming grpc 1.26 version. But beam 2.19
version is using grpc 1.21

You can find build output below.

Thanks
[1] http://dpaste.com/15X7429

Re: Building Dataflow Worker

Posted by Steve Niemitz <sn...@apache.org>.

I think you want the "legacy-worker" target instead:

./gradlew -Ppublishing -PnoSigning
:runners:google-cloud-dataflow-java:worker:legacy-worker:shadowJar

That's what I've always used at least.

On Mon, Jun 15, 2020 at 4:57 PM Luke Cwik <lc...@google.com> wrote:

> I noticed that you are not using the gradle wrapper but your own installed
> version. Apache Beam 2.19 is using gradle 5.2.1, is the installed version
> compatible with that?
>
> Try
> ./gradlew :runners:google-cloud-dataflow-java:worker:shadowJar
> in a clean workspace.
>
> On Fri, Jun 12, 2020 at 4:30 PM Talat Uyarer <tu...@paloaltonetworks.com>
> wrote:
>
>> Hi,
>>
>> I want to build the dataflow worker on apache beram 2.19. However I faced
>> a grpc issue. I did not change anything. Just checked release-2.19.0 branch
>> and run build command. Could you help me understand why it does not build.
>> [1]
>>
>> Additional information, Based on my limited knowledge Looks like it is
>> looking for a class which is coming grpc 1.26 version. But beam 2.19
>> version is using grpc 1.21
>>
>> You can find build output below.
>>
>> Thanks
>> [1] http://dpaste.com/15X7429
>>
>

Re: Building Dataflow Worker

Posted by Kyle Weaver <kc...@google.com>.

> Looks like there is an issue with Gradle Errorprone's plugin.

I saw the same error earlier today:
https://jira.apache.org/jira/browse/BEAM-10263

I haven't done a full investigation yet, but it seems this broke the build
for several (?) past Beam source releases. I'm not sure what Beam's policy
is about patching existing source releases, but in any case it is fixed in
2.22.0 and the master branch.

On Mon, Jun 15, 2020 at 4:04 PM Talat Uyarer <tu...@paloaltonetworks.com>
wrote:

> Thank you Luke and Steve for your help.
>
> Looks like there is an issue with Gradle Errorprone's plugin. I had to
> modify BeamModulePlugin.groovy [1], Without modification when i tried to
> build it I got this error[2]. Should I create a patch for it ?
>
> Last question. We are using streaming engine. Which module should we build
> to replace the dataflow worker jar file ?
>
> Thanks
>
> [1] http://dpaste.com/238QH9Z
> [2] http://dpaste.com/01J55J0
>
> On Mon, Jun 15, 2020 at 1:57 PM Luke Cwik <lc...@google.com> wrote:
>
>> I noticed that you are not using the gradle wrapper but your own
>> installed version. Apache Beam 2.19 is using gradle 5.2.1, is the installed
>> version compatible with that?
>>
>> Try
>> ./gradlew :runners:google-cloud-dataflow-java:worker:shadowJar
>> in a clean workspace.
>>
>> On Fri, Jun 12, 2020 at 4:30 PM Talat Uyarer <
>> tuyarer@paloaltonetworks.com> wrote:
>>
>>> Hi,
>>>
>>> I want to build the dataflow worker on apache beram 2.19. However I
>>> faced a grpc issue. I did not change anything. Just checked release-2.19.0
>>> branch and run build command. Could you help me understand why it does not
>>> build. [1]
>>>
>>> Additional information, Based on my limited knowledge Looks like it is
>>> looking for a class which is coming grpc 1.26 version. But beam 2.19
>>> version is using grpc 1.21
>>>
>>> You can find build output below.
>>>
>>> Thanks
>>> [1] http://dpaste.com/15X7429
>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__dpaste.com_15X7429&d=DwMFaQ&c=V9IgWpI5PvzTw83UyHGVSoW3Uc1MFWe5J8PTfkrzVSo&r=BkW1L6EF7ergAVYDXCo-3Vwkpy6qjsWAz7_GD7pAR8g&m=zDPYSCF59Q6opRD-3y98savwq8rZ5jCKso2wW2SE9l4&s=yuJLSEbioHDZlLC2x-Lazdg-4Q5CZPurmIRX5WqvYHg&e=>
>>>
>>

Re: Building Dataflow Worker

Posted by Talat Uyarer <tu...@paloaltonetworks.com>.

Thank you Luke and Steve for your help.

Looks like there is an issue with Gradle Errorprone's plugin. I had to
modify BeamModulePlugin.groovy [1], Without modification when i tried to
build it I got this error[2]. Should I create a patch for it ?

Last question. We are using streaming engine. Which module should we build
to replace the dataflow worker jar file ?

Thanks

[1] http://dpaste.com/238QH9Z
[2] http://dpaste.com/01J55J0

On Mon, Jun 15, 2020 at 1:57 PM Luke Cwik <lc...@google.com> wrote:

> I noticed that you are not using the gradle wrapper but your own installed
> version. Apache Beam 2.19 is using gradle 5.2.1, is the installed version
> compatible with that?
>
> Try
> ./gradlew :runners:google-cloud-dataflow-java:worker:shadowJar
> in a clean workspace.
>
> On Fri, Jun 12, 2020 at 4:30 PM Talat Uyarer <tu...@paloaltonetworks.com>
> wrote:
>
>> Hi,
>>
>> I want to build the dataflow worker on apache beram 2.19. However I faced
>> a grpc issue. I did not change anything. Just checked release-2.19.0 branch
>> and run build command. Could you help me understand why it does not build.
>> [1]
>>
>> Additional information, Based on my limited knowledge Looks like it is
>> looking for a class which is coming grpc 1.26 version. But beam 2.19
>> version is using grpc 1.21
>>
>> You can find build output below.
>>
>> Thanks
>> [1] http://dpaste.com/15X7429
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__dpaste.com_15X7429&d=DwMFaQ&c=V9IgWpI5PvzTw83UyHGVSoW3Uc1MFWe5J8PTfkrzVSo&r=BkW1L6EF7ergAVYDXCo-3Vwkpy6qjsWAz7_GD7pAR8g&m=zDPYSCF59Q6opRD-3y98savwq8rZ5jCKso2wW2SE9l4&s=yuJLSEbioHDZlLC2x-Lazdg-4Q5CZPurmIRX5WqvYHg&e=>
>>
>

Re: Building Dataflow Worker

Posted by Luke Cwik <lc...@google.com>.

I noticed that you are not using the gradle wrapper but your own installed
version. Apache Beam 2.19 is using gradle 5.2.1, is the installed version
compatible with that?

Try
./gradlew :runners:google-cloud-dataflow-java:worker:shadowJar
in a clean workspace.

On Fri, Jun 12, 2020 at 4:30 PM Talat Uyarer <tu...@paloaltonetworks.com>
wrote:

> Hi,
>
> I want to build the dataflow worker on apache beram 2.19. However I faced
> a grpc issue. I did not change anything. Just checked release-2.19.0 branch
> and run build command. Could you help me understand why it does not build.
> [1]
>
> Additional information, Based on my limited knowledge Looks like it is
> looking for a class which is coming grpc 1.26 version. But beam 2.19
> version is using grpc 1.21
>
> You can find build output below.
>
> Thanks
> [1] http://dpaste.com/15X7429
>