You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Jacek Laskowski <ja...@japila.pl> on 2020/05/23 17:29:27 UTC
How to build Beam locally (with just the bare minimum for Java SDK
and runners)?
Hi,
I've been wondering how the devs build Beam locally. I'm new to gradle and
can't seem to find it in the docs [1].
I thought the following would be enough, but turns out it executes go-,
website- and python-related tasks among others.
./gradlew build -x test
I thought about disabling the tasks I don't want and ended up with the
following:
./gradlew build -x test -x check -x website -x docker -x
:sdks:python:setupVirtualenv
but that still builds other projects I don't need and so I doubt if -x's is
the way to go. There must be a more clever approach. What's that? Please
help. Thanks.
[1] https://beam.apache.org/contribute/
Pozdrawiam,
Jacek Laskowski
----
https://about.me/JacekLaskowski
"The Internals Of" Online Books <https://books.japila.pl/>
Follow me on https://twitter.com/jaceklaskowski
<https://twitter.com/jaceklaskowski>
Re: How to build Beam locally (with just the bare minimum for Java
SDK and runners)?
Posted by Nathan Fisher <nf...@junctionbox.ca>.
There’s a comment on that wiki page that I was going to ask about.
Specifically it references the maven-publish plugin [1]. Is the team
amenable to adding it to the parent build file? It makes it a lot easier to
incorporate locally built beam artefacts into a local maven project.
Obviously not globally reproducible but useful for people wanting to submit
changes to beam and then test them in their own project.
1 -
https://docs.gradle.org/current/userguide/publishing_maven.html#publishing_maven:tasks
On Sat, May 23, 2020 at 15:17, Thomas Weise <th...@apache.org> wrote:
> Hi Jacek,
>
> Most of the developer documentation can be found on the cwiki, in this case
>
> https://cwiki.apache.org/confluence/display/BEAM/Gradle+Tips
>
> To build just a runner, for example, Flink:
>
> ./gradlew :runners:flink:1.10:job-server:runShadow
>
>
> Thomas
>
> On Sat, May 23, 2020 at 10:29 AM Jacek Laskowski <ja...@japila.pl> wrote:
>
>> Hi,
>>
>> I've been wondering how the devs build Beam locally. I'm new to gradle
>> and can't seem to find it in the docs [1].
>>
>> I thought the following would be enough, but turns out it executes go-,
>> website- and python-related tasks among others.
>>
>> ./gradlew build -x test
>>
>> I thought about disabling the tasks I don't want and ended up with the
>> following:
>>
>> ./gradlew build -x test -x check -x website -x docker -x
>> :sdks:python:setupVirtualenv
>>
>> but that still builds other projects I don't need and so I doubt if -x's
>> is the way to go. There must be a more clever approach. What's that? Please
>> help. Thanks.
>>
>> [1] https://beam.apache.org/contribute/
>>
>> Pozdrawiam,
>> Jacek Laskowski
>> ----
>> https://about.me/JacekLaskowski
>> "The Internals Of" Online Books <https://books.japila.pl/>
>> Follow me on https://twitter.com/jaceklaskowski
>>
>> <https://twitter.com/jaceklaskowski>
>>
> --
Nathan Fisher
w: http://junctionbox.ca/
Re: How to build Beam locally (with just the bare minimum for Java
SDK and runners)?
Posted by Jean-Baptiste Onofre <jb...@nanthrax.net>.
Hi,
Personally I’m use:
./gradlew :sdks:java:build …
And ./gradlew projects gives you the name of the module.
Regards
JB
> Le 24 mai 2020 à 11:32, Jacek Laskowski <ja...@japila.pl> a écrit :
>
> Hi Thomas,
>
> > Most of the developer documentation can be found on the cwiki, in this case
>
> That's the purpose of the email as I couldn't find the answer.
>
> My question is how to build Beam locally with just the bare minimum for Java SDK and the runners? After reading the aforementioned page I ended up with the following:
>
> ./gradlew -p sdks/java build -x test
>
> The reason I've been asking about it was that I hoped it would fix the issue(s) with IDEA (thinking that the issue could be with auto-generated files). I'm still struggling with importing the sources in IDEA and have to manually remove 2.20.0-SNAPSHOT directory-based deps and add the sdks/java module.
>
> Thanks for your help Thomas. Danke schon!
>
> Pozdrawiam,
> Jacek Laskowski
> ----
> https://about.me/JacekLaskowski <https://about.me/JacekLaskowski>
> "The Internals Of" Online Books <https://books.japila.pl/>
> Follow me on https://twitter.com/jaceklaskowski <https://twitter.com/jaceklaskowski>
>
> <https://twitter.com/jaceklaskowski>
>
> On Sat, May 23, 2020 at 8:17 PM Thomas Weise <thw@apache.org <ma...@apache.org>> wrote:
> Hi Jacek,
>
> Most of the developer documentation can be found on the cwiki, in this case
>
> https://cwiki.apache.org/confluence/display/BEAM/Gradle+Tips <https://cwiki.apache.org/confluence/display/BEAM/Gradle+Tips>
>
> To build just a runner, for example, Flink:
>
> ./gradlew :runners:flink:1.10:job-server:runShadow
>
> Thomas
>
> On Sat, May 23, 2020 at 10:29 AM Jacek Laskowski <jacek@japila.pl <ma...@japila.pl>> wrote:
> Hi,
>
> I've been wondering how the devs build Beam locally. I'm new to gradle and can't seem to find it in the docs [1].
>
> I thought the following would be enough, but turns out it executes go-, website- and python-related tasks among others.
>
> ./gradlew build -x test
>
> I thought about disabling the tasks I don't want and ended up with the following:
>
> ./gradlew build -x test -x check -x website -x docker -x :sdks:python:setupVirtualenv
>
> but that still builds other projects I don't need and so I doubt if -x's is the way to go. There must be a more clever approach. What's that? Please help. Thanks.
>
> [1] https://beam.apache.org/contribute/ <https://beam.apache.org/contribute/>
>
> Pozdrawiam,
> Jacek Laskowski
> ----
> https://about.me/JacekLaskowski <https://about.me/JacekLaskowski>
> "The Internals Of" Online Books <https://books.japila.pl/>
> Follow me on https://twitter.com/jaceklaskowski <https://twitter.com/jaceklaskowski>
>
> <https://twitter.com/jaceklaskowski>
Re: How to build Beam locally (with just the bare minimum for Java
SDK and runners)?
Posted by Jacek Laskowski <ja...@japila.pl>.
Hi Thomas,
> Most of the developer documentation can be found on the cwiki, in this
case
That's the purpose of the email as I couldn't find the answer.
My question is how to build Beam locally with just the bare minimum for
Java SDK and the runners? After reading the aforementioned page I ended up
with the following:
./gradlew -p sdks/java build -x test
The reason I've been asking about it was that I hoped it would fix the
issue(s) with IDEA (thinking that the issue could be with auto-generated
files). I'm still struggling with importing the sources in IDEA and have to
manually remove 2.20.0-SNAPSHOT directory-based deps and add the sdks/java
module.
Thanks for your help Thomas. Danke schon!
Pozdrawiam,
Jacek Laskowski
----
https://about.me/JacekLaskowski
"The Internals Of" Online Books <https://books.japila.pl/>
Follow me on https://twitter.com/jaceklaskowski
<https://twitter.com/jaceklaskowski>
On Sat, May 23, 2020 at 8:17 PM Thomas Weise <th...@apache.org> wrote:
> Hi Jacek,
>
> Most of the developer documentation can be found on the cwiki, in this case
>
> https://cwiki.apache.org/confluence/display/BEAM/Gradle+Tips
>
> To build just a runner, for example, Flink:
>
> ./gradlew :runners:flink:1.10:job-server:runShadow
>
> Thomas
>
> On Sat, May 23, 2020 at 10:29 AM Jacek Laskowski <ja...@japila.pl> wrote:
>
>> Hi,
>>
>> I've been wondering how the devs build Beam locally. I'm new to gradle
>> and can't seem to find it in the docs [1].
>>
>> I thought the following would be enough, but turns out it executes go-,
>> website- and python-related tasks among others.
>>
>> ./gradlew build -x test
>>
>> I thought about disabling the tasks I don't want and ended up with the
>> following:
>>
>> ./gradlew build -x test -x check -x website -x docker -x
>> :sdks:python:setupVirtualenv
>>
>> but that still builds other projects I don't need and so I doubt if -x's
>> is the way to go. There must be a more clever approach. What's that? Please
>> help. Thanks.
>>
>> [1] https://beam.apache.org/contribute/
>>
>> Pozdrawiam,
>> Jacek Laskowski
>> ----
>> https://about.me/JacekLaskowski
>> "The Internals Of" Online Books <https://books.japila.pl/>
>> Follow me on https://twitter.com/jaceklaskowski
>>
>> <https://twitter.com/jaceklaskowski>
>>
>
Re: How to build Beam locally (with just the bare minimum for Java
SDK and runners)?
Posted by Thomas Weise <th...@apache.org>.
Hi Jacek,
Most of the developer documentation can be found on the cwiki, in this case
https://cwiki.apache.org/confluence/display/BEAM/Gradle+Tips
To build just a runner, for example, Flink:
./gradlew :runners:flink:1.10:job-server:runShadow
Thomas
On Sat, May 23, 2020 at 10:29 AM Jacek Laskowski <ja...@japila.pl> wrote:
> Hi,
>
> I've been wondering how the devs build Beam locally. I'm new to gradle and
> can't seem to find it in the docs [1].
>
> I thought the following would be enough, but turns out it executes go-,
> website- and python-related tasks among others.
>
> ./gradlew build -x test
>
> I thought about disabling the tasks I don't want and ended up with the
> following:
>
> ./gradlew build -x test -x check -x website -x docker -x
> :sdks:python:setupVirtualenv
>
> but that still builds other projects I don't need and so I doubt if -x's
> is the way to go. There must be a more clever approach. What's that? Please
> help. Thanks.
>
> [1] https://beam.apache.org/contribute/
>
> Pozdrawiam,
> Jacek Laskowski
> ----
> https://about.me/JacekLaskowski
> "The Internals Of" Online Books <https://books.japila.pl/>
> Follow me on https://twitter.com/jaceklaskowski
>
> <https://twitter.com/jaceklaskowski>
>