You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Dan Hill <qu...@gmail.com> on 2020/06/11 21:20:59 UTC
Does anyone have an example of Bazel working with Flink?
I took the Flink playground and I'm trying to swap out Maven for Bazel. I
got to the point where I'm hitting the following error. I want to diff my
code with an existing, working setup.
Thanks! - Dan
client_1 |
org.apache.flink.client.program.ProgramInvocationException:
Neither a 'Main-Class', nor a 'program-class' entry was found in the jar
file.
client_1 | at
org.apache.flink.client.program.PackagedProgram.getEntryPointClassNameFromJar(PackagedProgram.java:596)
client_1 | at
org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:190)
client_1 | at
org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:128)
client_1 | at
org.apache.flink.client.cli.CliFrontend.buildProgram(CliFrontend.java:862)
client_1 | at
org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:204)
client_1 | at
org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1050)
client_1 | at
org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1126)
client_1 | at
org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
client_1 | at
org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1126)
Re: Does anyone have an example of Bazel working with Flink?
Posted by Austin Cawley-Edwards <au...@gmail.com>.
Ok, no worries Aaron, that's still good advice :)
One last question - are you using JAR-based or image-based deployments? The
only real problem using Flink & Bazel and a JAR-based deployment from our
experience is removing the Flink libs present in the deploy environment
from the deploy jar, and still having them present when we want to do local
debugging/ integration testing. Also possible that we're just not using
Bazel entirely correctly!
Thank you!
Austin
On Thu, Jun 18, 2020 at 12:32 PM Aaron Levin <aa...@stripe.com> wrote:
> Hi Austin,
>
> In our experience, `rules_scala` and `rules_java` are enough for us at
> this point.
>
> It's entirely possible I'm not thinking far enough into the future,
> though, so don't take our lack of investment as a sign it's not worth
> investing in :)
>
> Best,
>
> Aaron Levin
>
> On Thu, Jun 18, 2020 at 10:27 AM Austin Cawley-Edwards <
> austin.cawley@gmail.com> wrote:
>
>> Great to hear Dan!
>>
>> @Aaron - would you/ your team be interested in a `rules_flink` project?
>> I'm still fairly new to Bazel and know enough to contribute, but could
>> definitely use guidance on design as well.
>>
>> Best,
>> Austin
>>
>> On Mon, Jun 15, 2020 at 11:07 PM Dan Hill <qu...@gmail.com> wrote:
>>
>>> Thanks for the replies! I was able to use the provided answers to get a
>>> setup working (maybe not the most efficiently). The main change I made was
>>> to switch to including the deploy jar in the image (rather than the default
>>> one).
>>>
>>> I'm open to contributing to a "rules_flink" project. I don't know
>>> enough yet to help design it.
>>>
>>> On Sat, Jun 13, 2020 at 4:39 AM Till Rohrmann <tr...@apache.org>
>>> wrote:
>>>
>>>> Hi Dan,
>>>>
>>>> if you want to run a Flink job without specifying the main class via
>>>> `bin/flink run --class org.a.b.Foobar` then you have to add a MANIFEST.MF
>>>> file to your jar under META-INF and this file needs to contain `Main-Class:
>>>> org.a.b.Foobar`.
>>>>
>>>> Cheers,
>>>> Till
>>>>
>>>> On Fri, Jun 12, 2020 at 12:30 AM Austin Cawley-Edwards <
>>>> austin.cawley@gmail.com> wrote:
>>>>
>>>>> Hey all,
>>>>>
>>>>> Adding to Aaron's response, we use Bazel to build our Flink apps.
>>>>> We've open-sourced some of our setup here[1] though a bit outdated. There
>>>>> are definitely rough edges/ probably needs a good deal of work to fit other
>>>>> setups. We have written a wrapper around the `java_library` and
>>>>> `java_binary` and could do the same for `rules_scala`, though we just
>>>>> started using Bazel last November and have a lot to learn in terms of best
>>>>> practices there.
>>>>>
>>>>> If you're interested in contributing to a `rules_flink` project, I
>>>>> would be as well!
>>>>>
>>>>> Best,
>>>>> Austin
>>>>>
>>>>> [1]:
>>>>> https://github.com/fintechstudios/vp-operator-demo-ff-virtual-2020
>>>>>
>>>>> On Thu, Jun 11, 2020 at 6:14 PM Aaron Levin <aa...@stripe.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Dan,
>>>>>>
>>>>>> We use Bazel to compile our Flink applications. We're using
>>>>>> "rules_scala" (https://github.com/bazelbuild/rules_scala) to manage
>>>>>> the dependencies and produce jars. We haven't had any issues. However, I
>>>>>> have found that sometimes it's difficult to figure out exactly what Flink
>>>>>> target or dependency my application needs.
>>>>>>
>>>>>> Unfortunately I'm not sure what issue you're seeing here. I would
>>>>>> guess either your flink application wasn't compiled into the jar
>>>>>> you're executing. If you can paste the bazel target used to generate your
>>>>>> jar and how you're launching the application, that will be helpful
>>>>>> for diagnosis.
>>>>>>
>>>>>> On Thu, Jun 11, 2020 at 5:21 PM Dan Hill <qu...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I took the Flink playground and I'm trying to swap out Maven for
>>>>>>> Bazel. I got to the point where I'm hitting the following error. I want
>>>>>>> to diff my code with an existing, working setup.
>>>>>>>
>>>>>>> Thanks! - Dan
>>>>>>>
>>>>>>>
>>>>>>> client_1 | org.apache.flink.client.program.ProgramInvocationException:
>>>>>>> Neither a 'Main-Class', nor a 'program-class' entry was found in the jar
>>>>>>> file.
>>>>>>>
>>>>>>> client_1 | at
>>>>>>> org.apache.flink.client.program.PackagedProgram.getEntryPointClassNameFromJar(PackagedProgram.java:596)
>>>>>>>
>>>>>>> client_1 | at
>>>>>>> org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:190)
>>>>>>>
>>>>>>> client_1 | at
>>>>>>> org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:128)
>>>>>>>
>>>>>>> client_1 | at
>>>>>>> org.apache.flink.client.cli.CliFrontend.buildProgram(CliFrontend.java:862)
>>>>>>>
>>>>>>> client_1 | at
>>>>>>> org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:204)
>>>>>>>
>>>>>>> client_1 | at
>>>>>>> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1050)
>>>>>>>
>>>>>>> client_1 | at
>>>>>>> org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1126)
>>>>>>>
>>>>>>> client_1 | at
>>>>>>> org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
>>>>>>>
>>>>>>> client_1 | at
>>>>>>> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1126)
>>>>>>>
>>>>>>
Re: Does anyone have an example of Bazel working with Flink?
Posted by Aaron Levin <aa...@stripe.com>.
Hi Austin,
In our experience, `rules_scala` and `rules_java` are enough for us at this
point.
It's entirely possible I'm not thinking far enough into the future, though,
so don't take our lack of investment as a sign it's not worth investing in
:)
Best,
Aaron Levin
On Thu, Jun 18, 2020 at 10:27 AM Austin Cawley-Edwards <
austin.cawley@gmail.com> wrote:
> Great to hear Dan!
>
> @Aaron - would you/ your team be interested in a `rules_flink` project?
> I'm still fairly new to Bazel and know enough to contribute, but could
> definitely use guidance on design as well.
>
> Best,
> Austin
>
> On Mon, Jun 15, 2020 at 11:07 PM Dan Hill <qu...@gmail.com> wrote:
>
>> Thanks for the replies! I was able to use the provided answers to get a
>> setup working (maybe not the most efficiently). The main change I made was
>> to switch to including the deploy jar in the image (rather than the default
>> one).
>>
>> I'm open to contributing to a "rules_flink" project. I don't know enough
>> yet to help design it.
>>
>> On Sat, Jun 13, 2020 at 4:39 AM Till Rohrmann <tr...@apache.org>
>> wrote:
>>
>>> Hi Dan,
>>>
>>> if you want to run a Flink job without specifying the main class via
>>> `bin/flink run --class org.a.b.Foobar` then you have to add a MANIFEST.MF
>>> file to your jar under META-INF and this file needs to contain `Main-Class:
>>> org.a.b.Foobar`.
>>>
>>> Cheers,
>>> Till
>>>
>>> On Fri, Jun 12, 2020 at 12:30 AM Austin Cawley-Edwards <
>>> austin.cawley@gmail.com> wrote:
>>>
>>>> Hey all,
>>>>
>>>> Adding to Aaron's response, we use Bazel to build our Flink apps. We've
>>>> open-sourced some of our setup here[1] though a bit outdated. There are
>>>> definitely rough edges/ probably needs a good deal of work to fit other
>>>> setups. We have written a wrapper around the `java_library` and
>>>> `java_binary` and could do the same for `rules_scala`, though we just
>>>> started using Bazel last November and have a lot to learn in terms of best
>>>> practices there.
>>>>
>>>> If you're interested in contributing to a `rules_flink` project, I
>>>> would be as well!
>>>>
>>>> Best,
>>>> Austin
>>>>
>>>> [1]: https://github.com/fintechstudios/vp-operator-demo-ff-virtual-2020
>>>>
>>>> On Thu, Jun 11, 2020 at 6:14 PM Aaron Levin <aa...@stripe.com>
>>>> wrote:
>>>>
>>>>> Hi Dan,
>>>>>
>>>>> We use Bazel to compile our Flink applications. We're using
>>>>> "rules_scala" (https://github.com/bazelbuild/rules_scala) to manage
>>>>> the dependencies and produce jars. We haven't had any issues. However, I
>>>>> have found that sometimes it's difficult to figure out exactly what Flink
>>>>> target or dependency my application needs.
>>>>>
>>>>> Unfortunately I'm not sure what issue you're seeing here. I would
>>>>> guess either your flink application wasn't compiled into the jar
>>>>> you're executing. If you can paste the bazel target used to generate your
>>>>> jar and how you're launching the application, that will be helpful
>>>>> for diagnosis.
>>>>>
>>>>> On Thu, Jun 11, 2020 at 5:21 PM Dan Hill <qu...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I took the Flink playground and I'm trying to swap out Maven for
>>>>>> Bazel. I got to the point where I'm hitting the following error. I want
>>>>>> to diff my code with an existing, working setup.
>>>>>>
>>>>>> Thanks! - Dan
>>>>>>
>>>>>>
>>>>>> client_1 | org.apache.flink.client.program.ProgramInvocationException:
>>>>>> Neither a 'Main-Class', nor a 'program-class' entry was found in the jar
>>>>>> file.
>>>>>>
>>>>>> client_1 | at
>>>>>> org.apache.flink.client.program.PackagedProgram.getEntryPointClassNameFromJar(PackagedProgram.java:596)
>>>>>>
>>>>>> client_1 | at
>>>>>> org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:190)
>>>>>>
>>>>>> client_1 | at
>>>>>> org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:128)
>>>>>>
>>>>>> client_1 | at
>>>>>> org.apache.flink.client.cli.CliFrontend.buildProgram(CliFrontend.java:862)
>>>>>>
>>>>>> client_1 | at
>>>>>> org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:204)
>>>>>>
>>>>>> client_1 | at
>>>>>> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1050)
>>>>>>
>>>>>> client_1 | at
>>>>>> org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1126)
>>>>>>
>>>>>> client_1 | at
>>>>>> org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
>>>>>>
>>>>>> client_1 | at
>>>>>> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1126)
>>>>>>
>>>>>
Re: Does anyone have an example of Bazel working with Flink?
Posted by Austin Cawley-Edwards <au...@gmail.com>.
Great to hear Dan!
@Aaron - would you/ your team be interested in a `rules_flink` project? I'm
still fairly new to Bazel and know enough to contribute, but could
definitely use guidance on design as well.
Best,
Austin
On Mon, Jun 15, 2020 at 11:07 PM Dan Hill <qu...@gmail.com> wrote:
> Thanks for the replies! I was able to use the provided answers to get a
> setup working (maybe not the most efficiently). The main change I made was
> to switch to including the deploy jar in the image (rather than the default
> one).
>
> I'm open to contributing to a "rules_flink" project. I don't know enough
> yet to help design it.
>
> On Sat, Jun 13, 2020 at 4:39 AM Till Rohrmann <tr...@apache.org>
> wrote:
>
>> Hi Dan,
>>
>> if you want to run a Flink job without specifying the main class via
>> `bin/flink run --class org.a.b.Foobar` then you have to add a MANIFEST.MF
>> file to your jar under META-INF and this file needs to contain `Main-Class:
>> org.a.b.Foobar`.
>>
>> Cheers,
>> Till
>>
>> On Fri, Jun 12, 2020 at 12:30 AM Austin Cawley-Edwards <
>> austin.cawley@gmail.com> wrote:
>>
>>> Hey all,
>>>
>>> Adding to Aaron's response, we use Bazel to build our Flink apps. We've
>>> open-sourced some of our setup here[1] though a bit outdated. There are
>>> definitely rough edges/ probably needs a good deal of work to fit other
>>> setups. We have written a wrapper around the `java_library` and
>>> `java_binary` and could do the same for `rules_scala`, though we just
>>> started using Bazel last November and have a lot to learn in terms of best
>>> practices there.
>>>
>>> If you're interested in contributing to a `rules_flink` project, I would
>>> be as well!
>>>
>>> Best,
>>> Austin
>>>
>>> [1]: https://github.com/fintechstudios/vp-operator-demo-ff-virtual-2020
>>>
>>> On Thu, Jun 11, 2020 at 6:14 PM Aaron Levin <aa...@stripe.com>
>>> wrote:
>>>
>>>> Hi Dan,
>>>>
>>>> We use Bazel to compile our Flink applications. We're using
>>>> "rules_scala" (https://github.com/bazelbuild/rules_scala) to manage
>>>> the dependencies and produce jars. We haven't had any issues. However, I
>>>> have found that sometimes it's difficult to figure out exactly what Flink
>>>> target or dependency my application needs.
>>>>
>>>> Unfortunately I'm not sure what issue you're seeing here. I would guess
>>>> either your flink application wasn't compiled into the jar
>>>> you're executing. If you can paste the bazel target used to generate your
>>>> jar and how you're launching the application, that will be helpful
>>>> for diagnosis.
>>>>
>>>> On Thu, Jun 11, 2020 at 5:21 PM Dan Hill <qu...@gmail.com> wrote:
>>>>
>>>>> I took the Flink playground and I'm trying to swap out Maven for
>>>>> Bazel. I got to the point where I'm hitting the following error. I want
>>>>> to diff my code with an existing, working setup.
>>>>>
>>>>> Thanks! - Dan
>>>>>
>>>>>
>>>>> client_1 | org.apache.flink.client.program.ProgramInvocationException:
>>>>> Neither a 'Main-Class', nor a 'program-class' entry was found in the jar
>>>>> file.
>>>>>
>>>>> client_1 | at
>>>>> org.apache.flink.client.program.PackagedProgram.getEntryPointClassNameFromJar(PackagedProgram.java:596)
>>>>>
>>>>> client_1 | at
>>>>> org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:190)
>>>>>
>>>>> client_1 | at
>>>>> org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:128)
>>>>>
>>>>> client_1 | at
>>>>> org.apache.flink.client.cli.CliFrontend.buildProgram(CliFrontend.java:862)
>>>>>
>>>>> client_1 | at
>>>>> org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:204)
>>>>>
>>>>> client_1 | at
>>>>> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1050)
>>>>>
>>>>> client_1 | at
>>>>> org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1126)
>>>>>
>>>>> client_1 | at
>>>>> org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
>>>>>
>>>>> client_1 | at
>>>>> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1126)
>>>>>
>>>>
Re: Does anyone have an example of Bazel working with Flink?
Posted by Dan Hill <qu...@gmail.com>.
Thanks for the replies! I was able to use the provided answers to get a
setup working (maybe not the most efficiently). The main change I made was
to switch to including the deploy jar in the image (rather than the default
one).
I'm open to contributing to a "rules_flink" project. I don't know enough
yet to help design it.
On Sat, Jun 13, 2020 at 4:39 AM Till Rohrmann <tr...@apache.org> wrote:
> Hi Dan,
>
> if you want to run a Flink job without specifying the main class via
> `bin/flink run --class org.a.b.Foobar` then you have to add a MANIFEST.MF
> file to your jar under META-INF and this file needs to contain `Main-Class:
> org.a.b.Foobar`.
>
> Cheers,
> Till
>
> On Fri, Jun 12, 2020 at 12:30 AM Austin Cawley-Edwards <
> austin.cawley@gmail.com> wrote:
>
>> Hey all,
>>
>> Adding to Aaron's response, we use Bazel to build our Flink apps. We've
>> open-sourced some of our setup here[1] though a bit outdated. There are
>> definitely rough edges/ probably needs a good deal of work to fit other
>> setups. We have written a wrapper around the `java_library` and
>> `java_binary` and could do the same for `rules_scala`, though we just
>> started using Bazel last November and have a lot to learn in terms of best
>> practices there.
>>
>> If you're interested in contributing to a `rules_flink` project, I would
>> be as well!
>>
>> Best,
>> Austin
>>
>> [1]: https://github.com/fintechstudios/vp-operator-demo-ff-virtual-2020
>>
>> On Thu, Jun 11, 2020 at 6:14 PM Aaron Levin <aa...@stripe.com>
>> wrote:
>>
>>> Hi Dan,
>>>
>>> We use Bazel to compile our Flink applications. We're using
>>> "rules_scala" (https://github.com/bazelbuild/rules_scala) to manage the
>>> dependencies and produce jars. We haven't had any issues. However, I have
>>> found that sometimes it's difficult to figure out exactly what Flink target
>>> or dependency my application needs.
>>>
>>> Unfortunately I'm not sure what issue you're seeing here. I would guess
>>> either your flink application wasn't compiled into the jar
>>> you're executing. If you can paste the bazel target used to generate your
>>> jar and how you're launching the application, that will be helpful
>>> for diagnosis.
>>>
>>> On Thu, Jun 11, 2020 at 5:21 PM Dan Hill <qu...@gmail.com> wrote:
>>>
>>>> I took the Flink playground and I'm trying to swap out Maven for
>>>> Bazel. I got to the point where I'm hitting the following error. I want
>>>> to diff my code with an existing, working setup.
>>>>
>>>> Thanks! - Dan
>>>>
>>>>
>>>> client_1 | org.apache.flink.client.program.ProgramInvocationException:
>>>> Neither a 'Main-Class', nor a 'program-class' entry was found in the jar
>>>> file.
>>>>
>>>> client_1 | at
>>>> org.apache.flink.client.program.PackagedProgram.getEntryPointClassNameFromJar(PackagedProgram.java:596)
>>>>
>>>> client_1 | at
>>>> org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:190)
>>>>
>>>> client_1 | at
>>>> org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:128)
>>>>
>>>> client_1 | at
>>>> org.apache.flink.client.cli.CliFrontend.buildProgram(CliFrontend.java:862)
>>>>
>>>> client_1 | at
>>>> org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:204)
>>>>
>>>> client_1 | at
>>>> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1050)
>>>>
>>>> client_1 | at
>>>> org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1126)
>>>>
>>>> client_1 | at
>>>> org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
>>>>
>>>> client_1 | at
>>>> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1126)
>>>>
>>>
Re: Does anyone have an example of Bazel working with Flink?
Posted by Till Rohrmann <tr...@apache.org>.
Hi Dan,
if you want to run a Flink job without specifying the main class via
`bin/flink run --class org.a.b.Foobar` then you have to add a MANIFEST.MF
file to your jar under META-INF and this file needs to contain `Main-Class:
org.a.b.Foobar`.
Cheers,
Till
On Fri, Jun 12, 2020 at 12:30 AM Austin Cawley-Edwards <
austin.cawley@gmail.com> wrote:
> Hey all,
>
> Adding to Aaron's response, we use Bazel to build our Flink apps. We've
> open-sourced some of our setup here[1] though a bit outdated. There are
> definitely rough edges/ probably needs a good deal of work to fit other
> setups. We have written a wrapper around the `java_library` and
> `java_binary` and could do the same for `rules_scala`, though we just
> started using Bazel last November and have a lot to learn in terms of best
> practices there.
>
> If you're interested in contributing to a `rules_flink` project, I would
> be as well!
>
> Best,
> Austin
>
> [1]: https://github.com/fintechstudios/vp-operator-demo-ff-virtual-2020
>
> On Thu, Jun 11, 2020 at 6:14 PM Aaron Levin <aa...@stripe.com> wrote:
>
>> Hi Dan,
>>
>> We use Bazel to compile our Flink applications. We're using "rules_scala"
>> (https://github.com/bazelbuild/rules_scala) to manage the dependencies
>> and produce jars. We haven't had any issues. However, I have found that
>> sometimes it's difficult to figure out exactly what Flink target or
>> dependency my application needs.
>>
>> Unfortunately I'm not sure what issue you're seeing here. I would guess
>> either your flink application wasn't compiled into the jar
>> you're executing. If you can paste the bazel target used to generate your
>> jar and how you're launching the application, that will be helpful
>> for diagnosis.
>>
>> On Thu, Jun 11, 2020 at 5:21 PM Dan Hill <qu...@gmail.com> wrote:
>>
>>> I took the Flink playground and I'm trying to swap out Maven for Bazel.
>>> I got to the point where I'm hitting the following error. I want to diff
>>> my code with an existing, working setup.
>>>
>>> Thanks! - Dan
>>>
>>>
>>> client_1 | org.apache.flink.client.program.ProgramInvocationException:
>>> Neither a 'Main-Class', nor a 'program-class' entry was found in the jar
>>> file.
>>>
>>> client_1 | at
>>> org.apache.flink.client.program.PackagedProgram.getEntryPointClassNameFromJar(PackagedProgram.java:596)
>>>
>>> client_1 | at
>>> org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:190)
>>>
>>> client_1 | at
>>> org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:128)
>>>
>>> client_1 | at
>>> org.apache.flink.client.cli.CliFrontend.buildProgram(CliFrontend.java:862)
>>>
>>> client_1 | at
>>> org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:204)
>>>
>>> client_1 | at
>>> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1050)
>>>
>>> client_1 | at
>>> org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1126)
>>>
>>> client_1 | at
>>> org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
>>>
>>> client_1 | at
>>> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1126)
>>>
>>
Re: Does anyone have an example of Bazel working with Flink?
Posted by Austin Cawley-Edwards <au...@gmail.com>.
Hey all,
Adding to Aaron's response, we use Bazel to build our Flink apps. We've
open-sourced some of our setup here[1] though a bit outdated. There are
definitely rough edges/ probably needs a good deal of work to fit other
setups. We have written a wrapper around the `java_library` and
`java_binary` and could do the same for `rules_scala`, though we just
started using Bazel last November and have a lot to learn in terms of best
practices there.
If you're interested in contributing to a `rules_flink` project, I would be
as well!
Best,
Austin
[1]: https://github.com/fintechstudios/vp-operator-demo-ff-virtual-2020
On Thu, Jun 11, 2020 at 6:14 PM Aaron Levin <aa...@stripe.com> wrote:
> Hi Dan,
>
> We use Bazel to compile our Flink applications. We're using "rules_scala" (
> https://github.com/bazelbuild/rules_scala) to manage the dependencies and
> produce jars. We haven't had any issues. However, I have found that
> sometimes it's difficult to figure out exactly what Flink target or
> dependency my application needs.
>
> Unfortunately I'm not sure what issue you're seeing here. I would guess
> either your flink application wasn't compiled into the jar
> you're executing. If you can paste the bazel target used to generate your
> jar and how you're launching the application, that will be helpful
> for diagnosis.
>
> On Thu, Jun 11, 2020 at 5:21 PM Dan Hill <qu...@gmail.com> wrote:
>
>> I took the Flink playground and I'm trying to swap out Maven for Bazel.
>> I got to the point where I'm hitting the following error. I want to diff
>> my code with an existing, working setup.
>>
>> Thanks! - Dan
>>
>>
>> client_1 | org.apache.flink.client.program.ProgramInvocationException:
>> Neither a 'Main-Class', nor a 'program-class' entry was found in the jar
>> file.
>>
>> client_1 | at
>> org.apache.flink.client.program.PackagedProgram.getEntryPointClassNameFromJar(PackagedProgram.java:596)
>>
>> client_1 | at
>> org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:190)
>>
>> client_1 | at
>> org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:128)
>>
>> client_1 | at
>> org.apache.flink.client.cli.CliFrontend.buildProgram(CliFrontend.java:862)
>>
>> client_1 | at
>> org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:204)
>>
>> client_1 | at
>> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1050)
>>
>> client_1 | at
>> org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1126)
>>
>> client_1 | at
>> org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
>>
>> client_1 | at
>> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1126)
>>
>
Re: Does anyone have an example of Bazel working with Flink?
Posted by Aaron Levin <aa...@stripe.com>.
Hi Dan,
We use Bazel to compile our Flink applications. We're using "rules_scala" (
https://github.com/bazelbuild/rules_scala) to manage the dependencies and
produce jars. We haven't had any issues. However, I have found that
sometimes it's difficult to figure out exactly what Flink target or
dependency my application needs.
Unfortunately I'm not sure what issue you're seeing here. I would guess
either your flink application wasn't compiled into the jar
you're executing. If you can paste the bazel target used to generate your
jar and how you're launching the application, that will be helpful
for diagnosis.
On Thu, Jun 11, 2020 at 5:21 PM Dan Hill <qu...@gmail.com> wrote:
> I took the Flink playground and I'm trying to swap out Maven for Bazel. I
> got to the point where I'm hitting the following error. I want to diff my
> code with an existing, working setup.
>
> Thanks! - Dan
>
>
> client_1 | org.apache.flink.client.program.ProgramInvocationException:
> Neither a 'Main-Class', nor a 'program-class' entry was found in the jar
> file.
>
> client_1 | at
> org.apache.flink.client.program.PackagedProgram.getEntryPointClassNameFromJar(PackagedProgram.java:596)
>
> client_1 | at
> org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:190)
>
> client_1 | at
> org.apache.flink.client.program.PackagedProgram.<init>(PackagedProgram.java:128)
>
> client_1 | at
> org.apache.flink.client.cli.CliFrontend.buildProgram(CliFrontend.java:862)
>
> client_1 | at
> org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:204)
>
> client_1 | at
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1050)
>
> client_1 | at
> org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1126)
>
> client_1 | at
> org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
>
> client_1 | at
> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1126)
>