You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@beam.apache.org by Dumi Loghin <du...@gmail.com> on 2016/07/20 08:47:27 UTC
Flink Wordcount example on single node "No runner was specified"
Hi,
I'm trying to run Wordcount example on Flink single node but I'm getting
this error:
~/incubator-beam/runners/flink/examples$ mvn exec:exec
-Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt -Drunner=FlinkRunner
...
Exception in thread "main" java.lang.IllegalArgumentException: No Runner
was specified and the DirectRunner was not found on the classpath.
Specify a runner by either:
Explicitly specifying a runner by providing the 'runner' property
Adding the DirectRunner to the classpath
Calling 'PipelineOptions.setRunner(PipelineRunner)' directly
at
org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:282)
at
org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:273)
at
org.apache.beam.sdk.options.ProxyInvocationHandler.getDefault(ProxyInvocationHandler.java:482)
at
org.apache.beam.sdk.options.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:157)
at
org.apache.beam.sdk.options.PipelineOptionsValidator.validate(PipelineOptionsValidator.java:72)
at
org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:306)
at org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:106)
...
I've tried with -Drunner=FlinkRunner and without it and I get the same
error. Moreover, the runner is also set in the source:
options.setRunner(FlinkRunner.class);
Any suggestions?
Best regards,
Dumi
Re: Flink Wordcount example on single node "No runner was specified"
Posted by Maximilian Michels <mx...@apache.org>.
Hi Dumi,
Thanks for the feedback. The Readme has been updated in this PR
https://github.com/apache/incubator-beam/pull/821
Best,
Max
On Thu, Jul 28, 2016 at 7:54 AM, Dumi Loghin <du...@gmail.com> wrote:
> When running on a Flink cluster, it's working with the absolute path. This
> is the working command:
>
> ~/flink-1.0.3/bin$ ./flink run
> /home/hadoop/incubator-beam/runners/flink/beam-test/target/beam-test-1.0-SNAPSHOT.jar
> --input=/home/hadoop/kinglear.txt --output=wc-out.txt --runner=FlinkRunner
>
>
> By the way, I think the Readme can be improved because I was getting some
> errors when trying to build the jar for running on the Flink cluster. So I
> have created a new maven project and replaced the src folder with the one
> from examples. However I was getting:
>
> [ERROR]
> /home/hadoop/incubator-beam/runners/flink/cluster/src/main/java/org/apache/beam/runners/flink/examples/streaming/KafkaIOExamples.java:[78,38]
> diamond operator is not supported in -source 1.5
> (use -source 7 or higher to enable diamond operator)
>
> so I added
>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-compiler-plugin</artifactId>
> <configuration>
> <source>1.7</source>
> <target>1.7</target>
> </configuration>
> </plugin>
>
> to the plugins section of pom.xml.
>
>
> Then I was getting:
>
> [ERROR]
> /home/hadoop/incubator-beam/runners/flink/cluster/src/main/java/org/apache/beam/runners/flink/examples/streaming/KafkaIOExamples.java:[38,51]
> package org.apache.flink.streaming.connectors.kafka does not exist
>
> so I added
>
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-connector-kafka-0.8_2.10</artifactId>
> <version>1.0.3</version>
> </dependency>
>
> to the dependencies section of pom.xml.
>
> Best regards,
> Dumi
>
>
> On Wed, Jul 27, 2016 at 7:48 PM, Maximilian Michels <mx...@apache.org> wrote:
>>
>> Okay great!
>>
>> I think it was a permission problem before. It shouldn't matter where
>> the file is as long as Flink can access it. Still puzzled about the
>> error message, was that really the entire stack trace?
>>
>> On Wed, Jul 27, 2016 at 7:05 AM, Dumi Loghin <du...@gmail.com> wrote:
>> > Hi,
>> >
>> > The application is running if the input file is in the examples folder:
>> >
>> > ~/incubator-beam/runners/flink/examples$
>> > /home/hadoop/apache-maven-3.3.9/bin/mvn exec:java
>> > -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
>> > -Dinput=kinglear.txt -Doutput=wc-out.txt
>> >
>> > ( as opposed to "~/incubator-beam/runners/flink/examples$
>> > /home/hadoop/apache-maven-3.3.9/bin/mvn exec:java
>> > -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
>> > -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt" that is throwing
>> > the
>> > exception)
>> >
>> > Best regards,
>> > Dumi
>> >
>> >
>> >
>> > On Tue, Jul 26, 2016 at 11:53 PM, Maximilian Michels <mx...@apache.org>
>> > wrote:
>> >>
>> >> Hi Dumi,
>> >>
>> >> The execution fails during shutdown of the local Flink cluster. Does
>> >> the program execute before the exception occurs?
>> >>
>> >> Best,
>> >> Max
>> >>
>> >> On Tue, Jul 26, 2016 at 5:15 AM, Dumi Loghin <du...@gmail.com>
>> >> wrote:
>> >> > Hi,
>> >> >
>> >> > Thank you!
>> >> >
>> >> > I have manually added WordCount.java and pom.xml to the latest Beam
>> >> > code
>> >> > and
>> >> > try to run it. But now I'm getting the following exception:
>> >> >
>> >> > :~/incubator-beam/runners/flink/examples$
>> >> > /home/hadoop/apache-maven-3.3.9/bin/mvn exec:java
>> >> > -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
>> >> > -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt
>> >> >
>> >> > java.lang.reflect.InvocationTargetException
>> >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>> >> > Method)
>> >> > at
>> >> >
>> >> >
>> >> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> >> > at
>> >> >
>> >> >
>> >> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> >> > at java.lang.reflect.Method.invoke(Method.java:606)
>> >> > at
>> >> > org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
>> >> > at java.lang.Thread.run(Thread.java:745)
>> >> > Caused by: java.lang.RuntimeException: Pipeline execution failed
>> >> > at
>> >> > org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:113)
>> >> > at
>> >> > org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:48)
>> >> > at org.apache.beam.sdk.Pipeline.run(Pipeline.java:183)
>> >> > at
>> >> >
>> >> >
>> >> > org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:117)
>> >> > ... 6 more
>> >> > Caused by: java.util.concurrent.TimeoutException: Futures timed out
>> >> > after
>> >> > [10000 milliseconds]
>> >> > at
>> >> > scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
>> >> > at
>> >> > scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:153)
>> >> > at
>> >> > scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
>> >> > at
>> >> > scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
>> >> > at
>> >> >
>> >> >
>> >> > scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
>> >> > at scala.concurrent.Await$.ready(package.scala:86)
>> >> > at
>> >> >
>> >> >
>> >> > org.apache.flink.runtime.minicluster.FlinkMiniCluster.shutdown(FlinkMiniCluster.scala:340)
>> >> > at
>> >> >
>> >> >
>> >> > org.apache.flink.runtime.minicluster.FlinkMiniCluster.stop(FlinkMiniCluster.scala:319)
>> >> > at
>> >> > org.apache.flink.client.LocalExecutor.stop(LocalExecutor.java:126)
>> >> > at
>> >> >
>> >> >
>> >> > org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:194)
>> >> > at
>> >> >
>> >> >
>> >> > org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:90)
>> >> > at
>> >> >
>> >> >
>> >> > org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:116)
>> >> > at
>> >> > org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:110)
>> >> > ... 9 more
>> >> >
>> >> > Best regards,
>> >> > Dumi
>> >> >
>> >> > On Mon, Jul 25, 2016 at 7:14 PM, Maximilian Michels <mx...@apache.org>
>> >> > wrote:
>> >> >>
>> >> >> Hi Dumi,
>> >> >>
>> >> >> Somewhere on the way we broke the exec configuration. I've fixed
>> >> >> this
>> >> >> in this PR and updated the README:
>> >> >> https://github.com/apache/incubator-beam/pull/724
>> >> >>
>> >> >> Should be merged soon.
>> >> >>
>> >> >> Best,
>> >> >> Max
>> >> >>
>> >> >> On Fri, Jul 22, 2016 at 2:49 PM, Lukasz Cwik <lc...@google.com>
>> >> >> wrote:
>> >> >> > I think runner should be org.apache.beam.runners.flink.FlinkRunner
>> >> >> > and
>> >> >> > not
>> >> >> > org.apache.beam.runners.flink.FlinkRunner.class but that is
>> >> >> > probably
>> >> >> > unrelated to the failure that you are seeing.
>> >> >> >
>> >> >> > On Fri, Jul 22, 2016 at 7:42 AM, Dumi Loghin
>> >> >> > <du...@gmail.com>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> Hi,
>> >> >> >>
>> >> >> >> I'm still getting an exception related to arguments. I mention
>> >> >> >> that
>> >> >> >> I'm
>> >> >> >> working with commit adb472083460bc175a0685133eadb6a336f1a4a2 (22
>> >> >> >> Jul).
>> >> >> >>
>> >> >> >> ~/incubator-beam/runners/flink/examples$ mvn exec:java
>> >> >> >> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
>> >> >> >>
>> >> >> >>
>> >> >> >> -Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
>> >> >> >> --inputFile=/home/hadoop/kinglear.txt --output=wc-out.txt"
>> >> >> >> ...
>> >> >> >> [ERROR] Failed to execute goal
>> >> >> >> org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on
>> >> >> >> project
>> >> >> >> beam-runners-flink_2.10-examples: Unable to parse configuration
>> >> >> >> of
>> >> >> >> mojo
>> >> >> >> org.codehaus.mojo:exec-maven-plugin:1.4.0:java for parameter
>> >> >> >> arguments:
>> >> >> >> Cannot store value into array: ArrayStoreException -> [Help 1]
>> >> >> >> ...
>> >> >> >>
>> >> >> >> However, I was able to run it in early June. If I revert to, for
>> >> >> >> example,
>> >> >> >> commit c2146b9f9d6a1f39a5699725ccb51829d751b88a (6 Jun) I'm not
>> >> >> >> getting
>> >> >> >> the
>> >> >> >> above exceptions. Maybe something changed in the options
>> >> >> >> (arguments)
>> >> >> >> handling code?
>> >> >> >>
>> >> >> >> Best regards,
>> >> >> >> Dumi
>> >> >> >>
>> >> >> >> On Wed, Jul 20, 2016 at 7:35 PM, Aljoscha Krettek
>> >> >> >> <al...@apache.org>
>> >> >> >> wrote:
>> >> >> >>>
>> >> >> >>> I think it should work like this:
>> >> >> >>>
>> >> >> >>> mvn exec:java \
>> >> >> >>>
>> >> >> >>> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount \
>> >> >> >>>
>> >> >> >>>
>> >> >> >>> -Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
>> >> >> >>> --inputFile='"$input"' --output='"$outfile_prefix"'"'
>> >> >> >>>
>> >> >> >>> On Wed, 20 Jul 2016 at 10:47 Dumi Loghin <du...@gmail.com>
>> >> >> >>> wrote:
>> >> >> >>>>
>> >> >> >>>> Hi,
>> >> >> >>>>
>> >> >> >>>> I'm trying to run Wordcount example on Flink single node but
>> >> >> >>>> I'm
>> >> >> >>>> getting
>> >> >> >>>> this error:
>> >> >> >>>>
>> >> >> >>>> ~/incubator-beam/runners/flink/examples$ mvn exec:exec
>> >> >> >>>> -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt
>> >> >> >>>> -Drunner=FlinkRunner
>> >> >> >>>> ...
>> >> >> >>>> Exception in thread "main" java.lang.IllegalArgumentException:
>> >> >> >>>> No
>> >> >> >>>> Runner
>> >> >> >>>> was specified and the DirectRunner was not found on the
>> >> >> >>>> classpath.
>> >> >> >>>> Specify a runner by either:
>> >> >> >>>> Explicitly specifying a runner by providing the 'runner'
>> >> >> >>>> property
>> >> >> >>>> Adding the DirectRunner to the classpath
>> >> >> >>>> Calling 'PipelineOptions.setRunner(PipelineRunner)'
>> >> >> >>>> directly
>> >> >> >>>> at
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:282)
>> >> >> >>>> at
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:273)
>> >> >> >>>> at
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>> org.apache.beam.sdk.options.ProxyInvocationHandler.getDefault(ProxyInvocationHandler.java:482)
>> >> >> >>>> at
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>> org.apache.beam.sdk.options.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:157)
>> >> >> >>>> at
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>> org.apache.beam.sdk.options.PipelineOptionsValidator.validate(PipelineOptionsValidator.java:72)
>> >> >> >>>> at
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:306)
>> >> >> >>>> at
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>> org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:106)
>> >> >> >>>> ...
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>> I've tried with -Drunner=FlinkRunner and without it and I get
>> >> >> >>>> the
>> >> >> >>>> same
>> >> >> >>>> error. Moreover, the runner is also set in the source:
>> >> >> >>>>
>> >> >> >>>> options.setRunner(FlinkRunner.class);
>> >> >> >>>>
>> >> >> >>>> Any suggestions?
>> >> >> >>>>
>> >> >> >>>> Best regards,
>> >> >> >>>> Dumi
>> >> >> >>
>> >> >> >>
>> >> >> >
>> >> >
>> >> >
>> >
>> >
>
>
Re: Flink Wordcount example on single node "No runner was specified"
Posted by Dumi Loghin <du...@gmail.com>.
When running on a Flink cluster, it's working with the absolute path. This
is the working command:
~/flink-1.0.3/bin$ ./flink run
/home/hadoop/incubator-beam/runners/flink/beam-test/target/beam-test-1.0-SNAPSHOT.jar
--input=/home/hadoop/kinglear.txt --output=wc-out.txt --runner=FlinkRunner
By the way, I think the Readme can be improved because I was getting some
errors when trying to build the jar for running on the Flink cluster. So I
have created a new maven project and replaced the src folder with the one
from examples. However I was getting:
[ERROR]
/home/hadoop/incubator-beam/runners/flink/cluster/src/main/java/org/apache/beam/runners/flink/examples/streaming/KafkaIOExamples.java:[78,38]
diamond operator is not supported in -source 1.5
(use -source 7 or higher to enable diamond operator)
so I added
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
to the plugins section of pom.xml.
Then I was getting:
[ERROR]
/home/hadoop/incubator-beam/runners/flink/cluster/src/main/java/org/apache/beam/runners/flink/examples/streaming/KafkaIOExamples.java:[38,51]
package org.apache.flink.streaming.connectors.kafka does not exist
so I added
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-kafka-0.8_2.10</artifactId>
<version>1.0.3</version>
</dependency>
to the dependencies section of pom.xml.
Best regards,
Dumi
On Wed, Jul 27, 2016 at 7:48 PM, Maximilian Michels <mx...@apache.org> wrote:
> Okay great!
>
> I think it was a permission problem before. It shouldn't matter where
> the file is as long as Flink can access it. Still puzzled about the
> error message, was that really the entire stack trace?
>
> On Wed, Jul 27, 2016 at 7:05 AM, Dumi Loghin <du...@gmail.com> wrote:
> > Hi,
> >
> > The application is running if the input file is in the examples folder:
> >
> > ~/incubator-beam/runners/flink/examples$
> > /home/hadoop/apache-maven-3.3.9/bin/mvn exec:java
> > -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
> > -Dinput=kinglear.txt -Doutput=wc-out.txt
> >
> > ( as opposed to "~/incubator-beam/runners/flink/examples$
> > /home/hadoop/apache-maven-3.3.9/bin/mvn exec:java
> > -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
> > -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt" that is throwing
> the
> > exception)
> >
> > Best regards,
> > Dumi
> >
> >
> >
> > On Tue, Jul 26, 2016 at 11:53 PM, Maximilian Michels <mx...@apache.org>
> wrote:
> >>
> >> Hi Dumi,
> >>
> >> The execution fails during shutdown of the local Flink cluster. Does
> >> the program execute before the exception occurs?
> >>
> >> Best,
> >> Max
> >>
> >> On Tue, Jul 26, 2016 at 5:15 AM, Dumi Loghin <du...@gmail.com>
> wrote:
> >> > Hi,
> >> >
> >> > Thank you!
> >> >
> >> > I have manually added WordCount.java and pom.xml to the latest Beam
> code
> >> > and
> >> > try to run it. But now I'm getting the following exception:
> >> >
> >> > :~/incubator-beam/runners/flink/examples$
> >> > /home/hadoop/apache-maven-3.3.9/bin/mvn exec:java
> >> > -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
> >> > -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt
> >> >
> >> > java.lang.reflect.InvocationTargetException
> >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> > at
> >> >
> >> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >> > at
> >> >
> >> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> > at java.lang.reflect.Method.invoke(Method.java:606)
> >> > at
> >> > org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
> >> > at java.lang.Thread.run(Thread.java:745)
> >> > Caused by: java.lang.RuntimeException: Pipeline execution failed
> >> > at
> >> > org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:113)
> >> > at
> >> > org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:48)
> >> > at org.apache.beam.sdk.Pipeline.run(Pipeline.java:183)
> >> > at
> >> >
> >> >
> org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:117)
> >> > ... 6 more
> >> > Caused by: java.util.concurrent.TimeoutException: Futures timed out
> >> > after
> >> > [10000 milliseconds]
> >> > at
> >> > scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
> >> > at
> >> > scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:153)
> >> > at
> >> > scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
> >> > at
> >> > scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
> >> > at
> >> >
> >> >
> scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
> >> > at scala.concurrent.Await$.ready(package.scala:86)
> >> > at
> >> >
> >> >
> org.apache.flink.runtime.minicluster.FlinkMiniCluster.shutdown(FlinkMiniCluster.scala:340)
> >> > at
> >> >
> >> >
> org.apache.flink.runtime.minicluster.FlinkMiniCluster.stop(FlinkMiniCluster.scala:319)
> >> > at
> >> > org.apache.flink.client.LocalExecutor.stop(LocalExecutor.java:126)
> >> > at
> >> >
> >> >
> org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:194)
> >> > at
> >> >
> >> >
> org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:90)
> >> > at
> >> >
> >> >
> org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:116)
> >> > at
> >> > org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:110)
> >> > ... 9 more
> >> >
> >> > Best regards,
> >> > Dumi
> >> >
> >> > On Mon, Jul 25, 2016 at 7:14 PM, Maximilian Michels <mx...@apache.org>
> >> > wrote:
> >> >>
> >> >> Hi Dumi,
> >> >>
> >> >> Somewhere on the way we broke the exec configuration. I've fixed this
> >> >> in this PR and updated the README:
> >> >> https://github.com/apache/incubator-beam/pull/724
> >> >>
> >> >> Should be merged soon.
> >> >>
> >> >> Best,
> >> >> Max
> >> >>
> >> >> On Fri, Jul 22, 2016 at 2:49 PM, Lukasz Cwik <lc...@google.com>
> wrote:
> >> >> > I think runner should be org.apache.beam.runners.flink.FlinkRunner
> >> >> > and
> >> >> > not
> >> >> > org.apache.beam.runners.flink.FlinkRunner.class but that is
> probably
> >> >> > unrelated to the failure that you are seeing.
> >> >> >
> >> >> > On Fri, Jul 22, 2016 at 7:42 AM, Dumi Loghin <dumiloghin@gmail.com
> >
> >> >> > wrote:
> >> >> >>
> >> >> >> Hi,
> >> >> >>
> >> >> >> I'm still getting an exception related to arguments. I mention
> that
> >> >> >> I'm
> >> >> >> working with commit adb472083460bc175a0685133eadb6a336f1a4a2 (22
> >> >> >> Jul).
> >> >> >>
> >> >> >> ~/incubator-beam/runners/flink/examples$ mvn exec:java
> >> >> >> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
> >> >> >>
> >> >> >>
> -Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
> >> >> >> --inputFile=/home/hadoop/kinglear.txt --output=wc-out.txt"
> >> >> >> ...
> >> >> >> [ERROR] Failed to execute goal
> >> >> >> org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on
> >> >> >> project
> >> >> >> beam-runners-flink_2.10-examples: Unable to parse configuration of
> >> >> >> mojo
> >> >> >> org.codehaus.mojo:exec-maven-plugin:1.4.0:java for parameter
> >> >> >> arguments:
> >> >> >> Cannot store value into array: ArrayStoreException -> [Help 1]
> >> >> >> ...
> >> >> >>
> >> >> >> However, I was able to run it in early June. If I revert to, for
> >> >> >> example,
> >> >> >> commit c2146b9f9d6a1f39a5699725ccb51829d751b88a (6 Jun) I'm not
> >> >> >> getting
> >> >> >> the
> >> >> >> above exceptions. Maybe something changed in the options
> (arguments)
> >> >> >> handling code?
> >> >> >>
> >> >> >> Best regards,
> >> >> >> Dumi
> >> >> >>
> >> >> >> On Wed, Jul 20, 2016 at 7:35 PM, Aljoscha Krettek
> >> >> >> <al...@apache.org>
> >> >> >> wrote:
> >> >> >>>
> >> >> >>> I think it should work like this:
> >> >> >>>
> >> >> >>> mvn exec:java \
> >> >> >>>
> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount \
> >> >> >>>
> >> >> >>>
> -Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
> >> >> >>> --inputFile='"$input"' --output='"$outfile_prefix"'"'
> >> >> >>>
> >> >> >>> On Wed, 20 Jul 2016 at 10:47 Dumi Loghin <du...@gmail.com>
> >> >> >>> wrote:
> >> >> >>>>
> >> >> >>>> Hi,
> >> >> >>>>
> >> >> >>>> I'm trying to run Wordcount example on Flink single node but I'm
> >> >> >>>> getting
> >> >> >>>> this error:
> >> >> >>>>
> >> >> >>>> ~/incubator-beam/runners/flink/examples$ mvn exec:exec
> >> >> >>>> -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt
> >> >> >>>> -Drunner=FlinkRunner
> >> >> >>>> ...
> >> >> >>>> Exception in thread "main" java.lang.IllegalArgumentException:
> No
> >> >> >>>> Runner
> >> >> >>>> was specified and the DirectRunner was not found on the
> classpath.
> >> >> >>>> Specify a runner by either:
> >> >> >>>> Explicitly specifying a runner by providing the 'runner'
> >> >> >>>> property
> >> >> >>>> Adding the DirectRunner to the classpath
> >> >> >>>> Calling 'PipelineOptions.setRunner(PipelineRunner)' directly
> >> >> >>>> at
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:282)
> >> >> >>>> at
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:273)
> >> >> >>>> at
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> org.apache.beam.sdk.options.ProxyInvocationHandler.getDefault(ProxyInvocationHandler.java:482)
> >> >> >>>> at
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> org.apache.beam.sdk.options.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:157)
> >> >> >>>> at
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> org.apache.beam.sdk.options.PipelineOptionsValidator.validate(PipelineOptionsValidator.java:72)
> >> >> >>>> at
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:306)
> >> >> >>>> at
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:106)
> >> >> >>>> ...
> >> >> >>>>
> >> >> >>>>
> >> >> >>>> I've tried with -Drunner=FlinkRunner and without it and I get
> the
> >> >> >>>> same
> >> >> >>>> error. Moreover, the runner is also set in the source:
> >> >> >>>>
> >> >> >>>> options.setRunner(FlinkRunner.class);
> >> >> >>>>
> >> >> >>>> Any suggestions?
> >> >> >>>>
> >> >> >>>> Best regards,
> >> >> >>>> Dumi
> >> >> >>
> >> >> >>
> >> >> >
> >> >
> >> >
> >
> >
>
Re: Flink Wordcount example on single node "No runner was specified"
Posted by Dumi Loghin <du...@gmail.com>.
Hi,
Here is the entire output of the command (when running mvn -e):
~/incubator-beam/runners/flink/examples$
/home/hadoop/apache-maven-3.3.9/bin/mvn -e exec:java
-Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
-Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt
[INFO] Error stacktraces are turned on.
[INFO] Scanning for projects...
[INFO]
[INFO]
------------------------------------------------------------------------
[INFO] Building Apache Beam :: Runners :: Flink :: Examples
0.2.0-incubating-SNAPSHOT
[INFO]
------------------------------------------------------------------------
[INFO]
[INFO] --- exec-maven-plugin:1.4.0:java (default-cli) @
beam-runners-flink_2.10-examples ---
[WARNING]
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Pipeline execution failed
at
org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:113)
at
org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:48)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:183)
at
org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:117)
... 6 more
Caused by: java.util.concurrent.TimeoutException: Futures timed out after
[10000 milliseconds]
at
scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
at
scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:153)
at scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
at scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
at
scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
at scala.concurrent.Await$.ready(package.scala:86)
at
org.apache.flink.runtime.minicluster.FlinkMiniCluster.shutdown(FlinkMiniCluster.scala:340)
at
org.apache.flink.runtime.minicluster.FlinkMiniCluster.stop(FlinkMiniCluster.scala:319)
at
org.apache.flink.client.LocalExecutor.stop(LocalExecutor.java:126)
at
org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:194)
at
org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:90)
at
org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:116)
at
org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:110)
... 9 more
[INFO]
------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 49.646 s
[INFO] Finished at: 2016-07-28T12:44:39+08:00
[INFO] Final Memory: 34M/451M
[INFO]
------------------------------------------------------------------------
[ERROR] Failed to execute goal
org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project
beam-runners-flink_2.10-examples: An exception occured while executing the
Java class. null: InvocationTargetException: Pipeline execution failed:
Futures timed out after [10000 milliseconds] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on
project beam-runners-flink_2.10-examples: An exception occured while
executing the Java class. null
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at
org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at
org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at
org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at
org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: An exception
occured while executing the Java class. null
at
org.codehaus.mojo.exec.ExecJavaMojo.execute(ExecJavaMojo.java:345)
at
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
... 20 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Pipeline execution failed
at
org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:113)
at
org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:48)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:183)
at
org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:117)
... 6 more
Caused by: java.util.concurrent.TimeoutException: Futures timed out after
[10000 milliseconds]
at
scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
at
scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:153)
at scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
at scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
at
scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
at scala.concurrent.Await$.ready(package.scala:86)
at
org.apache.flink.runtime.minicluster.FlinkMiniCluster.shutdown(FlinkMiniCluster.scala:340)
at
org.apache.flink.runtime.minicluster.FlinkMiniCluster.stop(FlinkMiniCluster.scala:319)
at
org.apache.flink.client.LocalExecutor.stop(LocalExecutor.java:126)
at
org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:194)
at
org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:90)
at
org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:116)
at
org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:110)
... 9 more
[ERROR]
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions,
please read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
Best regards,
Dumi
On Wed, Jul 27, 2016 at 7:48 PM, Maximilian Michels <mx...@apache.org> wrote:
> Okay great!
>
> I think it was a permission problem before. It shouldn't matter where
> the file is as long as Flink can access it. Still puzzled about the
> error message, was that really the entire stack trace?
>
> On Wed, Jul 27, 2016 at 7:05 AM, Dumi Loghin <du...@gmail.com> wrote:
> > Hi,
> >
> > The application is running if the input file is in the examples folder:
> >
> > ~/incubator-beam/runners/flink/examples$
> > /home/hadoop/apache-maven-3.3.9/bin/mvn exec:java
> > -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
> > -Dinput=kinglear.txt -Doutput=wc-out.txt
> >
> > ( as opposed to "~/incubator-beam/runners/flink/examples$
> > /home/hadoop/apache-maven-3.3.9/bin/mvn exec:java
> > -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
> > -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt" that is throwing
> the
> > exception)
> >
> > Best regards,
> > Dumi
> >
> >
> >
> > On Tue, Jul 26, 2016 at 11:53 PM, Maximilian Michels <mx...@apache.org>
> wrote:
> >>
> >> Hi Dumi,
> >>
> >> The execution fails during shutdown of the local Flink cluster. Does
> >> the program execute before the exception occurs?
> >>
> >> Best,
> >> Max
> >>
> >> On Tue, Jul 26, 2016 at 5:15 AM, Dumi Loghin <du...@gmail.com>
> wrote:
> >> > Hi,
> >> >
> >> > Thank you!
> >> >
> >> > I have manually added WordCount.java and pom.xml to the latest Beam
> code
> >> > and
> >> > try to run it. But now I'm getting the following exception:
> >> >
> >> > :~/incubator-beam/runners/flink/examples$
> >> > /home/hadoop/apache-maven-3.3.9/bin/mvn exec:java
> >> > -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
> >> > -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt
> >> >
> >> > java.lang.reflect.InvocationTargetException
> >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> > at
> >> >
> >> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >> > at
> >> >
> >> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> > at java.lang.reflect.Method.invoke(Method.java:606)
> >> > at
> >> > org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
> >> > at java.lang.Thread.run(Thread.java:745)
> >> > Caused by: java.lang.RuntimeException: Pipeline execution failed
> >> > at
> >> > org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:113)
> >> > at
> >> > org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:48)
> >> > at org.apache.beam.sdk.Pipeline.run(Pipeline.java:183)
> >> > at
> >> >
> >> >
> org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:117)
> >> > ... 6 more
> >> > Caused by: java.util.concurrent.TimeoutException: Futures timed out
> >> > after
> >> > [10000 milliseconds]
> >> > at
> >> > scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
> >> > at
> >> > scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:153)
> >> > at
> >> > scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
> >> > at
> >> > scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
> >> > at
> >> >
> >> >
> scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
> >> > at scala.concurrent.Await$.ready(package.scala:86)
> >> > at
> >> >
> >> >
> org.apache.flink.runtime.minicluster.FlinkMiniCluster.shutdown(FlinkMiniCluster.scala:340)
> >> > at
> >> >
> >> >
> org.apache.flink.runtime.minicluster.FlinkMiniCluster.stop(FlinkMiniCluster.scala:319)
> >> > at
> >> > org.apache.flink.client.LocalExecutor.stop(LocalExecutor.java:126)
> >> > at
> >> >
> >> >
> org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:194)
> >> > at
> >> >
> >> >
> org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:90)
> >> > at
> >> >
> >> >
> org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:116)
> >> > at
> >> > org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:110)
> >> > ... 9 more
> >> >
> >> > Best regards,
> >> > Dumi
> >> >
> >> > On Mon, Jul 25, 2016 at 7:14 PM, Maximilian Michels <mx...@apache.org>
> >> > wrote:
> >> >>
> >> >> Hi Dumi,
> >> >>
> >> >> Somewhere on the way we broke the exec configuration. I've fixed this
> >> >> in this PR and updated the README:
> >> >> https://github.com/apache/incubator-beam/pull/724
> >> >>
> >> >> Should be merged soon.
> >> >>
> >> >> Best,
> >> >> Max
> >> >>
> >> >> On Fri, Jul 22, 2016 at 2:49 PM, Lukasz Cwik <lc...@google.com>
> wrote:
> >> >> > I think runner should be org.apache.beam.runners.flink.FlinkRunner
> >> >> > and
> >> >> > not
> >> >> > org.apache.beam.runners.flink.FlinkRunner.class but that is
> probably
> >> >> > unrelated to the failure that you are seeing.
> >> >> >
> >> >> > On Fri, Jul 22, 2016 at 7:42 AM, Dumi Loghin <dumiloghin@gmail.com
> >
> >> >> > wrote:
> >> >> >>
> >> >> >> Hi,
> >> >> >>
> >> >> >> I'm still getting an exception related to arguments. I mention
> that
> >> >> >> I'm
> >> >> >> working with commit adb472083460bc175a0685133eadb6a336f1a4a2 (22
> >> >> >> Jul).
> >> >> >>
> >> >> >> ~/incubator-beam/runners/flink/examples$ mvn exec:java
> >> >> >> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
> >> >> >>
> >> >> >>
> -Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
> >> >> >> --inputFile=/home/hadoop/kinglear.txt --output=wc-out.txt"
> >> >> >> ...
> >> >> >> [ERROR] Failed to execute goal
> >> >> >> org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on
> >> >> >> project
> >> >> >> beam-runners-flink_2.10-examples: Unable to parse configuration of
> >> >> >> mojo
> >> >> >> org.codehaus.mojo:exec-maven-plugin:1.4.0:java for parameter
> >> >> >> arguments:
> >> >> >> Cannot store value into array: ArrayStoreException -> [Help 1]
> >> >> >> ...
> >> >> >>
> >> >> >> However, I was able to run it in early June. If I revert to, for
> >> >> >> example,
> >> >> >> commit c2146b9f9d6a1f39a5699725ccb51829d751b88a (6 Jun) I'm not
> >> >> >> getting
> >> >> >> the
> >> >> >> above exceptions. Maybe something changed in the options
> (arguments)
> >> >> >> handling code?
> >> >> >>
> >> >> >> Best regards,
> >> >> >> Dumi
> >> >> >>
> >> >> >> On Wed, Jul 20, 2016 at 7:35 PM, Aljoscha Krettek
> >> >> >> <al...@apache.org>
> >> >> >> wrote:
> >> >> >>>
> >> >> >>> I think it should work like this:
> >> >> >>>
> >> >> >>> mvn exec:java \
> >> >> >>>
> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount \
> >> >> >>>
> >> >> >>>
> -Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
> >> >> >>> --inputFile='"$input"' --output='"$outfile_prefix"'"'
> >> >> >>>
> >> >> >>> On Wed, 20 Jul 2016 at 10:47 Dumi Loghin <du...@gmail.com>
> >> >> >>> wrote:
> >> >> >>>>
> >> >> >>>> Hi,
> >> >> >>>>
> >> >> >>>> I'm trying to run Wordcount example on Flink single node but I'm
> >> >> >>>> getting
> >> >> >>>> this error:
> >> >> >>>>
> >> >> >>>> ~/incubator-beam/runners/flink/examples$ mvn exec:exec
> >> >> >>>> -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt
> >> >> >>>> -Drunner=FlinkRunner
> >> >> >>>> ...
> >> >> >>>> Exception in thread "main" java.lang.IllegalArgumentException:
> No
> >> >> >>>> Runner
> >> >> >>>> was specified and the DirectRunner was not found on the
> classpath.
> >> >> >>>> Specify a runner by either:
> >> >> >>>> Explicitly specifying a runner by providing the 'runner'
> >> >> >>>> property
> >> >> >>>> Adding the DirectRunner to the classpath
> >> >> >>>> Calling 'PipelineOptions.setRunner(PipelineRunner)' directly
> >> >> >>>> at
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:282)
> >> >> >>>> at
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:273)
> >> >> >>>> at
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> org.apache.beam.sdk.options.ProxyInvocationHandler.getDefault(ProxyInvocationHandler.java:482)
> >> >> >>>> at
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> org.apache.beam.sdk.options.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:157)
> >> >> >>>> at
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> org.apache.beam.sdk.options.PipelineOptionsValidator.validate(PipelineOptionsValidator.java:72)
> >> >> >>>> at
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:306)
> >> >> >>>> at
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:106)
> >> >> >>>> ...
> >> >> >>>>
> >> >> >>>>
> >> >> >>>> I've tried with -Drunner=FlinkRunner and without it and I get
> the
> >> >> >>>> same
> >> >> >>>> error. Moreover, the runner is also set in the source:
> >> >> >>>>
> >> >> >>>> options.setRunner(FlinkRunner.class);
> >> >> >>>>
> >> >> >>>> Any suggestions?
> >> >> >>>>
> >> >> >>>> Best regards,
> >> >> >>>> Dumi
> >> >> >>
> >> >> >>
> >> >> >
> >> >
> >> >
> >
> >
>
Re: Flink Wordcount example on single node "No runner was specified"
Posted by Maximilian Michels <mx...@apache.org>.
Okay great!
I think it was a permission problem before. It shouldn't matter where
the file is as long as Flink can access it. Still puzzled about the
error message, was that really the entire stack trace?
On Wed, Jul 27, 2016 at 7:05 AM, Dumi Loghin <du...@gmail.com> wrote:
> Hi,
>
> The application is running if the input file is in the examples folder:
>
> ~/incubator-beam/runners/flink/examples$
> /home/hadoop/apache-maven-3.3.9/bin/mvn exec:java
> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
> -Dinput=kinglear.txt -Doutput=wc-out.txt
>
> ( as opposed to "~/incubator-beam/runners/flink/examples$
> /home/hadoop/apache-maven-3.3.9/bin/mvn exec:java
> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
> -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt" that is throwing the
> exception)
>
> Best regards,
> Dumi
>
>
>
> On Tue, Jul 26, 2016 at 11:53 PM, Maximilian Michels <mx...@apache.org> wrote:
>>
>> Hi Dumi,
>>
>> The execution fails during shutdown of the local Flink cluster. Does
>> the program execute before the exception occurs?
>>
>> Best,
>> Max
>>
>> On Tue, Jul 26, 2016 at 5:15 AM, Dumi Loghin <du...@gmail.com> wrote:
>> > Hi,
>> >
>> > Thank you!
>> >
>> > I have manually added WordCount.java and pom.xml to the latest Beam code
>> > and
>> > try to run it. But now I'm getting the following exception:
>> >
>> > :~/incubator-beam/runners/flink/examples$
>> > /home/hadoop/apache-maven-3.3.9/bin/mvn exec:java
>> > -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
>> > -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt
>> >
>> > java.lang.reflect.InvocationTargetException
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > at
>> >
>> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> > at
>> >
>> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> > at java.lang.reflect.Method.invoke(Method.java:606)
>> > at
>> > org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
>> > at java.lang.Thread.run(Thread.java:745)
>> > Caused by: java.lang.RuntimeException: Pipeline execution failed
>> > at
>> > org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:113)
>> > at
>> > org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:48)
>> > at org.apache.beam.sdk.Pipeline.run(Pipeline.java:183)
>> > at
>> >
>> > org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:117)
>> > ... 6 more
>> > Caused by: java.util.concurrent.TimeoutException: Futures timed out
>> > after
>> > [10000 milliseconds]
>> > at
>> > scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
>> > at
>> > scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:153)
>> > at
>> > scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
>> > at
>> > scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
>> > at
>> >
>> > scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
>> > at scala.concurrent.Await$.ready(package.scala:86)
>> > at
>> >
>> > org.apache.flink.runtime.minicluster.FlinkMiniCluster.shutdown(FlinkMiniCluster.scala:340)
>> > at
>> >
>> > org.apache.flink.runtime.minicluster.FlinkMiniCluster.stop(FlinkMiniCluster.scala:319)
>> > at
>> > org.apache.flink.client.LocalExecutor.stop(LocalExecutor.java:126)
>> > at
>> >
>> > org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:194)
>> > at
>> >
>> > org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:90)
>> > at
>> >
>> > org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:116)
>> > at
>> > org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:110)
>> > ... 9 more
>> >
>> > Best regards,
>> > Dumi
>> >
>> > On Mon, Jul 25, 2016 at 7:14 PM, Maximilian Michels <mx...@apache.org>
>> > wrote:
>> >>
>> >> Hi Dumi,
>> >>
>> >> Somewhere on the way we broke the exec configuration. I've fixed this
>> >> in this PR and updated the README:
>> >> https://github.com/apache/incubator-beam/pull/724
>> >>
>> >> Should be merged soon.
>> >>
>> >> Best,
>> >> Max
>> >>
>> >> On Fri, Jul 22, 2016 at 2:49 PM, Lukasz Cwik <lc...@google.com> wrote:
>> >> > I think runner should be org.apache.beam.runners.flink.FlinkRunner
>> >> > and
>> >> > not
>> >> > org.apache.beam.runners.flink.FlinkRunner.class but that is probably
>> >> > unrelated to the failure that you are seeing.
>> >> >
>> >> > On Fri, Jul 22, 2016 at 7:42 AM, Dumi Loghin <du...@gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> Hi,
>> >> >>
>> >> >> I'm still getting an exception related to arguments. I mention that
>> >> >> I'm
>> >> >> working with commit adb472083460bc175a0685133eadb6a336f1a4a2 (22
>> >> >> Jul).
>> >> >>
>> >> >> ~/incubator-beam/runners/flink/examples$ mvn exec:java
>> >> >> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
>> >> >>
>> >> >> -Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
>> >> >> --inputFile=/home/hadoop/kinglear.txt --output=wc-out.txt"
>> >> >> ...
>> >> >> [ERROR] Failed to execute goal
>> >> >> org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on
>> >> >> project
>> >> >> beam-runners-flink_2.10-examples: Unable to parse configuration of
>> >> >> mojo
>> >> >> org.codehaus.mojo:exec-maven-plugin:1.4.0:java for parameter
>> >> >> arguments:
>> >> >> Cannot store value into array: ArrayStoreException -> [Help 1]
>> >> >> ...
>> >> >>
>> >> >> However, I was able to run it in early June. If I revert to, for
>> >> >> example,
>> >> >> commit c2146b9f9d6a1f39a5699725ccb51829d751b88a (6 Jun) I'm not
>> >> >> getting
>> >> >> the
>> >> >> above exceptions. Maybe something changed in the options (arguments)
>> >> >> handling code?
>> >> >>
>> >> >> Best regards,
>> >> >> Dumi
>> >> >>
>> >> >> On Wed, Jul 20, 2016 at 7:35 PM, Aljoscha Krettek
>> >> >> <al...@apache.org>
>> >> >> wrote:
>> >> >>>
>> >> >>> I think it should work like this:
>> >> >>>
>> >> >>> mvn exec:java \
>> >> >>> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount \
>> >> >>>
>> >> >>> -Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
>> >> >>> --inputFile='"$input"' --output='"$outfile_prefix"'"'
>> >> >>>
>> >> >>> On Wed, 20 Jul 2016 at 10:47 Dumi Loghin <du...@gmail.com>
>> >> >>> wrote:
>> >> >>>>
>> >> >>>> Hi,
>> >> >>>>
>> >> >>>> I'm trying to run Wordcount example on Flink single node but I'm
>> >> >>>> getting
>> >> >>>> this error:
>> >> >>>>
>> >> >>>> ~/incubator-beam/runners/flink/examples$ mvn exec:exec
>> >> >>>> -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt
>> >> >>>> -Drunner=FlinkRunner
>> >> >>>> ...
>> >> >>>> Exception in thread "main" java.lang.IllegalArgumentException: No
>> >> >>>> Runner
>> >> >>>> was specified and the DirectRunner was not found on the classpath.
>> >> >>>> Specify a runner by either:
>> >> >>>> Explicitly specifying a runner by providing the 'runner'
>> >> >>>> property
>> >> >>>> Adding the DirectRunner to the classpath
>> >> >>>> Calling 'PipelineOptions.setRunner(PipelineRunner)' directly
>> >> >>>> at
>> >> >>>>
>> >> >>>>
>> >> >>>> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:282)
>> >> >>>> at
>> >> >>>>
>> >> >>>>
>> >> >>>> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:273)
>> >> >>>> at
>> >> >>>>
>> >> >>>>
>> >> >>>> org.apache.beam.sdk.options.ProxyInvocationHandler.getDefault(ProxyInvocationHandler.java:482)
>> >> >>>> at
>> >> >>>>
>> >> >>>>
>> >> >>>> org.apache.beam.sdk.options.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:157)
>> >> >>>> at
>> >> >>>>
>> >> >>>>
>> >> >>>> org.apache.beam.sdk.options.PipelineOptionsValidator.validate(PipelineOptionsValidator.java:72)
>> >> >>>> at
>> >> >>>>
>> >> >>>>
>> >> >>>> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:306)
>> >> >>>> at
>> >> >>>>
>> >> >>>>
>> >> >>>> org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:106)
>> >> >>>> ...
>> >> >>>>
>> >> >>>>
>> >> >>>> I've tried with -Drunner=FlinkRunner and without it and I get the
>> >> >>>> same
>> >> >>>> error. Moreover, the runner is also set in the source:
>> >> >>>>
>> >> >>>> options.setRunner(FlinkRunner.class);
>> >> >>>>
>> >> >>>> Any suggestions?
>> >> >>>>
>> >> >>>> Best regards,
>> >> >>>> Dumi
>> >> >>
>> >> >>
>> >> >
>> >
>> >
>
>
Re: Flink Wordcount example on single node "No runner was specified"
Posted by Dumi Loghin <du...@gmail.com>.
Hi,
The application is running if the input file is in the examples folder:
~/incubator-beam/runners/flink/examples$
/home/hadoop/apache-maven-3.3.9/bin/mvn exec:java
-Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
*-Dinput=kinglear.txt* -Doutput=wc-out.txt
( as opposed to "~/incubator-beam/runners/flink/examples$
/home/hadoop/apache-maven-3.3.9/bin/mvn exec:java
-Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
*-Dinput=/home/hadoop/kinglear.txt* -Doutput=wc-out.txt" that is throwing
the exception)
Best regards,
Dumi
On Tue, Jul 26, 2016 at 11:53 PM, Maximilian Michels <mx...@apache.org> wrote:
> Hi Dumi,
>
> The execution fails during shutdown of the local Flink cluster. Does
> the program execute before the exception occurs?
>
> Best,
> Max
>
> On Tue, Jul 26, 2016 at 5:15 AM, Dumi Loghin <du...@gmail.com> wrote:
> > Hi,
> >
> > Thank you!
> >
> > I have manually added WordCount.java and pom.xml to the latest Beam code
> and
> > try to run it. But now I'm getting the following exception:
> >
> > :~/incubator-beam/runners/flink/examples$
> > /home/hadoop/apache-maven-3.3.9/bin/mvn exec:java
> > -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
> > -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt
> >
> > java.lang.reflect.InvocationTargetException
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at
> org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
> > at java.lang.Thread.run(Thread.java:745)
> > Caused by: java.lang.RuntimeException: Pipeline execution failed
> > at
> > org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:113)
> > at
> > org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:48)
> > at org.apache.beam.sdk.Pipeline.run(Pipeline.java:183)
> > at
> > org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:117)
> > ... 6 more
> > Caused by: java.util.concurrent.TimeoutException: Futures timed out after
> > [10000 milliseconds]
> > at
> > scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
> > at
> > scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:153)
> > at
> scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
> > at
> scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
> > at
> >
> scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
> > at scala.concurrent.Await$.ready(package.scala:86)
> > at
> >
> org.apache.flink.runtime.minicluster.FlinkMiniCluster.shutdown(FlinkMiniCluster.scala:340)
> > at
> >
> org.apache.flink.runtime.minicluster.FlinkMiniCluster.stop(FlinkMiniCluster.scala:319)
> > at
> > org.apache.flink.client.LocalExecutor.stop(LocalExecutor.java:126)
> > at
> > org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:194)
> > at
> >
> org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:90)
> > at
> >
> org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:116)
> > at
> > org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:110)
> > ... 9 more
> >
> > Best regards,
> > Dumi
> >
> > On Mon, Jul 25, 2016 at 7:14 PM, Maximilian Michels <mx...@apache.org>
> wrote:
> >>
> >> Hi Dumi,
> >>
> >> Somewhere on the way we broke the exec configuration. I've fixed this
> >> in this PR and updated the README:
> >> https://github.com/apache/incubator-beam/pull/724
> >>
> >> Should be merged soon.
> >>
> >> Best,
> >> Max
> >>
> >> On Fri, Jul 22, 2016 at 2:49 PM, Lukasz Cwik <lc...@google.com> wrote:
> >> > I think runner should be org.apache.beam.runners.flink.FlinkRunner and
> >> > not
> >> > org.apache.beam.runners.flink.FlinkRunner.class but that is probably
> >> > unrelated to the failure that you are seeing.
> >> >
> >> > On Fri, Jul 22, 2016 at 7:42 AM, Dumi Loghin <du...@gmail.com>
> >> > wrote:
> >> >>
> >> >> Hi,
> >> >>
> >> >> I'm still getting an exception related to arguments. I mention that
> I'm
> >> >> working with commit adb472083460bc175a0685133eadb6a336f1a4a2 (22
> Jul).
> >> >>
> >> >> ~/incubator-beam/runners/flink/examples$ mvn exec:java
> >> >> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
> >> >> -Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
> >> >> --inputFile=/home/hadoop/kinglear.txt --output=wc-out.txt"
> >> >> ...
> >> >> [ERROR] Failed to execute goal
> >> >> org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on
> project
> >> >> beam-runners-flink_2.10-examples: Unable to parse configuration of
> mojo
> >> >> org.codehaus.mojo:exec-maven-plugin:1.4.0:java for parameter
> arguments:
> >> >> Cannot store value into array: ArrayStoreException -> [Help 1]
> >> >> ...
> >> >>
> >> >> However, I was able to run it in early June. If I revert to, for
> >> >> example,
> >> >> commit c2146b9f9d6a1f39a5699725ccb51829d751b88a (6 Jun) I'm not
> getting
> >> >> the
> >> >> above exceptions. Maybe something changed in the options (arguments)
> >> >> handling code?
> >> >>
> >> >> Best regards,
> >> >> Dumi
> >> >>
> >> >> On Wed, Jul 20, 2016 at 7:35 PM, Aljoscha Krettek <
> aljoscha@apache.org>
> >> >> wrote:
> >> >>>
> >> >>> I think it should work like this:
> >> >>>
> >> >>> mvn exec:java \
> >> >>> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount \
> >> >>>
> -Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
> >> >>> --inputFile='"$input"' --output='"$outfile_prefix"'"'
> >> >>>
> >> >>> On Wed, 20 Jul 2016 at 10:47 Dumi Loghin <du...@gmail.com>
> wrote:
> >> >>>>
> >> >>>> Hi,
> >> >>>>
> >> >>>> I'm trying to run Wordcount example on Flink single node but I'm
> >> >>>> getting
> >> >>>> this error:
> >> >>>>
> >> >>>> ~/incubator-beam/runners/flink/examples$ mvn exec:exec
> >> >>>> -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt
> >> >>>> -Drunner=FlinkRunner
> >> >>>> ...
> >> >>>> Exception in thread "main" java.lang.IllegalArgumentException: No
> >> >>>> Runner
> >> >>>> was specified and the DirectRunner was not found on the classpath.
> >> >>>> Specify a runner by either:
> >> >>>> Explicitly specifying a runner by providing the 'runner'
> property
> >> >>>> Adding the DirectRunner to the classpath
> >> >>>> Calling 'PipelineOptions.setRunner(PipelineRunner)' directly
> >> >>>> at
> >> >>>>
> >> >>>>
> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:282)
> >> >>>> at
> >> >>>>
> >> >>>>
> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:273)
> >> >>>> at
> >> >>>>
> >> >>>>
> org.apache.beam.sdk.options.ProxyInvocationHandler.getDefault(ProxyInvocationHandler.java:482)
> >> >>>> at
> >> >>>>
> >> >>>>
> org.apache.beam.sdk.options.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:157)
> >> >>>> at
> >> >>>>
> >> >>>>
> org.apache.beam.sdk.options.PipelineOptionsValidator.validate(PipelineOptionsValidator.java:72)
> >> >>>> at
> >> >>>>
> >> >>>>
> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:306)
> >> >>>> at
> >> >>>>
> >> >>>>
> org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:106)
> >> >>>> ...
> >> >>>>
> >> >>>>
> >> >>>> I've tried with -Drunner=FlinkRunner and without it and I get the
> >> >>>> same
> >> >>>> error. Moreover, the runner is also set in the source:
> >> >>>>
> >> >>>> options.setRunner(FlinkRunner.class);
> >> >>>>
> >> >>>> Any suggestions?
> >> >>>>
> >> >>>> Best regards,
> >> >>>> Dumi
> >> >>
> >> >>
> >> >
> >
> >
>
Re: Flink Wordcount example on single node "No runner was specified"
Posted by Maximilian Michels <mx...@apache.org>.
Hi Dumi,
The execution fails during shutdown of the local Flink cluster. Does
the program execute before the exception occurs?
Best,
Max
On Tue, Jul 26, 2016 at 5:15 AM, Dumi Loghin <du...@gmail.com> wrote:
> Hi,
>
> Thank you!
>
> I have manually added WordCount.java and pom.xml to the latest Beam code and
> try to run it. But now I'm getting the following exception:
>
> :~/incubator-beam/runners/flink/examples$
> /home/hadoop/apache-maven-3.3.9/bin/mvn exec:java
> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
> -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt
>
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: Pipeline execution failed
> at
> org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:113)
> at
> org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:48)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:183)
> at
> org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:117)
> ... 6 more
> Caused by: java.util.concurrent.TimeoutException: Futures timed out after
> [10000 milliseconds]
> at
> scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
> at
> scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:153)
> at scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
> at scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
> at
> scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
> at scala.concurrent.Await$.ready(package.scala:86)
> at
> org.apache.flink.runtime.minicluster.FlinkMiniCluster.shutdown(FlinkMiniCluster.scala:340)
> at
> org.apache.flink.runtime.minicluster.FlinkMiniCluster.stop(FlinkMiniCluster.scala:319)
> at
> org.apache.flink.client.LocalExecutor.stop(LocalExecutor.java:126)
> at
> org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:194)
> at
> org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:90)
> at
> org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:116)
> at
> org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:110)
> ... 9 more
>
> Best regards,
> Dumi
>
> On Mon, Jul 25, 2016 at 7:14 PM, Maximilian Michels <mx...@apache.org> wrote:
>>
>> Hi Dumi,
>>
>> Somewhere on the way we broke the exec configuration. I've fixed this
>> in this PR and updated the README:
>> https://github.com/apache/incubator-beam/pull/724
>>
>> Should be merged soon.
>>
>> Best,
>> Max
>>
>> On Fri, Jul 22, 2016 at 2:49 PM, Lukasz Cwik <lc...@google.com> wrote:
>> > I think runner should be org.apache.beam.runners.flink.FlinkRunner and
>> > not
>> > org.apache.beam.runners.flink.FlinkRunner.class but that is probably
>> > unrelated to the failure that you are seeing.
>> >
>> > On Fri, Jul 22, 2016 at 7:42 AM, Dumi Loghin <du...@gmail.com>
>> > wrote:
>> >>
>> >> Hi,
>> >>
>> >> I'm still getting an exception related to arguments. I mention that I'm
>> >> working with commit adb472083460bc175a0685133eadb6a336f1a4a2 (22 Jul).
>> >>
>> >> ~/incubator-beam/runners/flink/examples$ mvn exec:java
>> >> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
>> >> -Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
>> >> --inputFile=/home/hadoop/kinglear.txt --output=wc-out.txt"
>> >> ...
>> >> [ERROR] Failed to execute goal
>> >> org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project
>> >> beam-runners-flink_2.10-examples: Unable to parse configuration of mojo
>> >> org.codehaus.mojo:exec-maven-plugin:1.4.0:java for parameter arguments:
>> >> Cannot store value into array: ArrayStoreException -> [Help 1]
>> >> ...
>> >>
>> >> However, I was able to run it in early June. If I revert to, for
>> >> example,
>> >> commit c2146b9f9d6a1f39a5699725ccb51829d751b88a (6 Jun) I'm not getting
>> >> the
>> >> above exceptions. Maybe something changed in the options (arguments)
>> >> handling code?
>> >>
>> >> Best regards,
>> >> Dumi
>> >>
>> >> On Wed, Jul 20, 2016 at 7:35 PM, Aljoscha Krettek <al...@apache.org>
>> >> wrote:
>> >>>
>> >>> I think it should work like this:
>> >>>
>> >>> mvn exec:java \
>> >>> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount \
>> >>> -Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
>> >>> --inputFile='"$input"' --output='"$outfile_prefix"'"'
>> >>>
>> >>> On Wed, 20 Jul 2016 at 10:47 Dumi Loghin <du...@gmail.com> wrote:
>> >>>>
>> >>>> Hi,
>> >>>>
>> >>>> I'm trying to run Wordcount example on Flink single node but I'm
>> >>>> getting
>> >>>> this error:
>> >>>>
>> >>>> ~/incubator-beam/runners/flink/examples$ mvn exec:exec
>> >>>> -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt
>> >>>> -Drunner=FlinkRunner
>> >>>> ...
>> >>>> Exception in thread "main" java.lang.IllegalArgumentException: No
>> >>>> Runner
>> >>>> was specified and the DirectRunner was not found on the classpath.
>> >>>> Specify a runner by either:
>> >>>> Explicitly specifying a runner by providing the 'runner' property
>> >>>> Adding the DirectRunner to the classpath
>> >>>> Calling 'PipelineOptions.setRunner(PipelineRunner)' directly
>> >>>> at
>> >>>>
>> >>>> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:282)
>> >>>> at
>> >>>>
>> >>>> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:273)
>> >>>> at
>> >>>>
>> >>>> org.apache.beam.sdk.options.ProxyInvocationHandler.getDefault(ProxyInvocationHandler.java:482)
>> >>>> at
>> >>>>
>> >>>> org.apache.beam.sdk.options.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:157)
>> >>>> at
>> >>>>
>> >>>> org.apache.beam.sdk.options.PipelineOptionsValidator.validate(PipelineOptionsValidator.java:72)
>> >>>> at
>> >>>>
>> >>>> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:306)
>> >>>> at
>> >>>>
>> >>>> org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:106)
>> >>>> ...
>> >>>>
>> >>>>
>> >>>> I've tried with -Drunner=FlinkRunner and without it and I get the
>> >>>> same
>> >>>> error. Moreover, the runner is also set in the source:
>> >>>>
>> >>>> options.setRunner(FlinkRunner.class);
>> >>>>
>> >>>> Any suggestions?
>> >>>>
>> >>>> Best regards,
>> >>>> Dumi
>> >>
>> >>
>> >
>
>
Re: Flink Wordcount example on single node "No runner was specified"
Posted by Dumi Loghin <du...@gmail.com>.
Hi,
Thank you!
I have manually added WordCount.java and pom.xml to the latest Beam code
and try to run it. But now I'm getting the following exception:
:~/incubator-beam/runners/flink/examples$
/home/hadoop/apache-maven-3.3.9/bin/mvn exec:java
-Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
-Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Pipeline execution failed
at
org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:113)
at
org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:48)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:183)
at
org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:117)
... 6 more
Caused by: java.util.concurrent.TimeoutException: Futures timed out after
[10000 milliseconds]
at
scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
at
scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:153)
at scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
at scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
at
scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
at scala.concurrent.Await$.ready(package.scala:86)
at
org.apache.flink.runtime.minicluster.FlinkMiniCluster.shutdown(FlinkMiniCluster.scala:340)
at
org.apache.flink.runtime.minicluster.FlinkMiniCluster.stop(FlinkMiniCluster.scala:319)
at
org.apache.flink.client.LocalExecutor.stop(LocalExecutor.java:126)
at
org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:194)
at
org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:90)
at
org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:116)
at
org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:110)
... 9 more
Best regards,
Dumi
On Mon, Jul 25, 2016 at 7:14 PM, Maximilian Michels <mx...@apache.org> wrote:
> Hi Dumi,
>
> Somewhere on the way we broke the exec configuration. I've fixed this
> in this PR and updated the README:
> https://github.com/apache/incubator-beam/pull/724
>
> Should be merged soon.
>
> Best,
> Max
>
> On Fri, Jul 22, 2016 at 2:49 PM, Lukasz Cwik <lc...@google.com> wrote:
> > I think runner should be org.apache.beam.runners.flink.FlinkRunner and
> not
> > org.apache.beam.runners.flink.FlinkRunner.class but that is probably
> > unrelated to the failure that you are seeing.
> >
> > On Fri, Jul 22, 2016 at 7:42 AM, Dumi Loghin <du...@gmail.com>
> wrote:
> >>
> >> Hi,
> >>
> >> I'm still getting an exception related to arguments. I mention that I'm
> >> working with commit adb472083460bc175a0685133eadb6a336f1a4a2 (22 Jul).
> >>
> >> ~/incubator-beam/runners/flink/examples$ mvn exec:java
> >> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
> >> -Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
> >> --inputFile=/home/hadoop/kinglear.txt --output=wc-out.txt"
> >> ...
> >> [ERROR] Failed to execute goal
> >> org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project
> >> beam-runners-flink_2.10-examples: Unable to parse configuration of mojo
> >> org.codehaus.mojo:exec-maven-plugin:1.4.0:java for parameter arguments:
> >> Cannot store value into array: ArrayStoreException -> [Help 1]
> >> ...
> >>
> >> However, I was able to run it in early June. If I revert to, for
> example,
> >> commit c2146b9f9d6a1f39a5699725ccb51829d751b88a (6 Jun) I'm not getting
> the
> >> above exceptions. Maybe something changed in the options (arguments)
> >> handling code?
> >>
> >> Best regards,
> >> Dumi
> >>
> >> On Wed, Jul 20, 2016 at 7:35 PM, Aljoscha Krettek <al...@apache.org>
> >> wrote:
> >>>
> >>> I think it should work like this:
> >>>
> >>> mvn exec:java \
> >>> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount \
> >>> -Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
> >>> --inputFile='"$input"' --output='"$outfile_prefix"'"'
> >>>
> >>> On Wed, 20 Jul 2016 at 10:47 Dumi Loghin <du...@gmail.com> wrote:
> >>>>
> >>>> Hi,
> >>>>
> >>>> I'm trying to run Wordcount example on Flink single node but I'm
> getting
> >>>> this error:
> >>>>
> >>>> ~/incubator-beam/runners/flink/examples$ mvn exec:exec
> >>>> -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt
> -Drunner=FlinkRunner
> >>>> ...
> >>>> Exception in thread "main" java.lang.IllegalArgumentException: No
> Runner
> >>>> was specified and the DirectRunner was not found on the classpath.
> >>>> Specify a runner by either:
> >>>> Explicitly specifying a runner by providing the 'runner' property
> >>>> Adding the DirectRunner to the classpath
> >>>> Calling 'PipelineOptions.setRunner(PipelineRunner)' directly
> >>>> at
> >>>>
> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:282)
> >>>> at
> >>>>
> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:273)
> >>>> at
> >>>>
> org.apache.beam.sdk.options.ProxyInvocationHandler.getDefault(ProxyInvocationHandler.java:482)
> >>>> at
> >>>>
> org.apache.beam.sdk.options.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:157)
> >>>> at
> >>>>
> org.apache.beam.sdk.options.PipelineOptionsValidator.validate(PipelineOptionsValidator.java:72)
> >>>> at
> >>>>
> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:306)
> >>>> at
> >>>>
> org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:106)
> >>>> ...
> >>>>
> >>>>
> >>>> I've tried with -Drunner=FlinkRunner and without it and I get the same
> >>>> error. Moreover, the runner is also set in the source:
> >>>>
> >>>> options.setRunner(FlinkRunner.class);
> >>>>
> >>>> Any suggestions?
> >>>>
> >>>> Best regards,
> >>>> Dumi
> >>
> >>
> >
>
Re: Flink Wordcount example on single node "No runner was specified"
Posted by Maximilian Michels <mx...@apache.org>.
Hi Dumi,
Somewhere on the way we broke the exec configuration. I've fixed this
in this PR and updated the README:
https://github.com/apache/incubator-beam/pull/724
Should be merged soon.
Best,
Max
On Fri, Jul 22, 2016 at 2:49 PM, Lukasz Cwik <lc...@google.com> wrote:
> I think runner should be org.apache.beam.runners.flink.FlinkRunner and not
> org.apache.beam.runners.flink.FlinkRunner.class but that is probably
> unrelated to the failure that you are seeing.
>
> On Fri, Jul 22, 2016 at 7:42 AM, Dumi Loghin <du...@gmail.com> wrote:
>>
>> Hi,
>>
>> I'm still getting an exception related to arguments. I mention that I'm
>> working with commit adb472083460bc175a0685133eadb6a336f1a4a2 (22 Jul).
>>
>> ~/incubator-beam/runners/flink/examples$ mvn exec:java
>> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
>> -Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
>> --inputFile=/home/hadoop/kinglear.txt --output=wc-out.txt"
>> ...
>> [ERROR] Failed to execute goal
>> org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project
>> beam-runners-flink_2.10-examples: Unable to parse configuration of mojo
>> org.codehaus.mojo:exec-maven-plugin:1.4.0:java for parameter arguments:
>> Cannot store value into array: ArrayStoreException -> [Help 1]
>> ...
>>
>> However, I was able to run it in early June. If I revert to, for example,
>> commit c2146b9f9d6a1f39a5699725ccb51829d751b88a (6 Jun) I'm not getting the
>> above exceptions. Maybe something changed in the options (arguments)
>> handling code?
>>
>> Best regards,
>> Dumi
>>
>> On Wed, Jul 20, 2016 at 7:35 PM, Aljoscha Krettek <al...@apache.org>
>> wrote:
>>>
>>> I think it should work like this:
>>>
>>> mvn exec:java \
>>> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount \
>>> -Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
>>> --inputFile='"$input"' --output='"$outfile_prefix"'"'
>>>
>>> On Wed, 20 Jul 2016 at 10:47 Dumi Loghin <du...@gmail.com> wrote:
>>>>
>>>> Hi,
>>>>
>>>> I'm trying to run Wordcount example on Flink single node but I'm getting
>>>> this error:
>>>>
>>>> ~/incubator-beam/runners/flink/examples$ mvn exec:exec
>>>> -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt -Drunner=FlinkRunner
>>>> ...
>>>> Exception in thread "main" java.lang.IllegalArgumentException: No Runner
>>>> was specified and the DirectRunner was not found on the classpath.
>>>> Specify a runner by either:
>>>> Explicitly specifying a runner by providing the 'runner' property
>>>> Adding the DirectRunner to the classpath
>>>> Calling 'PipelineOptions.setRunner(PipelineRunner)' directly
>>>> at
>>>> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:282)
>>>> at
>>>> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:273)
>>>> at
>>>> org.apache.beam.sdk.options.ProxyInvocationHandler.getDefault(ProxyInvocationHandler.java:482)
>>>> at
>>>> org.apache.beam.sdk.options.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:157)
>>>> at
>>>> org.apache.beam.sdk.options.PipelineOptionsValidator.validate(PipelineOptionsValidator.java:72)
>>>> at
>>>> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:306)
>>>> at
>>>> org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:106)
>>>> ...
>>>>
>>>>
>>>> I've tried with -Drunner=FlinkRunner and without it and I get the same
>>>> error. Moreover, the runner is also set in the source:
>>>>
>>>> options.setRunner(FlinkRunner.class);
>>>>
>>>> Any suggestions?
>>>>
>>>> Best regards,
>>>> Dumi
>>
>>
>
Re: Flink Wordcount example on single node "No runner was specified"
Posted by Lukasz Cwik <lc...@google.com>.
I think runner should be org.apache.beam.runners.flink.FlinkRunner and not
org.apache.beam.runners.flink.FlinkRunner.class but that is probably
unrelated to the failure that you are seeing.
On Fri, Jul 22, 2016 at 7:42 AM, Dumi Loghin <du...@gmail.com> wrote:
> Hi,
>
> I'm still getting an exception related to arguments. I mention that I'm
> working with commit adb472083460bc175a0685133eadb6a336f1a4a2 (22 Jul).
>
> ~/incubator-beam/runners/flink/examples$ mvn exec:java
> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
> -Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
> --inputFile=/home/hadoop/kinglear.txt --output=wc-out.txt"
> ...
> [ERROR] Failed to execute goal
> org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project
> beam-runners-flink_2.10-examples: Unable to parse configuration of mojo
> org.codehaus.mojo:exec-maven-plugin:1.4.0:java for parameter arguments:
> Cannot store value into array: ArrayStoreException -> [Help 1]
> ...
>
> However, I was able to run it in early June. If I revert to, for example,
> commit c2146b9f9d6a1f39a5699725ccb51829d751b88a (6 Jun) I'm not getting the
> above exceptions. Maybe something changed in the options (arguments)
> handling code?
>
> Best regards,
> Dumi
>
> On Wed, Jul 20, 2016 at 7:35 PM, Aljoscha Krettek <al...@apache.org>
> wrote:
>
>> I think it should work like this:
>>
>> mvn exec:java \
>> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount \
>> -Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
>> --inputFile='"$input"' --output='"$outfile_prefix"'"'
>>
>> On Wed, 20 Jul 2016 at 10:47 Dumi Loghin <du...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I'm trying to run Wordcount example on Flink single node but I'm getting
>>> this error:
>>>
>>> ~/incubator-beam/runners/flink/examples$ mvn exec:exec
>>> -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt -Drunner=FlinkRunner
>>> ...
>>> Exception in thread "main" java.lang.IllegalArgumentException: No Runner
>>> was specified and the DirectRunner was not found on the classpath.
>>> Specify a runner by either:
>>> Explicitly specifying a runner by providing the 'runner' property
>>> Adding the DirectRunner to the classpath
>>> Calling 'PipelineOptions.setRunner(PipelineRunner)' directly
>>> at
>>> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:282)
>>> at
>>> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:273)
>>> at
>>> org.apache.beam.sdk.options.ProxyInvocationHandler.getDefault(ProxyInvocationHandler.java:482)
>>> at
>>> org.apache.beam.sdk.options.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:157)
>>> at
>>> org.apache.beam.sdk.options.PipelineOptionsValidator.validate(PipelineOptionsValidator.java:72)
>>> at
>>> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:306)
>>> at
>>> org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:106)
>>> ...
>>>
>>>
>>> I've tried with -Drunner=FlinkRunner and without it and I get the same
>>> error. Moreover, the runner is also set in the source:
>>>
>>> options.setRunner(FlinkRunner.class);
>>>
>>> Any suggestions?
>>>
>>> Best regards,
>>> Dumi
>>>
>>
>
Re: Flink Wordcount example on single node "No runner was specified"
Posted by Dumi Loghin <du...@gmail.com>.
Hi,
I'm still getting an exception related to arguments. I mention that I'm
working with commit adb472083460bc175a0685133eadb6a336f1a4a2 (22 Jul).
~/incubator-beam/runners/flink/examples$ mvn exec:java
-Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
-Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
--inputFile=/home/hadoop/kinglear.txt --output=wc-out.txt"
...
[ERROR] Failed to execute goal
org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project
beam-runners-flink_2.10-examples: Unable to parse configuration of mojo
org.codehaus.mojo:exec-maven-plugin:1.4.0:java for parameter arguments:
Cannot store value into array: ArrayStoreException -> [Help 1]
...
However, I was able to run it in early June. If I revert to, for example,
commit c2146b9f9d6a1f39a5699725ccb51829d751b88a (6 Jun) I'm not getting the
above exceptions. Maybe something changed in the options (arguments)
handling code?
Best regards,
Dumi
On Wed, Jul 20, 2016 at 7:35 PM, Aljoscha Krettek <al...@apache.org>
wrote:
> I think it should work like this:
>
> mvn exec:java \
> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount \
> -Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
> --inputFile='"$input"' --output='"$outfile_prefix"'"'
>
> On Wed, 20 Jul 2016 at 10:47 Dumi Loghin <du...@gmail.com> wrote:
>
>> Hi,
>>
>> I'm trying to run Wordcount example on Flink single node but I'm getting
>> this error:
>>
>> ~/incubator-beam/runners/flink/examples$ mvn exec:exec
>> -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt -Drunner=FlinkRunner
>> ...
>> Exception in thread "main" java.lang.IllegalArgumentException: No Runner
>> was specified and the DirectRunner was not found on the classpath.
>> Specify a runner by either:
>> Explicitly specifying a runner by providing the 'runner' property
>> Adding the DirectRunner to the classpath
>> Calling 'PipelineOptions.setRunner(PipelineRunner)' directly
>> at
>> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:282)
>> at
>> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:273)
>> at
>> org.apache.beam.sdk.options.ProxyInvocationHandler.getDefault(ProxyInvocationHandler.java:482)
>> at
>> org.apache.beam.sdk.options.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:157)
>> at
>> org.apache.beam.sdk.options.PipelineOptionsValidator.validate(PipelineOptionsValidator.java:72)
>> at
>> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:306)
>> at
>> org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:106)
>> ...
>>
>>
>> I've tried with -Drunner=FlinkRunner and without it and I get the same
>> error. Moreover, the runner is also set in the source:
>>
>> options.setRunner(FlinkRunner.class);
>>
>> Any suggestions?
>>
>> Best regards,
>> Dumi
>>
>
Re: Flink Wordcount example on single node "No runner was specified"
Posted by Aljoscha Krettek <al...@apache.org>.
I think it should work like this:
mvn exec:java \
-Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount \
-Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
--inputFile='"$input"' --output='"$outfile_prefix"'"'
On Wed, 20 Jul 2016 at 10:47 Dumi Loghin <du...@gmail.com> wrote:
> Hi,
>
> I'm trying to run Wordcount example on Flink single node but I'm getting
> this error:
>
> ~/incubator-beam/runners/flink/examples$ mvn exec:exec
> -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt -Drunner=FlinkRunner
> ...
> Exception in thread "main" java.lang.IllegalArgumentException: No Runner
> was specified and the DirectRunner was not found on the classpath.
> Specify a runner by either:
> Explicitly specifying a runner by providing the 'runner' property
> Adding the DirectRunner to the classpath
> Calling 'PipelineOptions.setRunner(PipelineRunner)' directly
> at
> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:282)
> at
> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:273)
> at
> org.apache.beam.sdk.options.ProxyInvocationHandler.getDefault(ProxyInvocationHandler.java:482)
> at
> org.apache.beam.sdk.options.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:157)
> at
> org.apache.beam.sdk.options.PipelineOptionsValidator.validate(PipelineOptionsValidator.java:72)
> at
> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:306)
> at
> org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:106)
> ...
>
>
> I've tried with -Drunner=FlinkRunner and without it and I get the same
> error. Moreover, the runner is also set in the source:
>
> options.setRunner(FlinkRunner.class);
>
> Any suggestions?
>
> Best regards,
> Dumi
>