You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@beam.apache.org by Punit Naik <na...@gmail.com> on 2016/05/25 07:14:23 UTC
SparkPipelineRunner ERROR
Hi Everyone
I compiled beam project with the command "mvn clean install -DskipTests"
and then ran the WordCount example with the following command:
mvn compile exec:java -pl examples/java
-Dexec.mainClass=org.apache.beam.examples.WordCount
-Dexec.args="--runner=SparkPipelineRunner --sparkMaster=local
--inputFile=/in --output=/beam-out"
But it gave me an error saying:
[ERROR] Failed to execute goal
org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project
java-examples-all: An exception occured while executing the Java class.
null: InvocationTargetException: Unknown 'runner' specified
'SparkPipelineRunner', supported pipeline runners
[BlockingDataflowPipelineRunner, DataflowPipelineRunner,
DirectPipelineRunner] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on
project java-examples-all: An exception occured while executing the Java
class. null
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at
org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at
org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at
org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at
org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: An exception
occured while executing the Java class. null
at org.codehaus.mojo.exec.ExecJavaMojo.execute(ExecJavaMojo.java:345)
at
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
... 20 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Unknown 'runner' specified
'SparkPipelineRunner', supported pipeline runners
[BlockingDataflowPipelineRunner, DataflowPipelineRunner,
DirectPipelineRunner]
at
org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1434)
at
org.apache.beam.sdk.options.PipelineOptionsFactory.access$400(PipelineOptionsFactory.java:107)
at
org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:292)
at org.apache.beam.examples.WordCount.main(WordCount.java:192)
... 6 more
Caused by: java.lang.ClassNotFoundException: SparkPipelineRunner
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at
org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1421)
... 9 more
What is the problem?
--
Thank You
Regards
Punit Naik
Re: SparkPipelineRunner ERROR
Posted by Punit Naik <na...@gmail.com>.
Okay got it. I think i'll have to run using spark-submit.
On Wed, May 25, 2016 at 3:06 PM, Punit Naik <na...@gmail.com> wrote:
> Hi Everyone
>
> I was able to run the example with Spark runner on files present in /tmp
> directory
> . What if I want to use the files from HDFS?
>
> On Wed, May 25, 2016 at 2:04 PM, Punit Naik <na...@gmail.com>
> wrote:
>
>> Hi Ismael
>>
>> I am using maven 3.3.9
>>
>> On Wed, May 25, 2016 at 1:31 PM, Ismaël Mejía <ie...@gmail.com> wrote:
>>
>>> Hi Punit,
>>>
>>> Can you confirm the version of Maven are you using ?
>>>
>>> I ask this because I had some issues building the project with the
>>> native version of maven that came with debian/ubuntu, but since I moved to
>>> the 'real' one I have no problmems. And I just did git pull from master and
>>> everything seems to work ok.
>>>
>>> Ismaël
>>>
>>>
>>> On Wed, May 25, 2016 at 10:00 AM, Amit Sela <am...@gmail.com>
>>> wrote:
>>>
>>>> I'll take a look first chance I get and get back to you.
>>>>
>>>> On Wed, May 25, 2016, 10:57 Punit Naik <na...@gmail.com> wrote:
>>>>
>>>>> Hi Amit
>>>>>
>>>>> Thanks, it ran but got stuck at:
>>>>>
>>>>> [INFO] --- exec-maven-plugin:1.4.0:exec (default-cli) @ spark ---
>>>>> log4j:WARN No appenders could be found for logger
>>>>> (org.apache.beam.sdk.options.PipelineOptionsFactory).
>>>>> log4j:WARN Please initialize the log4j system properly.
>>>>> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
>>>>> for more info.
>>>>>
>>>>>
>>>>> On Wed, May 25, 2016 at 1:17 PM, Amit Sela <am...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I'm not near a computer but are you running from the parent project
>>>>>> dir or the runner ? You need to run from the runner dir.
>>>>>>
>>>>>> On Wed, May 25, 2016, 10:44 Punit Naik <na...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Amit
>>>>>>>
>>>>>>> Just tried that and still got an error:
>>>>>>>
>>>>>>> [ERROR] Failed to execute goal
>>>>>>> org.codehaus.mojo:exec-maven-plugin:1.4.0:exec (default-cli) on project
>>>>>>> parent: The parameters 'executable' for goal
>>>>>>> org.codehaus.mojo:exec-maven-plugin:1.4.0:exec are missing or invalid ->
>>>>>>> [Help 1]
>>>>>>>
>>>>>>> On Wed, May 25, 2016 at 1:01 PM, Amit Sela <am...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Would you try running with mvn exec:exec ?
>>>>>>>> See here:
>>>>>>>> https://github.com/apache/incubator-beam/blob/master/runners/spark/README.md
>>>>>>>>
>>>>>>>> On Wed, May 25, 2016, 10:14 Punit Naik <na...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Everyone
>>>>>>>>>
>>>>>>>>> I compiled beam project with the command "mvn clean install
>>>>>>>>> -DskipTests" and then ran the WordCount example with the following command:
>>>>>>>>>
>>>>>>>>> mvn compile exec:java -pl examples/java
>>>>>>>>> -Dexec.mainClass=org.apache.beam.examples.WordCount
>>>>>>>>> -Dexec.args="--runner=SparkPipelineRunner --sparkMaster=local
>>>>>>>>> --inputFile=/in --output=/beam-out"
>>>>>>>>>
>>>>>>>>> But it gave me an error saying:
>>>>>>>>>
>>>>>>>>> [ERROR] Failed to execute goal
>>>>>>>>> org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project
>>>>>>>>> java-examples-all: An exception occured while executing the Java class.
>>>>>>>>> null: InvocationTargetException: Unknown 'runner' specified
>>>>>>>>> 'SparkPipelineRunner', supported pipeline runners
>>>>>>>>> [BlockingDataflowPipelineRunner, DataflowPipelineRunner,
>>>>>>>>> DirectPipelineRunner] -> [Help 1]
>>>>>>>>> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
>>>>>>>>> execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli)
>>>>>>>>> on project java-examples-all: An exception occured while executing the Java
>>>>>>>>> class. null
>>>>>>>>> at
>>>>>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
>>>>>>>>> at
>>>>>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
>>>>>>>>> at
>>>>>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
>>>>>>>>> at
>>>>>>>>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
>>>>>>>>> at
>>>>>>>>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
>>>>>>>>> at
>>>>>>>>> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
>>>>>>>>> at
>>>>>>>>> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
>>>>>>>>> at
>>>>>>>>> org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
>>>>>>>>> at
>>>>>>>>> org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
>>>>>>>>> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
>>>>>>>>> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
>>>>>>>>> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
>>>>>>>>> at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>>>> at
>>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>>>>>>> at
>>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>>>>>>> at
>>>>>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
>>>>>>>>> at
>>>>>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
>>>>>>>>> at
>>>>>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
>>>>>>>>> at
>>>>>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
>>>>>>>>> Caused by: org.apache.maven.plugin.MojoExecutionException: An
>>>>>>>>> exception occured while executing the Java class. null
>>>>>>>>> at
>>>>>>>>> org.codehaus.mojo.exec.ExecJavaMojo.execute(ExecJavaMojo.java:345)
>>>>>>>>> at
>>>>>>>>> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
>>>>>>>>> at
>>>>>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
>>>>>>>>> ... 20 more
>>>>>>>>> Caused by: java.lang.reflect.InvocationTargetException
>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>>>> at
>>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>>>>>>> at
>>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>>>>>>> at
>>>>>>>>> org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
>>>>>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>>>>>> Caused by: java.lang.IllegalArgumentException: Unknown 'runner'
>>>>>>>>> specified 'SparkPipelineRunner', supported pipeline runners
>>>>>>>>> [BlockingDataflowPipelineRunner, DataflowPipelineRunner,
>>>>>>>>> DirectPipelineRunner]
>>>>>>>>> at
>>>>>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1434)
>>>>>>>>> at
>>>>>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory.access$400(PipelineOptionsFactory.java:107)
>>>>>>>>> at
>>>>>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:292)
>>>>>>>>> at org.apache.beam.examples.WordCount.main(WordCount.java:192)
>>>>>>>>> ... 6 more
>>>>>>>>> Caused by: java.lang.ClassNotFoundException: SparkPipelineRunner
>>>>>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>>>>>>> at java.lang.Class.forName0(Native Method)
>>>>>>>>> at java.lang.Class.forName(Class.java:264)
>>>>>>>>> at
>>>>>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1421)
>>>>>>>>> ... 9 more
>>>>>>>>>
>>>>>>>>> What is the problem?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Thank You
>>>>>>>>>
>>>>>>>>> Regards
>>>>>>>>>
>>>>>>>>> Punit Naik
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Thank You
>>>>>>>
>>>>>>> Regards
>>>>>>>
>>>>>>> Punit Naik
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Thank You
>>>>>
>>>>> Regards
>>>>>
>>>>> Punit Naik
>>>>>
>>>>
>>>
>>
>>
>> --
>> Thank You
>>
>> Regards
>>
>> Punit Naik
>>
>
>
>
> --
> Thank You
>
> Regards
>
> Punit Naik
>
--
Thank You
Regards
Punit Naik
Re: SparkPipelineRunner ERROR
Posted by Punit Naik <na...@gmail.com>.
Hi Everyone
I was able to run the example with Spark runner on files present in /tmp
directory
. What if I want to use the files from HDFS?
On Wed, May 25, 2016 at 2:04 PM, Punit Naik <na...@gmail.com> wrote:
> Hi Ismael
>
> I am using maven 3.3.9
>
> On Wed, May 25, 2016 at 1:31 PM, Ismaël Mejía <ie...@gmail.com> wrote:
>
>> Hi Punit,
>>
>> Can you confirm the version of Maven are you using ?
>>
>> I ask this because I had some issues building the project with the native
>> version of maven that came with debian/ubuntu, but since I moved to the
>> 'real' one I have no problmems. And I just did git pull from master and
>> everything seems to work ok.
>>
>> Ismaël
>>
>>
>> On Wed, May 25, 2016 at 10:00 AM, Amit Sela <am...@gmail.com> wrote:
>>
>>> I'll take a look first chance I get and get back to you.
>>>
>>> On Wed, May 25, 2016, 10:57 Punit Naik <na...@gmail.com> wrote:
>>>
>>>> Hi Amit
>>>>
>>>> Thanks, it ran but got stuck at:
>>>>
>>>> [INFO] --- exec-maven-plugin:1.4.0:exec (default-cli) @ spark ---
>>>> log4j:WARN No appenders could be found for logger
>>>> (org.apache.beam.sdk.options.PipelineOptionsFactory).
>>>> log4j:WARN Please initialize the log4j system properly.
>>>> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
>>>> for more info.
>>>>
>>>>
>>>> On Wed, May 25, 2016 at 1:17 PM, Amit Sela <am...@gmail.com>
>>>> wrote:
>>>>
>>>>> I'm not near a computer but are you running from the parent project
>>>>> dir or the runner ? You need to run from the runner dir.
>>>>>
>>>>> On Wed, May 25, 2016, 10:44 Punit Naik <na...@gmail.com> wrote:
>>>>>
>>>>>> Hi Amit
>>>>>>
>>>>>> Just tried that and still got an error:
>>>>>>
>>>>>> [ERROR] Failed to execute goal
>>>>>> org.codehaus.mojo:exec-maven-plugin:1.4.0:exec (default-cli) on project
>>>>>> parent: The parameters 'executable' for goal
>>>>>> org.codehaus.mojo:exec-maven-plugin:1.4.0:exec are missing or invalid ->
>>>>>> [Help 1]
>>>>>>
>>>>>> On Wed, May 25, 2016 at 1:01 PM, Amit Sela <am...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Would you try running with mvn exec:exec ?
>>>>>>> See here:
>>>>>>> https://github.com/apache/incubator-beam/blob/master/runners/spark/README.md
>>>>>>>
>>>>>>> On Wed, May 25, 2016, 10:14 Punit Naik <na...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Everyone
>>>>>>>>
>>>>>>>> I compiled beam project with the command "mvn clean install
>>>>>>>> -DskipTests" and then ran the WordCount example with the following command:
>>>>>>>>
>>>>>>>> mvn compile exec:java -pl examples/java
>>>>>>>> -Dexec.mainClass=org.apache.beam.examples.WordCount
>>>>>>>> -Dexec.args="--runner=SparkPipelineRunner --sparkMaster=local
>>>>>>>> --inputFile=/in --output=/beam-out"
>>>>>>>>
>>>>>>>> But it gave me an error saying:
>>>>>>>>
>>>>>>>> [ERROR] Failed to execute goal
>>>>>>>> org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project
>>>>>>>> java-examples-all: An exception occured while executing the Java class.
>>>>>>>> null: InvocationTargetException: Unknown 'runner' specified
>>>>>>>> 'SparkPipelineRunner', supported pipeline runners
>>>>>>>> [BlockingDataflowPipelineRunner, DataflowPipelineRunner,
>>>>>>>> DirectPipelineRunner] -> [Help 1]
>>>>>>>> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
>>>>>>>> execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli)
>>>>>>>> on project java-examples-all: An exception occured while executing the Java
>>>>>>>> class. null
>>>>>>>> at
>>>>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
>>>>>>>> at
>>>>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
>>>>>>>> at
>>>>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
>>>>>>>> at
>>>>>>>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
>>>>>>>> at
>>>>>>>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
>>>>>>>> at
>>>>>>>> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
>>>>>>>> at
>>>>>>>> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
>>>>>>>> at
>>>>>>>> org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
>>>>>>>> at
>>>>>>>> org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
>>>>>>>> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
>>>>>>>> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
>>>>>>>> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
>>>>>>>> at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>>> at
>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>>>>>> at
>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>>>>>> at
>>>>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
>>>>>>>> at
>>>>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
>>>>>>>> at
>>>>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
>>>>>>>> at
>>>>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
>>>>>>>> Caused by: org.apache.maven.plugin.MojoExecutionException: An
>>>>>>>> exception occured while executing the Java class. null
>>>>>>>> at
>>>>>>>> org.codehaus.mojo.exec.ExecJavaMojo.execute(ExecJavaMojo.java:345)
>>>>>>>> at
>>>>>>>> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
>>>>>>>> at
>>>>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
>>>>>>>> ... 20 more
>>>>>>>> Caused by: java.lang.reflect.InvocationTargetException
>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>>> at
>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>>>>>> at
>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>>>>>> at
>>>>>>>> org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
>>>>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>>>>> Caused by: java.lang.IllegalArgumentException: Unknown 'runner'
>>>>>>>> specified 'SparkPipelineRunner', supported pipeline runners
>>>>>>>> [BlockingDataflowPipelineRunner, DataflowPipelineRunner,
>>>>>>>> DirectPipelineRunner]
>>>>>>>> at
>>>>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1434)
>>>>>>>> at
>>>>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory.access$400(PipelineOptionsFactory.java:107)
>>>>>>>> at
>>>>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:292)
>>>>>>>> at org.apache.beam.examples.WordCount.main(WordCount.java:192)
>>>>>>>> ... 6 more
>>>>>>>> Caused by: java.lang.ClassNotFoundException: SparkPipelineRunner
>>>>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>>>>>> at java.lang.Class.forName0(Native Method)
>>>>>>>> at java.lang.Class.forName(Class.java:264)
>>>>>>>> at
>>>>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1421)
>>>>>>>> ... 9 more
>>>>>>>>
>>>>>>>> What is the problem?
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Thank You
>>>>>>>>
>>>>>>>> Regards
>>>>>>>>
>>>>>>>> Punit Naik
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Thank You
>>>>>>
>>>>>> Regards
>>>>>>
>>>>>> Punit Naik
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Thank You
>>>>
>>>> Regards
>>>>
>>>> Punit Naik
>>>>
>>>
>>
>
>
> --
> Thank You
>
> Regards
>
> Punit Naik
>
--
Thank You
Regards
Punit Naik
Re: SparkPipelineRunner ERROR
Posted by Punit Naik <na...@gmail.com>.
Hi Ismael
I am using maven 3.3.9
On Wed, May 25, 2016 at 1:31 PM, Ismaël Mejía <ie...@gmail.com> wrote:
> Hi Punit,
>
> Can you confirm the version of Maven are you using ?
>
> I ask this because I had some issues building the project with the native
> version of maven that came with debian/ubuntu, but since I moved to the
> 'real' one I have no problmems. And I just did git pull from master and
> everything seems to work ok.
>
> Ismaël
>
>
> On Wed, May 25, 2016 at 10:00 AM, Amit Sela <am...@gmail.com> wrote:
>
>> I'll take a look first chance I get and get back to you.
>>
>> On Wed, May 25, 2016, 10:57 Punit Naik <na...@gmail.com> wrote:
>>
>>> Hi Amit
>>>
>>> Thanks, it ran but got stuck at:
>>>
>>> [INFO] --- exec-maven-plugin:1.4.0:exec (default-cli) @ spark ---
>>> log4j:WARN No appenders could be found for logger
>>> (org.apache.beam.sdk.options.PipelineOptionsFactory).
>>> log4j:WARN Please initialize the log4j system properly.
>>> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
>>> for more info.
>>>
>>>
>>> On Wed, May 25, 2016 at 1:17 PM, Amit Sela <am...@gmail.com> wrote:
>>>
>>>> I'm not near a computer but are you running from the parent project dir
>>>> or the runner ? You need to run from the runner dir.
>>>>
>>>> On Wed, May 25, 2016, 10:44 Punit Naik <na...@gmail.com> wrote:
>>>>
>>>>> Hi Amit
>>>>>
>>>>> Just tried that and still got an error:
>>>>>
>>>>> [ERROR] Failed to execute goal
>>>>> org.codehaus.mojo:exec-maven-plugin:1.4.0:exec (default-cli) on project
>>>>> parent: The parameters 'executable' for goal
>>>>> org.codehaus.mojo:exec-maven-plugin:1.4.0:exec are missing or invalid ->
>>>>> [Help 1]
>>>>>
>>>>> On Wed, May 25, 2016 at 1:01 PM, Amit Sela <am...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Would you try running with mvn exec:exec ?
>>>>>> See here:
>>>>>> https://github.com/apache/incubator-beam/blob/master/runners/spark/README.md
>>>>>>
>>>>>> On Wed, May 25, 2016, 10:14 Punit Naik <na...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Everyone
>>>>>>>
>>>>>>> I compiled beam project with the command "mvn clean install
>>>>>>> -DskipTests" and then ran the WordCount example with the following command:
>>>>>>>
>>>>>>> mvn compile exec:java -pl examples/java
>>>>>>> -Dexec.mainClass=org.apache.beam.examples.WordCount
>>>>>>> -Dexec.args="--runner=SparkPipelineRunner --sparkMaster=local
>>>>>>> --inputFile=/in --output=/beam-out"
>>>>>>>
>>>>>>> But it gave me an error saying:
>>>>>>>
>>>>>>> [ERROR] Failed to execute goal
>>>>>>> org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project
>>>>>>> java-examples-all: An exception occured while executing the Java class.
>>>>>>> null: InvocationTargetException: Unknown 'runner' specified
>>>>>>> 'SparkPipelineRunner', supported pipeline runners
>>>>>>> [BlockingDataflowPipelineRunner, DataflowPipelineRunner,
>>>>>>> DirectPipelineRunner] -> [Help 1]
>>>>>>> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
>>>>>>> execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli)
>>>>>>> on project java-examples-all: An exception occured while executing the Java
>>>>>>> class. null
>>>>>>> at
>>>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
>>>>>>> at
>>>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
>>>>>>> at
>>>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
>>>>>>> at
>>>>>>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
>>>>>>> at
>>>>>>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
>>>>>>> at
>>>>>>> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
>>>>>>> at
>>>>>>> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
>>>>>>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
>>>>>>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
>>>>>>> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
>>>>>>> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
>>>>>>> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
>>>>>>> at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>> at
>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>>>>> at
>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>>>>> at
>>>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
>>>>>>> at
>>>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
>>>>>>> at
>>>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
>>>>>>> at
>>>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
>>>>>>> Caused by: org.apache.maven.plugin.MojoExecutionException: An
>>>>>>> exception occured while executing the Java class. null
>>>>>>> at
>>>>>>> org.codehaus.mojo.exec.ExecJavaMojo.execute(ExecJavaMojo.java:345)
>>>>>>> at
>>>>>>> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
>>>>>>> at
>>>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
>>>>>>> ... 20 more
>>>>>>> Caused by: java.lang.reflect.InvocationTargetException
>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>> at
>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>>>>> at
>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>>>>> at
>>>>>>> org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
>>>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>>>> Caused by: java.lang.IllegalArgumentException: Unknown 'runner'
>>>>>>> specified 'SparkPipelineRunner', supported pipeline runners
>>>>>>> [BlockingDataflowPipelineRunner, DataflowPipelineRunner,
>>>>>>> DirectPipelineRunner]
>>>>>>> at
>>>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1434)
>>>>>>> at
>>>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory.access$400(PipelineOptionsFactory.java:107)
>>>>>>> at
>>>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:292)
>>>>>>> at org.apache.beam.examples.WordCount.main(WordCount.java:192)
>>>>>>> ... 6 more
>>>>>>> Caused by: java.lang.ClassNotFoundException: SparkPipelineRunner
>>>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>>>>> at java.lang.Class.forName0(Native Method)
>>>>>>> at java.lang.Class.forName(Class.java:264)
>>>>>>> at
>>>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1421)
>>>>>>> ... 9 more
>>>>>>>
>>>>>>> What is the problem?
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Thank You
>>>>>>>
>>>>>>> Regards
>>>>>>>
>>>>>>> Punit Naik
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Thank You
>>>>>
>>>>> Regards
>>>>>
>>>>> Punit Naik
>>>>>
>>>>
>>>
>>>
>>> --
>>> Thank You
>>>
>>> Regards
>>>
>>> Punit Naik
>>>
>>
>
--
Thank You
Regards
Punit Naik
Re: SparkPipelineRunner ERROR
Posted by Ismaël Mejía <ie...@gmail.com>.
Hi Punit,
Can you confirm the version of Maven are you using ?
I ask this because I had some issues building the project with the native
version of maven that came with debian/ubuntu, but since I moved to the
'real' one I have no problmems. And I just did git pull from master and
everything seems to work ok.
Ismaël
On Wed, May 25, 2016 at 10:00 AM, Amit Sela <am...@gmail.com> wrote:
> I'll take a look first chance I get and get back to you.
>
> On Wed, May 25, 2016, 10:57 Punit Naik <na...@gmail.com> wrote:
>
>> Hi Amit
>>
>> Thanks, it ran but got stuck at:
>>
>> [INFO] --- exec-maven-plugin:1.4.0:exec (default-cli) @ spark ---
>> log4j:WARN No appenders could be found for logger
>> (org.apache.beam.sdk.options.PipelineOptionsFactory).
>> log4j:WARN Please initialize the log4j system properly.
>> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
>> more info.
>>
>>
>> On Wed, May 25, 2016 at 1:17 PM, Amit Sela <am...@gmail.com> wrote:
>>
>>> I'm not near a computer but are you running from the parent project dir
>>> or the runner ? You need to run from the runner dir.
>>>
>>> On Wed, May 25, 2016, 10:44 Punit Naik <na...@gmail.com> wrote:
>>>
>>>> Hi Amit
>>>>
>>>> Just tried that and still got an error:
>>>>
>>>> [ERROR] Failed to execute goal
>>>> org.codehaus.mojo:exec-maven-plugin:1.4.0:exec (default-cli) on project
>>>> parent: The parameters 'executable' for goal
>>>> org.codehaus.mojo:exec-maven-plugin:1.4.0:exec are missing or invalid ->
>>>> [Help 1]
>>>>
>>>> On Wed, May 25, 2016 at 1:01 PM, Amit Sela <am...@gmail.com>
>>>> wrote:
>>>>
>>>>> Would you try running with mvn exec:exec ?
>>>>> See here:
>>>>> https://github.com/apache/incubator-beam/blob/master/runners/spark/README.md
>>>>>
>>>>> On Wed, May 25, 2016, 10:14 Punit Naik <na...@gmail.com> wrote:
>>>>>
>>>>>> Hi Everyone
>>>>>>
>>>>>> I compiled beam project with the command "mvn clean install
>>>>>> -DskipTests" and then ran the WordCount example with the following command:
>>>>>>
>>>>>> mvn compile exec:java -pl examples/java
>>>>>> -Dexec.mainClass=org.apache.beam.examples.WordCount
>>>>>> -Dexec.args="--runner=SparkPipelineRunner --sparkMaster=local
>>>>>> --inputFile=/in --output=/beam-out"
>>>>>>
>>>>>> But it gave me an error saying:
>>>>>>
>>>>>> [ERROR] Failed to execute goal
>>>>>> org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project
>>>>>> java-examples-all: An exception occured while executing the Java class.
>>>>>> null: InvocationTargetException: Unknown 'runner' specified
>>>>>> 'SparkPipelineRunner', supported pipeline runners
>>>>>> [BlockingDataflowPipelineRunner, DataflowPipelineRunner,
>>>>>> DirectPipelineRunner] -> [Help 1]
>>>>>> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
>>>>>> execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli)
>>>>>> on project java-examples-all: An exception occured while executing the Java
>>>>>> class. null
>>>>>> at
>>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
>>>>>> at
>>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
>>>>>> at
>>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
>>>>>> at
>>>>>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
>>>>>> at
>>>>>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
>>>>>> at
>>>>>> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
>>>>>> at
>>>>>> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
>>>>>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
>>>>>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
>>>>>> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
>>>>>> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
>>>>>> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
>>>>>> at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>> at
>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>>>> at
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>>>> at
>>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
>>>>>> at
>>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
>>>>>> at
>>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
>>>>>> at
>>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
>>>>>> Caused by: org.apache.maven.plugin.MojoExecutionException: An
>>>>>> exception occured while executing the Java class. null
>>>>>> at
>>>>>> org.codehaus.mojo.exec.ExecJavaMojo.execute(ExecJavaMojo.java:345)
>>>>>> at
>>>>>> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
>>>>>> at
>>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
>>>>>> ... 20 more
>>>>>> Caused by: java.lang.reflect.InvocationTargetException
>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>> at
>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>>>> at
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>>>> at
>>>>>> org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
>>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>>> Caused by: java.lang.IllegalArgumentException: Unknown 'runner'
>>>>>> specified 'SparkPipelineRunner', supported pipeline runners
>>>>>> [BlockingDataflowPipelineRunner, DataflowPipelineRunner,
>>>>>> DirectPipelineRunner]
>>>>>> at
>>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1434)
>>>>>> at
>>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory.access$400(PipelineOptionsFactory.java:107)
>>>>>> at
>>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:292)
>>>>>> at org.apache.beam.examples.WordCount.main(WordCount.java:192)
>>>>>> ... 6 more
>>>>>> Caused by: java.lang.ClassNotFoundException: SparkPipelineRunner
>>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>>>> at java.lang.Class.forName0(Native Method)
>>>>>> at java.lang.Class.forName(Class.java:264)
>>>>>> at
>>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1421)
>>>>>> ... 9 more
>>>>>>
>>>>>> What is the problem?
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Thank You
>>>>>>
>>>>>> Regards
>>>>>>
>>>>>> Punit Naik
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Thank You
>>>>
>>>> Regards
>>>>
>>>> Punit Naik
>>>>
>>>
>>
>>
>> --
>> Thank You
>>
>> Regards
>>
>> Punit Naik
>>
>
Re: SparkPipelineRunner ERROR
Posted by Amit Sela <am...@gmail.com>.
I'll take a look first chance I get and get back to you.
On Wed, May 25, 2016, 10:57 Punit Naik <na...@gmail.com> wrote:
> Hi Amit
>
> Thanks, it ran but got stuck at:
>
> [INFO] --- exec-maven-plugin:1.4.0:exec (default-cli) @ spark ---
> log4j:WARN No appenders could be found for logger
> (org.apache.beam.sdk.options.PipelineOptionsFactory).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
> more info.
>
>
> On Wed, May 25, 2016 at 1:17 PM, Amit Sela <am...@gmail.com> wrote:
>
>> I'm not near a computer but are you running from the parent project dir
>> or the runner ? You need to run from the runner dir.
>>
>> On Wed, May 25, 2016, 10:44 Punit Naik <na...@gmail.com> wrote:
>>
>>> Hi Amit
>>>
>>> Just tried that and still got an error:
>>>
>>> [ERROR] Failed to execute goal
>>> org.codehaus.mojo:exec-maven-plugin:1.4.0:exec (default-cli) on project
>>> parent: The parameters 'executable' for goal
>>> org.codehaus.mojo:exec-maven-plugin:1.4.0:exec are missing or invalid ->
>>> [Help 1]
>>>
>>> On Wed, May 25, 2016 at 1:01 PM, Amit Sela <am...@gmail.com> wrote:
>>>
>>>> Would you try running with mvn exec:exec ?
>>>> See here:
>>>> https://github.com/apache/incubator-beam/blob/master/runners/spark/README.md
>>>>
>>>> On Wed, May 25, 2016, 10:14 Punit Naik <na...@gmail.com> wrote:
>>>>
>>>>> Hi Everyone
>>>>>
>>>>> I compiled beam project with the command "mvn clean install
>>>>> -DskipTests" and then ran the WordCount example with the following command:
>>>>>
>>>>> mvn compile exec:java -pl examples/java
>>>>> -Dexec.mainClass=org.apache.beam.examples.WordCount
>>>>> -Dexec.args="--runner=SparkPipelineRunner --sparkMaster=local
>>>>> --inputFile=/in --output=/beam-out"
>>>>>
>>>>> But it gave me an error saying:
>>>>>
>>>>> [ERROR] Failed to execute goal
>>>>> org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project
>>>>> java-examples-all: An exception occured while executing the Java class.
>>>>> null: InvocationTargetException: Unknown 'runner' specified
>>>>> 'SparkPipelineRunner', supported pipeline runners
>>>>> [BlockingDataflowPipelineRunner, DataflowPipelineRunner,
>>>>> DirectPipelineRunner] -> [Help 1]
>>>>> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
>>>>> execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli)
>>>>> on project java-examples-all: An exception occured while executing the Java
>>>>> class. null
>>>>> at
>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
>>>>> at
>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
>>>>> at
>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
>>>>> at
>>>>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
>>>>> at
>>>>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
>>>>> at
>>>>> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
>>>>> at
>>>>> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
>>>>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
>>>>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
>>>>> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
>>>>> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
>>>>> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
>>>>> at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> at
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>>> at
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>>> at
>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
>>>>> at
>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
>>>>> at
>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
>>>>> at
>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
>>>>> Caused by: org.apache.maven.plugin.MojoExecutionException: An
>>>>> exception occured while executing the Java class. null
>>>>> at
>>>>> org.codehaus.mojo.exec.ExecJavaMojo.execute(ExecJavaMojo.java:345)
>>>>> at
>>>>> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
>>>>> at
>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
>>>>> ... 20 more
>>>>> Caused by: java.lang.reflect.InvocationTargetException
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> at
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>>> at
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>>> at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>> Caused by: java.lang.IllegalArgumentException: Unknown 'runner'
>>>>> specified 'SparkPipelineRunner', supported pipeline runners
>>>>> [BlockingDataflowPipelineRunner, DataflowPipelineRunner,
>>>>> DirectPipelineRunner]
>>>>> at
>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1434)
>>>>> at
>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory.access$400(PipelineOptionsFactory.java:107)
>>>>> at
>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:292)
>>>>> at org.apache.beam.examples.WordCount.main(WordCount.java:192)
>>>>> ... 6 more
>>>>> Caused by: java.lang.ClassNotFoundException: SparkPipelineRunner
>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>>> at java.lang.Class.forName0(Native Method)
>>>>> at java.lang.Class.forName(Class.java:264)
>>>>> at
>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1421)
>>>>> ... 9 more
>>>>>
>>>>> What is the problem?
>>>>>
>>>>>
>>>>> --
>>>>> Thank You
>>>>>
>>>>> Regards
>>>>>
>>>>> Punit Naik
>>>>>
>>>>
>>>
>>>
>>> --
>>> Thank You
>>>
>>> Regards
>>>
>>> Punit Naik
>>>
>>
>
>
> --
> Thank You
>
> Regards
>
> Punit Naik
>
Re: SparkPipelineRunner ERROR
Posted by Punit Naik <na...@gmail.com>.
Hi Amit
Thanks, it ran but got stuck at:
[INFO] --- exec-maven-plugin:1.4.0:exec (default-cli) @ spark ---
log4j:WARN No appenders could be found for logger
(org.apache.beam.sdk.options.PipelineOptionsFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
more info.
On Wed, May 25, 2016 at 1:17 PM, Amit Sela <am...@gmail.com> wrote:
> I'm not near a computer but are you running from the parent project dir or
> the runner ? You need to run from the runner dir.
>
> On Wed, May 25, 2016, 10:44 Punit Naik <na...@gmail.com> wrote:
>
>> Hi Amit
>>
>> Just tried that and still got an error:
>>
>> [ERROR] Failed to execute goal
>> org.codehaus.mojo:exec-maven-plugin:1.4.0:exec (default-cli) on project
>> parent: The parameters 'executable' for goal
>> org.codehaus.mojo:exec-maven-plugin:1.4.0:exec are missing or invalid ->
>> [Help 1]
>>
>> On Wed, May 25, 2016 at 1:01 PM, Amit Sela <am...@gmail.com> wrote:
>>
>>> Would you try running with mvn exec:exec ?
>>> See here:
>>> https://github.com/apache/incubator-beam/blob/master/runners/spark/README.md
>>>
>>> On Wed, May 25, 2016, 10:14 Punit Naik <na...@gmail.com> wrote:
>>>
>>>> Hi Everyone
>>>>
>>>> I compiled beam project with the command "mvn clean install
>>>> -DskipTests" and then ran the WordCount example with the following command:
>>>>
>>>> mvn compile exec:java -pl examples/java
>>>> -Dexec.mainClass=org.apache.beam.examples.WordCount
>>>> -Dexec.args="--runner=SparkPipelineRunner --sparkMaster=local
>>>> --inputFile=/in --output=/beam-out"
>>>>
>>>> But it gave me an error saying:
>>>>
>>>> [ERROR] Failed to execute goal
>>>> org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project
>>>> java-examples-all: An exception occured while executing the Java class.
>>>> null: InvocationTargetException: Unknown 'runner' specified
>>>> 'SparkPipelineRunner', supported pipeline runners
>>>> [BlockingDataflowPipelineRunner, DataflowPipelineRunner,
>>>> DirectPipelineRunner] -> [Help 1]
>>>> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
>>>> execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli)
>>>> on project java-examples-all: An exception occured while executing the Java
>>>> class. null
>>>> at
>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
>>>> at
>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
>>>> at
>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
>>>> at
>>>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
>>>> at
>>>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
>>>> at
>>>> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
>>>> at
>>>> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
>>>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
>>>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
>>>> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
>>>> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
>>>> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
>>>> at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>> at
>>>> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
>>>> at
>>>> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
>>>> at
>>>> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
>>>> at
>>>> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
>>>> Caused by: org.apache.maven.plugin.MojoExecutionException: An exception
>>>> occured while executing the Java class. null
>>>> at
>>>> org.codehaus.mojo.exec.ExecJavaMojo.execute(ExecJavaMojo.java:345)
>>>> at
>>>> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
>>>> at
>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
>>>> ... 20 more
>>>> Caused by: java.lang.reflect.InvocationTargetException
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>> at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
>>>> at java.lang.Thread.run(Thread.java:745)
>>>> Caused by: java.lang.IllegalArgumentException: Unknown 'runner'
>>>> specified 'SparkPipelineRunner', supported pipeline runners
>>>> [BlockingDataflowPipelineRunner, DataflowPipelineRunner,
>>>> DirectPipelineRunner]
>>>> at
>>>> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1434)
>>>> at
>>>> org.apache.beam.sdk.options.PipelineOptionsFactory.access$400(PipelineOptionsFactory.java:107)
>>>> at
>>>> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:292)
>>>> at org.apache.beam.examples.WordCount.main(WordCount.java:192)
>>>> ... 6 more
>>>> Caused by: java.lang.ClassNotFoundException: SparkPipelineRunner
>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>> at java.lang.Class.forName0(Native Method)
>>>> at java.lang.Class.forName(Class.java:264)
>>>> at
>>>> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1421)
>>>> ... 9 more
>>>>
>>>> What is the problem?
>>>>
>>>>
>>>> --
>>>> Thank You
>>>>
>>>> Regards
>>>>
>>>> Punit Naik
>>>>
>>>
>>
>>
>> --
>> Thank You
>>
>> Regards
>>
>> Punit Naik
>>
>
--
Thank You
Regards
Punit Naik
Re: SparkPipelineRunner ERROR
Posted by Amit Sela <am...@gmail.com>.
I'm not near a computer but are you running from the parent project dir or
the runner ? You need to run from the runner dir.
On Wed, May 25, 2016, 10:44 Punit Naik <na...@gmail.com> wrote:
> Hi Amit
>
> Just tried that and still got an error:
>
> [ERROR] Failed to execute goal
> org.codehaus.mojo:exec-maven-plugin:1.4.0:exec (default-cli) on project
> parent: The parameters 'executable' for goal
> org.codehaus.mojo:exec-maven-plugin:1.4.0:exec are missing or invalid ->
> [Help 1]
>
> On Wed, May 25, 2016 at 1:01 PM, Amit Sela <am...@gmail.com> wrote:
>
>> Would you try running with mvn exec:exec ?
>> See here:
>> https://github.com/apache/incubator-beam/blob/master/runners/spark/README.md
>>
>> On Wed, May 25, 2016, 10:14 Punit Naik <na...@gmail.com> wrote:
>>
>>> Hi Everyone
>>>
>>> I compiled beam project with the command "mvn clean install -DskipTests"
>>> and then ran the WordCount example with the following command:
>>>
>>> mvn compile exec:java -pl examples/java
>>> -Dexec.mainClass=org.apache.beam.examples.WordCount
>>> -Dexec.args="--runner=SparkPipelineRunner --sparkMaster=local
>>> --inputFile=/in --output=/beam-out"
>>>
>>> But it gave me an error saying:
>>>
>>> [ERROR] Failed to execute goal
>>> org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project
>>> java-examples-all: An exception occured while executing the Java class.
>>> null: InvocationTargetException: Unknown 'runner' specified
>>> 'SparkPipelineRunner', supported pipeline runners
>>> [BlockingDataflowPipelineRunner, DataflowPipelineRunner,
>>> DirectPipelineRunner] -> [Help 1]
>>> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
>>> execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli)
>>> on project java-examples-all: An exception occured while executing the Java
>>> class. null
>>> at
>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
>>> at
>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
>>> at
>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
>>> at
>>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
>>> at
>>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
>>> at
>>> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
>>> at
>>> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
>>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
>>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
>>> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
>>> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
>>> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
>>> at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>> at
>>> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
>>> at
>>> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
>>> at
>>> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
>>> at
>>> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
>>> Caused by: org.apache.maven.plugin.MojoExecutionException: An exception
>>> occured while executing the Java class. null
>>> at org.codehaus.mojo.exec.ExecJavaMojo.execute(ExecJavaMojo.java:345)
>>> at
>>> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
>>> at
>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
>>> ... 20 more
>>> Caused by: java.lang.reflect.InvocationTargetException
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>> at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
>>> at java.lang.Thread.run(Thread.java:745)
>>> Caused by: java.lang.IllegalArgumentException: Unknown 'runner'
>>> specified 'SparkPipelineRunner', supported pipeline runners
>>> [BlockingDataflowPipelineRunner, DataflowPipelineRunner,
>>> DirectPipelineRunner]
>>> at
>>> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1434)
>>> at
>>> org.apache.beam.sdk.options.PipelineOptionsFactory.access$400(PipelineOptionsFactory.java:107)
>>> at
>>> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:292)
>>> at org.apache.beam.examples.WordCount.main(WordCount.java:192)
>>> ... 6 more
>>> Caused by: java.lang.ClassNotFoundException: SparkPipelineRunner
>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>> at java.lang.Class.forName0(Native Method)
>>> at java.lang.Class.forName(Class.java:264)
>>> at
>>> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1421)
>>> ... 9 more
>>>
>>> What is the problem?
>>>
>>>
>>> --
>>> Thank You
>>>
>>> Regards
>>>
>>> Punit Naik
>>>
>>
>
>
> --
> Thank You
>
> Regards
>
> Punit Naik
>
Re: SparkPipelineRunner ERROR
Posted by Punit Naik <na...@gmail.com>.
Hi Amit
Just tried that and still got an error:
[ERROR] Failed to execute goal
org.codehaus.mojo:exec-maven-plugin:1.4.0:exec (default-cli) on project
parent: The parameters 'executable' for goal
org.codehaus.mojo:exec-maven-plugin:1.4.0:exec are missing or invalid ->
[Help 1]
On Wed, May 25, 2016 at 1:01 PM, Amit Sela <am...@gmail.com> wrote:
> Would you try running with mvn exec:exec ?
> See here:
> https://github.com/apache/incubator-beam/blob/master/runners/spark/README.md
>
> On Wed, May 25, 2016, 10:14 Punit Naik <na...@gmail.com> wrote:
>
>> Hi Everyone
>>
>> I compiled beam project with the command "mvn clean install -DskipTests"
>> and then ran the WordCount example with the following command:
>>
>> mvn compile exec:java -pl examples/java
>> -Dexec.mainClass=org.apache.beam.examples.WordCount
>> -Dexec.args="--runner=SparkPipelineRunner --sparkMaster=local
>> --inputFile=/in --output=/beam-out"
>>
>> But it gave me an error saying:
>>
>> [ERROR] Failed to execute goal
>> org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project
>> java-examples-all: An exception occured while executing the Java class.
>> null: InvocationTargetException: Unknown 'runner' specified
>> 'SparkPipelineRunner', supported pipeline runners
>> [BlockingDataflowPipelineRunner, DataflowPipelineRunner,
>> DirectPipelineRunner] -> [Help 1]
>> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
>> goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on
>> project java-examples-all: An exception occured while executing the Java
>> class. null
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
>> at
>> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
>> at
>> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
>> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
>> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
>> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
>> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
>> at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
>> at
>> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
>> Caused by: org.apache.maven.plugin.MojoExecutionException: An exception
>> occured while executing the Java class. null
>> at org.codehaus.mojo.exec.ExecJavaMojo.execute(ExecJavaMojo.java:345)
>> at
>> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
>> at
>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
>> ... 20 more
>> Caused by: java.lang.reflect.InvocationTargetException
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: java.lang.IllegalArgumentException: Unknown 'runner' specified
>> 'SparkPipelineRunner', supported pipeline runners
>> [BlockingDataflowPipelineRunner, DataflowPipelineRunner,
>> DirectPipelineRunner]
>> at
>> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1434)
>> at
>> org.apache.beam.sdk.options.PipelineOptionsFactory.access$400(PipelineOptionsFactory.java:107)
>> at
>> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:292)
>> at org.apache.beam.examples.WordCount.main(WordCount.java:192)
>> ... 6 more
>> Caused by: java.lang.ClassNotFoundException: SparkPipelineRunner
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>> at java.lang.Class.forName0(Native Method)
>> at java.lang.Class.forName(Class.java:264)
>> at
>> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1421)
>> ... 9 more
>>
>> What is the problem?
>>
>>
>> --
>> Thank You
>>
>> Regards
>>
>> Punit Naik
>>
>
--
Thank You
Regards
Punit Naik
Re: SparkPipelineRunner ERROR
Posted by Amit Sela <am...@gmail.com>.
Would you try running with mvn exec:exec ?
See here:
https://github.com/apache/incubator-beam/blob/master/runners/spark/README.md
On Wed, May 25, 2016, 10:14 Punit Naik <na...@gmail.com> wrote:
> Hi Everyone
>
> I compiled beam project with the command "mvn clean install -DskipTests"
> and then ran the WordCount example with the following command:
>
> mvn compile exec:java -pl examples/java
> -Dexec.mainClass=org.apache.beam.examples.WordCount
> -Dexec.args="--runner=SparkPipelineRunner --sparkMaster=local
> --inputFile=/in --output=/beam-out"
>
> But it gave me an error saying:
>
> [ERROR] Failed to execute goal
> org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project
> java-examples-all: An exception occured while executing the Java class.
> null: InvocationTargetException: Unknown 'runner' specified
> 'SparkPipelineRunner', supported pipeline runners
> [BlockingDataflowPipelineRunner, DataflowPipelineRunner,
> DirectPipelineRunner] -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on
> project java-examples-all: An exception occured while executing the Java
> class. null
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
> at
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
> at
> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
> at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
> at
> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
> Caused by: org.apache.maven.plugin.MojoExecutionException: An exception
> occured while executing the Java class. null
> at org.codehaus.mojo.exec.ExecJavaMojo.execute(ExecJavaMojo.java:345)
> at
> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
> ... 20 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.IllegalArgumentException: Unknown 'runner' specified
> 'SparkPipelineRunner', supported pipeline runners
> [BlockingDataflowPipelineRunner, DataflowPipelineRunner,
> DirectPipelineRunner]
> at
> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1434)
> at
> org.apache.beam.sdk.options.PipelineOptionsFactory.access$400(PipelineOptionsFactory.java:107)
> at
> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:292)
> at org.apache.beam.examples.WordCount.main(WordCount.java:192)
> ... 6 more
> Caused by: java.lang.ClassNotFoundException: SparkPipelineRunner
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:264)
> at
> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1421)
> ... 9 more
>
> What is the problem?
>
>
> --
> Thank You
>
> Regards
>
> Punit Naik
>