You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by David Swearingen <ds...@centrifugesystems.com> on 2014/02/10 17:49:35 UTC
"Failed to find Spark examples assembly..." error message
I'm running 0.9.0 and attempting to try the example described here:
https://spark.incubator.apache.org/docs/0.9.0/streaming-programming-guide.html#a-quick-example
./bin/run-example
org.apache.spark.streaming.examples.JavaNetworkWordCount local[2]
localhost 9999
But get the error:
./bin/run-example: line 47: [:
/home/centrifuge/spark-0.9.0-incubating-bin-hadoop1/examples/target/scala-2.10/spark-examples-assembly-0.9.0-incubating.jar:
binary operator expected
Failed to find Spark examples assembly in
/home/centrifuge/spark-0.9.0-incubating-bin-hadoop1/examples/target
You need to build Spark with sbt/sbt assembly before running this program
I've done the 'sbt/sbt assembly' step. I've tried to debug the shell
script but it seems like the first problem is the missing "assembly" file...
Any suggestions?
Thanks!
______________________________________________________________________
This email has been scanned by the Boundary Defense for Email Security System. For more information please visit http://www.apptix.com/email-security/antispam-virus
______________________________________________________________________
Re: "Failed to find Spark examples assembly..." error message
Posted by Mark Hamstra <ma...@clearstorydata.com>.
Blow away the examples/target directory, then do `./sbt/sbt
examples/assembly` -- you've got two assembly jars where there should be
only one in target/scala-2.10:
spark-examples-assembly-0.9.0-incubating.jar
On Mon, Feb 10, 2014 at 10:06 AM, David Swearingen <
dswearingen@centrifugesystems.com> wrote:
> The file examples-assembly is not there.
>
> I ran 'sbt/sbt examples assembly' and that didn't change anything.
>
> Note that the shell script is complaining about "binary operator expected"
> but I'm not positive that's a cause or a result for some other problem.
>
> Here's what's in the directory:
>
> ls -al examples/target/scala-2.10/
> total 199648
> drwxrwxr-x 4 centrifuge centrifuge 4096 Feb 9 14:51 .
> drwxrwxr-x 8 centrifuge centrifuge 4096 Feb 9 14:48 ..
> drwxrwxr-x 3 centrifuge centrifuge 4096 Feb 9 14:47 cache
> drwxrwxr-x 3 centrifuge centrifuge 4096 Feb 9 14:50 classes
> -rw-rw-r-- 1 centrifuge centrifuge 125932602 Feb 9 14:51
> spark-examples-assembly-0.9.0-incubating.jar
> -rw-rw-r-- 1 centrifuge centrifuge 78485738 Feb 2 19:05
> spark-examples_2.10-assembly-0.9.0-incubating.jar
>
> Thanks again.
>
>
> On Mon Feb 10 12:31:31 2014, Mark Hamstra wrote:
>
>> Is the assembly jar actually missing, or is the script somehow looking
>> in the wrong place? Check
>> /home/centrifuge/spark-0.9.0-incubating-bin-hadoop1/
>> examples/target/scala-2.10.
>> If the examples-assembly jar is not there, then either `./sbt/sbt
>> assembly` or `./sbt/sbt examples/assembly` should build it. Else you
>> need to be looking for why the script is not coming up with the right
>> path to that file.
>>
>>
>> On Mon, Feb 10, 2014 at 8:49 AM, David Swearingen
>> <dswearingen@centrifugesystems.com
>> <ma...@centrifugesystems.com>> wrote:
>>
>> I'm running 0.9.0 and attempting to try the example described
>> here:
>> https://spark.incubator.apache.org/docs/0.9.0/
>> streaming-programming-guide.html#a-quick-example
>>
>>
>> ./bin/run-example
>> org.apache.spark.streaming.examples.JavaNetworkWordCount local[2]
>> localhost 9999
>>
>> But get the error:
>>
>> ./bin/run-example: line 47: [:
>> /home/centrifuge/spark-0.9.0-incubating-bin-hadoop1/
>> examples/target/scala-2.10/spark-examples-assembly-0.9.0-incubating.jar:
>> binary operator expected
>> Failed to find Spark examples assembly in
>> /home/centrifuge/spark-0.9.0-incubating-bin-hadoop1/examples/target
>> You need to build Spark with sbt/sbt assembly before running this
>> program
>>
>> I've done the 'sbt/sbt assembly' step. I've tried to debug the
>> shell script but it seems like the first problem is the missing
>> "assembly" file...
>>
>> Any suggestions?
>>
>> Thanks!
>> ____________________________________________________________
>> __________
>> This email has been scanned by the Boundary Defense for Email
>> Security System. For more information please visit
>> http://www.apptix.com/email-security/antispam-virus
>> ____________________________________________________________
>> __________
>>
>>
>>
>> ______________________________________________________________________
>> This email has been scanned by the Boundary Defense for Email Security
>> System. For more information please visit
>> http://www.apptix.com/email-security/antispam-virus
>> ______________________________________________________________________
>>
>
> --
> ----------------------------------------------
> Centrifuge Systems
> Director, Professional Services
> http://www.centrifugesystems.com
> 7926 Jones Branch Drive McLean, VA 22102
> 703-517-6910 (m)
>
>
> ______________________________________________________________________
> This email has been scanned by the Boundary Defense for Email Security
> System. For more information please visit http://www.apptix.com/email-
> security/antispam-virus
> ______________________________________________________________________
>
Re: "Failed to find Spark examples assembly..." error message
Posted by David Swearingen <ds...@centrifugesystems.com>.
The file examples-assembly is not there.
I ran 'sbt/sbt examples assembly' and that didn't change anything.
Note that the shell script is complaining about "binary operator
expected" but I'm not positive that's a cause or a result for some
other problem.
Here's what's in the directory:
ls -al examples/target/scala-2.10/
total 199648
drwxrwxr-x 4 centrifuge centrifuge 4096 Feb 9 14:51 .
drwxrwxr-x 8 centrifuge centrifuge 4096 Feb 9 14:48 ..
drwxrwxr-x 3 centrifuge centrifuge 4096 Feb 9 14:47 cache
drwxrwxr-x 3 centrifuge centrifuge 4096 Feb 9 14:50 classes
-rw-rw-r-- 1 centrifuge centrifuge 125932602 Feb 9 14:51
spark-examples-assembly-0.9.0-incubating.jar
-rw-rw-r-- 1 centrifuge centrifuge 78485738 Feb 2 19:05
spark-examples_2.10-assembly-0.9.0-incubating.jar
Thanks again.
On Mon Feb 10 12:31:31 2014, Mark Hamstra wrote:
> Is the assembly jar actually missing, or is the script somehow looking
> in the wrong place? Check
> /home/centrifuge/spark-0.9.0-incubating-bin-hadoop1/examples/target/scala-2.10.
> If the examples-assembly jar is not there, then either `./sbt/sbt
> assembly` or `./sbt/sbt examples/assembly` should build it. Else you
> need to be looking for why the script is not coming up with the right
> path to that file.
>
>
> On Mon, Feb 10, 2014 at 8:49 AM, David Swearingen
> <dswearingen@centrifugesystems.com
> <ma...@centrifugesystems.com>> wrote:
>
> I'm running 0.9.0 and attempting to try the example described
> here:
> https://spark.incubator.apache.org/docs/0.9.0/streaming-programming-guide.html#a-quick-example
>
>
> ./bin/run-example
> org.apache.spark.streaming.examples.JavaNetworkWordCount local[2]
> localhost 9999
>
> But get the error:
>
> ./bin/run-example: line 47: [:
> /home/centrifuge/spark-0.9.0-incubating-bin-hadoop1/examples/target/scala-2.10/spark-examples-assembly-0.9.0-incubating.jar:
> binary operator expected
> Failed to find Spark examples assembly in
> /home/centrifuge/spark-0.9.0-incubating-bin-hadoop1/examples/target
> You need to build Spark with sbt/sbt assembly before running this
> program
>
> I've done the 'sbt/sbt assembly' step. I've tried to debug the
> shell script but it seems like the first problem is the missing
> "assembly" file...
>
> Any suggestions?
>
> Thanks!
> ______________________________________________________________________
> This email has been scanned by the Boundary Defense for Email
> Security System. For more information please visit
> http://www.apptix.com/email-security/antispam-virus
> ______________________________________________________________________
>
>
>
> ______________________________________________________________________
> This email has been scanned by the Boundary Defense for Email Security
> System. For more information please visit
> http://www.apptix.com/email-security/antispam-virus
> ______________________________________________________________________
--
----------------------------------------------
Centrifuge Systems
Director, Professional Services
http://www.centrifugesystems.com
7926 Jones Branch Drive McLean, VA 22102
703-517-6910 (m)
______________________________________________________________________
This email has been scanned by the Boundary Defense for Email Security System. For more information please visit http://www.apptix.com/email-security/antispam-virus
______________________________________________________________________
Re: "Failed to find Spark examples assembly..." error message
Posted by Mark Hamstra <ma...@clearstorydata.com>.
Is the assembly jar actually missing, or is the script somehow looking in
the wrong place? Check
/home/centrifuge/spark-0.9.0-incubating-bin-hadoop1/examples/target/scala-2.10.
If the examples-assembly jar is not there, then either `./sbt/sbt
assembly` or `./sbt/sbt examples/assembly` should build it. Else you need
to be looking for why the script is not coming up with the right path to
that file.
On Mon, Feb 10, 2014 at 8:49 AM, David Swearingen <
dswearingen@centrifugesystems.com> wrote:
> I'm running 0.9.0 and attempting to try the example described here:
> https://spark.incubator.apache.org/docs/0.9.0/streaming-programming-guide.html#a-quick-example
>
>
> ./bin/run-example org.apache.spark.streaming.examples.JavaNetworkWordCount
> local[2] localhost 9999
>
> But get the error:
>
> ./bin/run-example: line 47: [:
> /home/centrifuge/spark-0.9.0-incubating-bin-hadoop1/examples/target/scala-2.10/spark-examples-assembly-0.9.0-incubating.jar:
> binary operator expected
> Failed to find Spark examples assembly in
> /home/centrifuge/spark-0.9.0-incubating-bin-hadoop1/examples/target
> You need to build Spark with sbt/sbt assembly before running this program
>
> I've done the 'sbt/sbt assembly' step. I've tried to debug the shell
> script but it seems like the first problem is the missing "assembly" file...
>
> Any suggestions?
>
> Thanks!
> ______________________________________________________________________
> This email has been scanned by the Boundary Defense for Email Security
> System. For more information please visit
> http://www.apptix.com/email-security/antispam-virus
> ______________________________________________________________________
>