You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Adrian Mocanu <am...@verticalscope.com> on 2014/02/12 23:43:27 UTC
Spark Release 0.9.0 missing org.apache.spark.streaming package +
misleading documentation on
http://spark.incubator.apache.org/releases/spark-release-0-9-0.html
I pull l the following version of spark via sbt
"org.apache.spark" % "spark-core_2.10" % "0.9.0-incubating"
Which according to the green button at the bottom of this page http://spark.incubator.apache.org/streaming/ should have streaming.
On this page: http://spark.incubator.apache.org/releases/spark-release-0-9-0.html
It talks about improvements to spark:
"A few aspects of the API have been improved:
* DStream and PairDStream classes have been moved from org.apache.spark.streaming to org.apache.spark.streaming.dstream to keep it consistent with org.apache.spark.rdd.RDD.
* DStream.foreach has been renamed to foreachRDD to make it explicit that it works for every RDD, not every element"
These package don't exist in spark anymore. See my attachment. Where did it go? I also can't find org.apache.spark.streaming.StreamingContext
-Adrian
RE: Spark Release 0.9.0 missing org.apache.spark.streaming package
+ misleading documentation on
http://spark.incubator.apache.org/releases/spark-release-0-9-0.html
Posted by Adrian Mocanu <am...@verticalscope.com>.
Thank you guys. Including "org.apache.spark" % "spark-streaming_2.10" % "0.9.0-incubating" worked.
My confusion came from the fact that on the page (http://spark.incubator.apache.org/streaming/ ) on the big green button at the bottom it says "Download Spark includes Spark Streaming"
I think it would help to change the text on the button.
-A
From: Evan R. Sparks [mailto:evan.sparks@gmail.com]
Sent: February-12-14 6:03 PM
To: user@spark.incubator.apache.org
Subject: Re: Spark Release 0.9.0 missing org.apache.spark.streaming package + misleading documentation on http://spark.incubator.apache.org/releases/spark-release-0-9-0.html
Ah, good catch Aaron - my bad!
On Wed, Feb 12, 2014 at 2:50 PM, Aaron Davidson <il...@gmail.com>> wrote:
Be sure to link against the 0.9.0 version, though! Here are the updated docs:
https://spark.incubator.apache.org/docs/0.9.0/streaming-programming-guide.html#basics
On Wed, Feb 12, 2014 at 2:48 PM, Evan R. Sparks <ev...@gmail.com>> wrote:
You'll need to link against the spark-streaming artifact as well - see the linking section here:
https://spark.incubator.apache.org/docs/0.8.1/streaming-programming-guide.html
On Wed, Feb 12, 2014 at 2:43 PM, Adrian Mocanu <am...@verticalscope.com>> wrote:
I pull l the following version of spark via sbt
"org.apache.spark" % "spark-core_2.10" % "0.9.0-incubating"
Which according to the green button at the bottom of this page http://spark.incubator.apache.org/streaming/ should have streaming.
On this page: http://spark.incubator.apache.org/releases/spark-release-0-9-0.html
It talks about improvements to spark:
"A few aspects of the API have been improved:
* DStream and PairDStream classes have been moved from org.apache.spark.streaming to org.apache.spark.streaming.dstream to keep it consistent with org.apache.spark.rdd.RDD.
* DStream.foreach has been renamed to foreachRDD to make it explicit that it works for every RDD, not every element"
These package don't exist in spark anymore. See my attachment. Where did it go? I also can't find org.apache.spark.streaming.StreamingContext
-Adrian
Re: Spark Release 0.9.0 missing org.apache.spark.streaming package +
misleading documentation on http://spark.incubator.apache.org/releases/spark-release-0-9-0.html
Posted by "Evan R. Sparks" <ev...@gmail.com>.
Ah, good catch Aaron - my bad!
On Wed, Feb 12, 2014 at 2:50 PM, Aaron Davidson <il...@gmail.com> wrote:
> Be sure to link against the 0.9.0 version, though! Here are the updated
> docs:
>
> https://spark.incubator.apache.org/docs/0.9.0/streaming-programming-guide.html#basics
>
>
>
> On Wed, Feb 12, 2014 at 2:48 PM, Evan R. Sparks <ev...@gmail.com>wrote:
>
>> You'll need to link against the spark-streaming artifact as well - see
>> the linking section here:
>>
>> https://spark.incubator.apache.org/docs/0.8.1/streaming-programming-guide.html
>>
>>
>> On Wed, Feb 12, 2014 at 2:43 PM, Adrian Mocanu <amocanu@verticalscope.com
>> > wrote:
>>
>>> I pull l the following version of spark via sbt
>>>
>>> "org.apache.spark" % "spark-core_2.10" % "0.9.0-incubating"
>>>
>>> Which according to the green button at the bottom of this page
>>> http://spark.incubator.apache.org/streaming/ should have streaming.
>>>
>>>
>>>
>>>
>>>
>>> On this page:
>>> http://spark.incubator.apache.org/releases/spark-release-0-9-0.html
>>>
>>> It talks about improvements to spark:
>>>
>>> "A few aspects of the API have been improved:
>>>
>>> - DStream and PairDStream classes have been moved from
>>> org.apache.spark.streaming to org.apache.spark.streaming.dstream to
>>> keep it consistent with org.apache.spark.rdd.RDD.
>>> - DStream.foreach has been renamed to foreachRDD to make it explicit
>>> that it works for every RDD, not every element"
>>>
>>> *These package don't exist in spark anymore. *See my attachment. *Where
>>> did it go?* I also can't find
>>> *org.apache.spark.streaming.StreamingContext*
>>>
>>>
>>>
>>>
>>>
>>> -Adrian
>>>
>>>
>>>
>>
>>
>
Re: Spark Release 0.9.0 missing org.apache.spark.streaming package +
misleading documentation on http://spark.incubator.apache.org/releases/spark-release-0-9-0.html
Posted by Aaron Davidson <il...@gmail.com>.
Be sure to link against the 0.9.0 version, though! Here are the updated
docs:
https://spark.incubator.apache.org/docs/0.9.0/streaming-programming-guide.html#basics
On Wed, Feb 12, 2014 at 2:48 PM, Evan R. Sparks <ev...@gmail.com>wrote:
> You'll need to link against the spark-streaming artifact as well - see the
> linking section here:
>
> https://spark.incubator.apache.org/docs/0.8.1/streaming-programming-guide.html
>
>
> On Wed, Feb 12, 2014 at 2:43 PM, Adrian Mocanu <am...@verticalscope.com>wrote:
>
>> I pull l the following version of spark via sbt
>>
>> "org.apache.spark" % "spark-core_2.10" % "0.9.0-incubating"
>>
>> Which according to the green button at the bottom of this page
>> http://spark.incubator.apache.org/streaming/ should have streaming.
>>
>>
>>
>>
>>
>> On this page:
>> http://spark.incubator.apache.org/releases/spark-release-0-9-0.html
>>
>> It talks about improvements to spark:
>>
>> "A few aspects of the API have been improved:
>>
>> - DStream and PairDStream classes have been moved from
>> org.apache.spark.streaming to org.apache.spark.streaming.dstream to
>> keep it consistent with org.apache.spark.rdd.RDD.
>> - DStream.foreach has been renamed to foreachRDD to make it explicit
>> that it works for every RDD, not every element"
>>
>> *These package don't exist in spark anymore. *See my attachment. *Where
>> did it go?* I also can't find
>> *org.apache.spark.streaming.StreamingContext*
>>
>>
>>
>>
>>
>> -Adrian
>>
>>
>>
>
>
Re: Spark Release 0.9.0 missing org.apache.spark.streaming package +
misleading documentation on http://spark.incubator.apache.org/releases/spark-release-0-9-0.html
Posted by "Evan R. Sparks" <ev...@gmail.com>.
You'll need to link against the spark-streaming artifact as well - see the
linking section here:
https://spark.incubator.apache.org/docs/0.8.1/streaming-programming-guide.html
On Wed, Feb 12, 2014 at 2:43 PM, Adrian Mocanu <am...@verticalscope.com>wrote:
> I pull l the following version of spark via sbt
>
> "org.apache.spark" % "spark-core_2.10" % "0.9.0-incubating"
>
> Which according to the green button at the bottom of this page
> http://spark.incubator.apache.org/streaming/ should have streaming.
>
>
>
>
>
> On this page:
> http://spark.incubator.apache.org/releases/spark-release-0-9-0.html
>
> It talks about improvements to spark:
>
> "A few aspects of the API have been improved:
>
> - DStream and PairDStream classes have been moved from
> org.apache.spark.streaming to org.apache.spark.streaming.dstream to
> keep it consistent with org.apache.spark.rdd.RDD.
> - DStream.foreach has been renamed to foreachRDD to make it explicit
> that it works for every RDD, not every element"
>
> *These package don't exist in spark anymore. *See my attachment. *Where
> did it go?* I also can't find
> *org.apache.spark.streaming.StreamingContext*
>
>
>
>
>
> -Adrian
>
>
>
Re: Spark Release 0.9.0 missing org.apache.spark.streaming package +
misleading documentation on http://spark.incubator.apache.org/releases/spark-release-0-9-0.html
Posted by Mark Hamstra <ma...@clearstorydata.com>.
>
> *These package don't exist in spark anymore.*
They never did exist in spark-core. If your project depends on
spark-streaming, then it should pull in the spark-core dependency as well.
If you only depend on spark-core, then you won't get streaming if you
don't also ask for it. Similarly for other packages within Spark that
depend on spark-core.
On Wed, Feb 12, 2014 at 2:43 PM, Adrian Mocanu <am...@verticalscope.com>wrote:
> I pull l the following version of spark via sbt
>
> "org.apache.spark" % "spark-core_2.10" % "0.9.0-incubating"
>
> Which according to the green button at the bottom of this page
> http://spark.incubator.apache.org/streaming/ should have streaming.
>
>
>
>
>
> On this page:
> http://spark.incubator.apache.org/releases/spark-release-0-9-0.html
>
> It talks about improvements to spark:
>
> "A few aspects of the API have been improved:
>
> - DStream and PairDStream classes have been moved from
> org.apache.spark.streaming to org.apache.spark.streaming.dstream to
> keep it consistent with org.apache.spark.rdd.RDD.
> - DStream.foreach has been renamed to foreachRDD to make it explicit
> that it works for every RDD, not every element"
>
> *These package don't exist in spark anymore. *See my attachment. *Where
> did it go?* I also can't find
> *org.apache.spark.streaming.StreamingContext*
>
>
>
>
>
> -Adrian
>
>
>