You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by naresh Goud <na...@gmail.com> on 2018/02/19 01:17:33 UTC
KafkaUtils.createStream(..) is removed for API
Hello Team,
I see "KafkaUtils.createStream() " method not available in spark 2.2.1.
Can someone please confirm if these methods are removed?
below is my pom.xml entries.
<properties>
<scala.version>2.11.8</scala.version>
<scala.tools.version>2.11</scala.tools.version>
</properties>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_${scala.tools.version}</artifactId>
<version>2.2.1</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming-kafka-0-10_2.11</artifactId>
<version>2.2.1</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.2.1</version>
<scope>provided</scope>
</dependency>
<dependency>
Thank you,
Naresh
Re: KafkaUtils.createStream(..) is removed for API
Posted by Cody Koeninger <co...@koeninger.org>.
I can't speak for committers, but my guess is it's more likely for
DStreams in general to stop being supported before that particular
integration is removed.
On Sun, Feb 18, 2018 at 9:34 PM, naresh Goud <na...@gmail.com> wrote:
> Thanks Ted.
>
> I see createDirectStream is experimental as annotated with
> "org.apache.spark.annotation.Experimental".
>
> Is it possible to be this API will be removed in future? because we wanted
> to use this API in one of our production jobs. afraid if it will not be
> supported in future.
>
> Thank you,
> Naresh
>
>
>
>
> On Sun, Feb 18, 2018 at 7:47 PM, Ted Yu <yu...@gmail.com> wrote:
>>
>> createStream() is still in
>> external/kafka-0-8/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala
>> But it is not in
>> external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaUtils.scala
>>
>> FYI
>>
>> On Sun, Feb 18, 2018 at 5:17 PM, naresh Goud <na...@gmail.com>
>> wrote:
>>>
>>> Hello Team,
>>>
>>> I see "KafkaUtils.createStream() " method not available in spark 2.2.1.
>>>
>>> Can someone please confirm if these methods are removed?
>>>
>>> below is my pom.xml entries.
>>>
>>>
>>> <properties>
>>> <scala.version>2.11.8</scala.version>
>>> <scala.tools.version>2.11</scala.tools.version>
>>> </properties>
>>>
>>>
>>> <dependency>
>>> <groupId>org.apache.spark</groupId>
>>> <artifactId>spark-streaming_${scala.tools.version}</artifactId>
>>> <version>2.2.1</version>
>>> <scope>provided</scope>
>>> </dependency>
>>> <dependency>
>>> <groupId>org.apache.spark</groupId>
>>> <artifactId>spark-streaming-kafka-0-10_2.11</artifactId>
>>> <version>2.2.1</version>
>>> <scope>provided</scope>
>>> </dependency>
>>> <dependency>
>>> <groupId>org.apache.spark</groupId>
>>> <artifactId>spark-core_2.11</artifactId>
>>> <version>2.2.1</version>
>>> <scope>provided</scope>
>>> </dependency>
>>> <dependency>
>>>
>>>
>>>
>>>
>>>
>>> Thank you,
>>> Naresh
>>
>>
>
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org
Re: KafkaUtils.createStream(..) is removed for API
Posted by Cody Koeninger <co...@koeninger.org>.
I can't speak for committers, but my guess is it's more likely for
DStreams in general to stop being supported before that particular
integration is removed.
On Sun, Feb 18, 2018 at 9:34 PM, naresh Goud <na...@gmail.com> wrote:
> Thanks Ted.
>
> I see createDirectStream is experimental as annotated with
> "org.apache.spark.annotation.Experimental".
>
> Is it possible to be this API will be removed in future? because we wanted
> to use this API in one of our production jobs. afraid if it will not be
> supported in future.
>
> Thank you,
> Naresh
>
>
>
>
> On Sun, Feb 18, 2018 at 7:47 PM, Ted Yu <yu...@gmail.com> wrote:
>>
>> createStream() is still in
>> external/kafka-0-8/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala
>> But it is not in
>> external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaUtils.scala
>>
>> FYI
>>
>> On Sun, Feb 18, 2018 at 5:17 PM, naresh Goud <na...@gmail.com>
>> wrote:
>>>
>>> Hello Team,
>>>
>>> I see "KafkaUtils.createStream() " method not available in spark 2.2.1.
>>>
>>> Can someone please confirm if these methods are removed?
>>>
>>> below is my pom.xml entries.
>>>
>>>
>>> <properties>
>>> <scala.version>2.11.8</scala.version>
>>> <scala.tools.version>2.11</scala.tools.version>
>>> </properties>
>>>
>>>
>>> <dependency>
>>> <groupId>org.apache.spark</groupId>
>>> <artifactId>spark-streaming_${scala.tools.version}</artifactId>
>>> <version>2.2.1</version>
>>> <scope>provided</scope>
>>> </dependency>
>>> <dependency>
>>> <groupId>org.apache.spark</groupId>
>>> <artifactId>spark-streaming-kafka-0-10_2.11</artifactId>
>>> <version>2.2.1</version>
>>> <scope>provided</scope>
>>> </dependency>
>>> <dependency>
>>> <groupId>org.apache.spark</groupId>
>>> <artifactId>spark-core_2.11</artifactId>
>>> <version>2.2.1</version>
>>> <scope>provided</scope>
>>> </dependency>
>>> <dependency>
>>>
>>>
>>>
>>>
>>>
>>> Thank you,
>>> Naresh
>>
>>
>
Re: KafkaUtils.createStream(..) is removed for API
Posted by naresh Goud <na...@gmail.com>.
Thanks Ted.
I see createDirectStream is experimental as annotated with
"org.apache.spark.annotation.Experimental".
Is it possible to be this API will be removed in future? because we wanted
to use this API in one of our production jobs. afraid if it will not be
supported in future.
Thank you,
Naresh
On Sun, Feb 18, 2018 at 7:47 PM, Ted Yu <yu...@gmail.com> wrote:
> createStream() is still in external/kafka-0-8/src/main
> /scala/org/apache/spark/streaming/kafka/KafkaUtils.scala
> But it is not in external/kafka-0-10/src/main/scala/org/apache/spark/strea
> ming/kafka010/KafkaUtils.scala
>
> FYI
>
> On Sun, Feb 18, 2018 at 5:17 PM, naresh Goud <na...@gmail.com>
> wrote:
>
>> Hello Team,
>>
>> I see "KafkaUtils.createStream() " method not available in spark 2.2.1.
>>
>> Can someone please confirm if these methods are removed?
>>
>> below is my pom.xml entries.
>>
>>
>> <properties>
>> <scala.version>2.11.8</scala.version>
>> <scala.tools.version>2.11</scala.tools.version>
>> </properties>
>>
>>
>> <dependency>
>> <groupId>org.apache.spark</groupId>
>> <artifactId>spark-streaming_${scala.tools.version}</artifactId>
>> <version>2.2.1</version>
>> <scope>provided</scope>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.spark</groupId>
>> <artifactId>spark-streaming-kafka-0-10_2.11</artifactId>
>> <version>2.2.1</version>
>> <scope>provided</scope>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.spark</groupId>
>> <artifactId>spark-core_2.11</artifactId>
>> <version>2.2.1</version>
>> <scope>provided</scope>
>> </dependency>
>> <dependency>
>>
>>
>>
>>
>>
>> Thank you,
>> Naresh
>>
>
>
Re: KafkaUtils.createStream(..) is removed for API
Posted by naresh Goud <na...@gmail.com>.
Thanks Ted.
I see createDirectStream is experimental as annotated with
"org.apache.spark.annotation.Experimental".
Is it possible to be this API will be removed in future? because we wanted
to use this API in one of our production jobs. afraid if it will not be
supported in future.
Thank you,
Naresh
On Sun, Feb 18, 2018 at 7:47 PM, Ted Yu <yu...@gmail.com> wrote:
> createStream() is still in external/kafka-0-8/src/main
> /scala/org/apache/spark/streaming/kafka/KafkaUtils.scala
> But it is not in external/kafka-0-10/src/main/scala/org/apache/spark/strea
> ming/kafka010/KafkaUtils.scala
>
> FYI
>
> On Sun, Feb 18, 2018 at 5:17 PM, naresh Goud <na...@gmail.com>
> wrote:
>
>> Hello Team,
>>
>> I see "KafkaUtils.createStream() " method not available in spark 2.2.1.
>>
>> Can someone please confirm if these methods are removed?
>>
>> below is my pom.xml entries.
>>
>>
>> <properties>
>> <scala.version>2.11.8</scala.version>
>> <scala.tools.version>2.11</scala.tools.version>
>> </properties>
>>
>>
>> <dependency>
>> <groupId>org.apache.spark</groupId>
>> <artifactId>spark-streaming_${scala.tools.version}</artifactId>
>> <version>2.2.1</version>
>> <scope>provided</scope>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.spark</groupId>
>> <artifactId>spark-streaming-kafka-0-10_2.11</artifactId>
>> <version>2.2.1</version>
>> <scope>provided</scope>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.spark</groupId>
>> <artifactId>spark-core_2.11</artifactId>
>> <version>2.2.1</version>
>> <scope>provided</scope>
>> </dependency>
>> <dependency>
>>
>>
>>
>>
>>
>> Thank you,
>> Naresh
>>
>
>
Re: KafkaUtils.createStream(..) is removed for API
Posted by Ted Yu <yu...@gmail.com>.
createStream() is still
in external/kafka-0-8/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala
But it is not
in external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaUtils.scala
FYI
On Sun, Feb 18, 2018 at 5:17 PM, naresh Goud <na...@gmail.com>
wrote:
> Hello Team,
>
> I see "KafkaUtils.createStream() " method not available in spark 2.2.1.
>
> Can someone please confirm if these methods are removed?
>
> below is my pom.xml entries.
>
>
> <properties>
> <scala.version>2.11.8</scala.version>
> <scala.tools.version>2.11</scala.tools.version>
> </properties>
>
>
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-streaming_${scala.tools.version}</artifactId>
> <version>2.2.1</version>
> <scope>provided</scope>
> </dependency>
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-streaming-kafka-0-10_2.11</artifactId>
> <version>2.2.1</version>
> <scope>provided</scope>
> </dependency>
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-core_2.11</artifactId>
> <version>2.2.1</version>
> <scope>provided</scope>
> </dependency>
> <dependency>
>
>
>
>
>
> Thank you,
> Naresh
>
Re: KafkaUtils.createStream(..) is removed for API
Posted by Ted Yu <yu...@gmail.com>.
createStream() is still
in external/kafka-0-8/src/main/scala/org/apache/spark/streaming/kafka/KafkaUtils.scala
But it is not
in external/kafka-0-10/src/main/scala/org/apache/spark/streaming/kafka010/KafkaUtils.scala
FYI
On Sun, Feb 18, 2018 at 5:17 PM, naresh Goud <na...@gmail.com>
wrote:
> Hello Team,
>
> I see "KafkaUtils.createStream() " method not available in spark 2.2.1.
>
> Can someone please confirm if these methods are removed?
>
> below is my pom.xml entries.
>
>
> <properties>
> <scala.version>2.11.8</scala.version>
> <scala.tools.version>2.11</scala.tools.version>
> </properties>
>
>
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-streaming_${scala.tools.version}</artifactId>
> <version>2.2.1</version>
> <scope>provided</scope>
> </dependency>
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-streaming-kafka-0-10_2.11</artifactId>
> <version>2.2.1</version>
> <scope>provided</scope>
> </dependency>
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-core_2.11</artifactId>
> <version>2.2.1</version>
> <scope>provided</scope>
> </dependency>
> <dependency>
>
>
>
>
>
> Thank you,
> Naresh
>