You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by SRK <sw...@gmail.com> on 2017/06/08 22:03:27 UTC
Is Structured streaming ready for production usage
Hi,
Is structured streaming ready for production usage in Spark 2.2?
Thanks,
Swetha
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Is-Structured-streaming-ready-for-production-usage-tp28751.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org
Re: Is Structured streaming ready for production usage
Posted by "Shixiong(Ryan) Zhu" <sh...@databricks.com>.
Please take a look at
http://spark.apache.org/docs/latest/structured-streaming-kafka-integration.html
On Thu, Jun 8, 2017 at 4:46 PM, swetha kasireddy <sw...@gmail.com>
wrote:
> OK. Can we use Spark Kafka Direct with Structured Streaming?
>
> On Thu, Jun 8, 2017 at 4:46 PM, swetha kasireddy <
> swethakasireddy@gmail.com> wrote:
>
>> OK. Can we use Spark Kafka Direct as part of Structured Streaming?
>>
>> On Thu, Jun 8, 2017 at 3:35 PM, Tathagata Das <
>> tathagata.das1565@gmail.com> wrote:
>>
>>> YES. At Databricks, our customers have already been using Structured
>>> Streaming and in the last month alone processed over 3 trillion records.
>>>
>>> https://databricks.com/blog/2017/06/06/simple-super-fast-str
>>> eaming-engine-apache-spark.html
>>>
>>> On Thu, Jun 8, 2017 at 3:03 PM, SRK <sw...@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Is structured streaming ready for production usage in Spark 2.2?
>>>>
>>>> Thanks,
>>>> Swetha
>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context: http://apache-spark-user-list.
>>>> 1001560.n3.nabble.com/Is-Structured-streaming-ready-for-prod
>>>> uction-usage-tp28751.html
>>>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>>>>
>>>>
>>>
>>
>
Re: Is Structured streaming ready for production usage
Posted by swetha kasireddy <sw...@gmail.com>.
OK. Can we use Spark Kafka Direct with Structured Streaming?
On Thu, Jun 8, 2017 at 4:46 PM, swetha kasireddy <sw...@gmail.com>
wrote:
> OK. Can we use Spark Kafka Direct as part of Structured Streaming?
>
> On Thu, Jun 8, 2017 at 3:35 PM, Tathagata Das <tathagata.das1565@gmail.com
> > wrote:
>
>> YES. At Databricks, our customers have already been using Structured
>> Streaming and in the last month alone processed over 3 trillion records.
>>
>> https://databricks.com/blog/2017/06/06/simple-super-fast-str
>> eaming-engine-apache-spark.html
>>
>> On Thu, Jun 8, 2017 at 3:03 PM, SRK <sw...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Is structured streaming ready for production usage in Spark 2.2?
>>>
>>> Thanks,
>>> Swetha
>>>
>>>
>>>
>>> --
>>> View this message in context: http://apache-spark-user-list.
>>> 1001560.n3.nabble.com/Is-Structured-streaming-ready-for-prod
>>> uction-usage-tp28751.html
>>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>>>
>>>
>>
>
Re: Is Structured streaming ready for production usage
Posted by swetha kasireddy <sw...@gmail.com>.
OK. Can we use Spark Kafka Direct as part of Structured Streaming?
On Thu, Jun 8, 2017 at 3:35 PM, Tathagata Das <ta...@gmail.com>
wrote:
> YES. At Databricks, our customers have already been using Structured
> Streaming and in the last month alone processed over 3 trillion records.
>
> https://databricks.com/blog/2017/06/06/simple-super-fast-
> streaming-engine-apache-spark.html
>
> On Thu, Jun 8, 2017 at 3:03 PM, SRK <sw...@gmail.com> wrote:
>
>> Hi,
>>
>> Is structured streaming ready for production usage in Spark 2.2?
>>
>> Thanks,
>> Swetha
>>
>>
>>
>> --
>> View this message in context: http://apache-spark-user-list.
>> 1001560.n3.nabble.com/Is-Structured-streaming-ready-for-
>> production-usage-tp28751.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>>
>>
>
Re: Is Structured streaming ready for production usage
Posted by Tathagata Das <ta...@gmail.com>.
YES. At Databricks, our customers have already been using Structured
Streaming and in the last month alone processed over 3 trillion records.
https://databricks.com/blog/2017/06/06/simple-super-fast-streaming-engine-apache-spark.html
On Thu, Jun 8, 2017 at 3:03 PM, SRK <sw...@gmail.com> wrote:
> Hi,
>
> Is structured streaming ready for production usage in Spark 2.2?
>
> Thanks,
> Swetha
>
>
>
> --
> View this message in context: http://apache-spark-user-list.
> 1001560.n3.nabble.com/Is-Structured-streaming-ready-
> for-production-usage-tp28751.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>