You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Prashant Sharma <sc...@gmail.com> on 2020/03/02 10:52:10 UTC
[DISCUSS] Shall we mark spark streaming component as deprecated.
Hi All,
It is noticed that some of the users of Spark streaming do not immediately
realise that it is a deprecated component and it would be scary, if they
end up with it in production. Now that we are in a position to release
about Spark 3.0.0, may be we should discuss - should the spark streaming
carry an explicit notice? That it is deprecated and not under active
development.
I have opened an issue already, but I think a mailing list discussion would
be more appropriate. https://issues.apache.org/jira/browse/SPARK-31006
Thanks,
Prashant.
Re: [DISCUSS] Shall we mark spark streaming component as
deprecated.
Posted by Reynold Xin <rx...@databricks.com>.
It's a good discussion to have though: should we deprecate dstream, and what do we need to do to make that happen? My experience working with a lot of Spark users is that in general I recommend them staying away from dstream, due to a lot of design and architectural issues.
On Mon, Mar 02, 2020 at 9:32 AM, Prashant Sharma < scrapcodes@gmail.com > wrote:
>
> I may have speculated, or believed the unauthorised sources, nevertheless
> I am happy to be corrected.
>
> On Mon, Mar 2, 2020 at 8:05 PM Sean Owen < srowen@ gmail. com (
> srowen@gmail.com ) > wrote:
>
>
>> Er, who says it's deprecated? I have never heard anything like that.
>> Why would it be?
>>
>> On Mon, Mar 2, 2020 at 4:52 AM Prashant Sharma < scrapcodes@ gmail. com (
>> scrapcodes@gmail.com ) > wrote:
>> >
>> > Hi All,
>> >
>> > It is noticed that some of the users of Spark streaming do not
>> immediately realise that it is a deprecated component and it would be
>> scary, if they end up with it in production. Now that we are in a position
>> to release about Spark 3.0.0, may be we should discuss - should the spark
>> streaming carry an explicit notice? That it is deprecated and not under
>> active development.
>> >
>> > I have opened an issue already, but I think a mailing list discussion
>> would be more appropriate. https:/ / issues. apache. org/ jira/ browse/ SPARK-31006
>> ( https://issues.apache.org/jira/browse/SPARK-31006 )
>> >
>> > Thanks,
>> > Prashant.
>> >
>
>
>
Re: [DISCUSS] Shall we mark spark streaming component as deprecated.
Posted by Prashant Sharma <sc...@gmail.com>.
I may have speculated, or believed the unauthorised sources, nevertheless I
am happy to be corrected.
On Mon, Mar 2, 2020 at 8:05 PM Sean Owen <sr...@gmail.com> wrote:
> Er, who says it's deprecated? I have never heard anything like that.
> Why would it be?
>
> On Mon, Mar 2, 2020 at 4:52 AM Prashant Sharma <sc...@gmail.com>
> wrote:
> >
> > Hi All,
> >
> > It is noticed that some of the users of Spark streaming do not
> immediately realise that it is a deprecated component and it would be
> scary, if they end up with it in production. Now that we are in a position
> to release about Spark 3.0.0, may be we should discuss - should the spark
> streaming carry an explicit notice? That it is deprecated and not under
> active development.
> >
> > I have opened an issue already, but I think a mailing list discussion
> would be more appropriate.
> https://issues.apache.org/jira/browse/SPARK-31006
> >
> > Thanks,
> > Prashant.
> >
>
Re: [DISCUSS] Shall we mark spark streaming component as deprecated.
Posted by Sean Owen <sr...@gmail.com>.
Er, who says it's deprecated? I have never heard anything like that.
Why would it be?
On Mon, Mar 2, 2020 at 4:52 AM Prashant Sharma <sc...@gmail.com> wrote:
>
> Hi All,
>
> It is noticed that some of the users of Spark streaming do not immediately realise that it is a deprecated component and it would be scary, if they end up with it in production. Now that we are in a position to release about Spark 3.0.0, may be we should discuss - should the spark streaming carry an explicit notice? That it is deprecated and not under active development.
>
> I have opened an issue already, but I think a mailing list discussion would be more appropriate. https://issues.apache.org/jira/browse/SPARK-31006
>
> Thanks,
> Prashant.
>
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: [DISCUSS] Shall we mark spark streaming component as deprecated.
Posted by Ismaël Mejía <ie...@gmail.com>.
Is it really ready to be deprecated? The fact that we cannot do multiple
aggregations with structured streaming [1] is a serious runtime limitation that
the DStream API does not have,
Is it worth to deprecate without having an equivalent set of features?
[1] https://issues.apache.org/jira/browse/SPARK-26655
On Mon, Mar 2, 2020 at 11:52 AM Prashant Sharma <sc...@gmail.com> wrote:
>
> Hi All,
>
> It is noticed that some of the users of Spark streaming do not immediately realise that it is a deprecated component and it would be scary, if they end up with it in production. Now that we are in a position to release about Spark 3.0.0, may be we should discuss - should the spark streaming carry an explicit notice? That it is deprecated and not under active development.
>
> I have opened an issue already, but I think a mailing list discussion would be more appropriate. https://issues.apache.org/jira/browse/SPARK-31006
>
> Thanks,
> Prashant.
>
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org