You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@druid.apache.org by Jonathan Wei <jo...@apache.org> on 2020/01/19 21:16:30 UTC
Deprecate Tranquility?
Hi all,
For the upcoming 0.17.0 release, I'm wondering if this would be a good time
to officially declare Tranquility as deprecated/sunsetted in the release
notes.
This was discussed a few times in the past, such as:
https://lists.apache.org/thread.html/476fdd5b6c681854df38bc78e5fae11d3fa371345e1924694ea6be4a%40%3Cdev.druid.apache.org%3E
Tranquility hasn't had any updates and no on seems interested in
maintaining it, it's been de-emphasized in the docs, and I think the reply
from Gian in that thread gives a good breakdown of the rationale for
favoring Kafka ingestion instead:
> We should include some rationale as to why that
decision was made, too. To me the rationale boils down to: KIS style
ingestion doesn't have a windowPeriod restriction, doesn't lose data when
tasks fail, is lower footprint when reading from an external stream hub
like Kafka/Kinesis, and has generally proven to be easier to set up and
debug.
Any thoughts?
Best,
Jon
Re: Deprecate Tranquility?
Posted by Himanshu <g....@gmail.com>.
To give more context, Druid code has an "external discovery" mechanism to
let external systems discover stuff inside Druid cluster via zookeeper. It
is the set of things that are announced via "ServiceAnnouncer". It exists
solely to support tranquility and has a hard dependency on Zookeeeper. As I
am working towards https://github.com/apache/druid/issues/9053 , I am
planning to add configuration options to disable external discovery
because it can't exist without Zookeeper, however I wonder if it is now
right time to remove that code altogether and consequently officially
remove tranquility support in next Druid release.
On Mon, Mar 9, 2020 at 11:43 AM Jonathan Wei <jo...@apache.org> wrote:
> Hm, I looked at the commit history and that wording has been there since
> 0.16.0. The 0.17.0 release didn't have any changes to Tranquility status.
>
> On Sun, Mar 8, 2020 at 10:18 PM Himanshu <g....@gmail.com> wrote:
>
> > Hi Jon,
> >
> > Did we deprecate tranquility in 0.17.0 release and plan to drop
> tranquility
> > support in next Druid release?
> >
> > Language in, both,
> > https://druid.apache.org/docs/latest/ingestion/tranquility.html and
> > https://github.com/apache/druid/releases#docs recommend users to not use
> > tranquility but doesn't say anything regarding dropping its support from
> > next Druid release.
> >
> > -- Himanshu
> >
> >
> >
> >
> > On Sun, Jan 19, 2020 at 6:58 PM Aditya Mishra <ad...@gmail.com>
> > wrote:
> >
> > > +1
> > >
> > > afaik, even basic extensions have problems with tranquillity at this
> > point.
> > >
> > > I started off with druid 6 months ago & was struggling to make
> > datasketches
> > > work with tranquillity for a couple of days before realizing that a
> Kafka
> > > cluster in front is the right way.
> > >
> > > It'll only cause confusion for newcomers till clearly marked as
> > deprecated.
> > >
> > > On Mon, 20 Jan 2020, 07:19 Fangjin Yang, <fa...@imply.io> wrote:
> > >
> > > > +1
> > > >
> > > > On Sun, Jan 19, 2020 at 1:16 PM Jonathan Wei <jo...@apache.org>
> > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > For the upcoming 0.17.0 release, I'm wondering if this would be a
> > good
> > > > time
> > > > > to officially declare Tranquility as deprecated/sunsetted in the
> > > release
> > > > > notes.
> > > > >
> > > > > This was discussed a few times in the past, such as:
> > > > >
> > > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/476fdd5b6c681854df38bc78e5fae11d3fa371345e1924694ea6be4a%40%3Cdev.druid.apache.org%3E
> > > > >
> > > > > Tranquility hasn't had any updates and no on seems interested in
> > > > > maintaining it, it's been de-emphasized in the docs, and I think
> the
> > > > reply
> > > > > from Gian in that thread gives a good breakdown of the rationale
> for
> > > > > favoring Kafka ingestion instead:
> > > > >
> > > > > > We should include some rationale as to why that
> > > > > decision was made, too. To me the rationale boils down to: KIS
> style
> > > > > ingestion doesn't have a windowPeriod restriction, doesn't lose
> data
> > > when
> > > > > tasks fail, is lower footprint when reading from an external stream
> > hub
> > > > > like Kafka/Kinesis, and has generally proven to be easier to set up
> > and
> > > > > debug.
> > > > >
> > > > > Any thoughts?
> > > > >
> > > > > Best,
> > > > > Jon
> > > > >
> > > >
> > >
> >
>
Re: Deprecate Tranquility?
Posted by Jonathan Wei <jo...@apache.org>.
Hm, I looked at the commit history and that wording has been there since
0.16.0. The 0.17.0 release didn't have any changes to Tranquility status.
On Sun, Mar 8, 2020 at 10:18 PM Himanshu <g....@gmail.com> wrote:
> Hi Jon,
>
> Did we deprecate tranquility in 0.17.0 release and plan to drop tranquility
> support in next Druid release?
>
> Language in, both,
> https://druid.apache.org/docs/latest/ingestion/tranquility.html and
> https://github.com/apache/druid/releases#docs recommend users to not use
> tranquility but doesn't say anything regarding dropping its support from
> next Druid release.
>
> -- Himanshu
>
>
>
>
> On Sun, Jan 19, 2020 at 6:58 PM Aditya Mishra <ad...@gmail.com>
> wrote:
>
> > +1
> >
> > afaik, even basic extensions have problems with tranquillity at this
> point.
> >
> > I started off with druid 6 months ago & was struggling to make
> datasketches
> > work with tranquillity for a couple of days before realizing that a Kafka
> > cluster in front is the right way.
> >
> > It'll only cause confusion for newcomers till clearly marked as
> deprecated.
> >
> > On Mon, 20 Jan 2020, 07:19 Fangjin Yang, <fa...@imply.io> wrote:
> >
> > > +1
> > >
> > > On Sun, Jan 19, 2020 at 1:16 PM Jonathan Wei <jo...@apache.org>
> wrote:
> > >
> > > > Hi all,
> > > >
> > > > For the upcoming 0.17.0 release, I'm wondering if this would be a
> good
> > > time
> > > > to officially declare Tranquility as deprecated/sunsetted in the
> > release
> > > > notes.
> > > >
> > > > This was discussed a few times in the past, such as:
> > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/476fdd5b6c681854df38bc78e5fae11d3fa371345e1924694ea6be4a%40%3Cdev.druid.apache.org%3E
> > > >
> > > > Tranquility hasn't had any updates and no on seems interested in
> > > > maintaining it, it's been de-emphasized in the docs, and I think the
> > > reply
> > > > from Gian in that thread gives a good breakdown of the rationale for
> > > > favoring Kafka ingestion instead:
> > > >
> > > > > We should include some rationale as to why that
> > > > decision was made, too. To me the rationale boils down to: KIS style
> > > > ingestion doesn't have a windowPeriod restriction, doesn't lose data
> > when
> > > > tasks fail, is lower footprint when reading from an external stream
> hub
> > > > like Kafka/Kinesis, and has generally proven to be easier to set up
> and
> > > > debug.
> > > >
> > > > Any thoughts?
> > > >
> > > > Best,
> > > > Jon
> > > >
> > >
> >
>
Re: Deprecate Tranquility?
Posted by Himanshu <g....@gmail.com>.
Hi Jon,
Did we deprecate tranquility in 0.17.0 release and plan to drop tranquility
support in next Druid release?
Language in, both,
https://druid.apache.org/docs/latest/ingestion/tranquility.html and
https://github.com/apache/druid/releases#docs recommend users to not use
tranquility but doesn't say anything regarding dropping its support from
next Druid release.
-- Himanshu
On Sun, Jan 19, 2020 at 6:58 PM Aditya Mishra <ad...@gmail.com> wrote:
> +1
>
> afaik, even basic extensions have problems with tranquillity at this point.
>
> I started off with druid 6 months ago & was struggling to make datasketches
> work with tranquillity for a couple of days before realizing that a Kafka
> cluster in front is the right way.
>
> It'll only cause confusion for newcomers till clearly marked as deprecated.
>
> On Mon, 20 Jan 2020, 07:19 Fangjin Yang, <fa...@imply.io> wrote:
>
> > +1
> >
> > On Sun, Jan 19, 2020 at 1:16 PM Jonathan Wei <jo...@apache.org> wrote:
> >
> > > Hi all,
> > >
> > > For the upcoming 0.17.0 release, I'm wondering if this would be a good
> > time
> > > to officially declare Tranquility as deprecated/sunsetted in the
> release
> > > notes.
> > >
> > > This was discussed a few times in the past, such as:
> > >
> > >
> >
> https://lists.apache.org/thread.html/476fdd5b6c681854df38bc78e5fae11d3fa371345e1924694ea6be4a%40%3Cdev.druid.apache.org%3E
> > >
> > > Tranquility hasn't had any updates and no on seems interested in
> > > maintaining it, it's been de-emphasized in the docs, and I think the
> > reply
> > > from Gian in that thread gives a good breakdown of the rationale for
> > > favoring Kafka ingestion instead:
> > >
> > > > We should include some rationale as to why that
> > > decision was made, too. To me the rationale boils down to: KIS style
> > > ingestion doesn't have a windowPeriod restriction, doesn't lose data
> when
> > > tasks fail, is lower footprint when reading from an external stream hub
> > > like Kafka/Kinesis, and has generally proven to be easier to set up and
> > > debug.
> > >
> > > Any thoughts?
> > >
> > > Best,
> > > Jon
> > >
> >
>
Re: Deprecate Tranquility?
Posted by Aditya Mishra <ad...@gmail.com>.
+1
afaik, even basic extensions have problems with tranquillity at this point.
I started off with druid 6 months ago & was struggling to make datasketches
work with tranquillity for a couple of days before realizing that a Kafka
cluster in front is the right way.
It'll only cause confusion for newcomers till clearly marked as deprecated.
On Mon, 20 Jan 2020, 07:19 Fangjin Yang, <fa...@imply.io> wrote:
> +1
>
> On Sun, Jan 19, 2020 at 1:16 PM Jonathan Wei <jo...@apache.org> wrote:
>
> > Hi all,
> >
> > For the upcoming 0.17.0 release, I'm wondering if this would be a good
> time
> > to officially declare Tranquility as deprecated/sunsetted in the release
> > notes.
> >
> > This was discussed a few times in the past, such as:
> >
> >
> https://lists.apache.org/thread.html/476fdd5b6c681854df38bc78e5fae11d3fa371345e1924694ea6be4a%40%3Cdev.druid.apache.org%3E
> >
> > Tranquility hasn't had any updates and no on seems interested in
> > maintaining it, it's been de-emphasized in the docs, and I think the
> reply
> > from Gian in that thread gives a good breakdown of the rationale for
> > favoring Kafka ingestion instead:
> >
> > > We should include some rationale as to why that
> > decision was made, too. To me the rationale boils down to: KIS style
> > ingestion doesn't have a windowPeriod restriction, doesn't lose data when
> > tasks fail, is lower footprint when reading from an external stream hub
> > like Kafka/Kinesis, and has generally proven to be easier to set up and
> > debug.
> >
> > Any thoughts?
> >
> > Best,
> > Jon
> >
>
Re: Deprecate Tranquility?
Posted by Fangjin Yang <fa...@imply.io>.
+1
On Sun, Jan 19, 2020 at 1:16 PM Jonathan Wei <jo...@apache.org> wrote:
> Hi all,
>
> For the upcoming 0.17.0 release, I'm wondering if this would be a good time
> to officially declare Tranquility as deprecated/sunsetted in the release
> notes.
>
> This was discussed a few times in the past, such as:
>
> https://lists.apache.org/thread.html/476fdd5b6c681854df38bc78e5fae11d3fa371345e1924694ea6be4a%40%3Cdev.druid.apache.org%3E
>
> Tranquility hasn't had any updates and no on seems interested in
> maintaining it, it's been de-emphasized in the docs, and I think the reply
> from Gian in that thread gives a good breakdown of the rationale for
> favoring Kafka ingestion instead:
>
> > We should include some rationale as to why that
> decision was made, too. To me the rationale boils down to: KIS style
> ingestion doesn't have a windowPeriod restriction, doesn't lose data when
> tasks fail, is lower footprint when reading from an external stream hub
> like Kafka/Kinesis, and has generally proven to be easier to set up and
> debug.
>
> Any thoughts?
>
> Best,
> Jon
>