You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Harsha Chintalapani <ka...@harsha.io> on 2016/10/25 21:16:12 UTC
[VOTE] Add REST Server to Apache Kafka
Hi All,
We are proposing to have a REST Server as part of Apache Kafka
to provide producer/consumer/admin APIs. We Strongly believe having
REST server functionality with Apache Kafka will help a lot of users.
Here is the KIP that Mani Kumar wrote
https://cwiki.apache.org/confluence/display/KAFKA/KIP-80:+Kafka+Rest+Server.
There is a discussion thread in dev list that had differing opinions on
whether to include REST server in Apache Kafka or not. You can read more
about that in this thread
http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
This is a VOTE thread to check interest in the community for
adding REST Server implementation in Apache Kafka.
Thanks,
Harsha
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Roger Hoover <ro...@gmail.com>.
-1
I think a good set of HTTP APIs can simplify client implementations and
allow for easier integration with Kafka as well as simpler ops/admin
tools. Elasticsearch is a great example. However, I think it's too early
to put this into the core. The existing Kafka REST proxies (that I know
of) are based on HTTP 1.1 and require base64 encoding of binary data. I
hope/suspect that we will see HTTP2-based Kafka-proxies but they should be
external to the core project for rapid experimentation. Maybe someday, we
can standardize on one. As an example, even if we decide to support HTTP
2, we still need a message framing spec on top of it. gRPC is an option
but who knows what else will emerge. Then there's metadata serialization
(Protobuf for gRPC). It will be hard to get everyone to agree on these and
we probably don't need to all agree on them at this point.
For reference,
- this is the PubSub gRPC spec: https://github.com/
googleapis/googleapis/blob/master/google/pubsub/v1/pubsub.proto
- this is the message framing spec for gRPC on top of HTTP2 -
http://www.grpc.io/docs/guides/wire.html
Delimited-Message → Compressed-Flag Message-Length Message
Compressed-Flag → 0 / 1 # encoded as 1 byte unsigned integer
Message-Length → {length of Message} # encoded as 4 byte unsigned integer
Message → *{binary octet}
Cheers,
Roger
On Wed, Oct 26, 2016 at 10:21 PM, Jaikiran Pai <ja...@gmail.com>
wrote:
> -1.
>
> I would personally like Kafka core to be limited to the core aspects. I
> consider REST server like this more a tooling thing than a core aspect of
> Kafka and would like it to be outside of Kafka.
>
> -Jaikiran
>
>
> On Wednesday 26 October 2016 02:46 AM, Harsha Chintalapani wrote:
>
>> Hi All,
>> We are proposing to have a REST Server as part of Apache
>> Kafka
>> to provide producer/consumer/admin APIs. We Strongly believe having
>> REST server functionality with Apache Kafka will help a lot of users.
>> Here is the KIP that Mani Kumar wrote
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-80:+Ka
>> fka+Rest+Server.
>> There is a discussion thread in dev list that had differing opinions on
>> whether to include REST server in Apache Kafka or not. You can read more
>> about that in this thread
>> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mb
>> ox/%3CCAMVt_AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@ma
>> il.gmail.com%3E
>>
>> This is a VOTE thread to check interest in the community for
>> adding REST Server implementation in Apache Kafka.
>>
>> Thanks,
>> Harsha
>>
>>
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Jaikiran Pai <ja...@gmail.com>.
-1.
I would personally like Kafka core to be limited to the core aspects. I
consider REST server like this more a tooling thing than a core aspect
of Kafka and would like it to be outside of Kafka.
-Jaikiran
On Wednesday 26 October 2016 02:46 AM, Harsha Chintalapani wrote:
> Hi All,
> We are proposing to have a REST Server as part of Apache Kafka
> to provide producer/consumer/admin APIs. We Strongly believe having
> REST server functionality with Apache Kafka will help a lot of users.
> Here is the KIP that Mani Kumar wrote
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-80:+Kafka+Rest+Server.
> There is a discussion thread in dev list that had differing opinions on
> whether to include REST server in Apache Kafka or not. You can read more
> about that in this thread
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
>
> This is a VOTE thread to check interest in the community for
> adding REST Server implementation in Apache Kafka.
>
> Thanks,
> Harsha
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Guruditta Golani <gg...@hortonworks.com>.
+1
supporting this natively will help immensely in building operational tools on top of it and augment ease of use cases in large deployments.
-Guru
________________________________________
From: Shekar Tippur <ct...@gmail.com>
Sent: Wednesday, October 26, 2016 9:23 AM
To: users
Cc: dev@kafka.apache.org
Subject: Re: [VOTE] Add REST Server to Apache Kafka
+1
Thanks
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Guruditta Golani <gg...@hortonworks.com>.
+1
supporting this natively will help immensely in building operational tools on top of it and augment ease of use cases in large deployments.
-Guru
________________________________________
From: Shekar Tippur <ct...@gmail.com>
Sent: Wednesday, October 26, 2016 9:23 AM
To: users
Cc: dev@kafka.apache.org
Subject: Re: [VOTE] Add REST Server to Apache Kafka
+1
Thanks
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Dana Powers <da...@gmail.com>.
-1
On Wed, Oct 26, 2016 at 9:23 AM, Shekar Tippur <ct...@gmail.com> wrote:
> +1
>
> Thanks
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Dana Powers <da...@gmail.com>.
-1
On Wed, Oct 26, 2016 at 9:23 AM, Shekar Tippur <ct...@gmail.com> wrote:
> +1
>
> Thanks
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Shekar Tippur <ct...@gmail.com>.
+1
Thanks
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Shekar Tippur <ct...@gmail.com>.
+1
Thanks
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Jun Rao <ju...@confluent.io>.
Sorry a typo. -1 instead.
Thanks,
Jun
On Tue, Oct 25, 2016 at 4:24 PM, Jun Rao <ju...@confluent.io> wrote:
> +1
>
> Thanks,
>
> Jun
>
> On Tue, Oct 25, 2016 at 2:16 PM, Harsha Chintalapani <ka...@harsha.io>
> wrote:
>
>> Hi All,
>> We are proposing to have a REST Server as part of Apache Kafka
>> to provide producer/consumer/admin APIs. We Strongly believe having
>> REST server functionality with Apache Kafka will help a lot of users.
>> Here is the KIP that Mani Kumar wrote
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-80:+
>> Kafka+Rest+Server.
>> There is a discussion thread in dev list that had differing opinions on
>> whether to include REST server in Apache Kafka or not. You can read more
>> about that in this thread
>> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mb
>> ox/%3CCAMVt_AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@
>> mail.gmail.com%3E
>>
>> This is a VOTE thread to check interest in the community for
>> adding REST Server implementation in Apache Kafka.
>>
>> Thanks,
>> Harsha
>>
>
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Jun Rao <ju...@confluent.io>.
Sorry a typo. -1 instead.
Thanks,
Jun
On Tue, Oct 25, 2016 at 4:24 PM, Jun Rao <ju...@confluent.io> wrote:
> +1
>
> Thanks,
>
> Jun
>
> On Tue, Oct 25, 2016 at 2:16 PM, Harsha Chintalapani <ka...@harsha.io>
> wrote:
>
>> Hi All,
>> We are proposing to have a REST Server as part of Apache Kafka
>> to provide producer/consumer/admin APIs. We Strongly believe having
>> REST server functionality with Apache Kafka will help a lot of users.
>> Here is the KIP that Mani Kumar wrote
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-80:+
>> Kafka+Rest+Server.
>> There is a discussion thread in dev list that had differing opinions on
>> whether to include REST server in Apache Kafka or not. You can read more
>> about that in this thread
>> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mb
>> ox/%3CCAMVt_AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@
>> mail.gmail.com%3E
>>
>> This is a VOTE thread to check interest in the community for
>> adding REST Server implementation in Apache Kafka.
>>
>> Thanks,
>> Harsha
>>
>
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Jun Rao <ju...@confluent.io>.
+1
Thanks,
Jun
On Tue, Oct 25, 2016 at 2:16 PM, Harsha Chintalapani <ka...@harsha.io>
wrote:
> Hi All,
> We are proposing to have a REST Server as part of Apache Kafka
> to provide producer/consumer/admin APIs. We Strongly believe having
> REST server functionality with Apache Kafka will help a lot of users.
> Here is the KIP that Mani Kumar wrote
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 80:+Kafka+Rest+Server.
> There is a discussion thread in dev list that had differing opinions on
> whether to include REST server in Apache Kafka or not. You can read more
> about that in this thread
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
>
> This is a VOTE thread to check interest in the community for
> adding REST Server implementation in Apache Kafka.
>
> Thanks,
> Harsha
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by parth brahmbhatt <br...@gmail.com>.
+1.
On Tue, Oct 25, 2016 at 2:58 PM, Jay Kreps <ja...@confluent.io> wrote:
> -1
>
> I think the REST server for Kafka that already exists is quite good and
> getting contributions. Moving this into the core project doesn't solve a
> problem that I see.
>
> -Jay
>
> On Tue, Oct 25, 2016 at 2:16 PM, Harsha Chintalapani <ka...@harsha.io>
> wrote:
>
> > Hi All,
> > We are proposing to have a REST Server as part of Apache
> Kafka
> > to provide producer/consumer/admin APIs. We Strongly believe having
> > REST server functionality with Apache Kafka will help a lot of users.
> > Here is the KIP that Mani Kumar wrote
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 80:+Kafka+Rest+Server.
> > There is a discussion thread in dev list that had differing opinions on
> > whether to include REST server in Apache Kafka or not. You can read more
> > about that in this thread
> > http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> > AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
> >
> > This is a VOTE thread to check interest in the community for
> > adding REST Server implementation in Apache Kafka.
> >
> > Thanks,
> > Harsha
> >
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by parth brahmbhatt <br...@gmail.com>.
+1.
On Tue, Oct 25, 2016 at 2:58 PM, Jay Kreps <ja...@confluent.io> wrote:
> -1
>
> I think the REST server for Kafka that already exists is quite good and
> getting contributions. Moving this into the core project doesn't solve a
> problem that I see.
>
> -Jay
>
> On Tue, Oct 25, 2016 at 2:16 PM, Harsha Chintalapani <ka...@harsha.io>
> wrote:
>
> > Hi All,
> > We are proposing to have a REST Server as part of Apache
> Kafka
> > to provide producer/consumer/admin APIs. We Strongly believe having
> > REST server functionality with Apache Kafka will help a lot of users.
> > Here is the KIP that Mani Kumar wrote
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 80:+Kafka+Rest+Server.
> > There is a discussion thread in dev list that had differing opinions on
> > whether to include REST server in Apache Kafka or not. You can read more
> > about that in this thread
> > http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> > AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
> >
> > This is a VOTE thread to check interest in the community for
> > adding REST Server implementation in Apache Kafka.
> >
> > Thanks,
> > Harsha
> >
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Ben Davison <be...@7digital.com>.
I'm going to change my vote to a -1, great points made by all.
(Although I would want to have another discussion around a "management"
REST endpoint for Kafka, I think there would be value in that)
On Wed, Oct 26, 2016 at 3:56 PM, Samuel Taylor <st...@square-root.com>
wrote:
> -1
>
> I don't think the reasons from the KIP are good enough:
>
> 1 -- That other tools have a REST interface isn't a reason Kafka needs one.
> Further, "it is useful" is not a reason; *why* is it useful?
> 2 -- Why does a REST server *need* to be immediately available to every use
> that downloads Kafka?
> 3 -- Maintaining compatibility between Kafka and a REST proxy see seems
> like a burden that should be on the maintainers of the REST proxy, not on
> the core team.
>
> On Wed, Oct 26, 2016 at 2:48 AM, Jan Filipiak <Ja...@trivago.com>
> wrote:
>
> > And you also still need to find the correct broker, for each http call,
> > wich is also hard, when programming against the http api
> >
> >
> > On 26.10.2016 09:46, Jan Filipiak wrote:
> >
> >> So happy to see this reply.
> >> I do think the same, actually makes it way harder to properly batch up
> >> records on http, as kafka core would need to know how to split your
> payload.
> >> It would help people do the wrong thing IMO
> >>
> >> best Jan
> >>
> >> On 25.10.2016 23:58, Jay Kreps wrote:
> >>
> >>> -1
> >>>
> >>> I think the REST server for Kafka that already exists is quite good and
> >>> getting contributions. Moving this into the core project doesn't solve
> a
> >>> problem that I see.
> >>>
> >>> -Jay
> >>>
> >>> On Tue, Oct 25, 2016 at 2:16 PM, Harsha Chintalapani <ka...@harsha.io>
> >>> wrote:
> >>>
> >>> Hi All,
> >>>> We are proposing to have a REST Server as part of Apache
> >>>> Kafka
> >>>> to provide producer/consumer/admin APIs. We Strongly believe having
> >>>> REST server functionality with Apache Kafka will help a lot of users.
> >>>> Here is the KIP that Mani Kumar wrote
> >>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >>>> 80:+Kafka+Rest+Server.
> >>>> There is a discussion thread in dev list that had differing opinions
> on
> >>>> whether to include REST server in Apache Kafka or not. You can read
> more
> >>>> about that in this thread
> >>>> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mb
> >>>> ox/%3CCAMVt_
> >>>> AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
> >>>>
> >>>> This is a VOTE thread to check interest in the community
> for
> >>>> adding REST Server implementation in Apache Kafka.
> >>>>
> >>>> Thanks,
> >>>> Harsha
> >>>>
> >>>>
> >>
> >
>
>
> --
> *Samuel Taylor*
> Data Science
>
> *Square Root, Inc. <http://square-root.com/>*
> <http://square-root.com/>Square-Root.com <http://square-root.com/>
>
--
This email, including attachments, is private and confidential. If you have
received this email in error please notify the sender and delete it from
your system. Emails are not secure and may contain viruses. No liability
can be accepted for viruses that might be transferred by this email or any
attachment. Any unauthorised copying of this message or unauthorised
distribution and publication of the information contained herein are
prohibited.
7digital Limited. Registered office: 69 Wilson Street, London EC2A 2BB.
Registered in England and Wales. Registered No. 04843573.
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Samuel Taylor <st...@square-root.com>.
-1
I don't think the reasons from the KIP are good enough:
1 -- That other tools have a REST interface isn't a reason Kafka needs one.
Further, "it is useful" is not a reason; *why* is it useful?
2 -- Why does a REST server *need* to be immediately available to every use
that downloads Kafka?
3 -- Maintaining compatibility between Kafka and a REST proxy see seems
like a burden that should be on the maintainers of the REST proxy, not on
the core team.
On Wed, Oct 26, 2016 at 2:48 AM, Jan Filipiak <Ja...@trivago.com>
wrote:
> And you also still need to find the correct broker, for each http call,
> wich is also hard, when programming against the http api
>
>
> On 26.10.2016 09:46, Jan Filipiak wrote:
>
>> So happy to see this reply.
>> I do think the same, actually makes it way harder to properly batch up
>> records on http, as kafka core would need to know how to split your payload.
>> It would help people do the wrong thing IMO
>>
>> best Jan
>>
>> On 25.10.2016 23:58, Jay Kreps wrote:
>>
>>> -1
>>>
>>> I think the REST server for Kafka that already exists is quite good and
>>> getting contributions. Moving this into the core project doesn't solve a
>>> problem that I see.
>>>
>>> -Jay
>>>
>>> On Tue, Oct 25, 2016 at 2:16 PM, Harsha Chintalapani <ka...@harsha.io>
>>> wrote:
>>>
>>> Hi All,
>>>> We are proposing to have a REST Server as part of Apache
>>>> Kafka
>>>> to provide producer/consumer/admin APIs. We Strongly believe having
>>>> REST server functionality with Apache Kafka will help a lot of users.
>>>> Here is the KIP that Mani Kumar wrote
>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>>>> 80:+Kafka+Rest+Server.
>>>> There is a discussion thread in dev list that had differing opinions on
>>>> whether to include REST server in Apache Kafka or not. You can read more
>>>> about that in this thread
>>>> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mb
>>>> ox/%3CCAMVt_
>>>> AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
>>>>
>>>> This is a VOTE thread to check interest in the community for
>>>> adding REST Server implementation in Apache Kafka.
>>>>
>>>> Thanks,
>>>> Harsha
>>>>
>>>>
>>
>
--
*Samuel Taylor*
Data Science
*Square Root, Inc. <http://square-root.com/>*
<http://square-root.com/>Square-Root.com <http://square-root.com/>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Jan Filipiak <Ja...@trivago.com>.
And you also still need to find the correct broker, for each http call,
wich is also hard, when programming against the http api
On 26.10.2016 09:46, Jan Filipiak wrote:
> So happy to see this reply.
> I do think the same, actually makes it way harder to properly batch up
> records on http, as kafka core would need to know how to split your
> payload.
> It would help people do the wrong thing IMO
>
> best Jan
>
> On 25.10.2016 23:58, Jay Kreps wrote:
>> -1
>>
>> I think the REST server for Kafka that already exists is quite good and
>> getting contributions. Moving this into the core project doesn't solve a
>> problem that I see.
>>
>> -Jay
>>
>> On Tue, Oct 25, 2016 at 2:16 PM, Harsha Chintalapani <ka...@harsha.io>
>> wrote:
>>
>>> Hi All,
>>> We are proposing to have a REST Server as part of Apache
>>> Kafka
>>> to provide producer/consumer/admin APIs. We Strongly believe having
>>> REST server functionality with Apache Kafka will help a lot of users.
>>> Here is the KIP that Mani Kumar wrote
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>>> 80:+Kafka+Rest+Server.
>>> There is a discussion thread in dev list that had differing opinions on
>>> whether to include REST server in Apache Kafka or not. You can read
>>> more
>>> about that in this thread
>>> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
>>>
>>> AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
>>>
>>> This is a VOTE thread to check interest in the community for
>>> adding REST Server implementation in Apache Kafka.
>>>
>>> Thanks,
>>> Harsha
>>>
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Jan Filipiak <Ja...@trivago.com>.
So happy to see this reply.
I do think the same, actually makes it way harder to properly batch up
records on http, as kafka core would need to know how to split your payload.
It would help people do the wrong thing IMO
best Jan
On 25.10.2016 23:58, Jay Kreps wrote:
> -1
>
> I think the REST server for Kafka that already exists is quite good and
> getting contributions. Moving this into the core project doesn't solve a
> problem that I see.
>
> -Jay
>
> On Tue, Oct 25, 2016 at 2:16 PM, Harsha Chintalapani <ka...@harsha.io>
> wrote:
>
>> Hi All,
>> We are proposing to have a REST Server as part of Apache Kafka
>> to provide producer/consumer/admin APIs. We Strongly believe having
>> REST server functionality with Apache Kafka will help a lot of users.
>> Here is the KIP that Mani Kumar wrote
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>> 80:+Kafka+Rest+Server.
>> There is a discussion thread in dev list that had differing opinions on
>> whether to include REST server in Apache Kafka or not. You can read more
>> about that in this thread
>> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
>> AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
>>
>> This is a VOTE thread to check interest in the community for
>> adding REST Server implementation in Apache Kafka.
>>
>> Thanks,
>> Harsha
>>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Jay Kreps <ja...@confluent.io>.
-1
I think the REST server for Kafka that already exists is quite good and
getting contributions. Moving this into the core project doesn't solve a
problem that I see.
-Jay
On Tue, Oct 25, 2016 at 2:16 PM, Harsha Chintalapani <ka...@harsha.io>
wrote:
> Hi All,
> We are proposing to have a REST Server as part of Apache Kafka
> to provide producer/consumer/admin APIs. We Strongly believe having
> REST server functionality with Apache Kafka will help a lot of users.
> Here is the KIP that Mani Kumar wrote
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 80:+Kafka+Rest+Server.
> There is a discussion thread in dev list that had differing opinions on
> whether to include REST server in Apache Kafka or not. You can read more
> about that in this thread
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
>
> This is a VOTE thread to check interest in the community for
> adding REST Server implementation in Apache Kafka.
>
> Thanks,
> Harsha
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Hugo Picado <hu...@gmail.com>.
-1
From my experience, nothing good comes from merges similar to this one, as
it has huge implications in terms of codebase maintenance, CD pipeline
duration and stability as well as isolated testability. I don't see any
particular worth benefit on having the REST server in the core repository
as it is reasonably decoupled in design and have well defined
responsibilities. Also, I can't foresee any particular issue at the present
moment on keeping REST server right where it stands and contribute into it.
If then governance happens to be an observed issue, this can be
reconsidered again.
Best regards,
Hugo.
On Tue, Oct 25, 2016 at 10:16 PM, Harsha Chintalapani <ka...@harsha.io>
wrote:
> Hi All,
> We are proposing to have a REST Server as part of Apache Kafka
> to provide producer/consumer/admin APIs. We Strongly believe having
> REST server functionality with Apache Kafka will help a lot of users.
> Here is the KIP that Mani Kumar wrote
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 80:+Kafka+Rest+Server.
> There is a discussion thread in dev list that had differing opinions on
> whether to include REST server in Apache Kafka or not. You can read more
> about that in this thread
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
>
> This is a VOTE thread to check interest in the community for
> adding REST Server implementation in Apache Kafka.
>
> Thanks,
> Harsha
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by "Matthias J. Sax" <ma...@confluent.io>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
- -1
Kafka offers a protocol clients can implement, and REST server is only
one of many clients. If we include REST, we need to include any other
client, too, which seems not to be reasonable to me.
- -Matthias
On 10/25/16 5:29 PM, Haohui Mai wrote:
> +1
>
> In my personal experience Kafka REST server is one of essential
> parts on building a reliable real-time pipeline. It's okay to keep
> the code reasonably separated from the core, but would appreciate
> to see some official supports for the REST servers from the Kafka
> community.
>
> Regards, Haohui
>
> On Tue, Oct 25, 2016 at 4:42 PM Ismael Juma <is...@juma.me.uk>
> wrote:
>
>> -1
>>
>> I think it's good to avoid adding non-essential items to the
>> Apache Kafka project so that it can continue evolving at a fast
>> pace. Since a REST server can be naturally implemented as an
>> external project, I believe the disadvantages of including it
>> outweigh the advantages.
>>
>> Ismael
>>
>> On Tue, Oct 25, 2016 at 10:16 PM, Harsha Chintalapani
>> <ka...@harsha.io> wrote:
>>
>>> Hi All, We are proposing to have a REST Server as part of
>>> Apache
>> Kafka
>>> to provide producer/consumer/admin APIs. We Strongly believe
>>> having REST server functionality with Apache Kafka will help a
>>> lot of users. Here is the KIP that Mani Kumar wrote
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>>> 80:+Kafka+Rest+Server. There is a discussion thread in dev list
>>> that had differing opinions on whether to include REST server
>>> in Apache Kafka or not. You can read more about that in this
>>> thread
>>> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCA
MVt_
>>>
>>>
AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
>>>
>>> This is a VOTE thread to check interest in the community for
>>> adding REST Server implementation in Apache Kafka.
>>>
>>> Thanks, Harsha
>>>
>>
>
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - https://gpgtools.org
iQIcBAEBCgAGBQJYEAeOAAoJECnhiMLycopP9bwQAJqcljCiNSpdwcS8+7GilRkK
JBNEnlF5YFXPk4j2U2C6ZZtUqBUdpweQbDegwFsB/Fj04gIs+FiAzkGltXxk7yNM
AI6ZhCxHl6Zg6oNr3Jrfot1uRHaIypHvzJm+8X1QoVWgeOTJe+8lV3LkWBuhD2kN
hfktQBRe8F54URlahL+Q9gygvV6bN9DK7J+i+wUUe6EpZq5A7wiCMR7SkO9IyZpl
CzHL6/V767gQ/Ztq20FhIW0S7eYcyXDfMiZG7Pll5ipEc0AuD1COGf26asL0aW0f
axHARMiEgQLScbOVvzqRjLhEDHYl6MdPD9er5g5DRJG5/ZX4xxQs4jTKaht4l1vh
X5C3pXFZ5UIjJTud/GKYX72tencMX/V1glxQWX9+1xobj5tz7EfUec6x2CWk7whh
HsXjEzt42qXuhyfHtZB0sXbcSj23ODkt3JmqhAd1ievt8BOuVFLzWznfvHs8sYKM
Q0SwNm1K5n+wDAfrLg3ZaFZM/8hO4uVeenUXuVjSwiujVTWqFf2YShwdTUE9AZNj
b/98HTQCDc6L8xJKu0LLzXXO9VvOdd+ggV4nOBqyZwm3qZ3ax3qm/FU0WgGSira2
Mrc5AXnhymmH/J9/CvaHeiSGRHpIxPNojFatCZoYU7AULigzpAfyK7W3iTTUOKAS
NJEHojNYFlJ4VeGpAuxE
=JOal
-----END PGP SIGNATURE-----
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Harsha Ch <ha...@gmail.com>.
Neha,
As I stated in my earlier email . This is an interest check
vote thread and , not a KIP vote thread, to gather input from users mailing
list if there is any interest for REST server to be part of Kafka. Also,
this has been noted in the discuss thread that we should make a interest
vote thread to include users list .
Thanks,
Harsha
On Tue, Oct 25, 2016 at 6:51 PM Neha Narkhede <ne...@confluent.io> wrote:
> -1
>
> Let me make an observation and then state my reasons.
>
> First, I am a little surprised to see this vote thread as I think it is a
> bit premature. As a community, we've consistently invested in good
> communication. If there are differing opinions on a DISCUSS thread, we
> generally gather the community for a KIP call/ in-person meeting. If
> required, meet several times. Then send around notes and finally, call a
> vote. I've observed us resolve many issues and reach better decisions as a
> community by doing that. We've given higher bandwidth communication a
> chance and that has paid off. I would've liked to see us do the same here
> and would still offer the time to do so.
>
> There is a ton of context involved to understand the concerns of those of
> us who have been actively contributing to Kafka for 6 years -- We've worked
> very hard to keep Kafka simple, that is a big reason for its adoption and
> success so far. We've built a thoughtful community that engages in
> constructive communication versus reaching knee-jerk decisions. I see the
> latter happening here and would urge the new members to invest more time in
> sharing ideas before voting.
>
> Let me state the reasons for my vote since the thread was started:
>
> - REST proxy does not cover a meaningful % of usage on Kafka. When we ran a
> poll, it turned out that 50% users want non-java clients, 46% ish wanted
> the java clients and the rest were interested in other protocols (REST,
> MQTT, AMQP).
> - We have tried this approach and it did not work. Our objective is to
> deliver high-quality software and build an agile community. We failed at
> both those objectives with clients and have decided to instead invest in
> the ecosystem and not bloat Apache Kafka.
> - The Kafka ecosystem is large, thriving, open-source and non-Apache. That
> is working, why do we think this should be an exception? Never change
> something that works right? To me, this already is a big reason why the
> Apache argument doesn't make sense.
>
> Regarding the governance worries -- If you are worried about governance,
> why not first try to contribute, hit roadblocks around project direction
> and then raise concerns around governance? Doing that prematurely is
> solving a hypothetical problem and we need to invest time in solving real
> problems.
>
>
> On Tue, Oct 25, 2016 at 5:30 PM Haohui Mai <ri...@gmail.com> wrote:
>
> > +1
> >
> > In my personal experience Kafka REST server is one of essential parts on
> > building a reliable real-time pipeline. It's okay to keep the code
> > reasonably separated from the core, but would appreciate to see some
> > official supports for the REST servers from the Kafka community.
> >
> > Regards,
> > Haohui
> >
> > On Tue, Oct 25, 2016 at 4:42 PM Ismael Juma <is...@juma.me.uk> wrote:
> >
> > > -1
> > >
> > > I think it's good to avoid adding non-essential items to the Apache
> Kafka
> > > project so that it can continue evolving at a fast pace. Since a REST
> > > server can be naturally implemented as an external project, I believe
> the
> > > disadvantages of including it outweigh the advantages.
> > >
> > > Ismael
> > >
> > > On Tue, Oct 25, 2016 at 10:16 PM, Harsha Chintalapani <kafka@harsha.io
> >
> > > wrote:
> > >
> > > > Hi All,
> > > > We are proposing to have a REST Server as part of Apache
> > > Kafka
> > > > to provide producer/consumer/admin APIs. We Strongly believe having
> > > > REST server functionality with Apache Kafka will help a lot of users.
> > > > Here is the KIP that Mani Kumar wrote
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > 80:+Kafka+Rest+Server.
> > > > There is a discussion thread in dev list that had differing opinions
> on
> > > > whether to include REST server in Apache Kafka or not. You can read
> > more
> > > > about that in this thread
> > > >
> > http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> > > > AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
> > > >
> > > > This is a VOTE thread to check interest in the community
> for
> > > > adding REST Server implementation in Apache Kafka.
> > > >
> > > > Thanks,
> > > > Harsha
> > > >
> > >
> >
> --
> Thanks,
> Neha
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Neha Narkhede <ne...@confluent.io>.
Harsha -- Thanks for clarifying but I had noticed that and still gave those
suggestions.
Calling a VOTE thread on a discussion when there are varied viewpoints is
never something this community has resorted to. The optics of it force
binary decision making when in reality there are several tradeoffs to
consider (Ewen summarized those really well in his recent email). I think a
discussion on each of the points is worthwhile which is something a
conversation would allow us to do.
On Wed, Oct 26, 2016 at 12:15 AM Ewen Cheslack-Postava <ew...@confluent.io>
wrote:
> Perhaps in the future we can use a [POLL] or similar? This confused a lot
> of folks because they thought a VOTE was being called on KIP-80 when it
> wasn't. (Even if we say shame on them for not reading the body in full, the
> title caught me as well.)
>
> -Ewen
>
> On Tue, Oct 25, 2016 at 7:11 PM, Harsha Chintalapani <ka...@harsha.io>
> wrote:
>
> > Neha,
> > As I stated in my earlier email, this is an interest check
> > vote thread and, not a KIP vote thread, to gather input from users
> mailing
> > list if there is any interest for REST server to be part of Kafka. Also,
> > this has been noted in the discuss thread that we should make an interest
> > vote thread to include users list.
> >
> > Thanks,
> > Harsha
> >
> > On Tue, Oct 25, 2016 at 6:51 PM Neha Narkhede <ne...@confluent.io> wrote:
> >
> > > -1
> > >
> > > Let me make an observation and then state my reasons.
> > >
> > > First, I am a little surprised to see this vote thread as I think it
> is a
> > > bit premature. As a community, we've consistently invested in good
> > > communication. If there are differing opinions on a DISCUSS thread, we
> > > generally gather the community for a KIP call/ in-person meeting. If
> > > required, meet several times. Then send around notes and finally, call
> a
> > > vote. I've observed us resolve many issues and reach better decisions
> as
> > a
> > > community by doing that. We've given higher bandwidth communication a
> > > chance and that has paid off. I would've liked to see us do the same
> here
> > > and would still offer the time to do so.
> > >
> > > There is a ton of context involved to understand the concerns of those
> of
> > > us who have been actively contributing to Kafka for 6 years -- We've
> > worked
> > > very hard to keep Kafka simple, that is a big reason for its adoption
> and
> > > success so far. We've built a thoughtful community that engages in
> > > constructive communication versus reaching knee-jerk decisions. I see
> the
> > > latter happening here and would urge the new members to invest more
> time
> > in
> > > sharing ideas before voting.
> > >
> > > Let me state the reasons for my vote since the thread was started:
> > >
> > > - REST proxy does not cover a meaningful % of usage on Kafka. When we
> > ran a
> > > poll, it turned out that 50% users want non-java clients, 46% ish
> wanted
> > > the java clients and the rest were interested in other protocols (REST,
> > > MQTT, AMQP).
> > > - We have tried this approach and it did not work. Our objective is to
> > > deliver high-quality software and build an agile community. We failed
> at
> > > both those objectives with clients and have decided to instead invest
> in
> > > the ecosystem and not bloat Apache Kafka.
> > > - The Kafka ecosystem is large, thriving, open-source and non-Apache.
> > That
> > > is working, why do we think this should be an exception? Never change
> > > something that works right? To me, this already is a big reason why the
> > > Apache argument doesn't make sense.
> > >
> > > Regarding the governance worries -- If you are worried about
> governance,
> > > why not first try to contribute, hit roadblocks around project
> direction
> > > and then raise concerns around governance? Doing that prematurely is
> > > solving a hypothetical problem and we need to invest time in solving
> real
> > > problems.
> > >
> > >
> > > On Tue, Oct 25, 2016 at 5:30 PM Haohui Mai <ri...@gmail.com> wrote:
> > >
> > > > +1
> > > >
> > > > In my personal experience Kafka REST server is one of essential parts
> > on
> > > > building a reliable real-time pipeline. It's okay to keep the code
> > > > reasonably separated from the core, but would appreciate to see some
> > > > official supports for the REST servers from the Kafka community.
> > > >
> > > > Regards,
> > > > Haohui
> > > >
> > > > On Tue, Oct 25, 2016 at 4:42 PM Ismael Juma <is...@juma.me.uk>
> wrote:
> > > >
> > > > > -1
> > > > >
> > > > > I think it's good to avoid adding non-essential items to the Apache
> > > Kafka
> > > > > project so that it can continue evolving at a fast pace. Since a
> REST
> > > > > server can be naturally implemented as an external project, I
> believe
> > > the
> > > > > disadvantages of including it outweigh the advantages.
> > > > >
> > > > > Ismael
> > > > >
> > > > > On Tue, Oct 25, 2016 at 10:16 PM, Harsha Chintalapani <
> > kafka@harsha.io
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hi All,
> > > > > > We are proposing to have a REST Server as part of
> > Apache
> > > > > Kafka
> > > > > > to provide producer/consumer/admin APIs. We Strongly believe
> having
> > > > > > REST server functionality with Apache Kafka will help a lot of
> > users.
> > > > > > Here is the KIP that Mani Kumar wrote
> > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > > 80:+Kafka+Rest+Server.
> > > > > > There is a discussion thread in dev list that had differing
> > opinions
> > > on
> > > > > > whether to include REST server in Apache Kafka or not. You can
> read
> > > > more
> > > > > > about that in this thread
> > > > > >
> > > > http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.
> > mbox/%3CCAMVt_
> > > > > > AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
> > > > > >
> > > > > > This is a VOTE thread to check interest in the
> community
> > > for
> > > > > > adding REST Server implementation in Apache Kafka.
> > > > > >
> > > > > > Thanks,
> > > > > > Harsha
> > > > > >
> > > > >
> > > >
> > > --
> > > Thanks,
> > > Neha
> > >
> >
>
>
>
> --
> Thanks,
> Ewen
>
--
Thanks,
Neha
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Ewen Cheslack-Postava <ew...@confluent.io>.
Perhaps in the future we can use a [POLL] or similar? This confused a lot
of folks because they thought a VOTE was being called on KIP-80 when it
wasn't. (Even if we say shame on them for not reading the body in full, the
title caught me as well.)
-Ewen
On Tue, Oct 25, 2016 at 7:11 PM, Harsha Chintalapani <ka...@harsha.io>
wrote:
> Neha,
> As I stated in my earlier email, this is an interest check
> vote thread and, not a KIP vote thread, to gather input from users mailing
> list if there is any interest for REST server to be part of Kafka. Also,
> this has been noted in the discuss thread that we should make an interest
> vote thread to include users list.
>
> Thanks,
> Harsha
>
> On Tue, Oct 25, 2016 at 6:51 PM Neha Narkhede <ne...@confluent.io> wrote:
>
> > -1
> >
> > Let me make an observation and then state my reasons.
> >
> > First, I am a little surprised to see this vote thread as I think it is a
> > bit premature. As a community, we've consistently invested in good
> > communication. If there are differing opinions on a DISCUSS thread, we
> > generally gather the community for a KIP call/ in-person meeting. If
> > required, meet several times. Then send around notes and finally, call a
> > vote. I've observed us resolve many issues and reach better decisions as
> a
> > community by doing that. We've given higher bandwidth communication a
> > chance and that has paid off. I would've liked to see us do the same here
> > and would still offer the time to do so.
> >
> > There is a ton of context involved to understand the concerns of those of
> > us who have been actively contributing to Kafka for 6 years -- We've
> worked
> > very hard to keep Kafka simple, that is a big reason for its adoption and
> > success so far. We've built a thoughtful community that engages in
> > constructive communication versus reaching knee-jerk decisions. I see the
> > latter happening here and would urge the new members to invest more time
> in
> > sharing ideas before voting.
> >
> > Let me state the reasons for my vote since the thread was started:
> >
> > - REST proxy does not cover a meaningful % of usage on Kafka. When we
> ran a
> > poll, it turned out that 50% users want non-java clients, 46% ish wanted
> > the java clients and the rest were interested in other protocols (REST,
> > MQTT, AMQP).
> > - We have tried this approach and it did not work. Our objective is to
> > deliver high-quality software and build an agile community. We failed at
> > both those objectives with clients and have decided to instead invest in
> > the ecosystem and not bloat Apache Kafka.
> > - The Kafka ecosystem is large, thriving, open-source and non-Apache.
> That
> > is working, why do we think this should be an exception? Never change
> > something that works right? To me, this already is a big reason why the
> > Apache argument doesn't make sense.
> >
> > Regarding the governance worries -- If you are worried about governance,
> > why not first try to contribute, hit roadblocks around project direction
> > and then raise concerns around governance? Doing that prematurely is
> > solving a hypothetical problem and we need to invest time in solving real
> > problems.
> >
> >
> > On Tue, Oct 25, 2016 at 5:30 PM Haohui Mai <ri...@gmail.com> wrote:
> >
> > > +1
> > >
> > > In my personal experience Kafka REST server is one of essential parts
> on
> > > building a reliable real-time pipeline. It's okay to keep the code
> > > reasonably separated from the core, but would appreciate to see some
> > > official supports for the REST servers from the Kafka community.
> > >
> > > Regards,
> > > Haohui
> > >
> > > On Tue, Oct 25, 2016 at 4:42 PM Ismael Juma <is...@juma.me.uk> wrote:
> > >
> > > > -1
> > > >
> > > > I think it's good to avoid adding non-essential items to the Apache
> > Kafka
> > > > project so that it can continue evolving at a fast pace. Since a REST
> > > > server can be naturally implemented as an external project, I believe
> > the
> > > > disadvantages of including it outweigh the advantages.
> > > >
> > > > Ismael
> > > >
> > > > On Tue, Oct 25, 2016 at 10:16 PM, Harsha Chintalapani <
> kafka@harsha.io
> > >
> > > > wrote:
> > > >
> > > > > Hi All,
> > > > > We are proposing to have a REST Server as part of
> Apache
> > > > Kafka
> > > > > to provide producer/consumer/admin APIs. We Strongly believe having
> > > > > REST server functionality with Apache Kafka will help a lot of
> users.
> > > > > Here is the KIP that Mani Kumar wrote
> > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > > 80:+Kafka+Rest+Server.
> > > > > There is a discussion thread in dev list that had differing
> opinions
> > on
> > > > > whether to include REST server in Apache Kafka or not. You can read
> > > more
> > > > > about that in this thread
> > > > >
> > > http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.
> mbox/%3CCAMVt_
> > > > > AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
> > > > >
> > > > > This is a VOTE thread to check interest in the community
> > for
> > > > > adding REST Server implementation in Apache Kafka.
> > > > >
> > > > > Thanks,
> > > > > Harsha
> > > > >
> > > >
> > >
> > --
> > Thanks,
> > Neha
> >
>
--
Thanks,
Ewen
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Harsha Chintalapani <ka...@harsha.io>.
Neha,
As I stated in my earlier email, this is an interest check
vote thread and, not a KIP vote thread, to gather input from users mailing
list if there is any interest for REST server to be part of Kafka. Also,
this has been noted in the discuss thread that we should make an interest
vote thread to include users list.
Thanks,
Harsha
On Tue, Oct 25, 2016 at 6:51 PM Neha Narkhede <ne...@confluent.io> wrote:
> -1
>
> Let me make an observation and then state my reasons.
>
> First, I am a little surprised to see this vote thread as I think it is a
> bit premature. As a community, we've consistently invested in good
> communication. If there are differing opinions on a DISCUSS thread, we
> generally gather the community for a KIP call/ in-person meeting. If
> required, meet several times. Then send around notes and finally, call a
> vote. I've observed us resolve many issues and reach better decisions as a
> community by doing that. We've given higher bandwidth communication a
> chance and that has paid off. I would've liked to see us do the same here
> and would still offer the time to do so.
>
> There is a ton of context involved to understand the concerns of those of
> us who have been actively contributing to Kafka for 6 years -- We've worked
> very hard to keep Kafka simple, that is a big reason for its adoption and
> success so far. We've built a thoughtful community that engages in
> constructive communication versus reaching knee-jerk decisions. I see the
> latter happening here and would urge the new members to invest more time in
> sharing ideas before voting.
>
> Let me state the reasons for my vote since the thread was started:
>
> - REST proxy does not cover a meaningful % of usage on Kafka. When we ran a
> poll, it turned out that 50% users want non-java clients, 46% ish wanted
> the java clients and the rest were interested in other protocols (REST,
> MQTT, AMQP).
> - We have tried this approach and it did not work. Our objective is to
> deliver high-quality software and build an agile community. We failed at
> both those objectives with clients and have decided to instead invest in
> the ecosystem and not bloat Apache Kafka.
> - The Kafka ecosystem is large, thriving, open-source and non-Apache. That
> is working, why do we think this should be an exception? Never change
> something that works right? To me, this already is a big reason why the
> Apache argument doesn't make sense.
>
> Regarding the governance worries -- If you are worried about governance,
> why not first try to contribute, hit roadblocks around project direction
> and then raise concerns around governance? Doing that prematurely is
> solving a hypothetical problem and we need to invest time in solving real
> problems.
>
>
> On Tue, Oct 25, 2016 at 5:30 PM Haohui Mai <ri...@gmail.com> wrote:
>
> > +1
> >
> > In my personal experience Kafka REST server is one of essential parts on
> > building a reliable real-time pipeline. It's okay to keep the code
> > reasonably separated from the core, but would appreciate to see some
> > official supports for the REST servers from the Kafka community.
> >
> > Regards,
> > Haohui
> >
> > On Tue, Oct 25, 2016 at 4:42 PM Ismael Juma <is...@juma.me.uk> wrote:
> >
> > > -1
> > >
> > > I think it's good to avoid adding non-essential items to the Apache
> Kafka
> > > project so that it can continue evolving at a fast pace. Since a REST
> > > server can be naturally implemented as an external project, I believe
> the
> > > disadvantages of including it outweigh the advantages.
> > >
> > > Ismael
> > >
> > > On Tue, Oct 25, 2016 at 10:16 PM, Harsha Chintalapani <kafka@harsha.io
> >
> > > wrote:
> > >
> > > > Hi All,
> > > > We are proposing to have a REST Server as part of Apache
> > > Kafka
> > > > to provide producer/consumer/admin APIs. We Strongly believe having
> > > > REST server functionality with Apache Kafka will help a lot of users.
> > > > Here is the KIP that Mani Kumar wrote
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > 80:+Kafka+Rest+Server.
> > > > There is a discussion thread in dev list that had differing opinions
> on
> > > > whether to include REST server in Apache Kafka or not. You can read
> > more
> > > > about that in this thread
> > > >
> > http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> > > > AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
> > > >
> > > > This is a VOTE thread to check interest in the community
> for
> > > > adding REST Server implementation in Apache Kafka.
> > > >
> > > > Thanks,
> > > > Harsha
> > > >
> > >
> >
> --
> Thanks,
> Neha
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Neha Narkhede <ne...@confluent.io>.
-1
Let me make an observation and then state my reasons.
First, I am a little surprised to see this vote thread as I think it is a
bit premature. As a community, we've consistently invested in good
communication. If there are differing opinions on a DISCUSS thread, we
generally gather the community for a KIP call/ in-person meeting. If
required, meet several times. Then send around notes and finally, call a
vote. I've observed us resolve many issues and reach better decisions as a
community by doing that. We've given higher bandwidth communication a
chance and that has paid off. I would've liked to see us do the same here
and would still offer the time to do so.
There is a ton of context involved to understand the concerns of those of
us who have been actively contributing to Kafka for 6 years -- We've worked
very hard to keep Kafka simple, that is a big reason for its adoption and
success so far. We've built a thoughtful community that engages in
constructive communication versus reaching knee-jerk decisions. I see the
latter happening here and would urge the new members to invest more time in
sharing ideas before voting.
Let me state the reasons for my vote since the thread was started:
- REST proxy does not cover a meaningful % of usage on Kafka. When we ran a
poll, it turned out that 50% users want non-java clients, 46% ish wanted
the java clients and the rest were interested in other protocols (REST,
MQTT, AMQP).
- We have tried this approach and it did not work. Our objective is to
deliver high-quality software and build an agile community. We failed at
both those objectives with clients and have decided to instead invest in
the ecosystem and not bloat Apache Kafka.
- The Kafka ecosystem is large, thriving, open-source and non-Apache. That
is working, why do we think this should be an exception? Never change
something that works right? To me, this already is a big reason why the
Apache argument doesn't make sense.
Regarding the governance worries -- If you are worried about governance,
why not first try to contribute, hit roadblocks around project direction
and then raise concerns around governance? Doing that prematurely is
solving a hypothetical problem and we need to invest time in solving real
problems.
On Tue, Oct 25, 2016 at 5:30 PM Haohui Mai <ri...@gmail.com> wrote:
> +1
>
> In my personal experience Kafka REST server is one of essential parts on
> building a reliable real-time pipeline. It's okay to keep the code
> reasonably separated from the core, but would appreciate to see some
> official supports for the REST servers from the Kafka community.
>
> Regards,
> Haohui
>
> On Tue, Oct 25, 2016 at 4:42 PM Ismael Juma <is...@juma.me.uk> wrote:
>
> > -1
> >
> > I think it's good to avoid adding non-essential items to the Apache Kafka
> > project so that it can continue evolving at a fast pace. Since a REST
> > server can be naturally implemented as an external project, I believe the
> > disadvantages of including it outweigh the advantages.
> >
> > Ismael
> >
> > On Tue, Oct 25, 2016 at 10:16 PM, Harsha Chintalapani <ka...@harsha.io>
> > wrote:
> >
> > > Hi All,
> > > We are proposing to have a REST Server as part of Apache
> > Kafka
> > > to provide producer/consumer/admin APIs. We Strongly believe having
> > > REST server functionality with Apache Kafka will help a lot of users.
> > > Here is the KIP that Mani Kumar wrote
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 80:+Kafka+Rest+Server.
> > > There is a discussion thread in dev list that had differing opinions on
> > > whether to include REST server in Apache Kafka or not. You can read
> more
> > > about that in this thread
> > >
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> > > AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
> > >
> > > This is a VOTE thread to check interest in the community for
> > > adding REST Server implementation in Apache Kafka.
> > >
> > > Thanks,
> > > Harsha
> > >
> >
>
--
Thanks,
Neha
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Haohui Mai <ri...@gmail.com>.
+1
In my personal experience Kafka REST server is one of essential parts on
building a reliable real-time pipeline. It's okay to keep the code
reasonably separated from the core, but would appreciate to see some
official supports for the REST servers from the Kafka community.
Regards,
Haohui
On Tue, Oct 25, 2016 at 4:42 PM Ismael Juma <is...@juma.me.uk> wrote:
> -1
>
> I think it's good to avoid adding non-essential items to the Apache Kafka
> project so that it can continue evolving at a fast pace. Since a REST
> server can be naturally implemented as an external project, I believe the
> disadvantages of including it outweigh the advantages.
>
> Ismael
>
> On Tue, Oct 25, 2016 at 10:16 PM, Harsha Chintalapani <ka...@harsha.io>
> wrote:
>
> > Hi All,
> > We are proposing to have a REST Server as part of Apache
> Kafka
> > to provide producer/consumer/admin APIs. We Strongly believe having
> > REST server functionality with Apache Kafka will help a lot of users.
> > Here is the KIP that Mani Kumar wrote
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 80:+Kafka+Rest+Server.
> > There is a discussion thread in dev list that had differing opinions on
> > whether to include REST server in Apache Kafka or not. You can read more
> > about that in this thread
> > http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> > AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
> >
> > This is a VOTE thread to check interest in the community for
> > adding REST Server implementation in Apache Kafka.
> >
> > Thanks,
> > Harsha
> >
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Ismael Juma <is...@juma.me.uk>.
-1
I think it's good to avoid adding non-essential items to the Apache Kafka
project so that it can continue evolving at a fast pace. Since a REST
server can be naturally implemented as an external project, I believe the
disadvantages of including it outweigh the advantages.
Ismael
On Tue, Oct 25, 2016 at 10:16 PM, Harsha Chintalapani <ka...@harsha.io>
wrote:
> Hi All,
> We are proposing to have a REST Server as part of Apache Kafka
> to provide producer/consumer/admin APIs. We Strongly believe having
> REST server functionality with Apache Kafka will help a lot of users.
> Here is the KIP that Mani Kumar wrote
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 80:+Kafka+Rest+Server.
> There is a discussion thread in dev list that had differing opinions on
> whether to include REST server in Apache Kafka or not. You can read more
> about that in this thread
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
>
> This is a VOTE thread to check interest in the community for
> adding REST Server implementation in Apache Kafka.
>
> Thanks,
> Harsha
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Harsha Chintalapani <ka...@harsha.io>.
This Vote now closed. There is the majority who are not in favor of
including the REST server.
Thanks everyone for participating.
Vote tally:
+1 Votes:
Harsha Chintalapani
Parth Brahbhatt
Ken Jackson
Suresh Srinivas
Jungtaek Lim
Ali Akthar
Haohui Mai
Shekhar Tippur
Guruditta Golani
-1 Votes:
Jay Kreps
Ben Davidson
Jeff Widman
Stevo Slavic
Sriram Subramanian
Craig W
Jun Rao
Ismael Juma
Matthias J. Sax
Neha Narkhede
Jan Filipiak
Ofir Manor
Samuel Taylor
Dana Powers
Zakee
Andrew Otto
Hugo Picado
Jaikiran Pai
Roger Hoover
Gwen Shapira
Thanks,
Harsha
On Sat, Oct 29, 2016 at 8:16 AM Gwen Shapira <gw...@confluent.io> wrote:
> Oops. Sorry, didn't notice the 72h voting period has passed. You can
> disregard.
>
> Gwen
>
> On Sat, Oct 29, 2016 at 4:29 PM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > -1
> >
> > Kafka's development model is a good fit for critical path and
> > well-established APIs. It doesn't work as well for add-ons that need to
> > rapidly evolve. Merging communities with different development pace and
> > models rarely ends well - I think the REST Proxy will benefit from being
> a
> > separate project.
> >
> > On Tue, Oct 25, 2016 at 11:16 PM, Harsha Chintalapani <ka...@harsha.io>
> > wrote:
> >
> >> Hi All,
> >> We are proposing to have a REST Server as part of Apache
> Kafka
> >> to provide producer/consumer/admin APIs. We Strongly believe having
> >> REST server functionality with Apache Kafka will help a lot of users.
> >> Here is the KIP that Mani Kumar wrote
> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-80:+
> >> Kafka+Rest+Server.
> >> There is a discussion thread in dev list that had differing opinions on
> >> whether to include REST server in Apache Kafka or not. You can read more
> >> about that in this thread
> >> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mb
> >> ox/%3CCAMVt_AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@
> >> mail.gmail.com%3E
> >>
> >> This is a VOTE thread to check interest in the community for
> >> adding REST Server implementation in Apache Kafka.
> >>
> >> Thanks,
> >> Harsha
> >>
> >
> >
> >
> > --
> > *Gwen Shapira*
> > Product Manager | Confluent
> > 650.450.2760 <(650)%20450-2760> | @gwenshap
> > Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> > <http://www.confluent.io/blog>
> >
> >
>
>
> --
> *Gwen Shapira*
> Product Manager | Confluent
> 650.450.2760 <(650)%20450-2760> | @gwenshap
> Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> <http://www.confluent.io/blog>
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Harsha Chintalapani <ka...@harsha.io>.
This Vote now closed. There is the majority who are not in favor of
including the REST server.
Thanks everyone for participating.
Vote tally:
+1 Votes:
Harsha Chintalapani
Parth Brahbhatt
Ken Jackson
Suresh Srinivas
Jungtaek Lim
Ali Akthar
Haohui Mai
Shekhar Tippur
Guruditta Golani
-1 Votes:
Jay Kreps
Ben Davidson
Jeff Widman
Stevo Slavic
Sriram Subramanian
Craig W
Jun Rao
Ismael Juma
Matthias J. Sax
Neha Narkhede
Jan Filipiak
Ofir Manor
Samuel Taylor
Dana Powers
Zakee
Andrew Otto
Hugo Picado
Jaikiran Pai
Roger Hoover
Gwen Shapira
Thanks,
Harsha
On Sat, Oct 29, 2016 at 8:16 AM Gwen Shapira <gw...@confluent.io> wrote:
> Oops. Sorry, didn't notice the 72h voting period has passed. You can
> disregard.
>
> Gwen
>
> On Sat, Oct 29, 2016 at 4:29 PM, Gwen Shapira <gw...@confluent.io> wrote:
>
> > -1
> >
> > Kafka's development model is a good fit for critical path and
> > well-established APIs. It doesn't work as well for add-ons that need to
> > rapidly evolve. Merging communities with different development pace and
> > models rarely ends well - I think the REST Proxy will benefit from being
> a
> > separate project.
> >
> > On Tue, Oct 25, 2016 at 11:16 PM, Harsha Chintalapani <ka...@harsha.io>
> > wrote:
> >
> >> Hi All,
> >> We are proposing to have a REST Server as part of Apache
> Kafka
> >> to provide producer/consumer/admin APIs. We Strongly believe having
> >> REST server functionality with Apache Kafka will help a lot of users.
> >> Here is the KIP that Mani Kumar wrote
> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-80:+
> >> Kafka+Rest+Server.
> >> There is a discussion thread in dev list that had differing opinions on
> >> whether to include REST server in Apache Kafka or not. You can read more
> >> about that in this thread
> >> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mb
> >> ox/%3CCAMVt_AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@
> >> mail.gmail.com%3E
> >>
> >> This is a VOTE thread to check interest in the community for
> >> adding REST Server implementation in Apache Kafka.
> >>
> >> Thanks,
> >> Harsha
> >>
> >
> >
> >
> > --
> > *Gwen Shapira*
> > Product Manager | Confluent
> > 650.450.2760 <(650)%20450-2760> | @gwenshap
> > Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> > <http://www.confluent.io/blog>
> >
> >
>
>
> --
> *Gwen Shapira*
> Product Manager | Confluent
> 650.450.2760 <(650)%20450-2760> | @gwenshap
> Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> <http://www.confluent.io/blog>
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Gwen Shapira <gw...@confluent.io>.
Oops. Sorry, didn't notice the 72h voting period has passed. You can
disregard.
Gwen
On Sat, Oct 29, 2016 at 4:29 PM, Gwen Shapira <gw...@confluent.io> wrote:
> -1
>
> Kafka's development model is a good fit for critical path and
> well-established APIs. It doesn't work as well for add-ons that need to
> rapidly evolve. Merging communities with different development pace and
> models rarely ends well - I think the REST Proxy will benefit from being a
> separate project.
>
> On Tue, Oct 25, 2016 at 11:16 PM, Harsha Chintalapani <ka...@harsha.io>
> wrote:
>
>> Hi All,
>> We are proposing to have a REST Server as part of Apache Kafka
>> to provide producer/consumer/admin APIs. We Strongly believe having
>> REST server functionality with Apache Kafka will help a lot of users.
>> Here is the KIP that Mani Kumar wrote
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-80:+
>> Kafka+Rest+Server.
>> There is a discussion thread in dev list that had differing opinions on
>> whether to include REST server in Apache Kafka or not. You can read more
>> about that in this thread
>> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mb
>> ox/%3CCAMVt_AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@
>> mail.gmail.com%3E
>>
>> This is a VOTE thread to check interest in the community for
>> adding REST Server implementation in Apache Kafka.
>>
>> Thanks,
>> Harsha
>>
>
>
>
> --
> *Gwen Shapira*
> Product Manager | Confluent
> 650.450.2760 | @gwenshap
> Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> <http://www.confluent.io/blog>
>
>
--
*Gwen Shapira*
Product Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
<http://www.confluent.io/blog>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Gwen Shapira <gw...@confluent.io>.
Oops. Sorry, didn't notice the 72h voting period has passed. You can
disregard.
Gwen
On Sat, Oct 29, 2016 at 4:29 PM, Gwen Shapira <gw...@confluent.io> wrote:
> -1
>
> Kafka's development model is a good fit for critical path and
> well-established APIs. It doesn't work as well for add-ons that need to
> rapidly evolve. Merging communities with different development pace and
> models rarely ends well - I think the REST Proxy will benefit from being a
> separate project.
>
> On Tue, Oct 25, 2016 at 11:16 PM, Harsha Chintalapani <ka...@harsha.io>
> wrote:
>
>> Hi All,
>> We are proposing to have a REST Server as part of Apache Kafka
>> to provide producer/consumer/admin APIs. We Strongly believe having
>> REST server functionality with Apache Kafka will help a lot of users.
>> Here is the KIP that Mani Kumar wrote
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-80:+
>> Kafka+Rest+Server.
>> There is a discussion thread in dev list that had differing opinions on
>> whether to include REST server in Apache Kafka or not. You can read more
>> about that in this thread
>> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mb
>> ox/%3CCAMVt_AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@
>> mail.gmail.com%3E
>>
>> This is a VOTE thread to check interest in the community for
>> adding REST Server implementation in Apache Kafka.
>>
>> Thanks,
>> Harsha
>>
>
>
>
> --
> *Gwen Shapira*
> Product Manager | Confluent
> 650.450.2760 | @gwenshap
> Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> <http://www.confluent.io/blog>
>
>
--
*Gwen Shapira*
Product Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
<http://www.confluent.io/blog>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Gwen Shapira <gw...@confluent.io>.
-1
Kafka's development model is a good fit for critical path and
well-established APIs. It doesn't work as well for add-ons that need to
rapidly evolve. Merging communities with different development pace and
models rarely ends well - I think the REST Proxy will benefit from being a
separate project.
On Tue, Oct 25, 2016 at 11:16 PM, Harsha Chintalapani <ka...@harsha.io>
wrote:
> Hi All,
> We are proposing to have a REST Server as part of Apache Kafka
> to provide producer/consumer/admin APIs. We Strongly believe having
> REST server functionality with Apache Kafka will help a lot of users.
> Here is the KIP that Mani Kumar wrote
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 80:+Kafka+Rest+Server.
> There is a discussion thread in dev list that had differing opinions on
> whether to include REST server in Apache Kafka or not. You can read more
> about that in this thread
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
>
> This is a VOTE thread to check interest in the community for
> adding REST Server implementation in Apache Kafka.
>
> Thanks,
> Harsha
>
--
*Gwen Shapira*
Product Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
<http://www.confluent.io/blog>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Jay Kreps <ja...@confluent.io>.
-1
I think the REST server for Kafka that already exists is quite good and
getting contributions. Moving this into the core project doesn't solve a
problem that I see.
-Jay
On Tue, Oct 25, 2016 at 2:16 PM, Harsha Chintalapani <ka...@harsha.io>
wrote:
> Hi All,
> We are proposing to have a REST Server as part of Apache Kafka
> to provide producer/consumer/admin APIs. We Strongly believe having
> REST server functionality with Apache Kafka will help a lot of users.
> Here is the KIP that Mani Kumar wrote
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 80:+Kafka+Rest+Server.
> There is a discussion thread in dev list that had differing opinions on
> whether to include REST server in Apache Kafka or not. You can read more
> about that in this thread
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
>
> This is a VOTE thread to check interest in the community for
> adding REST Server implementation in Apache Kafka.
>
> Thanks,
> Harsha
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Ofir Manor <of...@equalum.io>.
-1
First, I think there was no technical discussion of the KIP details in the
DISCUSS until now, just a binary yes/no. Real discussion just started with
the excellent ten technical points from Ewen a few hours ago.
Generally,I don't see how a standard REST makes sense for format-agnostic
realtime streaming. It would likely be a poor choice for any high-volume
messaging, and encouraging its usage instead of the existing native clients
(or promoting it as an equivalent alternative) is likely a mistake and will
hurt users.
For those needing that functionality, they can get it today and can also
participate in that ecosystem project (or fork it).
Ofir Manor
Co-Founder & CTO | Equalum
Mobile: +972-54-7801286 | Email: ofir.manor@equalum.io
On Wed, Oct 26, 2016 at 12:16 AM, Harsha Chintalapani <ka...@harsha.io>
wrote:
> Hi All,
> We are proposing to have a REST Server as part of Apache Kafka
> to provide producer/consumer/admin APIs. We Strongly believe having
> REST server functionality with Apache Kafka will help a lot of users.
> Here is the KIP that Mani Kumar wrote
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 80:+Kafka+Rest+Server.
> There is a discussion thread in dev list that had differing opinions on
> whether to include REST server in Apache Kafka or not. You can read more
> about that in this thread
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
>
> This is a VOTE thread to check interest in the community for
> adding REST Server implementation in Apache Kafka.
>
> Thanks,
> Harsha
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Ben Davison <be...@7digital.com>.
+ 1
On Tue, Oct 25, 2016 at 10:16 PM, Harsha Chintalapani <ka...@harsha.io>
wrote:
> Hi All,
> We are proposing to have a REST Server as part of Apache Kafka
> to provide producer/consumer/admin APIs. We Strongly believe having
> REST server functionality with Apache Kafka will help a lot of users.
> Here is the KIP that Mani Kumar wrote
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 80:+Kafka+Rest+Server.
> There is a discussion thread in dev list that had differing opinions on
> whether to include REST server in Apache Kafka or not. You can read more
> about that in this thread
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
>
> This is a VOTE thread to check interest in the community for
> adding REST Server implementation in Apache Kafka.
>
> Thanks,
> Harsha
>
--
This email, including attachments, is private and confidential. If you have
received this email in error please notify the sender and delete it from
your system. Emails are not secure and may contain viruses. No liability
can be accepted for viruses that might be transferred by this email or any
attachment. Any unauthorised copying of this message or unauthorised
distribution and publication of the information contained herein are
prohibited.
7digital Limited. Registered office: 69 Wilson Street, London EC2A 2BB.
Registered in England and Wales. Registered No. 04843573.
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Ken Jackson <kj...@gmail.com>.
+1
On Tue, Oct 25, 2016 at 3:17 PM, Harsha Chintalapani <ka...@harsha.io>
wrote:
> Jeff,
> Thanks for participating. We already have the discussion thread going
> on. Please add your thoughts there . I'll keep this as interest check vote
> thread.
> Thanks,
> Harsha
>
> On Tue, Oct 25, 2016 at 3:12 PM Stevo Slavić <ss...@gmail.com> wrote:
>
> > -1
> >
> > I fully agree with Jay and Jeff.
> >
> > On Wed, Oct 26, 2016 at 12:03 AM, Jeff Widman <je...@netskope.com> wrote:
> >
> > > -1
> > >
> > > As an end-user, while I like the idea in theory, in practice I don't
> > think
> > > it's a good idea just yet.
> > >
> > > Certainly, it'd be useful, enabling things like
> > > https://github.com/Landoop/kafka-topics-ui to work without needing
> > > anything
> > > outside of Kafka core.
> > >
> > > But there are already enough things in the existing Kafka core that
> feel
> > a
> > > little half-baked, especially on the ops side... there's been a few
> > threads
> > > lately on the users mailing list where users have asked about
> > functionality
> > > that seems like it should be standard and all proffered solutions have
> > > seemed a bit hacky.
> > >
> > > Similarly, while I understand the argument that having a solid HTTP
> > > endpoint would obviate the need for other language bindings, I'm not
> sure
> > > that's what would happen. From a speed perspective, I'd often rather
> use
> > > the native bindings in my language of choice, even if it's unofficial
> and
> > > may be lagging behind on features. So this doesn't feel that compelling
> > to
> > > me.
> > >
> > > I'd rather the Kafka community focus on making Kafka rock-solid at what
> > it
> > > does, particularly having the existing CLI ops tooling and docs be
> really
> > > solid before trying to add fairly major new pieces to the project.
> > >
> > > Plus I think it's very beneficial for the Kafka community for Confluent
> > to
> > > have a strong business model--they provide many contributions to core
> > that
> > > most of us benefit from for free. Keeping a few things like the REST
> > > interface under their umbrella gives them free marketing exposure while
> > > still allowing the rest of us access to their tooling as
> > free/open-source.
> > >
> > >
> > > On Tue, Oct 25, 2016 at 2:16 PM, Harsha Chintalapani <ka...@harsha.io>
> > > wrote:
> > >
> > > > Hi All,
> > > > We are proposing to have a REST Server as part of Apache
> > > Kafka
> > > > to provide producer/consumer/admin APIs. We Strongly believe having
> > > > REST server functionality with Apache Kafka will help a lot of users.
> > > > Here is the KIP that Mani Kumar wrote
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > 80:+Kafka+Rest+Server.
> > > > There is a discussion thread in dev list that had differing opinions
> on
> > > > whether to include REST server in Apache Kafka or not. You can read
> > more
> > > > about that in this thread
> > > >
> > http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> > > > AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
> > > >
> > > > This is a VOTE thread to check interest in the community
> for
> > > > adding REST Server implementation in Apache Kafka.
> > > >
> > > > Thanks,
> > > > Harsha
> > > >
> > >
> >
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Harsha Chintalapani <ka...@harsha.io>.
Jeff,
Thanks for participating. We already have the discussion thread going
on. Please add your thoughts there . I'll keep this as interest check vote
thread.
Thanks,
Harsha
On Tue, Oct 25, 2016 at 3:12 PM Stevo Slavić <ss...@gmail.com> wrote:
> -1
>
> I fully agree with Jay and Jeff.
>
> On Wed, Oct 26, 2016 at 12:03 AM, Jeff Widman <je...@netskope.com> wrote:
>
> > -1
> >
> > As an end-user, while I like the idea in theory, in practice I don't
> think
> > it's a good idea just yet.
> >
> > Certainly, it'd be useful, enabling things like
> > https://github.com/Landoop/kafka-topics-ui to work without needing
> > anything
> > outside of Kafka core.
> >
> > But there are already enough things in the existing Kafka core that feel
> a
> > little half-baked, especially on the ops side... there's been a few
> threads
> > lately on the users mailing list where users have asked about
> functionality
> > that seems like it should be standard and all proffered solutions have
> > seemed a bit hacky.
> >
> > Similarly, while I understand the argument that having a solid HTTP
> > endpoint would obviate the need for other language bindings, I'm not sure
> > that's what would happen. From a speed perspective, I'd often rather use
> > the native bindings in my language of choice, even if it's unofficial and
> > may be lagging behind on features. So this doesn't feel that compelling
> to
> > me.
> >
> > I'd rather the Kafka community focus on making Kafka rock-solid at what
> it
> > does, particularly having the existing CLI ops tooling and docs be really
> > solid before trying to add fairly major new pieces to the project.
> >
> > Plus I think it's very beneficial for the Kafka community for Confluent
> to
> > have a strong business model--they provide many contributions to core
> that
> > most of us benefit from for free. Keeping a few things like the REST
> > interface under their umbrella gives them free marketing exposure while
> > still allowing the rest of us access to their tooling as
> free/open-source.
> >
> >
> > On Tue, Oct 25, 2016 at 2:16 PM, Harsha Chintalapani <ka...@harsha.io>
> > wrote:
> >
> > > Hi All,
> > > We are proposing to have a REST Server as part of Apache
> > Kafka
> > > to provide producer/consumer/admin APIs. We Strongly believe having
> > > REST server functionality with Apache Kafka will help a lot of users.
> > > Here is the KIP that Mani Kumar wrote
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 80:+Kafka+Rest+Server.
> > > There is a discussion thread in dev list that had differing opinions on
> > > whether to include REST server in Apache Kafka or not. You can read
> more
> > > about that in this thread
> > >
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> > > AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
> > >
> > > This is a VOTE thread to check interest in the community for
> > > adding REST Server implementation in Apache Kafka.
> > >
> > > Thanks,
> > > Harsha
> > >
> >
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Stevo Slavić <ss...@gmail.com>.
-1
I fully agree with Jay and Jeff.
On Wed, Oct 26, 2016 at 12:03 AM, Jeff Widman <je...@netskope.com> wrote:
> -1
>
> As an end-user, while I like the idea in theory, in practice I don't think
> it's a good idea just yet.
>
> Certainly, it'd be useful, enabling things like
> https://github.com/Landoop/kafka-topics-ui to work without needing
> anything
> outside of Kafka core.
>
> But there are already enough things in the existing Kafka core that feel a
> little half-baked, especially on the ops side... there's been a few threads
> lately on the users mailing list where users have asked about functionality
> that seems like it should be standard and all proffered solutions have
> seemed a bit hacky.
>
> Similarly, while I understand the argument that having a solid HTTP
> endpoint would obviate the need for other language bindings, I'm not sure
> that's what would happen. From a speed perspective, I'd often rather use
> the native bindings in my language of choice, even if it's unofficial and
> may be lagging behind on features. So this doesn't feel that compelling to
> me.
>
> I'd rather the Kafka community focus on making Kafka rock-solid at what it
> does, particularly having the existing CLI ops tooling and docs be really
> solid before trying to add fairly major new pieces to the project.
>
> Plus I think it's very beneficial for the Kafka community for Confluent to
> have a strong business model--they provide many contributions to core that
> most of us benefit from for free. Keeping a few things like the REST
> interface under their umbrella gives them free marketing exposure while
> still allowing the rest of us access to their tooling as free/open-source.
>
>
> On Tue, Oct 25, 2016 at 2:16 PM, Harsha Chintalapani <ka...@harsha.io>
> wrote:
>
> > Hi All,
> > We are proposing to have a REST Server as part of Apache
> Kafka
> > to provide producer/consumer/admin APIs. We Strongly believe having
> > REST server functionality with Apache Kafka will help a lot of users.
> > Here is the KIP that Mani Kumar wrote
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 80:+Kafka+Rest+Server.
> > There is a discussion thread in dev list that had differing opinions on
> > whether to include REST server in Apache Kafka or not. You can read more
> > about that in this thread
> > http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> > AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
> >
> > This is a VOTE thread to check interest in the community for
> > adding REST Server implementation in Apache Kafka.
> >
> > Thanks,
> > Harsha
> >
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Jeff Widman <je...@netskope.com>.
-1
As an end-user, while I like the idea in theory, in practice I don't think
it's a good idea just yet.
Certainly, it'd be useful, enabling things like
https://github.com/Landoop/kafka-topics-ui to work without needing anything
outside of Kafka core.
But there are already enough things in the existing Kafka core that feel a
little half-baked, especially on the ops side... there's been a few threads
lately on the users mailing list where users have asked about functionality
that seems like it should be standard and all proffered solutions have
seemed a bit hacky.
Similarly, while I understand the argument that having a solid HTTP
endpoint would obviate the need for other language bindings, I'm not sure
that's what would happen. From a speed perspective, I'd often rather use
the native bindings in my language of choice, even if it's unofficial and
may be lagging behind on features. So this doesn't feel that compelling to
me.
I'd rather the Kafka community focus on making Kafka rock-solid at what it
does, particularly having the existing CLI ops tooling and docs be really
solid before trying to add fairly major new pieces to the project.
Plus I think it's very beneficial for the Kafka community for Confluent to
have a strong business model--they provide many contributions to core that
most of us benefit from for free. Keeping a few things like the REST
interface under their umbrella gives them free marketing exposure while
still allowing the rest of us access to their tooling as free/open-source.
On Tue, Oct 25, 2016 at 2:16 PM, Harsha Chintalapani <ka...@harsha.io>
wrote:
> Hi All,
> We are proposing to have a REST Server as part of Apache Kafka
> to provide producer/consumer/admin APIs. We Strongly believe having
> REST server functionality with Apache Kafka will help a lot of users.
> Here is the KIP that Mani Kumar wrote
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 80:+Kafka+Rest+Server.
> There is a discussion thread in dev list that had differing opinions on
> whether to include REST server in Apache Kafka or not. You can read more
> about that in this thread
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
>
> This is a VOTE thread to check interest in the community for
> adding REST Server implementation in Apache Kafka.
>
> Thanks,
> Harsha
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Jun Rao <ju...@confluent.io>.
+1
Thanks,
Jun
On Tue, Oct 25, 2016 at 2:16 PM, Harsha Chintalapani <ka...@harsha.io>
wrote:
> Hi All,
> We are proposing to have a REST Server as part of Apache Kafka
> to provide producer/consumer/admin APIs. We Strongly believe having
> REST server functionality with Apache Kafka will help a lot of users.
> Here is the KIP that Mani Kumar wrote
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 80:+Kafka+Rest+Server.
> There is a discussion thread in dev list that had differing opinions on
> whether to include REST server in Apache Kafka or not. You can read more
> about that in this thread
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
>
> This is a VOTE thread to check interest in the community for
> adding REST Server implementation in Apache Kafka.
>
> Thanks,
> Harsha
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Gwen Shapira <gw...@confluent.io>.
-1
Kafka's development model is a good fit for critical path and
well-established APIs. It doesn't work as well for add-ons that need to
rapidly evolve. Merging communities with different development pace and
models rarely ends well - I think the REST Proxy will benefit from being a
separate project.
On Tue, Oct 25, 2016 at 11:16 PM, Harsha Chintalapani <ka...@harsha.io>
wrote:
> Hi All,
> We are proposing to have a REST Server as part of Apache Kafka
> to provide producer/consumer/admin APIs. We Strongly believe having
> REST server functionality with Apache Kafka will help a lot of users.
> Here is the KIP that Mani Kumar wrote
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 80:+Kafka+Rest+Server.
> There is a discussion thread in dev list that had differing opinions on
> whether to include REST server in Apache Kafka or not. You can read more
> about that in this thread
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
>
> This is a VOTE thread to check interest in the community for
> adding REST Server implementation in Apache Kafka.
>
> Thanks,
> Harsha
>
--
*Gwen Shapira*
Product Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
<http://www.confluent.io/blog>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Andrew Otto <ot...@wikimedia.org>.
-1 for http kafka client in core
Although a read only management interface, perhaps via http, sounds kinda
useful for things like health checks as mentioned.
On Wed, Oct 26, 2016 at 2:00 PM, Zakee <kz...@netzero.net> wrote:
> -1
>
> Thanks.
> > On Oct 25, 2016, at 2:16 PM, Harsha Chintalapani <ka...@harsha.io>
> wrote:
> >
> > Hi All,
> > We are proposing to have a REST Server as part of Apache Kafka
> > to provide producer/consumer/admin APIs. We Strongly believe having
> > REST server functionality with Apache Kafka will help a lot of users.
> > Here is the KIP that Mani Kumar wrote
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 80:+Kafka+Rest+Server.
> > There is a discussion thread in dev list that had differing opinions on
> > whether to include REST server in Apache Kafka or not. You can read more
> > about that in this thread
> > http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
> >
> > This is a VOTE thread to check interest in the community for
> > adding REST Server implementation in Apache Kafka.
> >
> > Thanks,
> > Harsha
>
> ____________________________________________________________
> How To Fix Your Fatigue (Do This Every Day)
> gundrymd.com
> http://thirdpartyoffers.netzero.net/TGL3231/5810ef50463346f4f12a0st02vuc
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Andrew Otto <ot...@wikimedia.org>.
-1 for http kafka client in core
Although a read only management interface, perhaps via http, sounds kinda
useful for things like health checks as mentioned.
On Wed, Oct 26, 2016 at 2:00 PM, Zakee <kz...@netzero.net> wrote:
> -1
>
> Thanks.
> > On Oct 25, 2016, at 2:16 PM, Harsha Chintalapani <ka...@harsha.io>
> wrote:
> >
> > Hi All,
> > We are proposing to have a REST Server as part of Apache Kafka
> > to provide producer/consumer/admin APIs. We Strongly believe having
> > REST server functionality with Apache Kafka will help a lot of users.
> > Here is the KIP that Mani Kumar wrote
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 80:+Kafka+Rest+Server.
> > There is a discussion thread in dev list that had differing opinions on
> > whether to include REST server in Apache Kafka or not. You can read more
> > about that in this thread
> > http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
> >
> > This is a VOTE thread to check interest in the community for
> > adding REST Server implementation in Apache Kafka.
> >
> > Thanks,
> > Harsha
>
> ____________________________________________________________
> How To Fix Your Fatigue (Do This Every Day)
> gundrymd.com
> http://thirdpartyoffers.netzero.net/TGL3231/5810ef50463346f4f12a0st02vuc
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Zakee <kz...@netzero.net>.
-1
Thanks.
> On Oct 25, 2016, at 2:16 PM, Harsha Chintalapani <ka...@harsha.io> wrote:
>
> Hi All,
> We are proposing to have a REST Server as part of Apache Kafka
> to provide producer/consumer/admin APIs. We Strongly believe having
> REST server functionality with Apache Kafka will help a lot of users.
> Here is the KIP that Mani Kumar wrote
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-80:+Kafka+Rest+Server.
> There is a discussion thread in dev list that had differing opinions on
> whether to include REST server in Apache Kafka or not. You can read more
> about that in this thread
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
>
> This is a VOTE thread to check interest in the community for
> adding REST Server implementation in Apache Kafka.
>
> Thanks,
> Harsha
____________________________________________________________
How To Fix Your Fatigue (Do This Every Day)
gundrymd.com
http://thirdpartyoffers.netzero.net/TGL3231/5810ef50463346f4f12a0st02vuc
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Zakee <kz...@netzero.net>.
-1
Thanks.
> On Oct 25, 2016, at 2:16 PM, Harsha Chintalapani <ka...@harsha.io> wrote:
>
> Hi All,
> We are proposing to have a REST Server as part of Apache Kafka
> to provide producer/consumer/admin APIs. We Strongly believe having
> REST server functionality with Apache Kafka will help a lot of users.
> Here is the KIP that Mani Kumar wrote
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-80:+Kafka+Rest+Server.
> There is a discussion thread in dev list that had differing opinions on
> whether to include REST server in Apache Kafka or not. You can read more
> about that in this thread
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
>
> This is a VOTE thread to check interest in the community for
> adding REST Server implementation in Apache Kafka.
>
> Thanks,
> Harsha
____________________________________________________________
How To Fix Your Fatigue (Do This Every Day)
gundrymd.com
http://thirdpartyoffers.netzero.net/TGL3231/5810ef50463346f4f12a0st02vuc
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Jeff Widman <je...@netskope.com>.
-1
As an end-user, while I like the idea in theory, in practice I don't think
it's a good idea just yet.
Certainly, it'd be useful, enabling things like
https://github.com/Landoop/kafka-topics-ui to work without needing anything
outside of Kafka core.
But there are already enough things in the existing Kafka core that feel a
little half-baked, especially on the ops side... there's been a few threads
lately on the users mailing list where users have asked about functionality
that seems like it should be standard and all proffered solutions have
seemed a bit hacky.
Similarly, while I understand the argument that having a solid HTTP
endpoint would obviate the need for other language bindings, I'm not sure
that's what would happen. From a speed perspective, I'd often rather use
the native bindings in my language of choice, even if it's unofficial and
may be lagging behind on features. So this doesn't feel that compelling to
me.
I'd rather the Kafka community focus on making Kafka rock-solid at what it
does, particularly having the existing CLI ops tooling and docs be really
solid before trying to add fairly major new pieces to the project.
Plus I think it's very beneficial for the Kafka community for Confluent to
have a strong business model--they provide many contributions to core that
most of us benefit from for free. Keeping a few things like the REST
interface under their umbrella gives them free marketing exposure while
still allowing the rest of us access to their tooling as free/open-source.
On Tue, Oct 25, 2016 at 2:16 PM, Harsha Chintalapani <ka...@harsha.io>
wrote:
> Hi All,
> We are proposing to have a REST Server as part of Apache Kafka
> to provide producer/consumer/admin APIs. We Strongly believe having
> REST server functionality with Apache Kafka will help a lot of users.
> Here is the KIP that Mani Kumar wrote
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 80:+Kafka+Rest+Server.
> There is a discussion thread in dev list that had differing opinions on
> whether to include REST server in Apache Kafka or not. You can read more
> about that in this thread
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
>
> This is a VOTE thread to check interest in the community for
> adding REST Server implementation in Apache Kafka.
>
> Thanks,
> Harsha
>
Re: [VOTE] Add REST Server to Apache Kafka
Posted by Ben Davison <be...@7digital.com>.
+ 1
On Tue, Oct 25, 2016 at 10:16 PM, Harsha Chintalapani <ka...@harsha.io>
wrote:
> Hi All,
> We are proposing to have a REST Server as part of Apache Kafka
> to provide producer/consumer/admin APIs. We Strongly believe having
> REST server functionality with Apache Kafka will help a lot of users.
> Here is the KIP that Mani Kumar wrote
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 80:+Kafka+Rest+Server.
> There is a discussion thread in dev list that had differing opinions on
> whether to include REST server in Apache Kafka or not. You can read more
> about that in this thread
> http://mail-archives.apache.org/mod_mbox/kafka-dev/201610.mbox/%3CCAMVt_
> AyMqeuDM39ZnSXGKtPDdE46sowmqhsXoP-+JMBCUV74Dw@mail.gmail.com%3E
>
> This is a VOTE thread to check interest in the community for
> adding REST Server implementation in Apache Kafka.
>
> Thanks,
> Harsha
>
--
This email, including attachments, is private and confidential. If you have
received this email in error please notify the sender and delete it from
your system. Emails are not secure and may contain viruses. No liability
can be accepted for viruses that might be transferred by this email or any
attachment. Any unauthorised copying of this message or unauthorised
distribution and publication of the information contained herein are
prohibited.
7digital Limited. Registered office: 69 Wilson Street, London EC2A 2BB.
Registered in England and Wales. Registered No. 04843573.