Questions reg. Kafka apis and proxy

[Achanta Vamsi Subhash <ac...@flipkart.com>]

Hi,

We are considering hosting Kafka as a service in our company (with multiple
clusters and a common interface) . We have the following questions with
regards to hosting Kafka as a service:

- The recommended Kafka clients and their api have changed since the last 2
releases - producer in 0.8.2 and consumer in 0.9. Are the apis considered
finalized with 0.9 or will there be any change in the apis that might come
in the future?
- Also, currently the client library requires the metadata-broker-list -
meaning that the clients have to know which Kafka cluster they need to
connect to. Is there any well-supported TCP layer proxy for it? (not the
Application layer REST-proxy as it will be used for the higher scale -
millions/sec)
​​
Does anyone have a similar setup? Could you share your opinions?

-- 
Regards
Vamsi Subhash

Re: Questions reg. Kafka apis and proxy

[Guozhang Wang <wa...@gmail.com>]

Achanta,

The Java producer APIs has been running in some production environment and
is considered stable; the Java consumer APIs in 0.9 is considered beta and
may have light modifications in the future.

About the metadata-broker-list: this config is only needed to bootstrap the
client, hence does not necessarily need to include all the broker hosts,
for example in practice people could set metadata-broker-list as the
address of a load balancer in front of the Kafka brokers that will be
forwarded, and once the client is up and running it can automatically fetch
the full broker list from a metadata request.

Guozhang

On Wed, Dec 9, 2015 at 12:52 AM, Achanta Vamsi Subhash <
achanta.vamsi@flipkart.com> wrote:

> Hi,
>
> We are considering hosting Kafka as a service in our company (with multiple
> clusters and a common interface) . We have the following questions with
> regards to hosting Kafka as a service:
>
> - The recommended Kafka clients and their api have changed since the last 2
> releases - producer in 0.8.2 and consumer in 0.9. Are the apis considered
> finalized with 0.9 or will there be any change in the apis that might come
> in the future?
> - Also, currently the client library requires the metadata-broker-list -
> meaning that the clients have to know which Kafka cluster they need to
> connect to. Is there any well-supported TCP layer proxy for it? (not the
> Application layer REST-proxy as it will be used for the higher scale -
> millions/sec)
> ​​
> Does anyone have a similar setup? Could you share your opinions?
>
> --
> Regards
> Vamsi Subhash
>



-- 
-- Guozhang