Kafka 1.0 upgrade

[Tolga Can <to...@snap.com.INVALID>]

Hi

We are in the process of upgrading our Kafka cluster from 0.9 to 1.0. We
will need a cluster similar to LinkedIn’s busiest clusters as described here
<https://docs.confluent.io/current/kafka/deployment.html>, 50K partitions
50 VMs. I could not find enough information about if the latest release is
being used at production at a big scale.

Is Kafka 1.0 production ready? Is there anyone using it at a very big
scale?

Regards
Tc

Re: Kafka 1.0 upgrade

[Brett Rann <br...@zendesk.com.INVALID>]

we run a 1.0.1 prerelease in production just fine, but the scale is
smaller. 20+ clusters with 3-10 brokers each, each cluster with about 120
topics and about 15k partitions. We have unusual messages sizes, so peaks
of around 40k messages, 60MB in,  400MB out, per sec in the largest one.

we run a pre-release because of this:
https://issues.apache.org/jira/browse/KAFKA-6185

Which seems applicable to you. I wouldn't recommend upgrading the
log.message.format without that fix.

Why not test it out?

On Fri, Jan 12, 2018 at 6:58 AM, Tolga Can <to...@snap.com.invalid>
wrote:

> Hi
>
> We are in the process of upgrading our Kafka cluster from 0.9 to 1.0. We
> will need a cluster similar to LinkedIn’s busiest clusters as described
> here
> <https://docs.confluent.io/current/kafka/deployment.html
> <https://docs.confluent.io/current/kafka/deployment.html>>, 50K
> partitions
> 50 VMs. I could not find enough information about if the latest release is
> being used at production at a big scale.
>
> Is Kafka 1.0 production ready? Is there anyone using it at a very big
> scale?
>
> Regards
> Tc
>