You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by John Stark <jo...@trimble.com> on 2017/03/20 04:21:46 UTC

Thoughts on AWS I3 series for brokers?

AWS question which may be of interest to other AWS Kafka users.  We are
looking at upgrading our brokers to the new I3
<https://aws.amazon.com/about-aws/whats-new/2017/02/now-available-amazon-ec2-i3-instances-next-generation-storage-optimized-high-i-o-instances/>series.
The I3.16xlarge is the biggest and has the following specs.

   - 488.0 GB memory
   - 64 vCPUs
   - 15200 GB (8 * 1900 GB NVMe SSD) storage
   - 20 Gigabit network performance


The cost ends up being more expensive that our current config but
reasonable for what you get. Does anyone have experience with the I3 series
yet?  Fast disks and 20GB connection will be great but what broker/java
settings should we tweak to maximize the benefits of the memory, CPU, 8
disks, etc.

To round out the cluster, what are AWS servers are people using for
Zookeeper now that offsets are stored in the brokers?  In case it matters,
we are upgrading to 0.10.1.

Thanks for your help,

-John