Best Go Client Library

["Dieterly, Deklan" <de...@hp.com>]

I’m looking at implementing Kafka consumers in Go. From the Kafka wiki, I see that there are two clients available.

1. https://github.com/Shopify/sarama
2. https://github.com/stealthly/go_kafka_client

Of the two which is more fully featured? Specifically, I’m looking for automatic load balancing and fail-over when consumer clients fail. I was disappointed with the Python client library (https://github.com/mumrah/kafka-python) that did not have this functionality built into it and I want to make sure that I don’t make another choice were I’ll be disappointed. Are there any reasons to prefer one of the listed Go clients over the other? Are there any gotchas with either of the two client libraries?

Thanks.


>From the wiki at https://cwiki.apache.org/confluence/display/KAFKA/Clients#Clients-Go%28AKAgolang%29
Go (AKA golang)

Pure Go implementation with full protocol support. Consumer and Producer implementations included, GZIP and Snappy compression supported.

Maintainer: Shopify<http://shopify.com>
License: MIT

https://github.com/Shopify/sarama

Another Pure Go implementation built to work as like the "Consumer Group" High Level Consumer with more features including plug-able partition ownership and work fan out.

Maintainer: Big Data Open Source Security<https://www.linkedin.com/company/big-data-open-source-security-llc>
License: Apache v2.0

https://github.com/stealthly/go_kafka_client


Regards.
--
Deklan Dieterly
Hewlett-Packard Company
Sr. Systems Software Engineer
HP Cloud

Re: Best Go Client Library

[Joe Stein <jo...@stealth.ly>]

The Go Kafka Client https://github.com/stealthly/go_kafka_client is a
wrapper around Sarama https://github.com/Shopify/sarama that implements
high level consumer functionality (including automatic load balancing and
fail-over when consumer clients fail) and other features for producing and
consuming with Kafka through Sarama.

/*******************************************
 Joe Stein
 Founder, Principal Consultant
 Big Data Open Source Security LLC
 http://www.stealth.ly
 Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
********************************************/

On Wed, Jan 14, 2015 at 6:42 PM, Dieterly, Deklan <de...@hp.com>
wrote:

> I’m looking at implementing Kafka consumers in Go. From the Kafka wiki, I
> see that there are two clients available.
>
> 1. https://github.com/Shopify/sarama
> 2. https://github.com/stealthly/go_kafka_client
>
> Of the two which is more fully featured? Specifically, I’m looking for
> automatic load balancing and fail-over when consumer clients fail. I was
> disappointed with the Python client library (
> https://github.com/mumrah/kafka-python) that did not have this
> functionality built into it and I want to make sure that I don’t make
> another choice were I’ll be disappointed. Are there any reasons to prefer
> one of the listed Go clients over the other? Are there any gotchas with
> either of the two client libraries?
>
> Thanks.
>
>
> From the wiki at
> https://cwiki.apache.org/confluence/display/KAFKA/Clients#Clients-Go%28AKAgolang%29
> Go (AKA golang)
>
> Pure Go implementation with full protocol support. Consumer and Producer
> implementations included, GZIP and Snappy compression supported.
>
> Maintainer: Shopify<http://shopify.com>
> License: MIT
>
> https://github.com/Shopify/sarama
>
> Another Pure Go implementation built to work as like the "Consumer Group"
> High Level Consumer with more features including plug-able partition
> ownership and work fan out.
>
> Maintainer: Big Data Open Source Security<
> https://www.linkedin.com/company/big-data-open-source-security-llc>
> License: Apache v2.0
>
> https://github.com/stealthly/go_kafka_client
>
>
> Regards.
> --
> Deklan Dieterly
> Hewlett-Packard Company
> Sr. Systems Software Engineer
> HP Cloud
>
>