.net and Kafka

[Richard Park <rp...@ancestry.com>]

At Ancestry we're starting a project using Kafka where a large majority of out producer processes are going to be in C#. 

I've done a bit of research and have found a few repos that have c# clients which all seem to be at different points of development. 

So I have two questions:

1. Is there a c# client that is in active or at least semi active development? We are starting with Kafka 0.7 first but eventually we will transition to 0.8. So if anyone is making inroads there that would also be useful information. 

2. Are there any c# based shops using Kafka producers? If so is there another arrangement besides sending messages directly from c# stacks that works well? We floated an idea of possibly writing a java based log scraping tool but direct sending would be cleaner. 

I asked if we could switch all of our code to Java and was threatened with being shot from a cannon into the sun. But any suggestions would still be great. :)

Richard 

Re: .net and Kafka

[Mark Farnan <de...@petrolink.com>]

All, 

We are also working on a C# Client for Kafka,   targeting 0.8 only.  It is a port of the C++ one at this stage.   

Separate from the C# Client, we are also using AVRO internally for ser format and interested in anyone elses experience with this.  I note Microsoft have also been doing some work in this area:  


I anticipate releasing the client as OS when complete,  and would be interested in co-operating if anyone is interested. 

Regards

Mark Farnan. 
CTO - Petrolink International. 


On Sep 14, 2013, at 2:18 AM, Richard Park <rp...@ancestry.com> wrote:

> We are currently trying out AVRO although we're not being very fancy with it.
> 
> We're actually trying to build on top of this client code right now:
> https://github.com/ExactTargetDev/kafka.git
> 
> Out of all the repos I looked at it seems to show the most promise and had some work within this year. We're primarily worrying about this for producers and it seems to work well enough for that.
> 
> I'd love to see anything for 0.8 as I think that's going to be close in our future. Do you have anything out there now?
> 
> Richard
> 
> -----Original Message-----
> From: Henrik Feldt [mailto:Henrik.Feldt@intelliplan.se] 
> Sent: Friday, September 13, 2013 9:36 AM
> To: users@kafka.apache.org
> Subject: Re: .net and Kafka
> 
> Q: Do you also use Avro?
> 
> Regards,
> Henrik
> 
> On 13 sep 2013, at 08:33, Grégoire Seux <g....@criteo.com> wrote:
> 
>> Hi Richard,
>> 
>> we are currently writing a C# driver as all c# drivers are not very active. It has started as a simple translation of the scala driver (producer only) and targets 0.8 protocol.
>> 
>> -- 
>> Grégoire
> 
> 

RE: .net and Kafka

[Richard Park <rp...@ancestry.com>]

We are currently trying out AVRO although we're not being very fancy with it.

We're actually trying to build on top of this client code right now:
https://github.com/ExactTargetDev/kafka.git

Out of all the repos I looked at it seems to show the most promise and had some work within this year. We're primarily worrying about this for producers and it seems to work well enough for that.

I'd love to see anything for 0.8 as I think that's going to be close in our future. Do you have anything out there now?

Richard

-----Original Message-----
From: Henrik Feldt [mailto:Henrik.Feldt@intelliplan.se] 
Sent: Friday, September 13, 2013 9:36 AM
To: users@kafka.apache.org
Subject: Re: .net and Kafka

Q: Do you also use Avro?

Regards,
Henrik

On 13 sep 2013, at 08:33, Grégoire Seux <g....@criteo.com> wrote:

> Hi Richard,
> 
> we are currently writing a C# driver as all c# drivers are not very active. It has started as a simple translation of the scala driver (producer only) and targets 0.8 protocol.
> 
> -- 
> Grégoire

Re: .net and Kafka

[Henrik Feldt <He...@intelliplan.se>]

Q: Do you also use Avro?

Regards,
Henrik

On 13 sep 2013, at 08:33, Grégoire Seux <g....@criteo.com> wrote:

> Hi Richard,
> 
> we are currently writing a C# driver as all c# drivers are not very active. It has started as a simple translation of the scala driver (producer only) and targets 0.8 protocol.
> 
> -- 
> Grégoire

RE: .net and Kafka

[Grégoire Seux <g....@criteo.com>]

Hi Richard,

we are currently writing a C# driver as all c# drivers are not very active. It has started as a simple translation of the scala driver (producer only) and targets 0.8 protocol.

-- 
Grégoire

Re: .net and Kafka

[Henrik Feldt <He...@intelliplan.se>]

We're writing an F# client for 0.8. Cooperate?

On 11 sep 2013, at 17:57, Richard Park <rp...@ancestry.com> wrote:

> At Ancestry we're starting a project using Kafka where a large majority of out producer processes are going to be in C#. 
> 
> I've done a bit of research and have found a few repos that have c# clients which all seem to be at different points of development. 
> 
> So I have two questions:
> 
> 1. Is there a c# client that is in active or at least semi active development? We are starting with Kafka 0.7 first but eventually we will transition to 0.8. So if anyone is making inroads there that would also be useful information. 
> 
> 2. Are there any c# based shops using Kafka producers? If so is there another arrangement besides sending messages directly from c# stacks that works well? We floated an idea of possibly writing a java based log scraping tool but direct sending would be cleaner. 
> 
> I asked if we could switch all of our code to Java and was threatened with being shot from a cannon into the sun. But any suggestions would still be great. :)
> 
> Richard 
>