event-shuttle

[Scott Clasen <sc...@heroku.com>]

Thought Id post this here, as Ive seen questions here around "What do I do
if kafka is down?"

here is one possibility.

https://github.com/sclasen/event-shuttle

this is a go service meant to run as a unix daemon, and apps running on the
same box can just post messages over http to a service on localhost. The
service journals messages through boltdb and delivers them to kafka when
its available.

It also works around the problem of producing to kafka from languages that
have *ahem* less robust client implementations, as hopefully any lang can
post over http.

Most appropriate for emmitters of higher-value, lower-volume events.

Thoughts?
SC

Re: event-shuttle

[Joe Stein <jo...@stealth.ly>]

This does come up often, interesting solution. And another reason to dig
more into Go so I can give this a spin.

/*******************************************
 Joe Stein
 Founder, Principal Consultant
 Big Data Open Source Security LLC
 http://www.stealth.ly
 Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
********************************************/


On Tue, Jun 3, 2014 at 4:09 PM, Scott Clasen <sc...@heroku.com> wrote:

> Thought Id post this here, as Ive seen questions here around "What do I do
> if kafka is down?"
>
> here is one possibility.
>
> https://github.com/sclasen/event-shuttle
>
> this is a go service meant to run as a unix daemon, and apps running on the
> same box can just post messages over http to a service on localhost. The
> service journals messages through boltdb and delivers them to kafka when
> its available.
>
> It also works around the problem of producing to kafka from languages that
> have *ahem* less robust client implementations, as hopefully any lang can
> post over http.
>
> Most appropriate for emmitters of higher-value, lower-volume events.
>
> Thoughts?
> SC
>