You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Todd Bilsborrow <tb...@rhythmnewmedia.com> on 2012/06/11 23:09:45 UTC
production deployment
Just curious - how are folks on this list deploying Kafka in a hosted production environment? Do you just unzip/build/run in place, on every broker machine? Or is there a more binary distribution approach that people are taking?
- Todd
Re: production deployment
Posted by Johan Rydberg <jo...@gmail.com>.
we're creating a .rpm. here's the spec-file if someone is interested:
https://gist.github.com/2052835
On Mon, Jun 11, 2012 at 11:09 PM, Todd Bilsborrow
<tb...@rhythmnewmedia.com> wrote:
> Just curious - how are folks on this list deploying Kafka in a hosted production environment? Do you just unzip/build/run in place, on every broker machine? Or is there a more binary distribution approach that people are taking?
>
> - Todd
Re: production deployment
Posted by Evan Chan <ev...@ooyala.com>.
It might help to have a build script create a .zip file with all the
necessary files in it.
At Ooyala we are still using the binary distribution of 0.6, but we use a
Ruby deploy script to rsync the distribution + our own config files.
-Evan
On Mon, Jun 18, 2012 at 10:07 PM, Jay Kreps <ja...@gmail.com> wrote:
> BTW it would be great to get feedback on how we could improve this. The
> linkedin folks are all using linkedin's internal deployment system, so we
> haven't done much thinking in this area and personally I am not up on the
> latest and greatest deployment tools.
>
> -Jay
>
> On Mon, Jun 18, 2012 at 6:41 PM, Chris Burroughs
> <ch...@gmail.com>wrote:
>
> > On 06/11/2012 05:09 PM, Todd Bilsborrow wrote:
> > > Just curious - how are folks on this list deploying Kafka in a hosted
> > production environment? Do you just unzip/build/run in place, on every
> > broker machine? Or is there a more binary distribution approach that
> people
> > are taking?
> > >
> > > - Todd
> > >
> >
> > At AddThis it looks something like:
> > - start with latest Apache release (or svn checkout)
> > - Make a new custom tarball (remove contrib, custom jars, maybe change
> > defaults).
> > - Have a separate set of templated config files (for brokers IDs etc.)
> > - rsync everywhere
> > - Pass off to whatever command and control tool [1] is cool these days.
> >
> >
> > [1] fabric, capistranto, shell scripts, whatever
> >
>
--
--
*Evan Chan*
Senior Software Engineer |
ev@ooyala.com | (650) 996-4600
www.ooyala.com | blog <http://www.ooyala.com/blog> |
@ooyala<http://www.twitter.com/ooyala>
Re: production deployment
Posted by Jay Kreps <ja...@gmail.com>.
BTW it would be great to get feedback on how we could improve this. The
linkedin folks are all using linkedin's internal deployment system, so we
haven't done much thinking in this area and personally I am not up on the
latest and greatest deployment tools.
-Jay
On Mon, Jun 18, 2012 at 6:41 PM, Chris Burroughs
<ch...@gmail.com>wrote:
> On 06/11/2012 05:09 PM, Todd Bilsborrow wrote:
> > Just curious - how are folks on this list deploying Kafka in a hosted
> production environment? Do you just unzip/build/run in place, on every
> broker machine? Or is there a more binary distribution approach that people
> are taking?
> >
> > - Todd
> >
>
> At AddThis it looks something like:
> - start with latest Apache release (or svn checkout)
> - Make a new custom tarball (remove contrib, custom jars, maybe change
> defaults).
> - Have a separate set of templated config files (for brokers IDs etc.)
> - rsync everywhere
> - Pass off to whatever command and control tool [1] is cool these days.
>
>
> [1] fabric, capistranto, shell scripts, whatever
>
Re: production deployment
Posted by Chris Burroughs <ch...@gmail.com>.
On 06/11/2012 05:09 PM, Todd Bilsborrow wrote:
> Just curious - how are folks on this list deploying Kafka in a hosted production environment? Do you just unzip/build/run in place, on every broker machine? Or is there a more binary distribution approach that people are taking?
>
> - Todd
>
At AddThis it looks something like:
- start with latest Apache release (or svn checkout)
- Make a new custom tarball (remove contrib, custom jars, maybe change
defaults).
- Have a separate set of templated config files (for brokers IDs etc.)
- rsync everywhere
- Pass off to whatever command and control tool [1] is cool these days.
[1] fabric, capistranto, shell scripts, whatever