You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Mingtao Zhang <ma...@gmail.com> on 2014/09/30 17:46:19 UTC
kafka & docker
Hi,
Any one has dockerized kafka working?
Should we specify the ip address?
I expected everything working on just localhost but saw this "SEVERE:
Producer connection to 172.17.0.3:9092 unsuccessful".
Thanks in advance!
Best Regards,
Mingtao
Re: kafka & docker
Posted by Joe Stein <jo...@stealth.ly>.
Yes, here is a vagrant virtual box setup
https://github.com/stealthly/scala-kafka
On Thu, Oct 2, 2014 at 3:51 PM, Mingtao Zhang <ma...@gmail.com>
wrote:
> Thanks for the response!
>
> Any one has it working on Virtualbox? which is the case for Winddos/Mac?
>
> How do we configure the network adapter?
>
> Best Regards,
> Mingtao
>
> Best Regards,
> Mingtao
>
> On Tue, Sep 30, 2014 at 3:31 PM, Joe Stein <jo...@stealth.ly> wrote:
>
> > << Is there a 'Kafka->HDFS with Camus' docker as well one can play
> around
> > with?
> >
> > Not that I know of. These folks
> > http://blog.sequenceiq.com/blog/2014/09/15/hadoop-2-5-1-docker/ have
> nice
> > Hadoop docker containers may be a good starting point.
> >
> > << What's the story for persisting data with Docker? Do you use a data
> > volume or do you just start fresh every time you start the Docker
> instance?
> >
> > We only use it for development & testing so starting fresh and
> > bootstrapping data is how we use it when we do. Volumes should work fine
> > though for persisting if need be.
> >
> > /*******************************************
> > Joe Stein
> > Founder, Principal Consultant
> > Big Data Open Source Security LLC
> > http://www.stealth.ly
> > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> > ********************************************/
> >
> > On Tue, Sep 30, 2014 at 3:00 PM, Daniel Compton <de...@danielcompton.net>
> > wrote:
> >
> > > Hi Joe
> > >
> > > What's the story for persisting data with Docker? Do you use a data
> > volume
> > > or do you just start fresh every time you start the Docker instance?
> > >
> > > Daniel.
> > >
> > > > On 1/10/2014, at 7:13 am, Buntu Dev <bu...@gmail.com> wrote:
> > > >
> > > > Thanks Joe.. seems quite handy. Is there a 'Kafka->HDFS with Camus'
> > > docker
> > > > as well one can play around with?
> > > >
> > > >> On Tue, Sep 30, 2014 at 9:00 AM, Joe Stein <jo...@stealth.ly>
> > > wrote:
> > > >>
> > > >> You need to change the advertised hostname.
> > > >>
> > > >> Take a look
> https://registry.hub.docker.com/u/stealthly/docker-kafka/
> > > and
> > > >> https://registry.hub.docker.com/u/stealthly/docker-zookeeper/ we
> use
> > it
> > > >> often for local testing here is how to start
> > > >>
> https://github.com/stealthly/docker-kafka/blob/master/start-broker.sh
> > > e.g.
> > > >>
> > https://github.com/stealthly/metrics-kafka/blob/master/bootstrap.sh#L13
> > > >>
> > > >> /*******************************************
> > > >> Joe Stein
> > > >> Founder, Principal Consultant
> > > >> Big Data Open Source Security LLC
> > > >> http://www.stealth.ly
> > > >> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> > > >> ********************************************/
> > > >>
> > > >> On Tue, Sep 30, 2014 at 11:46 AM, Mingtao Zhang <
> > mail2mingtao@gmail.com
> > > >
> > > >> wrote:
> > > >>
> > > >>> Hi,
> > > >>>
> > > >>> Any one has dockerized kafka working?
> > > >>>
> > > >>> Should we specify the ip address?
> > > >>>
> > > >>> I expected everything working on just localhost but saw this
> "SEVERE:
> > > >>> Producer connection to 172.17.0.3:9092 unsuccessful".
> > > >>>
> > > >>> Thanks in advance!
> > > >>>
> > > >>> Best Regards,
> > > >>> Mingtao
> > > >>
> > >
> >
>
Re: kafka & docker
Posted by Mingtao Zhang <ma...@gmail.com>.
Thanks for the response!
Any one has it working on Virtualbox? which is the case for Winddos/Mac?
How do we configure the network adapter?
Best Regards,
Mingtao
Best Regards,
Mingtao
On Tue, Sep 30, 2014 at 3:31 PM, Joe Stein <jo...@stealth.ly> wrote:
> << Is there a 'Kafka->HDFS with Camus' docker as well one can play around
> with?
>
> Not that I know of. These folks
> http://blog.sequenceiq.com/blog/2014/09/15/hadoop-2-5-1-docker/ have nice
> Hadoop docker containers may be a good starting point.
>
> << What's the story for persisting data with Docker? Do you use a data
> volume or do you just start fresh every time you start the Docker instance?
>
> We only use it for development & testing so starting fresh and
> bootstrapping data is how we use it when we do. Volumes should work fine
> though for persisting if need be.
>
> /*******************************************
> Joe Stein
> Founder, Principal Consultant
> Big Data Open Source Security LLC
> http://www.stealth.ly
> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> ********************************************/
>
> On Tue, Sep 30, 2014 at 3:00 PM, Daniel Compton <de...@danielcompton.net>
> wrote:
>
> > Hi Joe
> >
> > What's the story for persisting data with Docker? Do you use a data
> volume
> > or do you just start fresh every time you start the Docker instance?
> >
> > Daniel.
> >
> > > On 1/10/2014, at 7:13 am, Buntu Dev <bu...@gmail.com> wrote:
> > >
> > > Thanks Joe.. seems quite handy. Is there a 'Kafka->HDFS with Camus'
> > docker
> > > as well one can play around with?
> > >
> > >> On Tue, Sep 30, 2014 at 9:00 AM, Joe Stein <jo...@stealth.ly>
> > wrote:
> > >>
> > >> You need to change the advertised hostname.
> > >>
> > >> Take a look https://registry.hub.docker.com/u/stealthly/docker-kafka/
> > and
> > >> https://registry.hub.docker.com/u/stealthly/docker-zookeeper/ we use
> it
> > >> often for local testing here is how to start
> > >> https://github.com/stealthly/docker-kafka/blob/master/start-broker.sh
> > e.g.
> > >>
> https://github.com/stealthly/metrics-kafka/blob/master/bootstrap.sh#L13
> > >>
> > >> /*******************************************
> > >> Joe Stein
> > >> Founder, Principal Consultant
> > >> Big Data Open Source Security LLC
> > >> http://www.stealth.ly
> > >> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> > >> ********************************************/
> > >>
> > >> On Tue, Sep 30, 2014 at 11:46 AM, Mingtao Zhang <
> mail2mingtao@gmail.com
> > >
> > >> wrote:
> > >>
> > >>> Hi,
> > >>>
> > >>> Any one has dockerized kafka working?
> > >>>
> > >>> Should we specify the ip address?
> > >>>
> > >>> I expected everything working on just localhost but saw this "SEVERE:
> > >>> Producer connection to 172.17.0.3:9092 unsuccessful".
> > >>>
> > >>> Thanks in advance!
> > >>>
> > >>> Best Regards,
> > >>> Mingtao
> > >>
> >
>
Re: kafka & docker
Posted by Joe Stein <jo...@stealth.ly>.
<< Is there a 'Kafka->HDFS with Camus' docker as well one can play around
with?
Not that I know of. These folks
http://blog.sequenceiq.com/blog/2014/09/15/hadoop-2-5-1-docker/ have nice
Hadoop docker containers may be a good starting point.
<< What's the story for persisting data with Docker? Do you use a data
volume or do you just start fresh every time you start the Docker instance?
We only use it for development & testing so starting fresh and
bootstrapping data is how we use it when we do. Volumes should work fine
though for persisting if need be.
/*******************************************
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
********************************************/
On Tue, Sep 30, 2014 at 3:00 PM, Daniel Compton <de...@danielcompton.net>
wrote:
> Hi Joe
>
> What's the story for persisting data with Docker? Do you use a data volume
> or do you just start fresh every time you start the Docker instance?
>
> Daniel.
>
> > On 1/10/2014, at 7:13 am, Buntu Dev <bu...@gmail.com> wrote:
> >
> > Thanks Joe.. seems quite handy. Is there a 'Kafka->HDFS with Camus'
> docker
> > as well one can play around with?
> >
> >> On Tue, Sep 30, 2014 at 9:00 AM, Joe Stein <jo...@stealth.ly>
> wrote:
> >>
> >> You need to change the advertised hostname.
> >>
> >> Take a look https://registry.hub.docker.com/u/stealthly/docker-kafka/
> and
> >> https://registry.hub.docker.com/u/stealthly/docker-zookeeper/ we use it
> >> often for local testing here is how to start
> >> https://github.com/stealthly/docker-kafka/blob/master/start-broker.sh
> e.g.
> >> https://github.com/stealthly/metrics-kafka/blob/master/bootstrap.sh#L13
> >>
> >> /*******************************************
> >> Joe Stein
> >> Founder, Principal Consultant
> >> Big Data Open Source Security LLC
> >> http://www.stealth.ly
> >> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> >> ********************************************/
> >>
> >> On Tue, Sep 30, 2014 at 11:46 AM, Mingtao Zhang <mail2mingtao@gmail.com
> >
> >> wrote:
> >>
> >>> Hi,
> >>>
> >>> Any one has dockerized kafka working?
> >>>
> >>> Should we specify the ip address?
> >>>
> >>> I expected everything working on just localhost but saw this "SEVERE:
> >>> Producer connection to 172.17.0.3:9092 unsuccessful".
> >>>
> >>> Thanks in advance!
> >>>
> >>> Best Regards,
> >>> Mingtao
> >>
>
Re: kafka & docker
Posted by Daniel Compton <de...@danielcompton.net>.
Hi Joe
What's the story for persisting data with Docker? Do you use a data volume or do you just start fresh every time you start the Docker instance?
Daniel.
> On 1/10/2014, at 7:13 am, Buntu Dev <bu...@gmail.com> wrote:
>
> Thanks Joe.. seems quite handy. Is there a 'Kafka->HDFS with Camus' docker
> as well one can play around with?
>
>> On Tue, Sep 30, 2014 at 9:00 AM, Joe Stein <jo...@stealth.ly> wrote:
>>
>> You need to change the advertised hostname.
>>
>> Take a look https://registry.hub.docker.com/u/stealthly/docker-kafka/ and
>> https://registry.hub.docker.com/u/stealthly/docker-zookeeper/ we use it
>> often for local testing here is how to start
>> https://github.com/stealthly/docker-kafka/blob/master/start-broker.sh e.g.
>> https://github.com/stealthly/metrics-kafka/blob/master/bootstrap.sh#L13
>>
>> /*******************************************
>> Joe Stein
>> Founder, Principal Consultant
>> Big Data Open Source Security LLC
>> http://www.stealth.ly
>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>> ********************************************/
>>
>> On Tue, Sep 30, 2014 at 11:46 AM, Mingtao Zhang <ma...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Any one has dockerized kafka working?
>>>
>>> Should we specify the ip address?
>>>
>>> I expected everything working on just localhost but saw this "SEVERE:
>>> Producer connection to 172.17.0.3:9092 unsuccessful".
>>>
>>> Thanks in advance!
>>>
>>> Best Regards,
>>> Mingtao
>>
Re: kafka & docker
Posted by Buntu Dev <bu...@gmail.com>.
Thanks Joe.. seems quite handy. Is there a 'Kafka->HDFS with Camus' docker
as well one can play around with?
On Tue, Sep 30, 2014 at 9:00 AM, Joe Stein <jo...@stealth.ly> wrote:
> You need to change the advertised hostname.
>
> Take a look https://registry.hub.docker.com/u/stealthly/docker-kafka/ and
> https://registry.hub.docker.com/u/stealthly/docker-zookeeper/ we use it
> often for local testing here is how to start
> https://github.com/stealthly/docker-kafka/blob/master/start-broker.sh e.g.
> https://github.com/stealthly/metrics-kafka/blob/master/bootstrap.sh#L13
>
> /*******************************************
> Joe Stein
> Founder, Principal Consultant
> Big Data Open Source Security LLC
> http://www.stealth.ly
> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> ********************************************/
>
> On Tue, Sep 30, 2014 at 11:46 AM, Mingtao Zhang <ma...@gmail.com>
> wrote:
>
> > Hi,
> >
> > Any one has dockerized kafka working?
> >
> > Should we specify the ip address?
> >
> > I expected everything working on just localhost but saw this "SEVERE:
> > Producer connection to 172.17.0.3:9092 unsuccessful".
> >
> > Thanks in advance!
> >
> > Best Regards,
> > Mingtao
> >
>
Re: kafka & docker
Posted by Joe Stein <jo...@stealth.ly>.
You need to change the advertised hostname.
Take a look https://registry.hub.docker.com/u/stealthly/docker-kafka/ and
https://registry.hub.docker.com/u/stealthly/docker-zookeeper/ we use it
often for local testing here is how to start
https://github.com/stealthly/docker-kafka/blob/master/start-broker.sh e.g.
https://github.com/stealthly/metrics-kafka/blob/master/bootstrap.sh#L13
/*******************************************
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
********************************************/
On Tue, Sep 30, 2014 at 11:46 AM, Mingtao Zhang <ma...@gmail.com>
wrote:
> Hi,
>
> Any one has dockerized kafka working?
>
> Should we specify the ip address?
>
> I expected everything working on just localhost but saw this "SEVERE:
> Producer connection to 172.17.0.3:9092 unsuccessful".
>
> Thanks in advance!
>
> Best Regards,
> Mingtao
>