You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by "Marton, Elek" <hd...@anzix.net> on 2017/09/08 12:45:40 UTC
[DISCUSS] official docker image(s) for hadoop
TL;DR: I propose to create official hadoop images and upload them to the
dockerhub.
GOAL/SCOPE: I would like improve the existing documentation with
easy-to-use docker based recipes to start hadoop clusters with various
configuration.
The images also could be used to test experimental features. For example
ozone could be tested easily with these compose file and configuration:
https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
Or even the configuration could be included in the compose file:
https://github.com/elek/hadoop/blob/docker-2.8.0/example/docker-compose.yaml
I would like to create separated example compose files for federation,
ha, metrics usage, etc. to make it easier to try out and understand the
features.
CONTEXT: There is an existing Jira
https://issues.apache.org/jira/browse/HADOOP-13397
But it’s about a tool to generate production quality docker images
(multiple types, in a flexible way). If no objections, I will create a
separated issue to create simplified docker images for rapid prototyping
and investigating new features. And register the branch to the dockerhub
to create the images automatically.
MY BACKGROUND: I am working with docker based hadoop/spark clusters
quite a while and run them succesfully in different environments
(kubernetes, docker-swarm, nomad-based scheduling, etc.) My work is
available from here: https://github.com/flokkr but they could handle
more complex use cases (eg. instrumenting java processes with btrace, or
read/reload configuration from consul).
And IMHO in the official hadoop documentation it’s better to suggest
to use official apache docker images and not external ones (which could
be changed).
Please let me know if you have any comments.
Marton
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [DISCUSS] official docker image(s) for hadoop
Posted by "Marton, Elek" <hd...@anzix.net>.
Thanks all the feedbacks.
I created an issue:
https://issues.apache.org/jira/browse/HADOOP-14898
Let's continue the discussion there.
Thanks,
Marton
On 09/08/2017 02:45 PM, Marton, Elek wrote:
>
> TL;DR: I propose to create official hadoop images and upload them to the
> dockerhub.
>
> GOAL/SCOPE: I would like improve the existing documentation with
> easy-to-use docker based recipes to start hadoop clusters with various
> configuration.
>
> The images also could be used to test experimental features. For example
> ozone could be tested easily with these compose file and configuration:
>
> https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
>
> Or even the configuration could be included in the compose file:
>
> https://github.com/elek/hadoop/blob/docker-2.8.0/example/docker-compose.yaml
>
>
> I would like to create separated example compose files for federation,
> ha, metrics usage, etc. to make it easier to try out and understand the
> features.
>
> CONTEXT: There is an existing Jira
> https://issues.apache.org/jira/browse/HADOOP-13397
> But it’s about a tool to generate production quality docker images
> (multiple types, in a flexible way). If no objections, I will create a
> separated issue to create simplified docker images for rapid prototyping
> and investigating new features. And register the branch to the dockerhub
> to create the images automatically.
>
> MY BACKGROUND: I am working with docker based hadoop/spark clusters
> quite a while and run them succesfully in different environments
> (kubernetes, docker-swarm, nomad-based scheduling, etc.) My work is
> available from here: https://github.com/flokkr but they could handle
> more complex use cases (eg. instrumenting java processes with btrace, or
> read/reload configuration from consul).
> And IMHO in the official hadoop documentation it’s better to suggest
> to use official apache docker images and not external ones (which could
> be changed).
>
> Please let me know if you have any comments.
>
> Marton
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [DISCUSS] official docker image(s) for hadoop
Posted by Bharat Viswanadham <bv...@hortonworks.com>.
+1 (non-binding)
It would be really nice to have Docker to try different features of Hadoop (like HA, Federation Enabled, Erasure coding…), which will helpful for both developers and users.
Thanks,
Bharat
On 9/13/17, 11:31 AM, "Eric Badger" <eb...@oath.com.INVALID> wrote:
+1 definitely think an official Hadoop docker image (possibly 1 per major
or minor release) would be a positive both for contributors and for users
of Hadoop.
Eric
On Wed, Sep 13, 2017 at 1:19 PM, Wangda Tan <wh...@gmail.com> wrote:
> +1 to add Hadoop docker image for easier testing / prototyping, it gonna be
> super helpful!
>
> Thanks,
> Wangda
>
> On Wed, Sep 13, 2017 at 10:48 AM, Miklos Szegedi <
> miklos.szegedi@cloudera.com> wrote:
>
> > Marton, thank you for working on this. I think Official Docker images for
> > Hadoop would be very useful for a lot of reasons. I think that it is
> better
> > to have a coordinated effort with production ready base images with
> > dependent images for prototyping. Does anyone else have an opinion about
> > this?
> >
> > Thank you,
> > Miklos
> >
> > On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <hd...@anzix.net> wrote:
> >
> > >
> > > TL;DR: I propose to create official hadoop images and upload them to
> the
> > > dockerhub.
> > >
> > > GOAL/SCOPE: I would like improve the existing documentation with
> > > easy-to-use docker based recipes to start hadoop clusters with various
> > > configuration.
> > >
> > > The images also could be used to test experimental features. For
> example
> > > ozone could be tested easily with these compose file and configuration:
> > >
> > > https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
> > >
> > > Or even the configuration could be included in the compose file:
> > >
> > > https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc
> > > ker-compose.yaml
> > >
> > > I would like to create separated example compose files for federation,
> > ha,
> > > metrics usage, etc. to make it easier to try out and understand the
> > > features.
> > >
> > > CONTEXT: There is an existing Jira https://issues.apache.org/jira
> > > /browse/HADOOP-13397
> > > But it’s about a tool to generate production quality docker images
> > > (multiple types, in a flexible way). If no objections, I will create a
> > > separated issue to create simplified docker images for rapid
> prototyping
> > > and investigating new features. And register the branch to the
> dockerhub
> > to
> > > create the images automatically.
> > >
> > > MY BACKGROUND: I am working with docker based hadoop/spark clusters
> quite
> > > a while and run them succesfully in different environments (kubernetes,
> > > docker-swarm, nomad-based scheduling, etc.) My work is available from
> > here:
> > > https://github.com/flokkr but they could handle more complex use cases
> > > (eg. instrumenting java processes with btrace, or read/reload
> > configuration
> > > from consul).
> > > And IMHO in the official hadoop documentation it’s better to suggest
> to
> > > use official apache docker images and not external ones (which could be
> > > changed).
> > >
> > > Please let me know if you have any comments.
> > >
> > > Marton
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> > >
> > >
> >
>
Re: [DISCUSS] official docker image(s) for hadoop
Posted by Bharat Viswanadham <bv...@hortonworks.com>.
+1 (non-binding)
It would be really nice to have Docker to try different features of Hadoop (like HA, Federation Enabled, Erasure coding…), which will helpful for both developers and users.
Thanks,
Bharat
On 9/13/17, 11:31 AM, "Eric Badger" <eb...@oath.com.INVALID> wrote:
+1 definitely think an official Hadoop docker image (possibly 1 per major
or minor release) would be a positive both for contributors and for users
of Hadoop.
Eric
On Wed, Sep 13, 2017 at 1:19 PM, Wangda Tan <wh...@gmail.com> wrote:
> +1 to add Hadoop docker image for easier testing / prototyping, it gonna be
> super helpful!
>
> Thanks,
> Wangda
>
> On Wed, Sep 13, 2017 at 10:48 AM, Miklos Szegedi <
> miklos.szegedi@cloudera.com> wrote:
>
> > Marton, thank you for working on this. I think Official Docker images for
> > Hadoop would be very useful for a lot of reasons. I think that it is
> better
> > to have a coordinated effort with production ready base images with
> > dependent images for prototyping. Does anyone else have an opinion about
> > this?
> >
> > Thank you,
> > Miklos
> >
> > On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <hd...@anzix.net> wrote:
> >
> > >
> > > TL;DR: I propose to create official hadoop images and upload them to
> the
> > > dockerhub.
> > >
> > > GOAL/SCOPE: I would like improve the existing documentation with
> > > easy-to-use docker based recipes to start hadoop clusters with various
> > > configuration.
> > >
> > > The images also could be used to test experimental features. For
> example
> > > ozone could be tested easily with these compose file and configuration:
> > >
> > > https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
> > >
> > > Or even the configuration could be included in the compose file:
> > >
> > > https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc
> > > ker-compose.yaml
> > >
> > > I would like to create separated example compose files for federation,
> > ha,
> > > metrics usage, etc. to make it easier to try out and understand the
> > > features.
> > >
> > > CONTEXT: There is an existing Jira https://issues.apache.org/jira
> > > /browse/HADOOP-13397
> > > But it’s about a tool to generate production quality docker images
> > > (multiple types, in a flexible way). If no objections, I will create a
> > > separated issue to create simplified docker images for rapid
> prototyping
> > > and investigating new features. And register the branch to the
> dockerhub
> > to
> > > create the images automatically.
> > >
> > > MY BACKGROUND: I am working with docker based hadoop/spark clusters
> quite
> > > a while and run them succesfully in different environments (kubernetes,
> > > docker-swarm, nomad-based scheduling, etc.) My work is available from
> > here:
> > > https://github.com/flokkr but they could handle more complex use cases
> > > (eg. instrumenting java processes with btrace, or read/reload
> > configuration
> > > from consul).
> > > And IMHO in the official hadoop documentation it’s better to suggest
> to
> > > use official apache docker images and not external ones (which could be
> > > changed).
> > >
> > > Please let me know if you have any comments.
> > >
> > > Marton
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> > >
> > >
> >
>
Re: [DISCUSS] official docker image(s) for hadoop
Posted by Eric Badger <eb...@oath.com.INVALID>.
+1 definitely think an official Hadoop docker image (possibly 1 per major
or minor release) would be a positive both for contributors and for users
of Hadoop.
Eric
On Wed, Sep 13, 2017 at 1:19 PM, Wangda Tan <wh...@gmail.com> wrote:
> +1 to add Hadoop docker image for easier testing / prototyping, it gonna be
> super helpful!
>
> Thanks,
> Wangda
>
> On Wed, Sep 13, 2017 at 10:48 AM, Miklos Szegedi <
> miklos.szegedi@cloudera.com> wrote:
>
> > Marton, thank you for working on this. I think Official Docker images for
> > Hadoop would be very useful for a lot of reasons. I think that it is
> better
> > to have a coordinated effort with production ready base images with
> > dependent images for prototyping. Does anyone else have an opinion about
> > this?
> >
> > Thank you,
> > Miklos
> >
> > On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <hd...@anzix.net> wrote:
> >
> > >
> > > TL;DR: I propose to create official hadoop images and upload them to
> the
> > > dockerhub.
> > >
> > > GOAL/SCOPE: I would like improve the existing documentation with
> > > easy-to-use docker based recipes to start hadoop clusters with various
> > > configuration.
> > >
> > > The images also could be used to test experimental features. For
> example
> > > ozone could be tested easily with these compose file and configuration:
> > >
> > > https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
> > >
> > > Or even the configuration could be included in the compose file:
> > >
> > > https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc
> > > ker-compose.yaml
> > >
> > > I would like to create separated example compose files for federation,
> > ha,
> > > metrics usage, etc. to make it easier to try out and understand the
> > > features.
> > >
> > > CONTEXT: There is an existing Jira https://issues.apache.org/jira
> > > /browse/HADOOP-13397
> > > But it’s about a tool to generate production quality docker images
> > > (multiple types, in a flexible way). If no objections, I will create a
> > > separated issue to create simplified docker images for rapid
> prototyping
> > > and investigating new features. And register the branch to the
> dockerhub
> > to
> > > create the images automatically.
> > >
> > > MY BACKGROUND: I am working with docker based hadoop/spark clusters
> quite
> > > a while and run them succesfully in different environments (kubernetes,
> > > docker-swarm, nomad-based scheduling, etc.) My work is available from
> > here:
> > > https://github.com/flokkr but they could handle more complex use cases
> > > (eg. instrumenting java processes with btrace, or read/reload
> > configuration
> > > from consul).
> > > And IMHO in the official hadoop documentation it’s better to suggest
> to
> > > use official apache docker images and not external ones (which could be
> > > changed).
> > >
> > > Please let me know if you have any comments.
> > >
> > > Marton
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> > >
> > >
> >
>
Re: [DISCUSS] official docker image(s) for hadoop
Posted by Mingliang Liu <li...@gmail.com>.
> It would be very helpful for testing the RC.
For testing and voting, I have been using docker containers for a while, see code at: https://github.com/weiqingy/caochong <https://github.com/weiqingy/caochong>
> TL;DR: I propose to create official hadoop images and upload them to the dockerhub
I’m +1 on this idea. The “official” docker image basically means a commitment to maintain well documented and broadly tested images, which seems not a burden to us.
Ceph has a community docker project https://github.com/ceph/ceph-docker <https://github.com/ceph/ceph-docker> and I think our scope here is similar to it.
Mingliang
> On Sep 13, 2017, at 11:39 AM, Yufei Gu <fl...@gmail.com> wrote:
>
> It would be very helpful for testing the RC. To vote a RC, committers and
> PMCs usually spend lots of time to compile, deploy the RC, do several
> sanity tests, then +1 for the RC. The docker image potentially saves the
> compilation and deployment time, and people can do more tests.
>
> Best,
>
> Yufei
>
> On Wed, Sep 13, 2017 at 11:19 AM, Wangda Tan <wh...@gmail.com> wrote:
>
>> +1 to add Hadoop docker image for easier testing / prototyping, it gonna be
>> super helpful!
>>
>> Thanks,
>> Wangda
>>
>> On Wed, Sep 13, 2017 at 10:48 AM, Miklos Szegedi <
>> miklos.szegedi@cloudera.com> wrote:
>>
>>> Marton, thank you for working on this. I think Official Docker images for
>>> Hadoop would be very useful for a lot of reasons. I think that it is
>> better
>>> to have a coordinated effort with production ready base images with
>>> dependent images for prototyping. Does anyone else have an opinion about
>>> this?
>>>
>>> Thank you,
>>> Miklos
>>>
>>> On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <hd...@anzix.net> wrote:
>>>
>>>>
>>>> TL;DR: I propose to create official hadoop images and upload them to
>> the
>>>> dockerhub.
>>>>
>>>> GOAL/SCOPE: I would like improve the existing documentation with
>>>> easy-to-use docker based recipes to start hadoop clusters with various
>>>> configuration.
>>>>
>>>> The images also could be used to test experimental features. For
>> example
>>>> ozone could be tested easily with these compose file and configuration:
>>>>
>>>> https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
>>>>
>>>> Or even the configuration could be included in the compose file:
>>>>
>>>> https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc
>>>> ker-compose.yaml
>>>>
>>>> I would like to create separated example compose files for federation,
>>> ha,
>>>> metrics usage, etc. to make it easier to try out and understand the
>>>> features.
>>>>
>>>> CONTEXT: There is an existing Jira https://issues.apache.org/jira
>>>> /browse/HADOOP-13397
>>>> But it’s about a tool to generate production quality docker images
>>>> (multiple types, in a flexible way). If no objections, I will create a
>>>> separated issue to create simplified docker images for rapid
>> prototyping
>>>> and investigating new features. And register the branch to the
>> dockerhub
>>> to
>>>> create the images automatically.
>>>>
>>>> MY BACKGROUND: I am working with docker based hadoop/spark clusters
>> quite
>>>> a while and run them succesfully in different environments (kubernetes,
>>>> docker-swarm, nomad-based scheduling, etc.) My work is available from
>>> here:
>>>> https://github.com/flokkr but they could handle more complex use cases
>>>> (eg. instrumenting java processes with btrace, or read/reload
>>> configuration
>>>> from consul).
>>>> And IMHO in the official hadoop documentation it’s better to suggest
>> to
>>>> use official apache docker images and not external ones (which could be
>>>> changed).
>>>>
>>>> Please let me know if you have any comments.
>>>>
>>>> Marton
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>>
>>>>
>>>
>>
Re: [DISCUSS] official docker image(s) for hadoop
Posted by Mingliang Liu <li...@gmail.com>.
> It would be very helpful for testing the RC.
For testing and voting, I have been using docker containers for a while, see code at: https://github.com/weiqingy/caochong <https://github.com/weiqingy/caochong>
> TL;DR: I propose to create official hadoop images and upload them to the dockerhub
I’m +1 on this idea. The “official” docker image basically means a commitment to maintain well documented and broadly tested images, which seems not a burden to us.
Ceph has a community docker project https://github.com/ceph/ceph-docker <https://github.com/ceph/ceph-docker> and I think our scope here is similar to it.
Mingliang
> On Sep 13, 2017, at 11:39 AM, Yufei Gu <fl...@gmail.com> wrote:
>
> It would be very helpful for testing the RC. To vote a RC, committers and
> PMCs usually spend lots of time to compile, deploy the RC, do several
> sanity tests, then +1 for the RC. The docker image potentially saves the
> compilation and deployment time, and people can do more tests.
>
> Best,
>
> Yufei
>
> On Wed, Sep 13, 2017 at 11:19 AM, Wangda Tan <wh...@gmail.com> wrote:
>
>> +1 to add Hadoop docker image for easier testing / prototyping, it gonna be
>> super helpful!
>>
>> Thanks,
>> Wangda
>>
>> On Wed, Sep 13, 2017 at 10:48 AM, Miklos Szegedi <
>> miklos.szegedi@cloudera.com> wrote:
>>
>>> Marton, thank you for working on this. I think Official Docker images for
>>> Hadoop would be very useful for a lot of reasons. I think that it is
>> better
>>> to have a coordinated effort with production ready base images with
>>> dependent images for prototyping. Does anyone else have an opinion about
>>> this?
>>>
>>> Thank you,
>>> Miklos
>>>
>>> On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <hd...@anzix.net> wrote:
>>>
>>>>
>>>> TL;DR: I propose to create official hadoop images and upload them to
>> the
>>>> dockerhub.
>>>>
>>>> GOAL/SCOPE: I would like improve the existing documentation with
>>>> easy-to-use docker based recipes to start hadoop clusters with various
>>>> configuration.
>>>>
>>>> The images also could be used to test experimental features. For
>> example
>>>> ozone could be tested easily with these compose file and configuration:
>>>>
>>>> https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
>>>>
>>>> Or even the configuration could be included in the compose file:
>>>>
>>>> https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc
>>>> ker-compose.yaml
>>>>
>>>> I would like to create separated example compose files for federation,
>>> ha,
>>>> metrics usage, etc. to make it easier to try out and understand the
>>>> features.
>>>>
>>>> CONTEXT: There is an existing Jira https://issues.apache.org/jira
>>>> /browse/HADOOP-13397
>>>> But it’s about a tool to generate production quality docker images
>>>> (multiple types, in a flexible way). If no objections, I will create a
>>>> separated issue to create simplified docker images for rapid
>> prototyping
>>>> and investigating new features. And register the branch to the
>> dockerhub
>>> to
>>>> create the images automatically.
>>>>
>>>> MY BACKGROUND: I am working with docker based hadoop/spark clusters
>> quite
>>>> a while and run them succesfully in different environments (kubernetes,
>>>> docker-swarm, nomad-based scheduling, etc.) My work is available from
>>> here:
>>>> https://github.com/flokkr but they could handle more complex use cases
>>>> (eg. instrumenting java processes with btrace, or read/reload
>>> configuration
>>>> from consul).
>>>> And IMHO in the official hadoop documentation it’s better to suggest
>> to
>>>> use official apache docker images and not external ones (which could be
>>>> changed).
>>>>
>>>> Please let me know if you have any comments.
>>>>
>>>> Marton
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>>
>>>>
>>>
>>
Re: [DISCUSS] official docker image(s) for hadoop
Posted by Mingliang Liu <li...@gmail.com>.
> It would be very helpful for testing the RC.
For testing and voting, I have been using docker containers for a while, see code at: https://github.com/weiqingy/caochong <https://github.com/weiqingy/caochong>
> TL;DR: I propose to create official hadoop images and upload them to the dockerhub
I’m +1 on this idea. The “official” docker image basically means a commitment to maintain well documented and broadly tested images, which seems not a burden to us.
Ceph has a community docker project https://github.com/ceph/ceph-docker <https://github.com/ceph/ceph-docker> and I think our scope here is similar to it.
Mingliang
> On Sep 13, 2017, at 11:39 AM, Yufei Gu <fl...@gmail.com> wrote:
>
> It would be very helpful for testing the RC. To vote a RC, committers and
> PMCs usually spend lots of time to compile, deploy the RC, do several
> sanity tests, then +1 for the RC. The docker image potentially saves the
> compilation and deployment time, and people can do more tests.
>
> Best,
>
> Yufei
>
> On Wed, Sep 13, 2017 at 11:19 AM, Wangda Tan <wh...@gmail.com> wrote:
>
>> +1 to add Hadoop docker image for easier testing / prototyping, it gonna be
>> super helpful!
>>
>> Thanks,
>> Wangda
>>
>> On Wed, Sep 13, 2017 at 10:48 AM, Miklos Szegedi <
>> miklos.szegedi@cloudera.com> wrote:
>>
>>> Marton, thank you for working on this. I think Official Docker images for
>>> Hadoop would be very useful for a lot of reasons. I think that it is
>> better
>>> to have a coordinated effort with production ready base images with
>>> dependent images for prototyping. Does anyone else have an opinion about
>>> this?
>>>
>>> Thank you,
>>> Miklos
>>>
>>> On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <hd...@anzix.net> wrote:
>>>
>>>>
>>>> TL;DR: I propose to create official hadoop images and upload them to
>> the
>>>> dockerhub.
>>>>
>>>> GOAL/SCOPE: I would like improve the existing documentation with
>>>> easy-to-use docker based recipes to start hadoop clusters with various
>>>> configuration.
>>>>
>>>> The images also could be used to test experimental features. For
>> example
>>>> ozone could be tested easily with these compose file and configuration:
>>>>
>>>> https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
>>>>
>>>> Or even the configuration could be included in the compose file:
>>>>
>>>> https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc
>>>> ker-compose.yaml
>>>>
>>>> I would like to create separated example compose files for federation,
>>> ha,
>>>> metrics usage, etc. to make it easier to try out and understand the
>>>> features.
>>>>
>>>> CONTEXT: There is an existing Jira https://issues.apache.org/jira
>>>> /browse/HADOOP-13397
>>>> But it’s about a tool to generate production quality docker images
>>>> (multiple types, in a flexible way). If no objections, I will create a
>>>> separated issue to create simplified docker images for rapid
>> prototyping
>>>> and investigating new features. And register the branch to the
>> dockerhub
>>> to
>>>> create the images automatically.
>>>>
>>>> MY BACKGROUND: I am working with docker based hadoop/spark clusters
>> quite
>>>> a while and run them succesfully in different environments (kubernetes,
>>>> docker-swarm, nomad-based scheduling, etc.) My work is available from
>>> here:
>>>> https://github.com/flokkr but they could handle more complex use cases
>>>> (eg. instrumenting java processes with btrace, or read/reload
>>> configuration
>>>> from consul).
>>>> And IMHO in the official hadoop documentation it’s better to suggest
>> to
>>>> use official apache docker images and not external ones (which could be
>>>> changed).
>>>>
>>>> Please let me know if you have any comments.
>>>>
>>>> Marton
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>>
>>>>
>>>
>>
Re: [DISCUSS] official docker image(s) for hadoop
Posted by Mingliang Liu <li...@gmail.com>.
> It would be very helpful for testing the RC.
For testing and voting, I have been using docker containers for a while, see code at: https://github.com/weiqingy/caochong <https://github.com/weiqingy/caochong>
> TL;DR: I propose to create official hadoop images and upload them to the dockerhub
I’m +1 on this idea. The “official” docker image basically means a commitment to maintain well documented and broadly tested images, which seems not a burden to us.
Ceph has a community docker project https://github.com/ceph/ceph-docker <https://github.com/ceph/ceph-docker> and I think our scope here is similar to it.
Mingliang
> On Sep 13, 2017, at 11:39 AM, Yufei Gu <fl...@gmail.com> wrote:
>
> It would be very helpful for testing the RC. To vote a RC, committers and
> PMCs usually spend lots of time to compile, deploy the RC, do several
> sanity tests, then +1 for the RC. The docker image potentially saves the
> compilation and deployment time, and people can do more tests.
>
> Best,
>
> Yufei
>
> On Wed, Sep 13, 2017 at 11:19 AM, Wangda Tan <wh...@gmail.com> wrote:
>
>> +1 to add Hadoop docker image for easier testing / prototyping, it gonna be
>> super helpful!
>>
>> Thanks,
>> Wangda
>>
>> On Wed, Sep 13, 2017 at 10:48 AM, Miklos Szegedi <
>> miklos.szegedi@cloudera.com> wrote:
>>
>>> Marton, thank you for working on this. I think Official Docker images for
>>> Hadoop would be very useful for a lot of reasons. I think that it is
>> better
>>> to have a coordinated effort with production ready base images with
>>> dependent images for prototyping. Does anyone else have an opinion about
>>> this?
>>>
>>> Thank you,
>>> Miklos
>>>
>>> On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <hd...@anzix.net> wrote:
>>>
>>>>
>>>> TL;DR: I propose to create official hadoop images and upload them to
>> the
>>>> dockerhub.
>>>>
>>>> GOAL/SCOPE: I would like improve the existing documentation with
>>>> easy-to-use docker based recipes to start hadoop clusters with various
>>>> configuration.
>>>>
>>>> The images also could be used to test experimental features. For
>> example
>>>> ozone could be tested easily with these compose file and configuration:
>>>>
>>>> https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
>>>>
>>>> Or even the configuration could be included in the compose file:
>>>>
>>>> https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc
>>>> ker-compose.yaml
>>>>
>>>> I would like to create separated example compose files for federation,
>>> ha,
>>>> metrics usage, etc. to make it easier to try out and understand the
>>>> features.
>>>>
>>>> CONTEXT: There is an existing Jira https://issues.apache.org/jira
>>>> /browse/HADOOP-13397
>>>> But it’s about a tool to generate production quality docker images
>>>> (multiple types, in a flexible way). If no objections, I will create a
>>>> separated issue to create simplified docker images for rapid
>> prototyping
>>>> and investigating new features. And register the branch to the
>> dockerhub
>>> to
>>>> create the images automatically.
>>>>
>>>> MY BACKGROUND: I am working with docker based hadoop/spark clusters
>> quite
>>>> a while and run them succesfully in different environments (kubernetes,
>>>> docker-swarm, nomad-based scheduling, etc.) My work is available from
>>> here:
>>>> https://github.com/flokkr but they could handle more complex use cases
>>>> (eg. instrumenting java processes with btrace, or read/reload
>>> configuration
>>>> from consul).
>>>> And IMHO in the official hadoop documentation it’s better to suggest
>> to
>>>> use official apache docker images and not external ones (which could be
>>>> changed).
>>>>
>>>> Please let me know if you have any comments.
>>>>
>>>> Marton
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>>
>>>>
>>>
>>
Re: [DISCUSS] official docker image(s) for hadoop
Posted by Yufei Gu <fl...@gmail.com>.
It would be very helpful for testing the RC. To vote a RC, committers and
PMCs usually spend lots of time to compile, deploy the RC, do several
sanity tests, then +1 for the RC. The docker image potentially saves the
compilation and deployment time, and people can do more tests.
Best,
Yufei
On Wed, Sep 13, 2017 at 11:19 AM, Wangda Tan <wh...@gmail.com> wrote:
> +1 to add Hadoop docker image for easier testing / prototyping, it gonna be
> super helpful!
>
> Thanks,
> Wangda
>
> On Wed, Sep 13, 2017 at 10:48 AM, Miklos Szegedi <
> miklos.szegedi@cloudera.com> wrote:
>
> > Marton, thank you for working on this. I think Official Docker images for
> > Hadoop would be very useful for a lot of reasons. I think that it is
> better
> > to have a coordinated effort with production ready base images with
> > dependent images for prototyping. Does anyone else have an opinion about
> > this?
> >
> > Thank you,
> > Miklos
> >
> > On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <hd...@anzix.net> wrote:
> >
> > >
> > > TL;DR: I propose to create official hadoop images and upload them to
> the
> > > dockerhub.
> > >
> > > GOAL/SCOPE: I would like improve the existing documentation with
> > > easy-to-use docker based recipes to start hadoop clusters with various
> > > configuration.
> > >
> > > The images also could be used to test experimental features. For
> example
> > > ozone could be tested easily with these compose file and configuration:
> > >
> > > https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
> > >
> > > Or even the configuration could be included in the compose file:
> > >
> > > https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc
> > > ker-compose.yaml
> > >
> > > I would like to create separated example compose files for federation,
> > ha,
> > > metrics usage, etc. to make it easier to try out and understand the
> > > features.
> > >
> > > CONTEXT: There is an existing Jira https://issues.apache.org/jira
> > > /browse/HADOOP-13397
> > > But it’s about a tool to generate production quality docker images
> > > (multiple types, in a flexible way). If no objections, I will create a
> > > separated issue to create simplified docker images for rapid
> prototyping
> > > and investigating new features. And register the branch to the
> dockerhub
> > to
> > > create the images automatically.
> > >
> > > MY BACKGROUND: I am working with docker based hadoop/spark clusters
> quite
> > > a while and run them succesfully in different environments (kubernetes,
> > > docker-swarm, nomad-based scheduling, etc.) My work is available from
> > here:
> > > https://github.com/flokkr but they could handle more complex use cases
> > > (eg. instrumenting java processes with btrace, or read/reload
> > configuration
> > > from consul).
> > > And IMHO in the official hadoop documentation it’s better to suggest
> to
> > > use official apache docker images and not external ones (which could be
> > > changed).
> > >
> > > Please let me know if you have any comments.
> > >
> > > Marton
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> > >
> > >
> >
>
Re: [DISCUSS] official docker image(s) for hadoop
Posted by Eric Badger <eb...@oath.com.INVALID>.
+1 definitely think an official Hadoop docker image (possibly 1 per major
or minor release) would be a positive both for contributors and for users
of Hadoop.
Eric
On Wed, Sep 13, 2017 at 1:19 PM, Wangda Tan <wh...@gmail.com> wrote:
> +1 to add Hadoop docker image for easier testing / prototyping, it gonna be
> super helpful!
>
> Thanks,
> Wangda
>
> On Wed, Sep 13, 2017 at 10:48 AM, Miklos Szegedi <
> miklos.szegedi@cloudera.com> wrote:
>
> > Marton, thank you for working on this. I think Official Docker images for
> > Hadoop would be very useful for a lot of reasons. I think that it is
> better
> > to have a coordinated effort with production ready base images with
> > dependent images for prototyping. Does anyone else have an opinion about
> > this?
> >
> > Thank you,
> > Miklos
> >
> > On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <hd...@anzix.net> wrote:
> >
> > >
> > > TL;DR: I propose to create official hadoop images and upload them to
> the
> > > dockerhub.
> > >
> > > GOAL/SCOPE: I would like improve the existing documentation with
> > > easy-to-use docker based recipes to start hadoop clusters with various
> > > configuration.
> > >
> > > The images also could be used to test experimental features. For
> example
> > > ozone could be tested easily with these compose file and configuration:
> > >
> > > https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
> > >
> > > Or even the configuration could be included in the compose file:
> > >
> > > https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc
> > > ker-compose.yaml
> > >
> > > I would like to create separated example compose files for federation,
> > ha,
> > > metrics usage, etc. to make it easier to try out and understand the
> > > features.
> > >
> > > CONTEXT: There is an existing Jira https://issues.apache.org/jira
> > > /browse/HADOOP-13397
> > > But it’s about a tool to generate production quality docker images
> > > (multiple types, in a flexible way). If no objections, I will create a
> > > separated issue to create simplified docker images for rapid
> prototyping
> > > and investigating new features. And register the branch to the
> dockerhub
> > to
> > > create the images automatically.
> > >
> > > MY BACKGROUND: I am working with docker based hadoop/spark clusters
> quite
> > > a while and run them succesfully in different environments (kubernetes,
> > > docker-swarm, nomad-based scheduling, etc.) My work is available from
> > here:
> > > https://github.com/flokkr but they could handle more complex use cases
> > > (eg. instrumenting java processes with btrace, or read/reload
> > configuration
> > > from consul).
> > > And IMHO in the official hadoop documentation it’s better to suggest
> to
> > > use official apache docker images and not external ones (which could be
> > > changed).
> > >
> > > Please let me know if you have any comments.
> > >
> > > Marton
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> > >
> > >
> >
>
Re: [DISCUSS] official docker image(s) for hadoop
Posted by Eric Badger <eb...@oath.com.INVALID>.
+1 definitely think an official Hadoop docker image (possibly 1 per major
or minor release) would be a positive both for contributors and for users
of Hadoop.
Eric
On Wed, Sep 13, 2017 at 1:19 PM, Wangda Tan <wh...@gmail.com> wrote:
> +1 to add Hadoop docker image for easier testing / prototyping, it gonna be
> super helpful!
>
> Thanks,
> Wangda
>
> On Wed, Sep 13, 2017 at 10:48 AM, Miklos Szegedi <
> miklos.szegedi@cloudera.com> wrote:
>
> > Marton, thank you for working on this. I think Official Docker images for
> > Hadoop would be very useful for a lot of reasons. I think that it is
> better
> > to have a coordinated effort with production ready base images with
> > dependent images for prototyping. Does anyone else have an opinion about
> > this?
> >
> > Thank you,
> > Miklos
> >
> > On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <hd...@anzix.net> wrote:
> >
> > >
> > > TL;DR: I propose to create official hadoop images and upload them to
> the
> > > dockerhub.
> > >
> > > GOAL/SCOPE: I would like improve the existing documentation with
> > > easy-to-use docker based recipes to start hadoop clusters with various
> > > configuration.
> > >
> > > The images also could be used to test experimental features. For
> example
> > > ozone could be tested easily with these compose file and configuration:
> > >
> > > https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
> > >
> > > Or even the configuration could be included in the compose file:
> > >
> > > https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc
> > > ker-compose.yaml
> > >
> > > I would like to create separated example compose files for federation,
> > ha,
> > > metrics usage, etc. to make it easier to try out and understand the
> > > features.
> > >
> > > CONTEXT: There is an existing Jira https://issues.apache.org/jira
> > > /browse/HADOOP-13397
> > > But it’s about a tool to generate production quality docker images
> > > (multiple types, in a flexible way). If no objections, I will create a
> > > separated issue to create simplified docker images for rapid
> prototyping
> > > and investigating new features. And register the branch to the
> dockerhub
> > to
> > > create the images automatically.
> > >
> > > MY BACKGROUND: I am working with docker based hadoop/spark clusters
> quite
> > > a while and run them succesfully in different environments (kubernetes,
> > > docker-swarm, nomad-based scheduling, etc.) My work is available from
> > here:
> > > https://github.com/flokkr but they could handle more complex use cases
> > > (eg. instrumenting java processes with btrace, or read/reload
> > configuration
> > > from consul).
> > > And IMHO in the official hadoop documentation it’s better to suggest
> to
> > > use official apache docker images and not external ones (which could be
> > > changed).
> > >
> > > Please let me know if you have any comments.
> > >
> > > Marton
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> > >
> > >
> >
>
Re: [DISCUSS] official docker image(s) for hadoop
Posted by Yufei Gu <fl...@gmail.com>.
It would be very helpful for testing the RC. To vote a RC, committers and
PMCs usually spend lots of time to compile, deploy the RC, do several
sanity tests, then +1 for the RC. The docker image potentially saves the
compilation and deployment time, and people can do more tests.
Best,
Yufei
On Wed, Sep 13, 2017 at 11:19 AM, Wangda Tan <wh...@gmail.com> wrote:
> +1 to add Hadoop docker image for easier testing / prototyping, it gonna be
> super helpful!
>
> Thanks,
> Wangda
>
> On Wed, Sep 13, 2017 at 10:48 AM, Miklos Szegedi <
> miklos.szegedi@cloudera.com> wrote:
>
> > Marton, thank you for working on this. I think Official Docker images for
> > Hadoop would be very useful for a lot of reasons. I think that it is
> better
> > to have a coordinated effort with production ready base images with
> > dependent images for prototyping. Does anyone else have an opinion about
> > this?
> >
> > Thank you,
> > Miklos
> >
> > On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <hd...@anzix.net> wrote:
> >
> > >
> > > TL;DR: I propose to create official hadoop images and upload them to
> the
> > > dockerhub.
> > >
> > > GOAL/SCOPE: I would like improve the existing documentation with
> > > easy-to-use docker based recipes to start hadoop clusters with various
> > > configuration.
> > >
> > > The images also could be used to test experimental features. For
> example
> > > ozone could be tested easily with these compose file and configuration:
> > >
> > > https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
> > >
> > > Or even the configuration could be included in the compose file:
> > >
> > > https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc
> > > ker-compose.yaml
> > >
> > > I would like to create separated example compose files for federation,
> > ha,
> > > metrics usage, etc. to make it easier to try out and understand the
> > > features.
> > >
> > > CONTEXT: There is an existing Jira https://issues.apache.org/jira
> > > /browse/HADOOP-13397
> > > But it’s about a tool to generate production quality docker images
> > > (multiple types, in a flexible way). If no objections, I will create a
> > > separated issue to create simplified docker images for rapid
> prototyping
> > > and investigating new features. And register the branch to the
> dockerhub
> > to
> > > create the images automatically.
> > >
> > > MY BACKGROUND: I am working with docker based hadoop/spark clusters
> quite
> > > a while and run them succesfully in different environments (kubernetes,
> > > docker-swarm, nomad-based scheduling, etc.) My work is available from
> > here:
> > > https://github.com/flokkr but they could handle more complex use cases
> > > (eg. instrumenting java processes with btrace, or read/reload
> > configuration
> > > from consul).
> > > And IMHO in the official hadoop documentation it’s better to suggest
> to
> > > use official apache docker images and not external ones (which could be
> > > changed).
> > >
> > > Please let me know if you have any comments.
> > >
> > > Marton
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> > >
> > >
> >
>
Re: [DISCUSS] official docker image(s) for hadoop
Posted by Yufei Gu <fl...@gmail.com>.
It would be very helpful for testing the RC. To vote a RC, committers and
PMCs usually spend lots of time to compile, deploy the RC, do several
sanity tests, then +1 for the RC. The docker image potentially saves the
compilation and deployment time, and people can do more tests.
Best,
Yufei
On Wed, Sep 13, 2017 at 11:19 AM, Wangda Tan <wh...@gmail.com> wrote:
> +1 to add Hadoop docker image for easier testing / prototyping, it gonna be
> super helpful!
>
> Thanks,
> Wangda
>
> On Wed, Sep 13, 2017 at 10:48 AM, Miklos Szegedi <
> miklos.szegedi@cloudera.com> wrote:
>
> > Marton, thank you for working on this. I think Official Docker images for
> > Hadoop would be very useful for a lot of reasons. I think that it is
> better
> > to have a coordinated effort with production ready base images with
> > dependent images for prototyping. Does anyone else have an opinion about
> > this?
> >
> > Thank you,
> > Miklos
> >
> > On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <hd...@anzix.net> wrote:
> >
> > >
> > > TL;DR: I propose to create official hadoop images and upload them to
> the
> > > dockerhub.
> > >
> > > GOAL/SCOPE: I would like improve the existing documentation with
> > > easy-to-use docker based recipes to start hadoop clusters with various
> > > configuration.
> > >
> > > The images also could be used to test experimental features. For
> example
> > > ozone could be tested easily with these compose file and configuration:
> > >
> > > https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
> > >
> > > Or even the configuration could be included in the compose file:
> > >
> > > https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc
> > > ker-compose.yaml
> > >
> > > I would like to create separated example compose files for federation,
> > ha,
> > > metrics usage, etc. to make it easier to try out and understand the
> > > features.
> > >
> > > CONTEXT: There is an existing Jira https://issues.apache.org/jira
> > > /browse/HADOOP-13397
> > > But it’s about a tool to generate production quality docker images
> > > (multiple types, in a flexible way). If no objections, I will create a
> > > separated issue to create simplified docker images for rapid
> prototyping
> > > and investigating new features. And register the branch to the
> dockerhub
> > to
> > > create the images automatically.
> > >
> > > MY BACKGROUND: I am working with docker based hadoop/spark clusters
> quite
> > > a while and run them succesfully in different environments (kubernetes,
> > > docker-swarm, nomad-based scheduling, etc.) My work is available from
> > here:
> > > https://github.com/flokkr but they could handle more complex use cases
> > > (eg. instrumenting java processes with btrace, or read/reload
> > configuration
> > > from consul).
> > > And IMHO in the official hadoop documentation it’s better to suggest
> to
> > > use official apache docker images and not external ones (which could be
> > > changed).
> > >
> > > Please let me know if you have any comments.
> > >
> > > Marton
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> > >
> > >
> >
>
Re: [DISCUSS] official docker image(s) for hadoop
Posted by Yufei Gu <fl...@gmail.com>.
It would be very helpful for testing the RC. To vote a RC, committers and
PMCs usually spend lots of time to compile, deploy the RC, do several
sanity tests, then +1 for the RC. The docker image potentially saves the
compilation and deployment time, and people can do more tests.
Best,
Yufei
On Wed, Sep 13, 2017 at 11:19 AM, Wangda Tan <wh...@gmail.com> wrote:
> +1 to add Hadoop docker image for easier testing / prototyping, it gonna be
> super helpful!
>
> Thanks,
> Wangda
>
> On Wed, Sep 13, 2017 at 10:48 AM, Miklos Szegedi <
> miklos.szegedi@cloudera.com> wrote:
>
> > Marton, thank you for working on this. I think Official Docker images for
> > Hadoop would be very useful for a lot of reasons. I think that it is
> better
> > to have a coordinated effort with production ready base images with
> > dependent images for prototyping. Does anyone else have an opinion about
> > this?
> >
> > Thank you,
> > Miklos
> >
> > On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <hd...@anzix.net> wrote:
> >
> > >
> > > TL;DR: I propose to create official hadoop images and upload them to
> the
> > > dockerhub.
> > >
> > > GOAL/SCOPE: I would like improve the existing documentation with
> > > easy-to-use docker based recipes to start hadoop clusters with various
> > > configuration.
> > >
> > > The images also could be used to test experimental features. For
> example
> > > ozone could be tested easily with these compose file and configuration:
> > >
> > > https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
> > >
> > > Or even the configuration could be included in the compose file:
> > >
> > > https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc
> > > ker-compose.yaml
> > >
> > > I would like to create separated example compose files for federation,
> > ha,
> > > metrics usage, etc. to make it easier to try out and understand the
> > > features.
> > >
> > > CONTEXT: There is an existing Jira https://issues.apache.org/jira
> > > /browse/HADOOP-13397
> > > But it’s about a tool to generate production quality docker images
> > > (multiple types, in a flexible way). If no objections, I will create a
> > > separated issue to create simplified docker images for rapid
> prototyping
> > > and investigating new features. And register the branch to the
> dockerhub
> > to
> > > create the images automatically.
> > >
> > > MY BACKGROUND: I am working with docker based hadoop/spark clusters
> quite
> > > a while and run them succesfully in different environments (kubernetes,
> > > docker-swarm, nomad-based scheduling, etc.) My work is available from
> > here:
> > > https://github.com/flokkr but they could handle more complex use cases
> > > (eg. instrumenting java processes with btrace, or read/reload
> > configuration
> > > from consul).
> > > And IMHO in the official hadoop documentation it’s better to suggest
> to
> > > use official apache docker images and not external ones (which could be
> > > changed).
> > >
> > > Please let me know if you have any comments.
> > >
> > > Marton
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> > >
> > >
> >
>
Re: [DISCUSS] official docker image(s) for hadoop
Posted by Eric Badger <eb...@oath.com.INVALID>.
+1 definitely think an official Hadoop docker image (possibly 1 per major
or minor release) would be a positive both for contributors and for users
of Hadoop.
Eric
On Wed, Sep 13, 2017 at 1:19 PM, Wangda Tan <wh...@gmail.com> wrote:
> +1 to add Hadoop docker image for easier testing / prototyping, it gonna be
> super helpful!
>
> Thanks,
> Wangda
>
> On Wed, Sep 13, 2017 at 10:48 AM, Miklos Szegedi <
> miklos.szegedi@cloudera.com> wrote:
>
> > Marton, thank you for working on this. I think Official Docker images for
> > Hadoop would be very useful for a lot of reasons. I think that it is
> better
> > to have a coordinated effort with production ready base images with
> > dependent images for prototyping. Does anyone else have an opinion about
> > this?
> >
> > Thank you,
> > Miklos
> >
> > On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <hd...@anzix.net> wrote:
> >
> > >
> > > TL;DR: I propose to create official hadoop images and upload them to
> the
> > > dockerhub.
> > >
> > > GOAL/SCOPE: I would like improve the existing documentation with
> > > easy-to-use docker based recipes to start hadoop clusters with various
> > > configuration.
> > >
> > > The images also could be used to test experimental features. For
> example
> > > ozone could be tested easily with these compose file and configuration:
> > >
> > > https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
> > >
> > > Or even the configuration could be included in the compose file:
> > >
> > > https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc
> > > ker-compose.yaml
> > >
> > > I would like to create separated example compose files for federation,
> > ha,
> > > metrics usage, etc. to make it easier to try out and understand the
> > > features.
> > >
> > > CONTEXT: There is an existing Jira https://issues.apache.org/jira
> > > /browse/HADOOP-13397
> > > But it’s about a tool to generate production quality docker images
> > > (multiple types, in a flexible way). If no objections, I will create a
> > > separated issue to create simplified docker images for rapid
> prototyping
> > > and investigating new features. And register the branch to the
> dockerhub
> > to
> > > create the images automatically.
> > >
> > > MY BACKGROUND: I am working with docker based hadoop/spark clusters
> quite
> > > a while and run them succesfully in different environments (kubernetes,
> > > docker-swarm, nomad-based scheduling, etc.) My work is available from
> > here:
> > > https://github.com/flokkr but they could handle more complex use cases
> > > (eg. instrumenting java processes with btrace, or read/reload
> > configuration
> > > from consul).
> > > And IMHO in the official hadoop documentation it’s better to suggest
> to
> > > use official apache docker images and not external ones (which could be
> > > changed).
> > >
> > > Please let me know if you have any comments.
> > >
> > > Marton
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> > >
> > >
> >
>
Re: [DISCUSS] official docker image(s) for hadoop
Posted by Wangda Tan <wh...@gmail.com>.
+1 to add Hadoop docker image for easier testing / prototyping, it gonna be
super helpful!
Thanks,
Wangda
On Wed, Sep 13, 2017 at 10:48 AM, Miklos Szegedi <
miklos.szegedi@cloudera.com> wrote:
> Marton, thank you for working on this. I think Official Docker images for
> Hadoop would be very useful for a lot of reasons. I think that it is better
> to have a coordinated effort with production ready base images with
> dependent images for prototyping. Does anyone else have an opinion about
> this?
>
> Thank you,
> Miklos
>
> On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <hd...@anzix.net> wrote:
>
> >
> > TL;DR: I propose to create official hadoop images and upload them to the
> > dockerhub.
> >
> > GOAL/SCOPE: I would like improve the existing documentation with
> > easy-to-use docker based recipes to start hadoop clusters with various
> > configuration.
> >
> > The images also could be used to test experimental features. For example
> > ozone could be tested easily with these compose file and configuration:
> >
> > https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
> >
> > Or even the configuration could be included in the compose file:
> >
> > https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc
> > ker-compose.yaml
> >
> > I would like to create separated example compose files for federation,
> ha,
> > metrics usage, etc. to make it easier to try out and understand the
> > features.
> >
> > CONTEXT: There is an existing Jira https://issues.apache.org/jira
> > /browse/HADOOP-13397
> > But it’s about a tool to generate production quality docker images
> > (multiple types, in a flexible way). If no objections, I will create a
> > separated issue to create simplified docker images for rapid prototyping
> > and investigating new features. And register the branch to the dockerhub
> to
> > create the images automatically.
> >
> > MY BACKGROUND: I am working with docker based hadoop/spark clusters quite
> > a while and run them succesfully in different environments (kubernetes,
> > docker-swarm, nomad-based scheduling, etc.) My work is available from
> here:
> > https://github.com/flokkr but they could handle more complex use cases
> > (eg. instrumenting java processes with btrace, or read/reload
> configuration
> > from consul).
> > And IMHO in the official hadoop documentation it’s better to suggest to
> > use official apache docker images and not external ones (which could be
> > changed).
> >
> > Please let me know if you have any comments.
> >
> > Marton
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
> >
>
Re: [DISCUSS] official docker image(s) for hadoop
Posted by Wangda Tan <wh...@gmail.com>.
+1 to add Hadoop docker image for easier testing / prototyping, it gonna be
super helpful!
Thanks,
Wangda
On Wed, Sep 13, 2017 at 10:48 AM, Miklos Szegedi <
miklos.szegedi@cloudera.com> wrote:
> Marton, thank you for working on this. I think Official Docker images for
> Hadoop would be very useful for a lot of reasons. I think that it is better
> to have a coordinated effort with production ready base images with
> dependent images for prototyping. Does anyone else have an opinion about
> this?
>
> Thank you,
> Miklos
>
> On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <hd...@anzix.net> wrote:
>
> >
> > TL;DR: I propose to create official hadoop images and upload them to the
> > dockerhub.
> >
> > GOAL/SCOPE: I would like improve the existing documentation with
> > easy-to-use docker based recipes to start hadoop clusters with various
> > configuration.
> >
> > The images also could be used to test experimental features. For example
> > ozone could be tested easily with these compose file and configuration:
> >
> > https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
> >
> > Or even the configuration could be included in the compose file:
> >
> > https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc
> > ker-compose.yaml
> >
> > I would like to create separated example compose files for federation,
> ha,
> > metrics usage, etc. to make it easier to try out and understand the
> > features.
> >
> > CONTEXT: There is an existing Jira https://issues.apache.org/jira
> > /browse/HADOOP-13397
> > But it’s about a tool to generate production quality docker images
> > (multiple types, in a flexible way). If no objections, I will create a
> > separated issue to create simplified docker images for rapid prototyping
> > and investigating new features. And register the branch to the dockerhub
> to
> > create the images automatically.
> >
> > MY BACKGROUND: I am working with docker based hadoop/spark clusters quite
> > a while and run them succesfully in different environments (kubernetes,
> > docker-swarm, nomad-based scheduling, etc.) My work is available from
> here:
> > https://github.com/flokkr but they could handle more complex use cases
> > (eg. instrumenting java processes with btrace, or read/reload
> configuration
> > from consul).
> > And IMHO in the official hadoop documentation it’s better to suggest to
> > use official apache docker images and not external ones (which could be
> > changed).
> >
> > Please let me know if you have any comments.
> >
> > Marton
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
> >
>
Re: [DISCUSS] official docker image(s) for hadoop
Posted by Wangda Tan <wh...@gmail.com>.
+1 to add Hadoop docker image for easier testing / prototyping, it gonna be
super helpful!
Thanks,
Wangda
On Wed, Sep 13, 2017 at 10:48 AM, Miklos Szegedi <
miklos.szegedi@cloudera.com> wrote:
> Marton, thank you for working on this. I think Official Docker images for
> Hadoop would be very useful for a lot of reasons. I think that it is better
> to have a coordinated effort with production ready base images with
> dependent images for prototyping. Does anyone else have an opinion about
> this?
>
> Thank you,
> Miklos
>
> On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <hd...@anzix.net> wrote:
>
> >
> > TL;DR: I propose to create official hadoop images and upload them to the
> > dockerhub.
> >
> > GOAL/SCOPE: I would like improve the existing documentation with
> > easy-to-use docker based recipes to start hadoop clusters with various
> > configuration.
> >
> > The images also could be used to test experimental features. For example
> > ozone could be tested easily with these compose file and configuration:
> >
> > https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
> >
> > Or even the configuration could be included in the compose file:
> >
> > https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc
> > ker-compose.yaml
> >
> > I would like to create separated example compose files for federation,
> ha,
> > metrics usage, etc. to make it easier to try out and understand the
> > features.
> >
> > CONTEXT: There is an existing Jira https://issues.apache.org/jira
> > /browse/HADOOP-13397
> > But it’s about a tool to generate production quality docker images
> > (multiple types, in a flexible way). If no objections, I will create a
> > separated issue to create simplified docker images for rapid prototyping
> > and investigating new features. And register the branch to the dockerhub
> to
> > create the images automatically.
> >
> > MY BACKGROUND: I am working with docker based hadoop/spark clusters quite
> > a while and run them succesfully in different environments (kubernetes,
> > docker-swarm, nomad-based scheduling, etc.) My work is available from
> here:
> > https://github.com/flokkr but they could handle more complex use cases
> > (eg. instrumenting java processes with btrace, or read/reload
> configuration
> > from consul).
> > And IMHO in the official hadoop documentation it’s better to suggest to
> > use official apache docker images and not external ones (which could be
> > changed).
> >
> > Please let me know if you have any comments.
> >
> > Marton
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
> >
>
Re: [DISCUSS] official docker image(s) for hadoop
Posted by Wangda Tan <wh...@gmail.com>.
+1 to add Hadoop docker image for easier testing / prototyping, it gonna be
super helpful!
Thanks,
Wangda
On Wed, Sep 13, 2017 at 10:48 AM, Miklos Szegedi <
miklos.szegedi@cloudera.com> wrote:
> Marton, thank you for working on this. I think Official Docker images for
> Hadoop would be very useful for a lot of reasons. I think that it is better
> to have a coordinated effort with production ready base images with
> dependent images for prototyping. Does anyone else have an opinion about
> this?
>
> Thank you,
> Miklos
>
> On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <hd...@anzix.net> wrote:
>
> >
> > TL;DR: I propose to create official hadoop images and upload them to the
> > dockerhub.
> >
> > GOAL/SCOPE: I would like improve the existing documentation with
> > easy-to-use docker based recipes to start hadoop clusters with various
> > configuration.
> >
> > The images also could be used to test experimental features. For example
> > ozone could be tested easily with these compose file and configuration:
> >
> > https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
> >
> > Or even the configuration could be included in the compose file:
> >
> > https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc
> > ker-compose.yaml
> >
> > I would like to create separated example compose files for federation,
> ha,
> > metrics usage, etc. to make it easier to try out and understand the
> > features.
> >
> > CONTEXT: There is an existing Jira https://issues.apache.org/jira
> > /browse/HADOOP-13397
> > But it’s about a tool to generate production quality docker images
> > (multiple types, in a flexible way). If no objections, I will create a
> > separated issue to create simplified docker images for rapid prototyping
> > and investigating new features. And register the branch to the dockerhub
> to
> > create the images automatically.
> >
> > MY BACKGROUND: I am working with docker based hadoop/spark clusters quite
> > a while and run them succesfully in different environments (kubernetes,
> > docker-swarm, nomad-based scheduling, etc.) My work is available from
> here:
> > https://github.com/flokkr but they could handle more complex use cases
> > (eg. instrumenting java processes with btrace, or read/reload
> configuration
> > from consul).
> > And IMHO in the official hadoop documentation it’s better to suggest to
> > use official apache docker images and not external ones (which could be
> > changed).
> >
> > Please let me know if you have any comments.
> >
> > Marton
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
> >
>
Re: [DISCUSS] official docker image(s) for hadoop
Posted by Miklos Szegedi <mi...@cloudera.com>.
Marton, thank you for working on this. I think Official Docker images for
Hadoop would be very useful for a lot of reasons. I think that it is better
to have a coordinated effort with production ready base images with
dependent images for prototyping. Does anyone else have an opinion about
this?
Thank you,
Miklos
On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <hd...@anzix.net> wrote:
>
> TL;DR: I propose to create official hadoop images and upload them to the
> dockerhub.
>
> GOAL/SCOPE: I would like improve the existing documentation with
> easy-to-use docker based recipes to start hadoop clusters with various
> configuration.
>
> The images also could be used to test experimental features. For example
> ozone could be tested easily with these compose file and configuration:
>
> https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
>
> Or even the configuration could be included in the compose file:
>
> https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc
> ker-compose.yaml
>
> I would like to create separated example compose files for federation, ha,
> metrics usage, etc. to make it easier to try out and understand the
> features.
>
> CONTEXT: There is an existing Jira https://issues.apache.org/jira
> /browse/HADOOP-13397
> But it’s about a tool to generate production quality docker images
> (multiple types, in a flexible way). If no objections, I will create a
> separated issue to create simplified docker images for rapid prototyping
> and investigating new features. And register the branch to the dockerhub to
> create the images automatically.
>
> MY BACKGROUND: I am working with docker based hadoop/spark clusters quite
> a while and run them succesfully in different environments (kubernetes,
> docker-swarm, nomad-based scheduling, etc.) My work is available from here:
> https://github.com/flokkr but they could handle more complex use cases
> (eg. instrumenting java processes with btrace, or read/reload configuration
> from consul).
> And IMHO in the official hadoop documentation it’s better to suggest to
> use official apache docker images and not external ones (which could be
> changed).
>
> Please let me know if you have any comments.
>
> Marton
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
Re: [DISCUSS] official docker image(s) for hadoop
Posted by Miklos Szegedi <mi...@cloudera.com>.
Marton, thank you for working on this. I think Official Docker images for
Hadoop would be very useful for a lot of reasons. I think that it is better
to have a coordinated effort with production ready base images with
dependent images for prototyping. Does anyone else have an opinion about
this?
Thank you,
Miklos
On Fri, Sep 8, 2017 at 5:45 AM, Marton, Elek <hd...@anzix.net> wrote:
>
> TL;DR: I propose to create official hadoop images and upload them to the
> dockerhub.
>
> GOAL/SCOPE: I would like improve the existing documentation with
> easy-to-use docker based recipes to start hadoop clusters with various
> configuration.
>
> The images also could be used to test experimental features. For example
> ozone could be tested easily with these compose file and configuration:
>
> https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
>
> Or even the configuration could be included in the compose file:
>
> https://github.com/elek/hadoop/blob/docker-2.8.0/example/doc
> ker-compose.yaml
>
> I would like to create separated example compose files for federation, ha,
> metrics usage, etc. to make it easier to try out and understand the
> features.
>
> CONTEXT: There is an existing Jira https://issues.apache.org/jira
> /browse/HADOOP-13397
> But it’s about a tool to generate production quality docker images
> (multiple types, in a flexible way). If no objections, I will create a
> separated issue to create simplified docker images for rapid prototyping
> and investigating new features. And register the branch to the dockerhub to
> create the images automatically.
>
> MY BACKGROUND: I am working with docker based hadoop/spark clusters quite
> a while and run them succesfully in different environments (kubernetes,
> docker-swarm, nomad-based scheduling, etc.) My work is available from here:
> https://github.com/flokkr but they could handle more complex use cases
> (eg. instrumenting java processes with btrace, or read/reload configuration
> from consul).
> And IMHO in the official hadoop documentation it’s better to suggest to
> use official apache docker images and not external ones (which could be
> changed).
>
> Please let me know if you have any comments.
>
> Marton
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
Re: [DISCUSS] official docker image(s) for hadoop
Posted by "Marton, Elek" <hd...@anzix.net>.
Thanks all the feedbacks.
I created an issue:
https://issues.apache.org/jira/browse/HADOOP-14898
Let's continue the discussion there.
Thanks,
Marton
On 09/08/2017 02:45 PM, Marton, Elek wrote:
>
> TL;DR: I propose to create official hadoop images and upload them to the
> dockerhub.
>
> GOAL/SCOPE: I would like improve the existing documentation with
> easy-to-use docker based recipes to start hadoop clusters with various
> configuration.
>
> The images also could be used to test experimental features. For example
> ozone could be tested easily with these compose file and configuration:
>
> https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
>
> Or even the configuration could be included in the compose file:
>
> https://github.com/elek/hadoop/blob/docker-2.8.0/example/docker-compose.yaml
>
>
> I would like to create separated example compose files for federation,
> ha, metrics usage, etc. to make it easier to try out and understand the
> features.
>
> CONTEXT: There is an existing Jira
> https://issues.apache.org/jira/browse/HADOOP-13397
> But it’s about a tool to generate production quality docker images
> (multiple types, in a flexible way). If no objections, I will create a
> separated issue to create simplified docker images for rapid prototyping
> and investigating new features. And register the branch to the dockerhub
> to create the images automatically.
>
> MY BACKGROUND: I am working with docker based hadoop/spark clusters
> quite a while and run them succesfully in different environments
> (kubernetes, docker-swarm, nomad-based scheduling, etc.) My work is
> available from here: https://github.com/flokkr but they could handle
> more complex use cases (eg. instrumenting java processes with btrace, or
> read/reload configuration from consul).
> And IMHO in the official hadoop documentation it’s better to suggest
> to use official apache docker images and not external ones (which could
> be changed).
>
> Please let me know if you have any comments.
>
> Marton
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [DISCUSS] official docker image(s) for hadoop
Posted by "Marton, Elek" <hd...@anzix.net>.
Thanks all the feedbacks.
I created an issue:
https://issues.apache.org/jira/browse/HADOOP-14898
Let's continue the discussion there.
Thanks,
Marton
On 09/08/2017 02:45 PM, Marton, Elek wrote:
>
> TL;DR: I propose to create official hadoop images and upload them to the
> dockerhub.
>
> GOAL/SCOPE: I would like improve the existing documentation with
> easy-to-use docker based recipes to start hadoop clusters with various
> configuration.
>
> The images also could be used to test experimental features. For example
> ozone could be tested easily with these compose file and configuration:
>
> https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
>
> Or even the configuration could be included in the compose file:
>
> https://github.com/elek/hadoop/blob/docker-2.8.0/example/docker-compose.yaml
>
>
> I would like to create separated example compose files for federation,
> ha, metrics usage, etc. to make it easier to try out and understand the
> features.
>
> CONTEXT: There is an existing Jira
> https://issues.apache.org/jira/browse/HADOOP-13397
> But it’s about a tool to generate production quality docker images
> (multiple types, in a flexible way). If no objections, I will create a
> separated issue to create simplified docker images for rapid prototyping
> and investigating new features. And register the branch to the dockerhub
> to create the images automatically.
>
> MY BACKGROUND: I am working with docker based hadoop/spark clusters
> quite a while and run them succesfully in different environments
> (kubernetes, docker-swarm, nomad-based scheduling, etc.) My work is
> available from here: https://github.com/flokkr but they could handle
> more complex use cases (eg. instrumenting java processes with btrace, or
> read/reload configuration from consul).
> And IMHO in the official hadoop documentation it’s better to suggest
> to use official apache docker images and not external ones (which could
> be changed).
>
> Please let me know if you have any comments.
>
> Marton
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [DISCUSS] official docker image(s) for hadoop
Posted by "Marton, Elek" <hd...@anzix.net>.
Thanks all the feedbacks.
I created an issue:
https://issues.apache.org/jira/browse/HADOOP-14898
Let's continue the discussion there.
Thanks,
Marton
On 09/08/2017 02:45 PM, Marton, Elek wrote:
>
> TL;DR: I propose to create official hadoop images and upload them to the
> dockerhub.
>
> GOAL/SCOPE: I would like improve the existing documentation with
> easy-to-use docker based recipes to start hadoop clusters with various
> configuration.
>
> The images also could be used to test experimental features. For example
> ozone could be tested easily with these compose file and configuration:
>
> https://gist.github.com/elek/1676a97b98f4ba561c9f51fce2ab2ea6
>
> Or even the configuration could be included in the compose file:
>
> https://github.com/elek/hadoop/blob/docker-2.8.0/example/docker-compose.yaml
>
>
> I would like to create separated example compose files for federation,
> ha, metrics usage, etc. to make it easier to try out and understand the
> features.
>
> CONTEXT: There is an existing Jira
> https://issues.apache.org/jira/browse/HADOOP-13397
> But it’s about a tool to generate production quality docker images
> (multiple types, in a flexible way). If no objections, I will create a
> separated issue to create simplified docker images for rapid prototyping
> and investigating new features. And register the branch to the dockerhub
> to create the images automatically.
>
> MY BACKGROUND: I am working with docker based hadoop/spark clusters
> quite a while and run them succesfully in different environments
> (kubernetes, docker-swarm, nomad-based scheduling, etc.) My work is
> available from here: https://github.com/flokkr but they could handle
> more complex use cases (eg. instrumenting java processes with btrace, or
> read/reload configuration from consul).
> And IMHO in the official hadoop documentation it’s better to suggest
> to use official apache docker images and not external ones (which could
> be changed).
>
> Please let me know if you have any comments.
>
> Marton
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org