You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-dev@hadoop.apache.org by Tsuyoshi Ozawa <oz...@apache.org> on 2016/07/19 07:46:13 UTC
[DISCUSS] Official Docker Image at release time
Hi developers,
Klaus mentioned the availability of an official docker image of Apache
Hadoop. Is it time that we start to distribute an official docker
image at release time?
http://mail-archives.apache.org/mod_mbox/hadoop-user/201607.mbox/%3CSG2PR04MB162977CFE150444FA022510FB6370%40SG2PR04MB1629.apcprd04.prod.outlook.com%3E
Thoughts?
Thanks,
- Tsuyoshi
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [DISCUSS] Official Docker Image at release time
Posted by Roman Shaposhnik <ro...@shaposhnik.org>.
On Tue, Jul 19, 2016 at 8:12 AM, Sean Busbey <bu...@cloudera.com> wrote:
> FWIW, there is a "blessed" apache area on docker hub now, and it's
> just an INFRA request to point out the needed Dockerfile in the repo.
>
> PMCs can also request write access to bintray hosting of docker images
> for PMC members.
>
> Info on INFRA-8441, example on INFRA-12019.
>
> A Docker image that starts up a pseudo distributed instance would be
> great for new folks. I don't know if it's worth the investment to
> build images that do more complex multi-instance deployments (though
> there is some fledgling work in HBase for tooling that would do this
> for us).
I am with Sean on this one. To make it truly useful outside of a Quickstart VM
use case the community image has to deal with configuration management
in a much more explicit way. Just sticking java bits into a Docker container
won't really solve much.
Thanks,
Roman.
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [DISCUSS] Official Docker Image at release time
Posted by Roman Shaposhnik <ro...@shaposhnik.org>.
On Tue, Jul 19, 2016 at 8:12 AM, Sean Busbey <bu...@cloudera.com> wrote:
> FWIW, there is a "blessed" apache area on docker hub now, and it's
> just an INFRA request to point out the needed Dockerfile in the repo.
>
> PMCs can also request write access to bintray hosting of docker images
> for PMC members.
>
> Info on INFRA-8441, example on INFRA-12019.
>
> A Docker image that starts up a pseudo distributed instance would be
> great for new folks. I don't know if it's worth the investment to
> build images that do more complex multi-instance deployments (though
> there is some fledgling work in HBase for tooling that would do this
> for us).
I am with Sean on this one. To make it truly useful outside of a Quickstart VM
use case the community image has to deal with configuration management
in a much more explicit way. Just sticking java bits into a Docker container
won't really solve much.
Thanks,
Roman.
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [DISCUSS] Official Docker Image at release time
Posted by Roman Shaposhnik <ro...@shaposhnik.org>.
On Tue, Jul 19, 2016 at 8:12 AM, Sean Busbey <bu...@cloudera.com> wrote:
> FWIW, there is a "blessed" apache area on docker hub now, and it's
> just an INFRA request to point out the needed Dockerfile in the repo.
>
> PMCs can also request write access to bintray hosting of docker images
> for PMC members.
>
> Info on INFRA-8441, example on INFRA-12019.
>
> A Docker image that starts up a pseudo distributed instance would be
> great for new folks. I don't know if it's worth the investment to
> build images that do more complex multi-instance deployments (though
> there is some fledgling work in HBase for tooling that would do this
> for us).
I am with Sean on this one. To make it truly useful outside of a Quickstart VM
use case the community image has to deal with configuration management
in a much more explicit way. Just sticking java bits into a Docker container
won't really solve much.
Thanks,
Roman.
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [DISCUSS] Official Docker Image at release time
Posted by Sean Busbey <bu...@cloudera.com>.
FWIW, there is a "blessed" apache area on docker hub now, and it's
just an INFRA request to point out the needed Dockerfile in the repo.
PMCs can also request write access to bintray hosting of docker images
for PMC members.
Info on INFRA-8441, example on INFRA-12019.
A Docker image that starts up a pseudo distributed instance would be
great for new folks. I don't know if it's worth the investment to
build images that do more complex multi-instance deployments (though
there is some fledgling work in HBase for tooling that would do this
for us).
On Tue, Jul 19, 2016 at 2:46 AM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
> Hi developers,
>
> Klaus mentioned the availability of an official docker image of Apache
> Hadoop. Is it time that we start to distribute an official docker
> image at release time?
>
> http://mail-archives.apache.org/mod_mbox/hadoop-user/201607.mbox/%3CSG2PR04MB162977CFE150444FA022510FB6370%40SG2PR04MB1629.apcprd04.prod.outlook.com%3E
>
> Thoughts?
>
> Thanks,
> - Tsuyoshi
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
--
busbey
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [DISCUSS] Official Docker Image at release time
Posted by Sean Busbey <bu...@cloudera.com>.
FWIW, there is a "blessed" apache area on docker hub now, and it's
just an INFRA request to point out the needed Dockerfile in the repo.
PMCs can also request write access to bintray hosting of docker images
for PMC members.
Info on INFRA-8441, example on INFRA-12019.
A Docker image that starts up a pseudo distributed instance would be
great for new folks. I don't know if it's worth the investment to
build images that do more complex multi-instance deployments (though
there is some fledgling work in HBase for tooling that would do this
for us).
On Tue, Jul 19, 2016 at 2:46 AM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
> Hi developers,
>
> Klaus mentioned the availability of an official docker image of Apache
> Hadoop. Is it time that we start to distribute an official docker
> image at release time?
>
> http://mail-archives.apache.org/mod_mbox/hadoop-user/201607.mbox/%3CSG2PR04MB162977CFE150444FA022510FB6370%40SG2PR04MB1629.apcprd04.prod.outlook.com%3E
>
> Thoughts?
>
> Thanks,
> - Tsuyoshi
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
--
busbey
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [DISCUSS] Official Docker Image at release time
Posted by Sean Busbey <bu...@cloudera.com>.
FWIW, there is a "blessed" apache area on docker hub now, and it's
just an INFRA request to point out the needed Dockerfile in the repo.
PMCs can also request write access to bintray hosting of docker images
for PMC members.
Info on INFRA-8441, example on INFRA-12019.
A Docker image that starts up a pseudo distributed instance would be
great for new folks. I don't know if it's worth the investment to
build images that do more complex multi-instance deployments (though
there is some fledgling work in HBase for tooling that would do this
for us).
On Tue, Jul 19, 2016 at 2:46 AM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
> Hi developers,
>
> Klaus mentioned the availability of an official docker image of Apache
> Hadoop. Is it time that we start to distribute an official docker
> image at release time?
>
> http://mail-archives.apache.org/mod_mbox/hadoop-user/201607.mbox/%3CSG2PR04MB162977CFE150444FA022510FB6370%40SG2PR04MB1629.apcprd04.prod.outlook.com%3E
>
> Thoughts?
>
> Thanks,
> - Tsuyoshi
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
--
busbey
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [DISCUSS] Official Docker Image at release time
Posted by Tsuyoshi Ozawa <oz...@apache.org>.
> Create a JIRA to trace this requirement at: https://issues.apache.org/jira/browse/HADOOP-13397
Awesome. Thanks Klaus for filing the issue.
Do you have any idea to solve the concern Roman and Sean have? In
other words, could you clarify what your Docker script actually does?
> I am with Sean on this one. To make it truly useful outside of a Quickstart VM
> use case the community image has to deal with configuration management
> in a much more explicit way. Just sticking java bits into a Docker container
> won't really solve much.
If I understand your idea correctly, the dockerfile script doesn't
manage configuration: it will be copied from user-defined
configuration files, right?
Thanks
- Tsuyoshi
On Thu, Jul 21, 2016 at 6:38 PM, Klaus Ma <kl...@hotmail.com> wrote:
> Thanks, Tsuyoshi!
>
> Create a JIRA to trace this requirement at:
> https://issues.apache.org/jira/browse/HADOOP-13397
>
> ----
> Da (Klaus) Ma (马达), PMP® | Software Architect
> IBM Spectrum, STG, IBM GCG
> +86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
>
> On Jul 21, 2016, at 13:33, Tsuyoshi Ozawa <oz...@apache.org> wrote:
>
> Forwarding this discussion to Klaus.
>
> - Tsuyoshi
>
> On Tue, Jul 19, 2016 at 4:46 PM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
>
> Hi developers,
>
> Klaus mentioned the availability of an official docker image of Apache
> Hadoop. Is it time that we start to distribute an official docker
> image at release time?
>
> http://mail-archives.apache.org/mod_mbox/hadoop-user/201607.mbox/%3CSG2PR04MB162977CFE150444FA022510FB6370%40SG2PR04MB1629.apcprd04.prod.outlook.com%3E
>
> Thoughts?
>
> Thanks,
> - Tsuyoshi
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [DISCUSS] Official Docker Image at release time
Posted by Tsuyoshi Ozawa <oz...@apache.org>.
> Create a JIRA to trace this requirement at: https://issues.apache.org/jira/browse/HADOOP-13397
Awesome. Thanks Klaus for filing the issue.
Do you have any idea to solve the concern Roman and Sean have? In
other words, could you clarify what your Docker script actually does?
> I am with Sean on this one. To make it truly useful outside of a Quickstart VM
> use case the community image has to deal with configuration management
> in a much more explicit way. Just sticking java bits into a Docker container
> won't really solve much.
If I understand your idea correctly, the dockerfile script doesn't
manage configuration: it will be copied from user-defined
configuration files, right?
Thanks
- Tsuyoshi
On Thu, Jul 21, 2016 at 6:38 PM, Klaus Ma <kl...@hotmail.com> wrote:
> Thanks, Tsuyoshi!
>
> Create a JIRA to trace this requirement at:
> https://issues.apache.org/jira/browse/HADOOP-13397
>
> ----
> Da (Klaus) Ma (马达), PMP® | Software Architect
> IBM Spectrum, STG, IBM GCG
> +86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
>
> On Jul 21, 2016, at 13:33, Tsuyoshi Ozawa <oz...@apache.org> wrote:
>
> Forwarding this discussion to Klaus.
>
> - Tsuyoshi
>
> On Tue, Jul 19, 2016 at 4:46 PM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
>
> Hi developers,
>
> Klaus mentioned the availability of an official docker image of Apache
> Hadoop. Is it time that we start to distribute an official docker
> image at release time?
>
> http://mail-archives.apache.org/mod_mbox/hadoop-user/201607.mbox/%3CSG2PR04MB162977CFE150444FA022510FB6370%40SG2PR04MB1629.apcprd04.prod.outlook.com%3E
>
> Thoughts?
>
> Thanks,
> - Tsuyoshi
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [DISCUSS] Official Docker Image at release time
Posted by Tsuyoshi Ozawa <oz...@apache.org>.
> Create a JIRA to trace this requirement at: https://issues.apache.org/jira/browse/HADOOP-13397
Awesome. Thanks Klaus for filing the issue.
Do you have any idea to solve the concern Roman and Sean have? In
other words, could you clarify what your Docker script actually does?
> I am with Sean on this one. To make it truly useful outside of a Quickstart VM
> use case the community image has to deal with configuration management
> in a much more explicit way. Just sticking java bits into a Docker container
> won't really solve much.
If I understand your idea correctly, the dockerfile script doesn't
manage configuration: it will be copied from user-defined
configuration files, right?
Thanks
- Tsuyoshi
On Thu, Jul 21, 2016 at 6:38 PM, Klaus Ma <kl...@hotmail.com> wrote:
> Thanks, Tsuyoshi!
>
> Create a JIRA to trace this requirement at:
> https://issues.apache.org/jira/browse/HADOOP-13397
>
> ----
> Da (Klaus) Ma (马达), PMP® | Software Architect
> IBM Spectrum, STG, IBM GCG
> +86-10-8245 4084 | klaus1982.cn@gmail.com | http://k82.me
>
> On Jul 21, 2016, at 13:33, Tsuyoshi Ozawa <oz...@apache.org> wrote:
>
> Forwarding this discussion to Klaus.
>
> - Tsuyoshi
>
> On Tue, Jul 19, 2016 at 4:46 PM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
>
> Hi developers,
>
> Klaus mentioned the availability of an official docker image of Apache
> Hadoop. Is it time that we start to distribute an official docker
> image at release time?
>
> http://mail-archives.apache.org/mod_mbox/hadoop-user/201607.mbox/%3CSG2PR04MB162977CFE150444FA022510FB6370%40SG2PR04MB1629.apcprd04.prod.outlook.com%3E
>
> Thoughts?
>
> Thanks,
> - Tsuyoshi
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [DISCUSS] Official Docker Image at release time
Posted by Tsuyoshi Ozawa <oz...@apache.org>.
Forwarding this discussion to Klaus.
- Tsuyoshi
On Tue, Jul 19, 2016 at 4:46 PM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
> Hi developers,
>
> Klaus mentioned the availability of an official docker image of Apache
> Hadoop. Is it time that we start to distribute an official docker
> image at release time?
>
> http://mail-archives.apache.org/mod_mbox/hadoop-user/201607.mbox/%3CSG2PR04MB162977CFE150444FA022510FB6370%40SG2PR04MB1629.apcprd04.prod.outlook.com%3E
>
> Thoughts?
>
> Thanks,
> - Tsuyoshi
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [DISCUSS] Official Docker Image at release time
Posted by Tsuyoshi Ozawa <oz...@apache.org>.
Forwarding this discussion to Klaus.
- Tsuyoshi
On Tue, Jul 19, 2016 at 4:46 PM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
> Hi developers,
>
> Klaus mentioned the availability of an official docker image of Apache
> Hadoop. Is it time that we start to distribute an official docker
> image at release time?
>
> http://mail-archives.apache.org/mod_mbox/hadoop-user/201607.mbox/%3CSG2PR04MB162977CFE150444FA022510FB6370%40SG2PR04MB1629.apcprd04.prod.outlook.com%3E
>
> Thoughts?
>
> Thanks,
> - Tsuyoshi
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [DISCUSS] Official Docker Image at release time
Posted by Tsuyoshi Ozawa <oz...@apache.org>.
Forwarding this discussion to Klaus.
- Tsuyoshi
On Tue, Jul 19, 2016 at 4:46 PM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
> Hi developers,
>
> Klaus mentioned the availability of an official docker image of Apache
> Hadoop. Is it time that we start to distribute an official docker
> image at release time?
>
> http://mail-archives.apache.org/mod_mbox/hadoop-user/201607.mbox/%3CSG2PR04MB162977CFE150444FA022510FB6370%40SG2PR04MB1629.apcprd04.prod.outlook.com%3E
>
> Thoughts?
>
> Thanks,
> - Tsuyoshi
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [DISCUSS] Official Docker Image at release time
Posted by Sean Busbey <bu...@cloudera.com>.
Folks might want to take a look at
https://issues.apache.org/jira/browse/HBASE-12721
and its associated review board posting of aggregate work. One of hte
HBase community members has been creating infra (geared towards test
ATM) for spinning up clusters of docker images. from what I
understand, right now that includes both hdfs, hbase, and yarn
clusters.
On Thu, Jul 21, 2016 at 5:07 AM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
> Hi Kai,
>
> Thanks for your positive feedback!
>
>> I think also providing Dockerfile in some flexible form (like template
> file or fetching configuration from env variables?)
> is useful because Docker image has fixed configuration and there is no room
> unless overriding with FROM.
>
> Sounds good idea. What do you think about the difference of roles
> between master and workers? Should we prepare separate Dockerfiles for each
> role?
>
> Thanks,
> - Tsuyoshi
>
> On Tuesday, 19 July 2016, Sasaki Kai <sa...@treasure-data.com> wrote:
>
>> Hi Tsuyoshi
>>
>> Official docker image of Hadoop sounds very good to me.
>> In my use case, we usually use Docker image of Hadoop as PoC or
>> development cluster because it is easy to deploy and modify.
>> Currently we use below Docker image or Dockerfile to launch our
>> development cluster.
>> https://hub.docker.com/r/sequenceiq/hadoop-docker/
>>
>> But it does not use the latest Hadoop package and dependencies. There is
>> often some time lag to catch up and updating.
>>
>> So there is a reason to provide Docker image and Dockerfile from
>> community. It enables developers to try Hadoop easily.
>> I think also providing Dockerfile in some flexible form (like template
>> file or fetching configuration from env variables?)
>> is useful because Docker image has fixed configuration and there is no
>> room unless overriding with FROM.
>> I assume developers who want to try new Hadoop also would like to modify
>> configuration or dependencies.
>> This can be achieved by flexible Dockerfile which uses configuration
>> passed from environment variables.
>>
>> Thanks
>>
>> Kai Sasaki
>>
>>
>> On Jul 19, 2016, at 4:46 PM, Tsuyoshi Ozawa <ozawa@apache.org
>> <javascript:_e(%7B%7D,'cvml','ozawa@apache.org');>> wrote:
>>
>> Hi developers,
>>
>> Klaus mentioned the availability of an official docker image of Apache
>> Hadoop. Is it time that we start to distribute an official docker
>> image at release time?
>>
>>
>> http://mail-archives.apache.org/mod_mbox/hadoop-user/201607.mbox/%3CSG2PR04MB162977CFE150444FA022510FB6370%40SG2PR04MB1629.apcprd04.prod.outlook.com%3E
>>
>> Thoughts?
>>
>> Thanks,
>> - Tsuyoshi
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> <javascript:_e(%7B%7D,'cvml','yarn-dev-unsubscribe@hadoop.apache.org');>
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>> <javascript:_e(%7B%7D,'cvml','yarn-dev-help@hadoop.apache.org');>
>>
>>
>>
--
busbey
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [DISCUSS] Official Docker Image at release time
Posted by Sean Busbey <bu...@cloudera.com>.
Folks might want to take a look at
https://issues.apache.org/jira/browse/HBASE-12721
and its associated review board posting of aggregate work. One of hte
HBase community members has been creating infra (geared towards test
ATM) for spinning up clusters of docker images. from what I
understand, right now that includes both hdfs, hbase, and yarn
clusters.
On Thu, Jul 21, 2016 at 5:07 AM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
> Hi Kai,
>
> Thanks for your positive feedback!
>
>> I think also providing Dockerfile in some flexible form (like template
> file or fetching configuration from env variables?)
> is useful because Docker image has fixed configuration and there is no room
> unless overriding with FROM.
>
> Sounds good idea. What do you think about the difference of roles
> between master and workers? Should we prepare separate Dockerfiles for each
> role?
>
> Thanks,
> - Tsuyoshi
>
> On Tuesday, 19 July 2016, Sasaki Kai <sa...@treasure-data.com> wrote:
>
>> Hi Tsuyoshi
>>
>> Official docker image of Hadoop sounds very good to me.
>> In my use case, we usually use Docker image of Hadoop as PoC or
>> development cluster because it is easy to deploy and modify.
>> Currently we use below Docker image or Dockerfile to launch our
>> development cluster.
>> https://hub.docker.com/r/sequenceiq/hadoop-docker/
>>
>> But it does not use the latest Hadoop package and dependencies. There is
>> often some time lag to catch up and updating.
>>
>> So there is a reason to provide Docker image and Dockerfile from
>> community. It enables developers to try Hadoop easily.
>> I think also providing Dockerfile in some flexible form (like template
>> file or fetching configuration from env variables?)
>> is useful because Docker image has fixed configuration and there is no
>> room unless overriding with FROM.
>> I assume developers who want to try new Hadoop also would like to modify
>> configuration or dependencies.
>> This can be achieved by flexible Dockerfile which uses configuration
>> passed from environment variables.
>>
>> Thanks
>>
>> Kai Sasaki
>>
>>
>> On Jul 19, 2016, at 4:46 PM, Tsuyoshi Ozawa <ozawa@apache.org
>> <javascript:_e(%7B%7D,'cvml','ozawa@apache.org');>> wrote:
>>
>> Hi developers,
>>
>> Klaus mentioned the availability of an official docker image of Apache
>> Hadoop. Is it time that we start to distribute an official docker
>> image at release time?
>>
>>
>> http://mail-archives.apache.org/mod_mbox/hadoop-user/201607.mbox/%3CSG2PR04MB162977CFE150444FA022510FB6370%40SG2PR04MB1629.apcprd04.prod.outlook.com%3E
>>
>> Thoughts?
>>
>> Thanks,
>> - Tsuyoshi
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> <javascript:_e(%7B%7D,'cvml','yarn-dev-unsubscribe@hadoop.apache.org');>
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>> <javascript:_e(%7B%7D,'cvml','yarn-dev-help@hadoop.apache.org');>
>>
>>
>>
--
busbey
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [DISCUSS] Official Docker Image at release time
Posted by Sean Busbey <bu...@cloudera.com>.
Folks might want to take a look at
https://issues.apache.org/jira/browse/HBASE-12721
and its associated review board posting of aggregate work. One of hte
HBase community members has been creating infra (geared towards test
ATM) for spinning up clusters of docker images. from what I
understand, right now that includes both hdfs, hbase, and yarn
clusters.
On Thu, Jul 21, 2016 at 5:07 AM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
> Hi Kai,
>
> Thanks for your positive feedback!
>
>> I think also providing Dockerfile in some flexible form (like template
> file or fetching configuration from env variables?)
> is useful because Docker image has fixed configuration and there is no room
> unless overriding with FROM.
>
> Sounds good idea. What do you think about the difference of roles
> between master and workers? Should we prepare separate Dockerfiles for each
> role?
>
> Thanks,
> - Tsuyoshi
>
> On Tuesday, 19 July 2016, Sasaki Kai <sa...@treasure-data.com> wrote:
>
>> Hi Tsuyoshi
>>
>> Official docker image of Hadoop sounds very good to me.
>> In my use case, we usually use Docker image of Hadoop as PoC or
>> development cluster because it is easy to deploy and modify.
>> Currently we use below Docker image or Dockerfile to launch our
>> development cluster.
>> https://hub.docker.com/r/sequenceiq/hadoop-docker/
>>
>> But it does not use the latest Hadoop package and dependencies. There is
>> often some time lag to catch up and updating.
>>
>> So there is a reason to provide Docker image and Dockerfile from
>> community. It enables developers to try Hadoop easily.
>> I think also providing Dockerfile in some flexible form (like template
>> file or fetching configuration from env variables?)
>> is useful because Docker image has fixed configuration and there is no
>> room unless overriding with FROM.
>> I assume developers who want to try new Hadoop also would like to modify
>> configuration or dependencies.
>> This can be achieved by flexible Dockerfile which uses configuration
>> passed from environment variables.
>>
>> Thanks
>>
>> Kai Sasaki
>>
>>
>> On Jul 19, 2016, at 4:46 PM, Tsuyoshi Ozawa <ozawa@apache.org
>> <javascript:_e(%7B%7D,'cvml','ozawa@apache.org');>> wrote:
>>
>> Hi developers,
>>
>> Klaus mentioned the availability of an official docker image of Apache
>> Hadoop. Is it time that we start to distribute an official docker
>> image at release time?
>>
>>
>> http://mail-archives.apache.org/mod_mbox/hadoop-user/201607.mbox/%3CSG2PR04MB162977CFE150444FA022510FB6370%40SG2PR04MB1629.apcprd04.prod.outlook.com%3E
>>
>> Thoughts?
>>
>> Thanks,
>> - Tsuyoshi
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> <javascript:_e(%7B%7D,'cvml','yarn-dev-unsubscribe@hadoop.apache.org');>
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>> <javascript:_e(%7B%7D,'cvml','yarn-dev-help@hadoop.apache.org');>
>>
>>
>>
--
busbey
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [DISCUSS] Official Docker Image at release time
Posted by Tsuyoshi Ozawa <oz...@apache.org>.
Hi Kai,
Thanks for your positive feedback!
> I think also providing Dockerfile in some flexible form (like template
file or fetching configuration from env variables?)
is useful because Docker image has fixed configuration and there is no room
unless overriding with FROM.
Sounds good idea. What do you think about the difference of roles
between master and workers? Should we prepare separate Dockerfiles for each
role?
Thanks,
- Tsuyoshi
On Tuesday, 19 July 2016, Sasaki Kai <sa...@treasure-data.com> wrote:
> Hi Tsuyoshi
>
> Official docker image of Hadoop sounds very good to me.
> In my use case, we usually use Docker image of Hadoop as PoC or
> development cluster because it is easy to deploy and modify.
> Currently we use below Docker image or Dockerfile to launch our
> development cluster.
> https://hub.docker.com/r/sequenceiq/hadoop-docker/
>
> But it does not use the latest Hadoop package and dependencies. There is
> often some time lag to catch up and updating.
>
> So there is a reason to provide Docker image and Dockerfile from
> community. It enables developers to try Hadoop easily.
> I think also providing Dockerfile in some flexible form (like template
> file or fetching configuration from env variables?)
> is useful because Docker image has fixed configuration and there is no
> room unless overriding with FROM.
> I assume developers who want to try new Hadoop also would like to modify
> configuration or dependencies.
> This can be achieved by flexible Dockerfile which uses configuration
> passed from environment variables.
>
> Thanks
>
> Kai Sasaki
>
>
> On Jul 19, 2016, at 4:46 PM, Tsuyoshi Ozawa <ozawa@apache.org
> <javascript:_e(%7B%7D,'cvml','ozawa@apache.org');>> wrote:
>
> Hi developers,
>
> Klaus mentioned the availability of an official docker image of Apache
> Hadoop. Is it time that we start to distribute an official docker
> image at release time?
>
>
> http://mail-archives.apache.org/mod_mbox/hadoop-user/201607.mbox/%3CSG2PR04MB162977CFE150444FA022510FB6370%40SG2PR04MB1629.apcprd04.prod.outlook.com%3E
>
> Thoughts?
>
> Thanks,
> - Tsuyoshi
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> <javascript:_e(%7B%7D,'cvml','yarn-dev-unsubscribe@hadoop.apache.org');>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> <javascript:_e(%7B%7D,'cvml','yarn-dev-help@hadoop.apache.org');>
>
>
>
Re: [DISCUSS] Official Docker Image at release time
Posted by Tsuyoshi Ozawa <oz...@apache.org>.
Hi Kai,
Thanks for your positive feedback!
> I think also providing Dockerfile in some flexible form (like template
file or fetching configuration from env variables?)
is useful because Docker image has fixed configuration and there is no room
unless overriding with FROM.
Sounds good idea. What do you think about the difference of roles
between master and workers? Should we prepare separate Dockerfiles for each
role?
Thanks,
- Tsuyoshi
On Tuesday, 19 July 2016, Sasaki Kai <sa...@treasure-data.com> wrote:
> Hi Tsuyoshi
>
> Official docker image of Hadoop sounds very good to me.
> In my use case, we usually use Docker image of Hadoop as PoC or
> development cluster because it is easy to deploy and modify.
> Currently we use below Docker image or Dockerfile to launch our
> development cluster.
> https://hub.docker.com/r/sequenceiq/hadoop-docker/
>
> But it does not use the latest Hadoop package and dependencies. There is
> often some time lag to catch up and updating.
>
> So there is a reason to provide Docker image and Dockerfile from
> community. It enables developers to try Hadoop easily.
> I think also providing Dockerfile in some flexible form (like template
> file or fetching configuration from env variables?)
> is useful because Docker image has fixed configuration and there is no
> room unless overriding with FROM.
> I assume developers who want to try new Hadoop also would like to modify
> configuration or dependencies.
> This can be achieved by flexible Dockerfile which uses configuration
> passed from environment variables.
>
> Thanks
>
> Kai Sasaki
>
>
> On Jul 19, 2016, at 4:46 PM, Tsuyoshi Ozawa <ozawa@apache.org
> <javascript:_e(%7B%7D,'cvml','ozawa@apache.org');>> wrote:
>
> Hi developers,
>
> Klaus mentioned the availability of an official docker image of Apache
> Hadoop. Is it time that we start to distribute an official docker
> image at release time?
>
>
> http://mail-archives.apache.org/mod_mbox/hadoop-user/201607.mbox/%3CSG2PR04MB162977CFE150444FA022510FB6370%40SG2PR04MB1629.apcprd04.prod.outlook.com%3E
>
> Thoughts?
>
> Thanks,
> - Tsuyoshi
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> <javascript:_e(%7B%7D,'cvml','yarn-dev-unsubscribe@hadoop.apache.org');>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> <javascript:_e(%7B%7D,'cvml','yarn-dev-help@hadoop.apache.org');>
>
>
>
Re: [DISCUSS] Official Docker Image at release time
Posted by Tsuyoshi Ozawa <oz...@apache.org>.
Hi Kai,
Thanks for your positive feedback!
> I think also providing Dockerfile in some flexible form (like template
file or fetching configuration from env variables?)
is useful because Docker image has fixed configuration and there is no room
unless overriding with FROM.
Sounds good idea. What do you think about the difference of roles
between master and workers? Should we prepare separate Dockerfiles for each
role?
Thanks,
- Tsuyoshi
On Tuesday, 19 July 2016, Sasaki Kai <sa...@treasure-data.com> wrote:
> Hi Tsuyoshi
>
> Official docker image of Hadoop sounds very good to me.
> In my use case, we usually use Docker image of Hadoop as PoC or
> development cluster because it is easy to deploy and modify.
> Currently we use below Docker image or Dockerfile to launch our
> development cluster.
> https://hub.docker.com/r/sequenceiq/hadoop-docker/
>
> But it does not use the latest Hadoop package and dependencies. There is
> often some time lag to catch up and updating.
>
> So there is a reason to provide Docker image and Dockerfile from
> community. It enables developers to try Hadoop easily.
> I think also providing Dockerfile in some flexible form (like template
> file or fetching configuration from env variables?)
> is useful because Docker image has fixed configuration and there is no
> room unless overriding with FROM.
> I assume developers who want to try new Hadoop also would like to modify
> configuration or dependencies.
> This can be achieved by flexible Dockerfile which uses configuration
> passed from environment variables.
>
> Thanks
>
> Kai Sasaki
>
>
> On Jul 19, 2016, at 4:46 PM, Tsuyoshi Ozawa <ozawa@apache.org
> <javascript:_e(%7B%7D,'cvml','ozawa@apache.org');>> wrote:
>
> Hi developers,
>
> Klaus mentioned the availability of an official docker image of Apache
> Hadoop. Is it time that we start to distribute an official docker
> image at release time?
>
>
> http://mail-archives.apache.org/mod_mbox/hadoop-user/201607.mbox/%3CSG2PR04MB162977CFE150444FA022510FB6370%40SG2PR04MB1629.apcprd04.prod.outlook.com%3E
>
> Thoughts?
>
> Thanks,
> - Tsuyoshi
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> <javascript:_e(%7B%7D,'cvml','yarn-dev-unsubscribe@hadoop.apache.org');>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> <javascript:_e(%7B%7D,'cvml','yarn-dev-help@hadoop.apache.org');>
>
>
>