Cloud native images for ASF projects.

[jay vyas <ja...@gmail.com>]

Hi Bigtop !


As you all know, I used to work on bigtop and ever since have been spending
alot of time working on kubernetes and cloud native application engineering.

One thing I'm noticing is that the quality of docker images for ASF
projects varies, and* it seems like a distributed systems / cloud native
viewpoint isn't expressed in the Dockerfiles for some projects.*

- Some docker files expect ROOT or a specific UID as an option.
- Some docker files chmod/chown things unnecessarily, meaning they can't
run in all clusters.
- and so on ...

I feel like bigtop is a good place to mitigate some of these problems for
the ASF community.

*Is anyone in BigTop interested in BigTop curated dockerfiles for "real"
Zookeeper, SOLR, etc deployments ?  *

If so, we can curate our own official Docker images that honor the
distributed systems 'rules of thumb' , in the same way that we build
RPM/DEB packages for this purpose.

Our engineering teams would be happy to help in  effort to curate such
images as well as set up CI for them...

In particular, *I think bigtop-official images for SOLR and Zookeeper are a
good starting point, *as those are very common in containerized
environments.  After that I think Spark, HBase, Hadoop and so on might be
interesting.

Interested in thoughts around this and wether or not its valuable to the
community?

-- 
jay vyas

Re: Cloud native images for ASF projects.

[Jazz Yao-Tsung Wang <ja...@gmail.com>]

Hi Jay,


Have you file a JIRA for this issue?
I'll follow up on JIRA in future.

I'm interested in BigTop curated dockerfiles.
I've put automated build docker images on docker hub.
But it's currently based on BigTop 1.1.0.
Source are available at https://github.com/jazzwang/docker-pool

~$ docker search jazzwang | grep bigtop
jazzwang/bigtop-hdfs              Apache BigTop HDFS in
pseudo-distributed mode   1                    [OK]
jazzwang/bigtop-pig               Apache Pig Quick Start Image
           1                    [OK]
jazzwang/bigtop-sqoop             Apache Sqoop Quick Start Image
           1                    [OK]
jazzwang/bigtop-mahout            Apache Mahout Quick Start Image
           1                    [OK]
jazzwang/bigtop-hive              Apache Hive Quick Start Image
           1                    [OK]
jazzwang/bigtop-yarn              Apache BigTop YARN in
pseudo-distributed mode   1                    [OK]
jazzwang/bigtop-hbase             Apache HBase Quick Start Image
           1                    [OK]

​

- Jazz


On Wed, Aug 16, 2017 at 10:22 PM, jay vyas <ja...@gmail.com>
wrote:

> Hi Bigtop !
>
>
> As you all know, I used to work on bigtop and ever since have been
> spending alot of time working on kubernetes and cloud native application
> engineering.
>
> One thing I'm noticing is that the quality of docker images for ASF
> projects varies, and* it seems like a distributed systems / cloud native
> viewpoint isn't expressed in the Dockerfiles for some projects.*
>
> - Some docker files expect ROOT or a specific UID as an option.
> - Some docker files chmod/chown things unnecessarily, meaning they can't
> run in all clusters.
> - and so on ...
>
> I feel like bigtop is a good place to mitigate some of these problems for
> the ASF community.
>
> *Is anyone in BigTop interested in BigTop curated dockerfiles for "real"
> Zookeeper, SOLR, etc deployments ?  *
>
> If so, we can curate our own official Docker images that honor the
> distributed systems 'rules of thumb' , in the same way that we build
> RPM/DEB packages for this purpose.
>
> Our engineering teams would be happy to help in  effort to curate such
> images as well as set up CI for them...
>
> In particular, *I think bigtop-official images for SOLR and Zookeeper are
> a good starting point, *as those are very common in containerized
> environments.  After that I think Spark, HBase, Hadoop and so on might be
> interesting.
>
> Interested in thoughts around this and wether or not its valuable to the
> community?
>
> --
> jay vyas
>