You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by sjeandeaux <gi...@git.apache.org> on 2015/08/01 09:52:10 UTC
[GitHub] incubator-zeppelin pull request: Add dockerfile (and makefile) to ...
GitHub user sjeandeaux opened a pull request:
https://github.com/apache/incubator-zeppelin/pull/175
Add dockerfile (and makefile) to build local version
Dockerfile contains:
* jdk-7u80-linux-x64
* maven >= 3.10
* sbt >= 0.13
* nodejs >= 0.12
* git gcc-c++ make bzip2
Makefile contains goals:
* **build** the image
* **run** with image
* install-local runs goal maven => `mvn clean install -DskipTests`
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sjeandeaux/incubator-zeppelin dockerfile
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-zeppelin/pull/175.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #175
----
commit 99319d4bf02d092f60231645a4070e4eb57db189
Author: Stéphane Jeandeaux <st...@gmail.com>
Date: 2015-08-01T07:47:10Z
Add dockerfile (and makefile) to build local version
Dockerfile contains:
* jdk-7u80-linux-x64
* maven >= 3.10
* sbt >= 0.13
* nodejs >= 0.12
* git gcc-c++ make bzip2
Makefile contains goals:
* **build** the image
* **run** with image
* install-local runs goal maven => `mvn clean install -DskipTests`
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Add dockerfile (and makefile) to ...
Posted by jersub <gi...@git.apache.org>.
Github user jersub commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/175#issuecomment-155767085
@sjeandeaux
- I suggest to expose `conf`, `logs` and `notebook` as volumes.
- Recently the recommended way to start Zeppelin is to provide `SPARK_HOME` (and optionally `HADOOP_HOME`) so there's no need to manage an official build for every Spark/Hadoop combination.
- Running Zeppelin through user `zeppelin` would be considered as more secure. However, it used to make the use of RW volumes painful. We should check if it is still the case as of version 1.9 of Docker Engine. Work has moved forward in this field but I don't know if the user namespace feature is actually complete.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Add dockerfile (and makefile) to ...
Posted by jersub <gi...@git.apache.org>.
Github user jersub commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/175#issuecomment-151880934
Hi,
I would expect the default Docker image to run Zeppelin, not to build it. I mean building Zeppelin at image creation instead, so the service can be run as a container.
Thanks to such a Dockerfile, one could create an "automated build" repository on Docker Hub, triggering a build for each commit. Even better, there would be an official Zeppelin repository on Docker Hub ;)
FI here is how I'm doing it: https://github.com/jersub/zeppelin-docker
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Add dockerfile (and makefile) to ...
Posted by sjeandeaux <gi...@git.apache.org>.
Github user sjeandeaux commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/175#issuecomment-140651361
- [ ] change `MAINTAINER` into `Zeppelin Project (incubating)`
- [ ] Expose "~/.m2" repository in Dockerfile as `VOLUME`
- [ ] Add comments at the top of file => how to build it and how to use it
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Add dockerfile (and makefile) to ...
Posted by bzz <gi...@git.apache.org>.
Github user bzz commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/175#issuecomment-153590401
Thank your addressing all the feedback @sjeandeaux !
It looks great but I have one more question, sorry if I missed that one before. As @jersub mention, the built image does not contain the built binary of zeppelin itself right now, am I right?
So what the use case you have in mind? For now I can imagine one, looking like this:
```
docker build -t zeppelin
docker run -it ...
mvn package ...
./bin/zeppelin-daemon.sh
```
And then Zeppelin becomes accessible on the forwarded host ports, right?
This is all good, but what if we at the same time will support something like this:
```
docker build -t zeppelin
docker push
docker run ....
```
and then have Zeppelin accessible on the forwarded host ports?
This way people who do not want to build it - could just opt out for the fast
```
docker pull ..
docker run ..
```
For the second and third case to work just a few minor changes required, like
- expose `SPARK_HOME` and `HADOOP_HOME` to the container, as it affects the build
- actually build zeppelin like `RUN mvn package ....` (and you can still override it w/ your own `/home/zeppelin/incubator-zeppelin/` in `-it` mode, so we keep the ability to build inside the container, as in case 1)
- point default PID1 script of the image to `/home/zeppelin/incubator-zeppelin/bin/zeppelin.sh`
But I think that might provide a lot of value for users who do not wish to build Zeppelin manually.
What do you think?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Add dockerfile (and makefile) to ...
Posted by sjeandeaux <gi...@git.apache.org>.
Github user sjeandeaux commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/175#issuecomment-127566868
* **gcc-c++** i forget to delete it when i delete installation of node
* **bzip2** when i used installation of node, npm need to untar bz2 file. I will test without.
* **ENV** in one line, it's ridiculous i just want unless layout. For more readability, i will split in two lines.
* **yum clean all** i will do it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Add dockerfile (and makefile) to ...
Posted by sjeandeaux <gi...@git.apache.org>.
Github user sjeandeaux commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/175#issuecomment-153906721
@bzz I agree with you, it's import to have a image with build of zeppelin.
But the difficulty, it's to manage all possibilities (spark version and hadoop version).
I create two Dockerfile:
* In `dev-tools` contains all tools to build and run zeppelin for developers (like hadoop project)
* The root of project contains a Dockerfile which creates a image of zeppelin.
I have some actions to realize:
- [ ] determinate if i need to create a user `zeppelin` in Dockfile zeppelin.
- [ ] determinate if i need to explose explicitly the volume.
- [ ] manage version of spark and hadoop
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Add dockerfile (and makefile) to ...
Posted by sjeandeaux <gi...@git.apache.org>.
Github user sjeandeaux commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/175#issuecomment-127340777
I delete in Dockerfile:
* sbt
* node
I add **Apache 2.0 headers** in new files.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Add dockerfile (and makefile) to ...
Posted by sjeandeaux <gi...@git.apache.org>.
Github user sjeandeaux commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/175#issuecomment-153481041
In Resume this pull-request contains:
* `Dockerfile` : contains all tools to build `zeppelin`:
* The `MAINTAINER` is ` Zeppelin Project (incubating)`
* Add `VOLUME` on `~/.m2`
* Add a documentation in Dockerfile
* `docker-zeppelin.sh` : help to build, run `zeppelin`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Add dockerfile (and makefile) to ...
Posted by bzz <gi...@git.apache.org>.
Github user bzz commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/175#issuecomment-139987266
Sorry for by late reply, having a default Docker image for Zeppelin is a great idea, couple of things though:
- I think that `MAINTAINER` field should either be something like Zeppelin Project (incubating), or in case you are willing to volunteer to take care of it and leave as it is, may be this should be discussed on dev@ mailing list first, to let others know and get a consensus
- it might be a good idea to expose "~/.m2" repository in Dockerfile as `VOLUME` and advise users in comment at the top of the file to mount it before building an image (as you do in Makefile)
- generally it's a good practice to have a comment, at the top of the file, with explanation of what it is for, how to build it and how to use it
```sh
# Build:
# `make install-local` default
# `....` spark 1.5, hadoop 2.2
# Usage:
# docker run .... \
# .... <path-to>/bin/zeppelin.sh
#
# NOTE: you may want to use /root/.m2 volume to save time downloading dependencies
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Add dockerfile (and makefile) to ...
Posted by bzz <gi...@git.apache.org>.
Github user bzz commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/175#issuecomment-127110462
Thank you for contribution!
Can you please add Apache 2.0 headers to the new files that has been added?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---