You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hivemall.apache.org by my...@apache.org on 2018/09/04 06:30:22 UTC
incubator-hivemall git commit: [HIVEMALL-216] Fix Docker image based
on openjdk 8
Repository: incubator-hivemall
Updated Branches:
refs/heads/master 07eb707b7 -> 13e13027b
[HIVEMALL-216] Fix Docker image based on openjdk 8
## What changes were proposed in this pull request?
This PR fixes building Docker image from Docker file.
## What type of PR is it?
Bug Fix
## What is the Jira issue?
https://issues.apache.org/jira/browse/HIVEMALL-216
## How was this patch tested?
manual tests
## How to use this feature?
See [documentation](https://hivemall.incubator.apache.org/userguide/docker/getting_started.html)
Author: Aki Ariga <ar...@treasure-data.com>
Closes #161 from chezou/fix-dockerfile.
Project: http://git-wip-us.apache.org/repos/asf/incubator-hivemall/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hivemall/commit/13e13027
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hivemall/tree/13e13027
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hivemall/diff/13e13027
Branch: refs/heads/master
Commit: 13e13027b9ad601f0ea759ed024fdf25e0bd6616
Parents: 07eb707
Author: Aki Ariga <ar...@treasure-data.com>
Authored: Tue Sep 4 15:30:14 2018 +0900
Committer: Makoto Yui <my...@apache.org>
Committed: Tue Sep 4 15:30:14 2018 +0900
----------------------------------------------------------------------
docs/gitbook/docker/getting_started.md | 26 ++++++++++++++------------
resources/docker/Dockerfile | 19 +++++++++++--------
resources/docker/home/.hiverc | 2 +-
3 files changed, 26 insertions(+), 21 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hivemall/blob/13e13027/docs/gitbook/docker/getting_started.md
----------------------------------------------------------------------
diff --git a/docs/gitbook/docker/getting_started.md b/docs/gitbook/docker/getting_started.md
index d4b2991..314c82e 100644
--- a/docs/gitbook/docker/getting_started.md
+++ b/docs/gitbook/docker/getting_started.md
@@ -17,6 +17,8 @@
under the License.
-->
+# Getting started with Hivemall on Docker
+
This page introduces how to run Hivemall on Docker.
<!-- toc -->
@@ -24,22 +26,22 @@ This page introduces how to run Hivemall on Docker.
> #### Caution
> This docker image contains a single-node Hadoop enviroment for evaluating Hivemall. Not suited for production uses.
-# Requirements
+## Requirements
* Docker Engine 1.6+
* Docker Compose 1.10+
-# 1. Build image
+## Build image
You have two options in order to build a **hivemall** docker image:
-## Using `docker-compose`
+### Using `docker-compose`
```
$ docker-compose -f resources/docker/docker-compose.yml build
```
-## Using `docker` command
+### Using `docker` command
```
$ docker build -f resources/docker/Dockerfile .
@@ -48,11 +50,11 @@ $ docker build -f resources/docker/Dockerfile .
> #### Note
> You can [skip](./getting_started.html#running-pre-built-docker-image-in-docker-hub) building images if you try to use a pre-build docker image from Docker Hub. However, since the Docker Hub repository is experimental one, the distributed image is NOT built on the "latest" commit in [our master branch](https://github.com/apache/incubator-hivemall).
-# 2. Run container
+## Run container
If you built an image by yourself, it can be launched by either `docker-compose` or `docker` command:
-## By `docker-compose`
+### By `docker-compose`
```
$ docker-compose -f resources/docker/docker-compose.yml up -d && docker attach hivemall
@@ -68,7 +70,7 @@ volumes:
- "/path/to/data/:/root/data/" # mount resources to container-side `/root/data` directory
```
-## By `docker` command
+### By `docker` command
Find a local docker image by `docker images`, and hit:
@@ -84,7 +86,7 @@ Similarly to the `volumes` option in the `docker-compose` file, `docker run` has
$ docker run ... -v /path/to/local/hivemall:/opt/hivemall
```
-## Running pre-built Docker image in Docker Hub
+### Running pre-built Docker image in Docker Hub
> #### Caution
> This part is experimental. Hivemall in the pre-built image might be out-of-date compared to the latest version in [our master branch](https://github.com/apache/incubator-hivemall).
@@ -101,12 +103,12 @@ $ docker pull hivemall/latest:20170517
$ docker run -p 8088:8088 -p 50070:50070 -p 19888:19888 -it hivemall/latest:20170517
```
-# 3. Run Hivemall on Docker
+## Run Hivemall on Docker
1. Type `hive` to run (`.hiverc` automatically loads Hivemall functions)
2. Try your Hivemall queries!
-## Accessing Hadoop management GUIs
+### Accessing Hadoop management GUIs
* YARN http://localhost:8088/
* HDFS http://localhost:50070/
@@ -114,7 +116,7 @@ $ docker run -p 8088:8088 -p 50070:50070 -p 19888:19888 -it hivemall/latest:2017
Note that you need to expose local ports e.g., by `-p 8088:8088 -p 50070:50070 -p 19888:19888` on running docker image.
-## Load data into HDFS (optional)
+### Load data into HDFS (optional)
You can find an example script to load data into HDFS in `$HOME/bin/prepare_iris.sh`.
The script loads iris dataset into `iris` database:
@@ -137,7 +139,7 @@ OK
Once you prepared the `iris` database, you are ready to move on to [our multi-class classification tutorial](../multiclass/iris_dataset.html).
-## Build Hivemall (optional)
+### Build Hivemall (optional)
In the container, Hivemall resource is stored in `$HIVEMALL_PATH`.
You can build Hivemall package by `cd $HIVEMALL_PATH && ./bin/build.sh`.
http://git-wip-us.apache.org/repos/asf/incubator-hivemall/blob/13e13027/resources/docker/Dockerfile
----------------------------------------------------------------------
diff --git a/resources/docker/Dockerfile b/resources/docker/Dockerfile
index 6332f4c..39dc3f4 100644
--- a/resources/docker/Dockerfile
+++ b/resources/docker/Dockerfile
@@ -17,13 +17,13 @@
# under the License.
#
-FROM openjdk:7
+FROM openjdk:8
WORKDIR /root/
ARG PREBUILD=true
-ARG HADOOP_VERSION=2.7.3
-ARG HIVE_VERSION=2.1.1
+ARG HADOOP_VERSION=2.7.7
+ARG HIVE_VERSION=2.3.3
ENV BASE_URL='https://www.apache.org/dyn/mirrors/mirrors.cgi?action=download&filename='
ENV HADOOP_HOME='/usr/local/hadoop'
@@ -34,13 +34,16 @@ ENV HADOOP_OPTS=' \
-Dsystem:user.name=root \
-Dderby.stream.error.file=/root/derby.log'
ENV PATH="${HADOOP_HOME}/bin:${HIVE_HOME}/bin:${PATH}"
+ENV JAVA_HOME='/usr/lib/jvm/java-8-openjdk-amd64'
+ENV JAVA8_HOME='/usr/lib/jvm/java-8-openjdk-amd64'
COPY . ${HIVEMALL_PATH}/
RUN set -eux && \
- apt update && \
- apt install -y --no-install-recommends openssh-server maven g++ make ruby npm && \
- ln -s /usr/bin/nodejs /usr/bin/node && \
+ apt-get update && \
+ apt-get install -y --no-install-recommends openssh-server maven g++ make ruby && \
+ curl -sL https://deb.nodesource.com/setup_10.x | bash - && \
+ apt-get install -y nodejs && \
npm install -g gitbook-cli && \
\
wget ${BASE_URL}hadoop/common/hadoop-${HADOOP_VERSION}/hadoop-${HADOOP_VERSION}.tar.gz -O - \
@@ -72,8 +75,8 @@ RUN set -eux && \
-exec sh -c 'f={} && ln -s $f /root/bin/${f##*/}' \; && \
ln -s ${HIVEMALL_PATH}/resources/docker/home/.hiverc /root && \
ln -s ${HIVEMALL_PATH}/resources/ddl/define-all.hive /root/hivemall/define-all.hive && \
- ln -s ${HIVEMALL_PATH}/target/hivemall-core-${HIVEMALL_VERSION}-with-dependencies.jar \
- /root/hivemall/hivemall-core-with-dependencies.jar && \
+ ln -s ${HIVEMALL_PATH}/target/hivemall-all-${HIVEMALL_VERSION}.jar \
+ /root/hivemall/hivemall-all.jar && \
rm -rf /var/cache/apt/archives/* /var/lib/apt/lists/* /root/.m2/* /root/.npm/*
VOLUME ["/opt/hivemall/", "/root/data/"]
http://git-wip-us.apache.org/repos/asf/incubator-hivemall/blob/13e13027/resources/docker/home/.hiverc
----------------------------------------------------------------------
diff --git a/resources/docker/home/.hiverc b/resources/docker/home/.hiverc
index 0030939..0260b22 100644
--- a/resources/docker/home/.hiverc
+++ b/resources/docker/home/.hiverc
@@ -1,2 +1,2 @@
-add jar /root/hivemall/hivemall-core-with-dependencies.jar;
+add jar /root/hivemall/hivemall-all.jar;
source /root/hivemall/define-all.hive;