You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2021/03/26 05:55:56 UTC
[GitHub] [incubator-dolphinscheduler] Joder5 opened a new issue #5155: A Linux distribution for docker-compose
Joder5 opened a new issue #5155:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/5155
As I know，the image of apache/dolphinscheduler was base on the Alpine image。But when I install Python dependencies，It takes a lot of time，at least an hour。What's more，It will need more any other dependencies，for example
`apk update && apk add --no-cache gcc g++ python3-dev libffi-dev librdkafka librdkafka-dev mariadb-connector-c-dev musl-dev libxml2-utils libxslt libxslt-dev py3-numpy py3-pandas`
But in the end, I fail to install python dependencies，because of the pandas。
So can you build another version based on other Linux，such as slim。 [constrast](https://cloud.tencent.com/developer/news/600722)
Thank you very much ！
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] chengshiwen commented on issue #5155: A Linux distribution for docker-compose
Posted by GitBox <gi...@apache.org>.
chengshiwen commented on issue #5155:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/5155#issuecomment-807980257
@Joder5 This is not a problem. As for a any bare linux, none of the libraries you mentioned will exist.
In order to improve the update speed, you can set the mirror source of alpine as follows:
```
# 1. install command/library/software
# If install slowly, you can replcae alpine's mirror with aliyun's mirror, Example:
# RUN sed -i "s/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g" /etc/apk/repositories
# RUN sed -i 's/dl-cdn.alpinelinux.org/mirror.tuna.tsinghua.edu.cn/g' /etc/apk/repositories
```
As for `pip`, you can also use the mirror source like `https://pypi.tuna.tsinghua.edu.cn/simple`
```
pip install --no-cache-dir -i https://pypi.tuna.tsinghua.edu.cn/simple
```
As for `slim`, we will consider it later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] chengshiwen edited a comment on issue #5155: A Linux distribution for docker-compose
Posted by GitBox <gi...@apache.org>.
chengshiwen edited a comment on issue #5155:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/5155#issuecomment-808232047
I give you a version of Dockerfile based on `debian:slim`, which will be added to the warehouse after optimization in the future.
```
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
FROM openjdk:8-jdk-slim
ARG VERSION
ARG DEBIAN_FRONTEND=noninteractive
ENV TZ Asia/Shanghai
ENV LANG C.UTF-8
ENV DOCKER true
# 1. install command/library/software
# If install slowly, you can replcae alpine's mirror with aliyun's mirror, Example:
RUN echo \
"deb http://mirrors.tuna.tsinghua.edu.cn/debian/ buster main contrib non-free\n\
deb http://mirrors.tuna.tsinghua.edu.cn/debian/ buster-updates main contrib non-free\n\
deb http://mirrors.tuna.tsinghua.edu.cn/debian/ buster-backports main contrib non-free\n\
deb http://mirrors.tuna.tsinghua.edu.cn/debian-security buster/updates main contrib non-free" > /etc/apt/sources.list
RUN apt-get update && \
apt-get install -y tzdata dos2unix python python3 procps netcat sudo tini postgresql-client && \
echo "Asia/Shanghai" > /etc/timezone && \
rm -f /etc/localtime && \
dpkg-reconfigure tzdata && \
rm -rf /var/lib/apt/lists/* /tmp/*
# 2. add dolphinscheduler
ADD ./apache-dolphinscheduler-incubating-${VERSION}-dolphinscheduler-bin.tar.gz /opt/
RUN ln -s /opt/apache-dolphinscheduler-incubating-${VERSION}-dolphinscheduler-bin /opt/dolphinscheduler
ENV DOLPHINSCHEDULER_HOME /opt/dolphinscheduler
# 3. add configuration and modify permissions and set soft links
COPY ./checkpoint.sh /root/checkpoint.sh
COPY ./startup-init-conf.sh /root/startup-init-conf.sh
COPY ./startup.sh /root/startup.sh
COPY ./conf/dolphinscheduler/*.tpl /opt/dolphinscheduler/conf/
COPY ./conf/dolphinscheduler/logback/* /opt/dolphinscheduler/conf/
COPY ./conf/dolphinscheduler/env/dolphinscheduler_env.sh /opt/dolphinscheduler/conf/env/
RUN dos2unix /root/checkpoint.sh && \
dos2unix /root/startup-init-conf.sh && \
dos2unix /root/startup.sh && \
dos2unix /opt/dolphinscheduler/conf/env/dolphinscheduler_env.sh && \
dos2unix /opt/dolphinscheduler/script/*.sh && \
dos2unix /opt/dolphinscheduler/bin/*.sh && \
rm -rf /bin/sh && \
ln -s /bin/bash /bin/sh && \
mkdir -p /tmp/xls /usr/lib/jvm && \
ln -sf /usr/local/openjdk-8 /usr/lib/jvm/java-1.8-openjdk && \
echo "Set disable_coredump false" >> /etc/sudo.conf
# 4. expose port
EXPOSE 5678 1234 12345 50051
ENTRYPOINT ["/usr/bin/tini", "--", "/root/startup.sh"]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] chengshiwen commented on issue #5155: A Linux distribution for docker-compose
Posted by GitBox <gi...@apache.org>.
chengshiwen commented on issue #5155:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/5155#issuecomment-808232047
I give you a version of Dockerfile based on `debian:slim`, which will be added to the warehouse after optimization in the future.
```
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
FROM openjdk:8-jdk-slim
ARG VERSION
ARG DEBIAN_FRONTEND=noninteractive
ENV TZ Asia/Shanghai
ENV LANG C.UTF-8
ENV DOCKER true
# 1. install command/library/software
# If install slowly, you can replcae alpine's mirror with aliyun's mirror, Example:
RUN echo \
"deb http://mirrors.tuna.tsinghua.edu.cn/debian/ buster main contrib non-free\n\
deb http://mirrors.tuna.tsinghua.edu.cn/debian/ buster-updates main contrib non-free\n\
deb http://mirrors.tuna.tsinghua.edu.cn/debian/ buster-backports main contrib non-free\n\
deb http://mirrors.tuna.tsinghua.edu.cn/debian-security buster/updates main contrib non-free" > /etc/apt/sources.list
RUN apt-get update && \
apt-get install -y tzdata dos2unix python python3 procps netcat sudo tini postgresql-client && \
echo "Asia/Shanghai" > /etc/timezone && \
rm -f /etc/localtime && \
dpkg-reconfigure tzdata && \
rm -rf /var/lib/apt/lists/* /tmp/*
# 2. add dolphinscheduler
ADD ./apache-dolphinscheduler-incubating-${VERSION}-dolphinscheduler-bin.tar.gz /opt/
RUN ln -s /opt/apache-dolphinscheduler-incubating-${VERSION}-dolphinscheduler-bin /opt/dolphinscheduler
ENV DOLPHINSCHEDULER_HOME /opt/dolphinscheduler
# 3. add configuration and modify permissions and set soft links
COPY ./checkpoint.sh /root/checkpoint.sh
COPY ./startup-init-conf.sh /root/startup-init-conf.sh
COPY ./startup.sh /root/startup.sh
COPY ./conf/dolphinscheduler/*.tpl /opt/dolphinscheduler/conf/
COPY ./conf/dolphinscheduler/logback/* /opt/dolphinscheduler/conf/
COPY ./conf/dolphinscheduler/env/dolphinscheduler_env.sh /opt/dolphinscheduler/conf/env/
RUN dos2unix /root/checkpoint.sh && \
dos2unix /root/startup-init-conf.sh && \
dos2unix /root/startup.sh && \
dos2unix /opt/dolphinscheduler/conf/env/dolphinscheduler_env.sh && \
dos2unix /opt/dolphinscheduler/script/*.sh && \
dos2unix /opt/dolphinscheduler/bin/*.sh && \
rm -rf /bin/sh && \
ln -s /bin/bash /bin/sh && \
mkdir -p /tmp/xls /usr/lib/jvm && \
ln -sf /usr/local/openjdk-8 /usr/lib/jvm/java-1.8-openjdk && \
echo "Set disable_coredump false" >> /etc/sudo.conf
# 4. expose port
EXPOSE 5678 1234 12345 50051
ENTRYPOINT ["/usr/bin/tini", "--", "/root/startup.sh"]
``
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] chengshiwen edited a comment on issue #5155: A Linux distribution for docker-compose
Posted by GitBox <gi...@apache.org>.
chengshiwen edited a comment on issue #5155:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/5155#issuecomment-807980257
@Joder5 This is not a problem. As for a any bare linux, none of the libraries you mentioned will exist.
In order to improve the update speed, you can set the mirror source of alpine as follows:
```
# 1. install command/library/software
# If install slowly, you can replcae alpine's mirror with aliyun's mirror, Example:
# RUN sed -i "s/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g" /etc/apk/repositories
# RUN sed -i 's/dl-cdn.alpinelinux.org/mirror.tuna.tsinghua.edu.cn/g' /etc/apk/repositories
```
As for `pip`, you can also use the mirror source like `https://pypi.tuna.tsinghua.edu.cn/simple`
```
pip install --no-cache-dir -i https://pypi.tuna.tsinghua.edu.cn/simple pandas
```
As for `slim`, we will consider it later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] chengshiwen commented on issue #5155: A Linux distribution for docker-compose
Posted by GitBox <gi...@apache.org>.
chengshiwen commented on issue #5155:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/5155#issuecomment-812477564
close by #5158
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] chengshiwen closed issue #5155: A Linux distribution for docker-compose
Posted by GitBox <gi...@apache.org>.
chengshiwen closed issue #5155:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/5155
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org