You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2021/04/25 11:33:27 UTC
[GitHub] [dolphinscheduler] gaozhenfeng1 opened a new issue #5378: Use docker to test big data cluster
gaozhenfeng1 opened a new issue #5378:
URL: https://github.com/apache/dolphinscheduler/issues/5378
Now,I want to use docker to quickly deploy a dolphinscheduler ,then test whether to schedule my big data platform。
Do you modify this configuration file before starting the ds service, and then log in to the previous page and run the flow task to test the connectivity with the big data component?
/script/env/dolphinscheduler_env.sh
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] chengshiwen edited a comment on issue #5378: Use docker to test big data cluster
Posted by GitBox <gi...@apache.org>.
chengshiwen edited a comment on issue #5378:
URL: https://github.com/apache/dolphinscheduler/issues/5378#issuecomment-826325507
In 1.3.5 release, you need do additional things:
- Prepare the client binary of the big data platform, such as `hadoop-2.8.3`, `spark-2.4.3-bin-hadoop2.7` or `flink-1.11.3`. And put them into the directory like `/host/path/to/bigdata`
- Add volume `- /host/path/to/bigdata:/opt/soft/bigdata` for `dolphinscheduler-worker` in `docker/docker-swarm/docker-compose.yml`
- Modify the environment variables in `docker/docker-swarm/dolphinscheduler_env.sh`, such as `HADOOP_HOME`, `SPARK_HOME` or `FLINK_HOME`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] chengshiwen commented on issue #5378: Use docker to test big data cluster
Posted by GitBox <gi...@apache.org>.
chengshiwen commented on issue #5378:
URL: https://github.com/apache/dolphinscheduler/issues/5378#issuecomment-826325507
In 1.3.5 release, you need do additional things:
- Prepare the client binary of the big data platform, such as `hadoop-2.8.3`, `spark-2.4.3-bin-hadoop2.7` or `flink-1.11.3`. And put them in the directory like `/host/path/to/bigdata`
- Add volume `- /host/path/to/bigdata:/opt/soft/bigdata` for `dolphinscheduler-worker` in `docker/docker-swarm/docker-compose.yml`
- Modify the environment variables in `docker/docker-swarm/dolphinscheduler_env.sh`, such as `HADOOP_HOME`, `SPARK_HOME` or `FLINK_HOME`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] chengshiwen edited a comment on issue #5378: Use docker to test big data cluster
Posted by GitBox <gi...@apache.org>.
chengshiwen edited a comment on issue #5378:
URL: https://github.com/apache/dolphinscheduler/issues/5378#issuecomment-826325507
In 1.3.5 release, you need do additional things:
- Prepare the client binary of the big data platform, such as `hadoop-2.8.3`, `spark-2.4.3-bin-hadoop2.7` or `flink-1.11.3`. And put them into the directory like `/host/path/to/bigdata`
- Add volume `- /host/path/to/bigdata:/opt/soft/bigdata` for `dolphinscheduler-worker` in `docker/docker-swarm/docker-compose.yml`
- Modify the environment variables in `docker/docker-swarm/dolphinscheduler_env.sh`, such as `HADOOP_HOME`, `SPARK_HOME` or `FLINK_HOME`. For example, modify `HADOOP_HOME` to `/opt/soft/bigdata/hadoop-2.8.3` and `HADOOP_CONF_DIR` to `/opt/soft/bigdata/hadoop-2.8.3/etc/hadoop`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #5378: Use docker to test big data cluster
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #5378:
URL: https://github.com/apache/dolphinscheduler/issues/5378#issuecomment-826308992
Hi:
* Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can subscribe to the developer's email,Mail subscription steps reference https://dolphinscheduler.apache.org/zh-cn/community/development/subscribe.html ,Then write the issue URL in the email content and send question to dev@dolphinscheduler.apache.org.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #5378: Use docker to test big data cluster
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #5378:
URL: https://github.com/apache/dolphinscheduler/issues/5378#issuecomment-826308992
Hi:
* Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can subscribe to the developer's email,Mail subscription steps reference https://dolphinscheduler.apache.org/zh-cn/community/development/subscribe.html ,Then write the issue URL in the email content and send question to dev@dolphinscheduler.apache.org.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] chengshiwen commented on issue #5378: Use docker to test big data cluster
Posted by GitBox <gi...@apache.org>.
chengshiwen commented on issue #5378:
URL: https://github.com/apache/dolphinscheduler/issues/5378#issuecomment-826326182
In next 1.3.6 release, you can refer to the readme https://github.com/apache/dolphinscheduler/blob/3f47bd8425/docker/build/README.md#how-to-support-hadoop-spark-flink-hive-or-datax. The essential principles of these two methods are the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] gaozhenfeng1 commented on issue #5378: Use docker to test big data cluster
Posted by GitBox <gi...@apache.org>.
gaozhenfeng1 commented on issue #5378:
URL: https://github.com/apache/dolphinscheduler/issues/5378#issuecomment-828119714
Ok thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] chengshiwen commented on issue #5378: Use docker to test big data cluster
Posted by GitBox <gi...@apache.org>.
chengshiwen commented on issue #5378:
URL: https://github.com/apache/dolphinscheduler/issues/5378#issuecomment-826325507
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] chengshiwen edited a comment on issue #5378: Use docker to test big data cluster
Posted by GitBox <gi...@apache.org>.
chengshiwen edited a comment on issue #5378:
URL: https://github.com/apache/dolphinscheduler/issues/5378#issuecomment-826325507
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] gaozhenfeng1 closed issue #5378: Use docker to test big data cluster
Posted by GitBox <gi...@apache.org>.
gaozhenfeng1 closed issue #5378:
URL: https://github.com/apache/dolphinscheduler/issues/5378
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org