You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by "duhanmin (via GitHub)" <gi...@apache.org> on 2023/04/17 07:54:32 UTC
[GitHub] [dolphinscheduler] duhanmin opened a new pull request, #13937: [Feature-11488]Datax can submit to YARN
duhanmin opened a new pull request, #13937:
URL: https://github.com/apache/dolphinscheduler/pull/13937
https://github.com/apache/dolphinscheduler/issues/11488
Instructions:
Configure the environment variables DATAX_ON_YARN_JAR and YARN_BIN to run DATAX ON YARN
Parameter explanation:
DATAX_ON_YARN_JAR:
https://github.com/duhanmin/datax-on-yarn Compile and package
YARN_BIN:
The default value is yarn, which is used to limit the memory of yarn submission to the client
DATAX_HOME:
datax decompression directory, or s3/hdfs compression package directory
```shell
DATAX_HOME=/Users/duhanmin/IdeaProjects/daima/DataX/target/datax/datax
YARN_BIN=HADOOP_OPTS="-Xms32m -Xmx128m" /usr/bin/yarn
DATAX_ON_YARN_JAR=/opt/soft/datax/datax-on-yarn-1.0.0.jar
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] duhanmin closed pull request #13937: [Feature-11488][Datax]Datax can submit to YARN
Posted by "duhanmin (via GitHub)" <gi...@apache.org>.
duhanmin closed pull request #13937: [Feature-11488][Datax]Datax can submit to YARN
URL: https://github.com/apache/dolphinscheduler/pull/13937
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] duhanmin closed pull request #13937: [Feature-11488]Datax can submit to YARN
Posted by "duhanmin (via GitHub)" <gi...@apache.org>.
duhanmin closed pull request #13937: [Feature-11488]Datax can submit to YARN
URL: https://github.com/apache/dolphinscheduler/pull/13937
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] duhanmin closed pull request #13937: [Feature-11488]Datax can submit to YARN
Posted by "duhanmin (via GitHub)" <gi...@apache.org>.
duhanmin closed pull request #13937: [Feature-11488]Datax can submit to YARN
URL: https://github.com/apache/dolphinscheduler/pull/13937
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org