[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #13939: [Feature][plugin] Datax can submit to YARN

["SbloodyS (via GitHub)" <gi...@apache.org>]

SbloodyS commented on code in PR #13939:
URL: https://github.com/apache/dolphinscheduler/pull/13939#discussion_r1169844402


##########
dolphinscheduler-task-plugin/dolphinscheduler-task-datax/src/main/java/org/apache/dolphinscheduler/plugin/task/datax/DataxTask.java:
##########
@@ -87,7 +88,16 @@ public class DataxTask extends AbstractTask {
      * todo: Create a shell script to execute the datax task, and read the python version from the env, so we can support multiple versions of datax python
      */
     private static final String DATAX_PYTHON = Optional.ofNullable(System.getenv("DATAX_PYTHON")).orElse("python2.7");
-
+    /**
+     * datax on yarn jar path
+     * https://github.com/duhanmin/datax-on-yarn
+     */
+    private static final String dataxOnYarnJar = System.getenv("DATAX_ON_YARN_JAR");
+    /**
+     * example: HADOOP_OPTS="-Xms32m -Xmx128m" /usr/bin/yarn
+     */
+    private static final String YARN_BIN =
+            StringUtils.isEmpty(System.getenv("YARN_BIN")) ? "yarn" : System.getenv("YARN_BIN");

Review Comment:
   It's better to put this to `dolphinscheduler_env.sh` and docs so that users can use it more easily.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org