[GitHub] [incubator-hugegraph-toolchain] javeme commented on a diff in pull request #318: feat: support yarn-cluster mode for spark-loader

[GitBox <gi...@apache.org>]

javeme commented on code in PR #318:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/318#discussion_r934106290


##########
hugegraph-loader/assembly/static/bin/hugegraph-spark-loader.sh:
##########
@@ -5,7 +5,7 @@ APP_DIR=$(dirname ${BIN_DIR})
 LIB_DIR=${APP_DIR}/lib
 
 # get hugegraph_params and engine_params
-source "$BIN_DIR"/get_params.sh
+source "$BIN_DIR"/get-params.sh

Review Comment:
   assemblyJarName -- ditto



##########
hugegraph-loader/assembly/static/bin/get-params.sh:
##########
@@ -5,20 +5,36 @@ function get_params() {
   hugegraph_params=""
   while (("$#")); do
     case "$1" in
-      -–file | --graph | --schema | --host | --port | --username | --token | --protocol | \
+      --graph | --schema | --host | --port | --username | --token | --protocol | \
       --trust-store-file | --trust-store-password | --clear-all-data | --clear-timeout | \
       --incremental-mode | --failure-mode | --batch-insert-threads | --single-insert-threads | \
       --max-conn | --max-conn-per-route | --batch-size | --max-parse-errors | --max-insert-errors | \
       --timeout | --shutdown-timeout | --retry-times | --retry-interval | --check-vertex | \
       --print-progress | --dry-run | --help)
-      hugegraph_params="$hugegraph_params $1 $2"
-      shift 2
-      ;;
-
-    *) # preserve positional arguments
-      engine_params="$engine_params $1"
-      shift
-      ;;
+        hugegraph_params="$hugegraph_params $1 $2"

Review Comment:
   can we rename to $hugegraph_params to $HUGEGRAPH_PARAMS, and the same for other variable names.



##########
hugegraph-loader/assembly/static/bin/get-params.sh:
##########
@@ -5,20 +5,36 @@ function get_params() {
   hugegraph_params=""
   while (("$#")); do
     case "$1" in
-      -–file | --graph | --schema | --host | --port | --username | --token | --protocol | \
+      --graph | --schema | --host | --port | --username | --token | --protocol | \
       --trust-store-file | --trust-store-password | --clear-all-data | --clear-timeout | \
       --incremental-mode | --failure-mode | --batch-insert-threads | --single-insert-threads | \
       --max-conn | --max-conn-per-route | --batch-size | --max-parse-errors | --max-insert-errors | \
       --timeout | --shutdown-timeout | --retry-times | --retry-interval | --check-vertex | \
       --print-progress | --dry-run | --help)
-      hugegraph_params="$hugegraph_params $1 $2"
-      shift 2
-      ;;
-
-    *) # preserve positional arguments
-      engine_params="$engine_params $1"
-      shift
-      ;;
+        hugegraph_params="$hugegraph_params $1 $2"
+        shift 2
+        ;;
+      --file)
+        file=$2
+        shift 2
+        ;;
+      --deploy-mode)
+        mode=$2
+        engine_params="$engine_params $1 $2"
+        shift 2
+        ;;
+      *) # preserve positional arguments
+        engine_params="$engine_params $1"
+        shift
+        ;;
     esac
   done
+
+  if [ $mode = 'cluster' ];then
+    hugegraph_params="$hugegraph_params --file ${file##*/}"
+    engine_params="$engine_params --files ${file}"
+  else
+    hugegraph_params="$hugegraph_params --file ${file}"
+  fi
+

Review Comment:
   can remove the useless line



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hugegraph.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org