[GitHub] [samza] abhishekshivanna commented on a change in pull request #1182: SAMZA-2343: Remove original split deployment code

[GitBox <gi...@apache.org>]

abhishekshivanna commented on a change in pull request #1182: SAMZA-2343: Remove original split deployment code
URL: https://github.com/apache/samza/pull/1182#discussion_r333801611
 
 

 ##########
 File path: samza-shell/src/main/bash/run-class.sh
 ##########
 @@ -42,44 +42,16 @@ GC_LOG_ROTATION_OPTS="-XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GC
 DEFAULT_LOG4J_FILE=$base_dir/lib/log4j.xml
 DEFAULT_LOG4J2_FILE=$base_dir/lib/log4j2.xml
 BASE_LIB_DIR="$base_dir/lib"
-# JOB_LIB_DIR will be set for yarn container in ContainerUtil.java
-# for others we set it to home_dir/lib
-JOB_LIB_DIR="${JOB_LIB_DIR:-$base_dir/lib}"
 
-export JOB_LIB_DIR=$JOB_LIB_DIR
-
-echo JOB_LIB_DIR=$JOB_LIB_DIR
 echo BASE_LIB_DIR=$BASE_LIB_DIR
+
 CLASSPATH=""
-if [ -d "$JOB_LIB_DIR" ] && [ "$JOB_LIB_DIR" != "$BASE_LIB_DIR" ]; then
-  # build a common classpath
-  # this class path will contain all the jars from the framework and the job's libs.
-  # in case of different version of the same lib - we pick the highest
-
-  #all jars from the fwk
-  base_jars=`ls $BASE_LIB_DIR/*.[jw]ar`
-  #all jars from the job
-  job_jars=`for file in $JOB_LIB_DIR/*.[jw]ar; do name=\`basename $file\`; if [[ $base_jars != *"$name"* ]]; then echo "$file"; fi; done`
-  # get all lib jars and reverse sort it by versions
-  all_jars=`for file in $base_jars $job_jars; do echo \`basename $file|sed 's/.*[-]\([0-9]\+\..*\)[jw]ar$/\1/'\` $file; done|sort -t. -k 1,1nr -k 2,2nr -k 3,3nr -k 4,4nr|awk '{print $2}'`
-  # generate the class path based on the sorted result, all the jars need to be appended on newlines
-  # to ensure java argument length of 72 bytes is not violated
-  for jar in $all_jars; do CLASSPATH=$CLASSPATH" $jar \n"; done
-
-  # for debug only
-  echo base_jars=$base_jars
-  echo job_jars=$job_jars
-  echo all_jars=$all_jars
-  echo generated combined CLASSPATH=$CLASSPATH
-else
-  # default behavior, all the jars need to be appended on newlines
-  # to ensure line argument length of 72 bytes is not violated
-  for file in $BASE_LIB_DIR/*.[jw]ar;
-  do
-    CLASSPATH=$CLASSPATH" $file \n"
-  done
-  echo generated from BASE_LIB_DIR CLASSPATH=$CLASSPATH
-fi
+# all the jars need to be appended on newlines to ensure line argument length of 72 bytes is not violated
+for file in $BASE_LIB_DIR/*.[jw]ar;
+do
+  CLASSPATH=$CLASSPATH" $file \n"
+done
+echo generated from BASE_LIB_DIR CLASSPATH=$CLASSPATH
 
 Review comment:
   Minor:
   I'm assuming the reverse sorting by version number is done so that if conflicting jar versions are somehow included in the build - they are loaded in a deterministic order ?
   I think that changing this behavior may prevent these builds (with conflicts) from deploying deterministically.
   Although, I'm not sure if its the Samza framework's responsibility to address this..

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services