You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2022/10/19 21:32:26 UTC
[GitHub] [ozone] tanvipenumudy opened a new pull request, #3864: HDDS-7363. Changes to Ozone CLI to validate if the jars in classpath files are present on an install
tanvipenumudy opened a new pull request, #3864:
URL: https://github.com/apache/ozone/pull/3864
## What changes were proposed in this pull request?
To make changes to the Ozone CLI for validating if each of the jars as listed in the Ozone classpath files are present in the indicated location or not.
To introduce the following CLI parameters:
```
Usage I: ozone classpath <ARTIFACTNAME> --validate
Usage II: ozone --daemon start|status|stop csi|datanode|om|recon|s3g|scm --validate
Options:
-h, --help
This help text.
--validate
Validates if all jars as indicated in the corresponding OZONE_RUN_ARTIFACT_NAME classpath file are present.
```
```
Usage: ozone --daemon start|status|stop csi|datanode|om|recon|s3g|scm --validate --continue
Options:
-h, --help
This help text.
--continue
The command execution shall continue even if the validation fails.
```
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-7363
## How was this patch tested?
NA
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] tanvipenumudy commented on pull request #3864: HDDS-7363. Changes to Ozone CLI to validate if the jars in classpath files are present on an install
Posted by GitBox <gi...@apache.org>.
tanvipenumudy commented on PR #3864:
URL: https://github.com/apache/ozone/pull/3864#issuecomment-1294816166
Resolved the merge conflicts with the master branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on a diff in pull request #3864: HDDS-7363. Changes to Ozone CLI to validate if the jars in classpath files are present on an install
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on code in PR #3864:
URL: https://github.com/apache/ozone/pull/3864#discussion_r1004097200
##########
hadoop-ozone/dist/src/shell/ozone/ozone-functions.sh:
##########
@@ -2512,6 +2525,23 @@ function ozone_parse_args
ozone_exit_with_usage 1
fi
;;
+ --validate)
+ shift
+ OZONE_VALIDATE_CLASSPATH=true
+ if [[ "${1}" == "classpath" || "${1}" == "--daemon" ]]; then
+ ((OZONE_PARSE_COUNTER=OZONE_PARSE_COUNTER+1))
+ elif [[ "${1}" == "continue" && "${2}" == "--daemon" ]]; then
Review Comment:
However, the first part my comment still stands, `--daemon` and `--validate` are both options of `ozone`, could be in any order.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] tanvipenumudy commented on a diff in pull request #3864: HDDS-7363. Changes to Ozone CLI to validate if the jars in classpath files are present on an install
Posted by GitBox <gi...@apache.org>.
tanvipenumudy commented on code in PR #3864:
URL: https://github.com/apache/ozone/pull/3864#discussion_r1004143922
##########
hadoop-ozone/dist/src/shell/ozone/ozone-functions.sh:
##########
@@ -2512,6 +2525,23 @@ function ozone_parse_args
ozone_exit_with_usage 1
fi
;;
+ --validate)
+ shift
+ OZONE_VALIDATE_CLASSPATH=true
+ if [[ "${1}" == "classpath" || "${1}" == "--daemon" ]]; then
+ ((OZONE_PARSE_COUNTER=OZONE_PARSE_COUNTER+1))
+ elif [[ "${1}" == "continue" && "${2}" == "--daemon" ]]; then
Review Comment:
Sure @adoroszlai, will be making the changes to process the combination of options (`--validate` and `--daemon`) post-completion of the loop in any order, thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] tanvipenumudy commented on a diff in pull request #3864: HDDS-7363. Changes to Ozone CLI to validate if the jars in classpath files are present on an install
Posted by GitBox <gi...@apache.org>.
tanvipenumudy commented on code in PR #3864:
URL: https://github.com/apache/ozone/pull/3864#discussion_r1002617333
##########
hadoop-ozone/dist/src/shell/ozone/ozone:
##########
@@ -94,6 +94,7 @@ function ozonecmd_case
classpath)
if [[ "$#" -gt 0 ]]; then
OZONE_RUN_ARTIFACT_NAME="$1"
+ OZONE_VALIDATE_CLASSPATH="true"
Review Comment:
Set `OZONE_VALIDATE_CLASSPATH` (=`true`) under `ozone_parse_opts` only if the `--validate` flag is passed, thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] tanvipenumudy commented on a diff in pull request #3864: HDDS-7363. Changes to Ozone CLI to validate if the jars in classpath files are present on an install
Posted by GitBox <gi...@apache.org>.
tanvipenumudy commented on code in PR #3864:
URL: https://github.com/apache/ozone/pull/3864#discussion_r1001399442
##########
hadoop-ozone/dist/src/shell/ozone/ozone-functions.sh:
##########
@@ -2661,6 +2662,110 @@ function ozone_generic_java_subcmd_handler
fi
}
+## @description Validates if all jars as indicated in the corresponding ${OZONE_RUN_ARTIFACT_NAME} classpath file are present
+## @audience private
+## @stability evolving
+## @replaceable yes
+function validate_classpath
+{
+ if [ -n "$1" ]; then
+ case "$1" in
+ -h|--help)
+ echo "Usage I: ozone classpath <ARTIFACTNAME> --validate"
+ echo "Usage II: ozone --daemon start|status|stop csi|datanode|om|recon|s3g|scm --validate"
+ echo
+ echo "Options:"
+ echo
+ echo " -h, --help"
+ echo " This help text."
+ echo
+ echo " --validate"
+ echo " Validates if all jars as indicated in the corresponding OZONE_RUN_ARTIFACT_NAME classpath file are present."
+ echo
+ exit 0
+ ;;
+ --validate)
+ validate_classpath_util "$2"
+ ;;
+ *)
+ echo "Invalid option "$1". Use --help to see the valid options."
+ exit 1
+ ;;
+ esac
+ fi
+}
+
+## @description Utility function of validate_classpath
+## @audience private
+## @stability evolving
+## @replaceable yes
+function validate_classpath_util
+{
+ local FLAG_FAIL
+ FLAG_FAIL=1
Review Comment:
Made the changes, thank you for the suggestion!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on pull request #3864: HDDS-7363. Changes to Ozone CLI to validate if the jars in classpath files are present on an install
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on PR #3864:
URL: https://github.com/apache/ozone/pull/3864#issuecomment-1290116048
Thanks @tanvipenumudy for updating the patch. Please also check test failure:
```
Cli.Classpath :: Test ozone classpath command
==============================================================================
Ignores HADOOP_CLASSPATH if OZONE_CLASSPATH is set | FAIL |
1 != 0
------------------------------------------------------------------------------
Picks up items from OZONE_CLASSPATH | FAIL |
1 != 0
------------------------------------------------------------------------------
Adds optional dir entries | FAIL |
1 != 0
------------------------------------------------------------------------------
Cli.Classpath :: Test ozone classpath command | FAIL |
3 tests, 0 passed, 3 failed
```
https://github.com/apache/ozone/actions/runs/3318472966/jobs/5482559714#step:5:1009
You can find output of these test cases in `log.html` in the _acceptance-unsecure_ artifact: https://github.com/apache/ozone/actions/runs/3318472966#artifacts
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on a diff in pull request #3864: HDDS-7363. Changes to Ozone CLI to validate if the jars in classpath files are present on an install
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on code in PR #3864:
URL: https://github.com/apache/ozone/pull/3864#discussion_r1004096386
##########
hadoop-ozone/dist/src/shell/ozone/ozone-functions.sh:
##########
@@ -2512,6 +2525,23 @@ function ozone_parse_args
ozone_exit_with_usage 1
fi
;;
+ --validate)
+ shift
+ OZONE_VALIDATE_CLASSPATH=true
+ if [[ "${1}" == "classpath" || "${1}" == "--daemon" ]]; then
+ ((OZONE_PARSE_COUNTER=OZONE_PARSE_COUNTER+1))
+ elif [[ "${1}" == "continue" && "${2}" == "--daemon" ]]; then
Review Comment:
Oh, you are right, since `--validate` is listed as an option of the `ozone` command, not the `classpath` subcommand, we can keep it like this. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] tanvipenumudy commented on a diff in pull request #3864: HDDS-7363. Changes to Ozone CLI to validate if the jars in classpath files are present on an install
Posted by GitBox <gi...@apache.org>.
tanvipenumudy commented on code in PR #3864:
URL: https://github.com/apache/ozone/pull/3864#discussion_r1002617174
##########
hadoop-ozone/dist/src/shell/ozone/ozone-functions.sh:
##########
@@ -2661,6 +2662,110 @@ function ozone_generic_java_subcmd_handler
fi
}
+## @description Validates if all jars as indicated in the corresponding ${OZONE_RUN_ARTIFACT_NAME} classpath file are present
+## @audience private
+## @stability evolving
+## @replaceable yes
+function validate_classpath
+{
+ if [ -n "$1" ]; then
+ case "$1" in
+ -h|--help)
+ echo "Usage I: ozone classpath <ARTIFACTNAME> --validate"
+ echo "Usage II: ozone --daemon start|status|stop csi|datanode|om|recon|s3g|scm --validate"
+ echo
+ echo "Options:"
+ echo
+ echo " -h, --help"
+ echo " This help text."
+ echo
+ echo " --validate"
+ echo " Validates if all jars as indicated in the corresponding OZONE_RUN_ARTIFACT_NAME classpath file are present."
+ echo
+ exit 0
+ ;;
+ --validate)
+ validate_classpath_util "$2"
+ ;;
+ *)
+ echo "Invalid option "$1". Use --help to see the valid options."
+ exit 1
+ ;;
+ esac
+ fi
+}
Review Comment:
Thank you @adoroszlai for the review, incorporated the suggested changes!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on a diff in pull request #3864: HDDS-7363. Changes to Ozone CLI to validate if the jars in classpath files are present on an install
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on code in PR #3864:
URL: https://github.com/apache/ozone/pull/3864#discussion_r1006117169
##########
hadoop-ozone/dist/src/shell/ozone/ozone-functions.sh:
##########
@@ -42,6 +42,38 @@ function ozone_debug
fi
}
+## @description Displays usage text for the '--validate' option
+## @audience private
+## @stability evolving
+## @replaceable yes
+function ozone_validate_classpath_usage
+{
+ description=$'The --validate flag validates if all jars as indicated in the corresponding OZONE_RUN_ARTIFACT_NAME classpath file are present\n\n'
+ usage_text=$'Usage I: ozone --validate classpath <ARTIFACTNAME>\nUsage II: ozone --validate [OPTIONS] --daemon start|status|stop csi|datanode|om|recon|s3g|scm\n\n'
+ options=$' OPTIONS is none or any of:\n\ncontinue\tcommand execution shall continue even if validation fails'
+ ozone_error "${description}${usage_text}${options}"
+ exit 1
+}
+
+## @description Validates if all jars as indicated in the corresponding OZONE_RUN_ARTIFACT_NAME classpath file are present
+## @audience private
+## @stability evolving
+## @replaceable yes
+function ozone_validate_classpath
+{
+ local OZONE_OPTION_DAEMON
+ [[ "${OZONE_SUBCMD_SUPPORTDAEMONIZATION}" == true && "${OZONE_DAEMON_MODE}" =~ ^st(art|op|atus)$ ]] &&
+ OZONE_OPTION_DAEMON=true || OZONE_OPTION_DAEMON=false
+
+ if [[ "${OZONE_VALIDATE_CLASSPATH}" == true && ( ( "${OZONE_VALIDATE_FAIL_ON_MISSING_JARS}" == true &&
+ ( "${OZONE_OPTION_DAEMON}" == true || "${OZONE_SUBCMD}" == classpath ) ) ||
+ ( "${OZONE_VALIDATE_FAIL_ON_MISSING_JARS}" == false && "${OZONE_OPTION_DAEMON}" == true ) ) ]]; then
+ ozone_validate_classpath_util
+ else
+ ozone_validate_classpath_usage
+ fi
Review Comment:
Usage is printed if `--validate` is not used, preventing normal daemon startup (causing acceptance test failure across the board). The function should do nothing if `"${OZONE_VALIDATE_CLASSPATH}" != true`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] tanvipenumudy commented on a diff in pull request #3864: HDDS-7363. Changes to Ozone CLI to validate if the jars in classpath files are present on an install
Posted by GitBox <gi...@apache.org>.
tanvipenumudy commented on code in PR #3864:
URL: https://github.com/apache/ozone/pull/3864#discussion_r1004040931
##########
hadoop-ozone/dist/src/shell/ozone/ozone-functions.sh:
##########
@@ -2512,6 +2525,23 @@ function ozone_parse_args
ozone_exit_with_usage 1
fi
;;
+ --validate)
+ shift
+ OZONE_VALIDATE_CLASSPATH=true
+ if [[ "${1}" == "classpath" || "${1}" == "--daemon" ]]; then
+ ((OZONE_PARSE_COUNTER=OZONE_PARSE_COUNTER+1))
+ elif [[ "${1}" == "continue" && "${2}" == "--daemon" ]]; then
Review Comment:
Thank you for reviewing @adoroszlai, noted these points.
Regarding the ordering of the options - just wanted to check if we should be going by the following usage convention or not:
```
Usage: ozone [OPTIONS] SUBCOMMAND [SUBCOMMAND OPTIONS]
OPTIONS is none or any of:
--buildpaths attempt to add class files from build tree
--config dir Ozone config directory
--daemon (start|status|stop) operate on a daemon
--debug turn on shell script debug mode
--help usage information
--hostnames list[,of,host,names] hosts to use in worker mode
--hosts filename list of hosts to use in worker mode
--jvmargs arguments append JVM options to any existing options defined in the OZONE_OPTS environment variable. Any defined in OZONE_CLIENT_OPTS will be append after these jvmargs
--loglevel level set the log4j level for this command
--validate (continue) validates if all jars as indicated in the corresponding OZONE_RUN_ARTIFACT_NAME classpath file are present
--workers turn on worker mode
SUBCOMMAND is one of:
Admin Commands:
daemonlog get/set the log level for each daemon
jmxget get JMX exported values from NameNode or DataNode.
Client Commands:
admin Ozone admin tool
auditparser runs audit parser tool
classpath prints the class path needed for running ozone commands
debug Ozone debug tool
dtutil operations related to delegation tokens
envvars display computed Hadoop environment variables
freon runs an ozone data generator
fs run a filesystem command on Ozone file system. Equivalent to 'hadoop fs'
genconf generate minimally required ozone configs and output to ozone-site.xml in specified path
getconf get ozone config values from configuration
insight tool to get runtime operation information
s3 command line interface for s3 related operations
sh command line interface for object store operations
tenant command line interface for multi-tenant related operations
version print the version
Daemon Commands:
csi run the standalone CSI daemon
datanode run a HDDS datanode
om Ozone Manager
recon run the Recon service
s3g run the S3 compatible REST gateway
scm run the Storage Container Manager service
SUBCOMMAND may print help when invoked w/o parameters or with -h.
```
Usage: `ozone [OPTIONS] SUBCOMMAND [SUBCOMMAND OPTIONS]`
- `[OPTIONS]` = `--validate`
- `SUBCOMMAND` = `classpath`
- `[SUBCOMMAND OPTIONS]` `ozone-manager`
Command: `ozone --validate classpath ozone-manager`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] tanvipenumudy commented on a diff in pull request #3864: HDDS-7363. Changes to Ozone CLI to validate if the jars in classpath files are present on an install
Posted by GitBox <gi...@apache.org>.
tanvipenumudy commented on code in PR #3864:
URL: https://github.com/apache/ozone/pull/3864#discussion_r1004031885
##########
hadoop-ozone/dist/src/shell/ozone/ozone:
##########
@@ -275,6 +276,10 @@ else
ozonecmd_case "${OZONE_SUBCMD}" "${OZONE_SUBCMD_ARGS[@]}"
fi
+if [[ "${OZONE_SUBCMD_SUPPORTDAEMONIZATION}" == true && "${OZONE_DAEMON_MODE}" =~ ^st(art|op|atus)$ ]] || [[ "${OZONE_SUBCMD}" == classpath ]]; then
+ ozone_validate_classpath
+fi
Review Comment:
Thank you, defined the conditions for validation at a single place!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on a diff in pull request #3864: HDDS-7363. Changes to Ozone CLI to validate if the jars in classpath files are present on an install
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on code in PR #3864:
URL: https://github.com/apache/ozone/pull/3864#discussion_r1000778732
##########
hadoop-ozone/dist/src/shell/ozone/ozone-functions.sh:
##########
@@ -2661,6 +2662,110 @@ function ozone_generic_java_subcmd_handler
fi
}
+## @description Validates if all jars as indicated in the corresponding ${OZONE_RUN_ARTIFACT_NAME} classpath file are present
+## @audience private
+## @stability evolving
+## @replaceable yes
+function validate_classpath
+{
+ if [ -n "$1" ]; then
+ case "$1" in
+ -h|--help)
+ echo "Usage I: ozone classpath <ARTIFACTNAME> --validate"
+ echo "Usage II: ozone --daemon start|status|stop csi|datanode|om|recon|s3g|scm --validate"
+ echo
+ echo "Options:"
+ echo
+ echo " -h, --help"
+ echo " This help text."
+ echo
+ echo " --validate"
+ echo " Validates if all jars as indicated in the corresponding OZONE_RUN_ARTIFACT_NAME classpath file are present."
+ echo
+ exit 0
+ ;;
+ --validate)
+ validate_classpath_util "$2"
+ ;;
+ *)
+ echo "Invalid option "$1". Use --help to see the valid options."
+ exit 1
+ ;;
+ esac
+ fi
+}
Review Comment:
This way of parsing arguments is very fragile. Please add the new option in `ozone_parse_args` instead.
##########
hadoop-ozone/dist/src/shell/ozone/ozone-functions.sh:
##########
@@ -2661,6 +2662,110 @@ function ozone_generic_java_subcmd_handler
fi
}
+## @description Validates if all jars as indicated in the corresponding ${OZONE_RUN_ARTIFACT_NAME} classpath file are present
+## @audience private
+## @stability evolving
+## @replaceable yes
+function validate_classpath
+{
+ if [ -n "$1" ]; then
+ case "$1" in
+ -h|--help)
+ echo "Usage I: ozone classpath <ARTIFACTNAME> --validate"
+ echo "Usage II: ozone --daemon start|status|stop csi|datanode|om|recon|s3g|scm --validate"
+ echo
+ echo "Options:"
+ echo
+ echo " -h, --help"
+ echo " This help text."
+ echo
+ echo " --validate"
+ echo " Validates if all jars as indicated in the corresponding OZONE_RUN_ARTIFACT_NAME classpath file are present."
+ echo
+ exit 0
+ ;;
+ --validate)
+ validate_classpath_util "$2"
+ ;;
+ *)
+ echo "Invalid option "$1". Use --help to see the valid options."
+ exit 1
+ ;;
+ esac
+ fi
+}
+
+## @description Utility function of validate_classpath
+## @audience private
+## @stability evolving
+## @replaceable yes
+function validate_classpath_util
+{
+ local FLAG_FAIL
+ FLAG_FAIL=1
+
+ if [ -n "$1" ]; then
+ if [ "${OZONE_SUBCMD_SUPPORTDAEMONIZATION}" == false ]; then
+ echo "unrecognized option"
+ exit 1
+ fi
+ case "$1" in
+ -h|--help)
+ echo "Usage: ozone --daemon start|status|stop csi|datanode|om|recon|s3g|scm --validate --continue"
+ echo
+ echo "Options:"
+ echo
+ echo " -h, --help"
+ echo " This help text."
+ echo
+ echo " --continue"
+ echo " The command execution shall continue even if the validation fails."
+ echo
+ exit 0
+ ;;
+ --continue)
+ echo "The command execution shall continue even if the validation fails."
+ FLAG_FAIL=0
+ ;;
+ *)
+ echo "Invalid option '$1'. Use --help to see the valid options."
+ exit 1
+ ;;
+ esac
+ fi
+
+ local CLASSPATH_FILE
+ CLASSPATH_FILE="${OZONE_HOME}/share/ozone/classpath/${OZONE_RUN_ARTIFACT_NAME}.classpath"
+ echo "Validating classpath file: $(realpath "$CLASSPATH_FILE")"
+ if [[ ! -e "$CLASSPATH_FILE" ]]; then
+ echo "ERROR: Classpath file descriptor $CLASSPATH_FILE is missing"
+ exit 1
+ fi
+
+ source "$CLASSPATH_FILE"
+ OIFS=$IFS
+ IFS=':'
+
+ local flag
+ flag=1
+ for jar in $classpath; do
+ if [[ ! -e "$jar" ]]; then
+ flag=0
+ echo "ERROR: Jar file $(realpath "$jar") is missing"
+ fi
+ done
+
+ IFS=$OIFS
+
+ if [[ "$flag" == 0 ]] && [[ "$FLAG_FAIL" == 0 ]]; then
+ echo "Validation FAILED due to missing jar files! Continuing command execution..."
+ elif [[ "$flag" == 0 ]] && [[ "$FLAG_FAIL" == 1 ]]; then
+ echo "Validation FAILED due to missing jar files!"
+ exit 1
+ else
+ echo "Validation SUCCESSFUL, all the required jars are present!"
Review Comment:
Nit:
```suggestion
echo "Validation SUCCESSFUL, all required jars are present!"
```
##########
hadoop-ozone/dist/src/shell/ozone/ozone-functions.sh:
##########
@@ -2661,6 +2662,110 @@ function ozone_generic_java_subcmd_handler
fi
}
+## @description Validates if all jars as indicated in the corresponding ${OZONE_RUN_ARTIFACT_NAME} classpath file are present
+## @audience private
+## @stability evolving
+## @replaceable yes
+function validate_classpath
+{
+ if [ -n "$1" ]; then
+ case "$1" in
+ -h|--help)
+ echo "Usage I: ozone classpath <ARTIFACTNAME> --validate"
+ echo "Usage II: ozone --daemon start|status|stop csi|datanode|om|recon|s3g|scm --validate"
+ echo
+ echo "Options:"
+ echo
+ echo " -h, --help"
+ echo " This help text."
+ echo
+ echo " --validate"
+ echo " Validates if all jars as indicated in the corresponding OZONE_RUN_ARTIFACT_NAME classpath file are present."
+ echo
+ exit 0
+ ;;
+ --validate)
+ validate_classpath_util "$2"
+ ;;
+ *)
+ echo "Invalid option "$1". Use --help to see the valid options."
+ exit 1
+ ;;
+ esac
+ fi
+}
+
+## @description Utility function of validate_classpath
+## @audience private
+## @stability evolving
+## @replaceable yes
+function validate_classpath_util
+{
+ local FLAG_FAIL
+ FLAG_FAIL=1
+
+ if [ -n "$1" ]; then
+ if [ "${OZONE_SUBCMD_SUPPORTDAEMONIZATION}" == false ]; then
+ echo "unrecognized option"
+ exit 1
+ fi
+ case "$1" in
+ -h|--help)
+ echo "Usage: ozone --daemon start|status|stop csi|datanode|om|recon|s3g|scm --validate --continue"
+ echo
+ echo "Options:"
+ echo
+ echo " -h, --help"
+ echo " This help text."
+ echo
+ echo " --continue"
+ echo " The command execution shall continue even if the validation fails."
+ echo
+ exit 0
+ ;;
+ --continue)
+ echo "The command execution shall continue even if the validation fails."
+ FLAG_FAIL=0
+ ;;
+ *)
+ echo "Invalid option '$1'. Use --help to see the valid options."
+ exit 1
+ ;;
+ esac
+ fi
+
+ local CLASSPATH_FILE
+ CLASSPATH_FILE="${OZONE_HOME}/share/ozone/classpath/${OZONE_RUN_ARTIFACT_NAME}.classpath"
+ echo "Validating classpath file: $(realpath "$CLASSPATH_FILE")"
+ if [[ ! -e "$CLASSPATH_FILE" ]]; then
+ echo "ERROR: Classpath file descriptor $CLASSPATH_FILE is missing"
+ exit 1
+ fi
+
+ source "$CLASSPATH_FILE"
+ OIFS=$IFS
+ IFS=':'
+
+ local flag
+ flag=1
Review Comment:
Please find a more descriptive name. Suggestion: rename to `found_missing_jars` and use "true"/"false" instead of magic numbers 0/1.
##########
hadoop-ozone/dist/src/shell/ozone/ozone-functions.sh:
##########
@@ -2661,6 +2662,110 @@ function ozone_generic_java_subcmd_handler
fi
}
+## @description Validates if all jars as indicated in the corresponding ${OZONE_RUN_ARTIFACT_NAME} classpath file are present
+## @audience private
+## @stability evolving
+## @replaceable yes
+function validate_classpath
+{
+ if [ -n "$1" ]; then
+ case "$1" in
+ -h|--help)
+ echo "Usage I: ozone classpath <ARTIFACTNAME> --validate"
+ echo "Usage II: ozone --daemon start|status|stop csi|datanode|om|recon|s3g|scm --validate"
+ echo
+ echo "Options:"
+ echo
+ echo " -h, --help"
+ echo " This help text."
+ echo
+ echo " --validate"
+ echo " Validates if all jars as indicated in the corresponding OZONE_RUN_ARTIFACT_NAME classpath file are present."
+ echo
+ exit 0
+ ;;
+ --validate)
+ validate_classpath_util "$2"
+ ;;
+ *)
+ echo "Invalid option "$1". Use --help to see the valid options."
+ exit 1
+ ;;
+ esac
+ fi
+}
+
+## @description Utility function of validate_classpath
+## @audience private
+## @stability evolving
+## @replaceable yes
+function validate_classpath_util
+{
+ local FLAG_FAIL
+ FLAG_FAIL=1
Review Comment:
Please rename to more descriptive, e.g. `fail_on_missing`, use `true/false`.
##########
hadoop-ozone/dist/src/shell/ozone/ozone:
##########
@@ -94,6 +94,7 @@ function ozonecmd_case
classpath)
if [[ "$#" -gt 0 ]]; then
OZONE_RUN_ARTIFACT_NAME="$1"
+ OZONE_VALIDATE_CLASSPATH="true"
Review Comment:
These should be set, by `ozone_parse_opts`, only if `--validate` is present.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] tanvipenumudy commented on a diff in pull request #3864: HDDS-7363. Changes to Ozone CLI to validate if the jars in classpath files are present on an install
Posted by GitBox <gi...@apache.org>.
tanvipenumudy commented on code in PR #3864:
URL: https://github.com/apache/ozone/pull/3864#discussion_r1005857235
##########
hadoop-ozone/dist/src/shell/ozone/ozone-functions.sh:
##########
@@ -2512,6 +2525,23 @@ function ozone_parse_args
ozone_exit_with_usage 1
fi
;;
+ --validate)
+ shift
+ OZONE_VALIDATE_CLASSPATH=true
+ if [[ "${1}" == "classpath" || "${1}" == "--daemon" ]]; then
+ ((OZONE_PARSE_COUNTER=OZONE_PARSE_COUNTER+1))
+ elif [[ "${1}" == "continue" && "${2}" == "--daemon" ]]; then
Review Comment:
Made changes to support ordering among the options: `ozone --validate [continue] --daemon start om` and `ozone --daemon start --validate [continue] om`, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] kerneltime merged pull request #3864: HDDS-7363. Changes to Ozone CLI to validate if the jars in classpath files are present on an install
Posted by GitBox <gi...@apache.org>.
kerneltime merged PR #3864:
URL: https://github.com/apache/ozone/pull/3864
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] adoroszlai commented on a diff in pull request #3864: HDDS-7363. Changes to Ozone CLI to validate if the jars in classpath files are present on an install
Posted by GitBox <gi...@apache.org>.
adoroszlai commented on code in PR #3864:
URL: https://github.com/apache/ozone/pull/3864#discussion_r1002989292
##########
hadoop-ozone/dist/src/shell/ozone/ozone-functions.sh:
##########
@@ -2512,6 +2525,23 @@ function ozone_parse_args
ozone_exit_with_usage 1
fi
;;
+ --validate)
+ shift
+ OZONE_VALIDATE_CLASSPATH=true
+ if [[ "${1}" == "classpath" || "${1}" == "--daemon" ]]; then
+ ((OZONE_PARSE_COUNTER=OZONE_PARSE_COUNTER+1))
+ elif [[ "${1}" == "continue" && "${2}" == "--daemon" ]]; then
Review Comment:
We should be able to provide options in any order. Please process combination of options (`--validate` and `--daemon`) after the loop has completed.
Also, currently only the following (somewhat counterintuitive order) is supported:
```
$ ozone --validate classpath ozone-manager
Validating classpath file: hadoop-ozone/dist/target/ozone-1.3.0-SNAPSHOT/share/ozone/classpath/ozone-manager.classpath
...
```
The following commands should also be supported, but are currently failing:
```
$ ozone classpath ozone-manager --validate
unrecognized option
[main] INFO util.ExitUtil: Exiting with status 1: unrecognized option
$ ozone classpath --validate ozone-manager
ERROR: Classpath file descriptor hadoop-ozone/dist/target/ozone-1.3.0-SNAPSHOT/share/ozone/classpath/--validate.classpath is missing
```
##########
hadoop-ozone/dist/src/shell/ozone/ozone:
##########
@@ -275,6 +276,10 @@ else
ozonecmd_case "${OZONE_SUBCMD}" "${OZONE_SUBCMD_ARGS[@]}"
fi
+if [[ "${OZONE_SUBCMD_SUPPORTDAEMONIZATION}" == true && "${OZONE_DAEMON_MODE}" =~ ^st(art|op|atus)$ ]] || [[ "${OZONE_SUBCMD}" == classpath ]]; then
+ ozone_validate_classpath
+fi
Review Comment:
Currently we have conditions for validation in two places: here and in `ozone_validate_classpath`. I'd prefer if all conditions were in the same place (probably best in `ozone_validate_classpath` to simplify the `ozone` executable a bit).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] tanvipenumudy commented on pull request #3864: HDDS-7363. Changes to Ozone CLI to validate if the jars in classpath files are present on an install
Posted by GitBox <gi...@apache.org>.
tanvipenumudy commented on PR #3864:
URL: https://github.com/apache/ozone/pull/3864#issuecomment-1286660892
Sure Attila, filed a follow-up task (to introduce Robot acceptance tests): https://issues.apache.org/jira/browse/HDDS-7373, thank you for the suggestion!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] tanvipenumudy commented on a diff in pull request #3864: HDDS-7363. Changes to Ozone CLI to validate if the jars in classpath files are present on an install
Posted by GitBox <gi...@apache.org>.
tanvipenumudy commented on code in PR #3864:
URL: https://github.com/apache/ozone/pull/3864#discussion_r1001399251
##########
hadoop-ozone/dist/src/shell/ozone/ozone-functions.sh:
##########
@@ -2661,6 +2662,110 @@ function ozone_generic_java_subcmd_handler
fi
}
+## @description Validates if all jars as indicated in the corresponding ${OZONE_RUN_ARTIFACT_NAME} classpath file are present
+## @audience private
+## @stability evolving
+## @replaceable yes
+function validate_classpath
+{
+ if [ -n "$1" ]; then
+ case "$1" in
+ -h|--help)
+ echo "Usage I: ozone classpath <ARTIFACTNAME> --validate"
+ echo "Usage II: ozone --daemon start|status|stop csi|datanode|om|recon|s3g|scm --validate"
+ echo
+ echo "Options:"
+ echo
+ echo " -h, --help"
+ echo " This help text."
+ echo
+ echo " --validate"
+ echo " Validates if all jars as indicated in the corresponding OZONE_RUN_ARTIFACT_NAME classpath file are present."
+ echo
+ exit 0
+ ;;
+ --validate)
+ validate_classpath_util "$2"
+ ;;
+ *)
+ echo "Invalid option "$1". Use --help to see the valid options."
+ exit 1
+ ;;
+ esac
+ fi
+}
+
+## @description Utility function of validate_classpath
+## @audience private
+## @stability evolving
+## @replaceable yes
+function validate_classpath_util
+{
+ local FLAG_FAIL
+ FLAG_FAIL=1
+
+ if [ -n "$1" ]; then
+ if [ "${OZONE_SUBCMD_SUPPORTDAEMONIZATION}" == false ]; then
+ echo "unrecognized option"
+ exit 1
+ fi
+ case "$1" in
+ -h|--help)
+ echo "Usage: ozone --daemon start|status|stop csi|datanode|om|recon|s3g|scm --validate --continue"
+ echo
+ echo "Options:"
+ echo
+ echo " -h, --help"
+ echo " This help text."
+ echo
+ echo " --continue"
+ echo " The command execution shall continue even if the validation fails."
+ echo
+ exit 0
+ ;;
+ --continue)
+ echo "The command execution shall continue even if the validation fails."
+ FLAG_FAIL=0
+ ;;
+ *)
+ echo "Invalid option '$1'. Use --help to see the valid options."
+ exit 1
+ ;;
+ esac
+ fi
+
+ local CLASSPATH_FILE
+ CLASSPATH_FILE="${OZONE_HOME}/share/ozone/classpath/${OZONE_RUN_ARTIFACT_NAME}.classpath"
+ echo "Validating classpath file: $(realpath "$CLASSPATH_FILE")"
+ if [[ ! -e "$CLASSPATH_FILE" ]]; then
+ echo "ERROR: Classpath file descriptor $CLASSPATH_FILE is missing"
+ exit 1
+ fi
+
+ source "$CLASSPATH_FILE"
+ OIFS=$IFS
+ IFS=':'
+
+ local flag
+ flag=1
Review Comment:
Made the changes, thank you for the suggestion!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
[GitHub] [ozone] tanvipenumudy commented on a diff in pull request #3864: HDDS-7363. Changes to Ozone CLI to validate if the jars in classpath files are present on an install
Posted by GitBox <gi...@apache.org>.
tanvipenumudy commented on code in PR #3864:
URL: https://github.com/apache/ozone/pull/3864#discussion_r1006143925
##########
hadoop-ozone/dist/src/shell/ozone/ozone-functions.sh:
##########
@@ -42,6 +42,38 @@ function ozone_debug
fi
}
+## @description Displays usage text for the '--validate' option
+## @audience private
+## @stability evolving
+## @replaceable yes
+function ozone_validate_classpath_usage
+{
+ description=$'The --validate flag validates if all jars as indicated in the corresponding OZONE_RUN_ARTIFACT_NAME classpath file are present\n\n'
+ usage_text=$'Usage I: ozone --validate classpath <ARTIFACTNAME>\nUsage II: ozone --validate [OPTIONS] --daemon start|status|stop csi|datanode|om|recon|s3g|scm\n\n'
+ options=$' OPTIONS is none or any of:\n\ncontinue\tcommand execution shall continue even if validation fails'
+ ozone_error "${description}${usage_text}${options}"
+ exit 1
+}
+
+## @description Validates if all jars as indicated in the corresponding OZONE_RUN_ARTIFACT_NAME classpath file are present
+## @audience private
+## @stability evolving
+## @replaceable yes
+function ozone_validate_classpath
+{
+ local OZONE_OPTION_DAEMON
+ [[ "${OZONE_SUBCMD_SUPPORTDAEMONIZATION}" == true && "${OZONE_DAEMON_MODE}" =~ ^st(art|op|atus)$ ]] &&
+ OZONE_OPTION_DAEMON=true || OZONE_OPTION_DAEMON=false
+
+ if [[ "${OZONE_VALIDATE_CLASSPATH}" == true && ( ( "${OZONE_VALIDATE_FAIL_ON_MISSING_JARS}" == true &&
+ ( "${OZONE_OPTION_DAEMON}" == true || "${OZONE_SUBCMD}" == classpath ) ) ||
+ ( "${OZONE_VALIDATE_FAIL_ON_MISSING_JARS}" == false && "${OZONE_OPTION_DAEMON}" == true ) ) ]]; then
+ ozone_validate_classpath_util
+ else
+ ozone_validate_classpath_usage
+ fi
Review Comment:
Thank you, made the required changes!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org