You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/04/06 14:31:00 UTC
[GitHub] [hudi] xushiyan opened a new pull request, #5240: [HUDI-3340] Fix deploy_staging_jars for different profiles
xushiyan opened a new pull request, #5240:
URL: https://github.com/apache/hudi/pull/5240
Make `scripts/release/deploy_staging_jars.sh` to handle different build profiles for scala, spark and flink.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5240: [HUDI-3340] Fix deploy_staging_jars for different profiles
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5240:
URL: https://github.com/apache/hudi/pull/5240#issuecomment-1090408080
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16b77c3f30bda08dc368e312f979c1181833c341",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7867",
"triggerID" : "16b77c3f30bda08dc368e312f979c1181833c341",
"triggerType" : "PUSH"
}, {
"hash" : "9863f757368ac58a0e19ee66fd269d9402e5ce09",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9863f757368ac58a0e19ee66fd269d9402e5ce09",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 16b77c3f30bda08dc368e312f979c1181833c341 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7867)
* 9863f757368ac58a0e19ee66fd269d9402e5ce09 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5240: [HUDI-3340] Fix deploy_staging_jars for different profiles
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5240:
URL: https://github.com/apache/hudi/pull/5240#issuecomment-1090347994
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16b77c3f30bda08dc368e312f979c1181833c341",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7867",
"triggerID" : "16b77c3f30bda08dc368e312f979c1181833c341",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 16b77c3f30bda08dc368e312f979c1181833c341 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7867)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5240: [HUDI-3340] Fix deploy_staging_jars for different profiles
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5240:
URL: https://github.com/apache/hudi/pull/5240#issuecomment-1090455486
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16b77c3f30bda08dc368e312f979c1181833c341",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7867",
"triggerID" : "16b77c3f30bda08dc368e312f979c1181833c341",
"triggerType" : "PUSH"
}, {
"hash" : "9863f757368ac58a0e19ee66fd269d9402e5ce09",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9863f757368ac58a0e19ee66fd269d9402e5ce09",
"triggerType" : "PUSH"
}, {
"hash" : "6f719fef6826d71e0bc520e22e856df3a5e9d612",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7870",
"triggerID" : "6f719fef6826d71e0bc520e22e856df3a5e9d612",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9863f757368ac58a0e19ee66fd269d9402e5ce09 UNKNOWN
* 6f719fef6826d71e0bc520e22e856df3a5e9d612 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7870)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a diff in pull request #5240: [HUDI-3340] Fix deploy_staging_jars for different profiles
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on code in PR #5240:
URL: https://github.com/apache/hudi/pull/5240#discussion_r844027463
##########
scripts/release/deploy_staging_jars.sh:
##########
@@ -21,40 +21,28 @@
## Variables with defaults (if not overwritten by environment)
##
MVN=${MVN:-mvn}
-SPARK_VERSION=2
# fail immediately
set -o errexit
set -o nounset
# print command before executing
set -o xtrace
-CURR_DIR=`pwd`
-if [[ `basename $CURR_DIR` != "scripts" ]] ; then
- echo "You have to call the script from the scripts/ dir"
+CURR_DIR=$(pwd)
+if [ ! -d "$CURR_DIR/packaging" ] ; then
+ echo "You have to call the script from the repository root dir that contains 'packaging/'"
exit 1
fi
-if [[ $# -lt 1 ]]; then
- echo "This script will deploy artifacts to staging repositories"
- echo "There is one param required:"
- echo "--scala_version=\${SCALA_VERSION}"
- exit
-else
- for param in "$@"
- do
- if [[ $param =~ --scala_version\=(2\.1[1-2]) ]]; then
- SCALA_VERSION=${BASH_REMATCH[1]}
- elif [[ $param =~ --spark_version\=([2-3]) ]]; then
- SPARK_VERSION=${BASH_REMATCH[1]}
- fi
- done
-fi
-
-###########################
-
-cd ..
-
-echo "Deploying to repository.apache.org with scala version ${SCALA_VERSION}"
-
-COMMON_OPTIONS="-Dscala-${SCALA_VERSION} -Dspark${SPARK_VERSION} -Prelease -DskipTests -DretryFailedDeploymentCount=10 -DdeployArtifacts=true"
-$MVN clean deploy $COMMON_OPTIONS
+declare -a SUPPORTED_VERSION_OPTS=(
+"-Dscala-2.11 -Dspark2.4 -Dflink1.13"
+"-Dscala-2.12 -Dspark2.4 -Dflink1.13"
+"-Dscala-2.12 -Dspark3.1 -Dflink1.14"
+"-Dscala-2.12 -Dspark3.2 -Dflink1.14"
+)
+
+for v in "${SUPPORTED_VERSION_OPTS[@]}"
+do
Review Comment:
if for some reason, network issue or service unavailable, and if RM needs to retry, we can't go w/ for loop here. Can you de-couple that. or add an option to do everything. or give an option for user to specify on set of option.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5240: [HUDI-3340] Fix deploy_staging_jars for different profiles
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5240:
URL: https://github.com/apache/hudi/pull/5240#issuecomment-1090415418
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16b77c3f30bda08dc368e312f979c1181833c341",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7867",
"triggerID" : "16b77c3f30bda08dc368e312f979c1181833c341",
"triggerType" : "PUSH"
}, {
"hash" : "9863f757368ac58a0e19ee66fd269d9402e5ce09",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9863f757368ac58a0e19ee66fd269d9402e5ce09",
"triggerType" : "PUSH"
}, {
"hash" : "6f719fef6826d71e0bc520e22e856df3a5e9d612",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7870",
"triggerID" : "6f719fef6826d71e0bc520e22e856df3a5e9d612",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 16b77c3f30bda08dc368e312f979c1181833c341 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7867)
* 9863f757368ac58a0e19ee66fd269d9402e5ce09 UNKNOWN
* 6f719fef6826d71e0bc520e22e856df3a5e9d612 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7870)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5240: [HUDI-3340] Fix deploy_staging_jars for different profiles
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5240:
URL: https://github.com/apache/hudi/pull/5240#issuecomment-1090411688
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16b77c3f30bda08dc368e312f979c1181833c341",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7867",
"triggerID" : "16b77c3f30bda08dc368e312f979c1181833c341",
"triggerType" : "PUSH"
}, {
"hash" : "9863f757368ac58a0e19ee66fd269d9402e5ce09",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9863f757368ac58a0e19ee66fd269d9402e5ce09",
"triggerType" : "PUSH"
}, {
"hash" : "6f719fef6826d71e0bc520e22e856df3a5e9d612",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6f719fef6826d71e0bc520e22e856df3a5e9d612",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 16b77c3f30bda08dc368e312f979c1181833c341 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=7867)
* 9863f757368ac58a0e19ee66fd269d9402e5ce09 UNKNOWN
* 6f719fef6826d71e0bc520e22e856df3a5e9d612 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5240: [HUDI-3340] Fix deploy_staging_jars for different profiles
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5240:
URL: https://github.com/apache/hudi/pull/5240#issuecomment-1090344551
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "16b77c3f30bda08dc368e312f979c1181833c341",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "16b77c3f30bda08dc368e312f979c1181833c341",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 16b77c3f30bda08dc368e312f979c1181833c341 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan merged pull request #5240: [HUDI-3340] Fix deploy_staging_jars for different profiles
Posted by GitBox <gi...@apache.org>.
xushiyan merged PR #5240:
URL: https://github.com/apache/hudi/pull/5240
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org