You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by "shwstppr (via GitHub)" <gi...@apache.org> on 2026/09/02 06:53:44 UTC
[PR] engine-schema: add script to generate upgrade path chages [cloudstack]
shwstppr opened a new pull request, #14034:
URL: https://github.com/apache/cloudstack/pull/14034
### Description
Given from and to versions the script will generate the basic upgrade path changes with Java class and schema upgrade and cleanup files.
### Types of changes
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
- [ ] Build/CI
- [ ] Test (unit or integration test code)
### Feature/Enhancement Scale or Bug Severity
#### Feature/Enhancement Scale
- [ ] Major
- [ ] Minor
#### Bug Severity
- [ ] BLOCKER
- [ ] Critical
- [ ] Major
- [ ] Minor
- [ ] Trivial
### Screenshots (if appropriate):
### How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
#### How did you try to break this feature and the system with this change?
<!-- see how your change affects other areas of the code, etc. -->
<!-- Please read the [CONTRIBUTING](https://github.com/apache/cloudstack/blob/main/CONTRIBUTING.md) document -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] engine-schema: add script to generate upgrade path changes [cloudstack]
Posted by "DaanHoogland (via GitHub)" <gi...@apache.org>.
DaanHoogland commented on PR #14034:
URL: https://github.com/apache/cloudstack/pull/14034#issuecomment-5512945952
can we change the example to 4.23.0 -> 24.0.0?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] engine-schema: add script to generate upgrade path changes [cloudstack]
Posted by "shwstppr (via GitHub)" <gi...@apache.org>.
shwstppr closed pull request #14034: engine-schema: add script to generate upgrade path changes
URL: https://github.com/apache/cloudstack/pull/14034
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] engine-schema: add script to generate upgrade path chages [cloudstack]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #14034:
URL: https://github.com/apache/cloudstack/pull/14034#discussion_r3911859651
##########
engine/schema/create-upgrade-path.sh:
##########
@@ -0,0 +1,161 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Scaffolds a new database upgrade path, following the pattern used e.g. in
+# https://github.com/apache/cloudstack/pull/12048/files
+#
+# Creates:
+# - engine/schema/src/main/java/com/cloud/upgrade/dao/Upgrade<from>to<to>.java
+# - engine/schema/src/main/resources/META-INF/db/schema-<from>to<to>.sql
+# - engine/schema/src/main/resources/META-INF/db/schema-<from>to<to>-cleanup.sql
+# and wires the new class into DatabaseUpgradeChecker.java (import + .next() entry).
+#
+# Usage: engine/schema/create-upgrade-path.sh <fromVersion> <toVersion>
+# Example: engine/schema/create-upgrade-path.sh 4.23.0.0 4.24.0.0
+
+set -euo pipefail
+
+usage() {
+ echo "Usage: $0 <fromVersion> <toVersion>"
+ echo "Example: $0 4.23.0.0 4.24.0.0"
+ exit 1
+}
+
+[[ $# -eq 2 ]] || usage
+
+FROM_VERSION="$1"
+TO_VERSION="$2"
+
+VERSION_REGEX='^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$'
+[[ "$FROM_VERSION" =~ $VERSION_REGEX ]] || { echo "Invalid fromVersion '$FROM_VERSION' (expected x.y.z.w)"; exit 1; }
+[[ "$TO_VERSION" =~ $VERSION_REGEX ]] || { echo "Invalid toVersion '$TO_VERSION' (expected x.y.z.w)"; exit 1; }
+
+FROM_COMPACT="${FROM_VERSION//./}"
+TO_COMPACT="${TO_VERSION//./}"
+CLASS_NAME="Upgrade${FROM_COMPACT}to${TO_COMPACT}"
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
+DAO_DIR="$SCRIPT_DIR/src/main/java/com/cloud/upgrade/dao"
+SQL_DIR="$SCRIPT_DIR/src/main/resources/META-INF/db"
+CHECKER_FILE="$SCRIPT_DIR/src/main/java/com/cloud/upgrade/DatabaseUpgradeChecker.java"
+LICENSE_TEMPLATE="$REPO_ROOT/.github/workflows/license-templates/LICENSE.txt"
+
+[[ -f "$LICENSE_TEMPLATE" ]] || { echo "License template not found: $LICENSE_TEMPLATE"; exit 1; }
+
+JAVA_FILE="$DAO_DIR/${CLASS_NAME}.java"
+SQL_FILE="$SQL_DIR/schema-${FROM_COMPACT}to${TO_COMPACT}.sql"
+CLEANUP_SQL_FILE="$SQL_DIR/schema-${FROM_COMPACT}to${TO_COMPACT}-cleanup.sql"
+
+for f in "$JAVA_FILE" "$SQL_FILE" "$CLEANUP_SQL_FILE"; do
+ if [[ -e "$f" ]]; then
+ echo "Refusing to overwrite existing file: $f"
+ exit 1
+ fi
+done
+
+if grep -q "new ${CLASS_NAME}()" "$CHECKER_FILE"; then
+ echo "DatabaseUpgradeChecker.java already references ${CLASS_NAME}"
+ exit 1
+fi
+
+# Render the shared license template as line comments for the given comment prefix.
+render_license() {
+ local prefix="$1"
+ awk -v prefix="$prefix" '{ if (length($0) == 0) print prefix; else print prefix " " $0 }' "$LICENSE_TEMPLATE"
+}
+
+LICENSE_JAVA="$(render_license "//")"
+LICENSE_SQL="$(render_license "--")"
+
+mkdir -p "$DAO_DIR" "$SQL_DIR"
+
+# 1. Upgrade class
+cat > "$JAVA_FILE" <<EOF
+${LICENSE_JAVA}
+package com.cloud.upgrade.dao;
+
+public class ${CLASS_NAME} extends DbUpgradeAbstractImpl implements DbUpgrade, DbUpgradeSystemVmTemplate {
+
+ @Override
+ public String[] getUpgradableVersionRange() {
+ return new String[]{"${FROM_VERSION}", "${TO_VERSION}"};
+ }
+
+ @Override
+ public String getUpgradedVersion() {
+ return "${TO_VERSION}";
+ }
+}
+EOF
+
+# 2. Schema upgrade + cleanup SQL scripts
+cat > "$SQL_FILE" <<EOF
+${LICENSE_SQL}
+
+--;
+-- Schema upgrade from ${FROM_VERSION} to ${TO_VERSION}
+--;
+EOF
+
+cat > "$CLEANUP_SQL_FILE" <<EOF
+${LICENSE_SQL}
+
+--;
+-- Schema upgrade cleanup from ${FROM_VERSION} to ${TO_VERSION}
+--;
+EOF
+
+# 3. Wire the new class into DatabaseUpgradeChecker.java: add the import in its
+# existing sorted block, and append a .next() entry right before .build().
+IMPORT_LINE="import com.cloud.upgrade.dao.${CLASS_NAME};"
+FIRST_IMPORT_LINE=$(grep -n '^import com\.cloud\.upgrade\.dao\.Upgrade' "$CHECKER_FILE" | head -1 | cut -d: -f1)
+LAST_IMPORT_LINE=$(grep -n '^import com\.cloud\.upgrade\.dao\.Upgrade' "$CHECKER_FILE" | tail -1 | cut -d: -f1)
+
+SORTED_IMPORTS_FILE="$(mktemp)"
+trap 'rm -f "$SORTED_IMPORTS_FILE"' EXIT
+{
+ grep '^import com\.cloud\.upgrade\.dao\.Upgrade' "$CHECKER_FILE"
+ echo "$IMPORT_LINE"
+} | sed 's/;$//' | LC_ALL=C sort -u | sed 's/$/;/' > "$SORTED_IMPORTS_FILE"
+
+TMP_CHECKER="$(mktemp)"
+awk -v first="$FIRST_IMPORT_LINE" -v last="$LAST_IMPORT_LINE" -v importfile="$SORTED_IMPORTS_FILE" '
+ NR == first {
+ while ((getline line < importfile) > 0) print line
+ }
+ NR >= first && NR <= last { next }
+ { print }
+' "$CHECKER_FILE" > "$TMP_CHECKER"
Review Comment:
FIRST_IMPORT_LINE`/`LAST_IMPORT_LINE` are not validated. If `DatabaseUpgradeChecker.java` doesn’t contain any lines matching `^import com\.cloud\.upgrade\.dao\.Upgrade`, the script will proceed and (a) not add the import and (b) still inject the `.next(...)` line later, likely breaking compilation due to a missing import. Add an explicit guard that fails fast when either line number is empty (and print a helpful message about the expected import block format).
##########
engine/schema/create-upgrade-path.sh:
##########
@@ -0,0 +1,161 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Scaffolds a new database upgrade path, following the pattern used e.g. in
+# https://github.com/apache/cloudstack/pull/12048/files
+#
+# Creates:
+# - engine/schema/src/main/java/com/cloud/upgrade/dao/Upgrade<from>to<to>.java
+# - engine/schema/src/main/resources/META-INF/db/schema-<from>to<to>.sql
+# - engine/schema/src/main/resources/META-INF/db/schema-<from>to<to>-cleanup.sql
+# and wires the new class into DatabaseUpgradeChecker.java (import + .next() entry).
+#
+# Usage: engine/schema/create-upgrade-path.sh <fromVersion> <toVersion>
+# Example: engine/schema/create-upgrade-path.sh 4.23.0.0 4.24.0.0
+
+set -euo pipefail
+
+usage() {
+ echo "Usage: $0 <fromVersion> <toVersion>"
+ echo "Example: $0 4.23.0.0 4.24.0.0"
+ exit 1
+}
+
+[[ $# -eq 2 ]] || usage
+
+FROM_VERSION="$1"
+TO_VERSION="$2"
+
+VERSION_REGEX='^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$'
+[[ "$FROM_VERSION" =~ $VERSION_REGEX ]] || { echo "Invalid fromVersion '$FROM_VERSION' (expected x.y.z.w)"; exit 1; }
+[[ "$TO_VERSION" =~ $VERSION_REGEX ]] || { echo "Invalid toVersion '$TO_VERSION' (expected x.y.z.w)"; exit 1; }
+
+FROM_COMPACT="${FROM_VERSION//./}"
+TO_COMPACT="${TO_VERSION//./}"
+CLASS_NAME="Upgrade${FROM_COMPACT}to${TO_COMPACT}"
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
+DAO_DIR="$SCRIPT_DIR/src/main/java/com/cloud/upgrade/dao"
+SQL_DIR="$SCRIPT_DIR/src/main/resources/META-INF/db"
+CHECKER_FILE="$SCRIPT_DIR/src/main/java/com/cloud/upgrade/DatabaseUpgradeChecker.java"
+LICENSE_TEMPLATE="$REPO_ROOT/.github/workflows/license-templates/LICENSE.txt"
+
+[[ -f "$LICENSE_TEMPLATE" ]] || { echo "License template not found: $LICENSE_TEMPLATE"; exit 1; }
+
+JAVA_FILE="$DAO_DIR/${CLASS_NAME}.java"
+SQL_FILE="$SQL_DIR/schema-${FROM_COMPACT}to${TO_COMPACT}.sql"
+CLEANUP_SQL_FILE="$SQL_DIR/schema-${FROM_COMPACT}to${TO_COMPACT}-cleanup.sql"
+
+for f in "$JAVA_FILE" "$SQL_FILE" "$CLEANUP_SQL_FILE"; do
+ if [[ -e "$f" ]]; then
+ echo "Refusing to overwrite existing file: $f"
+ exit 1
+ fi
+done
+
+if grep -q "new ${CLASS_NAME}()" "$CHECKER_FILE"; then
+ echo "DatabaseUpgradeChecker.java already references ${CLASS_NAME}"
+ exit 1
+fi
+
+# Render the shared license template as line comments for the given comment prefix.
+render_license() {
+ local prefix="$1"
+ awk -v prefix="$prefix" '{ if (length($0) == 0) print prefix; else print prefix " " $0 }' "$LICENSE_TEMPLATE"
+}
+
+LICENSE_JAVA="$(render_license "//")"
Review Comment:
The generated Java files use `//` line comments for the ASF license header. Many Apache/Java codebases standardize on a `/* ... */` block comment for license headers, and some tooling/checks can be sensitive to header formatting. Consider rendering a Java block comment for license output (while keeping `--` for SQL), to better match common ASF Java source conventions and reduce risk of header verification failures.
##########
engine/schema/create-upgrade-path.sh:
##########
@@ -0,0 +1,161 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Scaffolds a new database upgrade path, following the pattern used e.g. in
+# https://github.com/apache/cloudstack/pull/12048/files
+#
+# Creates:
+# - engine/schema/src/main/java/com/cloud/upgrade/dao/Upgrade<from>to<to>.java
+# - engine/schema/src/main/resources/META-INF/db/schema-<from>to<to>.sql
+# - engine/schema/src/main/resources/META-INF/db/schema-<from>to<to>-cleanup.sql
+# and wires the new class into DatabaseUpgradeChecker.java (import + .next() entry).
+#
+# Usage: engine/schema/create-upgrade-path.sh <fromVersion> <toVersion>
+# Example: engine/schema/create-upgrade-path.sh 4.23.0.0 4.24.0.0
+
+set -euo pipefail
+
+usage() {
+ echo "Usage: $0 <fromVersion> <toVersion>"
+ echo "Example: $0 4.23.0.0 4.24.0.0"
+ exit 1
+}
+
+[[ $# -eq 2 ]] || usage
+
+FROM_VERSION="$1"
+TO_VERSION="$2"
+
+VERSION_REGEX='^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$'
+[[ "$FROM_VERSION" =~ $VERSION_REGEX ]] || { echo "Invalid fromVersion '$FROM_VERSION' (expected x.y.z.w)"; exit 1; }
+[[ "$TO_VERSION" =~ $VERSION_REGEX ]] || { echo "Invalid toVersion '$TO_VERSION' (expected x.y.z.w)"; exit 1; }
+
+FROM_COMPACT="${FROM_VERSION//./}"
+TO_COMPACT="${TO_VERSION//./}"
+CLASS_NAME="Upgrade${FROM_COMPACT}to${TO_COMPACT}"
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
+DAO_DIR="$SCRIPT_DIR/src/main/java/com/cloud/upgrade/dao"
+SQL_DIR="$SCRIPT_DIR/src/main/resources/META-INF/db"
+CHECKER_FILE="$SCRIPT_DIR/src/main/java/com/cloud/upgrade/DatabaseUpgradeChecker.java"
+LICENSE_TEMPLATE="$REPO_ROOT/.github/workflows/license-templates/LICENSE.txt"
+
+[[ -f "$LICENSE_TEMPLATE" ]] || { echo "License template not found: $LICENSE_TEMPLATE"; exit 1; }
+
+JAVA_FILE="$DAO_DIR/${CLASS_NAME}.java"
+SQL_FILE="$SQL_DIR/schema-${FROM_COMPACT}to${TO_COMPACT}.sql"
+CLEANUP_SQL_FILE="$SQL_DIR/schema-${FROM_COMPACT}to${TO_COMPACT}-cleanup.sql"
+
+for f in "$JAVA_FILE" "$SQL_FILE" "$CLEANUP_SQL_FILE"; do
+ if [[ -e "$f" ]]; then
+ echo "Refusing to overwrite existing file: $f"
+ exit 1
+ fi
+done
+
+if grep -q "new ${CLASS_NAME}()" "$CHECKER_FILE"; then
+ echo "DatabaseUpgradeChecker.java already references ${CLASS_NAME}"
+ exit 1
+fi
+
+# Render the shared license template as line comments for the given comment prefix.
+render_license() {
+ local prefix="$1"
+ awk -v prefix="$prefix" '{ if (length($0) == 0) print prefix; else print prefix " " $0 }' "$LICENSE_TEMPLATE"
+}
+
+LICENSE_JAVA="$(render_license "//")"
+LICENSE_SQL="$(render_license "--")"
+
+mkdir -p "$DAO_DIR" "$SQL_DIR"
+
+# 1. Upgrade class
+cat > "$JAVA_FILE" <<EOF
+${LICENSE_JAVA}
+package com.cloud.upgrade.dao;
+
+public class ${CLASS_NAME} extends DbUpgradeAbstractImpl implements DbUpgrade, DbUpgradeSystemVmTemplate {
+
+ @Override
+ public String[] getUpgradableVersionRange() {
+ return new String[]{"${FROM_VERSION}", "${TO_VERSION}"};
+ }
+
+ @Override
+ public String getUpgradedVersion() {
+ return "${TO_VERSION}";
+ }
+}
+EOF
+
+# 2. Schema upgrade + cleanup SQL scripts
+cat > "$SQL_FILE" <<EOF
+${LICENSE_SQL}
+
+--;
+-- Schema upgrade from ${FROM_VERSION} to ${TO_VERSION}
+--;
+EOF
+
+cat > "$CLEANUP_SQL_FILE" <<EOF
+${LICENSE_SQL}
+
+--;
+-- Schema upgrade cleanup from ${FROM_VERSION} to ${TO_VERSION}
+--;
+EOF
+
+# 3. Wire the new class into DatabaseUpgradeChecker.java: add the import in its
+# existing sorted block, and append a .next() entry right before .build().
+IMPORT_LINE="import com.cloud.upgrade.dao.${CLASS_NAME};"
+FIRST_IMPORT_LINE=$(grep -n '^import com\.cloud\.upgrade\.dao\.Upgrade' "$CHECKER_FILE" | head -1 | cut -d: -f1)
+LAST_IMPORT_LINE=$(grep -n '^import com\.cloud\.upgrade\.dao\.Upgrade' "$CHECKER_FILE" | tail -1 | cut -d: -f1)
+
Review Comment:
FIRST_IMPORT_LINE`/`LAST_IMPORT_LINE` are not validated. If `DatabaseUpgradeChecker.java` doesn’t contain any lines matching `^import com\.cloud\.upgrade\.dao\.Upgrade`, the script will proceed and (a) not add the import and (b) still inject the `.next(...)` line later, likely breaking compilation due to a missing import. Add an explicit guard that fails fast when either line number is empty (and print a helpful message about the expected import block format).
##########
engine/schema/create-upgrade-path.sh:
##########
@@ -0,0 +1,161 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Scaffolds a new database upgrade path, following the pattern used e.g. in
+# https://github.com/apache/cloudstack/pull/12048/files
+#
+# Creates:
+# - engine/schema/src/main/java/com/cloud/upgrade/dao/Upgrade<from>to<to>.java
+# - engine/schema/src/main/resources/META-INF/db/schema-<from>to<to>.sql
+# - engine/schema/src/main/resources/META-INF/db/schema-<from>to<to>-cleanup.sql
+# and wires the new class into DatabaseUpgradeChecker.java (import + .next() entry).
+#
+# Usage: engine/schema/create-upgrade-path.sh <fromVersion> <toVersion>
+# Example: engine/schema/create-upgrade-path.sh 4.23.0.0 4.24.0.0
+
+set -euo pipefail
+
+usage() {
+ echo "Usage: $0 <fromVersion> <toVersion>"
+ echo "Example: $0 4.23.0.0 4.24.0.0"
+ exit 1
+}
+
+[[ $# -eq 2 ]] || usage
+
+FROM_VERSION="$1"
+TO_VERSION="$2"
+
+VERSION_REGEX='^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$'
+[[ "$FROM_VERSION" =~ $VERSION_REGEX ]] || { echo "Invalid fromVersion '$FROM_VERSION' (expected x.y.z.w)"; exit 1; }
+[[ "$TO_VERSION" =~ $VERSION_REGEX ]] || { echo "Invalid toVersion '$TO_VERSION' (expected x.y.z.w)"; exit 1; }
+
+FROM_COMPACT="${FROM_VERSION//./}"
+TO_COMPACT="${TO_VERSION//./}"
+CLASS_NAME="Upgrade${FROM_COMPACT}to${TO_COMPACT}"
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
+DAO_DIR="$SCRIPT_DIR/src/main/java/com/cloud/upgrade/dao"
+SQL_DIR="$SCRIPT_DIR/src/main/resources/META-INF/db"
+CHECKER_FILE="$SCRIPT_DIR/src/main/java/com/cloud/upgrade/DatabaseUpgradeChecker.java"
+LICENSE_TEMPLATE="$REPO_ROOT/.github/workflows/license-templates/LICENSE.txt"
+
+[[ -f "$LICENSE_TEMPLATE" ]] || { echo "License template not found: $LICENSE_TEMPLATE"; exit 1; }
+
+JAVA_FILE="$DAO_DIR/${CLASS_NAME}.java"
+SQL_FILE="$SQL_DIR/schema-${FROM_COMPACT}to${TO_COMPACT}.sql"
+CLEANUP_SQL_FILE="$SQL_DIR/schema-${FROM_COMPACT}to${TO_COMPACT}-cleanup.sql"
+
+for f in "$JAVA_FILE" "$SQL_FILE" "$CLEANUP_SQL_FILE"; do
+ if [[ -e "$f" ]]; then
+ echo "Refusing to overwrite existing file: $f"
+ exit 1
+ fi
+done
+
+if grep -q "new ${CLASS_NAME}()" "$CHECKER_FILE"; then
+ echo "DatabaseUpgradeChecker.java already references ${CLASS_NAME}"
+ exit 1
+fi
+
+# Render the shared license template as line comments for the given comment prefix.
+render_license() {
+ local prefix="$1"
+ awk -v prefix="$prefix" '{ if (length($0) == 0) print prefix; else print prefix " " $0 }' "$LICENSE_TEMPLATE"
+}
+
+LICENSE_JAVA="$(render_license "//")"
+LICENSE_SQL="$(render_license "--")"
+
+mkdir -p "$DAO_DIR" "$SQL_DIR"
+
+# 1. Upgrade class
+cat > "$JAVA_FILE" <<EOF
+${LICENSE_JAVA}
+package com.cloud.upgrade.dao;
+
+public class ${CLASS_NAME} extends DbUpgradeAbstractImpl implements DbUpgrade, DbUpgradeSystemVmTemplate {
+
+ @Override
+ public String[] getUpgradableVersionRange() {
+ return new String[]{"${FROM_VERSION}", "${TO_VERSION}"};
+ }
+
+ @Override
+ public String getUpgradedVersion() {
+ return "${TO_VERSION}";
+ }
+}
+EOF
+
+# 2. Schema upgrade + cleanup SQL scripts
+cat > "$SQL_FILE" <<EOF
+${LICENSE_SQL}
+
+--;
+-- Schema upgrade from ${FROM_VERSION} to ${TO_VERSION}
+--;
+EOF
+
+cat > "$CLEANUP_SQL_FILE" <<EOF
+${LICENSE_SQL}
+
+--;
+-- Schema upgrade cleanup from ${FROM_VERSION} to ${TO_VERSION}
+--;
+EOF
+
+# 3. Wire the new class into DatabaseUpgradeChecker.java: add the import in its
+# existing sorted block, and append a .next() entry right before .build().
+IMPORT_LINE="import com.cloud.upgrade.dao.${CLASS_NAME};"
+FIRST_IMPORT_LINE=$(grep -n '^import com\.cloud\.upgrade\.dao\.Upgrade' "$CHECKER_FILE" | head -1 | cut -d: -f1)
+LAST_IMPORT_LINE=$(grep -n '^import com\.cloud\.upgrade\.dao\.Upgrade' "$CHECKER_FILE" | tail -1 | cut -d: -f1)
+
+SORTED_IMPORTS_FILE="$(mktemp)"
+trap 'rm -f "$SORTED_IMPORTS_FILE"' EXIT
+{
+ grep '^import com\.cloud\.upgrade\.dao\.Upgrade' "$CHECKER_FILE"
+ echo "$IMPORT_LINE"
+} | sed 's/;$//' | LC_ALL=C sort -u | sed 's/$/;/' > "$SORTED_IMPORTS_FILE"
+
+TMP_CHECKER="$(mktemp)"
+awk -v first="$FIRST_IMPORT_LINE" -v last="$LAST_IMPORT_LINE" -v importfile="$SORTED_IMPORTS_FILE" '
+ NR == first {
+ while ((getline line < importfile) > 0) print line
+ }
+ NR >= first && NR <= last { next }
+ { print }
+' "$CHECKER_FILE" > "$TMP_CHECKER"
+
+NEXT_LINE=" .next(\"${FROM_VERSION}\", new ${CLASS_NAME}())"
+awk -v nextline="$NEXT_LINE" '
+ /^[ \t]*\.build\(\);/ && !inserted {
+ print nextline
+ inserted = 1
+ }
+ { print }
Review Comment:
The injection point matches the first line that looks like `.build();` anywhere in the file. This is brittle and can place the new `.next(...)` in the wrong builder chain if `DatabaseUpgradeChecker.java` contains multiple `.build();` calls. Make the match more specific to the upgrade-path builder block (e.g., key off a nearby unique marker line, the builder variable name, or the specific method that constructs the upgrade path).
##########
engine/schema/create-upgrade-path.sh:
##########
@@ -0,0 +1,161 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Scaffolds a new database upgrade path, following the pattern used e.g. in
+# https://github.com/apache/cloudstack/pull/12048/files
+#
+# Creates:
+# - engine/schema/src/main/java/com/cloud/upgrade/dao/Upgrade<from>to<to>.java
+# - engine/schema/src/main/resources/META-INF/db/schema-<from>to<to>.sql
+# - engine/schema/src/main/resources/META-INF/db/schema-<from>to<to>-cleanup.sql
+# and wires the new class into DatabaseUpgradeChecker.java (import + .next() entry).
+#
+# Usage: engine/schema/create-upgrade-path.sh <fromVersion> <toVersion>
+# Example: engine/schema/create-upgrade-path.sh 4.23.0.0 4.24.0.0
+
+set -euo pipefail
+
+usage() {
+ echo "Usage: $0 <fromVersion> <toVersion>"
+ echo "Example: $0 4.23.0.0 4.24.0.0"
+ exit 1
+}
+
+[[ $# -eq 2 ]] || usage
+
+FROM_VERSION="$1"
+TO_VERSION="$2"
+
+VERSION_REGEX='^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$'
+[[ "$FROM_VERSION" =~ $VERSION_REGEX ]] || { echo "Invalid fromVersion '$FROM_VERSION' (expected x.y.z.w)"; exit 1; }
+[[ "$TO_VERSION" =~ $VERSION_REGEX ]] || { echo "Invalid toVersion '$TO_VERSION' (expected x.y.z.w)"; exit 1; }
+
+FROM_COMPACT="${FROM_VERSION//./}"
+TO_COMPACT="${TO_VERSION//./}"
+CLASS_NAME="Upgrade${FROM_COMPACT}to${TO_COMPACT}"
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
+DAO_DIR="$SCRIPT_DIR/src/main/java/com/cloud/upgrade/dao"
+SQL_DIR="$SCRIPT_DIR/src/main/resources/META-INF/db"
+CHECKER_FILE="$SCRIPT_DIR/src/main/java/com/cloud/upgrade/DatabaseUpgradeChecker.java"
+LICENSE_TEMPLATE="$REPO_ROOT/.github/workflows/license-templates/LICENSE.txt"
+
+[[ -f "$LICENSE_TEMPLATE" ]] || { echo "License template not found: $LICENSE_TEMPLATE"; exit 1; }
+
+JAVA_FILE="$DAO_DIR/${CLASS_NAME}.java"
+SQL_FILE="$SQL_DIR/schema-${FROM_COMPACT}to${TO_COMPACT}.sql"
+CLEANUP_SQL_FILE="$SQL_DIR/schema-${FROM_COMPACT}to${TO_COMPACT}-cleanup.sql"
+
+for f in "$JAVA_FILE" "$SQL_FILE" "$CLEANUP_SQL_FILE"; do
+ if [[ -e "$f" ]]; then
+ echo "Refusing to overwrite existing file: $f"
+ exit 1
+ fi
+done
+
+if grep -q "new ${CLASS_NAME}()" "$CHECKER_FILE"; then
+ echo "DatabaseUpgradeChecker.java already references ${CLASS_NAME}"
+ exit 1
+fi
+
+# Render the shared license template as line comments for the given comment prefix.
+render_license() {
+ local prefix="$1"
+ awk -v prefix="$prefix" '{ if (length($0) == 0) print prefix; else print prefix " " $0 }' "$LICENSE_TEMPLATE"
+}
+
+LICENSE_JAVA="$(render_license "//")"
+LICENSE_SQL="$(render_license "--")"
+
+mkdir -p "$DAO_DIR" "$SQL_DIR"
+
+# 1. Upgrade class
+cat > "$JAVA_FILE" <<EOF
+${LICENSE_JAVA}
+package com.cloud.upgrade.dao;
+
+public class ${CLASS_NAME} extends DbUpgradeAbstractImpl implements DbUpgrade, DbUpgradeSystemVmTemplate {
+
+ @Override
+ public String[] getUpgradableVersionRange() {
+ return new String[]{"${FROM_VERSION}", "${TO_VERSION}"};
+ }
+
+ @Override
+ public String getUpgradedVersion() {
+ return "${TO_VERSION}";
+ }
+}
+EOF
+
+# 2. Schema upgrade + cleanup SQL scripts
+cat > "$SQL_FILE" <<EOF
+${LICENSE_SQL}
+
+--;
+-- Schema upgrade from ${FROM_VERSION} to ${TO_VERSION}
+--;
+EOF
+
+cat > "$CLEANUP_SQL_FILE" <<EOF
+${LICENSE_SQL}
+
+--;
+-- Schema upgrade cleanup from ${FROM_VERSION} to ${TO_VERSION}
+--;
+EOF
+
+# 3. Wire the new class into DatabaseUpgradeChecker.java: add the import in its
+# existing sorted block, and append a .next() entry right before .build().
+IMPORT_LINE="import com.cloud.upgrade.dao.${CLASS_NAME};"
+FIRST_IMPORT_LINE=$(grep -n '^import com\.cloud\.upgrade\.dao\.Upgrade' "$CHECKER_FILE" | head -1 | cut -d: -f1)
+LAST_IMPORT_LINE=$(grep -n '^import com\.cloud\.upgrade\.dao\.Upgrade' "$CHECKER_FILE" | tail -1 | cut -d: -f1)
+
+SORTED_IMPORTS_FILE="$(mktemp)"
+trap 'rm -f "$SORTED_IMPORTS_FILE"' EXIT
+{
+ grep '^import com\.cloud\.upgrade\.dao\.Upgrade' "$CHECKER_FILE"
+ echo "$IMPORT_LINE"
+} | sed 's/;$//' | LC_ALL=C sort -u | sed 's/$/;/' > "$SORTED_IMPORTS_FILE"
+
+TMP_CHECKER="$(mktemp)"
+awk -v first="$FIRST_IMPORT_LINE" -v last="$LAST_IMPORT_LINE" -v importfile="$SORTED_IMPORTS_FILE" '
+ NR == first {
+ while ((getline line < importfile) > 0) print line
+ }
+ NR >= first && NR <= last { next }
+ { print }
+' "$CHECKER_FILE" > "$TMP_CHECKER"
+
+NEXT_LINE=" .next(\"${FROM_VERSION}\", new ${CLASS_NAME}())"
+awk -v nextline="$NEXT_LINE" '
+ /^[ \t]*\.build\(\);/ && !inserted {
+ print nextline
+ inserted = 1
+ }
+ { print }
+' "$TMP_CHECKER" > "$CHECKER_FILE"
+rm -f "$TMP_CHECKER"
Review Comment:
The script rewrites `DatabaseUpgradeChecker.java` in-place via shell redirection (`> "$CHECKER_FILE"`). If the process is interrupted (or an error occurs mid-write), it can leave the file truncated/corrupted. Prefer writing to a temporary output file and then `mv` atomically into place; also consider adding `TMP_CHECKER` to the `trap` cleanup so temp files don’t linger on failures.
##########
engine/schema/create-upgrade-path.sh:
##########
@@ -0,0 +1,161 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Scaffolds a new database upgrade path, following the pattern used e.g. in
+# https://github.com/apache/cloudstack/pull/12048/files
+#
+# Creates:
+# - engine/schema/src/main/java/com/cloud/upgrade/dao/Upgrade<from>to<to>.java
+# - engine/schema/src/main/resources/META-INF/db/schema-<from>to<to>.sql
+# - engine/schema/src/main/resources/META-INF/db/schema-<from>to<to>-cleanup.sql
+# and wires the new class into DatabaseUpgradeChecker.java (import + .next() entry).
+#
+# Usage: engine/schema/create-upgrade-path.sh <fromVersion> <toVersion>
+# Example: engine/schema/create-upgrade-path.sh 4.23.0.0 4.24.0.0
+
+set -euo pipefail
+
+usage() {
+ echo "Usage: $0 <fromVersion> <toVersion>"
+ echo "Example: $0 4.23.0.0 4.24.0.0"
+ exit 1
+}
+
+[[ $# -eq 2 ]] || usage
+
+FROM_VERSION="$1"
+TO_VERSION="$2"
+
+VERSION_REGEX='^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$'
+[[ "$FROM_VERSION" =~ $VERSION_REGEX ]] || { echo "Invalid fromVersion '$FROM_VERSION' (expected x.y.z.w)"; exit 1; }
+[[ "$TO_VERSION" =~ $VERSION_REGEX ]] || { echo "Invalid toVersion '$TO_VERSION' (expected x.y.z.w)"; exit 1; }
+
+FROM_COMPACT="${FROM_VERSION//./}"
+TO_COMPACT="${TO_VERSION//./}"
+CLASS_NAME="Upgrade${FROM_COMPACT}to${TO_COMPACT}"
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
+DAO_DIR="$SCRIPT_DIR/src/main/java/com/cloud/upgrade/dao"
+SQL_DIR="$SCRIPT_DIR/src/main/resources/META-INF/db"
+CHECKER_FILE="$SCRIPT_DIR/src/main/java/com/cloud/upgrade/DatabaseUpgradeChecker.java"
+LICENSE_TEMPLATE="$REPO_ROOT/.github/workflows/license-templates/LICENSE.txt"
+
+[[ -f "$LICENSE_TEMPLATE" ]] || { echo "License template not found: $LICENSE_TEMPLATE"; exit 1; }
+
+JAVA_FILE="$DAO_DIR/${CLASS_NAME}.java"
+SQL_FILE="$SQL_DIR/schema-${FROM_COMPACT}to${TO_COMPACT}.sql"
+CLEANUP_SQL_FILE="$SQL_DIR/schema-${FROM_COMPACT}to${TO_COMPACT}-cleanup.sql"
+
+for f in "$JAVA_FILE" "$SQL_FILE" "$CLEANUP_SQL_FILE"; do
+ if [[ -e "$f" ]]; then
+ echo "Refusing to overwrite existing file: $f"
+ exit 1
+ fi
+done
+
+if grep -q "new ${CLASS_NAME}()" "$CHECKER_FILE"; then
+ echo "DatabaseUpgradeChecker.java already references ${CLASS_NAME}"
+ exit 1
+fi
+
+# Render the shared license template as line comments for the given comment prefix.
+render_license() {
+ local prefix="$1"
+ awk -v prefix="$prefix" '{ if (length($0) == 0) print prefix; else print prefix " " $0 }' "$LICENSE_TEMPLATE"
+}
+
+LICENSE_JAVA="$(render_license "//")"
+LICENSE_SQL="$(render_license "--")"
+
+mkdir -p "$DAO_DIR" "$SQL_DIR"
+
+# 1. Upgrade class
+cat > "$JAVA_FILE" <<EOF
+${LICENSE_JAVA}
+package com.cloud.upgrade.dao;
Review Comment:
The generated Java files use `//` line comments for the ASF license header. Many Apache/Java codebases standardize on a `/* ... */` block comment for license headers, and some tooling/checks can be sensitive to header formatting. Consider rendering a Java block comment for license output (while keeping `--` for SQL), to better match common ASF Java source conventions and reduce risk of header verification failures.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] engine-schema: add script to generate upgrade path chages [cloudstack]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #14034:
URL: https://github.com/apache/cloudstack/pull/14034#issuecomment-5505782998
## [Codecov](https://app.codecov.io/gh/apache/cloudstack/pull/14034?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 3.41%. Comparing base ([`8fb3ad4`](https://app.codecov.io/gh/apache/cloudstack/commit/8fb3ad4c4e6f431657d018512eece326d704c5b5?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`c077c7c`](https://app.codecov.io/gh/apache/cloudstack/commit/c077c7c520df3369dda9886f82165d1860839a4b?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
> :exclamation: There is a different number of reports uploaded between BASE (8fb3ad4) and HEAD (c077c7c). Click for more details.
>
> <details><summary>HEAD has 1 upload less than BASE</summary>
>
>| Flag | BASE (8fb3ad4) | HEAD (c077c7c) |
>|------|------|------|
>|unittests|1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #14034 +/- ##
=============================================
- Coverage 19.73% 3.41% -16.33%
=============================================
Files 6371 487 -5884
Lines 575784 41863 -533921
Branches 70478 7912 -62566
=============================================
- Hits 113659 1429 -112230
+ Misses 449772 40234 -409538
+ Partials 12353 200 -12153
```
| [Flag](https://app.codecov.io/gh/apache/cloudstack/pull/14034/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [uitests](https://app.codecov.io/gh/apache/cloudstack/pull/14034/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `3.41% <ø> (ø)` | |
| [unittests](https://app.codecov.io/gh/apache/cloudstack/pull/14034/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/cloudstack/pull/14034?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org