You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by GitBox <gi...@apache.org> on 2022/08/09 20:53:58 UTC

[GitHub] [camel-k] phantomjinx opened a new pull request, #3527: Improvements to e2e tests and testing OLM bundle in test index

phantomjinx opened a new pull request, #3527:
URL: https://github.com/apache/camel-k/pull/3527

   <!-- Description -->
   
   
   
   
   <!--
   Enter your extended release note in the below block. If the PR requires
   additional action from users switching to the new release, include the string
   "action required". If no release note is required, write "NONE". 
   
   You can (optionally) mark this PR with labels "kind/bug" or "kind/feature" to make sure
   the text is added to the right section of the release notes. 
   -->
   
   **Release Note**
   ```release-note
   NONE
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [camel-k] tadayosi merged pull request #3527: Improvements to OLM bundle building of test index

Posted by GitBox <gi...@apache.org>.
tadayosi merged PR #3527:
URL: https://github.com/apache/camel-k/pull/3527


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [camel-k] tadayosi commented on a diff in pull request #3527: Improvements to e2e tests and testing OLM bundle in test index

Posted by GitBox <gi...@apache.org>.
tadayosi commented on code in PR #3527:
URL: https://github.com/apache/camel-k/pull/3527#discussion_r949785135


##########
script/build_bundle_index.sh:
##########
@@ -58,43 +60,82 @@ fi
 
 mkdir -p "${INDEX_DIR}"
 
-if [ ! -f ${INDEX_DIR}/bundles.yaml ]; then
-  ${OPM} render ${BUNDLE_INDEX} -o yaml > ${INDEX_DIR}/bundles.yaml
+if [ ! -f ${INDEX_BASE_YAML} ]; then
+  ${OPM} render ${BUNDLE_INDEX} -o yaml > ${INDEX_BASE_YAML}
   if [ $? != 0 ]; then
     echo "Error: failed to render the base catalog"
     exit 1
   fi
 fi
 
-${OPM} render --skip-tls -o yaml \
-  ${BUNDLE_IMAGE} > ${PACKAGE_YAML}
-if [ $? != 0 ]; then
-  echo "Error: failed to render the ${PACKAGE} bundle catalog"
-  exit 1
+if [ ! -f ${PACKAGE_YAML} ]; then
+  ${OPM} render --skip-tls -o yaml \
+    ${BUNDLE_IMAGE} > ${PACKAGE_YAML}
+  if [ $? != 0 ]; then
+    echo "Error: failed to render the ${PACKAGE} bundle catalog"
+    exit 1
+  fi
 fi
 
+#
+# Extract the camel-k channels
+#
+yq eval ". | select(.package == \"${PACKAGE}\" and .schema == \"olm.channel\")" ${INDEX_BASE_YAML} > ${CHANNELS_YAML}
+if [ $? != 0 ] || [ ! -f "${CHANNELS_YAML}" ]; then
+  echo "ERROR: Failed to extract camel-k entries from bundle catalog"
+  exit 1
+fi
 
+#
+# Filter out the channels in the bundles file
+#
+yq -i eval ". | select(.package != \"${PACKAGE}\" or .schema != \"olm.channel\")" ${INDEX_BASE_YAML}
+if [ $? != 0 ]; then
+  echo "ERROR: Failed to remove camel-k channel entries from bundles catalog"
+  exit 1
+fi
 
-cat << EOF >> ${PACKAGE_YAML}
----
-schema: olm.channel
-package: ${PACKAGE}
-name: ${CHANNEL}
-entries:
-  - name: ${CSV_NAME}
-EOF
+#
+# Split the channels and append/insert the bundle into each one
+#
+IFS=','
+#Read the split words into an array based on comma delimiter
+read -a CHANNEL_ARR <<< "${CHANNELS}"
 
-if [ -n "${CSV_REPLACES}" ]; then
-cat << EOF >> ${PACKAGE_YAML}
-    replaces: ${CSV_REPLACES}
-EOF
-fi
+for channel in "${CHANNEL_ARR[@]}";
+do
+  channel_props=$(yq eval ". | select(.name == \"${channel}\")" ${CHANNELS_YAML})
 
-if [ -n "${CSV_SKIPS}" ]; then
-cat << EOF >> ${PACKAGE_YAML}
-    skipRange: "\'${CSV_SKIPS}\'"
-EOF
-fi
+  entry="{ \"name\": \"${CSV_NAME}\""
+  if [ -n "${CSV_REPLACES}" ]; then
+    entry="${entry}, \"replaces\": \"${CSV_REPLACES}\""
+  fi
+  if [ -n "${CSV_SKIPS}" ]; then
+    entry="${entry}, \"skipRange\": \"${CSV_SKIPS}\""
+  fi
+  entry="${entry} }"
+
+  if [ -z "${channel_props}" ]; then
+    #
+    # Append a new channel
+    #
+    echo "Appending channel ${channel} ..."
+    object="{ \"entries\": [${entry}], \"name\": \"${channel}\", \"package\": \"${PACKAGE}\", \"schema\": \"olm.channel\" }"
+
+    channel_file=$(mktemp ${channel}-channel-XXX.yaml)
+    trap "rm -f ${channel_file}" EXIT

Review Comment:
   shellcheck says:
   ```
       trap "rm -f ${channel_file}" EXIT
                   ^-------------^ SC2064 (warning): Use single quotes, otherwise this expands now rather than when signalled.
   ```
   does it make sense?



##########
script/build_bundle_index.sh:
##########
@@ -58,43 +60,82 @@ fi
 
 mkdir -p "${INDEX_DIR}"
 
-if [ ! -f ${INDEX_DIR}/bundles.yaml ]; then
-  ${OPM} render ${BUNDLE_INDEX} -o yaml > ${INDEX_DIR}/bundles.yaml
+if [ ! -f ${INDEX_BASE_YAML} ]; then
+  ${OPM} render ${BUNDLE_INDEX} -o yaml > ${INDEX_BASE_YAML}
   if [ $? != 0 ]; then
     echo "Error: failed to render the base catalog"
     exit 1
   fi
 fi
 
-${OPM} render --skip-tls -o yaml \
-  ${BUNDLE_IMAGE} > ${PACKAGE_YAML}
-if [ $? != 0 ]; then
-  echo "Error: failed to render the ${PACKAGE} bundle catalog"
-  exit 1
+if [ ! -f ${PACKAGE_YAML} ]; then
+  ${OPM} render --skip-tls -o yaml \
+    ${BUNDLE_IMAGE} > ${PACKAGE_YAML}
+  if [ $? != 0 ]; then
+    echo "Error: failed to render the ${PACKAGE} bundle catalog"
+    exit 1
+  fi
 fi
 
+#
+# Extract the camel-k channels
+#
+yq eval ". | select(.package == \"${PACKAGE}\" and .schema == \"olm.channel\")" ${INDEX_BASE_YAML} > ${CHANNELS_YAML}
+if [ $? != 0 ] || [ ! -f "${CHANNELS_YAML}" ]; then
+  echo "ERROR: Failed to extract camel-k entries from bundle catalog"
+  exit 1
+fi
 
+#
+# Filter out the channels in the bundles file
+#
+yq -i eval ". | select(.package != \"${PACKAGE}\" or .schema != \"olm.channel\")" ${INDEX_BASE_YAML}
+if [ $? != 0 ]; then
+  echo "ERROR: Failed to remove camel-k channel entries from bundles catalog"
+  exit 1
+fi
 
-cat << EOF >> ${PACKAGE_YAML}
----
-schema: olm.channel
-package: ${PACKAGE}
-name: ${CHANNEL}
-entries:
-  - name: ${CSV_NAME}
-EOF
+#
+# Split the channels and append/insert the bundle into each one
+#
+IFS=','
+#Read the split words into an array based on comma delimiter
+read -a CHANNEL_ARR <<< "${CHANNELS}"

Review Comment:
   shellcheck says:
   ```
   read -a CHANNEL_ARR <<< "${CHANNELS}"
   ^--^ SC2162 (info): read without -r will mangle backslashes.
   ```



##########
script/build_bundle_index.sh:
##########
@@ -58,43 +60,82 @@ fi
 
 mkdir -p "${INDEX_DIR}"
 
-if [ ! -f ${INDEX_DIR}/bundles.yaml ]; then
-  ${OPM} render ${BUNDLE_INDEX} -o yaml > ${INDEX_DIR}/bundles.yaml
+if [ ! -f ${INDEX_BASE_YAML} ]; then
+  ${OPM} render ${BUNDLE_INDEX} -o yaml > ${INDEX_BASE_YAML}
   if [ $? != 0 ]; then
     echo "Error: failed to render the base catalog"
     exit 1
   fi
 fi
 
-${OPM} render --skip-tls -o yaml \
-  ${BUNDLE_IMAGE} > ${PACKAGE_YAML}
-if [ $? != 0 ]; then
-  echo "Error: failed to render the ${PACKAGE} bundle catalog"
-  exit 1
+if [ ! -f ${PACKAGE_YAML} ]; then
+  ${OPM} render --skip-tls -o yaml \
+    ${BUNDLE_IMAGE} > ${PACKAGE_YAML}
+  if [ $? != 0 ]; then
+    echo "Error: failed to render the ${PACKAGE} bundle catalog"
+    exit 1
+  fi
 fi
 
+#
+# Extract the camel-k channels
+#
+yq eval ". | select(.package == \"${PACKAGE}\" and .schema == \"olm.channel\")" ${INDEX_BASE_YAML} > ${CHANNELS_YAML}

Review Comment:
   I don't think `yq` has been used anywhere else in the scripts. It should be better to note somewhere that `yq` is a requirement now and check the existence of `yq` with `command -v` at the beginning of this script and fail and warn early if it's not found.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [camel-k] phantomjinx commented on a diff in pull request #3527: Improvements to OLM bundle building of test index

Posted by GitBox <gi...@apache.org>.
phantomjinx commented on code in PR #3527:
URL: https://github.com/apache/camel-k/pull/3527#discussion_r993297857


##########
script/build_bundle_index.sh:
##########
@@ -58,43 +60,82 @@ fi
 
 mkdir -p "${INDEX_DIR}"
 
-if [ ! -f ${INDEX_DIR}/bundles.yaml ]; then
-  ${OPM} render ${BUNDLE_INDEX} -o yaml > ${INDEX_DIR}/bundles.yaml
+if [ ! -f ${INDEX_BASE_YAML} ]; then
+  ${OPM} render ${BUNDLE_INDEX} -o yaml > ${INDEX_BASE_YAML}
   if [ $? != 0 ]; then
     echo "Error: failed to render the base catalog"
     exit 1
   fi
 fi
 
-${OPM} render --skip-tls -o yaml \
-  ${BUNDLE_IMAGE} > ${PACKAGE_YAML}
-if [ $? != 0 ]; then
-  echo "Error: failed to render the ${PACKAGE} bundle catalog"
-  exit 1
+if [ ! -f ${PACKAGE_YAML} ]; then
+  ${OPM} render --skip-tls -o yaml \
+    ${BUNDLE_IMAGE} > ${PACKAGE_YAML}
+  if [ $? != 0 ]; then
+    echo "Error: failed to render the ${PACKAGE} bundle catalog"
+    exit 1
+  fi
 fi
 
+#
+# Extract the camel-k channels
+#
+yq eval ". | select(.package == \"${PACKAGE}\" and .schema == \"olm.channel\")" ${INDEX_BASE_YAML} > ${CHANNELS_YAML}
+if [ $? != 0 ] || [ ! -f "${CHANNELS_YAML}" ]; then
+  echo "ERROR: Failed to extract camel-k entries from bundle catalog"
+  exit 1
+fi
 
+#
+# Filter out the channels in the bundles file
+#
+yq -i eval ". | select(.package != \"${PACKAGE}\" or .schema != \"olm.channel\")" ${INDEX_BASE_YAML}
+if [ $? != 0 ]; then
+  echo "ERROR: Failed to remove camel-k channel entries from bundles catalog"
+  exit 1
+fi
 
-cat << EOF >> ${PACKAGE_YAML}
----
-schema: olm.channel
-package: ${PACKAGE}
-name: ${CHANNEL}
-entries:
-  - name: ${CSV_NAME}
-EOF
+#
+# Split the channels and append/insert the bundle into each one
+#
+IFS=','
+#Read the split words into an array based on comma delimiter
+read -a CHANNEL_ARR <<< "${CHANNELS}"
 
-if [ -n "${CSV_REPLACES}" ]; then
-cat << EOF >> ${PACKAGE_YAML}
-    replaces: ${CSV_REPLACES}
-EOF
-fi
+for channel in "${CHANNEL_ARR[@]}";
+do
+  channel_props=$(yq eval ". | select(.name == \"${channel}\")" ${CHANNELS_YAML})
 
-if [ -n "${CSV_SKIPS}" ]; then
-cat << EOF >> ${PACKAGE_YAML}
-    skipRange: "\'${CSV_SKIPS}\'"
-EOF
-fi
+  entry="{ \"name\": \"${CSV_NAME}\""
+  if [ -n "${CSV_REPLACES}" ]; then
+    entry="${entry}, \"replaces\": \"${CSV_REPLACES}\""
+  fi
+  if [ -n "${CSV_SKIPS}" ]; then
+    entry="${entry}, \"skipRange\": \"${CSV_SKIPS}\""
+  fi
+  entry="${entry} }"
+
+  if [ -z "${channel_props}" ]; then
+    #
+    # Append a new channel
+    #
+    echo "Appending channel ${channel} ..."
+    object="{ \"entries\": [${entry}], \"name\": \"${channel}\", \"package\": \"${PACKAGE}\", \"schema\": \"olm.channel\" }"
+
+    channel_file=$(mktemp ${channel}-channel-XXX.yaml)
+    trap "rm -f ${channel_file}" EXIT

Review Comment:
   Done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [camel-k] tadayosi commented on pull request #3527: Improvements to OLM bundle building of test index

Posted by GitBox <gi...@apache.org>.
tadayosi commented on PR #3527:
URL: https://github.com/apache/camel-k/pull/3527#issuecomment-1278456750

   > The test is failing:
   > 
   > ```
   > ❌ TestKameletFromCustomRepository (10m10.77s)
   > ```
   
   OK, it's not the PR's fault. Let's merge it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [camel-k] tadayosi commented on a diff in pull request #3527: Improvements to e2e tests and testing OLM bundle in test index

Posted by GitBox <gi...@apache.org>.
tadayosi commented on code in PR #3527:
URL: https://github.com/apache/camel-k/pull/3527#discussion_r949784327


##########
script/build_bundle_index.sh:
##########
@@ -58,43 +60,82 @@ fi
 
 mkdir -p "${INDEX_DIR}"
 
-if [ ! -f ${INDEX_DIR}/bundles.yaml ]; then
-  ${OPM} render ${BUNDLE_INDEX} -o yaml > ${INDEX_DIR}/bundles.yaml
+if [ ! -f ${INDEX_BASE_YAML} ]; then
+  ${OPM} render ${BUNDLE_INDEX} -o yaml > ${INDEX_BASE_YAML}
   if [ $? != 0 ]; then
     echo "Error: failed to render the base catalog"
     exit 1
   fi
 fi
 
-${OPM} render --skip-tls -o yaml \
-  ${BUNDLE_IMAGE} > ${PACKAGE_YAML}
-if [ $? != 0 ]; then
-  echo "Error: failed to render the ${PACKAGE} bundle catalog"
-  exit 1
+if [ ! -f ${PACKAGE_YAML} ]; then
+  ${OPM} render --skip-tls -o yaml \
+    ${BUNDLE_IMAGE} > ${PACKAGE_YAML}
+  if [ $? != 0 ]; then
+    echo "Error: failed to render the ${PACKAGE} bundle catalog"
+    exit 1
+  fi
 fi
 
+#
+# Extract the camel-k channels
+#
+yq eval ". | select(.package == \"${PACKAGE}\" and .schema == \"olm.channel\")" ${INDEX_BASE_YAML} > ${CHANNELS_YAML}

Review Comment:
   I don't think `yq` has been used anywhere else in the scripts before. It should be better to note somewhere that `yq` is a requirement now and check the existence of `yq` with `command -v` at the beginning of this script and fail and warn early if it's not found.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [camel-k] phantomjinx commented on a diff in pull request #3527: Improvements to OLM bundle building of test index

Posted by GitBox <gi...@apache.org>.
phantomjinx commented on code in PR #3527:
URL: https://github.com/apache/camel-k/pull/3527#discussion_r993297676


##########
script/build_bundle_index.sh:
##########
@@ -58,43 +60,82 @@ fi
 
 mkdir -p "${INDEX_DIR}"
 
-if [ ! -f ${INDEX_DIR}/bundles.yaml ]; then
-  ${OPM} render ${BUNDLE_INDEX} -o yaml > ${INDEX_DIR}/bundles.yaml
+if [ ! -f ${INDEX_BASE_YAML} ]; then
+  ${OPM} render ${BUNDLE_INDEX} -o yaml > ${INDEX_BASE_YAML}
   if [ $? != 0 ]; then
     echo "Error: failed to render the base catalog"
     exit 1
   fi
 fi
 
-${OPM} render --skip-tls -o yaml \
-  ${BUNDLE_IMAGE} > ${PACKAGE_YAML}
-if [ $? != 0 ]; then
-  echo "Error: failed to render the ${PACKAGE} bundle catalog"
-  exit 1
+if [ ! -f ${PACKAGE_YAML} ]; then
+  ${OPM} render --skip-tls -o yaml \
+    ${BUNDLE_IMAGE} > ${PACKAGE_YAML}
+  if [ $? != 0 ]; then
+    echo "Error: failed to render the ${PACKAGE} bundle catalog"
+    exit 1
+  fi
 fi
 
+#
+# Extract the camel-k channels
+#
+yq eval ". | select(.package == \"${PACKAGE}\" and .schema == \"olm.channel\")" ${INDEX_BASE_YAML} > ${CHANNELS_YAML}

Review Comment:
   Done



##########
script/build_bundle_index.sh:
##########
@@ -58,43 +60,82 @@ fi
 
 mkdir -p "${INDEX_DIR}"
 
-if [ ! -f ${INDEX_DIR}/bundles.yaml ]; then
-  ${OPM} render ${BUNDLE_INDEX} -o yaml > ${INDEX_DIR}/bundles.yaml
+if [ ! -f ${INDEX_BASE_YAML} ]; then
+  ${OPM} render ${BUNDLE_INDEX} -o yaml > ${INDEX_BASE_YAML}
   if [ $? != 0 ]; then
     echo "Error: failed to render the base catalog"
     exit 1
   fi
 fi
 
-${OPM} render --skip-tls -o yaml \
-  ${BUNDLE_IMAGE} > ${PACKAGE_YAML}
-if [ $? != 0 ]; then
-  echo "Error: failed to render the ${PACKAGE} bundle catalog"
-  exit 1
+if [ ! -f ${PACKAGE_YAML} ]; then
+  ${OPM} render --skip-tls -o yaml \
+    ${BUNDLE_IMAGE} > ${PACKAGE_YAML}
+  if [ $? != 0 ]; then
+    echo "Error: failed to render the ${PACKAGE} bundle catalog"
+    exit 1
+  fi
 fi
 
+#
+# Extract the camel-k channels
+#
+yq eval ". | select(.package == \"${PACKAGE}\" and .schema == \"olm.channel\")" ${INDEX_BASE_YAML} > ${CHANNELS_YAML}
+if [ $? != 0 ] || [ ! -f "${CHANNELS_YAML}" ]; then
+  echo "ERROR: Failed to extract camel-k entries from bundle catalog"
+  exit 1
+fi
 
+#
+# Filter out the channels in the bundles file
+#
+yq -i eval ". | select(.package != \"${PACKAGE}\" or .schema != \"olm.channel\")" ${INDEX_BASE_YAML}
+if [ $? != 0 ]; then
+  echo "ERROR: Failed to remove camel-k channel entries from bundles catalog"
+  exit 1
+fi
 
-cat << EOF >> ${PACKAGE_YAML}
----
-schema: olm.channel
-package: ${PACKAGE}
-name: ${CHANNEL}
-entries:
-  - name: ${CSV_NAME}
-EOF
+#
+# Split the channels and append/insert the bundle into each one
+#
+IFS=','
+#Read the split words into an array based on comma delimiter
+read -a CHANNEL_ARR <<< "${CHANNELS}"

Review Comment:
   Done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [camel-k] phantomjinx commented on pull request #3527: Improvements to OLM bundle building of test index

Posted by GitBox <gi...@apache.org>.
phantomjinx commented on PR #3527:
URL: https://github.com/apache/camel-k/pull/3527#issuecomment-1275964044

   Much of this PR seems to have been superceded and already added so changes here are fixes in response to review comments.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [camel-k] tadayosi commented on pull request #3527: Improvements to OLM bundle building of test index

Posted by GitBox <gi...@apache.org>.
tadayosi commented on PR #3527:
URL: https://github.com/apache/camel-k/pull/3527#issuecomment-1278446690

   The test is failing:
   ```
   ❌ TestKameletFromCustomRepository (10m10.77s)
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org