You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "jojochuang (via GitHub)" <gi...@apache.org> on 2023/06/22 20:23:16 UTC

[GitHub] [ozone] jojochuang opened a new pull request, #4963: HDDS-8825. Use apache/hadoop 3.3.5 docker image

jojochuang opened a new pull request, #4963:
URL: https://github.com/apache/ozone/pull/4963

   ## What changes were proposed in this pull request?
   
   Our tests depend on flokkr/hadoop which has stopped developing. We should switch to apache/hadoop docker images which has the latest Hadoop 3.3.5 docker image. For the tests that rely on older Hadoop versions, keep using flokkr/hadoop because apache/hadoop currently only maintain 3.3.x images.
   
   ## What is the link to the Apache JIRA
   
   https://issues.apache.org/jira/browse/HDDS-8825
   
   ## How was this patch tested?
   
   Existing tests


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] smengcl commented on a diff in pull request #4963: HDDS-8825. Use apache/hadoop 3.3.5 docker image

Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on code in PR #4963:
URL: https://github.com/apache/ozone/pull/4963#discussion_r1240103394


##########
hadoop-ozone/dist/src/main/compose/common/hadoop-test.sh:
##########
@@ -33,9 +33,15 @@ export OZONE_DIR=/opt/ozone
 # shellcheck source=/dev/null
 source "$COMPOSE_DIR/../testlib.sh"
 
-for HADOOP_VERSION in 2.7.3 3.1.2 3.2.2 3.3.1; do
+for HADOOP_VERSION in 2.7.3 3.1.2 3.2.2 3.3.5; do
   export HADOOP_VERSION
   export HADOOP_MAJOR_VERSION=${HADOOP_VERSION%%.*}
+  # Check if $HADOOP_VERSION starts with the prefix "3.3."
+  if [[ $HADOOP_VERSION == 3.3.* ]]; then

Review Comment:
   Yup I'm alright with the current approach. Thanks @jojochuang @adoroszlai 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] smengcl commented on a diff in pull request #4963: HDDS-8825. Use apache/hadoop 3.3.5 docker image

Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on code in PR #4963:
URL: https://github.com/apache/ozone/pull/4963#discussion_r1239313559


##########
hadoop-ozone/dist/src/main/compose/common/hadoop-test.sh:
##########
@@ -33,9 +33,15 @@ export OZONE_DIR=/opt/ozone
 # shellcheck source=/dev/null
 source "$COMPOSE_DIR/../testlib.sh"
 
-for HADOOP_VERSION in 2.7.3 3.1.2 3.2.2 3.3.1; do
+for HADOOP_VERSION in 2.7.3 3.1.2 3.2.2 3.3.5; do
   export HADOOP_VERSION
   export HADOOP_MAJOR_VERSION=${HADOOP_VERSION%%.*}
+  # Check if $HADOOP_VERSION starts with the prefix "3.3."
+  if [[ $HADOOP_VERSION == 3.3.* ]]; then

Review Comment:
   Is there a relatively easy way to make this `$HADOOP_VERSION >= 3.3.*` ?
   
   If not it should work at least for now, until Hadoop 3.4.0 is released.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] adoroszlai commented on a diff in pull request #4963: HDDS-8825. Use apache/hadoop 3.3.5 docker image

Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #4963:
URL: https://github.com/apache/ozone/pull/4963#discussion_r1239813425


##########
hadoop-ozone/dist/src/main/compose/common/hadoop-test.sh:
##########
@@ -33,9 +33,15 @@ export OZONE_DIR=/opt/ozone
 # shellcheck source=/dev/null
 source "$COMPOSE_DIR/../testlib.sh"
 
-for HADOOP_VERSION in 2.7.3 3.1.2 3.2.2 3.3.1; do
+for HADOOP_VERSION in 2.7.3 3.1.2 3.2.2 3.3.5; do
   export HADOOP_VERSION
   export HADOOP_MAJOR_VERSION=${HADOOP_VERSION%%.*}
+  # Check if $HADOOP_VERSION starts with the prefix "3.3."
+  if [[ $HADOOP_VERSION == 3.3.* ]]; then

Review Comment:
   > it should work at least for now, until Hadoop 3.4.0 is released
   
   Rather until we start testing with the new docker image after Hadoop 3.4.0 release.  For that, we'll need to change this test script anyway, so I guess it's OK to defer until then.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] adoroszlai merged pull request #4963: HDDS-8825. Use apache/hadoop 3.3.5 docker image

Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai merged PR #4963:
URL: https://github.com/apache/ozone/pull/4963


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] adoroszlai commented on pull request #4963: HDDS-8825. Use apache/hadoop 3.3.5 docker image

Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on PR #4963:
URL: https://github.com/apache/ozone/pull/4963#issuecomment-1604280822

   Thanks @jojochuang for the patch, @smengcl for the review.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] jojochuang commented on a diff in pull request #4963: HDDS-8825. Use apache/hadoop 3.3.5 docker image

Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on code in PR #4963:
URL: https://github.com/apache/ozone/pull/4963#discussion_r1239455479


##########
hadoop-ozone/dist/src/main/compose/common/hadoop-test.sh:
##########
@@ -33,9 +33,15 @@ export OZONE_DIR=/opt/ozone
 # shellcheck source=/dev/null
 source "$COMPOSE_DIR/../testlib.sh"
 
-for HADOOP_VERSION in 2.7.3 3.1.2 3.2.2 3.3.1; do
+for HADOOP_VERSION in 2.7.3 3.1.2 3.2.2 3.3.5; do
   export HADOOP_VERSION
   export HADOOP_MAJOR_VERSION=${HADOOP_VERSION%%.*}
+  # Check if $HADOOP_VERSION starts with the prefix "3.3."
+  if [[ $HADOOP_VERSION == 3.3.* ]]; then

Review Comment:
   This is exactly what ChatGPT is for! I asked ChatGPT and it answered:
   
   
   ```
   #!/bin/bash
   
   # Extracting X, Y, and Z from HADOOP_VERSION
   IFS='.' read -r -a version_parts <<< "$HADOOP_VERSION"
   X=${version_parts[0]}
   Y=${version_parts[1]}
   Z=${version_parts[2]}
   
   # Checking the condition and printing "hello world" if met
   if [[ $X -eq 3 && $Y -ge 3 ]]; then
     echo "hello world"
   fi
   
   
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org