You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "jojochuang (via GitHub)" <gi...@apache.org> on 2023/06/26 21:42:47 UTC

[GitHub] [ozone] jojochuang opened a new pull request, #4985: HDDS-8911. Update Hadoop to 3.3.6

jojochuang opened a new pull request, #4985:
URL: https://github.com/apache/ozone/pull/4985

   ## What changes were proposed in this pull request?
   
   1. Update build hadoop.version dependency to 3.3.6
   2. Update tests to use Apache Hadoop 3.3.6 docker image.
   
   ## What is the link to the Apache JIRA
   
   https://issues.apache.org/jira/browse/HDDS-8911
   
   ## How was this patch tested?
   
   Existing tests. https://github.com/jojochuang/ozone/actions/runs/5382664472


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] adoroszlai commented on a diff in pull request #4985: HDDS-8911. Update Hadoop to 3.3.6

Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #4985:
URL: https://github.com/apache/ozone/pull/4985#discussion_r1243986727


##########
hadoop-ozone/dist/src/main/license/jar-report.txt:
##########
@@ -1,11 +1,10 @@
-share/ozone/lib/accessors-smart.jar
+share/ozone/lib/FastInfoset.jar

Review Comment:
   It sorts both old (reference) and new lists exactly to avoid false positives due to difference in `sort`:
   
   https://github.com/apache/ozone/blob/4559ef4956cc24eb3a049cf9664322b140e6853a/hadoop-ozone/dev-support/checks/dependency.sh#L31-L32
   
   So any order is fine as far as `dependency` check is concerned.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] adoroszlai commented on a diff in pull request #4985: HDDS-8911. Update Hadoop to 3.3.6

Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #4985:
URL: https://github.com/apache/ozone/pull/4985#discussion_r1243151852


##########
hadoop-ozone/dist/src/main/compose/ozonesecure-ha/.env:
##########
@@ -15,7 +15,7 @@
 # limitations under the License.
 
 HDDS_VERSION=${hdds.version}
-HADOOP_VERSION=3
+HADOOP_VERSION=3.3.6

Review Comment:
   Now that the Docker image has the same version as Hadoop itself, we can change this to avoid the need for future updates:
   
   ```suggestion
   HADOOP_VERSION=${hadoop.version}
   ```



##########
hadoop-ozone/dist/src/main/compose/ozonesecure/.env:
##########
@@ -15,7 +15,7 @@
 # limitations under the License.
 
 HDDS_VERSION=${hdds.version}
-HADOOP_VERSION=3
+HADOOP_VERSION=3.3.6

Review Comment:
   ```suggestion
   HADOOP_VERSION=${hadoop.version}
   ```



##########
hadoop-ozone/dist/src/main/license/jar-report.txt:
##########
@@ -1,11 +1,10 @@
-share/ozone/lib/accessors-smart.jar
+share/ozone/lib/FastInfoset.jar

Review Comment:
   Nit: Three kinds of changes to `jar-report.txt` are mixed in this PR.
   
   1. real dependency changes (see below)
   2. fixing strictly out of alphabetical order items (e.g. `hdds-rocks-native`)
   3. reordering due to different `sort` behavior (lowercase/uppercase, chars other than alphabetical) across implementations
   
   I would prefer to stick only to the first kind of change in this PR:
   
   ```diff
   @@ -1,11 +1,9 @@
   -share/ozone/lib/accessors-smart.jar
    share/ozone/lib/activation.jar
    share/ozone/lib/animal-sniffer-annotations.jar
    share/ozone/lib/annotations.jar
    share/ozone/lib/annotations.jar
    share/ozone/lib/aopalliance.jar
    share/ozone/lib/aopalliance-repackaged.jar
   -share/ozone/lib/asm.jar
    share/ozone/lib/aspectjrt.jar
    share/ozone/lib/aspectjweaver.jar
    share/ozone/lib/awaitility.jar
   @@ -156,7 +154,6 @@
    share/ozone/lib/jooq-meta.jar
    share/ozone/lib/jsch.jar
    share/ozone/lib/json-simple.jar
   -share/ozone/lib/json-smart.jar
    share/ozone/lib/jsp-api.jar
    share/ozone/lib/jsr305.jar
    share/ozone/lib/jsr311-api.jar
   ```
   
   Rationale: The changes are in transitive dependencies, visible only in this file (no POM change).  Reordering lines hides the real difference.
   
   Unfortunately `dependency` check instructs to update the file mention using `update-jar-report.sh`:
   
   ```
   Please update the hadoop-ozone/dist/src/main/license/bin/LICENSE.txt file with the modification
      AND execute hadoop-ozone/dist/src/main/license/update-jar-report.sh when you are ready (after a full build)
   ```
   
   which re-generates from scratch, causing such mixed changes.  To avoid those, one can simply manually remove the unnecessary entries.



##########
hadoop-ozone/dist/src/main/smoketest/.env:
##########
@@ -14,4 +14,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-HADOOP_VERSION=3
\ No newline at end of file
+HADOOP_VERSION=3.3.6

Review Comment:
   ```suggestion
   HADOOP_VERSION=${hadoop.version}
   ```



##########
hadoop-ozone/dist/src/main/compose/ozonesecure-mr/.env:
##########
@@ -16,7 +16,7 @@
 
 HDDS_VERSION=${hdds.version}
 HADOOP_IMAGE=apache/hadoop
-HADOOP_VERSION=3.3.5
+HADOOP_VERSION=3.3.6

Review Comment:
   ```suggestion
   HADOOP_VERSION=${hadoop.version}
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] adoroszlai commented on a diff in pull request #4985: HDDS-8911. Update Hadoop to 3.3.6

Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai commented on code in PR #4985:
URL: https://github.com/apache/ozone/pull/4985#discussion_r1243986727


##########
hadoop-ozone/dist/src/main/license/jar-report.txt:
##########
@@ -1,11 +1,10 @@
-share/ozone/lib/accessors-smart.jar
+share/ozone/lib/FastInfoset.jar

Review Comment:
   It sorts both old (reference) and new lists exactly to avoid false positives due to difference in `sort`:
   
   https://github.com/apache/ozone/blob/4559ef4956cc24eb3a049cf9664322b140e6853a/hadoop-ozone/dev-support/checks/dependency.sh#L31-L32



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] jojochuang commented on a diff in pull request #4985: HDDS-8911. Update Hadoop to 3.3.6

Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on code in PR #4985:
URL: https://github.com/apache/ozone/pull/4985#discussion_r1243917111


##########
hadoop-ozone/dist/src/main/license/jar-report.txt:
##########
@@ -1,11 +1,10 @@
-share/ozone/lib/accessors-smart.jar
+share/ozone/lib/FastInfoset.jar

Review Comment:
   Yeah... I wasn't sure how the jar report is generated. Let's see if it is happy with this.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org

[GitHub] [ozone] adoroszlai merged pull request #4985: HDDS-8911. Update Hadoop to 3.3.6

Posted by "adoroszlai (via GitHub)" <gi...@apache.org>.
adoroszlai merged PR #4985:
URL: https://github.com/apache/ozone/pull/4985


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org