You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "ayushtkn (via GitHub)" <gi...@apache.org> on 2024/09/18 07:38:53 UTC
[PR] HADOOP-19165. Drop protobuf 2.5.0 from the distribution [hadoop]
ayushtkn opened a new pull request, #7051:
URL: https://github.com/apache/hadoop/pull/7051
### Description of PR
Drop protobuf 2.5.0 from the distribution, Hadoop uses the one from hadoop-thirdparty, we ideally don't need it.
### How was this patch tested?
Manually
**1. The contents of the distro:**
**Before:**
```
ayushsaxena@ayushsaxena share % ls -l -R . | grep protobuf
-rw-r--r--@ 1 ayushsaxena staff 1952718 Sep 18 10:54 hadoop-shaded-protobuf_3_25-1.3.0.jar
-rw-r--r--@ 1 ayushsaxena staff 1952718 Sep 18 10:56 hadoop-shaded-protobuf_3_25-1.3.0.jar
-rw-r--r--@ 1 ayushsaxena staff 5105 Sep 18 10:16 grpc-protobuf-1.53.0.jar
-rw-r--r--@ 1 ayushsaxena staff 7555 Sep 18 10:16 grpc-protobuf-lite-1.53.0.jar
-rw-r--r--@ 1 ayushsaxena staff 1875927 Sep 18 10:16 hbase-shaded-protobuf-4.1.5.jar
-rw-r--r--@ 1 ayushsaxena staff 533455 Sep 18 10:16 protobuf-java-2.5.0.jar
```
**Now:**
```
ayushsaxena@ayushsaxena share % ls -l -R . | grep protobuf
-rw-r--r--@ 1 ayushsaxena staff 1952718 Sep 18 12:43 hadoop-shaded-protobuf_3_25-1.3.0.jar
-rw-r--r--@ 1 ayushsaxena staff 1952718 Sep 18 12:44 hadoop-shaded-protobuf_3_25-1.3.0.jar
-rw-r--r--@ 1 ayushsaxena staff 5105 Sep 18 12:48 grpc-protobuf-1.53.0.jar
-rw-r--r--@ 1 ayushsaxena staff 7555 Sep 18 12:48 grpc-protobuf-lite-1.53.0.jar
-rw-r--r--@ 1 ayushsaxena staff 1875927 Sep 18 12:48 hbase-shaded-protobuf-4.1.5.jar
```
No protbuf-java-2.5.0 now in the distro
**2. Maven dependency:tree**
**Before:**
```
ayushsaxena@ayushsaxena hadoop % mvn dependency:tree | grep protobuf-java:jar:2.5.0:compile
[INFO] | +- com.google.protobuf:protobuf-java:jar:2.5.0:compile
[INFO] | +- com.google.protobuf:protobuf-java:jar:2.5.0:compile
[INFO] | +- com.google.protobuf:protobuf-java:jar:2.5.0:compile
[INFO] | | +- com.google.protobuf:protobuf-java:jar:2.5.0:compile
ayushsaxena@ayushsaxena hadoop %
```
**Now:**
```
ayushsaxena@ayushsaxena hadoop % mvn dependency:tree | grep protobuf-java:jar:2.5.0:compile
ayushsaxena@ayushsaxena hadoop %
```
Ran some UT selectively, Waiting for the CI to report any errors, if any
### For code changes:
- [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19165. Drop protobuf 2.5.0 from the distribution [hadoop]
Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on PR #7051:
URL: https://github.com/apache/hadoop/pull/7051#issuecomment-2370199206
According to HBASE-27436 and HBASE-27436, protobuf 2.5 can be purged if Hadoop does not use the HBase co-processor feature.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19165. Drop protobuf 2.5.0 from the distribution [hadoop]
Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on PR #7051:
URL: https://github.com/apache/hadoop/pull/7051#issuecomment-2364352795
Hasn't #6629 done this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19165. Drop protobuf 2.5.0 from the distribution [hadoop]
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn commented on PR #7051:
URL: https://github.com/apache/hadoop/pull/7051#issuecomment-2364363790
Hbase-2 still declares protobuf-2.5.0, the original ticket has details
https://issues.apache.org/jira/browse/HADOOP-19107?focusedCommentId=17843349&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17843349
I couldn't figure out why HBase still defines it, but since you were mentioning to drop protobuf-2.5, so I thought of giving it a try if it doesn't break anything...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19165. Drop protobuf 2.5.0 from the distribution [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #7051:
URL: https://github.com/apache/hadoop/pull/7051#issuecomment-2357889146
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 20s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 42s | | trunk passed |
| +1 :green_heart: | compile | 0m 17s | | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | compile | 0m 16s | | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | mvnsite | 0m 19s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 20s | | trunk passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javadoc | 0m 17s | | trunk passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | shadedclient | 53m 47s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 10s | | the patch passed |
| +1 :green_heart: | compile | 0m 10s | | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javac | 0m 10s | | the patch passed |
| +1 :green_heart: | compile | 0m 10s | | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | javac | 0m 10s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | mvnsite | 0m 11s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 10s | | the patch passed with JDK Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 |
| +1 :green_heart: | javadoc | 0m 10s | | the patch passed with JDK Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| +1 :green_heart: | shadedclient | 20m 20s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 13s | | hadoop-project in the patch passed. |
| +1 :green_heart: | asflicense | 0m 25s | | The patch does not generate ASF License warnings. |
| | | 76m 55s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7051/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/7051 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint |
| uname | Linux 400320336ae4 5.15.0-117-generic #127-Ubuntu SMP Fri Jul 5 20:13:28 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 59c5539a26df0b7c3fb57b4c9abccee04b029142 |
| Default Java | Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.24+8-post-Ubuntu-1ubuntu320.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_422-8u422-b05-1~20.04-b05 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7051/1/testReport/ |
| Max. process+thread count | 561 (vs. ulimit of 5500) |
| modules | C: hadoop-project U: hadoop-project |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7051/1/console |
| versions | git=2.25.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19165. Drop protobuf 2.5.0 from the distribution [hadoop]
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn merged PR #7051:
URL: https://github.com/apache/hadoop/pull/7051
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org