You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2020/05/18 06:40:46 UTC
[GitHub] [hadoop] vinayakumarb opened a new pull request #2026: HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes
vinayakumarb opened a new pull request #2026:
URL: https://github.com/apache/hadoop/pull/2026
After upgrade/shade of protobuf to 3.7 version, existing Hadoop-Rpc client-server implementations using ProtobufRpcEngine will not work.
So, this Jira proposes to keep existing ProtobuRpcEngine as-is (without shading and with protobuf-2.5.0 implementation) to support downstream implementations.
Use new ProtobufRpcEngine2 to use shaded protobuf classes within Hadoop and later projects who wish to upgrade their protobufs to 3.x.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on a change in pull request #2026: HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes
Posted by GitBox <gi...@apache.org>.
aajisaka commented on a change in pull request #2026:
URL: https://github.com/apache/hadoop/pull/2026#discussion_r435150341
##########
File path: hadoop-common-project/hadoop-common/pom.xml
##########
@@ -1015,7 +1031,81 @@
</plugins>
</build>
</profile>
-
+ <!-- profile to use already generated protobuf code using 2.5.0 for aarch64-->
+ <profile>
+ <id>aarch64</id>
+ <activation>
+ <activeByDefault>false</activeByDefault>
+ <os>
+ <family>linux</family>
+ <arch>aarch64</arch>
+ </os>
+ </activation>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>build-helper-maven-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>add-source-legacy-protobuf</id>
+ <phase>generate-sources</phase>
+ <goals>
+ <goal>add-source</goal>
+ </goals>
+ <configuration>
+ <sources>
+ <source>${basedir}/src/main/arm-java</source>
+ </sources>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+ </profile>
+ <!-- profile to generate protobuf code using 2.5.0-->
+ <profile>
+ <id>x86_64</id>
+ <activation>
+ <activeByDefault>false</activeByDefault>
+ <os>
+ <family>linux</family>
+ <arch>amd64</arch>
Review comment:
The profile is not activated on Mac and compile fails.
```suggestion
<arch>!aarch64</arch>
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on a change in pull request #2026: HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes
Posted by GitBox <gi...@apache.org>.
aajisaka commented on a change in pull request #2026:
URL: https://github.com/apache/hadoop/pull/2026#discussion_r437242388
##########
File path: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java
##########
@@ -74,6 +75,11 @@
private static String taskName = "attempt_001_02_r03_04_05";
+ @After
+ public void after() throws Exception {
+ cleanTokenPasswordFile();
+ }
+
Review comment:
Big +1 for this change, but I think this issue can be fixed in a separate jira.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on a change in pull request #2026: HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes
Posted by GitBox <gi...@apache.org>.
aajisaka commented on a change in pull request #2026:
URL: https://github.com/apache/hadoop/pull/2026#discussion_r432300040
##########
File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java
##########
@@ -1295,6 +1296,22 @@ static URI trimUri(URI uri) {
*/
public static void addPBProtocol(Configuration conf, Class<?> protocol,
BlockingService service, RPC.Server server) throws IOException {
+ RPC.setProtocolEngine(conf, protocol, ProtobufRpcEngine2.class);
+ server.addProtocol(RPC.RpcKind.RPC_PROTOCOL_BUFFER, protocol, service);
+ }
+
+ /**
+ * Add protobuf based protocol to the {@link org.apache.hadoop.ipc.RPC.Server}
+ * @param conf configuration
+ * @param protocol Protocol interface
+ * @param service service that implements the protocol
+ * @param server RPC server to which the protocol & implementation is
+ * added to
+ * @throws IOException
+ */
+ @Deprecated
+ public static void addPBProtocol(Configuration conf, Class<?> protocol,
Review comment:
https://github.com/apache/hadoop-ozone/pull/987 is now available. If the PR is merged, I think we don't have to fix DFSUtil.java.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2026: HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2026:
URL: https://github.com/apache/hadoop/pull/2026#issuecomment-638512027
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 44s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +0 :ok: | shelldocs | 0m 1s | Shelldocs was not available. |
| +0 :ok: | prototool | 0m 0s | prototool was not available. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 23 new or modified test files. |
||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 0m 34s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 19s | trunk passed |
| +1 :green_heart: | compile | 17m 5s | trunk passed |
| +1 :green_heart: | checkstyle | 3m 19s | trunk passed |
| +1 :green_heart: | mvnsite | 17m 19s | trunk passed |
| +1 :green_heart: | shadedclient | 14m 1s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 5m 39s | trunk passed |
| +0 :ok: | spotbugs | 29m 21s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| -1 :x: | findbugs | 29m 16s | root in trunk has 3 extant findbugs warnings. |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 32s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 27m 40s | the patch passed |
| +1 :green_heart: | compile | 16m 41s | the patch passed |
| -1 :x: | cc | 16m 41s | root generated 31 new + 131 unchanged - 31 fixed = 162 total (was 162) |
| -1 :x: | javac | 16m 41s | root generated 1 new + 1857 unchanged - 0 fixed = 1858 total (was 1857) |
| -0 :warning: | checkstyle | 3m 26s | root: The patch generated 12 new + 1400 unchanged - 13 fixed = 1412 total (was 1413) |
| +1 :green_heart: | mvnsite | 17m 29s | the patch passed |
| +1 :green_heart: | shellcheck | 0m 1s | There were no new shellcheck issues. |
| -1 :x: | whitespace | 0m 1s | The patch has 103 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | xml | 0m 4s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedclient | 14m 13s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 5m 38s | the patch passed |
| +1 :green_heart: | findbugs | 48m 51s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 604m 55s | root in the patch passed. |
| -1 :x: | asflicense | 1m 52s | The patch generated 1 ASF License warnings. |
| | | 872m 53s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.TestGetFileChecksum |
| | hadoop.hdfs.server.namenode.TestAddStripedBlockInFBR |
| | hadoop.hdfs.server.datanode.TestBPOfferService |
| | hadoop.hdfs.TestReconstructStripedFile |
| | hadoop.hdfs.tools.TestDFSAdminWithHA |
| | hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy |
| | hadoop.tools.TestDistCpSync |
| | hadoop.tools.TestDistCpSystem |
| | hadoop.yarn.applications.distributedshell.TestDistributedShell |
| | hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2026 |
| Optional Tests | dupname asflicense shellcheck shelldocs xml compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc prototool |
| uname | Linux fe8dfdd17c03 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / e8cb2ae409b |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/3/artifact/out/branch-findbugs-root-warnings.html |
| cc | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/3/artifact/out/diff-compile-cc-root.txt |
| javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/3/artifact/out/diff-compile-javac-root.txt |
| checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/3/artifact/out/diff-checkstyle-root.txt |
| whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/3/artifact/out/whitespace-eol.txt |
| unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/3/artifact/out/patch-unit-root.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/3/testReport/ |
| asflicense | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/3/artifact/out/patch-asflicense-problems.txt |
| Max. process+thread count | 3839 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core . U: . |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/3/console |
| versions | git=2.17.1 maven=3.6.0 shellcheck=0.4.6 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on a change in pull request #2026: HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes
Posted by GitBox <gi...@apache.org>.
aajisaka commented on a change in pull request #2026:
URL: https://github.com/apache/hadoop/pull/2026#discussion_r435150587
##########
File path: hadoop-common-project/hadoop-common/pom.xml
##########
@@ -1015,7 +1031,81 @@
</plugins>
</build>
</profile>
-
+ <!-- profile to use already generated protobuf code using 2.5.0 for aarch64-->
+ <profile>
+ <id>aarch64</id>
+ <activation>
+ <activeByDefault>false</activeByDefault>
+ <os>
+ <family>linux</family>
Review comment:
`<family>linux</family>` can be removed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2026: HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2026:
URL: https://github.com/apache/hadoop/pull/2026#issuecomment-642272089
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 8s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +0 :ok: | prototool | 0m 0s | prototool was not available. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 22 new or modified test files. |
||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 1m 8s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 21m 37s | trunk passed |
| +1 :green_heart: | compile | 18m 18s | trunk passed |
| +1 :green_heart: | checkstyle | 3m 19s | trunk passed |
| +1 :green_heart: | mvnsite | 20m 19s | trunk passed |
| +1 :green_heart: | shadedclient | 16m 35s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 6m 42s | trunk passed |
| +0 :ok: | spotbugs | 39m 52s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| -1 :x: | findbugs | 39m 40s | root in trunk has 3 extant findbugs warnings. |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 34m 0s | the patch passed |
| +1 :green_heart: | compile | 21m 5s | the patch passed |
| -1 :x: | cc | 21m 5s | root generated 20 new + 142 unchanged - 20 fixed = 162 total (was 162) |
| -1 :x: | javac | 21m 5s | root generated 1 new + 1857 unchanged - 0 fixed = 1858 total (was 1857) |
| -0 :warning: | checkstyle | 3m 51s | root: The patch generated 2 new + 1369 unchanged - 13 fixed = 1371 total (was 1382) |
| +1 :green_heart: | mvnsite | 20m 54s | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | There were no new shellcheck issues. |
| +1 :green_heart: | shelldocs | 0m 15s | There were no new shelldocs issues. |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | xml | 0m 4s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedclient | 16m 32s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 6m 32s | the patch passed |
| +1 :green_heart: | findbugs | 62m 23s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 670m 34s | root in the patch passed. |
| -1 :x: | asflicense | 1m 47s | The patch generated 1 ASF License warnings. |
| | | 992m 57s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.mapreduce.v2.app.rm.TestRMCommunicator |
| | hadoop.yarn.server.timelineservice.storage.TestTimelineWriterHBaseDown |
| | hadoop.yarn.applications.distributedshell.TestDistributedShell |
| | hadoop.security.TestRaceWhenRelogin |
| | hadoop.hdfs.server.namenode.TestINodeFile |
| | hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA |
| | hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA |
| | hadoop.hdfs.TestDecommissionWithStriped |
| | hadoop.hdfs.server.namenode.TestAddStripedBlocks |
| | hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier |
| | hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints |
| | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| | hadoop.hdfs.server.balancer.TestBalancerRPCDelay |
| | hadoop.hdfs.TestReconstructStripedFile |
| | hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy |
| | hadoop.hdfs.TestSafeModeWithStripedFileWithRandomECPolicy |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/5/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2026 |
| Optional Tests | dupname asflicense shellcheck shelldocs xml compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc prototool |
| uname | Linux 80092813a2cb 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 56247db3022 |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/5/artifact/out/branch-findbugs-root-warnings.html |
| cc | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/5/artifact/out/diff-compile-cc-root.txt |
| javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/5/artifact/out/diff-compile-javac-root.txt |
| checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/5/artifact/out/diff-checkstyle-root.txt |
| unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/5/artifact/out/patch-unit-root.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/5/testReport/ |
| asflicense | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/5/artifact/out/patch-asflicense-problems.txt |
| Max. process+thread count | 3086 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core . U: . |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/5/console |
| versions | git=2.17.1 maven=3.6.0 shellcheck=0.4.6 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb closed pull request #2026: HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes
Posted by GitBox <gi...@apache.org>.
vinayakumarb closed pull request #2026:
URL: https://github.com/apache/hadoop/pull/2026
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on a change in pull request #2026: HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes
Posted by GitBox <gi...@apache.org>.
vinayakumarb commented on a change in pull request #2026:
URL: https://github.com/apache/hadoop/pull/2026#discussion_r434322138
##########
File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java
##########
@@ -1295,6 +1296,22 @@ static URI trimUri(URI uri) {
*/
public static void addPBProtocol(Configuration conf, Class<?> protocol,
BlockingService service, RPC.Server server) throws IOException {
+ RPC.setProtocolEngine(conf, protocol, ProtobufRpcEngine2.class);
+ server.addProtocol(RPC.RpcKind.RPC_PROTOCOL_BUFFER, protocol, service);
+ }
+
+ /**
+ * Add protobuf based protocol to the {@link org.apache.hadoop.ipc.RPC.Server}
+ * @param conf configuration
+ * @param protocol Protocol interface
+ * @param service service that implements the protocol
+ * @param server RPC server to which the protocol & implementation is
+ * added to
+ * @throws IOException
+ */
+ @Deprecated
+ public static void addPBProtocol(Configuration conf, Class<?> protocol,
Review comment:
But, I feel, we can still keep the method with deprecated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb merged pull request #2026: HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes
Posted by GitBox <gi...@apache.org>.
vinayakumarb merged pull request #2026:
URL: https://github.com/apache/hadoop/pull/2026
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on a change in pull request #2026: HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes
Posted by GitBox <gi...@apache.org>.
vinayakumarb commented on a change in pull request #2026:
URL: https://github.com/apache/hadoop/pull/2026#discussion_r436449548
##########
File path: hadoop-common-project/hadoop-common/pom.xml
##########
@@ -1015,7 +1031,81 @@
</plugins>
</build>
</profile>
-
+ <!-- profile to use already generated protobuf code using 2.5.0 for aarch64-->
+ <profile>
+ <id>aarch64</id>
+ <activation>
+ <activeByDefault>false</activeByDefault>
+ <os>
+ <family>linux</family>
Review comment:
sure
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on a change in pull request #2026: HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes
Posted by GitBox <gi...@apache.org>.
vinayakumarb commented on a change in pull request #2026:
URL: https://github.com/apache/hadoop/pull/2026#discussion_r434320866
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProtobufRpcEngineCallback.java
##########
@@ -18,7 +18,7 @@
package org.apache.hadoop.ipc;
-import org.apache.hadoop.thirdparty.protobuf.Message;
+import com.google.protobuf.Message;
public interface ProtobufRpcEngineCallback {
Review comment:
sure
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProtobufRpcEngine.java
##########
@@ -433,7 +438,15 @@ public Server(Class<?> protocolClass, Object protocolImpl,
registerProtocolAndImpl(RPC.RpcKind.RPC_PROTOCOL_BUFFER, protocolClass,
protocolImpl);
}
-
+
+ @Override
+ protected RpcInvoker getServerRpcInvoker(RpcKind rpcKind) {
+ if (rpcKind == RpcKind.RPC_PROTOCOL_BUFFER) {
+ return RPC_INVOKER;
+ }
+ return super.getServerRpcInvoker(rpcKind);
+ }
+
Review comment:
yes
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on pull request #2026: HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes
Posted by GitBox <gi...@apache.org>.
vinayakumarb commented on pull request #2026:
URL: https://github.com/apache/hadoop/pull/2026#issuecomment-641715517
I think, by mistake I had closed this PR, while replying to comment. Not able to re-open.
Will raise a new PR for this. Please review.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on pull request #2026: HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes
Posted by GitBox <gi...@apache.org>.
aajisaka commented on pull request #2026:
URL: https://github.com/apache/hadoop/pull/2026#issuecomment-641193896
Now I'm +1. Hi @jojochuang could you review this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2026: HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2026:
URL: https://github.com/apache/hadoop/pull/2026#issuecomment-631039936
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 8s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +0 :ok: | shelldocs | 0m 2s | Shelldocs was not available. |
| +0 :ok: | prototool | 0m 0s | prototool was not available. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 22 new or modified test files. |
||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 1m 0s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 21m 52s | trunk passed |
| +1 :green_heart: | compile | 18m 11s | trunk passed |
| +1 :green_heart: | checkstyle | 3m 23s | trunk passed |
| +1 :green_heart: | mvnsite | 18m 4s | trunk passed |
| +1 :green_heart: | shadedclient | 15m 22s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 5m 33s | trunk passed |
| +0 :ok: | spotbugs | 32m 24s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| -1 :x: | findbugs | 1m 40s | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common in trunk has 1 extant findbugs warnings. |
| -1 :x: | findbugs | 32m 19s | root in trunk has 4 extant findbugs warnings. |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 28m 51s | the patch passed |
| +1 :green_heart: | compile | 17m 40s | the patch passed |
| -1 :x: | cc | 17m 40s | root generated 25 new + 137 unchanged - 25 fixed = 162 total (was 162) |
| +1 :green_heart: | javac | 17m 40s | the patch passed |
| -0 :warning: | checkstyle | 3m 19s | root: The patch generated 39 new + 1370 unchanged - 12 fixed = 1409 total (was 1382) |
| +1 :green_heart: | mvnsite | 18m 6s | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | There were no new shellcheck issues. |
| -1 :x: | whitespace | 0m 0s | The patch has 9 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | xml | 0m 3s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedclient | 15m 26s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 5m 37s | the patch passed |
| -1 :x: | findbugs | 2m 12s | hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) |
| -1 :x: | findbugs | 32m 35s | root generated 1 new + 4 unchanged - 0 fixed = 5 total (was 4) |
||| _ Other Tests _ |
| -1 :x: | unit | 612m 34s | root in the patch passed. |
| -1 :x: | asflicense | 1m 21s | The patch generated 1 ASF License warnings. |
| | | 894m 3s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:hadoop-common-project/hadoop-common |
| | Useless control flow in org.apache.hadoop.ipc.protobuf.ProtobufRpcEngine2Protos$RequestHeaderProto$Builder.maybeForceBuilderInitialization() At ProtobufRpcEngine2Protos.java: At ProtobufRpcEngine2Protos.java:[line 643] |
| FindBugs | module:root |
| | Useless control flow in org.apache.hadoop.ipc.protobuf.ProtobufRpcEngine2Protos$RequestHeaderProto$Builder.maybeForceBuilderInitialization() At ProtobufRpcEngine2Protos.java: At ProtobufRpcEngine2Protos.java:[line 643] |
| Failed junit tests | hadoop.yarn.applications.distributedshell.TestDistributedShell |
| | hadoop.security.TestRaceWhenRelogin |
| | hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure |
| | hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes |
| | hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics |
| | hadoop.hdfs.TestRollingUpgrade |
| | hadoop.fs.http.client.TestHttpFSFWithSWebhdfsFileSystem |
| | hadoop.fs.azurebfs.services.TestAbfsClientThrottlingAnalyzer |
| | hadoop.yarn.sls.TestSLSRunner |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2026 |
| Optional Tests | dupname asflicense shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle cc prototool |
| uname | Linux bd63306be0e9 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / bdbd59cfa09 |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-common-warnings.html |
| findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/artifact/out/branch-findbugs-root-warnings.html |
| cc | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/artifact/out/diff-compile-cc-root.txt |
| checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/artifact/out/diff-checkstyle-root.txt |
| whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/artifact/out/whitespace-eol.txt |
| findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/artifact/out/new-findbugs-hadoop-common-project_hadoop-common.html |
| findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/artifact/out/new-findbugs-root.html |
| unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/artifact/out/patch-unit-root.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/testReport/ |
| asflicense | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/artifact/out/patch-asflicense-problems.txt |
| Max. process+thread count | 2861 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core . U: . |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/2/console |
| versions | git=2.17.1 maven=3.6.0 shellcheck=0.4.6 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on a change in pull request #2026: HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes
Posted by GitBox <gi...@apache.org>.
vinayakumarb commented on a change in pull request #2026:
URL: https://github.com/apache/hadoop/pull/2026#discussion_r436451465
##########
File path: hadoop-common-project/hadoop-common/pom.xml
##########
@@ -1015,7 +1031,81 @@
</plugins>
</build>
</profile>
-
+ <!-- profile to use already generated protobuf code using 2.5.0 for aarch64-->
+ <profile>
+ <id>aarch64</id>
+ <activation>
+ <activeByDefault>false</activeByDefault>
+ <os>
+ <family>linux</family>
+ <arch>aarch64</arch>
+ </os>
+ </activation>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>build-helper-maven-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>add-source-legacy-protobuf</id>
+ <phase>generate-sources</phase>
+ <goals>
+ <goal>add-source</goal>
+ </goals>
+ <configuration>
+ <sources>
+ <source>${basedir}/src/main/arm-java</source>
+ </sources>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+ </profile>
+ <!-- profile to generate protobuf code using 2.5.0-->
+ <profile>
+ <id>x86_64</id>
+ <activation>
+ <activeByDefault>false</activeByDefault>
+ <os>
+ <family>linux</family>
+ <arch>amd64</arch>
Review comment:
Updated the same. Thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on a change in pull request #2026: HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes
Posted by GitBox <gi...@apache.org>.
vinayakumarb commented on a change in pull request #2026:
URL: https://github.com/apache/hadoop/pull/2026#discussion_r437853175
##########
File path: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java
##########
@@ -74,6 +75,11 @@
private static String taskName = "attempt_001_02_r03_04_05";
+ @After
+ public void after() throws Exception {
+ cleanTokenPasswordFile();
+ }
+
Review comment:
Pushed the latest changes without this.
Please check
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on a change in pull request #2026: HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes
Posted by GitBox <gi...@apache.org>.
vinayakumarb commented on a change in pull request #2026:
URL: https://github.com/apache/hadoop/pull/2026#discussion_r434410613
##########
File path: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProtobufRpcEngine.java
##########
@@ -433,7 +438,15 @@ public Server(Class<?> protocolClass, Object protocolImpl,
registerProtocolAndImpl(RPC.RpcKind.RPC_PROTOCOL_BUFFER, protocolClass,
protocolImpl);
}
-
+
+ @Override
+ protected RpcInvoker getServerRpcInvoker(RpcKind rpcKind) {
+ if (rpcKind == RpcKind.RPC_PROTOCOL_BUFFER) {
+ return RPC_INVOKER;
+ }
+ return super.getServerRpcInvoker(rpcKind);
+ }
+
Review comment:
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #2026: HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes
Posted by GitBox <gi...@apache.org>.
steveloughran commented on pull request #2026:
URL: https://github.com/apache/hadoop/pull/2026#issuecomment-630928501
way beyond my area of expertise, i'm afraid
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2026: HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2026:
URL: https://github.com/apache/hadoop/pull/2026#issuecomment-643405454
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 0s | Docker mode activated. |
| -1 :x: | patch | 0m 5s | https://github.com/apache/hadoop/pull/2026 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. |
| Subsystem | Report/Notes |
|----------:|:-------------|
| GITHUB PR | https://github.com/apache/hadoop/pull/2026 |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/6/console |
| versions | git=2.17.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on pull request #2026: HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes
Posted by GitBox <gi...@apache.org>.
vinayakumarb commented on pull request #2026:
URL: https://github.com/apache/hadoop/pull/2026#issuecomment-641717069
> I think, by mistake I had closed this PR, while replying to comment. Not able to re-open.
> Will raise a new PR for this. Please review.
Had pushed trunk itself without mychanges to source branch, so PR got closed.
Now its back.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2026: HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes
Posted by GitBox <gi...@apache.org>.
hadoop-yetus commented on pull request #2026:
URL: https://github.com/apache/hadoop/pull/2026#issuecomment-640525296
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 27m 53s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. |
| +0 :ok: | prototool | 0m 0s | prototool was not available. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 23 new or modified test files. |
||| _ trunk Compile Tests _ |
| +0 :ok: | mvndep | 1m 14s | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 24m 20s | trunk passed |
| +1 :green_heart: | compile | 21m 36s | trunk passed |
| +1 :green_heart: | checkstyle | 3m 39s | trunk passed |
| +1 :green_heart: | mvnsite | 21m 12s | trunk passed |
| +1 :green_heart: | shadedclient | 16m 25s | branch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 6m 52s | trunk passed |
| +0 :ok: | spotbugs | 39m 7s | Used deprecated FindBugs config; considering switching to SpotBugs. |
| -1 :x: | findbugs | 39m 1s | root in trunk has 3 extant findbugs warnings. |
||| _ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 33m 23s | the patch passed |
| +1 :green_heart: | compile | 20m 42s | the patch passed |
| -1 :x: | cc | 20m 42s | root generated 4 new + 158 unchanged - 4 fixed = 162 total (was 162) |
| -1 :x: | javac | 20m 42s | root generated 1 new + 1857 unchanged - 0 fixed = 1858 total (was 1857) |
| -0 :warning: | checkstyle | 3m 36s | root: The patch generated 2 new + 1401 unchanged - 13 fixed = 1403 total (was 1414) |
| +1 :green_heart: | mvnsite | 21m 55s | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | There were no new shellcheck issues. |
| +1 :green_heart: | shelldocs | 0m 12s | There were no new shelldocs issues. |
| -1 :x: | whitespace | 0m 0s | The patch has 103 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | xml | 0m 4s | The patch has no ill-formed XML file. |
| +1 :green_heart: | shadedclient | 16m 56s | patch has no errors when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 6m 40s | the patch passed |
| +1 :green_heart: | findbugs | 63m 44s | the patch passed |
||| _ Other Tests _ |
| -1 :x: | unit | 19m 32s | root in the patch failed. |
| -1 :x: | asflicense | 1m 3s | The patch generated 1 ASF License warnings. |
| | | 377m 0s | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.security.TestRaceWhenRelogin |
| | hadoop.util.TestDataChecksum |
| | hadoop.util.curator.TestChildReaper |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/4/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/2026 |
| Optional Tests | dupname asflicense shellcheck shelldocs xml compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle cc prototool |
| uname | Linux 4437dd23e314 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / a8610c15c49 |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/4/artifact/out/branch-findbugs-root-warnings.html |
| cc | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/4/artifact/out/diff-compile-cc-root.txt |
| javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/4/artifact/out/diff-compile-javac-root.txt |
| checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/4/artifact/out/diff-checkstyle-root.txt |
| whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/4/artifact/out/whitespace-eol.txt |
| unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/4/artifact/out/patch-unit-root.txt |
| Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/4/testReport/ |
| asflicense | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/4/artifact/out/patch-asflicense-problems.txt |
| Max. process+thread count | 3204 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core . U: . |
| Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2026/4/console |
| versions | git=2.17.1 maven=3.6.0 shellcheck=0.4.6 findbugs=3.1.0-RC1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
[GitHub] [hadoop] vinayakumarb commented on pull request #2026: HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes
Posted by GitBox <gi...@apache.org>.
vinayakumarb commented on pull request #2026:
URL: https://github.com/apache/hadoop/pull/2026#issuecomment-638060134
Updated the PR with review comment fixes. + below changes
`ProtobufRpcEngineProto.java` is generated using protobuf 2.5.0, which does not available for arm. So I have added the generated `ProtobufRpcEngineProto.java` in a separare source directory in `hadoop-common\src\main\arm-java` and adding this to sources in case of arm. But Re-generating using protobuf-maven-plugin in case of x86 platform as before.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org