You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "slfan1989 (via GitHub)" <gi...@apache.org> on 2024/02/08 03:19:59 UTC
[PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.2. [hadoop]
slfan1989 opened a new pull request, #6537:
URL: https://github.com/apache/hadoop/pull/6537
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g., 'HADOOP-17799. Your PR title ...'.
-->
### Description of PR
JIRA: HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.2.
### How was this patch tested?
### For code changes:
- [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. [hadoop]
Posted by "slfan1989 (via GitHub)" <gi...@apache.org>.
slfan1989 commented on PR #6537:
URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1936744038
> +1 for the change on trunk/3.4; let's see what surprises surface...we only need to worry about build time issues, not production ones.
I will submit a pull request to trunk3.4 and hopefully everything goes smoothly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.2. [hadoop]
Posted by "slfan1989 (via GitHub)" <gi...@apache.org>.
slfan1989 commented on PR #6537:
URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1935076896
@sjlee @steveloughran Thank you for your attention to this PR! When I chose this version, I checked the dependencies of the Spark and Flink projects. As of now, Spark is using 3.1.2, and Flink is using 3.2.2, so I opted for 3.2.2. However, we can consider trying 3.2.5. I will proceed to test it immediately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. [hadoop]
Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on PR #6537:
URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1956544980
This is not good.
But looking at the failures I don't know whether to categorise as "test runner regression" or "brittle tests failing under new test runner".
Here are some of the ones I've looked at
`TestDirectoryScanner.testThrottling`
This test is measuring how long things took. it is way too brittle against timing changes, both slower and faster.
```
java.lang.AssertionError: Throttle is too permissive
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.assertTrue(Assert.java:42)
at org.apache.hadoop.hdfs.server.datanode.TestDirectoryScanner.testThrottling(TestDirectoryScanner.java:901)
```
I think the step here is to move to assertj so asserts fail with meaningful messages, see if the failure can be understood. Ideally you'd want a test which doesn't measure elapsed time, but instead uses counters in the code (here: of throttle events) to assert what took place.
Test` TestBlockListAsLongs.testFuzz`
See this painfully often else where -it means that the protobuf lib was built with a more recent version of java8 than the early oracle ones. Its fixable in your own build (use the older one) or cast ByteBuffer to Buffer. otherwise we need to make sure tests are on a more recent build.
```
java.lang.NoSuchMethodError: java.nio.ByteBuffer.position(I)Ljava/nio/ByteBuffer;
at org.apache.hadoop.thirdparty.protobuf.IterableByteBufferInputStream.read(IterableByteBufferInputStream.java:143)
at org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.read(CodedInputStream.java:2080)
at org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.tryRefillBuffer(CodedInputStream.java:2831)
at org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.refillBuffer(CodedInputStream.java:2777)
at org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.readRawByte(CodedInputStream.java:2859)
at org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.readRawVarint64SlowPath(CodedInputStream.java:2648)
at org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.readRawVarint64(CodedInputStream.java:2641)
at org.apache.hadoop.thirdparty.protobuf.CodedInputStream$StreamDecoder.readSInt64(CodedInputStream.java:2497)
at org.apache.hadoop.hdfs.protocol.BlockListAsLongs$BufferDecoder$1.next(BlockListAsLongs.java:419)
at org.apache.hadoop.hdfs.protocol.BlockListAsLongs$BufferDecoder$1.next(BlockListAsLongs.java:397)
at org.apache.hadoop.hdfs.protocol.BlockListAsLongs$BufferDecoder.getBlockListAsLongs(BlockListAsLongs.java:375)
at org.apache.hadoop.hdfs.protocol.TestBlockListAsLongs.checkReport(TestBlockListAsLongs.java:156)
at org.apache.hadoop.hdfs.protocol.TestBlockListAsLongs.testFuzz(TestBlockListAsLongs.java:139)
```
test `TestDFSAdmin.testDecommissionDataNodesReconfig`
```
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:87)
at org.junit.Assert.assertTrue(Assert.java:42)
at org.junit.Assert.assertTrue(Assert.java:53)
at org.apache.hadoop.hdfs.tools.TestDFSAdmin.testDecommissionDataNodesReconfig(TestDFSAdmin.java:1356)
```
not a very meaningful message. suspect that a different ordering of the threads is causing the assert to fail.
1. move to AssertJ
2. analyse error, see what the fix is.
Test `TestCacheDirectives`.
```
at org.apache.hadoop.test.GenericTestUtils.waitFor(GenericTestUtils.java:403)
at org.apache.hadoop.test.GenericTestUtils.waitFor(GenericTestUtils.java:362)
at org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives.waitForCachedBlocks(TestCacheDirectives.java:760)
at org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives.teardown(TestCacheDirectives.java:173)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
```
this is a timeout during teardown; after this subsequent tests are possibly going to fail. No obvious cause, though again I'd suspect race conditions.
Rather than say "hey, let's revert", I'd propose a "surefire update triggers test failures" and see what can be done about addressing them. because we can't stay frozen with surefire versions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. [hadoop]
Posted by "slfan1989 (via GitHub)" <gi...@apache.org>.
slfan1989 commented on PR #6537:
URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1960784052
@ayushtkn @tasanuma @steveloughran
I don't want to roll back this change. I would like to continue pushing for this upgrade. I have noticed several pull requests that seem to compile inaccurately.
Nevertheless, I have decided to roll back this change for now to minimize the impact on other members.
I'm working on fixing HADOOP-15984 and I'll follow up later with HADOOP-19071.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. [hadoop]
Posted by "steveloughran (via GitHub)" <gi...@apache.org>.
steveloughran commented on PR #6537:
URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1961569379
pity. I do think those failures need to be addressed, as they are less test runner problems than brittle tests *which will need to be fixed eventually*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. [hadoop]
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn commented on PR #6537:
URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1956562170
& some are failing due to protobuf upgrade as part of hadoop thirdparty upgrade, I am checking all the HDFS tests, but little slowly
https://issues.apache.org/jira/browse/HADOOP-18197?focusedCommentId=17818651&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17818651
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. [hadoop]
Posted by "slfan1989 (via GitHub)" <gi...@apache.org>.
slfan1989 commented on PR #6537:
URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1937354924
@steveloughran Thank you for reviewing the code! I will merge this PR into the trunk branch and continue to progress with the work to support Java 17 compilation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.2. [hadoop]
Posted by "slfan1989 (via GitHub)" <gi...@apache.org>.
slfan1989 commented on PR #6537:
URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1933872180
@steveloughran Can you help review this PR? Thank you very much! It looks like we can directly upgrade maven-surefire-plugin to 3.2.2.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. [hadoop]
Posted by "tasanuma (via GitHub)" <gi...@apache.org>.
tasanuma commented on PR #6537:
URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1959698960
There is another problem. After this PR, running any unit test using `mvn test` from the command line has always resulted in a failure.
```
$ mvn test -Dtest=TestCacheDirectives
...
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.2.5:test (default-test) on project hadoop-annotations: No tests matching pattern "TestCacheDirectives" were executed! (Set -Dsurefire.failIfNoSpecifiedTests=false to ignore this error.) -> [Help 1]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.2. [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #6537:
URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1933340199
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 20s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. |
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 8s | | trunk passed |
| +1 :green_heart: | compile | 0m 14s | | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 14s | | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | mvnsite | 0m 16s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 19s | | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 15s | | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | shadedclient | 53m 16s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 9s | | the patch passed |
| +1 :green_heart: | compile | 0m 8s | | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 8s | | the patch passed |
| +1 :green_heart: | compile | 0m 10s | | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 0m 10s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | mvnsite | 0m 11s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 9s | | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 11s | | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| -1 :x: | shadedclient | 1m 34s | | patch has errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 9s | | hadoop-project in the patch passed. |
| +1 :green_heart: | asflicense | 0m 19s | | The patch does not generate ASF License warnings. |
| | | 59m 11s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6537/1/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/6537 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint |
| uname | Linux bf22f2dfefeb 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 9d0c44e21b0b96fba00bda5aa84a71bc9d00d9dd |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6537/1/testReport/ |
| Max. process+thread count | 551 (vs. ulimit of 5500) |
| modules | C: hadoop-project U: hadoop-project |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6537/1/console |
| versions | git=2.25.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.2. [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #6537:
URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1933404637
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6537/2/console in case of problems.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. [hadoop]
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn commented on PR #6537:
URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1959710836
@tasanuma that problem is there even before this, I had a ticket: [HADOOP-18701](https://issues.apache.org/jira/browse/HADOOP-18701), and was trying to handle that along with several other stuff, now I don't remember why I left it :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. [hadoop]
Posted by "slfan1989 (via GitHub)" <gi...@apache.org>.
slfan1989 commented on PR #6537:
URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1960795762
@shahrs87 I rolled back #6537. If you have any questions, feel free to contact me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. [hadoop]
Posted by "slfan1989 (via GitHub)" <gi...@apache.org>.
slfan1989 commented on PR #6537:
URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1959545723
@shahrs87 I agree to revert this PR. I've noticed several PRs failing to compile, which is concerning.
I will continue to follow up on updating the maven-surefire-plugin; indeed, this requires more testing.
cc: @steveloughran @ayushtkn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.2. [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #6537:
URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1933467016
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 21s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 29s | | trunk passed |
| +1 :green_heart: | compile | 0m 14s | | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 13s | | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | mvnsite | 0m 16s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 17s | | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 14s | | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | shadedclient | 19m 38s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 10s | | the patch passed |
| +1 :green_heart: | compile | 0m 8s | | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 8s | | the patch passed |
| +1 :green_heart: | compile | 0m 8s | | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 0m 8s | | the patch passed |
| +1 :green_heart: | blanks | 0m 1s | | The patch has no blanks issues. |
| +1 :green_heart: | mvnsite | 0m 12s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 1s | | No new issues. |
| +1 :green_heart: | javadoc | 0m 9s | | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 11s | | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | shadedclient | 19m 44s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 11s | | hadoop-project in the patch passed. |
| +1 :green_heart: | asflicense | 0m 24s | | The patch does not generate ASF License warnings. |
| | | 75m 58s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6537/2/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/6537 |
| Optional Tests | dupname asflicense mvnsite unit codespell detsecrets shellcheck shelldocs compile javac javadoc mvninstall shadedclient xmllint |
| uname | Linux 4bbfeceb9378 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / a76f77e6adc0133b80f1753d459f03cf8d072f95 |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6537/2/testReport/ |
| Max. process+thread count | 558 (vs. ulimit of 5500) |
| modules | C: hadoop-project U: hadoop-project |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6537/2/console |
| versions | git=2.25.1 maven=3.6.3 shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.2. [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #6537:
URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1935142238
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 0m 21s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 23s | | trunk passed |
| +1 :green_heart: | compile | 0m 13s | | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 12s | | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | mvnsite | 0m 18s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 20s | | trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 15s | | trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | shadedclient | 19m 16s | | branch has no errors when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 9s | | the patch passed |
| +1 :green_heart: | compile | 0m 10s | | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 10s | | the patch passed |
| +1 :green_heart: | compile | 0m 9s | | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | javac | 0m 9s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. |
| +1 :green_heart: | mvnsite | 0m 12s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 1s | | No new issues. |
| +1 :green_heart: | javadoc | 0m 10s | | the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 10s | | the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| +1 :green_heart: | shadedclient | 19m 30s | | patch has no errors when building and testing our client artifacts. |
|||| _ Other Tests _ |
| +1 :green_heart: | unit | 0m 11s | | hadoop-project in the patch passed. |
| +1 :green_heart: | asflicense | 0m 23s | | The patch does not generate ASF License warnings. |
| | | 75m 18s | | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6537/3/artifact/out/Dockerfile |
| GITHUB PR | https://github.com/apache/hadoop/pull/6537 |
| Optional Tests | dupname asflicense mvnsite unit codespell detsecrets shellcheck shelldocs compile javac javadoc mvninstall shadedclient xmllint |
| uname | Linux 8574d6e96e29 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / bc34daaf6ec1d79310e80534b339cc7eae540253 |
| Default Java | Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08 |
| Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6537/3/testReport/ |
| Max. process+thread count | 729 (vs. ulimit of 5500) |
| modules | C: hadoop-project U: hadoop-project |
| Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6537/3/console |
| versions | git=2.25.1 maven=3.6.3 shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. [hadoop]
Posted by "slfan1989 (via GitHub)" <gi...@apache.org>.
slfan1989 merged PR #6537:
URL: https://github.com/apache/hadoop/pull/6537
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. [hadoop]
Posted by "shahrs87 (via GitHub)" <gi...@apache.org>.
shahrs87 commented on PR #6537:
URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1954917172
Hi @slfan1989 @steveloughran
I am seeing test failures in `TestCacheDirectives, TestBlockReportLease` in the daily build after this PR.
First build after merging this PR [here](https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1499/)
Now trunk branch has become unstable and tests are failing with OOM and/or build getting stuck for days.
Sample builds:
[Build 1](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6556/5/)
[Build 2](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6562/1/)
[Build 3](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6553/2/)
All the above builds are from different jiras but the same set of tests failing.
Can you please take a look and revert if needed? Thank you !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. [hadoop]
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn commented on PR #6537:
URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1956558036
Some tests are getting skipped as well as I mentioned on the jira
https://issues.apache.org/jira/browse/HADOOP-19071?focusedCommentId=17818734&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17818734
TestDirectoryScanner is a known flaky that ain’t related there is already a ticket somewhere
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. [hadoop]
Posted by "shahrs87 (via GitHub)" <gi...@apache.org>.
shahrs87 commented on PR #6537:
URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1957394370
Thank you @steveloughran for your response.
> Rather than say "hey, let's revert", I'd propose a "surefire update triggers test failures" and see what can be done about addressing them. because we can't stay frozen with surefire versions.
I didn't mean to sound rude. Apologies if you felt that way. I understand that when upgrading maven, surefire or protobuf's version, it is very reasonable to see test failures. But in this case, the jenkins build is not running all tests, some of the tests are failing and some of the builds are getting stuck for 20-24 hours and is affecting all the developers. I spent atleast 2-3 days debugging the test failures assuming that this is caused by my patch before posting a [message](https://the-asf.slack.com/archives/CAD7C52Q3/p1708118456027809) on the slack channel and then @ayushtkn helped me. I am sure other developers have followed the same path.
What I am suggesting is: Lets revert this change, unblock all developers and then we raise a new PR with the proposed surefire version change, make sure all the tests pass, make test improvements if needed and then push the change to trunk branch.
WDYT?
Cc @ayushtkn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.5. [hadoop]
Posted by "slfan1989 (via GitHub)" <gi...@apache.org>.
slfan1989 commented on PR #6537:
URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1959571283
I have carefully read the comments above. Some unit tests reported errors. We need to re-modify the unit tests. I will actively participate in the modification of this part.
cc: @steveloughran @ayushtkn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.2. [hadoop]
Posted by "hadoop-yetus (via GitHub)" <gi...@apache.org>.
hadoop-yetus commented on PR #6537:
URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1935080767
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6537/3/console in case of problems.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org
Re: [PR] HADOOP-19071. Update maven-surefire-plugin from 3.0.0 to 3.2.2. [hadoop]
Posted by "sjlee (via GitHub)" <gi...@apache.org>.
sjlee commented on PR #6537:
URL: https://github.com/apache/hadoop/pull/6537#issuecomment-1934674117
Out of curiosity, what was the reason to pick 3.2.2 as the version to upgrade? The latest version seems to be 3.2.5 and there seem to be quite a few fixes between these two versions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org