You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/03/04 09:30:32 UTC
[GitHub] [spark] Ngone51 opened a new pull request #27789: [SPARK-30541]
testRetry flaky KafkaDelegationTokenSuite
Ngone51 opened a new pull request #27789: [SPARK-30541] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://spark.apache.org/contributing.html
2. Ensure you have added or run the appropriate tests for your PR: https://spark.apache.org/developer-tools.html
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][SPARK-XXXX] Your PR title ...'.
4. Be sure to keep the PR description updated to reflect all changes.
5. Please write your PR title to summarize what this PR proposes.
6. If possible, provide a concise example to reproduce the issue for a faster review.
7. If you want to add a new configuration, please read the guideline first for naming configurations in
'core/src/main/scala/org/apache/spark/internal/config/ConfigEntry.scala'.
-->
### What changes were proposed in this pull request?
<!--
Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->
Retry test `KafkaDelegationTokenSuite` for 3 times.
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
`KafkaDelegationTokenSuite` is too flaky.
### Does this PR introduce any user-facing change?
<!--
If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
If no, write 'No'.
-->
No.
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
-->
Pass Jenkins.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Ngone51 commented on issue #27789: [SPARK-30541][TEST]
testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
Ngone51 commented on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-594593391
I see. 2 options:
1. move before/all into the test scope
2. increase `timeout` in `testUtils.setup()`
But I also don't have much strong prefer on both...I know they're not good enough...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gaborgsomogyi edited a comment on issue #27789:
[SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
gaborgsomogyi edited a comment on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-594524053
I'm not against the approach but I've couple of questions before we go on:
* Could you reproduce it somehow on your local environment (or any environment where not all the tests run to spare time)? I've tried it to run it for weeks but it has never failed. I would like to understand what causes the issue but since I was not able to find any code glitch + not able to reproduce it's hard.
* Which part of the test failed exactly? I'm asking it because at least 2 parts can fail: `beforeAll` and the test itself. I think this won't solve SPARK-30541 since it's failing in the `beforeAll`...
* Have you checked this code?
```
/**
* Note: this method doesn't support `BeforeAndAfter`. You must use `BeforeAndAfterEach` to
* set up and tear down resources.
*/
def testRetry(s: String, n: Int = 2)(body: => Unit): Unit = {
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #27789:
[SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#discussion_r388108959
##########
File path: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaDelegationTokenSuite.scala
##########
@@ -62,7 +62,7 @@ class KafkaDelegationTokenSuite extends StreamTest with SharedSparkSession with
}
}
- test("Roundtrip") {
+ testRetry("Roundtrip", 3) {
Review comment:
Sounds fine to me.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27789:
[SPARK-31050][TEST] Disable flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27789: [SPARK-31050][TEST] Disable flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-595083531
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #27789:
[SPARK-31050][TEST] Disable flaky `Roundtrip` test in
KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #27789: [SPARK-31050][TEST] Disable flaky `Roundtrip` test in KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27789:
[SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-594415701
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27789:
[SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-594415147
**[Test build #119298 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119298/testReport)** for PR 27789 at commit [`54aa6b4`](https://github.com/apache/spark/commit/54aa6b4359ef9d09a01698a9ab3c8e4fcf718f98).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Ngone51 commented on a change in pull request #27789:
[SPARK-31050][TEST] Disable flaky `Roundtrip` test in
KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
Ngone51 commented on a change in pull request #27789: [SPARK-31050][TEST] Disable flaky `Roundtrip` test in KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#discussion_r388159919
##########
File path: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaDelegationTokenSuite.scala
##########
@@ -62,7 +62,7 @@ class KafkaDelegationTokenSuite extends StreamTest with SharedSparkSession with
}
}
- test("Roundtrip") {
+ ignore("Roundtrip") {
Review comment:
Hi @gaborgsomogyi , after taking a look of `BeforeAndAfterAll`, I think it should be OK due to:
```
/**
* Flag to indicate whether to invoke beforeAll and afterAll even when there are no tests expected.
*
* <p>
* The default value is <code>false</code>, which means beforeAll and afterAll will not be invoked
* if there are no tests expected. Whether tests are expected is determined by invoking <code>expectedTestCount</code> passing in
* the passed filter. Because this count does not include tests excluded based on tags, such as ignored tests, this prevents
* any side effects in <code>beforeAll</code> or <code>afterAll</code> if no tests will ultimately be executed anyway.
* If you always want to see the side effects even if no tests are expected, override this <code>val</code> and set it to true.
* </p>
*/
val invokeBeforeAllAndAfterAllEvenIfNoTestsAreExpected = false
```
And I tested locally to verify it. So, we're temporary safe by now. But please be aware of it as it can be easily break when people add any new test. And, of course, your later fix should also get rid of it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Ngone51 edited a comment on issue #27789:
[SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
Ngone51 edited a comment on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-594593391
I see. 2 options:
1. move beforeAll/AfterAll into the test scope
2. increase `timeout` in `testUtils.setup()`
But I also don't have much strong prefer on both...I know they're not good enough...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gaborgsomogyi commented on issue #27789:
[SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
gaborgsomogyi commented on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-594580839
I don't know what's going wrong but it fails at least in https://issues.apache.org/jira/browse/SPARK-30541 in `beforeAll`. The actual stand of the patch is not covering this and since you're not able to provide information how this solves the problem I would say this shouldn't be merged as is.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #27789: [SPARK-30541][TEST]
testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-595070637
Reading from the error messages, yea, seems it's failed during `setup`. Let's disable this test case and enable it back in a separate PR.
Conditional test can be an option if we know this `setup` being failed isn't an issue but just being flaky for other external reasons, and it's difficult to fix.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gaborgsomogyi commented on issue #27789:
[SPARK-31050][TEST] Disable flaky `Roundtrip` test in
KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
gaborgsomogyi commented on issue #27789: [SPARK-31050][TEST] Disable flaky `Roundtrip` test in KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-595255235
Created a PR to see how things are going with `testRetry`: https://github.com/apache/spark/pull/27810
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27789:
[SPARK-31050][TEST] Disable flaky `Roundtrip` test in
KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on issue #27789: [SPARK-31050][TEST] Disable flaky `Roundtrip` test in KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-595091872
Merged to master/3.0.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27789:
[SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-594415711
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24039/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gaborgsomogyi commented on a change in pull request #27789:
[SPARK-31050][TEST] Disable flaky `Roundtrip` test in
KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
gaborgsomogyi commented on a change in pull request #27789: [SPARK-31050][TEST] Disable flaky `Roundtrip` test in KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#discussion_r388149299
##########
File path: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaDelegationTokenSuite.scala
##########
@@ -62,7 +62,7 @@ class KafkaDelegationTokenSuite extends StreamTest with SharedSparkSession with
}
}
- test("Roundtrip") {
+ ignore("Roundtrip") {
Review comment:
Just wanted to say :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27789:
[SPARK-31050][TEST] Disable flaky `Roundtrip` test in
KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27789: [SPARK-31050][TEST] Disable flaky `Roundtrip` test in KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-595101842
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon edited a comment on issue #27789:
[SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
HyukjinKwon edited a comment on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-595070637
Reading the error messages, yea, seems it's failed during `setup`. Let's disable this test case and enable it back in a separate PR.
Conditional test can be an option if we know this `setup` being failed isn't an issue but just being flaky for other external reasons, and it's difficult to fix.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27789:
[SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-594415701
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gaborgsomogyi commented on issue #27789:
[SPARK-31050][TEST] Disable flaky `Roundtrip` test in
KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
gaborgsomogyi commented on issue #27789: [SPARK-31050][TEST] Disable flaky `Roundtrip` test in KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-595101132
I have the feeling the reason why we haven't found anything on Spark side is that because Kafka broker is not coming up randomly. Starting a discussion with the Kafka guys related this issue...
As a temporary solution if we move beforeAll/afterAll into the test scope retry can be used.
I'm going to try it out with @zsxwing test approach in a PR...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Ngone51 commented on a change in pull request #27789:
[SPARK-31050][TEST] Disable flaky `Roundtrip` test in
KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
Ngone51 commented on a change in pull request #27789: [SPARK-31050][TEST] Disable flaky `Roundtrip` test in KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#discussion_r388159919
##########
File path: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaDelegationTokenSuite.scala
##########
@@ -62,7 +62,7 @@ class KafkaDelegationTokenSuite extends StreamTest with SharedSparkSession with
}
}
- test("Roundtrip") {
+ ignore("Roundtrip") {
Review comment:
Hi @gaborgsomogyi , after taking a look of `BeforeAndAfterAll`, I think it should be OK due to:
```
/**
* Flag to indicate whether to invoke beforeAll and afterAll even when there are no tests expected.
*
* <p>
* The default value is <code>false</code>, which means beforeAll and afterAll will not be invoked
* if there are no tests expected. Whether tests are expected is determined by invoking <code>expectedTestCount</code> passing in
* the passed filter. Because this count does not include tests excluded based on tags, such as ignored tests, this prevents
* any side effects in <code>beforeAll</code> or <code>afterAll</code> if no tests will ultimately be executed anyway.
* If you always want to see the side effects even if no tests are expected, override this <code>val</code> and set it to true.
* </p>
*/
val invokeBeforeAllAndAfterAllEvenIfNoTestsAreExpected = false
```
And I tested locally to verify it. So, we're temporary safe by now. But please be aware of it as it can be easily break when there's another test. And, of course, the later fix could also get rid of it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27789:
[SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-594434582
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27789:
[SPARK-31050][TEST] Disable flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27789: [SPARK-31050][TEST] Disable flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-595083531
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27789: [SPARK-30541][TEST]
Disable flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27789: [SPARK-30541][TEST] Disable flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-595083110
**[Test build #119373 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119373/testReport)** for PR 27789 at commit [`ea1e389`](https://github.com/apache/spark/commit/ea1e3899414b703bc2a06c70fe9f611f0f68a75c).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27789:
[SPARK-31050][TEST] Disable flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27789: [SPARK-31050][TEST] Disable flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-595083543
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24111/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gaborgsomogyi edited a comment on issue #27789:
[SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
gaborgsomogyi edited a comment on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-594524053
I'm not against the approach but I've couple of questions before we go on:
* Could you reproduce it somehow on your local environment (or any environment where not all the tests run to spare time)? I've tried it to run it for weeks but it has never failed. I would like to understand what causes the issue but since I was not able to find any code glitch + not able to reproduce it's hard.
* Which part of the test failed exactly? I'm asking it because at least 2 parts can fail: `beforeAll` and the test itself. I think this won't solve SPARK-30541 since it's failing in the setup...
* Have you checked this code?
```
/**
* Note: this method doesn't support `BeforeAndAfter`. You must use `BeforeAndAfterEach` to
* set up and tear down resources.
*/
def testRetry(s: String, n: Int = 2)(body: => Unit): Unit = {
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27789:
[SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-594434595
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119298/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Ngone51 edited a comment on issue #27789:
[SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
Ngone51 edited a comment on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-594593391
I see. 2 options:
1. move beforeAll/afterAll into the test scope
2. increase `timeout` in `testUtils.setup()`
But I also don't have much strong prefer on both...I know they're not good enough...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27789:
[SPARK-31050][TEST] Disable flaky `Roundtrip` test in
KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27789: [SPARK-31050][TEST] Disable flaky `Roundtrip` test in KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-595101856
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119373/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27789:
[SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-594415711
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24039/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27789: [SPARK-30541][TEST]
testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-594434359
**[Test build #119298 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119298/testReport)** for PR 27789 at commit [`54aa6b4`](https://github.com/apache/spark/commit/54aa6b4359ef9d09a01698a9ab3c8e4fcf718f98).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gaborgsomogyi commented on issue #27789:
[SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
gaborgsomogyi commented on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-594524053
I'm not against the approach but I've couple of questions before we go on:
* Could you reproduce it somehow on your local environment (or any environment where not all the tests run)? I've tried it to run it for weeks but it has never failed. I would like to understand what causes the issue but since I was not able to find any code glitch + not able to reproduce it's hard.
* Which part of the test failed exactly? I'm asking it because at least 2 parts can fail: `beforeAll` and the test itself. I think this won't solve SPARK-30541 since it's failing in the setup...
* Have you checked this code?
```
/**
* Note: this method doesn't support `BeforeAndAfter`. You must use `BeforeAndAfterEach` to
* set up and tear down resources.
*/
def testRetry(s: String, n: Int = 2)(body: => Unit): Unit = {
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zsxwing commented on issue #27789: [SPARK-30541][TEST]
testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
zsxwing commented on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-594710354
+1 on trying to reproduce it and fix the root issue (or have a confident workaround if that's an issue in an external library).
I usually just update `dev/run-tests` to something like:
```shell
set -e
for i in `seq 0 100`; do
build/sbt "sql-kafka-0-10/test-only *KafkaDelegationTokenSuite"
done
# exec python3 -u ./dev/run-tests.py "$@"
```
and open a PR to run the flaky test 100 (or more) times on Jenkins in one build. If this can reproduce the issue, we can add more logs to investigate further.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27789:
[SPARK-31050][TEST] Disable flaky `Roundtrip` test in
KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #27789: [SPARK-31050][TEST] Disable flaky `Roundtrip` test in KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-595083110
**[Test build #119373 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119373/testReport)** for PR 27789 at commit [`ea1e389`](https://github.com/apache/spark/commit/ea1e3899414b703bc2a06c70fe9f611f0f68a75c).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27789: [SPARK-31050][TEST]
Disable flaky `Roundtrip` test in KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27789: [SPARK-31050][TEST] Disable flaky `Roundtrip` test in KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-595101638
**[Test build #119373 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119373/testReport)** for PR 27789 at commit [`ea1e389`](https://github.com/apache/spark/commit/ea1e3899414b703bc2a06c70fe9f611f0f68a75c).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Ngone51 commented on a change in pull request #27789:
[SPARK-31050][TEST] Disable flaky `Roundtrip` test in
KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
Ngone51 commented on a change in pull request #27789: [SPARK-31050][TEST] Disable flaky `Roundtrip` test in KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#discussion_r388147525
##########
File path: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaDelegationTokenSuite.scala
##########
@@ -62,7 +62,7 @@ class KafkaDelegationTokenSuite extends StreamTest with SharedSparkSession with
}
}
- test("Roundtrip") {
+ ignore("Roundtrip") {
Review comment:
Oh sorry..we need to also disable `beforeAll/afterAll`. I'll make a follow-up right now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gaborgsomogyi edited a comment on issue #27789:
[SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
gaborgsomogyi edited a comment on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-594524053
I'm not against the approach but I've couple of questions before we go on:
* Could you reproduce it somehow on your local environment (or any environment where not all the tests run to spare time)? I've tried it to run it for weeks but it has never failed. I would like to understand what causes the issue but since I was not able to find any code glitch + not able to reproduce it's hard.
* Which part of the test failed exactly? I'm asking it because at least 2 parts can fail: `beforeAll` and the test itself. I think this won't solve https://issues.apache.org/jira/browse/SPARK-30541 since it's failing in the `beforeAll`...
* Have you checked this code?
```
/**
* Note: this method doesn't support `BeforeAndAfter`. You must use `BeforeAndAfterEach` to
* set up and tear down resources.
*/
def testRetry(s: String, n: Int = 2)(body: => Unit): Unit = {
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on issue #27789: [SPARK-30541][TEST]
testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-594415516
cc @HeartSaVioR, @zsxwing, @gaborgsomogyi
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27789: [SPARK-30541][TEST]
testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-594415147
**[Test build #119298 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/119298/testReport)** for PR 27789 at commit [`54aa6b4`](https://github.com/apache/spark/commit/54aa6b4359ef9d09a01698a9ab3c8e4fcf718f98).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gaborgsomogyi commented on a change in pull request #27789:
[SPARK-31050][TEST] Disable flaky `Roundtrip` test in
KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
gaborgsomogyi commented on a change in pull request #27789: [SPARK-31050][TEST] Disable flaky `Roundtrip` test in KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#discussion_r388143395
##########
File path: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaDelegationTokenSuite.scala
##########
@@ -62,7 +62,7 @@ class KafkaDelegationTokenSuite extends StreamTest with SharedSparkSession with
}
}
- test("Roundtrip") {
+ testRetry("Roundtrip", 3) {
Review comment:
Late +1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on issue #27789: [SPARK-30541][TEST]
testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
HeartSaVioR commented on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-595067025
Have we observed the case where the setup was passed and the test "roundtrip" failed? The errors what I have been observed are all setup failure.
If we don't mind about conditional test, maybe we can "catch" exception in beforeAll, and run the code "roundtrip" conditionally depending on the result of setup.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gaborgsomogyi edited a comment on issue #27789:
[SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
gaborgsomogyi edited a comment on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-594524053
I'm not against the approach but I've couple of questions before we go on:
* Could you reproduce it somehow on your local environment (or any environment where not all the tests run to spare time)? I've tried it to run it for weeks but it has never failed. I would like to understand what causes the issue but since I was not able to find any code glitch + not able to reproduce it's hard.
* Which part of the test failed exactly? I'm asking it because at least 2 parts can fail: `beforeAll` and the test itself. I think this won't solve https://issues.apache.org/jira/browse/SPARK-30541 since it's failing in the `beforeAll`...
* Have you checked this code?
```
/**
* Note: this method doesn't support `BeforeAndAfter`. You must use `BeforeAndAfterEach` to
* set up and tear down resources.
*/
def testRetry(s: String, n: Int = 2)(body: => Unit): Unit = {
```
What about `BeforeAndAfterAll` what used here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Ngone51 commented on issue #27789: [SPARK-30541][TEST]
testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
Ngone51 commented on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-594548734
To be honest, I'm not familiar with this part. So, actually, I cannot tell any details about this test.
I decide to give a try in this way:
1. after tracking the discussion/JIRA/PR to confirm that we do not have a solution yet
2. we do use retry for other flaky test
3. this test is really flaky for long time...
And of course, I'd like to listen to your opinions on this while a pure fix is always welcomed than this.
As for `BeforeAndAfter`, I noted the limitation. I tested locally but without a detail check. And I realized now that `KafkaDelegationTokenSuite` extends `BeforeAndAfterAll`. I'll take a look.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27789:
[SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-594434582
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #27789:
[SPARK-31050][TEST] Disable flaky `Roundtrip` test in
KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #27789: [SPARK-31050][TEST] Disable flaky `Roundtrip` test in KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#discussion_r388162745
##########
File path: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaDelegationTokenSuite.scala
##########
@@ -62,7 +62,7 @@ class KafkaDelegationTokenSuite extends StreamTest with SharedSparkSession with
}
}
- test("Roundtrip") {
+ ignore("Roundtrip") {
Review comment:
Ah, so we're safe. Thanks for investigation; I was about to point this out too.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gatorsmile commented on a change in pull request #27789:
[SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
gatorsmile commented on a change in pull request #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#discussion_r388107442
##########
File path: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaDelegationTokenSuite.scala
##########
@@ -62,7 +62,7 @@ class KafkaDelegationTokenSuite extends StreamTest with SharedSparkSession with
}
}
- test("Roundtrip") {
+ testRetry("Roundtrip", 3) {
Review comment:
@zsxwing @gaborgsomogyi Should we disable this flaky test first and create a blocker ticket for 3.0?
If we keep hitting the test failure, it will impact the productivity of the open source development.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27789:
[SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-594434595
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119298/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27789:
[SPARK-31050][TEST] Disable flaky `Roundtrip` test in
KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27789: [SPARK-31050][TEST] Disable flaky `Roundtrip` test in KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-595101842
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27789:
[SPARK-31050][TEST] Disable flaky `Roundtrip` test in
KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #27789: [SPARK-31050][TEST] Disable flaky `Roundtrip` test in KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-595101856
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/119373/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] Ngone51 commented on issue #27789: [SPARK-30541][TEST]
testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
Ngone51 commented on issue #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-595082356
Thanks all! I disabled it and marked [SPARK-30541](https://issues.apache.org/jira/browse/SPARK-30541) as a blocker for 3.0.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27789:
[SPARK-31050][TEST] Disable flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #27789: [SPARK-31050][TEST] Disable flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#issuecomment-595083543
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/24111/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] zsxwing commented on a change in pull request #27789:
[SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
Posted by GitBox <gi...@apache.org>.
zsxwing commented on a change in pull request #27789: [SPARK-30541][TEST] testRetry flaky KafkaDelegationTokenSuite
URL: https://github.com/apache/spark/pull/27789#discussion_r388110374
##########
File path: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaDelegationTokenSuite.scala
##########
@@ -62,7 +62,7 @@ class KafkaDelegationTokenSuite extends StreamTest with SharedSparkSession with
}
}
- test("Roundtrip") {
+ testRetry("Roundtrip", 3) {
Review comment:
+1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org