You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "manuzhang (via GitHub)" <gi...@apache.org> on 2025/04/02 16:41:37 UTC
[PR] Build: Retry flaky test [iceberg]
manuzhang opened a new pull request, #12707:
URL: https://github.com/apache/iceberg/pull/12707
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Build: Retry flaky test [iceberg]
Posted by "danielcweeks (via GitHub)" <gi...@apache.org>.
danielcweeks commented on PR #12707:
URL: https://github.com/apache/iceberg/pull/12707#issuecomment-2776284957
This actually looks like it might be an issue related to the test setup. We're allowing up to [1 second for retry wait](https://github.com/apache/iceberg/blob/14122cb736dcc0db52e48caf183d92eb1272da86/core/src/test/java/org/apache/iceberg/hadoop/TestHadoopCommits.java#L433) along with 5 concurrent threads each committing 10 files, but only giving `10s` awaitility time. We might want to drop the 1 second retry time to `100ms` or increate the wait time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Build: Retry flaky test [iceberg]
Posted by "danielcweeks (via GitHub)" <gi...@apache.org>.
danielcweeks commented on PR #12707:
URL: https://github.com/apache/iceberg/pull/12707#issuecomment-2776389996
@manuzhang looking at the test, I think part of it is how the test is designed. There are 5 threads committing, but each thread waits until all other threads have committed before moving on to the next commit. Given that and the wait times, it's very likely they'll exceed the overall time.
I'm not sure this is the best way to demonstrate concurrency and contention across commits. It's also a pretty long test (usually takes around 60s on my laptop to complete).
Increasing the awaitility time to 30 seconds worked for me (10 seconds was very hit/miss).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Build: Retry flaky test [iceberg]
Posted by "pvary (via GitHub)" <gi...@apache.org>.
pvary commented on PR #12707:
URL: https://github.com/apache/iceberg/pull/12707#issuecomment-2775527627
@manuzhang: Is it flaky because the waiting time is too short? Shall we just decrease the number of threads, or commits, or increase the timeout?
I usually prefer to avoid retries as it is prone to hide real problems.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Build: Retry flaky test [iceberg]
Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang closed pull request #12707: Build: Retry flaky test
URL: https://github.com/apache/iceberg/pull/12707
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Build: Retry flaky test [iceberg]
Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on PR #12707:
URL: https://github.com/apache/iceberg/pull/12707#issuecomment-2774457965
The flaky test is usually when we need to wait for a condition to come true. In this case,
```
AtomicInteger barrier = new AtomicInteger(0);
Tasks.range(threadsCount)
.stopOnFailure()
.throwFailureWhenFinished()
.executeWith(executorService)
.run(
index -> {
for (int numCommittedFiles = 0;
numCommittedFiles < numberOfCommitedFilesPerThread;
numCommittedFiles++) {
final int currentFilesCount = numCommittedFiles;
Awaitility.await()
.pollInterval(Duration.ofMillis(10))
.atMost(Duration.ofSeconds(10))
.until(() -> barrier.get() >= currentFilesCount * threadsCount);
tableWithHighRetries.newFastAppend().appendFile(file).commit();
barrier.incrementAndGet();
}
});
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Build: Retry flaky test [iceberg]
Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on PR #12707:
URL: https://github.com/apache/iceberg/pull/12707#issuecomment-2776330349
Let me try with https://github.com/apache/iceberg/pull/12714
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Build: Retry flaky test [iceberg]
Posted by "danielcweeks (via GitHub)" <gi...@apache.org>.
danielcweeks commented on PR #12707:
URL: https://github.com/apache/iceberg/pull/12707#issuecomment-2773623291
@manuzhang do we know why it is flaky? Is this an inherent issue that we can address without retries?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org