You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "wombatu-kun (via GitHub)" <gi...@apache.org> on 2026/05/25 07:21:15 UTC
[PR] Spark: Bound runaway serializable-isolation and concurrent-refresh tests [iceberg]
wombatu-kun opened a new pull request, #16562:
URL: https://github.com/apache/iceberg/pull/16562
Closes #16359
## Summary
Several `spark-extensions` concurrency tests ran two worker threads in a barrier-synchronized loop bounded by `Integer.MAX_VALUE` while the main thread blocked on `assertThatThrownBy(<op>Future::get)` with no timeout, expecting a conflict exception. When that exception was never thrown, nothing bounded either thread — the append thread kept committing data files until CI hit its wall-clock limit or ran out of disk (as in #16303, where the runaway filled the GitHub Actions disk and was retriggered several times before the cause was found). This makes those tests fail fast instead of relying on external limits.
## What changed
- Bounded both worker loops with a shared `MAX_OPERATIONS = 20` constant — the same value the sibling `*WithSnapshotIsolation` tests already use with this identical harness — so the loop can no longer run unbounded.
- Wrapped the wait in `Future.get(OPERATION_TIMEOUT_MINUTES, TimeUnit.MINUTES)` (5 minutes) and cancelled the operation future in `finally`, so a stuck operation is interrupted and the wait can't block forever.
- Hoisted `MAX_OPERATIONS` and `OPERATION_TIMEOUT_MINUTES` into `SparkRowLevelOperationsTestBase` so these concurrency tests share one bound (the snapshot-isolation siblings now reference the same constant).
Affected methods (across Spark v3.5/v4.0/v4.1 where present): `testMergeWithSerializableIsolation`, `testDeleteWithSerializableIsolation`, `testUpdateWithSerializableIsolation`, and the copy-on-write `testMergeWithConcurrentTableRefresh`, `testDeleteWithConcurrentTableRefresh`, `testUpdateWithConcurrentTableRefresh`.
In a passing run the conflict still fires within the first couple of iterations, so behavior is unchanged; in a regression the bounded loop plus timeout make the test fail fast with a clear assertion instead of exhausting CI resources.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Bound runaway serializable-isolation and concurrent-refresh tests [iceberg]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on PR #16562:
URL: https://github.com/apache/iceberg/pull/16562#issuecomment-4886144787
not stale
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Bound runaway serializable-isolation and concurrent-refresh tests [iceberg]
Posted by "wombatu-kun (via GitHub) via issues" <is...@iceberg.apache.org>.
wombatu-kun closed pull request #16562: Spark: Bound runaway serializable-isolation and concurrent-refresh tests
URL: https://github.com/apache/iceberg/pull/16562
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Bound runaway serializable-isolation and concurrent-refresh tests [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #16562:
URL: https://github.com/apache/iceberg/pull/16562#issuecomment-6103783869
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Bound runaway serializable-isolation and concurrent-refresh tests [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #16562:
URL: https://github.com/apache/iceberg/pull/16562#issuecomment-4884290568
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark: Bound runaway serializable-isolation and concurrent-refresh tests [iceberg]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on PR #16562:
URL: https://github.com/apache/iceberg/pull/16562#issuecomment-5199604263
@huaxingao thanks for merging #16770 - would you mind taking a look at this one as well? Test-only change in the same flaky-test area, CI is green. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org