You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@texera.apache.org by GitBox <gi...@apache.org> on 2026/08/13 04:04:23 UTC
[GH] (texera/gh-readonly-queue/main/pr-7579-374c8d883e59b8313ca41a58319d134310b1bd66): Workflow run "Required Checks" failed!
The GitHub Actions job "Required Checks" on texera.git/gh-readonly-queue/main/pr-7579-374c8d883e59b8313ca41a58319d134310b1bd66 has failed.
Run started by GitHub user aglinxinyuan (triggered by aglinxinyuan).
Head commit for run:
d1cbab3c7348dbc15ea58e3b7d4e470dd36f21fb / Eugene Gu <eu...@outlook.com>
test(pyamber): add unit tests for IcebergTableWriter (#7579)
### What changes were proposed in this PR?
This PR adds
`amber/src/test/python/core/storage/iceberg/test_iceberg_table_writer.py`
with 14 pure unit tests using mocked catalog/table objects (no real
Iceberg catalog, no Postgres, no network). Covered behaviors:
- Constructor: loads the table via
`catalog.load_table(f"{namespace}.{name}")` and takes `buffer_size` from
`StorageConfig.ICEBERG_TABLE_COMMIT_BATCH_SIZE`.
- Buffer-threshold flush: `put_one` below the threshold does not flush;
reaching `buffer_size` triggers a flush (serde called with schema +
buffered items, `table.append` called with serde's result, buffer
cleared); items added after a flush start a fresh buffer.
- `close()` flushes the remaining items when the buffer is non-empty,
and performs no append when the buffer is empty.
- `open()` clears a previously dirty buffer.
- `remove_one()` removes a buffered item; removing an item already
flushed out of the buffer raises `ValueError` (pinning the current
`list.remove` behavior).
- `_flush_buffer()` returns early on an empty buffer (no
serde/append/refresh calls).
- Retry path: `table.append` raising pyiceberg's `CommitFailedException`
twice then succeeding completes the flush, with `table.refresh()` called
once per attempt; a permanent failure is reraised after 10 attempts and
the buffer is NOT cleared.
- A serde failure propagates without touching the table: serde runs
outside the retry loop, so it is called exactly once, no refresh/append
happens, and the buffer is kept.
- The retry decorator sets no exception filter, so a non-conflict error
(e.g. `ValueError`) is also retried for all 10 attempts before being
reraised (pinning the current behavior).
The retry tests patch `tenacity.nap.time.sleep`, so the
exponential-backoff waits (`wait_random_exponential(0.001, 10)`) never
sleep for real; the whole file runs in about a second.
No production code is changed.
### Any related issues, documentation, discussions?
Closes #7575
### How was this PR tested?
This PR is itself test-only. The new spec was run with:
```
cd amber && pytest src/test/python/core/storage/iceberg/test_iceberg_table_writer.py -v
```
Result: 14 passed in about a second. The file is formatted with `black`
(unchanged by `--check`). The suite was mutation-checked: targeted
mutations of the writer (flipping the `>=` threshold to `>`, dropping
the `buffer.clear()` after append, removing `table.refresh()` in the
retry body, making `close()` skip the flush, removing the empty-buffer
early return, and lowering `stop_after_attempt`) each caused at least
one test to fail, and the source was restored afterwards.
### Was this PR authored or co-authored using generative AI tooling?
Co-authored by: Claude Code (Claude Fable 5)
Report URL: https://github.com/apache/texera/actions/runs/31665552140
With regards,
GitHub Actions via GitBox