You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "raghav-reglobe (via GitHub)" <gi...@apache.org> on 2026/06/19 21:07:53 UTC
[PR] feat(transaction): RowDelta merge-on-read deletion vectors (carries forward #2203) [iceberg-rust]
raghav-reglobe opened a new pull request, #2678:
URL: https://github.com/apache/iceberg-rust/pull/2678
## Summary
Completes `RowDelta` with the **merge-on-read (MoR)** path — V3 Puffin **deletion vectors** committed via a `content=Deletes` manifest — on top of @wirybeaver's copy-on-write `RowDelta` foundation from #2203 (rebased onto current `main`, authorship preserved). Together they give `RowDelta` both COW (rewrite data files) and MoR (deletion vectors) — the basis for `MERGE INTO` / `UPDATE` / `DELETE` on V3 tables.
#2203 has been stale and conflicting with `main`; per my note there, this carries it forward. The rebase conflicts were trivial (additive `mod` / `use` keep-both) plus one moved API (`Snapshot::load_manifest_list` → `Table::manifest_list_reader`).
## MoR additions (on top of #2203)
- `DeleteVector::{to_puffin_blob, from_puffin_blob, write_to_puffin_file}` — serialize/deserialize the `deletion-vector-v1` blob (magic `0xD1D33964`, 4-byte BE length + portable 64-bit roaring bitmap + 4-byte BE CRC32; `snapshot-id` / `sequence-number` = `-1`; `fields` = `[]`) and produce the `PositionDeletes` `DataFile`.
- `RowDeltaAction::add_delete_files` MoR commit — writes a `content=Deletes` manifest with `Operation::Delete` (replaces the previous `FeatureUnsupported` stub).
- `delete_vector` made `pub` (with rustdoc).
## Validation
- **Byte-identical to Apache Iceberg-Java** — `to_puffin_blob` output equals the Java-produced golden DV fixtures from `apache/iceberg` core test resources, for 3 vectors: empty, `{1, 3, 5, 7, 9}`, and small+large spanning two roaring containers (`crates/iceberg/testdata/puffin/*-position-index.bin`).
- **Cross-implementation parity** with `apache/iceberg-go` (`puffin/dv_golden_test.go`, `TestSerializeDV*`).
- **Real data, cross-engine** — a deletion vector written by this code was committed to a real V3 table via a REST catalog on S3 and read back correctly by an independent engine (Apache Doris): 10 → 7 rows, exact positions deleted.
- 14 `delete_vector` + 7 `transaction::row_delta` tests pass; `cargo fmt` + `cargo clippy` clean.
## Credit & follow-ups
- `RowDelta` copy-on-write foundation by @wirybeaver (#2203) — carried forward here with authorship preserved.
- Out of scope (follow-ups): a higher-level `DVWriter` accumulator (multi-blob-per-Puffin, dedup, partition capture) as in iceberg-go; reader-side DV invariant enforcement (`snapshot-id`/`sequence-number` = -1 on read).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): RowDelta merge-on-read deletion vectors (carries forward #2203) [iceberg-rust]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #2678: feat(transaction): RowDelta merge-on-read deletion vectors (carries forward #2203)
URL: https://github.com/apache/iceberg-rust/pull/2678
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): RowDelta merge-on-read deletion vectors (carries forward #2203) [iceberg-rust]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #2678:
URL: https://github.com/apache/iceberg-rust/pull/2678#issuecomment-5502460464
This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(transaction): RowDelta merge-on-read deletion vectors (carries forward #2203) [iceberg-rust]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #2678:
URL: https://github.com/apache/iceberg-rust/pull/2678#issuecomment-5418757259
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org