[PR] [Feature] Add manifest sidecar support [paimon-rust]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi opened a new pull request, #861:
URL: https://github.com/apache/paimon-rust/pull/861

   ### Purpose
   
   Add Java Paimon-compatible manifest block sidecars so scans can avoid decoding Avro blocks that cannot match partition, row-ID, or bucket filters.
   
   ### Brief change log
   
   - Implement the version-1 `ManifestSidecar` codec, builder, validation, CRC checking, block selection, and ranged block reads.
   - Record physical Avro OCF block offsets and lengths.
   - Generate and publish `.avro.sidecar` files through manifest `_EXTRA_FILES` during commits.
   - Use sidecars in table scans with conservative fallback for missing, corrupt, unsupported, or mismatched files.
   - Preserve incremental ADD/DELETE validation before row-range pruning.
   - Add `manifest.sidecar.enabled`, inheriting `manifest-sort.enabled` when unset.
   
   ### Tests
   
   - `cargo test -p paimon --lib --quiet` — 2859 passed, 0 failed, 2 ignored.
   - `cargo test -p paimon manifest_sidecar --lib --quiet` — 10 passed.
   - `cargo clippy -p paimon --lib --tests -- -D warnings`.
   - `cargo fmt --all -- --check` and `git diff --check`.
   
   ### API and Format
   
   Adds public `ManifestSidecar`, `ManifestSidecarBuilder`, selection/block types, and the `.avro.sidecar` suffix. The binary format is compatible with Java Paimon ManifestSidecar version 1 and is explicitly referenced from manifest metadata.
   
   ### Documentation
   
   Documents the table option, format compatibility, publication lifecycle, and fallback behavior in `docs/src/sql.md`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [Feature] Add manifest sidecar support [paimon-rust]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi merged PR #861:
URL: https://github.com/apache/paimon-rust/pull/861


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org