You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "danny0405 (via GitHub)" <gi...@apache.org> on 2026/10/22 05:08:12 UTC

[I] [RFC-112] Support Hudi materialized-view flows and repeated refreshes [hudi]

danny0405 opened a new issue, #20405:
URL: https://github.com/apache/hudi/issues/20405

   ### Task Description
   
   Implement and validate CREATE MATERIALIZED VIEW ... USING hudi through the existing catalog and batch writer.
   
   Follow the RFC's Spark 4.1 sequence: create/load target; for refresh, publish an empty insert_overwrite_table truncate commit, reconcile catalog metadata, then write the recomputed result with bulk_insert or insert through append-mode saveAsTable. Do not reinterpret every append as an overwrite or claim atomicity across the entire sequence.
   
   Use the existing shared fixes for catalog creation/reconciliation and existing-table append routing. Provide a runnable SQL example and expected outputs.
   
   ### Acceptance criteria
   
   - [ ] Initial materialization and repeated refresh succeed for managed and external targets.
   - [ ] Refresh output exactly matches recomputation, including rows removed since the prior run and a zero-row result.
   - [ ] Tests cover partitioned/unpartitioned targets, unchanged definitions, supported additive schema changes, and identity-change rejection.
   - [ ] Timeline assertions distinguish truncate replace commits from result writes.
   - [ ] Inject failures before/after truncate and during reconciliation/write; document the empty window and lack of cross-step rollback.
   - [ ] All scenarios run through the CLI/Connect harness.
   
   ### Task Type
   
   Code improvement/refactoring
   
   ### Related Issues
   
   Parent feature issue: #20401
   RFC: #19997 (under review; implementation must follow the agreed contract).
   Dependencies for end-to-end acceptance: #20402, #20404, #20369, #20371.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org