You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "sivabalan narayanan (Jira)" <ji...@apache.org> on 2025/04/09 20:30:00 UTC

[jira] [Updated] (HUDI-8469) Re-design DAG to write to DT and MDT

     [ https://issues.apache.org/jira/browse/HUDI-8469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

sivabalan narayanan updated HUDI-8469:
--------------------------------------
    Remaining Estimate: 24h
     Original Estimate: 24h

> Re-design DAG to write to DT and MDT
> ------------------------------------
>
>                 Key: HUDI-8469
>                 URL: https://issues.apache.org/jira/browse/HUDI-8469
>             Project: Apache Hudi
>          Issue Type: Sub-task
>          Components: metadata, writer-core
>            Reporter: sivabalan narayanan
>            Assignee: sivabalan narayanan
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 1.0.2
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Lets design a new dag so that writes to both DT and MDT gets written using single dag. 
> Current design of collecting after writing to DT and then triggering to MDT results in some inconsistencies when RLI is involved. So, we wanted to re-design the dag to ease the non-determinism and get it working smoothly 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)