You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Lokesh Jain (Jira)" <ji...@apache.org> on 2023/06/05 11:56:00 UTC

[jira] [Resolved] (HUDI-5980) Add tests to guard against repeated dag trigger using spark event listeners

     [ https://issues.apache.org/jira/browse/HUDI-5980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Lokesh Jain resolved HUDI-5980.
-------------------------------

> Add tests to guard against repeated dag trigger using spark event listeners
> ---------------------------------------------------------------------------
>
>                 Key: HUDI-5980
>                 URL: https://issues.apache.org/jira/browse/HUDI-5980
>             Project: Apache Hudi
>          Issue Type: Improvement
>          Components: tests-ci
>            Reporter: sivabalan narayanan
>            Assignee: Lokesh Jain
>            Priority: Major
>              Labels: pull-request-available
>
> as of now, we don't have a good way to guard repeated dag trigger. all of our existing tests only checks for data. but w/ reconcile strategy, the extra files will be removed if some dag was repeated. So, we might need to add more tests to catch them if incase someone changes the dag in future. 
>  
> Eg test: [https://github.com/apache/hudi/pull/8275/files]
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)