[jira] [Created] (HUDI-5980) Add tests to guard against repeated dag trigger using spark event listeners

["sivabalan narayanan (Jira)" <ji...@apache.org>]

sivabalan narayanan created HUDI-5980:
-----------------------------------------

             Summary: Add tests to guard against repeated dag trigger using spark event listeners
                 Key: HUDI-5980
                 URL: https://issues.apache.org/jira/browse/HUDI-5980
             Project: Apache Hudi
          Issue Type: Improvement
          Components: tests-ci
            Reporter: sivabalan narayanan


as of now, we don't have a good way to guard repeated dag trigger. all of our existing tests only checks for data. but w/ reconcile strategy, the extra files will be removed if some dag was repeated. So, we might need to add more tests to catch them if incase someone changes the dag in future. 

 

Eg test: [https://github.com/apache/hudi/pull/8275/files]

 

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)