You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@gobblin.apache.org by "Zihan Li (Jira)" <ji...@apache.org> on 2021/04/02 18:19:00 UTC

[jira] [Created] (GOBBLIN-1419) Error handling for compaction pipeline on GMCE emitted error

Zihan Li created GOBBLIN-1419:
---------------------------------

             Summary: Error handling for compaction pipeline on GMCE emitted error
                 Key: GOBBLIN-1419
                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1419
             Project: Apache Gobblin
          Issue Type: Task
            Reporter: Zihan Li


No compaction publish file and emit GMCE are happening separately. If file rename succeed but emit GMCE fails, we will end up with not emit GMCE for the compaction. So need  a strategy to do error handling for compaction that when emit GMCE fails, next job will treat the previous compaction as fail and re-compact the data.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)