You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@gobblin.apache.org by "Kip Kohn (Jira)" <ji...@apache.org> on 2023/09/06 22:44:00 UTC

[jira] [Created] (GOBBLIN-1900) Add option for MR job execution to calculate and persist all work units, then proceed no further

Kip Kohn created GOBBLIN-1900:
---------------------------------

             Summary: Add option for MR job execution to calculate and persist all work units, then proceed no further
                 Key: GOBBLIN-1900
                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1900
             Project: Apache Gobblin
          Issue Type: Improvement
          Components: gobblin-api
            Reporter: Kip Kohn
            Assignee: Hung Tran


MapReduce is an aging legacy technology, so it's worth exploring additional execution frameworks.  To simplify experimentation and evaluation, add the ability to "borrow" MR's WorkUnit preparation, so such forays may at first jump ahead to investigate WU execution.

The proposed option `mr.persist.workunits.then.cancel` would work as advertised.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)