You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@gobblin.apache.org by "Zhixiong Chen (JIRA)" <ji...@apache.org> on 2018/01/10 17:37:03 UTC

[jira] [Created] (GOBBLIN-364) Exclude JobState from WorkUnit created by PartitionedFileSourceBase

Zhixiong Chen created GOBBLIN-364:
-------------------------------------

             Summary: Exclude JobState from WorkUnit created by PartitionedFileSourceBase
                 Key: GOBBLIN-364
                 URL: https://issues.apache.org/jira/browse/GOBBLIN-364
             Project: Apache Gobblin
          Issue Type: Task
            Reporter: Zhixiong Chen
            Assignee: Zhixiong Chen


Currently, each `WorkUnit` created by a `PartitionedFileSourceBase` source has a copy of the entire job configurations. For the following 2 reasons, we want to exclude job configurations from `WorkUnit`:
  - It's redundant as the runtime counterpart of `WorkUnit`, which is `WorkUnitState`, would have a reference to all job configurations.
  - Adding job configurations to `WorkUnit` has the bad side effect of masking dynamic job level configurations in MR Task runner



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)