You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@gobblin.apache.org by "William Lo (Jira)" <ji...@apache.org> on 2021/07/26 23:38:00 UTC

[jira] [Created] (GOBBLIN-1497) Reduce the number of calls to create FlowSpecs and configToProperties

William Lo created GOBBLIN-1497:
-----------------------------------

             Summary: Reduce the number of calls to create FlowSpecs and configToProperties
                 Key: GOBBLIN-1497
                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1497
             Project: Apache Gobblin
          Issue Type: Improvement
            Reporter: William Lo


FlowSpecs contain both configs and property fields that duplicate the same key/value information, however there are slight differences with properties being mutable, configs being type-safe, and legacy code being compatible with properties but not configs.

As such, FlowSpec creation and configToProperties are memory expensive operations at scale, and we should aim to minimize them as much as possible.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)