[jira] [Created] (GOBBLIN-1984) Add Consensus FlowExecutionId to FlowSpec used for Compilation and Tracking Dag

["Urmi Mustafi (Jira)" <ji...@apache.org>]

Urmi Mustafi created GOBBLIN-1984:
-------------------------------------

             Summary: Add Consensus FlowExecutionId to FlowSpec used for Compilation and Tracking Dag
                 Key: GOBBLIN-1984
                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1984
             Project: Apache Gobblin
          Issue Type: Bug
          Components: gobblin-service
            Reporter: Urmi Mustafi
            Assignee: Abhishek Tiwari


In a previous change, we noticed the compiled flowExecutionId was different from the one during execution and tried to fix it by updating the flowMetadata used to send GTE relating to compilation. We succeeded in updating the compilation event to use the flowExecutionId but did not update the FlowSpec used by Compiler to create the Dag which is used during the execution phase of flow management. The compiler retrieves a flowExecutionId if one exists in the flowSpec (which it does for adhoc flows), otherwise generates a new one ([code|[https://jarvis.corp.linkedin.com/codesearch/result/?name=MultiHopFlowCompiler.java&path=gobblin-elr%2Fgobblin-service%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fgobblin%2Fservice%2Fmodules%2Fflow&reponame=linkedin%2Fgobblin-elr#301]).] We do need to modify the FlowSpec then with the consensus flowExecutionId before forwarding it to the DagManager.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)