You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@gobblin.apache.org by "Abhishek Tiwari (JIRA)" <ji...@apache.org> on 2017/07/19 07:40:01 UTC

[jira] [Created] (GOBBLIN-33) StateStores persists Task and WorkUnit state to state.store.fs.uri

Abhishek Tiwari created GOBBLIN-33:
--------------------------------------

             Summary: StateStores persists Task and WorkUnit state to state.store.fs.uri
                 Key: GOBBLIN-33
                 URL: https://issues.apache.org/jira/browse/GOBBLIN-33
             Project: Apache Gobblin
          Issue Type: Bug
            Reporter: Joel Baranick


Gobblin 0.9.0 persists Task and WorkUnit state to state.store.fs.uri when running under yarn.  The issue here is that the state store might be switched to something like s3a which is eventually consistent.  In 0.6.0, gobblin persisted the Task and WorkUnit state to HDFS and reserved the state.store.fs.uri for the final job state.  This  worked much better as the Task and WorkUnit state is only used for communication between the worker nodes and the master node.
 
*Github Url* : https://github.com/linkedin/gobblin/issues/1847 
*Github Reporter* : [~jbaranick] 
*Github Created At* : 2017-05-09T17:26:58Z 
*Github Updated At* : 2017-06-22T21:36:43Z 
h3. Comments 
----
[~jbaranick] wrote on 2017-06-22T21:36:42Z : @htran1 Are you able to look into this? 
 
*Github Url* : https://github.com/linkedin/gobblin/issues/1847#issuecomment-310509687



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)