[jira] [Resolved] (GOBBLIN-33) StateStores persists Task and WorkUnit state to state.store.fs.uri

["Abhishek Tiwari (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/GOBBLIN-33?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Abhishek Tiwari resolved GOBBLIN-33.
------------------------------------
    Resolution: Fixed

Issue resolved by pull request #2035
[https://github.com/apache/incubator-gobblin/pull/2035]

> StateStores persists Task and WorkUnit state to state.store.fs.uri
> ------------------------------------------------------------------
>
>                 Key: GOBBLIN-33
>                 URL: https://issues.apache.org/jira/browse/GOBBLIN-33
>             Project: Apache Gobblin
>          Issue Type: Bug
>            Reporter: Joel Baranick
>            Assignee: Joel Baranick
>
> Gobblin 0.9.0 persists Task and WorkUnit state to state.store.fs.uri when running under yarn.  The issue here is that the state store might be switched to something like s3a which is eventually consistent.  In 0.6.0, gobblin persisted the Task and WorkUnit state to HDFS and reserved the state.store.fs.uri for the final job state.  This  worked much better as the Task and WorkUnit state is only used for communication between the worker nodes and the master node.
>  
> *Github Url* : https://github.com/linkedin/gobblin/issues/1847 
> *Github Reporter* : [~jbaranick] 
> *Github Created At* : 2017-05-09T17:26:58Z 
> *Github Updated At* : 2017-06-22T21:36:43Z 
> h3. Comments 
> ----
> [~jbaranick] wrote on 2017-06-22T21:36:42Z : @htran1 Are you able to look into this? 
>  
> *Github Url* : https://github.com/linkedin/gobblin/issues/1847#issuecomment-310509687



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)