You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@gobblin.apache.org by "Hung Tran (JIRA)" <ji...@apache.org> on 2019/05/31 17:00:00 UTC

[jira] [Created] (GOBBLIN-791) Fix hanging stream on error in asynchronous execution model

Hung Tran created GOBBLIN-791:
---------------------------------

             Summary: Fix hanging stream on error in asynchronous execution model
                 Key: GOBBLIN-791
                 URL: https://issues.apache.org/jira/browse/GOBBLIN-791
             Project: Apache Gobblin
          Issue Type: Task
            Reporter: Hung Tran


The asynchronous task execution model uses ReactiveX streams with a ConnectableFlowable. This is  a hot flowable, so it does not terminate when all subscribers have exited. This results in the extractor continuing to emit records after downstream constructs have exited due to an error. This is very problematic for extractors that introduce waits on control message acks since the extractor may hang.

Another issue is the errors do not propagate upwards, so errors in the writer do not fail the fork. Change the state of the fork onCancel() to a failure state so that the task gets failed.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)