You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@gobblin.apache.org by "Hung Tran (JIRA)" <ji...@apache.org> on 2019/06/03 14:10:00 UTC

[jira] [Resolved] (GOBBLIN-791) Fix hanging stream on error in asynchronous execution model

     [ https://issues.apache.org/jira/browse/GOBBLIN-791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hung Tran resolved GOBBLIN-791.
-------------------------------
       Resolution: Fixed
    Fix Version/s: 0.15.0

Issue resolved by pull request #2659
[https://github.com/apache/incubator-gobblin/pull/2659]

> Fix hanging stream on error in asynchronous execution model
> -----------------------------------------------------------
>
>                 Key: GOBBLIN-791
>                 URL: https://issues.apache.org/jira/browse/GOBBLIN-791
>             Project: Apache Gobblin
>          Issue Type: Task
>            Reporter: Hung Tran
>            Priority: Major
>             Fix For: 0.15.0
>
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> The asynchronous task execution model uses ReactiveX streams with a ConnectableFlowable. This is  a hot flowable, so it does not terminate when all subscribers have exited. This results in the extractor continuing to emit records after downstream constructs have exited due to an error. This is very problematic for extractors that introduce waits on control message acks since the extractor may hang.
> Another issue is the errors do not propagate upwards, so errors in the writer do not fail the fork. Change the state of the fork onCancel() to a failure state so that the task gets failed.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)