[GitHub] [flink] tillrohrmann edited a comment on issue #8242: [FLINK-6227][network] Introduce the DataConsumptionException for downstream task failure

[GitBox <gi...@apache.org>]

tillrohrmann edited a comment on issue #8242: [FLINK-6227][network] Introduce the DataConsumptionException for downstream task failure
URL: https://github.com/apache/flink/pull/8242#issuecomment-490055496
 
 
   I think we actually need to handle case `f` one way or the other. It is not impossible that a `TaskExecutor` completely dies. So we either return a `DataConnectionException` which is handled by the failover strategy or say that after trying to connect to the remote host for some time we fail with a `PartitionNotFoundException`. Of course, it can still be the case that the `TaskExecutor` is reachable afterwards but at some point we simply need to consider the `TaskExecutor` as dead.
   
   So we could only address `b` and `c` in the context of this PR and then introduce (if this is not already the case) as a second PR a `DataConnectionException`. But then, we would also need a way to figure out which result partitions are actually affected by the `DataConnectionException`. The `PartitionNotFoundException` contains the intermediate result partition id for this purpose.
   
   In order to work around this problem, we could also say that a `DataConnectionException` will cause a `PartitionNotFoundException` to be thrown if we are currently trying to look it up.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services