Python pipeline on Flink not finishing cleanly

[Janek Bevendorff <ja...@uni-weimar.de>]

Hi,

When I run a Python pipeline with multiple concurrent TaskManagers on 
Flink, the job hardly ever (or never) finishes properly. At the end, 
Beam (or Flink?) always throws a seemingly random gRPC 
IllegalStateException after my last GlobalCombine, so Beam goes into 
some weird error handling mode and eventually fails to job, even though 
it should have finished cleanly.

This is only reproducible with parallelism set to at least 5 or 8. With 
1-4, I cannot reliably (or at all) reproduce it. It looks like a similar 
issue has already been reported on Jira 
(https://issues.apache.org/jira/browse/BEAM-8980), but it got marked as 
stale. Anyone else seeing this? Is there anything I can do? I don't want 
my job to restart after it's finished and I want a clean exit status, 
otherwise I don't really know if everything succeeded properly and I 
don't want to comb through hundreds of log files to find out.

I added a comment with the stacktraces that I get below the 
above-mentioned issue: 
https://issues.apache.org/jira/browse/BEAM-8980?focusedCommentId=17483174&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17483174

Janek