Jobmanager time out / long running batch job

[Jan Oelschlegel <oe...@integration-factory.de>]

Hi,

Im using Flink 1.11.3 and run a batch job. In the log of the jobmanager I see that all operators switched from running to finished. And then there is a timeout of the jobmanager. And after some pause the overall status is switched from running  to finished.

Why is there a big gap in between? The task managers have finished their job long ago and only after some time it is accepted as completed at jobmanager. Why is that so?

PS: I replaced some sensitive data with '*'

2021-03-11 09:55:02,099 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph       ****-> Sink: Unnamed (4/4) (5d74d463ecfa2a87128978da5f6fcdbd) switched from RUNNING to FINISHED.
2021-03-11 09:55:02,207 WARN  org.apache.hadoop.hive.conf.HiveConf                         [] - HiveConf of name hive.enforce.sorting does not exist
2021-03-11 09:55:02,207 WARN  org.apache.hadoop.hive.conf.HiveConf                         [] - HiveConf of name hive.enforce.bucketing does not exist
2021-03-11 09:55:02,209 INFO  hive.metastore                                               [] - Trying to connect to metastore with URI thrift://******
2021-03-11 09:55:02,212 INFO  hive.metastore                                               [] - Opened a connection to metastore, current connections: 3
2021-03-11 09:55:02,212 INFO  hive.metastore                                               [] - Connected to metastore.
2021-03-11 09:55:48,817 INFO  org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - The heartbeat of JobManager with id 8300d383b32d743e1af2f117e91f2e52 timed out.
2021-03-11 09:55:48,817 INFO  org.apache.flink.runtime.resourcemanager.StandaloneResourceManager [] - Disconnect job manager 00000000000000000000000000000000@akka.tcp://flink@******:6123/user/rpc/jobmanager_2<mailto:00000000000000000000000000000000@akka.tcp://flink@******:6123/user/rpc/jobmanager_2> for job 85f5e6e0af74b31e359168a25390af92 from the resource manager.
2021-03-11 09:57:08,406 INFO  hive.metastore                                               [] - Closed a connection to metastore, current connections: 2
2021-03-11 09:57:08,406 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph       [] - Job ****** (85f5e6e0af74b31e359168a25390af92) switched from state RUNNING to FINISHED.


Best,
Jan


HINWEIS: Dies ist eine vertrauliche Nachricht und nur f?r den Adressaten bestimmt. Es ist nicht erlaubt, diese Nachricht zu kopieren oder Dritten zug?nglich zu machen. Sollten Sie diese Nachricht irrt?mlich erhalten haben, bitte ich um Ihre Mitteilung per E-Mail oder unter der oben angegebenen Telefonnummer.