You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by "ren shangtao (Jira)" <ji...@apache.org> on 2022/03/11 02:23:00 UTC
[jira] [Created] (FLINK-26593) task manager can not free memory when jobs are finished caused the linux out of memory
ren shangtao created FLINK-26593:
------------------------------------
Summary: task manager can not free memory when jobs are finished caused the linux out of memory
Key: FLINK-26593
URL: https://issues.apache.org/jira/browse/FLINK-26593
Project: Flink
Issue Type: Bug
Components: API / Core
Affects Versions: 1.14.3
Environment: The flink cluster environment with five nodes
The iceberg is 0.13.1 for flink 1.14
The hadoop is 3.3.1
The hive is 3.1.2
!image-2022-03-11-10-20-41-739.png!
Reporter: ren shangtao
Attachments: image-2022-03-11-10-15-00-682.png, image-2022-03-11-10-16-00-438.png, image-2022-03-11-10-16-53-678.png, image-2022-03-11-10-18-32-940.png, image-2022-03-11-10-19-31-014.png, image-2022-03-11-10-20-41-739.png
I have A batch task that inserts 100 million data from table A into table B after the primary key hash. I use Flink standalone mode to execute this task. After several times of executing this task, I find that the Flink TaskManager process occupies more and more memory. Finally, the Flink TaskManager process is killed when the system runs out of memory.
Here is the message
!image-2022-03-11-10-15-00-682.png!
before run the job
!image-2022-03-11-10-16-00-438.png!
after serval times
!image-2022-03-11-10-16-53-678.png!
My configuration
!image-2022-03-11-10-18-32-940.png!
!image-2022-03-11-10-19-31-014.png!
--
This message was sent by Atlassian Jira
(v8.20.1#820001)