[jira] [Created] (FLINK-22734) ExtractionUtils#getClassReader causes "open too many files" error

["zhiping lin (Jira)" <ji...@apache.org>]

zhiping lin created FLINK-22734:
-----------------------------------

             Summary: ExtractionUtils#getClassReader causes "open too many files" error
                 Key: FLINK-22734
                 URL: https://issues.apache.org/jira/browse/FLINK-22734
             Project: Flink
          Issue Type: Bug
    Affects Versions: 1.12.2
            Reporter: zhiping lin


when we get classReader in ExtractionUtils 

 
{code:java}
    private static ClassReader getClassReader(Class<?> cls) {        
        final String className = cls.getName().replaceFirst("^.*\\.", "") + ".class";        
        try {            
            return new ClassReader(cls.getResourceAsStream(className));        
        } catch (IOException e) {
            throw new IllegalStateException("Could not instantiate ClassReader.", e);
        }
    }
{code}
we open a inputStream by "cls.getResourceAsStream(className)" and set it as a construct param for  ClassReader without any other steps to close it. Also ClassReader's construct method with only a outside inputStream, classReader won't close the inputStream while it finished read the file. This will leads to fd leak with "open too many files" error when we parse a lot independent sqls with different envs in one jvm(the case is that we want to extract the table/data lineage of our flink jobs).
{code:java}
lsof -p 8011 | wc -l 
10534
{code}
After checked all files and I found they are opened when loaded the udf jars for each loop to build the env. Then I test this method with try-with-source and it works well. So I wonder if it is necessary to change a little bit just like this

 
{code:java}
private static ClassReader getClassReader(Class<?> cls) {
   final String className = cls.getName().replaceFirst("^.*\\.", "") + ".class";
   try (InputStream inputStream = cls.getResourceAsStream(className)) {
      return new ClassReader(inputStream);
   } catch (IOException e) {
      throw new IllegalStateException("Could not instantiate ClassReader.", e);
   }
}
{code}
 

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)