[GitHub] [flink] StephanEwen opened a new pull request #10447: [FLINK-15084][runtime] Add shared resources tracking to MemoryManager

[GitBox <gi...@apache.org>]

StephanEwen opened a new pull request #10447: [FLINK-15084][runtime] Add shared resources tracking to MemoryManager
URL: https://github.com/apache/flink/pull/10447
 
 
   ## What is the purpose of the change
   
   This change allows the memory manager to track shared opaque resources form other components, like for example block caches from RocksDB. The memory manager is responsible for allocating and releasing those resources, as well as for assigning the memory budget from of the managed memory budget.
   
   The core is the `getSharedMemoryResourceForManagedMemory(type, initializer, fraction)` method.
   It looks up if a resource identified by `type` already exists. If not, it reserves `fraction` amount of the managed memory and uses `initializer` to create the resource.
   
   Every acquisition of the resource creates a new `OpaqueMemoryResource` objects which the users close. Once all are closed, the resource is disposed internally and the reserved memory is returned.
   
   ## Verifying this change
   
   This change adds new unit test classes for the memory manager: 
     - [MemoryManagerSharedResourcesTest](https://github.com/StephanEwen/flink/blob/fba7ae37fda85dbac2cd235e5c1c049fe33df3df/flink-runtime/src/test/java/org/apache/flink/runtime/memory/MemoryManagerSharedResourcesTest.java)
     - [SharedResourcesTest](https://github.com/StephanEwen/flink/blob/fba7ae37fda85dbac2cd235e5c1c049fe33df3df/flink-runtime/src/test/java/org/apache/flink/runtime/memory/SharedResourcesTest.java)
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): **no**
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: **no**
     - The serializers: **no**
     - The runtime per-record code paths (performance sensitive): **no**
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: **no**
     - The S3 file system connector: **no**
   
   ## Documentation
   
     - Does this pull request introduce a new feature? (**only internal, not user facing**)
     - If yes, how is the feature documented? **not applicable**
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services