[jira] [Work logged] (HADOOP-18105) Implement a variant of ElasticByteBufferPool which uses weak references for garbage collection.

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-18105?focusedWorklogId=766306&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-766306 ]

ASF GitHub Bot logged work on HADOOP-18105:
-------------------------------------------

                Author: ASF GitHub Bot
            Created on: 04/May/22 21:54
            Start Date: 04/May/22 21:54
    Worklog Time Spent: 10m 
      Work Description: mukund-thakur opened a new pull request, #4263:
URL: https://github.com/apache/hadoop/pull/4263

   ### Description of PR
   part of HADOOP-18103.
   Required for vectored IO feature. None of current buffer pool
   implementation is complete. ElasticByteBufferPool doesn't use
   weak refrences and could lead to memory leak errors and
   DirectBufferPool doesn't support caller prefrences of direct
   and heap buffers and has only fixed length buffer implementation.
   
   ### How was this patch tested?
   Added new unit tests and tested through vectored read api integration test.
   
   
   ### For code changes:
   
   - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
   - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
   - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
   
   




Issue Time Tracking
-------------------

            Worklog Id:     (was: 766306)
    Remaining Estimate: 0h
            Time Spent: 10m

> Implement a variant of ElasticByteBufferPool which uses weak references for garbage collection.
> -----------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-18105
>                 URL: https://issues.apache.org/jira/browse/HADOOP-18105
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: common, fs
>            Reporter: Mukund Thakur
>            Assignee: Mukund Thakur
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Currently in hadoop codebase, we have two classes which implements byte buffers pooling.
> One is ElasticByteBufferPool which doesn't use weak references and thus could cause memory leaks in production environment. 
> Other is DirectBufferPool which uses weak references but doesn't support caller's preference for either on-heap or off-heap buffers. 
>  
> The idea is to create an improved version of ElasticByteBufferPool by subclassing it ( as it is marked as public and stable and used widely in hdfs ) with essential functionalities required for effective buffer pooling. This is important for the parent Vectored IO work.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org