[jira] [Work logged] (HADOOP-18091) S3A auditing leaks memory through ThreadLocal references

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-18091?focusedWorklogId=713398&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713398 ]

ASF GitHub Bot logged work on HADOOP-18091:
-------------------------------------------

                Author: ASF GitHub Bot
            Created on: 23/Jan/22 19:16
            Start Date: 23/Jan/22 19:16
    Worklog Time Spent: 10m 
      Work Description: steveloughran opened a new pull request #3916:
URL: https://github.com/apache/hadoop/pull/3916


   This patch does not fix the problem, but it allows for auditing
   to be disabled such that no ThreadLocal fields are created.
   By disabling auditing by default, memory leaks will not surface.
   
   Auditing to S3 Logs can be enabled if required; short-lived
   applications and/or applications which use the same limited set of
   S3A instances should be safe to use with auditing.
   
   Change-Id: I9090d65c896a12feb6826e581935078ffb737bab
   
   <!--
     Thanks for sending a pull request!
       1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
       2. Make sure your PR title starts with JIRA issue id, e.g., 'HADOOP-17799. Your PR title ...'.
   -->
   
   ### Description of PR
   
   
   ### How was this patch tested?
   
   
   ### For code changes:
   
   - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
   - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
   - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
   - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files?
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

            Worklog Id:     (was: 713398)
    Remaining Estimate: 0h
            Time Spent: 10m

> S3A auditing leaks memory through ThreadLocal references
> --------------------------------------------------------
>
>                 Key: HADOOP-18091
>                 URL: https://issues.apache.org/jira/browse/HADOOP-18091
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.3.2
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>            Priority: Blocker
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> {{ActiveAuditManagerS3A}} uses thread locals to map to active audit spans, which (because they are wrapped) include back reference to the audit manager instance and the config it was created with.
> these *do not* get cleaned up when the FS instance is closed.
> if you have a long lived process creating and destroying many FS instances, then memory gets used up. l



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org