You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@hawq.apache.org by "Lav Jain (JIRA)" <ji...@apache.org> on 2018/07/02 19:08:00 UTC

[jira] [Created] (HAWQ-1635) Cache Hadoop UGI for subsequent requests when PXF impersonation is on

Lav Jain created HAWQ-1635:
------------------------------

             Summary: Cache Hadoop UGI for subsequent requests when PXF impersonation is on	
                 Key: HAWQ-1635
                 URL: https://issues.apache.org/jira/browse/HAWQ-1635
             Project: Apache HAWQ
          Issue Type: Improvement
          Components: PXF
            Reporter: Lav Jain
            Assignee: Ed Espino


To fix the memory leak issue as part of HAWQ-1621, we are closing the UGI reference at the end of each PXF request. However, `FileSystem.closeAllForUGI` is an expensive operation and is slowing down PXF almost by a factor of 4 (with impersonation on) compared with impersonation turned off.

The proposal is to cache the UGI object for each PXF query session. The getFragments() call will create a new UGI and cache it using SegmentId:TransactionId. The bridge calls can get this UGI from cache. The cache entry will be cleaned-up periodically based on some timeout after inactivity.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)