[jira] [Commented] (SENTRY-901) Possible memory leak in the sentry thrift server

["Sravya Tirukkovalur (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SENTRY-901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14906510#comment-14906510 ] 

Sravya Tirukkovalur commented on SENTRY-901:
--------------------------------------------

I think this might also be related to thrift client not being thread safe (SENTRY-893). 

> Possible memory leak in the sentry thrift server
> ------------------------------------------------
>
>                 Key: SENTRY-901
>                 URL: https://issues.apache.org/jira/browse/SENTRY-901
>             Project: Sentry
>          Issue Type: Bug
>    Affects Versions: 1.4.0
>            Reporter: Sravya Tirukkovalur
>
> Took memory dumps at various times and looks like there are 4 major objects holding almost 100% of the memory. Three of them are byte[] and one of them is a HashMapEntry[].
> 1. byte[]
> - Looks like there is a leak in the transport layer
> - The thread java.lang.Thread @ 0x787991fc8 pool-3-thread-2224 keeps local variables with total size 184,551,168 (24.84%) bytes. The memory is accumulated in one instance of "byte[]" loaded by "<system class loader>".
> - Thread stack:
> {noformat}
> pool-3-thread-2224
>   at java.net.SocketInputStream.socketRead0(Ljava/io/FileDescriptor;[BIII)I (Native Method)
>   at java.net.SocketInputStream.read([BIII)I (SocketInputStream.java:152)
>   at java.net.SocketInputStream.read([BII)I (SocketInputStream.java:122)
>   at java.io.BufferedInputStream.fill()V (BufferedInputStream.java:235)
>   at java.io.BufferedInputStream.read1([BII)I (BufferedInputStream.java:275)
>   at java.io.BufferedInputStream.read([BII)I (BufferedInputStream.java:334)
>   at sentry.org.apache.thrift.transport.TIOStreamTransport.read([BII)I (TIOStreamTransport.java:127)
>   at sentry.org.apache.thrift.transport.TTransport.readAll([BII)I (TTransport.java:84)
>   at sentry.org.apache.thrift.transport.TSaslTransport.readLength()I (TSaslTransport.java:346)
>   at sentry.org.apache.thrift.transport.TSaslTransport.readFrame()V (TSaslTransport.java:423)
>   at sentry.org.apache.thrift.transport.TSaslTransport.read([BII)I (TSaslTransport.java:405)
>   at sentry.org.apache.thrift.transport.TSaslServerTransport.read([BII)I (TSaslServerTransport.java:1)
>   at sentry.org.apache.thrift.transport.TTransport.readAll([BII)I (TTransport.java:84)
>   at sentry.org.apache.thrift.protocol.TBinaryProtocol.readStringBody(I)Ljava/lang/String; (TBinaryProtocol.java:354)
>   at sentry.org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin()Lsentry/org/apache/thrift/protocol/TMessage; (TBinaryProtocol.java:215)
>   at sentry.org.apache.thrift.TMultiplexedProcessor.process(Lsentry/org/apache/thrift/protocol/TProtocol;Lsentry/org/apache/thrift/protocol/TProtocol;)Z (TMultiplexedProcessor.java:92)
>   at sentry.org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run()V (TThreadPoolServer.java:244)
>   at java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V (ThreadPoolExecutor.java:1145)
>   at java.util.concurrent.ThreadPoolExecutor$Worker.run()V (ThreadPoolExecutor.java:615)
>   at java.lang.Thread.run()V (Thread.java:745)
> {noformat}
> - Value looks like is like a series of RPC requests concatenated together
> {noformat}
> ......................server1.......<db_name>.......<tablename>............................server1.......<db_name>......<table_name>.........<rolename>.......................7SentryPolicyService:list_sentry_privileges_for_provider....e..........server1.........xxx...............xxx..............xxxr.......7SentryPolicyService:list_sentry_privileges_for_provider...xx.....................xxx..........................server1.......xxx.......xxx..........7SentryPolicyService:list_sentry_privileges_for_provider...g......................polystar..........................server1.......xxx.......xxx..........7SentryPolicyService:list_sentry_privileges_for_provider...h......................xxx..........................server1.......xxx.......xxx.....................................................................................................................................
> {noformat}
> 2. HashMapEntry[]
> - One instance of "org.datanucleus.NucleusContext" loaded by "sun.misc.Launcher$AppClassLoader @ 0x77ffbb410" occupies 164,277,416 (22.11%) bytes. The memory is accumulated in one instance of "java.util.HashMap$Entry[]" loaded by "<system class loader>".
> - Some entires are null. It has total of ~16000 entries.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)