[jira] [Commented] (CASSANDRA-10837) Cluster/session should be closed in Cassandra Hadoop Input/Output classes

["Benjamin Lerer (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-10837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15058713#comment-15058713 ] 

Benjamin Lerer commented on CASSANDRA-10837:
--------------------------------------------

[~alexliu68] Thanks for the patch.

If I am not mistaken Netty is using {{HashedWheelTimer}} to schedule task within the {{EventLoop}} and a new {{EventLoop}} will be created for each new {{Cluster}} instance.

I ran some simple tests using only the java driver. They confirmed that the ERROR is logged only when too many {{Cluster}} instances are created.

Regarding the patch I have the following comments:
* The changes to {{CqlRecordWriter}} seems wrong to me. 
## The {{Cluster}} and the {{Session}} instances were both properly managed by the [try-with-resources|https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html] statement. In the patch version only the {{Cluster}} instance is closed. 
## Making the {{Cluster}} an instance variable in {{NativeRingCache}} will trigger an error when the {{write}} method will be called, as the {{Cluster}} has been closed at the end of the constructor.
* In {{CqlInputFormat}} could you use {{try-with-resources}} for both {{Cluster}} and {{Session}} instances. I think it is best to do things properly by closing both of them. 

 

> Cluster/session should be closed in Cassandra Hadoop Input/Output classes
> -------------------------------------------------------------------------
>
>                 Key: CASSANDRA-10837
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10837
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Alex Liu
>            Assignee: Alex Liu
>             Fix For: 3.0.x
>
>         Attachments: 10837-3.0-branch.txt, 10837-v2-3.0-branch.txt
>
>
> See a lot of following warnings during Hadoop job running
> {code}
> ERROR 11:37:45 LEAK: You are creating too many HashedWheelTimer instances.  HashedWheelTimer is a shared resource that must be reused across the JVM,so that only a few instances are created.
> {code}
> Each cluster/session needs be closed and a shared HashedWheelTimer may reduce the resource leakage.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)