[jira] [Commented] (DRILL-8077) Thousand of CLOSE_WAIT connections to HDFS Datanode

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-8077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17460077#comment-17460077 ] 

ASF GitHub Bot commented on DRILL-8077:
---------------------------------------

Z0ltrix opened a new pull request #2406:
URL: https://github.com/apache/drill/pull/2406


   # [DRILL-8077](https://issues.apache.org/jira/browse/DRILL-8077): Thousand of CLOSE_WAIT connections to HDFS Datanode 
   
   ## Description
   
   When Drill runs in an IPv6 environment, it tries to reach the HDFS Datanode over IPv6, but the Datanode does not close the connection correctly. This leads to thousands of CLOSE_WAIT ipv6 connections from Drillbit to Datanode and after a while the Machine runs out of ports and stop working.
   
   To avoid this situation, we can prever IPv4 over IPv6 in drill-env.sh
   
   ## Documentation
   Code is self-explaining
   
   ## Testing
   local build was successful
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Thousand of CLOSE_WAIT connections to HDFS Datanode 
> ----------------------------------------------------
>
>                 Key: DRILL-8077
>                 URL: https://issues.apache.org/jira/browse/DRILL-8077
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Storage - Other
>    Affects Versions: 1.16.0
>            Reporter: Christian Pfarr
>            Priority: Major
>
> In IPv6 environments after a while drill has a lot of connections to HDFS Datanode in CLOSE_WAIT state.
> To fix this we add '-Djava.net.preferIPv4Stack=true -Djava.net.preferIPv6Addresses=false' to our drill configuration and the problem vanished.
> we saw [https://community.cloudera.com/t5/Support-Questions/Over-40000-CLOSE-WAIT-after-hbase-cluster-restart/m-p/6403/highlight/true#M51847] from cloudera that helped.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)