You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "ken (JIRA)" <ji...@apache.org> on 2019/02/05 06:30:00 UTC
[jira] [Commented] (DRILL-6935) Apache drill show a lot of
CLOSE_WAIT states when we access https://ip address:8047 caused the URL is
not available
[ https://issues.apache.org/jira/browse/DRILL-6935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16760501#comment-16760501 ]
ken commented on DRILL-6935:
----------------------------
Hello,
Is there any update for this?
> Apache drill show a lot of CLOSE_WAIT states when we access https://ip address:8047 caused the URL is not available
> -------------------------------------------------------------------------------------------------------------------
>
> Key: DRILL-6935
> URL: https://issues.apache.org/jira/browse/DRILL-6935
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill
> Affects Versions: 1.13.0
> Reporter: ken
> Priority: Critical
>
> Hi Team,
> Hope all is good.
> We need your help.
> Here is the apache drill process which we installed in our server.
> drill 19220 1 17 16:48 ? 00:15:32 /usr/java/jdk/bin/java > -Xms8G -Xmx8G -XX:MaxDirectMemorySize=96G -XX:ReservedCodeCacheSize=1024m > -Ddrill.exec.enable-epoll=false -XX:+CMSClassUnloadingEnabled -XX:+UseG1GC > -Dlog.path=/var/log/drill/drillbit.log > -Dlog.query.path=/var/log/drill/drillbit_queries.json -cp > /usr/local/apache-drill-1.13.1/conf:/usr/local/apache-drill-1.13.1/jars/*:/usr/local/apache-drill-1.13.1/jars/ext/*:/usr/local/apache-drill-1.13.1/jars/3rdparty/*:/usr/local/apache-drill-1.13.1/jars/classb/*:/usr/local/apache-drill-1.13.1/jars/3rdparty/linux/* > org.apache.drill.exec.server.Drillbit > root 23651 23227 0 18:16 pts/1 00:00:00 grep --color=auto java
> Question 1:
> There are a lot of CLOSE_WAIT states when I access apache drill [https://ip|https://ip/] > address:8047 <[https://theremin.digitalalchemy.net.au:8047/]> I have changed > our server ip to xxxx for the secruity reason, this caused that we can't > access apache drill by [https://ip|https://ip/] address:8047 > <[https://theremin.digitalalchemy.net.au:8047/]>, so we can't check which SQL run failed.
> tcp6 0 0 :::8047 :::* LISTEN > 19220/java > tcp6 518 0 192.168.xxxx:8047 192.168.100.131:54132 > CLOSE_WAIT 19220/java > tcp6 1 0 192.168. xxxx :8047 192.168.100.222:52986
> CLOSE_WAIT 19220/java > tcp6 518 0 192.168. xxxx :8047 192.168.100.222:53009
> CLOSE_WAIT 19220/java > tcp6 518 0 192.168. xxxx :8047 192.168.100.131:54131
> CLOSE_WAIT 19220/java > tcp6 1 0 192.168. xxxx :8047 192.168.3.119:61202
> CLOSE_WAIT 19220/java > tcp6 518 0 192.168. xxxx :8047 192.168.100.131:54366
> CLOSE_WAIT 19220/java > tcp6 518 0 192.168. xxxx :8047 192.168.100.131:54129
> CLOSE_WAIT 19220/java > tcp6 518 0 192.168. xxxx :8047 192.168.100.131:58627
> CLOSE_WAIT 19220/java > tcp6 518 0 192.168. xxxx :8047 192.168.100.131:58486
> CLOSE_WAIT 19220/java > tcp6 518 0 192.168. xxxx :8047 192.168.100.131:54134
> CLOSE_WAIT 19220/java > tcp6 518 0 192.168. xxxx :8047 192.168.100.222:53008
> CLOSE_WAIT 19220/java > tcp6 1 0 192.168. xxxx :8047 192.168.3.119:56226
> CLOSE_WAIT 19220/java > tcp6 518 0 192.168. xxxx :8047 192.168.100.222:52991
> CLOSE_WAIT 19220/java > tcp6 1 0 192.168. xxxx :8047 192.168.3.119:51172
> CLOSE_WAIT 19220/java > tcp6 1 0 192.168. xxxx :8047 192.168.3.119:36136
> CLOSE_WAIT 19220/java > tcp6 518 0 192.168. xxxx :8047 192.168.100.131:54133
> CLOSE_WAIT 19220/java > tcp6 24 0 192.168. xxxx :8047 192.168.100.131:57474
> ESTABLISHED 19220/java > tcp6 518 0 192.168. xxxx :8047 192.168.100.131:54069 CLOSE_WAIT 19220/java > tcp6 518 0 192.168. xxxx :8047 192.168.100.131:54130
> CLOSE_WAIT 19220/java > tcp6 518 0 192.168. xxxx :8047 192.168.100.222:53001
> CLOSE_WAIT 19220/java > tcp6 518 0 192.168. xxxx :8047 192.168.100.222:52985
> CLOSE_WAIT 19220/java > tcp6 518 0 192.168. xxxx :8047 192.168.100.222:52990
> CLOSE_WAIT 19220/java > tcp6 518 0 192.168. xxxx :8047 192.168.100.131:54212
> CLOSE_WAIT 19220/java > tcp6 1 0 192.168. xxxx :8047 192.168.100.131:58628
> CLOSE_WAIT 19220/java > tcp6 1 0 192.168. xxxx :8047 192.168.100.131:53955
> CLOSE_WAIT 19220/java > tcp6 518 0 192.168. xxxx :8047 192.168.100.131:57391
> CLOSE_WAIT 19220/java > tcp6 1 0 192.168. xxxx :8047 192.168.3.119:41219
> CLOSE_WAIT 19220/java > tcp6 518 0 192.168. xxxx :8047 192.168.100.131:54307
> CLOSE_WAIT 19220/java > tcp6 518 0 192.168. xxxx :8047 192.168.100.222:53000
> CLOSE_WAIT 19220/java > tcp6 518 0 192.168 xxxx :8047 192.168.100.222:52984
> CLOSE_WAIT 19220/java > tcp6 518 0 192.168. xxxx :8047 192.168.100.131:54308
> CLOSE_WAIT 19220/java > tcp6 1 0 192.168. xxxx :8047 192.168.3.119:46189
> CLOSE_WAIT 19220/java > tcp6 518 0 192.168. xxxx :8047 192.168.100.131:54211
> CLOSE_WAIT 19220/java > > > >
> Question 2:
> Our apache drill was down frequently, it seems that it is due to memory leak. However, we have configured 96G memory for apache dirll, so can you please advise how can we identify which SQL took a lot of memory? and how can improve our performance? Error Id: 40d789a6-91ee-4e0b-bfc9-a26358a43df3 on server:31010] > org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: > IllegalStateException: Memory was leaked by query. Memory leaked: (67043328) > Allocator(op:14:0:0:HashPartitionSender) > 1000000/67043328/101535744/10000000000 (res/actual/peak/limit) > > > Fragment 14:0 > > [Error Id: 40d789a6-91ee-4e0b-bfc9-a26358a43df3 on > theremin.root.digitalalchemy:31010] > at > org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633) > ~[drill-common-1.13.0.jar:1.13.0] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:300) > [drill-java-exec-1.13.0.jar:1.13.0] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160) > [drill-java-exec-1.13.0.jar:1.13.0] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:266) > [drill-java-exec-1.13.0.jar:1.13.0] > at > org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) > [drill-common-1.13.0.jar:1.13.0] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [na:1.8.0_161] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [na:1.8.0_161] > at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161] > > ) > > Thank you. >
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)