You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2020/07/08 10:25:36 UTC
[GitHub] [druid] ibotheperfect opened a new issue #10153: Too many open files error
ibotheperfect opened a new issue #10153:
URL: https://github.com/apache/druid/issues/10153
Hi.
When I try to run a query I'm getting below error on query screen
> Unknown exception / java.lang.RuntimeException: java.io.FileNotFoundException: var/druid/processing/druid-groupBy-965baa0b-301e-485a-b569-fa6853af9a4f_bb9d8881-5677-406b-9b9c-22d2e996e929/00005516.tmp (Too many open files) / java.lang.RuntimeException / on host slave07:8083
And this is full stacktrace
> 2020-07-08T06:44:10,234 ERROR [processing-9] com.google.common.util.concurrent.Futures$CombinedFuture - input future failed.
> java.lang.RuntimeException: java.io.FileNotFoundException: var/druid/processing/druid-groupBy-fc7a996e-0346-4ed2-9057-882b101efc63_c577d967-e492-44e8-8f69-a0a433509f21/00006647.tmp (Too many open files)
> at org.apache.druid.query.groupby.epinephelinae.SpillingGrouper.read(SpillingGrouper.java:327) ~[druid-processing-0.18.1.jar:0.18.1]
> at org.apache.druid.query.groupby.epinephelinae.SpillingGrouper.iterator(SpillingGrouper.java:252) ~[druid-processing-0.18.1.jar:0.18.1]
> at org.apache.druid.query.groupby.epinephelinae.ConcurrentGrouper$1.call(ConcurrentGrouper.java:351) ~[druid-processing-0.18.1.jar:0.18.1]
> at org.apache.druid.query.groupby.epinephelinae.ConcurrentGrouper$1.call(ConcurrentGrouper.java:347) ~[druid-processing-0.18.1.jar:0.18.1]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_252]
> at org.apache.druid.query.PrioritizedListenableFutureTask.run(PrioritizedExecutorService.java:247) [druid-processing-0.18.1.jar:0.18.1]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_252]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_252]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_252]
> Caused by: java.io.FileNotFoundException: var/druid/processing/druid-groupBy-fc7a996e-0346-4ed2-9057-882b101efc63_c577d967-e492-44e8-8f69-a0a433509f21/00006647.tmp (Too many open files)
> at java.io.FileInputStream.open0(Native Method) ~[?:1.8.0_252]
> at java.io.FileInputStream.open(FileInputStream.java:195) ~[?:1.8.0_252]
> at java.io.FileInputStream.<init>(FileInputStream.java:138) ~[?:1.8.0_252]
> at org.apache.druid.query.groupby.epinephelinae.SpillingGrouper.read(SpillingGrouper.java:322) ~[druid-processing-0.18.1.jar:0.18.1]
> ... 8 more
>
I did a search and every solution I found suggest increasing ulimit but even I set all ulimit parameters to unlimited I keep getting this error.
What else can cause this error?
Druid version: 0.18.1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] ibotheperfect commented on issue #10153: Too many open files error
Posted by GitBox <gi...@apache.org>.
ibotheperfect commented on issue #10153:
URL: https://github.com/apache/druid/issues/10153#issuecomment-655687849
My os is debian 9.8 and here my ulimit and sysctl settings I set
> #ulimit -n
> 999999
> #ulimit -u
> 257220
and sysctl
> fs.file-max = 3233101
> vm.max_map_count = 655300
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] ibotheperfect commented on issue #10153: Too many open files error
Posted by GitBox <gi...@apache.org>.
ibotheperfect commented on issue #10153:
URL: https://github.com/apache/druid/issues/10153#issuecomment-655504627
Actually this query sent by superset as druid sql. Even if I convert it native query, it will not solve my problem.
> may be you have use group by V2 engine ,when there is many result group ,there are many small files split in to the disk;
> so you can limit the use case or use group by engine v1;
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] jihoonson commented on issue #10153: Too many open files error
Posted by GitBox <gi...@apache.org>.
jihoonson commented on issue #10153:
URL: https://github.com/apache/druid/issues/10153#issuecomment-655660736
@ibotheperfect sometimes there are a couple of different settings to set the ulimit properly depending on your OS distribution. Could you please double check if the setting has been applied properly?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] JackyYangPassion commented on issue #10153: Too many open files error
Posted by GitBox <gi...@apache.org>.
JackyYangPassion commented on issue #10153:
URL: https://github.com/apache/druid/issues/10153#issuecomment-655500324
may be you have use group by V2 engine ,when there is many result group ,there are many small files split in to the disk;
so you can limit the use case or use group by engine v1;
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org