You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "James Turton (Jira)" <ji...@apache.org> on 2022/08/24 05:27:00 UTC
[jira] [Updated] (DRILL-8283) Add a configurable recursive file listing size limit
[ https://issues.apache.org/jira/browse/DRILL-8283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
James Turton updated DRILL-8283:
--------------------------------
Summary: Add a configurable recursive file listing size limit (was: Implement a configurable file listing size limit)
> Add a configurable recursive file listing size limit
> ----------------------------------------------------
>
> Key: DRILL-8283
> URL: https://issues.apache.org/jira/browse/DRILL-8283
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - Other
> Affects Versions: 1.20.2
> Reporter: James Turton
> Assignee: James Turton
> Priority: Minor
> Fix For: 1.20.3
>
>
> Currently a malicious, or merely an unwitting user can crash their Drill foreman by sending
> {code:java}
> select * from dfs.huge_workspace limit 10
> {code}
> causing the query planner to recurse over every file in huge_workspace and culminating in
> {code:java}
> 2022-08-09 15:13:22,251 [1d0da29f-e50c-fd51-43d9-8a5086d52c4e:foreman] ERROR o.a.drill.common.CatastrophicFailure - Catastrophic Failure Occurred, exiting. Information message: Unable to handle out of memory condition in Foreman.java.lang.OutOfMemoryError: null {code}
> if there are enough files in huge_workspace. A SHOW FILES command can produce the same effect. This issue proposes a new BOOT option named drill.exec.storage.file.max_listing_size with a default value of, say 10 000. If a file listing task exceeds this limit then the current operation is terminated with a UserException and runaway resource usage is prevented.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)