You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Philip Lane (Jira)" <ji...@apache.org> on 2021/05/10 15:13:00 UTC
[jira] [Updated] (DRILL-7924) Drill Memory Issues in Kubernetes
[ https://issues.apache.org/jira/browse/DRILL-7924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Philip Lane updated DRILL-7924:
-------------------------------
Description:
{{I am specifying a {{DRILLBIT_MAX_PROC_MEM}} of {{9G}}, a {{DRILL_HEAP}} of {{4G}}, and a {{DRILL_MAX_DIRECT_MEMORY}} of {{4G}} and allowing Drill to determine the {{DRILLBIT_CODE_CACHE_SIZE}}. When I review the Drill startup logs, I can see that Drill is trying to use those values:}}
{{ drill-pod [INFO[] Attempting to start up Drill with the following settings}}
{{ drill-pod DRILL_HEAP=4G}}
{{ drill-pod DRILL_MAX_DIRECT_MEMORY=4G}}
{{ drill-pod DRILLBIT_CODE_CACHE_SIZE=768m}}
{{ drill-pod [WARN[] Total Memory Allocation for Drillbit (8GB) exceeds available free memory (3GB)}}
{{ drill-pod [WARN[] Drillbit will start up, but can potentially crash due to oversubscribing of system memory.}}
{{ drill-pod Starting drillbit, logging to /opt/drill/log/drillbit.out}}
{{ drill-pod [INFO[] Attempting to start up Drill with the following settings}}
{{ drill-pod DRILL_HEAP=4G}}
{{ drill-pod DRILL_MAX_DIRECT_MEMORY=4G}}
{{ drill-pod DRILLBIT_CODE_CACHE_SIZE=768m}}
{{ drill-pod [WARN[] Total Memory Allocation for Drillbit (8GB) exceeds available free memory (3GB)}}
{{ drill-pod [WARN[] Drillbit will start up, but can potentially crash due to oversubscribing of system memory.}}
{{ drill-pod drillbit is running.}}
{{But when it is running, the UI doesn't match for the Direct memory and instead indicates that 0GB of memory is available:}}
{{ !image-2021-05-10-15-50-03-469.png!}}
{{ Note that the errors in the log citing memory allocation exceeds available free memory is irrelevant - I get the same issue when I don't get that error.}}
{{ }}
{{ What can I do to check if this is a UI bug or if Drill really is allocating 0GB to the Direct Memory despite being told to use 4GB? Any idea how I can fix it if it really is happening?}}
was:
I am specifying a {{DRILLBIT_MAX_PROC_MEM}} of {{9G}}, a {{DRILL_HEAP}} of {{4G}}, and a {{DRILL_MAX_DIRECT_MEMORY}} of {{4G}} and allowing Drill to determine the {{DRILLBIT_CODE_CACHE_SIZE}}. When I review the Drill startup logs, I can see that Drill is trying to use those values:
drill-pod [INFO[] Attempting to start up Drill with the following settings
drill-pod DRILL_HEAP=4G
drill-pod DRILL_MAX_DIRECT_MEMORY=4G
drill-pod DRILLBIT_CODE_CACHE_SIZE=768m
drill-pod [WARN[] Total Memory Allocation for Drillbit (8GB) exceeds available free memory (3GB)
drill-pod [WARN[] Drillbit will start up, but can potentially crash due to oversubscribing of system memory.
drill-pod Starting drillbit, logging to /opt/drill/log/drillbit.out
drill-pod [INFO[] Attempting to start up Drill with the following settings
drill-pod DRILL_HEAP=4G
drill-pod DRILL_MAX_DIRECT_MEMORY=4G
drill-pod DRILLBIT_CODE_CACHE_SIZE=768m
drill-pod [WARN[] Total Memory Allocation for Drillbit (8GB) exceeds available free memory (3GB)
drill-pod [WARN[] Drillbit will start up, but can potentially crash due to oversubscribing of system memory.
drill-pod drillbit is running.
But when it is running, the UI doesn't match for the Direct memory and instead indicates that 0GB of memory is available:
!image-2021-05-10-15-50-03-469.png!
Note that the errors in the log citing memory allocation exceeds available free memory is irrelevant - I get the same issue when I don't get that error.
What can I do to check if this is a UI bug or if Drill really is allocating 0GB to the Direct Memory despite being told to use 4GB? Any idea how I can fix it if it really is happening?
> Drill Memory Issues in Kubernetes
> ---------------------------------
>
> Key: DRILL-7924
> URL: https://issues.apache.org/jira/browse/DRILL-7924
> Project: Apache Drill
> Issue Type: Bug
> Components: Client - HTTP, Functions - Drill
> Affects Versions: 1.18.0
> Reporter: Philip Lane
> Priority: Major
> Attachments: image-2021-05-10-15-50-03-469.png
>
>
> {{I am specifying a {{DRILLBIT_MAX_PROC_MEM}} of {{9G}}, a {{DRILL_HEAP}} of {{4G}}, and a {{DRILL_MAX_DIRECT_MEMORY}} of {{4G}} and allowing Drill to determine the {{DRILLBIT_CODE_CACHE_SIZE}}. When I review the Drill startup logs, I can see that Drill is trying to use those values:}}
> {{ drill-pod [INFO[] Attempting to start up Drill with the following settings}}
> {{ drill-pod DRILL_HEAP=4G}}
> {{ drill-pod DRILL_MAX_DIRECT_MEMORY=4G}}
> {{ drill-pod DRILLBIT_CODE_CACHE_SIZE=768m}}
> {{ drill-pod [WARN[] Total Memory Allocation for Drillbit (8GB) exceeds available free memory (3GB)}}
> {{ drill-pod [WARN[] Drillbit will start up, but can potentially crash due to oversubscribing of system memory.}}
> {{ drill-pod Starting drillbit, logging to /opt/drill/log/drillbit.out}}
> {{ drill-pod [INFO[] Attempting to start up Drill with the following settings}}
> {{ drill-pod DRILL_HEAP=4G}}
> {{ drill-pod DRILL_MAX_DIRECT_MEMORY=4G}}
> {{ drill-pod DRILLBIT_CODE_CACHE_SIZE=768m}}
> {{ drill-pod [WARN[] Total Memory Allocation for Drillbit (8GB) exceeds available free memory (3GB)}}
> {{ drill-pod [WARN[] Drillbit will start up, but can potentially crash due to oversubscribing of system memory.}}
> {{ drill-pod drillbit is running.}}
> {{But when it is running, the UI doesn't match for the Direct memory and instead indicates that 0GB of memory is available:}}
> {{ !image-2021-05-10-15-50-03-469.png!}}
> {{ Note that the errors in the log citing memory allocation exceeds available free memory is irrelevant - I get the same issue when I don't get that error.}}
> {{ }}
> {{ What can I do to check if this is a UI bug or if Drill really is allocating 0GB to the Direct Memory despite being told to use 4GB? Any idea how I can fix it if it really is happening?}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)