You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@drill.apache.org by br...@apache.org on 2016/11/01 20:49:56 UTC
drill git commit: edit drill memory page based on changes to product
Repository: drill
Updated Branches:
refs/heads/gh-pages 7345dd9e8 -> 05e201b7f
edit drill memory page based on changes to product
Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/05e201b7
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/05e201b7
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/05e201b7
Branch: refs/heads/gh-pages
Commit: 05e201b7f360bb51f3f25dea61b2bf8194780b70
Parents: 7345dd9
Author: Bridget Bevens <bb...@maprtech.com>
Authored: Tue Nov 1 13:49:06 2016 -0700
Committer: Bridget Bevens <bb...@maprtech.com>
Committed: Tue Nov 1 13:49:06 2016 -0700
----------------------------------------------------------------------
.../020-configuring-drill-memory.md | 27 ++++++++++----------
.../020-physical-operators.md | 13 ++++------
2 files changed, 19 insertions(+), 21 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/drill/blob/05e201b7/_docs/configure-drill/020-configuring-drill-memory.md
----------------------------------------------------------------------
diff --git a/_docs/configure-drill/020-configuring-drill-memory.md b/_docs/configure-drill/020-configuring-drill-memory.md
index 4dc1292..c490ebe 100644
--- a/_docs/configure-drill/020-configuring-drill-memory.md
+++ b/_docs/configure-drill/020-configuring-drill-memory.md
@@ -1,6 +1,6 @@
---
title: "Configuring Drill Memory"
-date: 2016-02-06 00:18:12 UTC
+date: 2016-11-01 20:49:09 UTC
parent: "Configure Drill"
---
@@ -23,9 +23,9 @@ The [`planner.memory.max_query_memory_per_node`]({{site.baseurl}}/docs/configura
## Modifying Drillbit Memory
-You can modify memory for each Drillbit node in your cluster. To modify the
-memory for a Drillbit, edit the `XX:MaxDirectMemorySize` parameter in the
-Drillbit startup script, `drill-env.sh`, located in `<drill_installation_directory>/conf`.
+You can modify memory for each Drillbit node in your cluster. To modify the memory for a Drillbit, set the DRILL_MAX_DIRECT_MEMORY variable in the Drillbit startup script, `drill-env.sh`, located in `<drill_installation_directory>/conf`, as follows:
+
+ export DRILL_MAX_DIRECT_MEMORY=${DRILL_MAX_DIRECT_MEMORY:-"<value>"}
{% include startnote.html %}If XX:MaxDirectMemorySize is not set, the limit depends on the amount of available system memory.{% include endnote.html %}
@@ -35,16 +35,17 @@ After you edit `<drill_installation_directory>/conf/drill-env.sh`, [restart the
The `drill-env.sh` file contains the following options:
- DRILL_MAX_DIRECT_MEMORY="8G"
- DRILL_MAX_HEAP="4G"
+ #export DRILL_HEAP=${DRILL_HEAP:-"4G\u201d}
+ #export DRILL_MAX_DIRECT_MEMORY=${DRILL_MAX_DIRECT_MEMORY:-"8G"}
+
+To customize memory limits, uncomment the line needed and change the setting:
- export DRILL_JAVA_OPTS="-Xms1G -Xmx$DRILL_MAX_HEAP -XX:MaxDirectMemorySize=$DRILL_MAX_DIRECT_MEMORY -XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=1G -ea"
+ export DRILL_HEAP=${DRILL_HEAP:-"<limit>\u201d}
+ export DRILL_MAX_DIRECT_MEMORY=${DRILL_MAX_DIRECT_MEMORY:-\u201c<limit>"}
-* DRILL_MAX_DIRECT_MEMORY is the Java direct memory limit per node.
-* DRILL_MAX_HEAP is the maximum theoretical heap limit for the JVM per node.
-* Xmx specifies the maximum memory allocation pool for a Java Virtual Machine (JVM).
-* Xms specifies the initial memory allocation pool.
+DRILL_MAX_HEAP is the maximum theoretical heap limit for the JVM per node.
+DRILL_MAX_DIRECT_MEMORY is the Java direct memory limit per node.
-If performance is an issue, replace the -ea flag with -Dbounds=false, as shown in the following example:
+If performance is an issue, add -Dbounds=false, as shown in the following example:
- export DRILL_JAVA_OPTS="-Xms1G -Xmx$DRILL_MAX_HEAP -XX:MaxDirectMemorySize=$DRILL_MAX_DIRECT_MEMORY -XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=1G -Dbounds=false"
+ export DRILL_JAVA_OPTS="$DRILL_JAVA_OPTS -Dbounds=false"
http://git-wip-us.apache.org/repos/asf/drill/blob/05e201b7/_docs/performance-tuning/performance-tuning-reference/020-physical-operators.md
----------------------------------------------------------------------
diff --git a/_docs/performance-tuning/performance-tuning-reference/020-physical-operators.md b/_docs/performance-tuning/performance-tuning-reference/020-physical-operators.md
index 215a427..e10fafd 100644
--- a/_docs/performance-tuning/performance-tuning-reference/020-physical-operators.md
+++ b/_docs/performance-tuning/performance-tuning-reference/020-physical-operators.md
@@ -1,6 +1,6 @@
---
title: "Physical Operators"
-date: 2016-09-30 23:20:32 UTC
+date: 2016-11-01 20:49:10 UTC
parent: "Performance Tuning Reference"
---
@@ -51,14 +51,11 @@ Drill uses the following sort and limiter operators:
## Projection Operators
-Drill uses the following projection operators:
+Drill uses the following projection operators:
-| Operator | Description |
-|--------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| Project | A Project operator projects columns and/or expressions involving columns and constants. This operator holds one incoming record batch plus any additional materialized projects for the same number of rows as the incoming record batch. |
-| ExternalSort | The ExternalSort operator can potentially hold the entire dataset in memory. This operator will also start spooling to the disk in the case that there is memory pressure. In this case, the external sort will continue to try to use as much memory as available. In all cases, external sort will hold at least one record batch in memory for each record spill. Spills are currently sized based on the amount of memory available to the external sort operator. |
-| TopN | A TopN operator is used to perform an ORDER BY with LIMIT. |
-| Limit | A Limit operator is used to restrict the number of rows to a value specified by the LIMIT clause. |
+| Operator | Description |
+|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Project | A Project operator projects columns and/or expressions involving columns and constants. This operator holds one incoming record batch plus any additional materialized projects for the same number of rows as the incoming record batch. |
## Filter and Related Operators