You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@drill.apache.org by br...@apache.org on 2016/02/06 01:25:14 UTC
drill-site git commit: updates/edits for drill 1.5
Repository: drill-site
Updated Branches:
refs/heads/asf-site 1ae8531a4 -> 2cb56a575
updates/edits for drill 1.5
Project: http://git-wip-us.apache.org/repos/asf/drill-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill-site/commit/2cb56a57
Tree: http://git-wip-us.apache.org/repos/asf/drill-site/tree/2cb56a57
Diff: http://git-wip-us.apache.org/repos/asf/drill-site/diff/2cb56a57
Branch: refs/heads/asf-site
Commit: 2cb56a5753f3c802fb82584c14ec039cc394a571
Parents: 1ae8531
Author: Bridget Bevens <bb...@maprtech.com>
Authored: Fri Feb 5 16:25:01 2016 -0800
Committer: Bridget Bevens <bb...@maprtech.com>
Committed: Fri Feb 5 16:25:01 2016 -0800
----------------------------------------------------------------------
docs/configuration-options-introduction/index.html | 4 ++--
docs/configuring-drill-memory/index.html | 6 ++++--
.../index.html | 6 +++---
docs/troubleshooting/index.html | 8 +++++---
feed.xml | 4 ++--
5 files changed, 16 insertions(+), 12 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/drill-site/blob/2cb56a57/docs/configuration-options-introduction/index.html
----------------------------------------------------------------------
diff --git a/docs/configuration-options-introduction/index.html b/docs/configuration-options-introduction/index.html
index 851655a..34cab78 100644
--- a/docs/configuration-options-introduction/index.html
+++ b/docs/configuration-options-introduction/index.html
@@ -1044,7 +1044,7 @@
</div>
-
+ Feb 6, 2016
<link href="/css/docpage.css" rel="stylesheet" type="text/css">
@@ -1293,7 +1293,7 @@ Drill sources the local <code><drill_installation_directory>/conf</code> d
<tr>
<td>planner.memory.max_query_memory_per_node</td>
<td>2147483648 bytes</td>
-<td>Sets the maximum estimate of memory for a query per node in bytes. If the estimate is too low, Drill re-plans the query without memory-constrained operators.</td>
+<td>Sets the maximum amount of direct memory allocated to the sort operator in each query on a node. If a query plan contains multiple sort operators, they all share this memory. If you encounter memory issues when running queries with sort operators, increase the value of this option.</td>
</tr>
<tr>
<td>planner.memory.non_blocking_operators_memory</td>
http://git-wip-us.apache.org/repos/asf/drill-site/blob/2cb56a57/docs/configuring-drill-memory/index.html
----------------------------------------------------------------------
diff --git a/docs/configuring-drill-memory/index.html b/docs/configuring-drill-memory/index.html
index 64e1a49..59b30b8 100644
--- a/docs/configuring-drill-memory/index.html
+++ b/docs/configuring-drill-memory/index.html
@@ -1042,7 +1042,7 @@
</div>
-
+ Feb 6, 2016
<link href="/css/docpage.css" rel="stylesheet" type="text/css">
@@ -1059,7 +1059,9 @@ is written to disk during each phase of a job.</p>
a Drillbit. The on-heap memory for Drill is typically set at 4-8G (default is 4), which should
suffice because Drill avoids having data sit in heap memory.</p>
-<p>The <a href="/docs/configuration-options-introduction/#system-options"><code>planner.memory.max_query_memory_per_node</code></a> system option value determines the memory limits per node for each running query, especially for those involving external sorts, such as window functions. When you have a large amount of direct memory allocated, but still encounter memory issues when running these queries, increase the value of the option.</p>
+<p>As of Drill 1.5, Drill uses a new allocator that improves an operator’s use of direct memory and tracks the memory use more accurately. Due to this change, the sort operator (in queries that ran successfully in previous releases) may not have enough memory, resulting in a failed query and out of memory error instead of spilling to disk.</p>
+
+<p>The <a href="/docs/configuration-options-introduction/#system-options"><code>planner.memory.max_query_memory_per_node</code></a> system option value sets the maximum amount of direct memory allocated to the sort operator in each query on a node. If a query plan contains multiple sort operators, they all share this memory. If you encounter memory issues when running queries with sort operators, increase the value of this option. If you continue to encounter memory issues after increasing this value, you can also reduce the value of the <a href="/docs/configuration-options-introduction/"><code>planner.width.max_per_node</code></a> option to reduce the level of parallelism per node. However, this may increase the amount of time required for a query to complete. </p>
<h2 id="modifying-drillbit-memory">Modifying Drillbit Memory</h2>
http://git-wip-us.apache.org/repos/asf/drill-site/blob/2cb56a57/docs/sort-based-and-hash-based-memory-constrained-operators/index.html
----------------------------------------------------------------------
diff --git a/docs/sort-based-and-hash-based-memory-constrained-operators/index.html b/docs/sort-based-and-hash-based-memory-constrained-operators/index.html
index 246cb6b..7aa1b23 100644
--- a/docs/sort-based-and-hash-based-memory-constrained-operators/index.html
+++ b/docs/sort-based-and-hash-based-memory-constrained-operators/index.html
@@ -1044,7 +1044,7 @@
</div>
-
+ Feb 6, 2016
<link href="/css/docpage.css" rel="stylesheet" type="text/css">
@@ -1077,8 +1077,8 @@ Default: 1073741824 Range: 0 - 1073741824</p></li>
Toggles the state of memory estimation and re-planning of the query. When enabled, Drill conservatively estimates memory requirements and typically excludes memory-constrained operators from the plan and negatively impacts performance.<br>
Default: false</p></li>
<li><p><strong>planner.memory.max_query_memory_per_node</strong><br>
-Sets the maximum estimate of memory for a query per node. If the estimate is too low, Drill re-plans the query without memory-constrained operators.<br>
-Default: 2147483648</p></li>
+Sets the maximum amount of direct memory allocated to the sort operator in each query on a node. If a query plan contains multiple sort operators, they all share this memory. If you encounter memory issues when running queries with sort operators, increase the value of this option.<br>
+Default: 2147483648 </p></li>
</ul>
http://git-wip-us.apache.org/repos/asf/drill-site/blob/2cb56a57/docs/troubleshooting/index.html
----------------------------------------------------------------------
diff --git a/docs/troubleshooting/index.html b/docs/troubleshooting/index.html
index 6cc6738..67f1f4e 100644
--- a/docs/troubleshooting/index.html
+++ b/docs/troubleshooting/index.html
@@ -1040,7 +1040,7 @@
</div>
- Jan 5, 2016
+ Feb 6, 2016
<link href="/css/docpage.css" rel="stylesheet" type="text/css">
@@ -1106,9 +1106,11 @@
<h3 id="memory-issues">Memory Issues</h3>
-<p>Symptom: Memory problems occur when you run certain queries, such as those that perform window functions.</p>
+<p>Symptom: Memory problems occur when you run certain queries, such as those with sort operators.</p>
-<p>Solution: The <a href="/docs/configuration-options-introduction/#system-options"><code>planner.memory.max_query_memory_per_node</code></a> system option value determines the memory limits per node for each running query, especially for those involving external sorts, such as window functions. When you have a large amount of direct memory allocated, but still encounter memory issues when running these queries, increase the value of the option.</p>
+<p>Solution: Increase the value of the <a href="/docs/configuration-options-introduction/#system-options"><code>planner.memory.max_query_memory_per_node</code></a> option, which sets the maximum amount of direct memory allocated to the sort operator in each query on a node. If a query plan contains multiple sort operators, they all share this memory.
+If you continue to encounter memory issues after increasing the <code>planner.memory.max_query_memory_per_node</code> value, you can also reduce the value of the <code>planner.width.max_per_node</code> option to reduce the level of parallelism per node. However, this may increase the amount of time required for a query to complete.
+See <a href="/docs/configuring-drill-memory/">Configuring Drill Memory</a>.</p>
<h3 id="query-parsing-errors">Query Parsing Errors</h3>
http://git-wip-us.apache.org/repos/asf/drill-site/blob/2cb56a57/feed.xml
----------------------------------------------------------------------
diff --git a/feed.xml b/feed.xml
index 0bacba3..f0b787d 100644
--- a/feed.xml
+++ b/feed.xml
@@ -6,8 +6,8 @@
</description>
<link>/</link>
<atom:link href="/feed.xml" rel="self" type="application/rss+xml"/>
- <pubDate>Tue, 02 Feb 2016 16:01:14 -0800</pubDate>
- <lastBuildDate>Tue, 02 Feb 2016 16:01:14 -0800</lastBuildDate>
+ <pubDate>Fri, 05 Feb 2016 16:22:11 -0800</pubDate>
+ <lastBuildDate>Fri, 05 Feb 2016 16:22:11 -0800</lastBuildDate>
<generator>Jekyll v2.5.2</generator>
<item>