You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hawq.apache.org by yo...@apache.org on 2016/10/14 23:28:17 UTC
[1/3] incubator-hawq-docs git commit: Note about work_mem
Repository: incubator-hawq-docs
Updated Branches:
refs/heads/develop f4bd3ef5a -> 3df5448c8
Note about work_mem
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/bf16a062
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/bf16a062
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/bf16a062
Branch: refs/heads/develop
Commit: bf16a0621bdd5783c5b1400023fa0ab3df9b28e0
Parents: e169704
Author: Jane Beckman <jb...@pivotal.io>
Authored: Wed Oct 12 12:17:36 2016 -0700
Committer: Jane Beckman <jb...@pivotal.io>
Committed: Wed Oct 12 12:17:36 2016 -0700
----------------------------------------------------------------------
bestpractices/querying_data_bestpractices.html.md.erb | 3 +++
query/query-profiling.html.md.erb | 2 ++
reference/sql/EXPLAIN.html.md.erb | 4 +++-
3 files changed, 8 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/bf16a062/bestpractices/querying_data_bestpractices.html.md.erb
----------------------------------------------------------------------
diff --git a/bestpractices/querying_data_bestpractices.html.md.erb b/bestpractices/querying_data_bestpractices.html.md.erb
index 8a76c6d..7a86176 100644
--- a/bestpractices/querying_data_bestpractices.html.md.erb
+++ b/bestpractices/querying_data_bestpractices.html.md.erb
@@ -20,6 +20,9 @@ If a query performs poorly, examine its query plan and ask the following questio
`Work_mem used: 23430K bytes avg, 23430K bytes max (seg0). Work_mem wanted: 33649K bytes avg, 33649K bytes max (seg0) to lessen workfile I/O affecting 2 workers.`
+**Note**
+The *work\_mem* property is not configurable. Use resource queues to manage memory use.
+
The "bytes wanted" message from `EXPLAIN ANALYZE` is based on the amount of data written to work files and is not exact. The minimum `work_mem` needed can differ from the suggested value.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/bf16a062/query/query-profiling.html.md.erb
----------------------------------------------------------------------
diff --git a/query/query-profiling.html.md.erb b/query/query-profiling.html.md.erb
index ca07f6d..8d770d3 100644
--- a/query/query-profiling.html.md.erb
+++ b/query/query-profiling.html.md.erb
@@ -88,6 +88,8 @@ The estimated startup cost for this plan is `00.00` (no cost) and a total cost o
Work_mem wanted: 90K bytes avg, 90K byes max (seg0) to lessen
workfile I/O affecting 2 workers.
```
+**Note**
+The *work\_mem* property is not configurable. Use resource queues to manage memory use.
- The time (in milliseconds) in which the segment that produced the most rows retrieved the first row, and the time taken for that segment to retrieve all rows. The result may omit *<time> to first row* if it is the same as the *<time> to end*.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/bf16a062/reference/sql/EXPLAIN.html.md.erb
----------------------------------------------------------------------
diff --git a/reference/sql/EXPLAIN.html.md.erb b/reference/sql/EXPLAIN.html.md.erb
index 6d0befd..ca0e908 100644
--- a/reference/sql/EXPLAIN.html.md.erb
+++ b/reference/sql/EXPLAIN.html.md.erb
@@ -41,7 +41,9 @@ It is important to note that the cost of an upper-level node includes the cost o
workfile
[seg0] pass 1: 263 groups made from 263 rows
```
-
+**Note**
+You cannot set the *work\_mem* property. The *work\_mem* property is for information only.
+
- The time (in milliseconds) it took to retrieve the first row from the segment that produced the most rows, and the total time taken to retrieve all rows from that segment. The `\<time\> to first row` may be omitted if it is the same as the `\<time\> to end`.
**Important:**
[2/3] incubator-hawq-docs git commit: Links to resource queue
sections.
Posted by yo...@apache.org.
Links to resource queue sections.
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/5edd907d
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/5edd907d
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/5edd907d
Branch: refs/heads/develop
Commit: 5edd907de2b353b6e13e954524fbca9d4a3a7787
Parents: bf16a06
Author: Jane Beckman <jb...@pivotal.io>
Authored: Fri Oct 14 14:31:41 2016 -0700
Committer: Jane Beckman <jb...@pivotal.io>
Committed: Fri Oct 14 14:31:41 2016 -0700
----------------------------------------------------------------------
bestpractices/querying_data_bestpractices.html.md.erb | 5 +++--
query/query-profiling.html.md.erb | 2 +-
2 files changed, 4 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/5edd907d/bestpractices/querying_data_bestpractices.html.md.erb
----------------------------------------------------------------------
diff --git a/bestpractices/querying_data_bestpractices.html.md.erb b/bestpractices/querying_data_bestpractices.html.md.erb
index 7a86176..15fecfe 100644
--- a/bestpractices/querying_data_bestpractices.html.md.erb
+++ b/bestpractices/querying_data_bestpractices.html.md.erb
@@ -16,12 +16,13 @@ If a query performs poorly, examine its query plan and ask the following questio
If the plan is not choosing the optimal join order, set `join_collapse_limit=1` and use explicit `JOIN` syntax in your SQL statement to force the legacy query optimizer (planner) to the specified join order. You can also collect more statistics on the relevant join columns.
- **Does the optimizer selectively scan partitioned tables?** If you use table partitioning, is the optimizer selectively scanning only the child tables required to satisfy the query predicates? Scans of the parent tables should return 0 rows since the parent tables do not contain any data. See [Verifying Your Partition Strategy](../ddl/ddl-partition.html#topic74) for an example of a query plan that shows a selective partition scan.
-- **Does the optimizer choose hash aggregate and hash join operations where applicable?** Hash operations are typically much faster than other types of joins or aggregations. Row comparison and sorting is done in memory rather than reading/writing from disk. To enable the query optimizer to choose hash operations, there must be sufficient memory available to hold the estimated number of rows. Try increasing work memory to improve performance for a query. If possible, run an `EXPLAIN ANALYZE` for the query to show which plan operations spilled to disk, how much work memory they used, and how much memory was required to avoid spilling to disk. For example:
+- **Does the optimizer choose hash aggregate and hash join operations where applicable?** Hash operations are typically much faster than other types of joins or aggregations. Row comparison and sorting is done in memory rather than reading/writing from disk. To enable the query optimizer to choose hash operations, there must be sufficient memory available to hold the estimated number of rows. Try increasing work memory to improve performance for a query. If possible, run an `EXPLAIN ANALYZE` for the query to show which plan operations spilled to disk, how much work memory they used, and how much memory was required to avoid spilling to disk. For example:
`Work_mem used: 23430K bytes avg, 23430K bytes max (seg0). Work_mem wanted: 33649K bytes avg, 33649K bytes max (seg0) to lessen workfile I/O affecting 2 workers.`
**Note**
-The *work\_mem* property is not configurable. Use resource queues to manage memory use.
+The *work\_mem* property is not configurable. Use resource queues to manage memory use. For more information on resource queues, see [Configuring Resource Management](../resourcemgmt/ConfigureResourceManagement.html) and [Working with Hierarchical Resource Queues](../resourcemgmt/ResourceQueues.html).
+
The "bytes wanted" message from `EXPLAIN ANALYZE` is based on the amount of data written to work files and is not exact. The minimum `work_mem` needed can differ from the suggested value.
http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/blob/5edd907d/query/query-profiling.html.md.erb
----------------------------------------------------------------------
diff --git a/query/query-profiling.html.md.erb b/query/query-profiling.html.md.erb
index 8d770d3..b3139cf 100644
--- a/query/query-profiling.html.md.erb
+++ b/query/query-profiling.html.md.erb
@@ -89,7 +89,7 @@ The estimated startup cost for this plan is `00.00` (no cost) and a total cost o
workfile I/O affecting 2 workers.
```
**Note**
-The *work\_mem* property is not configurable. Use resource queues to manage memory use.
+The *work\_mem* property is not configurable. Use resource queues to manage memory use. For more information on resource queues, see [Configuring Resource Management](../resourcemgmt/ConfigureResourceManagement.html) and [Working with Hierarchical Resource Queues](../resourcemgmt/ResourceQueues.html).
- The time (in milliseconds) in which the segment that produced the most rows retrieved the first row, and the time taken for that segment to retrieve all rows. The result may omit *<time> to first row* if it is the same as the *<time> to end*.
[3/3] incubator-hawq-docs git commit: Merge branch 'feature/work_mem'
of https://github.com/janebeckman/incubator-hawq-docs into develop
Posted by yo...@apache.org.
Merge branch 'feature/work_mem' of https://github.com/janebeckman/incubator-hawq-docs into develop
Project: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/commit/3df5448c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/tree/3df5448c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-hawq-docs/diff/3df5448c
Branch: refs/heads/develop
Commit: 3df5448c8f7296e7c3245cacbdc3a93f869859cb
Parents: f4bd3ef 5edd907
Author: David Yozie <yo...@apache.org>
Authored: Fri Oct 14 16:28:09 2016 -0700
Committer: David Yozie <yo...@apache.org>
Committed: Fri Oct 14 16:28:09 2016 -0700
----------------------------------------------------------------------
bestpractices/querying_data_bestpractices.html.md.erb | 6 +++++-
query/query-profiling.html.md.erb | 2 ++
reference/sql/EXPLAIN.html.md.erb | 4 +++-
3 files changed, 10 insertions(+), 2 deletions(-)
----------------------------------------------------------------------