You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@drill.apache.org by ts...@apache.org on 2015/05/19 18:30:07 UTC
[5/9] drill git commit: fix org of perf tune
fix org of perf tune
Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/fb1c6d4e
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/fb1c6d4e
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/fb1c6d4e
Branch: refs/heads/gh-pages
Commit: fb1c6d4e61504072c144994318e34eb22cacb5ff
Parents: decb1c9
Author: Kristine Hahn <kh...@maprtech.com>
Authored: Tue May 19 00:52:14 2015 -0700
Committer: Kristine Hahn <kh...@maprtech.com>
Committed: Tue May 19 00:52:14 2015 -0700
----------------------------------------------------------------------
_data/docs.json | 502 +++++++++++++------
.../040-query-plans-and-tuning.md | 4 +
.../050-identifying-performance-issues.md | 4 +
.../010-query-plans.md | 74 +++
.../020-query-profiles.md | 142 ++++++
.../010-query-plans-and-tuning-introduction.md | 2 +-
.../020-join-planning-guidelines.md | 2 +-
...030-guidelines-for-optimizing-aggregation.md | 2 +-
.../040-modifying-query-planning-options.md | 2 +-
...d-hash-based-memory-constrained-operators.md | 2 +-
.../060-enabling-query-queuing.md | 2 +-
...to-balance-performance-with-multi-tenancy.md | 2 +-
.../010-query-plans.md | 74 ---
.../020-query-profiles.md | 142 ------
14 files changed, 584 insertions(+), 372 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/drill/blob/fb1c6d4e/_data/docs.json
----------------------------------------------------------------------
diff --git a/_data/docs.json b/_data/docs.json
index 05c3ba3..cd651e9 100644
--- a/_data/docs.json
+++ b/_data/docs.json
@@ -605,8 +605,8 @@
}
],
"children": [],
- "next_title": "Query Plans and Tuning Introduction",
- "next_url": "/docs/query-plans-and-tuning-introduction/",
+ "next_title": "Query Plans and Tuning",
+ "next_url": "/docs/query-plans-and-tuning/",
"parent": "Performance Tuning",
"previous_title": "Partition Pruning",
"previous_url": "/docs/partition-pruning/",
@@ -1670,8 +1670,8 @@
"Controlling Parallelization to Balance Performance with Multi-Tenancy": {
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -1679,9 +1679,9 @@
}
],
"children": [],
- "next_title": "Query Plans",
- "next_url": "/docs/query-plans/",
- "parent": "Query Plans and Tuning Introduction",
+ "next_title": "Identifying Performance Issues",
+ "next_url": "/docs/identifying-performance-issues/",
+ "parent": "Query Plans and Tuning",
"previous_title": "Enabling Query Queuing",
"previous_url": "/docs/enabling-query-queuing/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/070-controlling-parallelization-to-balance-performance-with-multi-tenancy.md",
@@ -2829,8 +2829,8 @@
"Enabling Query Queuing": {
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -2840,7 +2840,7 @@
"children": [],
"next_title": "Controlling Parallelization to Balance Performance with Multi-Tenancy",
"next_url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/",
- "parent": "Query Plans and Tuning Introduction",
+ "parent": "Query Plans and Tuning",
"previous_title": "Sort-Based and Hash-Based Memory-Constrained Operators",
"previous_url": "/docs/sort-based-and-hash-based-memory-constrained-operators/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/060-enabling-query-queuing.md",
@@ -3032,8 +3032,8 @@
"Guidelines for Optimizing Aggregation": {
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -3043,7 +3043,7 @@
"children": [],
"next_title": "Modifying Query Planning Options",
"next_url": "/docs/modifying-query-planning-options/",
- "parent": "Query Plans and Tuning Introduction",
+ "parent": "Query Plans and Tuning",
"previous_title": "Join Planning Guidelines",
"previous_url": "/docs/join-planning-guidelines/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/030-guidelines-for-optimizing-aggregation.md",
@@ -3147,6 +3147,66 @@
"title": "How to Run the Drill Demo",
"url": "/docs/how-to-run-the-drill-demo/"
},
+ "Identifying Performance Issues": {
+ "breadcrumbs": [
+ {
+ "title": "Performance Tuning",
+ "url": "/docs/performance-tuning/"
+ }
+ ],
+ "children": [
+ {
+ "breadcrumbs": [
+ {
+ "title": "Identifying Performance Issues",
+ "url": "/docs/identifying-performance-issues/"
+ },
+ {
+ "title": "Performance Tuning",
+ "url": "/docs/performance-tuning/"
+ }
+ ],
+ "children": [],
+ "next_title": "Query Profiles",
+ "next_url": "/docs/query-profiles/",
+ "parent": "Identifying Performance Issues",
+ "previous_title": "Identifying Performance Issues",
+ "previous_url": "/docs/identifying-performance-issues/",
+ "relative_path": "_docs/performance-tuning/identifying-performance-issues/010-query-plans.md",
+ "title": "Query Plans",
+ "url": "/docs/query-plans/"
+ },
+ {
+ "breadcrumbs": [
+ {
+ "title": "Identifying Performance Issues",
+ "url": "/docs/identifying-performance-issues/"
+ },
+ {
+ "title": "Performance Tuning",
+ "url": "/docs/performance-tuning/"
+ }
+ ],
+ "children": [],
+ "next_title": "Log and Debug",
+ "next_url": "/docs/log-and-debug/",
+ "parent": "Identifying Performance Issues",
+ "previous_title": "Query Plans",
+ "previous_url": "/docs/query-plans/",
+ "relative_path": "_docs/performance-tuning/identifying-performance-issues/020-query-profiles.md",
+ "title": "Query Profiles",
+ "url": "/docs/query-profiles/"
+ }
+ ],
+ "next_title": "Query Plans",
+ "next_url": "/docs/query-plans/",
+ "parent": "Performance Tuning",
+ "previous_title": "Controlling Parallelization to Balance Performance with Multi-Tenancy",
+ "previous_url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/",
+ "relative_path": "_docs/performance-tuning/050-identifying-performance-issues.md",
+ "title": "Identifying Performance Issues",
+ "url": "/docs/identifying-performance-issues/"
+ },
"Install Drill": {
"breadcrumbs": [],
"children": [
@@ -3786,8 +3846,8 @@
"Join Planning Guidelines": {
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -3797,7 +3857,7 @@
"children": [],
"next_title": "Guidelines for Optimizing Aggregation",
"next_url": "/docs/guidelines-for-optimizing-aggregation/",
- "parent": "Query Plans and Tuning Introduction",
+ "parent": "Query Plans and Tuning",
"previous_title": "Query Plans and Tuning Introduction",
"previous_url": "/docs/query-plans-and-tuning-introduction/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/020-join-planning-guidelines.md",
@@ -4226,8 +4286,8 @@
"Modifying Query Planning Options": {
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -4237,7 +4297,7 @@
"children": [],
"next_title": "Sort-Based and Hash-Based Memory-Constrained Operators",
"next_url": "/docs/sort-based-and-hash-based-memory-constrained-operators/",
- "parent": "Query Plans and Tuning Introduction",
+ "parent": "Query Plans and Tuning",
"previous_title": "Guidelines for Optimizing Aggregation",
"previous_url": "/docs/guidelines-for-optimizing-aggregation/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/040-modifying-query-planning-options.md",
@@ -4997,8 +5057,8 @@
}
],
"children": [],
- "next_title": "Query Plans and Tuning Introduction",
- "next_url": "/docs/query-plans-and-tuning-introduction/",
+ "next_title": "Query Plans and Tuning",
+ "next_url": "/docs/query-plans-and-tuning/",
"parent": "Performance Tuning",
"previous_title": "Partition Pruning",
"previous_url": "/docs/partition-pruning/",
@@ -5017,8 +5077,29 @@
{
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
+ },
+ {
+ "title": "Performance Tuning",
+ "url": "/docs/performance-tuning/"
+ }
+ ],
+ "children": [],
+ "next_title": "Join Planning Guidelines",
+ "next_url": "/docs/join-planning-guidelines/",
+ "parent": "Query Plans and Tuning",
+ "previous_title": "Query Plans and Tuning",
+ "previous_url": "/docs/query-plans-and-tuning/",
+ "relative_path": "_docs/performance-tuning/query-plans-and-tuning/010-query-plans-and-tuning-introduction.md",
+ "title": "Query Plans and Tuning Introduction",
+ "url": "/docs/query-plans-and-tuning-introduction/"
+ },
+ {
+ "breadcrumbs": [
+ {
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -5028,7 +5109,7 @@
"children": [],
"next_title": "Guidelines for Optimizing Aggregation",
"next_url": "/docs/guidelines-for-optimizing-aggregation/",
- "parent": "Query Plans and Tuning Introduction",
+ "parent": "Query Plans and Tuning",
"previous_title": "Query Plans and Tuning Introduction",
"previous_url": "/docs/query-plans-and-tuning-introduction/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/020-join-planning-guidelines.md",
@@ -5038,8 +5119,8 @@
{
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -5049,7 +5130,7 @@
"children": [],
"next_title": "Modifying Query Planning Options",
"next_url": "/docs/modifying-query-planning-options/",
- "parent": "Query Plans and Tuning Introduction",
+ "parent": "Query Plans and Tuning",
"previous_title": "Join Planning Guidelines",
"previous_url": "/docs/join-planning-guidelines/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/030-guidelines-for-optimizing-aggregation.md",
@@ -5059,8 +5140,8 @@
{
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -5070,7 +5151,7 @@
"children": [],
"next_title": "Sort-Based and Hash-Based Memory-Constrained Operators",
"next_url": "/docs/sort-based-and-hash-based-memory-constrained-operators/",
- "parent": "Query Plans and Tuning Introduction",
+ "parent": "Query Plans and Tuning",
"previous_title": "Guidelines for Optimizing Aggregation",
"previous_url": "/docs/guidelines-for-optimizing-aggregation/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/040-modifying-query-planning-options.md",
@@ -5080,8 +5161,8 @@
{
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -5091,7 +5172,7 @@
"children": [],
"next_title": "Enabling Query Queuing",
"next_url": "/docs/enabling-query-queuing/",
- "parent": "Query Plans and Tuning Introduction",
+ "parent": "Query Plans and Tuning",
"previous_title": "Modifying Query Planning Options",
"previous_url": "/docs/modifying-query-planning-options/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/050-sort-based-and-hash-based-memory-constrained-operators.md",
@@ -5101,8 +5182,8 @@
{
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -5112,7 +5193,7 @@
"children": [],
"next_title": "Controlling Parallelization to Balance Performance with Multi-Tenancy",
"next_url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/",
- "parent": "Query Plans and Tuning Introduction",
+ "parent": "Query Plans and Tuning",
"previous_title": "Sort-Based and Hash-Based Memory-Constrained Operators",
"previous_url": "/docs/sort-based-and-hash-based-memory-constrained-operators/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/060-enabling-query-queuing.md",
@@ -5122,8 +5203,8 @@
{
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -5131,9 +5212,9 @@
}
],
"children": [],
- "next_title": "Query Plans",
- "next_url": "/docs/query-plans/",
- "parent": "Query Plans and Tuning Introduction",
+ "next_title": "Identifying Performance Issues",
+ "next_url": "/docs/identifying-performance-issues/",
+ "parent": "Query Plans and Tuning",
"previous_title": "Enabling Query Queuing",
"previous_url": "/docs/enabling-query-queuing/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/070-controlling-parallelization-to-balance-performance-with-multi-tenancy.md",
@@ -5141,14 +5222,14 @@
"url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/"
}
],
- "next_title": "Join Planning Guidelines",
- "next_url": "/docs/join-planning-guidelines/",
+ "next_title": "Query Plans and Tuning Introduction",
+ "next_url": "/docs/query-plans-and-tuning-introduction/",
"parent": "Performance Tuning",
"previous_title": "Choosing a Storage Format",
"previous_url": "/docs/choosing-a-storage-format/",
- "relative_path": "_docs/performance-tuning/query-plans-and-tuning/010-query-plans-and-tuning-introduction.md",
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "relative_path": "_docs/performance-tuning/040-query-plans-and-tuning.md",
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"breadcrumbs": [
@@ -5157,32 +5238,58 @@
"url": "/docs/performance-tuning/"
}
],
- "children": [],
- "next_title": "Query Profiles",
- "next_url": "/docs/query-profiles/",
- "parent": "Performance Tuning",
- "previous_title": "Controlling Parallelization to Balance Performance with Multi-Tenancy",
- "previous_url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/",
- "relative_path": "_docs/performance-tuning/where-to-identify-performance-issues/010-query-plans.md",
- "title": "Query Plans",
- "url": "/docs/query-plans/"
- },
- {
- "breadcrumbs": [
+ "children": [
{
- "title": "Performance Tuning",
- "url": "/docs/performance-tuning/"
+ "breadcrumbs": [
+ {
+ "title": "Identifying Performance Issues",
+ "url": "/docs/identifying-performance-issues/"
+ },
+ {
+ "title": "Performance Tuning",
+ "url": "/docs/performance-tuning/"
+ }
+ ],
+ "children": [],
+ "next_title": "Query Profiles",
+ "next_url": "/docs/query-profiles/",
+ "parent": "Identifying Performance Issues",
+ "previous_title": "Identifying Performance Issues",
+ "previous_url": "/docs/identifying-performance-issues/",
+ "relative_path": "_docs/performance-tuning/identifying-performance-issues/010-query-plans.md",
+ "title": "Query Plans",
+ "url": "/docs/query-plans/"
+ },
+ {
+ "breadcrumbs": [
+ {
+ "title": "Identifying Performance Issues",
+ "url": "/docs/identifying-performance-issues/"
+ },
+ {
+ "title": "Performance Tuning",
+ "url": "/docs/performance-tuning/"
+ }
+ ],
+ "children": [],
+ "next_title": "Log and Debug",
+ "next_url": "/docs/log-and-debug/",
+ "parent": "Identifying Performance Issues",
+ "previous_title": "Query Plans",
+ "previous_url": "/docs/query-plans/",
+ "relative_path": "_docs/performance-tuning/identifying-performance-issues/020-query-profiles.md",
+ "title": "Query Profiles",
+ "url": "/docs/query-profiles/"
}
],
- "children": [],
- "next_title": "Log and Debug",
- "next_url": "/docs/log-and-debug/",
+ "next_title": "Query Plans",
+ "next_url": "/docs/query-plans/",
"parent": "Performance Tuning",
- "previous_title": "Query Plans",
- "previous_url": "/docs/query-plans/",
- "relative_path": "_docs/performance-tuning/where-to-identify-performance-issues/020-query-profiles.md",
- "title": "Query Profiles",
- "url": "/docs/query-profiles/"
+ "previous_title": "Controlling Parallelization to Balance Performance with Multi-Tenancy",
+ "previous_url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/",
+ "relative_path": "_docs/performance-tuning/050-identifying-performance-issues.md",
+ "title": "Identifying Performance Issues",
+ "url": "/docs/identifying-performance-issues/"
}
],
"next_title": "Performance Tuning Introduction",
@@ -5782,6 +5889,10 @@
"Query Plans": {
"breadcrumbs": [
{
+ "title": "Identifying Performance Issues",
+ "url": "/docs/identifying-performance-issues/"
+ },
+ {
"title": "Performance Tuning",
"url": "/docs/performance-tuning/"
}
@@ -5789,14 +5900,14 @@
"children": [],
"next_title": "Query Profiles",
"next_url": "/docs/query-profiles/",
- "parent": "Performance Tuning",
- "previous_title": "Controlling Parallelization to Balance Performance with Multi-Tenancy",
- "previous_url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/",
- "relative_path": "_docs/performance-tuning/where-to-identify-performance-issues/010-query-plans.md",
+ "parent": "Identifying Performance Issues",
+ "previous_title": "Identifying Performance Issues",
+ "previous_url": "/docs/identifying-performance-issues/",
+ "relative_path": "_docs/performance-tuning/identifying-performance-issues/010-query-plans.md",
"title": "Query Plans",
"url": "/docs/query-plans/"
},
- "Query Plans and Tuning Introduction": {
+ "Query Plans and Tuning": {
"breadcrumbs": [
{
"title": "Performance Tuning",
@@ -5807,8 +5918,29 @@
{
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
+ },
+ {
+ "title": "Performance Tuning",
+ "url": "/docs/performance-tuning/"
+ }
+ ],
+ "children": [],
+ "next_title": "Join Planning Guidelines",
+ "next_url": "/docs/join-planning-guidelines/",
+ "parent": "Query Plans and Tuning",
+ "previous_title": "Query Plans and Tuning",
+ "previous_url": "/docs/query-plans-and-tuning/",
+ "relative_path": "_docs/performance-tuning/query-plans-and-tuning/010-query-plans-and-tuning-introduction.md",
+ "title": "Query Plans and Tuning Introduction",
+ "url": "/docs/query-plans-and-tuning-introduction/"
+ },
+ {
+ "breadcrumbs": [
+ {
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -5818,7 +5950,7 @@
"children": [],
"next_title": "Guidelines for Optimizing Aggregation",
"next_url": "/docs/guidelines-for-optimizing-aggregation/",
- "parent": "Query Plans and Tuning Introduction",
+ "parent": "Query Plans and Tuning",
"previous_title": "Query Plans and Tuning Introduction",
"previous_url": "/docs/query-plans-and-tuning-introduction/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/020-join-planning-guidelines.md",
@@ -5828,8 +5960,8 @@
{
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -5839,7 +5971,7 @@
"children": [],
"next_title": "Modifying Query Planning Options",
"next_url": "/docs/modifying-query-planning-options/",
- "parent": "Query Plans and Tuning Introduction",
+ "parent": "Query Plans and Tuning",
"previous_title": "Join Planning Guidelines",
"previous_url": "/docs/join-planning-guidelines/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/030-guidelines-for-optimizing-aggregation.md",
@@ -5849,8 +5981,8 @@
{
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -5860,7 +5992,7 @@
"children": [],
"next_title": "Sort-Based and Hash-Based Memory-Constrained Operators",
"next_url": "/docs/sort-based-and-hash-based-memory-constrained-operators/",
- "parent": "Query Plans and Tuning Introduction",
+ "parent": "Query Plans and Tuning",
"previous_title": "Guidelines for Optimizing Aggregation",
"previous_url": "/docs/guidelines-for-optimizing-aggregation/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/040-modifying-query-planning-options.md",
@@ -5870,8 +6002,8 @@
{
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -5881,7 +6013,7 @@
"children": [],
"next_title": "Enabling Query Queuing",
"next_url": "/docs/enabling-query-queuing/",
- "parent": "Query Plans and Tuning Introduction",
+ "parent": "Query Plans and Tuning",
"previous_title": "Modifying Query Planning Options",
"previous_url": "/docs/modifying-query-planning-options/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/050-sort-based-and-hash-based-memory-constrained-operators.md",
@@ -5891,8 +6023,8 @@
{
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -5902,7 +6034,7 @@
"children": [],
"next_title": "Controlling Parallelization to Balance Performance with Multi-Tenancy",
"next_url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/",
- "parent": "Query Plans and Tuning Introduction",
+ "parent": "Query Plans and Tuning",
"previous_title": "Sort-Based and Hash-Based Memory-Constrained Operators",
"previous_url": "/docs/sort-based-and-hash-based-memory-constrained-operators/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/060-enabling-query-queuing.md",
@@ -5912,8 +6044,8 @@
{
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -5921,9 +6053,9 @@
}
],
"children": [],
- "next_title": "Query Plans",
- "next_url": "/docs/query-plans/",
- "parent": "Query Plans and Tuning Introduction",
+ "next_title": "Identifying Performance Issues",
+ "next_url": "/docs/identifying-performance-issues/",
+ "parent": "Query Plans and Tuning",
"previous_title": "Enabling Query Queuing",
"previous_url": "/docs/enabling-query-queuing/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/070-controlling-parallelization-to-balance-performance-with-multi-tenancy.md",
@@ -5931,11 +6063,32 @@
"url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/"
}
],
- "next_title": "Join Planning Guidelines",
- "next_url": "/docs/join-planning-guidelines/",
+ "next_title": "Query Plans and Tuning Introduction",
+ "next_url": "/docs/query-plans-and-tuning-introduction/",
"parent": "Performance Tuning",
"previous_title": "Choosing a Storage Format",
"previous_url": "/docs/choosing-a-storage-format/",
+ "relative_path": "_docs/performance-tuning/040-query-plans-and-tuning.md",
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
+ },
+ "Query Plans and Tuning Introduction": {
+ "breadcrumbs": [
+ {
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
+ },
+ {
+ "title": "Performance Tuning",
+ "url": "/docs/performance-tuning/"
+ }
+ ],
+ "children": [],
+ "next_title": "Join Planning Guidelines",
+ "next_url": "/docs/join-planning-guidelines/",
+ "parent": "Query Plans and Tuning",
+ "previous_title": "Query Plans and Tuning",
+ "previous_url": "/docs/query-plans-and-tuning/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/010-query-plans-and-tuning-introduction.md",
"title": "Query Plans and Tuning Introduction",
"url": "/docs/query-plans-and-tuning-introduction/"
@@ -5943,6 +6096,10 @@
"Query Profiles": {
"breadcrumbs": [
{
+ "title": "Identifying Performance Issues",
+ "url": "/docs/identifying-performance-issues/"
+ },
+ {
"title": "Performance Tuning",
"url": "/docs/performance-tuning/"
}
@@ -5950,10 +6107,10 @@
"children": [],
"next_title": "Log and Debug",
"next_url": "/docs/log-and-debug/",
- "parent": "Performance Tuning",
+ "parent": "Identifying Performance Issues",
"previous_title": "Query Plans",
"previous_url": "/docs/query-plans/",
- "relative_path": "_docs/performance-tuning/where-to-identify-performance-issues/020-query-profiles.md",
+ "relative_path": "_docs/performance-tuning/identifying-performance-issues/020-query-profiles.md",
"title": "Query Profiles",
"url": "/docs/query-profiles/"
},
@@ -8830,8 +8987,8 @@
"Sort-Based and Hash-Based Memory-Constrained Operators": {
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -8841,7 +8998,7 @@
"children": [],
"next_title": "Enabling Query Queuing",
"next_url": "/docs/enabling-query-queuing/",
- "parent": "Query Plans and Tuning Introduction",
+ "parent": "Query Plans and Tuning",
"previous_title": "Modifying Query Planning Options",
"previous_url": "/docs/modifying-query-planning-options/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/050-sort-based-and-hash-based-memory-constrained-operators.md",
@@ -12109,8 +12266,8 @@
}
],
"children": [],
- "next_title": "Query Plans and Tuning Introduction",
- "next_url": "/docs/query-plans-and-tuning-introduction/",
+ "next_title": "Query Plans and Tuning",
+ "next_url": "/docs/query-plans-and-tuning/",
"parent": "Performance Tuning",
"previous_title": "Partition Pruning",
"previous_url": "/docs/partition-pruning/",
@@ -12129,8 +12286,29 @@
{
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
+ },
+ {
+ "title": "Performance Tuning",
+ "url": "/docs/performance-tuning/"
+ }
+ ],
+ "children": [],
+ "next_title": "Join Planning Guidelines",
+ "next_url": "/docs/join-planning-guidelines/",
+ "parent": "Query Plans and Tuning",
+ "previous_title": "Query Plans and Tuning",
+ "previous_url": "/docs/query-plans-and-tuning/",
+ "relative_path": "_docs/performance-tuning/query-plans-and-tuning/010-query-plans-and-tuning-introduction.md",
+ "title": "Query Plans and Tuning Introduction",
+ "url": "/docs/query-plans-and-tuning-introduction/"
+ },
+ {
+ "breadcrumbs": [
+ {
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -12140,7 +12318,7 @@
"children": [],
"next_title": "Guidelines for Optimizing Aggregation",
"next_url": "/docs/guidelines-for-optimizing-aggregation/",
- "parent": "Query Plans and Tuning Introduction",
+ "parent": "Query Plans and Tuning",
"previous_title": "Query Plans and Tuning Introduction",
"previous_url": "/docs/query-plans-and-tuning-introduction/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/020-join-planning-guidelines.md",
@@ -12150,8 +12328,8 @@
{
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -12161,7 +12339,7 @@
"children": [],
"next_title": "Modifying Query Planning Options",
"next_url": "/docs/modifying-query-planning-options/",
- "parent": "Query Plans and Tuning Introduction",
+ "parent": "Query Plans and Tuning",
"previous_title": "Join Planning Guidelines",
"previous_url": "/docs/join-planning-guidelines/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/030-guidelines-for-optimizing-aggregation.md",
@@ -12171,8 +12349,8 @@
{
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -12182,7 +12360,7 @@
"children": [],
"next_title": "Sort-Based and Hash-Based Memory-Constrained Operators",
"next_url": "/docs/sort-based-and-hash-based-memory-constrained-operators/",
- "parent": "Query Plans and Tuning Introduction",
+ "parent": "Query Plans and Tuning",
"previous_title": "Guidelines for Optimizing Aggregation",
"previous_url": "/docs/guidelines-for-optimizing-aggregation/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/040-modifying-query-planning-options.md",
@@ -12192,8 +12370,8 @@
{
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -12203,7 +12381,7 @@
"children": [],
"next_title": "Enabling Query Queuing",
"next_url": "/docs/enabling-query-queuing/",
- "parent": "Query Plans and Tuning Introduction",
+ "parent": "Query Plans and Tuning",
"previous_title": "Modifying Query Planning Options",
"previous_url": "/docs/modifying-query-planning-options/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/050-sort-based-and-hash-based-memory-constrained-operators.md",
@@ -12213,8 +12391,8 @@
{
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -12224,7 +12402,7 @@
"children": [],
"next_title": "Controlling Parallelization to Balance Performance with Multi-Tenancy",
"next_url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/",
- "parent": "Query Plans and Tuning Introduction",
+ "parent": "Query Plans and Tuning",
"previous_title": "Sort-Based and Hash-Based Memory-Constrained Operators",
"previous_url": "/docs/sort-based-and-hash-based-memory-constrained-operators/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/060-enabling-query-queuing.md",
@@ -12234,8 +12412,8 @@
{
"breadcrumbs": [
{
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"title": "Performance Tuning",
@@ -12243,9 +12421,9 @@
}
],
"children": [],
- "next_title": "Query Plans",
- "next_url": "/docs/query-plans/",
- "parent": "Query Plans and Tuning Introduction",
+ "next_title": "Identifying Performance Issues",
+ "next_url": "/docs/identifying-performance-issues/",
+ "parent": "Query Plans and Tuning",
"previous_title": "Enabling Query Queuing",
"previous_url": "/docs/enabling-query-queuing/",
"relative_path": "_docs/performance-tuning/query-plans-and-tuning/070-controlling-parallelization-to-balance-performance-with-multi-tenancy.md",
@@ -12253,14 +12431,14 @@
"url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/"
}
],
- "next_title": "Join Planning Guidelines",
- "next_url": "/docs/join-planning-guidelines/",
+ "next_title": "Query Plans and Tuning Introduction",
+ "next_url": "/docs/query-plans-and-tuning-introduction/",
"parent": "Performance Tuning",
"previous_title": "Choosing a Storage Format",
"previous_url": "/docs/choosing-a-storage-format/",
- "relative_path": "_docs/performance-tuning/query-plans-and-tuning/010-query-plans-and-tuning-introduction.md",
- "title": "Query Plans and Tuning Introduction",
- "url": "/docs/query-plans-and-tuning-introduction/"
+ "relative_path": "_docs/performance-tuning/040-query-plans-and-tuning.md",
+ "title": "Query Plans and Tuning",
+ "url": "/docs/query-plans-and-tuning/"
},
{
"breadcrumbs": [
@@ -12269,32 +12447,58 @@
"url": "/docs/performance-tuning/"
}
],
- "children": [],
- "next_title": "Query Profiles",
- "next_url": "/docs/query-profiles/",
- "parent": "Performance Tuning",
- "previous_title": "Controlling Parallelization to Balance Performance with Multi-Tenancy",
- "previous_url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/",
- "relative_path": "_docs/performance-tuning/where-to-identify-performance-issues/010-query-plans.md",
- "title": "Query Plans",
- "url": "/docs/query-plans/"
- },
- {
- "breadcrumbs": [
+ "children": [
{
- "title": "Performance Tuning",
- "url": "/docs/performance-tuning/"
+ "breadcrumbs": [
+ {
+ "title": "Identifying Performance Issues",
+ "url": "/docs/identifying-performance-issues/"
+ },
+ {
+ "title": "Performance Tuning",
+ "url": "/docs/performance-tuning/"
+ }
+ ],
+ "children": [],
+ "next_title": "Query Profiles",
+ "next_url": "/docs/query-profiles/",
+ "parent": "Identifying Performance Issues",
+ "previous_title": "Identifying Performance Issues",
+ "previous_url": "/docs/identifying-performance-issues/",
+ "relative_path": "_docs/performance-tuning/identifying-performance-issues/010-query-plans.md",
+ "title": "Query Plans",
+ "url": "/docs/query-plans/"
+ },
+ {
+ "breadcrumbs": [
+ {
+ "title": "Identifying Performance Issues",
+ "url": "/docs/identifying-performance-issues/"
+ },
+ {
+ "title": "Performance Tuning",
+ "url": "/docs/performance-tuning/"
+ }
+ ],
+ "children": [],
+ "next_title": "Log and Debug",
+ "next_url": "/docs/log-and-debug/",
+ "parent": "Identifying Performance Issues",
+ "previous_title": "Query Plans",
+ "previous_url": "/docs/query-plans/",
+ "relative_path": "_docs/performance-tuning/identifying-performance-issues/020-query-profiles.md",
+ "title": "Query Profiles",
+ "url": "/docs/query-profiles/"
}
],
- "children": [],
- "next_title": "Log and Debug",
- "next_url": "/docs/log-and-debug/",
+ "next_title": "Query Plans",
+ "next_url": "/docs/query-plans/",
"parent": "Performance Tuning",
- "previous_title": "Query Plans",
- "previous_url": "/docs/query-plans/",
- "relative_path": "_docs/performance-tuning/where-to-identify-performance-issues/020-query-profiles.md",
- "title": "Query Profiles",
- "url": "/docs/query-profiles/"
+ "previous_title": "Controlling Parallelization to Balance Performance with Multi-Tenancy",
+ "previous_url": "/docs/controlling-parallelization-to-balance-performance-with-multi-tenancy/",
+ "relative_path": "_docs/performance-tuning/050-identifying-performance-issues.md",
+ "title": "Identifying Performance Issues",
+ "url": "/docs/identifying-performance-issues/"
}
],
"next_title": "Performance Tuning Introduction",
http://git-wip-us.apache.org/repos/asf/drill/blob/fb1c6d4e/_docs/performance-tuning/040-query-plans-and-tuning.md
----------------------------------------------------------------------
diff --git a/_docs/performance-tuning/040-query-plans-and-tuning.md b/_docs/performance-tuning/040-query-plans-and-tuning.md
new file mode 100755
index 0000000..04322f7
--- /dev/null
+++ b/_docs/performance-tuning/040-query-plans-and-tuning.md
@@ -0,0 +1,4 @@
+---
+title: "Query Plans and Tuning"
+parent: "Performance Tuning"
+---
http://git-wip-us.apache.org/repos/asf/drill/blob/fb1c6d4e/_docs/performance-tuning/050-identifying-performance-issues.md
----------------------------------------------------------------------
diff --git a/_docs/performance-tuning/050-identifying-performance-issues.md b/_docs/performance-tuning/050-identifying-performance-issues.md
new file mode 100755
index 0000000..fe28869
--- /dev/null
+++ b/_docs/performance-tuning/050-identifying-performance-issues.md
@@ -0,0 +1,4 @@
+---
+title: "Identifying Performance Issues"
+parent: "Performance Tuning"
+---
http://git-wip-us.apache.org/repos/asf/drill/blob/fb1c6d4e/_docs/performance-tuning/identifying-performance-issues/010-query-plans.md
----------------------------------------------------------------------
diff --git a/_docs/performance-tuning/identifying-performance-issues/010-query-plans.md b/_docs/performance-tuning/identifying-performance-issues/010-query-plans.md
new file mode 100755
index 0000000..7332322
--- /dev/null
+++ b/_docs/performance-tuning/identifying-performance-issues/010-query-plans.md
@@ -0,0 +1,74 @@
+---
+title: "Query Plans"
+parent: "Identifying Performance Issues"
+---
+If you experience performance issues in Drill, you can typically identify the source of the issues in the query plans or profiles. This section describes the logical plan and physical plans.
+
+## Query Plans
+
+Drill has an optimizer and a parallelizer that work together to plan a query. Drill creates logical, physical, and execution plans based on the available statistics for an associated set of files or data sources. The number of running Drill nodes and configured runtime settings contribute to how Drill plans and executes a query.
+
+You can use [EXPLAIN commands]({{ site.baseurl }}/docs/explain-commands/) to view the logical and physical plans for a query, however you cannot view the execution plan. To see how Drill executed a query, you can view the query profile in the Drill Web UI at <drill_node_ip_address>:8047.
+
+### Logical Plan
+
+A logical plan is a collection of logical operators that describe the work required to generate query results and define which data sources and operators to apply. The parser in Drill converts SQL operators into a logical operator syntax that Drill understands to create the logical plan. You can view the logical plan to see the planned operators. Modifying and resubmitting the logical plan to Drill (through submit_plan) is not very useful because Drill has not determined parallelization at this stage of planning.
+
+### Physical Plan
+
+A physical plan describes the chosen physical execution plan for a query statement. The optimizer applies various types of rules to rearrange operators and functions into an optimal plan and then converts the logical plan into a physical plan that tells Drill how to execute the query.
+
+You can review a physical plan to troubleshoot issues, modify the plan, and then submit the plan back to Drill. For example, if you run into a casting error or you want to change the join ordering of tables to see if the query runs faster. You can modify the physical plan to address the issue and then submit it back to Drill and run the query.
+
+Drill transforms the physical plan into an execution tree of minor fragments that run simultaneously on the cluster to carry out execution tasks. See Query Execution. You can view the activity of the fragments that executed a query in the query profile. See Query Profiles.
+
+**Viewing the Physical Plan**
+
+You can run the EXPLAIN command to view the physical plan for a query with or without costing formation. See EXPLAIN for Physical Plans and Costing Information. Analyze the cost-based query plan to identify the types of operators that Drill plans to use for the query and how much memory they will require.
+
+Read the text output from bottom to top to understand the sequence of operators planned to execute the query. You can also view a visual representation of the physical plan in the Profile view of the Drill Web UI. See Query Profiles. You can modify the detailed JSON output, and submit it back to Drill through the Drill Web UI.
+
+The physical plan shows the major fragments and specific operators with correlating MajorFragmentIDs and OperatorIDs. See Operators. Major fragments are an abstract concept that represent a phase of the query execution. Major fragments do not perform any query tasks.
+
+The physical plan displays the IDs in the following format:
+
+<MajorFragmentID\> - <OperatorID\>
+
+For example, 00-02 where 00 is the MajorFragmentID and 02 is is the OperatorID.
+
+If you view the plan with costing information, you can see where the majority of resources, in terms of I/O, CPU, and memory, will be spent when Drill executes the query. If joining tables, your query plan should include broadcast joins.
+
+**Example EXPLAIN PLAN**
+
+
+ 0: jdbc:drill:zk=local> explain plan for select type t, count(distinct id) from dfs.`/home/donuts/donuts.json` where type='donut' group by type;
+ +------------+------------+
+ | text | json |
+ +------------+------------+
+ | 00-00 Screen
+ 00-01 Project(t=[$0], EXPR$1=[$1])
+ 00-02 Project(t=[$0], EXPR$1=[$1])
+ 00-03 HashAgg(group=[{0}], EXPR$1=[COUNT($1)])
+ 00-04 HashAgg(group=[{0, 1}])
+ 00-05 SelectionVectorRemover
+ 00-06 Filter(condition=[=($0, 'donut')])
+ 00-07 Scan(groupscan=[EasyGroupScan [selectionRoot=/home/donuts/donuts.json, numFiles=1, columns=[`type`, `id`], files=[file:/home/donuts/donuts.json]]])...
+ …
+
+
+**Modifying and Submitting a Physical Plan to Drill**
+
+You can test the performance of a physical plan that Drill generates, modify the plan and then re-submit it to Drill. For example, you can modify the plan to change the join ordering of tables. You can also submit physical plans created outside of Drill through the Drill Web UI.
+
+**Note:** Only advanced users who know about query planning should modify and re-submit a physical plan.
+
+To modify and re-submit a physical plan to Drill, complete the following steps:
+
+1. Run EXPLAIN PLAN FOR <query\> to see the physical plan for your query.
+2. Copy the JSON output of the physical plan, and modify as needed.
+3. Navigate to the Drill Web UI at <drill\_node\_ip_address\>:8047.
+4. Select **Query** in the menu bar.
+
+
+5. Select the **Physical Plan** radio button under Query Type.
+6. Paste the physical plan into the Query field, and click **Submit**. Drill runs the plan and executes the query.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/drill/blob/fb1c6d4e/_docs/performance-tuning/identifying-performance-issues/020-query-profiles.md
----------------------------------------------------------------------
diff --git a/_docs/performance-tuning/identifying-performance-issues/020-query-profiles.md b/_docs/performance-tuning/identifying-performance-issues/020-query-profiles.md
new file mode 100755
index 0000000..a4908f7
--- /dev/null
+++ b/_docs/performance-tuning/identifying-performance-issues/020-query-profiles.md
@@ -0,0 +1,142 @@
+---
+title: "Query Profiles"
+parent: "Identifying Performance Issues"
+---
+
+A profile is a summary of metrics collected for each query that Drill executes. Query profiles provide information that you can use to monitor and analyze query performance. Drill creates a query profile from major, minor, operator, and input stream profiles. Each major fragment profile consists of a list of minor fragment profiles. Each minor fragment profile consists of a list of operator profiles. An operator profile consists of a list of input stream profiles.
+
+You can view aggregate statistics across profile lists in the Profile tab of the Drill Web UI at <drill\_node\_ip_address\>:8047. You can modify and resubmit queries, or cancel queries. For debugging purposes, you can use profiles in conjunction with Drill logs. See Log and Debug.
+
+Metrics in a query profile are associated with a coordinate system of IDs. Drill uses a coordinate system comprised of query, fragment, and operator identifiers to track query execution activities and resources. Drill assigns a unique QueryID to each query received and then assigns IDs to each fragment and operator that executes the query.
+
+**Example IDs**
+
+QueryID: 2aa98add-15b3-e155-5669-603c03bfde86
+
+Fragment and operator IDs:
+
+
+
+## Viewing a Query Profile
+
+When you select the Profiles tab in the Drill Web UI at <drill\_node_ip\_address\>:8047, you see a list of the last 100 queries than have run or that are currently running in the cluster.
+
+
+
+
+You can click on any query to see its profile.
+
+
+
+When you select a profile, notice that the URL in the address bar contains the QueryID. For example, 2aa98add-15b3-e155-5669-603c03bfde86 in the following URL:
+
+ http://<drill_node>:8047/profiles/2aa98add-15b3-e155-5669-603c03bfde86
+
+The Query Profile section in the Query profile summarizes a few key details about the query, including:
+
+ * The state of the query, either running, completed, or failed.
+ * The node operating as the Foreman; the Drillbit that receives a query from the client or application becomes the Foreman and drives the entire query.
+ * The total number of minor fragments required to execute the query
+
+If you scroll down, you can see the Fragment Profiles and Operator Profiles sections.
+
+## Fragment Profiles
+
+Fragment profiles section provides an overview table, and a major fragment block for each major fragment that executed the query. Each row in the Overview table provides the number of minor fragments that Drill parallelized from each major fragment, as well as aggregate time and memory metrics for the minor fragments.
+
+
+
+See Major Fragment Profiles Table for column descriptions.
+
+When you look at the fragment profiles, you may notice that some major fragments were parallelized into substantially fewer minor fragments, but happen to have the highest runtime. Or, you may notice certain minor fragments have a higher peak memory than others. When you notice these variations in execution, you can delve deeper into the profile by looking at the major fragment blocks.
+
+Below the Overview table are major fragment blocks. Each of these blocks corresponds to a row in the Overview table. You can expand the blocks to see metrics for all of the minor fragments that were parallelized from each major fragment, including the host on which each minor fragment ran. Each row in the major fragment table presents the fragment state, time metrics, memory metrics, and aggregate input metrics of each minor fragment.
+
+
+
+When looking at the minor fragment metrics, verify the state of the fragment. A fragment can have a “failed” state which could indicate an issue on the host. If the query itself fails, an operator may have run out of memory. If fragments running on a particular node are under performing, there may be multi-tenancy issues that you can address.
+
+You can also see a graph that illustrates the activity of major and minor fragments for the duration of the query.
+
+
+
+If you see “stair steps” in the graph, this indicates that the execution work of the fragments is not distributed evenly. Stair steps in the graph typically occur for non-local reads on data. To address this issue, you can increase data replication, rewrite the data, or file a JIRA to get help with the issue.
+
+This graph correlates with the visualized plan graph in the Visualized Plan tab. Each color in the graph corresponds to the activity of one major fragment.
+
+
+
+The visualized plan illustrates color-coded major fragments divided and labeled with the names of the operators used to complete each phase of the query. Exchange operators separate each major fragment. These operators represent a point where Drill can execute operations below them in parallel.
+
+## Operator Profiles
+
+Operator profiles describe each operator that performed relational operations during query execution. The Operator Profiles section provides an Overview table of the aggregate time and memory metrics for each operator within a major fragment.
+
+
+
+See Operator Profiles Table for column descriptions.
+
+Identify the operations that consume a majority of time and memory. You can potentially modify options related to the specific operators to improve performance.
+
+Below the Overview table are operator blocks, which you can expand to see metrics for each operator. Each of these blocks corresponds to a row in the Overview table. Each row in the Operator block presents time and memory metrics, as well as aggregate input metrics for each minor fragment.
+
+
+
+See Operator Block for column descriptions.
+
+Drill uses batches of records as a basic unit of work. The batches are pipelined between each operation. Record batches are no larger than 64k records. While the target size of one record batch is generally 256k, they can scale to many megabytes depending on the query plan and the width of the records.
+
+The Max Records number for each minor fragment should be almost equivalent. If one, or a very small number of minor fragments, perform the majority of the work, there may be data skew. To address data skew, you may need change settings related to table joins or partition data to balance the work.
+
+### Data Skew Example
+The following query was run against TPC-DS data:
+
+ 0: jdbc:drill:zk=local> select ss_customer_sk, count(*) as cnt from store_sales where ss_customer_sk is null or ss_customer_sk in (1, 2, 3, 4, 5) group by ss_customer_sk;
+ +-----------------+---------+
+ | ss_customer_sk | cnt |
+ +-----------------+---------+
+ | null | 129752 |
+ | 5 | 47 |
+ | 1 | 9 |
+ | 2 | 43 |
+ | 4 | 10 |
+ | 3 | 11 |
+ +-----------------+---------+
+ 6 rows selected
+
+In the result set, notice that the 'null' group has 129752 values while others have roughly similar values.
+
+Looking at the operator profile for the hash aggregate in major fragment 00, you can see that out of 8 minor fragments, only minor fragment 1 is processing a substantially larger number of records when compared to the other minor fragments.
+
+
+
+In this example, there is inherent skew present in the data. Other types of skew may not strictly be data dependent, but can be introduced by a sub-optimal hash function or other issues in the product. In either case, examining the query profile helps understand why a query is slow. In the first scenario, it may be possible to run separate queries for the skewed and non-skewed values. In the second scenario, it is better to seek technical support.
+
+## Physical Plan View
+
+The physical plan view provides statistics about the actual cost of the query operations in terms of memory, I/O, and CPU processing. You can use this profile to identify which operations consumed the majority of the resources during a query, modify the physical plan to address the cost-intensive operations, and submit the updated plan back to Drill. See [Costing Information]({{ site.baseurl }}/docs/explain/#costing-information).
+
+
+
+## Canceling a Query
+
+You may want to cancel a query if it hangs or causes performance bottlenecks. You can cancel a query in the Profile tab of the Drill Web UI.
+
+To cancel a query from the Drill Web UI, complete the following steps:
+
+1. Navigate to the Drill Web UI at <drill\_node_ip\_address\>:8047.
+The Drill node from which you access the Drill Web UI must have an active Drillbit running.
+2. Select Profiles in the toolbar.
+A list of running and completed queries appears.
+3. Click the query for which you want to see the profile.
+4. Select **Edit Query**.
+5. Click **Cancel** query to cancel the query.
+
+The following message appears:
+
+ Cancelled query <QueryID\>
+
+
+
+
+
http://git-wip-us.apache.org/repos/asf/drill/blob/fb1c6d4e/_docs/performance-tuning/query-plans-and-tuning/010-query-plans-and-tuning-introduction.md
----------------------------------------------------------------------
diff --git a/_docs/performance-tuning/query-plans-and-tuning/010-query-plans-and-tuning-introduction.md b/_docs/performance-tuning/query-plans-and-tuning/010-query-plans-and-tuning-introduction.md
index 0461b06..1f42966 100755
--- a/_docs/performance-tuning/query-plans-and-tuning/010-query-plans-and-tuning-introduction.md
+++ b/_docs/performance-tuning/query-plans-and-tuning/010-query-plans-and-tuning-introduction.md
@@ -1,6 +1,6 @@
---
title: "Query Plans and Tuning Introduction"
-parent: "Performance Tuning"
+parent: "Query Plans and Tuning"
---
You can modify several options that affect how Drill plans a query. This section describes some options that you can modify to improve performance.
http://git-wip-us.apache.org/repos/asf/drill/blob/fb1c6d4e/_docs/performance-tuning/query-plans-and-tuning/020-join-planning-guidelines.md
----------------------------------------------------------------------
diff --git a/_docs/performance-tuning/query-plans-and-tuning/020-join-planning-guidelines.md b/_docs/performance-tuning/query-plans-and-tuning/020-join-planning-guidelines.md
index 9ebc77a..4c2b7ef 100755
--- a/_docs/performance-tuning/query-plans-and-tuning/020-join-planning-guidelines.md
+++ b/_docs/performance-tuning/query-plans-and-tuning/020-join-planning-guidelines.md
@@ -1,6 +1,6 @@
---
title: "Join Planning Guidelines"
-parent: "Query Plans and Tuning Introduction"
+parent: "Query Plans and Tuning"
---
Drill uses distributed and broadcast joins to join tables. You can modify configuration settings in Drill to control how Drill plans joins in a query.
http://git-wip-us.apache.org/repos/asf/drill/blob/fb1c6d4e/_docs/performance-tuning/query-plans-and-tuning/030-guidelines-for-optimizing-aggregation.md
----------------------------------------------------------------------
diff --git a/_docs/performance-tuning/query-plans-and-tuning/030-guidelines-for-optimizing-aggregation.md b/_docs/performance-tuning/query-plans-and-tuning/030-guidelines-for-optimizing-aggregation.md
index cbbb3e1..5f79017 100755
--- a/_docs/performance-tuning/query-plans-and-tuning/030-guidelines-for-optimizing-aggregation.md
+++ b/_docs/performance-tuning/query-plans-and-tuning/030-guidelines-for-optimizing-aggregation.md
@@ -1,6 +1,6 @@
---
title: "Guidelines for Optimizing Aggregation"
-parent: "Query Plans and Tuning Introduction"
+parent: "Query Plans and Tuning"
---
http://git-wip-us.apache.org/repos/asf/drill/blob/fb1c6d4e/_docs/performance-tuning/query-plans-and-tuning/040-modifying-query-planning-options.md
----------------------------------------------------------------------
diff --git a/_docs/performance-tuning/query-plans-and-tuning/040-modifying-query-planning-options.md b/_docs/performance-tuning/query-plans-and-tuning/040-modifying-query-planning-options.md
index e142b02..fed5c25 100755
--- a/_docs/performance-tuning/query-plans-and-tuning/040-modifying-query-planning-options.md
+++ b/_docs/performance-tuning/query-plans-and-tuning/040-modifying-query-planning-options.md
@@ -1,6 +1,6 @@
---
title: "Modifying Query Planning Options"
-parent: "Query Plans and Tuning Introduction"
+parent: "Query Plans and Tuning"
---
Planner options affect how Drill plans a query. You can use the ALTER SYSTEM|SESSION commands to modify certain planning options to optimize query plans and improve performance. Typically, you modify options at the session level. See [ALTER SESSION]({{ site.baseurl }}/docs/alter-session/) for details on how to run the command.
http://git-wip-us.apache.org/repos/asf/drill/blob/fb1c6d4e/_docs/performance-tuning/query-plans-and-tuning/050-sort-based-and-hash-based-memory-constrained-operators.md
----------------------------------------------------------------------
diff --git a/_docs/performance-tuning/query-plans-and-tuning/050-sort-based-and-hash-based-memory-constrained-operators.md b/_docs/performance-tuning/query-plans-and-tuning/050-sort-based-and-hash-based-memory-constrained-operators.md
index 45927bb..3325a2c 100755
--- a/_docs/performance-tuning/query-plans-and-tuning/050-sort-based-and-hash-based-memory-constrained-operators.md
+++ b/_docs/performance-tuning/query-plans-and-tuning/050-sort-based-and-hash-based-memory-constrained-operators.md
@@ -1,6 +1,6 @@
---
title: "Sort-Based and Hash-Based Memory-Constrained Operators"
-parent: "Query Plans and Tuning Introduction"
+parent: "Query Plans and Tuning"
---
Drill uses hash-based and sort-based operators depending on the query characteristics. Hash aggregation and hash join are hash-based operations. Streaming aggregation and merge join are sort-based operations. Both hash-based and sort-based operations consume memory, however the hash aggregate and hash join operators are the fastest and most memory intensive operators.
http://git-wip-us.apache.org/repos/asf/drill/blob/fb1c6d4e/_docs/performance-tuning/query-plans-and-tuning/060-enabling-query-queuing.md
----------------------------------------------------------------------
diff --git a/_docs/performance-tuning/query-plans-and-tuning/060-enabling-query-queuing.md b/_docs/performance-tuning/query-plans-and-tuning/060-enabling-query-queuing.md
index bafa7eb..ad06070 100755
--- a/_docs/performance-tuning/query-plans-and-tuning/060-enabling-query-queuing.md
+++ b/_docs/performance-tuning/query-plans-and-tuning/060-enabling-query-queuing.md
@@ -1,6 +1,6 @@
---
title: "Enabling Query Queuing"
-parent: "Query Plans and Tuning Introduction"
+parent: "Query Plans and Tuning"
---
Drill runs all queries concurrently by default. However, Drill performance increases when a small number of queries run concurrently. You can enable query queues to limit the maximum number of queries that run concurrently. Splitting large queries into multiple small queries and enabling query queuing improves query performance.
http://git-wip-us.apache.org/repos/asf/drill/blob/fb1c6d4e/_docs/performance-tuning/query-plans-and-tuning/070-controlling-parallelization-to-balance-performance-with-multi-tenancy.md
----------------------------------------------------------------------
diff --git a/_docs/performance-tuning/query-plans-and-tuning/070-controlling-parallelization-to-balance-performance-with-multi-tenancy.md b/_docs/performance-tuning/query-plans-and-tuning/070-controlling-parallelization-to-balance-performance-with-multi-tenancy.md
index fd162bc..2178d14 100755
--- a/_docs/performance-tuning/query-plans-and-tuning/070-controlling-parallelization-to-balance-performance-with-multi-tenancy.md
+++ b/_docs/performance-tuning/query-plans-and-tuning/070-controlling-parallelization-to-balance-performance-with-multi-tenancy.md
@@ -1,6 +1,6 @@
---
title: "Controlling Parallelization to Balance Performance with Multi-Tenancy"
-parent: "Query Plans and Tuning Introduction"
+parent: "Query Plans and Tuning"
---
When you run Drill in a multi-tenant environment, (in conjunction with other workloads in a cluster, such as MapReduce) you may need to modify Drill settings and options to maximize performance, or reduce the allocated resources to other applications. See [Configuring Multi-Tenant Resources]({{ site.baseurl }}/docs/configuring-multitenant-resources/).
http://git-wip-us.apache.org/repos/asf/drill/blob/fb1c6d4e/_docs/performance-tuning/where-to-identify-performance-issues/010-query-plans.md
----------------------------------------------------------------------
diff --git a/_docs/performance-tuning/where-to-identify-performance-issues/010-query-plans.md b/_docs/performance-tuning/where-to-identify-performance-issues/010-query-plans.md
deleted file mode 100755
index 9ebd0c2..0000000
--- a/_docs/performance-tuning/where-to-identify-performance-issues/010-query-plans.md
+++ /dev/null
@@ -1,74 +0,0 @@
----
-title: "Query Plans"
-parent: "Performance Tuning"
----
-If you experience performance issues in Drill, you can typically identify the source of the issues in the query plans or profiles. This section describes the logical plan and physical plans.
-
-## Query Plans
-
-Drill has an optimizer and a parallelizer that work together to plan a query. Drill creates logical, physical, and execution plans based on the available statistics for an associated set of files or data sources. The number of running Drill nodes and configured runtime settings contribute to how Drill plans and executes a query.
-
-You can use [EXPLAIN commands]({{ site.baseurl }}/docs/explain-commands/) to view the logical and physical plans for a query, however you cannot view the execution plan. To see how Drill executed a query, you can view the query profile in the Drill Web UI at <drill_node_ip_address>:8047.
-
-### Logical Plan
-
-A logical plan is a collection of logical operators that describe the work required to generate query results and define which data sources and operators to apply. The parser in Drill converts SQL operators into a logical operator syntax that Drill understands to create the logical plan. You can view the logical plan to see the planned operators. Modifying and resubmitting the logical plan to Drill (through submit_plan) is not very useful because Drill has not determined parallelization at this stage of planning.
-
-### Physical Plan
-
-A physical plan describes the chosen physical execution plan for a query statement. The optimizer applies various types of rules to rearrange operators and functions into an optimal plan and then converts the logical plan into a physical plan that tells Drill how to execute the query.
-
-You can review a physical plan to troubleshoot issues, modify the plan, and then submit the plan back to Drill. For example, if you run into a casting error or you want to change the join ordering of tables to see if the query runs faster. You can modify the physical plan to address the issue and then submit it back to Drill and run the query.
-
-Drill transforms the physical plan into an execution tree of minor fragments that run simultaneously on the cluster to carry out execution tasks. See Query Execution. You can view the activity of the fragments that executed a query in the query profile. See Query Profiles.
-
-**Viewing the Physical Plan**
-
-You can run the EXPLAIN command to view the physical plan for a query with or without costing formation. See EXPLAIN for Physical Plans and Costing Information. Analyze the cost-based query plan to identify the types of operators that Drill plans to use for the query and how much memory they will require.
-
-Read the text output from bottom to top to understand the sequence of operators planned to execute the query. You can also view a visual representation of the physical plan in the Profile view of the Drill Web UI. See Query Profiles. You can modify the detailed JSON output, and submit it back to Drill through the Drill Web UI.
-
-The physical plan shows the major fragments and specific operators with correlating MajorFragmentIDs and OperatorIDs. See Operators. Major fragments are an abstract concept that represent a phase of the query execution. Major fragments do not perform any query tasks.
-
-The physical plan displays the IDs in the following format:
-
-<MajorFragmentID\> - <OperatorID\>
-
-For example, 00-02 where 00 is the MajorFragmentID and 02 is is the OperatorID.
-
-If you view the plan with costing information, you can see where the majority of resources, in terms of I/O, CPU, and memory, will be spent when Drill executes the query. If joining tables, your query plan should include broadcast joins.
-
-**Example EXPLAIN PLAN**
-
-
- 0: jdbc:drill:zk=local> explain plan for select type t, count(distinct id) from dfs.`/home/donuts/donuts.json` where type='donut' group by type;
- +------------+------------+
- | text | json |
- +------------+------------+
- | 00-00 Screen
- 00-01 Project(t=[$0], EXPR$1=[$1])
- 00-02 Project(t=[$0], EXPR$1=[$1])
- 00-03 HashAgg(group=[{0}], EXPR$1=[COUNT($1)])
- 00-04 HashAgg(group=[{0, 1}])
- 00-05 SelectionVectorRemover
- 00-06 Filter(condition=[=($0, 'donut')])
- 00-07 Scan(groupscan=[EasyGroupScan [selectionRoot=/home/donuts/donuts.json, numFiles=1, columns=[`type`, `id`], files=[file:/home/donuts/donuts.json]]])...
- …
-
-
-**Modifying and Submitting a Physical Plan to Drill**
-
-You can test the performance of a physical plan that Drill generates, modify the plan and then re-submit it to Drill. For example, you can modify the plan to change the join ordering of tables. You can also submit physical plans created outside of Drill through the Drill Web UI.
-
-**Note:** Only advanced users who know about query planning should modify and re-submit a physical plan.
-
-To modify and re-submit a physical plan to Drill, complete the following steps:
-
-1. Run EXPLAIN PLAN FOR <query\> to see the physical plan for your query.
-2. Copy the JSON output of the physical plan, and modify as needed.
-3. Navigate to the Drill Web UI at <drill\_node\_ip_address\>:8047.
-4. Select **Query** in the menu bar.
-
-
-5. Select the **Physical Plan** radio button under Query Type.
-6. Paste the physical plan into the Query field, and click **Submit**. Drill runs the plan and executes the query.
\ No newline at end of file