You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@drill.apache.org by br...@apache.org on 2019/04/18 19:17:39 UTC
[drill] branch gh-pages updated: add web ui features to doc for 1.16
This is an automated email from the ASF dual-hosted git repository.
bridgetb pushed a commit to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/drill.git
The following commit(s) were added to refs/heads/gh-pages by this push:
new 1e45c19 add web ui features to doc for 1.16
1e45c19 is described below
commit 1e45c195cf4d7b4025c7e83c1674480c4900249c
Author: Bridget Bevens <bb...@maprtech.com>
AuthorDate: Thu Apr 18 12:16:39 2019 -0700
add web ui features to doc for 1.16
---
.../030-planning-and-exececution-options.md | 64 ++++++++++++++--
.../036-configuring-storage-plugins.md | 10 ++-
.../rest-api/010-rest-api-introduction.md | 7 +-
...on-is-enabled-and-authentication-is-disabled.md | 4 +-
_docs/img/ui-export-all.png | Bin 0 -> 39876 bytes
_docs/img/ui-limit-results-warn.png | Bin 0 -> 13718 bytes
_docs/img/ui-options.PNG | Bin 0 -> 41028 bytes
_docs/img/ui-order-duration.png | Bin 0 -> 56440 bytes
_docs/img/ui-results-page.png | Bin 0 -> 101904 bytes
_docs/img/ui-row-limit.png | Bin 0 -> 34559 bytes
_docs/img/ui-spilled-data.png | Bin 0 -> 318 bytes
_docs/img/ui-submit-popup.png | Bin 0 -> 25778 bytes
_docs/img/ui-turtle-wait.png | Bin 0 -> 677 bytes
_docs/img/ui-wait-op.png | Bin 0 -> 369 bytes
_docs/install/060-starting-the-web-ui.md | 27 +++++--
.../020-query-profiles.md | 82 ++++++++++++++++-----
_docs/query-data/010-query-data-introduction.md | 52 ++++++++++---
17 files changed, 195 insertions(+), 51 deletions(-)
diff --git a/_docs/configure-drill/configuration-options/030-planning-and-exececution-options.md b/_docs/configure-drill/configuration-options/030-planning-and-exececution-options.md
index f486435..b10816b 100644
--- a/_docs/configure-drill/configuration-options/030-planning-and-exececution-options.md
+++ b/_docs/configure-drill/configuration-options/030-planning-and-exececution-options.md
@@ -1,6 +1,6 @@
---
title: "Planning and Execution Options"
-date: 2018-12-28
+date: 2019-04-18
parent: "Configuration Options"
---
You can set Drill query planning and execution options per cluster, at the
@@ -10,7 +10,6 @@ system level affect the entire system and persist between restarts. Session
level settings override system level settings for the duration of the session.
Planning options are prepended by planner, for example `planner.enable_hashjoin`. Execution options are prepended by drill.exe, for example `drill.exec.functions.cast_empty_string_to_null`.
-
## Setting Planning and Execution Options
You can set planning and execution options for Drill from the Drill shell (SQLLine) or the Drill Web UI. Options set in the Drill Web UI are set at the system level. You can override system level options set in the Drill Web UI by setting the options at the session level from the Drill shell.
@@ -39,14 +38,14 @@ See [Querying the Options Table]({{site.baseurl}}/docs/querying-system-tables/#q
### Setting Options from the Drill Web UI
-When Drill is running, you can access the Drill Web UI at `http://<drill-hostname>:8047/`. The Drill Web UI has an Options button located in the upper right corner that you can click to display an Options page. The Options page lists all the Drill options that you can set. When you set options from this page, you are setting the options at the system level. To override an option for the duration of a session, set the option from the Drill shell using the SET command.
+When Drill is running, you can access the Drill Web UI at `http://<drill-hostname>:8047/`. The Drill Web UI has an Options button located in the upper right corner that you can click to display an Options page. The Options page lists all the Drill options that you can set. When you set options from this page, you are setting the options at the system level. To override an option for the duration of a session, set the option from the Drill shell using the SET command.
-
+
Starting in Drill 1.15, the Options page includes the following enhancements:
**Search Field**
-The search field enables you to quickly search across all available Drill configuration options to quickly find options you are interested in. For example, you can search on the keyword "memory" for a list of all options related to configuring Drill memory.
+The search field enables you to quickly search across all available Drill configuration options to quickly find options you are interested in. For example, you can search on the keyword "memory" for a list of all options related to configuring Drill memory. Starting in Drill 1.16, the Search field contains an x that clears the search field when clicked.
**Quick Filters**
Next to the search field are quick filter buttons that filter options by important keywords in one click. For example, the "planner" quick filter button quickly lists all options related to the query planner. The query planning options are prepended by the keyword "planner." Quick filters return options that include the keyword in the option name and/or description.
@@ -90,7 +89,60 @@ Issuing ` select timestamp '2008-2-23 12:23:34' from (values(1))` returns the fo
Issuing the query after setting the `web.display_format.timestamp` option to the format `yyyy-MM-dd HH:mm:ss.SS` returns the following results in the Web UI:
- 2008-02-23 12:23:34.00
+ 2008-02-23 12:23:34.00
+
+##Limiting the Number of Rows Returned for Result Sets
+The `exec.query.max_rows` option, introduced in Drill 1.16, automatically limits the number of rows that Drill returns from a result set. When you configure a limit, the limit is automatically applied by Drill when you run a query. You do not need to include the LIMIT clause. Configuring a limit on results reduces the amount of processing and streaming work for the web server when using the Drill WEb UI.
+
+The default setting for `exec.query.max_rows` is 0, which indicates no limit; queries will return all results. An administrator can modify the exec.query.max_rows setting at the system level (setting persists across restarts) from the Options page in the Drill Web UI or using the ALTER SYSTEM command. Alternatively, the option can be set at the session level (valid for a user session only) using the SET command or from SQLLine.
+
+**Session Level**
+To set the `exec.query.max_rows` option per user session, use the SET command or the SQLLine line command, as shown:
+
+ SET `exec.query.max_rows` = <value>;
+ !set rowlimit <value>
+
+**System Level**
+To set the `exec.query.max_rows` option such that it persists across users, change the setting in the Drill Web UI (`http://<drill-hostname>:8047/options`) or use the ALTER SYSTEM SET command, as shown:
+
+ ALTER SYSTEM SET `exec.query.max_rows` = <value>;
+
+***
+**NOTE**
+The `exec.query.max_rows` option overrides row limit settings configured in the Drill Web UI and `drill-override.conf` file, which are described below.
+***
+
+### Drill Web UI Row Limit Settings
+In addition to the `exec.query.max_rows` option, the Query page in the Drill Web UI provides a field where you can enter a limit on the number of rows returned from a result set when you run a query.
+
+**Note:** You can also configure the row limit for a REST client through the [autoLimit parameter in the REST API]({{site.baseurl}}/docs/rest-api-introduction/#post-query-json).
+
+
+When you submit the query, results display across one or multiple pages. You can set the number of results that display per page. When results span multiple pages, scroll through the pages using the links in the bottom right corner of the screen.
+
+Clicking the Query Profile tab at the top of the results page displays the query profile with a warning stating that the results were automatically limited.
+
+Note that the `exec.query.max_rows` setting overrides the setting in the Web UI. For example, if the `exec.query.max_rows` option is set to 10 and you limit results to 20 on the Query page in the Web UI, Drill only returns 10 rows from the result set.
+
+###Drill-override.conf Row Limit Settings
+The `drill-override.conf` file contains the same row limit setting as the Drill Web UI; however, the setting in the `drill-override.conf` file overrides the limit set in the Drill Web UI. Note that the `exec.query.max_rows` option overrides the setting in `drill-override.conf` and the Drill Web UI. You must restart the Drillbit after you modify `drill-override.conf`.
+
+Add the following configuration to `drill-override.conf`, changing the row limit setting and rows per page setting, as needed:
+
+ drill.exec.http: {
+ web.client.resultset: {
+ autolimit {
+ enable: true,
+ rows: 1000
+ },
+ defaultRowsPerPage: 10
+ }
+ }
+
+
+
+
+
diff --git a/_docs/connect-a-data-source/036-configuring-storage-plugins.md b/_docs/connect-a-data-source/036-configuring-storage-plugins.md
index 6e470df..658b137 100644
--- a/_docs/connect-a-data-source/036-configuring-storage-plugins.md
+++ b/_docs/connect-a-data-source/036-configuring-storage-plugins.md
@@ -1,6 +1,6 @@
---
title: "Configuring Storage Plugins"
-date: 2018-12-08
+date: 2019-04-18
parent: "Storage Plugin Configuration"
---
@@ -88,11 +88,15 @@ The file is removed after Drill uses the file for the first time.
## Exporting Storage Plugin Configurations
-Starting in Drill 1.14, you can export and save your storage plugin configurations from the Storage page in the Drill Web UI. The Export option next to each enabled storage plugin provides an easy way to export your storage plugin configurations to a JSON file when you want to save and reuse the configurations. For example, before an upgrade.
+Starting in Drill 1.14, you can export and save your storage plugin configurations from the Storage page in the Drill Web UI. The Export option next to each enabled storage plugin provides an easy way to export each storage plugin configuration to a JSON file when you want to save and reuse the configurations. For example, before an upgrade.
-You can also access the Export option on the Update page, by selecting the Update option next to an enabled or disabled storage plugin.
+You can also access the Export option on the Update page, by selecting the Update option next to an enabled or disabled storage plugin.
+
+Starting in Drill 1.16, you can export all storage plugin configurations at once, by clicking the Export All button. You have the option to export all storage plugin configurations to a HOCON format (.conf file) or a JSON file.
+
+
When you need to configure the storage plugin, copy and paste the contents of the JSON file into the Configuration field. You can access the Configuration field for a storage plugin by selecting Update next to the storage plugin.
diff --git a/_docs/developer-information/rest-api/010-rest-api-introduction.md b/_docs/developer-information/rest-api/010-rest-api-introduction.md
index 7ef22bc..be66bbc 100644
--- a/_docs/developer-information/rest-api/010-rest-api-introduction.md
+++ b/_docs/developer-information/rest-api/010-rest-api-introduction.md
@@ -1,6 +1,6 @@
---
title: "REST API Introduction"
-date: 2018-12-08
+date: 2019-04-18
parent: "REST API"
---
@@ -62,7 +62,8 @@ Submit a query and return results.
**Parameters**
* queryType--SQL, PHYSICAL, or LOGICAL are valid types. Use only "SQL". Other types are for internal use only.
-* query--A SQL query that runs in Drill.
+* query--A SQL query that runs in Drill.
+* autoLimit--Limits the number of rows returned from the result set. (Drill 1.16+)
**Request Body**
@@ -73,7 +74,7 @@ Submit a query and return results.
**Example**
- curl -X POST -H "Content-Type: application/json" -d '{"queryType":"SQL", "query": "select * from dfs.`/Users/joe-user/apache-drill-1.4.0/sample-data/donuts.json` where name= \u0027Cake\u0027"}' http://localhost:8047/query.json
+ curl -X POST -H "Content-Type: application/json" -d '{"queryType":"SQL", "query": "select * from dfs.`/Users/joe-user/apache-drill-1.4.0/sample-data/donuts.json` where name= \u0027Cake\u0027", "autoLimit":1}' http://localhost:8047/query.json
**Response Body**
diff --git a/_docs/developer-information/rest-api/011-submitting-queries-from-the-rest-api-when-impersonation-is-enabled-and-authentication-is-disabled.md b/_docs/developer-information/rest-api/011-submitting-queries-from-the-rest-api-when-impersonation-is-enabled-and-authentication-is-disabled.md
index d3984bf..e18941e 100644
--- a/_docs/developer-information/rest-api/011-submitting-queries-from-the-rest-api-when-impersonation-is-enabled-and-authentication-is-disabled.md
+++ b/_docs/developer-information/rest-api/011-submitting-queries-from-the-rest-api-when-impersonation-is-enabled-and-authentication-is-disabled.md
@@ -1,6 +1,6 @@
---
title: "Submitting Queries from the REST API when Impersonation is Enabled and Authentication is Disabled"
-date: 2018-12-08
+date: 2019-04-18
parent: "REST API"
---
@@ -14,7 +14,7 @@ This feature only works when impersonation is enabled and authentication is disa
A user can issue queries through the Drill Web UI, SQLLine, using curl commands, or Java code, as follows:
##Drill Web UI
-You can submit a query through the Query page in the Drill Web UI after entering a valid username in the Username field. The Drill Web UI is accessible through the URL http(s)://<ip-address>:8047.
+You can submit a query through the Query page in the Drill Web UI after entering a valid username in the Username field. The Drill Web UI is accessible through the URL `http(s)://<drill-hostname>:8047`.
To re-run a query from the Profiles page in the Drill Web UI, you must submit a username prior to re-running the query.
diff --git a/_docs/img/ui-export-all.png b/_docs/img/ui-export-all.png
new file mode 100644
index 0000000..cfb932c
Binary files /dev/null and b/_docs/img/ui-export-all.png differ
diff --git a/_docs/img/ui-limit-results-warn.png b/_docs/img/ui-limit-results-warn.png
new file mode 100644
index 0000000..1a10b7a
Binary files /dev/null and b/_docs/img/ui-limit-results-warn.png differ
diff --git a/_docs/img/ui-options.PNG b/_docs/img/ui-options.PNG
new file mode 100644
index 0000000..aea171e
Binary files /dev/null and b/_docs/img/ui-options.PNG differ
diff --git a/_docs/img/ui-order-duration.png b/_docs/img/ui-order-duration.png
new file mode 100644
index 0000000..acfc146
Binary files /dev/null and b/_docs/img/ui-order-duration.png differ
diff --git a/_docs/img/ui-results-page.png b/_docs/img/ui-results-page.png
new file mode 100644
index 0000000..ff7cc18
Binary files /dev/null and b/_docs/img/ui-results-page.png differ
diff --git a/_docs/img/ui-row-limit.png b/_docs/img/ui-row-limit.png
new file mode 100644
index 0000000..063c93b
Binary files /dev/null and b/_docs/img/ui-row-limit.png differ
diff --git a/_docs/img/ui-spilled-data.png b/_docs/img/ui-spilled-data.png
new file mode 100644
index 0000000..4e5a75d
Binary files /dev/null and b/_docs/img/ui-spilled-data.png differ
diff --git a/_docs/img/ui-submit-popup.png b/_docs/img/ui-submit-popup.png
new file mode 100644
index 0000000..50bb865
Binary files /dev/null and b/_docs/img/ui-submit-popup.png differ
diff --git a/_docs/img/ui-turtle-wait.png b/_docs/img/ui-turtle-wait.png
new file mode 100644
index 0000000..f83f60d
Binary files /dev/null and b/_docs/img/ui-turtle-wait.png differ
diff --git a/_docs/img/ui-wait-op.png b/_docs/img/ui-wait-op.png
new file mode 100644
index 0000000..49b813c
Binary files /dev/null and b/_docs/img/ui-wait-op.png differ
diff --git a/_docs/install/060-starting-the-web-ui.md b/_docs/install/060-starting-the-web-ui.md
index 54d6992..8876721 100644
--- a/_docs/install/060-starting-the-web-ui.md
+++ b/_docs/install/060-starting-the-web-ui.md
@@ -1,6 +1,6 @@
---
title: "Starting the Web UI"
-date: 2018-12-28
+date: 2019-04-18
parent: Install Drill
---
@@ -11,19 +11,30 @@ The Drill Web UI is one of several [client interfaces](/docs/architecture-introd
- [View query profiles]({{site.baseurl}}/docs/query-profiles/)
- [Update and configure storage plugins]({{site.baseurl}}/docs/storage-plugin-registration/)
- View [logs]({{site.baseurl}}/docs/log-and-debug-introduction/) and [metrics]({{site.baseurl}}/docs/monitoring-metrics/)
-- [Set configuration options]({{site.baseurl}}/docs/planning-and-execution-options/#setting-options-from-the-drill-web-ui)
-
-Starting in Drill 1.15, you can use a Meta+Enter key combination instead of clicking the Submit button to submit queries through the query editor (Query page) in Drill Web UI. On Mac keyboards, the combination is Ctrl+Meta or Ctrl+Enter. On Windows and Linux, the combination is Ctrl+Enter; however, on Linux, if you mapped the meta key to another physical key on the keyboard, use that key + Enter.
+- [Set configuration options]({{site.baseurl}}/docs/planning-and-execution-options/#setting-options-from-the-drill-web-ui)
## Accessing the Web UI
-To access the Drill Web UI, enter the URL appropriate for your Drill configuration. The following list describes the URLs for various Drill configurations:
+To access the Drill Web UI, enter `http(s)://<drill-hostname-or-ip-address>:8047` in your browser's address bar. Note that Drill must be running on the node in order to access the Drill Web UI.
+
+The following list describes the URLs for various Drill configurations:
-* `http://<IP address or host name>:8047`
+* `http://<drill-hostname-or-ip-address>:8047`
Use this URL when [HTTPS support]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security/#https-support) is disabled (the default).
-* `https://<IP address or host name>:8047`
+* `https://<drill-hostname-or-ip-address>:8047`
Use this URL when HTTPS support is enabled.
* `http://localhost:8047`
- Use this URL when running Drill in embedded mode (./drill-embedded).
+ Use this URL when running Drill in embedded mode (./drill-embedded).
+
+## Running Queries from the Web UI
+
+To run a query in the Web UI, click Query in the toolbar to access the Query page. Enter your query in the query editor and click Submit or press Ctrl+Enter to submit the query.
+
+Starting in Drill 1.15, you can use a Meta+Enter key combination to submit queries through the query editor. On Mac keyboards, the combination is Ctrl+Meta or Ctrl+Enter. On Windows and Linux, the combination is Ctrl+Enter; however, on Linux, if you mapped the meta key to another physical key on the keyboard, use that key + Enter.
+
+Starting in Drill 1.16, a Query Submitted pop-up window appears when you submit a query.
+
+The pop-up serves as an indicator that a query is in progress and prevents any further action until the query returns results. The pop-up includes a Check Status button that links to the Profiles page. If needed, you can [cancel the query from the Profiles page]({{site.baseurl}}/docs/monitoring-and-canceling-queries-in-the-drill-web-ui/).
+
## Web UI Security
diff --git a/_docs/performance-tuning/identifying-performance-issues/020-query-profiles.md b/_docs/performance-tuning/identifying-performance-issues/020-query-profiles.md
index 53da190..4d70091 100644
--- a/_docs/performance-tuning/identifying-performance-issues/020-query-profiles.md
+++ b/_docs/performance-tuning/identifying-performance-issues/020-query-profiles.md
@@ -1,41 +1,89 @@
---
title: "Query Profiles"
-date: 2018-12-08
+date: 2019-04-18
parent: "Identifying Performance Issues"
---
-A profile is a summary of metrics collected for each query that Drill executes. Query profiles provide information that you can use to monitor and analyze query performance. When Drill executes a query, Drill writes the profile of each query to disk, which is either the local filesystem or a distributed file system, such as HDFS. As of Drill 1.11, Drill can [store profiles in memory]({{site.baseurl}}/docs/start-up-options/#configuring-start-up-options) instead of writing them to disk. Yo [...]
+A profile is a summary of metrics collected for each query that Drill executes. Query profiles provide information that you can use to monitor and analyze query performance. When Drill executes a query, Drill writes the profile of each query to disk, which is either the local filesystem or a distributed file system, such as HDFS. As of Drill 1.11, Drill can [store profiles in memory]({{site.baseurl}}/docs/start-up-options/#configuring-start-up-options) instead of writing them to disk. Yo [...]
-##Query Profiles in the Drill Web UI
-The Drill Web UI provides aggregate statistics across profile lists. Profile lists consist of data from major and minor fragments, operators, and input streams. You can use profiles in conjunction with Drill logs for debugging purposes. In addition to viewing query profiles, you can modify, resubmit, or cancel queries from the Drill Web UI.
+The Drill Web UI provides aggregate statistics across profile lists. Profile lists consist of data from major and minor fragments, operators, and input streams. Use query profiles in conjunction with Drill logs to debug issues.
-###Query, Fragment, and Operator Identifiers
-
-Metrics in a query profile are associated with a coordinate system of identifiers. Drill uses a coordinate system comprised of query, fragment, and operator identifiers to track query execution activities and resources. Drill assigns a unique identifier, the QueryID, to each query received and then assigns an identifier to each fragment and operator that executes the query. An example of a QueryID is 2aa98add-15b3-e155-5669-603c03bfde86. The following images shows an example of fragment [...]
+## Viewing a Query Profile
-
+You can view query profiles in the Profiles tab of the Drill Web UI. When you select the Profiles tab, you see a list of queries than ran or are currently running in the cluster.
-### Viewing a Query Profile
+
-You can view query profiles in the Profiles tab of the Drill Web UI. When you select the Profiles tab, you see a list of the last 100 queries than ran or are currently running in the cluster.
+Starting in Drill 1.16, you can sort the columns on the Profiles page. For example, if you want to quickly locate queries with the longest running times, you can sort the Duration column in descending order such that the longest running queries appear at the top of the list. You can also use the Search Profiles field to filter the list of query profiles and then use the sort feature.
-
+For example, the following image shows the query profiles filtered by the mapr user on a specific date with the Duration column sorted in descending order:
-You must click on a query to see its profile.
+
-
+To see the profile for a specific query, click on the query in the profiles list.
-When you select a profile, notice that the URL in the address bar contains the QueryID, as shown in the following URL:
+
- http://<drill_node>:8047/profiles/2aa98add-15b3-e155-5669-603c03bfde86
-
The Query Profile section summarizes a few key details about the query, including:
* The state of the query, either running, completed, or failed.
* The node operating as the Foreman; the Drillbit that receives a query from the client or application becomes the Foreman and drives the entire query.
* The total number of minor fragments required to execute the query.
-Further down you can see the Fragment Profiles and Operator Profiles sections.
+When you select a profile, notice that the URL in the address bar contains the QueryID, as shown in the following URL:
+
+ http://<drill_node>:8047/profiles/2aa98add-15b3-e155-5669-603c03bfde86
+
+##Query, Fragment, and Operator Identifiers
+
+Metrics in a query profile are associated with a coordinate system of identifiers. Drill uses a coordinate system comprised of query, fragment, and operator identifiers to track query execution activities and resources. Drill assigns a unique identifier, the QueryID, to each query received and then assigns an identifier to each fragment and operator that executes the query. An example of a QueryID is 2aa98add-15b3-e155-5669-603c03bfde86. The following image shows an example of fragment a [...]
+
+
+
+
+## Query Profile Warnings
+When Drill executes a query, the workload should be uniformly distributed across the fragments and operators processing the data. When you evaluate a query profile in the Drill Web UI, seeing a disproportionate distribution of work across fragments (in terms of time) or excess memory use typically indicates performance issues and requires some performance tuning.
+
+Starting in Drill 1.16, the Drill Web UI displays warning messages in the query profile for the following issues:
+
+- No query progress within a certain amount of time
+- Operators spilling data to disk (operators do not have enough memory to complete operations completely in-memory)
+- Operators spending significantly more time waiting for data than processing it
+
+In addition to the warnings, the Operator Profiles section of the query profile also displays an icon in the column to indicate which operator is askew.
+
+The following table lists the warnings, icons, configurable options related to the warnings, and option descriptions:
+
+| Warning Message | Icon | Related Option(s) | Description [...]
+|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------|---------------------------------------------------------------------------- [...]
+| !WARNING: No fragments have made any progress in the last 300 seconds. (See Last Progress below.) | None | drill.exec.http.profile.warning.progress.threshold | If none of the fragments make any progress within the set time, a warning [...]
+| !WARNING: Some operators have data spilled to disk. This will result in performance loss. (See Avg Peak memory and Max Peak Memory below) |  Look for the icon in the Avg Peak Memory and Max Peak Memory columns to find the operators that spilled data to disk. | See [Sort-Based and Hash-Based Memory Constrained Operators](https://drill.apache.org/docs/sort-based-and-hash-based-memory-constrai [...]
+| !WARNING: Some of the operators spent more time waiting for data than processing it. (See AVG WAIT TIME as compared to Average Process Time for the operators below.) |  Look for the icon in the Max Process Time or Max Wait Time columns to locate the operators that waited or processed too long. | drill.exec.http.profile.warning.time.skew.min | Sets the minimum threshold for operators wi [...]
+| See warning for drill.exec.http.profile.warning.time.skew.min | | drill.exec.http.profile.warning.time.skew.ratio.process | Defines the threshold ratio for processing. When the maxProcessing: [...]
+| See warning for drill.exec.http.profile.warning.time.skew.min | | drill.exec.http.profile.warning.time.skew.ratio.wait | Defines the threshold ratio for waiting. When the maxWait:avgWait r [...]
+| !WARNING: Some of the SCAN operators spent more time waiting for data than processing it. (See AVG WAIT TIME as compared to Average Process Time for the scan operators below.) |  Look for the icon in the Avg Wait Time or Avg Process Time columns to locate the scan operators that waited too long. | drill.exec.http.profile.warning.scan.wait.min | Sets a minimum threshold ratio for the scan [...]
+
+### Configuring Warning Threshold Options
+You can configure thresholds for the warnings through configuration options in the drill-override.conf file, as shown:
+
+ http: {
+ profile.warning: {
+ progress.threshold: 300,
+ time.skew: {
+ min: 2,
+ ratio: {
+ process: 2
+ wait: 2
+ }
+ },
+ scan.wait.min: 60
+ },
+ ...
+ }
+
+You must restart Drillbits after you modify drill-override.conf.
+
+The following sections describe the Fragment Profiles and Operator Profiles sections of the Drill Web UI in more detail.
## Fragment Profiles
diff --git a/_docs/query-data/010-query-data-introduction.md b/_docs/query-data/010-query-data-introduction.md
index dbe81d5..36afe2a 100644
--- a/_docs/query-data/010-query-data-introduction.md
+++ b/_docs/query-data/010-query-data-introduction.md
@@ -1,16 +1,47 @@
---
title: "Query Data Introduction"
-date: 2018-12-08
+date: 2019-04-18
parent: "Query Data"
---
-You can query local and distributed file systems, Hive, HBase data, complex data, INFORMATION SCHEMA, and system tables as described in the subtopics of this section.
-
-The query specifies the data source location and includes data casting.
+You can submit SQL queries against various data sources from the [Drill shell (SQLLine)]({{site.baseurl}}/docs/configuring-the-drill-shell/), [Drill Web UI]({{site.baseurl}}/docs/starting-the-web-ui/), [REST API]({{site.baseurl}}/docs/rest-api/), and tools that connect to Drill via [ODBC or JDBC]({{site.baseurl}}/docs/odbc-jdbc-interfaces/). Drill has [several storage and format plugins]({{site.baseurl}}/docs/connect-a-data-source-introduction/) that enable queries against multiple data [...]
+
+The following sections provide some general information about Drill queries.
## Specifying the Data Source Location
-The optional [USE statement]({{site.baseurl}}/docs/use) runs subsequent queries against a particular [storage plugin]({{site.baseurl}}/docs/connect-a-data-source-introduction/). The USE statement typically saves typing some of the storage plugin information in the FROM statement. If you omit the USE statement, specify a storage plugin, such as dfs, and optionally a workspace, such as default, and a path to the data source using dot notation and back ticks. For example:
+The optional [USE command]({{site.baseurl}}/docs/use) runs subsequent queries against a particular [storage plugin or schema]({{site.baseurl}}/docs/connect-a-data-source-introduction/). When you run the USE command to switch to a particular storage plugin or schema, you do not have to include the full path to the data in the FROM clause, for example:
+
+The following query was run before switching to the dfs.schema. A workspace named "samples" was configured in the dfs storage plugin, creating a schema named `dfs.samples`. Notice that you have to use dot notation for the schema and back ticks around the table name. In some cases you may point to a directory or file in the schema, in which case you would put back ticks around the entire path, for example ```dfs.samples.`/nation/data/nation.parquet/````.
+
+
+ apache drill> select * from dfs.samples.`nation1`;
+ +-------------+----------------+-------------+----------------------+
+ | N_NATIONKEY | N_NAME | N_REGIONKEY | N_COMMENT |
+ +-------------+----------------+-------------+----------------------+
+ | 0 | ALGERIA | 0 | haggle. carefully f |
+ | 1 | ARGENTINA | 1 | al foxes promise sly |
+ ...
+ +-------------+----------------+-------------+----------------------+
+
+Running USE to switch to the `dfs.samples` schema:
+
+ apache drill> use dfs.samples;
+ +------+-----------------------------------------+
+ | ok | summary |
+ +------+-----------------------------------------+
+ | true | Default schema changed to [dfs.samples] |
+ +------+-----------------------------------------+
+
+Query written without identifying the schema (without dot notation or back ticks):
+
+ apache drill (dfs.samples)> select * from nation1;
+ +-------------+----------------+-------------+----------------------+
+ | N_NATIONKEY | N_NAME | N_REGIONKEY | N_COMMENT |
+ +-------------+----------------+-------------+----------------------+
+ | 0 | ALGERIA | 0 | haggle. carefully f |
+ | 1 | ARGENTINA | 1 | al foxes promise sly |
+ ...
+ +-------------+----------------+-------------+----------------------+
-``dfs.`default`.`/Users/drill-user/apache-drill-1.1.0/log/sqlline_queries.json```;
## Casting Data
In some cases, Drill converts schema-less data to correctly-typed data implicitly. In this case, you do not need to [cast the data]({{site.baseurl}}/docs/supported-data-types/#casting-and-converting-data-types) to another type. The file format of the data and the nature of your query determines the requirement for casting or converting. Differences in casting depend on the data source.
@@ -22,15 +53,12 @@ To query HBase data using Drill, convert every column of an HBase table to/from
## Troubleshooting Queries
-In addition to testing queries interactively in the Drill shell, and examining error messages, use the [EXPLAIN command]({{site.baseurl}}/docs/explain/) to analyze errors and troubleshoot queries
-that do not run. For example, if you run into a casting error, the query plan
-text may help you isolate the problem.
+In addition to analyzing error messages printed by the Drill shell, you can troubleshoot queries from the [Profiles page]({{ site.baseurl }}/docs/identifying-performance-issues/) in the Drill Web UI or run the [EXPLAIN command]({{site.baseurl}}/docs/explain/) to review the query plan for issues. For example, if you run into a casting error, the query plan text may help you isolate the problem.
0: jdbc:drill:zk=local> !set maxwidth 10000
- 0: jdbc:drill:zk=local> explain plan for select ... ;
+ 0: jdbc:drill:zk=local> explain plan for <query>;
-[Drill shell commands]({{site.baseurl}}/docs/configuring-the-drill-shell/) include the `!set <set variable> <value>` to increase the default text display (number of characters). By
-default, most of the plan output is hidden.
+[Drill shell commands]({{site.baseurl}}/docs/configuring-the-drill-shell/) include the `!set <set variable> <value>` to increase the default text display (number of characters). By default, most of the plan output is hidden.
## Query Syntax Tips