You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@drill.apache.org by br...@apache.org on 2018/12/08 02:51:39 UTC
[drill] branch gh-pages updated: doc updates for 1.15
This is an automated email from the ASF dual-hosted git repository.
bridgetb pushed a commit to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/drill.git
The following commit(s) were added to refs/heads/gh-pages by this push:
new 1d15439 doc updates for 1.15
1d15439 is described below
commit 1d15439d713418d61fe635f0678fffc8f867685d
Author: Bridget Bevens <bb...@maprtech.com>
AuthorDate: Fri Dec 7 18:49:23 2018 -0800
doc updates for 1.15
---
.../architecture/010-architecture-introduction.md | 4 +-
_docs/architecture/015-drill-query-execution.md | 6 +-
.../010-configure-drill-introduction.md | 4 +-
_docs/configure-drill/100-ports-used-by-drill.md | 6 +-
.../010-configuration-options-introduction.md | 4 +-
.../040-persistent-configuration-storage.md | 6 +-
.../020-secure-communication-paths.md | 10 +-
.../securing-drill/030-roles-privileges.md | 22 +-
...g-user-impersonation-with-hive-authorization.md | 4 +-
...-drill-to-use-spnego-for-http-authentication.md | 12 +-
...092-configuring-web-ui-and-rest-api-security.md | 30 +--
.../036-configuring-storage-plugins.md | 6 +-
.../080-drill-default-input-format.md | 4 +-
.../plugins/060-hbase-storage-plugin.md | 4 +-
.../plugins/070-hive-storage-plugin.md | 10 +-
.../plugins/080-rdbms-storage-plugin.md | 8 +-
.../plugins/090-mongodb-storage-plugin.md | 8 +-
.../plugins/112-opentsdb-storage-plugin.md | 6 +-
.../plugins/113-kafka-storage-plugin.md | 8 +-
.../rest-api/010-rest-api-introduction.md | 8 +-
...on-is-enabled-and-authentication-is-disabled.md | 10 +-
_docs/getting-started/010-drill-introduction.md | 8 +-
_docs/install/060-starting-the-web-ui.md | 26 +-
_docs/install/061-stopping-drill.md | 14 +-
...tifying-multiple-drill-versions-in-a-cluster.md | 12 +-
.../001-log-and-debug-introduction.md | 10 +-
_docs/log-and-debug/002-error-messages.md | 6 +-
.../010-interfaces-introduction.md | 4 +-
.../020-tableau-examples.md | 4 +-
.../024-aynchronous-parquet-reader.md | 6 +-
.../027-hive-metadata-caching.md | 4 +-
.../010-query-plans.md | 10 +-
.../020-query-profiles.md | 16 +-
.../query-plans-and-tuning/065-throttling.md | 8 +-
_docs/query-data/010-query-data-introduction.md | 4 +-
_docs/query-data/070-query-sys-tbl.md | 298 ++++++++++++++-------
...ng-and-canceling-queries-in-the-Drill-Web-UI.md | 16 +-
.../005-querying-complex-data-introduction.md | 4 +-
_docs/rn/001-1.14.0-rn.md | 2 +-
_docs/rn/009-1.12.0-rn.md | 2 +-
_docs/rn/011-1.10.0-rn.md | 2 +-
_docs/rn/019-1.2.0-rn.md | 2 +-
.../sql-commands/031-create-temporary-table-as.md | 4 +-
_docs/sql-reference/sql-commands/053-describe.md | 4 +-
.../sql-reference/sql-commands/081-from-clause.md | 4 +-
.../sql-commands/082-group-by-clause.md | 31 ++-
.../010-installing-the-apache-drill-sandbox.md | 10 +-
.../020-getting-to-know-the-drill-sandbox.md | 4 +-
.../030-lesson-1-learn-about-the-data-set.md | 4 +-
49 files changed, 415 insertions(+), 284 deletions(-)
diff --git a/_docs/architecture/010-architecture-introduction.md b/_docs/architecture/010-architecture-introduction.md
index cc73f0f..bbf60ff 100644
--- a/_docs/architecture/010-architecture-introduction.md
+++ b/_docs/architecture/010-architecture-introduction.md
@@ -1,6 +1,6 @@
---
title: "Architecture Introduction"
-date: 2018-11-02
+date: 2018-12-08
parent: "Architecture"
---
Apache Drill is a low latency distributed query engine for large-scale
@@ -36,7 +36,7 @@ See [Drill Query Execution]({{ site.baseurl }}/docs/drill-query-execution/).
You can access Drill through the following interfaces:
* [Drill shell]({{ site.baseurl }}/docs/configuring-the-drill-shell/)
- * [Drill Web Console]({{ site.baseurl }}/docs/monitoring-and-canceling-queries-in-the-drill-web-console)
+ * [Drill Web UI]({{ site.baseurl }}/docs/monitoring-and-canceling-queries-in-the-drill-web-console)
* [ODBC/JDBC]({{ site.baseurl }}/docs/interfaces-introduction/#using-odbc-to-access-apache-drill-from-bi-tools)
* C++ API
diff --git a/_docs/architecture/015-drill-query-execution.md b/_docs/architecture/015-drill-query-execution.md
index 4772561..42b8226 100644
--- a/_docs/architecture/015-drill-query-execution.md
+++ b/_docs/architecture/015-drill-query-execution.md
@@ -1,6 +1,6 @@
---
title: "Drill Query Execution"
-date: 2018-11-02
+date: 2018-12-08
parent: "Architecture"
---
@@ -32,7 +32,7 @@ Drill uses an exchange operator to separate major fragments. An exchange is a ch
Major fragments do not actually perform any query tasks. Each major fragment is divided into one or multiple minor fragments (discussed in the next section) that actually execute the operations required to complete the query and return results back to the client.
-You can work with major fragments within the physical plan by capturing a JSON representation of the plan in a file, manually modifying it, and then submitting it back to Drill using the SUBMIT PLAN command. You can also view major fragments in the query profile, which is visible in the Drill Web Console. See [EXPLAIN ]({{ site.baseurl }}/docs/explain/)and [Query Profiles]({{ site.baseurl }}/docs/query-profiles/) for more information.
+You can work with major fragments within the physical plan by capturing a JSON representation of the plan in a file, manually modifying it, and then submitting it back to Drill using the SUBMIT PLAN command. You can also view major fragments in the query profile, which is visible in the Drill Web UI. See [EXPLAIN ]({{ site.baseurl }}/docs/explain/)and [Query Profiles]({{ site.baseurl }}/docs/query-profiles/) for more information.
## Minor Fragments
Each major fragment is parallelized into minor fragments. A minor fragment is a logical unit of work that runs inside a thread. A logical unit of work in Drill is also referred to as a slice. The execution plan that Drill creates is composed of minor fragments. Drill assigns each minor fragment a MinorFragmentID.
@@ -49,7 +49,7 @@ Minor fragments contain one or more relational operators. An operator performs a
For example, when performing a hash aggregation of two files, Drill breaks the first phase dedicated to scanning into two minor fragments. Each minor fragment contains scan operators that scan the files. Drill breaks the second phase dedicated to aggregation into four minor fragments. Each of the four minor fragments contain hash aggregate operators that perform the hash aggregation operations on the data.
-You cannot modify the number of minor fragments within the execution plan. However, you can view the query profile in the Drill Web Console and modify some configuration options that change the behavior of minor fragments, such as the maximum number of slices. See [Configuration Options]({{ site.baseurl }}/docs/configuration-options-introduction/).
+You cannot modify the number of minor fragments within the execution plan. However, you can view the query profile in the Drill Web UI and modify some configuration options that change the behavior of minor fragments, such as the maximum number of slices. See [Configuration Options]({{ site.baseurl }}/docs/configuration-options-introduction/).
### Execution of Minor Fragments
Minor fragments can run as root, intermediate, or leaf fragments. An execution tree contains only one root fragment. The coordinates of the execution tree are numbered from the root, with the root being zero. Data flows downstream from the leaf fragments to the root fragment.
diff --git a/_docs/configure-drill/010-configure-drill-introduction.md b/_docs/configure-drill/010-configure-drill-introduction.md
index 3e3c855..4d55019 100644
--- a/_docs/configure-drill/010-configure-drill-introduction.md
+++ b/_docs/configure-drill/010-configure-drill-introduction.md
@@ -1,6 +1,6 @@
---
title: "Configure Drill Introduction"
-date: 2016-11-21 22:42:09 UTC
+date: 2018-12-08
parent: "Configure Drill"
---
@@ -26,4 +26,4 @@ statements is Parquet. Using a configuration option, you can modify Drill to sto
## Query Profile Data Storage Configuration
-To avoid problems working with the Web Console, you need to [configure the ZooKeeper PStore]({{site.baseurl}}/docs/persistent-configuration-storage/#configuring-zookeeper-pstore).
+To avoid problems working with the Web UI, you need to [configure the ZooKeeper PStore]({{site.baseurl}}/docs/persistent-configuration-storage/#configuring-zookeeper-pstore).
diff --git a/_docs/configure-drill/100-ports-used-by-drill.md b/_docs/configure-drill/100-ports-used-by-drill.md
index add40d9..35f7e44 100644
--- a/_docs/configure-drill/100-ports-used-by-drill.md
+++ b/_docs/configure-drill/100-ports-used-by-drill.md
@@ -1,6 +1,6 @@
---
title: "Ports Used by Drill"
-date: 2016-04-14 00:51:25 UTC
+date: 2018-12-08
parent: "Configure Drill"
---
@@ -9,8 +9,8 @@ The table below lists the default ports that Drill uses and provides description
| Default Port | Type | Configuration Option | Description |
|--------------|------|------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| 8047 | TCP | drill.exec.http.port | Needed for the Drill Web Console. |
-| 31010 | TCP | drill.exec.rpc.user.server.port | User port address. Used between nodes in a Drill cluster. Needed for an external client, such as Tableau, to connect into the cluster nodes. Also needed for the Drill Web Console. |
+| 8047 | TCP | drill.exec.http.port | Needed for the Drill Web UI. |
+| 31010 | TCP | drill.exec.rpc.user.server.port | User port address. Used between nodes in a Drill cluster. Needed for an external client, such as Tableau, to connect into the cluster nodes. Also needed for the Drill Web UI. |
| 31011 | TCP | drill.exec.rpc.bit.server.port | Control port address. Used between nodes in a Drill cluster. Needed for multi-node installation of Apache Drill. |
| 31012 | TCP | drill.exec.rpc.bit.server.port + 1 | Data port address. Used between nodes in a Drill cluster. Needed for multi-node installation of Apache Drill. |
diff --git a/_docs/configure-drill/configuration-options/010-configuration-options-introduction.md b/_docs/configure-drill/configuration-options/010-configuration-options-introduction.md
index ece82c5..abfdce5 100644
--- a/_docs/configure-drill/configuration-options/010-configuration-options-introduction.md
+++ b/_docs/configure-drill/configuration-options/010-configuration-options-introduction.md
@@ -1,6 +1,6 @@
---
title: "Configuration Options Introduction"
-date: 2018-09-20 21:56:52 UTC
+date: 2018-12-08
parent: "Configuration Options"
---
@@ -87,7 +87,7 @@ The sys.options table lists options that you can set at the system or session le
| planner.slice_target | 100000 | The number of records manipulated within a fragment before Drill parallelizes operations. [...]
| planner.width.max_per_node | 70% of the total number of processors on a node | Maximum number of threads that can run in parallel for a query on a node. A slice is an individual thread. This number indicates the maximum number of slices per query for the query’s major fragment on a node. [...]
| planner.width.max_per_query | 1000 | Same as max per node but applies to the query as executed by the entire cluster. For example, this value might be the number of active Drillbits, or a higher number to return results faster. [...]
-| security.admin.user_groups | n/a | Supported as of 1.4. A comma-separated list of administrator groups for Web Console security. [...]
+| security.admin.user_groups | n/a | Supported as of 1.4. A comma-separated list of administrator groups for Web UI security. [...]
| security.admin.users | | Supported as of 1.4. A comma-separated list of user names who you want to give administrator privileges. [...]
| store.format | parquet | Output format for data written to tables with the CREATE TABLE AS (CTAS) command. Allowed values are parquet, json, psv, csv, or tsv. [...]
| store.hive.parquet.optimize_scan_with_native_reader | FALSE | By default, Drill reads Hive tables using the native Hive reader. When you enable this option, Drill reads Hive tables using Drill native readers, which enables faster reads and enforces direct memory usage. Starting in Drill 1.14, this option also enables Drill to apply filter push down optimizations. Previously, this was the store.hive.optimize_scan_with_native_readers [...]
diff --git a/_docs/configure-drill/configuration-options/040-persistent-configuration-storage.md b/_docs/configure-drill/configuration-options/040-persistent-configuration-storage.md
index dd02056..2f0055c 100644
--- a/_docs/configure-drill/configuration-options/040-persistent-configuration-storage.md
+++ b/_docs/configure-drill/configuration-options/040-persistent-configuration-storage.md
@@ -1,6 +1,6 @@
---
title: "Persistent Configuration Storage"
-date: 2018-07-01 23:48:26 UTC
+date: 2018-12-08
parent: "Configuration Options"
---
Drill stores persistent configuration data in a persistent configuration store
@@ -23,11 +23,11 @@ store persistent configuration data. The ZooKeeper PStore provider stores all
of the persistent configuration data in ZooKeeper except for query profile
data. The ZooKeeper PStore provider offloads query profile data to the Drill log directory on Drill nodes.
-You need to configure the ZooKeeper PStore to use the Drill Web Console when running multiple Drillbits.
+You need to configure the ZooKeeper PStore to use the Drill Web UI when running multiple Drillbits.
### Why Configure the ZooKeeper PStore
-When you run multiple DrillBits, configure a specific location for ZooKeeper to offload the query profile data instead of accepting the default temporary location. All Drillbits in the cluster cannot access the temporary location. Consequently, when you do not configure a location on the distributed file system, queries sent to some Drillbits do not appear in the Completed section of the Drill Web Console. Also, some Running links that you click to get information about running queries a [...]
+When you run multiple DrillBits, configure a specific location for ZooKeeper to offload the query profile data instead of accepting the default temporary location. All Drillbits in the cluster cannot access the temporary location. Consequently, when you do not configure a location on the distributed file system, queries sent to some Drillbits do not appear in the Completed section of the Drill Web UI. Also, some Running links that you click to get information about running queries are br [...]
### How to Configure the ZooKeeper PStore
diff --git a/_docs/configure-drill/securing-drill/020-secure-communication-paths.md b/_docs/configure-drill/securing-drill/020-secure-communication-paths.md
index dbfd194..7d098fd 100644
--- a/_docs/configure-drill/securing-drill/020-secure-communication-paths.md
+++ b/_docs/configure-drill/securing-drill/020-secure-communication-paths.md
@@ -1,6 +1,6 @@
---
title: "Secure Communication Paths"
-date: 2017-08-17 18:54:32 UTC
+date: 2018-12-08
parent: "Securing Drill"
---
As illustrated in the following figure, Drill features five secure communication paths. Drill 1.11 introduces encryption between a Drill client and Drillbit.
@@ -21,7 +21,7 @@ Drillbit-to-drillbit encryption is not available yet.
## Web Client to Drillbit
-The Web Console and REST API clients are web clients. Web clients can:
+The Web UI and REST API clients are web clients. Web clients can:
- Submit and monitor queries
- Configure storage plugins
@@ -35,10 +35,10 @@ Impersonation and authorization are available through the web clients only when
| Security Capability | Description | Re [...]
|---------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--- [...]
-| Authentication | Users authenticate to a drillbit using a username and password form authenticator. By default, authentication is disabled. | [C [...]
-| Encryption | Drill usese SSL for HTTPS access to the Web Console. | [C [...]
+| Authentication | Users authenticate to a drillbit using a username and password form authenticator. By default, authentication is disabled. | [C [...]
+| Encryption | Drill usese SSL for HTTPS access to the Web UI. | [Config [...]
| Impersonation | Drill acts on behalf of the user on the session. This is usually the connection user (or the user that authenticates). This user can impersonate another user, which is allowed if the connection user is authorized to impersonate the target user based on the inbound impersonation policies (USER role). By default, impersonation is disabled. | [C [...]
-| Authorization | Queries execute on behalf of the web user. Users and administrators have different navigation bars. Various tabs are shown based on privileges. For example, only administrators can see the Storage tab and create/read/update/delete storage plugin configuration. | [C [...]
+| Authorization | Queries execute on behalf of the web user. Users and administrators have different navigation bars. Various tabs are shown based on privileges. For example, only administrators can see the Storage tab and create/read/update/delete storage plugin configuration. | [C [...]
## Java and C++ Client to Drillbit
diff --git a/_docs/configure-drill/securing-drill/030-roles-privileges.md b/_docs/configure-drill/securing-drill/030-roles-privileges.md
index e6fccd4..ec72dd2 100644
--- a/_docs/configure-drill/securing-drill/030-roles-privileges.md
+++ b/_docs/configure-drill/securing-drill/030-roles-privileges.md
@@ -1,28 +1,28 @@
---
title: "Roles and Privileges"
-date: 2017-03-15 00:30:47 UTC
+date: 2018-12-08
parent: "Securing Drill"
---
Drill has two roles that perform different functions:
* User (USER) role
-* Administrator (ADMIN) role
-
+* Administrator (ADMIN) role
+
## User Role
Users can execute queries on data that he/she has access to. Each storage plugin manages the read/write permissions. Users can create views on top of data to provide granular access to that data.
## Administrator Role
-When authentication is enabled, only Drill users who are assigned Drill cluster administrator privileges can perform the following tasks:
-
-- Change system-level options by issuing the ALTER SYSTEM command.
-- Update a storage plugin configuration through the REST API or Web Console.
-- Users and administrators have different navigation bars in the web console. Various tabs are shown based on privilege. For example, only administrators can see the Storage tab and create/read/update/delete storage plugin configuration.
-- View profiles of all queries that all users have run or are currently running in a cluster.
-- Cancel running queries that were launched by any user in the cluster.
+When authentication is enabled, only Drill users who are assigned Drill cluster administrator privileges can perform the following tasks:
+
+- Change system-level options by issuing the ALTER SYSTEM command.
+- Update a storage plugin configuration through the REST API or Web UI.
+- Users and administrators have different navigation bars in the Web UI. Various tabs are shown based on privilege. For example, only administrators can see the Storage tab and create/read/update/delete storage plugin configuration.
+- View profiles of all queries that all users have run or are currently running in a cluster.
+- Cancel running queries that were launched by any user in the cluster.
-See [Configuring Web Console and REST API Security]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security/) for more information.
+See [Configuring Web UI and REST API Security]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security/) for more information.
diff --git a/_docs/configure-drill/securing-drill/060-configuring-user-impersonation-with-hive-authorization.md b/_docs/configure-drill/securing-drill/060-configuring-user-impersonation-with-hive-authorization.md
index 3aaea9d..42cc045 100644
--- a/_docs/configure-drill/securing-drill/060-configuring-user-impersonation-with-hive-authorization.md
+++ b/_docs/configure-drill/securing-drill/060-configuring-user-impersonation-with-hive-authorization.md
@@ -1,6 +1,6 @@
---
title: "Configuring User Impersonation with Hive Authorization"
-date: 2016-01-05
+date: 2018-12-08
parent: "Securing Drill"
---
As of Drill 1.1, you can enable impersonation in Drill and configure authorization in Hive version 1.0 to authorize access to metadata in the Hive metastore repository and data in the Hive warehouse. Impersonation allows a service to act on behalf of a client while performing the action requested by the client. See [Configuring User Impersonation]({{site.baseurl}}/docs/configuring-user-impersonation).
@@ -215,7 +215,7 @@ Add the following required authorization parameters in hive-site.xml to configur
## Step 3: Modifying the Hive Storage Plugin
-Modify the Hive storage plugin configuration in the Drill Web Console to include specific authorization settings. The Drillbit that you use to access the Web Console must be running.
+Modify the Hive storage plugin configuration in the Drill Web UI to include specific authorization settings. The Drillbit that you use to access the Web UI must be running.
Complete the following steps to modify the Hive storage plugin:
diff --git a/_docs/configure-drill/securing-drill/091-configuring-drill-to-use-spnego-for-http-authentication.md b/_docs/configure-drill/securing-drill/091-configuring-drill-to-use-spnego-for-http-authentication.md
index 0de0910..75860cb 100644
--- a/_docs/configure-drill/securing-drill/091-configuring-drill-to-use-spnego-for-http-authentication.md
+++ b/_docs/configure-drill/securing-drill/091-configuring-drill-to-use-spnego-for-http-authentication.md
@@ -1,6 +1,6 @@
---
title: "Configuring Drill to use SPNEGO for HTTP Authentication"
-date: 2018-06-08 02:01:22 UTC
+date: 2018-12-08
parent: "Securing Drill"
---
@@ -85,7 +85,7 @@ To configure SPNEGO on the web server, complete the following steps:
An administrator or user can configure SPNEGO on the client (web browser or client tools, such as curl). To configure SPNEGO on the client, a Kerberos Ticket Granting Ticket must exist for the user accessing the web server. The Kerberos Ticket Granting Ticket generated on the client side is used by the web client to get a service ticket from the KDC. This service ticket is used to generate a SPNEGO token, which is presented to the web server for authentication.
-The client should use the same web server hostname (as configured in the server-side principal) to access the Drill Web Console. If the server hostname differs, SPNEGO authentication will fail. For example, if the server principal is `"HTTP/example.QA.LAB@QA.LAB”`, the client should use `http://example.QA.LAB:8047` as the Drill Web Console URL.
+The client should use the same web server hostname (as configured in the server-side principal) to access the Drill Web UI. If the server hostname differs, SPNEGO authentication will fail. For example, if the server principal is `"HTTP/example.QA.LAB@QA.LAB”`, the client should use `http://example.QA.LAB:8047` as the Drill Web UI URL.
The following sections provide instructions for configuring the supported client-side browsers:
@@ -134,15 +134,15 @@ Issue the following `curl` command to log in using SPNEGO, and save the authenti
curl -v --negotiate -c cookie.txt -u : http://<hostname>:8047/spnegoLogin
-Use the authenticated session cookie stored in the file, for example `cookie.txt`, to access the Drill Web Console pages, as shown in the following example:
+Use the authenticated session cookie stored in the file, for example `cookie.txt`, to access the Drill Web UI pages, as shown in the following example:
curl -v --negotiate -b cookie.txt -u : http://<hostname>:8047/query
Example: curl -v --negotiate -b cookie.txt -u : http://example.QA.LAB:8047/query
-##Logging in to the Drill Web Console
-With the addition of SPNEGO authentication in Drill 1.13, an administrator can configure FORM and/or SPNEGO authentication mechanisms. The Drill Web Console provides two possible log in options for a user depending on the configuration.
+##Logging in to the Drill Web UI
+With the addition of SPNEGO authentication in Drill 1.13, an administrator can configure FORM and/or SPNEGO authentication mechanisms. The Drill Web UI provides two possible log in options for a user depending on the configuration.
-If a user selects FORM, he/she must enter their username and password to access restricted pages in the Drill Web Console. The user is authenticated through PAM.
+If a user selects FORM, he/she must enter their username and password to access restricted pages in the Drill Web UI. The user is authenticated through PAM.
If the user selects SPNEGO, the user is automatically logged in if they are an authenticated Kerberos user.
diff --git a/_docs/configure-drill/securing-drill/092-configuring-web-ui-and-rest-api-security.md b/_docs/configure-drill/securing-drill/092-configuring-web-ui-and-rest-api-security.md
index b65ef39..3616a95 100644
--- a/_docs/configure-drill/securing-drill/092-configuring-web-ui-and-rest-api-security.md
+++ b/_docs/configure-drill/securing-drill/092-configuring-web-ui-and-rest-api-security.md
@@ -1,21 +1,21 @@
---
-title: "Configuring Web Console and REST API Security"
-date: 2018-09-28 23:05:25 UTC
+title: "Configuring Web UI and REST API Security"
+date: 2018-12-08
parent: "Securing Drill"
---
-Drill 1.5 extends [Drill user security]({{site.baseurl}}/docs/configuring-user-security/) to the Web Console and underlying REST API. As administrator, you can control the extent of access to the Web Console and REST API client applications. For example,
-you can limit the access of certain users to Web Console functionality, such as viewing the in-progress or completed queries of other users. You can limit users from viewing other users' query profiles, who can cancel queries of other users, and other functionality.
+Drill 1.5 extends [Drill user security]({{site.baseurl}}/docs/configuring-user-security/) to the Web UI and underlying REST API. As administrator, you can control the extent of access to the Web UI and REST API client applications. For example,
+you can limit the access of certain users to Web UI functionality, such as viewing the in-progress or completed queries of other users. You can limit users from viewing other users' query profiles, who can cancel queries of other users, and other functionality.
-With Web Console security in place, users who do not have administrator privileges need to use the SHOW SCHEMAS command instead of the Web Console for storage plugin configuration information.
+With Web UI security in place, users who do not have administrator privileges need to use the SHOW SCHEMAS command instead of the Web UI for storage plugin configuration information.
## HTTPS Support
-Drill 1.2 uses code-level support for transport layer security (TLS) to secure the Web Console and REST API. By default, the Web Console and REST API support the HTTP protocol. You set the following start-up option to TRUE to enable HTTPS support:
+Drill 1.2 uses code-level support for transport layer security (TLS) to secure the Web UI and REST API. By default, the Web UI and REST API support the HTTP protocol. You set the following start-up option to TRUE to enable HTTPS support:
`drill.exec.http.ssl_enabled`
By default this start-up option is set to FALSE.
-Drill generates a self-signed certificate that works with SSL for HTTPS access to the Web Console. Because Drill uses a self-signed certificate, you see a warning in the browser when you go to `https://<node IP address>:8047`. The Chrome browser, for example, requires you to click `Advanced`, and then `Proceed to <address> (unsafe)`. If you have a signed certificate by an authority, you can set up a custom SSL to avoid this warning. You can set up SSL to specify the keystore or truststor [...]
+Drill generates a self-signed certificate that works with SSL for HTTPS access to the Web UI. Because Drill uses a self-signed certificate, you see a warning in the browser when you go to `https://<node IP address>:8047`. The Chrome browser, for example, requires you to click `Advanced`, and then `Proceed to <address> (unsafe)`. If you have a signed certificate by an authority, you can set up a custom SSL to avoid this warning. You can set up SSL to specify the keystore or truststore, or [...]
## Setting Up a Custom SSL Configuration
@@ -32,15 +32,15 @@ As cluster administrator, you can set the following SSL configuration parameters
See [SSL Certificates in a Drill Cluster]({{site.baseurl}}/docs/configuring-ssl-tls-for-encryption/#ssl-certificates-in-a-drill-cluster) for more information.
-## Prerequisites for Web Console and REST API Security
+## Prerequisites for Web UI and REST API Security
-You need to perform the following configuration tasks using Web Console and REST API security.
+You need to perform the following configuration tasks using Web UI and REST API security.
* Configure [user security]({{site.baseurl}}/docs/configuring-user-security/)
-* Set up Web Console administrators
- Optionally, you can set up Web Console administrator-user groups to facilitate management of multiple Web Console administrators.
+* Set up Web UI administrators
+ Optionally, you can set up Web UI administrator-user groups to facilitate management of multiple Web UI administrators.
-## Setting up Web Console Administrators and Administrator-User Groups
+## Setting up Web UI Administrators and Administrator-User Groups
Configure the following system options using the [ALTER SYSTEM]({{site.baseurl}}/docs/alter-system/) command:
@@ -49,11 +49,11 @@ Configure the following system options using the [ALTER SYSTEM]({{site.baseurl}}
* security.admin.user_groups
Set the value of this option to a comma-separated list of administrator groups.
-Any user who is a member of any group listed in security.admin.user.groups is a Drill cluster administrator. Any user for whom you have configured Drill user authentication, but not set up as a Drill cluster administrator, has only user privileges to access the Web Console and REST API client applications.
+Any user who is a member of any group listed in security.admin.user.groups is a Drill cluster administrator. Any user for whom you have configured Drill user authentication, but not set up as a Drill cluster administrator, has only user privileges to access the Web UI and REST API client applications.
-## Web Console and REST API Privileges
+## Web UI and REST API Privileges
-The following table and subsections describe the privilege levels for accessing the REST API methods and corresponding Web Console functions:
+The following table and subsections describe the privilege levels for accessing the REST API methods and corresponding Web UI functions:
* Administrator (ADMIN)
* User (USER)
diff --git a/_docs/connect-a-data-source/036-configuring-storage-plugins.md b/_docs/connect-a-data-source/036-configuring-storage-plugins.md
index 6aa4a37..6e470df 100644
--- a/_docs/connect-a-data-source/036-configuring-storage-plugins.md
+++ b/_docs/connect-a-data-source/036-configuring-storage-plugins.md
@@ -1,6 +1,6 @@
---
title: "Configuring Storage Plugins"
-date: 2018-07-18 22:29:15 UTC
+date: 2018-12-08
parent: "Storage Plugin Configuration"
---
@@ -19,7 +19,7 @@ You can use the Drill Web UI to update or add a new storage plugin configuration
To create a name and new configuration:
1. [Start the Drill shell]({{site.baseurl}}/docs/starting-drill-on-linux-and-mac-os-x/).
-2. [Start the Web Console]({{site.baseurl}}/docs/starting-the-web-console/). The Storage tab appears in the Web Console if you are [authorized]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security/) to view, update, or add storage plugins.
+2. [Start the Web UI]({{site.baseurl}}/docs/starting-the-web-console/). The Storage tab appears in the Web UI if you are [authorized]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security/) to view, update, or add storage plugins.
3. On the Storage tab, enter a name in **New Storage Plugin**.
Each configuration registered with Drill must have a distinct
name. Names are case-sensitive.
@@ -51,7 +51,7 @@ If you need to add a storage plugin configuration to Drill and do not want to us
The storage plugin configuration name.
* config
- The attribute settings as entered in the Web Console.
+ The attribute settings as entered in the Web UI.
For example, this command creates a storage plugin named myplugin for reading files of an unknown type located on the root of the file system:
diff --git a/_docs/connect-a-data-source/080-drill-default-input-format.md b/_docs/connect-a-data-source/080-drill-default-input-format.md
index 1105670..8e4d8c7 100644
--- a/_docs/connect-a-data-source/080-drill-default-input-format.md
+++ b/_docs/connect-a-data-source/080-drill-default-input-format.md
@@ -1,6 +1,6 @@
---
title: "Drill Default Input Format"
-date: 2018-02-08 02:32:55 UTC
+date: 2018-12-08
parent: "Storage Plugin Configuration"
---
You can define a default input format to tell Drill what file type exists in a
@@ -25,7 +25,7 @@ You must have a [defined workspace]({{ site.baseurl }}/docs/workspaces) before y
To define a default input format for a workspace:
- 1. Navigate to the [Drill Web Console]({{ site.baseurl }}/docs/plugin-configuration-basics/#using-the-drill-web-console). The Drillbit process must be running on the node before you connect to the Drill Web Console.
+ 1. Navigate to the [Drill Web UI]({{ site.baseurl }}/docs/plugin-configuration-basics/#using-the-drill-web-console). The Drillbit process must be running on the node before you connect to the Drill Web UI.
2. Select **Storage** in the toolbar.
3. Click **Update** next to the storage plugin configuration for which you want to define a default input format for a workspace.
4. In the Configuration area, locate the workspace, and change the `defaultInputFormat` attribute to any of the supported file types.
diff --git a/_docs/connect-a-data-source/plugins/060-hbase-storage-plugin.md b/_docs/connect-a-data-source/plugins/060-hbase-storage-plugin.md
index 347cc12..1c91333 100644
--- a/_docs/connect-a-data-source/plugins/060-hbase-storage-plugin.md
+++ b/_docs/connect-a-data-source/plugins/060-hbase-storage-plugin.md
@@ -1,11 +1,11 @@
---
title: "HBase Storage Plugin"
-date: 2018-11-02
+date: 2018-12-08
parent: "Connect a Data Source"
---
When connecting Drill to an HBase data source using the HBase storage plugin installed with Drill, you need to specify a ZooKeeper quorum. Drill supports HBase version 1.x.
-To view or change the HBase storage plugin configuration, use the [Drill Web Console]({{ site.baseurl }}/docs/plugin-configuration-basics/#using-the-drill-web-console). In the Web Console, select the **Storage** tab, and then click the **Update** button for the `hbase` storage plugin configuration. The following example shows a typical HBase storage plugin:
+To view or change the HBase storage plugin configuration, use the [Drill Web UI]({{ site.baseurl }}/docs/plugin-configuration-basics/#using-the-drill-web-console). In the Web UI, select the **Storage** tab, and then click the **Update** button for the `hbase` storage plugin configuration. The following example shows a typical HBase storage plugin:
{
"type": "hbase",
diff --git a/_docs/connect-a-data-source/plugins/070-hive-storage-plugin.md b/_docs/connect-a-data-source/plugins/070-hive-storage-plugin.md
index 4a944d2..cfea93a 100644
--- a/_docs/connect-a-data-source/plugins/070-hive-storage-plugin.md
+++ b/_docs/connect-a-data-source/plugins/070-hive-storage-plugin.md
@@ -1,6 +1,6 @@
---
title: "Hive Storage Plugin"
-date: 2018-07-11 21:37:17 UTC
+date: 2018-12-08
parent: "Connect a Data Source"
---
Prior to Drill 1.13, Drill supported Hive 1.0. Drill 1.13 and later includes version 2.3.2 of the Hive client, which adds support for queries on transactional (ACID) and non-transactional Hive bucketed ORC tables. The updated Hive libraries are backward compatible with earlier versions of the Hive server and metastore.
@@ -69,8 +69,8 @@ To connect Drill to a remote Hive metastore:
1. Issue the following command to start the Hive metastore service on the system specified in the `hive.metastore.uris`:
`hive --service metastore`
-2. In the [Drill Web Console]({{ site.baseurl }}/docs/plugin-configuration-basics/#using-the-drill-web-console), select the **Storage** tab.
-3. In the list of disabled storage plugins in the Drill Web Console, click **Update** next to `hive`.
+2. In the [Drill Web UI]({{ site.baseurl }}/docs/plugin-configuration-basics/#using-the-drill-web-console), select the **Storage** tab.
+3. In the list of disabled storage plugins in the Drill Web UI, click **Update** next to `hive`.
4. In the configuration window, add the `Thrift URI` and port to `hive.metastore.uris`. For example:
...
@@ -110,7 +110,7 @@ To connect Drill to a remote Hive metastore:
The Hive metastore configuration is embedded within the Drill process. Configure an embedded metastore only in a cluster that runs a single Drillbit and only for testing purposes. Do not embed the Hive metastore in production systems.
-Provide the metastore database configuration settings in the Drill Web Console. Before you configure an embedded Hive metastore, verify that the driver you use to connect to the Hive metastore is in the Drill classpath located in `/<drill installation directory>/lib/.` If the driver is not there, copy the driver to `/<drill
+Provide the metastore database configuration settings in the Drill Web UI. Before you configure an embedded Hive metastore, verify that the driver you use to connect to the Hive metastore is in the Drill classpath located in `/<drill installation directory>/lib/.` If the driver is not there, copy the driver to `/<drill
installation directory>/lib` on the Drill node. For more information about storage types and configurations, refer to ["Hive Metastore Administration"](https://cwiki.apache.org/confluence/display/Hive/AdminManual+MetastoreAdmin).
### Hive Embedded Metastore Configuration
@@ -118,7 +118,7 @@ installation directory>/lib` on the Drill node. For more information about stora
To configure an embedded Hive metastore, complete the following
steps:
-1. In the [Drill Web Console]({{ site.baseurl }}/docs/plugin-configuration-basics/#using-the-drill-web-console), and select the **Storage** tab.
+1. In the [Drill Web UI]({{ site.baseurl }}/docs/plugin-configuration-basics/#using-the-drill-web-console), and select the **Storage** tab.
2. In the disabled storage plugin configurations section, click **Update** next to `hive`.
3. In the configuration window, add the database configuration settings.
diff --git a/_docs/connect-a-data-source/plugins/080-rdbms-storage-plugin.md b/_docs/connect-a-data-source/plugins/080-rdbms-storage-plugin.md
index 11b5b82..ea84e87 100644
--- a/_docs/connect-a-data-source/plugins/080-rdbms-storage-plugin.md
+++ b/_docs/connect-a-data-source/plugins/080-rdbms-storage-plugin.md
@@ -1,6 +1,6 @@
---
title: "RDBMS Storage Plugin"
-date: 2018-06-08 02:01:23 UTC
+date: 2018-12-08
parent: "Connect a Data Source"
---
Apache Drill supports querying a number of RDBMS instances. This allows you to connect your traditional databases to your Drill cluster so you can have a single view of both your relational and NoSQL datasources in a single system.
@@ -14,18 +14,18 @@ Drill is designed to work with any relational datastore that provides a JDBC dri
1. [Install Drill]({{ site.baseurl }}/docs/installing-drill-in-embedded-mode), if you do not already have it installed.
2. Copy your database's JDBC driver into the jars/3rdparty directory. (You'll need to do this on every node.)
3. Restart Drill. See [Starting Drill in Distributed Mode]({{site.baseurl}}/docs/starting-drill-in-distributed-mode/).
- 4. Add a new storage configuration to Drill through the Web Console. Example configurations for [Oracle](#Example-Oracle-Configuration), [SQL Server](#Example-SQL-Server-Configuration), [MySQL](#Example-MySQL-Configuration) and [Postgres](#Example-Postgres-Configuration) are provided below.
+ 4. Add a new storage configuration to Drill through the Web UI. Example configurations for [Oracle](#Example-Oracle-Configuration), [SQL Server](#Example-SQL-Server-Configuration), [MySQL](#Example-MySQL-Configuration) and [Postgres](#Example-Postgres-Configuration) are provided below.
**Example: Working with MySQL**
-Drill communicates with MySQL through the JDBC driver using the configuration that you specify in the Web Console or through the [REST API]({{site.baseurl}}/docs/plugin-configuration-basics/#storage-plugin-rest-api).
+Drill communicates with MySQL through the JDBC driver using the configuration that you specify in the Web UI or through the [REST API]({{site.baseurl}}/docs/plugin-configuration-basics/#storage-plugin-rest-api).
{% include startnote.html %}Verify that MySQL is running and the MySQL driver is in place before you configure the JDBC storage plugin.{% include endnote.html %}
To configure the JDBC storage plugin:
1. [Start the Drill shell]({{site.baseurl}}/docs/starting-drill-on-linux-and-mac-os-x/).
-1. [Start the Web Console]({{site.baseurl}}/docs/starting-the-web-console/).
+1. [Start the Web UI]({{site.baseurl}}/docs/starting-the-web-console/).
1. On the Storage tab, enter a name in **New Storage Plugin**. For example, enter `myplugin`.
Each configuration registered with Drill must have a distinct name. Names are case-sensitive.
diff --git a/_docs/connect-a-data-source/plugins/090-mongodb-storage-plugin.md b/_docs/connect-a-data-source/plugins/090-mongodb-storage-plugin.md
index c83773c..a032e4b 100644
--- a/_docs/connect-a-data-source/plugins/090-mongodb-storage-plugin.md
+++ b/_docs/connect-a-data-source/plugins/090-mongodb-storage-plugin.md
@@ -1,6 +1,6 @@
---
title: "MongoDB Storage Plugin"
-date: 2018-11-02
+date: 2018-12-08
parent: "Connect a Data Source"
---
## Overview
@@ -25,12 +25,12 @@ To query MongoDB with Drill, you install Drill and MongoDB, and then you import
## Configuring MongoDB
-Drill must be running in order to access the Web Console to configure a storage plugin configuration. Start Drill and view and enable the MongoDB storage plugin configuration as described in the following procedure:
+Drill must be running in order to access the Web UI to configure a storage plugin configuration. Start Drill and view and enable the MongoDB storage plugin configuration as described in the following procedure:
1. [Start the Drill shell]({{site.baseurl}}/docs/starting-drill-on-linux-and-mac-os-x/).
- The Drill shell needs to be running to access the Drill Web Console.
- 2. In the [Drill Web Console]({{ site.baseurl }}/docs/plugin-configuration-basics/#using-the-drill-web-console), select the **Storage** tab.
+ The Drill shell needs to be running to access the Drill Web UI.
+ 2. In the [Drill Web UI]({{ site.baseurl }}/docs/plugin-configuration-basics/#using-the-drill-web-console), select the **Storage** tab.
4. Under Disabled Storage Plugins, select **Update** to choose the `mongo` storage plugin configuration.
5. In the Configuration window, take a look at the default configuration:
diff --git a/_docs/connect-a-data-source/plugins/112-opentsdb-storage-plugin.md b/_docs/connect-a-data-source/plugins/112-opentsdb-storage-plugin.md
index 690bccb..40745eb 100644
--- a/_docs/connect-a-data-source/plugins/112-opentsdb-storage-plugin.md
+++ b/_docs/connect-a-data-source/plugins/112-opentsdb-storage-plugin.md
@@ -1,18 +1,18 @@
---
title: "OpenTSDB Storage Plugin"
-date: 2018-02-08 02:32:57 UTC
+date: 2018-12-08
parent: "Connect a Data Source"
---
As of Drill 1.12, Drill provides a storage plugin for OpenTSDB. The OpenTSDB storage plugin uses the REST API with OpenTSDB.
-When you install Drill, a preconfigured OpenTSDB storage plugin is available on the Storage page in the Drill Web Console. Once you enable the storage plugin, you can query OpenTSDB from Drill.
+When you install Drill, a preconfigured OpenTSDB storage plugin is available on the Storage page in the Drill Web UI. Once you enable the storage plugin, you can query OpenTSDB from Drill.
The following sections describe how to enable the OpenTSDB storage plugin in Drill and the types of queries Drill currently supports on OpenTSDB. You can refer to the [README](https://github.com/apache/drill/blob/master/contrib/storage-opentsdb/README.md) and [OpenTSDB](http://opentsdb.net/) for additional information.
##Enabling the OpenTSDB Storage Plugin
-To enable the OpenTSDB storage plugin, enter the following URL in the address bar of your browser to access the Storage page in the Drill Web Console:
+To enable the OpenTSDB storage plugin, enter the following URL in the address bar of your browser to access the Storage page in the Drill Web UI:
http://<drill-node-ip-address>:8047/storage/
diff --git a/_docs/connect-a-data-source/plugins/113-kafka-storage-plugin.md b/_docs/connect-a-data-source/plugins/113-kafka-storage-plugin.md
index 9e0953e..4c973c0 100644
--- a/_docs/connect-a-data-source/plugins/113-kafka-storage-plugin.md
+++ b/_docs/connect-a-data-source/plugins/113-kafka-storage-plugin.md
@@ -1,10 +1,10 @@
---
title: "Kafka Storage Plugin"
-date: 2018-07-18 22:29:15 UTC
+date: 2018-12-08
parent: "Connect a Data Source"
---
-As of Drill 1.12, Drill provides a storage plugin for Kafka. The Kafka storage plugin enables you to run SQL queries on Apache Kafka and perform interactive analysis on the data. When you install Drill, a preconfigured Kafka storage plugin is available on the Storage page in the Drill Web Console. Once you enable and configure the storage plugin, you can query Kafka from Drill.
+As of Drill 1.12, Drill provides a storage plugin for Kafka. The Kafka storage plugin enables you to run SQL queries on Apache Kafka and perform interactive analysis on the data. When you install Drill, a preconfigured Kafka storage plugin is available on the Storage page in the Drill Web UI. Once you enable and configure the storage plugin, you can query Kafka from Drill.
The following sections provide information about the Kafka storage plugin, how to enable and configure the Kafka storage plugin in Drill, options that you can set at the system or session level, and example queries on a Kafka data source. You can refer to the [README](https://github.com/apache/drill/tree/master/contrib/storage-kafka#drill-kafka-plugin) and [Apache Kafka](https://kafka.apache.org/) for additional information.
@@ -84,11 +84,11 @@ This API is used to determine the startOffset for each partition in a topic. Not
## Enabling and Configuring the Kafka Storage Plugin
-Enable the Kafka storage plugin on the Storage page of the Drill Web Console and then edit the configuration as needed.
+Enable the Kafka storage plugin on the Storage page of the Drill Web UI and then edit the configuration as needed.
The Kafka storage plugin configuration contains the `kafkaConsumerProps` property which supports typical Kafka consumer properties, as described in [Kafka Consumer Configs](https://kafka.apache.org/documentation/#consumerconfigs).
-To enable the Kafka storage plugin, enter the following URL in the address bar of your browser to access the Storage page in the Drill Web Console:
+To enable the Kafka storage plugin, enter the following URL in the address bar of your browser to access the Storage page in the Drill Web UI:
http://<drill-node-ip-address>:8047/storage/
diff --git a/_docs/developer-information/rest-api/010-rest-api-introduction.md b/_docs/developer-information/rest-api/010-rest-api-introduction.md
index 7db18ad..7ef22bc 100644
--- a/_docs/developer-information/rest-api/010-rest-api-introduction.md
+++ b/_docs/developer-information/rest-api/010-rest-api-introduction.md
@@ -1,12 +1,12 @@
---
title: "REST API Introduction"
-date: 2018-02-09 00:16:00 UTC
+date: 2018-12-08
parent: "REST API"
---
-The Drill REST API provides programmatic access to Drill through the [Web Console]({{site.baseurl}}/docs/starting-the-web-console/). Using HTTP requests, you can run queries, perform storage plugin tasks, such as creating a storage plugin, obtain profiles of queries, and get current memory metrics.
+The Drill REST API provides programmatic access to Drill through the [Web UI]({{site.baseurl}}/docs/starting-the-web-console/). Using HTTP requests, you can run queries, perform storage plugin tasks, such as creating a storage plugin, obtain profiles of queries, and get current memory metrics.
-AN HTTP request uses the familiar Web Console URI:
+AN HTTP request uses the familiar Web UI URI:
`http://<IP address or host name>:8047`
@@ -16,7 +16,7 @@ Before making HTTP requests, [start Drill]({{site.baseurl}}/docs/starting-drill-
Several examples in the document use the donuts.json file. To download this file, go to [Drill test resources](https://github.com/apache/drill/blob/master/exec/java-exec/src/test/resources) page, locate donuts.json in the list of files, and download it. When using cURL, use unicode \u0027 for the single quotation mark as shown in the Query example.
-This documentation presents HTTP methods in the same order as functions appear in the Web Console:
+This documentation presents HTTP methods in the same order as functions appear in the Web UI:
[**Query**]({{site.baseurl}}/docs/rest-api-introduction/#query)
diff --git a/_docs/developer-information/rest-api/011-submitting-queries-from-the-rest-api-when-impersonation-is-enabled-and-authentication-is-disabled.md b/_docs/developer-information/rest-api/011-submitting-queries-from-the-rest-api-when-impersonation-is-enabled-and-authentication-is-disabled.md
index 84dfcbf..d3984bf 100644
--- a/_docs/developer-information/rest-api/011-submitting-queries-from-the-rest-api-when-impersonation-is-enabled-and-authentication-is-disabled.md
+++ b/_docs/developer-information/rest-api/011-submitting-queries-from-the-rest-api-when-impersonation-is-enabled-and-authentication-is-disabled.md
@@ -1,6 +1,6 @@
---
title: "Submitting Queries from the REST API when Impersonation is Enabled and Authentication is Disabled"
-date: 2018-02-08 00:38:56 UTC
+date: 2018-12-08
parent: "REST API"
---
@@ -11,12 +11,12 @@ Prior to Drill 1.12, there was no way to provide a username when running queries
This feature only works when impersonation is enabled and authentication is disabled. If impersonation and authentication are both disabled, a user cannot enter a username and all queries run as the anonymous user.
-A user can issue queries through the Drill Web Console, SQLLine, using curl commands, or Java code, as follows:
+A user can issue queries through the Drill Web UI, SQLLine, using curl commands, or Java code, as follows:
-##Drill Web Console
-You can submit a query through the Query page in the Drill Web Console after entering a valid username in the Username field. The Drill Web Console is accessible through the URL http(s)://<ip-address>:8047.
+##Drill Web UI
+You can submit a query through the Query page in the Drill Web UI after entering a valid username in the Username field. The Drill Web UI is accessible through the URL http(s)://<ip-address>:8047.
-To re-run a query from the Profiles page in the Drill Web Console, you must submit a username prior to re-running the query.
+To re-run a query from the Profiles page in the Drill Web UI, you must submit a username prior to re-running the query.
##SQLLine
diff --git a/_docs/getting-started/010-drill-introduction.md b/_docs/getting-started/010-drill-introduction.md
index 8ae054e..23cadcd 100644
--- a/_docs/getting-started/010-drill-introduction.md
+++ b/_docs/getting-started/010-drill-introduction.md
@@ -1,6 +1,6 @@
---
title: "Drill Introduction"
-date: 2018-08-05 04:52:48 UTC
+date: 2018-12-08
parent: "Getting Started"
---
Drill is an Apache open-source SQL query engine for Big Data exploration.
@@ -69,7 +69,7 @@ Drill 1.12 provides the following new features and improvements:
- System options improvements, including a new internal system options table (DRILL-5723)
- Ability to prevent users from accessing a path outside the current workspace (DRILL-5964)
- Ability to put the server in quiescent mode for a graceful shutdown (DRILL-4286)
-- The Drill Web Console lists the completion of successfully completed queries as "successful" (DRILL-5923)
+- The Drill Web UI lists the completion of successfully completed queries as "successful" (DRILL-5923)
## What's New in Apache Drill 1.11
@@ -93,7 +93,7 @@ Drill 1.10 provides the following new features and improvements:
* Support for the [CREATE TEMPORARY TABLE AS (CTTAS)]({{site.baseurl}}/docs/create-temporary-table-as-cttas//) command.
* A [JDBC connection option]({{site.baseurl}}/docs/using-the-jdbc-driver/#using-the-jdbc-url-format-for-a-direct-drillbit-connection) that improves fault tolerance when connecting directly to a Drill node from a client.
-* The [Web Console]({{site.baseurl}}/docs/identifying-multiple-drill-versions-in-a-cluster) displays the Drill version and additional query profile statistics.
+* The [Web UI]({{site.baseurl}}/docs/identifying-multiple-drill-versions-in-a-cluster) displays the Drill version and additional query profile statistics.
* Drill implicitly interprets the [INT96]({{site.baseurl}}/docs/parquet-format/#about-int96-support/) timestamp data type in Parquet files.
* Support for Kerberos authentication between the client and drillbit.
@@ -178,7 +178,7 @@ Javadocs and better application dependency compatibility
* NTILE
* LAG and LEAD
* FIRST_VALUE and LAST_VALUE
-* [HTTPS support]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security/) for Web Console operations
+* [HTTPS support]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security/) for Web UI operations
* Performance improvements for [querying HBase]({{site.baseurl}}/docs/querying-hbase/#querying-big-endian-encoded-data), which includes leveraging [ordered byte encoding]({{site.baseurl}}/docs/querying-hbase/#leveraging-hbase-ordered-byte-encoding)
* [Optimized reads]({{site.baseurl}}/docs/querying-hive/#optimizing-reads-of-parquet-backed-tables) of Parquet-backed, Hive tables
* Read support for the [Parquet INT96 type]({{site.baseurl}}/docs/parquet-format/#about-int96-support) and a new TIMESTAMP_IMPALA type used with the [CONVERT_FROM]({{site.baseurl}}/docs/supported-data-types/#data-types-for-convert_to-and-convert_from-functions) function decodes a timestamp from Hive or Impala.
diff --git a/_docs/install/060-starting-the-web-ui.md b/_docs/install/060-starting-the-web-ui.md
index da695b2..2059333 100644
--- a/_docs/install/060-starting-the-web-ui.md
+++ b/_docs/install/060-starting-the-web-ui.md
@@ -1,12 +1,12 @@
---
-title: "Starting the Web Console"
-date: 2018-04-19 01:45:25 UTC
+title: "Starting the Web UI"
+date: 2018-12-08
parent: Install Drill
---
-The Drill Web Console is one of several [client interfaces](/docs/architecture-introduction/#drill-clients) you can use to access Drill.
+The Drill Web UI is one of several [client interfaces](/docs/architecture-introduction/#drill-clients) you can use to access Drill.
-To open the Drill Web Console, launch a web browser, and go to one of the following URLs depending on the configuration of HTTPS support:
+To open the Drill Web UI, launch a web browser, and go to one of the following URLs depending on the configuration of HTTPS support:
* `http://<IP address or host name>:8047`
Use this URL when [HTTPS support]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security/#https-support) is disabled (the default).
@@ -17,24 +17,24 @@ Use this URL when running ./drill-embedded.
## Drill 1.2 and Later
-If [user authentication]({{site.baseurl}}/docs/configuring-user-authentication/) is not enabled, all the Web Console controls appear to users as well as administrators:
+If [user authentication]({{site.baseurl}}/docs/configuring-user-authentication/) is not enabled, all the Web UI controls appear to users as well as administrators:
-
+
-**Note:** As of Drill 1.12, users must enter a username to issue queries through the Query page in the Drill Web Console if user impersonation is enabled and authentication is disabled. To re-run a query from the Profiles page, users must also submit a username. See [REST API]({{site.baseurl}}/docs/submitting-queries-from-the-rest-api-when-impersonation-is-enabled-and-authentication-is-disabled/) for more information.
+**Note:** As of Drill 1.12, users must enter a username to issue queries through the Query page in the Drill Web UI if user impersonation is enabled and authentication is disabled. To re-run a query from the Profiles page, users must also submit a username. See [REST API]({{site.baseurl}}/docs/submitting-queries-from-the-rest-api-when-impersonation-is-enabled-and-authentication-is-disabled/) for more information.
-**Note:** As of Drill 1.13, an administrator can configure FORM and/or SPNEGO authentication mechanisms. The Drill Web Console provides two possible log in options for a user depending on the configuration. If a user selects FORM, s/he must enter their username and password to access restricted pages in the Drill Web Console. The user is authenticated through PAM. If the user selects SPNEGO, the user is automatically logged in if they are an authenticated Kerberos user. If accessing a pr [...]
+**Note:** As of Drill 1.13, an administrator can configure FORM and/or SPNEGO authentication mechanisms. The Drill Web UI provides two possible log in options for a user depending on the configuration. If a user selects FORM, s/he must enter their username and password to access restricted pages in the Drill Web UI. The user is authenticated through PAM. If the user selects SPNEGO, the user is automatically logged in if they are an authenticated Kerberos user. If accessing a protected pa [...]
If [user authentication]({{site.baseurl}}/docs/configuring-user-authentication/) is enabled, Drill prompts you for a user name/password:
-
+
-If an [administrator]({{ site.baseurl }}/docs/configuring-user-authentication/#administrator-privileges) logs in, all the Web Console controls appear: Query, Profiles, Storage, Metrics, Threads, and Options. The Profiles page for administrators contains the profiles of all queries executed on a cluster. Only administrators can see and use the Storage tab to view, update, or add a new [storage plugin configuration]({{site.baseurl}}/docs/plugin-configuration-basics/). Only administrators c [...]
+If an [administrator]({{ site.baseurl }}/docs/configuring-user-authentication/#administrator-privileges) logs in, all the Web UI controls appear: Query, Profiles, Storage, Metrics, Threads, and Options. The Profiles page for administrators contains the profiles of all queries executed on a cluster. Only administrators can see and use the Storage tab to view, update, or add a new [storage plugin configuration]({{site.baseurl}}/docs/plugin-configuration-basics/). Only administrators can se [...]
-
+
-If a user, who is not an administrator, logs in, the Web Console controls are limited to Query, Metrics, and Profiles. The Profiles tab for a non-administrator user contains the profiles of all queries the user issued either through ODBC, JDBC, or the Web Console.
+If a user, who is not an administrator, logs in, the Web UI controls are limited to Query, Metrics, and Profiles. The Profiles tab for a non-administrator user contains the profiles of all queries the user issued either through ODBC, JDBC, or the Web UI.
-
+
diff --git a/_docs/install/061-stopping-drill.md b/_docs/install/061-stopping-drill.md
index 23609ae..1bf4d1b 100644
--- a/_docs/install/061-stopping-drill.md
+++ b/_docs/install/061-stopping-drill.md
@@ -1,6 +1,6 @@
---
title: "Stopping Drill"
-date: 2018-04-03 23:37:00 UTC
+date: 2018-12-08
parent: Install Drill
---
@@ -13,9 +13,9 @@ To stop the Drill process on the node, issue the `drillbit.sh stop` command, as
###Gracefully Shutting Down the Drill Process
-Graceful Shutdown is enabled by default. You can gracefully shut down a Drillbit from the command line or the Drill Web Console.
+Graceful Shutdown is enabled by default. You can gracefully shut down a Drillbit from the command line or the Drill Web UI.
-You can only use the Graceful Shutdown option in the Drill Web Console to shut down the Drillbit from which you accessed the Drill Web Console. For example, if you accessed the Drill Web Console at `http://drillbit1:8047`, you can only use the Graceful Shutdown option to shut down drillbit1. When you initiate graceful shutdown from the Drill Web Console, the console posts an alert stating that a graceful shutdown was triggered. You can see the progress of the shut down as the Drillbit co [...]
+You can only use the Graceful Shutdown option in the Drill Web UI to shut down the Drillbit from which you accessed the Drill Web UI. For example, if you accessed the Drill Web UI at `http://drillbit1:8047`, you can only use the Graceful Shutdown option to shut down drillbit1. When you initiate graceful shutdown from the Drill Web UI, the console posts an alert stating that a graceful shutdown was triggered. You can see the progress of the shut down as the Drillbit completes queries and [...]
**Note:** If security ([https]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security/)) is enabled, only and administrator can perform a graceful shutdown.
@@ -29,22 +29,22 @@ The following list describes the various states that a Drillbit transitions thro
- **Online**: The Drillbit has started and registered with the cluster coordinator, such as ZooKeeper. ZooKeeper then shares the state of the Drillbit with other Drillbits in the cluster. Drillbits in the online state can all accept and process queries.
- **Quiescent**: When a Drillbit receives a graceful shutdown request, the Drillbit transitions into the quiescent state and shares its status change with the ZooKeeper. The ZooKeeper notifies the other Drillbits in the cluster of the Drillbit’s status change. Once the other Drillbits get the status update, they do not assign work to the Drillbit. However, if the Foreman assigns work to the Drillbit as the status update occurs, the Drillbit waits to complete work before shutting down. Th [...]
- **Grace**: A period in which a Drillbit can accept new queries from the Foreman. You can configure (at the system level) the amount of time a Drillbit can remain in this phase using the `drill.exe.grace_period_ms` option. Set this value in milliseconds. The default value is 0. There is no maximum limit. Ideally, you should give as little time as possible or no longer than twice the default heartbeat time of the ZooKeeper. For example, if the heartbeat is 5 seconds, set the value [...]
- - **Draining**: When the grace period ends, the Drillbit enters the draining phase of the quiescent state. In this phase, the Drillbit cannot accept new queries, but waits for the running queries to complete. You can view the draining queries in the Drill Web Console. The Index page in the Web Console shows the queries and fragments currently running on the node.
+ - **Draining**: When the grace period ends, the Drillbit enters the draining phase of the quiescent state. In this phase, the Drillbit cannot accept new queries, but waits for the running queries to complete. You can view the draining queries in the Drill Web UI. The Index page in the Web UI shows the queries and fragments currently running on the node.
- **Offline**: The phase the Drillbit enters after completing all queries.
- **Shutdown**: The final state in which a Drillbit removes itself from the ZooKeeper registry.
**Shutting Down a Drillbit Gracefully**
-You can gracefully shut down a Drillbit from the command line or the Drill Web Console.
+You can gracefully shut down a Drillbit from the command line or the Drill Web UI.
From the command line, run the following command on the node you want to shut down:
drillbit.sh graceful_stop
-From the Drill Web Console, enter the following URL in your browser’s address bar:
+From the Drill Web UI, enter the following URL in your browser’s address bar:
http://<IP address or host name>:8047 or https://<IP address or host name>:8047
-In the Drill Web Console, open the Index page, and click Shutdown next to the Drillbit you want to shut down.
+In the Drill Web UI, open the Index page, and click Shutdown next to the Drillbit you want to shut down.
diff --git a/_docs/install/070-identifying-multiple-drill-versions-in-a-cluster.md b/_docs/install/070-identifying-multiple-drill-versions-in-a-cluster.md
index 5e0099d..037ff9b 100644
--- a/_docs/install/070-identifying-multiple-drill-versions-in-a-cluster.md
+++ b/_docs/install/070-identifying-multiple-drill-versions-in-a-cluster.md
@@ -1,10 +1,10 @@
---
title: "Identifying Multiple Drill Versions in a Cluster"
-date: 2018-02-08 00:38:57 UTC
+date: 2018-12-08
parent: Install Drill
---
-As of Drill 1.10, the Web Console displays the Drill version running on each Drill node in the cluster, as shown in the following image:
+As of Drill 1.10, the Web UI displays the Drill version running on each Drill node in the cluster, as shown in the following image:
@@ -14,13 +14,13 @@ You can also retrieve the version information by running the following query:
If the version of Drill differs between nodes, a warning message appears. The nodes running the current version have a green label, while the nodes running another version have a red label, as shown in the image above.
-The Drill node from which you access the Web Console defines the current version. For example, assume you have two Drill nodes in a cluster with the following IP addresses, versions, and Web Console access:
+The Drill node from which you access the Web UI defines the current version. For example, assume you have two Drill nodes in a cluster with the following IP addresses, versions, and Web UI access:
-| Drill Node | Drill Version | Web Console |
+| Drill Node | Drill Version | Web UI |
|--------------|---------------|---------------------------|
| 10.10.123.88 | 1.9.0 | http:// 10.10.123.88:8047 |
| 10.10.136.25 | 1.10.0 | http://10.10.136.25:8047 |
-Accessing the Web Console for Drill node 10.10.123.88 displays Drill version 1.9.0 as the current version with a green label, while also displaying the Drill version for Drill node 10.10.136.25, but with a red label. Accessing the Web Console for Drill node 10.10.136.25 displays 1.10.0 as the current version with a green label, while also displaying the Drill version for Drill node 10.10.123.88, but with a red label. In both cases, the Web Console generates a warning to state that the Dr [...]
+Accessing the Web UI for Drill node 10.10.123.88 displays Drill version 1.9.0 as the current version with a green label, while also displaying the Drill version for Drill node 10.10.136.25, but with a red label. Accessing the Web UI for Drill node 10.10.136.25 displays 1.10.0 as the current version with a green label, while also displaying the Drill version for Drill node 10.10.123.88, but with a red label. In both cases, the Web UI generates a warning to state that the Drill versions do [...]
-The Web Console sorts the Drill nodes by version, starting with the current Drill node, followed by Drill nodes with Drill versions that match the current version, followed by Drill nodes that do not match the current version. Drill nodes marked as having an “undefined” version may be incorrectly defined or have a pre-1.10.0 version of Drill installed.
+The Web UI sorts the Drill nodes by version, starting with the current Drill node, followed by Drill nodes with Drill versions that match the current version, followed by Drill nodes that do not match the current version. Drill nodes marked as having an “undefined” version may be incorrectly defined or have a pre-1.10.0 version of Drill installed.
diff --git a/_docs/log-and-debug/001-log-and-debug-introduction.md b/_docs/log-and-debug/001-log-and-debug-introduction.md
index 321e466..5ff356f 100644
--- a/_docs/log-and-debug/001-log-and-debug-introduction.md
+++ b/_docs/log-and-debug/001-log-and-debug-introduction.md
@@ -1,6 +1,6 @@
---
title: "Log and Debug Introduction"
-date: 2016-06-29 01:29:06 UTC
+date: 2018-12-08
parent: "Log and Debug"
---
@@ -13,15 +13,15 @@ Drill outputs two standard log files:
* `drillbit.out`
* `drill.log`
-Drill provides a special file, `drillbit_queries.json`, on each Drill node. This log provides the QueryID and profile for every query run on a Drillbit. The Profile view in the Drill Web Console lists the last one-hundred queries that Drill ran. To see information for queries beyond the last one-hundred, you can view the `drillbit_queries.json` file on each Drill node.
+Drill provides a special file, `drillbit_queries.json`, on each Drill node. This log provides the QueryID and profile for every query run on a Drillbit. The Profile view in the Drill Web UI lists the last one-hundred queries that Drill ran. To see information for queries beyond the last one-hundred, you can view the `drillbit_queries.json` file on each Drill node.
Drill provides [audit logging]({{site.baseurl}}/docs/query-audit-logging/) of queries executed by various drillbits in the cluster.
-As of Drill 1.7, you can access the log files generated by the Drillbit process running on a particular node from the Logs page in the Drill Web Console. Currently, you can only see the logs for the Drillbit process running on the node used to access the Web Console.
+As of Drill 1.7, you can access the log files generated by the Drillbit process running on a particular node from the Logs page in the Drill Web UI. Currently, you can only see the logs for the Drillbit process running on the node used to access the Web UI.
-In a web browser, navigate to the Drill Web Console at `http://<host_ip_address>:8047` and click on **Logs** to access the Logs page. The Logs page provides a list of links that correlate with the log files stored in `$DRILL_HOME/logs`.
+In a web browser, navigate to the Drill Web UI at `http://<host_ip_address>:8047` and click on **Logs** to access the Logs page. The Logs page provides a list of links that correlate with the log files stored in `$DRILL_HOME/logs`.
-
+
diff --git a/_docs/log-and-debug/002-error-messages.md b/_docs/log-and-debug/002-error-messages.md
index 73b0a9b..3f92d3c 100644
--- a/_docs/log-and-debug/002-error-messages.md
+++ b/_docs/log-and-debug/002-error-messages.md
@@ -1,6 +1,6 @@
---
title: "Error Messages"
-date: 2018-11-02
+date: 2018-12-08
parent: "Log and Debug"
---
@@ -21,6 +21,6 @@ The following table provides descriptions for the IDs included in a thread:
| ID Type | Description |
|-----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| QueryID | The identifier assigned to the query. You can locate a query in Drill Web Console by the QueryID and then cancel the query if needed. See [Query Profiles]({{ site.baseurl }}/docs/query-profiles/). |
-| MajorFragmentID | The identifier assigned to a major fragment. Major fragments map to the physical plan. You can see major fragment activity for a query in the Drill Web Console. See [Query Profiles]({{ site.baseurl }}/docs/query-profiles) for more information. |
+| QueryID | The identifier assigned to the query. You can locate a query in Drill Web UI by the QueryID and then cancel the query if needed. See [Query Profiles]({{ site.baseurl }}/docs/query-profiles/). |
+| MajorFragmentID | The identifier assigned to a major fragment. Major fragments map to the physical plan. You can see major fragment activity for a query in the Drill Web UI. See [Query Profiles]({{ site.baseurl }}/docs/query-profiles) for more information. |
| MinorFragmentID | The identifier assigned to the minor fragment. Minor fragments map to the parallelization of major fragments. See [Query Profiles]({{ site.baseurl }}/docs/query-profiles). |
diff --git a/_docs/odbc-jdbc-interfaces/010-interfaces-introduction.md b/_docs/odbc-jdbc-interfaces/010-interfaces-introduction.md
index 9214961..9d741e4 100644
--- a/_docs/odbc-jdbc-interfaces/010-interfaces-introduction.md
+++ b/_docs/odbc-jdbc-interfaces/010-interfaces-introduction.md
@@ -1,12 +1,12 @@
---
title: "Interfaces Introduction"
-date: 2018-10-30 21:47:12 UTC
+date: 2018-12-08
parent: "ODBC/JDBC Interfaces"
---
You can connect to Apache Drill through the following interfaces:
* Drill shell
- * Drill Web Console
+ * Drill Web UI
* [ODBC]({{ site.baseurl }}/docs/installing-the-odbc-driver/)*
* [JDBC]({{ site.baseurl }}/docs/using-jdbc-with-squirrel-on-windows/)
* C++ API
diff --git a/_docs/odbc-jdbc-interfaces/using-drill-with-bi-tools/020-tableau-examples.md b/_docs/odbc-jdbc-interfaces/using-drill-with-bi-tools/020-tableau-examples.md
index 77b203c..2d3022d 100644
--- a/_docs/odbc-jdbc-interfaces/using-drill-with-bi-tools/020-tableau-examples.md
+++ b/_docs/odbc-jdbc-interfaces/using-drill-with-bi-tools/020-tableau-examples.md
@@ -1,6 +1,6 @@
---
title: "Tableau Examples"
-date: 2018-11-02
+date: 2018-12-08
parent: "Using Drill with BI Tools"
---
You can generate reports in Tableau using ODBC connections to Drill data
@@ -15,7 +15,7 @@ This section includes the following examples:
The steps and results of these examples assume pre-configured schemas and
source data. You define schemas by configuring storage plugins on the Storage
-tab of the [Drill Web Console]({{ site.baseurl }}/docs/getting-to-know-the-drill-sandbox/#storage-plugin-overview). Also, the examples assume you [enabled the DECIMAL data type]({{site.baseurl}}/docs/supported-data-types/#enabling-the-decimal-type) in Drill.
+tab of the [Drill Web UI]({{ site.baseurl }}/docs/getting-to-know-the-drill-sandbox/#storage-plugin-overview). Also, the examples assume you [enabled the DECIMAL data type]({{site.baseurl}}/docs/supported-data-types/#enabling-the-decimal-type) in Drill.
## Example: Connect to a Hive Table in Tableau
diff --git a/_docs/performance-tuning/024-aynchronous-parquet-reader.md b/_docs/performance-tuning/024-aynchronous-parquet-reader.md
index 9b6698d..58985bf 100644
--- a/_docs/performance-tuning/024-aynchronous-parquet-reader.md
+++ b/_docs/performance-tuning/024-aynchronous-parquet-reader.md
@@ -1,6 +1,6 @@
---
title: "Asynchronous Parquet Reader"
-date: 2016-11-21 21:25:59 UTC
+date: 2018-12-08
parent: "Performance Tuning"
---
@@ -13,7 +13,7 @@ Typically, the Drill default settings provide the best performance for a wide va
##Tuning the Parquet Scan Operator
The `store.parquet.reader.pagereader.async` option turns the asynchronous Parquet reader on or off. The option is turned on by default. You can use the [ALTER SESSION SET command]({{site.baseurl}}/docs/alter-session-command/) to enable the asynchronous Parquet reader option, as well as the options that control buffering and parallel decoding.
-When the asynchronous Page reader option is enabled, the Parquet Scan operator no longer reports operator wait time. Instead, it reports additional operator metrics that you can view in the query profile in the Drill Web Console.
+When the asynchronous Page reader option is enabled, the Parquet Scan operator no longer reports operator wait time. Instead, it reports additional operator metrics that you can view in the query profile in the Drill Web UI.
The `drill.exec.scan.threadpool_size` and `drill.exec.scan.decode_threadpool_size` parameters in the `drill-override.conf` file control the size of the threadpools that read and decode Parquet data when the asynchronous Parquet reader is enabled.
@@ -44,7 +44,7 @@ The following table lists and describes the drillbit configuration parameters in
| drill.exec.scan.decode_threadpool_size | The size of the thread pool used for decoding Parquet data. | (number of cores+1)/2 |
###Operator Metrics
-When the asynchronous Parquet reader option is enabled, Drill provides the following additional operator metrics, which you can access in the query profile from the Drill Web Console:
+When the asynchronous Parquet reader option is enabled, Drill provides the following additional operator metrics, which you can access in the query profile from the Drill Web UI:
**Note:** Time is measured in nanoseconds.
diff --git a/_docs/performance-tuning/027-hive-metadata-caching.md b/_docs/performance-tuning/027-hive-metadata-caching.md
index fd9d6c2..5b07dc5 100644
--- a/_docs/performance-tuning/027-hive-metadata-caching.md
+++ b/_docs/performance-tuning/027-hive-metadata-caching.md
@@ -1,6 +1,6 @@
---
title: "Hive Metadata Caching"
-date: 2016-02-02 23:56:57 UTC
+date: 2018-12-08
parent: "Performance Tuning"
---
@@ -22,7 +22,7 @@ As of Drill 1.5, you can modify the Hive storage plugin to change the rate at wh
To configure the Hive metastore client cache in Drill, complete the following steps:
-1. Start the [Drill Web Console]({{site.baseurl}}/docs/starting-the-web-console/).
+1. Start the [Drill Web UI]({{site.baseurl}}/docs/starting-the-web-console/).
2. Select the **Storage** tab.
3. Click **Update** next to the “hive” storage plugin.
4. Add the following parameters:
diff --git a/_docs/performance-tuning/identifying-performance-issues/010-query-plans.md b/_docs/performance-tuning/identifying-performance-issues/010-query-plans.md
index 9efb9a3..09aa943 100644
--- a/_docs/performance-tuning/identifying-performance-issues/010-query-plans.md
+++ b/_docs/performance-tuning/identifying-performance-issues/010-query-plans.md
@@ -1,6 +1,6 @@
---
title: "Query Plans"
-date: 2016-11-21 22:28:40 UTC
+date: 2018-12-08
parent: "Identifying Performance Issues"
---
If you experience performance issues in Drill, you can typically identify the source of the issues in the query plans or profiles. This section describes the logical plan and physical plans.
@@ -9,7 +9,7 @@ If you experience performance issues in Drill, you can typically identify the so
Drill has an optimizer and a parallelizer that work together to plan a query. Drill creates logical, physical, and execution plans based on the available statistics for an associated set of files or data sources. The number of running Drill nodes and configured runtime settings contribute to how Drill plans and executes a query.
-You can use [EXPLAIN commands]({{ site.baseurl }}/docs/explain-commands/) to view the logical and physical plans for a query, however you cannot view the execution plan. To see how Drill executed a query, you can view the query profile in the Drill Web Console at `<drill_node_ip_address>:8047`.
+You can use [EXPLAIN commands]({{ site.baseurl }}/docs/explain-commands/) to view the logical and physical plans for a query, however you cannot view the execution plan. To see how Drill executed a query, you can view the query profile in the Drill Web UI at `<drill_node_ip_address>:8047`.
### Logical Plan
@@ -27,7 +27,7 @@ Drill transforms the physical plan into an execution tree of minor fragments tha
You can run the EXPLAIN command to view the physical plan for a query with or without costing formation. See EXPLAIN for Physical Plans and Costing Information. Analyze the cost-based query plan to identify the types of operators that Drill plans to use for the query and how much memory they will require.
-Read the text output from bottom to top to understand the sequence of operators planned to execute the query. You can also view a visual representation of the physical plan in the Profile view of the Drill Web Console. See Query Profiles. You can modify the detailed JSON output, and submit it back to Drill through the Drill Web Console.
+Read the text output from bottom to top to understand the sequence of operators planned to execute the query. You can also view a visual representation of the physical plan in the Profile view of the Drill Web UI. See Query Profiles. You can modify the detailed JSON output, and submit it back to Drill through the Drill Web UI.
The physical plan shows the major fragments and specific operators with correlating MajorFragmentIDs and OperatorIDs. See Operators. Major fragments are an abstract concept that represent a phase of the query execution. Major fragments do not perform any query tasks.
@@ -59,7 +59,7 @@ If you view the plan with costing information, you can see where the majority of
**Modifying and Submitting a Physical Plan to Drill**
-You can test the performance of a physical plan that Drill generates, modify the plan and then re-submit it to Drill. For example, you can modify the plan to change the join ordering of tables. You can also submit physical plans created outside of Drill through the Drill Web Console.
+You can test the performance of a physical plan that Drill generates, modify the plan and then re-submit it to Drill. For example, you can modify the plan to change the join ordering of tables. You can also submit physical plans created outside of Drill through the Drill Web UI.
**Note:** Only advanced users who know about query planning should modify and re-submit a physical plan.
@@ -67,7 +67,7 @@ To modify and re-submit a physical plan to Drill, complete the following steps:
1. Run EXPLAIN PLAN FOR `<query>` to see the physical plan for your query.
2. Copy the JSON output of the physical plan, and modify as needed.
-3. Navigate to the Drill Web Console at `<drill_node_ip_address>:8047`.
+3. Navigate to the Drill Web UI at `<drill_node_ip_address>:8047`.
4. Select **Query** in the menu bar.
diff --git a/_docs/performance-tuning/identifying-performance-issues/020-query-profiles.md b/_docs/performance-tuning/identifying-performance-issues/020-query-profiles.md
index 326746e..53da190 100644
--- a/_docs/performance-tuning/identifying-performance-issues/020-query-profiles.md
+++ b/_docs/performance-tuning/identifying-performance-issues/020-query-profiles.md
@@ -1,13 +1,13 @@
---
title: "Query Profiles"
-date: 2017-08-08 02:33:05 UTC
+date: 2018-12-08
parent: "Identifying Performance Issues"
---
-A profile is a summary of metrics collected for each query that Drill executes. Query profiles provide information that you can use to monitor and analyze query performance. When Drill executes a query, Drill writes the profile of each query to disk, which is either the local filesystem or a distributed file system, such as HDFS. As of Drill 1.11, Drill can [store profiles in memory]({{site.baseurl}}/docs/start-up-options/#configuring-start-up-options) instead of writing them to disk. Yo [...]
+A profile is a summary of metrics collected for each query that Drill executes. Query profiles provide information that you can use to monitor and analyze query performance. When Drill executes a query, Drill writes the profile of each query to disk, which is either the local filesystem or a distributed file system, such as HDFS. As of Drill 1.11, Drill can [store profiles in memory]({{site.baseurl}}/docs/start-up-options/#configuring-start-up-options) instead of writing them to disk. Yo [...]
-##Query Profiles in the Drill Web Console
-The Drill Web Console provides aggregate statistics across profile lists. Profile lists consist of data from major and minor fragments, operators, and input streams. You can use profiles in conjunction with Drill logs for debugging purposes. In addition to viewing query profiles, you can modify, resubmit, or cancel queries from the Drill Web Console.
+##Query Profiles in the Drill Web UI
+The Drill Web UI provides aggregate statistics across profile lists. Profile lists consist of data from major and minor fragments, operators, and input streams. You can use profiles in conjunction with Drill logs for debugging purposes. In addition to viewing query profiles, you can modify, resubmit, or cancel queries from the Drill Web UI.
###Query, Fragment, and Operator Identifiers
@@ -17,7 +17,7 @@ Metrics in a query profile are associated with a coordinate system of identifier
### Viewing a Query Profile
-You can view query profiles in the Profiles tab of the Drill Web Console. When you select the Profiles tab, you see a list of the last 100 queries than ran or are currently running in the cluster.
+You can view query profiles in the Profiles tab of the Drill Web UI. When you select the Profiles tab, you see a list of the last 100 queries than ran or are currently running in the cluster.
@@ -109,12 +109,12 @@ The Physical Plan view provides statistics about the actual cost of the query op
### Canceling a Query
-You may want to cancel a query if it hangs or causes performance bottlenecks. You can cancel a query from the Profile tab of the Drill Web Console.
+You may want to cancel a query if it hangs or causes performance bottlenecks. You can cancel a query from the Profile tab of the Drill Web UI.
To cancel a query, complete the following steps:
-1. Navigate to the Drill Web Console at `<drill_node_ip_address>:8047`.
-The Drill node from which you access the Drill Web Console must have an active Drillbit running.
+1. Navigate to the Drill Web UI at `<drill_node_ip_address>:8047`.
+The Drill node from which you access the Drill Web UI must have an active Drillbit running.
2. Select **Profiles** in the toolbar.
A list of running and completed queries appears.
3. Click the query for which you want to see the profile.
diff --git a/_docs/performance-tuning/query-plans-and-tuning/065-throttling.md b/_docs/performance-tuning/query-plans-and-tuning/065-throttling.md
index 25048c5..3ff6b3e 100644
--- a/_docs/performance-tuning/query-plans-and-tuning/065-throttling.md
+++ b/_docs/performance-tuning/query-plans-and-tuning/065-throttling.md
@@ -1,6 +1,6 @@
---
title: "Throttling"
-date: 2017-11-14 21:36:24 UTC
+date: 2018-12-08
parent: "Query Plans and Tuning"
---
@@ -11,7 +11,7 @@ If throttling is disabled, you most likely need to increase the amount of memory
## Configuring Throttling
-You can enable and configure query queuing using system options that you set in the Drill Web Console or through SQL statements. The examples in this document use SQL statements.
+You can enable and configure query queuing using system options that you set in the Drill Web UI or through SQL statements. The examples in this document use SQL statements.
**Note:** The memory for the small and large query queues is calculated as:
@@ -26,7 +26,7 @@ You can turn throttling on and off using the exec.queue.enable option, as shown:
ALTER SYSTEM SET `exec.queue.enable` = true
-The main page of the Drill Web Console shows the current queue configuration when throttling is enabled.
+The main page of the Drill Web UI shows the current queue configuration when throttling is enabled.
@@ -60,7 +60,7 @@ The queue threshold tells Drill how to sort queries into the small and large que
The value that you set represents the total query cost, as computed by the planner. The value does not directly map to physical numbers, however it is derived from row count, CPU, network cost, and so on.
-Before you set the value, experiment by running queries that you consider small and large, then look at the total cost per query in the Drill Web Console, and pick a number in between when setting the threshold option.
+Before you set the value, experiment by running queries that you consider small and large, then look at the total cost per query in the Drill Web UI, and pick a number in between when setting the threshold option.
diff --git a/_docs/query-data/010-query-data-introduction.md b/_docs/query-data/010-query-data-introduction.md
index 672e424..dbe81d5 100644
--- a/_docs/query-data/010-query-data-introduction.md
+++ b/_docs/query-data/010-query-data-introduction.md
@@ -1,6 +1,6 @@
---
title: "Query Data Introduction"
-date: 2018-03-21 01:38:37 UTC
+date: 2018-12-08
parent: "Query Data"
---
You can query local and distributed file systems, Hive, HBase data, complex data, INFORMATION SCHEMA, and system tables as described in the subtopics of this section.
@@ -55,7 +55,7 @@ In addition to syntax highlighting, an autocomplete feature enables you to use s
**Using the Autocomplete Feature**
-On the Query page in the Drill Web Console, place your cursor in the Query window and press `ctrl+space`. A drop-down menu of Drill keywords, functions, and templates appears. Use the up and down arrows on the keyboard to scroll through the list.
+On the Query page in the Drill Web UI, place your cursor in the Query window and press `ctrl+space`. A drop-down menu of Drill keywords, functions, and templates appears. Use the up and down arrows on the keyboard to scroll through the list.
**Note:** The `s*` option provides the template for a SELECT query.
diff --git a/_docs/query-data/070-query-sys-tbl.md b/_docs/query-data/070-query-sys-tbl.md
index aaab925..19d0f31 100644
--- a/_docs/query-data/070-query-sys-tbl.md
+++ b/_docs/query-data/070-query-sys-tbl.md
@@ -1,19 +1,16 @@
---
title: "Querying System Tables"
-date: 2018-11-02
+date: 2018-12-08
parent: "Query Data"
----
-Drill has a sys database that contains system tables. You can query the system
-tables for information about Drill, including Drill ports, the Drill version
-running on the system, and available Drill options. View the databases in
-Drill to identify the sys database, and then use the sys database to view
-system tables that you can query.
+---
+
+Drill has a sys database that contains system tables. You can query the system tables for information about Drill, such as available configuration options and the Drill version running on the system. Starting in Drill 1.15, you can query the functions table expose the available built-in SQL functions and user-defined functions.
-## View Drill Databases
+## Viewing Drill Databases
-Issue the `SHOW DATABASES` command to view Drill databases.
+When you run the `SHOW DATABASES` command, Drill returns the list of available databases/schemas that you can query. In the following example, you can see that Drill returns the sys database/schema:
- 0: jdbc:drill:zk=10.10.100.113:5181> show databases;
+ SHOW DATABASES;
+--------------------+
| SCHEMA_NAME |
+--------------------+
@@ -28,17 +25,15 @@ Issue the `SHOW DATABASES` command to view Drill databases.
| cp.default |
| hbase |
| INFORMATION_SCHEMA |
- +--------------------+
- 11 rows selected (0.162 seconds)
+ +--------------------+
-Drill returns `sys` in the database results.
-## Use the Sys Database
+## Switching to the System Database/Schema
-Issue the `USE` command to select the sys database for subsequent SQL
-requests.
+The USE command changes the schema context to the specified schema. When you switch to the sys schema, as shown, Drill issues all subsequent queries against that schema only:
+
- 0: jdbc:drill:zk=10.10.100.113:5181> use sys;
+ USE sys;
+-------+----------------------------------+
| ok | summary |
+-------+----------------------------------+
@@ -46,31 +41,37 @@ requests.
+-------+----------------------------------+
1 row selected (0.101 seconds)
-## View Tables
+## Viewing System Tables
-Issue the `SHOW TABLES` command to view the tables in the sys database.
+When you run the `SHOW TABLES` command, Drill returns the list of system tables that you can query for information about Drill:
- 0: jdbc:drill:zk=10.10.100.113:5181> show tables;
- +---------------+-------------+
- | TABLE_SCHEMA | TABLE_NAME |
- +---------------+-------------+
- | sys | boot |
- | sys | drillbits |
- | sys | memory |
- | sys | options |
- | sys | threads |
- | sys | version |
- +---------------+-------------+
- 3 rows selected (0.934 seconds)
- 0: jdbc:drill:zk=10.10.100.113:5181>
+ SHOW TABLES;
+ +---------------+-----------------------+
+ | TABLE_SCHEMA | TABLE_NAME |
+ +---------------+-----------------------+
+ | sys | memory |
+ | sys | functions |
+ | sys | internal_options_old |
+ | sys | profiles |
+ | sys | internal_options |
+ | sys | threads |
+ | sys | version |
+ | sys | options_old |
+ | sys | profiles_json |
+ | sys | options |
+ | sys | drillbits |
+ | sys | boot |
+ | sys | connections |
+ +---------------+-----------------------+
-## Query System Tables
-Query the drillbits, version, options, boot, threads, and memory tables in the sys database.
+## Querying System Tables
-###Query the drillbits table.
+The following sections show examples of queries against the system tables available in the sys database/schema.
- 0: jdbc:drill:zk=10.10.100.113:5181> select * from drillbits;
+###Querying the drillbits table
+
+ SELECT * FROM drillbits;
+-------------------+------------+--------------+------------+---------+
| hostname | user_port | control_port | data_port | current|
+-------------------+------------+--------------+------------+--------+
@@ -78,53 +79,52 @@ Query the drillbits, version, options, boot, threads, and memory tables in the s
| qa-node114.qa.lab | 31010 | 31011 | 31012 | false |
| qa-node116.qa.lab | 31010 | 31011 | 31012 | false |
+-------------------+------------+--------------+------------+---------+
- 3 rows selected (0.146 seconds)
+
- * hostname
+ * **hostname**
The name of the node running the Drillbit service.
- * user_port
+ * **user_port**
The user port address, used between nodes in a cluster for connecting to
-external clients and for the Drill Web Console.
- * control_port
+external clients and for the Drill Web UI.
+ * **control_port**
The control port address, used between nodes for multi-node installation of
Apache Drill.
- * data_port
+ * **data_port**
The data port address, used between nodes for multi-node installation of
Apache Drill.
- * current
+ * **current**
True means the Drillbit is connected to the session or client running the
query. This Drillbit is the Foreman for the current session.
-### Query the version table.
+### Querying the version table
- 0: jdbc:drill:zk=10.10.100.113:5181> select * from version;
+ SELECT * FROM version;
+-------------------------------------------+--------------------------------------------------------------------+----------------------------+--------------+----------------------------+
| commit_id | commit_message | commit_time | build_email | build_time |
+-------------------------------------------+--------------------------------------------------------------------+----------------------------+--------------+----------------------------+
| d8b19759657698581cc0d01d7038797952888123 | DRILL-3100: TestImpersonationDisabledWithMiniDFS fails on Windows | 15.05.2015 @ 05:18:03 UTC | Unknown | 15.05.2015 @ 06:52:32 UTC |
+-------------------------------------------+--------------------------------------------------------------------+----------------------------+--------------+----------------------------+
- 1 row selected (0.099 seconds)
- * commit_id
-The github id of the release you are running. For example, <https://github.com
-/apache/drill/commit/e3ab2c1760ad34bda80141e2c3108f7eda7c9104>
- * commit_message
+ * **commit_id**
+The github id of the release you are running. For example, `https://github.com
+/apache/drill/commit/e3ab2c1760ad34bda80141e2c3108f7eda7c9104`
+ * **commit_message**
The message explaining the change.
- * commit_time
+ * **commit_time**
The date and time of the change.
- * build_email
+ * **build_email**
The email address of the person who made the change, which is unknown in this
example.
- * build_time
+ * **build_time**
The time that the release was built.
-### Query the options table.
+### Querying the options table
Drill provides system, session, and boot options that you can query.
The following example shows a query on the system options:
- 0: jdbc:drill:zk=10.10.100.113:5181> select * from options where type='SYSTEM' limit 10;
+ SELECT * FROM options WHERE type='SYSTEM' LIMIT 10;
+-------------------------------------------------+----------+---------+----------+-------------+-------------+-----------+------------+
| name | kind | type | status | num_val | string_val | bool_val | float_val |
+-------------------------------------------------+----------+---------+----------+-------------+-------------+-----------+------------+
@@ -141,27 +141,27 @@ The following example shows a query on the system options:
+-------------------------------------------------+----------+---------+----------+-------------+-------------+-----------+------------+
10 rows selected (0.216 seconds)
- * name
+ * **name**
The name of the option.
- * kind
+ * **kind**
The data type of the option value.
- * type
+ * **type**
The type of options in the output: system or session.
- * status
+ * **status**
The status of the option: default or changed.
- * num_val
+ * **num_val**
The default value, which is of the long or int data type; otherwise, null.
- * string_val
+ * **string_val**
The default value, which is a string; otherwise, null.
- * bool_val
+ * **bool_val**
The default value, which is true or false; otherwise, null.
- * float_val
+ * **float_val**
The default value, which is of the double, float, or long double data type;
otherwise, null.
-### Query the boot table.
+### Querying the boot table
- 0: jdbc:drill:zk=10.10.100.113:5181> select * from boot limit 10;
+ SELECT * FROM boot LIMIT 10;
+--------------------------------------+----------+-------+---------+------------+-------------------------+-----------+------------+
| name | kind | type | status | num_val | string_val | bool_val | float_val |
+--------------------------------------+----------+-------+---------+------------+-------------------------+-----------+------------+
@@ -178,27 +178,27 @@ otherwise, null.
+--------------------------------------+----------+-------+---------+------------+-------------------------+-----------+------------+
10 rows selected (0.192 seconds)
- * name
+ * **name**
The name of the boot option.
- * kind
+ * **kind**
The data type of the option value.
- * type
+ * **type**
This is always boot.
- * status
+ * **status**
This is always boot.
- * num_val
+ * **num_val**
The default value, which is of the long or int data type; otherwise, null.
- * string_val
+ * **string_val**
The default value, which is a string; otherwise, null.
- * bool_val
+ * **bool_val**
The default value, which is true or false; otherwise, null.
- * float_val
+ * **float_val**
The default value, which is of the double, float, or long double data type;
otherwise, null.
-### Query the threads table.
+### Querying the threads table
- 0: jdbc:drill:zk=10.10.100.113:5181> select * from threads;
+ SELECT * FROM threads;
+--------------------+------------+----------------+---------------+
| hostname | user_port | total_threads | busy_threads |
+--------------------+------------+----------------+---------------+
@@ -208,43 +208,155 @@ otherwise, null.
+--------------------+------------+----------------+---------------+
3 rows selected (0.618 seconds)
- * hostname
+ * **hostname**
The name of the node running the Drillbit service.
- * user_port
+ * **user_port**
The user port address, used between nodes in a cluster for connecting to
-external clients and for the Drill Web Console.
- * total_threads
+external clients and for the Drill Web UI.
+ * **total_threads**
The peak thread count on the node.
- * busy_threads
+ * **busy_threads**
The current number of live threads (daemon and non-daemon) on the node.
-### Query the memory table.
+### Querying the memory table
- 0: jdbc:drill:zk=10.10.100.113:5181> select * from memory;
+ SELECT * FROM memory;
+--------------------+------------+---------------+-------------+-----------------+---------------------+-------------+
| hostname | user_port | heap_current | heap_max | direct_current | jvm_direct_current | direct_max |
+--------------------+------------+---------------+-------------+-----------------+---------------------+-------------+
| qa-node115.qa.lab | 31010 | 443549712 | 4294967296 | 11798941 | 167772974 | 8589934592 |
| qa-node114.qa.lab | 31010 | 149948432 | 4294967296 | 7750365 | 134218542 | 8589934592 |
| qa-node116.qa.lab | 31010 | 358612992 | 4294967296 | 7750365 | 83886894 | 8589934592 |
- +--------------------+------------+---------------+-------------+-----------------+---------------------+-------------+
- 3 rows selected (0.172 seconds)
+ +--------------------+------------+---------------+-------------+-----------------+---------------------+-------------+
+
- * hostname
+ * **hostname**
The name of the node running the Drillbit service.
- * user_port
+ * **user_port**
The user port address, used between nodes in a cluster for connecting to
-external clients and for the Drill Web Console.
- * heap_current
+external clients and for the Drill Web UI.
+ * **heap_current**
The amount of memory being used on the heap, in bytes.
- * heap_max
+ * **heap_max**
The maximum amount of memory available on the heap, in bytes.
- * direct_current
+ * **direct_current**
The current direct memory being used by the allocator, in bytes.
- * jvm_direct_current
+ * **jvm_direct_current**
The current JVM direct memory allocation, in bytes.
- * direct_max
-The maximum direct memory available to the allocator, in bytes.
+ * **direct_max**
+The maximum direct memory available to the allocator, in bytes.
+
+
+### Querying the connections table
+
+ SELECT * FROM connections limit 3;
+ +------------+--------------+------------+--------------------------+------------------------+----------+------------------+--------------+-----------+---------------------------------------+
+ | user | client | drillbit | established | duration | queries | isAuthenticated | isEncrypted | usingSSL | session |
+ +------------+--------------+------------+--------------------------+------------------------+----------+------------------+--------------+-----------+---------------------------------------+
+ | anonymous | 10.10.10.10 | test.lab | 2018-12-07 15:40:50.857 | 1 hr 32 min 7.670 sec | 20 | false | false | false | 484c7c8c-2ecd-4f1e-b42d-e3c8750b4ce5 |
+ +------------+--------------+------------+--------------------------+------------------------+----------+------------------+--------------+-----------+---------------------------------------+
+
+The connections table provides the following information about the connection to the Drillbits, including:
+
+
+- name of the user connecting to the Drillbit from the client
+- ip address of the client
+- name of the Drillbit to which the client has connected
+- time at which the connection was established
+- duration of the connection
+- number of queries run during the connection
+- security for the connection
+- session ID
+
+### Querying the profiles_json table
+
+ SELECT * FROM profiles_json LIMIT 1;
+ +---------------------------------------+----------------------------------------------------------------------------------+
+ | queryId | json |
+ +---------------------------------------+----------------------------------------------------------------------------------+
+ | 23f4e2a7-5d60-a766-0b03-d7a03e99033e | {"id":{"part1":2590944894098909030,"part2":793715042592293694},"type":1,"start":1544232280280,"end":1544232280309,"query":"SELECT * FROM profiles_json LIMIT 0","plan":"00-00 Screen : rowType = RecordType(VARCHAR(65536) queryId, VARCHAR(65536) json): rowcount = 1.0, cumulative cost = {1.1 rows, 2.1 cpu, 1.0 io, 0.0 network, 0.0 memory}, id = 2115\n00-01 Project(queryId=[$0], json=[$1]) : rowType = RecordType(VARCHAR(65536) queryId, VARCH [...]
+ +---------------------------------------+----------------------------------------------------------------------------------+
+
+The profiles_json provides the query profile in JSON format for all queries, by queryID.
+
+### Querying the options table
+
+The options table contains configuration options available in Drill. You can set most of the options at the system or session level. Starting in Drill 1.15, a new options table lists option descriptions, the value that the option is currently set to, and the status of the option. Status shows if the value for the option has been changed from the default. You can query the new and old options table, as shown in the examples below.
+
+The following query selects all of the options related to memory from the new options table:
+
+**Note:** Option names are case-sensitive.
+
+ SELECT * FROM options WHERE name LIKE '%memory%';
+ +----------------------------------------------------------------+---------+-------------------+-------------+----------+--------------+----------------------------------------------------------------------------------+
+ | name | kind | accessibleScopes | val | status | optionScope | description |
+ +----------------------------------------------------------------+---------+-------------------+-------------+----------+--------------+----------------------------------------------------------------------------------+
+ | drill.exec.memory.operator.output_batch_size | BIGINT | SYSTEM | 16777216 | DEFAULT | BOOT | Available as of Drill 1.13. Limits the amount of memory that the Flatten, Merge Join, and External Sort ope... |
+ | drill.exec.memory.operator.output_batch_size_avail_mem_factor | FLOAT | SYSTEM | 0.1 | DEFAULT | BOOT | Based on the available system memory, adjusts the output batch size for buffered operators by the factor set. |
+ | exec.hashagg.use_memory_prediction | BIT | ALL | true | DEFAULT | BOOT | Enables Hash Aggregates to use memory predictions to proactively spill early. Default is true. |
+ | exec.queue.memory_ratio | FLOAT | SYSTEM | 10.0 | DEFAULT | BOOT | |
+ | exec.queue.memory_reserve_ratio | FLOAT | SYSTEM | 0.2 | DEFAULT | BOOT | |
+ | planner.memory.average_field_width | BIGINT | ALL | 8 | DEFAULT | BOOT | Used in estimating memory requirements. |
+ | planner.memory.enable_memory_estimation | BIT | ALL | false | DEFAULT | BOOT | Toggles the state of memory estimation and re-planning of the query. When enabled, Drill conservatively est... |
+ | planner.memory.hash_agg_table_factor | FLOAT | ALL | 1.1 | DEFAULT | BOOT | A heuristic value for influencing the size of the hash aggregation table. |
+ | planner.memory.hash_join_table_factor | FLOAT | ALL | 1.1 | DEFAULT | BOOT | A heuristic value for influencing the size of the hash aggregation table. |
+ | planner.memory.max_query_memory_per_node | BIGINT | ALL | 2147483750 | CHANGED | SYSTEM | Sets the maximum amount of direct memory allocated to the Sort and Hash Aggregate operators during each que... |
+ | planner.memory.min_memory_per_buffered_op | BIGINT | ALL | 41943040 | DEFAULT | BOOT | Minimum memory allocated to each buffered operator instance |
+ | planner.memory.non_blocking_operators_memory | BIGINT | ALL | 64 | DEFAULT | BOOT | Extra query memory per node for non-blocking operators. This option is currently used only for memory estim... |
+ | planner.memory.percent_per_query | FLOAT | ALL | 0.05 | DEFAULT | BOOT | Sets the memory as a percentage of the total direct memory. |
+ | planner.memory_limit | BIGINT | ALL | 268435456 | DEFAULT | BOOT | Defines the maximum amount of direct memory allocated to a query for planning. When multiple queries run co... |
+ +----------------------------------------------------------------+---------+-------------------+-------------+----------+--------------+----------------------------------------------------------------------------------+
+
+
+The following query selects three of the options related to memory from the old options table:
+
+**Note:** Notice that the old options table contains _val columns, which have been removed from the options table introduced in Drill 1.15.
+
+ SELECT * FROM options_old WHERE name LIKE '%memory%' LIMIT 3;
+ +----------------------------------------------------------------+----------+-------------------+--------------+----------+-----------+-------------+-----------+------------+
+ | name | kind | accessibleScopes | optionScope | status | num_val | string_val | bool_val | float_val |
+ +----------------------------------------------------------------+----------+-------------------+--------------+----------+-----------+-------------+-----------+------------+
+ | drill.exec.memory.operator.output_batch_size | LONG | SYSTEM | BOOT | DEFAULT | 16777216 | null | null | null |
+ | drill.exec.memory.operator.output_batch_size_avail_mem_factor | DOUBLE | SYSTEM | BOOT | DEFAULT | null | null | null | 0.1 |
+ | exec.hashagg.use_memory_prediction | BOOLEAN | ALL | BOOT | DEFAULT | null | null | true | null |
+ +----------------------------------------------------------------+----------+-------------------+--------------+----------+-----------+-------------+-----------+------------+
+
+
+### Querying the functions table
+
+The functions table is available starting in Drill 1.15. The functions table exposes the available SQL functions in Drill and also detects UDFs that have been dynamically loaded into Drill. You can query the functions table to see the following information about built-in and user-defined functions in Drill:
+
+ SELECT * FROM functions LIMIT 0;
+ +-------+------------+-------------+---------+-----------+
+ | name | signature | returnType | source | internal |
+ +-------+------------+-------------+---------+-----------+
+ +-------+------------+-------------+---------+-----------+
+
+
+The following query shows the count of built-in functions and UDFs:
+**Note:** In this example, UDFs were dynamically loaded into Drill, as indicated by the JAR files.
+
+ SELECT source, COUNT(*) AS functionCount FROM sys.functions GROUP BY source;
+ +-----------------------------------------+----------------+
+ | source | functionCount |
+ +-----------------------------------------+----------------+
+ | built-in | 2704 |
+ | simple-drill-function-1.0-SNAPSHOT.jar | 12 |
+ | drill-url-tools-1.0.jar | 1 |
+ +-----------------------------------------+----------------+
+
+You can search the functions table to see if Drill supports specific functions. For example, the following query shows all the functions with names that include sql:
+
+ SELECT * FROM functions WHERE name LIKE '%sql%';
+ +-------------------+------------------------------------+-------------+-----------+-----------+
+ | name | signature | returnType | source | internal |
+ +-------------------+------------------------------------+-------------+-----------+-----------+
+ | sqlTypeOf | LATE-REQUIRED | VARCHAR | built-in | false |
+ | sql_to_date | VARCHAR-REQUIRED,VARCHAR-REQUIRED | DATE | built-in | false |
+ | sql_to_time | VARCHAR-REQUIRED,VARCHAR-REQUIRED | TIME | built-in | false |
+ | sql_to_timestamp | VARCHAR-REQUIRED,VARCHAR-REQUIRED | TIMESTAMP | built-in | false |
+ +-------------------+------------------------------------+-------------+-----------+-----------+
+
For information about how to configure Drill system and session options, see [Planning and Execution Options]({{ site.baseurl }}/docs/planning-and-execution-options).
diff --git a/_docs/query-data/080-monitoring-and-canceling-queries-in-the-Drill-Web-UI.md b/_docs/query-data/080-monitoring-and-canceling-queries-in-the-Drill-Web-UI.md
index 9d646f6..07fdf29 100644
--- a/_docs/query-data/080-monitoring-and-canceling-queries-in-the-Drill-Web-UI.md
+++ b/_docs/query-data/080-monitoring-and-canceling-queries-in-the-Drill-Web-UI.md
@@ -1,17 +1,17 @@
---
-title: "Monitoring and Canceling Queries in the Drill Web Console"
-date: 2018-11-02
+title: "Monitoring and Canceling Queries in the Drill Web UI"
+date: 2018-12-08
parent: "Query Data"
---
-You can monitor and cancel queries from the Drill Web Console if you are [authorized]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security/) to do so. To access the Drill
-Web Console, the Drillbit process must be running on the Drill node that you use to
-access the Drill Web Console.
+You can monitor and cancel queries from the Drill Web UI if you are [authorized]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security/) to do so. To access the Drill
+Web UI, the Drillbit process must be running on the Drill node that you use to
+access the Drill Web UI.
-To monitor or cancel a query from the Drill Web Console, complete the following
+To monitor or cancel a query from the Drill Web UI, complete the following
steps:
- 1. Navigate to the Drill Web Console at `<drill_node_ip_address>:8047.`
-When you access the Drill Web Console, you see some general information about Drill
+ 1. Navigate to the Drill Web UI at `<drill_node_ip_address>:8047.`
+When you access the Drill Web UI, you see some general information about Drill
running in your cluster, such as the nodes running the Drillbit process, the
various ports Drill is using, and the amount of direct memory assigned to
Drill.
diff --git a/_docs/query-data/querying-complex-data/005-querying-complex-data-introduction.md b/_docs/query-data/querying-complex-data/005-querying-complex-data-introduction.md
index afd2786..b88a5a5 100644
--- a/_docs/query-data/querying-complex-data/005-querying-complex-data-introduction.md
+++ b/_docs/query-data/querying-complex-data/005-querying-complex-data-introduction.md
@@ -1,6 +1,6 @@
---
title: "Querying Complex Data Introduction"
-date: 2018-02-09 00:16:04 UTC
+date: 2018-12-08
parent: "Querying Complex Data"
---
Apache Drill queries do not require prior knowledge of the actual data you are
@@ -50,7 +50,7 @@ your own queries, you need to be aware of the basic data types in these files:
* boolean values: true, false
Check that you have the following configuration setting for JSON files in the
-Drill Web Console (`dfs` storage plugin configuration):
+Drill Web UI (`dfs` storage plugin configuration):
"json" : {
"type" : "json"
diff --git a/_docs/rn/001-1.14.0-rn.md b/_docs/rn/001-1.14.0-rn.md
index a63d924..94d123e 100644
--- a/_docs/rn/001-1.14.0-rn.md
+++ b/_docs/rn/001-1.14.0-rn.md
@@ -459,7 +459,7 @@ The following sections list all the fixes and improvements in this release:
</li>
<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6503'>DRILL-6503</a>] - Performance improvements in lateral
</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6505'>DRILL-6505</a>] - Drill web console query: support back button or add "edit query"
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6505'>DRILL-6505</a>] - Drill Web UI query: support back button or add "edit query"
</li>
<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6515'>DRILL-6515</a>] - Render a linkage between the Unnest operator and its source operator
</li>
diff --git a/_docs/rn/009-1.12.0-rn.md b/_docs/rn/009-1.12.0-rn.md
index 693bba0..26f131c 100644
--- a/_docs/rn/009-1.12.0-rn.md
+++ b/_docs/rn/009-1.12.0-rn.md
@@ -25,7 +25,7 @@ This release of Drill provides the following new features and improvements:
- System options improvements, including a new internal system options table (DRILL-5723)
- Ability to prevent users from accessing a path outside the current workspace (DRILL-5964)
- Ability to put the server in quiescent mode for a graceful shutdown (DRILL-4286)
-- The Drill Web Console lists the completion of successfully completed queries as "successful" (DRILL-5923)
+- The Drill Web UI lists the completion of successfully completed queries as "successful" (DRILL-5923)
diff --git a/_docs/rn/011-1.10.0-rn.md b/_docs/rn/011-1.10.0-rn.md
index 0f83cf3..b30baf8 100644
--- a/_docs/rn/011-1.10.0-rn.md
+++ b/_docs/rn/011-1.10.0-rn.md
@@ -12,7 +12,7 @@ This release of Drill provides the following new features and improvements:
- Support for the CREATE TEMPORARY TABLE AS (CTTAS) command.
- A JDBC connection option that improves fault tolerance when connecting directly to a Drill node from a client.
-- The Web Console displays the Drill version and additional query profile statistics.
+- The Web UI displays the Drill version and additional query profile statistics.
- Drill implicitly interprets the INT96 timestamp data type in Parquet files.
- Support for Kerberos authentication between the client and drillbit.
diff --git a/_docs/rn/019-1.2.0-rn.md b/_docs/rn/019-1.2.0-rn.md
index 572db45..7793c9d 100644
--- a/_docs/rn/019-1.2.0-rn.md
+++ b/_docs/rn/019-1.2.0-rn.md
@@ -25,7 +25,7 @@ This release of Drill introduces a number of enhancements, including the followi
* NTILE
* LAG and LEAD
* FIRST_VALUE and LAST_VALUE
-* [HTTPS support]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security/) for Web Console operations
+* [HTTPS support]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security/) for Web UI operations
* Performance improvements for [querying HBase]({{site.baseurl}}/docs/querying-hbase/#querying-big-endian-encoded-data), which includes leveraging [ordered byte encoding]({{site.baseurl}}/docs/querying-hbase/#leveraging-hbase-ordered-byte-encoding)
* [Optimized reads]({{site.baseurl}}/docs/querying-hive/#optimizing-reads-of-parquet-backed-tables) of Parquet-backed, Hive tables
* Read support for the [Parquet INT96 type]({{site.baseurl}}/docs/parquet-format/#about-int96-support) and a new TIMESTAMP_IMPALA type used with the [CONVERT_FROM]({{site.baseurl}}/docs/supported-data-types/#data-types-for-convert_to-and-convert_from-functions) function decodes a timestamp from Hive or Impala.
diff --git a/_docs/sql-reference/sql-commands/031-create-temporary-table-as.md b/_docs/sql-reference/sql-commands/031-create-temporary-table-as.md
index 7755730..bd996f0 100644
--- a/_docs/sql-reference/sql-commands/031-create-temporary-table-as.md
+++ b/_docs/sql-reference/sql-commands/031-create-temporary-table-as.md
@@ -1,6 +1,6 @@
---
title: "CREATE TEMPORARY TABLE AS (CTTAS)"
-date: 2017-03-27 18:12:49 UTC
+date: 2018-12-08
parent: "SQL Commands"
---
As of Drill 1.10, you can use the CREATE TEMPORARY TABLE AS (CTTAS) command to store the results of a query in a temporary table. You can reference the temporary table in subsequent queries within the same session, thereby improving query performance. Data written to the temporary table is not permanently stored on the filesystem. Drill automatically drops the temporary table once the session ends or the Drillbit process fails. Therefore, you do not have to manually drop the table.
@@ -46,7 +46,7 @@ Example dfs.tmp workspace in the dfs storage plugin:
} ...
To override the default temporary workspace, define `drill.exec.default_temporary_workspace` in the `drill-override.conf file` and then restart the Drillbit. For more information, see [Configuration Option Introduction]({{site.baseurl}}/docs/configuration-options-introduction/#system-options).
-To change the default temporary workspace connection or directory path, update the dfs storage plugin on the Storage page in the Drill Web Console. For example, dfs connection attribute can be `file:///` or `maprfs:///` and the `dfs.tmp` location attribute can be `/tmp` or `/tmp2`. For more information, see [Plugin Configuration Basics]({{site.baseurl}}/docs/plugin-configuration-basics).
+To change the default temporary workspace connection or directory path, update the dfs storage plugin on the Storage page in the Drill Web UI. For example, dfs connection attribute can be `file:///` or `maprfs:///` and the `dfs.tmp` location attribute can be `/tmp` or `/tmp2`. For more information, see [Plugin Configuration Basics]({{site.baseurl}}/docs/plugin-configuration-basics).
Note: When you run Drill in distributed mode, verify that the default temporary workspace connection points to a distributed filesystem. If temporary tables are created on the local filesystem, they can only be accessed by the local Drillbit that created the temporary table.
###Setting the Storage Format
diff --git a/_docs/sql-reference/sql-commands/053-describe.md b/_docs/sql-reference/sql-commands/053-describe.md
index 4ff935b..86961e5 100644
--- a/_docs/sql-reference/sql-commands/053-describe.md
+++ b/_docs/sql-reference/sql-commands/053-describe.md
@@ -1,6 +1,6 @@
---
title: "DESCRIBE"
-date: 2018-04-19 23:14:01 UTC
+date: 2018-12-08
parent: "SQL Commands"
---
The DESCRIBE command returns information about columns in a table, view, or schema.
@@ -50,7 +50,7 @@ Drill only supports SQL data types. Verify that all data types in an external da
###DESCRIBE SCHEMA
- You can issue the DESCRIBE SCHEMA command on any schema. However, you can only include workspaces for file schemas, such as `dfs.myworkspace`.
-- When you issue the DESCRIBE SCHEMA command on a particular schema, Drill returns all of the schema properties. The schema properties correlate with the configuration information in the Storage tab of the Drill Web Console for that schema.
+- When you issue the DESCRIBE SCHEMA command on a particular schema, Drill returns all of the schema properties. The schema properties correlate with the configuration information in the Storage tab of the Drill Web UI for that schema.
- When you issue DESCRIBE SCHEMA against a schema and workspace, such as `dfs.myworkspace`, Drill returns the workspace properties in addition to all of the schema properties.
- When you issue DESCRIBE SCHEMA against the `dfs` schema, Drill also returns the properties of the “default” workspace. Issuing DESCRIBE SCHEMA against `dfs` or `` dfs.`default` `` returns the same results.
diff --git a/_docs/sql-reference/sql-commands/081-from-clause.md b/_docs/sql-reference/sql-commands/081-from-clause.md
index 0ae17dd..1a7732e 100644
--- a/_docs/sql-reference/sql-commands/081-from-clause.md
+++ b/_docs/sql-reference/sql-commands/081-from-clause.md
@@ -1,6 +1,6 @@
---
title: "FROM Clause"
-date: 2018-12-04
+date: 2018-12-08
parent: "SQL Commands"
---
The FROM clause lists the references (tables, views, and subqueries) that data is selected from. Drill expands the traditional concept of a “table reference” in a standard SQL FROM clause to refer to files and directories in a local or distributed file system.
@@ -158,7 +158,7 @@ The following example joins two tables on the table id:
In the following example, assume you have the following two tables that you want to join using a cross join:
-**Note:** These tables were created from the region.parquet and nation.parquet files from the sample-data folder included with the Drill installation.
+**Note:** These tables were created from the region.parquet and nation.parquet files in the sample-data folder included with the Drill installation.
SELECT * FROM tmp.`n_name`;
+----------+-----------------------+
diff --git a/_docs/sql-reference/sql-commands/082-group-by-clause.md b/_docs/sql-reference/sql-commands/082-group-by-clause.md
index d770d0a..daca485 100644
--- a/_docs/sql-reference/sql-commands/082-group-by-clause.md
+++ b/_docs/sql-reference/sql-commands/082-group-by-clause.md
@@ -1,10 +1,9 @@
---
title: "GROUP BY Clause"
-date: 2018-11-02
+date: 2018-12-08
parent: "SQL Commands"
---
-The GROUP BY clause identifies the grouping columns for the query. You typically use a GROUP BY clause in conjunction with an aggregate expression. Grouping columns must be declared when the query computes aggregates with standard functions such as SUM, AVG, and COUNT. Currently, Drill does not support grouping on aliases.
-
+The GROUP BY clause identifies the grouping columns for the query. You typically use a GROUP BY clause in conjunction with an aggregate expression. Grouping columns must be declared when the query computes aggregates with standard functions such as SUM, AVG, and COUNT. Starting in 1.15, Drill supports aliases in the GROUP BY clause.
## Syntax
The GROUP BY clause supports the following syntax:
@@ -45,8 +44,28 @@ The following query returns sales totals grouped by month:
| February | 532901 |
| September | 373100 |
| January | 346536 |
- +------------+------------+
-
-
+ +------------+------------+
+
+The following query extracts the year from the date\_hired column and groups the years in an aliased column, yrs\_hired:
+
+ USE cp;
+ +-------+---------------------------------+
+ | ok | summary |
+ +-------+---------------------------------+
+ | true | Default schema changed to [cp] |
+ +-------+---------------------------------+
+
+ SELECT EXTRACT(year FROM hire_date) AS yrs_hired FROM `employee.json` GROUP BY yrs_hired;
+ +------------+
+ | yrs_hired |
+ +------------+
+ | 1994 |
+ | 1998 |
+ | 1996 |
+ | 1995 |
+ | 1997 |
+ | 1993 |
+ +------------+
+
diff --git a/_docs/tutorials/learn-drill-with-the-mapr-sandbox/010-installing-the-apache-drill-sandbox.md b/_docs/tutorials/learn-drill-with-the-mapr-sandbox/010-installing-the-apache-drill-sandbox.md
index d029ebc..ec82c92 100644
--- a/_docs/tutorials/learn-drill-with-the-mapr-sandbox/010-installing-the-apache-drill-sandbox.md
+++ b/_docs/tutorials/learn-drill-with-the-mapr-sandbox/010-installing-the-apache-drill-sandbox.md
@@ -1,6 +1,6 @@
---
title: "Installing the Apache Drill Sandbox"
-date: 2018-11-02
+date: 2018-12-08
parent: "Learn Drill with the MapR Sandbox"
---
## Prerequisites
@@ -55,7 +55,7 @@ VMware Player or VMware Fusion:
Open a browser on your host machine and enter the URL in the browser's address field.
You can access the host via SSH by ssh mapr@localhost -p 2222
Log in to this virtual machine: Linux/Windows <Alt+F2>, Mac OS X <Options+F5>
-**Note:** The URL provided corresponds to the Web Console in Apache Drill.
+**Note:** The URL provided corresponds to the Web UI in Apache Drill.
6. Verify that a DNS entry was created on the host machine for the virtual machine. If not, create the entry.
* For Linux and Mac, create the entry in `/etc/hosts`.
@@ -63,7 +63,7 @@ VMware Player or VMware Fusion:
Example: `127.0.1.1 <vm_hostname>`
-7. Navigate to [localhost:8047](http://localhost:8047) to experience the Drill Web Console, or log in to the sandbox through the command line.
+7. Navigate to [localhost:8047](http://localhost:8047) to experience the Drill Web UI, or log in to the sandbox through the command line.
* Login using ssh as described in ["Getting to Know the Sandbox"]({{site.baseurl}}/docs/getting-to-know-the-drill-sandbox). When prompted, enter `mapr` as the login name and password.
* Alternatively, access the command line on the VM: Press Alt+F2 on Windows or Option+F5 on Mac.
@@ -110,9 +110,9 @@ VirtualBox:
You can access the host via SSH by ssh mapr@localhost -p 2222
Log in to this virtual machine: Linux/Windows <Alt+F2>, Mac OS X <Options+F5>
- **Note:** The URL provided corresponds to the Web Console in Apache Drill.
+ **Note:** The URL provided corresponds to the Web UI in Apache Drill.
-10. Navigate to [localhost:8047](http://localhost:8047) to experience the Drill Web Console, or log into the sandbox through the command line.
+10. Navigate to [localhost:8047](http://localhost:8047) to experience the Drill Web UI, or log into the sandbox through the command line.
* Login using ssh as described in ["Getting to Know the Sandbox"]({{site.baseurl}}/docs/getting-to-know-the-drill-sandbox). When prompted, enter `mapr` as the login name and password.
* Alternatively, access the command line on the VM: Press Alt+F2 on Windows or Option+F5 on Mac.
diff --git a/_docs/tutorials/learn-drill-with-the-mapr-sandbox/020-getting-to-know-the-drill-sandbox.md b/_docs/tutorials/learn-drill-with-the-mapr-sandbox/020-getting-to-know-the-drill-sandbox.md
index c2db78b..9fcae54 100644
--- a/_docs/tutorials/learn-drill-with-the-mapr-sandbox/020-getting-to-know-the-drill-sandbox.md
+++ b/_docs/tutorials/learn-drill-with-the-mapr-sandbox/020-getting-to-know-the-drill-sandbox.md
@@ -1,6 +1,6 @@
---
title: "Getting to Know the Drill Sandbox"
-date: 2018-08-20 18:59:28 UTC
+date: 2018-12-08
parent: "Learn Drill with the MapR Sandbox"
---
This section covers key information about the Apache Drill tutorial. After [installing the Drill sandbox]({{ site.baseurl }}/docs/installing-the-apache-drill-sandbox) and starting the sandbox, you can open another terminal window (Linux) or Command Prompt (Windows) and use the secure shell (ssh) to connect to the VM, assuming ssh is installed. Use the following login name and password: mapr/mapr.
@@ -24,7 +24,7 @@ Drill includes a shell for connecting to relational databases and executing SQL
In this tutorial you query a number of data sets, including Hive and HBase, and files on the file system, such as CSV, JSON, and Parquet files. To access these diverse data sources, you connect Drill to storage plugins.
## Storage Plugin Overview
-You use a [storage plugin]({{ site.baseurl }}/docs/connect-a-data-source-introduction) to connect to a data source, such as a file or the Hive metastore. Take a look at the storage plugin definitions by opening the Storage tab in the Drill Web Console. Launch a web browser and go to: `http://<IP address>:8047/storage`.
+You use a [storage plugin]({{ site.baseurl }}/docs/connect-a-data-source-introduction) to connect to a data source, such as a file or the Hive metastore. Take a look at the storage plugin definitions by opening the Storage tab in the Drill Web UI. Launch a web browser and go to: `http://<IP address>:8047/storage`.
The control panel for managing storage plugins appears.
diff --git a/_docs/tutorials/learn-drill-with-the-mapr-sandbox/030-lesson-1-learn-about-the-data-set.md b/_docs/tutorials/learn-drill-with-the-mapr-sandbox/030-lesson-1-learn-about-the-data-set.md
index 43b3418..34c4ef2 100644
--- a/_docs/tutorials/learn-drill-with-the-mapr-sandbox/030-lesson-1-learn-about-the-data-set.md
+++ b/_docs/tutorials/learn-drill-with-the-mapr-sandbox/030-lesson-1-learn-about-the-data-set.md
@@ -1,6 +1,6 @@
---
title: "Lesson 1: Learn about the Data Set"
-date: 2018-02-08 02:32:57 UTC
+date: 2018-12-08
parent: "Learn Drill with the MapR Sandbox"
---
## Goal
@@ -33,7 +33,7 @@ the Drill shell, type:
`0: jdbc:drill:> !quit`
-Examples in this tutorial use the Drill shell. You can also execute queries using the Drill Web Console.
+Examples in this tutorial use the Drill shell. You can also execute queries using the Drill Web UI.
### Enable the DECIMAL Data Type