You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@drill.apache.org by br...@apache.org on 2017/03/15 02:43:28 UTC
[1/7] drill git commit: post content for AD 1.10 release
Repository: drill
Updated Branches:
refs/heads/gh-pages 5c2ee7fa8 -> c75795000
post content for AD 1.10 release
Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/74eabf4d
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/74eabf4d
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/74eabf4d
Branch: refs/heads/gh-pages
Commit: 74eabf4da0c28a94b2356969755963893b896193
Parents: 5c2ee7f
Author: Bridget Bevens <bb...@maprtech.com>
Authored: Mon Mar 13 15:49:06 2017 -0700
Committer: Bridget Bevens <bb...@maprtech.com>
Committed: Mon Mar 13 15:49:06 2017 -0700
----------------------------------------------------------------------
.../040-parquet-format.md | 50 +++++++++++++++----
_docs/developer-information/009-rest-api.md | 6 +--
_docs/img/jdbc_connection_tries.png | Bin 0 -> 7999 bytes
_docs/img/multiple_drill_versions.jpg | Bin 0 -> 81613 bytes
...ying-multiple-drill-versions-in-a-cluster.md | 26 ++++++++++
.../015-using-jdbc-driver.md | 24 +++++++--
.../005-querying-a-file-system-introduction.md | 43 +++++++++++++++-
7 files changed, 130 insertions(+), 19 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/drill/blob/74eabf4d/_docs/data-sources-and-file-formats/040-parquet-format.md
----------------------------------------------------------------------
diff --git a/_docs/data-sources-and-file-formats/040-parquet-format.md b/_docs/data-sources-and-file-formats/040-parquet-format.md
index 56e48e7..0bf6db0 100644
--- a/_docs/data-sources-and-file-formats/040-parquet-format.md
+++ b/_docs/data-sources-and-file-formats/040-parquet-format.md
@@ -1,6 +1,6 @@
---
title: "Parquet Format"
-date:
+date: 2017-03-13 22:49:07 UTC
parent: "Data Sources and File Formats"
---
[Apache Parquet](http://parquet.incubator.apache.org/documentation/latest) has the following characteristics:
@@ -22,9 +22,27 @@ Apache Drill includes the following support for Parquet:
When a read of Parquet data occurs, Drill loads only the necessary columns of data, which reduces I/O. Reading only a small piece of the Parquet data from a data file or table, Drill can examine and analyze all values for a column across multiple files. You can create a Drill table from one format and store the data in another format, including Parquet.
## Writing Parquet Files
-CREATE TABLE AS (CTAS) can use any data source provided by the storage plugin. To write Parquet data using the CTAS command, set the session store.format option as shown in the next section. Alternatively, configure the storage plugin to point to the directory containing the Parquet files.
+CREATE TABLE AS (CTAS) can use any data source provided by the storage plugin. To write Parquet data using the CTAS command, set the `session store.format` option as shown in [Configuring the Parquet Storage Format]({{site.baseurl}}/docs/parquet-format/#configuring-the-parquet-storage-format). Alternatively, configure the storage plugin to point to the directory containing the Parquet files.
-Although the data resides in a single table, Parquet output generally consists of multiple files that resemble MapReduce output having numbered file names, such as 0_0_0.parquet in a directory.
+Although the data resides in a single table, Parquet output generally consists of multiple files that resemble MapReduce output having numbered file names, such as 0_0_0.parquet in a directory.
+
+###Date Value Auto-Correction
+As of Drill 1.10, Drill writes standard Parquet date values. Drill also has an automatic correction feature that automatically detects and corrects corrupted date values that Drill wrote into Parquet files prior to Drill 1.10.
+
+By default, the automatic correction feature is turned on and works for dates up to 5,000 years into the future. In the unlikely event that Drill needs to write dates thousands of years into the future, turn the auto-correction feature off.
+
+To disable the auto-correction feature, navigate to the storage plugin configuration and change the `autoCorrectCorruptDates` option in the Parquet configuration to \u201cfalse\u201d, as shown in the example below:
+
+ "formats": {
+ "parquet": {
+ "type": "parquet",
+ "autoCorrectCorruptDates": false
+ }
+
+Alternatively, you can set the option to false when you issue a query, as shown in the following example:
+
+ SELECT l_shipdate, l_commitdate FROM table(dfs.`/drill/testdata/parquet_date/dates_nodrillversion/drillgen2_lineitem`
+ (type => 'parquet', autoCorrectCorruptDates => false)) LIMIT 1;
### Configuring the Parquet Storage Format
To read or write Parquet data, you need to include the Parquet format in the storage plugin format definitions. The `dfs` plugin definition includes the Parquet format.
@@ -130,20 +148,30 @@ The first table in this section maps SQL data types to Parquet data types, limit
| INTEGER | INT32 | 4-byte signed integer |
| VARBINARY(12)* | INT96 | 12-byte signed int |
-\* Drill 1.2 and later supports reading the Parquet INT96 type.
+\* Drill 1.10 and later can implicitly interpret the Parquet INT96 type as TIMESTAMP (with standard 8 byte/millisecond precision) when the `store.parquet.int96_as_timestamp` option is enabled. In earlier versions of Drill (1.2 through 1.9) or when the `store.parquet.int96_as_timestamp` option is disabled, you must use the CONVERT_FROM function for Drill to correctly interpret INT96 values as TIMESTAMP values.
+
+## About INT96 Support
+As of Drill 1.10, Drill can implicitly interpret the INT96 timestamp data type in Parquet files when the `store.parquet.int96_as_timestamp` option is enabled. For earlier versions of Drill, or when the `store.parquet.int96_as_timestamp` option is disabled, you must use the CONVERT_FROM function,
+
+The `store.parquet.int96_as_timestamp` option is disabled by default. Use the [ALTER SYSTEM|SESSION SET]({{site.baseurl}}/docs/alter-system/) command to enable the option. Unnecessarily enabling this option can cause queries to fail because the CONVERT_FROM(col, 'TIMESTAMP_IMPALA') function does not work when `store.parquet.int96_as_timestamp` is enabled.
+
+###Using CONVERT_FROM to Interpret INT96
+In earlier versions of Drill (1.2 through 1.9), you must use the CONVERT_FROM function for Drill to interpret the Parquet INT96 type. For example, to decode a timestamp from Hive or Impala, which is of type INT96, use the CONVERT_FROM function and the [TIMESTAMP_IMPALA]({{site.baseurl}}/docs/supported-data-types/#data-types-for-convert_to-and-convert_from-functions) type argument:
+
+``SELECT CONVERT_FROM(timestamp_field, 'TIMESTAMP_IMPALA') as timestamp_field FROM `dfs.file_with_timestamp.parquet`;``
+
+Because INT96 is supported for reads only, you cannot use the TIMESTAMP_IMPALA as a data type argument with CONVERT_TO. You can convert a SQL TIMESTAMP to VARBINARY using the CAST function, but the resultant VARBINARY is not the same as INT96.
-## About INT96 Support
-Drill 1.2 and later supports reading the Parquet INT96 type. For example, to decode a timestamp from Hive or Impala, which is of type INT96, use the CONVERT_FROM function and the [TIMESTAMP_IMPALA]({{site.baseurl}}/docs/supported-data-types/#data-types-for-convert_to-and-convert_from-functions) type argument:
+For example, create a Drill table after reading INT96 and converting some data to a timestamp.
-``SELECT CONVERT_FROM(timestamp_field, 'TIMESTAMP_IMPALA') as timestamp_field FROM `dfs.file_with_timestamp.parquet`;``
-Because INT96 is supported for reads only, you cannot use the TIMESTAMP_IMPALA as a data type argument with CONVERT_TO.
+ CREATE TABLE t2(c1) AS SELECT CONVERT_FROM(created_ts, 'TIMESTAMP_IMPALA') FROM t1 ORDER BY 1 LIMIT 1;
-You can convert a SQL TIMESTAMP to VARBINARY using the CAST function, but the resultant VARBINARY is not the same as the INT96. For example, create a Drill table after reading an INT96 and converting some data to a timestamp.
+t1.created_ts is an INT96 (or Hive/Impala timestamp) , t2.created_ts is a SQL timestamp. These types are not comparable. You cannot use a condition like t1.created_ts = t2.created_ts.
-`CREATE TABLE t2(c1) AS SELECT CONVERT_FROM(created_ts, 'TIMESTAMP_IMPALA') FROM t1 ORDER BY 1 LIMIT 1;`
+###Configuring the Timezone
+By default, INT96 timestamp values represent the local date and time, which is similar to Hive. To get INT96 timestamp values in UTC, configure Drill for [UTC time]({{site.baseurl}}/docs/data-type-conversion/#time-zone-limitation).
-t1.created_ts is an INT96 (or Hive/Impala timestamp) , t2.created_ts is a SQL timestamp. These types are not comparable--you cannot use a condition like t1.created_ts = t2.created_ts.
### SQL Types to Parquet Logical Types
Parquet also supports logical types, fully described on the [Apache Parquet site](https://github.com/Parquet/parquet-format/blob/master/LogicalTypes.md). Embedded types, JSON and BSON, annotate a binary primitive type representing a JSON or BSON document. The logical types and their mapping to SQL types are:
http://git-wip-us.apache.org/repos/asf/drill/blob/74eabf4d/_docs/developer-information/009-rest-api.md
----------------------------------------------------------------------
diff --git a/_docs/developer-information/009-rest-api.md b/_docs/developer-information/009-rest-api.md
index ce9e03e..86b37df 100644
--- a/_docs/developer-information/009-rest-api.md
+++ b/_docs/developer-information/009-rest-api.md
@@ -1,6 +1,6 @@
---
title: "REST API"
-date: 2016-11-21 22:14:41 UTC
+date: 2017-03-13 22:49:08 UTC
parent: "Developer Information"
---
@@ -306,13 +306,13 @@ Gets metric information.
----------
-### GET /stats.json
+### GET /cluster.json
Get Drillbit information, such as ports numbers.
**Example**
-`curl http://localhost:8047/stats.json`
+`curl http://localhost:8047/cluster.json`
**Response Body**
http://git-wip-us.apache.org/repos/asf/drill/blob/74eabf4d/_docs/img/jdbc_connection_tries.png
----------------------------------------------------------------------
diff --git a/_docs/img/jdbc_connection_tries.png b/_docs/img/jdbc_connection_tries.png
new file mode 100644
index 0000000..5d5fef8
Binary files /dev/null and b/_docs/img/jdbc_connection_tries.png differ
http://git-wip-us.apache.org/repos/asf/drill/blob/74eabf4d/_docs/img/multiple_drill_versions.jpg
----------------------------------------------------------------------
diff --git a/_docs/img/multiple_drill_versions.jpg b/_docs/img/multiple_drill_versions.jpg
new file mode 100644
index 0000000..8db9234
Binary files /dev/null and b/_docs/img/multiple_drill_versions.jpg differ
http://git-wip-us.apache.org/repos/asf/drill/blob/74eabf4d/_docs/install/070-identifying-multiple-drill-versions-in-a-cluster.md
----------------------------------------------------------------------
diff --git a/_docs/install/070-identifying-multiple-drill-versions-in-a-cluster.md b/_docs/install/070-identifying-multiple-drill-versions-in-a-cluster.md
new file mode 100644
index 0000000..e5971bb
--- /dev/null
+++ b/_docs/install/070-identifying-multiple-drill-versions-in-a-cluster.md
@@ -0,0 +1,26 @@
+---
+title: Identifying Multiple Drill Versions in a Cluster
+date:
+parent: Install Drill
+---
+
+As of Drill 1.10, the Web Console displays the Drill version running on each Drill node in the cluster, as shown in the following image:
+
+
+
+You can also retrieve the version information by running the following query:
+
+ SELECT * FROM sys.drillbits;
+
+If the version of Drill differs between nodes, a warning message appears. The nodes running the current version have a green label, while the nodes running another version have a red label, as shown in the image above.
+
+The Drill node from which you access the Web Console defines the current version. For example, assume you have two Drill nodes in a cluster with the following IP addresses, versions, and Web Console access:
+
+| Drill Node | Drill Version | Web Console |
+|--------------|---------------|---------------------------|
+| 10.10.123.88 | 1.9.0 | http:// 10.10.123.88:8047 |
+| 10.10.136.25 | 1.10.0 | http://10.10.136.25:8047 |
+
+Accessing the Web Console for Drill node 10.10.123.88 displays Drill version 1.9.0 as the current version with a green label, while also displaying the Drill version for Drill node 10.10.136.25, but with a red label. Accessing the Web Console for Drill node 10.10.136.25 displays 1.10.0 as the current version with a green label, while also displaying the Drill version for Drill node 10.10.123.88, but with a red label. In both cases, the Web Console generates a warning to state that the Drill versions do not match.
+
+The Web Console sorts the Drill nodes by version, starting with the current Drill node, followed by Drill nodes with Drill versions that match the current version, followed by Drill nodes that do not match the current version. Drill nodes marked as having an \u201cundefined\u201d version may be incorrectly defined or have a pre-1.10.0 version of Drill installed.
http://git-wip-us.apache.org/repos/asf/drill/blob/74eabf4d/_docs/odbc-jdbc-interfaces/015-using-jdbc-driver.md
----------------------------------------------------------------------
diff --git a/_docs/odbc-jdbc-interfaces/015-using-jdbc-driver.md b/_docs/odbc-jdbc-interfaces/015-using-jdbc-driver.md
index 1ace1cc..e572adc 100644
--- a/_docs/odbc-jdbc-interfaces/015-using-jdbc-driver.md
+++ b/_docs/odbc-jdbc-interfaces/015-using-jdbc-driver.md
@@ -1,6 +1,6 @@
---
title: "Using the JDBC Driver"
-date:
+date: 2017-03-13 22:49:08 UTC
parent: "ODBC/JDBC Interfaces"
---
This section explains how to install and use the JDBC driver for Apache Drill. To use the JDBC driver, you have to:
@@ -79,14 +79,32 @@ drill.exec: {
## Using the JDBC URL Format for a Direct Drillbit Connection
-If you want to connect directly to a Drillbit instead of using ZooKeeper to choose the Drillbit, replace `zk=<zk name>` with `drillbit=<node>` as shown in the following URL.
+If you want to connect directly to a Drillbit instead of using ZooKeeper to choose the Drillbit, replace `zk=<zk name>` with `drillbit=<node name>` as shown in the following URL:
`jdbc:drill:drillbit=<node name>[:<port>][,<node name2>[:<port>]... `
`<directory>/<cluster ID>[schema=<storage plugin>]`
where
-`drillbit=<node name>` specifies one or more host names or IP addresses of cluster nodes running Drill.
+`drillbit=<node name>` specifies one or more host names or IP addresses of cluster nodes running Drill.
+
+###`tries` Parameter
+
+As of Drill 1.10, you can include the optional `tries=<value>` parameter in the connection string, as shown in the following URL:
+
+
+ jdbc:drill:drillbit=<node name>[:<port>][,<node name2>[:<port>]...
+ <directory>/<cluster ID>;[schema=<storage plugin>];tries=5
+
+The \u201ctries\u201d option represents the maximum number of unique drillbits to which the client can try to establish a successful connection. The default value is 5. This option improves the fault tolerance in the Drill client when first trying to connect with a drillbit, which will then act as the Foreman (the node that drives the query).
+
+The order in which the client tries to connect to the drillbits may not occur in the order listed in the connection string. If the first try results in an authentication failure, the client does not attempt any additional tries. If the number of unique drillbits listed in the `drillbit` parameter is less than the \u201ctries\u201d value, the client tries to connect to each drillbit one time.
+
+For example, if there are three unique drillbits listed in the connection string, and the \u201ctries\u201d value is set to 5, the client can try to connect to each drillbit once, until a successful connection is made, as shown in the image below:
+
+
+
+If the client cannot successfully connect to any of the drillbits, Drill returns a failure message.
For definitions of other URL components, see [Using the JDBC URL for a Random Drillbit Connection]({{site.baseurl}}/docs/using-the-jdbc-driver/#using-the-jdbc-url-for-a-random-drillbit-connection).
http://git-wip-us.apache.org/repos/asf/drill/blob/74eabf4d/_docs/query-data/query-a-file-system/005-querying-a-file-system-introduction.md
----------------------------------------------------------------------
diff --git a/_docs/query-data/query-a-file-system/005-querying-a-file-system-introduction.md b/_docs/query-data/query-a-file-system/005-querying-a-file-system-introduction.md
index 98e107c..436cc75 100644
--- a/_docs/query-data/query-a-file-system/005-querying-a-file-system-introduction.md
+++ b/_docs/query-data/query-a-file-system/005-querying-a-file-system-introduction.md
@@ -1,6 +1,6 @@
---
title: "Querying a File System Introduction"
-date: 2016-04-12 18:29:30 UTC
+date: 2017-03-13 22:49:09 UTC
parent: "Querying a File System"
---
Files and directories are like standard SQL tables to Drill. You can specify a
@@ -33,4 +33,43 @@ Drill supports the following file types:
The extensions for these file types must match the configuration settings for
your registered storage plugins. For example, PSV files may be defined with a
-`.tbl` extension, while CSV files are defined with a `.csv` extension.
+`.tbl` extension, while CSV files are defined with a `.csv` extension.
+
+##Implicit Columns
+Drill 1.8 introduces implicit columns. Implicit columns provide file information, such as the directory path to a file and the file extension. You can query implicit columns in files, directories, nested directories, and files.
+
+The following table lists the implicit columns available and their descriptions:
+
+| Implicit Column Name | Description |
+|------------------------|--------------------------------------------------------------------------------------------|
+| FQN | The fully qualified name. Contains the full path to the file, including the file name. |
+| FILEPATH | The full path to the file, without the file name. |
+| FILENAME | The file name with the file extension. Does not include the path to the file. |
+| SUFFIX | The file suffix without the dot (.) at the beginning. |
+
+To access implicit columns, you must explicitly include the columns in a query, as shown in the following example:
+
+ 0: jdbc:drill:zk=local> SELECT fqn, filepath, filename, suffix FROM dfs.`/dev/data/files/test.csvh` LIMIT 1;
+
+ +-------------------------------------+--------------------------+---------------+----------------+
+ | fqn | filepath | filename | suffix |
+ +-------------------------------------+--------------------------+---------------+----------------+
+ | /dev/data/files/test.csvh | /dev/data/files | test.csvh | csvh |
+ +-------------------------------------+--------------------------+---------------+----------------+
+
+{% include startnote.html %}If a table has a column with the same name as an implicit column, such as \u201csuffix,\u201d the implicit column overrides the table column.{% include endnote.html %}
+
+If a column name has the same name as an implicit column, you can change the default implicit column name using the [ALTER SYSTEM|SESSION SET]({{site.baseurl}}/docs/alter-system/) command with the appropriate parameter, as shown in the following example:
+
+ ALTER SYSTEM SET `drill.exec.storage.implicit.suffix.column.label` = appendix;
+
+Use the following configuration options to change the default implicit column names:
+
+ drill.exec.storage.implicit.fqn.column.label
+ drill.exec.storage.implicit.filepath.column.label
+ drill.exec.storage.implicit.filename.column.label
+ drill.exec.storage.implicit.suffix.column.label
+
+
+
+
[3/7] drill git commit: Doc updates for Drill 1.10
Posted by br...@apache.org.
http://git-wip-us.apache.org/repos/asf/drill/blob/b7b848e6/_docs/rn/001-1.10.0-rn.md
----------------------------------------------------------------------
diff --git a/_docs/rn/001-1.10.0-rn.md b/_docs/rn/001-1.10.0-rn.md
new file mode 100644
index 0000000..be2cc1b
--- /dev/null
+++ b/_docs/rn/001-1.10.0-rn.md
@@ -0,0 +1,307 @@
+---
+title: "Apache Drill 1.10.0 Release Notes"
+parent: "Release Notes"
+---
+
+**Release date:** March 15, 2017
+
+Today, we're happy to announce the availability of Drill 1.10.0. You can download it [here](https://drill.apache.org/download/).
+
+## New Features
+This release of Drill provides the following new features:
+
+- Support for the Create Temporary Table As (CTTAS) command.
+- JDBC connection option that improves fault tolerance when connecting directly to a Drill node from a client.
+- Drill writes standard Parquet date values and automatically detects and corrects any corrupted date values.
+- The Drill Web Console displays the Drill version running on each Drill node in the cluster.
+- Drill can implicitly interpret the INT96 timestamp data type in Parquet files.
+- Support for Kerberos authentication.
+
+The following sections list additional bug fixes and improvements:
+
+<h2> Sub-task
+</h2>
+<ul>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4272'>DRILL-4272</a>] - When sort runs out of memory and query fails, resources are seemingly not freed
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4301'>DRILL-4301</a>] - OOM : Unable to allocate sv2 for 1000 records, and not enough batchGroups to spill.
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4730'>DRILL-4730</a>] - Update JDBC DatabaseMetaData implementation to use new Metadata APIs
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5008'>DRILL-5008</a>] - Refactor, document and simplify ExternalSortBatch
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5011'>DRILL-5011</a>] - External Sort Batch memory use depends on record width
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5014'>DRILL-5014</a>] - ExternalSortBatch cache size, spill count differs from config setting
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5017'>DRILL-5017</a>] - Config param drill.exec.sort.external.batch.size is not used
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5019'>DRILL-5019</a>] - ExternalSortBatch spills all batches to disk even if even one spills
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5020'>DRILL-5020</a>] - ExternalSortBatch has inconsistent notions of the memory limit
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5022'>DRILL-5022</a>] - ExternalSortBatch sets two different limits for "copier" memory
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5023'>DRILL-5023</a>] - ExternalSortBatch does not spill fully, throws off spill calculations
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5025'>DRILL-5025</a>] - ExternalSortBatch provides weak control over spill file size
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5026'>DRILL-5026</a>] - ExternalSortBatch uses two memory allocators; one will do
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5027'>DRILL-5027</a>] - ExternalSortBatch is inefficient: rewrites data unnecessarily
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5055'>DRILL-5055</a>] - External Sort does not delete spill file if error occurs during close
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5062'>DRILL-5062</a>] - External sort refers to the deprecated HDFS fs.default.name param
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5066'>DRILL-5066</a>] - External sort attempts to retry sv2 memory alloc, even if can never succeed
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5210'>DRILL-5210</a>] - External Sort BatchGroup leaks memory if an OOM occurs during read
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5262'>DRILL-5262</a>] - NPE in managed external sort while spilling to disk
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5264'>DRILL-5264</a>] - Managed External Sort fails with OOM
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5267'>DRILL-5267</a>] - Managed external sort spills too often with Parquet data
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5285'>DRILL-5285</a>] - Provide detailed, accurate estimate of size consumed by a record batch
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5294'>DRILL-5294</a>] - Managed External Sort throws an OOM during the merge and spill phase
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5310'>DRILL-5310</a>] - Memory leak in managed sort if OOM during sv2 allocation
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5312'>DRILL-5312</a>] - "Record batch sizer" does not include overhead for variable-sized vectors
+</li>
+</ul>
+
+<h2> Bug
+</h2>
+<ul>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1808'>DRILL-1808</a>] - Large compilation unit tests fails due to high memory allocation
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-2293'>DRILL-2293</a>] - CTAS does not clean up when it fails
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3562'>DRILL-3562</a>] - Query fails when using flatten on JSON data where some documents have an empty array
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4578'>DRILL-4578</a>] - "children" missing from results of full scan over JSON data
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4764'>DRILL-4764</a>] - Parquet file with INT_16, etc. logical types not supported by simple SELECT
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4812'>DRILL-4812</a>] - Wildcard queries fail on Windows
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4850'>DRILL-4850</a>] - TPCDS Query 33 failed in the second and 3rd runs, but succeeded in the 1st run
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4872'>DRILL-4872</a>] - NPE from CTAS partitioned by a projected casted null
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4919'>DRILL-4919</a>] - Fix select count(1) / count(*) on csv with header
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4938'>DRILL-4938</a>] - Report UserException when constant expression reduction fails
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4963'>DRILL-4963</a>] - Issues when overloading Drill native functions with dynamic UDFs
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4982'>DRILL-4982</a>] - Hive Queries degrade when queries switch between different formats
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4994'>DRILL-4994</a>] - Prepared statement stopped working between 1.8.0 client and < 1.8.0 server
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4995'>DRILL-4995</a>] - Allow lazy init when dynamic UDF support is disabled
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4996'>DRILL-4996</a>] - Parquet Date auto-correction is not working in auto-partitioned parquet files generated by drill-1.6
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5005'>DRILL-5005</a>] - Potential issues with external sort info in query profile
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5015'>DRILL-5015</a>] - As per documentation, when issuing a list of drillbits in the connection string, we always attempt to connect only to the first one
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5032'>DRILL-5032</a>] - Drill query on hive parquet table failed with OutOfMemoryError: Java heap space
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5034'>DRILL-5034</a>] - Select timestamp from hive generated parquet always return in UTC
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5039'>DRILL-5039</a>] - NPE - CTAS PARTITION BY (<char-type-column>)
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5040'>DRILL-5040</a>] - Interrupted CTAS should not succeed & should not create physical file on disk
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5044'>DRILL-5044</a>] - Fix retry logic to handle VersionMismatchException by not deleting jars in remote UDFs area
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5048'>DRILL-5048</a>] - Fix type mismatch error in case statement with null timestamp
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5050'>DRILL-5050</a>] - C++ client library has symbol resolution issues when loaded by a process that already uses boost::asio
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5051'>DRILL-5051</a>] - DRILL-5051: Fix incorrect result returned in nest query with offset specified
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5070'>DRILL-5070</a>] - Code gen: create methods in fixed order to allow test verification
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5081'>DRILL-5081</a>] - Excessive info level logging introduced in DRILL-4203
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5086'>DRILL-5086</a>] - ClassCastException when filter pushdown is used with a bigint or float column and metadata caching.
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5088'>DRILL-5088</a>] - Error when reading DBRef column
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5091'>DRILL-5091</a>] - JDBC unit test fail on Java 8
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5094'>DRILL-5094</a>] - Assure Comparator to be transitive
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5097'>DRILL-5097</a>] - Using store.parquet.reader.int96_as_timestamp gives IOOB whereas convert_from works
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5104'>DRILL-5104</a>] - Foreman sets external sort memory allocation even for a physical plan
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5112'>DRILL-5112</a>] - Unit tests derived from PopUnitTestBase fail in IDE due to config errors
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5113'>DRILL-5113</a>] - Upgrade Maven RAT plugin to avoid annoying XML errors
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5117'>DRILL-5117</a>] - Compile error when query a json file with 1000+columns
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5119'>DRILL-5119</a>] - Update MapR version to 5.2.0.40963-mapr
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5121'>DRILL-5121</a>] - A memory leak is observed when exact case is not specified for a column in a filter condition
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5127'>DRILL-5127</a>] - Revert the fix for DRILL-4831
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5157'>DRILL-5157</a>] - Multiple Snappy versions on class path; causes unit test failures
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5159'>DRILL-5159</a>] - ProjectMergeRule in Drill should operate on RelNodes with same convention trait.
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5164'>DRILL-5164</a>] - Equi-join query results in CompileException when inputs have large number of columns
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5167'>DRILL-5167</a>] - C++ connector does not set escape string for metadata search pattern
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5190'>DRILL-5190</a>] - Display planning and queued time for a query in its profile page
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5196'>DRILL-5196</a>] - Could not run a single MongoDB unit test case through command line or IDE
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5207'>DRILL-5207</a>] - Improve Parquet scan pipelining
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5208'>DRILL-5208</a>] - Finding path to java executable should be deterministic
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5218'>DRILL-5218</a>] - Support Disabling Heartbeats in C++ Client
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5224'>DRILL-5224</a>] - CTTAS: fix errors connected with system path delimiters (Windows)
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5230'>DRILL-5230</a>] - Translation of millisecond duration into hours is incorrect
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5238'>DRILL-5238</a>] - CTTAS: unable to resolve temporary table if workspace is indicated without schema
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5242'>DRILL-5242</a>] - The UI breaks when trying to render profiles having unknown metrics
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5243'>DRILL-5243</a>] - Fix TestContextFunctions.sessionIdUDFWithinSameSession unit test
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5252'>DRILL-5252</a>] - A condition returns always true
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5263'>DRILL-5263</a>] - Prevent left NLJoin with non scalar subqueries
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5266'>DRILL-5266</a>] - Parquet Reader produces "low density" record batches - bits vs. bytes
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5273'>DRILL-5273</a>] - CompliantTextReader exhausts 4 GB memory when reading 5000 small files
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5274'>DRILL-5274</a>] - Exception thrown in Drillbit shutdown in UDF cleanup code
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5275'>DRILL-5275</a>] - Sort spill serialization is slow due to repeated buffer allocations
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5284'>DRILL-5284</a>] - Roll-up of final fixes for managed sort
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5287'>DRILL-5287</a>] - Provide option to skip updates of ephemeral state changes in Zookeeper
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5293'>DRILL-5293</a>] - Poor performance of Hash Table due to same hash value as distribution below
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5304'>DRILL-5304</a>] - Queries fail intermittently when there is skew in data distribution
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5313'>DRILL-5313</a>] - C++ client build failure on linux
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5326'>DRILL-5326</a>] - Unit tests failures related to the SERVER_METADTA
+</li>
+</ul>
+
+<h2> Improvement
+</h2>
+<ul>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3510'>DRILL-3510</a>] - Add ANSI_QUOTES option so that Drill's SQL Parser will recognize ANSI_SQL identifiers
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4217'>DRILL-4217</a>] - Query parquet file treat INT_16 & INT_8 as INT32
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4280'>DRILL-4280</a>] - Kerberos Authentication
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4373'>DRILL-4373</a>] - Drill and Hive have incompatible timestamp representations in parquet
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4604'>DRILL-4604</a>] - Generate warning on Web UI if drillbits version mismatch is detected
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4864'>DRILL-4864</a>] - Add ANSI format for date/time functions
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4956'>DRILL-4956</a>] - Temporary tables support
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4980'>DRILL-4980</a>] - Upgrading of the approach of parquet date correctness status detection
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4987'>DRILL-4987</a>] - Use ImpersonationUtil in RemoteFunctionRegistry
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5043'>DRILL-5043</a>] - Function that returns a unique id per session/connection similar to MySQL's CONNECTION_ID()
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5052'>DRILL-5052</a>] - Option to debug generated Java code using an IDE
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5056'>DRILL-5056</a>] - UserException does not write full message to log
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5065'>DRILL-5065</a>] - Optimize count(*) queries on MapR-DB JSON Tables
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5080'>DRILL-5080</a>] - Create a memory-managed version of the External Sort operator
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5085'>DRILL-5085</a>] - Add / update description for dynamic UDFs directories in drill-env.sh and drill-module.conf
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5098'>DRILL-5098</a>] - Improving fault tolerance for connection between client and foreman node.
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5108'>DRILL-5108</a>] - Reduce output from Maven git-commit-id-plugin
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5116'>DRILL-5116</a>] - Enable generated code debugging in each Drill operator
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5123'>DRILL-5123</a>] - Write query profile after sending final response to client to improve latency
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5126'>DRILL-5126</a>] - Provide simplified, unified "cluster fixture" for tests
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5172'>DRILL-5172</a>] - Display elapsed time for queries in the UI
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5195'>DRILL-5195</a>] - Publish Operator and MajorFragment Stats in Profile page
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5215'>DRILL-5215</a>] - CTTAS: disallow temp tables in view expansion logic
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5221'>DRILL-5221</a>] - cancel message is delayed until queryid or data is received
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5254'>DRILL-5254</a>] - Enhance default reduction factors in optimizer
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5255'>DRILL-5255</a>] - Unit tests fail due to CTTAS temporary name space checks
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5257'>DRILL-5257</a>] - Provide option to save query profiles sync, async or not at all
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5258'>DRILL-5258</a>] - Allow "extended" mock tables access from SQL queries
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5259'>DRILL-5259</a>] - Allow listing a user-defined number of profiles
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5260'>DRILL-5260</a>] - Refinements to new "Cluster Fixture" test framework
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5270'>DRILL-5270</a>] - Improve loading of profiles listing in the WebUI
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5290'>DRILL-5290</a>] - Provide an option to build operator table once for built-in static functions and reuse it across queries.
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5301'>DRILL-5301</a>] - Add server metadata API
+</li>
+</ul>
+
+<h2> New Feature
+</h2>
+<ul>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4935'>DRILL-4935</a>] - Allow drillbits to advertise a configurable host address to Zookeeper
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4979'>DRILL-4979</a>] - Make dataport configurable
+</li>
+</ul>
+
+<h2> Task
+</h2>
+<ul>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5354'>DRILL-5354</a>] - Create CTTAS Documentaion
+</li>
+</ul>
+
+<h2> Test
+</h2>
+<ul>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4253'>DRILL-4253</a>] - Some functional tests are failing because sort limit is too low
+</li>
+</ul>
+
+
+
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/drill/blob/b7b848e6/_docs/sql-reference/sql-commands/005-supported-sql-commands.md
----------------------------------------------------------------------
diff --git a/_docs/sql-reference/sql-commands/005-supported-sql-commands.md b/_docs/sql-reference/sql-commands/005-supported-sql-commands.md
index 5a87073..6605469 100644
--- a/_docs/sql-reference/sql-commands/005-supported-sql-commands.md
+++ b/_docs/sql-reference/sql-commands/005-supported-sql-commands.md
@@ -1,6 +1,6 @@
---
title: Supported SQL Commands
-date: 2016-07-22 00:26:58 UTC
+date: 2017-03-14 22:11:02 UTC
parent: "SQL Commands"
---
The following table provides a list of the SQL commands that Drill supports,
@@ -13,6 +13,7 @@ with their descriptions and example syntax:
| [ALTER SYSTEM RESET]({{site.baseurl}}/docs/alter-system) | Changes a system setting back to its default system setting. For a list of Drill options and their descriptions, see [Planning and Execution Options]{{site.baseurl/docs/planning-and-execution-options/}}. | ALTER SYSTEM RESET \`option_name`; |
| [ALTER SYSTEM RESET ALL]({{site.baseurl}}/docs/alter-system) | Changes all system settings back to their default system values. For a list of Drill options and their descriptions, see [Planning and Execution Options]{{site.baseurl/docs/planning-and-execution-options/}}. | ALTER SYSTEM RESET ALL; |
| [CREATE TABLE AS(CTAS)]({{site.baseurl}}/docs/create-table-as-ctas/) | Creates a new table and populates the new table with rows returned from a SELECT query. Use the CREATE TABLE AS (CTAS) statement in place of INSERT INTO. | CREATE TABLE name AS query; |
+| [CREATE TEMPORARY TABLE AS(CTTAS)]({{site.baseurl}}/docs/create-temp-table-as-cttas/) | Stores the results of a query in a temporary table. | CREATE TEMPORARY TABLE name AS query; |
| [CREATE VIEW]({{site.baseurl}}/docs/create-view) | Creates a virtual structure for the result set of a stored query.- | CREATE [OR REPLACE] VIEW [workspace.]view_name [ (column_name [, ...]) ] AS query; |
| [DESCRIBE ]({{site.baseurl}}/docs/describe) | Returns information about columns in a table or view. | DESCRIBE [workspace.]table_name |
| [DROP VIEW]({{site.baseurl}}/docs/drop-view) | Removes a view. | DROP VIEW [workspace.]view_name ; |
http://git-wip-us.apache.org/repos/asf/drill/blob/b7b848e6/_docs/sql-reference/sql-commands/079-select.md
----------------------------------------------------------------------
diff --git a/_docs/sql-reference/sql-commands/079-select.md b/_docs/sql-reference/sql-commands/079-select.md
index 9d7d781..96a0087 100644
--- a/_docs/sql-reference/sql-commands/079-select.md
+++ b/_docs/sql-reference/sql-commands/079-select.md
@@ -1,6 +1,6 @@
---
title: "SELECT"
-date:
+date: 2017-03-14 22:11:02 UTC
parent: "SQL Commands"
---
Drill supports the following ANSI standard clauses in the SELECT statement:
@@ -19,6 +19,7 @@ Drill supports the following ANSI standard clauses in the SELECT statement:
You can use the same SELECT syntax in the following commands:
* CREATE TABLE AS (CTAS)
+ * CREATE TEMPORARY TABLES AS (CTTAS)
* CREATE VIEW
INSERT INTO SELECT is not yet supported.
http://git-wip-us.apache.org/repos/asf/drill/blob/b7b848e6/_docs/sql-reference/sql-commands/120-use.md
----------------------------------------------------------------------
diff --git a/_docs/sql-reference/sql-commands/120-use.md b/_docs/sql-reference/sql-commands/120-use.md
index f9f5899..395f716 100644
--- a/_docs/sql-reference/sql-commands/120-use.md
+++ b/_docs/sql-reference/sql-commands/120-use.md
@@ -1,11 +1,12 @@
---
title: "USE"
-date:
+date: 2017-03-14 22:11:03 UTC
parent: "SQL Commands"
---
The USE command changes the schema context to the specified schema. When you
issue the USE command to switch to a particular schema, Drill queries that
schema only.
+**Note:** You must include the full path to a file or table whenever there is a temporary table in the session with the same name. See [CTTAS]({{site.baseurl}}/docs/create-temp-table-as).
## Syntax
@@ -26,7 +27,7 @@ workspaces in Drill. See [Storage Plugin Registration]({{ site.baseurl }}/docs/s
## Usage Notes
Issue the USE command to change to a particular schema. When you use a schema,
-you do not have to include the full path to a file or table in your query.
+you do not have to include the full path to a file or table in your query.
For example, to query a file named `donuts.json` in the
`/users/max/drill/json/` directory, you must include the full file path in
[5/7] drill git commit: Doc updates for Drill 1.10
Posted by br...@apache.org.
Doc updates for Drill 1.10
Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/b7b848e6
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/b7b848e6
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/b7b848e6
Branch: refs/heads/gh-pages
Commit: b7b848e6a3af1de808d201aa9b3c16f167cf0610
Parents: 4fee2dc
Author: Bridget Bevens <bb...@maprtech.com>
Authored: Tue Mar 14 15:11:00 2017 -0700
Committer: Bridget Bevens <bb...@maprtech.com>
Committed: Tue Mar 14 15:11:00 2017 -0700
----------------------------------------------------------------------
.../010-configuration-options-introduction.md | 166 +++++-----
_docs/rn/001-1.10.0-rn.md | 307 +++++++++++++++++++
.../sql-commands/005-supported-sql-commands.md | 3 +-
_docs/sql-reference/sql-commands/079-select.md | 3 +-
_docs/sql-reference/sql-commands/120-use.md | 5 +-
5 files changed, 401 insertions(+), 83 deletions(-)
----------------------------------------------------------------------
[7/7] drill git commit: update docs for 1.10
Posted by br...@apache.org.
update docs for 1.10
Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/c7579500
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/c7579500
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/c7579500
Branch: refs/heads/gh-pages
Commit: c75795000e62accf97e42d6c1d06d316d58a94ca
Parents: 0aae69b
Author: Bridget Bevens <bb...@maprtech.com>
Authored: Tue Mar 14 19:39:13 2017 -0700
Committer: Bridget Bevens <bb...@maprtech.com>
Committed: Tue Mar 14 19:39:13 2017 -0700
----------------------------------------------------------------------
_data/version.json | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/drill/blob/c7579500/_data/version.json
----------------------------------------------------------------------
diff --git a/_data/version.json b/_data/version.json
index 51479e3..c5ea3a9 100644
--- a/_data/version.json
+++ b/_data/version.json
@@ -1,7 +1,7 @@
{
- "display_version": "1.9",
- "full_version": "1.9.0",
- "release_date": "November 29, 2016",
- "blog_post":"/blog/2016/11/29/drill-1.9-released",
- "release_notes": "https://drill.apache.org/docs/apache-drill-1-9-0-release-notes/"
+ "display_version": "1.10",
+ "full_version": "1.10.0",
+ "release_date": "March 15, 2017",
+ "blog_post":"/blog/2017/03/15/drill-1.10-released",
+ "release_notes": "https://drill.apache.org/docs/apache-drill-1-10-0-release-notes/"
}
[4/7] drill git commit: Doc updates for Drill 1.10
Posted by br...@apache.org.
http://git-wip-us.apache.org/repos/asf/drill/blob/b7b848e6/_docs/configure-drill/configuration-options/010-configuration-options-introduction.md
----------------------------------------------------------------------
diff --git a/_docs/configure-drill/configuration-options/010-configuration-options-introduction.md b/_docs/configure-drill/configuration-options/010-configuration-options-introduction.md
index 4ea7540..a841d99 100644
--- a/_docs/configure-drill/configuration-options/010-configuration-options-introduction.md
+++ b/_docs/configure-drill/configuration-options/010-configuration-options-introduction.md
@@ -1,10 +1,10 @@
---
title: "Configuration Options Introduction"
-date: 2016-10-03 22:44:17 UTC
+date: 2017-03-14 22:11:01 UTC
parent: "Configuration Options"
---
-Drill provides many configuration options that you can enable, disable, or modify. Modifying certain configuration options can impact Drill performance. Many of the configuration options reside in the `drill-env.sh` script and the `drill-override.conf` configuration file located in the `$DRILL_HOME/conf` directory. Drill loads these files from `/etc/drill/conf`, if it exists. Otherwise, Drill loads the files from the `$DRILL_HOME/conf` directory.
+Drill provides many configuration options that you can enable, disable, or modify. Modifying certain configuration options can impact performance. Many of the configuration options reside in the `drill-env.sh` script and the `drill-override.conf` configuration file located in the `$DRILL_HOME/conf` directory. Drill loads these files from `/etc/drill/conf`, if it exists. Otherwise, Drill loads the files from the `$DRILL_HOME/conf` directory.
{% include startnote.html %}As of Drill 1.8, most of the system options listed below are set in $DRILL_HOME/conf/drill-override.conf. Settings controlled through environment variables reside in $DRILL_HOME/bin/drill-config.sh. You can create an entry in $DRILL_HOME/conf/drill-env.sh to override many of the settings in drill-config.sh.{% include endnote.html %}
@@ -13,81 +13,89 @@ See [Configuration and Launch Script Changes]({{site.baseurl}}/docs/apache-drill
The sys.options table contains information about system and session options. The sys.boot table contains information about Drill start-up options. The section, ["Start-up Options"]({{site.baseurl}}/docs/start-up-options), covers how to configure and view key boot options. The following table lists the system options in alphabetical order and provides a brief description of supported options.
## System Options
-The sys.options table lists the following options that you can set as a system or session option as described in the section, ["Planning and Execution Options"]({{site.baseurl}}/docs/planning-and-execution-options).
+The sys.options table lists the following options that you can set as a system or session option as described in the section, ["Planning and Execution Options"]({{site.baseurl}}/docs/planning-and-execution-options).
+
+| Name | Default | Comments
|
+|--------------------------------------------------------|--------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| drill.exec.default_temporary_workspace | dfs.tmp | Available as of Drill 1.10. Sets the workspace for temporary tables. The workspace must be writable, file-based, and point to a location that already exists. This option requires the following format: `<schema>.<workspace`
|
+| drill.exec.storage.implicit.filename.column.label | filename | Available as of Drill 1.10. Sets the implicit column name for the filename column.
|
+| drill.exec.storage.implicit.filepath.column.label | filepath | Available as of Drill 1.10. Sets the implicit column name for the filepath column.
|
+| drill.exec.storage.implicit.fqn.column.label | fqn | Available as of Drill 1.10. Sets the implicit column name for the fqn column.
|
+| drill.exec.storage.implicit.suffix.column.label | suffix | Available as of Drill 1.10. Sets the implicit column name for the suffix column.
|
+| drill.exec.functions.cast_empty_string_to_null | FALSE | In a text file, treat empty fields as NULL values instead of empty string.
|
+| drill.exec.storage.file.partition.column.label | dir | The column label for directory levels in results of queries of files in a directory. Accepts a string input.
|
+| exec.enable_union_type | FALSE | Enable support for Avro union type.
|
+| exec.errors.verbose | FALSE | Toggles verbose output of executable error messages
|
+| exec.java_compiler | DEFAULT | Switches between DEFAULT, JDK, and JANINO mode for the current session. Uses Janino by default for generated source code of less than exec.java_compiler_janino_maxsize; otherwise, switches to the JDK compiler.
|
+| exec.java_compiler_debug | TRUE | Toggles the output of debug-level compiler error messages in runtime generated code.
|
+| exec.java_compiler_janino_maxsize | 262144 | See the exec.java_compiler option comment. Accepts inputs of type LONG.
|
+| exec.max_hash_table_size | 1073741824 | Ending size in buckets for hash tables. Range: 0 - 1073741824.
|
+| exec.min_hash_table_size | 65536 | Starting size in bucketsfor hash tables. Increase according to available memory to improve performance. Increasing for very large aggregations or joins when you have large amounts of memory for Drill to use. Range: 0 - 1073741824.
|
+| exec.queue.enable | FALSE | Changes the state of query queues. False allows unlimited concurrent queries.
|
+| exec.queue.large | 10 | Sets the number of large queries that can run concurrently in the cluster. Range: 0-1000
|
+| exec.queue.small | 100 | Sets the number of small queries that can run concurrently in the cluster. Range: 0-1001
|
+| exec.queue.threshold | 30000000 | Sets the cost threshold, which depends on the complexity of the queries in queue, for determining whether query is large or small. Complex queries have higher thresholds. Range: 0-9223372036854775807
|
+| exec.queue.timeout_millis | 300000 | Indicates how long a query can wait in queue before the query fails. Range: 0-9223372036854775807
|
+| exec.schedule.assignment.old | FALSE | Used to prevent query failure when no work units are assigned to a minor fragment, particularly when the number of files is much larger than the number of leaf fragments.
|
+| exec.storage.enable_new_text_reader | TRUE | Enables the text reader that complies with the RFC 4180 standard for text/csv files.
|
+| new_view_default_permissions | 700 | Sets view permissions using an octal code in the Unix tradition.
|
+| planner.add_producer_consumer | FALSE | Increase prefetching of data from disk. Disable for in-memory reads.
|
+| planner.affinity_factor | 1.2 | Factor by which a node with endpoint affinity is favored while creating assignment. Accepts inputs of type DOUBLE.
|
+| planner.broadcast_factor | 1 | A heuristic parameter for influencing the broadcast of records as part of a query.
|
+| planner.broadcast_threshold | 10000000 | The maximum number of records allowed to be broadcast as part of a query. After one million records, Drill reshuffles data rather than doing a broadcast to one side of the join. Range: 0-2147483647
|
+| planner.disable_exchanges | FALSE | Toggles the state of hashing to a random exchange.
|
+| planner.enable_broadcast_join | TRUE | Changes the state of aggregation and join operators. The broadcast join can be used for hash join, merge join and nested loop join. Use to join a large (fact) table to relatively smaller (dimension) tables. Do not disable.
|
+| planner.enable_constant_folding | TRUE | If one side of a filter condition is a constant expression, constant folding evaluates the expression in the planning phase and replaces the expression with the constant value. For example, Drill can rewrite WHERE age + 5 < 42 as WHERE age < 37.
|
+| planner.enable_decimal_data_type | FALSE | False disables the DECIMAL data type, including casting to DECIMAL and reading DECIMAL types from Parquet and Hive.
|
+| planner.enable_demux_exchange | FALSE | Toggles the state of hashing to a demulitplexed exchange.
|
+| planner.enable_hash_single_key | TRUE | Each hash key is associated with a single value.
|
+| planner.enable_hashagg | TRUE | Enable hash aggregation; otherwise, Drill does a sort-based aggregation. Does not write to disk. Enable is recommended.
|
+| planner.enable_hashjoin | TRUE | Enable the memory hungry hash join. Drill assumes that a query with have adequate memory to complete and tries to use the fastest operations possible to complete the planned inner, left, right, or full outer joins using a hash table. Does not write to disk. Disabling hash join allows Drill to manage arbitrarily large data in a small memory footprint.
|
+| planner.enable_hashjoin_swap | TRUE | Enables consideration of multiple join order sequences during the planning phase. Might negatively affect the performance of some queries due to inaccuracy of estimated row count especially after a filter, join, or aggregation.
|
+| planner.enable_hep_join_opt | | Enables the heuristic planner for joins.
|
+| planner.enable_mergejoin | TRUE | Sort-based operation. A merge join is used for inner join, left and right outer joins. Inputs to the merge join must be sorted. It reads the sorted input streams from both sides and finds matching rows. Writes to disk.
|
+| planner.enable_multiphase_agg | TRUE | Each minor fragment does a local aggregation in phase 1, distributes on a hash basis using GROUP-BY keys partially aggregated results to other fragments, and all the fragments perform a total aggregation using this data.
|
+| planner.enable_mux_exchange | TRUE | Toggles the state of hashing to a multiplexed exchange.
|
+| planner.enable_nestedloopjoin | TRUE | Sort-based operation. Writes to disk.
|
+| planner.enable_nljoin_for_scalar_only | TRUE | Supports nested loop join planning where the right input is scalar in order to enable NOT-IN, Inequality, Cartesian, and uncorrelated EXISTS planning.
|
+| planner.enable_streamagg | TRUE | Sort-based operation. Writes to disk.
|
+| planner.filter.max_selectivity_estimate_factor | 1 | Available as of Drill 1.8. Sets the maximum filter selectivity estimate. The selectivity can vary between 0 and 1. For more details, see planner.filter.min_selectivity_estimate_factor.
|
+| planner.filter.min_selectivity_estimate_factor | 0 | Available as of Drill 1.8. Sets the minimum filter selectivity estimate to increase the parallelization of the major fragment performing a join. This option is useful for deeply nested queries with complicated predicates and serves as a workaround when statistics are insufficient or unavailable. The selectivity can vary between 0 and 1. The value of this option caps the estimated SELECTIVITY. The estimated ROWCOUNT is derived by multiplying the estimated SELECTIVITY by the estimated ROWCOUNT of the upstream operator. The estimated ROWCOUNT displays when you use the EXPLAIN PLAN INCLUDING ALL ATTRIBUTES FOR command. This option does not control the estimated ROWCOUNT of downstream operators (post FILTER). However, estimated ROWCOUNTs may change because the operator ROWCOUNTs depend on their downstream operators. The FILTER operator relies on the input of its immediate upstream oper
ator, for example SCAN, AGGREGATE. If two filters are present in a plan, each filter may have a different estimated ROWCOUNT based on the immediate upstream operator's estimated ROWCOUNT. |
+| planner.identifier_max_length | 1024 | A minimum length is needed because option names are identifiers themselves.
|
+| planner.join.hash_join_swap_margin_factor | 10 | The number of join order sequences to consider during the planning phase.
|
+| planner.join.row_count_estimate_factor | 1 | The factor for adjusting the estimated row count when considering multiple join order sequences during the planning phase.
|
+| planner.memory.average_field_width | 8 | Used in estimating memory requirements.
|
+| planner.memory.enable_memory_estimation | FALSE | Toggles the state of memory estimation and re-planning of the query. When enabled, Drill conservatively estimates memory requirements and typically excludes these operators from the plan and negatively impacts performance.
|
+| planner.memory.hash_agg_table_factor | 1.1 | A heuristic value for influencing the size of the hash aggregation table.
|
+| planner.memory.hash_join_table_factor | 1.1 | A heuristic value for influencing the size of the hash aggregation table.
|
+| planner.memory.max_query_memory_per_node | 2147483648 bytes | Sets the maximum amount of direct memory allocated to the sort operator in each query on a node. If a query plan contains multiple sort operators, they all share this memory. If you encounter memory issues when running queries with sort operators, increase the value of this option.
|
+| planner.memory.non_blocking_operators_memory | 64 | Extra query memory per node for non-blocking operators. This option is currently used only for memory estimation. Range: 0-2048 MB
|
+| planner.memory_limit | 268435456 bytes | Defines the maximum amount of direct memory allocated to a query for planning. When multiple queries run concurrently, each query is allocated the amount of memory set by this parameter.Increase the value of this parameter and rerun the query if partition pruning failed due to insufficient memory.
|
+| planner.nestedloopjoin_factor | 100 | A heuristic value for influencing the nested loop join.
|
+| planner.partitioner_sender_max_threads | 8 | Upper limit of threads for outbound queuing.
|
+| planner.partitioner_sender_set_threads | -1 | Overwrites the number of threads used to send out batches of records. Set to -1 to disable. Typically not changed.
|
+| planner.partitioner_sender_threads_factor | 2 | A heuristic param to use to influence final number of threads. The higher the value the fewer the number of threads.
|
+| planner.producer_consumer_queue_size | 10 | How much data to prefetch from disk in record batches out-of-band of query execution. The larger the queue size, the greater the amount of memory that the queue and overall query execution consumes.
|
+| planner.slice_target | 100000 | The number of records manipulated within a fragment before Drill parallelizes operations.
|
+| planner.width.max_per_node | 3 | Maximum number of threads that can run in parallel for a query on a node. A slice is an individual thread. This number indicates the maximum number of slices per query for the query\u2019s major fragment on a node.
|
+| planner.width.max_per_query | 1000 | Same as max per node but applies to the query as executed by the entire cluster. For example, this value might be the number of active Drillbits, or a higher number to return results faster.
|
+| security.admin.user_groups | n/a | Unsupported as of 1.4. A comma-separated list of administrator groups for Web Console security.
|
+| security.admin.users | | Unsupported as of 1.4. A comma-separated list of user names who you want to give administrator privileges.
|
+| store.format | parquet | Output format for data written to tables with the CREATE TABLE AS (CTAS) command. Allowed values are parquet, json, psv, csv, or tsv.
|
+| store.hive.optimize_scan_with_native_readers | FALSE | Optimize reads of Parquet-backed external tables from Hive by using Drill native readers instead of the Hive Serde interface. (Drill 1.2 and later)
|
+| store.json.all_text_mode | FALSE | Drill reads all data from the JSON files as VARCHAR. Prevents schema change errors.
|
+| store.json.extended_types | FALSE | Turns on special JSON structures that Drill serializes for storing more type information than the four basic JSON types.
|
+| store.json.read_numbers_as_double | FALSE | Reads numbers with or without a decimal point as DOUBLE. Prevents schema change errors.
|
+| store.mongo.all_text_mode | FALSE | Similar to store.json.all_text_mode for MongoDB.
|
+| store.mongo.read_numbers_as_double | FALSE | Similar to store.json.read_numbers_as_double.
<TRUNCATED>
[2/7] drill git commit: add CTTAS doc for Drill 1.10 release
Posted by br...@apache.org.
add CTTAS doc for Drill 1.10 release
Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/4fee2dc1
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/4fee2dc1
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/4fee2dc1
Branch: refs/heads/gh-pages
Commit: 4fee2dc10e60e108896528ea5c40104dd18f8256
Parents: 74eabf4
Author: Bridget Bevens <bb...@maprtech.com>
Authored: Mon Mar 13 22:06:03 2017 -0700
Committer: Bridget Bevens <bb...@maprtech.com>
Committed: Mon Mar 13 22:06:03 2017 -0700
----------------------------------------------------------------------
_docs/sql-reference/040-operators.md | 6 +-
.../031-create-temporary-table-as.md | 146 +++++++++++++++++++
2 files changed, 149 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/drill/blob/4fee2dc1/_docs/sql-reference/040-operators.md
----------------------------------------------------------------------
diff --git a/_docs/sql-reference/040-operators.md b/_docs/sql-reference/040-operators.md
index a6c26d7..d67d6ce 100644
--- a/_docs/sql-reference/040-operators.md
+++ b/_docs/sql-reference/040-operators.md
@@ -1,6 +1,6 @@
---
title: "Operators"
-date: 2016-01-14
+date: 2017-03-14 05:06:04 UTC
parent: "SQL Reference"
---
You can use various types of operators in your Drill queries to perform
@@ -11,7 +11,7 @@ operations on your data.
You can use the following logical operators in your Drill queries:
* AND
- * BETWEEN
+ * BETWEEN (Includes end points. For example, if a query states WHERE age BETWEEN 10 AND 20, Drill returns both 10 and 20 in the result.)
* IN
* LIKE
* NOT
@@ -85,7 +85,7 @@ The following table shows the precedence of operators in decreasing order:
| IS NOT NULL | | test for not null |
| (any other) | left | all other native and user-defined operators |
| IN | | set membership |
-| BETWEEN | | range containment |
+| BETWEEN | | range containment, includes end points |
| OVERLAPS | | time interval overlap |
| LIKE ILIKE SIMILAR TO NOT SIMILAR TO | | string pattern matching |
| < > | | less than, greater than |
http://git-wip-us.apache.org/repos/asf/drill/blob/4fee2dc1/_docs/sql-reference/sql-commands/031-create-temporary-table-as.md
----------------------------------------------------------------------
diff --git a/_docs/sql-reference/sql-commands/031-create-temporary-table-as.md b/_docs/sql-reference/sql-commands/031-create-temporary-table-as.md
new file mode 100644
index 0000000..0191701
--- /dev/null
+++ b/_docs/sql-reference/sql-commands/031-create-temporary-table-as.md
@@ -0,0 +1,146 @@
+---
+title: "CREATE TEMPORARY TABLE AS"
+date:
+parent: "SQL Commands"
+---
+As of Drill 1.10, you can use the CREATE TEMPORARY TABLE AS (CTTAS) command to store the results of a query in a temporary table. You can reference the temporary table in subsequent queries within the same session, thereby improving query performance. Data written to the temporary table is not permanently stored on the filesystem. Drill automatically drops the temporary table once the session ends or the Drillbit process fails. Therefore, you do not have to manually drop the table.
+
+{% include startnote.html %}You cannot create a view over a temporary table.{% include endnote.html %}
+
+## Syntax
+
+ CREATE TEMPORARY TABLE name [ (column list) ][ PARTITION BY (column list) ] AS query
+
+## Parameters ##
+*name* - A directory name that is unique.
+
+*column list* - An optional list of column names or aliases in the new table. If specified, the number of columns for the temporary table must be the same as the number of columns in the SELECT statement of the query. When columns are not specified for the temporary table, the column names in the temporary table are set based on the source table column name or alias. If a column name has an alias, the alias is used.
+
+[PARTITION BY]({{site.baseurl}}/docs/partition-by-clause) - An optional parameter that can only be used to create temporary tables with the Parquet data format. When used, it must specify the column or list of columns to be partitioned.
+
+*query* - A SELECT statement that needs to include aliases for ambiguous column names, such as COLUMNS[0]. Using SELECT * is [not recommended]({{site.baseurl}}/docs/text-files-csv-tsv-psv/#tips-for-performant-querying) when selecting CSV, TSV, and PSV data.
+
+## Usage Notes
+
+###Workspace for Temporary Tables
+
+By default, Drill creates temporary tables within the default temporary workspace, `dfs.tmp`. The default temporary workspace must be writable, file-based, and point to a location that
+
+already exists, otherwise temporary tables creation will fail. You cannot create a temporary table outside of the default temporary workspace.
+
+Example dfs.tmp workspace in the dfs storage plugin:
+
+ {
+ "type": "file",
+ "enabled": true,
+ "connection": "maprfs:///",
+ "config": null,
+ "workspaces": {
+ "root": {
+ "location": "/",
+ "writable": false,
+ "defaultInputFormat": null
+ },
+ "tmp": {
+ "location": "/tmp",
+ "writable": true,
+ "defaultInputFormat": null
+ } ...
+To override the default temporary workspace, define `drill.exec.default_temporary_workspace` in the `drill-override.conf file` and then restart the Drillbit. For more information, see [Configuration Option Introduction]({{site.baseurl}}/docs/configuration-options-introduction/#system-options).
+
+To change the default temporary workspace connection or directory path, update the dfs storage plugin on the Storage page in the Drill Web Console. For example, dfs connection attribute can be `file:///` or `maprfs:///` and the `dfs.tmp` location attribute can be `/tmp` or `/tmp2`. For more information, see [Plugin Configuration Basics]({{site.baseurl}}/docs/plugin-configuration-basics).
+
+Note: When you run Drill in distributed mode, verify that the default temporary workspace connection points to a distributed filesystem. If temporary tables are created on the local filesystem, they can only be accessed by the local Drillbit that created the temporary table.
+###Setting the Storage Format
+The default storage format for temporary tables is parquet. However, you can create temporary tables in one of the following formats:
+
+* csv, tsv, psv
+* parquet
+* json
+
+To change the storage format, set the `store.format` option before you create the temporary table. For example, you can use the ALTER SESSION SET command to set the store.format option to JSON:
+
+ ALTER SESSION SET `store.format`='json';
+
+###Creation of Temporary Tables and User Access
+
+In general, the user that creates a temporary table can run queries against the table, as long as the session in which the table was created is active. Although temporary tables are actually directories, you query the temporary table directory as you would query a table.
+
+When you create a temporary table, Drill creates a temporary location named after the session ID to store temporary tables associated with the session. Drill writes temporary table files with masked filenames, such as 0_0_0.parquet, to temporary table directories within the session\u2019s temporary location. Internally, Drill masks each temporary table directory name. Therefore, when you submit a query with the temporary table name, Drill resolves the temporary table name to the internal, masked directory name.
+
+Drill creates the session and temporary table directories and files with the following permissions:
+
+* Only the owner can perform read, write, and execute operations on the folder directories.
+* Only the owner can create and read the files within the folders.
+
+###Authorization
+
+When authorization is enabled, the user that started the Drillbit will own the temporary tables created within that session. However, the user that created the table will have the ability to query the data. For example, the *mapr* user starts the Drillbit and *Sally* starts the Drill shell. When authorization is enabled, *Sally* can submit a query to create a temporary table and she can query the temporary table data. However, the *mapr* user is the owner of the temporary table directory and files. In this case, *Tom* can access the temporary table if he knows the full path to the temporary table directory and the session is still active.
+
+###Impersonation
+
+When impersonation is enabled, only the user that created the temporary table can query the temporary table. For example, if the Drillbit was started by the *mapr* user and user *Tom* starts the Drill shell and creates a temporary table, *Tom* will have permission to query the temporary table. *Tom* can also access temporary tables that he created in a different session as long as he has the full path to the temporary table directory and the session is still active. However, *Sally* cannot query the temporary table even if she has the full path to the temporary table directory and the session is active.
+
+###Selection of Tables
+
+When you mention a table name in a SELECT statement, any temporary table with that name takes priority over a table with the same name in the current workspace.
+
+For example, when you issue a SELECT statement on a table name that is common among the default temporary tables workspace and the current workspace, the temporary table is returned:
+
+ USE dfs.json;
+
+ SELECT * FROM donuts; //returns table from dfs.tmp SELECT* FROM dfs.json.donuts; //returns table from dfs.json
+
+###Drop a Temporary Table
+
+Once a session ends or the drillbit process fails, Drill will drop the session\u2019s temporary location which includes any associated temporary table directories and files. However, you can also drop a temporary table using the DROP TABLE command. When you drop a temporary table within a session, Drill deletes the temporary table directory and its files.
+
+When you drop a temporary table, it is not required to specify the workspace. Any temporary table with the specified table name takes priority over a table with the same name in the current workspace.
+
+In the following example, cust_promotions is the name of a persistent table under dfs.json and it is also the name of a temporary table.
+
+ USE dfs.json;
+ DROP TABLE cust_promotions;
+ +-------+-----------------------------------------------------+
+ | ok | summary |
+ +-------+-----------------------------------------------------+
+ | true | Temporary table [cust_promotions] dropped |
+ +-------+-----------------------------------------------------+
+
+Even though the workspace was set to dfs.json, the temporary table from dfs.tmp was dropped.
+
+##CTTAS Example
+
+In the following example, the user has .csv file with the following data:
+
+ "1","Bob","200","false"
+ "2","Kate","150","true"
+ "3","Tim","20","false"
+ "4","Roger","500","true"
+
+The first column is the customer id, the second column is the customer name, the third column is the sale amount, and the fourth column indicates if the customer got a discount:
+
+The user wants to create a temporary table which sums the amount of sales with a discount and sums the amount of sales without a discount. The user runs the following command to create a table which has one column with the sales amount and another column which indicates if the sales were associated with a discount or not:
+
+ CREATE TEMPORARY TABLE total_amount_by_discount as SELECT cast(columns[3] as boolean) discount, sum(cast(columns[2] as double)) total_amount FROM dfs.`/emp/amount.csv` GROUP BY columns[3];
+
+ +--------------+-------------------------------------+
+ | Fragment | Number of records written |
+ +-------------+--------------------------------------+
+ | 0_0 | 2 |
+ +-------------+--------------------------------------+
+
+The temporary table `total_amount_by_discount` was created in the default temporary workspace with the default storage format.
+
+To view the temporary table, the user can issue a SELECT statement on the entire table:
+
+ select * from total_amount_by_discount;
+
+ +-----------+-------------------+
+ | discount | total_amount |
+ +-----------+-------------------+
+ | false | 220.0 |
+ | true | 650.0 |
+ +-----------+-------------------+
+
+Within this session, the user can also query data from this temporary table using SELECT statements.
\ No newline at end of file
[6/7] drill git commit: docs for the Drill 1.10 release
Posted by br...@apache.org.
docs for the Drill 1.10 release
Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/0aae69b2
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/0aae69b2
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/0aae69b2
Branch: refs/heads/gh-pages
Commit: 0aae69b20226539288a31a04ceb147d4ccc042d6
Parents: b7b848e
Author: Bridget Bevens <bb...@maprtech.com>
Authored: Tue Mar 14 19:28:06 2017 -0700
Committer: Bridget Bevens <bb...@maprtech.com>
Committed: Tue Mar 14 19:28:06 2017 -0700
----------------------------------------------------------------------
...configuring-drill-to-read-web-server-logs.md | 86 ++++++++++++++++++++
_docs/getting-started/010-drill-introduction.md | 13 ++-
_docs/rn/001-1.10.0-rn.md | 15 ++--
blog/_posts/2017-03-15-drill-1.10-released.md | 29 +++++++
4 files changed, 134 insertions(+), 9 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/drill/blob/0aae69b2/_docs/configure-drill/079-configuring-drill-to-read-web-server-logs.md
----------------------------------------------------------------------
diff --git a/_docs/configure-drill/079-configuring-drill-to-read-web-server-logs.md b/_docs/configure-drill/079-configuring-drill-to-read-web-server-logs.md
new file mode 100644
index 0000000..0c53c59
--- /dev/null
+++ b/_docs/configure-drill/079-configuring-drill-to-read-web-server-logs.md
@@ -0,0 +1,86 @@
+---
+title: "Configuring Drill to Read Web Server Logs"
+date:
+parent: "Configure Drill"
+---
+
+As of version 1.9, Drill can natively ingest and query web server logs. To configure Drill to read server logs, you must modify the extensions section in the dfs storage plugin configuration, as shown below:
+
+ "httpd": {
+ "type": "httpd",
+ "logFormat": "%h %t \"%r\" %>s %b \"%{Referer}i\" \"%{user-agent}i\"",
+ "timestampFormat": null
+ }
+
+{% include startnote.html %}The `logFormat` section must match the format of the log files, otherwise Drill cannot correctly parse the logs.{% include endnote.html %}
+
+## HTTPD Format Strings
+The following table lists the fields that log files can include. The `timestampformat` is optional, but you can include a format for the time stamp and Drill will parse the times in the log files into Drill dates.
+
+
+|Format String | Variable Name |
+|--------------|---------------|
+|%a | connection.client.ip |
+|%{c}a | connection.client.peerip |
+|%A | connection.server.ip |
+|%B | response.body.bytes |
+|%b | response.body.bytesclf |
+|%{Foobar}C | request.cookies.* |
+|%D | server.process.time |
+|%{Foobar}e | server.environment.* |
+|%f | server.filename |
+|%h | connection.client.host |
+|%H | request.protocol |
+|%{Foobar}i | request.header. |
+|%k | connection.keepalivecount |
+|%l | connection.client.logname |
+|%L | request.errorlogid STRING
+|%m | request.method |
+|%{Foobar}n | server.module_note.* |
+|%{Foobar}o | response.header.* |
+|%p | request.server.port.canonical |
+|%{canonical}p | connection.server.port.canonical |
+|%{local}p | connection.server.port |
+|%{remote}p | connection.client.port |
+|%P | connection.server.child.processid |
+|%{pid}P | connection.server.child.processid |
+|%{tid}P | connection.server.child.threadid |
+|%{hextid}P | connection.server.child.hexthreadid |
+|%q | request.querystring |
+|%r | request.firstline |
+|%R | request.handler |
+|%s | request.status.original |
+|%>s | request.status.last |
+|%t | request.receive.time |
+|%{msec}t | request.receive.time.begin.msec |
+|%{begin:msec}t | request.receive.time.begin.msec |
+|%{end:msec}t | request.receive.time.end.msec |
+|%{usec}t | request.receive.time.begin.usec |
+|%{begin:usec}t | request.receive.time.begin.usec |
+|%{end:usec}t | request.receive.time.end.usec |
+|%{msec_frac}t | request.receive.time.begin.msec_frac |
+|%{begin:msec_frac}t | request.receive.time.begin.msec_frac TIME.EPOCH
+|%{end:msec_frac}t | request.receive.time.end.msec_frac |
+|%{usec_frac}t | request.receive.time.begin.usec_frac |
+|%{begin:usec_frac}t | request.receive.time.begin.usec_frac |
+|%{end:usec_frac}t | request.receive.time.end.usec_frac |
+|%T | response.server.processing.time |
+|%u | connection.client.user |
+|%U | request.urlpath |
+|%v | connection.server.name.canonical |
+|%V | connection.server.name |
+|%X | response.connection.status |
+|%I | request.bytes |
+|%O | response.bytes |
+|%{cookie}i | request.cookies |
+|%{set-cookie}o | response.cookies |
+|%{user-agent}i | request.user-agent |
+|%{referer}i | request.referer |
+
+## Additional Functionality
+In addition to reading raw log files, the following functions are also useful when analyzing log files:
+
+* `parse_url(<url>)`: This function accepts a URL as an argument and returns a map of the URL's protocol, authority, host, and path.
+* `parse_query( <query_string> )`: This function accepts a query string and returns a key/value pairing of the variables submitted in the request.
+
+A function that parses User Agent strings and returns a map of all the pertinent information is available at: https://github.com/cgivre/drill-useragent-function
http://git-wip-us.apache.org/repos/asf/drill/blob/0aae69b2/_docs/getting-started/010-drill-introduction.md
----------------------------------------------------------------------
diff --git a/_docs/getting-started/010-drill-introduction.md b/_docs/getting-started/010-drill-introduction.md
index 30c74d4..85e4eeb 100644
--- a/_docs/getting-started/010-drill-introduction.md
+++ b/_docs/getting-started/010-drill-introduction.md
@@ -1,6 +1,6 @@
---
title: "Drill Introduction"
-date: 2016-11-21 21:25:58 UTC
+date: 2017-03-15 02:28:07 UTC
parent: "Getting Started"
---
Drill is an Apache open-source SQL query engine for Big Data exploration.
@@ -10,6 +10,17 @@ applications, while still providing the familiarity and ecosystem of ANSI SQL,
the industry-standard query language. Drill provides plug-and-play integration
with existing Apache Hive and Apache HBase deployments.
+## What's New in Apache Drill 1.10
+
+Drill 1.10 provides the following new features and improvements:
+
+* Support for the [CREATE TEMPORARY TABLE AS (CTTAS)]({{site.baseurl}}/docs/create-temporary-table-as/) command.
+* A [JDBC connection option]({{site.baseurl}}/docs/using-the-jdbc-driver/#using-the-jdbc-url-format-for-a-direct-drillbit-connection) that improves fault tolerance when connecting directly to a Drill node from a client.
+* The [Web Console]({{site.baseurl}}/docs/identifying-multiple-drill-versions-in-a-cluster) displays the Drill version and additional query profile statistics.
+* Drill implicitly interprets the [INT96]({{site.baseurl}}/docs/parquet-format/#about-int96-support/) timestamp data type in Parquet files.
+* Support for Kerberos authentication between the client and drillbit.
+
+
## What's New in Apache Drill 1.9
Drill 1.9 provides the following new features:
http://git-wip-us.apache.org/repos/asf/drill/blob/0aae69b2/_docs/rn/001-1.10.0-rn.md
----------------------------------------------------------------------
diff --git a/_docs/rn/001-1.10.0-rn.md b/_docs/rn/001-1.10.0-rn.md
index be2cc1b..70a948c 100644
--- a/_docs/rn/001-1.10.0-rn.md
+++ b/_docs/rn/001-1.10.0-rn.md
@@ -7,15 +7,14 @@ parent: "Release Notes"
Today, we're happy to announce the availability of Drill 1.10.0. You can download it [here](https://drill.apache.org/download/).
-## New Features
-This release of Drill provides the following new features:
+## New Features and Improvements
+This release of Drill provides the following new features and improvements:
-- Support for the Create Temporary Table As (CTTAS) command.
-- JDBC connection option that improves fault tolerance when connecting directly to a Drill node from a client.
-- Drill writes standard Parquet date values and automatically detects and corrects any corrupted date values.
-- The Drill Web Console displays the Drill version running on each Drill node in the cluster.
-- Drill can implicitly interpret the INT96 timestamp data type in Parquet files.
-- Support for Kerberos authentication.
+- Support for the CREATE TEMPORARY TABLE AS (CTTAS) command.
+- A JDBC connection option that improves fault tolerance when connecting directly to a Drill node from a client.
+- The Web Console displays the Drill version and additional query profile statistics.
+- Drill implicitly interprets the INT96 timestamp data type in Parquet files.
+- Support for Kerberos authentication between the client and drillbit.
The following sections list additional bug fixes and improvements:
http://git-wip-us.apache.org/repos/asf/drill/blob/0aae69b2/blog/_posts/2017-03-15-drill-1.10-released.md
----------------------------------------------------------------------
diff --git a/blog/_posts/2017-03-15-drill-1.10-released.md b/blog/_posts/2017-03-15-drill-1.10-released.md
new file mode 100644
index 0000000..c46fa3b
--- /dev/null
+++ b/blog/_posts/2017-03-15-drill-1.10-released.md
@@ -0,0 +1,29 @@
+---
+layout: post
+title: "Drill 1.10 Released"
+code: drill-1.10-released
+excerpt: Apache Drill 1.10's highlights are: CTTAS, improved fault tolerance, Drill version and statistics in Web Console, implicit interpretation of INT96, and Kerberos authentication.
+authors: ["bbevens"]
+---
+
+Today, we're happy to announce the availability of Drill 1.10.0. You can download it [here](https://drill.apache.org/download/).
+
+The release provides the following bug fixes and improvements:
+
+## CTTAS
+The CREATE TEMPORARY TABLE AS (CTTAS) command stores the results of a query in a temporary table. See [CTTAS]({{site.baseurl}}/docs/create-temporary-table-as/).
+
+## Improved Fault Tolerance
+An optional `tries=<value>` parameter included in the JDBC connection string indicates the number of unique drillbits to which the client can try to connect. See [Using the JDBC URL Format for a Direct Drillbit Connection]({{site.baseurl}}/docs/using-the-jdbc-driver/#using-the-jdbc-url-format-for-a-direct-drillbit-connection).
+
+## Drill Version and Statistics in Web Console
+The Web Console displays the Drill version running on each Drill node in the cluster. See [Identifying Multiple Drill Versions in a Cluster]({{site.baseurl}}/docs/identifying-multiple-drill-versions-in-a-cluster).
+
+## Implicit Interpretation of INT96
+Drill implicitly interprets the INT96 timestamp data type in Parquet files when the new `store.parquet.int96_as_timestamp` option is enabled. See [About INT96 Support]({{site.baseurl}}/docs/parquet-format/#about-int96-support/).
+
+## Kerberos Authentication
+Drill supports Kerberos authentication between the client and drillbit.
+
+A complete list of JIRAs resolved in the 1.10.0 release can be found [here](https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12338769&styleName=Html&projectId=12313820&Create=Create&atl_token=A5KQ-2QAV-T4JA-FDED%7C264858c85b35c3b8ac66b0573aa7e88ffa802c9d%7Clin).
+