You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@drill.apache.org by br...@apache.org on 2017/03/15 02:43:33 UTC
[6/7] drill git commit: docs for the Drill 1.10 release
docs for the Drill 1.10 release
Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/0aae69b2
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/0aae69b2
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/0aae69b2
Branch: refs/heads/gh-pages
Commit: 0aae69b20226539288a31a04ceb147d4ccc042d6
Parents: b7b848e
Author: Bridget Bevens <bb...@maprtech.com>
Authored: Tue Mar 14 19:28:06 2017 -0700
Committer: Bridget Bevens <bb...@maprtech.com>
Committed: Tue Mar 14 19:28:06 2017 -0700
----------------------------------------------------------------------
...configuring-drill-to-read-web-server-logs.md | 86 ++++++++++++++++++++
_docs/getting-started/010-drill-introduction.md | 13 ++-
_docs/rn/001-1.10.0-rn.md | 15 ++--
blog/_posts/2017-03-15-drill-1.10-released.md | 29 +++++++
4 files changed, 134 insertions(+), 9 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/drill/blob/0aae69b2/_docs/configure-drill/079-configuring-drill-to-read-web-server-logs.md
----------------------------------------------------------------------
diff --git a/_docs/configure-drill/079-configuring-drill-to-read-web-server-logs.md b/_docs/configure-drill/079-configuring-drill-to-read-web-server-logs.md
new file mode 100644
index 0000000..0c53c59
--- /dev/null
+++ b/_docs/configure-drill/079-configuring-drill-to-read-web-server-logs.md
@@ -0,0 +1,86 @@
+---
+title: "Configuring Drill to Read Web Server Logs"
+date:
+parent: "Configure Drill"
+---
+
+As of version 1.9, Drill can natively ingest and query web server logs. To configure Drill to read server logs, you must modify the extensions section in the dfs storage plugin configuration, as shown below:
+
+ "httpd": {
+ "type": "httpd",
+ "logFormat": "%h %t \"%r\" %>s %b \"%{Referer}i\" \"%{user-agent}i\"",
+ "timestampFormat": null
+ }
+
+{% include startnote.html %}The `logFormat` section must match the format of the log files, otherwise Drill cannot correctly parse the logs.{% include endnote.html %}
+
+## HTTPD Format Strings
+The following table lists the fields that log files can include. The `timestampformat` is optional, but you can include a format for the time stamp and Drill will parse the times in the log files into Drill dates.
+
+
+|Format String | Variable Name |
+|--------------|---------------|
+|%a | connection.client.ip |
+|%{c}a | connection.client.peerip |
+|%A | connection.server.ip |
+|%B | response.body.bytes |
+|%b | response.body.bytesclf |
+|%{Foobar}C | request.cookies.* |
+|%D | server.process.time |
+|%{Foobar}e | server.environment.* |
+|%f | server.filename |
+|%h | connection.client.host |
+|%H | request.protocol |
+|%{Foobar}i | request.header. |
+|%k | connection.keepalivecount |
+|%l | connection.client.logname |
+|%L | request.errorlogid STRING
+|%m | request.method |
+|%{Foobar}n | server.module_note.* |
+|%{Foobar}o | response.header.* |
+|%p | request.server.port.canonical |
+|%{canonical}p | connection.server.port.canonical |
+|%{local}p | connection.server.port |
+|%{remote}p | connection.client.port |
+|%P | connection.server.child.processid |
+|%{pid}P | connection.server.child.processid |
+|%{tid}P | connection.server.child.threadid |
+|%{hextid}P | connection.server.child.hexthreadid |
+|%q | request.querystring |
+|%r | request.firstline |
+|%R | request.handler |
+|%s | request.status.original |
+|%>s | request.status.last |
+|%t | request.receive.time |
+|%{msec}t | request.receive.time.begin.msec |
+|%{begin:msec}t | request.receive.time.begin.msec |
+|%{end:msec}t | request.receive.time.end.msec |
+|%{usec}t | request.receive.time.begin.usec |
+|%{begin:usec}t | request.receive.time.begin.usec |
+|%{end:usec}t | request.receive.time.end.usec |
+|%{msec_frac}t | request.receive.time.begin.msec_frac |
+|%{begin:msec_frac}t | request.receive.time.begin.msec_frac TIME.EPOCH
+|%{end:msec_frac}t | request.receive.time.end.msec_frac |
+|%{usec_frac}t | request.receive.time.begin.usec_frac |
+|%{begin:usec_frac}t | request.receive.time.begin.usec_frac |
+|%{end:usec_frac}t | request.receive.time.end.usec_frac |
+|%T | response.server.processing.time |
+|%u | connection.client.user |
+|%U | request.urlpath |
+|%v | connection.server.name.canonical |
+|%V | connection.server.name |
+|%X | response.connection.status |
+|%I | request.bytes |
+|%O | response.bytes |
+|%{cookie}i | request.cookies |
+|%{set-cookie}o | response.cookies |
+|%{user-agent}i | request.user-agent |
+|%{referer}i | request.referer |
+
+## Additional Functionality
+In addition to reading raw log files, the following functions are also useful when analyzing log files:
+
+* `parse_url(<url>)`: This function accepts a URL as an argument and returns a map of the URL's protocol, authority, host, and path.
+* `parse_query( <query_string> )`: This function accepts a query string and returns a key/value pairing of the variables submitted in the request.
+
+A function that parses User Agent strings and returns a map of all the pertinent information is available at: https://github.com/cgivre/drill-useragent-function
http://git-wip-us.apache.org/repos/asf/drill/blob/0aae69b2/_docs/getting-started/010-drill-introduction.md
----------------------------------------------------------------------
diff --git a/_docs/getting-started/010-drill-introduction.md b/_docs/getting-started/010-drill-introduction.md
index 30c74d4..85e4eeb 100644
--- a/_docs/getting-started/010-drill-introduction.md
+++ b/_docs/getting-started/010-drill-introduction.md
@@ -1,6 +1,6 @@
---
title: "Drill Introduction"
-date: 2016-11-21 21:25:58 UTC
+date: 2017-03-15 02:28:07 UTC
parent: "Getting Started"
---
Drill is an Apache open-source SQL query engine for Big Data exploration.
@@ -10,6 +10,17 @@ applications, while still providing the familiarity and ecosystem of ANSI SQL,
the industry-standard query language. Drill provides plug-and-play integration
with existing Apache Hive and Apache HBase deployments.
+## What's New in Apache Drill 1.10
+
+Drill 1.10 provides the following new features and improvements:
+
+* Support for the [CREATE TEMPORARY TABLE AS (CTTAS)]({{site.baseurl}}/docs/create-temporary-table-as/) command.
+* A [JDBC connection option]({{site.baseurl}}/docs/using-the-jdbc-driver/#using-the-jdbc-url-format-for-a-direct-drillbit-connection) that improves fault tolerance when connecting directly to a Drill node from a client.
+* The [Web Console]({{site.baseurl}}/docs/identifying-multiple-drill-versions-in-a-cluster) displays the Drill version and additional query profile statistics.
+* Drill implicitly interprets the [INT96]({{site.baseurl}}/docs/parquet-format/#about-int96-support/) timestamp data type in Parquet files.
+* Support for Kerberos authentication between the client and drillbit.
+
+
## What's New in Apache Drill 1.9
Drill 1.9 provides the following new features:
http://git-wip-us.apache.org/repos/asf/drill/blob/0aae69b2/_docs/rn/001-1.10.0-rn.md
----------------------------------------------------------------------
diff --git a/_docs/rn/001-1.10.0-rn.md b/_docs/rn/001-1.10.0-rn.md
index be2cc1b..70a948c 100644
--- a/_docs/rn/001-1.10.0-rn.md
+++ b/_docs/rn/001-1.10.0-rn.md
@@ -7,15 +7,14 @@ parent: "Release Notes"
Today, we're happy to announce the availability of Drill 1.10.0. You can download it [here](https://drill.apache.org/download/).
-## New Features
-This release of Drill provides the following new features:
+## New Features and Improvements
+This release of Drill provides the following new features and improvements:
-- Support for the Create Temporary Table As (CTTAS) command.
-- JDBC connection option that improves fault tolerance when connecting directly to a Drill node from a client.
-- Drill writes standard Parquet date values and automatically detects and corrects any corrupted date values.
-- The Drill Web Console displays the Drill version running on each Drill node in the cluster.
-- Drill can implicitly interpret the INT96 timestamp data type in Parquet files.
-- Support for Kerberos authentication.
+- Support for the CREATE TEMPORARY TABLE AS (CTTAS) command.
+- A JDBC connection option that improves fault tolerance when connecting directly to a Drill node from a client.
+- The Web Console displays the Drill version and additional query profile statistics.
+- Drill implicitly interprets the INT96 timestamp data type in Parquet files.
+- Support for Kerberos authentication between the client and drillbit.
The following sections list additional bug fixes and improvements:
http://git-wip-us.apache.org/repos/asf/drill/blob/0aae69b2/blog/_posts/2017-03-15-drill-1.10-released.md
----------------------------------------------------------------------
diff --git a/blog/_posts/2017-03-15-drill-1.10-released.md b/blog/_posts/2017-03-15-drill-1.10-released.md
new file mode 100644
index 0000000..c46fa3b
--- /dev/null
+++ b/blog/_posts/2017-03-15-drill-1.10-released.md
@@ -0,0 +1,29 @@
+---
+layout: post
+title: "Drill 1.10 Released"
+code: drill-1.10-released
+excerpt: Apache Drill 1.10's highlights are: CTTAS, improved fault tolerance, Drill version and statistics in Web Console, implicit interpretation of INT96, and Kerberos authentication.
+authors: ["bbevens"]
+---
+
+Today, we're happy to announce the availability of Drill 1.10.0. You can download it [here](https://drill.apache.org/download/).
+
+The release provides the following bug fixes and improvements:
+
+## CTTAS
+The CREATE TEMPORARY TABLE AS (CTTAS) command stores the results of a query in a temporary table. See [CTTAS]({{site.baseurl}}/docs/create-temporary-table-as/).
+
+## Improved Fault Tolerance
+An optional `tries=<value>` parameter included in the JDBC connection string indicates the number of unique drillbits to which the client can try to connect. See [Using the JDBC URL Format for a Direct Drillbit Connection]({{site.baseurl}}/docs/using-the-jdbc-driver/#using-the-jdbc-url-format-for-a-direct-drillbit-connection).
+
+## Drill Version and Statistics in Web Console
+The Web Console displays the Drill version running on each Drill node in the cluster. See [Identifying Multiple Drill Versions in a Cluster]({{site.baseurl}}/docs/identifying-multiple-drill-versions-in-a-cluster).
+
+## Implicit Interpretation of INT96
+Drill implicitly interprets the INT96 timestamp data type in Parquet files when the new `store.parquet.int96_as_timestamp` option is enabled. See [About INT96 Support]({{site.baseurl}}/docs/parquet-format/#about-int96-support/).
+
+## Kerberos Authentication
+Drill supports Kerberos authentication between the client and drillbit.
+
+A complete list of JIRAs resolved in the 1.10.0 release can be found [here](https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12338769&styleName=Html&projectId=12313820&Create=Create&atl_token=A5KQ-2QAV-T4JA-FDED%7C264858c85b35c3b8ac66b0573aa7e88ffa802c9d%7Clin).
+