You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@drill.apache.org by br...@apache.org on 2017/03/31 21:36:04 UTC
drill git commit: move httpd content to storage plugin content
Repository: drill
Updated Branches:
refs/heads/gh-pages 040d4e1aa -> e10bbd81b
move httpd content to storage plugin content
Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/e10bbd81
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/e10bbd81
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/e10bbd81
Branch: refs/heads/gh-pages
Commit: e10bbd81bc7d99a41e9e064f868c474684a5b1e8
Parents: 040d4e1
Author: Bridget Bevens <bb...@maprtech.com>
Authored: Fri Mar 31 14:34:13 2017 -0700
Committer: Bridget Bevens <bb...@maprtech.com>
Committed: Fri Mar 31 14:34:13 2017 -0700
----------------------------------------------------------------------
...configuring-drill-to-read-web-server-logs.md | 86 --------------------
.../plugins/111-httpd-storage-plugin.md | 86 ++++++++++++++++++++
2 files changed, 86 insertions(+), 86 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/drill/blob/e10bbd81/_docs/configure-drill/079-configuring-drill-to-read-web-server-logs.md
----------------------------------------------------------------------
diff --git a/_docs/configure-drill/079-configuring-drill-to-read-web-server-logs.md b/_docs/configure-drill/079-configuring-drill-to-read-web-server-logs.md
deleted file mode 100644
index 95a841e..0000000
--- a/_docs/configure-drill/079-configuring-drill-to-read-web-server-logs.md
+++ /dev/null
@@ -1,86 +0,0 @@
----
-title: "Configuring Drill to Read Web Server Logs"
-date: 2017-03-15 19:34:55 UTC
-parent: "Configure Drill"
----
-
-As of version 1.9, Drill can natively ingest and query web server logs. To configure Drill to read server logs, you must modify the extensions section in the dfs storage plugin configuration, as shown below:
-
- "httpd": {
- "type": "httpd",
- "logFormat": "%h %t \"%r\" %>s %b \"%{Referer}i\" \"%{user-agent}i\"",
- "timestampFormat": null
- }
-
-{% include startnote.html %}The logFormat section must match the format of the log files, otherwise Drill cannot correctly parse the logs.{% include endnote.html %}
-
-## HTTPD Format Strings
-The following table lists the fields that log files can include. The `timestampformat` is optional, but you can include a format for the time stamp and Drill will parse the times in the log files into Drill dates.
-
-
-|Format String | Variable Name |
-|--------------|---------------|
-|%a | connection.client.ip |
-|%{c}a | connection.client.peerip |
-|%A | connection.server.ip |
-|%B | response.body.bytes |
-|%b | response.body.bytesclf |
-|%{Foobar}C | request.cookies.* |
-|%D | server.process.time |
-|%{Foobar}e | server.environment.* |
-|%f | server.filename |
-|%h | connection.client.host |
-|%H | request.protocol |
-|%{Foobar}i | request.header. |
-|%k | connection.keepalivecount |
-|%l | connection.client.logname |
-|%L | request.errorlogid STRING
-|%m | request.method |
-|%{Foobar}n | server.module_note.* |
-|%{Foobar}o | response.header.* |
-|%p | request.server.port.canonical |
-|%{canonical}p | connection.server.port.canonical |
-|%{local}p | connection.server.port |
-|%{remote}p | connection.client.port |
-|%P | connection.server.child.processid |
-|%{pid}P | connection.server.child.processid |
-|%{tid}P | connection.server.child.threadid |
-|%{hextid}P | connection.server.child.hexthreadid |
-|%q | request.querystring |
-|%r | request.firstline |
-|%R | request.handler |
-|%s | request.status.original |
-|%>s | request.status.last |
-|%t | request.receive.time |
-|%{msec}t | request.receive.time.begin.msec |
-|%{begin:msec}t | request.receive.time.begin.msec |
-|%{end:msec}t | request.receive.time.end.msec |
-|%{usec}t | request.receive.time.begin.usec |
-|%{begin:usec}t | request.receive.time.begin.usec |
-|%{end:usec}t | request.receive.time.end.usec |
-|%{msec_frac}t | request.receive.time.begin.msec_frac |
-|%{begin:msec_frac}t | request.receive.time.begin.msec_frac TIME.EPOCH
-|%{end:msec_frac}t | request.receive.time.end.msec_frac |
-|%{usec_frac}t | request.receive.time.begin.usec_frac |
-|%{begin:usec_frac}t | request.receive.time.begin.usec_frac |
-|%{end:usec_frac}t | request.receive.time.end.usec_frac |
-|%T | response.server.processing.time |
-|%u | connection.client.user |
-|%U | request.urlpath |
-|%v | connection.server.name.canonical |
-|%V | connection.server.name |
-|%X | response.connection.status |
-|%I | request.bytes |
-|%O | response.bytes |
-|%{cookie}i | request.cookies |
-|%{set-cookie}o | response.cookies |
-|%{user-agent}i | request.user-agent |
-|%{referer}i | request.referer |
-
-## Additional Functionality
-In addition to reading raw log files, the following functions are also useful when analyzing log files:
-
-* `parse_url(<url>)`: This function accepts a URL as an argument and returns a map of the URL's protocol, authority, host, and path.
-* `parse_query( <query_string> )`: This function accepts a query string and returns a key/value pairing of the variables submitted in the request.
-
-A function that parses User Agent strings and returns a map of all the pertinent information is available at: https://github.com/cgivre/drill-useragent-function
http://git-wip-us.apache.org/repos/asf/drill/blob/e10bbd81/_docs/connect-a-data-source/plugins/111-httpd-storage-plugin.md
----------------------------------------------------------------------
diff --git a/_docs/connect-a-data-source/plugins/111-httpd-storage-plugin.md b/_docs/connect-a-data-source/plugins/111-httpd-storage-plugin.md
new file mode 100644
index 0000000..1aa5184
--- /dev/null
+++ b/_docs/connect-a-data-source/plugins/111-httpd-storage-plugin.md
@@ -0,0 +1,86 @@
+---
+title: "HTTPD Storage Plugin"
+date: 2017-03-15 19:34:55 UTC
+parent: "Configure Drill"
+---
+
+As of version 1.9, Drill can natively ingest and query web server logs. To configure Drill to read server logs, you must modify the extensions section in the dfs storage plugin configuration, as shown below:
+
+ "httpd": {
+ "type": "httpd",
+ "logFormat": "%h %t \"%r\" %>s %b \"%{Referer}i\" \"%{user-agent}i\"",
+ "timestampFormat": null
+ }
+
+{% include startnote.html %}The logFormat section must match the format of the log files, otherwise Drill cannot correctly parse the logs.{% include endnote.html %}
+
+## HTTPD Format Strings
+The following table lists the fields that log files can include. The `timestampformat` is optional, but you can include a format for the time stamp and Drill will parse the times in the log files into Drill dates.
+
+
+|Format String | Variable Name |
+|--------------|---------------|
+|%a | connection.client.ip |
+|%{c}a | connection.client.peerip |
+|%A | connection.server.ip |
+|%B | response.body.bytes |
+|%b | response.body.bytesclf |
+|%{Foobar}C | request.cookies.* |
+|%D | server.process.time |
+|%{Foobar}e | server.environment.* |
+|%f | server.filename |
+|%h | connection.client.host |
+|%H | request.protocol |
+|%{Foobar}i | request.header. |
+|%k | connection.keepalivecount |
+|%l | connection.client.logname |
+|%L | request.errorlogid STRING
+|%m | request.method |
+|%{Foobar}n | server.module_note.* |
+|%{Foobar}o | response.header.* |
+|%p | request.server.port.canonical |
+|%{canonical}p | connection.server.port.canonical |
+|%{local}p | connection.server.port |
+|%{remote}p | connection.client.port |
+|%P | connection.server.child.processid |
+|%{pid}P | connection.server.child.processid |
+|%{tid}P | connection.server.child.threadid |
+|%{hextid}P | connection.server.child.hexthreadid |
+|%q | request.querystring |
+|%r | request.firstline |
+|%R | request.handler |
+|%s | request.status.original |
+|%>s | request.status.last |
+|%t | request.receive.time |
+|%{msec}t | request.receive.time.begin.msec |
+|%{begin:msec}t | request.receive.time.begin.msec |
+|%{end:msec}t | request.receive.time.end.msec |
+|%{usec}t | request.receive.time.begin.usec |
+|%{begin:usec}t | request.receive.time.begin.usec |
+|%{end:usec}t | request.receive.time.end.usec |
+|%{msec_frac}t | request.receive.time.begin.msec_frac |
+|%{begin:msec_frac}t | request.receive.time.begin.msec_frac TIME.EPOCH
+|%{end:msec_frac}t | request.receive.time.end.msec_frac |
+|%{usec_frac}t | request.receive.time.begin.usec_frac |
+|%{begin:usec_frac}t | request.receive.time.begin.usec_frac |
+|%{end:usec_frac}t | request.receive.time.end.usec_frac |
+|%T | response.server.processing.time |
+|%u | connection.client.user |
+|%U | request.urlpath |
+|%v | connection.server.name.canonical |
+|%V | connection.server.name |
+|%X | response.connection.status |
+|%I | request.bytes |
+|%O | response.bytes |
+|%{cookie}i | request.cookies |
+|%{set-cookie}o | response.cookies |
+|%{user-agent}i | request.user-agent |
+|%{referer}i | request.referer |
+
+## Additional Functionality
+In addition to reading raw log files, the following functions are also useful when analyzing log files:
+
+* `parse_url(<url>)`: This function accepts a URL as an argument and returns a map of the URL's protocol, authority, host, and path.
+* `parse_query( <query_string> )`: This function accepts a query string and returns a key/value pairing of the variables submitted in the request.
+
+A function that parses User Agent strings and returns a map of all the pertinent information is available at: https://github.com/cgivre/drill-useragent-function