You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@drill.apache.org by br...@apache.org on 2019/04/09 01:23:54 UTC
[drill] branch gh-pages updated: add syslog format plugin doc
This is an automated email from the ASF dual-hosted git repository.
bridgetb pushed a commit to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/drill.git
The following commit(s) were added to refs/heads/gh-pages by this push:
new 0a0b93b add syslog format plugin doc
0a0b93b is described below
commit 0a0b93bcea4ba8851001ab93c11bd41362cafb0b
Author: Bridget Bevens <bb...@maprtech.com>
AuthorDate: Mon Apr 8 18:22:48 2019 -0700
add syslog format plugin doc
---
.../035-plugin-configuration-basics.md | 4 +-
.../plugins/116-sys-log-format-plugin.md | 58 ++++++++++++++++++++++
2 files changed, 60 insertions(+), 2 deletions(-)
diff --git a/_docs/connect-a-data-source/035-plugin-configuration-basics.md b/_docs/connect-a-data-source/035-plugin-configuration-basics.md
index 2925d22..d058867 100644
--- a/_docs/connect-a-data-source/035-plugin-configuration-basics.md
+++ b/_docs/connect-a-data-source/035-plugin-configuration-basics.md
@@ -1,6 +1,6 @@
---
title: "Plugin Configuration Basics"
-date: 2019-02-13
+date: 2019-04-09
parent: "Storage Plugin Configuration"
---
There are several ways you can configure storage plugins. For example, you can configure storage plugins in the Drill Web UI, using REST API, or through configuration files. See [Configuring Storage Plugins]({{site.baseurl}}/docs/configuring-storage-plugins/) for more information.
@@ -79,7 +79,7 @@ The following table describes the attributes you configure for storage plugins i
</tr>
<tr>
<td>"formats" . . . "type"</td>
- <td>"pcap"<br>"pcapng"<br>"text"<br>"parquet"<br>"json"<br>"maprdb"<br>"avro"<br>"image"<br>"sequencefile"<br>"httpd"</td>
+ <td>"pcap"<br>"pcapng"<br>"text"<br>"parquet"<br>"json"<br>"maprdb"<br>"avro"<br>"image"<br>"sequencefile"<br>"httpd"<br>"[syslog]({{site.baseurl}}/docs/sys-log-format-plugin/)"</td>
<td>yes</td>
<td>Format type. You can define two formats, csv and psv, as type "Text", but having different delimiters. </td>
</tr>
diff --git a/_docs/connect-a-data-source/plugins/116-sys-log-format-plugin.md b/_docs/connect-a-data-source/plugins/116-sys-log-format-plugin.md
new file mode 100644
index 0000000..bb7211b
--- /dev/null
+++ b/_docs/connect-a-data-source/plugins/116-sys-log-format-plugin.md
@@ -0,0 +1,58 @@
+---
+title: "Syslog Format Plugin"
+date: 2019-04-08
+parent: "Connect a Data Source"
+---
+
+Starting in Drill 1.16, Drill provides a syslog format plugin, which enables Drill to query syslog formatted data as specified in RFC-5424, as shown:
+
+ <165>1 2003-10-11T22:14:15.003Z mymachine.example.com evntslog - ID47 [exampleSDID@32473 iut="3" eventSource="Application" eventID="1011"][examplePriority@32473 class="high"]
+
+## Configuration Options
+This syslog format plugin has the following configuration options:
+
+
+
+- **maxErrors**
+Sets the maximum number of malformatted lines that the format plugin will tolerate before throwing an error and halting execution.
+- **flattenStructuredData**
+Syslog data optionally contains a series of key/value pairs known as the structured data. By default, Drill will parse these into a map.
+
+ "syslog": {
+ "type": "syslog",
+ "extensions": [ "syslog" ],
+ "maxErrors": 10,
+ "flattenStructuredData": false
+ }
+
+## Fields
+
+In terms of data types, the `event_date` field is a datetime, the `severity_code`, `facility_code`, and `proc_id` are integers and all other fields are VARCHARs.
+
+**Note:** All fields, with the exception of the `event_date` field, are not required; therefore, all fields may not be present at all times.
+
+- **event_date**
+This is the time of the event
+- **severity_code**
+The severity code of the event
+- **facility_code**
+The facility code of the incident
+- **severity**
+The severity of the event
+- **facility**
+- **ip**
+The IP address or hostname of the source machine
+- **app_name**
+The name of the application that is generating the event
+- **proc_id**
+The process ID of the event that generated the event
+- **msg_id**
+The identifier of the message
+- **message**
+The actual message text of the event
+- **raw**
+The full text of the event
+
+## Structured Data
+
+Syslog data can contain a list of key/value pairs which Drill will extract in a field called `structured_data`. This field is a Drill map.
\ No newline at end of file