You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@drill.apache.org by dz...@apache.org on 2021/05/09 12:43:58 UTC
[drill] branch gh-pages updated (3396cd8 -> e758222)
This is an automated email from the ASF dual-hosted git repository.
dzamo pushed a change to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/drill.git.
from 3396cd8 Document BIT_AND and BIT_OR bitwise logic aggregate functions.
new cfd7ee0 Add LMT (last modified time) to the implicit columns returned for files.
new e758222 Document IP networking functions.
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
_data/docs.json | 112 ++++++++++++++++++---
.../005-querying-a-file-system-introduction.md | 61 ++++++-----
.../sql-functions/100-ip-net-functions.md | 26 +++++
3 files changed, 154 insertions(+), 45 deletions(-)
create mode 100644 _docs/sql-reference/sql-functions/100-ip-net-functions.md
[drill] 01/02: Add LMT (last modified time) to the implicit columns
returned for files.
Posted by dz...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dzamo pushed a commit to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/drill.git
commit cfd7ee0e520df68c2f16dea4406948ce71dc7dd0
Author: James Turton <ja...@somecomputer.xyz>
AuthorDate: Thu May 6 15:28:18 2021 +0200
Add LMT (last modified time) to the implicit columns returned for files.
---
.../005-querying-a-file-system-introduction.md | 61 +++++++++++-----------
1 file changed, 30 insertions(+), 31 deletions(-)
diff --git a/_docs/query-data/query-a-file-system/005-querying-a-file-system-introduction.md b/_docs/query-data/query-a-file-system/005-querying-a-file-system-introduction.md
index c9f4eb8..b45b882 100644
--- a/_docs/query-data/query-a-file-system/005-querying-a-file-system-introduction.md
+++ b/_docs/query-data/query-a-file-system/005-querying-a-file-system-introduction.md
@@ -3,6 +3,7 @@ title: "Querying a File System Introduction"
slug: "Querying a File System Introduction"
parent: "Querying a File System"
---
+
Files and directories are like standard SQL tables to Drill. You can specify a
file system "database" as a prefix in queries when you refer to objects across
databases. In Drill, a file system database consists of a storage plugin name
@@ -22,54 +23,52 @@ more information.
Drill supports the following file types:
- * Plain text files, including:
- * Comma-separated values (CSV, type: text)
- * Tab-separated values (TSV, type: text)
- * Pipe-separated values (PSV, type: text)
- * Structured data files:
- * [Avro]({{site.baseurl}}/docs/querying-avro-files/) (type: avro)
- * [JSON]({{site.baseurl}}/docs/querying-json-files/) (type: json)
- * [Parquet]({{site.baseurl}}/docs/querying-parquet-files/) (type: parquet)
+- Plain text files, including:
+ - Comma-separated values (CSV, type: text)
+ - Tab-separated values (TSV, type: text)
+ - Pipe-separated values (PSV, type: text)
+- Structured data files:
+ - [Avro]({{site.baseurl}}/docs/querying-avro-files/) (type: avro)
+ - [JSON]({{site.baseurl}}/docs/querying-json-files/) (type: json)
+ - [Parquet]({{site.baseurl}}/docs/querying-parquet-files/) (type: parquet)
The extensions for these file types must match the configuration settings for
your registered storage plugins. For example, PSV files may be defined with a
-`.tbl` extension, while CSV files are defined with a `.csv` extension.
+`.tbl` extension, while CSV files are defined with a `.csv` extension.
+
+## Implicit Columns
+
+Drill 1.8 introduces implicit columns. Implicit columns provide file information, such as the directory path to a file and the file extension. You can query implicit columns in files, directories, nested directories, and files.
+
+The following table lists the implicit columns available and their descriptions:
-## Implicit Columns
-Drill 1.8 introduces implicit columns. Implicit columns provide file information, such as the directory path to a file and the file extension. You can query implicit columns in files, directories, nested directories, and files.
+| Implicit Column Name | Description |
+| -------------------- | -------------------------------------------------------------------------------------- |
+| FQN | The fully qualified name. Contains the full path to the file, including the file name. |
+| FILEPATH | The full path to the file, without the file name. |
+| FILENAME | The file name with the file extension. Does not include the path to the file. |
+| SUFFIX | The file suffix without the dot (.) at the beginning. |
+| LMT | The last modification time of the file as recorded by the containing file system |
-The following table lists the implicit columns available and their descriptions:
-
-| Implicit Column Name | Description |
-|------------------------|--------------------------------------------------------------------------------------------|
-| FQN | The fully qualified name. Contains the full path to the file, including the file name. |
-| FILEPATH | The full path to the file, without the file name. |
-| FILENAME | The file name with the file extension. Does not include the path to the file. |
-| SUFFIX | The file suffix without the dot (.) at the beginning. |
+To access implicit columns, you must explicitly include the columns in a query, as shown in the following example:
-To access implicit columns, you must explicitly include the columns in a query, as shown in the following example:
+ 0: jdbc:drill:zk=local> SELECT fqn, filepath, filename, suffix FROM dfs.`/dev/data/files/test.csvh` LIMIT 1;
- 0: jdbc:drill:zk=local> SELECT fqn, filepath, filename, suffix FROM dfs.`/dev/data/files/test.csvh` LIMIT 1;
-
|-------------------------------------|--------------------------|---------------|----------------|
| fqn | filepath | filename | suffix |
|-------------------------------------|--------------------------|---------------|----------------|
| /dev/data/files/test.csvh | /dev/data/files | test.csvh | csvh |
- |-------------------------------------|--------------------------|---------------|----------------|
+ |-------------------------------------|--------------------------|---------------|----------------|
-{% include startnote.html %}If a table has a column with the same name as an implicit column, such as “suffix,” the implicit column overrides the table column.{% include endnote.html %}
+{% include startnote.html %}If a table has a column with the same name as an implicit column, such as “suffix,” the implicit column overrides the table column.{% include endnote.html %}
-If a column name has the same name as an implicit column, you can change the default implicit column name using the [ALTER SYSTEM|SESSION SET]({{site.baseurl}}/docs/alter-system/) command with the appropriate parameter, as shown in the following example:
+If a column name has the same name as an implicit column, you can change the default implicit column name using the [ALTER SYSTEM|SESSION SET]({{site.baseurl}}/docs/alter-system/) command with the appropriate parameter, as shown in the following example:
- ALTER SYSTEM SET `drill.exec.storage.implicit.suffix.column.label` = appendix;
+ ALTER SYSTEM SET `drill.exec.storage.implicit.suffix.column.label` = appendix;
-Use the following configuration options to change the default implicit column names:
+Use the following configuration options to change the default implicit column names:
drill.exec.storage.implicit.fqn.column.label
drill.exec.storage.implicit.filepath.column.label
drill.exec.storage.implicit.filename.column.label
drill.exec.storage.implicit.suffix.column.label
-
-
-
-
[drill] 02/02: Document IP networking functions.
Posted by dz...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dzamo pushed a commit to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/drill.git
commit e758222d6bd64274627c8f9c961f030a455a0f4b
Author: James Turton <ja...@somecomputer.xyz>
AuthorDate: Sun May 9 14:42:35 2021 +0200
Document IP networking functions.
---
_data/docs.json | 112 ++++++++++++++++++---
.../sql-functions/100-ip-net-functions.md | 26 +++++
2 files changed, 124 insertions(+), 14 deletions(-)
diff --git a/_data/docs.json b/_data/docs.json
index 2f127f6..5977f34 100644
--- a/_data/docs.json
+++ b/_data/docs.json
@@ -5827,6 +5827,27 @@
"title": "How to Partition Data",
"url": "/docs/how-to-partition-data/"
},
+ "IP Network functions": {
+ "breadcrumbs": [
+ {
+ "title": "SQL Functions",
+ "url": "/docs/sql-functions/"
+ },
+ {
+ "title": "SQL Reference",
+ "url": "/docs/sql-reference/"
+ }
+ ],
+ "children": [],
+ "next_title": "SQL Window Functions",
+ "next_url": "/docs/sql-window-functions/",
+ "parent": "SQL Functions",
+ "previous_title": "Time Series Analysis Functions",
+ "previous_url": "/docs/time-series-analysis-functions/",
+ "relative_path": "_docs/sql-reference/sql-functions/100-ip-net-functions.md",
+ "title": "IP Network functions",
+ "url": "/docs/ip-network-functions/"
+ },
"Identifying Multiple Drill Versions in a Cluster": {
"breadcrumbs": [
{
@@ -13042,14 +13063,35 @@
}
],
"children": [],
- "next_title": "SQL Window Functions",
- "next_url": "/docs/sql-window-functions/",
+ "next_title": "IP Network functions",
+ "next_url": "/docs/ip-network-functions/",
"parent": "SQL Functions",
"previous_title": "GIS functions",
"previous_url": "/docs/gis-functions/",
"relative_path": "_docs/sql-reference/sql-functions/090-time-series-analysis-functions.md",
"title": "Time Series Analysis Functions",
"url": "/docs/time-series-analysis-functions/"
+ },
+ {
+ "breadcrumbs": [
+ {
+ "title": "SQL Functions",
+ "url": "/docs/sql-functions/"
+ },
+ {
+ "title": "SQL Reference",
+ "url": "/docs/sql-reference/"
+ }
+ ],
+ "children": [],
+ "next_title": "SQL Window Functions",
+ "next_url": "/docs/sql-window-functions/",
+ "parent": "SQL Functions",
+ "previous_title": "Time Series Analysis Functions",
+ "previous_url": "/docs/time-series-analysis-functions/",
+ "relative_path": "_docs/sql-reference/sql-functions/100-ip-net-functions.md",
+ "title": "IP Network functions",
+ "url": "/docs/ip-network-functions/"
}
],
"next_title": "About SQL Function Examples",
@@ -13489,14 +13531,35 @@
}
],
"children": [],
- "next_title": "SQL Window Functions",
- "next_url": "/docs/sql-window-functions/",
+ "next_title": "IP Network functions",
+ "next_url": "/docs/ip-network-functions/",
"parent": "SQL Functions",
"previous_title": "GIS functions",
"previous_url": "/docs/gis-functions/",
"relative_path": "_docs/sql-reference/sql-functions/090-time-series-analysis-functions.md",
"title": "Time Series Analysis Functions",
"url": "/docs/time-series-analysis-functions/"
+ },
+ {
+ "breadcrumbs": [
+ {
+ "title": "SQL Functions",
+ "url": "/docs/sql-functions/"
+ },
+ {
+ "title": "SQL Reference",
+ "url": "/docs/sql-reference/"
+ }
+ ],
+ "children": [],
+ "next_title": "SQL Window Functions",
+ "next_url": "/docs/sql-window-functions/",
+ "parent": "SQL Functions",
+ "previous_title": "Time Series Analysis Functions",
+ "previous_url": "/docs/time-series-analysis-functions/",
+ "relative_path": "_docs/sql-reference/sql-functions/100-ip-net-functions.md",
+ "title": "IP Network functions",
+ "url": "/docs/ip-network-functions/"
}
],
"next_title": "About SQL Function Examples",
@@ -13625,8 +13688,8 @@
"next_title": "SQL Window Functions Introduction",
"next_url": "/docs/sql-window-functions-introduction/",
"parent": "SQL Reference",
- "previous_title": "Time Series Analysis Functions",
- "previous_url": "/docs/time-series-analysis-functions/",
+ "previous_title": "IP Network functions",
+ "previous_url": "/docs/ip-network-functions/",
"relative_path": "_docs/sql-reference/055-sql-window-functions.md",
"title": "SQL Window Functions",
"url": "/docs/sql-window-functions/"
@@ -14720,8 +14783,8 @@
"next_title": "SQL Window Functions Introduction",
"next_url": "/docs/sql-window-functions-introduction/",
"parent": "SQL Reference",
- "previous_title": "Time Series Analysis Functions",
- "previous_url": "/docs/time-series-analysis-functions/",
+ "previous_title": "IP Network functions",
+ "previous_url": "/docs/ip-network-functions/",
"relative_path": "_docs/sql-reference/055-sql-window-functions.md",
"title": "SQL Window Functions",
"url": "/docs/sql-window-functions/"
@@ -15805,8 +15868,8 @@
}
],
"children": [],
- "next_title": "SQL Window Functions",
- "next_url": "/docs/sql-window-functions/",
+ "next_title": "IP Network functions",
+ "next_url": "/docs/ip-network-functions/",
"parent": "SQL Functions",
"previous_title": "GIS functions",
"previous_url": "/docs/gis-functions/",
@@ -21378,14 +21441,35 @@
}
],
"children": [],
- "next_title": "SQL Window Functions",
- "next_url": "/docs/sql-window-functions/",
+ "next_title": "IP Network functions",
+ "next_url": "/docs/ip-network-functions/",
"parent": "SQL Functions",
"previous_title": "GIS functions",
"previous_url": "/docs/gis-functions/",
"relative_path": "_docs/sql-reference/sql-functions/090-time-series-analysis-functions.md",
"title": "Time Series Analysis Functions",
"url": "/docs/time-series-analysis-functions/"
+ },
+ {
+ "breadcrumbs": [
+ {
+ "title": "SQL Functions",
+ "url": "/docs/sql-functions/"
+ },
+ {
+ "title": "SQL Reference",
+ "url": "/docs/sql-reference/"
+ }
+ ],
+ "children": [],
+ "next_title": "SQL Window Functions",
+ "next_url": "/docs/sql-window-functions/",
+ "parent": "SQL Functions",
+ "previous_title": "Time Series Analysis Functions",
+ "previous_url": "/docs/time-series-analysis-functions/",
+ "relative_path": "_docs/sql-reference/sql-functions/100-ip-net-functions.md",
+ "title": "IP Network functions",
+ "url": "/docs/ip-network-functions/"
}
],
"next_title": "About SQL Function Examples",
@@ -21514,8 +21598,8 @@
"next_title": "SQL Window Functions Introduction",
"next_url": "/docs/sql-window-functions-introduction/",
"parent": "SQL Reference",
- "previous_title": "Time Series Analysis Functions",
- "previous_url": "/docs/time-series-analysis-functions/",
+ "previous_title": "IP Network functions",
+ "previous_url": "/docs/ip-network-functions/",
"relative_path": "_docs/sql-reference/055-sql-window-functions.md",
"title": "SQL Window Functions",
"url": "/docs/sql-window-functions/"
diff --git a/_docs/sql-reference/sql-functions/100-ip-net-functions.md b/_docs/sql-reference/sql-functions/100-ip-net-functions.md
new file mode 100644
index 0000000..019e10a
--- /dev/null
+++ b/_docs/sql-reference/sql-functions/100-ip-net-functions.md
@@ -0,0 +1,26 @@
+---
+title: "IP Networking functions"
+slug: "IP Networking functions"
+parent: "SQL Functions"
+---
+
+**Introduced in release:** 1.12.
+
+A set of functions for common computations on IP network addresses.
+
+| Function | Output | Description |
+| --------------------------- | ------- | --------------------------------------------------------- |
+| inet_aton(<ip>) | INT | Converts an IPv4 address into an integer |
+| inet_ntoa( <int>) | VARCHAR | Converts an integer IP into dotted decimal notation |
+| in_network( <ip>,<cidr> ) | BOOLEAN | Returns true if the IP address is in the given CIDR block |
+| address_count( <cidr> ) | INT | Returns the number of IPs in a given CIDR block |
+| broadcast_address( <cidr> ) | VARCHAR | Returns the broadcast address for a given CIDR block |
+| netmask(<cidr> ) | VARCHAR | Returns the netmask for a given CIDR block |
+| low_address(<cidr>) | VARCHAR | Returns the first address in a given CIDR block |
+| high_address(<cidr>) | VARCHAR | Returns the last address in a given CIDR block |
+| url_encode( <url> ) | VARCHAR | Returns a URL encoded string |
+| url_decode( <url> ) | VARCHAR | Decodes a URL encoded string |
+| is_valid_IP(<ip>) | BOOLEAN | Returns true if the IP is a valid IP address |
+| is_private_ip(<ip>) | BOOLEAN | Returns true if the IP is a private IPv4 address |
+| is_valid_IPv4(<ip>) | BOOLEAN | Returns true if the IP is a valid IPv4 address |
+| is_valid_IPv6(<ip>) | BOOLEAN | Returns true if the IP is a valid IPv6 address |