You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@drill.apache.org by br...@apache.org on 2018/12/28 06:32:07 UTC
[drill-site] branch asf-site updated: doc updates and edits for
Drill 1.15
This is an automated email from the ASF dual-hosted git repository.
bridgetb pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/drill-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 1b4ce95 doc updates and edits for Drill 1.15
1b4ce95 is described below
commit 1b4ce9583e1e4099552063dbcf4fb5d59a86bd1c
Author: Bridget Bevens <bb...@maprtech.com>
AuthorDate: Thu Dec 27 22:31:55 2018 -0800
doc updates and edits for Drill 1.15
---
.../index.html | 28 +++++++--
docs/configuring-the-drill-shell/index.html | 30 +++++++---
docs/date-time-functions-and-arithmetic/index.html | 67 ++++++++++++----------
docs/start-up-options/index.html | 13 ++++-
feed.xml | 4 +-
5 files changed, 94 insertions(+), 48 deletions(-)
diff --git a/docs/configuring-custom-acls-to-secure-znodes/index.html b/docs/configuring-custom-acls-to-secure-znodes/index.html
index f6177a3..f82bd00 100644
--- a/docs/configuring-custom-acls-to-secure-znodes/index.html
+++ b/docs/configuring-custom-acls-to-secure-znodes/index.html
@@ -1293,17 +1293,21 @@
</div>
- Dec 8, 2018
+ Dec 28, 2018
<link href="/css/docpage.css" rel="stylesheet" type="text/css">
<div class="int_text" align="left">
- <p>Drill uses ZooKeeper for dynamic service discovery in a cluster; ZooKeeper uses the Curator framework and Service Discovery recipe to discover services. In addition to discovering services, Drill uses ZooKeeper to store certain cluster-level configuration and query profile information in znodes. A znode is an internal data tree in ZooKeeper that stores coordination- and execution-related information. Each time a Drillbit starts up and establish a new session with Zookeeper (us [...]
+ <p>Drill uses the Curator Framework to interact with ZooKeeper to discover services in a cluster. In addition to discovering services, Drill uses ZooKeeper to store certain cluster-level configuration and query profile information in znodes. A znode is an internal data tree in ZooKeeper that stores coordination- and execution-related information. If information in the znodes is not properly secured, cluster privacy and/or security is compromised. </p>
-<p>ZooKeeper uses ACLs (access control lists) to control access to znodes and secure the information they store. ACLs specify sets of ids and permissions that are associated with the ids. The ZooKeeper ACLs are set such that only the Drillbit process user can access (create, delete, read, write, administer) all the ZooKeeper nodes in a Drill cluster, except for the service discovery znodes. When a Drillbit shuts-down, the ZooKeeper session ends and the znode is removed. </p>
+<p>Drill allows users to create a custom ACL (access control list) on the znodes to secure data. ACLs specify sets of ids and permissions that are associated with the ids. ZooKeeper uses ACLs to control access to znodes and secure the information they store. </p>
-<p>Prior to Drill 1.15, ZooKeeper ACLs in secure and unsecure clusters were set to [world:all], meaning that all users had create, delete, read, write, and administrator access to the zknodes. Starting in Drill 1.15, ACLs in unsecure clusters are set to [world:all]. ACLs in secure clusters are set to [authid: all], which provides only the authenticated user that created the znode with full access. Discovery znodes (znodes with the list of Drillbits) have an additional ACL set to [world:r [...]
+<p>Prior to Drill 1.15, ZooKeeper ACLs in secure and unsecure clusters were set to [world:all], meaning that all users had create, delete, read, write, and administrator access to the zknodes. Starting in Drill 1.15, ACLs in unsecure clusters are set to [world:all]. ACLs in secure clusters are set to [authid: all], which provides only the authenticated user that created the znode with full access. Discovery znodes (znodes with the list of Drillbits) have an additional ACL set to [world:r [...]
+
+<ul>
+<li>View the <a href="https://github.com/apache/drill/blob/master/distribution/src/resources/drill-override-example.conf">drill-override-example.conf</a> file to see example ACL configurations.</li>
+</ul>
<h2 id="securing-znodes">Securing znodes</h2>
@@ -1320,7 +1324,21 @@
<li><p>Restart Drill.</p></li>
</ol>
-<p>When you restart Drill, the ACL, as mentioned in your custom class, is applied to the znode created when Drill starts.</p>
+<p>When you restart Drill, the ACL, as mentioned in your custom class, is applied to the znode created when Drill starts. </p>
+
+<hr>
+
+<p><strong>NOTE</strong><br>
+Existing ACLs for persistent znodes will not be affected if a Drillbit is restarted with a different ACL setting. ACLs are applied only at znode creation time, and Drill does not recreate any znode that is already present. If you want to change an ACL for existing znodes, connect to the ZooKeeper server using zkCli and then use option a or b, as described: </p>
+
+<ul>
+<li><p>a) Shutdown Drillbits, delete the persistent znodes, change the ACL settings and then restart the Drillbit </p></li>
+<li><p>b) Manually change the ACLs on the existing znodes to reflect the new ACL settings, using the setAcl command in the zkCli.</p></li>
+</ul>
+
+<p>For either option to work, an authenticated connection between the zkCli and ZooKeeper Server must be established. </p>
+
+<hr>
<p>For additional information, refer to: </p>
diff --git a/docs/configuring-the-drill-shell/index.html b/docs/configuring-the-drill-shell/index.html
index 30c076c..83120f5 100644
--- a/docs/configuring-the-drill-shell/index.html
+++ b/docs/configuring-the-drill-shell/index.html
@@ -1291,7 +1291,7 @@
</div>
- Dec 27, 2018
+ Dec 28, 2018
<link href="/css/docpage.css" rel="stylesheet" type="text/css">
@@ -1299,7 +1299,7 @@
<p>Drill uses SQLLine as the Drill shell. SQLLine is a pure-Java console-based utility for connecting to relational databases and running SQL commands. </p>
-<p>Starting in Drill 1.15, Drill uses SQLLine 1.6, which you can customize through the Drill <a href="/docs/configuring-the-drill-shell/#customizing-sqlline-in-the-drill-sqlline-override-conf-file">configuration file, drill-sqlline-override.conf</a>. Before installing and running Drill with SQLLine 1.6, delete the old SQLLine history file The history file is located in the following location: </p>
+<p>Starting in Drill 1.15, Drill uses SQLLine 1.6, which you can customize through the Drill <a href="/docs/configuring-the-drill-shell/#customizing-sqlline-in-the-drill-sqlline-override-conf-file">configuration file, drill-sqlline-override.conf</a>. Before installing and running Drill with SQLLine 1.6, delete the old SQLLine history file located in: </p>
<ul>
<li>$HOME/.sqlline/history (UNIX, Linux, Mac OS)</li>
@@ -1312,12 +1312,14 @@
<h2 id="drill-shell-commands">Drill Shell Commands</h2>
-<p>The following table lists the Drill shell commands that you can run from the command line:</p>
+<p>The following table lists some of the Drill shell commands that you can run from the command line. To see a complete list of shell commands issue the <code>!help</code> command. </p>
+
+<p><strong>Note:</strong> Some new shell commands, such as <code>!reset</code>, and key-strokes were introduced when Drill upgraded to SQLLine 1.6 (Drill 1.15 and later); however, not all of the commands are supported. </p>
<table><thead>
<tr>
-<th>Command</th>
-<th>Description</th>
+<th><strong>Command</strong></th>
+<th>*<em>Description *</em></th>
</tr>
</thead><tbody>
<tr>
@@ -1369,6 +1371,18 @@
<td>Record all output to the specified file.</td>
</tr>
<tr>
+<td></td>
+<td></td>
+</tr>
+<tr>
+<td>!reset</td>
+<td>Reset a sqlline variable</td>
+</tr>
+<tr>
+<td></td>
+<td></td>
+</tr>
+<tr>
<td>!run</td>
<td>Run a script from the specified file.</td>
</tr>
@@ -1420,9 +1434,9 @@
<table><thead>
<tr>
-<th>Variable Name</th>
-<th>Valid Variable Values</th>
-<th>Description</th>
+<th><strong>Variable Name</strong></th>
+<th><strong>Valid Variable Values</strong></th>
+<th><strong>Description</strong></th>
</tr>
</thead><tbody>
<tr>
diff --git a/docs/date-time-functions-and-arithmetic/index.html b/docs/date-time-functions-and-arithmetic/index.html
index e8370b6..7c094a1 100644
--- a/docs/date-time-functions-and-arithmetic/index.html
+++ b/docs/date-time-functions-and-arithmetic/index.html
@@ -1293,7 +1293,7 @@
</div>
- Dec 27, 2018
+ Dec 28, 2018
<link href="/css/docpage.css" rel="stylesheet" type="text/css">
@@ -1899,61 +1899,66 @@ SELECT UNIX_TIMESTAMP('2015-05-29 08:18:53.0', 'yyyy-MM-dd HH:mm:ss.
<h3 id="timestampadd-syntax">TIMESTAMPADD Syntax</h3>
-<p>TIMESTAMPADD(<em>time_unit,interval,date_expression</em>) </p>
+<p>TIMESTAMPADD(<em>time_unit,interval, keyword date_expression</em>) </p>
<h3 id="timestampadd-usage-notes">TIMESTAMPADD Usage Notes</h3>
<ul>
-<li>A date expressions is in the format YYYY-MM-DD.</li>
-<li>Supports the following time units: Day, Week, Month, Quarter, Year</li>
+<li><em>Keyword</em> is the type of <em>date_expression</em>: date, time, or timestamp</li>
+<li>Supports date, time, and timestamp values in the following formats:
+
+<ul>
+<li>Date format: YYYY-MM-DD</li>
+<li>Time format: HH:MI:SS</li>
+<li>Timestamp format: YYYY-MM-DD HH:MI:SS</li>
+</ul></li>
+<li>Supports the following time units: Nanosecond, Microsecond, Second, Minute, Hour, Day, Month, Year, Week, Quarter </li>
<li>Drill uses the unit of time to infer the return type.</li>
<li><p>You can include the SQL_TSI_ prefix with the any of the supported time units, as shown: </p>
-<div class="highlight"><pre><code class="language-text" data-lang="text">SELECT TIMESTAMPADD(SQL_TSI_YEAR,3,'1982-05-06');
+<div class="highlight"><pre><code class="language-text" data-lang="text">SELECT TIMESTAMPADD(SQL_TSI_MINUTE,5,TIME '05:05:05');
++-----------+
+| EXPR$0 |
++-----------+
+| 05:10:05 |
++-----------+
</code></pre></div></li>
</ul>
<h3 id="timestampadd-examples">TIMESTAMPADD Examples</h3>
<p>Add three years to the given date: </p>
-<div class="highlight"><pre><code class="language-text" data-lang="text">SELECT TIMESTAMPADD(YEAR,3,'1982-05-06');
+<div class="highlight"><pre><code class="language-text" data-lang="text">SELECT TIMESTAMPADD(YEAR,3,DATE '1982-05-06');
+------------------------+
| EXPR$0 |
+------------------------+
| 1985-05-06 00:00:00.0 |
+------------------------+
</code></pre></div>
-<p>Add one quarter of a year (3 months) to the given date: </p>
-<div class="highlight"><pre><code class="language-text" data-lang="text">SELECT TIMESTAMPADD(QUARTER,1,'1982-05-06');
+<p>Add one quarter of a year (3 months) to the given date: </p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">SELECT TIMESTAMPADD(QUARTER,1,DATE'1982-05-06');
+------------------------+
| EXPR$0 |
+------------------------+
| 1982-08-06 00:00:00.0 |
+------------------------+
</code></pre></div>
-<p>Add three months to the given date: </p>
-<div class="highlight"><pre><code class="language-text" data-lang="text">SELECT TIMESTAMPADD(MONTH,3,'1982-05-06');
-+------------------------+
-| EXPR$0 |
-+------------------------+
-| 1982-08-06 00:00:00.0 |
-+------------------------+
-</code></pre></div>
-<p>Add three weeks to the given date: </p>
-<div class="highlight"><pre><code class="language-text" data-lang="text">SELECT TIMESTAMPADD(WEEK,3,'1982-05-06');
-+------------------------+
-| EXPR$0 |
-+------------------------+
-| 1982-05-27 00:00:00.0 |
-+------------------------+
-</code></pre></div>
-<p>Add three days to the given date: </p>
-<div class="highlight"><pre><code class="language-text" data-lang="text">SELECT TIMESTAMPADD(DAY,3,'1982-05-06');
-+------------------------+
-| EXPR$0 |
-+------------------------+
-| 1982-05-09 00:00:00.0 |
-+------------------------+
+<p>Add 225 seconds to the given time: </p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">SELECT TIMESTAMPADD(SECOND,225,TIME '02:02:02');
++-----------+
+| EXPR$0 |
++-----------+
+| 02:05:47 |
++-----------+
</code></pre></div>
+<p>Add 5555500000 microseconds to the given timestamp value:</p>
+
+<p>SELECT TIMESTAMPADD(MICROSECOND,5555500000, TIMESTAMP '2003-02-01 12:05:35');
++--------------------------+
+| EXPR$0 |
++--------------------------+
+| 2003-02-01 12:26:35.532 |
++--------------------------+</p>
+
<h2 id="timestampdiff">TIMESTAMPDIFF</h2>
<p>Calculates an interval of time, in the given time units, by subtracting <em>datetime_expression1</em> from <em>datetime_expression2</em> (<em>datetime_expression2</em> − <em>datetime_expression1</em>). </p>
diff --git a/docs/start-up-options/index.html b/docs/start-up-options/index.html
index 87b50c3..c5bbcdf 100644
--- a/docs/start-up-options/index.html
+++ b/docs/start-up-options/index.html
@@ -1293,7 +1293,7 @@
</div>
- Jul 6, 2018
+ Dec 28, 2018
<link href="/css/docpage.css" rel="stylesheet" type="text/css">
@@ -1334,11 +1334,20 @@ file tells Drill to scan that JAR file or associated object and include it.</p>
</code></pre></div>
<h2 id="configuring-start-up-options">Configuring Start-Up Options</h2>
-<p>You can configure start-up options for each drillbit in <code><drill_home>/conf/drill-override.conf</code> .</p>
+<p>You can configure start-up options for each Drillbit in <code><drill_home>/conf/drill-override.conf</code>.</p>
<p>The summary of start-up options, also known as boot options, lists default values. The following descriptions provide more detail on key options that are frequently reconfigured:</p>
<ul>
+<li><p><strong>store.parquet.complex.batch.num_records</strong><br>
+Introduced in Drill 1.15. Sets batch sizing (number of rows per batch) for the Parquet reader. The default value is 4,000 rows. If rows are large and queries return out-of-memory errors, reduce the value. You can set this option at the system level in <code><drill_home>/conf/drill-override.conf</code> or at the session level using the SET command, as shown: </p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">SET `store.parquet.complex.batch.num_records`=2000;
++-------+---------------------------------------------------+
+| ok | summary |
++-------+---------------------------------------------------+
+| true | store.parquet.complex.batch.num_records updated. |
++-------+---------------------------------------------------+
+</code></pre></div></li>
<li><p><strong>drill.exec.http.ssl_enabled</strong><br>
Introduced in Drill 1.2. Enables or disables <a href="/docs/configuring-web-console-and-rest-api-security/#https-support">HTTPS support</a>. Settings are TRUE and FALSE, respectively. The default is FALSE. </p></li>
<li><p><strong>drill.exec.sys.store.provider.class</strong><br>
diff --git a/feed.xml b/feed.xml
index f1266f1..1cb158b 100644
--- a/feed.xml
+++ b/feed.xml
@@ -6,8 +6,8 @@
</description>
<link>/</link>
<atom:link href="/feed.xml" rel="self" type="application/rss+xml"/>
- <pubDate>Thu, 27 Dec 2018 16:40:33 -0800</pubDate>
- <lastBuildDate>Thu, 27 Dec 2018 16:40:33 -0800</lastBuildDate>
+ <pubDate>Thu, 27 Dec 2018 22:27:51 -0800</pubDate>
+ <lastBuildDate>Thu, 27 Dec 2018 22:27:51 -0800</lastBuildDate>
<generator>Jekyll v2.5.2</generator>
<item>