You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@drill.apache.org by br...@apache.org on 2017/07/31 21:04:12 UTC
drill-site git commit: Doc edits for Drill 1.11
Repository: drill-site
Updated Branches:
refs/heads/asf-site e168c6134 -> 7866d03e2
Doc edits for Drill 1.11
Project: http://git-wip-us.apache.org/repos/asf/drill-site/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill-site/commit/7866d03e
Tree: http://git-wip-us.apache.org/repos/asf/drill-site/tree/7866d03e
Diff: http://git-wip-us.apache.org/repos/asf/drill-site/diff/7866d03e
Branch: refs/heads/asf-site
Commit: 7866d03e2ca893e29175a850fee831856edbd54b
Parents: e168c61
Author: Bridget Bevens <bb...@maprtech.com>
Authored: Mon Jul 31 14:03:57 2017 -0700
Committer: Bridget Bevens <bb...@maprtech.com>
Committed: Mon Jul 31 14:03:57 2017 -0700
----------------------------------------------------------------------
blog/2017/07/31/drill-1.11-released/index.html | 13 ++++--
.../index.html | 46 ++++++++++++++++----
docs/configuring-user-authentication/index.html | 4 +-
docs/secure-communication-paths/index.html | 9 +++-
docs/securing-drill-introduction/index.html | 4 +-
feed.xml | 17 +++++---
6 files changed, 68 insertions(+), 25 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/drill-site/blob/7866d03e/blog/2017/07/31/drill-1.11-released/index.html
----------------------------------------------------------------------
diff --git a/blog/2017/07/31/drill-1.11-released/index.html b/blog/2017/07/31/drill-1.11-released/index.html
index 7701412..be42b37 100644
--- a/blog/2017/07/31/drill-1.11-released/index.html
+++ b/blog/2017/07/31/drill-1.11-released/index.html
@@ -146,7 +146,7 @@
<h2 id="spill-to-disk-for-hash-aggregate-operator-(drill-5457)">Spill to Disk for Hash Aggregate Operator (DRILL-5457)</h2>
-<p>The Hash aggregate operator can spill data to disk in cases where the operation exceeds the set memory limit. </p>
+<p>The Hash aggregate operator can spill data to disk in cases where the operation exceeds the set memory limit. Note that you may need to increase the default value of the <code>planner.memory.max_query_memory_per_node</code> option due to insufficient memory. </p>
<h2 id="format-plugin-support-for-pcap-files-(drill-5432)">Format Plugin Support for PCAP Files (DRILL-5432)</h2>
@@ -168,7 +168,7 @@
<h2 id="configurable-ctas-directory-and-file-permissions-option-(drill-5391)">Configurable CTAS Directory and File Permissions Option (DRILL-5391)</h2>
-<p>You can use the <code>exec.persistent_table.umask</code> configuration option, at the system or session level, to modify permissions on directories and files that result from running the CTAS command. By default, the option is set to 002, which sets the default directory permissions to 775 and default file permissions to -664. </p>
+<p>You can use the <code>exec.persistent_table.umask</code> configuration option, at the system or session level, to modify permissions on directories and files that result from running the CTAS command. By default, the option is set to 002, which sets the default directory permissions to 775 and default file permissions to 664. </p>
<h2 id="support-for-network-encryption-(drill-4335)">Support for Network Encryption (DRILL-4335)</h2>
@@ -178,9 +178,14 @@
<p>Drill now stores the relative path in the metadata file (versus the absolute path), which enables you to move partitioned Parquet directories from one location in DFS to another without having to rebuild the Parquet metadata files; the metadata remains valid in the new location. </p>
-<h2 id="support-for-ansi_quotes-(drill-3510)">Support for ANSI_QUOTES (DRILL-3510)</h2>
+<h2 id="support-for-additional-quoting-identifiers-(drill-3510)">Support for Additional Quoting Identifiers (DRILL-3510)</h2>
-<p>In addition to back ticks, the SQL parser in Drill can use double quotes as identifier quotes. Use the <code>planner.parser.quoting_identifiers</code> configuration option, at the system or session level, to set the type of identifier quotes that the SQL parser in Drill uses. </p>
+<p>In addition to back ticks, the SQL parser in Drill can use double quotes and square brackets as identifier quotes. Use the <code>planner.parser.quoting_identifiers</code> configuration option, at the system or session level, to set the type of identifier quotes that the SQL parser in Drill uses, as shown: </p>
+<div class="highlight"><pre><code class="language-text" data-lang="text"> ALTER SESSION SET planner.parser.quoting_identifiers = '"';
+ ALTER SESSION SET planner.parser.quoting_identifiers = '[';
+ ALTER SESSION SET planner.parser.quoting_identifiers = '`';
+</code></pre></div>
+<p>The default setting is back ticks. The quoting identifier used in queries must match the setting. If you use another type of quoting identifier, Drill returns an error. </p>
<p>You can find a complete list of JIRAs resolved in the 1.11.0 release <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313820&version=12339943">here</a>.</p>
http://git-wip-us.apache.org/repos/asf/drill-site/blob/7866d03e/docs/configuring-kerberos-authentication/index.html
----------------------------------------------------------------------
diff --git a/docs/configuring-kerberos-authentication/index.html b/docs/configuring-kerberos-authentication/index.html
index 6e671f0..01ea0be 100644
--- a/docs/configuring-kerberos-authentication/index.html
+++ b/docs/configuring-kerberos-authentication/index.html
@@ -1126,13 +1126,13 @@
</div>
- May 17, 2017
+ Jul 31, 2017
<link href="/css/docpage.css" rel="stylesheet" type="text/css">
<div class="int_text" align="left">
- <p>In release 1.10 Drill supports Kerberos v5 network security authentication. To use Kerberos with Drill and establish connectivity, use the JDBC driver packaged with Drill 1.10.</p>
+ <p>In release 1.11 Drill supports Kerberos v5 network security authentication and client-to-drillbit encryption. To use Kerberos with Drill and establish connectivity, use the JDBC driver packaged with Drill 1.11.</p>
<p>Kerberos allows trusted hosts to prove their identity over a network to an information system. A Kerberos <em>realm</em> is unique authentication domain. A centralized <em>key distribution center (KDC)</em> coordinates authentication between a clients and servers. Clients and servers obtain and use tickets from the KDC using a special <em>keytab</em> file to communicate with the KDC and prove their identity to gain access to a drillbit. Administrators must create <em>principal</em> (user or server) identities and passwords to ensure the secure exchange of mutual authentication information passed to and from the drillbit. </p>
@@ -1145,7 +1145,7 @@
<h2 id="prerequisites">Prerequisites</h2>
-<p>The required Kerberos (JDBC) plugin is part of the 1.10 Drill package. To use it, you must have a working Kerberos infrastructure, which Drill does not provide. You must be working in a Linux-based or Windows Active Directory (AD) Kerberos environment with secure clusters and have a Drill server configured for Kerberos. See <a href="/docs/configuring-kerberos-authentication/#enabling-authentication">Enabling Authentication</a>.</p>
+<p>The required Kerberos (JDBC) plugin is part of the 1.11 Drill package. To use it, you must have a working Kerberos infrastructure, which Drill does not provide. You must be working in a Linux-based or Windows Active Directory (AD) Kerberos environment with secure clusters and have a Drill server configured for Kerberos. See <a href="/docs/configuring-kerberos-authentication/#enabling-authentication">Enabling Authentication</a>.</p>
<h2 id="client-authentication-process">Client Authentication Process</h2>
@@ -1166,10 +1166,17 @@
<p>For Kerberos server authentication information, see the <a href="http://web.mit.edu/kerberos/" title="MIT Kerberos">MIT Kerberos</a> administration documentation. </p>
-<h2 id="enabling-authentication">Enabling Authentication</h2>
+<h2 id="enabling-authentication-and-encryption">Enabling Authentication and Encryption</h2>
<p>During startup, a drillbit service must authenticate. At runtime, Drill uses the keytab file. Trust is based on the keytab file; its secrets are shared with the KDC. The drillbit service also uses this keytab credential to validate service tickets from clients. Based on this information, the drillbit determines whether the client’s identity can be verified to use its service. </p>
+<p>To enable encryption,set the following parameters in the <code>drill-override.conf</code> file (as shown in the second example below): </p>
+
+<ul>
+<li><p><code>security.user.encryption.sasl.enabled</code> to true. This parameter determines if the drillbit is enabled for encryption. Only Drill 1.11 drillbits support encryption. </p></li>
+<li><p><code>security.user.encryption.sasl.max_wrapped_size</code>. This parameter specifies the maximum size of encoded buffer in bytes (maxbuffer parameter in sasl) that the client and server will receive. Using this the SASL framework exposes maximum buffer size that the wrap function will accept, so that Drill client/server can chop the Outbound RPC message with the size. The maximum recommended value is 16777215. The default is 65536.</p></li>
+</ul>
+
<p><img src="/docs/img/kerberos-client-server.png" alt="kerberos client server"></p>
<p> 1. Create a Kerberos principal identity and a keytab file. You can create one principal for each drillbit or one principal for all drillbits in a cluster. The <code>drill.keytab</code> file must be owned by and readable by the administrator user. </p>
@@ -1222,11 +1229,13 @@
auth.principal:“drill/<clustername>@<REALM>.COM”,
auth.keytab:“/etc/drill/conf/drill.keytab”
}
- security.user.auth: {
- enabled: true,
- packages += "org.apache.drill.exec.rpc.user.security",
- impl: "pam",
- pam_profiles: ["sudo", "login"]
+ security.user: {
+ auth.enabled: true,
+ auth.packages += "org.apache.drill.exec.rpc.user.security",
+ auth.impl: "pam",
+ auth.pam_profiles: ["sudo", "login"],
+ encryption.sasl.enabled: true,
+ encryption.sasl.max_wrapped_size: 65536,
}
}
</code></pre></div></li>
@@ -1290,6 +1299,12 @@
<td></td>
</tr>
<tr>
+<td>sasl_encrypt</td>
+<td>When set to true, ensures that a client connects to a server with encryption capabilities. For example, Drill 1.11 drillbits, which support client-to-drillbit encryption.</td>
+<td>Optional</td>
+<td>false</td>
+</tr>
+<tr>
<td>service_name</td>
<td>Primary name of the drillbit service principal.</td>
<td>Optional</td>
@@ -1309,6 +1324,19 @@
</tr>
</tbody></table>
+<h3 id="client-encryption">Client Encryption</h3>
+
+<p>A client can specify that it requires a server with encryption capabilities only by setting the <code>sasl_encrypt</code> connection parameter to <strong>true</strong>. If the cluster to which client is connecting has encryption disabled, the client will fail to connect to that server.</p>
+<div class="highlight"><pre><code class="language-text" data-lang="text">drill.exec {
+ security: {
+ user.auth.enabled: true,
+ auth.mechanisms: ["KERBEROS"],
+ auth.principal: "drill/serverhostname@REALM.COM",
+ auth.keytab: "/etc/drill/conf/drill.keytab",
+ user.encryption.sasl.enabled: true
+ }
+}
+</code></pre></div>
<h3 id="connection-url-examples">Connection URL Examples</h3>
<p>The following five examples show the JDBC connection URL that the embedded JDBC client uses for Kerberos authentication. The first section, Example of a Simple Connection URL, includes a simple connection string and the second section, Examples of Connection URLs Used with Previously Generated TGTs, includes examples to use with previously generated TGTs.</p>
http://git-wip-us.apache.org/repos/asf/drill-site/blob/7866d03e/docs/configuring-user-authentication/index.html
----------------------------------------------------------------------
diff --git a/docs/configuring-user-authentication/index.html b/docs/configuring-user-authentication/index.html
index 16025ec..5eaa394 100644
--- a/docs/configuring-user-authentication/index.html
+++ b/docs/configuring-user-authentication/index.html
@@ -1126,7 +1126,7 @@
</div>
- May 17, 2017
+ Jul 31, 2017
<link href="/css/docpage.css" rel="stylesheet" type="text/css">
@@ -1135,7 +1135,7 @@
<p>Authentication is the process of establishing confidence of authenticity. A Drill client user is authenticated when a drillbit process running in a Drill cluster confirms the identity it is presented with. Drill 1.10 supports several authentication mechanisms through which users can prove their identity before accessing cluster data: </p>
<ul>
-<li><strong>Kerberos</strong> - New in Drill 1.10. See <a href="/docs/configuring-kerberos-authentication/">Configuring Kerberos Authentication</a>.</li>
+<li><strong>Kerberos</strong> - Featuring Drill client to Drillbit encryption in Drill 1.11. See <a href="/docs/configuring-kerberos-authentication/">Configuring Kerberos Authentication</a>.</li>
<li><strong>Plain</strong> [also known as basic authentication (auth), which is username and password-based authentication, through the Linux Pluggable Authentication Module (PAM)] - See <a href="/docs/configuring-plain-authentication/">Configuring Plain Authentication</a>.</li>
<li><strong>Custom authenticators</strong> - See <a href="/docs/creating-custom-authenticators">Creating Custom Authenticators</a>.</li>
</ul>
http://git-wip-us.apache.org/repos/asf/drill-site/blob/7866d03e/docs/secure-communication-paths/index.html
----------------------------------------------------------------------
diff --git a/docs/secure-communication-paths/index.html b/docs/secure-communication-paths/index.html
index a1e53b1..68e542a 100644
--- a/docs/secure-communication-paths/index.html
+++ b/docs/secure-communication-paths/index.html
@@ -1126,7 +1126,7 @@
</div>
- Mar 17, 2017
+ Jul 31, 2017
<link href="/css/docpage.css" rel="stylesheet" type="text/css">
@@ -1157,7 +1157,7 @@
<p><strong>Note</strong></p>
-<p>Impersonation and authorization are available through the web clients only when authentication is enabled. Otherwise, the user identity is unknown.</p>
+<p>Impersonation, authorization, and encryption are available through the web clients only when authentication and encryption are enabled. Otherwise, the user identity is unknown and encryption is not used.</p>
<hr>
@@ -1174,6 +1174,11 @@
<td><a href="/docs/configuring-web-console-and-rest-api-security">Configuring Web Console and REST API Security</a></td>
</tr>
<tr>
+<td>Encryption</td>
+<td>Drill 1.11 supports encryption between a Drill client and Drillbit using the Kerberos mechanism over a Java SASL framework. Encrypting the client-to-drillbit communication pathway ensures data integrity and prevents data tampering as well as snooping. On the server side, enable encryption in the drill-override.conf file with the security.user.encryption.sasl.enabled parameter. On the client side, use the sasl_encrypt parameter in the connection string.</td>
+<td><a href="/docs/configuring-kerberos-authentication/">Configuring Kerberos Authentication</a></td>
+</tr>
+<tr>
<td>Impersonation</td>
<td>Drill acts on behalf of the user on the session. This is usually the connection user (or the user that authenticates). This user can impersonate another user, which is allowed if the connection user is authorized to impersonate the target user based on the inbound impersonation policies (USER role). By default, impersonation is disabled.</td>
<td><a href="/docs/configuring-user-impersonation/#impersonation-and-views">Configuring User Impersonation</a> and <a href="/docs/configuring-inbound-impersonation">Configuring Inbound Impersonation</a></td>
http://git-wip-us.apache.org/repos/asf/drill-site/blob/7866d03e/docs/securing-drill-introduction/index.html
----------------------------------------------------------------------
diff --git a/docs/securing-drill-introduction/index.html b/docs/securing-drill-introduction/index.html
index b5465dc..8b4c8d7 100644
--- a/docs/securing-drill-introduction/index.html
+++ b/docs/securing-drill-introduction/index.html
@@ -1126,7 +1126,7 @@
</div>
- Mar 16, 2017
+ Jul 31, 2017
<link href="/css/docpage.css" rel="stylesheet" type="text/css">
@@ -1155,7 +1155,7 @@ See <a href="/docs/configuring-kerberos-authentication/">Configuring Kerberos Au
- <a href="/docs/configuring-user-impersonation/">Configuring User Impersonation</a><br>
- <a href="/docs/configuring-inbound-impersonation/">Configuring Inbound Impersonation</a><br>
- <a href="/docs/configuring-user-impersonation-with-hive-authorization/">Configuring User Impersonation with Hive Authorization</a><br></li>
-<li><strong>Encryption</strong> - Drill does not support encryption as of Drill 1.10.</li>
+<li><strong>Encryption</strong> - Drill supports client-to-drillbit encryption in Drill 1.11.</li>
</ul>
http://git-wip-us.apache.org/repos/asf/drill-site/blob/7866d03e/feed.xml
----------------------------------------------------------------------
diff --git a/feed.xml b/feed.xml
index 1477bf3..3febaac 100644
--- a/feed.xml
+++ b/feed.xml
@@ -6,8 +6,8 @@
</description>
<link>/</link>
<atom:link href="/feed.xml" rel="self" type="application/rss+xml"/>
- <pubDate>Sun, 30 Jul 2017 21:54:25 -0700</pubDate>
- <lastBuildDate>Sun, 30 Jul 2017 21:54:25 -0700</lastBuildDate>
+ <pubDate>Mon, 31 Jul 2017 14:01:58 -0700</pubDate>
+ <lastBuildDate>Mon, 31 Jul 2017 14:01:58 -0700</lastBuildDate>
<generator>Jekyll v2.5.2</generator>
<item>
@@ -31,7 +31,7 @@
<h2 id="spill-to-disk-for-hash-aggregate-operator-(drill-5457)">Spill to Disk for Hash Aggregate Operator (DRILL-5457)</h2>
-<p>The Hash aggregate operator can spill data to disk in cases where the operation exceeds the set memory limit. </p>
+<p>The Hash aggregate operator can spill data to disk in cases where the operation exceeds the set memory limit. Note that you may need to increase the default value of the <code>planner.memory.max_query_memory_per_node</code> option due to insufficient memory. </p>
<h2 id="format-plugin-support-for-pcap-files-(drill-5432)">Format Plugin Support for PCAP Files (DRILL-5432)</h2>
@@ -53,7 +53,7 @@
<h2 id="configurable-ctas-directory-and-file-permissions-option-(drill-5391)">Configurable CTAS Directory and File Permissions Option (DRILL-5391)</h2>
-<p>You can use the <code>exec.persistent_table.umask</code> configuration option, at the system or session level, to modify permissions on directories and files that result from running the CTAS command. By default, the option is set to 002, which sets the default directory permissions to 775 and default file permissions to -664. </p>
+<p>You can use the <code>exec.persistent_table.umask</code> configuration option, at the system or session level, to modify permissions on directories and files that result from running the CTAS command. By default, the option is set to 002, which sets the default directory permissions to 775 and default file permissions to 664. </p>
<h2 id="support-for-network-encryption-(drill-4335)">Support for Network Encryption (DRILL-4335)</h2>
@@ -63,9 +63,14 @@
<p>Drill now stores the relative path in the metadata file (versus the absolute path), which enables you to move partitioned Parquet directories from one location in DFS to another without having to rebuild the Parquet metadata files; the metadata remains valid in the new location. </p>
-<h2 id="support-for-ansi_quotes-(drill-3510)">Support for ANSI_QUOTES (DRILL-3510)</h2>
+<h2 id="support-for-additional-quoting-identifiers-(drill-3510)">Support for Additional Quoting Identifiers (DRILL-3510)</h2>
-<p>In addition to back ticks, the SQL parser in Drill can use double quotes as identifier quotes. Use the <code>planner.parser.quoting_identifiers</code> configuration option, at the system or session level, to set the type of identifier quotes that the SQL parser in Drill uses. </p>
+<p>In addition to back ticks, the SQL parser in Drill can use double quotes and square brackets as identifier quotes. Use the <code>planner.parser.quoting_identifiers</code> configuration option, at the system or session level, to set the type of identifier quotes that the SQL parser in Drill uses, as shown: </p>
+<div class="highlight"><pre><code class="language-text" data-lang="text"> ALTER SESSION SET planner.parser.quoting_identifiers = &#39;&quot;&#39;;
+ ALTER SESSION SET planner.parser.quoting_identifiers = &#39;[&#39;;
+ ALTER SESSION SET planner.parser.quoting_identifiers = &#39;`&#39;;
+</code></pre></div>
+<p>The default setting is back ticks. The quoting identifier used in queries must match the setting. If you use another type of quoting identifier, Drill returns an error. </p>
<p>You can find a complete list of JIRAs resolved in the 1.11.0 release <a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313820&amp;version=12339943">here</a>.</p>
</description>