You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@drill.apache.org by br...@apache.org on 2017/08/17 04:13:15 UTC
drill git commit: Drill odbc and security updates from C.Skrbina +
spill to disk updates from B.Bevens
Repository: drill
Updated Branches:
refs/heads/gh-pages 3272b2e74 -> a63f2e347
Drill odbc and security updates from C.Skrbina + spill to disk updates from B.Bevens
Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/a63f2e34
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/a63f2e34
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/a63f2e34
Branch: refs/heads/gh-pages
Commit: a63f2e347f551f60e72973fa45688d1504f303b5
Parents: 3272b2e
Author: Bridget Bevens <bb...@maprtech.com>
Authored: Wed Aug 16 21:11:12 2017 -0700
Committer: Bridget Bevens <bb...@maprtech.com>
Committed: Wed Aug 16 21:11:12 2017 -0700
----------------------------------------------------------------------
.../020-secure-communication-paths.md | 17 +-
.../090-configuring-kerberos-security.md | 99 +++++-----
.../drill-channel-pipeline-with-handlers.png | Bin 32476 -> 14051 bytes
_docs/img/kerberos-clnt-svr.png | Bin 46792 -> 55818 bytes
_docs/img/odbctrace.png | Bin 0 -> 20938 bytes
.../010-odbc-configuration-reference.md | 55 +-----
.../configuring-odbc/011-logging-tracing.md | 179 +++++++++++++++++++
.../020-configuring-odbc-on-linux.md | 26 +--
.../025-configuring-odbc-on-mac-os-x.md | 21 +--
.../030-configuring-odbc-on-windows.md | 5 +-
.../040-testing-the-odbc-connection.md | 16 +-
.../010-installing-the-driver-on-linux.md | 23 ++-
.../020-installing-the-driver-on-mac-os-x.md | 26 ++-
.../030-installing-the-driver-on-windows.md | 24 ++-
...d-hash-based-memory-constrained-operators.md | 72 ++++++--
15 files changed, 345 insertions(+), 218 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/drill/blob/a63f2e34/_docs/configure-drill/securing-drill/020-secure-communication-paths.md
----------------------------------------------------------------------
diff --git a/_docs/configure-drill/securing-drill/020-secure-communication-paths.md b/_docs/configure-drill/securing-drill/020-secure-communication-paths.md
index dcdc4e2..359ee74 100644
--- a/_docs/configure-drill/securing-drill/020-secure-communication-paths.md
+++ b/_docs/configure-drill/securing-drill/020-secure-communication-paths.md
@@ -1,19 +1,24 @@
---
title: "Secure Communication Paths"
-date: 2017-08-07 19:02:29 UTC
+date: 2017-08-17 04:11:19 UTC
parent: "Securing Drill"
---
As illustrated in the following figure, Drill features five secure communication paths. Drill 1.11 introduces encryption between a Drill client and Drillbit.
-1. [Web Client to Drillbit]({{site.baseurl}}/docs/server-communication-paths/#web-client-to-drillbit)
-1. [C++ Client to Drillbit]({{site.baseurl}}/docs/server-communication-paths/#c++-client-to-drillbit)
-1. [Java Client to Drillbit]({{site.baseurl}}/docs/server-communication-paths/#java-client-to-drillbit)
-1. [Java Client and Drillbit to ZooKeeper]({{site.baseurl}}/docs/server-communication-paths/#java-client-and-drillbit-to-zookeeper)
-1. [Drillbit to Hive Storage Plugin]({{site.baseurl}}/docs/server-communication-paths/#drillbit-to-hive-storage-plugin)
+1. [Web Client to Drillbit]({{site.baseurl}}/docs/secure-communication-paths/#web-client-to-drillbit)
+1. [Java and C++ Client to Drillbit]({{site.baseurl}}/docs/secure-communication-paths/#java-and-c++-client-to-drillbit)
+1. [Java Client and Drillbit to ZooKeeper]({{site.baseurl}}/docs/secure-communication-paths/#drill-client-and-drillbit-to-zookeeper)
+1. [Drillbit to Hive Storage Plugin]({{site.baseurl}}/docs/secure-communication-paths/#drillbit-to-hive-storage-plugin)
+---
+**Note**
+
+Drillbit-to-drillbit encryption is not available yet.
+
+
## Web Client to Drillbit
The Web Console and REST API clients are web clients. Web clients can:
http://git-wip-us.apache.org/repos/asf/drill/blob/a63f2e34/_docs/configure-drill/securing-drill/090-configuring-kerberos-security.md
----------------------------------------------------------------------
diff --git a/_docs/configure-drill/securing-drill/090-configuring-kerberos-security.md b/_docs/configure-drill/securing-drill/090-configuring-kerberos-security.md
index 3cea262..1a65d69 100644
--- a/_docs/configure-drill/securing-drill/090-configuring-kerberos-security.md
+++ b/_docs/configure-drill/securing-drill/090-configuring-kerberos-security.md
@@ -1,9 +1,9 @@
---
title: "Configuring Kerberos Security"
-date: 2017-05-17 01:38:52 UTC
+date: 2017-08-17 04:11:23 UTC
parent: "Securing Drill"
---
-Drill 1.11 supports Kerberos v5 network security authentication and client-to-drillbit encryption for Kerberos. To use Kerberos with Drill and establish connectivity, use the JDBC driver packaged with Drill.
+Drill 1.11 supports Kerberos v5 network security authentication and encryption for Kerberos. To use Kerberos with Drill and establish connectivity, use the JDBC driver packaged with Drill.
Kerberos allows trusted hosts to prove their identity over a network to an information system. A Kerberos *realm* is unique authentication domain. A centralized *key distribution center (KDC)* coordinates authentication between a clients and servers. Clients and servers obtain and use tickets from the KDC using a special *keytab* file to communicate with the KDC and prove their identity to gain access to a drillbit. Administrators must create *principal* (user or server) identities and passwords to ensure the secure exchange of mutual authentication information passed to and from the drillbit.
@@ -40,7 +40,7 @@ For Kerberos server authentication information, see the [MIT Kerberos](http://we
### Enabling Authentication and Encryption
During startup, a drillbit service must authenticate. At runtime, Drill uses the keytab file. Trust is based on the keytab file; its secrets are shared with the KDC. The drillbit service also uses this keytab credential to validate service tickets from clients. Based on this information, the drillbit determines whether the client’s identity can be verified to use its service.
-With encryption enabled, negotiation occurs for the most secure level of encryption such that a AES-256 cipher is used (if it's available as a KDC-supported encyption type). Setting the `security.user.encryption.sasl.enabled` parameter in the `drill-override.conf` file to **true** enables encryption for Kerberos.
+With encryption enabled, negotiation occurs for the most secure level of encryption. A strong cipher is used from the available KDC-supported encryption types. Set the `security.user.encryption.sasl.enabled` property to **true** as shown in step 3. This property facilitates the SASL negotiation with the Kerberos mechanism between the client and drillbit with the quality of protection (qop) set to the authentication with confidentiality (auth-conf) value.
1. Create a Kerberos principal identity and a keytab file. You can create one principal for each drillbit or one principal for all drillbits in a cluster. The `drill.keytab` file must be owned by and readable by the administrator user.
@@ -60,31 +60,31 @@ With encryption enabled, negotiation occurs for the most secure level of encrypt
: ktadd -k /opt/mapr/conf/drill.keytab <username>/<FQDN>@<REALM>.COM
-2. Add the Kerberos principal identity and keytab file to the `drill-override.conf` file. The instance name must be lowercase. Also, if \_HOST is set as the instance name in the principal, it is replaced with the fully qualified domain name of that host for the instance name. For example, if a drillbit running on `host01.aws.lab` uses `drill/_HOST@<EXAMPLE>.COM` as the principal, the canonicalized principal is `drill/host01.aws.lab@<EXAMPLE>.COM`.
-
-To configure multiple mechanisms, extend the mechanisms list and provide additional configuration parameters. For example, the following configuration enables Kerberos and Plain (username and password) mechanisms. See [Installing and Connfiguring Plain Authentication]({{site.baseurl}}/docs/configuring-plain-authentication/#installing-and-configuring-plain-authentication) for Plain PAM configuration instructions.
-
- drill.exec: {
- cluster-id: "drillbits1",
- zk.connect: "qa102-81.qa.lab:2181,qa102-82.qa.lab:2181,qa102-83.qa.lab:2181",
- impersonation: {
- enabled: true,
- max_chained_user_hops: 3
- },
- security: {
- **auth.mechanisms**:["KERBEROS","PLAIN"],
- **auth.principal**:“drill/<clustername>@<REALM>.COM”,
- **auth.keytab**:“/etc/drill/conf/drill.keytab”
- }
- security.user: {
- auth.enabled: true,
- auth.packages += "org.apache.drill.exec.rpc.user.security",
- auth.impl: "pam",
- auth.pam_profiles: ["sudo", "login"],
- }
- }
-
-3. Set the `security.user.encryption.sasl.enabled` parameter to **true**. (Only Kerberos supports encryption.)
+2. Add the Kerberos principal identity and keytab file to the `drill-override.conf` file. The instance name must be lowercase. Also, if \_HOST is set as the instance name in the principal, it is replaced with the fully qualified domain name of that host for the instance name. For example, if a drillbit running on `host01.aws.lab` uses `drill/_HOST@<EXAMPLE>.COM` as the principal, the canonicalized principal is `drill/host01.aws.lab@<EXAMPLE>.COM`.
+
+To configure multiple mechanisms, extend the mechanisms list and provide additional configuration parameters. For example, the following configuration enables Kerberos and Plain (username and password) mechanisms. See [Installing and Connfiguring Plain Authentication]({{site.baseurl}}/docs/configuring-plain-authentication/#installing-and-configuring-plain-authentication) for Plain PAM configuration instructions.
+
+ drill.exec: {
+ cluster-id: "drillbits1",
+ zk.connect: "qa102-81.qa.lab:2181,qa102-82.qa.lab:2181,qa102-83.qa.lab:2181",
+ impersonation: {
+ enabled: true,
+ max_chained_user_hops: 3
+ },
+ security.auth: {
+ mechanisms:["KERBEROS","PLAIN"],
+ principal:“drill/<clustername>@<REALM>.COM”,
+ keytab:“/etc/drill/conf/drill.keytab”
+ }
+ security.user: {
+ auth.enabled: true,
+ auth.packages += "org.apache.drill.exec.rpc.user.security",
+ auth.impl: "pam",
+ auth.pam_profiles: ["sudo", "login"],
+ }
+ }
+
+2. a. To enable encryption with the Kerberos mechanism, set the `security.user.encryption.sasl.enabled` parameter to **true**. (Only Kerberos supports encryption.)
drill.exec: {
@@ -94,13 +94,15 @@ To configure multiple mechanisms, extend the mechanisms list and provide additio
enabled: true,
max_chained_user_hops: 3
},
- security: {
- user.auth.enabled: true,
- user.encryption.sasl.enabled: true,
- auth.mechanisms: [“KERBEROS”],
- auth.principal: “drill/<clustername>@<REALM>.COM”,
- auth.keytab: “/etc/drill/conf/drill.keytab”,
- }
+ security.auth: {
+ mechanisms: [“KERBEROS”],
+ principal: “drill/<clustername>@<REALM>.COM”,
+ keytab: “/etc/drill/conf/drill.keytab”,
+ }
+ security.user: {
+ auth.enabled: true,
+ **encryption.sasl.enabled: true,**
+ }
}
@@ -136,29 +138,28 @@ A client can provide its credentials in two ways:
### Configuration Options
The following table lists configuration options for connection URLs. See the Connection URL Examples section for sample URLs.
-| Connection Parameter | Description | Mandatory/Optional | Default Value |
-|----------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| auth | Authentication mechanism. The value is deduced if not specified. Kerberos if principal is provided. Plain if a user and password is provided. A Drill client can also explicitly specify a particular authentication mechanism to use using this parameter. For example, <auth=kerberos> for Kerberos along with service_name, service_host or principal and <auth=plain> for the Plain authentication with username and password. | Optional | The preference order is Kerberos and Plain. |
-| principal | Drillbit service principal. The format of the principal is primary/instance@realm. For Kerberos, the Drill service principal is derived if the value is not provided using this configuration. service_name (primary) and service_host (instance) are used to generate a valid principal. Since the ticket or keytab contains the realm information, the realm is optional. | Optional | |
-| keytab | For Kerberos, if the client chooses to authenticate using a keytab rather than a ticket, set the keytab parameter to the location of the keytab file. The client principal must be provided through the user parameter. A Kerberos ticket is used as the default credential (It is assumed to be present on client-side. The Drill client does not generate the required credentials.) | Optional | |
-| sasl_encrypt | When set to **true**, ensures that a client connects to a server with encryption capabilities. For example, Drill 1.11 drillbits, which support client-to-drillbit encryption. | Optional | false |
-| service_name | Primary name of the drillbit service principal. | Optional | drill |
-| service_host | Instance name of the drillbit service principal. | Optional | Since this value is usually the hostname of the node where a drillbit is running, the default value is the drillbit hostname is provided either through ZooKeeper or through a direct connection string. |
-| realm | Kerberos realm name for the drillbit service principal. The ticket or keytab contains the realm information. | Optional | |
-
+| Connection Parameter | Description | Mandatory/Optional | Default Value |
+|----------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| auth | Authentication mechanism. The value is deduced if not specified. Kerberos if principal is provided. Plain if a user and password is provided. A Drill client can also explicitly specify a particular authentication mechanism to use using this parameter. For example, <auth=kerberos> for Kerberos along with service_name, service_host or principal and <auth=plain> for the Plain authentication with username and password. | Optional | The preference order is Kerberos and Plain. |
+| principal | Drillbit service principal. The format of the principal is primary/instance@realm. For Kerberos, the Drill service principal is derived if the value is not provided using this configuration. service_name (primary) and service_host (instance) are used to generate a valid principal. Since the ticket or keytab contains the realm information, the realm is optional. | Optional | |
+| keytab | For Kerberos, if the client chooses to authenticate using a keytab rather than a ticket, set the keytab parameter to the location of the keytab file. The client principal must be provided through the user parameter. A Kerberos ticket is used as the default credential (It is assumed to be present on client-side. The Drill client does not generate the required credentials.) | Optional | |
+| sasl_encrypt | When set to **true**, ensures that a client connects to a server with encryption capabilities. For example, Drill 1.11 drillbits, which support client-to-drillbit encryption. | Optional | false |
+| service_name | Primary name of the drillbit service principal. | Optional | drill |
+| service_host | Instance name of the drillbit service principal. | Optional | Since this value is usually the hostname of the node where a drillbit is running, the default value is the drillbit hostname is provided either through ZooKeeper or through a direct connection string. |
+| realm | Kerberos realm name for the drillbit service principal. The ticket or keytab contains the realm information. | Optional | |
### Client Encryption
-A client can specify that it requires a server with encryption capabilities only by setting the `sasl_encrypt` connection parameter to **true**. If the cluster to which client is connecting has encryption disabled, the client will fail to connect to that server.
+A client can specify that it requires a drillbit with encryption capabilities only if the `sasl_encrypt` connection parameter is set to **true**. If the drillbit to which the client is connecting has encryption disabled, the client will fail to connect to that drillbit. By default, the client negotiates for a connection either with or without encryption capabilities based on whether or not encryption is enabled on the drillbit.
See *Client Compatibility* in [Configuring User Security]({{site.baseurl}}/docs/configuring-user-security/) for information about client version and Drill version compatibility.
### Connection URL Examples
-The following five examples show the JDBC connection URL that the embedded JDBC client uses for Kerberos authentication.
+The following five examples contain the JDBC connection URL that the embedded JDBC client uses for Kerberos authentication security. If encryption is enabled in a drillbit configuration, then the negotiation between the client and the drillbit will occur with encryption capabilities such that all traffic after a successful connection is encrypted.
- Example of a Simple Connection URL-a simple connection string
- - Example 1: TGT for Client Credentials
-
+ - Example 1: TGT for Client Credentials
+
- Examples of Connection URLs Used with Previously Generated TGTs-examples to use with previously generated TGTs
- Example 2: Drillbit Provided by Direct Connection String and Configured with a Unique Service Principal
- Example 3: Drillbit Selected by ZooKeeper and Configured with a Unique Service Principal
http://git-wip-us.apache.org/repos/asf/drill/blob/a63f2e34/_docs/img/drill-channel-pipeline-with-handlers.png
----------------------------------------------------------------------
diff --git a/_docs/img/drill-channel-pipeline-with-handlers.png b/_docs/img/drill-channel-pipeline-with-handlers.png
index 99aca9a..6272cc1 100644
Binary files a/_docs/img/drill-channel-pipeline-with-handlers.png and b/_docs/img/drill-channel-pipeline-with-handlers.png differ
http://git-wip-us.apache.org/repos/asf/drill/blob/a63f2e34/_docs/img/kerberos-clnt-svr.png
----------------------------------------------------------------------
diff --git a/_docs/img/kerberos-clnt-svr.png b/_docs/img/kerberos-clnt-svr.png
index 1012fda..cfee502 100644
Binary files a/_docs/img/kerberos-clnt-svr.png and b/_docs/img/kerberos-clnt-svr.png differ
http://git-wip-us.apache.org/repos/asf/drill/blob/a63f2e34/_docs/img/odbctrace.png
----------------------------------------------------------------------
diff --git a/_docs/img/odbctrace.png b/_docs/img/odbctrace.png
new file mode 100644
index 0000000..7e02657
Binary files /dev/null and b/_docs/img/odbctrace.png differ
http://git-wip-us.apache.org/repos/asf/drill/blob/a63f2e34/_docs/odbc-jdbc-interfaces/configuring-odbc/010-odbc-configuration-reference.md
----------------------------------------------------------------------
diff --git a/_docs/odbc-jdbc-interfaces/configuring-odbc/010-odbc-configuration-reference.md b/_docs/odbc-jdbc-interfaces/configuring-odbc/010-odbc-configuration-reference.md
index b4972ee..24f4b7e 100644
--- a/_docs/odbc-jdbc-interfaces/configuring-odbc/010-odbc-configuration-reference.md
+++ b/_docs/odbc-jdbc-interfaces/configuring-odbc/010-odbc-configuration-reference.md
@@ -1,9 +1,9 @@
---
title: "ODBC Configuration Reference"
-date: 2017-06-06 23:26:11 UTC
+date: 2017-08-17 04:11:39 UTC
parent: "Configuring ODBC"
---
-You can use various configuration options to control the behavior of the MapR Drill ODBC Driver. You can use these options in a connection string or in the `odbc.ini` configuration file for the Mac OS X version or the driver.
+You can use various configuration options to control the behavior of the Drill ODBC Driver. You can use these options in a connection string or in the `odbc.ini` configuration file for the Mac OS X version or the driver.
{% include startnote.html %}If you use a connection string to connect to your data source, set these configuration properties in the connection string instead of the .odbc.ini file.{% include endnote.html %}
@@ -14,7 +14,7 @@ The following table provides a list of the configuration options and a brief des
| Property | Default Values | Brief Description |
|-------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| AdvancedProperties | `CastAnyToVarchar=true; HandshakeTimeout=5; QueryTimeout=180; TimestampTZDisplayTimezone=local; ExcludedSchemas= sys,INFORMATION_SCHEMA; NumberOfPrefetchBuffers=5` | Not required. Advanced properties for configuring the driver. You can set custom connection properties by specifying them as advanced properties. If you specify a property that the driver does not explicitly support, the driver still accepts the property, and passes it to the server for processing. Separate advanced properties using a semi-colon (;) and then surround all advanced properties in a connection string using braces { and }. For example, `{<property>;<property>; . . .}` In addition, the following Advanced Properties string excludes the schemas named `test` and `abc`, sets the timeout to 30 seconds, and sets the time zone to Coordinated Universal Time:`HandshakeTimeout=30;QueryTimeout=30;TimestampTZDisplayTimezone=utc;ExcludedSchemas=test,abc`. |
-| AuthenticationType | No Authentication | Not required. This option specifies how the driver authenticates the connection to Drill. No Authentication: The driver does not authenticate the connection to Drill. Kerberos: The driver authenticates the connection using the Kerberos protocol. Plain: The driver authenticates the connection using a user name and a password. MapRSASL: The driver authenticates the connection using the MapR-SASL protocol. |
+| AuthenticationType | No Authentication | Not required. This option specifies how the driver authenticates the connection to Drill. No Authentication: The driver does not authenticate the connection to Drill. Kerberos: The driver authenticates the connection using the Kerberos protocol. Plain: The driver authenticates the connection using a user name and a password. |
| Catalog | The default catalog name specified in the driver's .did file (typically, DRILL). | Not required. The name of the synthetic catalog under which all of the schemas/databases are organized. This catalog name is used as the value for SQL_DATABASE_NAME or CURRENT CATALOG. |
| ConnectionType | Direct to Drillbit (Direct) | Required. This option specifies whether the driver connects to a single server or a ZooKeeper cluster. Direct to Drillbit (Direct): The driver connects to a single Drill server. ZooKeeper Quorum (ZooKeeper): The driver connects to a ZooKeeper cluster. |
| DelegationUID | none | Not required. If a value is specified for this setting, the driver delegates all operations against Drill to the specified user, rather than to the authenticated user for the connection. This option is applicable only when Plain authentication is enabled. |
@@ -126,55 +126,6 @@ The following is an example connection string for the Zookeeper connection type:
DRIVER=MapR Drill ODBC Driver;AdvancedProperties={HandshakeTimeout=0;QueryTimeout=0;TimestampTZDisplayTimezone=utc;ExcludedSchemas=sys, INFORMATION_SCHEMA;};Catalog=DRILL;Schema=;ConnectionType=ZooKeeper;ZKQuorum=192.168.39.43:5181;ZKClusterID=drillbits1
-## Logging Options
-
-Configure logging to troubleshoot issues. To configure logging, click the **Logging Options** button on the ODBC DSN Setup dialog and then set a log leveland a log path.
-
-If logging is enabled, the MapR Drill ODBC driver logs events in following log files in the log path that you configure:
-
-* `driver.log` provides a log of driver events.
-* `drillclient.log` provides a log of the Drill client events.
-
-
-### Logging Levels
-
-The following log levels are available:
-
-* (0) OFF: Disables logging.
-* (1) FATAL: Logs severe error events that may cause the driver to stop running.
-* (2) ERROR: Logs error events that may allow the driver to continue running.
-* (3) WARNING: Logs events about potentially harmful situations.
-* (4) INFO: Logs high-level events about driver processes.
-* (5) DEBUG: Logs detailed events that may help to debug issues.
-* (6) TRACE: Logs finer-grained events than the DEBUG level.
-
-### Enabling Logging
-
-To enable logging:
-
-1. Open the `.mapr.drillodbc.ini` configuration file in a text editor. (On Mac OS X, the default installation will install a .mapr.drillodbc.ini to $HOME.)
-
-2. Set the LogLevel key to the desired level of information to include in log files.
-For example:
-
- `LogLevel=2`
-
-3. Set the LogPath key to the full path to the folder where you want to save log files.
-For example:
-
- `LogPath=/localhome/employee/Documents`
-
-4. Save the `.mapr.drillodbc.ini` configuration file. The MapR Drill ODBC Driver produces two log files at the location you specify using the Log Path field:
- * `driver.log` provides a log of driver activities.
- * `drillclient.log` provides a log of Drill client activities.
-
-### Disabling Logging
-
-To disable logging:
-
-1. Open the `.mapr.drillodbc.ini` configuration file in a text editor.
-2. Set the LogLevel key to zero (`0`).
-3. Save the `.mapr.drillodbc.ini` configuration file.
http://git-wip-us.apache.org/repos/asf/drill/blob/a63f2e34/_docs/odbc-jdbc-interfaces/configuring-odbc/011-logging-tracing.md
----------------------------------------------------------------------
diff --git a/_docs/odbc-jdbc-interfaces/configuring-odbc/011-logging-tracing.md b/_docs/odbc-jdbc-interfaces/configuring-odbc/011-logging-tracing.md
new file mode 100644
index 0000000..f643c0c
--- /dev/null
+++ b/_docs/odbc-jdbc-interfaces/configuring-odbc/011-logging-tracing.md
@@ -0,0 +1,179 @@
+---
+title: "ODBC Configuration Reference"
+date: 2017-06-06 23:26:11 UTC
+parent: "Configuring ODBC"
+---
+
+## Logging Options
+
+Only enable logging long enough to capture information required to resolve an issue. Logging decreases performance and can consume a large quantity of disk space.
+
+If logging is enabled, the Drill ODBC driver logs events in the following log files in the log path that you configure:
+
+* `driver.log` provides a log of driver events.
+* `drillclient.log` provides a log of the Drill client events.
+
+### Logging Levels
+
+The following log levels are available:
+
+* (0) OFF: Disables logging.
+* (1) FATAL: Logs severe error events that may cause the driver to stop running.
+* (2) ERROR: Logs error events that may allow the driver to continue running.
+* (3) WARNING: Logs events that might result in an error if action is not taken.
+* (4) INFO: Logs general information that describes the progress of the driver.
+* (5) DEBUG: Logs detailed events that may help to debug issues.
+* (6) TRACE: Logs all driver activity, which includes more fine-grained events than the DEBUG level.
+
+### Non-Windows Platforms
+
+On non-Windows platforms, logging is configured through the driver-wide settings in the `mapr.drillodbc.ini` file, which apply to all connections using the driver.
+
+
+#### Enable Logging
+
+To enable logging:
+
+1. Open the `.mapr.drillodbc.ini` configuration file in a text editor. (On Mac OS X, the default installation will install a .mapr.drillodbc.ini to $HOME.)
+
+2. Set the **LogLevel** key to the desired level of information to include in log files.
+For example:
+
+ `LogLevel=2`
+
+3. Set the **LogPath** key to the full path to the folder where you want to save log files.
+For example:
+
+ `LogPath=/localhome/employee/logs`
+
+4. Save the `.mapr.drillodbc.ini` configuration file. The Drill ODBC Driver produces two log files at the location you specify using the **Log Path** field:
+ * `driver.log` provides a log of driver activities.
+ * `drillclient.log` provides a log of Drill client activities.
+
+
+6. Restart the applciation to make sure that the new settings take effect. Configuration changes will not be picked up until the application reloads the driver.
+
+#### Disable Logging
+
+To disable logging:
+
+1. Open the `.mapr.drillodbc.ini` configuration file in a text editor.
+2. Set the **LogLevel** key to zero (`0`).
+3. Save the `.mapr.drillodbc.ini` configuration file.
+4. Restart your ODBC application to make sure that the new settings take effect.
+
+### Windows Platforms
+
+On Windows, logging is available in the Windows **ODBC Data Source Administrator** where you created the DSN. You must run `C:\Windows\SysWOW64\odbcad32.exe` to access and modify 32-bit DSNs on 64-bit Windows.
+
+#### Enable Logging
+
+To enable logging:
+
+1. Click **Start**, **All Programs**, and then click the program group corresponding to the driver.
+
+2. Select the DSN for which you want to log activity.
+
+3. Click **Configure**.
+
+4. In the **DSN Setup** dialog box, click **Logging Options**.
+
+6. From the **Log Level** drop-down list, select the logging level corresponding to the amount of information that you want to include in log files.
+7. In the **Log Path** (or Log Directory) field, specify the full path to the folder where you want to save log files.
+
+8. If necessary (for example, if requested by a Support team), type the name of the component for which to log messages in the **Log Namespace** field. Otherwise, do not type a value in the field.
+9. Click **OK** to close the Logging Options dialog box.
+
+7. Click **OK** to save your settings and close the **DSN Configuration** dialog box. Configuration changes will not be saved of picked up by the driver until you have clicked **OK** in the **DSN Configuration **dialog box. Click **Cancel** (or the X button) to discard changes.
+
+8. Restart the application to make sure that the new settings take effect. Configuration changes will not be picked up by until the application reloads the driver.
+
+
+#### Disable Logging
+
+To disable logging:
+
+1. Select the DSN.
+2. Click **Configure**.
+3. Click **Logging Options**.
+4. From the **Log Level** drop-down list, select **LOG_OFF**.
+5. Click **OK**.
+6. Restart your ODBC application to make sure that the new settings take effect.
+
+## Driver Manager Tracing
+
+The driver manager trace facility, is a useful way to troubleshoot ODBC driver issues.You can choose from several driver managers, depending on which platform you use.
+
+{% include startimportant.html %}Tracing is active only for applications started after tracing has started. Currently-executing applications will not have tracing enabled. You must restart applications to enable tracing. You may need to restart services for applications, or you may have to restart the machine to properly enable tracing.{% include endimportant.html %}
+
+
+### OSX and Other Non-Windows Platforms
+
+iODBC is the default driver manager on OSX and can sometimes be found on other non-Windows platforms.
+
+
+#### Enable Trace Logging
+
+To enable trace logging:
+
+1. Locate your `maprdrill.odbc.ini` file. This often located in your home directory. It might be a hidden file.
+2. Open the file and add the following key-value pairs under the section heading **[ODBC]**. If the heading does not exist, add it. For example:
+
+ `[ODBC]`
+
+ `Trace=1`
+
+ `TraceFile=/path/to/file/traceFile.log`
+
+ `[ODBC Data Sources]`
+
+#### Disable Trace Logging
+
+When the trace is complete, disable tracing because tracing will consume disk space and significantly impact performance.
+
+1. Locate your `maprdrill.odbc.ini` file. This often located in your home directory. It might be a hidden file.
+2. Open the file and add the following key-value pairs under the section heading **[ODBC]**. For example:
+
+ `[ODBC]`
+
+ `Trace=0`
+
+ `TraceFile=/path/to/file/traceFile.log`
+
+ `[ODBC Data Sources]`
+
+1. (Again, this action will only impact applications that have just started, not currently-executing applications.) Restart your application.
+
+
+### OSX and Other Non-Windows Platforms
+
+#### Enable Trace Logging
+
+To enable tracing on Windows:
+
+1. Open the **ODBC Data Source Administrator**.
+1. Go to the **Tracing** tab.
+
+
+
+
+
+1. Change the **Log File Path** field to be the location and name of the file to which you’d like to write the trace entries.
+
+
+1. (Optional) Check **Machine-wide tracing** for all user identities if you want the tracing to affect all users on the machine. If you are unsure, check this box.
+
+
+1. Press **Start Tracing Now**.
+
+#### Disable Trace Logging
+
+When the trace is complete, disable tracing because tracing will consume disk space and significantly impact performance.
+
+1. Open the **ODBC Data Source Administrator**.
+1. Go to the **Tracing** tab.
+1. Press Stop **Tracing Now**. (Again, this action will only impact applications that have just started, not currently-executing applications.)
+1. Restart your application.
+
+For more information about generating ODBC traces, see [How To Generate an ODBC Trace with ODBC Data Source Administrator.
+](https://support.microsoft.com/en-us/help/274551/how-to-generate-an-odbc-trace-with-odbc-data-source-administrator)
http://git-wip-us.apache.org/repos/asf/drill/blob/a63f2e34/_docs/odbc-jdbc-interfaces/configuring-odbc/020-configuring-odbc-on-linux.md
----------------------------------------------------------------------
diff --git a/_docs/odbc-jdbc-interfaces/configuring-odbc/020-configuring-odbc-on-linux.md b/_docs/odbc-jdbc-interfaces/configuring-odbc/020-configuring-odbc-on-linux.md
index 7885485..eb0be2c 100644
--- a/_docs/odbc-jdbc-interfaces/configuring-odbc/020-configuring-odbc-on-linux.md
+++ b/_docs/odbc-jdbc-interfaces/configuring-odbc/020-configuring-odbc-on-linux.md
@@ -1,6 +1,6 @@
---
title: "Configuring ODBC on Linux"
-date: 2017-06-06 23:26:12 UTC
+date: 2017-08-17 04:11:44 UTC
parent: "Configuring ODBC"
---
@@ -11,7 +11,7 @@ steps:
* [Step 1: Set Environment Variables]({{site.baseurl}}/docs/configuring-odbc-on-linux/#step-1:-set-environment-variables)
* [Step 2: Define the ODBC Data Sources in odbc.ini]({{site.baseurl}}/docs/configuring-odbc-on-linux/#step-2:-define-the-odbc-data-sources-in-.odbc.ini)
* [Step 3: (Optional) Define the ODBC Driver in odbcinst.ini]({{site.baseurl}}/docs/configuring-odbc-on-linux/#step-3:-(optional)-define-the-odbc-driver-in-.odbcinst.ini)
-* [Step 4: Configure the MapR Drill ODBC Driver]({{site.baseurl}}/docs/configuring-odbc-on-linux/#configuring-.mapr.drillodbc.ini)
+* [Step 4: Configure the Drill ODBC Driver]({{site.baseurl}}/docs/configuring-odbc-on-linux/#configuring-.mapr.drillodbc.ini)
## Sample Configuration Files
@@ -128,7 +128,7 @@ The following Linux sample shows a possible configuration for using Drill in dis
# Values for ConnectionType, AdvancedProperties, Catalog, Schema should be set here.
# If ConnectionType is Direct, include Host and Port. If ConnectionType is ZooKeeper, include ZKQuorum and ZKClusterID
# They can also be specified on the connection string.
- # AuthenticationType:No authentication;Plain;Kerberos;MapRSASL;
+ # AuthenticationType:No authentication;Plain;Kerberos;
ConnectionType=Direct
HOST=[HOST]
PORT=[PORT]
@@ -147,11 +147,6 @@ The following Linux sample shows a possible configuration for using Drill in dis
### Authentication Properties
If the Drillbit requires authentication, uncomment the AuthenticationType, add an AuthenticationType, and configure properties. If the Drillbit does not require authentication (or to configure no password protection), you can use the No Authentication option. You do not need to configure additional settings.
-
-* **MapRSASL**
- * The MapR login utility must be used to obtain a MapR ticket for MapR SASL authentication.
- * You must install and configure the MapR login utility before you can use the MapR SASL authentication mechanism. See <a href="http://maprdocs.mapr.com/home/SecurityGuide/SecurityArchitecture-AuthenticationArchitecture.html" title="MapR Login Utilty">Authentication Architecture: The maprlogin Utility</a> and <a href="http://maprdocs.mapr.com/home/SecurityGuide/Tickets.html">Tickets</a>.
-
* **Kerberos**
* See the <a href="http://web.mit.edu/kerberos/" title="MIT Kerberos">MIT Kerberos</a> documentation for installing and configuring a Kerberos environment, which is beyond the scope of the information provided here.
* To specify the Kerberos mechanism:
@@ -180,7 +175,7 @@ To use Drill in embedded mode, do not define the ZKQuorum and ZKClusterID proper
* `PORT=31010`
[Driver Configuration Options]({{ site.baseurl }}/docs/odbc-configuration-reference/#configuration-options) describes configuration options available for controlling the
-behavior of DSNs using the MapR Drill ODBC Driver.
+behavior of DSNs using the Drill ODBC Driver.
----------
@@ -206,9 +201,9 @@ directly in the` .odbc.ini` configuration file. The `.odbinst.ini` file contains
----------
-## Step 4: Configure the MapR Drill ODBC Driver
+## Step 4: Configure the Drill ODBC Driver
-Configure the MapR Drill ODBC Driver for your environment by modifying the `.mapr.drillodbc.ini` configuration
+Configure the Drill ODBC Driver for your environment by modifying the `.mapr.drillodbc.ini` configuration
file. This configures the driver to work with your ODBC driver manager. The following sample shows a possible configuration, which you can use as is if you installed the default iODBC driver manager.
**Example**
@@ -227,7 +222,7 @@ file. This configures the driver to work with your ODBC driver manager. The foll
### Configuring .mapr.drillodbc.ini
-To configure the MapR Drill ODBC Driver in the `.mapr.drillodbc.ini` configuration file, complete the following steps:
+To configure the Drill ODBC Driver in the `.mapr.drillodbc.ini` configuration file, complete the following steps:
1. Open the `.mapr.drillodbc.ini` configuration file in a text editor.
@@ -240,9 +235,4 @@ To configure the MapR Drill ODBC Driver in the `.mapr.drillodbc.ini` configurati
### Next Step
-[Testing the ODBC Connection]({{ site.baseurl }}/docs/testing-the-odbc-connection/)
-
-
-
-
-
+[Testing the ODBC Connection]({{ site.baseurl }}/docs/testing-the-odbc-connection/)
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/drill/blob/a63f2e34/_docs/odbc-jdbc-interfaces/configuring-odbc/025-configuring-odbc-on-mac-os-x.md
----------------------------------------------------------------------
diff --git a/_docs/odbc-jdbc-interfaces/configuring-odbc/025-configuring-odbc-on-mac-os-x.md b/_docs/odbc-jdbc-interfaces/configuring-odbc/025-configuring-odbc-on-mac-os-x.md
index 11939b2..a607050 100644
--- a/_docs/odbc-jdbc-interfaces/configuring-odbc/025-configuring-odbc-on-mac-os-x.md
+++ b/_docs/odbc-jdbc-interfaces/configuring-odbc/025-configuring-odbc-on-mac-os-x.md
@@ -1,6 +1,6 @@
---
title: "Configuring ODBC on Mac OS X"
-date: 2017-06-06 23:26:12 UTC
+date: 2017-08-17 04:11:50 UTC
parent: "Configuring ODBC"
---
ODBC driver managers use configuration files to define and configure ODBC data
@@ -10,7 +10,7 @@ procedure:
* [Step 1: Driver Installer Updates Sample Configuration Files]({{site.baseurl}}/docs/configuring-odbc-on-mac-os-x/#step-1:-driver-installer-updates-sample-configuration-files)
* [Step 2: Set Environment Variables]({{site.baseurl}}/docs/configuring-odbc-on-mac-os-x/#step-2:-set-environment-variables)
* [Step 3: Define the ODBC Data Sources in odbc.ini]({{site.baseurl}}/docs/configuring-odbc-on-mac-os-x/#step-3:-define-the-odbc-data-sources-in-.odbc.ini)
-* [Step 4: Configure the MapR Drill ODBC Driver]({{site.baseurl}}/docs/configuring-odbc-on-mac-os-x/#step-4:-configure-the-mapr-drill-odbc-driver)
+* [Step 4: Configure the Drill ODBC Driver]({{site.baseurl}}/docs/configuring-odbc-on-mac-os-x/#step-4:-configure-the-drill-odbc-driver)
## Step 1: Driver Installer Updates Sample Configuration Files
@@ -73,11 +73,6 @@ You set the following properties for using Drill in distributed mode:
### Authentication Properties
If the Drillbit requires authentication, uncomment the AuthenticationType, add an AuthenticationType, and configure properties. If the Drillbit does not require authentication (or to configure no password protection), you can use the No Authentication option. You do not need to configure additional settings.
-
-* **MapRSASL**
- * The MapR login utility must be used to obtain a MapR ticket for MapR SASL authentication.
- * You must install and configure the MapR login utility before you can use the MapR SASL authentication mechanism. See <a href="http://maprdocs.mapr.com/home/SecurityGuide/SecurityArchitecture-AuthenticationArchitecture.html" title="MapR Login Utilty">Authentication Architecture: The maprlogin Utility</a> and <a href="http://maprdocs.mapr.com/home/SecurityGuide/Tickets.html">Tickets</a>.
-
* **Kerberos**
* See the <a href="http://web.mit.edu/kerberos/" title="MIT Kerberos">MIT Kerberos</a> documentation for installing and configuring a Kerberos environment, which is beyond the scope of the information provided here.
* To specify the Kerberos mechanism:
@@ -106,7 +101,7 @@ To use Drill in local mode, do not define the ZKQuorum and ZKClusterID propertie
* `PORT=31010`
[Driver Configuration Options]({{ site.baseurl }}/docs/odbc-configuration-reference/#configuration-options) describes configuration options available for controlling the
-behavior of DSNs using the MapR Drill ODBC Driver.
+behavior of DSNs using the Drill ODBC Driver.
###Sample Configuration File
The following sample shows a possible configuration for using Drill in embedded mode.
@@ -136,7 +131,7 @@ The following sample shows a possible configuration for using Drill in embedded
# If ConnectionType is Direct, include Host and Port. If ConnectionType is ZooKeeper, include ZKQuorum and ZKClusterID
# They can also be specified on the connection string.
- # AuthenticationType: No authentication; Plain; Kerberos; MapRSASL
+ # AuthenticationType: No authentication; Plain; Kerberos;
ConnectionType=Direct
HOST=localhost
PORT=31010
@@ -155,13 +150,11 @@ The following sample shows a possible configuration for using Drill in embedded
# The DriverUnicodeEncoding setting is only used for MapRDM
-
-
----------
-## Step 4: Configure the MapR Drill ODBC Driver
+## Step 4: Configure the Drill ODBC Driver
-Configure the MapR Drill ODBC Driver for your environment by modifying the `.mapr.drillodbc.ini` configuration
+Configure the Drill ODBC Driver for your environment by modifying the `.mapr.drillodbc.ini` configuration
file. This configures the driver to work with your ODBC driver manager. The following sample shows a possible configuration, which you can use as is if you installed the default iODBC driver manager.
**Example**
@@ -177,7 +170,7 @@ file. This configures the driver to work with your ODBC driver manager. The foll
### Configuring .mapr.drillodbc.ini
-To configure the MapR Drill ODBC Driver in the `.mapr.drillodbc.ini` configuration file, complete the following steps:
+To configure the Drill ODBC Driver in the `.mapr.drillodbc.ini` configuration file, complete the following steps:
1. Open the `.mapr.drillodbc.ini` configuration file in a text editor.
http://git-wip-us.apache.org/repos/asf/drill/blob/a63f2e34/_docs/odbc-jdbc-interfaces/configuring-odbc/030-configuring-odbc-on-windows.md
----------------------------------------------------------------------
diff --git a/_docs/odbc-jdbc-interfaces/configuring-odbc/030-configuring-odbc-on-windows.md b/_docs/odbc-jdbc-interfaces/configuring-odbc/030-configuring-odbc-on-windows.md
index c63a6e1..cc63e27 100644
--- a/_docs/odbc-jdbc-interfaces/configuring-odbc/030-configuring-odbc-on-windows.md
+++ b/_docs/odbc-jdbc-interfaces/configuring-odbc/030-configuring-odbc-on-windows.md
@@ -1,6 +1,6 @@
---
title: "Configuring ODBC on Windows"
-date: 2017-06-22 20:42:15 UTC
+date: 2017-08-17 04:11:54 UTC
parent: "Configuring ODBC"
---
To create an ODBC connection to Drill data sources on Windows, complete the following steps:
@@ -38,14 +38,11 @@ version of the ODBC Data Source Administrator to create the DSN.
### Step 2: Select an Authentication Option
To password protect the DSN, select the appropriate authentication type in the **Authentication Type** dropdown. If the Drillbit does not require authentication (or to configure no password protection), you can use the No Authentication option. You do not need to configure additional settings.
-* **MapR-SASL**
- * The maprlogin utility must be used to obtain a MapR ticket. To install and use the MapR login utility, see <a href="http://maprdocs.mapr.com/home/SecurityGuide/SecurityArchitecture-AuthenticationArchitecture.html" title="MapR Login Utilty">Authentication Architecture: The maprlogin Utility</a> and <a href="http://maprdocs.mapr.com/home/SecurityGuide/Tickets.html/">Tickets</a>.
* **Kerberos** - configure Host FQDN and Service Name properties.
* To specify the default Kerberos mechanism, select the Use Only SSPI checkbox.
* To use MIT Kerberos by default and only use the SSPI plugin if the GSSAPI library is not available, clear the Use Only SSPI checkbox.
* **Plain Authentication** - configure UID and PWD properties.
-
### Step 3: Configure the Connection Type
In the **Connection Type** section, **Direct to Drillbit** is selected for using Drill in embedded mode. To use Drill in embedded mode, set the connection type to **Direct** and define HOST and PORT properties. For example:
http://git-wip-us.apache.org/repos/asf/drill/blob/a63f2e34/_docs/odbc-jdbc-interfaces/configuring-odbc/040-testing-the-odbc-connection.md
----------------------------------------------------------------------
diff --git a/_docs/odbc-jdbc-interfaces/configuring-odbc/040-testing-the-odbc-connection.md b/_docs/odbc-jdbc-interfaces/configuring-odbc/040-testing-the-odbc-connection.md
index 2d49016..8566f18 100644
--- a/_docs/odbc-jdbc-interfaces/configuring-odbc/040-testing-the-odbc-connection.md
+++ b/_docs/odbc-jdbc-interfaces/configuring-odbc/040-testing-the-odbc-connection.md
@@ -1,6 +1,6 @@
---
title: "Testing the ODBC Connection"
-date: 2017-06-06 23:26:13 UTC
+date: 2017-08-17 04:11:59 UTC
parent: "Configuring ODBC"
---
You can use DSN connection strings and DSN-less connection strings for your connections.
@@ -74,20 +74,6 @@ Example:
AuthenticationType=Kerberos;
KrbServiceHost=maprdriverdemo.example.com; KrbServiceName=drill
-###MapR-SASL Authentication Connections
-
-The following is the format of a DSN-less connection string for a Drillbit that requires MapR-SASL authentication:
-
-Driver=MapR Drill ODBC Driver;ConnectionType=Direct; Host=[*Server*];Port=[*PortNumber*];AuthenticationType=MapRSASL
-
-Example:
-
- Driver=MapR Drill ODBC Driver;
- ConnectionType=Direct;
- Host=192.168.227.169;
- Port=31010;
- AuthenticationType=MapRSASL
-
###Plain Authentication Connections
The following is the format of a DSN-less connection string for a Drillbit that requires Plain authentication:
http://git-wip-us.apache.org/repos/asf/drill/blob/a63f2e34/_docs/odbc-jdbc-interfaces/installing-the-odbc-driver/010-installing-the-driver-on-linux.md
----------------------------------------------------------------------
diff --git a/_docs/odbc-jdbc-interfaces/installing-the-odbc-driver/010-installing-the-driver-on-linux.md b/_docs/odbc-jdbc-interfaces/installing-the-odbc-driver/010-installing-the-driver-on-linux.md
index 4f7a691..8671cc1 100644
--- a/_docs/odbc-jdbc-interfaces/installing-the-odbc-driver/010-installing-the-driver-on-linux.md
+++ b/_docs/odbc-jdbc-interfaces/installing-the-odbc-driver/010-installing-the-driver-on-linux.md
@@ -1,9 +1,9 @@
---
title: "Installing the Driver on Linux"
-date: 2017-06-21 23:59:00 UTC
+date: 2017-08-17 04:12:04 UTC
parent: "Installing the ODBC Driver"
---
-Install the MapR Drill ODBC Driver on the machine from which you connect to
+Install the Drill ODBC Driver on the machine from which you connect to
the Drill service. You can install the 32- or 64-bit driver on Linux. Install
the version of the driver that matches the architecture of the client
application that you use to access Drill. The 64-bit editions of Linux support
@@ -11,15 +11,12 @@ application that you use to access Drill. The 64-bit editions of Linux support
Install the MapR Drill ODBC Driver on a system that meets the [system requirements]({{site.baseurl}}/docs/installing-the-driver-on-linux/#system-requirements), and then complete the following steps described in detail in this document:
- * [Step 1: Download the MapR Drill ODBC Driver]({{site.baseurl}}/docs/installing-the-driver-on-linux/#step-1:-download-the-mapr-drill-odbc-driver)
- * [Step 2: Install the MapR Drill ODBC Driver]({{site.baseurl}}/docs/installing-the-driver-on-linux/#step-2:-install-the-mapr-drill-odbc-driver)
- * [Step 3: Check the MapR Drill ODBC Driver version]({{site.baseurl}}/docs/installing-the-driver-on-linux/#step-3:-check-the-mapr-drill-odbc-driver-version)
+ * [Step 1: Download the Drill ODBC Driver]({{site.baseurl}}/docs/installing-the-driver-on-linux/#step-1:-download-the-mapr-drill-odbc-driver)
+ * [Step 2: Install the Drill ODBC Driver]({{site.baseurl}}/docs/installing-the-driver-on-linux/#step-2:-install-the-mapr-drill-odbc-driver)
+ * [Step 3: Check the Drill ODBC Driver version]({{site.baseurl}}/docs/installing-the-driver-on-linux/#step-3:-check-the-mapr-drill-odbc-driver-version)
Verify that your system meets the system requirements before you start.
-## Known Issue
-
-MapR-SASL is not supported on 32-bit machines.
## System Requirements
@@ -43,10 +40,10 @@ MapR-SASL is not supported on 32-bit machines.
To install the driver, you need Administrator privileges on the computer.
-## Step 1: Download the MapR Drill ODBC Driver
+## Step 1: Download the Drill ODBC Driver
Download the driver from the [download site](http://package.mapr.com/tools/MapR-ODBC/MapR_Drill/). The current version is 1.3.8.
-## Step 2: Install the MapR Drill ODBC Driver
+## Step 2: Install the Drill ODBC Driver
To install the driver, complete the following steps:
@@ -65,10 +62,10 @@ To install the driver, complete the following steps:
`zypper install RPMFileName`
-The MapR Drill ODBC Driver typically resolves dependencies automatically. If not, error messages during the installation indicate that the package manager in your Linux distribution cannot resolve the
+The Drill ODBC Driver typically resolves dependencies automatically. If not, error messages during the installation indicate that the package manager in your Linux distribution cannot resolve the
dependencies automatically. In this case, manually install the packages.
-The following table provides a list of the MapR Drill ODBC Driver file
+The following table provides a list of the Drill ODBC Driver file
locations and descriptions:
File| Description
@@ -78,7 +75,7 @@ File| Description
`/opt/mapr/drill/lib/32 `| 32-bit shared libraries directory (will be created if you install the 32-bit driver).
`/opt/mapr/drill/lib/64`| 64-bit shared libraries directory (will be created if you install the 64-bit driver)..
-## Step 3: Check the MapR Drill ODBC Driver Version
+## Step 3: Check the Drill ODBC Driver Version
To check the version of the driver you installed, use the following case-sensitive command on the terminal command line:
http://git-wip-us.apache.org/repos/asf/drill/blob/a63f2e34/_docs/odbc-jdbc-interfaces/installing-the-odbc-driver/020-installing-the-driver-on-mac-os-x.md
----------------------------------------------------------------------
diff --git a/_docs/odbc-jdbc-interfaces/installing-the-odbc-driver/020-installing-the-driver-on-mac-os-x.md b/_docs/odbc-jdbc-interfaces/installing-the-odbc-driver/020-installing-the-driver-on-mac-os-x.md
index f31551c..c29250f 100644
--- a/_docs/odbc-jdbc-interfaces/installing-the-odbc-driver/020-installing-the-driver-on-mac-os-x.md
+++ b/_docs/odbc-jdbc-interfaces/installing-the-odbc-driver/020-installing-the-driver-on-mac-os-x.md
@@ -1,16 +1,16 @@
---
title: "Installing the Driver on Mac OS X"
-date: 2017-06-21 23:59:00 UTC
+date: 2017-08-17 04:12:08 UTC
parent: "Installing the ODBC Driver"
---
-Install the MapR Drill ODBC Driver on the machine from which you connect to
+Install the Drill ODBC Driver on the machine from which you connect to
the Drill service.
-Install the MapR Drill ODBC Driver on a system that meets the [system requirements]({{site.baseurl}}/docs/installing-the-driver-on-mac-os-x/#system-requirements). Complete the following steps, described in detail in this document:
+Install the Drill ODBC Driver on a system that meets the [system requirements]({{site.baseurl}}/docs/installing-the-driver-on-mac-os-x/#system-requirements). Complete the following steps, described in detail in this document:
- * [Step 1: Download the MapR Drill ODBC Driver]({{site.baseurl}}/docs/installing-the-driver-on-mac-os-x/#step-1-download-the-mapr-drill-odbc-driver)
- * [Step 2: Install the MapR Drill ODBC Driver]({{site.baseurl}}/docs/installing-the-driver-on-mac-os-x/#step-2:-install-the-mapr-drill-odbc-driver)
- * [Step 3: Check the MapR Drill ODBC Driver Version]({{site.baseurl}}/docs/installing-the-driver-on-mac-os-x/#step-3:-check-the-mapr-drill-odbc-driver-version)
+ * [Step 1: Download the Drill ODBC Driver]({{site.baseurl}}/docs/installing-the-driver-on-mac-os-x/#step-1-download-the-drill-odbc-driver)
+ * [Step 2: Install the Drill ODBC Driver]({{site.baseurl}}/docs/installing-the-driver-on-mac-os-x/#step-2:-install-the-drill-odbc-driver)
+ * [Step 3: Check the Drill ODBC Driver Version]({{site.baseurl}}/docs/installing-the-driver-on-mac-os-x/#step-3:-check-the-drill-odbc-driver-version)
## System Requirements
@@ -24,16 +24,14 @@ To install the driver, you need Administrator privileges on the computer.
Example: `127.0.0.1 localhost`
-----------
-## Step 1: Download the MapR Drill ODBC Driver
+## Step 1: Download the Drill ODBC Driver
To download ODBC drivers that support both 32- and 64-bit client applications, click
-[MapR Drill ODBC Driver for Mac](http://package.mapr.com/tools/MapR-ODBC/MapR_Drill/).
+[Drill ODBC Driver for Mac](http://package.mapr.com/tools/MapR-ODBC/MapR_Drill/).
-----------
-## Step 2: Install the MapR Drill ODBC Driver
+## Step 2: Install the Drill ODBC Driver
To install the driver, complete the following steps:
@@ -43,14 +41,13 @@ To install the driver, complete the following steps:
4. When the installation completes, click **Close.**
-
-MapR Drill ODBC Driver files install in the following locations:
+Drill ODBC Driver files install in the following locations:
* `/Library/mapr/drill/ErrorMessages` – Error messages files directory
* `/Library/mapr/drill/Setup` – Sample configuration files directory
* `/Library/mapr/drill/lib` – Binaries directory
-## Step 3: Check the MapR Drill ODBC Driver Version
+## Step 3: Check the Drill ODBC Driver Version
To check the version of the driver you installed, use the following command on the terminal command line:
@@ -65,7 +62,6 @@ To display information about the iODBC driver manager installed on the machine,
/usr/local/iODBC/bin/iodbc-config
-
### Next Step
[Configuring ODBC on Mac OS X]({{ site.baseurl }}/docs/configuring-odbc-on-mac-os-x/).
http://git-wip-us.apache.org/repos/asf/drill/blob/a63f2e34/_docs/odbc-jdbc-interfaces/installing-the-odbc-driver/030-installing-the-driver-on-windows.md
----------------------------------------------------------------------
diff --git a/_docs/odbc-jdbc-interfaces/installing-the-odbc-driver/030-installing-the-driver-on-windows.md b/_docs/odbc-jdbc-interfaces/installing-the-odbc-driver/030-installing-the-driver-on-windows.md
index bd6ce3d..0ce2d5d 100644
--- a/_docs/odbc-jdbc-interfaces/installing-the-odbc-driver/030-installing-the-driver-on-windows.md
+++ b/_docs/odbc-jdbc-interfaces/installing-the-odbc-driver/030-installing-the-driver-on-windows.md
@@ -1,28 +1,24 @@
---
title: "Installing the Driver on Windows"
-date: 2017-06-21 23:59:01 UTC
+date: 2017-08-17 04:12:13 UTC
parent: "Installing the ODBC Driver"
---
-The MapR Drill ODBC Driver installer is available for 32- and 64-bit
+The Drill ODBC Driver installer is available for 32- and 64-bit
applications on Windows®. On 64-bit Windows operating systems, you can execute both 32- and 64-bit applications. However, 64-bit applications must use 64-bit drivers, and 32-bit applications must use 32-bit drivers. Make sure that you use the driver version that matches the bitness of the client application machine.
- * MapR Drill 1.3 32-bit.msi for 32-bit applications
- * MapR Drill 1.3 64-bit.msi for 64-bit applications
+ * Drill 1.3 32-bit.msi for 32-bit applications
+ * Drill 1.3 64-bit.msi for 64-bit applications
{% include startnote.html %}Currently Drill does not support a 32-bit Windows machine. However, the 32- or 64-bit MapR Drill ODBC Driver is supported on a 64-bit machine.{% include endnote.html %}
-Install the MapR Drill ODBC Driver on a system that meets the system requirements. Complete the following steps, described in detail in this document:
+To install the Drill ODBC Driver on a system that meets the system requirements, complete the following steps:
- * [Step 1: Download the MapR Drill ODBC Driver]({{site.baseurl}}/docs/installing-the-driver-on-windows/#step-1:-download-the-mapr-drill-odbc-driver)
- * [Step 2: Install the MapR Drill ODBC Driver]({{site.baseurl}}/docs/installing-the-driver-on-windows/#step-2:-install-the-mapr-drill-odbc-driver)
+ * [Step 1: Download the Drill ODBC Driver]({{site.baseurl}}/docs/installing-the-driver-on-windows/#step-1:-download-the-drill-odbc-driver)
+ * [Step 2: Install the Drill ODBC Driver]({{site.baseurl}}/docs/installing-the-driver-on-windows/#step-2:-install-the-drill-odbc-driver)
* [Step 3: Verify the installation]({{site.baseurl}}/docs/installing-the-driver-on-windows/#step-3:-verify-the-installation)
-## Known Issue
-
-The 32-bit driver does not support MapR-SASL. The 32-bit driver supports Plain and Kerberos authentication only.
-
## System Requirements
Each computer where you install the driver must meet the following system
@@ -49,7 +45,7 @@ To install the driver, you need Administrator privileges on the computer.
----------
-## Step 1: Download the MapR Drill ODBC Driver
+## Step 1: Download the Drill ODBC Driver
Download the installer that corresponds to the bitness of the client application from which you want to create an ODBC connection. The current version is 1.3.8.
@@ -58,7 +54,7 @@ Download the installer that corresponds to the bitness of the client application
----------
-## Step 2: Install the MapR Drill ODBC Driver
+## Step 2: Install the Drill ODBC Driver
1. Double-click the installer from the location where you downloaded it.
2. Click **Next.**
@@ -76,7 +72,7 @@ To verify the installation on Windows 10, perform the following steps:
1\. Go to **Settings** and enter **odbc**.
-2\. Select **Set up ODBC data sources `<version>`**. The **ODBC Data Source Administrator `<version>`** dialog appears. Click the **System DSN** to view the MapR Drill data source.
+2\. Select **Set up ODBC data sources `<version>`**. The **ODBC Data Source Administrator `<version>`** dialog appears. Click the **System DSN** to view the Drill data source.
http://git-wip-us.apache.org/repos/asf/drill/blob/a63f2e34/_docs/performance-tuning/query-plans-and-tuning/050-sort-based-and-hash-based-memory-constrained-operators.md
----------------------------------------------------------------------
diff --git a/_docs/performance-tuning/query-plans-and-tuning/050-sort-based-and-hash-based-memory-constrained-operators.md b/_docs/performance-tuning/query-plans-and-tuning/050-sort-based-and-hash-based-memory-constrained-operators.md
index 9b7aabc..ed2a91c 100644
--- a/_docs/performance-tuning/query-plans-and-tuning/050-sort-based-and-hash-based-memory-constrained-operators.md
+++ b/_docs/performance-tuning/query-plans-and-tuning/050-sort-based-and-hash-based-memory-constrained-operators.md
@@ -1,41 +1,77 @@
---
title: "Sort-Based and Hash-Based Memory-Constrained Operators"
-date: 2016-02-06 00:18:13 UTC
+date: 2017-08-17 04:12:18 UTC
parent: "Query Plans and Tuning"
---
-Drill uses hash-based and sort-based operators depending on the query characteristics. Hash aggregation and hash join are hash-based operations. Streaming aggregation and merge join are sort-based operations. Both hash-based and sort-based operations consume memory, however the hash aggregate and hash join operators are the fastest and most memory intensive operators.
+Drill uses hash-based and sort-based operators depending on the query characteristics. Hash aggregation and hash join are hash-based operations. Streaming aggregation and merge join are sort-based operations. Both hash-based and sort-based operations consume memory, however the hash aggregate and hash join operators are the fastest and most memory intensive operators.
+
+When planning a query with sort- and hash-based operators, Drill evaluates the available memory multiplied by a configurable reduction constant (for parallelization purposes) and then limits the operations to the maximum of this amount of memory. Drill spills data to disk if the sort and hash aggregate operations cannot be performed in memory. Alternatively, you can disable large hash operations if they do not fit in memory on your system. When disabled, Drill creates alternative plans. You can also modify the minimum hash table size, increasing the size for very large aggregations or joins when you have large amounts of memory for Drill to use. If you have large data sets, you can increase the hash table size to improve performance.
+
+##Memory Options
+The `planner.memory.max_query_memory_per_node` option sets the maximum amount of direct memory allocated to the sort and hash aggregate operators during each query on a node. The default limit is 2147483648 bytes (2GB), which is quite conservative. This memory is split between operators. If a query plan contains multiple sort and/or hash aggregate operators, the memory is divided between them.
+
+When a query is parallelized, the number of operators is multiplied, which reduces the amount of memory given to each instance of the sort and hash aggregate operators during a query. If you encounter memory issues when running queries with sort and hash aggregate operators, calculate the memory requirements for your queries and the amount of available memory on each node. Based on the information, increase the value for the `planner.memory.max_query_memory_per_node` option using the ALTER SYSTEM|SESSION SET command, as shown:
+
+ ALTER SYSTEM|SESSION SET `planner.memory.max_query_memory_per_node` = 8147483648
+
+
+The `planner.memory.enable_memory_estimation` option toggles the state of memory estimation and re-planning of the query. When enabled, Drill conservatively estimates memory requirements and typically excludes memory-constrained operators from the query plan, which can negatively impact performance. The default setting is false. If you want Drill to use very conservative memory estimates, use the ALTER SYSTEM|SESSION SET command to change the setting, as shown:
+
+ ALTER SYSTEM|SESSION SET `planner.memory.enable_memory_estimation` = true
+
-Currently, hash-based operations do not spill to disk as needed, but the sort-based operations do. When Drill plans a sort-based query, it evaluates the size of available memory multiplied by a configurable reduction constant (for parallelization purposes) and then limits the sort-based operations to the maximum of this amount of memory.
+##Spill to Disk
+The "Spill to Disk" feature prevents queries that use memory-intensive sort and hash aggregate operations from failing with out-of-memory errors. Drill automatically writes excess data to a temporary directory on disk when queries with sort or hash aggregate operations exceed the set memory limit on a Drill node. When the operators finish processing the in-memory data, Drill reads the spilled data back from disk, and the operators finish processing the data. When the operations complete, Drill removes the data from disk.
+
+Spilling to disk enables queries to run uninterrupted while Drill performs the spill operations in the background. However, there can be performance impact due to the time required to spill data and then read the data back from disk.
-If the hash-based operators run out of memory during execution, the query fails. If large hash operations do not fit in memory on your system, you can disable these operations. When disabled, Drill creates alternative plans that allow spilling to disk.
+{% include startnote.html %}Drill 1.11 and later supports spilling to disk for the Hash Aggregate operator in addition to the Sort operator. Previous releases of Drill only supported spilling to disk for the Sort operator.{% include endnote.html %}
-You can also modify the minimum hash table size, increasing the size for very large aggregations or joins when you have large amounts of memory for Drill to use. If you have large data sets, you can increase this hash table size to improve performance.
+###Spill Locations
+Drill writes data to a temporary work area on disk. The default location of the temporary work area is /tmp/drill/spill on the local file system. The /tmp/drill/spill directory should suffice for small workloads or examples, however it is highly recommended that you redirect the default spill location to a location with enough disk space to support spilling for large workloads.
-Use the ALTER SYSTEM or ALTER SESSION commands with the options in the table below to disable the hash aggregate and hash join operators, modify the hash table size, disable memory estimation, or set the estimated maximum amount of memory for a query. Typically, you set the options at the session level unless you want the setting to persist across all sessions.
+{% include startnote.html %}Spilled data may require more space than the table referenced in the query that is spilling the data. For example, if a table is 100 GB per node, the spill directory should have the capacity to hold more than 100 GB.{% include endnote.html %}
+
+When you configure the spill location, you can specify a single directory, or a list of directories into which the sort and hash aggregate operators both spill. Alternatively, you can set specific spill directories for each type of operator, however this is not recommended as these options will be deprecated in future releases of Drill. For more information, see the Spill to Disk Configuration Options section below.
+
+###Spill to Disk Configuration Options
+The spill to disk options reside in the drill-override.conf file on each Drill node. An administrator or someone familiar with storage and disks should manage these settings.
+
+{% include startnote.html %}You can see examples of these configuration options in the drill-override-example.conf file located in the <drill_installation>/conf directory.{% include endnote.html %}
+
+The following table lists the configuration options for spilling data:
+
+| Option | Description | Default | Example |
+|--------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------|-----------------------------------------------------------------------|
+| drill.exe.spill.fs | Introduced in Drill 1.11. The default file system on the local machine into which the sort and hash aggregate operators spill data. This is the recommended option to use for spilling. You can configure this option so that data spills into a distributed file system, such as hdfs. | "file:///" | "hdfs:///" |
+| drill.exec.spill.directories | Introduced in Drill 1.11. The list of directories into which the sort and hash aggregate operators spill data. The list must be an array with directories separated by a comma. This is the recommended option for spilling to multiple directories. | [ "/tmp/drill/spill" ], | [ "/fs1/drill/spill" , "/fs2/drill/spill" , "/fs3/drill/spill" ], |
+| drill.exec.sort.external.spill.fs | Overrides the default location into which the sort operator spills data. Instead of spilling into the location set by the drill.exec.spill.fs option, the sort operators spill into the location specified by this option. Note: As of Drill 1.11, this option is supported for backward compatibility, however in future releases, this option will be deprecated. It is highly recommended that you use the drill.exec.spill.fs option to set the spill location instead. | "file:///" | "hdfs:///" |
+| drill.exec.sort.external.spill.directories | Overrides the location into which the sort operator spills data. Instead of spilling into the location set by the drill.exec.spill.directories option, the sort operators spill into the directories specified by this option. The list must be an array with directories separated by a comma. Note: As of Drill 1.11, this option is supported for backward compatibility, however in future releases, this option will be deprecated. It is highly recommended that you use the drill.exec.spill.directories option to set the spill location instead. | [ "/tmp/drill/spill" ] | ["/fs1/drill/spill" , "/fs2/drill/spill" , "/fs3/drill/spill" ], |
+| drill.exec.hashagg.spill.fs | Overrides the location into which the hash aggregate operator spills data. Instead of spilling into the location set by the drill.exec.spill.fs option, the hash aggregate operator spills into the location specified by this option. Setting this option to 1 disables spilling for the hash aggregate operator. Note: As of Drill 1.11, this option is supported for backward compatibility, however in future releases, this option will be deprecated. It is highly recommended that you use the drill.exec.spill.fs option to set the spill location instead. | "file:///" | "hdfs:///" |
+| drill.exec.hashagg.spill.directories | Overrides the location into which the hash aggregate operator spills data. Instead of spilling into the location set by the drill.exec.spill.directories option, the hash aggregate operator spills to the directories specified by this option. The list must be an array with directories separated by a comma. Note: As of Drill 1.11, this option is supported for backward compatibility, however in future releases, this option will be deprecated. It is highly recommended that you use the drill.exec.spill. directories option to set the spill location instead. | [ "/tmp/drill/spill" ], | ["/fs1/drill/spill" , "/fs2/drill/spill" , "/fs3/drill/spill" ], |
+
+##Hash-Based Operator Settings
+Use the ALTER SYSTEM|SESSION SET commands with the options below to disable the hash aggregate and hash join operators, modify the hash table size, disable memory estimation, or set the estimated maximum amount of memory for a query. Typically, you set the options at the session level unless you want the setting to persist across all sessions.
The following options control the hash-based operators:
* **planner.enable_hashagg**
- Enable hash aggregation; otherwise, Drill does a sort-based aggregation. Does not write to disk. Enable is recommended. Default: true
+ Enables or disables hash aggregation; otherwise, Drill does a sort-based aggregation. This option is enabled by default. The default setting is true, which is recommended.
* **planner.enable_hashjoin**
- Enable the memory hungry hash join. Drill assumes that a query will have adequate memory to complete and tries to use the fastest operations possible to complete the planned inner, left, right, or full outer joins using a hash table. Does not write to disk. Disabling hash join allows Drill to manage arbitrarily large data in a small memory footprint. Default: true
+ Enables or disables the memory hungry hash join. Drill assumes that a query will have adequate memory to complete and tries to use the fastest operations possible to complete the planned inner, left, right, or full outer joins using a hash table. Currently, this operator does not write to disk. Disabling hash join allows Drill to manage arbitrarily large data in a small memory footprint. This option is enabled by default. The default setting is true.
* **exec.min_hash_table_size**
- Starting size for hash tables. Increase according to available memory to improve performance.
- Default: 65536 Range: 0 - 1073741824
+ Starting size for hash tables. Increase this setting based on the memory available to improve performance.
+ The default setting for this option is 65536. The setting can range from 0 to 1073741824.
* **exec.max\_hash\_table_size**
- Ending size for hash tables.
- Default: 1073741824 Range: 0 - 1073741824
+ Ending size for hash tables. The default setting for this option is 1073741824. The setting can range from 0 to 1073741824.
+
+
+
-* **planner.memory.enable\_memory_estimation**
- Toggles the state of memory estimation and re-planning of the query. When enabled, Drill conservatively estimates memory requirements and typically excludes memory-constrained operators from the plan and negatively impacts performance.
- Default: false
-* **planner.memory.max\_query\_memory\_per_node**
- Sets the maximum amount of direct memory allocated to the sort operator in each query on a node. If a query plan contains multiple sort operators, they all share this memory. If you encounter memory issues when running queries with sort operators, increase the value of this option.
- Default: 2147483648