You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@drill.apache.org by kr...@apache.org on 2016/01/07 01:40:58 UTC
drill git commit: redirect example
Repository: drill
Updated Branches:
refs/heads/gh-pages 263c11334 -> 3f0a7d609
redirect example
add config for redirect
add instructions for redirect
MD-643
Rahul's review
Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/3f0a7d60
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/3f0a7d60
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/3f0a7d60
Branch: refs/heads/gh-pages
Commit: 3f0a7d609af0ef2848f102ca4e5d9f3fd66b143c
Parents: 263c113
Author: Kris Hahn <kr...@apache.org>
Authored: Wed Jan 6 15:45:23 2016 -0800
Committer: Kris Hahn <kr...@apache.org>
Committed: Wed Jan 6 16:40:26 2016 -0800
----------------------------------------------------------------------
README.md | 22 ++++++-
_config.yml | 3 +
.../plugins/070-hive-storage-plugin.md | 61 ++++++++++++++------
_docs/sample-datasets/001-aol.md | 6 +-
4 files changed, 71 insertions(+), 21 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/drill/blob/3f0a7d60/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index 49df1fa..2d3f0b7 100644
--- a/README.md
+++ b/README.md
@@ -11,6 +11,8 @@ jekyll serve --config _config.yml,_config-prod.yml
```
Note that you can skip the first two commands (and only run `jekyll serve`) if you haven't changed the title or path of any of the documentation pages.
+## One Time Setup for Last-Modified-Date
+
To automatically add the last-modified-on date, a one-time local setup is required:
1. In your cloned directory of Drill, in drill/.git/hooks, create a file named pre-commit (no extension) that contains this script:
@@ -28,7 +30,7 @@ To automatically add the last-modified-on date, a one-time local setup is requir
chmod +x pre-commit
-In addition to the title: and parent:, you now need to add date: to the front matter of any file you create. For example:
+On any page you create, in addition to the title: and parent:, you now need to add date: to the front matter of any file you create. For example:
---
title: "Configuring Multitenant Resources"
@@ -36,7 +38,23 @@ In addition to the title: and parent:, you now need to add date: to the front ma
date:
---
-Do not fill in or alter the date: field. Jekyll and git take care of that when you commit the file.
+Do not fill in or alter the date: field. Jekyll and git take care of that when you commit the file.
+
+## One Time Setup for Redirecting gh-pages
+
+Locally install the jekyll-redirect-from gem:
+
+ gem install jekyll-redirect-from
+
+On any page you want to redirect, add the redirect_to: and the URL to the front matter. For example:
+
+ ---
+ title: "Configuring Multitenant Resources"
+ parent: "Configuring a Multitenant Cluster"
+ date:
+ redirect_to:
+ - http://<new_url>
+ ---
# Compiling the Website
http://git-wip-us.apache.org/repos/asf/drill/blob/3f0a7d60/_config.yml
----------------------------------------------------------------------
diff --git a/_config.yml b/_config.yml
index f6ffe66..9a6aa16 100644
--- a/_config.yml
+++ b/_config.yml
@@ -30,3 +30,6 @@ defaults:
sass:
style: :compressed
+
+gems:
+ - jekyll-redirect-from
http://git-wip-us.apache.org/repos/asf/drill/blob/3f0a7d60/_docs/connect-a-data-source/plugins/070-hive-storage-plugin.md
----------------------------------------------------------------------
diff --git a/_docs/connect-a-data-source/plugins/070-hive-storage-plugin.md b/_docs/connect-a-data-source/plugins/070-hive-storage-plugin.md
index 32643e5..4c6b699 100644
--- a/_docs/connect-a-data-source/plugins/070-hive-storage-plugin.md
+++ b/_docs/connect-a-data-source/plugins/070-hive-storage-plugin.md
@@ -1,6 +1,6 @@
---
title: "Hive Storage Plugin"
-date:
+date: 2016-01-07
parent: "Connect a Data Source"
---
Drill 1.1 and later supports Hive 1.0. To access Hive tables
@@ -8,22 +8,12 @@ using custom SerDes or InputFormat/OutputFormat, all nodes running Drillbits
must have the SerDes or InputFormat/OutputFormat `JAR` files in the
`<drill_installation_directory>/jars/3rdparty` folder.
-## Hive Remote Metastore Configuration
+You can run Hive queries in the following ways by configuring the Hive storage plugin as described in this document:
-The Hive metastore runs as a separate service outside
-of Hive. Drill communicates with the Hive metastore through Thrift. The
-metastore service communicates with the Hive database over JDBC. Point Drill
-to the Hive metastore service address, and provide the connection parameters
-in a Hive storage plugin configuration to configure a connection to Drill.
-
-{% include startnote.html %}Verify that the Hive metastore service is running before you register the Hive metastore.{% include endnote.html %}
+* [Connect Drill to the Hive remote metastore]({{site.baseurl}}/docs/hive-storage-plugin/#connect-drill-to-the-hive-remote-metastore-directly)
+* [Connect to the Hive embedded metastore]({{site.baseurl}}/docs/hive-storage-plugin/#connect-to-the-hive-embedded-metastore)
-To register a remote Hive metastore with Drill:
-
-1. Issue the following command to start the Hive metastore service on the system specified in the `hive.metastore.uris`:
- `hive --service metastore`
-2. In the [Drill Web Console]({{ site.baseurl }}/docs/plugin-configuration-basics/#using-the-drill-web-console), select the **Storage** tab.
-3. In the list of disabled storage plugins in the Drill Web Console, click **Update** next to `hive`. The Hive storage plugin configuration appears:
+You update the Hive storage plugin by selecting the **Storage tab** on the [Drill Web Console]({{ site.baseurl }}/docs/plugin-configuration-basics/#using-the-drill-web-console). From the list of disabled storage plugins in the Drill Web Console, click **Update** next to `hive`. The default Hive storage plugin configuration appears as follows:
{
"type": "hive",
@@ -36,6 +26,27 @@ To register a remote Hive metastore with Drill:
"hive.metastore.sasl.enabled": "false"
}
}
+
+## Connect Drill to the Hive Remote Metastore
+
+The Hive metastore runs as a separate service outside
+of Hive. Drill can query the Hive metastore through Thrift. The
+metastore service communicates with the Hive database over JDBC.
+
+Follow the steps in the next section to point Drill
+to the Hive metastore service address. Provide the connection parameters
+in a Hive storage plugin configuration to configure a connection to Drill. At this point, if you query data sources that Drill supports other than HBase (or MapR), you are finished configuring the Hive storage plugin. If you query HBase using Hive, you need to add ZooKeeper quorum and port properties. The HBaseStorageHandler requires these properties. Drill discovers HBase services using these properties. If you use the HBase storage plugin, the ZooKeeper quorum and port properties in the Hive storage plugin are the same as those in the HBase storage plugin, assuming you want to use the same HBase database.
+
+{% include startnote.html %}Verify that the Hive metastore service is running before you register the Hive metastore.{% include endnote.html %}
+
+### Hive Remote Metastore Configuration
+
+To connect Drill to a remote Hive metastore:
+
+1. Issue the following command to start the Hive metastore service on the system specified in the `hive.metastore.uris`:
+ `hive --service metastore`
+2. In the [Drill Web Console]({{ site.baseurl }}/docs/plugin-configuration-basics/#using-the-drill-web-console), select the **Storage** tab.
+3. In the list of disabled storage plugins in the Drill Web Console, click **Update** next to `hive`.
4. In the configuration window, add the `Thrift URI` and port to `hive.metastore.uris`. For example:
...
@@ -55,15 +66,31 @@ To register a remote Hive metastore with Drill:
}
}
-6. Click **Enable**.
+6. If you do not query HBase, skip this step. If you query HBase, in the configuration window, add the names of the ZooKeeper quorum hosts and the ZooKeeper port, for example 2181.
-## Hive Embedded Metastore Configuration
+ {
+ "type": "hive",
+ "enabled": false,
+ "configProps": {
+ .
+ .
+ .
+ "hbase.zookeeper.quorum": "zkhost1,zkhost2,zkhost3",
+ "hbase.zookeeper.property.clientPort:" "2181"
+ }
+ }
+
+7. Click **Enable**.
+
+## Connect to the Hive embedded metastore
The Hive metastore configuration is embedded within the Drill process. Configure an embedded metastore only in a cluster that runs a single Drillbit and only for testing purposes. Do not embed the Hive metastore in production systems.
Provide the metastore database configuration settings in the Drill Web Console. Before you configure an embedded Hive metastore, verify that the driver you use to connect to the Hive metastore is in the Drill classpath located in `/<drill installation directory>/lib/.` If the driver is not there, copy the driver to `/<drill
installation directory>/lib` on the Drill node. For more information about storage types and configurations, refer to ["Hive Metastore Administration"](https://cwiki.apache.org/confluence/display/Hive/AdminManual+MetastoreAdmin).
+### Hive Embedded Metastore Configuration
+
To configure an embedded Hive metastore, complete the following
steps:
http://git-wip-us.apache.org/repos/asf/drill/blob/3f0a7d60/_docs/sample-datasets/001-aol.md
----------------------------------------------------------------------
diff --git a/_docs/sample-datasets/001-aol.md b/_docs/sample-datasets/001-aol.md
index 4948cbf..d9b8eed 100644
--- a/_docs/sample-datasets/001-aol.md
+++ b/_docs/sample-datasets/001-aol.md
@@ -1,6 +1,8 @@
---
title: "AOL Search"
-date:
+date: 2016-01-06
+redirect_to:
+ - http://gregsadetsky.com/aol-data
parent: "Sample Datasets"
---
## Quick Stats
@@ -12,7 +14,7 @@ a collection of real query log data that is based on real users.
The dataset consists of 20M Web queries from 650k users over a period of three
months, 440MB in total and available [for
-download](http://zola.di.unipi.it/smalltext/datasets.html). The format used in
+download](http://gregsadetsky.com/aol-data/). The format used in
the dataset is:
AnonID, Query, QueryTime, ItemRank, ClickURL