You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@drill.apache.org by br...@apache.org on 2018/03/18 20:04:11 UTC
[1/3] drill git commit: test
Repository: drill
Updated Branches:
refs/heads/gh-pages c7e28e75e -> d8dd5d3db
test
Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/35fa5541
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/35fa5541
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/35fa5541
Branch: refs/heads/gh-pages
Commit: 35fa55418cdb4bfff829faab0bd29d7f802ffb7e
Parents: c7e28e7
Author: Bridget Bevens <bb...@maprtech.com>
Authored: Thu Mar 15 10:08:42 2018 -0700
Committer: Bridget Bevens <bb...@maprtech.com>
Committed: Thu Mar 15 10:08:42 2018 -0700
----------------------------------------------------------------------
...d-hash-based-memory-constrained-operators.md | 4 +-
_docs/rn/001-1.12.0-rn.md | 430 -------------------
2 files changed, 2 insertions(+), 432 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/drill/blob/35fa5541/_docs/performance-tuning/query-plans-and-tuning/050-sort-based-and-hash-based-memory-constrained-operators.md
----------------------------------------------------------------------
diff --git a/_docs/performance-tuning/query-plans-and-tuning/050-sort-based-and-hash-based-memory-constrained-operators.md b/_docs/performance-tuning/query-plans-and-tuning/050-sort-based-and-hash-based-memory-constrained-operators.md
index 3e5b305..b8d1dda 100644
--- a/_docs/performance-tuning/query-plans-and-tuning/050-sort-based-and-hash-based-memory-constrained-operators.md
+++ b/_docs/performance-tuning/query-plans-and-tuning/050-sort-based-and-hash-based-memory-constrained-operators.md
@@ -1,6 +1,6 @@
---
title: "Sort-Based and Hash-Based Memory-Constrained Operators"
-date: 2018-03-14 01:21:10 UTC
+date: 2018-03-15 17:08:44 UTC
parent: "Query Plans and Tuning"
---
@@ -19,7 +19,7 @@ When the Sort, Hash Aggregate, and Hash Join operators finish processing the dat
Ideally, you want to allocate enough memory for Drill to perform all operations in memory. When data spills to disk, you will not see any difference in terms of how queries run, however spilling to disk can impact performance due to the additional I/O required to write data to disk and read the data back. See Memory Allocation (page 4) for more information.
-**Note:** Drill 1.14 and later supports spilling to disk for the Hash Join, Hash Aggregate, and Sort operators. Drill 1.11 and 1.12 supports spilling to disk for the Hash Aggregate and Sort operators. Releases of Drill prior to 1.11 only support spilling to disk for the Sort operator.
+**Note:** Drill 1.14 and later supports spilling to disk for the Hash Join, Hash Aggregate, and Sort operators. Drill 1.11, 1.12, and 1.13 supports spilling to disk for the Hash Aggregate and Sort operators. Releases of Drill prior to 1.11 only support spilling to disk for the Sort operator.
**Spill Locations**
http://git-wip-us.apache.org/repos/asf/drill/blob/35fa5541/_docs/rn/001-1.12.0-rn.md
----------------------------------------------------------------------
diff --git a/_docs/rn/001-1.12.0-rn.md b/_docs/rn/001-1.12.0-rn.md
deleted file mode 100644
index 3c8fe4e..0000000
--- a/_docs/rn/001-1.12.0-rn.md
+++ /dev/null
@@ -1,430 +0,0 @@
----
-title: "Apache Drill 1.12.0 Release Notes"
-parent: "Release Notes"
----
-
-**Release date:** December 15, 2017
-
-Today, we're happy to announce the availability of Drill 1.12.0. You can download it [here](https://drill.apache.org/download/).
-
-## New Features and Improvements
-This release of Drill provides the following new features and improvements:
-
-- Kafka and OpenTSDB storage plugins (DRILL-4779, DRILL-5337)
-- SSL/TLS support (DRILL-5431)
-- Network encryption support (DRILL-5682)
-- Queue-based memory assignment for buffering operators (DRILL-5716)
-- A collection of networking functions that facilitate network analysis using Drill (DRILL-5834)
-- Support for the libpam4j PAM authenticator (DRILL-5820)
-- Filter pushdown for Parquet can handle files with multiple rowgroups (DRILL-5795)
-- UTF-8 is enabled in the query string by default (DRILL-5772)
-- IF NOT EXISTS support for CREATE TABLE and CREATE VIEWS (DRILL-5952)
-- Geometry functions, `ST_AsGeoJSON` and `ST_AsJSON`, that return GeoJSON and JSON representations (DRILL-5962, DRILL-5960)
-- JMX metrics for failed and canceled queries (DRILL-5909)
-- Syntax highlighting and error checking for storage plugin configurations (DRILL-5981)
-- System options improvements, including a new internal system options table (DRILL-5723)
-- Ability to prevent users from accessing a path outside the current workspace (DRILL-5964)
-- Ability to put the server in quiescent mode for a graceful shutdown (DRILL-4286)
-- The Drill Web Console lists the completion of successfully completed queries as "successful" (DRILL-5923)
-
-
-
-The following sections list additional bug fixes and improvements:
-
-<h2> Sub-task
-</h2>
-<ul>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5253'>DRILL-5253</a>] - External sort fails with OOM error (Fails to allocate sv2)
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5801'>DRILL-5801</a>] - Label the Gantt chart that indicates the lifespan details of the various minor fragments
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5802'>DRILL-5802</a>] - Provide a sortable table for tables within a query profile
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5803'>DRILL-5803</a>] - Show the hostname for each minor fragment in operator table
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5867'>DRILL-5867</a>] - List profiles in pages rather than a long verbose listing
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5881'>DRILL-5881</a>] - Java Client: [Threat Modeling] Drillbit may be spoofed by an attacker and this may lead to data being written to the attacker's target instead of Drillbit
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5882'>DRILL-5882</a>] - C++ Client: [Threat Modeling] Drillbit may be spoofed by an attacker and this may lead to data being written to the attacker's target instead of Drillbit
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6031'>DRILL-6031</a>] - Document support for dots in field names
-</li>
-</ul>
-
-<h2> Bug
-</h2>
-<ul>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1051'>DRILL-1051</a>] - Casting timestamp as date gives wrong result for dates earlier than 1883
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1499'>DRILL-1499</a>] - Different column order could appear in the result set for a schema-less select * query, even there are no changing schemas.
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3241'>DRILL-3241</a>] - Query with window function runs out of direct memory and does not report back to client that it did
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3407'>DRILL-3407</a>] - CTAS Auto Partition : The plan for count(*) should show the list of files scanned
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3449'>DRILL-3449</a>] - When Foreman node dies, the FragmentExecutor still tries to send status updates to Foreman
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3827'>DRILL-3827</a>] - Empty metadata file causes queries on the table to fail
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3829'>DRILL-3829</a>] - Metadata Caching : Drill should ignore a corrupted cache file
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4139'>DRILL-4139</a>] - Fix parquet partition pruning for BIT, INTERVAL and DECIMAL types
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4255'>DRILL-4255</a>] - SELECT DISTINCT query over JSON data returns UNSUPPORTED OPERATION
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4640'>DRILL-4640</a>] - Unable to submit physical plan from Web UI on Windows
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4686'>DRILL-4686</a>] - Aggregation query over HBase table results in IllegalStateException: Failure while reading vector
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4734'>DRILL-4734</a>] - Query against HBase table on a 5 node cluster fails with SchemaChangeException
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4735'>DRILL-4735</a>] - Count(dir0) on parquet returns 0 result
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5002'>DRILL-5002</a>] - Using hive's date functions on top of date column gives wrong results for local time-zone
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5146'>DRILL-5146</a>] - Unnecessary spilling to disk by sort when we only have 5000 rows with one column
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5166'>DRILL-5166</a>] - Select with options returns NPE
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5185'>DRILL-5185</a>] - Union all not passing type info when the output contains 0 rows
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5268'>DRILL-5268</a>] - SYSTEM ERROR: UnsupportedOperationException: Unable to get size for minor type [MAP] and mode [REQUIRED]
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5269'>DRILL-5269</a>] - SYSTEM ERROR: JsonMappingException: No suitable constructor found for type [simple type, class org.apache.drill.exec.store.direct.DirectSubScan]
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5327'>DRILL-5327</a>] - Hash aggregate can return empty batch which can cause schema change exception
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5339'>DRILL-5339</a>] - Web UI: flaws in query profile display on error
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5341'>DRILL-5341</a>] - Web UI: remove duplicating link to documentation in Options page
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5346'>DRILL-5346</a>] - Web UI: remove link on user in query profile list
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5357'>DRILL-5357</a>] - Partition pruning information not available in query plan for COUNT aggregate query
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5416'>DRILL-5416</a>] - Vectors read from disk report incorrect memory sizes
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5442'>DRILL-5442</a>] - Managed Sort: IndexOutOfBounds with a join over an inlist
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5443'>DRILL-5443</a>] - Managed External Sort fails with OOM while spilling to disk
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5445'>DRILL-5445</a>] - Assertion Error in Managed External Sort when dealing with repeated maps
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5447'>DRILL-5447</a>] - Managed External Sort : Unable to allocate sv2 vector
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5464'>DRILL-5464</a>] - Fix JSON reader when it deals with empty file
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5465'>DRILL-5465</a>] - Managed external sort results in an OOM
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5478'>DRILL-5478</a>] - Spill file size parameter is not honored by the managed external sort
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5480'>DRILL-5480</a>] - Empty batch returning from HBase may cause SchemaChangeException even when data does not have different schema
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5507'>DRILL-5507</a>] - Millions of "Failure finding Drillbit running on host" info messages in foreman logs
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5513'>DRILL-5513</a>] - Managed External Sort : OOM error during the merge phase
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5519'>DRILL-5519</a>] - Sort fails to spill and results in an OOM
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5525'>DRILL-5525</a>] - Inconsistent, unhelpful semantics for batch, field schema comparison
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5546'>DRILL-5546</a>] - Schema change problems caused by empty batch
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5547'>DRILL-5547</a>] - Drill config options and session options do not work as intended
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5564'>DRILL-5564</a>] - IllegalStateException: allocator[op:21:1:5:HashJoinPOP]: buffer space (16674816) + prealloc space (0) + child space (0) != allocated (16740352)
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5582'>DRILL-5582</a>] - [Threat Modeling] Drillbit may be spoofed by an attacker and this may lead to data being written to the attacker's target instead of Drillbit
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5594'>DRILL-5594</a>] - Excessive buffer reallocations during merge phase of external sort
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5597'>DRILL-5597</a>] - Incorrect "bits" vector allocation in nullable vectors allocateNew()
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5602'>DRILL-5602</a>] - Repeated List Vector fails to initialize the offset vector
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5617'>DRILL-5617</a>] - Spill file name collisions when spill file is on a shared file system
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5645'>DRILL-5645</a>] - negation of expression causes null pointer exception
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5660'>DRILL-5660</a>] - Drill 1.10 queries fail due to Parquet Metadata "corruption" from DRILL-3867
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5663'>DRILL-5663</a>] - Drillbit fails to start when only keystore path is provided without keystore password.
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5675'>DRILL-5675</a>] - Drill C++ Client Date Time Literals Support Metadata Mapping is Incorrect
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5686'>DRILL-5686</a>] - Warning for sasl.max_wrapped_size contain incorrect syntax
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5687'>DRILL-5687</a>] - Disable TestMergeJoinWithSchemaChanges#testMissingAndNewColumns
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5694'>DRILL-5694</a>] - hash agg spill to disk, second phase OOM
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5698'>DRILL-5698</a>] - Drill should start in embedded mode using java 1.8.0_144
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5699'>DRILL-5699</a>] - Drill Web UI Page Source Has Links To External Sites
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5701'>DRILL-5701</a>] - drill.connections.rpc.<user/control/data>.<encrypted/unencrypted> metric not behaving correctly
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5702'>DRILL-5702</a>] - Jdbc Driver Class not found
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5710'>DRILL-5710</a>] - drill-config.sh incorrectly exits with Java 1.7 or later is required to run Apache Drill
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5714'>DRILL-5714</a>] - Fix NPE when mapr-db plugin is used in table function
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5715'>DRILL-5715</a>] - Performance of refactored HashAgg operator regressed
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5717'>DRILL-5717</a>] - change some date time unit cases with specific timezone or Local
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5721'>DRILL-5721</a>] - Query with only root fragment and no non-root fragment hangs when Drillbit to Drillbit Control Connection has network issues
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5725'>DRILL-5725</a>] - Update Jackson version to 2.7.8
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5727'>DRILL-5727</a>] - Update release profile to generate SHA-512 checksum.
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5729'>DRILL-5729</a>] - Fix Travis Checks
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5737'>DRILL-5737</a>] - Hash Agg uses more than the allocated memory under certain low memory conditions
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5740'>DRILL-5740</a>] - hash agg fail to read spill file
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5743'>DRILL-5743</a>] - Using order by clause in a select * query on hbase table returns only the row_key and order by field(s)
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5744'>DRILL-5744</a>] - External sort fails with OOM error
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5745'>DRILL-5745</a>] - Invalid "location" information in Drill web server
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5746'>DRILL-5746</a>] - Pcap PR manually edited Protobuf files, values lost on next build
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5749'>DRILL-5749</a>] - Foreman and Netty threads occure deadlock
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5751'>DRILL-5751</a>] - Fix unit tests to use local file system even if it is not set by default
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5753'>DRILL-5753</a>] - Managed External Sort: One or more nodes ran out of memory while executing the query.
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5755'>DRILL-5755</a>] - TOP_N_SORT operator does not free memory while running
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5757'>DRILL-5757</a>] - CONVERT_TO_JSON function is failed while using non-existence field as a parameter.
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5761'>DRILL-5761</a>] - Disable Lilith ClassicMultiplexSocketAppender by default
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5763'>DRILL-5763</a>] - Fix NPE during MapRDBSubScan deserialization
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5765'>DRILL-5765</a>] - Json query profile is not shown on Web UI
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5766'>DRILL-5766</a>] - Stored XSS in APACHE DRILL
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5771'>DRILL-5771</a>] - Fix serDe errors for format plugins
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5775'>DRILL-5775</a>] - Select * query on a maprdb binary table fails
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5790'>DRILL-5790</a>] - PCAP format explicitly opens local file
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5792'>DRILL-5792</a>] - CONVERT_FROM_JSON on an empty file throws runtime exception
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5804'>DRILL-5804</a>] - External Sort times out, may be infinite loop
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5811'>DRILL-5811</a>] - Large number of "Failure finding Drillbit" messages when using MFS
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5812'>DRILL-5812</a>] - Restore the minor fragment Gantt chart Canvas
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5816'>DRILL-5816</a>] - Hash function produces skewed results on String values with same leading prefix
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5819'>DRILL-5819</a>] - Default value of security.admin.user_groups and security.admin.users is "true"
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5822'>DRILL-5822</a>] - The query with "SELECT *" with "ORDER BY" clause and `planner.slice_target`=1 doesn't preserve column order
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5824'>DRILL-5824</a>] - 1st phase Hash Aggregate allocates more memory than the limit
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5830'>DRILL-5830</a>] - Resolve regressions to MapR DB from DRILL-5546
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5838'>DRILL-5838</a>] - Fix MaprDB filter pushdown for the case of nested field (reg. of DRILL-4264)
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5839'>DRILL-5839</a>] - Handle Empty Batches in Merge Receiver
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5840'>DRILL-5840</a>] - A query that includes sort completes, and then loses Drill connection. Drill becomes unresponsive, and cannot restart because it cannot communicate with Zookeeper
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5845'>DRILL-5845</a>] - Columns returned by select with "ORDER BY" and "LIMIT" clauses are not in correct order.
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5849'>DRILL-5849</a>] - Add freemarker lib to dependencyManagement to ensure proper version is used when resolving dependency version conflicts
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5853'>DRILL-5853</a>] - Sort removal based on NULL direction
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5854'>DRILL-5854</a>] - IllegalStateException when empty batch is received.
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5857'>DRILL-5857</a>] - Fix NumberFormatException in Hive unit tests
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5859'>DRILL-5859</a>] - Time for query queuing timeout not display correctly in WebUI
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5863'>DRILL-5863</a>] - Sortable table incorrectly sorts minor fragments and time elements lexically instead of sorting by implicit value
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5864'>DRILL-5864</a>] - Selecting a non-existing field from a MapR-DB JSON table fails with NPE
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5872'>DRILL-5872</a>] - Deserialization of profile JSON fails due to totalCost being reported as "NaN"
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5874'>DRILL-5874</a>] - NPE in AnonWebUserConnection.cleanupSession()
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5878'>DRILL-5878</a>] - TableNotFound exception is being reported for a wrong storage plugin.
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5887'>DRILL-5887</a>] - Display process user/ groups in Drill UI
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5888'>DRILL-5888</a>] - jdbc-all-jar unit tests broken because of dependency on hadoop.security
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5890'>DRILL-5890</a>] - Tests Leak Many Open File Descriptors
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5895'>DRILL-5895</a>] - Fix unit tests for mongo storage plugin
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5896'>DRILL-5896</a>] - Handle vector creation in HbaseRecordReader to avoid NullableInt vectors later
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5898'>DRILL-5898</a>] - Query returns columns in the wrong order
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5906'>DRILL-5906</a>] - java.lang.NullPointerException while quering Hive ORC tables on MapR cluster.
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5923'>DRILL-5923</a>] - State of a successfully completed query shown as "COMPLETED"
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5941'>DRILL-5941</a>] - Skip header / footer logic works incorrectly for Hive tables when file has several input splits
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5972'>DRILL-5972</a>] - Slow performance for query on INFORMATION_SCHEMA.TABLE
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5979'>DRILL-5979</a>] - unnable to access temporary table from drill web interface
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5986'>DRILL-5986</a>] - Update jackson-databind version to 2.7.9.1
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5987'>DRILL-5987</a>] - Two versions of javassist on the classpath
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6006'>DRILL-6006</a>] - Label current is missing on Web UI since the same drillbits with different status are considered to be different
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6007'>DRILL-6007</a>] - web ui index page is refreshed at high pace
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6017'>DRILL-6017</a>] - Fix for SHUTDOWN button being visible for non Admin users
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6019'>DRILL-6019</a>] - Only admin should be able to access shutdown resources
-</li>
-<li>Order by doesn't sort columns when window function is involved in the query</li>
-<li>MIN MAX DIR tests fail and return incorrect results</li>
-<li>header.line.count does not work for Hive table with data file size > chunk size</li>
-<li>lang.NoClassDefFoundError exception not handled in org.apache.drill.exec.rpc.security.ClientAuthenticatorProvider</li>
-<li>Investigate jackson-databind vulnerabilities CVE-2017-15095 & CVE-2017-7525</li>
-<li>Null Pointer Exception with query using table function</li>
-<li>Fix XSS vulnerabilities in Drill</li>
-<li>Some queries in concurrent execution get stuck in STARTING phase</li>
-<li>hashagg.num_partitions cannot be set by "alter session" or "alter system"</li>
-<li>Query with 2-way JOIN fails with "Hash join does not support schema changes"</li>
-<li>RowKeyJoin is generated instead of IndexScan for queries with derived table/ table functions</li>
-<li>ConvertCountToDirectScan rule enhancements</li>
-<li>Drill wrong old date-time values displaying</li>
-<li>Warning message "Exception while trying to prune partition" when query on view</li>
-<li>Drill not able to read MapR-DB when FQDN is longer than 64 characters</li>
-<li>Pam authentication with Centrify doesn't work by JPam restriction</li>
-<li>Hash aggregate does not support schema changes</li>
-<li>Simple query with only one join condition failed with "Hash join does not support schema changes"</li>
-<li>UnsupportedOperationException: Unable to get holder type for minor type [LATE] and mode [OPTIONAL].</li>
-<li>Query on TPC-H SF100 dataset fails with "Hash join does not support schema changes" [MapR-DB JSON Tables]</li>
-</ul>
-
-<h2> Improvement
-</h2>
-<ul>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1691'>DRILL-1691</a>] - ConvertCountToDirectScan rule should be applicable for 2 or more COUNT aggregates
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4264'>DRILL-4264</a>] - Allow field names to include dots
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5089'>DRILL-5089</a>] - Skip initializing all enabled storage plugins for every query
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5106'>DRILL-5106</a>] - Refactor SkipRecordsInspector to exclude check for predefined file formats
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5259'>DRILL-5259</a>] - Allow listing a user-defined number of profiles
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5338'>DRILL-5338</a>] - Web UI: add better visual indication for PLANNING, QUEUED, EXECUTION in Query Profile that they are part of total time duration
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5704'>DRILL-5704</a>] - Improve error message on client side when queries fail with "Failed to create schema tree." when Impersonation is enabled and logins are anonymous
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5709'>DRILL-5709</a>] - Provide a value vector method to convert a vector to nullable
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5716'>DRILL-5716</a>] - Queue-based memory assignment for buffering operators
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5726'>DRILL-5726</a>] - Support Impersonation without authentication for REST API
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5795'>DRILL-5795</a>] - Filter pushdown for parquet handles multi rowgroup file
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5808'>DRILL-5808</a>] - Reduce memory allocator strictness for "managed" operators
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5815'>DRILL-5815</a>] - Provide option to set query memory as percent of total
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5832'>DRILL-5832</a>] - Migrate OperatorFixture to use SystemOptionManager rather than mock
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5834'>DRILL-5834</a>] - Add Networking Functions
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5842'>DRILL-5842</a>] - Refactor and simplify the fragment, operator contexts for testing
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5862'>DRILL-5862</a>] - Update project parent pom xml to the latest ASF version
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5893'>DRILL-5893</a>] - Maven forkCount property is too aggressive causing builds to fail on some machines.
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5899'>DRILL-5899</a>] - Simple pattern matchers can work with DrillBuf directly
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5909'>DRILL-5909</a>] - need new JMX metrics for (FAILED and CANCELED) queries
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5910'>DRILL-5910</a>] - Logging exception when custom AuthenticatorFactory not found
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5921'>DRILL-5921</a>] - Counters metrics should be listed in table
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5943'>DRILL-5943</a>] - Avoid the strong check introduced by DRILL-5582 for PLAIN mechanism
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5952'>DRILL-5952</a>] - Implement "CREATE TABLE IF NOT EXISTS"
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5960'>DRILL-5960</a>] - Add function STAsGeoJSON to extend GIS support
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5962'>DRILL-5962</a>] - Add function STAsJSON to extend GIS support
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5964'>DRILL-5964</a>] - Do not allow queries to access paths outside the current workspace root
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5980'>DRILL-5980</a>] - Make queryType param for REST API case insensitive
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5981'>DRILL-5981</a>] - Add Syntax Highlighting and Error Checking to Storage Plugin Config Page
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5989'>DRILL-5989</a>] - Run Smoke Tests On Travis
-</li>
-</ul>
-
-<h2> New Feature
-</h2>
-<ul>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4286'>DRILL-4286</a>] - Have an ability to put server in quiescent mode of operation
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4779'>DRILL-4779</a>] - Kafka storage plugin support
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5337'>DRILL-5337</a>] - OpenTSDB storage plugin
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5431'>DRILL-5431</a>] - Support SSL
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5682'>DRILL-5682</a>] - Apache Drill should support network encryption
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5723'>DRILL-5723</a>] - Support System/Session Internal Options And Additional Option System Fixes
-</li>
-</ul>
-
-<h2> Task
-</h2>
-<ul>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5601'>DRILL-5601</a>] - Rollup of External Sort memory management fixes
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5685'>DRILL-5685</a>] - Provide a way to set common environment variable between sqlline and Drillbit differently.
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5758'>DRILL-5758</a>] - Rollup of external sort fixes to issues found by QA
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5772'>DRILL-5772</a>] - Enable UTF-8 support in query string by default
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5781'>DRILL-5781</a>] - Fix unit test failures to use tests config even if default config is available
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5820'>DRILL-5820</a>] - Add support for libpam4j Pam Authenticator
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5911'>DRILL-5911</a>] - Upgrade esri-geometry-api version to 2.0.0 to avoid dependency on org.json library
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5917'>DRILL-5917</a>] - Ban org.json:json library in Drill
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6000'>DRILL-6000</a>] - Graceful Shutdown Unit Tests Should Not Be Run On Travis
-</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6011'>DRILL-6011</a>] - Update log info message in DrillClient
-</li>
-</ul>
-
\ No newline at end of file
[3/3] drill git commit: Doc and website updates for the 1.13 release
Posted by br...@apache.org.
Doc and website updates for the 1.13 release
Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/d8dd5d3d
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/d8dd5d3d
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/d8dd5d3d
Branch: refs/heads/gh-pages
Commit: d8dd5d3db4bd34afeeae9c2381aa518286354f5f
Parents: 35fa554
Author: Bridget Bevens <bb...@maprtech.com>
Authored: Sun Mar 18 13:02:35 2018 -0700
Committer: Bridget Bevens <bb...@maprtech.com>
Committed: Sun Mar 18 13:02:35 2018 -0700
----------------------------------------------------------------------
_data/version.json | 10 +-
_docs/031-drill-on-yarn.md | 4 +
.../010-drill-on-yarn-introduction.md | 47 ++
.../020-creating-a-basic-drill-cluster.md | 237 ++++++++++
.../030-launch-drill-under-yarn.md | 30 ++
.../040-configuration-reference.md | 84 ++++
.../050-drill-on-yarn-command-line-tool.md | 104 +++++
.../060-using-the-drill-on-yarn-web-ui.md | 83 ++++
.../070-multiple-drill-clusters.md | 70 +++
.../080-enabling-web-ui-security.md | 32 ++
.../090-appendix-a-release-note-issues.md | 173 ++++++++
.../091-appendix-b-drill-env.sh-settings.md | 18 +
.../092-appendix-c-troubleshooting.md | 96 +++++
...093-appendix-d-recreate-the-drill-archive.md | 31 ++
_docs/getting-started/010-drill-introduction.md | 23 +-
_docs/img/doy-envsh-mapping.PNG | Bin 0 -> 52771 bytes
.../047-installing-drill-on-the-cluster.md | 8 +-
...20-installing-drill-on-linux-and-mac-os-x.md | 8 +-
_docs/rn/001-1.13.0-rn.md | 294 +++++++++++++
_docs/rn/009-1.12.0-rn.md | 430 +++++++++++++++++++
_docs/tutorials/020-drill-in-10-minutes.md | 10 +-
.../050-analyzing-highly-dynamic-datasets.md | 4 +-
blog/_posts/2018-3-18-drill-1.13-released.md | 39 ++
23 files changed, 1814 insertions(+), 21 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/drill/blob/d8dd5d3d/_data/version.json
----------------------------------------------------------------------
diff --git a/_data/version.json b/_data/version.json
index 7342da9..afd7a0b 100644
--- a/_data/version.json
+++ b/_data/version.json
@@ -1,7 +1,7 @@
{
- "display_version": "1.12",
- "full_version": "1.12.0",
- "release_date": "December 15, 2017",
- "blog_post":"/blog/2017/12/15/drill-1.12-released",
- "release_notes": "https://drill.apache.org/docs/apache-drill-1-12-0-release-notes/"
+ "display_version": "1.13",
+ "full_version": "1.13.0",
+ "release_date": "March 18, 2018",
+ "blog_post":"/blog/2018/03/18/drill-1.13-released",
+ "release_notes": "https://drill.apache.org/docs/apache-drill-1-13-0-release-notes/"
}
http://git-wip-us.apache.org/repos/asf/drill/blob/d8dd5d3d/_docs/031-drill-on-yarn.md
----------------------------------------------------------------------
diff --git a/_docs/031-drill-on-yarn.md b/_docs/031-drill-on-yarn.md
new file mode 100644
index 0000000..f69713c
--- /dev/null
+++ b/_docs/031-drill-on-yarn.md
@@ -0,0 +1,4 @@
+---
+title: "Drill-on-YARN"
+nocontent: true
+---
http://git-wip-us.apache.org/repos/asf/drill/blob/d8dd5d3d/_docs/drill-on-yarn/010-drill-on-yarn-introduction.md
----------------------------------------------------------------------
diff --git a/_docs/drill-on-yarn/010-drill-on-yarn-introduction.md b/_docs/drill-on-yarn/010-drill-on-yarn-introduction.md
new file mode 100644
index 0000000..0803b21
--- /dev/null
+++ b/_docs/drill-on-yarn/010-drill-on-yarn-introduction.md
@@ -0,0 +1,47 @@
+---
+title: "Drill-on-YARN Introduction"
+date:
+parent: "Drill-on-YARN"
+---
+
+As of Drill 1.13, Drill integrates with YARN to manage resources. Drill becomes a long running application with YARN. When you launch Drill, YARN automatically deploys (“localizes”) the Drill software onto each node, avoiding the need to preinstall Drill on each node. Resource management is simplified because YARN is aware of the resources dedicated to Drill.
+
+Before you install and deploy Drill under YARN, you should be familiar with YARN concepts, such as the Resource Manager (RM), Node Manager (NM), and so on. You should also have a working Drill cluster that you want to launch under YARN. Drill configuration is best tested by launching Drill directly. You can launch Drill
+under YARN when the configuration becomes stable.
+
+##YARN Resource Settings
+
+Drill uses all available resources to run queries at optimal speed. When
+running Drill under YARN, you inform YARN of the resources that Drill will consume. Drill does not limit itself to the YARN settings; instead the YARN settings inform YARN of the resources that Drill will consume so that YARN
+does not over-allocate those same resources to other tasks.
+
+All YARN distributions provide settings for memory and CPU (called “vcores” by YARN). Some
+distributions also provide disk settings.
+
+For memory, you first configure Drill’s memory as described below, then you inform YARN of the Drill configuration.
+
+Drill will use all available disk I/Os. Drill will also use all available CPU. Consider enabling Linux
+cgroups to limit Drill's CPU usage to match the YARN vcores allocation.
+
+##Drill-on-YARN Components
+
+Drill-on-YARN uses the following components:
+
+- **Drill distribution archive:** The original .tar.gz file for your Drill distribution. DrillonYARN
+uploads this archive to your distributed file system (DFS). YARN downloads it (localized
+it) to each worker node.
+- **Drill site directory:** A directory that contains your Drill configuration and custom jar files.
+DrillonYARN copies this directory to each worker node.
+- **Configuration:** A configuration file which tells DrillonYARN
+how to manage your Drill cluster. This file is separate from your configuration files for Drill itself.
+- **DrillonYARN client:** A command line program to start, stop and monitor your YARN-managed Drill cluster.
+- **Drill Application Master (AM):** The software that works with YARN to request resources, launch Drillbits, and so on. The AM provides a web UI to manage your Drill cluster.
+- **Drillbit:** The Drill daemon software that YARN runs on each node.
+
+##Overview of Steps Required to Run Drill Under YARN
+To launch Drill under YARN, you will complete the following key steps. Each step is explained in detail in the following sections of the Drill-on-YARN documentation.
+
+- Create a Drill site directory with your site-specific files.
+- Configure Drill-on-YARN using the the drill-on-yarn.conf configuration file.
+- Use the Drill-on-YARN client tool to launch your Drill cluster.
+- Use the Drill-on-YARN client or web UI to monitor and shut down the Drill cluster.
http://git-wip-us.apache.org/repos/asf/drill/blob/d8dd5d3d/_docs/drill-on-yarn/020-creating-a-basic-drill-cluster.md
----------------------------------------------------------------------
diff --git a/_docs/drill-on-yarn/020-creating-a-basic-drill-cluster.md b/_docs/drill-on-yarn/020-creating-a-basic-drill-cluster.md
new file mode 100644
index 0000000..b721161
--- /dev/null
+++ b/_docs/drill-on-yarn/020-creating-a-basic-drill-cluster.md
@@ -0,0 +1,237 @@
+---
+title: "Creating a Basic Drill Cluster"
+date:
+parent: "Drill-on-YARN"
+---
+
+This topic walks you through the steps needed to create a basic Drill cluster.
+
+##The Client Machine
+
+YARN works by launching an application using a “client” application. For Drill, this is the
+Drill-on-YARN client. The client can run on any machine that has both the Drill and Hadoop
+software. Any host from which you currently launch YARN jobs can be the client. The client is not required to be part of the YARN cluster.
+
+When running Drill outside of YARN, you must install Drill on every node in the cluster. With YARN, you only need install Drill only on the client machine; Drill-on-YARN automatically deploys (“localizes”) Drill to the worker nodes.
+
+When running Drill without YARN, many users place their configuration files and custom code
+within the Drill distribution directory. When running under YARN, all your configuration and
+custom code resides in the site directory; do not change anything in the Drill install.
+(This allows Drill-on-YARN to upload your original Drill install archive without rebuilding it.)
+
+Complete the following steps on the client machine:
+
+###Create a Master Directory
+
+To localize Drill files, the client tool requires a copy of the original Drill distribution archive and the location of your site directory. Assume all these components reside in a single “master directory” described as $MASTER_DIR . On the client machine, create the master directory, as shown:
+
+ export MASTER_DIR=/path/to/master/dir
+ mkdir $MASTER_DIR
+ cd $MASTER_DIR
+
+To build the master directory, you will:
+- Download the Drill archive to $MASTER_DRILL.
+- Unpack the archive to create $DRILL_HOME.
+- Create the site directory with the required configuration files.
+
+The following steps provide the instructions for building the master directory:
+
+###Install Drill
+These instructions assume you are installing Drill as part of the Drill-on-YARN
+setup. You can use your existing Drill 1.8 or later install as long as it meets the required criteria.
+
+Follow the Drill [install directions]({{site.baseurl}}/docs/install-drill-introduction/) to install Drill on your client host. The install steps are different for YARN than for the Embedded or Cluster install.
+
+1. Select a Drill version. The name is used in multiple places below. For convenience, define an environment variable for the name:
+
+ export DRILL_NAME=apachedrillx.y.z
+Replace x.y.z with the selected version.
+2. Download the Drill version.
+
+ wget \ http://apache.mesi.com.ar/drill/drillx.y.z/$DRILL_NAME.tar.gz
+Or use
+
+ curl o $DRILL_NAME.tar.gz \http://apache.mesi.com.ar/drill/drillx.y.z/$DRILL_NAME.tar.gz
+Again, replace x.y.z with the selected version.
+
+3. Expand the Drill distribution into this folder to create the master directory
+
+ tar -xzf $DRILL_NAME.tar.gz
+4. For ease of following the remaining steps, call your expanded Drill folder $DRILL_HOME :
+
+ export DRILL_HOME=$MASTER_DIR/$DRILL_NAME
+
+Your master directory should now contain the original Drill archive along with an expanded copy
+of that archive.
+
+###Create the Site Directory
+
+The site directory contains your site-specific files for Drill. If you are converting an existing Drill install, see the “Site Directory” section.
+
+Create the site directory within your master directory:
+
+ export DRILL_SITE=$MASTER_DIR/site
+ mkdir $DRILL_SITE
+
+When you do a fresh install, Drill includes a conf directory under $DRILL_HOME. Use the files
+in that directory to create your site directory.
+
+ cp $DRILL_HOME/conf/drilloverrideexample.conf \
+ $DRILL_SITE/drilloverride.conf
+ cp $DRILL_HOME/conf/drill-on-yarnexample.conf \
+ $DRILL_SITE/drill-on-yarn.conf
+ cp $DRILL_HOME/conf/drillenv.sh $DRILL_SITE
+
+Edit the above configuration files as per the Drill install instructions, and the Drill-on-YARN
+instructions below. (Note that, under YARN, you set the Drill memory limits in
+drill-on-yarn.sh instead of drillenv.sh.)
+
+If you develop custom code (data sources or user-defined functions (UDFs)), place the Java JAR
+files in $DRILL_SITE/jars.
+
+Your master directory should now contain the Drill software and your site directory with default files. You will use the site directory each time you start Drill by using the --site
+(or --config) option. The following are examples, do not run these yet:
+
+ drillbit.sh --site $DRILL_SITE
+ drill-on-yarn.sh --site $DRILL_SITE
+
+Once you have created your site directory, upgrades are trivial. Simply delete the old Drill
+distribution and install the new one. Your files remain unchanged in the site directory.
+
+###Configure Drill-on-YARN using Existing Settings
+
+The next step is to configure Drill. If you have used Drill, start with Drill
+in distributed mode to learn which configuration options you need. YARN is an awkward
+environment in which to learn Drill configuration. These instructions assume that you have already worked out the required configuration on a separate Drill install. Let's call that location $PROD_DRILL_HOME.
+
+From $PROD_DRILL_HOME, copy the following to corresponding locations in $DRILL_SITE:
+
+ cp $PROD_DRILL_HOME/conf/drilloverride.conf $DRILL_SITE
+ cp $PROD_DRILL_HOME/conf/drillenv.sh $DRILL_SITE
+ cp $PROD_DRILL_HOME/jars/3rdparty/ yourJarName .jar $DRILL_SITE/jars
+
+###Create Your Cluster Configuration File
+The next step is to specify additional configuration which Drill-on-YARN requires to launch your Drill cluster.
+
+Start by editing $DRILL_SITE/drill-on-yarn.conf using your favorite editor. This file is
+in the same HOCON format used by drill-override.conf.
+
+Consult $DRILL_HOME/conf/drill-on-yarn-example.conf as an example. However,
+do not just copy the example file; instead, copy only the specific configuration settings that you need; the others will automatically take the Drill-defined
+default values.
+
+The following sections discuss each configuration option that you must set.
+
+###Drill Resource Configuration
+The two key Drill memory parameters are Java heap size and direct memory. In a non-YARN
+cluster, you set these in $DRILL_HOME/conf/drillenv.sh as follows (shown with the
+default values):
+
+ DRILL_MAX_DIRECT_MEMORY="8G"
+ DRILL_HEAP="4G"
+
+Drill-on-YARN uses a different mechanism to set these values. You set the values in
+drill-on-yarn.conf , then Drill-on-YARN copies the values into the environment variables
+when launching each Drillbit.
+
+ drillbit: {
+ heap: "4G"
+ max-direct-memory: "8G"
+ }
+
+To create the Drill-on-YARN setup, simply copy the values directly from your pre-YARN
+drillenv.sh file into the above configuration. (Drill-on-YARN copies the values back into
+the environment variables when launching Drill.)
+
+Next, determine the container size needed to run Drill under YARN. Typically this size
+is simply the sum of the heap and direct memory. However, if you are using custom libraries that perform their own memory allocation, or launch sub-processes, you must account for that
+memory usage as well. The YARN memory is expressed in MB. For example, for the default
+settings above, we need 12G of memory or 12288MB:
+
+ drillbit: {
+ memory-mb: 6144
+ }
+
+Finally, you must determine how much CPU to grant to Drill. Drill is a CPU intensive
+operation and greatly benefits from each additional core. However, you can limit Drill’s CPU usage under YARN by specifying the number of YARN virtual cores (vcores) to allocate to Drill:
+
+ drillbit: {
+ vcores: 4
+ }
+
+Note that in the above, each configuration setting was shown separately. In your actual file,
+however, they appear within a single group as follows:
+
+ drillbit: {
+ heap: "4G"
+ max-direct-memory: "8G"
+ memory-mb: 6144
+ vcores: 4
+ }
+
+
+###Drillbit Cluster Configuration
+Drill-on-YARN uses the concept of a “cluster group” of Drillbits to describe the set of Drillbits to launch. A group can be one of three kinds:
+
+
+
+- Basic: launches drillbits anywhere in the YARN cluster where a container is available.
+- Labeled: Uses YARN labels to identify the set of nodes that should run Drill.
+
+This section describes how to create a basic group suitable for testing. See later sections for the type.
+
+For a basic group, you need only specify the group type and the number of Drillbits to launch:
+
+ cluster: [
+ {
+ name: "mypool"
+ type: "basic"
+ count: 1
+ }
+ ]
+
+
+The above syntax says that pools is a list that contains a series of pool objects contained in
+braces. In this release, however, Drill supports just one pool.
+
+###ZooKeeper Configuration
+Drill uses ZooKeeper to coordinate between Drillbits. When run under YARN, the Drill
+Application Master uses ZooKeeper to monitor Drillbit health. Drill-on-YARN reads your
+$DRILL_SITE/drilloverride.conf file for ZooKeeper settings.
+
+
+###Configure the Drill Distribution Archive
+Next configure the name of the Drill distribution archive that you downloaded earlier.
+
+ drill-install: {
+ client-path: "archive-path"
+ }
+
+Where archive-path is the location of your archive. In our example, this is $MASTER_DIR/apache-drill.x.y.z.tar.gz. Use the full name of the master directory, not the environment variable. (Substitute your actual version number for x.y.z.)
+
+###Select the Distributed File System Location
+Drill copies your archive onto your distributed file system (such as HDFS) in a location you
+provide. Set the DFS options as follows:
+
+ dfs: {
+ connection: "hdfs://localhost/"
+ dir: "/user/drill"
+ }
+
+Drill can read the connection information from your Hadoop configuration files ($HADOOP_HOME/etc/hadoop/coresite.xml ). Or, you can specify a connection directly in the Drill cluster configuration file using the connection attribute. Then, choose a DFS file system location. Drill uses “ /user/drill ” by default.
+
+###Hadoop Location
+Apache Drill users must tell Drill-on-YARN the location of your Hadoop install. Set the
+HADOOP_HOME environment variable in $DRILL_SITE/drillenv.sh to point to your Hadoop installation:
+
+ export HADOOP_HOME= /path/to/hadoop-home
+
+This assumes that Hadoop configuration is in the default location:
+
+ $HADOOP_HOME/etc/hadoop
+
+If your configuration is elsewhere, set HADOOP_CONF_DIR instead:
+
+ export HADOOP_CONF_DIR= /path/to/hadoop-config
+
+
http://git-wip-us.apache.org/repos/asf/drill/blob/d8dd5d3d/_docs/drill-on-yarn/030-launch-drill-under-yarn.md
----------------------------------------------------------------------
diff --git a/_docs/drill-on-yarn/030-launch-drill-under-yarn.md b/_docs/drill-on-yarn/030-launch-drill-under-yarn.md
new file mode 100644
index 0000000..6b95de2
--- /dev/null
+++ b/_docs/drill-on-yarn/030-launch-drill-under-yarn.md
@@ -0,0 +1,30 @@
+---
+title: "Launch Drill Under YARN"
+date:
+parent: "Drill-on-YARN"
+---
+
+Use the client tool to launch your new Drill cluster, as shown:
+
+ $DRILL_HOME/bin/drill-on-yarn.sh site
+ $DRILL_SITE start
+
+
+A number of lines describing the start-up process appear. The tool automatically archives
+and uploads your site directory, which YARN copies (along with Drill) onto each node. If all goes well, the tool prints a URL for the Drill Application Master process that you can use to monitor the cluster. Your Drillbits should now be up and running. (If not, see the Troubleshooting section.)
+
+Check the status of your Drill cluster, as shown:
+
+ $DRILL_HOME/bin/drill-on-yarn.sh site
+ $DRILL_SITE status
+
+Stop your cluster, as shown:
+
+ $DRILL_HOME/bin/drill-on-yarn.sh site
+ $DRILL_SITE stop
+
+
+Note, to avoid typing the site argument each time, you can set an environment variable, as shown:
+
+ export DRILL_CONF_DIR=$DRILL_SITE
+ $DRILL_HOME/bin/drill-on-yarn.sh start
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/drill/blob/d8dd5d3d/_docs/drill-on-yarn/040-configuration-reference.md
----------------------------------------------------------------------
diff --git a/_docs/drill-on-yarn/040-configuration-reference.md b/_docs/drill-on-yarn/040-configuration-reference.md
new file mode 100644
index 0000000..5b0a2dc
--- /dev/null
+++ b/_docs/drill-on-yarn/040-configuration-reference.md
@@ -0,0 +1,84 @@
+---
+title: "Configuration Reference"
+date:
+parent: "Drill-on-YARN"
+---
+
+The Creating a Basic Drill Cluster topic presented the minimum configuration needed to launch Drill under YARN. Additional configuration options are available for specialized cases. Refer to the drill-on-yarn-example.conf for information about the other options.
+
+##Application Name
+The application name appears when starting or stopping the Drill cluster and in the
+Drill-on-YARN web UI. Choose a name helpful to you:
+
+ app-name: "My Drill Cluster"
+
+##Drill Distribution Archive
+The Drill distribution archive is assumed to expand to create a folder that has the same name as the archive itself (minus the .tar.gz suffix). That is, the archive apache-drill-x.y.z.tar.gz is assumed to expand to a directory named apache-drill-x.y.z. Apache Drill archives follow this pattern. In specialized cases, you may have to create your own archive. If you do, it is most convenient if you follow the same pattern. However, if cannot follow the pattern, you can configure Drill-on-YARN to follow a custom pattern using the drill-install.dir-name option:
+
+ drill-install:{
+ clientpath: "/path/to/ your-custom-archive.tar.gz"
+ dirname: "your-drill-directory"
+ }
+
+Where:
+
+`/path/to/ your-custom-archive.tar.gz` is the location of your archive. `your-drill-directory`
+is the name of your Drill directory within the archive.
+
+##Customize Web UI Port
+If you run multiple Drill clusters per YARN cluster, then YARN may choose to place two Drill AM
+processes on the same node. To avoid port conflicts, change the HTTP port for one or both of
+the Drill clusters:
+
+ drill.yarn:{
+ http: {
+ port: 12345
+ }
+ }
+
+##Customize Application Master Settings
+The following settings apply to the Application Master. All are prefixed with `drill.yarn.am.`
+
+| Name | Description | Default |
+|-----------------|--------------------------------------------------------------------|---------|
+| memorymb | Memory, in MB, to allocate to the AM. | 512 |
+| vcores | Number of CPUS to allocate to the AM. | 1 |
+| heap | Java heap for the AM. | 450M |
+| node-label-expr | YARN node label expression to use to select nodes to run the AM. | None |
+
+
+##Drillbit Customization
+
+The following Drill-on-YARN configuration options control the Drillbit processes. All properties
+start with drill.yarn.drillbit.
+
+| Name | Description | Default |
+|-------------------|---------------------------------------------------------------------------------------------------|----------|
+| memory-mb | Memory, in MB, to allocate to the Drillbit. | 13000 |
+| vcores | Number of CPUS to allocatet to the AM. | 4 |
+| disks | Number of disk equivalents consumed by Drill (on versions of YARN that support disk resources.) | 1 |
+| heap | Java heap memory. | 4G |
+| max-direct-memory | Direct (offheap) memory for the Drillbit. | 8G |
+| log-gc | Enables Java garbage collector logging. | FALSE |
+| class-path | Additional classpath entries. | blank |
+
+Note that the Drillbit node expression is set in the labeled pool below.
+
+##Cluster Groups
+YARN was originally designed for MapReduce jobs that can run on any node, and that often can be combined onto a single node. Compared to the traditional MapReduce jobs, Drill has additional constraints:
+
+- Only one Drillbit (per Drill cluster) can run per host (to avoid port conflict.)
+- Drillbits work best when launched on the same host as the data that the Drillbit is to scan.
+
+
+###Basic Cluster
+A basic cluster launches n Drillbits on distinct nodes anywhere in your YARN cluster. The basic cluster is great for testing and other informal tasks: just configure the desired vcores and memory, along with a number of nodes, then launch Drill. YARN will locate a set of suitable hosts anywhere on the YARN cluster.
+
+###Labeled Hosts
+Drill-on-YARN can handle node placement directly without the use of labeled queues. You use the “labeled” pool type. Then, set the drillbit-label-expr property to a YARN label expression that matches the nodes on which Drill should run. You will most often care only about Drillbit placement. Finally, indicate the number of Drillbits to run on the selected nodes.
+
+###Named Hosts
+You can configure Drill-on-YARN to run on a specific set of hosts. However, you must keep the list synchronized with your YARN cluster. If you list a host that is not available to YARN, then Drill cannot start a Drillbit on that host. Also, if the host does
+not have sufficient resources available, the Drillbit will not run.
+
+
http://git-wip-us.apache.org/repos/asf/drill/blob/d8dd5d3d/_docs/drill-on-yarn/050-drill-on-yarn-command-line-tool.md
----------------------------------------------------------------------
diff --git a/_docs/drill-on-yarn/050-drill-on-yarn-command-line-tool.md b/_docs/drill-on-yarn/050-drill-on-yarn-command-line-tool.md
new file mode 100644
index 0000000..d3044c1
--- /dev/null
+++ b/_docs/drill-on-yarn/050-drill-on-yarn-command-line-tool.md
@@ -0,0 +1,104 @@
+---
+title: "Drill-on-YARN Command-Line Tool"
+date:
+parent: "Drill-on-YARN"
+---
+
+Drill provides the drill-on-yarn command-line tool to start, stop, resize and check the status of your Drill cluster. The tool is located in:
+
+ $DRILL_HOME/bin/drill-on-yarn.sh site $DRILL_SITE command
+
+Where command is one of those described below.
+
+##Start the Drill Cluster
+
+Start your drill cluster with the start command:
+
+ $DRILL_HOME/bin/drill-on-yarn.sh start
+
+The command shows the startup status followed by a summary of the application:
+
+ Launching DrillonYARN...
+ Application ID: application_1462842354064_0001
+ Application State: ACCEPTED
+ Starting......
+ Application State: RUNNING
+ Tracking URL:
+ http://10.250.50.31:8088/proxy/application_1462842354064_0001/
+ Application Master URL: http://10.250.50.31:8048/
+
+
+The first line confirms which cluster is starting by displaying the cluster name from your
+configuration file. The next line shows YARN’s application ID and tracks the job status from
+Accepted to Running. Once the job starts, you’ll see YARN’s job tracking URL along with
+Drill-on-YARN’s web UI url. Use this URL to visit the web UI described below. Once the application starts, Drill-on-YARN writes an “appid” file into your master directory:
+
+ ls $MASTER_DIR
+ …
+ drillbits1.appid
+
+The file name is the same as your Drill cluster ID. The file contains the id if the Drill-on-YARN application for use by the other commands described below. You can run only one Drill AM at a time. If you attempt to start as second one from the same client machine on which you started the first, the client command will complain that the appid file already exists. If you attempt to start the cluster from a different node, then the second AM will detect the conflict and will shut down again.
+
+##Drill Cluster Status
+
+You can retrieve basic information about the Drill cluster as follows:
+
+ $DRILL_HOME/bin/drill-on-yarn.sh status
+
+You will see output something like the following:
+
+ Application ID: application_1462842354064_0001
+ Application State: RUNNING
+ Host: yosemite/10.250.50.31
+ Tracking URL:
+ http://10.250.50.31:8088/proxy/application_1462842354064_0001/
+ Queue: default
+ User: drilluser
+ Start Time: 20160509
+ 16:56:40
+ Application Name: DrillonYARN
+ AM State: LIVE
+ Target Drillbit Count: 1
+ Live Drillbit Count: 1
+ For more information, visit: http://10.250.50.31:8048/
+
+
+The first two several lines give you information about YARN’s state: the application ID, the
+application state and YARN’s tracking URL for the application. Next is the host on which the Drill AM is running, the queue on which the application was placed and the user who submitted the application. The start time tells you when YARN started the application.
+The next few lines are specific to Drill: the name of the application (which you configured in the Drill-on-YARN configuration file), the Drill application master URL, the number of Drillbits you requested to run and the number actually running. Finally, the last line gives you the URL to use to access the Drill-on-YARN web UI described below.
+
+##Stop the Drill Cluster
+You can stop the Drill cluster from the command line:
+
+ $DRILL_HOME/bin/drill-on-yarn.sh stop
+
+Note that this command is “forceful”, it kills any in-flight queries. The output tracks the shutdown and displays the final YARN application status:
+
+ Stopping Application ID: application_1462842354064_0001
+ Stopping...
+ Stopped.
+ Final status: SUCCEEDED
+
+##Resize the Drill Cluster
+You can add or remove nodes to your Drill cluster while the cluster runs using the re-size
+command. You can specify the change either by giving the number of nodes you want to run:
+
+ $DRILL_HOME/bin/drill-on-yarn.sh resize 10
+
+Or by specifying the change in node count: + for increase, for decrease. To add two nodes:
+
+ $DRILL_HOME/bin/drill-on-yarn.sh resize +2
+
+To remove three nodes:
+
+ $DRILL_HOME/bin/drill-on-yarn.sh resize 3
+
+Drill will add nodes only if additional nodes are available from YARN. If you request to stop more nodes than are running, Drill stops all the running nodes. Note that in the current version of Drill, stopping nodes is a forceful operation: any in-flight queries will fail.
+
+##Clean the DFS Files
+
+If you run Drill-on-YARN for a temporary cluster, Drill will leave the Drill software archive in your designated DFS directory. You can remove those files with the following:
+
+ $DRILL_HOME/bin/drill-on-yarn.sh clean
+
+Specifically, the first start uploads your Drill archive to DFS. Stop leaves the archive in DFS. Subsequent start commands reuse the cached archive if it is the same size as the version on the local disk. Clean removes the cached file, forcing Drill to upload a fresh copy if you again restart the Drill cluster.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/drill/blob/d8dd5d3d/_docs/drill-on-yarn/060-using-the-drill-on-yarn-web-ui.md
----------------------------------------------------------------------
diff --git a/_docs/drill-on-yarn/060-using-the-drill-on-yarn-web-ui.md b/_docs/drill-on-yarn/060-using-the-drill-on-yarn-web-ui.md
new file mode 100644
index 0000000..89d4dc3
--- /dev/null
+++ b/_docs/drill-on-yarn/060-using-the-drill-on-yarn-web-ui.md
@@ -0,0 +1,83 @@
+---
+title: "Using the Drill-on-YARN Web UI"
+date:
+parent: "Drill-on-YARN"
+---
+
+Applications that run under YARN provide an Application Master (AM) process to manage the
+application’s task. Drill provides the Drill-on-YARN Application Master for this purpose. When
+you launch Drill using the command line tool, the tool asks YARN to launch Drill’s AM, which in
+turn launches your Drillbits.
+
+The Drill application master provides a web UI to monitor cluster status and to perform simple
+operations such as increasing or decreasing cluster size, or stop the cluster. You can reach the UI using the URL provided when the application starts. You can also follow the link from the YARN Resource Manager UI. Find the page for the Drill application. Click on
+the Tracking URL link.
+
+The UI provides five pages:
+1. A main page that provides overall cluster status.
+2. A configuration page where you view the complete set of configuration variables which
+the Drill AM uses.
+3. Detailed list of the running Drillbits.
+4. Management page with a number of simple operations to resize or stop the cluster.
+5. A history of stopped, killed or failed Drillbits. Use this to diagnose problems.
+
+##Main Page
+The main page shows the state of the Drill cluster.
+
+Drill Cluster Status : the state of the Drill cluster, one of the following:
+
+- LIVE: Normal state: shows that your Drill cluster is running.
+- ENDING: The cluster is in the process of shutting down.
+
+
+There is no “ENDED” state: once the cluster is shut down, the AM itself exists and the web UI is no longer available.
+
+
+
+- **Target Drillbit Count:** The number of Drillbits to run in the cluster. The actual number may be less if Drillbits have not yet started, or if YARN cannot allocate enough containers.
+- **Live Drillbit Count:** Number of Drillbits that are ready for use. These have successfully started, have registered with ZooKeeper, and are ready for use. You can see the detail of all Drillbits (including those in the process of starting or stopping) using the Drillbits page. Each Drillbit must run on a separate node, so this is also the number of nodes in the cluster running Drill.
+- **Total Drillbit Memory and Virtual Cores:** The total number of YARN resources currently
+allocated to running Drillbits.
+- **YARN Node Count, Memory and Virtual Cores :** Reports general information about YARN
+itself including the number of nodes, the total cluster memory and total number of virtual cores.
+- **Groups:** Lists the cluster groups defined in the configuration file (of which only one is currently supported), along with the target and actual number of Drillbits in that group.
+
+##Configuration Page
+The configuration page shows the complete set of configuration values used for the current run.
+The values come from your own configuration along with Drill-provided defaults. Use this page
+to diagnose configuration-related issues. Names are shown in fully-expanded form. That is the
+name “drill.yarn.http.port” refers to the parameter defined as follows in your configuration file:
+
+ drill.yarn:{
+ http: {
+ port: 8048
+ }
+ }
+
+##Drillbits Page
+The Drillbits page lists all drillbits in all states.
+
+
+
+- **ID:** A sequential number assigned to each new Drillbit. Numbers may not start with 1 if you have previously shut down some Drillbits.
+- **Group:** The cluster group that started the Drillbit. (Cluster groups are from the configuration file.)
+- **Host:** The host name or IP address on which the Drillbit runs. If the Drillbit is in normal operating state, this field is also a hyperlink to the Web UI for the Drillbit.
+- **State:** The operating state of the Drillbit. The normal state is “Running.” The drillbit passes through a number of states as YARN allocates a container and launches a process, as the AM waits for the Drillbit to become registered in ZooKeeper, and so on. Similarly, the Drillbit passes through a different set of states during shutdown. Use this value to diagnose problems.
+If the Drillbit is in a live state, then this field shows an “[X]” link that you can use to kill this particular Drillbit. Use this if the Drillbit has startup problems or seems unresponsive. During the shutdown process, the kill link disappears and is replaced with a “(Cancelled)” note.
+- **ZK State:** The ZooKeeper handshake state. Normal state is “START_ACK”, meaning that the
+Drillbit has registered with ZooKeeper. This state is useful when diagnosing problems.
+- **Container ID:** The YARNassigned container ID for the Drillbit task. The ID is a link, it takes you to the YARN Node Manager UI for the Drillbit task.
+- **Memory and Virtual Cores:** The amount of resources actually allocated to the Drillbit by YARN.
+- **Start Time:** The date and time (in your local timezone, displayed in ISO format) when the
+Drillbit launch started. This page will also display un-managed Drillbits, if present. An un-manage Drillbit is one that is running, has registered with ZooKeeper, but was not started by the Drill Application Master. Likely the Drillbit was launched using the drillbit.sh script directly. Use the host name to locate the machine running the Drillbit if you want to convert that Drillbit to run under YARN.
+
+##Manage Page
+The Manage page allows you to re-size or stop the cluster. You can re-size the cluster by adding Drillbits, removing Drillbits or setting the cluster to a desired size.
+
+Drill is a long-running application. In normal practice, you leave Drill running indefinitely. You would shut down your Drill cluster only to, say, perform an upgrade of the Drill software or to change configuration options. When you terminate your Drill cluster, any in-progress queries will fail. Therefore, a good practice is to perform the shut down with users so that Drill is not processing any queries at the time of the shutdown.
+
+When removing or shutting-down the cluster, you will receive a confirmation page asking if you
+really do want to stop Drillbit processes. Click Confirm to continue.
+
+##History Page
+The History page lists all Drillbits which have failed, been killed, or been restarted. The History page allows you to detect failures and diagnose problems. Use the YARN container ID listed on this page to locate the log files for the Drillbit.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/drill/blob/d8dd5d3d/_docs/drill-on-yarn/070-multiple-drill-clusters.md
----------------------------------------------------------------------
diff --git a/_docs/drill-on-yarn/070-multiple-drill-clusters.md b/_docs/drill-on-yarn/070-multiple-drill-clusters.md
new file mode 100644
index 0000000..07294af
--- /dev/null
+++ b/_docs/drill-on-yarn/070-multiple-drill-clusters.md
@@ -0,0 +1,70 @@
+---
+title: "Multiple Drill Clusters"
+date:
+parent: "Drill-on-YARN"
+---
+
+Drill-on-YARN allows you to easily define multiple Drill clusters on a single YARN cluster. Each Drill cluster is a collection of Drillbits that work as an independent unit. For example, you might define one test cluster of a few machines on the same physical cluster that runs larger clusters for, say, Development and Marketing.
+
+Drill clusters coordinate using ZooKeeper, so you must assign each cluster a distinct ZooKeeper
+entry. YARN may launch Drillbits from different clusters on the same physical node, so each
+Drill cluster must use a distinct set of ports. Since each cluster requires its own setup, you must create a separate site directory for each. The instructions below explain the required setup.
+
+##Create a New Site Directory
+Create a new site directory for your new cluster. Let’s say that your new cluster has the name
+“second”. Using the same structure as before, create a new site directory under your master
+directory:
+
+ export SECOND_SITE=$MASTER_DIR/second
+ mkdir $SECOND_SITE
+
+Copy files into this new site directory as you did to create the first one. You can copy and modify an existing set of files, or create the site from scratch.
+
+At a minimum, you must set the following configuration options in drill-override.conf:
+
+ drill.exec: {
+ cluster-id: "drillbits",
+ zk: {
+ root: "second"
+ connect: "zkhost: 2181"
+ }
+ rpc {
+ user.server.port: 41010
+ bit.server.port: 41011
+ }
+ http.port: 9047
+ }
+
+You have two options for how your new cluster interacts with the existing cluster. The normal
+case is a shared-nothing scenario in which the two clusters are entirely independent at the
+configuration level. For this case, ensure that the zk.root name is distinct from any existing
+cluster.
+
+In the more advanced case, if both clusters share the same zk.root value, then they will
+share settings such as storage plugins. If the clusters share the same root, then they must have distinct cluster-id values.
+
+In addition, the three ports must have values distinct from all other clusters. In the example
+above, we’ve added a 1 to the first digit of the default port numbers; you can choose any available ports.
+
+##Drill-on-YARN Configuration
+Create the drill-on-yarn.conf file as described before. The following must be distinct
+for your cluster:
+
+ drill.yarn: {
+ app-name: "Second Cluster"
+ dfs: {
+ app-dir: "/user/drill2"
+ }
+ http : {
+ port: 9048
+ }
+ }
+
+That is, give your cluster a distinct name, a distinct upload directory in DFS, and a distinct port number.
+
+##Start the Cluster
+Use the site directory for the second cluster to start the cluster:
+
+ $DRILL_HOME/bin/drill-on-yarn.sh site
+ $SECOND_SITE start
+
http://git-wip-us.apache.org/repos/asf/drill/blob/d8dd5d3d/_docs/drill-on-yarn/080-enabling-web-ui-security.md
----------------------------------------------------------------------
diff --git a/_docs/drill-on-yarn/080-enabling-web-ui-security.md b/_docs/drill-on-yarn/080-enabling-web-ui-security.md
new file mode 100644
index 0000000..e8f772d
--- /dev/null
+++ b/_docs/drill-on-yarn/080-enabling-web-ui-security.md
@@ -0,0 +1,32 @@
+---
+title: "Enabling Web UI Security"
+date:
+parent: "Drill-on-YARN"
+---
+
+Drill-on-YARN provides a web UI as described earlier. By default, the UI is open to everyone.
+You can secure the UI using either a simple predefined user name and password, or using
+Drill’s user authentication.
+
+##Simple Security
+Simple security is enabled using three configuration settings:
+
+ drill.yarn.http: {
+ auth-type: "simple"
+ user-name: "bob"
+ password: "secret"
+ }
+
+Restart the Drill-on-YARN Application Master. When you visit the web UI, a login page should
+appear, prompting you to log in. Only the above user and password are valid. Simple security is not highly secure; but it is useful for testing, prototypes and the like.
+
+##Using Drill’s User Authentication
+Drill-on-YARN can use Drill’s authentication system. In this mode, the user name and password
+must match that of the user that started the Drill-on-YARN application. To enable Drill security:
+
+ drill.yarn.http: {
+ auth-type: "drill"
+ }
+
+You must have previously enabled Drill user authentication, as described in the [Drill
+Documentation]({{site.baseurl}}/docs/configuring-user-authentication/) .
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/drill/blob/d8dd5d3d/_docs/drill-on-yarn/090-appendix-a-release-note-issues.md
----------------------------------------------------------------------
diff --git a/_docs/drill-on-yarn/090-appendix-a-release-note-issues.md b/_docs/drill-on-yarn/090-appendix-a-release-note-issues.md
new file mode 100644
index 0000000..d93f185
--- /dev/null
+++ b/_docs/drill-on-yarn/090-appendix-a-release-note-issues.md
@@ -0,0 +1,173 @@
+---
+title: "Appendix A: Release Note Issues"
+date:
+parent: "Drill-on-YARN"
+---
+
+Drill-on-YARN creates a tighter coupling between Drill and Hadoop than did previous Drill
+versions. You should be aware of the following compatibility issues:
+
+##Migrating the $DRILL_HOME/conf/drill-env.sh Script
+Prior to Drill 1.8, the drill-env.sh script contained Drill defaults, distribution-specific
+settings, and configuration specific to your application (“site”.) In Drill 1.8, the Drill and distribution settings are moved to other locations. The site-specific settings change in format to allow YARN to override them. The following section details the changes you must make if you reuse a drill-env.sh file from a prior release. (If you create a new file, you can skip this section.)
+
+At the end of this process, your file should contain just two lines for memory settings, plus any additional custom settings you may have added.
+
+##Memory Settings
+Most Drill configuration is done via the Drill configuration file and the configuration registry. However, certain options must be set at the time that Drill starts; such options are configured in the $DRILL_HOME/conf/drill-env.sh file. Under YARN, these settings are set in the
+YARN configuration. To ensure that the YARN configuration options are used, you must modify
+your existing drill-env.sh file as follows. (If you are installing Drill fresh, and don’t have an existing file, you can skip these steps. The Drill 1.8 and later files already have the correct format.)
+
+Find the two lines that look like this:
+
+ DRILL_MAX_DIRECT_MEMORY="8G"
+ DRILL_HEAP="4G"
+
+Replace them with the following two lines:
+
+ export DRILL_MAX_DIRECT_MEMORY=${DRILL_MAX_DIRECT_MEMORY:" 8G"}
+ export DRILL_HEAP=${DRILL_HEAP:" 4G"}
+
+
+Copy the actual values from the old lines to the new ones (e.g. the “8G” and “4G” values.)
+Those are the values that Drill when use if you launch it outside of YARN. The new lines ensure
+that these values are replaced by those set by Drill-on-YARN when running under YARN.
+
+If you omit this change, then Drill will ignore your memory settings in Drill-on-YARN, resulting in a potential mismatch between the Drill memory settings and the amount of memory requested from YARN.
+
+##Remove General Drill Settings
+
+If you are reusing the drill-env.sh from a prior release, find lines similar to the following:
+
+ export DRILL_JAVA_OPTS="-Xms$DRILL_HEAP -Xmx$DRILL_HEAP
+ -XX:MaxDirectMemorySize=$DRILL_MAX_DIRECT_MEMORY \
+ -XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=1G \
+ -Ddrill.exec.enable-epoll=true"
+
+Compare these lines to the original drill-env.sh to determine if you modified the lines.
+These general Drill defaults now reside in other Drill scripts and can be remove from your
+site-specific version of drill-env.sh.
+
+##Remove Distribution-Specific Settings
+Some Drill distributions added distribution-specific settings to the drill-env.sh script. Drill 1.8 moves these settings to a new $DRILL_HOME/conf/distrib-env.sh file. Compare drill-env.sh and distrib-env.sh . Lines that occur in both files should be removed from
+drill-env.sh.
+
+If you later find you need to change the settings in distrib-env.sh , copy the line to drill-env.sh and modify the line. Drill reads drill-env.sh after distrib-env.sh, so your site-specific settings will replace the default distribution settings.
+
+##Hadoop Jar Files
+Drill depends on certain Hadoop Java JAR files which the Drill distribution includes in the
+$DRILL_HOME/jars/3rdparty directory. Although YARN offers Drill a Java class-path with
+the Hadoop jars, Drill uses its own copies instead to ensure Drill runs under the same
+configuration with which it was tested. Drill distributions that are part of a complete Hadoop distribution (such as the MapR distribution) have already verified version compatibility for you. If you are assembling your own Hadoop and Drill combination, you should verify that the Hadoop version packaged with Drill is compatible with the version running our YARN cluster.
+
+##$DRILL_HOME/conf/core-site.xml Issue
+Prior versions of Drill included a file in the $DRILL_HOME/conf directory called
+core-site.xml. YARN relies on a file with the same name in the Hadoop configuration directory. The Drill copy hides the YARN copy, preventing YARN from operating correctly. For this reason, version 1.8 of Drill renames the example file to core-site-example.xml. When upgrading an existing Drill installation, do not copy the file from your current version of Drill to the new version. If you modified core-site.xml, you should merge your changes with Hadoop’s core-site.xml file.
+
+##Mac OS setsid Issue
+YARN has a bug which prevents Drillbits from properly shutting down when run under YARN on
+Mac OS.
+
+
+
+- [YARN-3066](https://issues.apache.org/jira/browse/YARN-3066): Hadoop leaves orphaned tasks running after job is killed.
+
+
+You may encounter this problem if you use a Mac to try out the YARN integration for Drill. The
+symptom is that you:
+● Start Drill as described below
+● Attempt to stop the Drill cluster as described below
+● Afterwards use jps to list Java processes and find that Drillbit is still running.
+
+The problem is that the setsid command is not available under MacOS. The workaround is to
+use the open source equivalent:
+
+● Install the [XCode command line tools](https://developer.apple.com/library/content/technotes/tn2339/_index.html).
+● Using git, clone ersatz-ssid from https://github.com/jerrykuch/ersatz-setsid
+● Cd into the ersatz-ssid directory and type: `make`
+● Copy the resulting executable into `/usr/bin : sudo cp setsid /usr/bin`
+
+##Apache YARN Node Labels and Labeled Queues
+The Drill-on-YARN feature should work with Apache YARN node labels, but such support is
+currently not tested. Early indications are that the Apache YARN label documentation does not
+quite match the implementation, and that labels are very tricky. The Drill team is looking forward to community assistance to better support Apache YARN node labels.
+
+##Apache YARN RM Failure and Recovery
+Drill-on-YARN currently does not gracefully handle Apache YARN Resource Manager failure
+and recovery. According to the Apache YARN documentation, a failed RM may restart any
+in-flight Application Masters, then alert the AM of any in-flight tasks. Drill-on-YARN is not
+currently aware of this restart capability. Existing Drillbits will continue to run, at least for a time.They may be reported in the Drill-on-YARN web UI as unmanaged. Presumably, eventually YARN will kill the old Drillbits at which time Drill-on-YARN should start replacements. This is an area for future improvement based on community experience.
+
+##Configuring User Authentication
+The Drill Documentation describes how to configure user authentication using PAM. Two
+revisions are needed for Drill-on-YARN:
+● Configure user authentication for Drill using a site directory
+● Configure Drill-on-YARN authentication
+
+The existing instructions explain how to configure PAM authentication by changing Drill config
+files and adding libraries to the Drill distribution directory. If you use that approach, you must rebuild the Drill software archive as described elsewhere in this document. However, you can simply configure security using the site directory as explained below.
+
+###Configuring User Authentication for the Drillbit
+Existing instructions:
+
+Untar the file, and copy the libjpam.so file into a directory that does not contain other
+Hadoop components.
+
+Example: /opt/pam/
+
+Revised instructions: You have the option of deploying the library to each node, or allowing YARN to distribute the library. To have YARN do the distribution:
+
+Create the following directory:
+
+ $DRILL_SITE/lib
+
+Untar the file and copy libjpam.so into $DRILL_SITE/lib.
+
+Existing instructions: Add the following line to <DRILL_HOME>/conf/drill-env.sh, including the
+directory where the libjpam.so file is located:
+
+ export DRILLBIT_JAVA_OPTS="-Djava.library.path=<directory>"
+ Example: export DRILLBIT_JAVA_OPTS="-Djava.library.path=/opt/pam/"
+
+Revised instructions: If you are not using Drill-on-YARN, set a new environment variable in drill-env.sh:
+
+ export DRILL_JAVA_LIB_PATH=”<directory>”
+
+If you install the library yourself, either set DRILL_JAVA_LIB_PATH as above, or set the
+following in drill-on-yarn.conf:
+
+ drill.yarn.files: {
+ librarypath: "<directory>"
+ }
+
+**Note:** Do not explicitly set DRILLBIT_JAVA_OPTS as you may have done in previous releases; Drill will not know how to add your $DRILL_SITE/lib directory or how to interpret the librarypath item above.
+
+If you put the library in the $DRILL_SITE/lib directory, Drill-on-YARN automatically
+does the necessary configuration; there is nothing more for you to do.
+
+###Implementing and Configuring a Custom Authenticator
+Most of the existing steps are fine, except for step 3.
+
+Current text: Add the JAR file that you built to the following directory on each Drill node:
+
+ <DRILLINSTALL_HOME>/jars
+
+Revised text: Add the JAR file that you built to the following directory on each Drill node:
+
+ $DRILL_SITE/jars
+
+If running under YARN, you only need to add the jar to the site directory on the node from which you start Drill-on-YARN (which we’ve referred to as $MASTER_DIR.)
+
+Also, step 5: Restart the Drillbit process on each Drill node.
+
+ <DRILLINSTALL_HOME>/bin/drillbit.sh restart
+
+Under YARN, restart the YARN cluster:
+
+ $DRILL_HOME/bin/drill-on-yarn.sh --site $DRILL_SITE restart
+
+###Configuring User Authentication for the Application Master
+If you configure user authentication for Drill, then user authentication is automatically configured in the Application Master also. Only users with admin privileges can use the AM web UI.
+
+###Testing User Authentication on the Mac
+The [Drill Documentation]({{site.baseurl}}/docs/configuring-user-authentication/) describes how to configure user authentication using PAM, including instructions for downloading a required native library. However, if you are testing security on the Mac, the referenced library does not work on modern Macs. Instead, see the workaround in [DRILL-4756](https://issues.apache.org/jira/browse/DRILL-4756).
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/drill/blob/d8dd5d3d/_docs/drill-on-yarn/091-appendix-b-drill-env.sh-settings.md
----------------------------------------------------------------------
diff --git a/_docs/drill-on-yarn/091-appendix-b-drill-env.sh-settings.md b/_docs/drill-on-yarn/091-appendix-b-drill-env.sh-settings.md
new file mode 100644
index 0000000..6d86dc1
--- /dev/null
+++ b/_docs/drill-on-yarn/091-appendix-b-drill-env.sh-settings.md
@@ -0,0 +1,18 @@
+---
+title: "Appendix B: drill-env.sh Settings"
+date:
+parent: "Drill-on-YARN"
+---
+
+When running Drill outside of YARN, you can set many start-up options in drill-env.sh. Most
+users accept the defaults. However, some users require specialized settings.
+
+Under YARN, Drill still reads your $DRILL_SITE/drill-env.sh file to pick up configuration.
+However, for most options, Drill-on-YARN provides configuration options in drill-on-yarn.conf to set options that were formerly set in drill-env.sh.
+
+The following table provides a mapping:
+
+
+
+*If you set these options in both places, the value in drill-env.sh takes precedence. Note that EXTN_CLASSPATH (and drill.yarn.drillbit.extn-class-path) are a newer, more general way to add extensions. Rather than setting specific Hadoop or HBase variables, you can combine any number of extensions into the single extension classpath.
+
http://git-wip-us.apache.org/repos/asf/drill/blob/d8dd5d3d/_docs/drill-on-yarn/092-appendix-c-troubleshooting.md
----------------------------------------------------------------------
diff --git a/_docs/drill-on-yarn/092-appendix-c-troubleshooting.md b/_docs/drill-on-yarn/092-appendix-c-troubleshooting.md
new file mode 100644
index 0000000..dade3c7
--- /dev/null
+++ b/_docs/drill-on-yarn/092-appendix-c-troubleshooting.md
@@ -0,0 +1,96 @@
+---
+title: "Appendix C: Troubleshooting"
+date:
+parent: "Drill-on-YARN"
+---
+
+Drill-on-YARN starts a complex chain of events: the client starts the AM and the AM starts
+Drillbits, both using YARN. Many opportunities exist for configuration issues to derail the
+process. Below are a number of items to check if things go wrong.
+
+##Client Start
+The Drill-on-YARN client prints all messages to the console. Some common errors are:
+
+**Missing HADOOP_HOME**
+Drill-on-YARN requires access to your Hadoop configuration as described above. The client will
+display an error and fail if it is unable to load the DFS or YARN configuration, typically because
+HADOOP_HOME is not set.
+
+**Missing/wrong Drill Archive**
+Drill-on-YARN uploads your Drill archive to DFS. The client will fail if the archive configuration is missing, if the archive does not exist, is not readable, or is not in the correct format. Check that the drill.yarn.drill-install.client-path provides the full path name to the Drill archive.
+
+**DFS Configuration or Connection Problems**
+
+The Drill-on-YARN client uploads the Drill archive and your site directory to your DFS. Possible
+problems here include:
+● Missing DFS configuration in the Hadoop configuration folder
+● Incorrect DFS configuration (wrong host or port)
+● Missing permissions to write to the folder identified by the drill.yarn.dfs.app-dir configuration property (“/user/drill” by default.)
+
+**Wrong Version of the Drill Archive**
+
+Drill-on-YARN uploads a new copy of your site archive each time you start your Drill cluster.
+However, the Drill software archive is large, so Drill-on-YARN uploads the archive only when it
+changes. Drill detects changes by comparing the size of the DFS copy with your local copy.
+Most of the time this works fine. However, if you suspect that Drill has not uploaded the most
+recent copy, you can force the client to perform an upload by either manually deleting the Drill
+archive from DFS, or using the f option:
+
+ $DRILL_HOME/bin/drill-on-yarn.sh --site $DRILL_SITE start -f
+
+**Site Directory Problems**
+
+Drill creates a tar archive of your site directory using the following command:
+
+ tar -C $DRILL_SITE -czf /some/tmp/dir/temp-name.tar.gz
+
+For some temporary directory selected by Java. This command can fail if your version of tar
+does not recognize the above arguments, if the site directory is not readable, or the temporary
+file cannot be created.
+
+**YARN Application Launch Failure**
+
+YARN may fail to launch the Drill AM for a number of reasons. The user running the
+Drill-on-YARN client may not have permission to launch YARN applications. YARN itself may
+encounter errors. Check the YARN log files for possible causes.
+
+**Diagnosing Post-Launch Problems**
+If the Drill AM starts, but does not launch Drillbits, your next step is to check the Drill AM web UI using the link provided when the application starts. If the AM exits quickly, then the URL may
+not be valid.
+
+Instead, you can use YARN’s own Resource Manager UI to check the status of the application,
+using the Application ID provided when the application starts. Look at the application's log files
+for possible problems.
+
+**Application Master Port Conflict**
+
+The Drill-on-YARN Application Master provides a web UI on port 8048 (one greater than the Drill
+web UI port) by default. However, if another application is already bound to that port, the AM will
+fail to launch. Select a different port, as follows:
+
+ drill.yarn.http.port: 12345
+
+**Multiple AMs**
+It is easy to accidentally start multiple AMs for the same Drill cluster. Two lines of defense
+protect against this fault:
+
+The Drill-on-YARN client look for an existing appid file and refuses to start a new AM when the
+file is present. (Use the f file if the file is not valid.) The AM registers with ZK and will automatically shut down if another AM is already registered.
+
+
+**Drillbit Failure**
+● Finding launch, container, log directories
+● Double drillbit with same ports
+
+Diagnostics
+● Client
+● Client provides AM web UI
+● Finding launch, container, log directories
+● Client status
+
+Fixing Problems
+● Change config files
+● If Drill archive changes, force new upload
+● Missing app ID file (use a option)
+● Stop existing app (f to force stop)
+● Restart DoY (reuploads config)
http://git-wip-us.apache.org/repos/asf/drill/blob/d8dd5d3d/_docs/drill-on-yarn/093-appendix-d-recreate-the-drill-archive.md
----------------------------------------------------------------------
diff --git a/_docs/drill-on-yarn/093-appendix-d-recreate-the-drill-archive.md b/_docs/drill-on-yarn/093-appendix-d-recreate-the-drill-archive.md
new file mode 100644
index 0000000..796531b
--- /dev/null
+++ b/_docs/drill-on-yarn/093-appendix-d-recreate-the-drill-archive.md
@@ -0,0 +1,31 @@
+---
+title: "Appendix D: Recreate the Drill Archive"
+date:
+parent: "Drill-on-YARN"
+---
+
+Previous instructions assume that you make no changes to your Drill installation; that all your site-specific files reside in a separate site directory. Prior Drill versions put all configurations within the Drill directory. If you chose to continue that pattern, or if you change the Drill installation, you must rebuild the Drill archive and configure Drill-on-YARN
+to upload your custom archive in place of the standard archive. The steps below explain the process.
+
+To change the contents of the Drill archive, you must perform two steps:
+
+1. Create an archive of the Drill install directory.
+2. Configure Drill-on-YARN to use that archive.
+
+##Create the Drill Archive
+The first step is to create the master archive of your Drill files. Do the following with the master directory as the current directory.
+
+ cd $MASTER_DIR tar -czf archive-name.tar.gz $DRILL_HOME
+
+//Replace “archivename” with the name you created.
+
+##Configure Drill-on-YARN
+To use the archive, modify your drill-on-yarn.conf file to identify the archive you must created:
+
+ drill.yarn.drill-install.client-path: “/path/to/archive-name.tar.gz”
+
+YARN expects that, when extracting the master directory, that it creates a directory called
+archive-name that contains the Drill directories conf, jars, and so on. However, if archive-name is different than the name of the $DRILL_HOME directory, simply configure the correct name of the expanded folder:
+
+ drill.yarn.drill-install.dir-name: “your-dir-name”
+
http://git-wip-us.apache.org/repos/asf/drill/blob/d8dd5d3d/_docs/getting-started/010-drill-introduction.md
----------------------------------------------------------------------
diff --git a/_docs/getting-started/010-drill-introduction.md b/_docs/getting-started/010-drill-introduction.md
index e249017..3d60a3b 100644
--- a/_docs/getting-started/010-drill-introduction.md
+++ b/_docs/getting-started/010-drill-introduction.md
@@ -1,6 +1,6 @@
---
title: "Drill Introduction"
-date: 2017-12-16 06:24:22 UTC
+date: 2018-03-18 20:02:37 UTC
parent: "Getting Started"
---
Drill is an Apache open-source SQL query engine for Big Data exploration.
@@ -10,6 +10,27 @@ applications, while still providing the familiarity and ecosystem of ANSI SQL,
the industry-standard query language. Drill provides plug-and-play integration
with existing Apache Hive and Apache HBase deployments.
+## What's New in Apache Drill 1.13
+- JDK 8 support. ([DRILL-1491](https://issues.apache.org/jira/browse/DRILL-1491))
+- Upgrade to [Calcite version 1.15](https://calcite.apache.org/docs/history.html#v1-15-0). ([DRILL-3993](https://issues.apache.org/jira/browse/DRILL-3993))
+- JDBC Statement.setQueryTimeout(int) support to cancel queries if they do not complete within the specified time. ([DRILL-3640](https://issues.apache.org/jira/browse/DRILL-3640))
+- Batch processing improvements that enable you to [limit the amount of memory]({{site.baseurl}}/docs/configuring-drill-memory/#modifying-memory-allocated-to-queries) that the Flatten, Merge Join, and External Sort operators allocate to outgoing batches. ([DRILL-6123](https://issues.apache.org/jira/browse/DRILL-6123))
+- Enhanced DESCRIBE command. ([DRILL-4559](https://issues.apache.org/jira/browse/DRILL-4559))
+- Support for SPNEGO to extend Kerberos to Web applications through HTTP. ([DRILL-5425](https://issues.apache.org/jira/browse/DRILL-5425))
+- Ability to run [Drill under YARN]({{site.baseurl}}/docs/drill-on-yarn/). ([DRILL-1170](https://issues.apache.org/jira/browse/DRILL-1170))
+- Parquet filter pushdown support for IS [NOT] NULL, TRUE, and FALSE operators and implicit and explicit casts for timestamp, date, and time data types. ([DRILL-6174](https://issues.apache.org/jira/browse/DRILL-6174))
+- Performance improvements with support for project push down, filter push down, and partition pruning on dynamically expanded columns when represented as a star in the ITEM operator. ([DRILL-6118](https://issues.apache.org/jira/browse/DRILL-6118))
+- Updated Hive libraries and the Drill Hive client updated to 2.3.2 with support for querying Hive transactional ORC bucketed tables. ([DRILL-5978](https://issues.apache.org/jira/browse/DRILL-5978))
+- Ability to automatically manage memory allocations during Drill startup. ([DRILL-5741](https://issues.apache.org/jira/browse/DRILL-5741))
+- Ability to query an empty directory and use it for queries with any JOIN and UNION (UNION ALL) operators. ([Drill-4185](https://issues.apache.org/jira/browse/DRILL-4185))
+- Non-numeric support for JSON processing. ([Drill-5919](https://issues.apache.org/jira/browse/DRILL-5919))
+- New options to that enable you to configure the number of Jetty acceptors and selectors ([DRILL-5994](https://issues.apache.org/jira/browse/DRILL-5994))
+- Support SQL syntax highlighting of queries, auto-complete support in SQL editors, and snippets. ([DRILL-5868](https://issues.apache.org/jira/browse/DRILL-5868))
+- Improved performance of the Single Merge Exchange operator. ([DRILL-6115](https://issues.apache.org/jira/browse/DRILL-6115))
+- Like operator optimization. [DRILL-5879](https://issues.apache.org/jira/browse/DRILL-5879)
+- User/Distribution-specific configuration checks during startup ([DRILL-5741](https://issues.apache.org/jira/browse/DRILL-5741)).
+
+
## What's New in Apache Drill 1.12
Drill 1.12 provides the following new features and improvements:
http://git-wip-us.apache.org/repos/asf/drill/blob/d8dd5d3d/_docs/img/doy-envsh-mapping.PNG
----------------------------------------------------------------------
diff --git a/_docs/img/doy-envsh-mapping.PNG b/_docs/img/doy-envsh-mapping.PNG
new file mode 100644
index 0000000..c80fa8a
Binary files /dev/null and b/_docs/img/doy-envsh-mapping.PNG differ
http://git-wip-us.apache.org/repos/asf/drill/blob/d8dd5d3d/_docs/install/047-installing-drill-on-the-cluster.md
----------------------------------------------------------------------
diff --git a/_docs/install/047-installing-drill-on-the-cluster.md b/_docs/install/047-installing-drill-on-the-cluster.md
index 21ab2f0..9b5acd8 100644
--- a/_docs/install/047-installing-drill-on-the-cluster.md
+++ b/_docs/install/047-installing-drill-on-the-cluster.md
@@ -1,13 +1,13 @@
---
title: "Installing Drill on the Cluster"
-date: 2017-12-16 06:24:22 UTC
+date: 2018-03-18 20:02:37 UTC
parent: "Installing Drill in Distributed Mode"
---
You install Drill on nodes in the cluster, configure a cluster ID, and add Zookeeper information, as described in the following steps:
- 1. Download the latest version of Apache Drill [here](http://apache.mirrors.hoobly.com/drill/drill-1.12.0/apache-drill-1.12.0.tar.gz) or from the [Apache Drill mirror site](http://www.apache.org/dyn/closer.cgi/drill/drill-1.12.0/apache-drill-1.12.0.tar.gz) with the command appropriate for your system:
- * `wget http://apache.mirrors.hoobly.com/drill/drill-1.12.0/apache-drill-1.12.0.tar.gz`
- * `curl -o apache-drill-1.12.0.tar.gz http://apache.mirrors.hoobly.com/drill/drill-1.12.0/apache-drill-1.12.0.tar.gz`
+ 1. Download the latest version of Apache Drill [here](http://apache.mirrors.hoobly.com/drill/drill-1.13.0/apache-drill-1.13.0.tar.gz) or from the [Apache Drill mirror site](http://www.apache.org/dyn/closer.cgi/drill/drill-1.13.0/apache-drill-1.13.0.tar.gz) with the command appropriate for your system:
+ * `wget http://apache.mirrors.hoobly.com/drill/drill-1.13.0/apache-drill-1.13.0.tar.gz`
+ * `curl -o apache-drill-1.13.0.tar.gz http://apache.mirrors.hoobly.com/drill/drill-1.13.0/apache-drill-1.13.0.tar.gz`
2. Extract the tarball to the directory of your choice, such as `/opt`:
`tar -xzvf apache-drill-<version>.tar.gz`
3. In `drill-override.conf,` use the Drill `cluster ID`, and provide ZooKeeper host names and port numbers to configure a connection to your ZooKeeper quorum.
http://git-wip-us.apache.org/repos/asf/drill/blob/d8dd5d3d/_docs/install/installing-drill-in-embedded-mode/020-installing-drill-on-linux-and-mac-os-x.md
----------------------------------------------------------------------
diff --git a/_docs/install/installing-drill-in-embedded-mode/020-installing-drill-on-linux-and-mac-os-x.md b/_docs/install/installing-drill-in-embedded-mode/020-installing-drill-on-linux-and-mac-os-x.md
index 4342ef6..c560965 100644
--- a/_docs/install/installing-drill-in-embedded-mode/020-installing-drill-on-linux-and-mac-os-x.md
+++ b/_docs/install/installing-drill-in-embedded-mode/020-installing-drill-on-linux-and-mac-os-x.md
@@ -1,6 +1,6 @@
---
title: "Installing Drill on Linux and Mac OS X"
-date: 2017-12-16 06:24:23 UTC
+date: 2018-03-18 20:02:37 UTC
parent: "Installing Drill in Embedded Mode"
---
First, check that you [meet the prerequisites]({{site.baseurl}}/docs/embedded-mode-prerequisites), and then install Apache Drill on Linux or Mac OS X:
@@ -8,9 +8,9 @@ First, check that you [meet the prerequisites]({{site.baseurl}}/docs/embedded-mo
Complete the following steps to install Drill:
1. In a terminal window, change to the directory where you want to install Drill.
-2. Download the latest version of Apache Drill [here](http://apache.mirrors.hoobly.com/drill/drill-1.12.0/apache-drill-1.12.0.tar.gz) or from the [Apache Drill mirror site](http://www.apache.org/dyn/closer.cgi/drill/drill-1.12.0/apache-drill-1.12.0.tar.gz) with the command appropriate for your system:
- * `wget http://apache.mirrors.hoobly.com/drill/drill-1.12.0/apache-drill-1.12.0.tar.gz`
- * `curl -o apache-drill-1.12.0.tar.gz http://www.apache.org/dyn/closer.cgi/drill/drill-1.12.0/apache-drill-1.12.0.tar.gz`
+2. Download the latest version of Apache Drill [here](http://apache.mirrors.hoobly.com/drill/drill-1.13.0/apache-drill-1.13.0.tar.gz) or from the [Apache Drill mirror site](http://www.apache.org/dyn/closer.cgi/drill/drill-1.13.0/apache-drill-1.13.0.tar.gz) with the command appropriate for your system:
+ * `wget http://apache.mirrors.hoobly.com/drill/drill-1.13.0/apache-drill-1.13.0.tar.gz`
+ * `curl -o apache-drill-1.13.0.tar.gz http://www.apache.org/dyn/closer.cgi/drill/drill-1.13.0/apache-drill-1.13.0.tar.gz`
3. Copy the downloaded file to the directory where you want to install Drill.
4. Extract the contents of the Drill `.tar.gz` file. Use sudo only if necessary:
`tar -xvzf <.tar.gz file name>`
[2/3] drill git commit: Doc and website updates for the 1.13 release
Posted by br...@apache.org.
http://git-wip-us.apache.org/repos/asf/drill/blob/d8dd5d3d/_docs/rn/001-1.13.0-rn.md
----------------------------------------------------------------------
diff --git a/_docs/rn/001-1.13.0-rn.md b/_docs/rn/001-1.13.0-rn.md
new file mode 100644
index 0000000..01c4ae6
--- /dev/null
+++ b/_docs/rn/001-1.13.0-rn.md
@@ -0,0 +1,294 @@
+---
+title: "Apache Drill 1.13.0 Release Notes"
+parent: "Release Notes"
+---
+
+**Release date:** March 18, 2018
+
+Today, we're happy to announce the availability of Drill 1.13.0. You can download it [here](https://drill.apache.org/download/).
+
+## New Features and Improvements
+This release of Drill provides the following new features and improvements:
+
+- JDK 8 support. ([DRILL-1491](https://issues.apache.org/jira/browse/DRILL-1491))
+- Upgrade to [Calcite version 1.15](https://calcite.apache.org/docs/history.html#v1-15-0). ([DRILL-3993](https://issues.apache.org/jira/browse/DRILL-3993))
+- JDBC Statement.setQueryTimeout(int) support to cancel queries if they do not complete within the specified time. ([DRILL-3640](https://issues.apache.org/jira/browse/DRILL-3640))
+- Batch processing improvements that enable you to [limit the amount of memory]({{site.baseurl}}/docs/configuring-drill-memory/#modifying-memory-allocated-to-queries) that the Flatten, Merge Join, and External Sort operators allocate to outgoing batches. ([DRILL-6123](https://issues.apache.org/jira/browse/DRILL-6123))
+- Enhanced DESCRIBE command. ([DRILL-4559](https://issues.apache.org/jira/browse/DRILL-4559))
+- Support for SPNEGO to extend Kerberos to Web applications through HTTP. ([DRILL-5425](https://issues.apache.org/jira/browse/DRILL-5425))
+- Ability to run [Drill under YARN]({{site.baseurl}}/docs/drill-on-yarn/). ([DRILL-1170](https://issues.apache.org/jira/browse/DRILL-1170))
+- Parquet filter pushdown support for IS [NOT] NULL, TRUE, and FALSE operators and implicit and explicit casts for timestamp, date, and time data types. ([DRILL-6174](https://issues.apache.org/jira/browse/DRILL-6174))
+- Performance improvements with support for project push down, filter push down, and partition pruning on dynamically expanded columns when represented as a star in the ITEM operator. ([DRILL-6118](https://issues.apache.org/jira/browse/DRILL-6118))
+- Updated Hive libraries and the Drill Hive client updated to 2.3.2 with support for querying Hive transactional ORC bucketed tables. ([DRILL-5978](https://issues.apache.org/jira/browse/DRILL-5978))
+- Ability to automatically manage memory allocations during Drill startup. ([DRILL-5741](https://issues.apache.org/jira/browse/DRILL-5741))
+- Ability to query an empty directory and use it for queries with any JOIN and UNION (UNION ALL) operators. ([Drill-4185](https://issues.apache.org/jira/browse/DRILL-4185))
+- Non-numeric support for JSON processing. ([Drill-5919](https://issues.apache.org/jira/browse/DRILL-5919))
+- New options to that enable you to configure the number of Jetty acceptors and selectors ([DRILL-5994](https://issues.apache.org/jira/browse/DRILL-5994))
+- Support SQL syntax highlighting of queries, auto-complete support in SQL editors, and snippets. ([DRILL-5868](https://issues.apache.org/jira/browse/DRILL-5868))
+- Improved performance of the Single Merge Exchange operator. ([DRILL-6115](https://issues.apache.org/jira/browse/DRILL-6115))
+- Like operator optimization. [DRILL-5879](https://issues.apache.org/jira/browse/DRILL-5879)
+- User/Distribution-specific configuration checks during startup ([DRILL-5741](https://issues.apache.org/jira/browse/DRILL-5741)).
+
+
+The following sections list all fixes and improvements:
+
+<h2> Sub-task
+</h2>
+<ul>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4333'>DRILL-4333</a>] - tests in Drill2489CallsAfterCloseThrowExceptionsTest fail in Java 8
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5068'>DRILL-5068</a>] - Create a system table for completed profiles - sys.profiles
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6036'>DRILL-6036</a>] - Create a sys.connections table
+</li>
+</ul>
+
+<h2> Bug
+</h2>
+<ul>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4120'>DRILL-4120</a>] - dir0 does not work when the directory structure contains Avro files
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4329'>DRILL-4329</a>] - 13 Unit tests are failing with JDK 8
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4469'>DRILL-4469</a>] - SUM window query returns incorrect results over integer data
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4708'>DRILL-4708</a>] - connection closed unexpectedly
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4923'>DRILL-4923</a>] - Use of CASE WHEN inside a sub-query results in AssertionError
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4942'>DRILL-4942</a>] - incorrect result - case when (not null is null) then true else false end
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5170'>DRILL-5170</a>] - JMockit-based unit tests fail when run under Java 8
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5286'>DRILL-5286</a>] - When rel and target candidate set is the same, planner should not need to do convert for the relNode since it must have been done
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5377'>DRILL-5377</a>] - Five-digit year dates are displayed incorrectly via jdbc
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5690'>DRILL-5690</a>] - RepeatedDecimal18Vector does not pass scale, precision to data vector
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5730'>DRILL-5730</a>] - Fix Unit Test failures on JDK 8 And Some JDK 7 versions
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5768'>DRILL-5768</a>] - Drill planer should not allow select * with group by clause
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5833'>DRILL-5833</a>] - Parquet reader fails with assertion error for Decimal9, Decimal18 types
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5851'>DRILL-5851</a>] - Empty table during a join operation with a non empty table produces cast exception
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5880'>DRILL-5880</a>] - java.sql.SQLException: UNSUPPORTED_OPERATION ERROR: This query cannot be planned possibly due to either a cartesian join or an inequality join
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5902'>DRILL-5902</a>] - Queries encounter random failure due to RPC connection timed out
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5926'>DRILL-5926</a>] - TestValueVector tests fail sporadically
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5961'>DRILL-5961</a>] - For long running queries (> 10 min) Drill may raise FragmentSetupException for completed/cancelled fragments
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5963'>DRILL-5963</a>] - Canceling a query hung in planning state, leaves the query in ENQUEUED state for ever.
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5967'>DRILL-5967</a>] - Memory leak by HashPartitionSender
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5971'>DRILL-5971</a>] - Fix INT64, INT32 logical types in complex parquet reader
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5996'>DRILL-5996</a>] - Unable to re-run queries from Profiles tab with impersonation and without authentication
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6020'>DRILL-6020</a>] - NullPointerException with Union setting on when querying JSON untyped path
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6021'>DRILL-6021</a>] - Show shutdown button when authentication is not enabled
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6040'>DRILL-6040</a>] - Need to add usage for graceful_stop to drillbit.sh
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6044'>DRILL-6044</a>] - Shutdown button does not work from WebUI
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6054'>DRILL-6054</a>] - Issues in FindPartitionConditions
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6070'>DRILL-6070</a>] - Hash join with empty tables should not do casting of data types to INT<OPTIONAL>
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6079'>DRILL-6079</a>] - Memory leak caused by ParquetRowGroupScan
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6080'>DRILL-6080</a>] - Sort incorrectly limits batch size to 65535 records rather than 65536
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6081'>DRILL-6081</a>] - Duration of completed queries is continuously increasing
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6083'>DRILL-6083</a>] - RestClientFixture does not connect to the correct webserver port
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6085'>DRILL-6085</a>] - Travis build sometimes fails becomes vm suddenly exits.
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6088'>DRILL-6088</a>] - MainLoginPageModel errors out when http.auth.mechanisms is not configured
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6090'>DRILL-6090</a>] - While connecting to drill-bits using JDBC Driver through Zookeeper, a lot of "Curator-Framework-0" threads are created if connection to drill-bit is not successful(no drill-bits are up/reachable)
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6093'>DRILL-6093</a>] - Unneeded columns in Drill logical project
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6099'>DRILL-6099</a>] - Drill does not push limit past project (flatten) if it cannot be pushed into scan
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6100'>DRILL-6100</a>] - Intermittent failure while reading Parquet file footer during planning phase
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6111'>DRILL-6111</a>] - NullPointerException with Kafka Storage Plugin
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6117'>DRILL-6117</a>] - Query End time is not being set
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6119'>DRILL-6119</a>] - The OpenTSDB storage plugin is not included in the Drill distribution
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6124'>DRILL-6124</a>] - testCountDownLatch can be null in PartitionerDecorator depending on user's injection controls config
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6127'>DRILL-6127</a>] - NullPointerException happens when submitting physical plan to the Hive storage plugin
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6128'>DRILL-6128</a>] - Wrong Result with Nested Loop Join
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6129'>DRILL-6129</a>] - Query fails on nested data type schema change
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6137'>DRILL-6137</a>] - Join Failure When Some Json File Partitions Empty
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6140'>DRILL-6140</a>] - Operators listed in Profiles Page doesn't always correspond with operator specified in Physical Plan
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6143'>DRILL-6143</a>] - Make Fragment Runner's RPC Timeout a SystemOption
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6144'>DRILL-6144</a>] - Make directMemory amount configurable in tests
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6148'>DRILL-6148</a>] - TestSortSpillWithException is sometimes failing.
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6151'>DRILL-6151</a>] - Fragment executors may terminate without sending final batch to a downstream causing query to hang
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6154'>DRILL-6154</a>] - NaN, Infinity issues
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6164'>DRILL-6164</a>] - Heap memory leak during parquet scan and OOM
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6172'>DRILL-6172</a>] - setValueCount of VariableLengthVectors throws IOB exception when called with 0 value after clearing vectors
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6185'>DRILL-6185</a>] - Error is displaying while accessing query profiles via the Web-UI
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6187'>DRILL-6187</a>] - Exception in RPC communication between DataClient/ControlClient and respective servers when bit-to-bit security is on
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6188'>DRILL-6188</a>] - Fix C++ client build on Centos 7 and OSX
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6189'>DRILL-6189</a>] - Security: passwords logging and file permisions
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6190'>DRILL-6190</a>] - Packets can be bigger than strictly legal
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6191'>DRILL-6191</a>] - Need more information on TCP flags
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6192'>DRILL-6192</a>] - Drill is vulnerable to CVE-2017-12197
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6195'>DRILL-6195</a>] - Quering Hive non-partitioned transactional tables via Drill
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6197'>DRILL-6197</a>] - Duplicate entries in inputProfiles of minor fragments for specific operators
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6198'>DRILL-6198</a>] - OpenTSDB unit tests fail when Lilith client is run
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6204'>DRILL-6204</a>] - Pass tables columns without partition columns to empty Hive reader
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6205'>DRILL-6205</a>] - Reduce memory consumption of testFlattenUpperLimit test
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6216'>DRILL-6216</a>] - Metadata mismatch when connecting to a Drill 1.12.0 with a Drill-1.13.0-SNAPSHOT driver
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6217'>DRILL-6217</a>] - NaN/Inf: NestedLoopJoin processes NaN values incorrectly
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6226'>DRILL-6226</a>] - Projection pushdown does not occur with Calcite upgrade
+</li>
+</ul>
+
+<h2> New Feature
+</h2>
+<ul>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1170'>DRILL-1170</a>] - YARN support for Drill
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5425'>DRILL-5425</a>] - Support HTTP Kerberos auth using SPNEGO
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5868'>DRILL-5868</a>] - Support SQL syntax highlighting of queries
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6068'>DRILL-6068</a>] - Drill should support user/distribution specific configuration checks during startup
+</li>
+</ul>
+
+<h2> Improvement
+</h2>
+<ul>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3640'>DRILL-3640</a>] - Drill JDBC driver support Statement.setQueryTimeout(int)
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3958'>DRILL-3958</a>] - Improve error message when JDBC driver not found
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4185'>DRILL-4185</a>] - UNION ALL involving empty directory on any side of union all results in Failed query
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4559'>DRILL-4559</a>] - Re-work DESCRIBE command after moving to Calcite master branch
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5688'>DRILL-5688</a>] - Add repeated map support to column accessors
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5741'>DRILL-5741</a>] - Automatically manage memory allocations during startup
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5879'>DRILL-5879</a>] - Optimize "Like" operator
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5919'>DRILL-5919</a>] - Add non-numeric support for JSON processing
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5966'>DRILL-5966</a>] - Upgrade DRILL to Calcite 1.13.0
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5973'>DRILL-5973</a>] - Support injection of time-bound pauses in server
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5974'>DRILL-5974</a>] - Read JSON non-relational fields using text mode
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5978'>DRILL-5978</a>] - Updating of Apache and MapR Hive libraries to 2.3.2 and 2.1.1-mapr-1710 versions respectively
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5993'>DRILL-5993</a>] - Improve Performance of Copiers used by SV Remover, Top N etc.
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5994'>DRILL-5994</a>] - enable configuring number of Jetty acceptors and selectors
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6002'>DRILL-6002</a>] - Avoid memory copy from direct buffer to heap while spilling to local disk
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6025'>DRILL-6025</a>] - Execution time of a running query shown as 'NOT AVAILABLE'
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6028'>DRILL-6028</a>] - Allow splitting generated code in ChainedHashTable into blocks to avoid "code too large" error
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6049'>DRILL-6049</a>] - Rollup of hygiene changes from "batch size" project
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6063'>DRILL-6063</a>] - Set correct ThreadContext ClassLoader before using Hadoop Configuration class in DrillClient
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6071'>DRILL-6071</a>] - Limit batch size for flatten operator
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6089'>DRILL-6089</a>] - Validate That Planner Does Not Assume HashJoin Preserves Ordering for FS, MaprDB, or Hive
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6102'>DRILL-6102</a>] - CurrentModification Exception in BaseAllocator when debugging
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6106'>DRILL-6106</a>] - Use valueOf method instead of constructor since valueOf has a higher performance by caching frequently requested values.
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6114'>DRILL-6114</a>] - Complete internal metadata layer for improved batch handling
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6115'>DRILL-6115</a>] - SingleMergeExchange is not scaling up when many minor fragments are allocated for a query.
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6118'>DRILL-6118</a>] - Handle item star columns during project / filter push down and directory pruning
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6123'>DRILL-6123</a>] - Limit batch size for Merge Join based on memory
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6126'>DRILL-6126</a>] - Allocate memory for value vectors upfront in flatten operator
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6153'>DRILL-6153</a>] - Revised operator framework
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6163'>DRILL-6163</a>] - Switch Travis To Java 8
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6174'>DRILL-6174</a>] - Parquet pushdown planning improvements
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6177'>DRILL-6177</a>] - Merge Join - Allocate memory for outgoing value vectors based on sizes of incoming batches.
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6180'>DRILL-6180</a>] - Use System Option "output_batch_size" for External Sort
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6210'>DRILL-6210</a>] - Enhance the test schema builder for remaining types
+</li>
+</ul>
+
+<h2> Task
+</h2>
+<ul>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1491'>DRILL-1491</a>] - Support for JDK 8
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3993'>DRILL-3993</a>] - Rebase Drill on Calcite master branch
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6130'>DRILL-6130</a>] - Fix NPE during physical plan submission for various storage plugins
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6138'>DRILL-6138</a>] - Move RecordBatchSizer to org.apache.drill.exec.record package
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6208'>DRILL-6208</a>] - Fix FunctionInitializerTest#testConcurrentFunctionBodyLoad to use Mockito instead of JMockit
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6213'>DRILL-6213</a>] - During restart drillbit should be killed forcefully if exceeds allowed timeout
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6218'>DRILL-6218</a>] - Update release profile to not generate MD5 checksum
+</li>
+</ul>
+
+
http://git-wip-us.apache.org/repos/asf/drill/blob/d8dd5d3d/_docs/rn/009-1.12.0-rn.md
----------------------------------------------------------------------
diff --git a/_docs/rn/009-1.12.0-rn.md b/_docs/rn/009-1.12.0-rn.md
new file mode 100644
index 0000000..3c8fe4e
--- /dev/null
+++ b/_docs/rn/009-1.12.0-rn.md
@@ -0,0 +1,430 @@
+---
+title: "Apache Drill 1.12.0 Release Notes"
+parent: "Release Notes"
+---
+
+**Release date:** December 15, 2017
+
+Today, we're happy to announce the availability of Drill 1.12.0. You can download it [here](https://drill.apache.org/download/).
+
+## New Features and Improvements
+This release of Drill provides the following new features and improvements:
+
+- Kafka and OpenTSDB storage plugins (DRILL-4779, DRILL-5337)
+- SSL/TLS support (DRILL-5431)
+- Network encryption support (DRILL-5682)
+- Queue-based memory assignment for buffering operators (DRILL-5716)
+- A collection of networking functions that facilitate network analysis using Drill (DRILL-5834)
+- Support for the libpam4j PAM authenticator (DRILL-5820)
+- Filter pushdown for Parquet can handle files with multiple rowgroups (DRILL-5795)
+- UTF-8 is enabled in the query string by default (DRILL-5772)
+- IF NOT EXISTS support for CREATE TABLE and CREATE VIEWS (DRILL-5952)
+- Geometry functions, `ST_AsGeoJSON` and `ST_AsJSON`, that return GeoJSON and JSON representations (DRILL-5962, DRILL-5960)
+- JMX metrics for failed and canceled queries (DRILL-5909)
+- Syntax highlighting and error checking for storage plugin configurations (DRILL-5981)
+- System options improvements, including a new internal system options table (DRILL-5723)
+- Ability to prevent users from accessing a path outside the current workspace (DRILL-5964)
+- Ability to put the server in quiescent mode for a graceful shutdown (DRILL-4286)
+- The Drill Web Console lists the completion of successfully completed queries as "successful" (DRILL-5923)
+
+
+
+The following sections list additional bug fixes and improvements:
+
+<h2> Sub-task
+</h2>
+<ul>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5253'>DRILL-5253</a>] - External sort fails with OOM error (Fails to allocate sv2)
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5801'>DRILL-5801</a>] - Label the Gantt chart that indicates the lifespan details of the various minor fragments
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5802'>DRILL-5802</a>] - Provide a sortable table for tables within a query profile
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5803'>DRILL-5803</a>] - Show the hostname for each minor fragment in operator table
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5867'>DRILL-5867</a>] - List profiles in pages rather than a long verbose listing
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5881'>DRILL-5881</a>] - Java Client: [Threat Modeling] Drillbit may be spoofed by an attacker and this may lead to data being written to the attacker's target instead of Drillbit
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5882'>DRILL-5882</a>] - C++ Client: [Threat Modeling] Drillbit may be spoofed by an attacker and this may lead to data being written to the attacker's target instead of Drillbit
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6031'>DRILL-6031</a>] - Document support for dots in field names
+</li>
+</ul>
+
+<h2> Bug
+</h2>
+<ul>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1051'>DRILL-1051</a>] - Casting timestamp as date gives wrong result for dates earlier than 1883
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1499'>DRILL-1499</a>] - Different column order could appear in the result set for a schema-less select * query, even there are no changing schemas.
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3241'>DRILL-3241</a>] - Query with window function runs out of direct memory and does not report back to client that it did
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3407'>DRILL-3407</a>] - CTAS Auto Partition : The plan for count(*) should show the list of files scanned
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3449'>DRILL-3449</a>] - When Foreman node dies, the FragmentExecutor still tries to send status updates to Foreman
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3827'>DRILL-3827</a>] - Empty metadata file causes queries on the table to fail
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-3829'>DRILL-3829</a>] - Metadata Caching : Drill should ignore a corrupted cache file
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4139'>DRILL-4139</a>] - Fix parquet partition pruning for BIT, INTERVAL and DECIMAL types
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4255'>DRILL-4255</a>] - SELECT DISTINCT query over JSON data returns UNSUPPORTED OPERATION
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4640'>DRILL-4640</a>] - Unable to submit physical plan from Web UI on Windows
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4686'>DRILL-4686</a>] - Aggregation query over HBase table results in IllegalStateException: Failure while reading vector
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4734'>DRILL-4734</a>] - Query against HBase table on a 5 node cluster fails with SchemaChangeException
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4735'>DRILL-4735</a>] - Count(dir0) on parquet returns 0 result
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5002'>DRILL-5002</a>] - Using hive's date functions on top of date column gives wrong results for local time-zone
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5146'>DRILL-5146</a>] - Unnecessary spilling to disk by sort when we only have 5000 rows with one column
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5166'>DRILL-5166</a>] - Select with options returns NPE
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5185'>DRILL-5185</a>] - Union all not passing type info when the output contains 0 rows
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5268'>DRILL-5268</a>] - SYSTEM ERROR: UnsupportedOperationException: Unable to get size for minor type [MAP] and mode [REQUIRED]
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5269'>DRILL-5269</a>] - SYSTEM ERROR: JsonMappingException: No suitable constructor found for type [simple type, class org.apache.drill.exec.store.direct.DirectSubScan]
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5327'>DRILL-5327</a>] - Hash aggregate can return empty batch which can cause schema change exception
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5339'>DRILL-5339</a>] - Web UI: flaws in query profile display on error
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5341'>DRILL-5341</a>] - Web UI: remove duplicating link to documentation in Options page
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5346'>DRILL-5346</a>] - Web UI: remove link on user in query profile list
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5357'>DRILL-5357</a>] - Partition pruning information not available in query plan for COUNT aggregate query
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5416'>DRILL-5416</a>] - Vectors read from disk report incorrect memory sizes
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5442'>DRILL-5442</a>] - Managed Sort: IndexOutOfBounds with a join over an inlist
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5443'>DRILL-5443</a>] - Managed External Sort fails with OOM while spilling to disk
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5445'>DRILL-5445</a>] - Assertion Error in Managed External Sort when dealing with repeated maps
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5447'>DRILL-5447</a>] - Managed External Sort : Unable to allocate sv2 vector
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5464'>DRILL-5464</a>] - Fix JSON reader when it deals with empty file
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5465'>DRILL-5465</a>] - Managed external sort results in an OOM
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5478'>DRILL-5478</a>] - Spill file size parameter is not honored by the managed external sort
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5480'>DRILL-5480</a>] - Empty batch returning from HBase may cause SchemaChangeException even when data does not have different schema
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5507'>DRILL-5507</a>] - Millions of "Failure finding Drillbit running on host" info messages in foreman logs
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5513'>DRILL-5513</a>] - Managed External Sort : OOM error during the merge phase
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5519'>DRILL-5519</a>] - Sort fails to spill and results in an OOM
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5525'>DRILL-5525</a>] - Inconsistent, unhelpful semantics for batch, field schema comparison
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5546'>DRILL-5546</a>] - Schema change problems caused by empty batch
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5547'>DRILL-5547</a>] - Drill config options and session options do not work as intended
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5564'>DRILL-5564</a>] - IllegalStateException: allocator[op:21:1:5:HashJoinPOP]: buffer space (16674816) + prealloc space (0) + child space (0) != allocated (16740352)
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5582'>DRILL-5582</a>] - [Threat Modeling] Drillbit may be spoofed by an attacker and this may lead to data being written to the attacker's target instead of Drillbit
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5594'>DRILL-5594</a>] - Excessive buffer reallocations during merge phase of external sort
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5597'>DRILL-5597</a>] - Incorrect "bits" vector allocation in nullable vectors allocateNew()
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5602'>DRILL-5602</a>] - Repeated List Vector fails to initialize the offset vector
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5617'>DRILL-5617</a>] - Spill file name collisions when spill file is on a shared file system
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5645'>DRILL-5645</a>] - negation of expression causes null pointer exception
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5660'>DRILL-5660</a>] - Drill 1.10 queries fail due to Parquet Metadata "corruption" from DRILL-3867
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5663'>DRILL-5663</a>] - Drillbit fails to start when only keystore path is provided without keystore password.
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5675'>DRILL-5675</a>] - Drill C++ Client Date Time Literals Support Metadata Mapping is Incorrect
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5686'>DRILL-5686</a>] - Warning for sasl.max_wrapped_size contain incorrect syntax
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5687'>DRILL-5687</a>] - Disable TestMergeJoinWithSchemaChanges#testMissingAndNewColumns
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5694'>DRILL-5694</a>] - hash agg spill to disk, second phase OOM
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5698'>DRILL-5698</a>] - Drill should start in embedded mode using java 1.8.0_144
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5699'>DRILL-5699</a>] - Drill Web UI Page Source Has Links To External Sites
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5701'>DRILL-5701</a>] - drill.connections.rpc.<user/control/data>.<encrypted/unencrypted> metric not behaving correctly
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5702'>DRILL-5702</a>] - Jdbc Driver Class not found
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5710'>DRILL-5710</a>] - drill-config.sh incorrectly exits with Java 1.7 or later is required to run Apache Drill
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5714'>DRILL-5714</a>] - Fix NPE when mapr-db plugin is used in table function
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5715'>DRILL-5715</a>] - Performance of refactored HashAgg operator regressed
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5717'>DRILL-5717</a>] - change some date time unit cases with specific timezone or Local
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5721'>DRILL-5721</a>] - Query with only root fragment and no non-root fragment hangs when Drillbit to Drillbit Control Connection has network issues
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5725'>DRILL-5725</a>] - Update Jackson version to 2.7.8
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5727'>DRILL-5727</a>] - Update release profile to generate SHA-512 checksum.
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5729'>DRILL-5729</a>] - Fix Travis Checks
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5737'>DRILL-5737</a>] - Hash Agg uses more than the allocated memory under certain low memory conditions
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5740'>DRILL-5740</a>] - hash agg fail to read spill file
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5743'>DRILL-5743</a>] - Using order by clause in a select * query on hbase table returns only the row_key and order by field(s)
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5744'>DRILL-5744</a>] - External sort fails with OOM error
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5745'>DRILL-5745</a>] - Invalid "location" information in Drill web server
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5746'>DRILL-5746</a>] - Pcap PR manually edited Protobuf files, values lost on next build
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5749'>DRILL-5749</a>] - Foreman and Netty threads occure deadlock
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5751'>DRILL-5751</a>] - Fix unit tests to use local file system even if it is not set by default
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5753'>DRILL-5753</a>] - Managed External Sort: One or more nodes ran out of memory while executing the query.
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5755'>DRILL-5755</a>] - TOP_N_SORT operator does not free memory while running
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5757'>DRILL-5757</a>] - CONVERT_TO_JSON function is failed while using non-existence field as a parameter.
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5761'>DRILL-5761</a>] - Disable Lilith ClassicMultiplexSocketAppender by default
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5763'>DRILL-5763</a>] - Fix NPE during MapRDBSubScan deserialization
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5765'>DRILL-5765</a>] - Json query profile is not shown on Web UI
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5766'>DRILL-5766</a>] - Stored XSS in APACHE DRILL
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5771'>DRILL-5771</a>] - Fix serDe errors for format plugins
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5775'>DRILL-5775</a>] - Select * query on a maprdb binary table fails
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5790'>DRILL-5790</a>] - PCAP format explicitly opens local file
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5792'>DRILL-5792</a>] - CONVERT_FROM_JSON on an empty file throws runtime exception
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5804'>DRILL-5804</a>] - External Sort times out, may be infinite loop
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5811'>DRILL-5811</a>] - Large number of "Failure finding Drillbit" messages when using MFS
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5812'>DRILL-5812</a>] - Restore the minor fragment Gantt chart Canvas
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5816'>DRILL-5816</a>] - Hash function produces skewed results on String values with same leading prefix
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5819'>DRILL-5819</a>] - Default value of security.admin.user_groups and security.admin.users is "true"
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5822'>DRILL-5822</a>] - The query with "SELECT *" with "ORDER BY" clause and `planner.slice_target`=1 doesn't preserve column order
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5824'>DRILL-5824</a>] - 1st phase Hash Aggregate allocates more memory than the limit
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5830'>DRILL-5830</a>] - Resolve regressions to MapR DB from DRILL-5546
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5838'>DRILL-5838</a>] - Fix MaprDB filter pushdown for the case of nested field (reg. of DRILL-4264)
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5839'>DRILL-5839</a>] - Handle Empty Batches in Merge Receiver
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5840'>DRILL-5840</a>] - A query that includes sort completes, and then loses Drill connection. Drill becomes unresponsive, and cannot restart because it cannot communicate with Zookeeper
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5845'>DRILL-5845</a>] - Columns returned by select with "ORDER BY" and "LIMIT" clauses are not in correct order.
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5849'>DRILL-5849</a>] - Add freemarker lib to dependencyManagement to ensure proper version is used when resolving dependency version conflicts
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5853'>DRILL-5853</a>] - Sort removal based on NULL direction
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5854'>DRILL-5854</a>] - IllegalStateException when empty batch is received.
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5857'>DRILL-5857</a>] - Fix NumberFormatException in Hive unit tests
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5859'>DRILL-5859</a>] - Time for query queuing timeout not display correctly in WebUI
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5863'>DRILL-5863</a>] - Sortable table incorrectly sorts minor fragments and time elements lexically instead of sorting by implicit value
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5864'>DRILL-5864</a>] - Selecting a non-existing field from a MapR-DB JSON table fails with NPE
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5872'>DRILL-5872</a>] - Deserialization of profile JSON fails due to totalCost being reported as "NaN"
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5874'>DRILL-5874</a>] - NPE in AnonWebUserConnection.cleanupSession()
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5878'>DRILL-5878</a>] - TableNotFound exception is being reported for a wrong storage plugin.
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5887'>DRILL-5887</a>] - Display process user/ groups in Drill UI
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5888'>DRILL-5888</a>] - jdbc-all-jar unit tests broken because of dependency on hadoop.security
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5890'>DRILL-5890</a>] - Tests Leak Many Open File Descriptors
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5895'>DRILL-5895</a>] - Fix unit tests for mongo storage plugin
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5896'>DRILL-5896</a>] - Handle vector creation in HbaseRecordReader to avoid NullableInt vectors later
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5898'>DRILL-5898</a>] - Query returns columns in the wrong order
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5906'>DRILL-5906</a>] - java.lang.NullPointerException while quering Hive ORC tables on MapR cluster.
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5923'>DRILL-5923</a>] - State of a successfully completed query shown as "COMPLETED"
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5941'>DRILL-5941</a>] - Skip header / footer logic works incorrectly for Hive tables when file has several input splits
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5972'>DRILL-5972</a>] - Slow performance for query on INFORMATION_SCHEMA.TABLE
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5979'>DRILL-5979</a>] - unnable to access temporary table from drill web interface
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5986'>DRILL-5986</a>] - Update jackson-databind version to 2.7.9.1
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5987'>DRILL-5987</a>] - Two versions of javassist on the classpath
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6006'>DRILL-6006</a>] - Label current is missing on Web UI since the same drillbits with different status are considered to be different
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6007'>DRILL-6007</a>] - web ui index page is refreshed at high pace
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6017'>DRILL-6017</a>] - Fix for SHUTDOWN button being visible for non Admin users
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6019'>DRILL-6019</a>] - Only admin should be able to access shutdown resources
+</li>
+<li>Order by doesn't sort columns when window function is involved in the query</li>
+<li>MIN MAX DIR tests fail and return incorrect results</li>
+<li>header.line.count does not work for Hive table with data file size > chunk size</li>
+<li>lang.NoClassDefFoundError exception not handled in org.apache.drill.exec.rpc.security.ClientAuthenticatorProvider</li>
+<li>Investigate jackson-databind vulnerabilities CVE-2017-15095 & CVE-2017-7525</li>
+<li>Null Pointer Exception with query using table function</li>
+<li>Fix XSS vulnerabilities in Drill</li>
+<li>Some queries in concurrent execution get stuck in STARTING phase</li>
+<li>hashagg.num_partitions cannot be set by "alter session" or "alter system"</li>
+<li>Query with 2-way JOIN fails with "Hash join does not support schema changes"</li>
+<li>RowKeyJoin is generated instead of IndexScan for queries with derived table/ table functions</li>
+<li>ConvertCountToDirectScan rule enhancements</li>
+<li>Drill wrong old date-time values displaying</li>
+<li>Warning message "Exception while trying to prune partition" when query on view</li>
+<li>Drill not able to read MapR-DB when FQDN is longer than 64 characters</li>
+<li>Pam authentication with Centrify doesn't work by JPam restriction</li>
+<li>Hash aggregate does not support schema changes</li>
+<li>Simple query with only one join condition failed with "Hash join does not support schema changes"</li>
+<li>UnsupportedOperationException: Unable to get holder type for minor type [LATE] and mode [OPTIONAL].</li>
+<li>Query on TPC-H SF100 dataset fails with "Hash join does not support schema changes" [MapR-DB JSON Tables]</li>
+</ul>
+
+<h2> Improvement
+</h2>
+<ul>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-1691'>DRILL-1691</a>] - ConvertCountToDirectScan rule should be applicable for 2 or more COUNT aggregates
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4264'>DRILL-4264</a>] - Allow field names to include dots
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5089'>DRILL-5089</a>] - Skip initializing all enabled storage plugins for every query
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5106'>DRILL-5106</a>] - Refactor SkipRecordsInspector to exclude check for predefined file formats
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5259'>DRILL-5259</a>] - Allow listing a user-defined number of profiles
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5338'>DRILL-5338</a>] - Web UI: add better visual indication for PLANNING, QUEUED, EXECUTION in Query Profile that they are part of total time duration
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5704'>DRILL-5704</a>] - Improve error message on client side when queries fail with "Failed to create schema tree." when Impersonation is enabled and logins are anonymous
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5709'>DRILL-5709</a>] - Provide a value vector method to convert a vector to nullable
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5716'>DRILL-5716</a>] - Queue-based memory assignment for buffering operators
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5726'>DRILL-5726</a>] - Support Impersonation without authentication for REST API
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5795'>DRILL-5795</a>] - Filter pushdown for parquet handles multi rowgroup file
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5808'>DRILL-5808</a>] - Reduce memory allocator strictness for "managed" operators
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5815'>DRILL-5815</a>] - Provide option to set query memory as percent of total
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5832'>DRILL-5832</a>] - Migrate OperatorFixture to use SystemOptionManager rather than mock
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5834'>DRILL-5834</a>] - Add Networking Functions
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5842'>DRILL-5842</a>] - Refactor and simplify the fragment, operator contexts for testing
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5862'>DRILL-5862</a>] - Update project parent pom xml to the latest ASF version
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5893'>DRILL-5893</a>] - Maven forkCount property is too aggressive causing builds to fail on some machines.
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5899'>DRILL-5899</a>] - Simple pattern matchers can work with DrillBuf directly
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5909'>DRILL-5909</a>] - need new JMX metrics for (FAILED and CANCELED) queries
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5910'>DRILL-5910</a>] - Logging exception when custom AuthenticatorFactory not found
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5921'>DRILL-5921</a>] - Counters metrics should be listed in table
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5943'>DRILL-5943</a>] - Avoid the strong check introduced by DRILL-5582 for PLAIN mechanism
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5952'>DRILL-5952</a>] - Implement "CREATE TABLE IF NOT EXISTS"
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5960'>DRILL-5960</a>] - Add function STAsGeoJSON to extend GIS support
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5962'>DRILL-5962</a>] - Add function STAsJSON to extend GIS support
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5964'>DRILL-5964</a>] - Do not allow queries to access paths outside the current workspace root
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5980'>DRILL-5980</a>] - Make queryType param for REST API case insensitive
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5981'>DRILL-5981</a>] - Add Syntax Highlighting and Error Checking to Storage Plugin Config Page
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5989'>DRILL-5989</a>] - Run Smoke Tests On Travis
+</li>
+</ul>
+
+<h2> New Feature
+</h2>
+<ul>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4286'>DRILL-4286</a>] - Have an ability to put server in quiescent mode of operation
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4779'>DRILL-4779</a>] - Kafka storage plugin support
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5337'>DRILL-5337</a>] - OpenTSDB storage plugin
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5431'>DRILL-5431</a>] - Support SSL
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5682'>DRILL-5682</a>] - Apache Drill should support network encryption
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5723'>DRILL-5723</a>] - Support System/Session Internal Options And Additional Option System Fixes
+</li>
+</ul>
+
+<h2> Task
+</h2>
+<ul>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5601'>DRILL-5601</a>] - Rollup of External Sort memory management fixes
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5685'>DRILL-5685</a>] - Provide a way to set common environment variable between sqlline and Drillbit differently.
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5758'>DRILL-5758</a>] - Rollup of external sort fixes to issues found by QA
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5772'>DRILL-5772</a>] - Enable UTF-8 support in query string by default
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5781'>DRILL-5781</a>] - Fix unit test failures to use tests config even if default config is available
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5820'>DRILL-5820</a>] - Add support for libpam4j Pam Authenticator
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5911'>DRILL-5911</a>] - Upgrade esri-geometry-api version to 2.0.0 to avoid dependency on org.json library
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-5917'>DRILL-5917</a>] - Ban org.json:json library in Drill
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6000'>DRILL-6000</a>] - Graceful Shutdown Unit Tests Should Not Be Run On Travis
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-6011'>DRILL-6011</a>] - Update log info message in DrillClient
+</li>
+</ul>
+
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/drill/blob/d8dd5d3d/_docs/tutorials/020-drill-in-10-minutes.md
----------------------------------------------------------------------
diff --git a/_docs/tutorials/020-drill-in-10-minutes.md b/_docs/tutorials/020-drill-in-10-minutes.md
index b3c252e..96e6a39 100644
--- a/_docs/tutorials/020-drill-in-10-minutes.md
+++ b/_docs/tutorials/020-drill-in-10-minutes.md
@@ -1,6 +1,6 @@
---
title: "Drill in 10 Minutes"
-date: 2017-12-16 06:24:23 UTC
+date: 2018-03-18 20:02:37 UTC
parent: "Tutorials"
description: Get started with Drill in 10 minutes or less.
---
@@ -45,9 +45,9 @@ The output looks something like this:
Complete the following steps to install Drill:
1. In a terminal window, change to the directory where you want to install Drill.
-2. Download the latest version of Apache Drill [here](http://apache.mirrors.hoobly.com/drill/drill-1.12.0/apache-drill-1.12.0.tar.gz) or from the [Apache Drill mirror site](http://www.apache.org/dyn/closer.cgi/drill/drill-1.12.0/apache-drill-1.12.0.tar.gz) with the command appropriate for your system:
- * `wget http://apache.mirrors.hoobly.com/drill/drill-1.12.0/apache-drill-1.12.0.tar.gz`
- * `curl -o apache-drill-1.12.0.tar.gz http://apache.mirrors.hoobly.com/drill/drill-1.12.0/apache-drill-1.12.0.tar.gz`
+2. Download the latest version of Apache Drill [here](http://apache.mirrors.hoobly.com/drill/drill-1.13.0/apache-drill-1.13.0.tar.gz) or from the [Apache Drill mirror site](http://www.apache.org/dyn/closer.cgi/drill/drill-1.13.0/apache-drill-1.13.0.tar.gz) with the command appropriate for your system:
+ * `wget http://apache.mirrors.hoobly.com/drill/drill-1.13.0/apache-drill-1.13.0.tar.gz`
+ * `curl -o apache-drill-1.13.0.tar.gz http://apache.mirrors.hoobly.com/drill/drill-1.13.0/apache-drill-1.13.0.tar.gz`
3. Copy the downloaded file to the directory where you want to install Drill.
4. Extract the contents of the Drill .tar.gz file. Use `sudo` if necessary:
`tar -xvzf <.tar.gz file name>`
@@ -75,7 +75,7 @@ Start Drill in embedded mode using the `drill-embedded` command:
You can install Drill on Windows. First, set the JAVA_HOME environment variable, and then install Drill. Complete the following steps to install Drill:
-1. Download the latest version of Apache Drill [here](http://apache.mirrors.hoobly.com/drill/drill-1.12.0/apache-drill-1.12.0.tar.gz) or from the [Apache Drill mirror site](http://www.apache.org/dyn/closer.cgi/drill/drill-1.12.0/apache-drill-1.12.0.tar.gz).
+1. Download the latest version of Apache Drill [here](http://apache.mirrors.hoobly.com/drill/drill-1.13.0/apache-drill-1.13.0.tar.gz) or from the [Apache Drill mirror site](http://www.apache.org/dyn/closer.cgi/drill/drill-1.13.0/apache-drill-1.13.0.tar.gz).
2. Move the `apache-drill-<version>.tar.gz` file to a directory where you want to install Drill.
3. Unzip the `TAR.GZ` file using a third-party tool. If the tool you use does not unzip the TAR file as well as the `TAR.GZ` file, unzip the `apache-drill-<version>.tar` to extract the Drill software. The extraction process creates the installation directory named apache-drill-<version> containing the Drill software.
http://git-wip-us.apache.org/repos/asf/drill/blob/d8dd5d3d/_docs/tutorials/050-analyzing-highly-dynamic-datasets.md
----------------------------------------------------------------------
diff --git a/_docs/tutorials/050-analyzing-highly-dynamic-datasets.md b/_docs/tutorials/050-analyzing-highly-dynamic-datasets.md
index bc17acf..052d4fa 100644
--- a/_docs/tutorials/050-analyzing-highly-dynamic-datasets.md
+++ b/_docs/tutorials/050-analyzing-highly-dynamic-datasets.md
@@ -1,6 +1,6 @@
---
title: "Analyzing Highly Dynamic Datasets"
-date: 2018-02-08 02:32:57 UTC
+date: 2018-03-18 20:02:38 UTC
parent: "Tutorials"
---
@@ -33,7 +33,7 @@ The element names that you see in the `checkin_info` are unknown upfront and can
Step 1: First download Drill, if you have not yet done so, onto your machine
http://drill.apache.org/download/
- tar -xvf apache-drill-1.12.0.tar
+ tar -xvf apache-drill-1.13.0.tar
Install Drill locally on your desktop (embedded mode). You don’t need Hadoop.
http://git-wip-us.apache.org/repos/asf/drill/blob/d8dd5d3d/blog/_posts/2018-3-18-drill-1.13-released.md
----------------------------------------------------------------------
diff --git a/blog/_posts/2018-3-18-drill-1.13-released.md b/blog/_posts/2018-3-18-drill-1.13-released.md
new file mode 100644
index 0000000..6b8ea3a
--- /dev/null
+++ b/blog/_posts/2018-3-18-drill-1.13-released.md
@@ -0,0 +1,39 @@
+---
+layout: post
+title: "Drill 1.13 Released"
+code: drill-1.13-released
+excerpt: Apache Drill 1.13's highlights are: YARN support, support for HTTP Kerberos authentication using SPNEGO, SQL syntax highlighting of queries, and user and distribution specific configuration checks during startup.
+authors: ["bbevens"]
+---
+
+Today, we're happy to announce the availability of Drill 1.13.0. You can download it [here](https://drill.apache.org/download/).
+
+The release provides the following bug fixes and improvements:
+
+## Ability to Run Drill Under YARN (DRILL-1170)
+You can run Drill as a YARN application ([Drill-on-YARN]({{site.baseurl}}/docs/drill-on-yarn/)) if you want Drill to work alongside other applications, such as Hadoop and Spark, in a YARN-managed cluster. YARN assigns resources, such as memory and CPU, to applications in the cluster and eliminates the manual steps associated with installation and resource allocation for stand-alone applications in a multi-tenant environment. YARN automatically deploys (localizes) the Drill software onto each Drill node and manages the Drill cluster. Drill becomes a long-running application with YARN. You can monitor the Drill-on-YARN cluster using the Application Master web UI.
+
+
+
+## SPNEGO Support (DRILL-5425)
+
+You can use SPNEGO to extend Kerberos authentication to Web applications through HTTP.
+
+## SQL Syntax Support (DRILL-5868)
+Query syntax appears highlighted in the Drill Web Console. In addition to syntax highlighting, auto-complete is supported in all SQL editors, including the Edit Query tab within an existing profile to rerun the query. For browsers like Chrome, you can type Ctrl+Space for a drop-down list and then use arrow keys for navigating through options. An auto-complete feature that specifies Drill keywords and functions, and the ability to write SQL from templates using snippets.
+
+
+## User/Distribution-Specific Configuration Checks During Startup (DRILL-5741)
+
+ You can define the maximum amount of cumulative memory allocated to the Drill process during startup through the `DRILLBIT_MAX_PROC_MEM` environment variable. For example, if you set `DRILLBIT_MAX_PROC_MEM to 40G`, the total amount of memory allocated to the following memory parameters cannot exceed 40G:
+
+- `DRILL_HEAP=8G `
+- `DRILL_MAX_DIRECT_MEMORY=10G`
+- `DRILLBIT_CODE_CACHE_SIZE=1024M`
+
+At startup, an auto-setup.sh script performs a check to see if these memory parameters are declared. If the parameters are declared, the script performs a check to verify that the cumulative memory of the parameters does not exceed the value specified by `DRILLBIT_MAX_PROC_MEM`. If the cumulative memory exceeds the total amount of memory defined by `DRILLBIT_MAX_PROC_MEM`, Drill returns an error message with instructions.
+
+
+
+You can find a complete list of JIRAs resolved in the 1.13.0 release [here]({{site.baseurl}}/docs/apache-drill-1-13-0-release-notes/).
+