You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@drill.apache.org by br...@apache.org on 2016/08/30 22:29:28 UTC
[13/17] drill git commit: Updates to docs for Drill 1.8
Updates to docs for Drill 1.8
Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/5465a443
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/5465a443
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/5465a443
Branch: refs/heads/gh-pages
Commit: 5465a443bc50047ac1fc49965dd31a2ab4b2ae1f
Parents: ddb7fcf
Author: Bridget Bevens <bb...@maprtech.com>
Authored: Sat Aug 13 12:26:05 2016 -0700
Committer: Bridget Bevens <bb...@maprtech.com>
Committed: Sat Aug 13 12:26:05 2016 -0700
----------------------------------------------------------------------
.../010-partition-pruning-introduction.md | 4 +-
_docs/rn/003-1.8.0-rn.md | 47 +++++++++++++++-----
2 files changed, 38 insertions(+), 13 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/drill/blob/5465a443/_docs/performance-tuning/partition-pruning/010-partition-pruning-introduction.md
----------------------------------------------------------------------
diff --git a/_docs/performance-tuning/partition-pruning/010-partition-pruning-introduction.md b/_docs/performance-tuning/partition-pruning/010-partition-pruning-introduction.md
index e5f4e5f..f64a633 100644
--- a/_docs/performance-tuning/partition-pruning/010-partition-pruning-introduction.md
+++ b/_docs/performance-tuning/partition-pruning/010-partition-pruning-introduction.md
@@ -1,12 +1,12 @@
---
title: "Partition Pruning Introduction"
-date: 2016-08-11 19:02:20 UTC
+date: 2016-08-08 18:42:19 UTC
parent: "Partition Pruning"
---
Partition pruning is a performance optimization that limits the number of files and partitions that Drill reads when querying file systems and Hive tables. When you partition data, Drill only reads a subset of the files that reside in a file system or a subset of the partitions in a Hive table when a query matches certain filter criteria.
-As of Drill 1.8, partition pruning also applies to the Parquet metadata cache. When data is partitioned in a directory hierarchy, Drill attempts to read the metadata cache file from a sub-partition, based on matching filter criteria instead of reading from the top level partition, to reduce the amount of metadata read during the query planning time. If you created a metadata cache file in a previous version of Drill, you must issue the REFRESH TABLE METADATA command to regenerate the metadata cache file before running queries for partition pruning to occur. See [Optimizing Parquet Metadata Reading]({{site.baseurl}}/docs/optimizing-parquet-metadata-reading/) for more information.
+As of Drill 1.8, partition pruning also applies to the Parquet metadata cache. When data is partitioned in a directory hierarchy, Drill attempts to read the metadata cache file from a sub-partition, based on matching filter criteria instead of reading from the top level partition, to reduce the amount of metadata read during the query planning time. If you created a metadata cache file in a previous version of Drill, you must issue the REFRESH TABLE METADATA command to regenerate the metadata cache file before running queries for metadata cache pruning to occur. See [Optimizing Parquet Metadata Reading]({{site.baseurl}}/docs/optimizing-parquet-metadata-reading/) for more information.
The query planner in Drill performs partition pruning by evaluating the filters. If no partition filters are present, the underlying Scan operator reads all files in all directories and then sends the data to operators, such as Filter, downstream. When partition filters are present, the query planner pushes the filters down to the Scan if possible. The Scan reads only the directories that match the partition filters, thus reducing disk I/O.
http://git-wip-us.apache.org/repos/asf/drill/blob/5465a443/_docs/rn/003-1.8.0-rn.md
----------------------------------------------------------------------
diff --git a/_docs/rn/003-1.8.0-rn.md b/_docs/rn/003-1.8.0-rn.md
index 65de9a2..78db742 100644
--- a/_docs/rn/003-1.8.0-rn.md
+++ b/_docs/rn/003-1.8.0-rn.md
@@ -3,24 +3,33 @@ title: "Apache Drill 1.8.0 Release Notes"
parent: "Release Notes"
---
-**Release date:** August, 2016
+**Release date:** August 15, 2016
Today, we're happy to announce the availability of Drill 1.8.0. You can download it [here](https://drill.apache.org/download/).
-This release provides metadata cache pruning, support for the IF EXISTS parameter with the DROP TABLE and DROP VIEW commands, support for the DESCRIBE SCHEMA command, multi-byte delimiter support, new parameters for filter selectivity estimates, and the following bug fixes and improvements:
+This release provides metadata cache pruning, support for the IF EXISTS parameter with the DROP TABLE and DROP VIEW commands, support for the DESCRIBE SCHEMA command, multi-byte delimiter support, and new parameters for filter selectivity estimates.
+
+## Configuration and Launch Script Changes
+This release of Drill also includes the following changes to the configuration and launch scripts:
+
+- Default Drill settings now reside in `$DRILL_HOME/bin/drill-config.sh`. You can override many settings by creating an entry in `$DRILL_HOME/conf/drill-env.sh`. The file includes descriptions of the options that you can set. ([DRILL-4581](https://issues.apache.org/jira/browse/DRILL-4581))
+- Due to issues at high concurrency, the native Linux epoll transport is now disabled by default. ([DRILL-4623](https://issues.apache.org/jira/browse/DRILL-4623))
+
+If you upgrade to Drill 1.8, you must merge your custom settings with the latest settings in the `drill-override.conf` and `drill-env.sh` file that ships with Drill. As of Drill 1.8, all Drill defaults reside in the Drill scripts. The `drill-env.sh` script contains only your customizations. When you merge your existing `drill-env.sh` file with the 1.8 version of the file, you can remove all of the settings in your file except for those you created yourself. Consult the original `drill-env.sh` file from the prior Drill release to determine which settings you can remove.
+
+
+
+Drill 1.8 provides the following bug fixes and improvements:
-
<h2> Sub-task
</h2>
<ul>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4560'>DRILL-4560</a>] - ZKClusterCoordinator does not call DrillbitStatusListener.drillbitRegistered for new bits
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4581'>DRILL-4581</a>] - Various problems in the Drill startup scripts
</li>
<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4728'>DRILL-4728</a>] - Add support for new metadata fetch APIs
</li>
<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4729'>DRILL-4729</a>] - Add support for prepared statement implementation on server side
</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4730'>DRILL-4730</a>] - Update JDBC DatabaseMetaData implementation to use new Metadata APIs
-</li>
<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4732'>DRILL-4732</a>] - Update JDBC driver to use the new prepared statement APIs on DrillClient
</li>
</ul>
@@ -38,12 +47,16 @@ This release provides metadata cache pruning, support for the IF EXISTS paramete
</li>
<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4574'>DRILL-4574</a>] - Avro Plugin: Flatten does not work correctly on record items
</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4623'>DRILL-4623</a>] - Disable Epoll by Default
+</li>
<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4658'>DRILL-4658</a>] - cannot specify tab as a fieldDelimiter in table function
</li>
<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4664'>DRILL-4664</a>] - ScanBatch.isNewSchema() returns wrong result for map datatype
</li>
<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4665'>DRILL-4665</a>] - Partition pruning not working for hive partitioned table with 'LIKE' and '=' filter
</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4704'>DRILL-4704</a>] - select statement behavior is inconsistent for decimal values in parquet
+</li>
<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4707'>DRILL-4707</a>] - Conflicting columns names under case-insensitive policy lead to either memory leak or incorrect result
</li>
<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4715'>DRILL-4715</a>] - Java compilation error for a query with large number of expressions
@@ -68,6 +81,10 @@ This release provides metadata cache pruning, support for the IF EXISTS paramete
</li>
<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4825'>DRILL-4825</a>] - Wrong data with UNION ALL when querying different sub-directories under the same table
</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4836'>DRILL-4836</a>] - ZK Issue during Drillbit startup, possibly due to race condition
+</li>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4846'>DRILL-4846</a>] - Eliminate extra operations during metadata cache pruning
+</li>
</ul>
<h2> Improvement
@@ -83,11 +100,11 @@ This release provides metadata cache pruning, support for the IF EXISTS paramete
</li>
<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4751'>DRILL-4751</a>] - Remove dumpcat script from Drill distribution
</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4752'>DRILL-4752</a>] - Remove submit_plan script from Drill distribution
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4766'>DRILL-4766</a>] - FragmentExecutor should use EventProcessor and avoid blocking rpc threads
</li>
<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4786'>DRILL-4786</a>] - Improve metadata cache performance for queries with multiple partitions
</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4792'>DRILL-4792</a>] - Include session options used for a query as part of the profile
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4822'>DRILL-4822</a>] - Extend distrib-env.sh search to consider site directory
</li>
</ul>
@@ -98,9 +115,17 @@ This release provides metadata cache pruning, support for the IF EXISTS paramete
</li>
<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4673'>DRILL-4673</a>] - Implement "DROP TABLE IF EXISTS" for drill to prevent FAILED status on command return
</li>
-<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4714'>DRILL-4714</a>] - Add metadata and prepared statement APIs to DrillClient<->Drillbit interface
-</li>
<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4819'>DRILL-4819</a>] - Update MapR version to 5.2.0
</li>
</ul>
-
\ No newline at end of file
+
+<h2> Task
+</h2>
+<ul>
+<li>[<a href='https://issues.apache.org/jira/browse/DRILL-4499'>DRILL-4499</a>] - Remove unused classes
+</li>
+</ul>
+
+
+
+
\ No newline at end of file