You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by mi...@apache.org on 2018/12/06 23:14:58 UTC
[32/51] [partial] impala git commit: [DOCS] Impala 3.1 Docs to be
published
http://git-wip-us.apache.org/repos/asf/impala/blob/b4ad38a9/docs/build/html/topics/impala_drop_stats.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_drop_stats.html b/docs/build/html/topics/impala_drop_stats.html
index d50ac73..179c3af 100644
--- a/docs/build/html/topics/impala_drop_stats.html
+++ b/docs/build/html/topics/impala_drop_stats.html
@@ -1,8 +1,28 @@
+<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html
- SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="drop_stats"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>DROP STATS Statement</title></head><body id="drop_stats"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+ PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+
+<meta name="copyright" content="(C) Copyright 2018" />
+<meta name="DC.rights.owner" content="(C) Copyright 2018" />
+<meta name="DC.Type" content="concept" />
+<meta name="DC.Title" content="DROP STATS Statement" />
+<meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html" />
+<meta name="prodname" content="Impala" />
+<meta name="prodname" content="Impala" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="DC.Format" content="XHTML" />
+<meta name="DC.Identifier" content="drop_stats" />
+<link rel="stylesheet" type="text/css" href="../commonltr.css" />
+<title>DROP STATS Statement</title>
+</head>
+<body id="drop_stats">
+
<h1 class="title topictitle1" id="ariaid-title1">DROP STATS Statement</h1>
+
@@ -14,10 +34,12 @@
<code class="ph codeph">COMPUTE STATS</code> or <code class="ph codeph">COMPUTE INCREMENTAL STATS</code> statement.
</p>
+
<p class="p">
<strong class="ph b">Syntax:</strong>
</p>
+
<pre class="pre codeblock"><code>DROP STATS [<var class="keyword varname">database_name</var>.]<var class="keyword varname">table_name</var>
DROP INCREMENTAL STATS [<var class="keyword varname">database_name</var>.]<var class="keyword varname">table_name</var> PARTITION (<var class="keyword varname">partition_spec</var>)
@@ -33,11 +55,13 @@ DROP INCREMENTAL STATS [<var class="keyword varname">database_name</var>.]<var c
specification, and specify constant values for all the partition key columns.
</p>
+
<p class="p">
<code class="ph codeph">DROP STATS</code> removes all statistics from the table, whether created by <code class="ph codeph">COMPUTE
STATS</code> or <code class="ph codeph">COMPUTE INCREMENTAL STATS</code>.
</p>
+
<p class="p">
<code class="ph codeph">DROP INCREMENTAL STATS</code> only affects incremental statistics for a single partition, specified
through the <code class="ph codeph">PARTITION</code> clause. The incremental stats are marked as outdated, so that they are
@@ -46,10 +70,12 @@ DROP INCREMENTAL STATS [<var class="keyword varname">database_name</var>.]<var c
+
<p class="p">
<strong class="ph b">Usage notes:</strong>
</p>
+
<p class="p">
You typically use this statement when the statistics for a table or a partition have become stale due to data
files being added to or removed from the associated HDFS data directories, whether by manual HDFS operations
@@ -57,6 +83,7 @@ DROP INCREMENTAL STATS [<var class="keyword varname">database_name</var>.]<var c
adding or dropping partitions.
</p>
+
<p class="p">
When a table or partition has no associated statistics, Impala treats it as essentially zero-sized when
constructing the execution plan for a query. In particular, the statistics influence the order in which
@@ -65,6 +92,7 @@ DROP INCREMENTAL STATS [<var class="keyword varname">database_name</var>.]<var c
the table or partition after removing any stale statistics.
</p>
+
<p class="p">
Dropping the statistics is not required for an unpartitioned table or a partitioned table covered by the
original type of statistics. A subsequent <code class="ph codeph">COMPUTE STATS</code> statement replaces any existing
@@ -72,6 +100,7 @@ DROP INCREMENTAL STATS [<var class="keyword varname">database_name</var>.]<var c
this statement was rarely used before the introduction of incremental statistics.
</p>
+
<p class="p">
Dropping the statistics is required for a partitioned table containing incremental statistics, to make a
subsequent <code class="ph codeph">COMPUTE INCREMENTAL STATS</code> statement rescan an existing partition. See
@@ -79,17 +108,21 @@ DROP INCREMENTAL STATS [<var class="keyword varname">database_name</var>.]<var c
available in Impala 2.1.0 and higher.
</p>
+
<p class="p">
<strong class="ph b">Statement type:</strong> DDL
</p>
+
<p class="p">
<strong class="ph b">Cancellation:</strong> Cannot be cancelled.
</p>
+
<p class="p">
<strong class="ph b">HDFS permissions:</strong>
</p>
+
<p class="p">
The user ID that the <span class="keyword cmdname">impalad</span> daemon runs under,
typically the <code class="ph codeph">impala</code> user, does not need any
@@ -98,22 +131,26 @@ DROP INCREMENTAL STATS [<var class="keyword varname">database_name</var>.]<var c
not HDFS files and directories.
</p>
+
<p class="p">
<strong class="ph b">Examples:</strong>
</p>
+
<p class="p">
The following example shows a partitioned table that has associated statistics produced by the
<code class="ph codeph">COMPUTE INCREMENTAL STATS</code> statement, and how the situation evolves as statistics are dropped
from specific partitions, then the entire table.
</p>
+
<p class="p">
Initially, all table and column statistics are filled in.
</p>
+
<pre class="pre codeblock"><code>show table stats item_partitioned;
+-------------+-------+--------+----------+--------------+---------+-----------------
| i_category | #Rows | #Files | Size | Bytes Cached | Format | Incremental stats
@@ -166,12 +203,14 @@ show column stats item_partitioned;
the column-level statistics are unaffected.
</p>
- <div class="note note note_note"><span class="note__title notetitle">Note:</span>
+
+ <div class="note note"><span class="notetitle">Note:</span>
(It is possible that the row count might be preserved in future after a <code class="ph codeph">DROP INCREMENTAL
STATS</code> statement. Check the resolution of the issue
<a class="xref" href="https://issues.apache.org/jira/browse/IMPALA-1615" target="_blank">IMPALA-1615</a>.)
</div>
+
<pre class="pre codeblock"><code>drop incremental stats item_partitioned partition (i_category='Sports');
drop incremental stats item_partitioned partition (i_category='Electronics');
@@ -226,6 +265,7 @@ show column stats item_partitioned
<code class="ph codeph">INCREMENTAL</code> clause). Now, both table-level and column-level statistics are reset.
</p>
+
<pre class="pre codeblock"><code>drop stats item_partitioned;
show table stats item_partitioned
@@ -277,9 +317,17 @@ show column stats item_partitioned
<strong class="ph b">Related information:</strong>
</p>
+
<p class="p">
<a class="xref" href="impala_compute_stats.html#compute_stats">COMPUTE STATS Statement</a>, <a class="xref" href="impala_show.html#show_table_stats">SHOW TABLE STATS Statement</a>,
<a class="xref" href="impala_show.html#show_column_stats">SHOW COLUMN STATS Statement</a>, <a class="xref" href="impala_perf_stats.html#perf_stats">Table and Column Statistics</a>
</p>
+
</div>
-<nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_langref_sql.html">Impala SQL Statements</a></div></div></nav></article></main></body></html>
\ No newline at end of file
+
+<div class="related-links">
+<div class="familylinks">
+<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_langref_sql.html">Impala SQL Statements</a></div>
+</div>
+</div></body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/impala/blob/b4ad38a9/docs/build/html/topics/impala_drop_table.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_drop_table.html b/docs/build/html/topics/impala_drop_table.html
index ac156bd..16321fc 100644
--- a/docs/build/html/topics/impala_drop_table.html
+++ b/docs/build/html/topics/impala_drop_table.html
@@ -1,8 +1,28 @@
+<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html
- SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="drop_table"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>DROP TABLE Statement</title></head><body id="drop_table"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+ PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+
+<meta name="copyright" content="(C) Copyright 2018" />
+<meta name="DC.rights.owner" content="(C) Copyright 2018" />
+<meta name="DC.Type" content="concept" />
+<meta name="DC.Title" content="DROP TABLE Statement" />
+<meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html" />
+<meta name="prodname" content="Impala" />
+<meta name="prodname" content="Impala" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="DC.Format" content="XHTML" />
+<meta name="DC.Identifier" content="drop_table" />
+<link rel="stylesheet" type="text/css" href="../commonltr.css" />
+<title>DROP TABLE Statement</title>
+</head>
+<body id="drop_table">
+
<h1 class="title topictitle1" id="ariaid-title1">DROP TABLE Statement</h1>
+
@@ -14,16 +34,19 @@
external tables.
</p>
+
<p class="p">
<strong class="ph b">Syntax:</strong>
</p>
+
<pre class="pre codeblock"><code>DROP TABLE [IF EXISTS] [<var class="keyword varname">db_name</var>.]<var class="keyword varname">table_name</var> <span class="ph">[PURGE]</span></code></pre>
<p class="p">
<strong class="ph b">IF EXISTS clause:</strong>
</p>
+
<p class="p">
The optional <code class="ph codeph">IF EXISTS</code> clause makes the statement succeed whether or not the table exists.
If the table does exist, it is dropped; if it does not exist, the statement has no effect. This capability is
@@ -34,10 +57,12 @@
already exist).
</p>
+
<p class="p">
<strong class="ph b">PURGE clause:</strong>
</p>
+
<p class="p"> The optional <code class="ph codeph">PURGE</code> keyword, available in
<span class="keyword">Impala 2.3</span> and higher, causes Impala to remove the associated
HDFS data files immediately, rather than going through the HDFS trashcan
@@ -46,19 +71,23 @@
problem with the trashcan, such as the trash cannot being configured or
being in a different HDFS encryption zone than the data files. </p>
+
<p class="p">
<strong class="ph b">Statement type:</strong> DDL
</p>
+
<p class="p">
<strong class="ph b">Usage notes:</strong>
</p>
+
<p class="p">
By default, Impala removes the associated HDFS directory and data files for the table. If you issue a
<code class="ph codeph">DROP TABLE</code> and the data files are not deleted, it might be for the following reasons:
</p>
+
<ul class="ul">
<li class="li">
If the table was created with the
@@ -67,29 +96,35 @@
components, and Impala is only used to query the data files from their original locations.
</li>
+
<li class="li">
Impala might leave the data files behind unintentionally, if there is no HDFS location available to hold
the HDFS trashcan for the <code class="ph codeph">impala</code> user. See
<a class="xref" href="impala_prereqs.html#prereqs_account">User Account Requirements</a> for the procedure to set up the required HDFS home
directory.
</li>
+
</ul>
+
<p class="p">
Make sure that you are in the correct database before dropping a table, either by issuing a
<code class="ph codeph">USE</code> statement first or by using a fully qualified name
<code class="ph codeph"><var class="keyword varname">db_name</var>.<var class="keyword varname">table_name</var></code>.
</p>
+
<p class="p">
If you intend to issue a <code class="ph codeph">DROP DATABASE</code> statement, first issue <code class="ph codeph">DROP TABLE</code>
statements to remove all the tables in that database.
</p>
+
<p class="p">
<strong class="ph b">Examples:</strong>
</p>
+
<pre class="pre codeblock"><code>create database temporary;
use temporary;
create table unimportant (x int);
@@ -109,9 +144,11 @@ drop table temporary.trivial;</code></pre>
<a class="xref" href="../shared/../topics/impala_disk_space.html#disk_space">Managing Disk Space for Impala Data</a>.
</p>
+
<p class="p">
<strong class="ph b">Amazon S3 considerations:</strong>
</p>
+
<p class="p">
The <code class="ph codeph">DROP TABLE</code> statement can remove data files from S3
if the associated S3 table is an internal table.
@@ -121,13 +158,17 @@ drop table temporary.trivial;</code></pre>
See <a class="xref" href="impala_s3.html#s3">Using Impala with the Amazon S3 Filesystem</a> for details about working with S3 tables.
</p>
+
<div class="p">
For best compatibility with the S3 write support in <span class="keyword">Impala 2.6</span>
and higher:
<ul class="ul">
<li class="li">Use native Hadoop techniques to create data files in S3 for querying through Impala.</li>
+
<li class="li">Use the <code class="ph codeph">PURGE</code> clause of <code class="ph codeph">DROP TABLE</code> when dropping internal (managed) tables.</li>
+
</ul>
+
By default, when you drop an internal (managed) table, the data files are
moved to the HDFS trashcan. This operation is expensive for tables that
reside on the Amazon S3 filesystem. Therefore, for S3 tables, prefer to use
@@ -139,6 +180,7 @@ drop table temporary.trivial;</code></pre>
<code class="ph codeph">hadoop fs -cp</code>, or <code class="ph codeph">INSERT</code> in Impala or Hive.
</div>
+
<p class="p">
In <span class="keyword">Impala 2.6</span> and higher, Impala DDL statements such as
<code class="ph codeph">CREATE DATABASE</code>, <code class="ph codeph">CREATE TABLE</code>, <code class="ph codeph">DROP DATABASE CASCADE</code>,
@@ -148,27 +190,33 @@ drop table temporary.trivial;</code></pre>
See <a class="xref" href="../shared/../topics/impala_s3.html#s3">Using Impala with the Amazon S3 Filesystem</a> for details about reading and writing S3 data with Impala.
</p>
+
<p class="p">
<strong class="ph b">Cancellation:</strong> Cannot be cancelled.
</p>
+
<p class="p">
<strong class="ph b">HDFS permissions:</strong>
</p>
+
<p class="p">
For an internal table, the user ID that the <span class="keyword cmdname">impalad</span> daemon runs under,
typically the <code class="ph codeph">impala</code> user, must have write
permission for all the files and directories that make up the table.
</p>
+
<p class="p">
For an external table, dropping the table only involves changes to metadata in the metastore database.
Because Impala does not remove any HDFS files or directories when external tables are dropped,
no particular permissions are needed for the associated HDFS files or directories.
</p>
+
<p class="p">
<strong class="ph b">Kudu considerations:</strong>
</p>
+
<p class="p">
Kudu tables can be managed or external, the same as with HDFS-based
tables. For a managed table, the underlying Kudu table and its data
@@ -177,10 +225,12 @@ drop table temporary.trivial;</code></pre>
<code class="ph codeph">DROP TABLE</code>.
</p>
+
<p class="p">
<strong class="ph b">Related information:</strong>
</p>
+
<p class="p">
<a class="xref" href="impala_tables.html#tables">Overview of Impala Tables</a>,
<a class="xref" href="impala_alter_table.html#alter_table">ALTER TABLE Statement</a>, <a class="xref" href="impala_create_table.html#create_table">CREATE TABLE Statement</a>,
@@ -188,5 +238,12 @@ drop table temporary.trivial;</code></pre>
<a class="xref" href="impala_tables.html#external_tables">External Tables</a>
</p>
+
</div>
-<nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_langref_sql.html">Impala SQL Statements</a></div></div></nav></article></main></body></html>
\ No newline at end of file
+
+<div class="related-links">
+<div class="familylinks">
+<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_langref_sql.html">Impala SQL Statements</a></div>
+</div>
+</div></body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/impala/blob/b4ad38a9/docs/build/html/topics/impala_drop_view.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_drop_view.html b/docs/build/html/topics/impala_drop_view.html
index bb671b9..9724a8d 100644
--- a/docs/build/html/topics/impala_drop_view.html
+++ b/docs/build/html/topics/impala_drop_view.html
@@ -1,8 +1,28 @@
+<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html
- SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="drop_view"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>DROP VIEW Statement</title></head><body id="drop_view"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+ PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+
+<meta name="copyright" content="(C) Copyright 2018" />
+<meta name="DC.rights.owner" content="(C) Copyright 2018" />
+<meta name="DC.Type" content="concept" />
+<meta name="DC.Title" content="DROP VIEW Statement" />
+<meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html" />
+<meta name="prodname" content="Impala" />
+<meta name="prodname" content="Impala" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="DC.Format" content="XHTML" />
+<meta name="DC.Identifier" content="drop_view" />
+<link rel="stylesheet" type="text/css" href="../commonltr.css" />
+<title>DROP VIEW Statement</title>
+</head>
+<body id="drop_view">
+
<h1 class="title topictitle1" id="ariaid-title1">DROP VIEW Statement</h1>
+
@@ -16,29 +36,35 @@
HDFS.
</p>
+
<p class="p">
<strong class="ph b">Syntax:</strong>
</p>
+
<pre class="pre codeblock"><code>DROP VIEW [IF EXISTS] [<var class="keyword varname">db_name</var>.]<var class="keyword varname">view_name</var></code></pre>
<p class="p">
<strong class="ph b">Statement type:</strong> DDL
</p>
+
<p class="p">
<strong class="ph b">Cancellation:</strong> Cannot be cancelled.
</p>
+
<p class="p">
<strong class="ph b">HDFS permissions:</strong> This statement does not touch any HDFS files or directories,
therefore no HDFS permissions are required.
</p>
+
<p class="p">
<strong class="ph b">Examples:</strong>
</p>
+
<div class="p">
The following example creates a series of views and then drops them. These examples illustrate how views
are associated with a particular database, and both the view definitions and the view names for
@@ -68,13 +94,22 @@ DROP VIEW db2.v1;
</code></pre>
</div>
+
<p class="p">
<strong class="ph b">Related information:</strong>
</p>
+
<p class="p">
<a class="xref" href="impala_views.html#views">Overview of Impala Views</a>, <a class="xref" href="impala_create_view.html#create_view">CREATE VIEW Statement</a>,
<a class="xref" href="impala_alter_view.html#alter_view">ALTER VIEW Statement</a>
</p>
+
</div>
-<nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_langref_sql.html">Impala SQL Statements</a></div></div></nav></article></main></body></html>
\ No newline at end of file
+
+<div class="related-links">
+<div class="familylinks">
+<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_langref_sql.html">Impala SQL Statements</a></div>
+</div>
+</div></body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/impala/blob/b4ad38a9/docs/build/html/topics/impala_enable_expr_rewrites.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_enable_expr_rewrites.html b/docs/build/html/topics/impala_enable_expr_rewrites.html
new file mode 100644
index 0000000..32481cd
--- /dev/null
+++ b/docs/build/html/topics/impala_enable_expr_rewrites.html
@@ -0,0 +1,88 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE html
+ PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+
+<meta name="copyright" content="(C) Copyright 2018" />
+<meta name="DC.rights.owner" content="(C) Copyright 2018" />
+<meta name="DC.Type" content="concept" />
+<meta name="DC.Title" content="ENABLE_EXPR_REWRITES Query Option" />
+<meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html" />
+<meta name="prodname" content="Impala" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="DC.Format" content="XHTML" />
+<meta name="DC.Identifier" content="enable_expr_rewrites" />
+<link rel="stylesheet" type="text/css" href="../commonltr.css" />
+<title>ENABLE_EXPR_REWRITES Query Option</title>
+</head>
+<body id="enable_expr_rewrites">
+
+
+ <h1 class="title topictitle1" id="ariaid-title1">ENABLE_EXPR_REWRITES Query Option</h1>
+
+
+
+
+
+
+ <div class="body conbody">
+
+ <p class="p">
+ The <code class="ph codeph">ENABLE_EXPR_REWRITES</code> query option controls whether to enable or
+ disable the query compile time optimizations. These optimizations rewrite the expression
+ trees to a more compact and optimized form that helps avoid redundant expression
+ evaluation at run time. Performance optimizations controlled by this query option include:
+ </p>
+
+
+ <ul class="ul">
+ <li class="li">
+ Constant folding (added in <span class="keyword">)</span>
+ </li>
+
+
+ <li class="li">
+ Extracting common conjuncts from disjunctions (added in
+ <span class="keyword">)</span>
+ </li>
+
+
+ <li class="li">
+ Simplify conditionals with constant conditions (added in
+ <span class="keyword">)</span>
+ </li>
+
+ </ul>
+
+
+ <p class="p">
+ Set the option to <code class="ph codeph">false</code> or <code class="ph codeph">0</code> to disable the performance
+ optimizations.
+ </p>
+
+
+ <p class="p">
+ <strong class="ph b">Type: </strong><code class="ph codeph">boolean</code>
+ </p>
+
+
+ <p class="p">
+ <strong class="ph b">Default:</strong> <code class="ph codeph">true</code> (<code class="ph codeph">1</code>)
+ </p>
+
+
+ <p class="p">
+ <strong class="ph b">Added in:</strong> <span class="keyword">Impala 2.8</span>
+ </p>
+
+
+ </div>
+
+
+<div class="related-links">
+<div class="familylinks">
+<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_query_options.html">Query Options for the SET Statement</a></div>
+</div>
+</div></body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/impala/blob/b4ad38a9/docs/build/html/topics/impala_exec_single_node_rows_threshold.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_exec_single_node_rows_threshold.html b/docs/build/html/topics/impala_exec_single_node_rows_threshold.html
index 95bb859..5347f1c 100644
--- a/docs/build/html/topics/impala_exec_single_node_rows_threshold.html
+++ b/docs/build/html/topics/impala_exec_single_node_rows_threshold.html
@@ -1,8 +1,28 @@
+<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html
- SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="exec_single_node_rows_threshold"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>EXEC_SINGLE_NODE_ROWS_THRESHOLD Query Option (Impala 2.1 or higher only)</title></head><body id="exec_single_node_rows_threshold"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+ PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+
+<meta name="copyright" content="(C) Copyright 2018" />
+<meta name="DC.rights.owner" content="(C) Copyright 2018" />
+<meta name="DC.Type" content="concept" />
+<meta name="DC.Title" content="EXEC_SINGLE_NODE_ROWS_THRESHOLD Query Option (Impala 2.1 or higher only)" />
+<meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html" />
+<meta name="prodname" content="Impala" />
+<meta name="prodname" content="Impala" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="DC.Format" content="XHTML" />
+<meta name="DC.Identifier" content="exec_single_node_rows_threshold" />
+<link rel="stylesheet" type="text/css" href="../commonltr.css" />
+<title>EXEC_SINGLE_NODE_ROWS_THRESHOLD Query Option (Impala 2.1 or higher only)</title>
+</head>
+<body id="exec_single_node_rows_threshold">
+
<h1 class="title topictitle1" id="ariaid-title1">EXEC_SINGLE_NODE_ROWS_THRESHOLD Query Option (<span class="keyword">Impala 2.1</span> or higher only)</h1>
+
@@ -14,24 +34,28 @@
as a <span class="q">"small"</span> query, turning off optimizations such as parallel execution and native code generation. The
overhead for these optimizations is applicable for queries involving substantial amounts of data, but it
makes sense to skip them for queries involving tiny amounts of data. Reducing the overhead for small queries
- allows Impala to complete them more quickly, keeping YARN resources, admission control slots, and so on
- available for data-intensive queries.
+ allows Impala to complete them more quickly, keeping admission control slots, CPU, memory, and so on
+ available for resource-intensive queries.
</p>
+
<p class="p">
<strong class="ph b">Syntax:</strong>
</p>
+
<pre class="pre codeblock"><code>SET EXEC_SINGLE_NODE_ROWS_THRESHOLD=<var class="keyword varname">number_of_rows</var></code></pre>
<p class="p">
<strong class="ph b">Type:</strong> numeric
</p>
+
<p class="p">
<strong class="ph b">Default:</strong> 100
</p>
+
<p class="p">
<strong class="ph b">Usage notes:</strong> Typically, you increase the default value to make this optimization apply to more queries.
If incorrect or corrupted table and column statistics cause Impala to apply this optimization
@@ -41,16 +65,20 @@
statistics, you can turn this feature off by setting the value to -1.
</p>
+
<p class="p">
<strong class="ph b">Internal details:</strong>
</p>
+
<p class="p">
- This setting applies to query fragments where the amount of data to scan can be accurately determined, either
- through table and column statistics, or by the presence of a <code class="ph codeph">LIMIT</code> clause. If Impala cannot
- accurately estimate the size of the input data, this setting does not apply.
+ This setting applies to queries where the number of rows processed can be accurately
+ determined, either through table and column statistics, or by the presence of a
+ <code class="ph codeph">LIMIT</code> clause. If Impala cannot accurately estimate the number of rows,
+ then this setting does not apply.
</p>
+
<p class="p">
In <span class="keyword">Impala 2.3</span> and higher, where Impala supports the complex data types <code class="ph codeph">STRUCT</code>,
<code class="ph codeph">ARRAY</code>, and <code class="ph codeph">MAP</code>, if a query refers to any column of those types,
@@ -58,26 +86,31 @@
<code class="ph codeph">EXEC_SINGLE_NODE_ROWS_THRESHOLD</code> setting.
</p>
+
<p class="p">
For a query that is determined to be <span class="q">"small"</span>, all work is performed on the coordinator node. This might
result in some I/O being performed by remote reads. The savings from not distributing the query work and not
generating native code are expected to outweigh any overhead from the remote reads.
</p>
+
<p class="p">
<strong class="ph b">Added in:</strong> <span class="keyword">Impala 2.10</span>
</p>
+
<p class="p">
<strong class="ph b">Examples:</strong>
</p>
+
<p class="p">
A common use case is to query just a few rows from a table to inspect typical data values. In this example,
Impala does not parallelize the query or perform native code generation because the result set is guaranteed
to be smaller than the threshold value from this query option:
</p>
+
<pre class="pre codeblock"><code>SET EXEC_SINGLE_NODE_ROWS_THRESHOLD=500;
SELECT * FROM enormous_table LIMIT 300;
</code></pre>
@@ -86,4 +119,10 @@ SELECT * FROM enormous_table LIMIT 300;
</div>
-<nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_query_options.html">Query Options for the SET Statement</a></div></div></nav></article></main></body></html>
\ No newline at end of file
+
+<div class="related-links">
+<div class="familylinks">
+<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_query_options.html">Query Options for the SET Statement</a></div>
+</div>
+</div></body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/impala/blob/b4ad38a9/docs/build/html/topics/impala_exec_time_limit_s.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_exec_time_limit_s.html b/docs/build/html/topics/impala_exec_time_limit_s.html
index c904a01..42e2a13 100644
--- a/docs/build/html/topics/impala_exec_time_limit_s.html
+++ b/docs/build/html/topics/impala_exec_time_limit_s.html
@@ -1,6 +1,25 @@
+<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html
- SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="exec_time_limit_s"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>EXEC_TIME_LIMIT_S Query Option (Impala 2.12 or higher only)</title></head><body id="exec_time_limit_s"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+ PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+
+<meta name="copyright" content="(C) Copyright 2018" />
+<meta name="DC.rights.owner" content="(C) Copyright 2018" />
+<meta name="DC.Type" content="concept" />
+<meta name="DC.Title" content="EXEC_TIME_LIMIT_S Query Option (Impala 2.12 or higher only)" />
+<meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html" />
+<meta name="prodname" content="Impala" />
+<meta name="prodname" content="Impala" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="DC.Format" content="XHTML" />
+<meta name="DC.Identifier" content="exec_time_limit_s" />
+<link rel="stylesheet" type="text/css" href="../commonltr.css" />
+<title>EXEC_TIME_LIMIT_S Query Option (Impala 2.12 or higher only)</title>
+</head>
+<body id="exec_time_limit_s">
+
<h1 class="title topictitle1" id="ariaid-title1">EXEC_TIME_LIMIT_S Query Option (<span class="keyword">Impala 2.12</span> or higher only)</h1>
@@ -8,6 +27,7 @@
+
<div class="body conbody">
<p class="p">
@@ -16,6 +36,7 @@
option is intended to prevent runaway queries that execute for much longer than intended.
</p>
+
<p class="p">
For example, an Impala administrator could set a default value of
<code class="ph codeph">EXEC_TIME_LIMIT_S=3600</code> for a resource pool to automatically kill queries
@@ -28,7 +49,8 @@
session.
</p>
- <div class="note note note_note"><span class="note__title notetitle">Note:</span>
+
+ <div class="note note"><span class="notetitle">Note:</span>
<p class="p">
The time limit only starts once the query is executing. Time spent planning the query,
scheduling the query, or in admission control is not counted towards the execution time
@@ -36,35 +58,49 @@
the client has fetched all result rows. DML queries are eligible for automatic
cancellation until the DML statement has finished.
</p>
+
</div>
+
<p class="p">
<strong class="ph b">Syntax:</strong>
</p>
+
<pre class="pre codeblock"><code>SET EXEC_TIME_LIMIT_S=<var class="keyword varname">seconds</var>;</code></pre>
<p class="p">
<strong class="ph b">Type:</strong> numeric
</p>
+
<p class="p">
<strong class="ph b">Default:</strong> 0 (no time limit )
</p>
+
<p class="p">
<strong class="ph b">Added in:</strong>
<span class="keyword">Impala 2.12</span>
</p>
+
<p class="p">
<strong class="ph b">Related information:</strong>
</p>
+
<p class="p">
<a class="xref" href="impala_timeouts.html#timeouts">Setting Timeout Periods for Daemons, Queries, and Sessions</a>
</p>
+
</div>
-<nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_query_options.html">Query Options for the SET Statement</a></div></div></nav></article></main></body></html>
+
+<div class="related-links">
+<div class="familylinks">
+<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_query_options.html">Query Options for the SET Statement</a></div>
+</div>
+</div></body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/impala/blob/b4ad38a9/docs/build/html/topics/impala_explain.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_explain.html b/docs/build/html/topics/impala_explain.html
index 29d7c47..240eb06 100644
--- a/docs/build/html/topics/impala_explain.html
+++ b/docs/build/html/topics/impala_explain.html
@@ -1,8 +1,28 @@
+<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html
- SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="explain"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>EXPLAIN Statement</title></head><body id="explain"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+ PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+
+<meta name="copyright" content="(C) Copyright 2018" />
+<meta name="DC.rights.owner" content="(C) Copyright 2018" />
+<meta name="DC.Type" content="concept" />
+<meta name="DC.Title" content="EXPLAIN Statement" />
+<meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html" />
+<meta name="prodname" content="Impala" />
+<meta name="prodname" content="Impala" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="DC.Format" content="XHTML" />
+<meta name="DC.Identifier" content="explain" />
+<link rel="stylesheet" type="text/css" href="../commonltr.css" />
+<title>EXPLAIN Statement</title>
+</head>
+<body id="explain">
+
<h1 class="title topictitle1" id="ariaid-title1">EXPLAIN Statement</h1>
+
@@ -15,10 +35,12 @@
network. Use <code class="ph codeph">explain</code> followed by a complete <code class="ph codeph">SELECT</code> query. For example:
</p>
+
<p class="p">
<strong class="ph b">Syntax:</strong>
</p>
+
<pre class="pre codeblock"><code>EXPLAIN { <var class="keyword varname">select_query</var> | <var class="keyword varname">ctas_stmt</var> | <var class="keyword varname">insert_stmt</var> }
</code></pre>
@@ -27,22 +49,26 @@
<code class="ph codeph">WITH</code> clause. See <a class="xref" href="impala_select.html#select">SELECT Statement</a> for details.
</p>
+
<p class="p">
The <var class="keyword varname">insert_stmt</var> is an <code class="ph codeph">INSERT</code> statement that inserts into or overwrites an
existing table. It can use either the <code class="ph codeph">INSERT ... SELECT</code> or <code class="ph codeph">INSERT ...
VALUES</code> syntax. See <a class="xref" href="impala_insert.html#insert">INSERT Statement</a> for details.
</p>
+
<p class="p">
The <var class="keyword varname">ctas_stmt</var> is a <code class="ph codeph">CREATE TABLE</code> statement using the <code class="ph codeph">AS
SELECT</code> clause, typically abbreviated as a <span class="q">"CTAS"</span> operation. See
<a class="xref" href="impala_create_table.html#create_table">CREATE TABLE Statement</a> for details.
</p>
+
<p class="p">
<strong class="ph b">Usage notes:</strong>
</p>
+
<p class="p">
You can interpret the output to judge whether the query is performing efficiently, and adjust the query
and/or the schema if not. For example, you might change the tests in the <code class="ph codeph">WHERE</code> clause, add
@@ -51,6 +77,7 @@
performance tuning steps.
</p>
+
<p class="p">
The <code class="ph codeph">EXPLAIN</code> output reminds you if table or column statistics are missing from any table
involved in the query. These statistics are important for optimizing queries involving large tables or
@@ -58,6 +85,7 @@
and <a class="xref" href="impala_perf_stats.html#perf_stats">Table and Column Statistics</a> for how to use this information for query tuning.
</p>
+
<div class="p">
Read the <code class="ph codeph">EXPLAIN</code> plan from bottom to top:
<ul class="ul">
@@ -67,25 +95,31 @@
take to scan a table based on total data size and the size of the cluster.
</li>
+
<li class="li">
As you work your way up, next you see the operations that will be parallelized and performed on each
Impala node.
</li>
+
<li class="li">
At the higher levels, you see how data flows when intermediate result sets are combined and transmitted
from one node to another.
</li>
+
<li class="li">
See <a class="xref" href="../shared/../topics/impala_explain_level.html#explain_level">EXPLAIN_LEVEL Query Option</a> for details about the
<code class="ph codeph">EXPLAIN_LEVEL</code> query option, which lets you customize how much detail to show in the
<code class="ph codeph">EXPLAIN</code> plan depending on whether you are doing high-level or low-level tuning,
dealing with logical or physical aspects of the query.
</li>
+
</ul>
+
</div>
+
<p class="p">
If you come from a traditional database background and are not familiar with data warehousing, keep in mind
that Impala is optimized for full table scans across very large tables. The structure and distribution of
@@ -96,15 +130,18 @@
able to optimize a query so that it executes in seconds rather than minutes.
</p>
+
<p class="p">
For more information and examples to help you interpret <code class="ph codeph">EXPLAIN</code> output, see
<a class="xref" href="impala_explain_plan.html#perf_explain">Using the EXPLAIN Plan for Performance Tuning</a>.
</p>
+
<p class="p">
<strong class="ph b">Extended EXPLAIN output:</strong>
</p>
+
<p class="p">
For performance tuning of complex queries, and capacity planning (such as using the admission control and
resource management features), you can enable more detailed and informative output for the
@@ -114,20 +151,24 @@
<code class="ph codeph">extended</code>, or <code class="ph codeph">verbose</code>.
</p>
+
<p class="p">
When extended <code class="ph codeph">EXPLAIN</code> output is enabled, <code class="ph codeph">EXPLAIN</code> statements print
information about estimated memory requirements, minimum number of virtual cores, and so on.
</p>
+
<p class="p">
See <a class="xref" href="impala_explain_level.html#explain_level">EXPLAIN_LEVEL Query Option</a> for details and examples.
</p>
+
<p class="p">
<strong class="ph b">Examples:</strong>
</p>
+
<p class="p">
This example shows how the standard <code class="ph codeph">EXPLAIN</code> output moves from the lowest (physical) level to
the higher (logical) levels. The query begins by scanning a certain amount of data; each node performs an
@@ -136,6 +177,7 @@
<code class="ph codeph">EXCHANGE</code> node); lastly, the intermediate results are summed to display the final result.
</p>
+
<pre class="pre codeblock" id="explain__explain_plan_simple"><code>[impalad-host:21000] > explain select count(*) from customer_address;
+----------------------------------------------------------+
| Explain String |
@@ -164,6 +206,7 @@
statement has additional information to use in deciding how to optimize the distributed query.
</p>
+
<pre class="pre codeblock"><code>[localhost:21000] > set explain_level=extended;
@@ -216,19 +259,23 @@ EXPLAIN_LEVEL set to extended
<p class="p">
<strong class="ph b">Security considerations:</strong>
</p>
+
<p class="p">
If these statements in your environment contain sensitive literal values such as credit card numbers or tax
identifiers, Impala can redact this sensitive information when displaying the statements in log files and
other administrative contexts. See <span class="xref">the documentation for your Apache Hadoop distribution</span> for details.
</p>
+
<p class="p">
<strong class="ph b">Cancellation:</strong> Cannot be cancelled.
</p>
+
<p class="p">
<strong class="ph b">HDFS permissions:</strong>
</p>
+
<p class="p">
The user ID that the <span class="keyword cmdname">impalad</span> daemon runs under,
@@ -239,15 +286,18 @@ EXPLAIN_LEVEL set to extended
if the source table is partitioned.)
</p>
+
<p class="p">
<strong class="ph b">Kudu considerations:</strong>
</p>
+
<p class="p">
The <code class="ph codeph">EXPLAIN</code> statement displays equivalent plan
information for queries against Kudu tables as for queries
against HDFS-based tables.
</p>
+
<p class="p">
To see which predicates Impala can <span class="q">"push down"</span> to Kudu for
efficient evaluation, without transmitting unnecessary rows back
@@ -262,6 +312,7 @@ EXPLAIN_LEVEL set to extended
immediately by Kudu and others are evaluated later by Impala:
</p>
+
<pre class="pre codeblock"><code>
EXPLAIN SELECT x,y from kudu_table WHERE
x = 1 AND y NOT IN (2,3) AND z = 1
@@ -282,9 +333,11 @@ EXPLAIN SELECT x,y from kudu_table WHERE
require any casting, can be pushed to Kudu.
</p>
+
<p class="p">
<strong class="ph b">Related information:</strong>
</p>
+
<p class="p">
<a class="xref" href="impala_select.html#select">SELECT Statement</a>,
<a class="xref" href="impala_insert.html#insert">INSERT Statement</a>,
@@ -292,5 +345,12 @@ EXPLAIN SELECT x,y from kudu_table WHERE
<a class="xref" href="impala_explain_plan.html#explain_plan">Understanding Impala Query Performance - EXPLAIN Plans and Query Profiles</a>
</p>
+
</div>
-<nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_langref_sql.html">Impala SQL Statements</a></div></div></nav></article></main></body></html>
\ No newline at end of file
+
+<div class="related-links">
+<div class="familylinks">
+<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_langref_sql.html">Impala SQL Statements</a></div>
+</div>
+</div></body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/impala/blob/b4ad38a9/docs/build/html/topics/impala_explain_level.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_explain_level.html b/docs/build/html/topics/impala_explain_level.html
index 8d592aa..cac58a2 100644
--- a/docs/build/html/topics/impala_explain_level.html
+++ b/docs/build/html/topics/impala_explain_level.html
@@ -1,8 +1,28 @@
+<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html
- SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="explain_level"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>EXPLAIN_LEVEL Query Option</title></head><body id="explain_level"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+ PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+
+<meta name="copyright" content="(C) Copyright 2018" />
+<meta name="DC.rights.owner" content="(C) Copyright 2018" />
+<meta name="DC.Type" content="concept" />
+<meta name="DC.Title" content="EXPLAIN_LEVEL Query Option" />
+<meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html" />
+<meta name="prodname" content="Impala" />
+<meta name="prodname" content="Impala" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="DC.Format" content="XHTML" />
+<meta name="DC.Identifier" content="explain_level" />
+<link rel="stylesheet" type="text/css" href="../commonltr.css" />
+<title>EXPLAIN_LEVEL Query Option</title>
+</head>
+<body id="explain_level">
+
<h1 class="title topictitle1" id="ariaid-title1">EXPLAIN_LEVEL Query Option</h1>
+
@@ -17,22 +37,27 @@
<code class="ph codeph">WHERE</code> clauses are implemented within a distributed query.
</p>
+
<p class="p">
<strong class="ph b">Type:</strong> <code class="ph codeph">STRING</code> or <code class="ph codeph">INT</code>
</p>
+
<p class="p">
<strong class="ph b">Default:</strong> <code class="ph codeph">1</code>
</p>
+
<p class="p">
<strong class="ph b">Arguments:</strong>
</p>
+
<p class="p">
The allowed range of numeric values for this option is 0 to 3:
</p>
+
<ul class="ul">
<li class="li">
<code class="ph codeph">0</code> or <code class="ph codeph">MINIMAL</code>: A barebones list, one line per operation. Primarily useful
@@ -40,26 +65,31 @@
long to read easily.
</li>
+
<li class="li">
<code class="ph codeph">1</code> or <code class="ph codeph">STANDARD</code>: The default level of detail, showing the logical way that
work is split up for the distributed query.
</li>
+
<li class="li">
<code class="ph codeph">2</code> or <code class="ph codeph">EXTENDED</code>: Includes additional detail about how the query planner
uses statistics in its decision-making process, to understand how a query could be tuned by gathering
statistics, using query hints, adding or removing predicates, and so on.
</li>
+
<li class="li">
<code class="ph codeph">3</code> or <code class="ph codeph">VERBOSE</code>: The maximum level of detail, showing how work is split up
within each node into <span class="q">"query fragments"</span> that are connected in a pipeline. This extra detail is
primarily useful for low-level performance testing and tuning within Impala itself, rather than for
rewriting the SQL code at the user level.
</li>
+
</ul>
- <div class="note note note_note"><span class="note__title notetitle">Note:</span>
+
+ <div class="note note"><span class="notetitle">Note:</span>
Prior to Impala 1.3, the allowed argument range for <code class="ph codeph">EXPLAIN_LEVEL</code> was 0 to 1: level 0 had
the mnemonic <code class="ph codeph">NORMAL</code>, and level 1 was <code class="ph codeph">VERBOSE</code>. In Impala 1.3 and higher,
<code class="ph codeph">NORMAL</code> is not a valid mnemonic value, and <code class="ph codeph">VERBOSE</code> still applies to the
@@ -67,6 +97,7 @@
older <code class="ph codeph">impala-shell</code> script files that set the <code class="ph codeph">EXPLAIN_LEVEL</code> query option.
</div>
+
<p class="p">
Changing the value of this option controls the amount of detail in the output of the <code class="ph codeph">EXPLAIN</code>
statement. The extended information from level 2 or 3 is especially useful during performance tuning, when
@@ -79,10 +110,12 @@
<a class="xref" href="impala_explain_plan.html#perf_explain">Using the EXPLAIN Plan for Performance Tuning</a> for details about how to use the extended information.
</p>
+
<p class="p">
<strong class="ph b">Usage notes:</strong>
</p>
+
<p class="p">
As always, read the <code class="ph codeph">EXPLAIN</code> output from bottom to top. The lowest lines represent the
initial work of the query (scanning data files), the lines in the middle represent calculations done on each
@@ -90,12 +123,14 @@
the final results being sent back to the coordinator node.
</p>
+
<p class="p">
The numbers in the left column are generated internally during the initial planning phase and do not
represent the actual order of operations, so it is not significant if they appear out of order in the
<code class="ph codeph">EXPLAIN</code> output.
</p>
+
<p class="p">
At all <code class="ph codeph">EXPLAIN</code> levels, the plan contains a warning if any tables in the query are missing
statistics. Use the <code class="ph codeph">COMPUTE STATS</code> statement to gather statistics for each table and suppress
@@ -103,6 +138,7 @@
query performance.
</p>
+
<p class="p">
The <code class="ph codeph">PROFILE</code> command in <span class="keyword cmdname">impala-shell</span> always starts with an explain plan
showing full detail, the same as with <code class="ph codeph">EXPLAIN_LEVEL=3</code>. <span class="ph">After the explain
@@ -110,15 +146,18 @@
<span class="keyword cmdname">impala-shell</span>.</span>
</p>
+
<p class="p">
<strong class="ph b">Examples:</strong>
</p>
+
<p class="p">
These examples use a trivial, empty table to illustrate how the essential aspects of query planning are shown
in <code class="ph codeph">EXPLAIN</code> output:
</p>
+
<pre class="pre codeblock"><code>[localhost:21000] > create table t1 (x int, s string);
[localhost:21000] > set explain_level=1;
[localhost:21000] > explain select count(*) from t1;
@@ -208,6 +247,7 @@
<code class="ph codeph">COMPUTE STATS</code> statement for the table:
</p>
+
<pre class="pre codeblock"><code>[localhost:21000] > compute stats t1;
+-----------------------------------------+
| summary |
@@ -244,6 +284,7 @@
join to a shuffle join.
</p>
+
<pre class="pre codeblock"><code>[localhost:21000] > set explain_level=1;
[localhost:21000] > explain select one.*, two.*, three.* from t1 one, t1 two, t1 three where one.x = two.x and two.x = three.x;
+---------------------------------------------------------+
@@ -315,6 +356,7 @@
final stage of join processing.
</p>
+
<pre class="pre codeblock"><code>[localhost:21000] > set explain_level=0;
[localhost:21000] > explain select one.*, two.*, three.*
> from t1 one join [shuffle] t1 two join t1 three
@@ -339,4 +381,10 @@
</div>
-<nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_query_options.html">Query Options for the SET Statement</a></div></div></nav></article></main></body></html>
\ No newline at end of file
+
+<div class="related-links">
+<div class="familylinks">
+<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_query_options.html">Query Options for the SET Statement</a></div>
+</div>
+</div></body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/impala/blob/b4ad38a9/docs/build/html/topics/impala_explain_plan.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_explain_plan.html b/docs/build/html/topics/impala_explain_plan.html
index 6ff87ab..511ff5b 100644
--- a/docs/build/html/topics/impala_explain_plan.html
+++ b/docs/build/html/topics/impala_explain_plan.html
@@ -1,8 +1,34 @@
+<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html
- SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_performance.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="explain_plan"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Understanding Impala Query Performance - EXPLAIN Plans and Query Profiles</title>
</head><body id="explain_plan"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+ PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+
+<meta name="copyright" content="(C) Copyright 2018" />
+<meta name="DC.rights.owner" content="(C) Copyright 2018" />
+<meta name="DC.Type" content="concept" />
+<meta name="DC.Title" content="Understanding Impala Query Performance - EXPLAIN Plans and Query Profiles" />
+<meta name="DC.Relation" scheme="URI" content="../topics/impala_performance.html" />
+<meta name="prodname" content="Impala" />
+<meta name="prodname" content="Impala" />
+<meta name="prodname" content="Impala" />
+<meta name="prodname" content="Impala" />
+<meta name="prodname" content="Impala" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="DC.Format" content="XHTML" />
+<meta name="DC.Identifier" content="explain_plan" />
+<link rel="stylesheet" type="text/css" href="../commonltr.css" />
+<title>Understanding Impala Query Performance - EXPLAIN Plans and Query Profiles</title>
+</head>
+<body id="explain_plan">
+
<h1 class="title topictitle1" id="ariaid-title1">Understanding Impala Query Performance - EXPLAIN Plans and Query Profiles</h1>
+
@@ -14,6 +40,7 @@
actually running the query itself.
</p>
+
<p class="p">
For an overview of the physical performance characteristics for a query, issue the <code class="ph codeph">SUMMARY</code>
statement in <span class="keyword cmdname">impala-shell</span> immediately after executing a query. This condensed information
@@ -21,6 +48,7 @@
at each phase compare to the actual values.
</p>
+
<p class="p">
To understand the detailed performance characteristics for a query, issue the <code class="ph codeph">PROFILE</code>
statement in <span class="keyword cmdname">impala-shell</span> immediately after executing a query. This low-level information
@@ -28,8 +56,10 @@
query is actually run.
</p>
+
<p class="p toc inpage"></p>
+
<p class="p">
Also, see <a class="xref" href="impala_hbase.html#hbase_performance">Performance Considerations for the Impala-HBase Integration</a>
and <a class="xref" href="impala_s3.html#s3_performance">Understanding and Tuning Impala Query Performance for S3 Data</a>
@@ -37,12 +67,19 @@
<code class="ph codeph">EXPLAIN</code> plans for queries against HBase tables
<span class="ph">and data stored in the Amazon Simple Storage System (S3)</span>.
</p>
+
</div>
- <nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_performance.html">Tuning Impala for Performance</a></div></div></nav><article class="topic concept nested1" aria-labelledby="ariaid-title2" id="explain_plan__perf_explain">
+
+ <div class="related-links">
+<div class="familylinks">
+<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_performance.html">Tuning Impala for Performance</a></div>
+</div>
+</div><div class="topic concept nested1" aria-labelledby="ariaid-title2" id="perf_explain">
<h2 class="title topictitle2" id="ariaid-title2">Using the EXPLAIN Plan for Performance Tuning</h2>
+
<div class="body conbody">
<p class="p">
@@ -55,6 +92,7 @@
+
<pre class="pre codeblock"><code>[impalad-host:21000] > explain select count(*) from customer_address;
+----------------------------------------------------------+
| Explain String |
@@ -83,31 +121,38 @@
take to scan a table based on total data size and the size of the cluster.
</li>
+
<li class="li">
As you work your way up, next you see the operations that will be parallelized and performed on each
Impala node.
</li>
+
<li class="li">
At the higher levels, you see how data flows when intermediate result sets are combined and transmitted
from one node to another.
</li>
+
<li class="li">
See <a class="xref" href="../shared/../topics/impala_explain_level.html#explain_level">EXPLAIN_LEVEL Query Option</a> for details about the
<code class="ph codeph">EXPLAIN_LEVEL</code> query option, which lets you customize how much detail to show in the
<code class="ph codeph">EXPLAIN</code> plan depending on whether you are doing high-level or low-level tuning,
dealing with logical or physical aspects of the query.
</li>
+
</ul>
+
</div>
+
<p class="p">
The <code class="ph codeph">EXPLAIN</code> plan is also printed at the beginning of the query profile report described in
<a class="xref" href="#perf_profile">Using the Query Profile for Performance Tuning</a>, for convenience in examining both the logical and physical aspects of the
query side-by-side.
</p>
+
<p class="p">
The amount of detail displayed in the <code class="ph codeph">EXPLAIN</code> output is controlled by the
<a class="xref" href="impala_explain_level.html#explain_level">EXPLAIN_LEVEL</a> query option. You typically
@@ -116,14 +161,18 @@
tuning, or when estimating query resource usage in conjunction with the resource management features.
</p>
+
</div>
- </article>
- <article class="topic concept nested1" aria-labelledby="ariaid-title3" id="explain_plan__perf_summary">
+ </div>
+
+
+ <div class="topic concept nested1" aria-labelledby="ariaid-title3" id="perf_summary">
<h2 class="title topictitle2" id="ariaid-title3">Using the SUMMARY Report for Performance Tuning</h2>
+
<div class="body conbody">
<p class="p">
@@ -134,12 +183,14 @@
query is run and so displays actual timing numbers.
</p>
+
<p class="p">
The <code class="ph codeph">SUMMARY</code> report is also printed at the beginning of the query profile report described
in <a class="xref" href="#perf_profile">Using the Query Profile for Performance Tuning</a>, for convenience in examining high-level and low-level aspects of the query
side-by-side.
</p>
+
<p class="p">
For example, here is a query involving an aggregate function, on a single-node VM. The different stages of
the query and their timings are shown (rolled up for all nodes), along with estimated and actual values
@@ -151,6 +202,7 @@
for the largest numbers.)
</p>
+
<pre class="pre codeblock"><code>[localhost:21000] > select avg(ss_sales_price) from store_sales where ss_coupon_amt = 0;
+---------------------+
| avg(ss_sales_price) |
@@ -173,11 +225,13 @@
smaller intermediate results are measured in milliseconds (ms) or even nanoseconds (ns).
</p>
+
<p class="p">
Here is an example from a more complicated query, as it would appear in the <code class="ph codeph">PROFILE</code>
output:
</p>
+
<pre class="pre codeblock"><code>Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail
------------------------------------------------------------------------------------------------------------------------
09:MERGING-EXCHANGE 1 79.738us 79.738us 5 5 0 -1.00 B UNPARTITIONED
@@ -192,12 +246,15 @@
00:SCAN HDFS 3 8s032ms 8s558ms 3.79M 600.12K 32.29 MB 264.00 MB tpch.lineitem l
</code></pre>
</div>
- </article>
- <article class="topic concept nested1" aria-labelledby="ariaid-title4" id="explain_plan__perf_profile">
+ </div>
+
+
+ <div class="topic concept nested1" aria-labelledby="ariaid-title4" id="perf_profile">
<h2 class="title topictitle2" id="ariaid-title4">Using the Query Profile for Performance Tuning</h2>
+
<div class="body conbody">
<p class="p">
@@ -211,6 +268,7 @@
reads are in effect.
</p>
+
<p class="p">
By default, time values in the profile output reflect the wall-clock time taken by an operation.
For values denoting system time or user time, the measurement unit is reflected in the metric
@@ -222,6 +280,7 @@
is actively using CPU cycles.
</p>
+
<p class="p">
The <a class="xref" href="impala_explain_plan.html#perf_explain"><code class="ph codeph">EXPLAIN</code> plan</a> is also printed
at the beginning of the query profile report, for convenience in examining both the logical and physical
@@ -230,6 +289,7 @@
verbosity of the <code class="ph codeph">EXPLAIN</code> output printed by the <code class="ph codeph">PROFILE</code> command.
</p>
+
<p class="p">
@@ -237,6 +297,7 @@
pseudo-distributed cluster to keep the output relatively brief.
</p>
+
<pre class="pre codeblock"><code>[localhost:21000] > profile;
Query Runtime Profile:
Query (id=6540a03d4bee0691:4963d6269b210ebd):
@@ -588,5 +649,8 @@ PLAN FRAGMENT 2
- TotalRawHdfsReadTime(*): 7.199ms
- TotalReadThroughput: 0.00 /sec</code></pre>
</div>
- </article>
-</article></main></body></html>
\ No newline at end of file
+
+ </div>
+
+</body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/impala/blob/b4ad38a9/docs/build/html/topics/impala_faq.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_faq.html b/docs/build/html/topics/impala_faq.html
index 117680e..bca4812 100644
--- a/docs/build/html/topics/impala_faq.html
+++ b/docs/build/html/topics/impala_faq.html
@@ -1,8 +1,27 @@
+<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html
- SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="faq"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Impala Frequently Asked Questions</title></head><body id="faq"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+ PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+
+<meta name="copyright" content="(C) Copyright 2018" />
+<meta name="DC.rights.owner" content="(C) Copyright 2018" />
+<meta name="DC.Type" content="concept" />
+<meta name="DC.Title" content="Impala Frequently Asked Questions" />
+<meta name="prodname" content="Impala" />
+<meta name="prodname" content="Impala" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="DC.Format" content="XHTML" />
+<meta name="DC.Identifier" content="faq" />
+<link rel="stylesheet" type="text/css" href="../commonltr.css" />
+<title>Impala Frequently Asked Questions</title>
+</head>
+<body id="faq">
+
<h1 class="title topictitle1" id="ariaid-title1">Impala Frequently Asked Questions</h1>
+
<div class="body conbody">
@@ -12,10 +31,14 @@
the interactive SQL engine for Hadoop.
</p>
+
<p class="p">
This section is under construction.
</p>
+
</div>
-</article></main></body></html>
\ No newline at end of file
+
+</body>
+</html>
\ No newline at end of file