You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by mi...@apache.org on 2018/05/01 21:30:14 UTC
[14/15] impala git commit: [DOCS] Add Impala docs from branch 2.x
http://git-wip-us.apache.org/repos/asf/impala/blob/592fd85d/docs/build/html/topics/impala_analytic_functions.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_analytic_functions.html b/docs/build/html/topics/impala_analytic_functions.html
index 1662afb..e798427 100644
--- a/docs/build/html/topics/impala_analytic_functions.html
+++ b/docs/build/html/topics/impala_analytic_functions.html
@@ -1,6 +1,6 @@
<!DOCTYPE html
SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_functions.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Im
pala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="an
alytic_functions"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Impala Analytic Functions</title></head><body id="analytic_functions"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_functions.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Im
pala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="an
alytic_functions"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Impala Analytic Functions</title></head><body id="analytic_functions"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
<h1 class="title topictitle1" id="ariaid-title1">Impala Analytic Functions</h1>
@@ -126,7 +126,7 @@ window_clause: See <a class="xref" href="#window_clause">Window Clause</a>
<p class="p">
In queries involving both analytic functions and partitioned tables, partition pruning only occurs for columns named in the <code class="ph codeph">PARTITION BY</code>
clause of the analytic function call. For example, if an analytic function query has a clause such as <code class="ph codeph">WHERE year=2016</code>,
- the way to make the query prune all other <code class="ph codeph">YEAR</code> partitions is to include <code class="ph codeph">PARTITION BY year</code>in the analytic function call;
+ the way to make the query prune all other <code class="ph codeph">YEAR</code> partitions is to include <code class="ph codeph">PARTITION BY year</code> in the analytic function call;
for example, <code class="ph codeph">OVER (PARTITION BY year,<var class="keyword varname">other_columns</var> <var class="keyword varname">other_analytic_clauses</var>)</code>.
</p>
@@ -749,7 +749,7 @@ order by kind, ordering desc, name;
<p class="p">
Partitioning by the <code class="ph codeph">X</code> column groups all the duplicate numbers together and returns the
- place each each value within the group; because each value occurs only 1 or 2 times,
+ place each value within the group; because each value occurs only 1 or 2 times,
<code class="ph codeph">DENSE_RANK()</code> designates each <code class="ph codeph">X</code> value as either first or second within its
group.
</p>
@@ -1585,7 +1585,7 @@ insert into animals values ('Fire-breathing dragon', 'Mythical', NULL);
<p class="p">
Partitioning by the <code class="ph codeph">X</code> column groups all the duplicate numbers together and returns the
- place each each value within the group; because each value occurs only 1 or 2 times,
+ place each value within the group; because each value occurs only 1 or 2 times,
<code class="ph codeph">RANK()</code> designates each <code class="ph codeph">X</code> value as either first or second within its
group.
</p>
http://git-wip-us.apache.org/repos/asf/impala/blob/592fd85d/docs/build/html/topics/impala_appx_count_distinct.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_appx_count_distinct.html b/docs/build/html/topics/impala_appx_count_distinct.html
index 3b9e011..4a36093 100644
--- a/docs/build/html/topics/impala_appx_count_distinct.html
+++ b/docs/build/html/topics/impala_appx_count_distinct.html
@@ -1,6 +1,6 @@
<!DOCTYPE html
SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="appx_count_distinct"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>APPX_COUNT_DISTINCT Query Option (Impala 2.0 or higher only)</title></head><body id="appx_count_distinct"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="appx_count_distinct"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>APPX_COUNT_DISTINCT Query Option (Impala 2.0 or higher only)</title></head><body id="appx_count_distinct"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
<h1 class="title topictitle1" id="ariaid-title1">APPX_COUNT_DISTINCT Query Option (<span class="keyword">Impala 2.0</span> or higher only)</h1>
http://git-wip-us.apache.org/repos/asf/impala/blob/592fd85d/docs/build/html/topics/impala_appx_median.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_appx_median.html b/docs/build/html/topics/impala_appx_median.html
index 7cc84a2..a838341 100644
--- a/docs/build/html/topics/impala_appx_median.html
+++ b/docs/build/html/topics/impala_appx_median.html
@@ -1,6 +1,6 @@
<!DOCTYPE html
SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_aggregate_functions.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="appx_median"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>APPX_MEDIAN Function</title></head><body id="appx_median"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_aggregate_functions.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="appx_median"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>APPX_MEDIAN Function</title></head><body id="appx_median"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
<h1 class="title topictitle1" id="ariaid-title1">APPX_MEDIAN Function</h1>
http://git-wip-us.apache.org/repos/asf/impala/blob/592fd85d/docs/build/html/topics/impala_array.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_array.html b/docs/build/html/topics/impala_array.html
index 3ccc018..a20ffaa 100644
--- a/docs/build/html/topics/impala_array.html
+++ b/docs/build/html/topics/impala_array.html
@@ -1,6 +1,6 @@
<!DOCTYPE html
SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_datatypes.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="array"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>ARRAY Complex Type (Impala 2.3 or higher only)</title></head><body id="array"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_datatypes.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="array"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>ARRAY Complex Type (Impala 2.3 or higher only)</title></head><body id="array"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
<h1 class="title topictitle1" id="ariaid-title1">ARRAY Complex Type (<span class="keyword">Impala 2.3</span> or higher only)</h1>
@@ -118,7 +118,7 @@ type ::= <var class="keyword varname">primitive_type</var> | <var class="keyword
</p>
</li>
<li class="li">
- <p class="p" id="array__d6e3207">
+ <p class="p" id="array__d6e3246">
The maximum length of the column definition for any complex type, including declarations for any nested types,
is 4000 characters.
</p>
@@ -135,7 +135,7 @@ type ::= <var class="keyword varname">primitive_type</var> | <var class="keyword
<strong class="ph b">Kudu considerations:</strong>
</p>
<p class="p">
- Currently, the data types <code class="ph codeph">DECIMAL</code>, <code class="ph codeph">CHAR</code>, <code class="ph codeph">VARCHAR</code>,
+ Currently, the data types <code class="ph codeph">CHAR</code>, <code class="ph codeph">VARCHAR</code>,
<code class="ph codeph">ARRAY</code>, <code class="ph codeph">MAP</code>, and <code class="ph codeph">STRUCT</code> cannot be used with Kudu tables.
</p>
http://git-wip-us.apache.org/repos/asf/impala/blob/592fd85d/docs/build/html/topics/impala_auditing.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_auditing.html b/docs/build/html/topics/impala_auditing.html
index 126f04e..b2af1e2 100644
--- a/docs/build/html/topics/impala_auditing.html
+++ b/docs/build/html/topics/impala_auditing.html
@@ -1,6 +1,6 @@
<!DOCTYPE html
SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_security.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="auditing"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Auditing Impala Operations</title></head><body id="auditing"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_security.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="auditing"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Auditing Impala Operations</title></head><body id="auditing"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
<h1 class="title topictitle1" id="ariaid-title1">Auditing Impala Operations</h1>
http://git-wip-us.apache.org/repos/asf/impala/blob/592fd85d/docs/build/html/topics/impala_authentication.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_authentication.html b/docs/build/html/topics/impala_authentication.html
index 3a334ec..3839b4a 100644
--- a/docs/build/html/topics/impala_authentication.html
+++ b/docs/build/html/topics/impala_authentication.html
@@ -1,6 +1,6 @@
<!DOCTYPE html
SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_security.html"><meta name="DC.Relation" scheme="URI" content="../topics/impala_kerberos.html"><meta name="DC.Relation" scheme="URI" content="../topics/impala_ldap.html"><meta name="DC.Relation" scheme="URI" content="../topics/impala_mixed_security.html"><meta name="DC.Relation" scheme="URI" content="../topics/impala_delegation.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="authentication"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Impala Auth
entication</title></head><body id="authentication"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_security.html"><meta name="DC.Relation" scheme="URI" content="../topics/impala_kerberos.html"><meta name="DC.Relation" scheme="URI" content="../topics/impala_ldap.html"><meta name="DC.Relation" scheme="URI" content="../topics/impala_mixed_security.html"><meta name="DC.Relation" scheme="URI" content="../topics/impala_delegation.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="authentication"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Impala Auth
entication</title></head><body id="authentication"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
<h1 class="title topictitle1" id="ariaid-title1">Impala Authentication</h1>
http://git-wip-us.apache.org/repos/asf/impala/blob/592fd85d/docs/build/html/topics/impala_authorization.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_authorization.html b/docs/build/html/topics/impala_authorization.html
index e87f9c4..bbc3548 100644
--- a/docs/build/html/topics/impala_authorization.html
+++ b/docs/build/html/topics/impala_authorization.html
@@ -1,6 +1,6 @@
<!DOCTYPE html
SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_security.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="authorization"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Enabling Sentry Authorization for Impala</title></head><body id="authorization"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_security.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="authorization"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Enabling Sentry Authorization for Impala</title></head><body id="authorization"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
<h1 class="title topictitle1" id="ariaid-title1">Enabling Sentry Authorization for Impala</h1>
http://git-wip-us.apache.org/repos/asf/impala/blob/592fd85d/docs/build/html/topics/impala_avg.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_avg.html b/docs/build/html/topics/impala_avg.html
index 1f8cfbe..d860a02 100644
--- a/docs/build/html/topics/impala_avg.html
+++ b/docs/build/html/topics/impala_avg.html
@@ -1,6 +1,6 @@
<!DOCTYPE html
SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_aggregate_functions.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="avg"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>AVG Function</title></head><body id="avg"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_aggregate_functions.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="avg"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>AVG Function</title></head><body id="avg"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
<h1 class="title topictitle1" id="ariaid-title1">AVG Function</h1>
http://git-wip-us.apache.org/repos/asf/impala/blob/592fd85d/docs/build/html/topics/impala_avro.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_avro.html b/docs/build/html/topics/impala_avro.html
index 42cf762..cf0f4e2 100644
--- a/docs/build/html/topics/impala_avro.html
+++ b/docs/build/html/topics/impala_avro.html
@@ -1,6 +1,6 @@
<!DOCTYPE html
SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_file_formats.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta
name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="avro"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Using the Avro File Format with Impala Tables</title></head><body id="avro"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_file_formats.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta
name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="avro"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Using the Avro File Format with Impala Tables</title></head><body id="avro"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
<h1 class="title topictitle1" id="ariaid-title1">Using the Avro File Format with Impala Tables</h1>
http://git-wip-us.apache.org/repos/asf/impala/blob/592fd85d/docs/build/html/topics/impala_batch_size.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_batch_size.html b/docs/build/html/topics/impala_batch_size.html
index ee7f80e..41d60be 100644
--- a/docs/build/html/topics/impala_batch_size.html
+++ b/docs/build/html/topics/impala_batch_size.html
@@ -1,6 +1,6 @@
<!DOCTYPE html
SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="batch_size"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>BATCH_SIZE Query Option</title></head><body id="batch_size"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="batch_size"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>BATCH_SIZE Query Option</title></head><body id="batch_size"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
<h1 class="title topictitle1" id="ariaid-title1">BATCH_SIZE Query Option</h1>
http://git-wip-us.apache.org/repos/asf/impala/blob/592fd85d/docs/build/html/topics/impala_bigint.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_bigint.html b/docs/build/html/topics/impala_bigint.html
index 0b70a9d..eea1af7 100644
--- a/docs/build/html/topics/impala_bigint.html
+++ b/docs/build/html/topics/impala_bigint.html
@@ -1,6 +1,6 @@
<!DOCTYPE html
SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_datatypes.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="bigint"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>BIGINT Data Type</title></head><body id="bigint"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_datatypes.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="bigint"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>BIGINT Data Type</title></head><body id="bigint"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
<h1 class="title topictitle1" id="ariaid-title1">BIGINT Data Type</h1>
@@ -31,11 +31,13 @@
<strong class="ph b">Conversions:</strong> Impala automatically converts to a floating-point type (<code class="ph codeph">FLOAT</code> or
<code class="ph codeph">DOUBLE</code>) automatically. Use <code class="ph codeph">CAST()</code> to convert to <code class="ph codeph">TINYINT</code>,
<code class="ph codeph">SMALLINT</code>, <code class="ph codeph">INT</code>, <code class="ph codeph">STRING</code>, or <code class="ph codeph">TIMESTAMP</code>.
- <span class="ph">Casting an integer or floating-point value <code class="ph codeph">N</code> to
+ <span class="ph">
+ Casting an integer or floating-point value <code class="ph codeph">N</code> to
<code class="ph codeph">TIMESTAMP</code> produces a value that is <code class="ph codeph">N</code> seconds past the start of the epoch
date (January 1, 1970). By default, the result value represents a date and time in the UTC time zone.
- If the setting <code class="ph codeph">-use_local_tz_for_unix_timestamp_conversions=true</code> is in effect,
- the resulting <code class="ph codeph">TIMESTAMP</code> represents a date and time in the local time zone.</span>
+ If the setting <code class="ph codeph">--use_local_tz_for_unix_timestamp_conversions=true</code> is in effect,
+ the resulting <code class="ph codeph">TIMESTAMP</code> represents a date and time in the local time zone.
+ </span>
</p>
<p class="p">
http://git-wip-us.apache.org/repos/asf/impala/blob/592fd85d/docs/build/html/topics/impala_bit_functions.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_bit_functions.html b/docs/build/html/topics/impala_bit_functions.html
index 5d36c3d..6aa878a 100644
--- a/docs/build/html/topics/impala_bit_functions.html
+++ b/docs/build/html/topics/impala_bit_functions.html
@@ -1,6 +1,6 @@
<!DOCTYPE html
SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_functions.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="bit_functions"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Impala Bit Functions</title></head><body id="bit_functions"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_functions.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="bit_functions"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Impala Bit Functions</title></head><body id="bit_functions"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
<h1 class="title topictitle1" id="ariaid-title1">Impala Bit Functions</h1>
http://git-wip-us.apache.org/repos/asf/impala/blob/592fd85d/docs/build/html/topics/impala_boolean.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_boolean.html b/docs/build/html/topics/impala_boolean.html
index 3c74ef5..d702a16 100644
--- a/docs/build/html/topics/impala_boolean.html
+++ b/docs/build/html/topics/impala_boolean.html
@@ -1,6 +1,6 @@
<!DOCTYPE html
SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_datatypes.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="boolean"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>BOOLEAN Data Type</title></head><body id="boolean"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_datatypes.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="boolean"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>BOOLEAN Data Type</title></head><body id="boolean"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
<h1 class="title topictitle1" id="ariaid-title1">BOOLEAN Data Type</h1>
http://git-wip-us.apache.org/repos/asf/impala/blob/592fd85d/docs/build/html/topics/impala_breakpad.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_breakpad.html b/docs/build/html/topics/impala_breakpad.html
index 1653563..e1e6526 100644
--- a/docs/build/html/topics/impala_breakpad.html
+++ b/docs/build/html/topics/impala_breakpad.html
@@ -1,6 +1,6 @@
<!DOCTYPE html
SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_troubleshooting.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="breakpad"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Breakpad Minidumps for Impala (Impala 2.6 or higher only)</title></head><body id="breakpad"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_troubleshooting.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="breakpad"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Breakpad Minidumps for Impala (Impala 2.6 or higher only)</title></head><body id="breakpad"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
<h1 class="title topictitle1" id="ariaid-title1">Breakpad Minidumps for Impala (<span class="keyword">Impala 2.6</span> or higher only)</h1>
http://git-wip-us.apache.org/repos/asf/impala/blob/592fd85d/docs/build/html/topics/impala_buffer_pool_limit.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_buffer_pool_limit.html b/docs/build/html/topics/impala_buffer_pool_limit.html
index 44f4808..5baa799 100644
--- a/docs/build/html/topics/impala_buffer_pool_limit.html
+++ b/docs/build/html/topics/impala_buffer_pool_limit.html
@@ -1,6 +1,6 @@
<!DOCTYPE html
SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="buffer_pool_limit"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>BUFFER_POOL_LIMIT Query Option</title></head><body id="buffer_pool_limit"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="buffer_pool_limit"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>BUFFER_POOL_LIMIT Query Option</title></head><body id="buffer_pool_limit"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
<h1 class="title topictitle1" id="ariaid-title1">BUFFER_POOL_LIMIT Query Option</h1>
http://git-wip-us.apache.org/repos/asf/impala/blob/592fd85d/docs/build/html/topics/impala_char.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_char.html b/docs/build/html/topics/impala_char.html
index 281a07c..f8b3a6e 100644
--- a/docs/build/html/topics/impala_char.html
+++ b/docs/build/html/topics/impala_char.html
@@ -1,6 +1,6 @@
<!DOCTYPE html
SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_datatypes.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="char"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>CHAR Data Type (Impala 2.0 or higher only)</title></head><body id="char"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_datatypes.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="char"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>CHAR Data Type (Impala 2.0 or higher only)</title></head><body id="char"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
<h1 class="title topictitle1" id="ariaid-title1">CHAR Data Type (<span class="keyword">Impala 2.0</span> or higher only)</h1>
@@ -240,7 +240,7 @@ select concat('[',a,']') as a, concat('[',b,']') as b, concat('[',c,']') as c fr
<strong class="ph b">Kudu considerations:</strong>
</p>
<p class="p">
- Currently, the data types <code class="ph codeph">DECIMAL</code>, <code class="ph codeph">CHAR</code>, <code class="ph codeph">VARCHAR</code>,
+ Currently, the data types <code class="ph codeph">CHAR</code>, <code class="ph codeph">VARCHAR</code>,
<code class="ph codeph">ARRAY</code>, <code class="ph codeph">MAP</code>, and <code class="ph codeph">STRUCT</code> cannot be used with Kudu tables.
</p>
http://git-wip-us.apache.org/repos/asf/impala/blob/592fd85d/docs/build/html/topics/impala_cluster_sizing.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_cluster_sizing.html b/docs/build/html/topics/impala_cluster_sizing.html
deleted file mode 100644
index 1bd5c97..0000000
--- a/docs/build/html/topics/impala_cluster_sizing.html
+++ /dev/null
@@ -1,318 +0,0 @@
-<!DOCTYPE html
- SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_planning.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="cluster_sizing"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Cluster Sizing Guidelines for Impala</title></head><body id="cluster_sizing"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
-
- <h1 class="title topictitle1" id="ariaid-title1">Cluster Sizing Guidelines for Impala</h1>
-
-
-
- <div class="body conbody">
-
- <p class="p">
-
- This document provides a very rough guideline to estimate the size of a cluster needed for a specific
- customer application. You can use this information when planning how much and what type of hardware to
- acquire for a new cluster, or when adding Impala workloads to an existing cluster.
- </p>
-
- <div class="note note note_note"><span class="note__title notetitle">Note:</span>
- Before making purchase or deployment decisions, consult organizations with relevant experience
- to verify the conclusions about hardware requirements based on your data volume and workload.
- </div>
-
-
-
- <p class="p">
- Always use hosts with identical specifications and capacities for all the nodes in the cluster. Currently,
- Impala divides the work evenly between cluster nodes, regardless of their exact hardware configuration.
- Because work can be distributed in different ways for different queries, if some hosts are overloaded
- compared to others in terms of CPU, memory, I/O, or network, you might experience inconsistent performance
- and overall slowness
- </p>
-
- <p class="p">
- For analytic workloads with star/snowflake schemas, and using consistent hardware for all nodes (64 GB RAM,
- 12 2 TB hard drives, 2x E5-2630L 12 cores total, 10 GB network), the following table estimates the number of
- DataNodes needed in the cluster based on data size and the number of concurrent queries, for workloads
- similar to TPC-DS benchmark queries:
- </p>
-
- <table class="table"><caption><span class="table--title-label">Table 1. </span><span class="title">Cluster size estimation based on the number of concurrent queries and data size with a 20 second average query response time</span></caption><colgroup><col><col><col><col><col><col></colgroup><thead class="thead">
- <tr class="row">
- <th class="entry nocellnorowborder" id="cluster_sizing__entry__1">
- Data Size
- </th>
- <th class="entry nocellnorowborder" id="cluster_sizing__entry__2">
- 1 query
- </th>
- <th class="entry nocellnorowborder" id="cluster_sizing__entry__3">
- 10 queries
- </th>
- <th class="entry nocellnorowborder" id="cluster_sizing__entry__4">
- 100 queries
- </th>
- <th class="entry nocellnorowborder" id="cluster_sizing__entry__5">
- 1000 queries
- </th>
- <th class="entry nocellnorowborder" id="cluster_sizing__entry__6">
- 2000 queries
- </th>
- </tr>
- </thead><tbody class="tbody">
- <tr class="row">
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__1 ">
- <strong class="ph b">250 GB</strong>
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__2 ">
- 2
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__3 ">
- 2
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__4 ">
- 5
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__5 ">
- 35
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__6 ">
- 70
- </td>
- </tr>
- <tr class="row">
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__1 ">
- <strong class="ph b">500 GB</strong>
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__2 ">
- 2
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__3 ">
- 2
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__4 ">
- 10
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__5 ">
- 70
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__6 ">
- 135
- </td>
- </tr>
- <tr class="row">
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__1 ">
- <strong class="ph b">1 TB</strong>
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__2 ">
- 2
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__3 ">
- 2
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__4 ">
- 15
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__5 ">
- 135
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__6 ">
- 270
- </td>
- </tr>
- <tr class="row">
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__1 ">
- <strong class="ph b">15 TB</strong>
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__2 ">
- 2
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__3 ">
- 20
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__4 ">
- 200
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__5 ">
- N/A
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__6 ">
- N/A
- </td>
- </tr>
- <tr class="row">
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__1 ">
- <strong class="ph b">30 TB</strong>
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__2 ">
- 4
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__3 ">
- 40
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__4 ">
- 400
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__5 ">
- N/A
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__6 ">
- N/A
- </td>
- </tr>
- <tr class="row">
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__1 ">
- <strong class="ph b">60 TB</strong>
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__2 ">
- 8
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__3 ">
- 80
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__4 ">
- 800
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__5 ">
- N/A
- </td>
- <td class="entry nocellnorowborder" headers="cluster_sizing__entry__6 ">
- N/A
- </td>
- </tr>
- </tbody></table>
-
- <section class="section" id="cluster_sizing__sizing_factors"><h2 class="title sectiontitle">Factors Affecting Scalability</h2>
-
-
-
- <p class="p">
- A typical analytic workload (TPC-DS style queries) using recommended hardware is usually CPU-bound. Each
- node can process roughly 1.6 GB/sec. Both CPU-bound and disk-bound workloads can scale almost linearly with
- cluster size. However, for some workloads, the scalability might be bounded by the network, or even by
- memory.
- </p>
-
- <p class="p">
- If the workload is already network bound (on a 10 GB network), increasing the cluster size won’t reduce
- the network load; in fact, a larger cluster could increase network traffic because some queries involve
- <span class="q">"broadcast"</span> operations to all DataNodes. Therefore, boosting the cluster size does not improve query
- throughput in a network-constrained environment.
- </p>
-
- <p class="p">
- Let’s look at a memory-bound workload. A workload is memory-bound if Impala cannot run any additional
- concurrent queries because all memory allocated has already been consumed, but neither CPU, disk, nor
- network is saturated yet. This can happen because currently Impala uses only a single core per node to
- process join and aggregation queries. For a node with 128 GB of RAM, if a join node takes 50 GB, the system
- cannot run more than 2 such queries at the same time.
- </p>
-
- <p class="p">
- Therefore, at most 2 cores are used. Throughput can still scale almost linearly even for a memory-bound
- workload. It’s just that the CPU will not be saturated. Per-node throughput will be lower than 1.6
- GB/sec. Consider increasing the memory per node.
- </p>
-
- <p class="p">
- As long as the workload is not network- or memory-bound, we can use the 1.6 GB/second per node as the
- throughput estimate.
- </p>
- </section>
-
- <section class="section" id="cluster_sizing__sizing_details"><h2 class="title sectiontitle">A More Precise Approach</h2>
-
-
-
- <p class="p">
- A more precise sizing estimate would require not only queries per minute (QPM), but also an average data
- size scanned per query (D). With the proper partitioning strategy, D is usually a fraction of the total
- data size. The following equation can be used as a rough guide to estimate the number of nodes (N) needed:
- </p>
-
-<pre class="pre codeblock"><code>Eq 1: N > QPM * D / 100 GB
-</code></pre>
-
- <p class="p">
- Here is an example. Suppose, on average, a query scans 50 GB of data and the average response time is
- required to be 15 seconds or less when there are 100 concurrent queries. The QPM is 100/15*60 = 400. We can
- estimate the number of node using our equation above.
- </p>
-
-<pre class="pre codeblock"><code>N > QPM * D / 100GB
-N > 400 * 50GB / 100GB
-N > 200
-</code></pre>
-
- <p class="p">
- Because this figure is a rough estimate, the corresponding number of nodes could be between 100 and 500.
- </p>
-
- <p class="p">
- Depending on the complexity of the query, the processing rate of query might change. If the query has more
- joins, aggregation functions, or CPU-intensive functions such as string processing or complex UDFs, the
- process rate will be lower than 1.6 GB/second per node. On the other hand, if the query only does scan and
- filtering on numbers, the processing rate can be higher.
- </p>
- </section>
-
- <section class="section" id="cluster_sizing__sizing_mem_estimate"><h2 class="title sectiontitle">Estimating Memory Requirements</h2>
-
-
-
-
- <p class="p">
- Impala can handle joins between multiple large tables. Make sure that statistics are collected for all the
- joined tables, using the <code class="ph codeph"><a class="xref" href="impala_compute_stats.html#compute_stats">COMPUTE
- STATS</a></code> statement. However, joining big tables does consume more memory. Follow the steps
- below to calculate the minimum memory requirement.
- </p>
-
- <p class="p">
- Suppose you are running the following join:
- </p>
-
-<pre class="pre codeblock"><code>select a.*, b.col_1, b.col_2, … b.col_n
-from a, b
-where a.key = b.key
-and b.col_1 in (1,2,4...)
-and b.col_4 in (....);
-</code></pre>
-
- <p class="p">
- And suppose table <code class="ph codeph">B</code> is smaller than table <code class="ph codeph">A</code> (but still a large table).
- </p>
-
- <p class="p">
- The memory requirement for the query is the right-hand table (<code class="ph codeph">B</code>), after decompression,
- filtering (<code class="ph codeph">b.col_n in ...</code>) and after projection (only using certain columns) must be less
- than the total memory of the entire cluster.
- </p>
-
-<pre class="pre codeblock"><code>Cluster Total Memory Requirement = Size of the smaller table *
- selectivity factor from the predicate *
- projection factor * compression ratio
-</code></pre>
-
- <p class="p">
- In this case, assume that table <code class="ph codeph">B</code> is 100 TB in Parquet format with 200 columns. The
- predicate on <code class="ph codeph">B</code> (<code class="ph codeph">b.col_1 in ...and b.col_4 in ...</code>) will select only 10% of
- the rows from <code class="ph codeph">B</code> and for projection, we are only projecting 5 columns out of 200 columns.
- Usually, Snappy compression gives us 3 times compression, so we estimate a 3x compression factor.
- </p>
-
-<pre class="pre codeblock"><code>Cluster Total Memory Requirement = Size of the smaller table *
- selectivity factor from the predicate *
- projection factor * compression ratio
- = 100TB * 10% * 5/200 * 3
- = 0.75TB
- = 750GB
-</code></pre>
-
- <p class="p">
- So, if you have a 10-node cluster, each node has 128 GB of RAM and you give 80% to Impala, then you have 1
- TB of usable memory for Impala, which is more than 750GB. Therefore, your cluster can handle join queries
- of this magnitude.
- </p>
- </section>
- </div>
-<nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_planning.html">Planning for Impala Deployment</a></div></div></nav></article></main></body></html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/impala/blob/592fd85d/docs/build/html/topics/impala_comments.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_comments.html b/docs/build/html/topics/impala_comments.html
index 547f292..1db7a5f 100644
--- a/docs/build/html/topics/impala_comments.html
+++ b/docs/build/html/topics/impala_comments.html
@@ -1,6 +1,6 @@
<!DOCTYPE html
SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_langref.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="comments"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Comments</title></head><body id="comments"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_langref.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="comments"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Comments</title></head><body id="comments"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
<h1 class="title topictitle1" id="ariaid-title1">Comments</h1>
http://git-wip-us.apache.org/repos/asf/impala/blob/592fd85d/docs/build/html/topics/impala_complex_types.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_complex_types.html b/docs/build/html/topics/impala_complex_types.html
index ebaaa88..1920363 100644
--- a/docs/build/html/topics/impala_complex_types.html
+++ b/docs/build/html/topics/impala_complex_types.html
@@ -1,6 +1,6 @@
<!DOCTYPE html
SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_datatypes.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="complex_types"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Complex Types (Impala 2.3 or higher only)</title></head><body id="complex_types"><main role="main"><article role="article" aria-labelledby="complex_types__nested_types">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_datatypes.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="complex_types"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Complex Types (Impala 2.3 or higher only)</title></head><body id="complex_types"><main role="main"><article role="article" aria-labelledby="complex_types__nested_types">
<h1 class="title topictitle1" id="complex_types__nested_types">Complex Types (<span class="keyword">Impala 2.3</span> or higher only)</h1>
http://git-wip-us.apache.org/repos/asf/impala/blob/592fd85d/docs/build/html/topics/impala_components.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_components.html b/docs/build/html/topics/impala_components.html
index 5a20f4e..c6ee7fb 100644
--- a/docs/build/html/topics/impala_components.html
+++ b/docs/build/html/topics/impala_components.html
@@ -1,6 +1,6 @@
<!DOCTYPE html
SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_concepts.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="intro_components"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Components of the Impala Server</title></head><body id="intro_components"><main
role="main"><article role="article" aria-labelledby="ariaid-title1">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_concepts.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="intro_components"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Components of the Impala Server</title></head><body id="intro_components"><main
role="main"><article role="article" aria-labelledby="ariaid-title1">
<h1 class="title topictitle1" id="ariaid-title1">Components of the Impala Server</h1>
@@ -119,7 +119,7 @@
<p class="p">
The Impala component known as the <dfn class="term">catalog service</dfn> relays the metadata changes from Impala SQL
- statements to all the DataNodes in a cluster. It is physically represented by a daemon process named
+ statements to all the Impala daemons in a cluster. It is physically represented by a daemon process named
<code class="ph codeph">catalogd</code>; you only need such a process on one host in the cluster. Because the requests
are passed through the statestore daemon, it makes sense to run the <span class="keyword cmdname">statestored</span> and
<span class="keyword cmdname">catalogd</span> services on the same host.
@@ -162,12 +162,41 @@
</li>
</ul>
- <p class="p">
- By default, the metadata loading and caching on startup happens asynchronously, so Impala can begin
- accepting requests promptly. To enable the original behavior, where Impala waited until all metadata was
- loaded before accepting any requests, set the <span class="keyword cmdname">catalogd</span> configuration option
- <code class="ph codeph">--load_catalog_in_background=false</code>.
- </p>
+ <div class="p">
+ Use <code class="ph codeph">--load_catalog_in_background</code> option to control when
+ the metadata of a table is loaded.
+ <ul class="ul">
+ <li class="li">
+ If set to <code class="ph codeph">false</code>, the metadata of a table is
+ loaded when it is referenced for the first time. This means that the
+ first run of a particular query can be slower than subsequent runs.
+ Starting in Impala 2.2, the default for
+ <code class="ph codeph">load_catalog_in_background</code> is
+ <code class="ph codeph">false</code>.
+ </li>
+ <li class="li">
+ If set to <code class="ph codeph">true</code>, the catalog service attempts to
+ load metadata for a table even if no query needed that metadata. So
+ metadata will possibly be already loaded when the first query that
+ would need it is run. However, for the following reasons, we
+ recommend not to set the option to <code class="ph codeph">true</code>.
+ <ul class="ul">
+ <li class="li">
+ Background load can interfere with query-specific metadata
+ loading. This can happen on startup or after invalidating
+ metadata, with a duration depending on the amount of metadata,
+ and can lead to a seemingly random long running queries that are
+ difficult to diagnose.
+ </li>
+ <li class="li">
+ Impala may load metadata for tables that are possibly never
+ used, potentially increasing catalog size and consequently memory
+ usage for both catalog service and Impala Daemon.
+ </li>
+ </ul>
+ </li>
+ </ul>
+ </div>
<p class="p">
Most considerations for load balancing and high availability apply to the <span class="keyword cmdname">impalad</span> daemon.
http://git-wip-us.apache.org/repos/asf/impala/blob/592fd85d/docs/build/html/topics/impala_compression_codec.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_compression_codec.html b/docs/build/html/topics/impala_compression_codec.html
index b4aefea..2251cfa 100644
--- a/docs/build/html/topics/impala_compression_codec.html
+++ b/docs/build/html/topics/impala_compression_codec.html
@@ -1,6 +1,6 @@
<!DOCTYPE html
SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="compression_codec"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>COMPRESSION_CODEC Query Option (Impala 2.0 or higher only)</title></head><body id="compression_codec"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="compression_codec"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>COMPRESSION_CODEC Query Option (Impala 2.0 or higher only)</title></head><body id="compression_codec"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
<h1 class="title topictitle1" id="ariaid-title1">COMPRESSION_CODEC Query Option (<span class="keyword">Impala 2.0</span> or higher only)</h1>
http://git-wip-us.apache.org/repos/asf/impala/blob/592fd85d/docs/build/html/topics/impala_compute_stats.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_compute_stats.html b/docs/build/html/topics/impala_compute_stats.html
index 040f90e..1ee09e0 100644
--- a/docs/build/html/topics/impala_compute_stats.html
+++ b/docs/build/html/topics/impala_compute_stats.html
@@ -1,6 +1,6 @@
<!DOCTYPE html
SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.8.x"><meta name="version" content="Impala 2.8.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="compute_stats"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>COMPUTE STATS Statement</title></head><body id="compute_stats"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="compute_stats"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>COMPUTE STATS Statement</title></head><body id="compute_stats"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
<h1 class="title topictitle1" id="ariaid-title1">COMPUTE STATS Statement</h1>
@@ -9,19 +9,25 @@
<div class="body conbody">
<p class="p">
-
- Gathers information about volume and distribution of data in a table and all associated columns and
- partitions. The information is stored in the metastore database, and used by Impala to help optimize queries.
- For example, if Impala can determine that a table is large or small, or has many or few distinct values it
- can organize parallelize the work appropriately for a join query or insert operation. For details about the
- kinds of information gathered by this statement, see <a class="xref" href="impala_perf_stats.html#perf_stats">Table and Column Statistics</a>.
+ The
+ COMPUTE STATS statement gathers information about volume and distribution
+ of data in a table and all associated columns and partitions. The
+ information is stored in the metastore database, and used by Impala to
+ help optimize queries. For example, if Impala can determine that a table
+ is large or small, or has many or few distinct values it can organize and
+ parallelize the work appropriately for a join query or insert operation.
+ For details about the kinds of information gathered by this statement, see
+ <a class="xref" href="impala_perf_stats.html#perf_stats">Table and Column Statistics</a>.
</p>
<p class="p">
<strong class="ph b">Syntax:</strong>
</p>
-<pre class="pre codeblock"><code>COMPUTE STATS [<var class="keyword varname">db_name</var>.]<var class="keyword varname">table_name</var>
+<pre class="pre codeblock"><code><span class="ph">COMPUTE STATS [<var class="keyword varname">db_name</var>.]<var class="keyword varname">table_name</var> [ ( <var class="keyword varname">column_list</var> ) ] [TABLESAMPLE SYSTEM(<var class="keyword varname">percentage</var>) [REPEATABLE(<var class="keyword varname">seed</var>)]]</span>
+
+<var class="keyword varname">column_list</var> ::= <var class="keyword varname">column_name</var> [ , <var class="keyword varname">column_name</var>, ... ]
+
COMPUTE INCREMENTAL STATS [<var class="keyword varname">db_name</var>.]<var class="keyword varname">table_name</var> [PARTITION (<var class="keyword varname">partition_spec</var>)]
<var class="keyword varname">partition_spec</var> ::= <var class="keyword varname">simple_partition_spec</var> | <span class="ph"><var class="keyword varname">complex_partition_spec</var></span>
@@ -45,30 +51,70 @@ COMPUTE INCREMENTAL STATS [<var class="keyword varname">db_name</var>.]<var clas
</p>
<p class="p">
- Originally, Impala relied on users to run the Hive <code class="ph codeph">ANALYZE TABLE</code> statement, but that method
- of gathering statistics proved unreliable and difficult to use. The Impala <code class="ph codeph">COMPUTE STATS</code>
- statement is built from the ground up to improve the reliability and user-friendliness of this operation.
- <code class="ph codeph">COMPUTE STATS</code> does not require any setup steps or special configuration. You only run a
- single Impala <code class="ph codeph">COMPUTE STATS</code> statement to gather both table and column statistics, rather
- than separate Hive <code class="ph codeph">ANALYZE TABLE</code> statements for each kind of statistics.
+ Originally, Impala relied on users to run the Hive <code class="ph codeph">ANALYZE
+ TABLE</code> statement, but that method of gathering statistics proved
+ unreliable and difficult to use. The Impala <code class="ph codeph">COMPUTE STATS</code>
+ statement was built to improve the reliability and user-friendliness of
+ this operation. <code class="ph codeph">COMPUTE STATS</code> does not require any setup
+ steps or special configuration. You only run a single Impala
+ <code class="ph codeph">COMPUTE STATS</code> statement to gather both table and column
+ statistics, rather than separate Hive <code class="ph codeph">ANALYZE TABLE</code>
+ statements for each kind of statistics.
+ </p>
+
+ <p class="p">
+ For non-incremental <code class="ph codeph">COMPUTE STATS</code>
+ statement, the columns for which statistics are computed can be specified
+ with an optional comma-separate list of columns.
+ </p>
+
+ <p class="p">
+ If no column list is given, the <code class="ph codeph">COMPUTE STATS</code> statement
+ computes column-level statistics for all columns of the table. This adds
+ potentially unneeded work for columns whose stats are not needed by
+ queries. It can be especially costly for very wide tables and unneeded
+ large string fields.
+ </p>
+ <p class="p">
+ <code class="ph codeph">COMPUTE STATS</code> returns an error when a specified column
+ cannot be analyzed, such as when the column does not exist, the column is
+ of an unsupported type for COMPUTE STATS, e.g. colums of complex types,
+ or the column is a partitioning column.
+
+ </p>
+ <p class="p">
+ If an empty column list is given, no column is analyzed by <code class="ph codeph">COMPUTE
+ STATS</code>.
+ </p>
+
+ <p class="p">
+ In <span class="keyword">Impala 2.12</span> and
+ higher, an optional <code class="ph codeph">TABLESAMPLE</code> clause immediately after
+ a table reference specifies that the <code class="ph codeph">COMPUTE STATS</code>
+ operation only processes a specified percentage of the table data. For
+ tables that are so large that a full <code class="ph codeph">COMPUTE STATS</code>
+ operation is impractical, you can use <code class="ph codeph">COMPUTE STATS</code> with
+ a <code class="ph codeph">TABLESAMPLE</code> clause to extrapolate statistics from a
+ sample of the table data. See <a href="impala_perf_stats.html"><span class="keyword">Table and Column Statistics</span></a>about the
+ experimental stats extrapolation and sampling features.
</p>
<p class="p">
The <code class="ph codeph">COMPUTE INCREMENTAL STATS</code> variation is a shortcut for partitioned tables that works on a
subset of partitions rather than the entire table. The incremental nature makes it suitable for large tables
with many partitions, where a full <code class="ph codeph">COMPUTE STATS</code> operation takes too long to be practical
- each time a partition is added or dropped. See <a class="xref" href="impala_perf_stats.html#perf_stats_incremental">Overview of Incremental Statistics</a>
+ each time a partition is added or dropped. See <a class="xref" href="impala_perf_stats.html#perf_stats_incremental">impala_perf_stats.html#perf_stats_incremental</a>
for full usage details.
</p>
<div class="note important note_important"><span class="note__title importanttitle">Important:</span>
<p class="p">
For a particular table, use either <code class="ph codeph">COMPUTE STATS</code> or
- <code class="ph codeph">COMPUTE INCREMENTAL STATS</code>, but never combine the two or alternate
- between them. If you switch from <code class="ph codeph">COMPUTE STATS</code> to
- <code class="ph codeph">COMPUTE INCREMENTAL STATS</code> during the lifetime of a table, or vice
- versa, drop all statistics (by running both <code class="ph codeph">DROP STATS</code> and
- <code class="ph codeph">DROP INCREMENTAL STATS</code>) before making the switch.
+ <code class="ph codeph">COMPUTE INCREMENTAL STATS</code>, but never combine the two or
+ alternate between them. If you switch from <code class="ph codeph">COMPUTE STATS</code> to
+ <code class="ph codeph">COMPUTE INCREMENTAL STATS</code> during the lifetime of a table, or
+ vice versa, drop all statistics by running <code class="ph codeph">DROP STATS</code> before
+ making the switch.
</p>
<p class="p">
When you run <code class="ph codeph">COMPUTE INCREMENTAL STATS</code> on a table for the first time,
@@ -92,39 +138,44 @@ COMPUTE INCREMENTAL STATS [<var class="keyword varname">db_name</var>.]<var clas
<code class="ph codeph">COMPUTE STATS</code> statement. Such tables display <code class="ph codeph">false</code> under the
<code class="ph codeph">Incremental stats</code> column of the <code class="ph codeph">SHOW TABLE STATS</code> output.
</p>
-
<div class="note note note_note"><span class="note__title notetitle">Note:</span>
- Because many of the most performance-critical and resource-intensive operations rely on table and column
- statistics to construct accurate and efficient plans, <code class="ph codeph">COMPUTE STATS</code> is an important step at
- the end of your ETL process. Run <code class="ph codeph">COMPUTE STATS</code> on all tables as your first step during
- performance tuning for slow queries, or troubleshooting for out-of-memory conditions:
- <ul class="ul">
- <li class="li">
- Accurate statistics help Impala construct an efficient query plan for join queries, improving performance
- and reducing memory usage.
- </li>
-
- <li class="li">
- Accurate statistics help Impala distribute the work effectively for insert operations into Parquet
- tables, improving performance and reducing memory usage.
- </li>
-
- <li class="li">
- Accurate statistics help Impala estimate the memory required for each query, which is important when you
- use resource management features, such as admission control and the YARN resource management framework.
- The statistics help Impala to achieve high concurrency, full utilization of available memory, and avoid
- contention with workloads from other Hadoop components.
- </li>
- <li class="li">
- In <span class="keyword">Impala 2.8</span> and higher, when you run the
- <code class="ph codeph">COMPUTE STATS</code> or <code class="ph codeph">COMPUTE INCREMENTAL STATS</code>
- statement against a Parquet table, Impala automatically applies the query
- option setting <code class="ph codeph">MT_DOP=4</code> to increase the amount of intra-node
- parallelism during this CPU-intensive operation. See <a class="xref" href="impala_mt_dop.html">MT_DOP Query Option</a>
- for details about what this query option does and how to use it with
- CPU-intensive <code class="ph codeph">SELECT</code> statements.
- </li>
- </ul>
+ <div class="p">
+ Because many of the most performance-critical and resource-intensive
+ operations rely on table and column statistics to construct accurate and
+ efficient plans, <code class="ph codeph">COMPUTE STATS</code> is an important step at
+ the end of your ETL process. Run <code class="ph codeph">COMPUTE STATS</code> on all
+ tables as your first step during performance tuning for slow queries, or
+ troubleshooting for out-of-memory conditions:
+ <ul class="ul">
+ <li class="li">
+ Accurate statistics help Impala construct an efficient query plan
+ for join queries, improving performance and reducing memory usage.
+ </li>
+ <li class="li">
+ Accurate statistics help Impala distribute the work effectively
+ for insert operations into Parquet tables, improving performance and
+ reducing memory usage.
+ </li>
+ <li class="li">
+ Accurate statistics help Impala estimate the memory
+ required for each query, which is important when you use resource
+ management features, such as admission control and the YARN resource
+ management framework. The statistics help Impala to achieve high
+ concurrency, full utilization of available memory, and avoid
+ contention with workloads from other Hadoop components.
+ </li>
+ <li class="li">
+ In <span class="keyword">Impala 2.8</span> and
+ higher, when you run the <code class="ph codeph">COMPUTE STATS</code> or
+ <code class="ph codeph">COMPUTE INCREMENTAL STATS</code> statement against a
+ Parquet table, Impala automatically applies the query option setting
+ <code class="ph codeph">MT_DOP=4</code> to increase the amount of intra-node
+ parallelism during this CPU-intensive operation. See <a class="xref" href="impala_mt_dop.html">MT_DOP Query Option</a> for details about what this query option does
+ and how to use it with CPU-intensive <code class="ph codeph">SELECT</code>
+ statements.
+ </li>
+ </ul>
+ </div>
</div>
<p class="p">