You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by mi...@apache.org on 2018/05/09 21:10:45 UTC
[36/51] [partial] impala git commit: [DOCS] Impala doc site update
for 3.0
http://git-wip-us.apache.org/repos/asf/impala/blob/fae51ec2/docs/build3x/html/topics/impala_ddl.html
----------------------------------------------------------------------
diff --git a/docs/build3x/html/topics/impala_ddl.html b/docs/build3x/html/topics/impala_ddl.html
new file mode 100644
index 0000000..e9737bf
--- /dev/null
+++ b/docs/build3x/html/topics/impala_ddl.html
@@ -0,0 +1,141 @@
+<!DOCTYPE html
+ SYSTEM "about:legacy-compat">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 3.0.x"><meta name="version" content="Impala 3.0.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="ddl"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>DDL Statements</title></head><body id="ddl"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+
+ <h1 class="title topictitle1" id="ariaid-title1">DDL Statements</h1>
+
+
+ <div class="body conbody">
+
+ <p class="p">
+ DDL refers to <span class="q">"Data Definition Language"</span>, a subset of SQL statements that change the structure of the
+ database schema in some way, typically by creating, deleting, or modifying schema objects such as databases,
+ tables, and views. Most Impala DDL statements start with the keywords <code class="ph codeph">CREATE</code>,
+ <code class="ph codeph">DROP</code>, or <code class="ph codeph">ALTER</code>.
+ </p>
+
+ <p class="p">
+ The Impala DDL statements are:
+ </p>
+
+ <ul class="ul">
+ <li class="li">
+ <a class="xref" href="impala_alter_table.html#alter_table">ALTER TABLE Statement</a>
+ </li>
+
+ <li class="li">
+ <a class="xref" href="impala_alter_view.html#alter_view">ALTER VIEW Statement</a>
+ </li>
+
+ <li class="li">
+ <a class="xref" href="impala_compute_stats.html#compute_stats">COMPUTE STATS Statement</a>
+ </li>
+
+ <li class="li">
+ <a class="xref" href="impala_create_database.html#create_database">CREATE DATABASE Statement</a>
+ </li>
+
+ <li class="li">
+ <a class="xref" href="impala_create_function.html#create_function">CREATE FUNCTION Statement</a>
+ </li>
+
+ <li class="li">
+ <a class="xref" href="impala_create_role.html#create_role">CREATE ROLE Statement (Impala 2.0 or higher only)</a>
+ </li>
+
+ <li class="li">
+ <a class="xref" href="impala_create_table.html#create_table">CREATE TABLE Statement</a>
+ </li>
+
+ <li class="li">
+ <a class="xref" href="impala_create_view.html#create_view">CREATE VIEW Statement</a>
+ </li>
+
+ <li class="li">
+ <a class="xref" href="impala_drop_database.html#drop_database">DROP DATABASE Statement</a>
+ </li>
+
+ <li class="li">
+ <a class="xref" href="impala_drop_function.html#drop_function">DROP FUNCTION Statement</a>
+ </li>
+
+ <li class="li">
+ <a class="xref" href="impala_drop_role.html#drop_role">DROP ROLE Statement (Impala 2.0 or higher only)</a>
+ </li>
+
+ <li class="li">
+ <a class="xref" href="impala_drop_table.html#drop_table">DROP TABLE Statement</a>
+ </li>
+
+ <li class="li">
+ <a class="xref" href="impala_drop_view.html#drop_view">DROP VIEW Statement</a>
+ </li>
+
+ <li class="li">
+ <a class="xref" href="impala_grant.html#grant">GRANT Statement (Impala 2.0 or higher only)</a>
+ </li>
+
+ <li class="li">
+ <a class="xref" href="impala_revoke.html#revoke">REVOKE Statement (Impala 2.0 or higher only)</a>
+ </li>
+ </ul>
+
+ <p class="p">
+ After Impala executes a DDL command, information about available tables, columns, views, partitions, and so
+ on is automatically synchronized between all the Impala nodes in a cluster. (Prior to Impala 1.2, you had to
+ issue a <code class="ph codeph">REFRESH</code> or <code class="ph codeph">INVALIDATE METADATA</code> statement manually on the other
+ nodes to make them aware of the changes.)
+ </p>
+
+ <p class="p">
+ If the timing of metadata updates is significant, for example if you use round-robin scheduling where each
+ query could be issued through a different Impala node, you can enable the
+ <a class="xref" href="impala_sync_ddl.html#sync_ddl">SYNC_DDL</a> query option to make the DDL statement wait until
+ all nodes have been notified about the metadata changes.
+ </p>
+
+ <p class="p">
+ See <a class="xref" href="impala_s3.html#s3">Using Impala with the Amazon S3 Filesystem</a> for details about how Impala DDL statements interact with
+ tables and partitions stored in the Amazon S3 filesystem.
+ </p>
+
+ <p class="p">
+ Although the <code class="ph codeph">INSERT</code> statement is officially classified as a DML (data manipulation language)
+ statement, it also involves metadata changes that must be broadcast to all Impala nodes, and so is also
+ affected by the <code class="ph codeph">SYNC_DDL</code> query option.
+ </p>
+
+ <p class="p">
+ Because the <code class="ph codeph">SYNC_DDL</code> query option makes each DDL operation take longer than normal, you
+ might only enable it before the last DDL operation in a sequence. For example, if you are running a script
+ that issues multiple of DDL operations to set up an entire new schema, add several new partitions, and so on,
+ you might minimize the performance overhead by enabling the query option only before the last
+ <code class="ph codeph">CREATE</code>, <code class="ph codeph">DROP</code>, <code class="ph codeph">ALTER</code>, or <code class="ph codeph">INSERT</code> statement.
+ The script only finishes when all the relevant metadata changes are recognized by all the Impala nodes, so
+ you could connect to any node and issue queries through it.
+ </p>
+
+ <p class="p">
+ The classification of DDL, DML, and other statements is not necessarily the same between Impala and Hive.
+ Impala organizes these statements in a way intended to be familiar to people familiar with relational
+ databases or data warehouse products. Statements that modify the metastore database, such as <code class="ph codeph">COMPUTE
+ STATS</code>, are classified as DDL. Statements that only query the metastore database, such as
+ <code class="ph codeph">SHOW</code> or <code class="ph codeph">DESCRIBE</code>, are put into a separate category of utility statements.
+ </p>
+
+ <div class="note note note_note"><span class="note__title notetitle">Note:</span>
+ The query types shown in the Impala debug web user interface might not match exactly the categories listed
+ here. For example, currently the <code class="ph codeph">USE</code> statement is shown as DDL in the debug web UI. The
+ query types shown in the debug web UI are subject to change, for improved consistency.
+ </div>
+
+ <p class="p">
+ <strong class="ph b">Related information:</strong>
+ </p>
+
+ <p class="p">
+ The other major classifications of SQL statements are data manipulation language (see
+ <a class="xref" href="impala_dml.html#dml">DML Statements</a>) and queries (see <a class="xref" href="impala_select.html#select">SELECT Statement</a>).
+ </p>
+ </div>
+<nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_langref_sql.html">Impala SQL Statements</a></div></div></nav></article></main></body></html>
http://git-wip-us.apache.org/repos/asf/impala/blob/fae51ec2/docs/build3x/html/topics/impala_debug_action.html
----------------------------------------------------------------------
diff --git a/docs/build3x/html/topics/impala_debug_action.html b/docs/build3x/html/topics/impala_debug_action.html
new file mode 100644
index 0000000..f39c89f
--- /dev/null
+++ b/docs/build3x/html/topics/impala_debug_action.html
@@ -0,0 +1,24 @@
+<!DOCTYPE html
+ SYSTEM "about:legacy-compat">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 3.0.x"><meta name="version" content="Impala 3.0.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="debug_action"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>DEBUG_ACTION Query Option</title></head><body id="debug_action"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+
+ <h1 class="title topictitle1" id="ariaid-title1">DEBUG_ACTION Query Option</h1>
+
+
+
+ <div class="body conbody">
+
+ <p class="p">
+
+ Introduces artificial problem conditions within queries. For internal debugging and troubleshooting.
+ </p>
+
+ <p class="p">
+ <strong class="ph b">Type:</strong> <code class="ph codeph">STRING</code>
+ </p>
+
+ <p class="p">
+ <strong class="ph b">Default:</strong> empty string
+ </p>
+ </div>
+<nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_query_options.html">Query Options for the SET Statement</a></div></div></nav></article></main></body></html>
http://git-wip-us.apache.org/repos/asf/impala/blob/fae51ec2/docs/build3x/html/topics/impala_decimal.html
----------------------------------------------------------------------
diff --git a/docs/build3x/html/topics/impala_decimal.html b/docs/build3x/html/topics/impala_decimal.html
new file mode 100644
index 0000000..3c0a917
--- /dev/null
+++ b/docs/build3x/html/topics/impala_decimal.html
@@ -0,0 +1,907 @@
+<!DOCTYPE html
+ SYSTEM "about:legacy-compat">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_datatypes.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 3.0.x"><meta name="version" content="Impala 3.0.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="decimal"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>DECIMAL Data Type (Impala 3.0 or higher only)</title></head><body id="decimal"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+
+ <h1 class="title topictitle1" id="ariaid-title1">DECIMAL Data Type (<span class="keyword">Impala 3.0</span> or higher only)</h1>
+
+
+
+
+
+ <div class="body conbody">
+
+ <p class="p">
+ The <code class="ph codeph">DECIMAL</code> data type is a numeric data type with fixed scale and
+ precision.
+ </p>
+
+ <p class="p">
+ The data type is useful for storing and doing operations on precise decimal values.
+ </p>
+
+ <p class="p">
+ <strong class="ph b">Syntax:</strong>
+ </p>
+
+<pre class="pre codeblock"><code>DECIMAL[(<var class="keyword varname">precision</var>[, <var class="keyword varname">scale</var>])]</code></pre>
+
+ <p class="p">
+ <strong class="ph b">Precision:</strong>
+ </p>
+
+ <p class="p">
+ <var class="keyword varname">precision</var> represents the total number of digits that can be represented
+ regardless of the location of the decimal point.
+ </p>
+
+ <p class="p">
+ This value must be between 1 and 38, specified as an integer literal.
+ </p>
+
+ <p class="p">
+ The default precision is 9.
+ </p>
+
+ <p class="p">
+ <strong class="ph b">Scale:</strong>
+ </p>
+
+ <p class="p">
+ <var class="keyword varname">scale</var> represents the number of fractional digits.
+ </p>
+
+ <p class="p">
+ This value must be less than or equal to the precision, specified as an integer literal.
+ </p>
+
+ <p class="p">
+ The default scale is 0.
+ </p>
+
+ <p class="p">
+ When the precision and the scale are omitted, a <code class="ph codeph">DECIMAL</code> is treated as
+ <code class="ph codeph">DECIMAL(9, 0)</code>.
+ </p>
+
+ <p class="p">
+ <strong class="ph b">Range:</strong>
+ </p>
+
+ <p class="p">
+ The range of <code class="ph codeph">DECIMAL</code> type is -10^38 +1 through 10^38 –1.
+ </p>
+
+ <p class="p">
+ The largest value is represented by <code class="ph codeph">DECIMAL(38, 0)</code>.
+ </p>
+
+ <p class="p">
+ The most precise fractional value (between 0 and 1, or 0 and -1) is represented by
+ <code class="ph codeph">DECIMAL(38, 38)</code>, with 38 digits to the right of the decimal point. The
+ value closest to 0 would be .0000...1 (37 zeros and the final 1). The value closest to 1
+ would be .999... (9 repeated 38 times).
+ </p>
+
+ <p class="p">
+ <strong class="ph b">Memory and disk storage:</strong>
+ </p>
+
+ <p class="p">
+ Only the precision determines the storage size for <code class="ph codeph">DECIMAL</code> values, and
+ the scale setting has no effect on the storage size. The following table describes the
+ in-memory storage once the values are loaded into memory.
+ </p>
+
+ <div class="p">
+ <table class="simpletable frame-all" id="decimal__simpletable_tty_3y2_mdb"><col style="width:50%"><col style="width:50%"><thead><tr class="sthead">
+
+ <th class="stentry" id="decimal__simpletable_tty_3y2_mdb__stentry__1">Precision</th>
+
+ <th class="stentry" id="decimal__simpletable_tty_3y2_mdb__stentry__2">In-memory Storage</th>
+
+ </tr></thead><tbody><tr class="strow">
+
+ <td class="stentry" headers="decimal__simpletable_tty_3y2_mdb__stentry__1">1 - 9</td>
+
+ <td class="stentry" headers="decimal__simpletable_tty_3y2_mdb__stentry__2">4 bytes</td>
+
+ </tr><tr class="strow">
+
+ <td class="stentry" headers="decimal__simpletable_tty_3y2_mdb__stentry__1">10 - 18</td>
+
+ <td class="stentry" headers="decimal__simpletable_tty_3y2_mdb__stentry__2">8 bytes</td>
+
+ </tr><tr class="strow">
+
+ <td class="stentry" headers="decimal__simpletable_tty_3y2_mdb__stentry__1">19 - 38</td>
+
+ <td class="stentry" headers="decimal__simpletable_tty_3y2_mdb__stentry__2">16 bytes</td>
+
+ </tr></tbody></table>
+ </div>
+
+ <p class="p">
+ The on-disk representation varies depending on the file format of the table.
+ </p>
+
+ <p class="p">
+ Text, RCFile, and SequenceFile tables use ASCII-based formats as below:
+ </p>
+
+ <div class="p">
+ <ul class="ul">
+ <li class="li">
+ Leading zeros are not stored.
+ </li>
+
+ <li class="li">
+ Trailing zeros are stored.
+ </li>
+
+ <li class="li">
+ <p class="p">
+ Each <code class="ph codeph">DECIMAL</code> value takes up as many bytes as the precision of the
+ value, plus:
+ </p>
+ <ul class="ul">
+ <li class="li">
+ One extra byte if the decimal point is present.
+ </li>
+
+ <li class="li">
+ One extra byte for negative values.
+ </li>
+ </ul>
+ </li>
+ </ul>
+ </div>
+
+ <p class="p">
+ Parquet and Avro tables use binary formats and offer more compact storage for
+ <code class="ph codeph">DECIMAL</code> values. In these tables, Impala stores each value in fewer bytes
+ where possible depending on the precision specified for the <code class="ph codeph">DECIMAL</code>
+ column. To conserve space in large tables, use the smallest-precision
+ <code class="ph codeph">DECIMAL</code> type.
+ </p>
+
+ <p class="p">
+ <strong class="ph b">Precision and scale in arithmetic operations:</strong>
+ </p>
+
+ <p class="p">
+ For all arithmetic operations, the resulting precision is at most 38.
+ </p>
+
+ <p class="p">
+ If the resulting precision would be greater than 38, Impala truncates the result from the
+ back, but keeps at least 6 fractional digits in scale and rounds.
+ </p>
+
+ <p class="p">
+ For example, <code class="ph codeph">DECIMAL(38, 20) * DECIMAL(38, 20)</code> returns
+ <code class="ph codeph">DECIMAL(38, 6)</code>. According to the table below, the resulting precision and
+ scale would be <code class="ph codeph">(77, 40)</code>, but they are higher than the maximum precision
+ and scale for <code class="ph codeph">DECIMAL</code>. So, Impala sets the precision to the maximum
+ allowed 38, and truncates the scale to 6.
+ </p>
+
+ <div class="p">
+ When you use <code class="ph codeph">DECIMAL</code> values in arithmetic operations, the precision and
+ scale of the result value are determined as follows. For better readability, the following
+ terms are used in the table below:
+ <ul class="ul">
+ <li dir="ltr" class="li">
+ <p dir="ltr" class="p">
+ P1, P2: Input precisions
+ </p>
+ </li>
+
+ <li dir="ltr" class="li">
+ <p dir="ltr" class="p">
+ S1, S2: Input scales
+ </p>
+ </li>
+
+ <li dir="ltr" class="li">
+ <p dir="ltr" class="p">
+ L1, L2: Leading digits in input <code class="ph codeph">DECIMAL</code>s, i.e., L1 = P1 - S1 and L2
+ = P2 - S2
+ </p>
+ </li>
+ </ul>
+ </div>
+
+ <div class="p">
+ <table class="table frame-all" id="decimal__table_inl_sz2_mdb"><caption></caption><colgroup><col><col><col></colgroup><tbody class="tbody">
+ <tr class="row">
+ <td class="entry cellrowborder align-left colsep-1 rowsep-1">
+ <strong class="ph b">Operation</strong>
+ </td>
+ <td class="entry cellrowborder align-left colsep-1 rowsep-1">
+ <strong class="ph b">Resulting Precision</strong>
+ </td>
+ <td class="entry cellrowborder align-left colsep-1 rowsep-1">
+ <strong class="ph b">Resulting Scale</strong>
+ </td>
+ </tr>
+ <tr class="row">
+ <td class="entry cellrowborder align-left colsep-1 rowsep-1">
+ Addition and Subtraction
+ </td>
+ <td class="entry cellrowborder align-left colsep-1 rowsep-1">
+ <p class="p">
+ max (L1, L2) + max (S1, S2) + 1
+ </p>
+
+
+
+ <p class="p">
+ 1 is for carry-over.
+ </p>
+ </td>
+ <td class="entry cellrowborder align-left colsep-1 rowsep-1">
+ max (S1, S2)
+ </td>
+ </tr>
+ <tr class="row">
+ <td class="entry cellrowborder align-left colsep-1 rowsep-1">
+ Multiplication
+ </td>
+ <td class="entry cellrowborder align-left colsep-1 rowsep-1">
+ P1 + P2 + 1
+ </td>
+ <td class="entry cellrowborder align-left colsep-1 rowsep-1">
+ S1 + S2
+ </td>
+ </tr>
+ <tr class="row">
+ <td class="entry cellrowborder align-left colsep-1 rowsep-1">
+ Division
+ </td>
+ <td class="entry cellrowborder align-left colsep-1 rowsep-1">
+ L1 + S2 + max (S1 + P2 + 1, 6)
+ </td>
+ <td class="entry cellrowborder align-left colsep-1 rowsep-1">
+ max (S1 + P2 + 1, 6)
+ </td>
+ </tr>
+ <tr class="row">
+ <td class="entry cellrowborder align-left colsep-1 rowsep-1">
+ Modulo
+ </td>
+ <td class="entry cellrowborder align-left colsep-1 rowsep-1">
+ min (L1, L2) + max (S1, S2)
+ </td>
+ <td class="entry cellrowborder align-left colsep-1 rowsep-1">
+ max (S1, S2)
+ </td>
+ </tr>
+ </tbody></table>
+ </div>
+
+ <p class="p">
+ <strong class="ph b">Precision and scale in functions:</strong>
+ </p>
+
+ <div class="p">
+ When you use <code class="ph codeph">DECIMAL</code> values in built-in functions, the precision and
+ scale of the result value are determined as follows:
+ <ul class="ul">
+ <li dir="ltr" class="li">
+ The result of the <code class="ph codeph">SUM</code> aggregate function on a
+ <code class="ph codeph">DECIMAL</code> value is:
+ <ul class="ul">
+ <li class="li">
+ <p dir="ltr" class="p">
+ Precision: 38
+ </p>
+ </li>
+
+ <li class="li">
+ <p dir="ltr" class="p">
+ Scale: The same scale as the input column
+ </p>
+ </li>
+ </ul>
+ </li>
+
+ <li dir="ltr" class="li">
+ <p dir="ltr" class="p">
+ The result of <code class="ph codeph">AVG</code> aggregate function on a <code class="ph codeph">DECIMAL</code>
+ value is:
+ </p>
+ <ul class="ul">
+ <li class="li">
+ <p dir="ltr" class="p">
+ Precision: 38
+ </p>
+ </li>
+
+ <li class="li">
+ <p dir="ltr" class="p">
+ Scale: max(Scale of input column, 6)
+ </p>
+ </li>
+ </ul>
+ </li>
+ </ul>
+ </div>
+
+ <p class="p">
+ <strong class="ph b">Implicit conversions in DECIMAL assignments:</strong>
+ </p>
+
+ <p class="p">
+ Impala enforces strict conversion rules in decimal assignments like in
+ <code class="ph codeph">INSERT</code> and <code class="ph codeph">UNION</code> statements, or in functions like
+ <code class="ph codeph">COALESCE</code>.
+ </p>
+
+ <p class="p">
+ If there is not enough precision and scale in the destination, Impala fails with an error.
+ </p>
+
+ <div class="p">
+ Impala performs implicit conversions between <code class="ph codeph">DECIMAL</code> and other numeric
+ types as below:
+ <ul class="ul">
+ <li class="li">
+ <code class="ph codeph">DECIMAL</code> is implicitly converted to <code class="ph codeph">DOUBLE</code> or
+ <code class="ph codeph">FLOAT</code> when necessary even with a loss of precision. It can be
+ necessary, for example when inserting a <code class="ph codeph">DECIMAL</code> value into a
+ <code class="ph codeph">DOUBLE</code> column. For example:
+<pre class="pre codeblock"><code>CREATE TABLE flt(c FLOAT);
+INSERT INTO flt SELECT CAST(1e37 AS DECIMAL(38, 0));
+SELECT CAST(c AS DECIMAL(38, 0)) FROM flt;
+
+Result: 9999999933815812510711506376257961984</code></pre>
+ <p dir="ltr" class="p">
+ The result has a loss of information due to implicit casting. This is why we
+ discourage using the <code class="ph codeph">DOUBLE</code> and <code class="ph codeph">FLOAT</code> types in
+ general.
+ </p>
+ </li>
+
+ <li class="li">
+ <code class="ph codeph">DOUBLE</code> and <code class="ph codeph">FLOAT</code> cannot be implicitly converted to
+ <code class="ph codeph">DECIMAL</code>. An error is returned.
+ </li>
+
+ <li class="li">
+ <code class="ph codeph">DECIMAL</code> is implicitly converted to <code class="ph codeph">DECIMAL</code> if all
+ digits fit in the resulting <code class="ph codeph">DECIMAL</code>.
+ <div class="p">
+ For example, the following query returns an error because the resulting type that
+ guarantees that all digits fit cannot be determined .
+<pre class="pre codeblock"><code>SELECT GREATEST (CAST(1 AS DECIMAL(38, 0)), CAST(2 AS DECIMAL(38, 37)));</code></pre>
+ </div>
+ </li>
+
+ <li class="li">
+ Integer values can be implicitly converted to <code class="ph codeph">DECIMAL</code> when there is
+ enough room in the <code class="ph codeph">DECIMAL</code> to guarantee that all digits fit. The
+ integer types require the following numbers of digits to the left of the decimal point
+ when converted to <code class="ph codeph">DECIMAL</code>:
+ <ul class="ul">
+ <li class="li">
+ <p dir="ltr" class="p">
+ <code class="ph codeph">BIGINT</code>: 19 digits
+ </p>
+ </li>
+
+ <li class="li">
+ <p dir="ltr" class="p">
+ <code class="ph codeph">INT</code>: 10 digits
+ </p>
+ </li>
+
+ <li class="li">
+ <p dir="ltr" class="p">
+ <code class="ph codeph">SMALLINT</code>: 5 digits
+ </p>
+ </li>
+
+ <li class="li">
+ <p dir="ltr" class="p">
+ <code class="ph codeph">TINYINT</code>: 3 digits
+ </p>
+ </li>
+ </ul>
+ <p class="p">
+ For example:
+ </p>
+
+ <div class="p">
+<pre class="pre codeblock"><code>CREATE TABLE decimals_10_8 (x DECIMAL(10, 8));
+INSERT INTO decimals_10_8 VALUES (CAST(1 AS TINYINT));</code></pre>
+ </div>
+
+ <p class="p">
+ The above <code class="ph codeph">INSERT</code> statement fails because <code class="ph codeph">TINYINT</code>
+ requires room for 3 digits to the left of the decimal point in the
+ <code class="ph codeph">DECIMAL</code>.
+ </p>
+
+ <div class="p">
+<pre class="pre codeblock"><code>CREATE TABLE decimals_11_8(x DECIMAL(11, 8));
+INSERT INTO decimals_11_8 VALUES (CAST(1 AS TINYINT));</code></pre>
+ </div>
+
+ <p class="p">
+ The above <code class="ph codeph">INSERT</code> statement succeeds because there is enough room
+ for 3 digits to the left of the decimal point that <code class="ph codeph">TINYINT</code>
+ requires.
+ </p>
+ </li>
+ </ul>
+ </div>
+
+ <div class="p">
+ In <code class="ph codeph">UNION</code>, the resulting precision and scales are determined as follows.
+ <ul class="ul">
+ <li class="li">
+ Precision: max (L1, L2) + max (S1, S2)
+ <p class="p">
+ If the resulting type does not fit in the <code class="ph codeph">DECIMAL</code> type, an error is
+ returned. See the first example below.
+ </p>
+ </li>
+
+ <li class="li">
+ Scale: max (S1, S2)
+ </li>
+ </ul>
+ </div>
+
+ <div class="p">
+ Examples for <code class="ph codeph">UNION</code>:
+ <ul class="ul">
+ <li class="li">
+ <code class="ph codeph">DECIMAL(20, 0) UNION DECIMAL(20, 20)</code> would require a
+ <code class="ph codeph">DECIMAL(40, 20)</code> to fit all the digits. Since this is larger than the
+ max precision for <code class="ph codeph">DECIMAL</code>, Impala returns an error. One way to fix
+ the error is to cast both operands to the desired type, for example
+ <code class="ph codeph">DECIMAL(38, 18)</code>.
+ </li>
+
+ <li dir="ltr" class="li">
+ <p dir="ltr" class="p">
+ <code class="ph codeph">DECIMAL(20, 2) UNION DECIMAL(8, 6)</code> returns <code class="ph codeph">DECIMAL(24,
+ 6)</code>.
+ </p>
+ </li>
+
+ <li dir="ltr" class="li">
+ <p dir="ltr" class="p">
+ <code class="ph codeph">INT UNION DECIMAL(9, 4)</code> returns <code class="ph codeph">DECIMAL(14, 4)</code>.
+ </p>
+
+ <p class="p">
+ <code class="ph codeph">INT</code> has the precision 10 and the scale 0, so it is treated as
+ <code class="ph codeph">DECIMAL(10, 0) UNION DECIMAL(9. 4)</code>.
+ </p>
+ </li>
+ </ul>
+ </div>
+
+ <p class="p">
+ <strong class="ph b">Casting between DECIMAL and other data types:</strong>
+ </p>
+
+ <div class="p">
+ To avoid potential conversion errors, use <code class="ph codeph">CAST</code> to explicitly convert
+ between <code class="ph codeph">DECIMAL</code> and other types in decimal assignments like in
+ <code class="ph codeph">INSERT</code> and <code class="ph codeph">UNION</code> statements, or in functions like
+ <code class="ph codeph">COALESCE</code>:
+ <ul class="ul">
+ <li dir="ltr" class="li">
+ <p dir="ltr" class="p">
+ You can cast the following types to <code class="ph codeph">DECIMAL</code>:
+ <code class="ph codeph">FLOAT</code>, <code class="ph codeph">TINYINT</code>, <code class="ph codeph">SMALLINT</code>,
+ <code class="ph codeph">INT</code>, <code class="ph codeph">BIGINT</code>, <code class="ph codeph">STRING</code>
+ </p>
+ </li>
+
+ <li dir="ltr" class="li">
+ <p dir="ltr" class="p">
+ You can cast <code class="ph codeph">DECIMAL</code> to the following types:
+ <code class="ph codeph">FLOAT</code>, <code class="ph codeph">TINYINT</code>, <code class="ph codeph">SMALLINT</code>,
+ <code class="ph codeph">INT</code>, <code class="ph codeph">BIGINT</code>, <code class="ph codeph">STRING</code>,
+ <code class="ph codeph">BOOLEAN</code>, <code class="ph codeph">TIMESTAMP</code>
+ </p>
+ </li>
+ </ul>
+ </div>
+
+ <div class="p">
+ Impala performs <code class="ph codeph">CAST</code> between <code class="ph codeph">DECIMAL</code> and other numeric
+ types as below:
+ <ul class="ul">
+ <li dir="ltr" class="li">
+ <p dir="ltr" class="p">
+ Precision: If you cast a value with bigger precision than the precision of the
+ destination type, Impala returns an error. For example, <code class="ph codeph">CAST(123456 AS
+ DECIMAL(3,0))</code> returns an error because all digits do not fit into
+ <code class="ph codeph">DECIMAL(3, 0)</code>
+ </p>
+ </li>
+
+ <li dir="ltr" class="li">
+ <p dir="ltr" class="p">
+ Scale: If you cast a value with more fractional digits than the scale of the
+ destination type, the fractional digits are rounded. For example, <code class="ph codeph">CAST(1.239
+ AS DECIMAL(3, 2))</code> returns <code class="ph codeph">1.24</code>.
+ </p>
+ </li>
+ </ul>
+ </div>
+
+ <p class="p">
+ <strong class="ph b">Casting STRING to DECIMAL:</strong>
+ </p>
+
+ <div class="p">
+ You can cast <code class="ph codeph">STRING</code> of numeric characters in columns, literals, or
+ expressions to <code class="ph codeph">DECIMAL</code> as long as number fits within the specified target
+ <code class="ph codeph">DECIMAL</code> type without overflow.
+ <ul class="ul">
+ <li dir="ltr" class="li">
+ <p dir="ltr" class="p">
+ If scale in <code class="ph codeph">STRING</code> > scale in <code class="ph codeph">DECIMAL</code>, the
+ fractional digits are rounded to the <code class="ph codeph">DECIMAL</code> scale.
+ </p>
+
+ <p dir="ltr" class="p">
+ For example, <code class="ph codeph">CAST('98.678912' AS DECIMAL(15, 1))</code> returns
+ <code class="ph codeph">98.7</code>.
+ </p>
+ </li>
+
+ <li dir="ltr" class="li">
+ <p dir="ltr" class="p">
+ If # leading digits in <code class="ph codeph">STRING</code> > # leading digits in
+ <code class="ph codeph">DECIMAL</code>, an error is returned.
+ </p>
+
+ <p dir="ltr" class="p">
+ For example, <code class="ph codeph">CAST('123.45' AS DECIMAL(2, 2))</code> returns an error.
+ </p>
+ </li>
+ </ul>
+ </div>
+
+ <p class="p">
+ Exponential notation is supported when casting from <code class="ph codeph">STRING</code>.
+ </p>
+
+ <p class="p">
+ For example, <code class="ph codeph">CAST('1.0e6' AS DECIMAL(32, 0))</code> returns
+ <code class="ph codeph">1000000</code>.
+ </p>
+
+ <p class="p">
+ Casting any non-numeric value, such as <code class="ph codeph">'ABC'</code> to the
+ <code class="ph codeph">DECIMAL</code> type returns an error.
+ </p>
+
+ <p class="p">
+ <strong class="ph b">Casting DECIMAL to TIMESTAMP:</strong>
+ </p>
+
+ <p class="p">
+ Casting a <code class="ph codeph">DECIMAL</code> value N to <code class="ph codeph">TIMESTAMP</code> produces a value
+ that is N seconds past the start of the epoch date (January 1, 1970).
+ </p>
+
+ <p class="p">
+ <strong class="ph b">DECIMAL vs FLOAT consideration:</strong>
+ </p>
+
+ <p class="p">
+ The <code class="ph codeph">FLOAT</code> and <code class="ph codeph">DOUBLE</code> types can cause problems or
+ unexpected behavior due to inability to precisely represent certain fractional values, for
+ example dollar and cents values for currency. You might find output values slightly
+ different than you inserted, equality tests that do not match precisely, or unexpected
+ values for <code class="ph codeph">GROUP BY</code> columns. The <code class="ph codeph">DECIMAL</code> type can help
+ reduce unexpected behavior and rounding errors, but at the expense of some performance
+ overhead for assignments and comparisons.
+ </p>
+
+ <p class="p">
+ <strong class="ph b">Literals and expressions:</strong>
+ </p>
+
+ <div class="p">
+ <ul class="ul">
+ <li dir="ltr" class="li">
+ <p dir="ltr" class="p">
+ Numeric literals without a decimal point
+ </p>
+ <ul class="ul">
+ <li class="li">
+ The literals are treated as the smallest integer that would fit the literal. For
+ example, <code class="ph codeph">111</code> is a <code class="ph codeph">TINYINT</code>, and
+ <code class="ph codeph">1111</code> is a <code class="ph codeph">SMALLINT</code>.
+ </li>
+
+ <li class="li">
+ Large literals that do not fit into any integer type are treated as
+ <code class="ph codeph">DECIMAL</code>.
+ </li>
+
+ <li class="li">
+ The literals too large to fit into a <code class="ph codeph">DECIMAL(38, 0)</code> are treated
+ as <code class="ph codeph">DOUBLE</code>.
+ </li>
+ </ul>
+ </li>
+
+ <li dir="ltr" class="li">
+ <p dir="ltr" class="p">
+ Numeric literals with a decimal point
+ </p>
+ <ul class="ul">
+ <li class="li">
+ The literal with less than 38 digits are treated as <code class="ph codeph">DECIMAL</code>.
+ </li>
+
+ <li class="li">
+ The literals with 38 or more digits are treated as a <code class="ph codeph">DOUBLE</code>.
+ </li>
+ </ul>
+ </li>
+
+ <li class="li">
+ Exponential notation is supported in <code class="ph codeph">DECIMAL</code> literals.
+ </li>
+
+ <li dir="ltr" class="li">
+ <p class="p">
+ To represent a very large or precise <code class="ph codeph">DECIMAL</code> value as a literal,
+ for example one that contains more digits than can be represented by a
+ <code class="ph codeph">BIGINT</code> literal, use a quoted string or a floating-point value for
+ the number and <code class="ph codeph">CAST</code> the string to the desired
+ <code class="ph codeph">DECIMAL</code> type.
+ </p>
+
+ <p class="p">
+ For example: <code class="ph codeph">CAST('999999999999999999999999999999' AS DECIMAL(38,
+ 5)))</code>
+ </p>
+ </li>
+ </ul>
+ </div>
+
+ <p class="p">
+ <strong class="ph b">File format considerations:</strong>
+ </p>
+
+ <div class="p" dir="ltr">
+ The <code class="ph codeph">DECIMAL</code> data type can be stored in any of the file formats supported
+ by Impala.
+ <ul class="ul">
+ <li dir="ltr" class="li">
+ <p dir="ltr" class="p">
+ Impala can query Avro, RCFile, or SequenceFile tables that contain
+ <code class="ph codeph">DECIMAL</code> columns, created by other Hadoop components.
+ </p>
+ </li>
+
+ <li dir="ltr" class="li">
+ <p dir="ltr" class="p">
+ Impala can query and insert into Kudu tables that contain <code class="ph codeph">DECIMAL</code>
+ columns.
+ </p>
+ </li>
+
+ <li dir="ltr" class="li">
+ <p dir="ltr" class="p">
+ The <code class="ph codeph">DECIMAL</code> data type is fully compatible with HBase tables.
+ </p>
+ </li>
+
+ <li dir="ltr" class="li">
+ <p dir="ltr" class="p">
+ The <code class="ph codeph">DECIMAL</code> data type is fully compatible with Parquet tables.
+ </p>
+ </li>
+
+ <li dir="ltr" class="li">
+ <p dir="ltr" class="p">
+ Values of the <code class="ph codeph">DECIMAL</code> data type are potentially larger in text
+ tables than in tables using Parquet or other binary formats.
+ </p>
+ </li>
+ </ul>
+ </div>
+
+ <p class="p">
+ <strong class="ph b">UDF consideration:</strong>
+ </p>
+
+ <p class="p">
+ When writing a C++ UDF, use the <code class="ph codeph">DecimalVal</code> data type defined in
+ <span class="ph filepath">/usr/include/impala_udf/udf.h</span>.
+ </p>
+
+ <p class="p">
+ <strong class="ph b">Changing precision and scale:</strong>
+ </p>
+
+ <div class="p">
+ You can issue an <code class="ph codeph">ALTER TABLE ... REPLACE COLUMNS</code> statement to change the
+ precision and scale of an existing <code class="ph codeph">DECIMAL</code> column.
+ <ul class="ul">
+ <li dir="ltr" class="li">
+ <p dir="ltr" class="p">
+ For text-based formats (text, RCFile, and SequenceFile tables)
+ </p>
+ <ul class="ul">
+ <li class="li">
+ <p dir="ltr" class="p">
+ If the values in the column fit within the new precision and scale, they are
+ returned correctly by a query.
+ </p>
+ </li>
+
+ <li class="li">
+ <div class="p" dir="ltr">
+ If any values that do not fit within the new precision and scale:
+ <ul class="ul">
+ <li class="li">
+ Impala returns an error if the query option <code class="ph codeph">ABORT_ON_ERROR</code>
+ is set to <code class="ph codeph">true</code>.
+ </li>
+
+ <li class="li">
+ Impala returns a <code class="ph codeph">NULL</code> and warning that conversion failed if
+ the query option <code class="ph codeph">ABORT_ON_ERROR</code> is set to
+ <code class="ph codeph">false</code>.
+ </li>
+ </ul>
+ </div>
+ </li>
+
+ <li class="li">
+ <p class="p">
+ Leading zeros do not count against the precision value, but trailing zeros after
+ the decimal point do.
+ </p>
+ </li>
+ </ul>
+ </li>
+
+ <li dir="ltr" class="li">
+ <p dir="ltr" class="p">
+ For binary formats (Parquet and Avro tables)
+ </p>
+ <ul class="ul">
+ <li class="li">
+ <p dir="ltr" class="p">
+ Although an <code class="ph codeph">ALTER TABLE ... REPLACE COLUMNS</code> statement that
+ changes the precision or scale of a <code class="ph codeph">DECIMAL</code> column succeeds,
+ any subsequent attempt to query the changed column results in a fatal error.
+ This is because the metadata about the columns is stored in the data files
+ themselves, and <code class="ph codeph">ALTER TABLE</code> does not actually make any updates
+ to the data files. The other unaltered columns can still be queried
+ successfully.
+ </p>
+ </li>
+
+ <li class="li">
+ <p dir="ltr" class="p">
+ If the metadata in the data files disagrees with the metadata in the metastore
+ database, Impala cancels the query.
+ </p>
+ </li>
+ </ul>
+ </li>
+ </ul>
+ </div>
+
+ <p class="p">
+ <strong class="ph b">Partitioning:</strong>
+ </p>
+
+ <p class="p">
+ Using a <code class="ph codeph">DECIMAL</code> column as a partition key provides you a better match
+ between the partition key values and the HDFS directory names than using a
+ <code class="ph codeph">DOUBLE</code> or <code class="ph codeph">FLOAT</code> partitioning column.
+ </p>
+
+ <p class="p">
+ <strong class="ph b">Column statistics considerations:</strong>
+ </p>
+
+ <p class="p">
+ Because the <code class="ph codeph">DECIMAL</code> type has a fixed size, the maximum and average size
+ fields are always filled in for column statistics, even before you run the <code class="ph codeph">COMPUTE
+ STATS</code> statement.
+ </p>
+
+ <p class="p">
+ <strong class="ph b">Compatibility with older version of DECIMAL:</strong>
+ </p>
+
+ <p class="p">
+ This version of <code class="ph codeph">DECIMAL</code> type is the default in
+ <span class="keyword">Impala 3.0</span> and higher. The key differences between this
+ version of <code class="ph codeph">DECIMAL</code> and the previous <code class="ph codeph">DECIMAL</code> V1 in Impala
+ 2.x include the following.
+ </p>
+
+ <div class="p">
+ <table class="simpletable frame-all" id="decimal__simpletable_bwl_khm_rdb"><col style="width:33.33333333333333%"><col style="width:33.33333333333333%"><col style="width:33.33333333333333%"><thead><tr class="sthead">
+
+ <th class="stentry" id="decimal__simpletable_bwl_khm_rdb__stentry__1"></th>
+
+ <th class="stentry" id="decimal__simpletable_bwl_khm_rdb__stentry__2">DECIMAL in <span class="keyword">Impala 3.0</span> or
+ higher</th>
+
+ <th class="stentry" id="decimal__simpletable_bwl_khm_rdb__stentry__3">DECIMAL in <span class="keyword">Impala 2.12</span> or lower
+ </th>
+
+ </tr></thead><tbody><tr class="strow">
+
+ <td class="stentry" headers="decimal__simpletable_bwl_khm_rdb__stentry__1">Overall behavior</td>
+
+ <td class="stentry" headers="decimal__simpletable_bwl_khm_rdb__stentry__2">Returns either the result or an error.</td>
+
+ <td class="stentry" headers="decimal__simpletable_bwl_khm_rdb__stentry__3">Returns either the result or <code class="ph codeph">NULL</code> with a
+ warning.</td>
+
+ </tr><tr class="strow">
+
+ <td class="stentry" headers="decimal__simpletable_bwl_khm_rdb__stentry__1">Overflow behavior</td>
+
+ <td class="stentry" headers="decimal__simpletable_bwl_khm_rdb__stentry__2">Aborts with an error.</td>
+
+ <td class="stentry" headers="decimal__simpletable_bwl_khm_rdb__stentry__3">Issues a warning and returns <code class="ph codeph">NULL</code>.</td>
+
+ </tr><tr class="strow">
+
+ <td class="stentry" headers="decimal__simpletable_bwl_khm_rdb__stentry__1">Truncation / rounding behavior in arithmetic</td>
+
+ <td class="stentry" headers="decimal__simpletable_bwl_khm_rdb__stentry__2">Truncates and rounds digits from the back.</td>
+
+ <td class="stentry" headers="decimal__simpletable_bwl_khm_rdb__stentry__3">Truncates digits from the front.</td>
+
+ </tr><tr class="strow">
+
+ <td class="stentry" headers="decimal__simpletable_bwl_khm_rdb__stentry__1">String cast</td>
+
+ <td class="stentry" headers="decimal__simpletable_bwl_khm_rdb__stentry__2">Truncates from the back and rounds.</td>
+
+ <td class="stentry" headers="decimal__simpletable_bwl_khm_rdb__stentry__3">Truncates from the back.</td>
+
+ </tr></tbody></table>
+ </div>
+
+ <div class="p">
+ If you need to continue using the first version of the <code class="ph codeph">DECIMAL</code> type for
+ the backward compatibility of your queries, set the <code class="ph codeph">DECIMAL_V2</code> query
+ option to <code class="ph codeph">FALSE</code>:
+<pre class="pre codeblock"><code>SET DECIMAL_V2=FALSE;</code></pre>
+ </div>
+
+ <p class="p">
+ <strong class="ph b">Compatibility with other databases:</strong>
+ </p>
+
+ <p dir="ltr" class="p">
+ Use the <code class="ph codeph">DECIMAL</code> data type in Impala for applications where you used the
+ <code class="ph codeph">NUMBER</code> data type in Oracle.
+ </p>
+
+ <p dir="ltr" class="p">
+ The Impala <code class="ph codeph">DECIMAL</code> type does not support the Oracle idioms of
+ <code class="ph codeph">*</code> for scale.
+ </p>
+
+ <p dir="ltr" class="p">
+ The Impala <code class="ph codeph">DECIMAL</code> type does not support negative values for precision.
+ </p>
+
+ </div>
+
+<nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_datatypes.html">Data Types</a></div></div></nav></article></main></body></html>
http://git-wip-us.apache.org/repos/asf/impala/blob/fae51ec2/docs/build3x/html/topics/impala_decimal_v2.html
----------------------------------------------------------------------
diff --git a/docs/build3x/html/topics/impala_decimal_v2.html b/docs/build3x/html/topics/impala_decimal_v2.html
new file mode 100644
index 0000000..b26c3e7
--- /dev/null
+++ b/docs/build3x/html/topics/impala_decimal_v2.html
@@ -0,0 +1,32 @@
+<!DOCTYPE html
+ SYSTEM "about:legacy-compat">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html"><meta name="prodname" content="Impala"><meta name="version" content="Impala 3.0.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="decimal_v2"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>DECIMAL_V2 Query Option</title></head><body id="decimal_v2"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+
+ <h1 class="title topictitle1" id="ariaid-title1">DECIMAL_V2 Query Option</h1>
+
+
+
+ <div class="body conbody">
+
+ <p class="p">
+ A query option that changes behavior related to the <code class="ph codeph">DECIMAL</code>
+ data type.
+ </p>
+
+ <div class="note important note_important"><span class="note__title importanttitle">Important:</span>
+ <p class="p">
+ This query option is currently unsupported.
+ Its precise behavior is currently undefined and might change
+ in the future.
+ </p>
+ </div>
+
+ <p class="p">
+ <strong class="ph b">Type:</strong> Boolean; recognized values are 1 and 0, or <code class="ph codeph">true</code> and <code class="ph codeph">false</code>;
+ any other value interpreted as <code class="ph codeph">false</code>
+ </p>
+ <p class="p">
+ <strong class="ph b">Default:</strong> <code class="ph codeph">false</code> (shown as 0 in output of <code class="ph codeph">SET</code> statement)
+ </p>
+ </div>
+<nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_query_options.html">Query Options for the SET Statement</a></div></div></nav></article></main></body></html>
http://git-wip-us.apache.org/repos/asf/impala/blob/fae51ec2/docs/build3x/html/topics/impala_default_join_distribution_mode.html
----------------------------------------------------------------------
diff --git a/docs/build3x/html/topics/impala_default_join_distribution_mode.html b/docs/build3x/html/topics/impala_default_join_distribution_mode.html
new file mode 100644
index 0000000..95ae29b
--- /dev/null
+++ b/docs/build3x/html/topics/impala_default_join_distribution_mode.html
@@ -0,0 +1,113 @@
+<!DOCTYPE html
+ SYSTEM "about:legacy-compat">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 3.0.x"><meta name="version" content="Impala 3.0.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="default_join_distribution_mode"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>DEFAULT_JOIN_DISTRIBUTION_MODE Query Option</title></head><body id="default_join_distribution_mode"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+
+ <h1 class="title topictitle1" id="ariaid-title1">DEFAULT_JOIN_DISTRIBUTION_MODE Query Option</h1>
+
+
+
+ <div class="body conbody">
+
+ <p class="p">
+
+ This option determines the join distribution that Impala uses when any of the tables
+ involved in a join query is missing statistics.
+ </p>
+
+ <p class="p">
+ Impala optimizes join queries based on the presence of table statistics,
+ which are produced by the Impala <code class="ph codeph">COMPUTE STATS</code> statement.
+ By default, when a table involved in the join query does not have statistics,
+ Impala uses the <span class="q">"broadcast"</span> technique that transmits the entire contents
+ of the table to all executor nodes participating in the query. If one table
+ involved in a join has statistics and the other does not, the table without
+ statistics is broadcast. If both tables are missing statistics, the table
+ that is referenced second in the join order is broadcast. This behavior
+ is appropriate when the table involved is relatively small, but can lead to
+ excessive network, memory, and CPU overhead if the table being broadcast is
+ large.
+ </p>
+
+ <p class="p">
+ Because Impala queries frequently involve very large tables, and suboptimal
+ joins for such tables could result in spilling or out-of-memory errors,
+ the setting <code class="ph codeph">DEFAULT_JOIN_DISTRIBUTION_MODE=SHUFFLE</code> lets you
+ override the default behavior. The shuffle join mechanism divides the corresponding rows
+ of each table involved in a join query using a hashing algorithm, and transmits
+ subsets of the rows to other nodes for processing. Typically, this kind of join is
+ more efficient for joins between large tables of similar size.
+ </p>
+
+ <p class="p">
+ The setting <code class="ph codeph">DEFAULT_JOIN_DISTRIBUTION_MODE=SHUFFLE</code> is
+ recommended when setting up and deploying new clusters, because it is less likely
+ to result in serious consequences such as spilling or out-of-memory errors if
+ the query plan is based on incomplete information. This setting is not the default,
+ to avoid changing the performance characteristics of join queries for clusters that
+ are already tuned for their existing workloads.
+ </p>
+
+ <p class="p">
+ <strong class="ph b">Type:</strong> integer
+ </p>
+ <p class="p">
+ The allowed values are <code class="ph codeph">BROADCAST</code> (equivalent to 0)
+ or <code class="ph codeph">SHUFFLE</code> (equivalent to 1).
+ </p>
+
+ <p class="p">
+ <strong class="ph b">Examples:</strong>
+ </p>
+ <p class="p">
+ The following examples demonstrate appropriate scenarios for each
+ setting of this query option.
+ </p>
+
+<pre class="pre codeblock"><code>
+-- Create a billion-row table.
+create table big_table stored as parquet
+ as select * from huge_table limit 1e9;
+
+-- For a big table with no statistics, the
+-- shuffle join mechanism is appropriate.
+set default_join_distribution_mode=shuffle;
+
+...join queries involving the big table...
+</code></pre>
+
+<pre class="pre codeblock"><code>
+-- Create a hundred-row table.
+create table tiny_table stored as parquet
+ as select * from huge_table limit 100;
+
+-- For a tiny table with no statistics, the
+-- broadcast join mechanism is appropriate.
+set default_join_distribution_mode=broadcast;
+
+...join queries involving the tiny table...
+</code></pre>
+
+<pre class="pre codeblock"><code>
+compute stats tiny_table;
+compute stats big_table;
+
+-- Once the stats are computed, the query option has
+-- no effect on join queries involving these tables.
+-- Impala can determine the absolute and relative sizes
+-- of each side of the join query by examining the
+-- row size, cardinality, and so on of each table.
+
+...join queries involving both of these tables...
+</code></pre>
+
+ <p class="p">
+ <strong class="ph b">Related information:</strong>
+ </p>
+ <p class="p">
+ <a class="xref" href="impala_compute_stats.html">COMPUTE STATS Statement</a>,
+ <a class="xref" href="impala_joins.html">Joins in Impala SELECT Statements</a>,
+ <a class="xref" href="impala_perf_joins.html">Performance Considerations for Join Queries</a>
+ </p>
+
+ </div>
+<nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_query_options.html">Query Options for the SET Statement</a></div></div></nav></article></main></body></html>
http://git-wip-us.apache.org/repos/asf/impala/blob/fae51ec2/docs/build3x/html/topics/impala_default_spillable_buffer_size.html
----------------------------------------------------------------------
diff --git a/docs/build3x/html/topics/impala_default_spillable_buffer_size.html b/docs/build3x/html/topics/impala_default_spillable_buffer_size.html
new file mode 100644
index 0000000..3eb3689
--- /dev/null
+++ b/docs/build3x/html/topics/impala_default_spillable_buffer_size.html
@@ -0,0 +1,87 @@
+<!DOCTYPE html
+ SYSTEM "about:legacy-compat">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 3.0.x"><meta name="version" content="Impala 3.0.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="default_spillable_buffer_size"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>DEFAULT_SPILLABLE_BUFFER_SIZE Query Option</title></head><body id="default_spillable_buffer_size"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+
+ <h1 class="title topictitle1" id="ariaid-title1">DEFAULT_SPILLABLE_BUFFER_SIZE Query Option</h1>
+
+
+
+ <div class="body conbody">
+
+ <p class="p">
+
+ Specifies the default size for a memory buffer used when the
+ spill-to-disk mechanism is activated, for example for queries against
+ a large table with no statistics, or large join operations.
+ </p>
+
+ <p class="p">
+ <strong class="ph b">Type:</strong> integer
+ </p>
+
+
+
+ <p class="p">
+ <strong class="ph b">Default:</strong>
+ </p>
+ <p class="p">
+ <code class="ph codeph">2097152</code> (2 MB)
+ </p>
+
+ <p class="p">
+ <strong class="ph b">Units:</strong> A numeric argument represents a size in bytes; you can also use a suffix of <code class="ph codeph">m</code>
+ or <code class="ph codeph">mb</code> for megabytes, or <code class="ph codeph">g</code> or <code class="ph codeph">gb</code> for gigabytes. If you
+ specify a value with unrecognized formats, subsequent queries fail with an error.
+ </p>
+
+ <p class="p">
+ <strong class="ph b">Added in:</strong> <span class="keyword">Impala 2.10.0</span>
+ </p>
+
+ <p class="p">
+ <strong class="ph b">Usage notes:</strong>
+ </p>
+ <p class="p">
+ This query option sets an upper bound on the size of the internal
+ buffer size that can be used during spill-to-disk operations. The
+ actual size of the buffer is chosen by the query planner.
+ </p>
+ <p class="p">
+ If overall query performance is limited by the time needed for spilling,
+ consider increasing the <code class="ph codeph">DEFAULT_SPILLABLE_BUFFER_SIZE</code> setting.
+ Larger buffer sizes result in Impala issuing larger I/O requests to storage
+ devices, which might result in higher throughput, particularly on rotational
+ disks.
+ </p>
+ <p class="p">
+ The tradeoff with a large value for this setting is increased memory usage during
+ spill-to-disk operations. Reducing this value may reduce memory consumption.
+ </p>
+ <p class="p">
+ To determine if the value for this setting is having an effect by capping the
+ spillable buffer size, you can see the buffer size chosen by the query planner for
+ a particular query. <code class="ph codeph">EXPLAIN</code> the query while the setting
+ <code class="ph codeph">EXPLAIN_LEVEL=2</code> is in effect.
+ </p>
+
+ <p class="p">
+ <strong class="ph b">Examples:</strong>
+ </p>
+
+<pre class="pre codeblock"><code>
+set default_spillable_buffer_size=4MB;
+
+</code></pre>
+
+ <p class="p">
+ <strong class="ph b">Related information:</strong>
+ </p>
+ <p class="p">
+ <a class="xref" href="impala_buffer_pool_limit.html">BUFFER_POOL_LIMIT Query Option</a>,
+ <a class="xref" href="impala_max_row_size.html">MAX_ROW_SIZE Query Option</a>,
+ <a class="xref" href="impala_min_spillable_buffer_size.html">MIN_SPILLABLE_BUFFER_SIZE Query Option</a>,
+ <a class="xref" href="impala_scalability.html">Scalability Considerations for Impala</a>
+ </p>
+
+ </div>
+<nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_query_options.html">Query Options for the SET Statement</a></div></div></nav></article></main></body></html>
http://git-wip-us.apache.org/repos/asf/impala/blob/fae51ec2/docs/build3x/html/topics/impala_delegation.html
----------------------------------------------------------------------
diff --git a/docs/build3x/html/topics/impala_delegation.html b/docs/build3x/html/topics/impala_delegation.html
new file mode 100644
index 0000000..696af37
--- /dev/null
+++ b/docs/build3x/html/topics/impala_delegation.html
@@ -0,0 +1,70 @@
+<!DOCTYPE html
+ SYSTEM "about:legacy-compat">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_authentication.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 3.0.x"><meta name="version" content="Impala 3.0.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="delegation"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Configuring Impala Delegation for Hue and BI Tools</title></head><body id="delegation"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+
+ <h1 class="title topictitle1" id="ariaid-title1">Configuring Impala Delegation for Hue and BI Tools</h1>
+
+
+
+ <div class="body conbody">
+
+ <p class="p">
+
+ When users submit Impala queries through a separate application, such as Hue or a business intelligence tool,
+ typically all requests are treated as coming from the same user. In Impala 1.2 and higher, authentication is
+ extended by a new feature that allows applications to pass along credentials for the users that connect to
+ them (known as <span class="q">"delegation"</span>), and issue Impala queries with the privileges for those users. Currently,
+ the delegation feature is available only for Impala queries submitted through application interfaces such as
+ Hue and BI tools; for example, Impala cannot issue queries using the privileges of the HDFS user.
+ </p>
+
+ <p class="p">
+ The delegation feature is enabled by a startup option for <span class="keyword cmdname">impalad</span>:
+ <code class="ph codeph">--authorized_proxy_user_config</code>. When you specify this option, users whose names you specify
+ (such as <code class="ph codeph">hue</code>) can delegate the execution of a query to another user. The query runs with the
+ privileges of the delegated user, not the original user such as <code class="ph codeph">hue</code>. The name of the
+ delegated user is passed using the HiveServer2 configuration property <code class="ph codeph">impala.doas.user</code>.
+ </p>
+
+ <p class="p">
+ You can specify a list of users that the application user can delegate to, or <code class="ph codeph">*</code> to allow a
+ superuser to delegate to any other user. For example:
+ </p>
+
+<pre class="pre codeblock"><code>impalad --authorized_proxy_user_config 'hue=user1,user2;admin=*' ...</code></pre>
+
+ <div class="note note note_note"><span class="note__title notetitle">Note:</span>
+ Make sure to use single quotes or escape characters to ensure that any <code class="ph codeph">*</code> characters do not
+ undergo wildcard expansion when specified in command-line arguments.
+ </div>
+
+ <p class="p">
+ See <a class="xref" href="impala_config_options.html#config_options">Modifying Impala Startup Options</a> for details about adding or changing
+ <span class="keyword cmdname">impalad</span> startup options. See
+ <a class="xref" href="http://blog.cloudera.com/blog/2013/07/how-hiveserver2-brings-security-and-concurrency-to-apache-hive/" target="_blank">this
+ blog post</a> for background information about the delegation capability in HiveServer2.
+ </p>
+ <p class="p">
+ To set up authentication for the delegated users:
+ </p>
+
+ <ul class="ul">
+ <li class="li">
+ <p class="p">
+ On the server side, configure either user/password authentication through LDAP, or Kerberos
+ authentication, for all the delegated users. See <a class="xref" href="impala_ldap.html#ldap">Enabling LDAP Authentication for Impala</a> or
+ <a class="xref" href="impala_kerberos.html#kerberos">Enabling Kerberos Authentication for Impala</a> for details.
+ </p>
+ </li>
+
+ <li class="li">
+ <p class="p">
+ On the client side, to learn how to enable delegation, consult the documentation
+ for the ODBC driver you are using.
+ </p>
+ </li>
+ </ul>
+
+ </div>
+
+<nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_authentication.html">Impala Authentication</a></div></div></nav></article></main></body></html>
http://git-wip-us.apache.org/repos/asf/impala/blob/fae51ec2/docs/build3x/html/topics/impala_delete.html
----------------------------------------------------------------------
diff --git a/docs/build3x/html/topics/impala_delete.html b/docs/build3x/html/topics/impala_delete.html
new file mode 100644
index 0000000..668970e
--- /dev/null
+++ b/docs/build3x/html/topics/impala_delete.html
@@ -0,0 +1,177 @@
+<!DOCTYPE html
+ SYSTEM "about:legacy-compat">
+<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 3.0.x"><meta name="version" content="Impala 3.0.x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="delete"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>DELETE Statement (Impala 2.8 or higher only)</title></head><body id="delete"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+
+ <h1 class="title topictitle1" id="ariaid-title1">DELETE Statement (<span class="keyword">Impala 2.8</span> or higher only)</h1>
+
+
+
+ <div class="body conbody">
+
+ <p class="p">
+
+ Deletes an arbitrary number of rows from a Kudu table.
+ This statement only works for Impala tables that use the Kudu storage engine.
+ </p>
+
+ <p class="p">
+ <strong class="ph b">Syntax:</strong>
+ </p>
+
+<pre class="pre codeblock"><code>
+DELETE [FROM] [<var class="keyword varname">database_name</var>.]<var class="keyword varname">table_name</var> [ WHERE <var class="keyword varname">where_conditions</var> ]
+
+DELETE <var class="keyword varname">table_ref</var> FROM [<var class="keyword varname">joined_table_refs</var>] [ WHERE <var class="keyword varname">where_conditions</var> ]
+</code></pre>
+
+ <p class="p">
+ The first form evaluates rows from one table against an optional
+ <code class="ph codeph">WHERE</code> clause, and deletes all the rows that
+ match the <code class="ph codeph">WHERE</code> conditions, or all rows if
+ <code class="ph codeph">WHERE</code> is omitted.
+ </p>
+
+ <p class="p">
+ The second form evaluates one or more join clauses, and deletes
+ all matching rows from one of the tables. The join clauses can
+ include non-Kudu tables, but the table from which the rows
+ are deleted must be a Kudu table. The <code class="ph codeph">FROM</code>
+ keyword is required in this case, to separate the name of
+ the table whose rows are being deleted from the table names
+ of the join clauses.
+ </p>
+
+ <p class="p">
+ <strong class="ph b">Usage notes:</strong>
+ </p>
+
+ <p class="p">
+ The conditions in the <code class="ph codeph">WHERE</code> clause are the same ones allowed
+ for the <code class="ph codeph">SELECT</code> statement. See <a class="xref" href="impala_select.html#select">SELECT Statement</a>
+ for details.
+ </p>
+
+ <p class="p">
+ The conditions in the <code class="ph codeph">WHERE</code> clause can refer to
+ any combination of primary key columns or other columns. Referring to
+ primary key columns in the <code class="ph codeph">WHERE</code> clause is more efficient
+ than referring to non-primary key columns.
+ </p>
+
+ <p class="p">
+ If the <code class="ph codeph">WHERE</code> clause is omitted, all rows are removed from the table.
+ </p>
+
+ <p class="p">
+ Because Kudu currently does not enforce strong consistency during concurrent DML operations,
+ be aware that the results after this statement finishes might be different than you
+ intuitively expect:
+ </p>
+ <ul class="ul">
+ <li class="li">
+ <p class="p">
+ If some rows cannot be deleted because their
+ some primary key columns are not found, due to their being deleted
+ by a concurrent <code class="ph codeph">DELETE</code> operation,
+ the statement succeeds but returns a warning.
+ </p>
+ </li>
+ <li class="li">
+ <p class="p">
+ A <code class="ph codeph">DELETE</code> statement might also overlap with
+ <code class="ph codeph">INSERT</code>, <code class="ph codeph">UPDATE</code>,
+ or <code class="ph codeph">UPSERT</code> statements running concurrently on the same table.
+ After the statement finishes, there might be more or fewer rows than expected in the table
+ because it is undefined whether the <code class="ph codeph">DELETE</code> applies to rows that are
+ inserted or updated while the <code class="ph codeph">DELETE</code> is in progress.
+ </p>
+ </li>
+ </ul>
+
+ <p class="p">
+ The number of affected rows is reported in an <span class="keyword cmdname">impala-shell</span> message
+ and in the query profile.
+ </p>
+
+ <p class="p">
+ <strong class="ph b">Statement type:</strong> DML
+ </p>
+
+ <div class="note important note_important"><span class="note__title importanttitle">Important:</span>
+ After adding or replacing data in a table used in performance-critical queries, issue a <code class="ph codeph">COMPUTE
+ STATS</code> statement to make sure all statistics are up-to-date. Consider updating statistics for a
+ table after any <code class="ph codeph">INSERT</code>, <code class="ph codeph">LOAD DATA</code>, or <code class="ph codeph">CREATE TABLE AS
+ SELECT</code> statement in Impala, or after loading data through Hive and doing a <code class="ph codeph">REFRESH
+ <var class="keyword varname">table_name</var></code> in Impala. This technique is especially important for tables that
+ are very large, used in join queries, or both.
+ </div>
+
+ <p class="p">
+ <strong class="ph b">Examples:</strong>
+ </p>
+
+ <p class="p">
+ The following examples show how to delete rows from a specified
+ table, either all rows or rows that match a <code class="ph codeph">WHERE</code>
+ clause:
+ </p>
+
+<pre class="pre codeblock"><code>
+-- Deletes all rows. The FROM keyword is optional.
+DELETE FROM kudu_table;
+DELETE kudu_table;
+
+-- Deletes 0, 1, or more rows.
+-- (If c1 is a single-column primary key, the statement could only
+-- delete 0 or 1 rows.)
+DELETE FROM kudu_table WHERE c1 = 100;
+
+-- Deletes all rows that match all the WHERE conditions.
+DELETE FROM kudu_table WHERE
+ (c1 > c2 OR c3 IN ('hello','world')) AND c4 IS NOT NULL;
+DELETE FROM t1 WHERE
+ (c1 IN (1,2,3) AND c2 > c3) OR c4 IS NOT NULL;
+DELETE FROM time_series WHERE
+ year = 2016 AND month IN (11,12) AND day > 15;
+
+-- WHERE condition with a subquery.
+DELETE FROM t1 WHERE
+ c5 IN (SELECT DISTINCT other_col FROM other_table);
+
+-- Does not delete any rows, because the WHERE condition is always false.
+DELETE FROM kudu_table WHERE 1 = 0;
+</code></pre>
+
+ <p class="p">
+ The following examples show how to delete rows that are part
+ of the result set from a join:
+ </p>
+
+<pre class="pre codeblock"><code>
+-- Remove _all_ rows from t1 that have a matching X value in t2.
+DELETE t1 FROM t1 JOIN t2 ON t1.x = t2.x;
+
+-- Remove _some_ rows from t1 that have a matching X value in t2.
+DELETE t1 FROM t1 JOIN t2 ON t1.x = t2.x
+ WHERE t1.y = FALSE and t2.z > 100;
+
+-- Delete from a Kudu table based on a join with a non-Kudu table.
+DELETE t1 FROM kudu_table t1 JOIN non_kudu_table t2 ON t1.x = t2.x;
+
+-- The tables can be joined in any order as long as the Kudu table
+-- is specified as the deletion target.
+DELETE t2 FROM non_kudu_table t1 JOIN kudu_table t2 ON t1.x = t2.x;
+</code></pre>
+
+ <p class="p">
+ <strong class="ph b">Related information:</strong>
+ </p>
+
+ <p class="p">
+ <a class="xref" href="impala_kudu.html#impala_kudu">Using Impala to Query Kudu Tables</a>, <a class="xref" href="impala_insert.html#insert">INSERT Statement</a>,
+ <a class="xref" href="impala_update.html#update">UPDATE Statement (Impala 2.8 or higher only)</a>, <a class="xref" href="impala_upsert.html#upsert">UPSERT Statement (Impala 2.8 or higher only)</a>
+ </p>
+
+ </div>
+
+<nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_langref_sql.html">Impala SQL Statements</a></div></div></nav></article></main></body></html>