You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by mi...@apache.org on 2018/12/06 23:14:51 UTC
[25/51] [partial] impala git commit: [DOCS] Impala 3.1 Docs to be
published
http://git-wip-us.apache.org/repos/asf/impala/blob/b4ad38a9/docs/build/html/topics/impala_insert.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_insert.html b/docs/build/html/topics/impala_insert.html
index d81bcfb..883e851 100644
--- a/docs/build/html/topics/impala_insert.html
+++ b/docs/build/html/topics/impala_insert.html
@@ -1,8 +1,30 @@
+<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html
- SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="insert"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>INSERT Statement</title></head><body id="insert"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+ PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+
+<meta name="copyright" content="(C) Copyright 2018" />
+<meta name="DC.rights.owner" content="(C) Copyright 2018" />
+<meta name="DC.Type" content="concept" />
+<meta name="DC.Title" content="INSERT Statement" />
+<meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html" />
+<meta name="prodname" content="Impala" />
+<meta name="prodname" content="Impala" />
+<meta name="prodname" content="Impala" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="DC.Format" content="XHTML" />
+<meta name="DC.Identifier" content="insert" />
+<link rel="stylesheet" type="text/css" href="../commonltr.css" />
+<title>INSERT Statement</title>
+</head>
+<body id="insert">
+
<h1 class="title topictitle1" id="ariaid-title1">INSERT Statement</h1>
+
@@ -14,10 +36,12 @@
TABLE</code> statement, or pre-defined tables and partitions created through Hive.
</p>
+
<p class="p">
<strong class="ph b">Syntax:</strong>
</p>
+
<pre class="pre codeblock"><code>[<var class="keyword varname">with_clause</var>]
INSERT <span class="ph">[<var class="keyword varname">hint_clause</var>]</span> { INTO | OVERWRITE } [TABLE] <var class="keyword varname">table_name</var>
[(<var class="keyword varname">column_list</var>)]
@@ -42,21 +66,24 @@ hint_with_brackets ::= [SHUFFLE] | [NOSHUFFLE]
(With this hint format, the square brackets are part of the syntax.)
</code></pre>
- <div class="note note note_note"><span class="note__title notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
The square bracket style of hint is now deprecated and might be removed in
a future release. For that reason, any newly added hints are not available
with the square bracket syntax.
</div>
+
<p class="p">
<strong class="ph b">Appending or replacing (INTO and OVERWRITE clauses):</strong>
</p>
+
<p class="p">
The <code class="ph codeph">INSERT INTO</code> syntax appends data to a table. The existing data files are left as-is, and
the inserted data is put into one or more new data files.
</p>
+
<p class="p">
The <code class="ph codeph">INSERT OVERWRITE</code> syntax replaces the data in a table.
@@ -65,10 +92,12 @@ hint_with_brackets ::= [SHUFFLE] | [NOSHUFFLE]
mechanism.
</p>
+
<p class="p">
<strong class="ph b">Complex type considerations:</strong>
</p>
+
<p class="p">
The <code class="ph codeph">INSERT</code> statement currently does not support writing data files
containing complex types (<code class="ph codeph">ARRAY</code>, <code class="ph codeph">STRUCT</code>, and <code class="ph codeph">MAP</code>).
@@ -78,14 +107,17 @@ hint_with_brackets ::= [SHUFFLE] | [NOSHUFFLE]
See <a class="xref" href="impala_complex_types.html#complex_types">Complex Types (Impala 2.3 or higher only)</a> for details about working with complex types.
</p>
+
<p class="p">
<strong class="ph b">Kudu considerations:</strong>
</p>
+
<p class="p">
Currently, the <code class="ph codeph">INSERT OVERWRITE</code> syntax cannot be used with Kudu tables.
</p>
+
<p class="p">
Kudu tables require a unique primary key for each row. If an <code class="ph codeph">INSERT</code>
statement attempts to insert a row with the same values for the primary key columns
@@ -98,6 +130,7 @@ hint_with_brackets ::= [SHUFFLE] | [NOSHUFFLE]
syntax.)
</p>
+
<p class="p">
For situations where you prefer to replace rows with duplicate primary key values,
rather than discarding the new data, you can use the <code class="ph codeph">UPSERT</code>
@@ -107,24 +140,29 @@ hint_with_brackets ::= [SHUFFLE] | [NOSHUFFLE]
<span class="q">"upserted"</span> data.
</p>
+
<p class="p">
If you really want to store new rows, not replace existing ones, but cannot do so
because of the primary key uniqueness constraint, consider recreating the table
with additional columns included in the primary key.
</p>
+
<p class="p">
See <a class="xref" href="impala_kudu.html#impala_kudu">Using Impala to Query Kudu Tables</a> for more details about using Impala with Kudu.
</p>
+
<p class="p">
<strong class="ph b">Usage notes:</strong>
</p>
+
<p class="p">
Impala currently supports:
</p>
+
<ul class="ul">
<li class="li">
Copy data from another table using <code class="ph codeph">SELECT</code> query. In Impala 1.2.1 and higher, you can
@@ -132,27 +170,32 @@ hint_with_brackets ::= [SHUFFLE] | [NOSHUFFLE]
<code class="ph codeph">CREATE TABLE AS SELECT</code> syntax, which bypasses the actual <code class="ph codeph">INSERT</code> keyword.
</li>
+
<li class="li">
An optional <a class="xref" href="impala_with.html#with"><code class="ph codeph">WITH</code> clause</a> before the
<code class="ph codeph">INSERT</code> keyword, to define a subquery referenced in the <code class="ph codeph">SELECT</code> portion.
</li>
+
<li class="li">
Create one or more new rows using constant expressions through <code class="ph codeph">VALUES</code> clause. (The
<code class="ph codeph">VALUES</code> clause was added in Impala 1.0.1.)
</li>
+
<li class="li">
<p class="p">
By default, the first column of each newly inserted row goes into the first column of the table, the
second column into the second column, and so on.
</p>
+
<p class="p">
You can also specify the columns to be inserted, an arbitrarily ordered subset of the columns in the
destination table, by specifying a column list immediately after the name of the destination table. This
feature lets you adjust the inserted columns to match the layout of a <code class="ph codeph">SELECT</code> statement,
rather than the other way around. (This feature was added in Impala 1.1.)
</p>
+
<p class="p">
The number of columns mentioned in the column list (known as the <span class="q">"column permutation"</span>) must match
the number of columns in the <code class="ph codeph">SELECT</code> list or the <code class="ph codeph">VALUES</code> tuples. The
@@ -160,8 +203,10 @@ hint_with_brackets ::= [SHUFFLE] | [NOSHUFFLE]
of each input row are reordered to match. If the number of columns in the column permutation is less than
in the destination table, all unmentioned columns are set to <code class="ph codeph">NULL</code>.
</p>
+
</li>
+
<li class="li">
An optional hint clause immediately either before the <code class="ph codeph">SELECT</code> keyword or after the
<code class="ph codeph">INSERT</code> keyword, to fine-tune the behavior when doing an <code class="ph codeph">INSERT ... SELECT</code>
@@ -172,9 +217,11 @@ hint_with_brackets ::= [SHUFFLE] | [NOSHUFFLE]
<span class="ph">large</span> memory buffers being allocated to buffer the data for each
partition. For usage details, see <a class="xref" href="impala_parquet.html#parquet_etl">Loading Data into Parquet Tables</a>.
</li>
+
</ul>
- <div class="note note note_note"><span class="note__title notetitle">Note:</span>
+
+ <div class="note note"><span class="notetitle">Note:</span>
<ul class="ul">
<li class="li">
Insert commands that partition or add files result in changes to Hive metadata. Because Impala uses Hive
@@ -182,28 +229,35 @@ hint_with_brackets ::= [SHUFFLE] | [NOSHUFFLE]
<a class="xref" href="impala_refresh.html#refresh">REFRESH</a> function.
</li>
+
<li class="li">
Currently, Impala can only insert data into tables that use the text and Parquet formats. For other file
formats, insert the data using Hive and use Impala to query it.
</li>
+
<li class="li">
As an alternative to the <code class="ph codeph">INSERT</code> statement, if you have existing data files elsewhere in
HDFS, the <code class="ph codeph">LOAD DATA</code> statement can move those files into a table. This statement works
with tables of any file format.
</li>
+
</ul>
+
</div>
+
<p class="p">
<strong class="ph b">Statement type:</strong> DML (but still affected by
<a class="xref" href="../shared/../topics/impala_sync_ddl.html#sync_ddl">SYNC_DDL</a> query option)
</p>
+
<p class="p">
<strong class="ph b">Usage notes:</strong>
</p>
+
<p class="p">
When you insert the results of an expression, particularly of a built-in function call, into a small numeric
column such as <code class="ph codeph">INT</code>, <code class="ph codeph">SMALLINT</code>, <code class="ph codeph">TINYINT</code>, or
@@ -213,10 +267,12 @@ hint_with_brackets ::= [SHUFFLE] | [NOSHUFFLE]
in the <code class="ph codeph">INSERT</code> statement to make the conversion explicit.
</p>
+
<p class="p">
<strong class="ph b">File format considerations:</strong>
</p>
+
<p class="p">
Because Impala can read certain file formats that it cannot write,
the <code class="ph codeph">INSERT</code> statement does not work for all kinds of
@@ -225,6 +281,7 @@ hint_with_brackets ::= [SHUFFLE] | [NOSHUFFLE]
<code class="ph codeph">INSERT</code> statement.
</p>
+
<p class="p">
Any <code class="ph codeph">INSERT</code> statement for a Parquet table requires enough free space in the HDFS filesystem
to write one block. Because Parquet data files use a block size of 1 GB by default, an
@@ -232,6 +289,7 @@ hint_with_brackets ::= [SHUFFLE] | [NOSHUFFLE]
space.
</p>
+
<p class="p">
If you connect to different Impala nodes within an <span class="keyword cmdname">impala-shell</span> session for
load-balancing purposes, you can enable the <code class="ph codeph">SYNC_DDL</code> query option to make each DDL
@@ -239,7 +297,8 @@ hint_with_brackets ::= [SHUFFLE] | [NOSHUFFLE]
nodes. See <a class="xref" href="../shared/../topics/impala_sync_ddl.html#sync_ddl">SYNC_DDL Query Option</a> for details.
</p>
- <div class="note important note_important"><span class="note__title importanttitle">Important:</span>
+
+ <div class="note important"><span class="importanttitle">Important:</span>
After adding or replacing data in a table used in performance-critical queries, issue a <code class="ph codeph">COMPUTE
STATS</code> statement to make sure all statistics are up-to-date. Consider updating statistics for a
table after any <code class="ph codeph">INSERT</code>, <code class="ph codeph">LOAD DATA</code>, or <code class="ph codeph">CREATE TABLE AS
@@ -248,10 +307,12 @@ hint_with_brackets ::= [SHUFFLE] | [NOSHUFFLE]
are very large, used in join queries, or both.
</div>
+
<p class="p">
<strong class="ph b">Examples:</strong>
</p>
+
<p class="p">
The following example sets up new tables with the same definition as the <code class="ph codeph">TAB1</code> table from the
<a class="xref" href="impala_tutorial.html#tutorial">Tutorial</a> section, using different file
@@ -259,6 +320,7 @@ hint_with_brackets ::= [SHUFFLE] | [NOSHUFFLE]
and <code class="ph codeph">STORED AS PARQUET</code> clauses:
</p>
+
<pre class="pre codeblock"><code>CREATE DATABASE IF NOT EXISTS file_formats;
USE file_formats;
@@ -279,6 +341,7 @@ STORED AS PARQUET;</code></pre>
statements with 5 rows each, the table contains 10 rows total:
</p>
+
<pre class="pre codeblock"><code>[localhost:21000] > insert into table text_table select * from default.tab1;
Inserted 5 rows in 0.41s
@@ -301,12 +364,14 @@ Returned 1 row(s) in 0.26s</code></pre>
efficient form to perform intensive analysis on that subset.
</p>
+
<p class="p">
For example, here we insert 5 rows into a table using the <code class="ph codeph">INSERT INTO</code> clause, then replace
the data by inserting 3 rows with the <code class="ph codeph">INSERT OVERWRITE</code> clause. Afterward, the table only
contains the 3 rows from the final <code class="ph codeph">INSERT</code> statement.
</p>
+
<pre class="pre codeblock"><code>[localhost:21000] > insert into table parquet_table select * from default.tab1;
Inserted 5 rows in 0.35s
@@ -326,7 +391,8 @@ Returned 1 row(s) in 0.43s</code></pre>
match the table definition.
</p>
- <div class="note note note_note" id="insert__insert_values_warning"><span class="note__title notetitle">Note:</span>
+
+ <div class="note note" id="insert__insert_values_warning"><span class="notetitle">Note:</span>
The <code class="ph codeph">INSERT ... VALUES</code> technique is not suitable for loading large quantities of data into
HDFS-based tables, because the insert operations cannot be parallelized, and each one produces a separate
data file. Use it for setting up small dimension tables or tiny amounts of data for experimenting with SQL
@@ -337,11 +403,13 @@ Returned 1 row(s) in 0.43s</code></pre>
and use a separate database to make cleanup easier if the operation does produce many tiny files.
</div>
+
<p class="p">
The following example shows how to insert one row or multiple rows, with expressions of different types,
using literal values, expressions, and function return values:
</p>
+
<pre class="pre codeblock"><code>create table val_test_1 (c1 int, c2 float, c3 string, c4 boolean, c5 timestamp);
insert into val_test_1 values (100, 99.9/10, 'abc', true, now());
create table val_test_2 (id int, token string);
@@ -352,6 +420,7 @@ insert overwrite val_test_2 values (1, 'a'), (2, 'b'), (-1,'xyzzy');</code></pre
a table with a file format that Impala currently does not write to:
</p>
+
<pre class="pre codeblock"><code>DROP TABLE IF EXISTS sequence_table;
CREATE TABLE sequence_table
( id INT, col_1 BOOLEAN, col_2 DOUBLE, col_3 TIMESTAMP )
@@ -376,6 +445,7 @@ Backend 0:SEQUENCE_FILE not implemented. </code></pre>
actually appear in the table:
</p>
+
<pre class="pre codeblock"><code>-- Start with 2 identical tables.
create table t1 (c1 int, c2 int);
create table t2 like t1;
@@ -407,6 +477,7 @@ insert into t2 (y) select c1 from t1;
stored in sorted order is impractical.
</p>
+
<p class="p">
<strong class="ph b">Concurrency considerations:</strong> Each <code class="ph codeph">INSERT</code> operation creates new data files with unique
names, so you can run multiple <code class="ph codeph">INSERT INTO</code> statements simultaneously without filename
@@ -419,12 +490,19 @@ insert into t2 (y) select c1 from t1;
issuing an <code class="ph codeph">hdfs dfs -rm -r</code> command, specifying the full path of the work subdirectory, whose
name ends in <code class="ph codeph">_dir</code>.
</p>
+
</div>
- <nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_langref_sql.html">Impala SQL Statements</a></div></div></nav><article class="topic concept nested1" aria-labelledby="ariaid-title2" id="insert__values">
+
+ <div class="related-links">
+<div class="familylinks">
+<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_langref_sql.html">Impala SQL Statements</a></div>
+</div>
+</div><div class="topic concept nested1" aria-labelledby="ariaid-title2" id="values">
<h2 class="title topictitle2" id="ariaid-title2">VALUES Clause</h2>
+
<div class="body conbody">
<p class="p">
@@ -432,7 +510,8 @@ insert into t2 (y) select c1 from t1;
typically within an <code class="ph codeph"><a class="xref" href="impala_insert.html#insert">INSERT</a></code> statement.
</p>
- <div class="note note note_note"><span class="note__title notetitle">Note:</span>
+
+ <div class="note note"><span class="notetitle">Note:</span>
The <code class="ph codeph">INSERT ... VALUES</code> technique is not suitable for loading large quantities of data into
HDFS-based tables, because the insert operations cannot be parallelized, and each one produces a separate
data file. Use it for setting up small dimension tables or tiny amounts of data for experimenting with SQL
@@ -443,25 +522,30 @@ insert into t2 (y) select c1 from t1;
clause, and use a separate database to make cleanup easier if the operation does produce many tiny files.
</div>
+
<p class="p">
The following examples illustrate:
</p>
+
<ul class="ul">
<li class="li">
How to insert a single row using a <code class="ph codeph">VALUES</code> clause.
</li>
+
<li class="li">
How to insert multiple rows using a <code class="ph codeph">VALUES</code> clause.
</li>
+
<li class="li">
How the row or rows from a <code class="ph codeph">VALUES</code> clause can be appended to a table through
<code class="ph codeph">INSERT INTO</code>, or replace the contents of the table through <code class="ph codeph">INSERT
OVERWRITE</code>.
</li>
+
<li class="li">
How the entries in a <code class="ph codeph">VALUES</code> clause can be literals, function results, or any other kind
of expression. See <a class="xref" href="impala_literals.html#literals">Literals</a> for the notation to use for literal
@@ -470,8 +554,10 @@ insert into t2 (y) select c1 from t1;
<a class="xref" href="impala_functions.html#builtins">Impala Built-In Functions</a> for other things you can include in expressions with the
<code class="ph codeph">VALUES</code> clause.
</li>
+
</ul>
+
<pre class="pre codeblock"><code>[localhost:21000] > describe val_example;
Query: describe val_example
Query finished, fetching results ...
@@ -509,6 +595,7 @@ Inserted 2 rows in 0.16s
use the syntax:
</p>
+
<pre class="pre codeblock"><code>INSERT INTO <var class="keyword varname">destination</var>
(<var class="keyword varname">col_x</var>, <var class="keyword varname">col_y</var>, <var class="keyword varname">col_z</var>)
VALUES
@@ -522,11 +609,13 @@ Inserted 2 rows in 0.16s
+
<p class="p">
To use a <code class="ph codeph">VALUES</code> clause like a table in other statements, wrap it in parentheses and use
<code class="ph codeph">AS</code> clauses to specify aliases for the entire object and any columns you need to refer to:
</p>
+
<pre class="pre codeblock"><code>[localhost:21000] > select * from (values(4,5,6),(7,8,9)) as t;
+---+---+---+
| 4 | 5 | 6 |
@@ -547,16 +636,19 @@ Inserted 2 rows in 0.16s
values as part of a longer statement involving joins or <code class="ph codeph">UNION ALL</code>.
</p>
+
<p class="p">
<strong class="ph b">HDFS considerations:</strong>
</p>
+
<p class="p">
Impala physically writes all inserted files under the ownership of its default user, typically
<code class="ph codeph">impala</code>. Therefore, this user must have HDFS write permission in the corresponding table
directory.
</p>
+
<p class="p">
The permission requirement is independent of the authorization performed by the Sentry framework. (If the
connected user is not authorized to insert into a table, Sentry blocks that operation immediately,
@@ -564,6 +656,7 @@ Inserted 2 rows in 0.16s
not owned by and do not inherit permissions from the connected user.
</p>
+
<p class="p">
The number of data files produced by an <code class="ph codeph">INSERT</code> statement depends on the size of the
cluster, the number of data blocks that are processed, the partition key columns in a partitioned table,
@@ -574,6 +667,7 @@ Inserted 2 rows in 0.16s
of megabytes are considered <span class="q">"tiny"</span>.)
</p>
+
<p class="p">
The <code class="ph codeph">INSERT</code> statement has always left behind a hidden work directory inside the data
directory of the table. Formerly, this hidden work directory was named
@@ -584,14 +678,17 @@ Inserted 2 rows in 0.16s
adjust them to use the new name.
</p>
+
<p class="p">
<strong class="ph b">HBase considerations:</strong>
</p>
+
<p class="p">
You can use the <code class="ph codeph">INSERT</code> statement with HBase tables as follows:
</p>
+
<ul class="ul">
<li class="li">
<p class="p">
@@ -599,15 +696,19 @@ Inserted 2 rows in 0.16s
VALUES</code> syntax. This is a good use case for HBase tables with Impala, because HBase tables are
not subject to the same kind of fragmentation from many small insert operations as HDFS tables are.
</p>
+
</li>
+
<li class="li">
<p class="p">
You can insert any number of rows at once into an HBase table using the <code class="ph codeph">INSERT ...
SELECT</code> syntax.
</p>
+
</li>
+
<li class="li">
<p class="p">
If more than one inserted row has the same value for the HBase key column, only the last inserted row
@@ -617,14 +718,18 @@ Inserted 2 rows in 0.16s
copying from an HDFS table, the HBase table might contain fewer rows than were inserted, if the key
column in the source table contained duplicate values.
</p>
+
</li>
+
<li class="li">
<p class="p">
You cannot <code class="ph codeph">INSERT OVERWRITE</code> into an HBase table. New rows are always appended.
</p>
+
</li>
+
<li class="li">
<p class="p">
When you create an Impala or Hive table that maps to an HBase table, the column order you specify with
@@ -636,16 +741,21 @@ Inserted 2 rows in 0.16s
<code class="ph codeph">DESCRIBE</code> statement for the table, and adjust the order of the select list in the
<code class="ph codeph">INSERT</code> statement.
</p>
+
</li>
+
</ul>
+
<p class="p">
See <a class="xref" href="impala_hbase.html#impala_hbase">Using Impala to Query HBase Tables</a> for more details about using Impala with HBase.
</p>
+
<p class="p">
<strong class="ph b">Amazon S3 considerations:</strong>
</p>
+
<p class="p">
In <span class="keyword">Impala 2.6</span> and higher, the Impala DML statements (<code class="ph codeph">INSERT</code>, <code class="ph codeph">LOAD DATA</code>,
and <code class="ph codeph">CREATE TABLE AS SELECT</code>) can write data into a table or partition that resides in the
@@ -657,6 +767,7 @@ Inserted 2 rows in 0.16s
If you bring data into S3 using the normal S3 transfer mechanisms instead of Impala DML statements,
issue a <code class="ph codeph">REFRESH</code> statement for the table before using Impala to query the S3 data.
</p>
+
<p class="p">
Because of differences between S3 and traditional filesystems, DML operations
for S3 tables can take longer than for tables on HDFS. For example, both the
@@ -672,33 +783,43 @@ Inserted 2 rows in 0.16s
It does not apply to <code class="ph codeph">INSERT OVERWRITE</code> or <code class="ph codeph">LOAD DATA</code> statements.
See <a class="xref" href="../shared/../topics/impala_s3_skip_insert_staging.html#s3_skip_insert_staging">S3_SKIP_INSERT_STAGING Query Option (Impala 2.6 or higher only)</a> for details.
</p>
+
<p class="p">See <a class="xref" href="impala_s3.html#s3">Using Impala with the Amazon S3 Filesystem</a> for details about reading and writing S3 data with Impala.</p>
+
<p class="p">
<strong class="ph b">ADLS considerations:</strong>
</p>
- <p class="p">
- In <span class="keyword">Impala 2.9</span> and higher, the Impala DML statements (<code class="ph codeph">INSERT</code>, <code class="ph codeph">LOAD DATA</code>,
- and <code class="ph codeph">CREATE TABLE AS SELECT</code>) can write data into a table or partition that resides in the
- Azure Data Lake Store (ADLS).
- The syntax of the DML statements is the same as for any other tables, because the ADLS location for tables and
- partitions is specified by an <code class="ph codeph">adl://</code> prefix in the
- <code class="ph codeph">LOCATION</code> attribute of
- <code class="ph codeph">CREATE TABLE</code> or <code class="ph codeph">ALTER TABLE</code> statements.
- If you bring data into ADLS using the normal ADLS transfer mechanisms instead of Impala DML statements,
- issue a <code class="ph codeph">REFRESH</code> statement for the table before using Impala to query the ADLS data.
- </p>
+
+ <p class="p"> In <span class="keyword">Impala 2.9</span> and higher, the Impala DML statements
+ (<code class="ph codeph">INSERT</code>, <code class="ph codeph">LOAD DATA</code>, and
+ <code class="ph codeph">CREATE TABLE AS SELECT</code>) can write data into a table
+ or partition that resides in the Azure Data Lake Store (ADLS). ADLS Gen2
+ is supported in <span class="keyword">Impala 3.1</span> and higher.</p>
+<p class="p">In the<code class="ph codeph">CREATE TABLE</code> or
+ <code class="ph codeph">ALTER TABLE</code> statements, specify the ADLS location for
+ tables and partitions with the <code class="ph codeph">adl://</code> prefix for ADLS
+ Gen1 and <code class="ph codeph">abfs://</code> or <code class="ph codeph">abfss://</code> for ADLS
+ Gen2 in the <code class="ph codeph">LOCATION</code> attribute.</p>
+<p class="p">If you bring data into ADLS
+ using the normal ADLS transfer mechanisms instead of Impala DML
+ statements, issue a <code class="ph codeph">REFRESH</code> statement for the table
+ before using Impala to query the ADLS data. </p>
+
<p class="p">See <a class="xref" href="impala_adls.html#adls">Using Impala with the Azure Data Lake Store (ADLS)</a> for details about reading and writing ADLS data with Impala.</p>
+
<p class="p">
<strong class="ph b">Security considerations:</strong>
</p>
+
<p class="p">
If these statements in your environment contain sensitive literal values such as credit card numbers or tax
identifiers, Impala can redact this sensitive information when displaying the statements in log files and
other administrative contexts. See <span class="xref">the documentation for your Apache Hadoop distribution</span> for details.
</p>
+
<p class="p">
<strong class="ph b">Cancellation:</strong> Can be cancelled. To cancel this statement, use Ctrl-C from the
<span class="keyword cmdname">impala-shell</span> interpreter, the <span class="ph uicontrol">Cancel</span> button from the
@@ -707,9 +828,11 @@ Inserted 2 rows in 0.16s
(port 25000).
</p>
+
<p class="p">
<strong class="ph b">HDFS permissions:</strong>
</p>
+
<p class="p">
The user ID that the <span class="keyword cmdname">impalad</span> daemon runs under,
typically the <code class="ph codeph">impala</code> user, must have read
@@ -723,44 +846,56 @@ Inserted 2 rows in 0.16s
the original data files in the table, only on the table directories themselves.
</p>
+
<p class="p">
<strong class="ph b">Restrictions:</strong>
</p>
+
<p class="p">
For <code class="ph codeph">INSERT</code> operations into <code class="ph codeph">CHAR</code> or <code class="ph codeph">VARCHAR</code> columns, you
must cast all <code class="ph codeph">STRING</code> literals or expressions returning <code class="ph codeph">STRING</code> to to a
<code class="ph codeph">CHAR</code> or <code class="ph codeph">VARCHAR</code> type with the appropriate length.
</p>
+
<p class="p">
<strong class="ph b">Related startup options:</strong>
</p>
+
<p class="p">
By default, if an <code class="ph codeph">INSERT</code> statement creates any new subdirectories underneath a partitioned
table, those subdirectories are assigned default HDFS permissions for the <code class="ph codeph">impala</code> user. To
make each subdirectory have the same permissions as its parent directory in HDFS, specify the
<code class="ph codeph">--insert_inherit_permissions</code> startup option for the <span class="keyword cmdname">impalad</span> daemon.
</p>
+
</div>
- </article>
- <article class="topic concept nested1" aria-labelledby="ariaid-title3" id="insert__partition_insert">
+
+ </div>
+
+ <div class="topic concept nested1" aria-labelledby="ariaid-title3" id="partition_insert">
<h2 class="title topictitle2" id="ariaid-title3">Inserting Into Partitioned Tables with PARTITION Clause</h2>
+
<div class="body conbody">
<p class="p">
For a partitioned table, the optional <code class="ph codeph">PARTITION</code> clause
identifies which partition or partitions the values are inserted
into.
</p>
+
<p class="p">
All examples in this section will use the table declared as below:
</p>
+
<pre class="pre codeblock"><code>CREATE TABLE t1 (w INT) PARTITIONED BY (x INT, y STRING);</code></pre>
</div>
- <article class="topic concept nested2" aria-labelledby="ariaid-title4" id="partition_insert__static_partition_insert">
+
+ <div class="topic concept nested2" aria-labelledby="ariaid-title4" id="static_partition_insert">
<h3 class="title topictitle3" id="ariaid-title4">Static Partition Inserts</h3>
+
<div class="body conbody">
<p class="p">
In a static partition insert where a partition key column is given a
@@ -768,17 +903,21 @@ Inserted 2 rows in 0.16s
<code class="ph codeph">(year=2012, month=2)</code>, the rows are inserted with the
same values specified for those partition key columns.
</p>
+
<p class="p">
The number of columns in the <code class="ph codeph">SELECT</code> list must equal
the number of columns in the column permutation.
</p>
+
<p class="p">
The <code class="ph codeph">PARTITION</code> clause must be used for static
partitioning inserts.
</p>
+
<p class="p">
Example:
</p>
+
<div class="p">
The following statement will insert the
<code class="ph codeph">some_other_table.c1</code> values for the
@@ -788,10 +927,14 @@ Inserted 2 rows in 0.16s
<code class="ph codeph">‘a’</code>.<pre class="pre codeblock"><code>INSERT INTO t1 PARTITION (x=10, y='a')
SELECT c1 FROM some_other_table;</code></pre>
</div>
+
</div>
- </article>
- <article class="topic concept nested2" aria-labelledby="ariaid-title5" id="partition_insert__dynamic_partition_insert">
+
+ </div>
+
+ <div class="topic concept nested2" aria-labelledby="ariaid-title5" id="dynamic_partition_insert">
<h3 class="title topictitle3" id="ariaid-title5">Dynamic Partition Inserts</h3>
+
<div class="body conbody">
<p class="p">
In a dynamic partition insert where a partition key
@@ -805,50 +948,75 @@ Inserted 2 rows in 0.16s
in the column permutation plus the number of partition key columns not
assigned a constant value.
</p>
+
<p class="p">
- See <a class="xref" href="https://www.cloudera.com/documentation/enterprise/latest/topics/impala_partitioning.html#partition_static_dynamic" target="_blank"><u class="ph u">Static and Dynamic Partitioning
- Clauses</u></a> for examples and performance characteristics
- of static and dynamic partitioned inserts.
+ See <a class="xref" href="impala_partitioning.html#partition_static_dynamic">Static and Dynamic Partitioning Clauses</a> for examples and performance characteristics of static and dynamic
+ partitioned inserts.
</p>
+
<p class="p">
The following rules apply to dynamic partition
inserts.
</p>
+
<ul class="ul">
<li class="li">
<p class="p">
The columns are bound in the order they appear in the
<code class="ph codeph">INSERT</code> statement.
</p>
+
<p class="p">
The table below shows the values inserted with the
<code class="ph codeph">INSERT</code> statements of different column
orders.
</p>
+
</li>
+
</ul>
- <table class="table frame-all" id="dynamic_partition_insert__table_vyx_dp3_ldb"><caption></caption><colgroup><col><col><col><col></colgroup><tbody class="tbody">
+
+
+<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="dynamic_partition_insert__table_vyx_dp3_ldb" class="table" frame="border" border="1" rules="all"><colgroup><col /><col /><col /><col /></colgroup><tbody class="tbody">
<tr class="row">
- <td class="entry cellrowborder align-left colsep-1 rowsep-1"></td>
- <td class="entry cellrowborder align-left colsep-1 rowsep-1">Column <code class="ph codeph">w</code> Value</td>
- <td class="entry cellrowborder align-left colsep-1 rowsep-1">Column <code class="ph codeph">x</code> Value</td>
- <td class="entry cellrowborder align-left colsep-1 rowsep-1">Column <code class="ph codeph">y</code> Value</td>
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;"> </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;">Column <code class="ph codeph">w</code> Value</td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;">Column <code class="ph codeph">x</code> Value</td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;">Column <code class="ph codeph">y</code> Value</td>
+
</tr>
+
<tr class="row">
- <td class="entry cellrowborder align-left colsep-1 rowsep-1"><code class="ph codeph">INSERT INTO t1 (w, x, y) VALUES (1, 2,
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;"><code class="ph codeph">INSERT INTO t1 (w, x, y) VALUES (1, 2,
'c');</code></td>
- <td class="entry cellrowborder align-left colsep-1 rowsep-1"><code class="ph codeph">1</code></td>
- <td class="entry cellrowborder align-left colsep-1 rowsep-1"><code class="ph codeph">2</code></td>
- <td class="entry cellrowborder align-left colsep-1 rowsep-1"><code class="ph codeph">‘c’</code></td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;"><code class="ph codeph">1</code></td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;"><code class="ph codeph">2</code></td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;"><code class="ph codeph">‘c’</code></td>
+
</tr>
+
<tr class="row">
- <td class="entry cellrowborder align-left colsep-1 rowsep-1"><code class="ph codeph">INSERT INTO t1 (x,w) PARTITION (y) VALUES (1,
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;"><code class="ph codeph">INSERT INTO t1 (x,w) PARTITION (y) VALUES (1,
2, 'c');</code></td>
- <td class="entry cellrowborder align-left colsep-1 rowsep-1"><code class="ph codeph">2</code></td>
- <td class="entry cellrowborder align-left colsep-1 rowsep-1"><code class="ph codeph">1</code></td>
- <td class="entry cellrowborder align-left colsep-1 rowsep-1"><code class="ph codeph">‘c’</code></td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;"><code class="ph codeph">2</code></td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;"><code class="ph codeph">1</code></td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;"><code class="ph codeph">‘c’</code></td>
+
</tr>
- </tbody></table>
+
+ </tbody>
+</table>
+</div>
+
<ul class="ul">
<li class="li">
When a partition clause is specified but the non-partition
@@ -863,10 +1031,12 @@ Inserted 2 rows in 0.16s
and <code class="ph codeph">‘c’</code> to <code class="ph codeph">y</code>
columns.
</p>
+
<pre class="pre codeblock"><code>INSERT INTO t1 PARTITION (x,y) VALUES (1, 2, ‘c’);
INSERT INTO t1 (w) PARTITION (x, y) VALUES (1, 2, ‘c’);
INSERT INTO t1 PARTITION (x, y='c') VALUES (1, 2);</code></pre>
</li>
+
<li class="li">
The <code class="ph codeph">PARTITION</code> clause is not required for
dynamic partition, but all the partition columns must be explicitly
@@ -876,12 +1046,14 @@ INSERT INTO t1 PARTITION (x, y='c') VALUES (1, 2);</code></pre>
<p class="p">
Example:
</p>
+
<p class="p">The following statements are valid because the partition
columns, <code class="ph codeph">x</code> and <code class="ph codeph">y</code>, are present in
the <code class="ph codeph">INSERT</code> statements, either in the
<code class="ph codeph">PARTITION</code> clause or in the column
list.
</p>
+
<pre class="pre codeblock"><code>INSERT INTO t1 PARTITION (x,y) VALUES (1, 2, ‘c’);
INSERT INTO t1 (w, x) PARTITION (y) VALUES (1, 2, ‘c’);</code></pre>
<p class="p">
@@ -890,8 +1062,10 @@ INSERT INTO t1 (w, x) PARTITION (y) VALUES (1, 2, ‘c’);</code></pre>
and <code class="ph codeph">y</code>, are not present in the
<code class="ph codeph">INSERT</code> statement.
</p>
+
<pre class="pre codeblock"><code>INSERT INTO t1 VALUES (1, 2, 'c');</code></pre>
</li>
+
<li class="li">
If partition columns do not exist in the source table, you can
specify a specific value for that column in the
@@ -902,10 +1076,17 @@ INSERT INTO t1 (w, x) PARTITION (y) VALUES (1, 2, ‘c’);</code></pre>
<code class="ph codeph">20</code>, specified in the <code class="ph codeph">PARTITION</code>
clause, is inserted into the <code class="ph codeph">x</code> column.
</p>
+
<pre class="pre codeblock"><code>INSERT INTO t1 PARTITION (x=20, y) SELECT * FROM source;</code></pre>
</li>
+
</ul>
+
</div>
- </article>
- </article>
- </article></main></body></html>
\ No newline at end of file
+
+ </div>
+
+ </div>
+
+ </body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/impala/blob/b4ad38a9/docs/build/html/topics/impala_install.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_install.html b/docs/build/html/topics/impala_install.html
index 05a00c4..91a5dc2 100644
--- a/docs/build/html/topics/impala_install.html
+++ b/docs/build/html/topics/impala_install.html
@@ -1,8 +1,27 @@
+<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html
- SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="install"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Installing Impala</title></head><body id="install"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+ PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+
+<meta name="copyright" content="(C) Copyright 2018" />
+<meta name="DC.rights.owner" content="(C) Copyright 2018" />
+<meta name="DC.Type" content="concept" />
+<meta name="DC.Title" content="Installing Impala" />
+<meta name="prodname" content="Impala" />
+<meta name="prodname" content="Impala" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="DC.Format" content="XHTML" />
+<meta name="DC.Identifier" content="install" />
+<link rel="stylesheet" type="text/css" href="../commonltr.css" />
+<title>Installing Impala</title>
+</head>
+<body id="install">
+
<h1 class="title topictitle1" id="ariaid-title1"><span class="ph">Installing Impala</span></h1>
+
<div class="body conbody">
@@ -20,12 +39,14 @@
that returns rapid responses to queries.
</p>
+
<p class="p">
Follow these steps to set up Impala on a cluster by building from source:
</p>
+
<ul class="ul">
<li class="li">
<p class="p">
@@ -33,18 +54,24 @@
<a class="xref" href="http://impala.apache.org/downloads.html" target="_blank">the Impala downloads page</a>
for the link to the latest release.
</p>
+
</li>
+
<li class="li">
<p class="p">
Check the <span class="ph filepath">README.md</span> file for a pointer
to the build instructions.
</p>
+
</li>
+
<li class="li">
<p class="p">
Please check the MD5 and SHA1 and GPG signature, the latter by using the code signing keys of the release managers.
</p>
+
</li>
+
<li class="li">
<div class="p">
Developers interested in working on Impala can clone the Impala source repository:
@@ -52,15 +79,20 @@
git clone https://git-wip-us.apache.org/repos/asf/impala.git
</code></pre>
</div>
+
</li>
+
</ul>
+
</div>
- <article class="topic concept nested1" aria-labelledby="ariaid-title2" id="install__install_details">
+
+ <div class="topic concept nested1" aria-labelledby="ariaid-title2" id="install_details">
<h2 class="title topictitle2" id="ariaid-title2">What is Included in an Impala Installation</h2>
+
<div class="body conbody">
<p class="p">
@@ -69,10 +101,12 @@ git clone https://git-wip-us.apache.org/repos/asf/impala.git
most of the query processing work) on <em class="ph i">all</em> DataNodes in the cluster.
</p>
+
<p class="p">
Impala primarily consists of these executables, which should be available after you build from source:
</p>
+
<ul class="ul">
<li class="li">
<p class="p">
@@ -80,8 +114,10 @@ git clone https://git-wip-us.apache.org/repos/asf/impala.git
<a class="xref" href="impala_processes.html#processes">Run one impalad process</a> on each node in the cluster
that has a DataNode.
</p>
+
</li>
+
<li class="li">
<p class="p">
<span class="keyword cmdname">statestored</span> - Name service that tracks location and status of all
@@ -89,8 +125,10 @@ git clone https://git-wip-us.apache.org/repos/asf/impala.git
instance of this daemon</a> on a node in your cluster. Most production deployments run this daemon
on the namenode.
</p>
+
</li>
+
<li class="li">
<p class="p">
<span class="keyword cmdname">catalogd</span> - Metadata coordination service that broadcasts changes from Impala DDL and
@@ -104,8 +142,10 @@ git clone https://git-wip-us.apache.org/repos/asf/impala.git
<a class="xref" href="impala_processes.html#processes">Run one instance of this daemon</a> on a node in your cluster,
preferably on the same host as the <code class="ph codeph">statestored</code> daemon.
</p>
+
</li>
+
<li class="li">
<p class="p">
<span class="keyword cmdname">impala-shell</span> - <a class="xref" href="impala_impala_shell.html#impala_shell">Command-line
@@ -113,14 +153,21 @@ git clone https://git-wip-us.apache.org/repos/asf/impala.git
anywhere on your network, not necessarily DataNodes or even within the same cluster as Impala. It can
connect remotely to any instance of the Impala daemon.
</p>
+
</li>
+
</ul>
+
<p class="p">
Before starting working with Impala, ensure that you have all necessary prerequisites. See
<a class="xref" href="impala_prereqs.html#prereqs">Impala Requirements</a> for details.
</p>
+
</div>
- </article>
-</article></main></body></html>
\ No newline at end of file
+ </div>
+
+
+</body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/impala/blob/b4ad38a9/docs/build/html/topics/impala_int.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_int.html b/docs/build/html/topics/impala_int.html
index ee10109..5182fef 100644
--- a/docs/build/html/topics/impala_int.html
+++ b/docs/build/html/topics/impala_int.html
@@ -1,8 +1,28 @@
+<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html
- SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="DC.Relation" scheme="URI" content="../topics/impala_datatypes.html"><meta name="prodname" content="Impala"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="int"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>INT Data Type</title></head><body id="int"><main role="main"><article role="article" aria-labelledby="ariaid-title1">
+ PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+
+<meta name="copyright" content="(C) Copyright 2018" />
+<meta name="DC.rights.owner" content="(C) Copyright 2018" />
+<meta name="DC.Type" content="concept" />
+<meta name="DC.Title" content="INT Data Type" />
+<meta name="DC.Relation" scheme="URI" content="../topics/impala_datatypes.html" />
+<meta name="prodname" content="Impala" />
+<meta name="prodname" content="Impala" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="DC.Format" content="XHTML" />
+<meta name="DC.Identifier" content="int" />
+<link rel="stylesheet" type="text/css" href="../commonltr.css" />
+<title>INT Data Type</title>
+</head>
+<body id="int">
+
<h1 class="title topictitle1" id="ariaid-title1">INT Data Type</h1>
+
@@ -12,20 +32,24 @@
A 4-byte integer data type used in <code class="ph codeph">CREATE TABLE</code> and <code class="ph codeph">ALTER TABLE</code> statements.
</p>
+
<p class="p">
<strong class="ph b">Syntax:</strong>
</p>
+
<p class="p">
In the column definition of a <code class="ph codeph">CREATE TABLE</code> statement:
</p>
+
<pre class="pre codeblock"><code><var class="keyword varname">column_name</var> INT</code></pre>
<p class="p">
<strong class="ph b">Range:</strong> -2147483648 .. 2147483647. There is no <code class="ph codeph">UNSIGNED</code> subtype.
</p>
+
<p class="p">
<strong class="ph b">Conversions:</strong> Impala automatically converts to a larger integer type (<code class="ph codeph">BIGINT</code>) or a
floating-point type (<code class="ph codeph">FLOAT</code> or <code class="ph codeph">DOUBLE</code>) automatically. Use
@@ -40,33 +64,40 @@
</span>
</p>
+
<p class="p">
<strong class="ph b">Usage notes:</strong>
</p>
+
<p class="p">
The data type <code class="ph codeph">INTEGER</code> is an alias for <code class="ph codeph">INT</code>.
</p>
+
<p class="p">
For a convenient and automated way to check the bounds of the <code class="ph codeph">INT</code> type, call the functions
<code class="ph codeph">MIN_INT()</code> and <code class="ph codeph">MAX_INT()</code>.
</p>
+
<p class="p">
If an integer value is too large to be represented as a <code class="ph codeph">INT</code>, use a <code class="ph codeph">BIGINT</code>
instead.
</p>
+
<p class="p">
<strong class="ph b">NULL considerations:</strong> Casting any non-numeric value to this type produces a <code class="ph codeph">NULL</code>
value.
</p>
+
<p class="p">
<strong class="ph b">Examples:</strong>
</p>
+
<pre class="pre codeblock"><code>CREATE TABLE t1 (x INT);
SELECT CAST(1000 AS INT);
</code></pre>
@@ -76,14 +107,17 @@ SELECT CAST(1000 AS INT);
type more efficiently than a <code class="ph codeph">STRING</code> representation of the value.
</p>
+
<p class="p">
<strong class="ph b">HBase considerations:</strong> This data type is fully compatible with HBase tables.
</p>
+
<p class="p">
<strong class="ph b">Parquet considerations:</strong>
</p>
+
<p class="p">
<strong class="ph b">Text table considerations:</strong> Values of this type are potentially larger in text tables than in tables
using Parquet or other binary formats.
@@ -91,14 +125,17 @@ SELECT CAST(1000 AS INT);
+
<p class="p">
<strong class="ph b">Internal details:</strong> Represented in memory as a 4-byte value.
</p>
+
<p class="p">
<strong class="ph b">Added in:</strong> Available in all versions of Impala.
</p>
+
<p class="p">
<strong class="ph b">Column statistics considerations:</strong> Because this type has a fixed size, the maximum and average size
fields are always filled in for column statistics, even before you run the <code class="ph codeph">COMPUTE STATS</code>
@@ -107,15 +144,24 @@ SELECT CAST(1000 AS INT);
+
<p class="p">
<strong class="ph b">Related information:</strong>
</p>
+
<p class="p">
<a class="xref" href="impala_literals.html#numeric_literals">Numeric Literals</a>, <a class="xref" href="impala_tinyint.html#tinyint">TINYINT Data Type</a>,
<a class="xref" href="impala_smallint.html#smallint">SMALLINT Data Type</a>, <a class="xref" href="impala_int.html#int">INT Data Type</a>,
- <a class="xref" href="impala_bigint.html#bigint">BIGINT Data Type</a>, <a class="xref" href="impala_decimal.html#decimal">DECIMAL Data Type (Impala 1.4 or higher only)</a>,
+ <a class="xref" href="impala_bigint.html#bigint">BIGINT Data Type</a>, <a class="xref" href="impala_decimal.html#decimal">DECIMAL Data Type (Impala 3.0 or higher only)</a>,
<a class="xref" href="impala_math_functions.html#math_functions">Impala Mathematical Functions</a>
</p>
+
</div>
-<nav role="navigation" class="related-links"><div class="familylinks"><div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_datatypes.html">Data Types</a></div></div></nav></article></main></body></html>
\ No newline at end of file
+
+<div class="related-links">
+<div class="familylinks">
+<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_datatypes.html">Data Types</a></div>
+</div>
+</div></body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/impala/blob/b4ad38a9/docs/build/html/topics/impala_intro.html
----------------------------------------------------------------------
diff --git a/docs/build/html/topics/impala_intro.html b/docs/build/html/topics/impala_intro.html
index 62e5c50..7f96ec7 100644
--- a/docs/build/html/topics/impala_intro.html
+++ b/docs/build/html/topics/impala_intro.html
@@ -1,11 +1,28 @@
+<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html
- SYSTEM "about:legacy-compat">
-<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta charset="UTF-8"><meta name="copyright" content="(C) Copyright 2018"><meta name="DC.rights.owner" content="(C) Copyright 2018"><meta name="DC.Type" content="concept"><meta name="prodname" content="Impala"><meta name="version" content="Impala 2.12x"><meta name="DC.Format" content="XHTML"><meta name="DC.Identifier" content="intro"><link rel="stylesheet" type="text/css" href="../commonltr.css"><title>Introducing Apache Impala</title></head><body id="intro"><main role="main"><article role="article" aria-labelledby="intro__impala">
+ PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+
+<meta name="copyright" content="(C) Copyright 2018" />
+<meta name="DC.rights.owner" content="(C) Copyright 2018" />
+<meta name="DC.Type" content="concept" />
+<meta name="DC.Title" content="Introducing Apache Impala" />
+<meta name="prodname" content="Impala" />
+<meta name="version" content="Impala 3.0.x" />
+<meta name="DC.Format" content="XHTML" />
+<meta name="DC.Identifier" content="intro" />
+<link rel="stylesheet" type="text/css" href="../commonltr.css" />
+<title>Introducing Apache Impala</title>
+</head>
+<body id="intro">
+
<h1 class="title topictitle1" id="intro__impala"><span class="ph">Introducing Apache Impala</span></h1>
+
- <div class="body conbody" id="intro__intro_body">
+ <div class="body conbody" id="intro_body">
<p class="p">
Impala provides fast, interactive SQL queries directly on your Apache Hadoop data stored in HDFS,
@@ -16,6 +33,7 @@
provides a familiar and unified platform for real-time or batch-oriented queries.
</p>
+
<p class="p">
Impala is an addition to tools available for querying big data. Impala does not replace the batch
processing frameworks built on MapReduce such as Hive. Hive and other frameworks built on MapReduce are
@@ -23,18 +41,22 @@
and Load (ETL) type jobs.
</p>
- <div class="note note note_note"><span class="note__title notetitle">Note:</span>
+
+ <div class="note note"><span class="notetitle">Note:</span>
Impala graduated from the Apache Incubator on November 15, 2017.
In places where the documentation formerly referred to <span class="q">"Cloudera Impala"</span>,
now the official name is <span class="q">"Apache Impala"</span>.
</div>
+
</div>
- <article class="topic concept nested1" aria-labelledby="ariaid-title2" id="intro__benefits">
+
+ <div class="topic concept nested1" aria-labelledby="ariaid-title2" id="benefits">
<h2 class="title topictitle2" id="ariaid-title2">Impala Benefits</h2>
+
<div class="body conbody">
<div class="p">
@@ -45,34 +67,44 @@
Familiar SQL interface that data scientists and analysts already know.
</li>
+
<li class="li">
Ability to query high volumes of data (<span class="q">"big data"</span>) in Apache Hadoop.
</li>
+
<li class="li">
Distributed queries in a cluster environment, for convenient scaling and to make use of cost-effective
commodity hardware.
</li>
+
<li class="li">
Ability to share data files between different components with no copy or export/import step; for example,
to write with Pig, transform with Hive and query with Impala. Impala can read from and write to Hive
tables, enabling simple data interchange using Impala for analytics on Hive-produced data.
</li>
+
<li class="li">
Single system for big data processing and analytics, so customers can avoid costly modeling and ETL just
for analytics.
</li>
+
</ul>
+
</div>
+
</div>
- </article>
- <article class="topic concept nested1" aria-labelledby="ariaid-title3" id="intro__impala_hadoop">
+ </div>
+
+
+ <div class="topic concept nested1" aria-labelledby="ariaid-title3" id="impala_hadoop">
<h2 class="title topictitle2" id="ariaid-title3">How Impala Works with <span class="keyword">Apache Hadoop</span></h2>
+
<div class="body conbody">
@@ -88,6 +120,7 @@
as connecting to Impala.
</li>
+
<li class="li">
Hive Metastore - Stores information about the data available to Impala. For example, the metastore lets
Impala know what databases are available and what the structure of those databases is. As you create,
@@ -96,6 +129,7 @@
service introduced in Impala 1.2.
</li>
+
<li class="li">
Impala - This process, which runs on DataNodes, coordinates and executes queries. Each
instance of Impala can receive, plan, and coordinate queries from Impala clients. Queries are
@@ -103,12 +137,16 @@
fragments.
</li>
+
<li class="li">
HBase and HDFS - Storage for data to be queried.
</li>
+
</ul>
+
</div>
+
<div class="p">
Queries executed using Impala are handled as follows:
<ol class="ol">
@@ -118,29 +156,38 @@
<code class="ph codeph">impalad</code> becomes the coordinator for the query.
</li>
+
<li class="li">
Impala parses the query and analyzes it to determine what tasks need to be performed by
<code class="ph codeph">impalad</code> instances across the cluster. Execution is planned for optimal efficiency.
</li>
+
<li class="li">
Services such as HDFS and HBase are accessed by local <code class="ph codeph">impalad</code> instances to provide
data.
</li>
+
<li class="li">
Each <code class="ph codeph">impalad</code> returns data to the coordinating <code class="ph codeph">impalad</code>, which sends
these results to the client.
</li>
+
</ol>
+
</div>
+
</div>
- </article>
- <article class="topic concept nested1" aria-labelledby="ariaid-title4" id="intro__features">
+ </div>
+
+
+ <div class="topic concept nested1" aria-labelledby="ariaid-title4" id="features">
<h2 class="title topictitle2" id="ariaid-title4">Primary Impala Features</h2>
+
<div class="body conbody">
<div class="p">
@@ -153,6 +200,7 @@
<a class="xref" href="../shared/../topics/impala_aggregate_functions.html#aggregate_functions">aggregate functions</a>.
</li>
+
<li class="li">
HDFS, HBase, <span class="ph">and Amazon Simple Storage System (S3)</span> storage, including:
<ul class="ul">
@@ -161,12 +209,16 @@
Avro, SequenceFile, and RCFile.
</li>
+
<li class="li">
Compression codecs: Snappy, GZIP, Deflate, BZIP.
</li>
+
</ul>
+
</li>
+
<li class="li">
Common data access interfaces including:
<ul class="ul">
@@ -174,25 +226,37 @@
<a class="xref" href="../shared/../topics/impala_jdbc.html#impala_jdbc">JDBC driver</a>.
</li>
+
<li class="li">
<a class="xref" href="../shared/../topics/impala_odbc.html#impala_odbc">ODBC driver</a>.
</li>
+
<li class="li">
Hue Beeswax and the Impala Query UI.
</li>
+
</ul>
+
</li>
+
<li class="li">
<a class="xref" href="../shared/../topics/impala_impala_shell.html#impala_shell">impala-shell command-line interface</a>.
</li>
+
<li class="li">
<a class="xref" href="../shared/../topics/impala_security.html#security">Kerberos authentication</a>.
</li>
+
</ul>
+
</div>
+
</div>
- </article>
-</article></main></body></html>
\ No newline at end of file
+
+ </div>
+
+</body>
+</html>
\ No newline at end of file