You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by jo...@apache.org on 2019/11/13 22:10:07 UTC
[impala] branch master updated (e0a98df -> f8c8fa5)
This is an automated email from the ASF dual-hosted git repository.
joemcdonnell pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git.
from e0a98df IMPALA-2112: Support primary key/foreign key constraints as part of create table in Impala.
new 186bc64 IMPALA-7506: [DOCS] Global INVALIDATE is supported in local catalog mode
new 4c5015d IMPALA-8855: [DOCS] Document the generic VALUES clause
new f8c8fa5 IMPALA-9150: Use HBase's stop-hbase.sh script for minicluster
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs/impala.ditamap | 1 +
docs/topics/impala_datatypes.xml | 53 ++++++++-------
docs/topics/impala_insert.xml | 35 ----------
docs/topics/impala_langref_unsupported.xml | 51 +++++++--------
docs/topics/impala_metadata.xml | 11 ----
docs/topics/impala_values.xml | 101 +++++++++++++++++++++++++++++
testdata/bin/create-load-data.sh | 5 ++
testdata/bin/kill-hbase.sh | 6 +-
8 files changed, 165 insertions(+), 98 deletions(-)
create mode 100644 docs/topics/impala_values.xml
[impala] 01/03: IMPALA-7506: [DOCS] Global INVALIDATE is supported
in local catalog mode
Posted by jo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
joemcdonnell pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 186bc64652dc386417716ae0c3b47c8bf078141c
Author: Alex Rodoni <ar...@cloudera.com>
AuthorDate: Tue Nov 12 15:23:13 2019 -0800
IMPALA-7506: [DOCS] Global INVALIDATE is supported in local catalog mode
Change-Id: I30ca29995a1b6667e2738803fc0a0639f8f08fe9
Reviewed-on: http://gerrit.cloudera.org:8080/14700
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Quanlong Huang <hu...@gmail.com>
---
docs/topics/impala_metadata.xml | 11 -----------
1 file changed, 11 deletions(-)
diff --git a/docs/topics/impala_metadata.xml b/docs/topics/impala_metadata.xml
index e810a54..fb5e856 100644
--- a/docs/topics/impala_metadata.xml
+++ b/docs/topics/impala_metadata.xml
@@ -127,15 +127,6 @@ under the License.
</dl>
</p>
- <p>
- <b>Limitation:</b>
- </p>
-
- <p>
- Global <codeph>INVALIDATES</codeph> are not supported when this feature is enabled. If
- your workload requires global <codeph>INVALIDATES</codeph>, do not use this feature.
- </p>
-
</conbody>
</concept>
@@ -454,8 +445,6 @@ under the License.
<title>Metrics for Event Based Automatic Metadata Sync</title>
-<!--AR 3/29/2019 Incorporate this section to the web UI doc when this feature goes GA.-->
-
<conbody>
<p>
[impala] 03/03: IMPALA-9150: Use HBase's stop-hbase.sh script for
minicluster
Posted by jo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
joemcdonnell pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit f8c8fa5b454b7c56c8383dd103a6f6b87d231327
Author: Joe McDonnell <jo...@cloudera.com>
AuthorDate: Tue Nov 12 11:56:19 2019 -0800
IMPALA-9150: Use HBase's stop-hbase.sh script for minicluster
testdata/bin/kill-hbase.sh currently uses the generic
kill-java-service.sh script to kill the region servers,
then the master, and then the zookeeper. Recent versions
of HBase become unusable after performing this type of
shutdown. The master seems to get stuck trying to recover,
even after restarting the minicluster.
The root cause in HBase is unclear, but HBase provides the
stop-hbase.sh script, which does a more graceful shutdown.
This switches tesdata/bin/kill-hbase.sh to use this script,
which avoids the recovery problems.
Testing:
- Ran the test-with-docker.py tests (which does a minicluster
restart). Before the change, the HBase tests timed out due
to HBase getting stuck recovering. After the change, tests
ran normally.
- Added a minicluster restart after dataload so that this
is tested.
Change-Id: I67283f9098c73c849023af8bfa7af62308bf3ed3
Reviewed-on: http://gerrit.cloudera.org:8080/14697
Reviewed-by: Vihang Karajgaonkar <vi...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
testdata/bin/create-load-data.sh | 5 +++++
testdata/bin/kill-hbase.sh | 6 ++++--
2 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/testdata/bin/create-load-data.sh b/testdata/bin/create-load-data.sh
index a081280..6798567 100755
--- a/testdata/bin/create-load-data.sh
+++ b/testdata/bin/create-load-data.sh
@@ -699,3 +699,8 @@ fi
if [[ $SKIP_RANGER -eq 0 ]]; then
run-step "Setting up Ranger" setup-ranger.log setup-ranger
fi
+
+# Restart the minicluster. This is strictly to provide a sanity check that
+# restarting the minicluster works and doesn't impact the tests. This is a common
+# operation for developers, so it is nice to test it.
+restart-cluster
diff --git a/testdata/bin/kill-hbase.sh b/testdata/bin/kill-hbase.sh
index 0a5fcda..902e869 100755
--- a/testdata/bin/kill-hbase.sh
+++ b/testdata/bin/kill-hbase.sh
@@ -23,8 +23,10 @@ setup_report_build_error
DIR=$(dirname "$0")
echo Stopping Hbase
-# Kill region server first, then hmaster, and zookeeper.
-"$DIR"/kill-java-service.sh -c HRegionServer -c HMaster -c HQuorumPeer -s 2
+# Use the stop-hbase.sh script provided by HBase. This does a more graceful shutdown
+# that using our kill-java-service.sh script.
+${HBASE_HOME}/bin/stop-hbase.sh
# Clear up data so that zookeeper/hbase won't do recovery when it starts.
+# TODO: is this still needed when using bin/stop-hbase.sh?
rm -rf /tmp/hbase-*
[impala] 02/03: IMPALA-8855: [DOCS] Document the generic VALUES
clause
Posted by jo...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
joemcdonnell pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 4c5015d5a2d7ebbb85afae35d0ca94f4961af32d
Author: Alex Rodoni <ar...@cloudera.com>
AuthorDate: Thu Nov 7 14:47:55 2019 -0800
IMPALA-8855: [DOCS] Document the generic VALUES clause
- Added a paragraph on implicit conversion in impala_datatypes.xml
Change-Id: I2568450993323236535a8f1d022dee7d09ecf62b
Reviewed-on: http://gerrit.cloudera.org:8080/14661
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
---
docs/impala.ditamap | 1 +
docs/topics/impala_datatypes.xml | 53 ++++++++-------
docs/topics/impala_insert.xml | 35 ----------
docs/topics/impala_langref_unsupported.xml | 51 +++++++--------
docs/topics/impala_values.xml | 101 +++++++++++++++++++++++++++++
5 files changed, 156 insertions(+), 85 deletions(-)
diff --git a/docs/impala.ditamap b/docs/impala.ditamap
index 8e11b89..ec8e03a 100644
--- a/docs/impala.ditamap
+++ b/docs/impala.ditamap
@@ -258,6 +258,7 @@ under the License.
<topicref href="topics/impala_update.xml"/>
<topicref href="topics/impala_upsert.xml"/>
<topicref href="topics/impala_use.xml"/>
+ <topicref href="topics/impala_values.xml"/>
<topicref href="topics/impala_hints.xml"/>
</topicref>
<topicref href="topics/impala_functions.xml">
diff --git a/docs/topics/impala_datatypes.xml b/docs/topics/impala_datatypes.xml
index 8076df6..a1bfe18 100644
--- a/docs/topics/impala_datatypes.xml
+++ b/docs/topics/impala_datatypes.xml
@@ -34,28 +34,37 @@ under the License.
</prolog>
<conbody>
-
- <p>
- <indexterm audience="hidden">data types</indexterm>
- Impala supports a set of data types that you can use for table columns, expression values, and function
- arguments and return values.
- </p>
-
- <note>
- Currently, Impala supports only scalar types, not composite or nested types. Accessing a table containing any
- columns with unsupported types causes an error.
- </note>
-
+ <p>Impala supports a set of data types that you can use for table columns,
+ expression values, and function arguments and return values. </p>
+ <note> Currently, Impala supports only scalar types, not composite or nested
+ types. Accessing a table containing any columns with unsupported types
+ causes an error. </note>
<p outputclass="toc"/>
-
- <p>
- For the notation to write literals of each of these data types, see
- <xref href="impala_literals.xml#literals"/>.
- </p>
-
- <p>
- See <xref href="impala_langref_unsupported.xml#langref_hiveql_delta"/> for differences between Impala and
- Hive data types.
- </p>
+ <p> For the notation to write literals of each of these data types, see
+ <xref href="impala_literals.xml#literals"/>. </p>
+
+ <p> Impala supports a limited set of implicit casts to avoid undesired
+ results from unexpected casting behavior. </p>
+ <ul>
+ <li> Impala does not implicitly cast between string and numeric or Boolean
+ types. Always use <codeph>CAST()</codeph> for these conversions. </li>
+ <li> Impala does perform implicit casts among the numeric types, when
+ going from a smaller or less precise type to a larger or more precise
+ one. For example, Impala will implicitly convert a
+ <codeph>SMALLINT</codeph> to a <codeph>BIGINT</codeph> or
+ <codeph>FLOAT</codeph>, but to convert from <codeph>DOUBLE</codeph> to
+ <codeph>FLOAT</codeph> or <codeph>INT</codeph> to
+ <codeph>TINYINT</codeph> requires a call to <codeph>CAST()</codeph> in
+ the query. </li>
+ <li> Impala does perform implicit casts from <codeph>STRING</codeph> to
+ <codeph>TIMESTAMP</codeph>. Impala has a restricted set of literal
+ formats for the <codeph>TIMESTAMP</codeph> data type and the
+ <codeph>FROM_UNIXTIME()</codeph> format string; see <xref
+ href="impala_timestamp.xml#timestamp"/> for details. </li>
+ </ul>
+ <p>See the topics under this section for full details on implicit and
+ explicit casting for each data type, and see <xref
+ href="impala_conversion_functions.xml#conversion_functions"/> for
+ details about the <codeph>CAST()</codeph> function.</p>
</conbody>
</concept>
diff --git a/docs/topics/impala_insert.xml b/docs/topics/impala_insert.xml
index 58b5169..c1db6d5 100644
--- a/docs/topics/impala_insert.xml
+++ b/docs/topics/impala_insert.xml
@@ -504,41 +504,6 @@ Inserted 2 rows in 0.16s
<codeph>NULL</codeph>.
</p>
-<!--
- <p>
- does not support specifying a subset of the
- columns in the table or specifying the columns in a different order. Use a
- <codeph>VALUES</codeph> clause with all the column values in the same order as
- the table definition, using <codeph>NULL</codeph> values for any columns you
- want to omit from the <codeph>INSERT</codeph> operation.
- </p>
--->
-
- <p>
- To use a <codeph>VALUES</codeph> clause like a table in other statements, wrap it in parentheses and use
- <codeph>AS</codeph> clauses to specify aliases for the entire object and any columns you need to refer to:
- </p>
-
-<codeblock>[localhost:21000] > select * from (values(4,5,6),(7,8,9)) as t;
-+---+---+---+
-| 4 | 5 | 6 |
-+---+---+---+
-| 4 | 5 | 6 |
-| 7 | 8 | 9 |
-+---+---+---+
-[localhost:21000] > select * from (values(1 as c1, true as c2, 'abc' as c3),(100,false,'xyz')) as t;
-+-----+-------+-----+
-| c1 | c2 | c3 |
-+-----+-------+-----+
-| 1 | true | abc |
-| 100 | false | xyz |
-+-----+-------+-----+</codeblock>
-
- <p>
- For example, you might use a tiny table constructed like this from constant literals or function return
- values as part of a longer statement involving joins or <codeph>UNION ALL</codeph>.
- </p>
-
<p conref="../shared/impala_common.xml#common/hdfs_blurb"/>
<p>
diff --git a/docs/topics/impala_langref_unsupported.xml b/docs/topics/impala_langref_unsupported.xml
index 9746827..af81a90 100644
--- a/docs/topics/impala_langref_unsupported.xml
+++ b/docs/topics/impala_langref_unsupported.xml
@@ -260,34 +260,29 @@ under the License.
</p>
<ul>
- <li>
- Impala supports a limited set of implicit casts. This can help avoid undesired results from unexpected
- casting behavior.
- <ul>
- <li>
- Impala does not implicitly cast between string and numeric or Boolean types. Always use
- <codeph>CAST()</codeph> for these conversions.
- </li>
-
- <li>
- Impala does perform implicit casts among the numeric types, when going from a smaller or less precise
- type to a larger or more precise one. For example, Impala will implicitly convert a
- <codeph>SMALLINT</codeph> to a <codeph>BIGINT</codeph> or <codeph>FLOAT</codeph>, but to convert from
- <codeph>DOUBLE</codeph> to <codeph>FLOAT</codeph> or <codeph>INT</codeph> to <codeph>TINYINT</codeph>
- requires a call to <codeph>CAST()</codeph> in the query.
- </li>
-
- <li>
- Impala does perform implicit casts from string to timestamp. Impala has a restricted set of literal
- formats for the <codeph>TIMESTAMP</codeph> data type and the <codeph>from_unixtime()</codeph> format
- string; see <xref href="impala_timestamp.xml#timestamp"/> for details.
- </li>
- </ul>
- <p>
- See <xref href="impala_datatypes.xml#datatypes"/> for full details on implicit and explicit casting for
- all types, and <xref href="impala_conversion_functions.xml#conversion_functions"/> for details about
- the <codeph>CAST()</codeph> function.
- </p>
+ <li> Impala supports a limited set of implicit casts. This can help
+ avoid undesired results from unexpected casting behavior. <ul>
+ <li> Impala does not implicitly cast between string and numeric or
+ Boolean types. Always use <codeph>CAST()</codeph> for these
+ conversions. </li>
+ <li> Impala does perform implicit casts among the numeric types,
+ when going from a smaller or less precise type to a larger or more
+ precise one. For example, Impala will implicitly convert a
+ <codeph>SMALLINT</codeph> to a <codeph>BIGINT</codeph> or
+ <codeph>FLOAT</codeph>, but to convert from
+ <codeph>DOUBLE</codeph> to <codeph>FLOAT</codeph> or
+ <codeph>INT</codeph> to <codeph>TINYINT</codeph> requires a call
+ to <codeph>CAST()</codeph> in the query. </li>
+ <li> Impala does perform implicit casts from string to timestamp.
+ Impala has a restricted set of literal formats for the
+ <codeph>TIMESTAMP</codeph> data type and the
+ <codeph>from_unixtime()</codeph> format string; see <xref
+ href="impala_timestamp.xml#timestamp"/> for details. </li>
+ </ul><p> See the topics under <xref
+ href="impala_datatypes.xml#datatypes"/> for full details on
+ implicit and explicit casting for each data type, and <xref
+ href="impala_conversion_functions.xml#conversion_functions"/> for
+ details about the <codeph>CAST()</codeph> function. </p>
</li>
<li>
diff --git a/docs/topics/impala_values.xml b/docs/topics/impala_values.xml
new file mode 100644
index 0000000..a32f1c1
--- /dev/null
+++ b/docs/topics/impala_values.xml
@@ -0,0 +1,101 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<!DOCTYPE concept PUBLIC "-//OASIS//DTD DITA Concept//EN" "concept.dtd">
+<concept id="values">
+
+ <title>VALUES Statement</title>
+ <titlealts audience="PDF"><navtitle>VALUES</navtitle></titlealts>
+ <prolog>
+ <metadata>
+ <data name="Category" value="Impala"/>
+ <data name="Category" value="SQL"/>
+ <data name="Category" value="Developers"/>
+ <data name="Category" value="Data Analysts"/>
+ </metadata>
+ </prolog>
+
+ <conbody>
+ <p> In addition to being part of the <codeph>INSERT</codeph> statement, the
+ <codeph>VALUES</codeph> clause can be used as stand-alone statement or
+ with the <codeph>SELECT</codeph> statement to construct a data set without
+ creating a table. For example, the following statement returns a data set
+ of 2 rows and 3 columns.
+ <codeblock>
+VALUES ('r1_c1', 'r1_c2', 'r1_c3')
+ , ('r2_c1', 'r2_c2', 'r2_c3');</codeblock></p>
+ <p><b>Syntax:</b></p>
+ <codeblock>VALUES (<varname>row</varname>)[, (<varname>row</varname>), ...];
+
+SELECT <varname>select_list</varname> FROM (VALUES (<varname>row</varname>)[, (<varname>row</varname>), ...]) AS <varname>alias</varname>;
+
+<varname>row</varname> ::= <varname>column</varname> [[AS <varname>alias</varname>], <varname>column</varname> [AS <varname>alias</varname>], ...]</codeblock>
+ <ul>
+ <li>The <codeph>VALUES</codeph> keyword is followed by a comma separated
+ list of one or more <varname>row</varname>s.</li>
+ <li><varname>row</varname> is a comma-separated list of one or more
+ <varname>column</varname>s.</li>
+ <li>Each <varname>row</varname> must have the same number of
+ <varname>column</varname>s.</li>
+ <li><varname>column</varname> can be a constant, a variable, or an
+ expression.</li>
+ <li>The corresponding <varname>column</varname>s must have compatible data
+ types in all <varname>row</varname>s. See the third query in the
+ Examples section below.</li>
+ <li>By default, the first row is used to name columns. But using the
+ <codeph>AS</codeph> keyword, you can optionally give the column an
+ <varname>alias</varname>. </li>
+ <li>If used in the <codeph>SELECT</codeph> statement, the
+ <codeph>AS</codeph> keyword with an <varname>alias</varname> is
+ required.</li>
+ <li>
+ <varname>select_list</varname> is the columns to be selected for the
+ result set.</li>
+ </ul>
+ <p><b>Examples:</b></p>
+ <p>
+ <codeblock>> SELECT * FROM (VALUES(4,5,6),(7,8,9)) AS t;
++---+---+---+
+| 4 | 5 | 6 |
++---+---+---+
+| 4 | 5 | 6 |
+| 7 | 8 | 9 |
++---+---+---+
+
+> SELECT * FROM (VALUES(1 AS c1, true AS c2, 'abc' AS c3),(100,false,'xyz')) AS t;
++-----+-------+-----+
+| c1 | c2 | c3 |
++-----+-------+-----+
+| 1 | true | abc |
+| 100 | false | xyz |
++-----+-------+-----+
+
+> VALUES (CAST('2019-01-01' AS TIMESTAMP)), ('2019-02-02');
++---------------------------------+
+| cast('2019-01-01' as timestamp) |
++---------------------------------+
+| 2019-01-01 00:00:00 |
+| 2019-02-02 00:00:00 |
++---------------------------------+</codeblock>
+ </p>
+ <p><b>Related information:</b></p>
+ <p>
+ <xref href="impala_select.xml#select"/></p>
+ </conbody>
+</concept>