You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by mi...@apache.org on 2018/11/20 16:28:38 UTC
[1/2] impala git commit: IMPALA-7861: [DOCS] TLS enabled by default
regardless of URI scheme
Repository: impala
Updated Branches:
refs/heads/master 8f151d0e6 -> a78a8d62a
IMPALA-7861: [DOCS] TLS enabled by default regardless of URI scheme
Change-Id: I88f615cf23f406035e544e68adacdd0393f69ab3
Reviewed-on: http://gerrit.cloudera.org:8080/11948
Reviewed-by: Joe McDonnell <jo...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Project: http://git-wip-us.apache.org/repos/asf/impala/repo
Commit: http://git-wip-us.apache.org/repos/asf/impala/commit/09c25144
Tree: http://git-wip-us.apache.org/repos/asf/impala/tree/09c25144
Diff: http://git-wip-us.apache.org/repos/asf/impala/diff/09c25144
Branch: refs/heads/master
Commit: 09c25144fddcfc9b72c9c4914e41675f865857ac
Parents: 8f151d0
Author: Alex Rodoni <ar...@cloudera.com>
Authored: Fri Nov 16 15:20:49 2018 -0800
Committer: Alex Rodoni <ar...@cloudera.com>
Committed: Tue Nov 20 01:18:29 2018 +0000
----------------------------------------------------------------------
docs/topics/impala_adls.xml | 66 ++++++++++++++++++++++++++--------------
1 file changed, 43 insertions(+), 23 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/impala/blob/09c25144/docs/topics/impala_adls.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_adls.xml b/docs/topics/impala_adls.xml
index f5103f4..d50b011 100644
--- a/docs/topics/impala_adls.xml
+++ b/docs/topics/impala_adls.xml
@@ -35,12 +35,14 @@ under the License.
<conbody>
- <p> You can use Impala to query data residing on the Azure Data Lake Store
+ <p>
+ You can use Impala to query data residing on the Azure Data Lake Store
(ADLS) filesystem. This capability allows convenient access to a storage
system that is remotely managed, accessible from anywhere, and integrated
with various cloud-based services. Impala can query files in any supported
file format from ADLS. The ADLS storage location can be for an entire
- table, or individual partitions in a partitioned table. </p>
+ table or individual partitions in a partitioned table.
+ </p>
<p>
The default Impala tables use data files stored on HDFS, which are ideal for bulk loads and queries using
@@ -89,14 +91,12 @@ under the License.
<ul>
<li><p> The <xref href="impala_create_table.xml#create_table"/> or <xref
href="impala_alter_table.xml#alter_table"/> statements can specify
- that a table resides on the ADLS filesystem by specifying an ADLS
- prefix for the <codeph>LOCATION</codeph> property.<ul>
- <li><codeph>adl://</codeph> for ADLS Gen1</li>
- <li><codeph>abfs://</codeph> for ADLS Gen2</li>
- <li><codeph>abfss://</codeph> for ADLS Gen2 with a secure socket
- layer connection</li>
- </ul>
- <codeph>ALTER TABLE</codeph> can also set the
+ that a table resides on the ADLS filesystem by using one of the
+ following ADLS prefixes in the <codeph>LOCATION</codeph> property.<ul>
+ <li>For ADLS Gen1: <codeph>adl://</codeph></li>
+ <li>For ADLS Gen2: <codeph>abfs://</codeph> or
+ <codeph>abfss://</codeph></li>
+ </ul></p><p><codeph>ALTER TABLE</codeph> can also set the
<codeph>LOCATION</codeph> property for an individual partition, so
that some data in a table resides on ADLS and other data in the same
table resides on HDFS. </p> See <xref href="impala_adls.xml#ddl"/>
@@ -269,24 +269,44 @@ under the License.
<conbody>
- <p> Impala reads data for a table or partition from ADLS based on the
+ <p>
+ Impala reads data for a table or partition from ADLS based on the
<codeph>LOCATION</codeph> attribute for the table or partition.
Specify the ADLS details in the <codeph>LOCATION</codeph> clause of a
<codeph>CREATE TABLE</codeph> or <codeph>ALTER TABLE</codeph>
- statement. The syntax for the <codeph>LOCATION</codeph> clause is:<ul>
- <li>For ADLS Gen1,
- <codeph>adl://<varname>account</varname>.azuredatalakestore.net/<varname>path/file</varname></codeph>
+ statement. The syntax for the <codeph>LOCATION</codeph> clause is:
+ <ul>
+ <li>
+ For ADLS Gen1:
+<codeblock>adl://<varname>account</varname>.azuredatalakestore.net/<varname>path/file</varname></codeblock></li>
+ <li>
+ For ADLS Gen2:
+<codeblock>abfs://<varname>container</varname>@<varname>account</varname>.dfs.core.windows.net/<varname>path</varname>/<varname>file</varname></codeblock>
+ <p>
+ or
+ </p>
+<codeblock>abfss://<varname>container</varname>@<varname>account</varname>.dfs.core.windows.net/<varname>path</varname>/<varname>file</varname></codeblock>
</li>
- <li>For ADLS Gen2,
- <codeph>abfs://<varname>container</varname>@<varname>account</varname>.dfs.core.windows.net/<varname>path</varname>/<varname>file</varname></codeph></li>
- <li>For ADLS Gen2 with a secure socket layer connection,
- <codeph>abfss://<varname>container</varname>@<varname>account</varname>.dfs.core.windows.net/<varname>path</varname>/<varname>file</varname></codeph></li>
- </ul></p>
- <p><codeph><varname>container</varname></codeph> denotes the parent
+ </ul>
+ </p>
+ <p>
+ <codeph><varname>container</varname></codeph> denotes the parent
location that holds the files and folders, which is the Containers in
- the Azure Storage Blobs service.</p>
- <p><codeph><varname>account</varname></codeph> is the name given for your
- storage account.</p>
+ the Azure Storage Blobs service.
+ </p>
+ <p>
+ <codeph><varname>account</varname></codeph> is the name given for your
+ storage account.
+ </p>
+ <note>
+ <p> By default, TLS is enabled both with <codeph>abfs://</codeph> and
+ <codeph>abfss://</codeph>. </p>
+ <p>
+ When you set the <codeph>fs.azure.always.use.https=false</codeph>
+ property, TLS is disabled with <codeph>abfs://</codeph>, and TLS is
+ enabled with <codeph>abfss://</codeph>
+ </p>
+ </note>
<p>
For a partitioned table, either specify a separate <codeph>LOCATION</codeph> clause for each new partition,
[2/2] impala git commit: IMPALA-5031: Fix use of uninitialised var in
SendTopicUpdate()
Posted by mi...@apache.org.
IMPALA-5031: Fix use of uninitialised var in SendTopicUpdate()
When SendTopicUpdate() returns an error, it does not set update_skipped,
so the variable is uninitialised. This means that the duration between
topic updates will depend on the uninitialised value.
Change-Id: I373c6f65854244fe90889412b38b0260fe1d1f13
Reviewed-on: http://gerrit.cloudera.org:8080/11957
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Project: http://git-wip-us.apache.org/repos/asf/impala/repo
Commit: http://git-wip-us.apache.org/repos/asf/impala/commit/a78a8d62
Tree: http://git-wip-us.apache.org/repos/asf/impala/tree/a78a8d62
Diff: http://git-wip-us.apache.org/repos/asf/impala/diff/a78a8d62
Branch: refs/heads/master
Commit: a78a8d62ae7a7fe6800c4eec93eed0e082a3ff07
Parents: 09c2514
Author: Tim Armstrong <ta...@cloudera.com>
Authored: Mon Nov 19 11:52:00 2018 -0800
Committer: Impala Public Jenkins <im...@cloudera.com>
Committed: Tue Nov 20 01:50:58 2018 +0000
----------------------------------------------------------------------
be/src/statestore/statestore.cc | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/impala/blob/a78a8d62/be/src/statestore/statestore.cc
----------------------------------------------------------------------
diff --git a/be/src/statestore/statestore.cc b/be/src/statestore/statestore.cc
index 70183d7..234c63c 100644
--- a/be/src/statestore/statestore.cc
+++ b/be/src/statestore/statestore.cc
@@ -930,7 +930,9 @@ void Statestore::DoSubscriberUpdate(UpdateKind update_kind, int thread_id,
deadline_ms = UnixMillis() + FLAGS_statestore_heartbeat_frequency_ms;
} else {
- bool update_skipped;
+ // Initialize to false so that we don't consider the update skipped when
+ // SendTopicUpdate() fails.
+ bool update_skipped = false;
status = SendTopicUpdate(subscriber.get(), update_kind, &update_skipped);
if (status.code() == TErrorCode::RPC_RECV_TIMEOUT) {
// Add details to status to make it more useful, while preserving the stack