You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by mi...@apache.org on 2018/08/03 21:23:32 UTC
[4/7] impala git commit: [DOCS] Added the part 1 of IMPALA-5607 to
the upgrade guide
[DOCS] Added the part 1 of IMPALA-5607 to the upgrade guide
Change-Id: Ib0e53959bef4c629a31868e16b03b6abc11c9f8d
Reviewed-on: http://gerrit.cloudera.org:8080/11051
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
Project: http://git-wip-us.apache.org/repos/asf/impala/repo
Commit: http://git-wip-us.apache.org/repos/asf/impala/commit/479c83cf
Tree: http://git-wip-us.apache.org/repos/asf/impala/tree/479c83cf
Diff: http://git-wip-us.apache.org/repos/asf/impala/diff/479c83cf
Branch: refs/heads/master
Commit: 479c83cf72f82b86fc08bde93a7910bf456dcae1
Parents: d0ec011
Author: Alex Rodoni <ar...@cloudera.com>
Authored: Wed Jul 25 13:44:14 2018 -0700
Committer: Alex Rodoni <ar...@cloudera.com>
Committed: Fri Aug 3 02:38:27 2018 +0000
----------------------------------------------------------------------
docs/topics/impala_upgrading.xml | 344 ++++++++++++++++++++++++----------
1 file changed, 245 insertions(+), 99 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/impala/blob/479c83cf/docs/topics/impala_upgrading.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_upgrading.xml b/docs/topics/impala_upgrading.xml
index 7bfc0a4..679b77d 100644
--- a/docs/topics/impala_upgrading.xml
+++ b/docs/topics/impala_upgrading.xml
@@ -21,6 +21,7 @@ under the License.
<concept id="upgrading">
<title>Upgrading Impala</title>
+
<prolog>
<metadata>
<data name="Category" value="Impala"/>
@@ -32,11 +33,11 @@ under the License.
<conbody>
<p>
- Upgrading Impala involves building or acquiring new Impala-related binaries,
- and then restarting Impala services.
+ Upgrading Impala involves building or acquiring new Impala-related binaries, and then
+ restarting Impala services.
</p>
- </conbody>
+ </conbody>
<concept id="upgrade_manual">
@@ -47,8 +48,7 @@ under the License.
<ul>
<li>
<p>
- Shut down all Impala-related daemons on
- all relevant hosts in the cluster:
+ Shut down all Impala-related daemons on all relevant hosts in the cluster:
</p>
<ol>
<li>
@@ -67,47 +67,52 @@ under the License.
</li>
</ol>
</li>
+
<li>
<p>
- Follow the build procedure in the
- <filepath>README.md</filepath> file
- to produce new Impala binaries.
+ Follow the build procedure in the <filepath>README.md</filepath> file to produce new
+ Impala binaries.
</p>
</li>
+
<li>
<p>
- Replace the binaries for all Impala-related
- daemons on all relevant hosts in the cluster.
+ Replace the binaries for all Impala-related daemons on all relevant hosts in the
+ cluster.
</p>
</li>
+
<li>
- Check if there are new recommended or required configuration settings to put into place in the
- configuration files, typically under <filepath>/etc/impala/conf</filepath>. See
- <xref href="impala_config_performance.xml#config_performance"/> for settings related to performance and
- scalability.
+ Check if there are new recommended or required configuration settings to put into
+ place in the configuration files, typically under
+ <filepath>/etc/impala/conf</filepath>. See
+ <xref href="impala_config_performance.xml#config_performance"/> for settings related
+ to performance and scalability.
</li>
+
<li>
<p>
- Restart all Impala-related daemons on
- all relevant hosts in the cluster:
+ Restart all Impala-related daemons on all relevant hosts in the cluster:
</p>
<ol>
<li>
- Restart the Impala state store service on the desired nodes in your cluster. Expect to see a process
- named <codeph>statestored</codeph> if the service started successfully.
+ Restart the Impala state store service on the desired nodes in your cluster.
+ Expect to see a process named <codeph>statestored</codeph> if the service started
+ successfully.
<codeblock>$ sudo service impala-state-store start
$ ps ax | grep [s]tatestored
6819 ? Sl 0:07 /usr/lib/impala/sbin/statestored -log_dir=/var/log/impala -state_store_port=24000
</codeblock>
<p>
- Restart the state store service <i>before</i> the Impala server service to avoid <q>Not
- connected</q> errors when you run <codeph>impala-shell</codeph>.
+ Restart the state store service <i>before</i> the Impala server service to avoid
+ <q>Not connected</q> errors when you run <codeph>impala-shell</codeph>.
</p>
</li>
<li rev="1.2">
- Restart the Impala catalog service on whichever host it runs on in your cluster. Expect to see a
- process named <codeph>catalogd</codeph> if the service started successfully.
+ Restart the Impala catalog service on whichever host it runs on in your cluster.
+ Expect to see a process named <codeph>catalogd</codeph> if the service started
+ successfully.
<codeblock>$ sudo service impala-catalog restart
$ ps ax | grep [c]atalogd
6068 ? Sl 4:06 /usr/lib/impala/sbin/catalogd
@@ -115,8 +120,8 @@ $ ps ax | grep [c]atalogd
</li>
<li>
- Restart the Impala daemon service on each node in your cluster. Expect to see a process named
- <codeph>impalad</codeph> if the service started successfully.
+ Restart the Impala daemon service on each node in your cluster. Expect to see a
+ process named <codeph>impalad</codeph> if the service started successfully.
<codeblock>$ sudo service impala-server start
$ ps ax | grep [i]mpalad
7936 ? Sl 0:12 /usr/lib/impala/sbin/impalad -log_dir=/var/log/impala -state_store_port=24000
@@ -129,192 +134,333 @@ $ ps ax | grep [i]mpalad
<note>
<p>
- If the services did not start successfully (even though the <codeph>sudo service</codeph> command might
- display <codeph>[OK]</codeph>), check for errors in the Impala log file, typically in
- <filepath>/var/log/impala</filepath>.
+ If the services did not start successfully (even though the <codeph>sudo
+ service</codeph> command might display <codeph>[OK]</codeph>), check for errors in the
+ Impala log file, typically in <filepath>/var/log/impala</filepath>.
</p>
</note>
+
</conbody>
+
</concept>
<concept id="concept_a2p_szq_jdb">
+
<title>Impala Upgrade Considerations</title>
+
<concept id="IMPALA-3916">
+
<title>List of Reserved Words Updated in <keyword keyref="impala30_full"
/></title>
+
<conbody>
+
<p>
- The list of <keyword keyref="reserved_words">reserved
- words</keyword> in Impala was updated in <keyword
- keyref="impala30_full"/>. If you need to use a reserved word as an
- identifier, e.g. a table name, enclose the word in back-ticks.
+ The list of <keyword keyref="reserved_words">reserved words</keyword> in Impala was
+ updated in <keyword
+ keyref="impala30_full"/>. If you need to use a
+ reserved word as an identifier, e.g. a table name, enclose the word in back-ticks.
</p>
<p>
- If you need to use the reserved words from previous versions of
- Impala, set the <codeph>impalad</codeph> and <codeph>catalogd</codeph>
- startup flag. Note that this startup option will be deprecated in a
- future release.
+ If you need to use the reserved words from previous versions of Impala, set the
+ <codeph>impalad</codeph> and <codeph>catalogd</codeph> startup flag. Note that this
+ startup option will be deprecated in a future release.
<codeblock>--reserved_words_version=2.11.0</codeblock>
</p>
+
</conbody>
+
</concept>
<concept id="IMPALA-4924">
+
<title>Decimal V2 Used by Default in <keyword keyref="impala30_full"/></title>
+
<conbody>
+
<p>
- In Impala, two different implementations of <codeph>DECIMAL</codeph>
- types are supported. Starting in <keyword keyref="impala30_full"/>,
- <codeph>DECIMAL</codeph> V2 is used by default. See <keyword
- keyref="decimal">DECIMAL Type</keyword> for detail information.
+ In Impala, two different implementations of <codeph>DECIMAL</codeph> types are
+ supported. Starting in <keyword keyref="impala30_full"/>, <codeph>DECIMAL</codeph> V2
+ is used by default. See <keyword
+ keyref="decimal">DECIMAL Type</keyword>
+ for detail information.
</p>
<p>
- If you need to continue using the first version of the
- <codeph>DECIMAL</codeph> type for the backward compatibility of your
- queries, set the <codeph>DECIMAL_V2</codeph> query option to
- <codeph>FALSE</codeph>:
+ If you need to continue using the first version of the <codeph>DECIMAL</codeph> type
+ for the backward compatibility of your queries, set the <codeph>DECIMAL_V2</codeph>
+ query option to <codeph>FALSE</codeph>:
<codeblock>SET DECIMAL_V2=FALSE;</codeblock>
</p>
+
</conbody>
+
</concept>
+
<concept id="IMPALA-5191">
+
<title>Behavior of Column Aliases Changed in <keyword
keyref="impala30_full"/></title>
+
<conbody>
+
<p>
- To conform to the SQL standard, Impala no longer performs alias
- substitution in the subexpressions of <codeph>GROUP BY</codeph>,
- <codeph>HAVING</codeph>, and <codeph>ORDER BY</codeph>. See <keyword
- keyref="aliases"/> for examples of supported and unsupported aliases
- syntax.
+ To conform to the SQL standard, Impala no longer performs alias substitution in the
+ subexpressions of <codeph>GROUP BY</codeph>, <codeph>HAVING</codeph>, and
+ <codeph>ORDER BY</codeph>. See <keyword
+ keyref="aliases"/> for examples of
+ supported and unsupported aliases syntax.
</p>
+
</conbody>
+
</concept>
<concept id="IMPALA-5037">
+
<title>Default PARQUET_ARRAY_RESOLUTION Changed in <keyword
keyref="impala30_full"/></title>
+
<conbody>
+
<p>
- The default value for the <codeph>PARQUET_ARRAY_RESOLUTION</codeph>
- was changed to <codeph>THREE_LEVEL</codeph> in <keyword
- keyref="impala30_full"/>, to match the Parquet standard 3-level
- encoding.
+ The default value for the <codeph>PARQUET_ARRAY_RESOLUTION</codeph> was changed to
+ <codeph>THREE_LEVEL</codeph> in <keyword
+ keyref="impala30_full"/>, to
+ match the Parquet standard 3-level encoding.
</p>
<p>
- See <keyword keyref="parquet_array_resolution"/> for the information
- about the query option.
+ See <keyword keyref="parquet_array_resolution"/> for the information about the query
+ option.
</p>
+
</conbody>
+
</concept>
+
<concept id="IMPALA-5293">
+
<title>Enable Clustering Hint for Inserts</title>
+
<conbody>
+
<p>
- In <keyword keyref="impala30_full"/>, the <keyword keyref="hints"
- >clustered</keyword> hint is enabled by default. The hint adds a
- local sort by the partitioning columns to a query plan. </p>
- <p> The <codeph>clustered</codeph> hint is only effective for HDFS and
- Kudu tables.
+ In <keyword keyref="impala30_full"/>, the
+ <keyword keyref="hints"
+ >clustered</keyword> hint is enabled by default.
+ The hint adds a local sort by the partitioning columns to a query plan.
</p>
<p>
- As in previous versions, the <codeph>noclustered</codeph> hint
- prevents clustering. If a table has ordering columns defined, the
- <codeph>noclustered</codeph> hint is ignored with a warning.
+ The <codeph>clustered</codeph> hint is only effective for HDFS and Kudu tables.
</p>
+
+ <p>
+ As in previous versions, the <codeph>noclustered</codeph> hint prevents clustering. If
+ a table has ordering columns defined, the <codeph>noclustered</codeph> hint is ignored
+ with a warning.
+ </p>
+
</conbody>
+
</concept>
<concept id="IMPALA-4319">
+
<title>Deprecated Query Options Removed in <keyword keyref="impala30_full"
/></title>
+
<conbody>
- <p> The following query options have been deprecated for several
- releases and removed: <ul>
- <li><codeph>DEFAULT_ORDER_BY_LIMIT</codeph></li>
- <li><codeph>ABORT_ON_DEFAULT_LIMIT_EXCEEDED</codeph></li>
- <li><codeph>V_CPU_CORES</codeph></li>
- <li><codeph>RESERVATION_REQUEST_TIMEOUT</codeph></li>
- <li><codeph>RM_INITIAL_MEM</codeph></li>
- <li><codeph>SCAN_NODE_CODEGEN_THRESHOLD</codeph></li>
- <li><codeph>MAX_IO_BUFFERS</codeph></li>
- <li><codeph>RM_INITIAL_MEM</codeph></li>
- <li><codeph>DISABLE_CACHED_READS</codeph></li>
+
+ <p>
+ The following query options have been deprecated for several releases and removed:
+ <ul>
+ <li>
+ <codeph>DEFAULT_ORDER_BY_LIMIT</codeph>
+ </li>
+
+ <li>
+ <codeph>ABORT_ON_DEFAULT_LIMIT_EXCEEDED</codeph>
+ </li>
+
+ <li>
+ <codeph>V_CPU_CORES</codeph>
+ </li>
+
+ <li>
+ <codeph>RESERVATION_REQUEST_TIMEOUT</codeph>
+ </li>
+
+ <li>
+ <codeph>RM_INITIAL_MEM</codeph>
+ </li>
+
+ <li>
+ <codeph>SCAN_NODE_CODEGEN_THRESHOLD</codeph>
+ </li>
+
+ <li>
+ <codeph>MAX_IO_BUFFERS</codeph>
+ </li>
+
+ <li>
+ <codeph>RM_INITIAL_MEM</codeph>
+ </li>
+
+ <li>
+ <codeph>DISABLE_CACHED_READS</codeph>
+ </li>
</ul>
</p>
+
</conbody>
+
</concept>
<concept id="impala-6648">
+
<title>Fine-grained Privileges Added in <keyword keyref="impala30_full"
/></title>
+
<conbody>
+
<p>
- Starting in <keyword keyref="impala30_full"/>, finer grained
- privileges are enforced, such as the <codeph>REFRESH</codeph>,
- <codeph>CREATE</codeph>, <codeph>DROP</codeph>, and
- <codeph>ALTER</codeph> privileges. In particular, running
- <codeph>REFRESH</codeph> or <codeph>INVALIDATE METADATA</codeph> now
- requires the new <codeph>REFRESH</codeph> privilege. Users who did not
- previously have the <codeph>ALL</codeph> privilege will no longer be
- able to run <codeph>REFRESH</codeph> or <codeph>INVALIDATE
- METADATA</codeph> after an upgrade. Those users need to have the
- <codeph>REFRESH</codeph> or <codeph>ALL</codeph> privilege granted
- to run <codeph>REFRESH</codeph> or <codeph>INVALIDATE
- METADATA</codeph>.
+ Starting in <keyword keyref="impala30_full"/>, finer grained privileges are enforced,
+ such as the <codeph>REFRESH</codeph>, <codeph>CREATE</codeph>, <codeph>DROP</codeph>,
+ and <codeph>ALTER</codeph> privileges. In particular, running <codeph>REFRESH</codeph>
+ or <codeph>INVALIDATE METADATA</codeph> now requires the new <codeph>REFRESH</codeph>
+ privilege. Users who did not previously have the <codeph>ALL</codeph> privilege will
+ no longer be able to run <codeph>REFRESH</codeph> or <codeph>INVALIDATE
+ METADATA</codeph> after an upgrade. Those users need to have the
+ <codeph>REFRESH</codeph> or <codeph>ALL</codeph> privilege granted to run
+ <codeph>REFRESH</codeph> or <codeph>INVALIDATE METADATA</codeph>.
</p>
<p>
- See <keyword keyref="grant"/> for the new privileges, the scope, and
- other information about the new privileges.
+ See <keyword keyref="grant"/> for the new privileges, the scope, and other information
+ about the new privileges.
</p>
+
</conbody>
+
</concept>
<concept id="IMPALA-3998">
+
<title>refresh_after_connect Impala Shell Option Removed in <keyword
keyref="impala30_full"/></title>
+
+ <conbody>
+
+ <p>
+ The deprecated <codeph>refresh_after_connect</codeph> option was removed from Impala
+ Shell in <keyword keyref="impala30_full"/>
+ </p>
+
+ </conbody>
+
+ </concept>
+
+ <concept id="impala-5607">
+
+ <title>Return Type Changed for EXTRACT and DATE_PART Functions in <keyword
+ keyref="impala30_full"/></title>
+
<conbody>
+
<p>
- The deprecated <codeph>refresh_after_connect</codeph> option was
- removed from Impala Shell in <keyword keyref="impala30_full"/>
+ The following changes were made to the <codeph>EXTRACT</codeph> and
+ <codeph>DATE_PART</codeph> functions:
+ <ul>
+ <li>
+ The output type of the <codeph>EXTRACT</codeph> and <codeph>DATE_PART</codeph>
+ functions was changed to <codeph>BIGINT</codeph>.
+ </li>
+
+ <li>
+ <p>
+ Extracting the millisecond part from a <codeph>TIMESTAMP</codeph> returns the
+ seconds component and the milliseconds component. For example, <codeph>EXTRACT
+ (CAST('2006-05-12 18:27:28.123456789' AS TIMESTAMP), 'MILLISECOND')</codeph>
+ will return <codeph>28123</codeph>.
+ </p>
+ </li>
+ </ul>
</p>
+
</conbody>
+
</concept>
<concept id="concept_mkn_ygr_jdb">
+
<title>Default Setting Changes</title>
+
<conbody>
+
<simpletable frame="all" id="simpletable_x55_ghr_jdb">
+
<sthead>
+
<stentry>Release Changed</stentry>
+
<stentry>Setting</stentry>
+
<stentry>Default Value</stentry>
+
</sthead>
+
<strow>
- <stentry><keyword keyref="impala212_full"/></stentry>
- <stentry><codeph>compact_catalog_topic</codeph>
- <codeph>impalad</codeph> flag</stentry>
- <stentry><codeph>true</codeph></stentry>
+
+ <stentry><keyword keyref="impala212_full"/>
+
+ </stentry>
+
+ <stentry><codeph>compact_catalog_topic</codeph><codeph>impalad</codeph> flag</stentry>
+
+ <stentry><codeph>true</codeph>
+
+ </stentry>
+
</strow>
+
<strow>
- <stentry><keyword keyref="impala212_full"/></stentry>
- <stentry><codeph>max_cached_file_handles</codeph>
- <codeph>impalad</codeph> flag</stentry>
- <stentry><codeph>20000</codeph></stentry>
+
+ <stentry><keyword keyref="impala212_full"/>
+
+ </stentry>
+
+ <stentry><codeph>max_cached_file_handles</codeph><codeph>impalad</codeph> flag</stentry>
+
+ <stentry><codeph>20000</codeph>
+
+ </stentry>
+
</strow>
+
<strow>
- <stentry><keyword keyref="impala30_full"/></stentry>
+
+ <stentry><keyword keyref="impala30_full"/>
+
+ </stentry>
+
<stentry><codeph>PARQUET_ARRAY_RESOLUTION</codeph> query
option</stentry>
- <stentry><codeph>THREE_LEVEL</codeph></stentry>
+
+ <stentry><codeph>THREE_LEVEL</codeph>
+
+ </stentry>
+
</strow>
+
</simpletable>
+
</conbody>
+
</concept>
+
</concept>
+
</concept>