You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by bo...@apache.org on 2019/02/27 10:27:26 UTC
[impala] 04/06: IMPALA-5981: [DOCS] Documented SET=""
This is an automated email from the ASF dual-hosted git repository.
boroknagyz pushed a commit to branch 2.x
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 6005194e3152af3defda80a9403286dd37341d44
Author: Alex Rodoni <ar...@cloudera.com>
AuthorDate: Mon Jun 25 16:30:14 2018 -0700
IMPALA-5981: [DOCS] Documented SET=""
Also, refactored the Impala SET doc and moved the command SET to
the Impala Shell Commands doc.
Change-Id: I7211405d5cc0a548c05ea5218798591873c14417
Reviewed-on: http://gerrit.cloudera.org:8080/10816
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
docs/topics/impala_set.xml | 305 +++++++++++-----------------------
docs/topics/impala_shell_commands.xml | 35 ++--
2 files changed, 110 insertions(+), 230 deletions(-)
diff --git a/docs/topics/impala_set.xml b/docs/topics/impala_set.xml
index 0020188..ddf89da 100644
--- a/docs/topics/impala_set.xml
+++ b/docs/topics/impala_set.xml
@@ -21,7 +21,13 @@ under the License.
<concept rev="2.0.0" id="set">
<title>SET Statement</title>
- <titlealts audience="PDF"><navtitle>SET</navtitle></titlealts>
+
+ <titlealts audience="PDF">
+
+ <navtitle>SET</navtitle>
+
+ </titlealts>
+
<prolog>
<metadata>
<data name="Category" value="Impala"/>
@@ -36,255 +42,130 @@ under the License.
<conbody>
<p rev="2.0.0">
- <indexterm audience="hidden">SET statement</indexterm>
- Specifies values for query options that control the runtime behavior of other statements within the same
- session.
+ The <codeph>SET</codeph> statement specifies values for query options that control the
+ runtime behavior of other statements within the same session.
</p>
- <p rev="2.5.0 IMPALA-2180">
- In <keyword keyref="impala25_full"/> and higher, <codeph>SET</codeph> also defines user-specified substitution variables for
- the <cmdname>impala-shell</cmdname> interpreter. This feature uses the <codeph>SET</codeph> command
- built into <cmdname>impala-shell</cmdname> instead of the SQL <codeph>SET</codeph> statement.
- Therefore the substitution mechanism only works with queries processed by <cmdname>impala-shell</cmdname>,
- not with queries submitted through JDBC or ODBC.
+ <p>
+ When issued in <codeph>impala-shell</codeph>, the <codeph>SET</codeph> command is
+ interpreted as an <codeph>impala-shell</codeph> command that has differences from the SQL
+ <codeph>SET</codeph> statement. See
+ <xref
+ href="impala_shell_commands.xml#shell_commands/set_cmd"/> for the
+ information about the <codeph>SET</codeph> command in <codeph>impala-shell</codeph>.
</p>
- <note type="important" rev="2.11.0 IMPALA-2181">
- <p>
- In <keyword keyref="impala211_full"/> and higher, the output of the <codeph>SET</codeph>
- statement changes in some important ways:
- </p>
- <ul>
- <li>
- <p>
- The options are divided into groups: <codeph>Regular Query Options</codeph>,
- <codeph>Advanced Query Options</codeph>, <codeph>Development Query Options</codeph>, and
- <codeph>Deprecated Query Options</codeph>.
- </p>
- </li>
- <li>
- <p>
- The advanced options are intended for use in specific
- kinds of performance tuning and debugging scenarios. The development options are
- related to internal development of Impala or features that are not yet finalized;
- these options might be changed or removed without notice.
- The deprecated options are related to features that are removed or changed so that
- the options no longer have any purpose; these options might be removed in future
- versions.
- </p>
- </li>
- <li>
- <p>
- By default, only the first two groups (regular and advanced) are
- displayed by the <codeph>SET</codeph> command. Use the syntax <codeph>SET ALL</codeph>
- to see all groups of options.
- </p>
- </li>
- <li>
- <p>
- <cmdname>impala-shell</cmdname> options and user-specified variables are always displayed
- at the end of the list of query options, after all appropriate option groups.
- </p>
- </li>
- <li>
- <p>
- When the <codeph>SET</codeph> command is run through the JDBC or ODBC interfaces,
- the result set has a new third column, <codeph>level</codeph>, indicating which
- group each option belongs to. The same distinction of <codeph>SET</codeph>
- returning the regular and advanced options, and <codeph>SET ALL</codeph>
- returning all option groups, applies to JDBC and ODBC also.
- </p>
- </li>
- </ul>
- </note>
-
<p conref="../shared/impala_common.xml#common/syntax_blurb"/>
-<codeblock>SET [<varname>query_option</varname>=<varname>option_value</varname>]
+<codeblock>SET
<ph rev="2.11.0 IMPALA-2181">SET ALL</ph>
+SET <varname>query_option</varname>=<varname>option_value</varname>
+SET <varname>query_option</varname>=""
</codeblock>
<p rev="2.11.0 IMPALA-2181">
- <codeph>SET</codeph> and <codeph>SET ALL</codeph> with no arguments return a
- result set consisting of all the applicable query options and their current values.
+ <codeph>SET</codeph> and <codeph>SET ALL</codeph> with no arguments return a result set
+ consisting of all the applicable query options and their current values.
</p>
<p>
- The query option name and any string argument values are case-insensitive.
+ The <varname>query_option</varname> and <varname>option_value</varname> are
+ case-insensitive.
</p>
<p>
- Each query option has a specific allowed notation for its arguments. Boolean options can be enabled and
- disabled by assigning values of either <codeph>true</codeph> and <codeph>false</codeph>, or
- <codeph>1</codeph> and <codeph>0</codeph>. Some numeric options accept a final character signifying the unit,
- such as <codeph>2g</codeph> for 2 gigabytes or <codeph>100m</codeph> for 100 megabytes. See
- <xref href="impala_query_options.xml#query_options"/> for the details of each query option.
+ Unlike the <codeph>impala-shell</codeph> command version of <codeph>SET</codeph>, when
+ used as a SQL statement, the string values for <varname>option_value</varname> need to be
+ quoted, e.g. <codeph>SET option="new_value"</codeph>.
</p>
<p>
- <b>Setting query options during impala-shell invocation:</b>
- </p>
-
- <p rev="2.11.0 IMPALA-5736">
- In <keyword keyref="impala211_full"/> and higher, you can use one or more command-line options
- of the form <codeph>--query_option=<varname>option</varname>=<varname>value</varname></codeph>
- when running the <cmdname>impala-shell</cmdname> command. The corresponding query option settings
- take effect for that <cmdname>impala-shell</cmdname> session.
+ The <codeph>SET <varname>query_option</varname> = ""</codeph> statement unsets the value
+ of the <varname>query_option</varname> in the current session, reverting it to the default
+ state. In <codeph>impala-shell</codeph>, use the <codeph>UNSET</codeph> command to set a
+ query option to it default.
</p>
<p>
- <b>User-specified substitution variables:</b>
- </p>
-
- <p rev="2.5.0 IMPALA-2180">
- In <keyword keyref="impala25_full"/> and higher, you can specify your own names and string substitution values
- within the <cmdname>impala-shell</cmdname> interpreter. Once a substitution variable is set up,
- its value is inserted into any SQL statement in that same <cmdname>impala-shell</cmdname> session
- that contains the notation <codeph>${var:<varname>varname</varname>}</codeph>.
- Using <codeph>SET</codeph> in an interactive <cmdname>impala-shell</cmdname> session overrides
- any value for that same variable passed in through the <codeph>--var=<varname>varname</varname>=<varname>value</varname></codeph>
- command-line option.
- </p>
-
- <p rev="2.5.0 IMPALA-2180">
- For example, to set up some default parameters for report queries, but then override those default
- within an <cmdname>impala-shell</cmdname> session, you might issue commands and statements such as
- the following:
- </p>
-
-<codeblock rev="2.5.0 IMPALA-2180">
--- Initial setup for this example.
-create table staging_table (s string);
-insert into staging_table values ('foo'), ('bar'), ('bletch');
-
-create table production_table (s string);
-insert into production_table values ('North America'), ('EMEA'), ('Asia');
-quit;
-
--- Start impala-shell with user-specified substitution variables,
--- run a query, then override the variables with SET and run the query again.
-$ impala-shell --var=table_name=staging_table --var=cutoff=2
-... <varname>banner message</varname> ...
-[localhost:21000] > select s from ${var:table_name} order by s limit ${var:cutoff};
-Query: select s from staging_table order by s limit 2
-+--------+
-| s |
-+--------+
-| bar |
-| bletch |
-+--------+
-Fetched 2 row(s) in 1.06s
-
-[localhost:21000] > set var:table_name=production_table;
-Variable TABLE_NAME set to production_table
-[localhost:21000] > set var:cutoff=3;
-Variable CUTOFF set to 3
-
-[localhost:21000] > select s from ${var:table_name} order by s limit ${var:cutoff};
-Query: select s from production_table order by s limit 3
-+---------------+
-| s |
-+---------------+
-| Asia |
-| EMEA |
-| North America |
-+---------------+
-</codeblock>
-
- <p rev="2.5.0 IMPALA-2180">
- The following example shows how <codeph>SET ALL</codeph> with no parameters displays
- all user-specified substitution variables, and how <codeph>UNSET</codeph> removes
- the substitution variable entirely:
- </p>
-
-<codeblock rev="2.11.0 IMPALA-2181">
-[localhost:21000] > set all;
-Query options (defaults shown in []):
-ABORT_ON_ERROR: [0]
-COMPRESSION_CODEC: []
-DISABLE_CODEGEN: [0]
-...
-
-Advanced Query Options:
-APPX_COUNT_DISTINCT: [0]
-BUFFER_POOL_LIMIT: []
-DEFAULT_JOIN_DISTRIBUTION_MODE: [0]
-...
-
-Development Query Options:
-BATCH_SIZE: [0]
-DEBUG_ACTION: []
-DECIMAL_V2: [0]
-...
-
-Deprecated Query Options:
-ABORT_ON_DEFAULT_LIMIT_EXCEEDED: [0]
-ALLOW_UNSUPPORTED_FORMATS: [0]
-DEFAULT_ORDER_BY_LIMIT: [-1]
-...
-
-Shell Options
- LIVE_PROGRESS: False
- LIVE_SUMMARY: False
-
-Variables:
- CUTOFF: 3
- TABLE_NAME: staging_table
-
-[localhost:21000] > unset var:cutoff;
-Unsetting variable CUTOFF
-[localhost:21000] > select s from ${var:table_name} order by s limit ${var:cutoff};
-Error: Unknown variable CUTOFF
-</codeblock>
-
- <p rev="2.5.0 IMPALA-2180">
- See <xref href="impala_shell_running_commands.xml"/> for more examples of using the
- <codeph>--var</codeph>, <codeph>SET</codeph>, and <codeph>${var:<varname>varname</varname>}</codeph>
- substitution technique in <cmdname>impala-shell</cmdname>.
+ Each query option has a specific allowed notation for its arguments. See
+ <xref href="impala_query_options.xml#query_options"/> for the details of each query
+ option.
</p>
<p conref="../shared/impala_common.xml#common/usage_notes_blurb"/>
<p>
- <codeph>MEM_LIMIT</codeph> is probably the most commonly used query option. You can specify a high value to
- allow a resource-intensive query to complete. For testing how queries would work on memory-constrained
- systems, you might specify an artificially low value.
- </p>
-
- <p conref="../shared/impala_common.xml#common/complex_types_blurb"/>
-
- <p conref="../shared/impala_common.xml#common/example_blurb"/>
-
- <p>
- The following example sets some numeric and some Boolean query options to control usage of memory, disk
- space, and timeout periods, then runs a query whose success could depend on the options in effect:
- </p>
-
-<codeblock>set mem_limit=64g;
-set DISABLE_UNSAFE_SPILLS=true;
-set parquet_file_size=400m;
-set RESERVATION_REQUEST_TIMEOUT=900000;
-insert overwrite parquet_table select c1, c2, count(c3) from text_table group by c1, c2, c3;
-</codeblock>
+ In <keyword keyref="impala211_full"/> and higher, the outputs of the <codeph>SET</codeph>
+ and <codeph>SET ALL</codeph> statements were reorganized as below:
+ </p>
+
+ <ul>
+ <li>
+ <p>
+ The options are divided into groups: <codeph>Regular Query Options</codeph>,
+ <codeph>Advanced Query Options</codeph>, <codeph>Development Query Options</codeph>,
+ and <codeph>Deprecated Query Options</codeph>.
+ </p>
+ <ul>
+ <li>
+ <p>
+ The advanced options are intended for use in specific kinds of performance tuning
+ and debugging scenarios.
+ </p>
+ </li>
+
+ <li>
+ <p>
+ The development options are related to internal development of Impala or features
+ that are not yet finalized. These options might be changed or removed without
+ notice.
+ </p>
+ </li>
+
+ <li>
+ <p>
+ The deprecated options are related to features that are removed or changed so that
+ the options no longer have any purpose. These options might be removed in future
+ versions.
+ </p>
+ </li>
+ </ul>
+ </li>
+
+ <li>
+ <p>
+ By default, only the first two groups, regular and advanced, are displayed by the
+ <codeph>SET</codeph> command. Use <codeph>SET ALL</codeph> to see all groups of
+ options.
+ </p>
+ </li>
+
+ <li>
+ <p>
+ <cmdname>impala-shell</cmdname> options and user-specified variables are always
+ displayed at the end of the list of query options, after all appropriate option
+ groups.
+ </p>
+ </li>
+ </ul>
<p conref="../shared/impala_common.xml#common/added_in_20"/>
<p>
- <codeph>SET</codeph> has always been available as an <cmdname>impala-shell</cmdname> command. Promoting it to
- a SQL statement lets you use this feature in client applications through the JDBC and ODBC APIs.
+ <codeph>SET</codeph> has always been available as an <cmdname>impala-shell</cmdname>
+ command. Promoting it to a SQL statement lets you use this feature in client applications
+ through the JDBC and ODBC APIs.
</p>
-<!-- <p conref="../shared/impala_common.xml#common/jdbc_blurb"/> -->
-
- <p conref="../shared/impala_common.xml#common/cancel_blurb_no"/>
-
<p conref="../shared/impala_common.xml#common/permissions_blurb_no"/>
<p conref="../shared/impala_common.xml#common/related_info"/>
<p>
- See <xref href="impala_query_options.xml#query_options"/> for the query options you can adjust using this
- statement.
+ See <xref href="impala_query_options.xml#query_options"/> for the query options you can
+ adjust using this statement.
</p>
+
</conbody>
+
</concept>
diff --git a/docs/topics/impala_shell_commands.xml b/docs/topics/impala_shell_commands.xml
index f9a48d9..91c6d13 100644
--- a/docs/topics/impala_shell_commands.xml
+++ b/docs/topics/impala_shell_commands.xml
@@ -291,29 +291,28 @@ under the License.
</entry>
<entry>
<p>
- Manages query options for an <cmdname>impala-shell</cmdname> session. The available options are the
- ones listed in <xref href="impala_query_options.xml#query_options"/>. These options are used for
- query tuning and troubleshooting. Issue <codeph>SET</codeph> with no arguments to see the current
- query options, either based on the <cmdname>impalad</cmdname> defaults, as specified by you at
- <cmdname>impalad</cmdname> startup, or based on earlier <codeph>SET</codeph> statements in the same
- session. To modify option values, issue commands with the syntax <codeph>set
- <varname>option</varname>=<varname>value</varname></codeph>. To restore an option to its default,
- use the <codeph>unset</codeph> command. Some options take Boolean values of <codeph>true</codeph>
- and <codeph>false</codeph>. Others take numeric arguments, or quoted string values.
+ Manages query options for an <cmdname>impala-shell</cmdname>
+ session. The available options are the ones listed in <xref
+ href="impala_query_options.xml#query_options"/>. These options
+ are used for query tuning and troubleshooting. Issue
+ <codeph>SET</codeph> with no arguments to see the current
+ query options, either based on the <cmdname>impalad</cmdname>
+ defaults, as specified by you at <cmdname>impalad</cmdname>
+ startup, or based on earlier <codeph>SET</codeph> statements in
+ the same session. To modify option values, issue commands with
+ the syntax <codeph>set
+ <varname>option</varname>=<varname>value</varname></codeph>.
+ To restore an option to its default, use the
+ <codeph>unset</codeph> command.
</p>
<p conref="../shared/impala_common.xml#common/set_vs_connect"/>
<p rev="2.0.0">
- In Impala 2.0 and later, <codeph>SET</codeph> is available as a SQL statement for any kind of
- application, not only through <cmdname>impala-shell</cmdname>. See
- <xref href="impala_set.xml#set"/> for details.
- </p>
-
- <p rev="2.5.0 IMPALA-2180">
- In Impala 2.5 and later, you can use <codeph>SET</codeph> to define your own substitution variables
- within an <cmdname>impala-shell</cmdname> session.
- Within a SQL statement, you substitute the value by using the notation <codeph>${var:<varname>variable_name</varname>}</codeph>.
+ In Impala 2.0 and later, <codeph>SET</codeph> is
+ available as a SQL statement for any kind of application as well
+ as in <cmdname>impala-shell</cmdname>. See <xref
+ href="impala_set.xml#set"/> for details.
</p>
</entry>
</row>