You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by ta...@apache.org on 2019/03/23 23:38:56 UTC
[impala] 05/10: IMPALA-3956: [DOCS] Escape variables with '\' in
impala-shell
This is an automated email from the ASF dual-hosted git repository.
tarmstrong pushed a commit to branch 2.x
in repository https://gitbox.apache.org/repos/asf/impala.git
commit c240bf11c0a3e01ca484723be5cd05021bab7dfe
Author: Alex Rodoni <ar...@cloudera.com>
AuthorDate: Tue Jul 3 09:32:38 2018 -0700
IMPALA-3956: [DOCS] Escape variables with '\' in impala-shell
Change-Id: Ifb95785a143939a94d55d3565364afe1e26c1f3d
Reviewed-on: http://gerrit.cloudera.org:8080/10861
Reviewed-by: Adam Holley <ah...@cloudera.com>
Reviewed-by: Fredy Wijaya <fw...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-on: http://gerrit.cloudera.org:8080/12802
---
docs/topics/impala_shell_commands.xml | 14 +-
docs/topics/impala_shell_running_commands.xml | 397 ++++++++------------------
2 files changed, 120 insertions(+), 291 deletions(-)
diff --git a/docs/topics/impala_shell_commands.xml b/docs/topics/impala_shell_commands.xml
index 91c6d13..f957f2f 100644
--- a/docs/topics/impala_shell_commands.xml
+++ b/docs/topics/impala_shell_commands.xml
@@ -260,12 +260,14 @@ under the License.
or <codeph>profile</codeph>.
</p>
<p>
- Specify an integer argument. A positive integer <codeph>N</codeph>
- represents the command labelled <codeph>N</codeph> in the history list.
- A negative integer <codeph>-N</codeph> represents the <codeph>N</codeph>th
- command from the end of the list, such as -1 for the most recent command.
- Commands that are executed again do not produce new entries in the
- history list.
+ Specify an integer argument. A positive integer
+ <codeph>N</codeph> represents the command labelled
+ <codeph>N</codeph> in the output of the
+ <codeph>HISTORY</codeph> command. A negative integer
+ <codeph>-N</codeph> represents the <codeph>N</codeph>th
+ command from the end of the list, such as -1 for the most recent
+ command. Commands that are executed again do not produce new
+ entries in the <codeph>HISTORY</codeph> output list.
</p>
</entry>
</row>
diff --git a/docs/topics/impala_shell_running_commands.xml b/docs/topics/impala_shell_running_commands.xml
index 75a0758..519d705 100644
--- a/docs/topics/impala_shell_running_commands.xml
+++ b/docs/topics/impala_shell_running_commands.xml
@@ -21,7 +21,13 @@ under the License.
<concept id="shell_running_commands">
<title>Running Commands and SQL Statements in impala-shell</title>
- <titlealts audience="PDF"><navtitle>Running Commands and SQL Statements</navtitle></titlealts>
+
+ <titlealts audience="PDF">
+
+ <navtitle>Running Commands and SQL Statements</navtitle>
+
+ </titlealts>
+
<prolog>
<metadata>
<data name="Category" value="Impala"/>
@@ -35,314 +41,135 @@ under the License.
<conbody>
<p>
- For information on available commands, see
- <xref href="impala_shell_commands.xml#shell_commands"/>. You can see the full set of available
- commands by pressing TAB twice, for example:
+ The following are a few of the key syntax and usage rules for running commands and SQL
+ statements in <codeph>impala-shell</codeph>.
</p>
-<codeblock>[impalad-host:21000] >
-connect describe explain help history insert quit refresh select set shell show use version
-[impalad-host:21000] ></codeblock>
+ <ul>
+ <li>
+ To see the full set of available commands, press <b>TAB</b> twice.
+ </li>
+
+ <li>
+ To cycle through and edit previous commands, click the up-arrow and down-arrow keys.
+ </li>
+
+ <li>
+ Use the standard set of keyboard shortcuts in GNU Readline library for editing and
+ cursor movement, such as <b>Ctrl-A</b> for the beginning of line and <b>Ctrl-E</b> for
+ the end of line.
+ </li>
+
+ <li>
+ Commands and SQL statements must be terminated by a semi-colon.
+ </li>
+
+ <li>
+ Commands and SQL statements can span multiple lines.
+ </li>
+
+ <li>
+ Use <codeph>--</codeph> to denote a single-line comment and /* */ to
+ denote a multi-line comment.
+
+ <p> A comment is considered part of the
+ statement it precedes, so when you enter a <codeph>--</codeph> or
+ <codeph>/* */</codeph> comment, you get a continuation prompt until
+ you finish entering a statement ending with a semicolon. For example:
+ </p>
+<codeblock>[impala] > -- This is a test comment
+ > SHOW TABLES LIKE 't*';
+</codeblock>
+ </li>
- <note>
- Commands must be terminated by a semi-colon. A command can span multiple lines.
- </note>
+ <li>
+ If a comment contains the
+ <codeph>${<varname>variable_name</varname>}</codeph> character and it
+ is not for a variable substitution, the <codeph>$</codeph> character
+ must be escaped, e.g. <codeph>-- \${hello}</codeph>.
+ </li>
+ </ul>
<p>
- For example:
+ For information on available <codeph>impala-shell</codeph> commands, see
+ <xref href="impala_shell_commands.xml#shell_commands"/>.
</p>
-<codeblock>[localhost:21000] > select *
- > from t1
- > limit 5;
-+---------+-----------+
-| s1 | s2 |
-+---------+-----------+
-| hello | world |
-| goodbye | cleveland |
-+---------+-----------+
-</codeblock>
+ </conbody>
- <p>
- A comment is considered part of the statement it precedes, so when you enter a <codeph>--</codeph> or
- <codeph>/* */</codeph> comment, you get a continuation prompt until you finish entering a statement ending
- with a semicolon:
- </p>
+ <concept id="impala-shell_variable">
-<codeblock>[localhost:21000] > -- This is a test comment
- > show tables like 't*';
-+--------+
-| name |
-+--------+
-| t1 |
-| t2 |
-| tab1 |
-| tab2 |
-| tab3 |
-| text_t |
-+--------+
-</codeblock>
+ <title>Variable Substitution in impala-shell</title>
- <p>
- Use the up-arrow and down-arrow keys to cycle through and edit previous commands.
- <cmdname>impala-shell</cmdname> uses the <codeph>readline</codeph> library and so supports a standard set of
- keyboard shortcuts for editing and cursor movement, such as <codeph>Ctrl-A</codeph> for beginning of line and
- <codeph>Ctrl-E</codeph> for end of line.
- </p>
+ <conbody>
- <p rev="2.5.0 IMPALA-2179 IMPALA-2180">
- In <keyword keyref="impala25_full"/> and higher, you can define substitution variables to be used within SQL statements
- processed by <cmdname>impala-shell</cmdname>. On the command line, you specify the option
- <codeph>--var=<varname>variable_name</varname>=<varname>value</varname></codeph>.
- Within an interactive session or a script file processed by the <codeph>-f</codeph> option, you specify
- a <codeph>SET</codeph> command using the notation <codeph>SET VAR:<varname>variable_name</varname>=<varname>value</varname></codeph>.
- Within a SQL statement, you substitute the value by using the notation <codeph>${var:<varname>variable_name</varname>}</codeph>.
- </p>
+ <p rev="2.5.0 IMPALA-2179 IMPALA-2180">
+ In <keyword keyref="impala25_full"/> and higher, you can define substitution variables
+ to be used within SQL statements processed by <cmdname>impala-shell</cmdname>.
+ <ol>
+ <li>
+ You specify the variable and its value as below.
+ <ul>
+ <li>
+ On the command line, you specify the option
+ <codeph>--var=<varname>variable_name</varname>=<varname>value</varname></codeph>
+ </li>
+
+ <li>
+ Within an interactive session or a script file processed by the
+ <codeph>-f</codeph> option, use the <codeph>SET
+ VAR:<varname>variable_name</varname>=<varname>value</varname></codeph> command.
+ </li>
+ </ul>
+ </li>
+
+ <li>
+ Use the above variable in SQL statements in the <codeph>impala-shell</codeph>
+ session using the notation:
+ <codeph>${VAR:<varname>variable_name</varname>}</codeph>.
+ </li>
+ </ol>
+ </p>
- <note>
- Because this feature is part of <cmdname>impala-shell</cmdname> rather than the <cmdname>impalad</cmdname>
- backend, make sure the client system you are connecting from has the most recent <cmdname>impala-shell</cmdname>.
- You can use this feature with a new <cmdname>impala-shell</cmdname> connecting to an older <cmdname>impalad</cmdname>,
- but not the reverse.
- </note>
-
- <p rev="2.5.0 IMPALA-2179 IMPALA-2180">
- For example, here are some <cmdname>impala-shell</cmdname> commands that define substitution variables and then
- use them in SQL statements executed through the <codeph>-q</codeph> and <codeph>-f</codeph> options.
- Notice how the <codeph>-q</codeph> argument strings are single-quoted to prevent shell expansion of the
- <codeph>${var:value}</codeph> notation, and any string literals within the queries are enclosed by double quotation marks.
- </p>
+ <note>
+ Because this feature is part of <cmdname>impala-shell</cmdname> rather than the
+ <cmdname>impalad</cmdname> backend, make sure the client system you are connecting from
+ has the most recent <cmdname>impala-shell</cmdname>. You can use this feature with a new
+ <cmdname>impala-shell</cmdname> connecting to an older <cmdname>impalad</cmdname>, but
+ not the reverse.
+ </note>
+
+ <p rev="2.5.0 IMPALA-2179 IMPALA-2180">
+ For example, here are some <cmdname>impala-shell</cmdname> commands that define
+ substitution variables and then use them in SQL statements executed through the
+ <codeph>-q</codeph> and <codeph>-f</codeph> options. Notice how the <codeph>-q</codeph>
+ argument strings are single-quoted to prevent shell expansion of the
+ <codeph>${var:value}</codeph> notation, and any string literals within the queries are
+ enclosed by double quotation marks.
+ </p>
<codeblock rev="2.5.0 IMPALA-2179 IMPALA-2180">
-$ impala-shell --var=tname=table1 --var=colname=x --var=coltype=string -q 'create table ${var:tname} (${var:colname} ${var:coltype}) stored as parquet'
-Starting Impala Shell without Kerberos authentication
-Connected to <varname>hostname</varname>
-Server version: <varname>impalad_version</varname>
-Query: create table table1 (x string) stored as parquet
-
-$ NEW_STRING="hello world"
-$ impala-shell --var=tname=table1 --var=insert_val="$NEW_STRING" -q 'insert into ${var:tname} values ("${var:insert_val}")'
-Starting Impala Shell without Kerberos authentication
-Connected to <varname>hostname</varname>
-Server version: <varname>impalad_version</varname>
-Query: insert into table1 values ("hello world")
-Inserted 1 row(s) in 1.40s
-
-$ for VAL in foo bar bletch
-do
- impala-shell --var=tname=table1 --var=insert_val="$VAL" -q 'insert into ${var:tname} values ("${var:insert_val}")'
-done
-...
-Query: insert into table1 values ("foo")
-Inserted 1 row(s) in 0.22s
-Query: insert into table1 values ("bar")
-Inserted 1 row(s) in 0.11s
-Query: insert into table1 values ("bletch")
-Inserted 1 row(s) in 0.21s
-
-$ echo "Search for what substring?" ; read answer
-Search for what substring?
-b
-$ impala-shell --var=tname=table1 -q 'select x from ${var:tname} where x like "%${var:answer}%"'
-Starting Impala Shell without Kerberos authentication
-Connected to <varname>hostname</varname>
-Server version: <varname>impalad_version</varname>
-Query: select x from table1 where x like "%b%"
-+--------+
-| x |
-+--------+
-| bletch |
-| bar |
-+--------+
-Fetched 2 row(s) in 0.83s
+$ impala-shell --var=tname=table1 --var=colname=x --var=coltype=string -q 'CREATE TABLE ${var:tname} (${var:colname} ${var:coltype}) STORED AS PARQUET'
+Query: CREATE TABLE table1 (x STRING) STORED AS PARQUET
</codeblock>
- <p rev="2.5.0 IMPALA-2179 IMPALA-2180">
- Here is a substitution variable passed in by the <codeph>--var</codeph> option,
- and then referenced by statements issued interactively. Then the variable is
- cleared with the <codeph>UNSET</codeph> command, and defined again with the
- <codeph>SET</codeph> command.
- </p>
+ <p rev="2.5.0 IMPALA-2179 IMPALA-2180">
+ The below example shows a substitution variable passed in by the <codeph>--var</codeph>
+ option, and then referenced by statements issued interactively. Then the variable is
+ reset with the <codeph>SET</codeph> command.
+ </p>
<codeblock rev="2.5.0 IMPALA-2179 IMPALA-2180">
$ impala-shell --quiet --var=tname=table1
-Starting Impala Shell without Kerberos authentication
-***********************************************************************************
-<varname>banner_message</varname>
-***********************************************************************************
-[<varname>hostname</varname>:21000] > select count(*) from ${var:tname};
-+----------+
-| count(*) |
-+----------+
-| 4 |
-+----------+
-[<varname>hostname</varname>:21000] > unset var:tname;
-Unsetting variable TNAME
-[<varname>hostname</varname>:21000] > select count(*) from ${var:tname};
-Error: Unknown variable TNAME
-[<varname>hostname</varname>:21000] > set var:tname=table1;
-[<varname>hostname</varname>:21000] > select count(*) from ${var:tname};
-+----------+
-| count(*) |
-+----------+
-| 4 |
-+----------+
-</codeblock>
- <p rev="IMPALA-3397">
- The following example shows how the <codeph>SOURCE</codeph> command can execute
- a series of statements from a file:
- </p>
+[impala] > SELECT COUNT(*) FROM ${var:tname};
-<codeblock rev="IMPALA-3397">
-$ cat commands.sql
-show databases;
-show tables in default;
-show functions in _impala_builtins like '*minute*';
-
-$ impala-shell -i localhost
-...
-[localhost:21000] > source commands.sql;
-Query: show databases
-+------------------+----------------------------------------------+
-| name | comment |
-+------------------+----------------------------------------------+
-| _impala_builtins | System database for Impala builtin functions |
-| default | Default Hive database |
-+------------------+----------------------------------------------+
-Fetched 2 row(s) in 0.06s
-Query: show tables in default
-+-----------+
-| name |
-+-----------+
-| customers |
-| sample_07 |
-| sample_08 |
-| web_logs |
-+-----------+
-Fetched 4 row(s) in 0.02s
-Query: show functions in _impala_builtins like '*minute*'
-+-------------+--------------------------------+-------------+---------------+
-| return type | signature | binary type | is persistent |
-+-------------+--------------------------------+-------------+---------------+
-| INT | minute(TIMESTAMP) | BUILTIN | true |
-| TIMESTAMP | minutes_add(TIMESTAMP, BIGINT) | BUILTIN | true |
-| TIMESTAMP | minutes_add(TIMESTAMP, INT) | BUILTIN | true |
-| TIMESTAMP | minutes_sub(TIMESTAMP, BIGINT) | BUILTIN | true |
-| TIMESTAMP | minutes_sub(TIMESTAMP, INT) | BUILTIN | true |
-+-------------+--------------------------------+-------------+---------------+
-Fetched 5 row(s) in 0.03s
-</codeblock>
-
- <p rev="IMPALA-3397">
- The following example shows how a file that is run by the <codeph>SOURCE</codeph> command,
- or through the <codeph>-q</codeph> or <codeph>-f</codeph> options of <cmdname>impala-shell</cmdname>,
- can contain additional <codeph>SOURCE</codeph> commands.
- The first file, <filepath>nested1.sql</filepath>, runs an <cmdname>impala-shell</cmdname> command
- and then also runs the commands from <filepath>nested2.sql</filepath>.
- This ability for scripts to call each other is often useful for code that sets up schemas for applications
- or test environments.
- </p>
-
-<codeblock rev="IMPALA-3397">
-$ cat nested1.sql
-show functions in _impala_builtins like '*minute*';
-source nested2.sql
-$ cat nested2.sql
-show functions in _impala_builtins like '*hour*'
-
-$ impala-shell -i localhost -f nested1.sql
-Starting Impala Shell without Kerberos authentication
-Connected to localhost:21000
-...
-Query: show functions in _impala_builtins like '*minute*'
-+-------------+--------------------------------+-------------+---------------+
-| return type | signature | binary type | is persistent |
-+-------------+--------------------------------+-------------+---------------+
-| INT | minute(TIMESTAMP) | BUILTIN | true |
-| TIMESTAMP | minutes_add(TIMESTAMP, BIGINT) | BUILTIN | true |
-| TIMESTAMP | minutes_add(TIMESTAMP, INT) | BUILTIN | true |
-| TIMESTAMP | minutes_sub(TIMESTAMP, BIGINT) | BUILTIN | true |
-| TIMESTAMP | minutes_sub(TIMESTAMP, INT) | BUILTIN | true |
-+-------------+--------------------------------+-------------+---------------+
-Fetched 5 row(s) in 0.01s
-Query: show functions in _impala_builtins like '*hour*'
-+-------------+------------------------------+-------------+---------------+
-| return type | signature | binary type | is persistent |
-+-------------+------------------------------+-------------+---------------+
-| INT | hour(TIMESTAMP) | BUILTIN | true |
-| TIMESTAMP | hours_add(TIMESTAMP, BIGINT) | BUILTIN | true |
-| TIMESTAMP | hours_add(TIMESTAMP, INT) | BUILTIN | true |
-| TIMESTAMP | hours_sub(TIMESTAMP, BIGINT) | BUILTIN | true |
-| TIMESTAMP | hours_sub(TIMESTAMP, INT) | BUILTIN | true |
-+-------------+------------------------------+-------------+---------------+
-Fetched 5 row(s) in 0.01s
-</codeblock>
-
- </conbody>
-
- <concept id="rerun" rev="2.10.0 IMPALA-992">
- <title>Rerunning impala-shell Commands</title>
- <conbody>
-
- <p>
- In <keyword keyref="impala210_full"/> and higher, you can use the
- <codeph>rerun</codeph> command, or its abbreviation <codeph>@</codeph>,
- to re-execute commands from the history list. The argument can be
- a positive integer (reflecting the number shown in <codeph>history</codeph>
- output) or a negative integer (reflecting the N'th last command in the
- <codeph>history</codeph> output. For example:
- </p>
-
-<codeblock><![CDATA[
-[localhost:21000] > select * from p1 order by t limit 5;
-...
-[localhost:21000] > show table stats p1;
-+-----------+--------+--------+------------------------------------------------------------+
-| #Rows | #Files | Size | Location |
-+-----------+--------+--------+------------------------------------------------------------+
-| 134217728 | 50 | 4.66MB | hdfs://test.example.com:8020/user/hive/warehouse/jdr.db/p1 |
-+-----------+--------+--------+------------------------------------------------------------+
-[localhost:21000] > compute stats p1;
-+-----------------------------------------+
-| summary |
-+-----------------------------------------+
-| Updated 1 partition(s) and 3 column(s). |
-+-----------------------------------------+
-[localhost:21000] > history;
-[1]: use jdr;
-[2]: history;
-[3]: show tables;
-[4]: select * from p1 order by t limit 5;
-[5]: show table stats p1;
-[6]: compute stats p1;
-[7]: history;
-[localhost:21000] > @-2; <- Rerun the 2nd last command in the history list
-Rerunning compute stats p1;
-+-----------------------------------------+
-| summary |
-+-----------------------------------------+
-| Updated 1 partition(s) and 3 column(s). |
-+-----------------------------------------+
-[localhost:21000] > history; <- History list is not updated by rerunning commands
- or by repeating the last command, in this case 'history'.
-[1]: use jdr;
-[2]: history;
-[3]: show tables;
-[4]: select * from p1 order by t limit 5;
-[5]: show table stats p1;
-[6]: compute stats p1;
-[7]: history;
-[localhost:21000] > @4; <- Rerun command #4 in the history list using short form '@'.
-Rerunning select * from p1 order by t limit 5;
-...
-[localhost:21000] > rerun 4; <- Rerun command #4 using long form 'rerun'.
-Rerunning select * from p1 order by t limit 5;
-...
-]]>
+[impala] > SET VAR:tname=table2;
+[impala] > SELECT COUNT(*) FROM ${var:tname};
</codeblock>
</conbody>
+
</concept>
</concept>