You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by jr...@apache.org on 2017/03/09 23:42:20 UTC
[5/5] incubator-impala git commit: IMPALA-3401: [DOCS] Physically
remove Cloudera Manager info
IMPALA-3401: [DOCS] Physically remove Cloudera Manager info
Followup from Laurel's code reviews, to physically
remove references to Cloudera Manager that were hidden.
Remove a few stray instances of Cloudera Manager that I found
still remaining in the source.
Fix up trailing spaces introduced during earlier
Cloudera Manager-related edits.
Also remove stray 'Cloudera' references, or stale/commented
Cloudera-specific info, noticed near other spots being edited.
Change-Id: Ifc4a84527ae42c39b3717190b6cf669e17fff04b
Reviewed-on: http://gerrit.cloudera.org:8080/6325
Reviewed-by: Ambreen Kazi <am...@cloudera.com>
Reviewed-by: John Russell <jr...@cloudera.com>
Tested-by: Impala Public Jenkins
Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/0467b0d5
Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/0467b0d5
Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/0467b0d5
Branch: refs/heads/master
Commit: 0467b0d54a70ab8d08d5a86efa835a8fda139920
Parents: 1bda28c
Author: John Russell <jr...@cloudera.com>
Authored: Wed Mar 8 16:31:07 2017 -0800
Committer: Impala Public Jenkins <im...@gerrit.cloudera.org>
Committed: Thu Mar 9 23:27:53 2017 +0000
----------------------------------------------------------------------
docs/shared/impala_common.xml | 102 +++-------
docs/topics/impala_admission.xml | 69 +------
docs/topics/impala_auditing.xml | 7 +-
docs/topics/impala_authorization.xml | 3 +-
docs/topics/impala_breakpad.xml | 57 +-----
docs/topics/impala_config_options.xml | 8 +-
docs/topics/impala_config_performance.xml | 43 +----
docs/topics/impala_faq.xml | 4 +-
docs/topics/impala_fixed_issues.xml | 93 +--------
docs/topics/impala_impala_shell.xml | 13 +-
docs/topics/impala_incompatible_changes.xml | 17 +-
docs/topics/impala_isilon.xml | 2 +-
docs/topics/impala_kerberos.xml | 5 +-
docs/topics/impala_logging.xml | 12 +-
docs/topics/impala_new_features.xml | 52 +----
docs/topics/impala_noncm_installation.xml | 11 +-
docs/topics/impala_perf_resources.xml | 11 +-
docs/topics/impala_perf_skew.xml | 11 +-
docs/topics/impala_perf_testing.xml | 10 +-
docs/topics/impala_prereqs.xml | 15 +-
docs/topics/impala_proxy.xml | 235 ++---------------------
docs/topics/impala_resource_management.xml | 21 +-
docs/topics/impala_s3.xml | 4 +-
docs/topics/impala_scalability.xml | 48 +----
docs/topics/impala_schema_design.xml | 6 -
docs/topics/impala_timeouts.xml | 17 +-
docs/topics/impala_troubleshooting.xml | 3 +-
docs/topics/impala_txtfile.xml | 36 +---
docs/topics/impala_udf.xml | 18 +-
docs/topics/impala_webui.xml | 29 +--
30 files changed, 123 insertions(+), 839 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/shared/impala_common.xml
----------------------------------------------------------------------
diff --git a/docs/shared/impala_common.xml b/docs/shared/impala_common.xml
index af62603..40a6c2b 100644
--- a/docs/shared/impala_common.xml
+++ b/docs/shared/impala_common.xml
@@ -574,14 +574,6 @@ under the License.
in the <cmdname>impalad</cmdname> and <cmdname>catalogd</cmdname> configuration settings.
</p>
</li>
- <li audience="hidden">
- <p>
- For clusters managed by Cloudera Manager, select the
- <uicontrol>Use HDFS Rules to Map Kerberos Principals to Short Names</uicontrol>
- checkbox to enable the service-wide <codeph>load_auth_to_local_rules</codeph> configuration setting.
- Then restart the Impala service.
- </p>
- </li>
</ul>
</p>
@@ -616,46 +608,22 @@ under the License.
indicating <q>Access Denied</q> .
<!--
[1]
-Impala: Impala Daemon -> Advanced -> Impala Daemon Logging Safety Valve
-Hive: Hive Server 2 -> Advanced -> HiveServer2 Logging Safety Valve
+Impala: Impala Daemon -> Advanced -> Impala Daemon Logging Safety Valve
+Hive: Hive Server 2 -> Advanced -> HiveServer2 Logging Safety Valve
Search: Solr Server -> Advanced -> HiveServer2 Logging Safety Valve
-->
</p>
</section>
- <section id="cm">
-
- <title>Cloudera Manager Terminology</title>
-
- <p>
- Especially during the transition from CM 4 to CM 5, we'll use some stock phraseology to talk about fields
- and such. Also there are some task steps etc. to conref under the Impala Service page that are easier
- to keep track of here instead of in cm_common_elements.xml. (Although as part of Apache work, anything
- CM might naturally move out of this file.)
- </p>
-
- <p>
- <ph id="safety_valve"> In Cloudera Manager 4, these fields are labelled <uicontrol>Safety
- Valve</uicontrol>; in Cloudera Manager 5, they are called <uicontrol>Advanced Configuration
- Snippet</uicontrol>. </ph>
- </p>
-
- <ul>
- <li id="go_impala_service">Go to the Impala service.</li>
- <li id="restart_impala_service">Restart the Impala service.</li>
- </ul>
-
- </section>
-
- <section id="citi">
+ <section id="restrictions">
- <title>Items from the Citibank Escalation Spreadsheet</title>
+ <title>Restrictions and Limitations</title>
<p>
- Paragraphs with IDs are intended to be reused both in the FAQ and the User's Guide. They refer to feature
- requests or misunderstandings encountered by Citibank, captured in the escalation spreadsheet here:
- <xref href="https://docs.google.com/a/cloudera.com/spreadsheet/ccc?key=0AplfwQJKyyTWdFdhY0E5WHVwNXZSTG9sMEZwQy1QZ1E&usp=drive_web#gid=0" scope="external" format="html"/>.
+ Potential misunderstandings for people familiar with other
+ database systems. Currently not referenced anywhere, because
+ they were only conref'ed from the FAQ page.
</p>
<p id="string_concatenation">
@@ -835,7 +803,7 @@ set PARQUET_FALLBACK_SCHEMA_RESOLUTION=position;
PARQUET_FALLBACK_SCHEMA_RESOLUTION set to position
select * from t2;
-WARNINGS:
+WARNINGS:
File 'schema_evolution.db/t2/45331705_data.0.parq'
has an incompatible Parquet schema for column 'schema_evolution.t2.c4'.
Column type: TIMESTAMP, Parquet schema: optional int32 c1 [i:0 d:1 r:0]
@@ -868,7 +836,7 @@ select * from t2;
</note>
<p rev="IMPALA-3732" id="avro_2gb_strings">
- The Avro specification allows string values up to 2**64 bytes in length.
+ The Avro specification allows string values up to 2**64 bytes in length.
Impala queries for Avro tables use 32-bit integers to hold string lengths.
In <keyword keyref="impala25_full"/> and higher, Impala truncates <codeph>CHAR</codeph>
and <codeph>VARCHAR</codeph> values in Avro tables to (2**31)-1 bytes.
@@ -971,7 +939,7 @@ alter table partitioned_data set tblproperties ('numRows'='1030000', 'STATS_GENE
<p id="impala_shell_progress_reports_shell_only_caveat">
Because the <codeph>LIVE_PROGRESS</codeph> and <codeph>LIVE_SUMMARY</codeph> query options
- are available only within the <cmdname>impala-shell</cmdname> interpreter:
+ are available only within the <cmdname>impala-shell</cmdname> interpreter:
<ul>
<li>
<p>
@@ -995,7 +963,7 @@ alter table partitioned_data set tblproperties ('numRows'='1030000', 'STATS_GENE
Likewise, the <cmdname>impala-shell</cmdname> command relies on
some information only available in <keyword keyref="impala23_full"/> and higher
to prepare live progress reports and query summaries. The
- <codeph>LIVE_PROGRESS</codeph> and <codeph>LIVE_SUMMARY</codeph>
+ <codeph>LIVE_PROGRESS</codeph> and <codeph>LIVE_SUMMARY</codeph>
query options have no effect when <cmdname>impala-shell</cmdname> connects
to a cluster running an older version of Impala.
</p>
@@ -1035,7 +1003,7 @@ use default;
-- Before dropping a database, first drop all the tables inside it,
<ph rev="2.3.0">-- or in <keyword keyref="impala23_full"/> and higher use the CASCADE clause.</ph>
drop database temp;
-ERROR: ImpalaRuntimeException: Error making 'dropDatabase' RPC to Hive Metastore:
+ERROR: ImpalaRuntimeException: Error making 'dropDatabase' RPC to Hive Metastore:
CAUSED BY: InvalidOperationException: Database temp is not empty
show tables in temp;
+------+
@@ -1462,14 +1430,14 @@ explain select s from yy2 where year in (select year from yy where year between
operation finishes. (Multiple concurrent queries can perform operations that use the <q>spill to disk</q>
technique, without any name conflicts for these temporary files.) You can specify a different location by
starting the <cmdname>impalad</cmdname> daemon with the
- <codeph>--scratch_dirs="<varname>path_to_directory</varname>"</codeph> configuration option.
- You can specify a single directory, or a comma-separated list of directories. The scratch directories must
- be on the local filesystem, not in HDFS. You might specify different directory paths for different hosts,
+ <codeph>--scratch_dirs="<varname>path_to_directory</varname>"</codeph> configuration option.
+ You can specify a single directory, or a comma-separated list of directories. The scratch directories must
+ be on the local filesystem, not in HDFS. You might specify different directory paths for different hosts,
depending on the capacity and speed
of the available storage devices. In <keyword keyref="impala23_full"/> or higher, Impala successfully starts (with a warning
- Impala successfully starts (with a warning written to the log) if it cannot create or read and write files
- in one of the scratch directories. If there is less than 1 GB free on the filesystem where that directory resides,
- Impala still runs, but writes a warning message to its log. If Impala encounters an error reading or writing
+ Impala successfully starts (with a warning written to the log) if it cannot create or read and write files
+ in one of the scratch directories. If there is less than 1 GB free on the filesystem where that directory resides,
+ Impala still runs, but writes a warning message to its log. If Impala encounters an error reading or writing
files in a scratch directory during a query, Impala logs the error and the query fails.
</p>
@@ -1642,7 +1610,7 @@ show functions in _impala_builtins like '*<varname>substring</varname>*';
</p>
<p rev="" id="hive_column_stats_caveat">
- If you run the Hive statement <codeph>ANALYZE TABLE COMPUTE STATISTICS FOR COLUMNS</codeph>,
+ If you run the Hive statement <codeph>ANALYZE TABLE COMPUTE STATISTICS FOR COLUMNS</codeph>,
Impala can only use the resulting column statistics if the table is unpartitioned.
Impala cannot use Hive-generated column statistics for a partitioned table.
</p>
@@ -1727,7 +1695,7 @@ show functions in _impala_builtins like '*<varname>substring</varname>*';
following technique for queries involving a single table:
</p>
<codeblock xml:space="preserve">select v1.c1 result1, v2.c1 result2 from
- (select count(distinct col1) as c1 from t1) v1
+ (select count(distinct col1) as c1 from t1) v1
cross join
(select count(distinct col2) as c1 from t1) v2;
</codeblock>
@@ -2211,7 +2179,7 @@ message schema {
$ parquet-tools meta sample.parq
creator: impala version 2.2.0-...
-file schema: schema
+file schema: schema
-------------------------------------------------------------------
year: OPTIONAL INT32 R:0 D:1
month: OPTIONAL INT32 R:0 D:1
@@ -2225,7 +2193,7 @@ carrier: OPTIONAL BINARY R:0 D:1
flight_num: OPTIONAL INT32 R:0 D:1
...
-row group 1: RC:20636601 TS:265103674
+row group 1: RC:20636601 TS:265103674
-------------------------------------------------------------------
year: INT32 SNAPPY DO:4 FPO:35 SZ:10103/49723/4.92 VC:20636601 ENC:PLAIN_DICTIONARY,RLE,PLAIN
month: INT32 SNAPPY DO:10147 FPO:10210 SZ:11380/35732/3.14 VC:20636601 ENC:PLAIN_DICTIONARY,RLE,PLAIN
@@ -2265,7 +2233,7 @@ flight_num: INT32 SNAPPY DO:83456393 FPO:83488603 SZ:10216514/11474301
<p id="impala_parquet_encodings_caveat">
Impala can query Parquet files that use the <codeph>PLAIN</codeph>, <codeph>PLAIN_DICTIONARY</codeph>,
- <codeph>BIT_PACKED</codeph>, and <codeph>RLE</codeph> encodings.
+ <codeph>BIT_PACKED</codeph>, and <codeph>RLE</codeph> encodings.
Currently, Impala does not support <codeph>RLE_DICTIONARY</codeph> encoding.
When creating files outside of Impala for use by Impala, make sure to use one of the supported encodings.
In particular, for MapReduce jobs, <codeph>parquet.writer.version</codeph> must not be defined
@@ -3124,15 +3092,6 @@ select * from header_line limit 10;
worked on SSSE3-enabled processors.
</p>
- <p id="logbuflevel_caveat">
- Due to a change to the implementation of logging in Impala 1.1.1 and higher, currently you should change
- the default setting for the <codeph>logbuflevel</codeph> property for the Impala service after installing
- through Cloudera Manager. In Cloudera Manager, on the log settings page for the Impala service, change the
- setting <uicontrol>Impala Daemon Log Buffer Level (logbuflevel)</uicontrol> from -1 to 0. You might change
- this setting to a value higher than 0, if you prefer to reduce the I/O overhead for logging, at the expense
- of possibly losing some lower-priority log messages in the event of a crash.
- </p>
-
<p id="rhel5_kerberos">
On version 5 of Red Hat Enterprise Linux and comparable distributions, some additional setup is needed for
the <cmdname>impala-shell</cmdname> interpreter to connect to a Kerberos-enabled Impala cluster:
@@ -3198,7 +3157,7 @@ sudo pip-python install ssl</codeblock>
<section id="admin_conrefs">
<title>Administration</title>
-
+
<p id="statestored_catalogd_ha_blurb" rev="">
Most considerations for load balancing and high availability apply to the <cmdname>impalad</cmdname> daemon.
The <cmdname>statestored</cmdname> and <cmdname>catalogd</cmdname> daemons do not have special
@@ -3206,7 +3165,7 @@ sudo pip-python install ssl</codeblock>
If those daemons become unavailable due to an outage on a particular
host, you can stop the Impala service, delete the <uicontrol>Impala StateStore</uicontrol> and
<uicontrol>Impala Catalog Server</uicontrol> roles, add the roles on a different host, and restart the
- Impala service.
+ Impala service.
</p>
<p id="hdfs_caching_encryption_caveat" rev="IMPALA-3679">
@@ -3412,15 +3371,6 @@ sudo pip-python install ssl</codeblock>
JAVA_TOOL_OPTIONS="-Xmx8g"
</codeblock>
</li>
- <li audience="hidden">
- <p rev="OPSAPS-26483">
- On systems managed by Cloudera Manager, include this value in the configuration field
- <uicontrol>Java Heap Size of Catalog Server in Bytes</uicontrol> (Cloudera Manager 5.7 and higher), or
- <uicontrol>Impala Catalog Server Environment Advanced Configuration Snippet (Safety Valve)</uicontrol>
- (prior to Cloudera Manager 5.7).
- Then restart the Impala service.
- </p>
- </li>
<li>
<p>
On systems not using cluster management software, put this environment variable setting into the
@@ -3500,7 +3450,7 @@ sudo pip-python install ssl</codeblock>
such as adding or dropping a column, by a mechanism other than
Impala.
</p>
-
+
<p id="kudu_internal_external_tables">
The distinction between internal and external tables has some special
details for Kudu tables. Tables created entirely through Impala are
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_admission.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_admission.xml b/docs/topics/impala_admission.xml
index 4f5388b..dbee3e2 100644
--- a/docs/topics/impala_admission.xml
+++ b/docs/topics/impala_admission.xml
@@ -213,19 +213,13 @@ under the License.
The Impala admission control feature uses the same configuration mechanism as the YARN resource manager to map users to
pools and authenticate them.
</p>
-
+
<p>
Although the Impala admission control feature uses a <codeph>fair-scheduler.xml</codeph> configuration file
behind the scenes, this file does not depend on which scheduler is used for YARN. You still use this file
even when YARN is using the capacity scheduler.
</p>
- <p rev="DOCS-648" audience="hidden">
- Although the Impala admission control feature uses a <codeph>fair-scheduler.xml</codeph> configuration file
- behind the scenes, this file does not depend on which scheduler is used for YARN. You still use this file,
- and Cloudera Manager can generate it for you, even when YARN is using the capacity scheduler.
- </p>
-
</conbody>
</concept>
@@ -400,31 +394,6 @@ under the License.
set of options) to the complex (multiple resource pools with different options, each pool handling queries
for a different set of users and groups).
</p>
-
-
- <p audience="hidden">
- The configuration options for admission control range from the simple (a single resource pool with a single
- set of options) to the complex (multiple resource pools with different options, each pool handling queries
- for a different set of users and groups). <ph rev="upstream">Cloudera</ph> recommends configuring the settings through the Cloudera Manager user
- interface.
- <!--
- , or on a system without Cloudera Manager by editing configuration files or through startup
- options to the <cmdname>impalad</cmdname> daemon.
- -->
- </p>
-
-<!-- To do: reconcile the similar notes in impala_admission.xml and admin_impala_admission_control.xml
- and make into a conref in both places. -->
- <note type="important" audience="hidden">
- Although the following options are still present in the Cloudera Manager interface under the
- <uicontrol>Admission Control</uicontrol> configuration settings dialog,
- <ph rev="upstream">Cloudera</ph> recommends you not use them in <keyword keyref="impala25_full"/> and higher.
- These settings only apply if you enable admission control but leave dynamic resource pools disabled.
- In <keyword keyref="impala25_full"/> and higher, prefer to set up dynamic resource pools and
- customize the settings for each pool, as described in
- <ph audience="integrated"><xref href="cm_mc_resource_pools.xml#concept_xkk_l1d_wr/section_p15_mhn_2v"/> and <xref href="cm_mc_resource_pools.xml#concept_xkk_l1d_wr/section_gph_tnk_lm"/></ph>
- <xref audience="standalone" href="http://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_resource_pools.html" scope="external" format="html"/>.
- </note>
<section id="admission_flags">
@@ -689,7 +658,7 @@ impala.admission-control.pool-queue-timeout-ms.<varname>queue_name</varname></ph
<title>Example of Admission Control Configuration</title>
<conbody>
-
+
<p> Here are sample <filepath>fair-scheduler.xml</filepath> and
<filepath>llama-site.xml</filepath> files that define resource pools
<codeph>root.default</codeph>, <codeph>root.development</codeph>, and
@@ -697,28 +666,28 @@ impala.admission-control.pool-queue-timeout-ms.<varname>queue_name</varname></ph
deployment they might contain other settings for use with various aspects of the YARN
component. The settings shown here are the significant ones for the Impala admission
control feature. </p>
-
+
<p>
<b>fair-scheduler.xml:</b>
</p>
-
+
<p>
Although Impala does not use the <codeph>vcores</codeph> value, you must still specify it to satisfy
YARN requirements for the file contents.
</p>
-
+
<p>
Each <codeph><aclSubmitApps></codeph> tag (other than the one for <codeph>root</codeph>) contains
a comma-separated list of users, then a space, then a comma-separated list of groups; these are the
users and groups allowed to submit Impala statements to the corresponding resource pool.
</p>
-
+
<p>
If you leave the <codeph><aclSubmitApps></codeph> element empty for a pool, nobody can submit
directly to that pool; child pools can specify their own <codeph><aclSubmitApps></codeph> values
to authorize users and groups to submit to those pools.
</p>
-
+
<codeblock><![CDATA[<allocations>
<queue name="root">
@@ -743,11 +712,11 @@ impala.admission-control.pool-queue-timeout-ms.<varname>queue_name</varname></ph
</allocations>
]]>
</codeblock>
-
+
<p>
<b>llama-site.xml:</b>
</p>
-
+
<codeblock rev="2.5.0 IMPALA-2538"><![CDATA[
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
@@ -821,21 +790,6 @@ impala.admission-control.pool-queue-timeout-ms.<varname>queue_name</varname></ph
</configuration>
]]>
</codeblock>
-
-
-
-
-
- <section id="section_fqn_qgb_rq" audience="hidden">
-
- <title>Example Admission Control Configurations Using Cloudera Manager</title>
-
- <p>
- For full instructions about configuring dynamic resource pools through Cloudera Manager, see
- <xref audience="integrated" href="cm_mc_resource_pools.xml#xd_583c10bfdbd326ba--43d5fd93-1410993f8c2--7ff2"/><xref audience="standalone" href="http://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_resource_pools.html" scope="external" format="html"/>.
- </p>
-
- </section>
</conbody>
</concept>
@@ -865,11 +819,6 @@ impala.admission-control.pool-queue-timeout-ms.<varname>queue_name</varname></ph
usage for the query, so you can fine-tune the configuration for the memory limits of the resource pools.
</p>
- <p audience="hidden">
- Where practical, use Cloudera Manager to configure the admission control parameters. The Cloudera Manager
- GUI is much simpler than editing the configuration files directly.
- </p>
-
<p>
Remember that the limits imposed by admission control are <q>soft</q> limits.
The decentralized nature of this mechanism means that each Impala node makes its own decisions about whether
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_auditing.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_auditing.xml b/docs/topics/impala_auditing.xml
index 2d5e46c..8f60f44 100644
--- a/docs/topics/impala_auditing.xml
+++ b/docs/topics/impala_auditing.xml
@@ -53,11 +53,10 @@ under the License.
<li>
Decide how many queries will be represented in each log file. By default,
- Impala starts a new log file every 5000 queries. To specify a different number, <ph
- audience="standalone">include
+ Impala starts a new log file every 5000 queries. To specify a different number,
+ <ph audience="standalone">include
the option <codeph>-max_audit_event_log_file_size=<varname>number_of_queries</varname></codeph>
- in the <cmdname>impalad</cmdname> startup options</ph>
- <xref href="cn_iu_audit_log.xml#xd_583c10bfdbd326ba--6eed2fb8-14349d04bee--7d6f/section_v25_lmy_bn" audience="integrated">configure Impala Daemon logging in Cloudera Manager</xref>.
+ in the <cmdname>impalad</cmdname> startup options</ph>.
</li>
<li>
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_authorization.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_authorization.xml b/docs/topics/impala_authorization.xml
index 7a360fa..1996b49 100644
--- a/docs/topics/impala_authorization.xml
+++ b/docs/topics/impala_authorization.xml
@@ -826,8 +826,7 @@ sales = hdfs://ha-nn-uri/etc/access/sales.ini
<p>
To enable URIs in per-DB policy files, the Java configuration option <codeph>sentry.allow.uri.db.policyfile</codeph>
must be set to <codeph>true</codeph>.
- For example: <!-- in the Cloudera Manager field <uicontrol>Impala Service Environment
- Advanced Configuration Snippet (Safety Valve)</uicontrol>: -->
+ For example:
</p>
<codeblock>JAVA_TOOL_OPTIONS="-Dsentry.allow.uri.db.policyfile=true"
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_breakpad.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_breakpad.xml b/docs/topics/impala_breakpad.xml
index d171ef7..056290a 100644
--- a/docs/topics/impala_breakpad.xml
+++ b/docs/topics/impala_breakpad.xml
@@ -134,43 +134,14 @@ under the License.
<concept id="breakpad_minidump_logging">
<title>Detecting Crash Events</title>
<conbody>
-
+
<p>
- You can see in the Impala log files when crash events occur that generate
- minidump files. Because each restart begins a new log file, the <q>crashed</q> message
- is always at or near the bottom of the log file. There might be another later message
+ You can see in the Impala log files when crash events occur that generate
+ minidump files. Because each restart begins a new log file, the <q>crashed</q> message
+ is always at or near the bottom of the log file. There might be another later message
if core dumps are also enabled.
</p>
-
- <p audience="hidden">
- You can see in the Impala log files or in the Cloudera Manager charts for Impala
- when crash events occur that generate minidump files. Because each restart begins
- a new log file, the <q>crashed</q> message is always at or near the bottom of the
- log file. (There might be another later message if core dumps are also enabled.)
- </p>
- </conbody>
- </concept>
- <concept id="breakpad_support_process" rev="CDH-39818" audience="hidden">
- <title>Using the Minidump Files for Problem Resolution</title>
- <conbody>
- <p>
- Typically, you provide minidump files to <keyword keyref="support_org"/> as part of problem resolution,
- in the same way that you might provide a core dump. The <uicontrol>Send Diagnostic Data</uicontrol>
- under the <uicontrol>Support</uicontrol> menu in Cloudera Manager guides you through the
- process of selecting a time period and volume of diagnostic data, then collects the data
- from all hosts and transmits the relevant information for you.
- </p>
- <fig id="fig_pqw_gvx_pr">
- <title>Send Diagnostic Data choice under Support menu</title>
- <image href="../images/support_send_diagnostic_data.png" scalefit="yes" placement="break"/>
- </fig>
- <p>
- You might get additional instructions from <keyword keyref="support_org"/> about collecting minidumps to better isolate a specific problem.
- Because the information in the minidump files is limited to stack traces and register contents,
- the possibility of including sensitive information is much lower than with core dump files.
- If any sensitive information is included in the minidump, <keyword keyref="support_org"/> preserves the confidentiality of that information.
- </p>
</conbody>
</concept>
@@ -182,22 +153,17 @@ under the License.
simulate a <codeph>SIGSEGV</codeph> crash for an <cmdname>impalad</cmdname>
process on a single DataNode, then examines the relevant log files and minidump file.
</p>
-
+
<p>
First, as root on a worker node, kill the <cmdname>impalad</cmdname> process with a
<codeph>SIGSEGV</codeph> error. The original process ID was 23114.
</p>
-
- <p audience="hidden">
- First, as root on a worker node, we kill the <cmdname>impalad</cmdname> process with a
- <codeph>SIGSEGV</codeph> error. The original process ID was 23114. (Cloudera Manager
- restarts the process with a new pid, as shown by the second <cmdname>ps</cmdname> command.)
- </p>
+
<codeblock><![CDATA[
# ps ax | grep impalad
23114 ? Sl 0:18 /opt/cloudera/parcels/<parcel_version>/lib/impala/sbin-retail/impalad --flagfile=/var/run/cloudera-scm-agent/process/114-impala-IMPALAD/impala-conf/impalad_flags
31259 pts/0 S+ 0:00 grep impalad
-#
+#
# kill -11 23114
#
# ps ax | grep impalad
@@ -262,17 +228,12 @@ E0623 14:03:43.911002 23114 logging.cc:118] stderr will be logged to this file.
Wrote minidump to /var/log/impala-minidumps/impalad/0980da2d-a905-01e1-25ff883a-04ee027a.dmp
]]>
</codeblock>
-
+
<p>
The resulting minidump file is much smaller than the corresponding core file,
- making it much easier to supply diagnostic information to <keyword keyref="support_org"/>.
- </p>
-
- <p audience="hidden">
- The resulting minidump file is much smaller than the corresponding core file,
making it much easier to supply diagnostic information to <keyword keyref="support_org"/>.
- The transmission process for the minidump files is automated through Cloudera Manager.
</p>
+
<codeblock><![CDATA[
# pwd
/var/log/impalad
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_config_options.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_config_options.xml b/docs/topics/impala_config_options.xml
index 3c1ae1f..89af100 100644
--- a/docs/topics/impala_config_options.xml
+++ b/docs/topics/impala_config_options.xml
@@ -153,8 +153,8 @@ Starting Impala Catalog Server: [ OK ]</codeblock>
<li rev="1.2">
<p>
Catalog server address (including both the hostname and the port number). Update the
- value of the <codeph>IMPALA_CATALOG_SERVICE_HOST</codeph> variable. Cloudera
- recommends the catalog server be on the same host as the statestore. In that
+ value of the <codeph>IMPALA_CATALOG_SERVICE_HOST</codeph> variable. Where
+ practical, run the catalog server on the same host as the statestore. In that
recommended configuration, the <cmdname>impalad</cmdname> daemon cannot refer to the
catalog server using the loopback address. If the catalog service is hosted on a
machine with an IP address of 192.168.0.27, add the following line:
@@ -333,8 +333,8 @@ Starting Impala Catalog Server: [ OK ]</codeblock>
<li rev="1.2">
<p>
Catalog server address. Update the <codeph>IMPALA_CATALOG_SERVICE_HOST</codeph>
- variable, including both the hostname and the port number in the value. Cloudera
- recommends the catalog server be on the same host as the statestore. In that
+ variable, including both the hostname and the port number in the value. Where
+ practical, run the catalog server on the same host as the statestore. In that
recommended configuration, the <cmdname>impalad</cmdname> daemon cannot refer to
the catalog server using the loopback address. If the catalog service is hosted on
a machine with an IP address of 192.168.0.27, add the following line:
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_config_performance.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_config_performance.xml b/docs/topics/impala_config_performance.xml
index ce9718b..af86b53 100644
--- a/docs/topics/impala_config_performance.xml
+++ b/docs/topics/impala_config_performance.xml
@@ -37,41 +37,16 @@ under the License.
installed using cluster management software, some of these configurations might be completed automatically; you must still
configure short-circuit reads manually. If you want to customize your environment, consider making the changes described in this topic.
</p>
-
- <p id="p_twenty-four" audience="hidden">
- This section describes the mandatory and recommended configuration settings for Impala. If Impala is
- installed using Cloudera Manager, some of these configurations are completed automatically; you must still
- configure short-circuit reads manually. If you installed Impala without Cloudera Manager, or if you want to
- customize your environment, consider making the changes described in this topic.
- </p>
-
- <p audience="hidden">
-<!-- Could conref this paragraph from ciiu_install.xml. -->
- In some cases, depending on the level of Impala, CDH, and Cloudera Manager, you might need to add particular
- component configuration details in one of the free-form fields on the Impala configuration pages within
- Cloudera Manager. <ph conref="../shared/impala_common.xml#common/safety_valve"/>
- </p>
<ul>
<li>
You must enable short-circuit reads, whether or not Impala was installed with cluster
management software. This setting goes in the Impala configuration settings, not the Hadoop-wide settings.
</li>
-
+
<li>
You must enable block location tracking, and you can optionally enable native checksumming for optimal performance.
</li>
-
-
- <li audience="hidden">
- If you installed Impala in an environment that is not managed by Cloudera Manager, you must enable block
- location tracking, and you can optionally enable native checksumming for optimal performance.
- </li>
-
- <li audience="hidden">
- If you deployed Impala using Cloudera Manager see
- <xref href="impala_perf_testing.xml#performance_testing"/> to confirm proper configuration.
- </li>
</ul>
<section id="section_fhq_wyv_ls">
@@ -81,19 +56,11 @@ under the License.
DataNodes, improving performance. This setting also minimizes the number
of additional copies of data. Short-circuit reads requires
<codeph>libhadoop.so</codeph>
- <!-- This link went stale. Not obvious how to keep it in sync with whatever Hadoop CDH is using behind the scenes. So hide the link for now. -->
- <!-- (the <xref href="http://hadoop.apache.org/docs/r0.19.1/native_libraries.html" scope="external" format="html">Hadoop Native Library</xref>) -->
(the Hadoop Native Library) to be accessible to both the server and the
client. <codeph>libhadoop.so</codeph> is not available if you have
installed from a tarball. You must install from an
<codeph>.rpm</codeph>, <codeph>.deb</codeph>, or parcel to use
- short-circuit local reads.
-
- <note audience="hidden">
- If you use Cloudera Manager, you can
- enable short-circuit reads through a checkbox in the user interface
- and that setting takes effect for Impala as well.
- </note>
+ short-circuit local reads.
</p>
<p>
<b>To configure DataNodes for short-circuit reads:</b>
@@ -105,11 +72,9 @@ under the License.
Impala configuration directory. The default Impala configuration
location is <codeph>/etc/impala/conf</codeph>. </li>
<li>
- <indexterm audience="hidden"
- >dfs.client.read.shortcircuit</indexterm>
+ <indexterm audience="hidden">dfs.client.read.shortcircuit</indexterm>
<indexterm audience="hidden">dfs.domain.socket.path</indexterm>
- <indexterm audience="hidden"
- >dfs.client.file-block-storage-locations.timeout.millis</indexterm>
+ <indexterm audience="hidden">dfs.client.file-block-storage-locations.timeout.millis</indexterm>
On all Impala nodes, configure the following properties in <!-- Exact timing is unclear, since we say farther down to copy /etc/hadoop/conf/hdfs-site.xml to /etc/impala/conf.
Which wouldn't work if we already modified the Impala version of the file here. Not to mention that this
doesn't take the CM interface into account, where these /etc files might not exist in those locations. -->
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_faq.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_faq.xml b/docs/topics/impala_faq.xml
index 4dd5d4f..754528e 100644
--- a/docs/topics/impala_faq.xml
+++ b/docs/topics/impala_faq.xml
@@ -39,13 +39,11 @@ under the License.
This section lists frequently asked questions for Apache Impala (incubating),
the interactive SQL engine for Hadoop.
</p>
-
+
<p>
This section is under construction.
</p>
-
-
</conbody>
</concept>
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_fixed_issues.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_fixed_issues.xml b/docs/topics/impala_fixed_issues.xml
index d0af303..d70327c 100644
--- a/docs/topics/impala_fixed_issues.xml
+++ b/docs/topics/impala_fixed_issues.xml
@@ -3784,7 +3784,7 @@ db3362f IMPALA-1801: external-data-source-executor leaking global jni refs
<b>Bug:</b>
<xref href="https://issues.cloudera.org/browse/IMPALA-1019" scope="external" format="html">IMPALA-1019</xref>
</p>
-
+
<p>
<b>Resolution:</b> This issue is fixed in Impala 1.3.2. The addition of HDFS caching
support in Impala 1.4 means that this issue does not apply to any new level of Impala.
@@ -4018,7 +4018,7 @@ db3362f IMPALA-1801: external-data-source-executor leaking global jni refs
<b>Bug:</b>
<xref href="https://issues.cloudera.org/browse/IMPALA-1019" scope="external" format="html">IMPALA-1019</xref>
</p>
-
+
<p>
<b>Resolution:</b> This issue is fixed in Impala 1.3.2. The addition of HDFS caching
support in Impala 1.4 means that this issue does not apply to any new level of Impala.
@@ -4770,11 +4770,6 @@ Bad stats:
key.
</p>
- <p audience="hidden">
- <b>Bug:</b>
- <xref href="https://jira.cloudera.com/browse/IMP-1188" scope="external" format="html">IMP-1188</xref>
- </p>
-
<p>
<b>Resolution:</b> Queries now return appropriate results when function calls are used in the row key
comparison. For queries involving non-existent row keys, such as <codeph>WHERE <varname>row_key</varname>
@@ -5614,12 +5609,6 @@ hive> NULL</codeblock>
Insert INTO TABLE SELECT <constant> will not insert any data and may return an error.
</p>
- <p audience="hidden">
- <b>Cloudera Bug:</b>
- <xref href="https://jira.cloudera.com/browse/IMP-231" scope="external" format="html"/> ; KI added 0.1
- <i>Cloudera internal only</i>
- </p>
-
<p>
<b>Anticipated Resolution</b>: Fixed
</p>
@@ -6076,12 +6065,6 @@ hive> NULL</codeblock>
<conbody>
- <p audience="hidden">
- <b>Cloudera Bug:</b>
- <xref href="https://jira.cloudera.com/browse/IMP-474" scope="external" format="html"/>; KI added 0.1
- <i>Cloudera internal only</i>
- </p>
-
<p>
<b>Resolution:</b> Fixed in 0.7
</p>
@@ -6191,12 +6174,6 @@ hive> NULL</codeblock>
does not propagate to Impala.
</p>
- <p audience="hidden">
- <b>Cloudera Bug:</b>
- <xref href="https://jira.cloudera.com/browse/IMP-56" scope="external" format="html"/>; KI added 0.1
- <i>Cloudera internal only</i>
- </p>
-
<p>
<b>Severity:</b> Low
</p>
@@ -6435,12 +6412,6 @@ hive> NULL</codeblock>
impala-shell will incorrectly report that the failed metadata refresh completed successfully.
</p>
- <p audience="hidden">
- <b>Cloudera Bug:</b>
- <xref href="https://jira.cloudera.com/browse/IMP-611" scope="external" format="html"/>; KI added 0.1
- <i>Cloudera internal only</i>
- </p>
-
<p>
<b>Anticipated Resolution:</b> To be fixed in a future release
</p>
@@ -6511,12 +6482,6 @@ hive> NULL</codeblock>
<conbody>
- <p audience="hidden">
- <b>Cloudera Bug:</b>
- <xref href="https://jira.cloudera.com/browse/IMP-601" scope="external" format="html"/>; KI added 0.1
- <i>Cloudera internal only</i>
- </p>
-
<p>
<b>Resolution:</b> Fixed in 0.4
</p>
@@ -6541,12 +6506,6 @@ hive> NULL</codeblock>
table will return an <codeph>unknown table</codeph> error message, even if the table is known.
</p>
- <p audience="hidden">
- <b>Cloudera Bug:</b>
- <xref href="https://jira.cloudera.com/browse/IMP-298" scope="external" format="html"/>; KI added 0.1
- <i>Cloudera internal only</i>
- </p>
-
<p>
<b>Resolution:</b> Fixed in 0.3
</p>
@@ -6565,12 +6524,6 @@ hive> NULL</codeblock>
even if the metadata for that table is fixed.
</p>
- <p audience="hidden">
- <b>Cloudera Bug:</b>
- <xref href="https://jira.cloudera.com/browse/IMP-298" scope="external" format="html"/>; KI added 0.1
- <i>Cloudera internal only</i>
- </p>
-
<p>
<b>Resolution:</b> Fixed in 0.3
</p>
@@ -6587,12 +6540,6 @@ hive> NULL</codeblock>
Attempting to select from these tables fails.
</p>
- <p audience="hidden">
- <b>Cloudera Bug:</b>
- <xref href="https://jira.cloudera.com/browse/IMP-581" scope="external" format="html"/>; KI added 0.1
- <i>Cloudera internal only</i>
- </p>
-
<p>
<b>Resolution:</b> Fixed in 0.3
</p>
@@ -6610,12 +6557,6 @@ hive> NULL</codeblock>
any of the joined tables in the WHERE clause.
</p>
- <p audience="hidden">
- <b>Cloudera Bug:</b>
- <xref href="https://jira.cloudera.com/browse/IMP-137" scope="external" format="html"/>; KI added 0.1
- <i>Cloudera internal only</i>
- </p>
-
<p>
<b>Resolution:</b> Fixed in 0.3.
</p>
@@ -6642,12 +6583,6 @@ hive> NULL</codeblock>
<codeblock>SELECT * FROM (SELECT sum(col1) FROM some_table GROUP BY col1) t1 JOIN other_table ON (...);</codeblock>
- <p audience="hidden">
- <b>Cloudera Bug:</b>
- <xref href="https://jira.cloudera.com/browse/IMP-491" scope="external" format="html"/>; KI added 0.1
- <i>Cloudera internal only</i>
- </p>
-
<p>
<b>Resolution:</b> Fixed in 0.2
</p>
@@ -6666,12 +6601,6 @@ hive> NULL</codeblock>
<codeblock>INSERT OVERWRITE TABLE test SELECT * FROM test2 LIMIT 1;</codeblock>
- <p audience="hidden">
- <b>Cloudera Bug:</b>
- <xref href="https://jira.cloudera.com/browse/IMP-497" scope="external" format="html"/>; KI added 0.1
- <i>Cloudera internal only</i>
- </p>
-
<p>
<b>Resolution:</b> Fixed in 0.2
</p>
@@ -6690,12 +6619,6 @@ hive> NULL</codeblock>
<codeblock>SELECT * FROM test2 LIMIT 1;</codeblock>
- <p audience="hidden">
- <b>Cloudera Bug:</b>
- <xref href="https://jira.cloudera.com/browse/IMP-535" scope="external" format="html"/>; KI added 0.1
- <i>Cloudera internal only</i>
- </p>
-
<p>
<b>Resolution:</b> Fixed in 0.2
</p>
@@ -6712,12 +6635,6 @@ hive> NULL</codeblock>
Attempting to read such files does not generate a diagnostic.
</p>
- <p audience="hidden">
- <b>Cloudera Bug:</b>
- <xref href="https://jira.cloudera.com/browse/IMP-270" scope="external" format="html"/>; KI added 0.1
- <i>Cloudera internal only</i>
- </p>
-
<p>
<b>Resolution:</b> Fixed in 0.2
</p>
@@ -6735,12 +6652,6 @@ hive> NULL</codeblock>
exception.
</p>
- <p audience="hidden">
- <b>Cloudera Bug:</b>
- <xref href="https://jira.cloudera.com/browse/IMP-522" scope="external" format="html"/>; KI added 0.1
- <i>Cloudera internal only</i>
- </p>
-
<p>
<b>Resolution:</b> Fixed in 0.2
</p>
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_impala_shell.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_impala_shell.xml b/docs/topics/impala_impala_shell.xml
index 4780728..6f74a41 100644
--- a/docs/topics/impala_impala_shell.xml
+++ b/docs/topics/impala_impala_shell.xml
@@ -93,20 +93,9 @@ under the License.
and edit previous commands.
</li>
</ul>
-
+
<p>
-
For information on installing the Impala shell, see <xref href="impala_install.xml#install"/>.
-
- </p>
-
-
- <p audience="hidden">
- For information on installing the Impala shell, see <xref href="impala_install.xml#install"/>. In Cloudera
- Manager 4.1 and higher, Cloudera Manager installs <cmdname>impala-shell</cmdname> automatically. You might
- install <cmdname>impala-shell</cmdname> manually on other systems not managed by Cloudera Manager, so that
- you can issue queries from client systems that are not also running the Impala daemon or other Apache Hadoop
- components.
</p>
<p>
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_incompatible_changes.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_incompatible_changes.xml b/docs/topics/impala_incompatible_changes.xml
index af47094..40d9d9b 100644
--- a/docs/topics/impala_incompatible_changes.xml
+++ b/docs/topics/impala_incompatible_changes.xml
@@ -1229,12 +1229,6 @@ select * from `cross`;</codeblock>
</li>
</ul>
- <p audience="hidden">
- Impala 1.2.1 goes along with CDH 4.5 and Cloudera Manager 4.8. If you used the beta version Impala 1.2.0
- that came with the beta of CDH 5, Impala 1.2.1 includes all the features of Impala 1.2.0 except for
- resource management, which relies on the YARN framework from CDH 5.
- </p>
-
<p>
The new <cmdname>catalogd</cmdname> service might require changes to any user-written scripts that stop,
start, or restart Impala services, install or upgrade Impala packages, or issue <codeph>REFRESH</codeph> or
@@ -1267,7 +1261,7 @@ select * from `cross`;</codeblock>
<ul conref="../shared/impala_common.xml#common/catalogd_xrefs">
<li/>
</ul>
-
+
<p>
The new resource management feature interacts with both YARN and Llama services.
<ph audience="PDF">See
@@ -1366,17 +1360,8 @@ ALTER TABLE <varname>table_name</varname> SET FILEFORMAT
<codeph>impala-lzo-cdh4</codeph> to the latest level. See <xref href="impala_txtfile.xml#lzo"/> for
details.
</li>
-
- <li audience="hidden">
- Cloudera Manager 4.5.2 and higher only supports Impala 1.0 and higher, and vice versa. If you upgrade to
- Impala 1.0 or higher managed by Cloudera Manager, you must also upgrade Cloudera Manager to version 4.5.2
- or higher. If you upgrade from an earlier version of Cloudera Manager, and were using Impala, you must
- also upgrade Impala to version 1.0 or higher. The beta versions of Impala are no longer supported as of
- the release of Impala 1.0.
- </li>
</ul>
</conbody>
</concept>
-
</concept>
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_isilon.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_isilon.xml b/docs/topics/impala_isilon.xml
index f475ae8..a26539d 100644
--- a/docs/topics/impala_isilon.xml
+++ b/docs/topics/impala_isilon.xml
@@ -87,7 +87,7 @@ CREATE TABLE d1.t2 (a TINYINT, b BOOLEAN);
the temporary data for the spill-to-disk feature, that configuration is
not recommended due to the need to transfer the data both ways using remote I/O.
</p>
-
+
<p>
When tuning Impala queries on HDFS, you typically try to avoid any remote reads.
When the data resides on Isilon storage, all the I/O consists of remote reads.
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_kerberos.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_kerberos.xml b/docs/topics/impala_kerberos.xml
index 480a861..643e3de 100644
--- a/docs/topics/impala_kerberos.xml
+++ b/docs/topics/impala_kerberos.xml
@@ -148,9 +148,8 @@ under the License.
</li>
<li>
- Editing <codeph>/etc/default/impala</codeph> <ph audience="hidden">(in cluster not managed by Cloudera Manager), or editing the
- <uicontrol>Security</uicontrol> settings in the Cloudera Manager interface,</ph>to accommodate Kerberos
- authentication.
+ Editing <codeph>/etc/default/impala</codeph>
+ to accommodate Kerberos authentication.
</li>
</ul>
</conbody>
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_logging.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_logging.xml b/docs/topics/impala_logging.xml
index d209c27..7c27a61 100644
--- a/docs/topics/impala_logging.xml
+++ b/docs/topics/impala_logging.xml
@@ -178,7 +178,6 @@ under the License.
<prolog>
<metadata>
<data name="Category" value="Administrators"/>
- <data name="Category" value="Cloudera Manager"/>
</metadata>
</prolog>
@@ -358,19 +357,18 @@ I0107 08:42:12.292706 14876 logging.cc:76] Flags (see also /varz are on debug we
<title>Setting Logging Levels</title>
<conbody>
-
+
<p>
Impala uses the GLOG system, which supports three logging levels. You can adjust logging levels
by exporting variable settings. To change logging settings manually, use a command
similar to the following on each node before starting <codeph>impalad</codeph>:
</p>
-
-
<codeblock>export GLOG_v=1</codeblock>
<note>
- For performance reasons, Cloudera highly recommends not enabling the most verbose logging level of 3.
+ For performance reasons, do not enable the most verbose logging level of 3 unless there is
+ no other alternative for troubleshooting.
</note>
<p>
@@ -390,7 +388,7 @@ I0107 08:42:12.292706 14876 logging.cc:76] Flags (see also /varz are on debug we
</p>
<p>
- Increasing logging levels imposes performance overhead and increases log size. <ph rev="upstream">Cloudera</ph> recommends using
+ Increasing logging levels imposes performance overhead and increases log size. Where practical, use
GLOG_v=1 for most cases: this level has minimal performance impact but still captures useful
troubleshooting information.
</p>
@@ -443,7 +441,7 @@ I0107 08:42:12.292706 14876 logging.cc:76] Flags (see also /varz are on debug we
system, such as credit card numbers or tax IDs, and literals matching these patterns are obfuscated wherever
they would normally be recorded in log files or displayed in administration or debugging user interfaces.
</p>
-
+
<p>
In a security context, the log redaction feature is complementary to the Sentry authorization framework.
Sentry prevents unauthorized users from being able to directly access table data. Redaction prevents
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_new_features.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_new_features.xml b/docs/topics/impala_new_features.xml
index 91638a9..1c5678c 100644
--- a/docs/topics/impala_new_features.xml
+++ b/docs/topics/impala_new_features.xml
@@ -1044,7 +1044,7 @@ under the License.
now use an optimized code path.
</p>
</li>
-
+
<li>
<p rev="IMPALA-3044 IMPALA-2538 IMPALA-1168">
Improvements to the memory reservation mechanism for the Impala
@@ -1055,20 +1055,7 @@ under the License.
setting) is now unlimited instead of 200.
</p>
</li>
-
- <li audience="hidden">
- <p rev="IMPALA-3044 IMPALA-2538 IMPALA-1168 CDH-33289 CDH-34603">
- Improvements to the memory reservation mechanism for the Impala
- admission control feature. You can specify more settings, such
- as the timeout period and maximum aggregate memory used, for each
- resource pool instead of globally for the Impala instance. The
- default limit for concurrent queries (the <uicontrol>max requests</uicontrol>
- setting) is now unlimited instead of 200.
- The Cloudera Manager user interface for admission control has been
- reworked, with the settings available under the
- <uicontrol>Dynamic Resource Pools</uicontrol> window.
- </p>
- </li>
+
<li>
<p rev="IMPALA-1755">
Performance improvements related to code generation.
@@ -3395,14 +3382,6 @@ under the License.
information about Impala queries that succeed or are blocked due to insufficient privileges. For details,
see <xref href="impala_security.xml#security"/>.
</li>
-
-
- <li audience="hidden">
- Additional security feature: auditing. New startup options for <cmdname>impalad</cmdname> let you capture
- information about Impala queries that succeed or are blocked due to insufficient privileges. To take full
- advantage of this feature with Cloudera Manager, upgrade to Cloudera Manager 4.7 or higher. For details,
- see <xref href="impala_security.xml#security"/>.
- </li>
<li>
Parquet data files generated by Impala 1.1.1 are now compatible with the Parquet support in Hive. See
@@ -3445,13 +3424,6 @@ under the License.
groups. By assigning privileges for views, you can control access to table data at the column level. For
details, see <xref href="impala_security.xml#security"/>.
</li>
-
-
- <li audience="hidden">
- Impala 1.1 works with Cloudera Manager 4.6 or higher. To use Cloudera Manager to manage authorization for
- the Impala web UI (the web pages served from port 25000 by default), use Cloudera Manager 4.6.2 or
- higher.
- </li>
<li>
Impala can now create, alter, drop, and query views. Views provide a flexible way to set up simple
@@ -3570,12 +3542,6 @@ under the License.
<conbody>
- <p audience="hidden">
- The primary enhancements in Impala 1.0.1 are internal, for compatibility with the new Cloudera Manager 4.6
- release. Try out the new <uicontrol>Impala Query Monitoring</uicontrol> feature in Cloudera Manager 4.6,
- which requires Impala 1.0.1.
- </p>
-
<p>
New user-visible features include:
</p>
@@ -3694,12 +3660,6 @@ under the License.
</li>
</ul>
- <p audience="hidden">
- In this version, both CDH 4.1 and 4.2 are supported, but due to performance improvements added, we highly
- recommend you use CDH 4.2 or higher to see the full benefit. If you are using Cloudera Manager, version 4.5
- is required.
- </p>
-
</conbody>
</concept>
@@ -3732,10 +3692,6 @@ under the License.
</ul>
</li>
- <li audience="hidden">
- Cloudera Manager 4.5 and CDH 4.2 support Impala 0.6.
- </li>
-
<li>
Support for the RCFile file format. For more information on file formats, see
<xref href="impala_file_formats.xml#file_formats">Understanding File Formats</xref>.
@@ -3784,10 +3740,6 @@ under the License.
Added support for Impala on RHEL5.7/Centos5.7. Impala is now supported on RHEL5.7/6.2 and Centos5.7/6.2.
</li>
- <li audience="hidden">
- Cloudera Manager 4.1.3 supports Impala 0.4.
- </li>
-
<li>
The Impala debug webserver now has the ability to serve static files from
<codeph>${IMPALA_HOME}/www</codeph>. This can be disabled by setting
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_noncm_installation.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_noncm_installation.xml b/docs/topics/impala_noncm_installation.xml
index b00cbfb..d443b6e 100644
--- a/docs/topics/impala_noncm_installation.xml
+++ b/docs/topics/impala_noncm_installation.xml
@@ -173,16 +173,7 @@ $ sudo apt-get install impala-catalog # Service start/stop script
<xref href="impala_config_performance.xml#config_performance"/>. Some of these configuration changes are
mandatory.
</li>
-
- <li audience="hidden">
- Complete any required or recommended configuration, as described in
- <xref href="impala_config_performance.xml#config_performance"/>. Some of these configuration changes are
- mandatory. (They are applied automatically when you install using Cloudera Manager.)
- </li>
-
-
-
-
+
</ol>
<p>
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_perf_resources.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_perf_resources.xml b/docs/topics/impala_perf_resources.xml
index 907f36c..174e8f0 100644
--- a/docs/topics/impala_perf_resources.xml
+++ b/docs/topics/impala_perf_resources.xml
@@ -64,22 +64,13 @@ under the License.
hold cached metadata.
</p>
</li>
-
+
<li>
<p>
For production deployments, implement resource isolation using your cluster management
tool.
</p>
</li>
-
- <li audience="hidden">
- <p>
- For production deployment, <ph rev="upstream">Cloudera</ph> recommends that you implement resource isolation using mechanisms
- such as cgroups, which you can configure using Cloudera Manager. For details, see the
- <xref href="http://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_service_pools.html" scope="external" format="html">Static
- Resource Pools</xref> in the Cloudera Manager documentation.
- </p>
- </li>
</ul>
</conbody>
</concept>
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_perf_skew.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_perf_skew.xml b/docs/topics/impala_perf_skew.xml
index 277ee9e..510a1d6 100644
--- a/docs/topics/impala_perf_skew.xml
+++ b/docs/topics/impala_perf_skew.xml
@@ -67,22 +67,13 @@ under the License.
by a detailed performance analysis.
</p>
</li>
-
+
<li>
<p>
In the Impala debug web UI, click on the <uicontrol>Profile</uicontrol> link associated with the query after it is
complete. The executive summary information is displayed early in the profile output.
</p>
</li>
-
-
- <li audience="hidden">
- <p>
- In the Cloudera Manager interface or the Impala debug web UI, click on the <uicontrol>Profile</uicontrol>
- link associated with the query after it is complete. The executive summary information is displayed early
- in the profile output.
- </p>
- </li>
</ul>
<p>
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_perf_testing.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_perf_testing.xml b/docs/topics/impala_perf_testing.xml
index 858095f..e2fbfd5 100644
--- a/docs/topics/impala_perf_testing.xml
+++ b/docs/topics/impala_perf_testing.xml
@@ -39,18 +39,10 @@ under the License.
<conbody>
<p>
- Test to ensure that Impala is configured for optimal performance. If you have installed Impala with cluster
+ Test to ensure that Impala is configured for optimal performance. If you have installed Impala with cluster
management software, complete the processes described in this topic to help ensure a proper
configuration. These procedures can be used to verify that Impala is set up correctly.
</p>
-
-
- <p audience="hidden">
- Test to ensure that Impala is configured for optimal performance. If you have installed Impala without
- Cloudera Manager, complete the processes described in this topic to help ensure a proper configuration. Even
- if you installed Impala with Cloudera Manager, which automatically applies appropriate configurations, these
- procedures can be used to verify that Impala is set up correctly.
- </p>
<section id="checking_config_performance">
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_prereqs.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_prereqs.xml b/docs/topics/impala_prereqs.xml
index 50ac726..3fa6429 100644
--- a/docs/topics/impala_prereqs.xml
+++ b/docs/topics/impala_prereqs.xml
@@ -106,23 +106,14 @@ under the License.
the metastore database. For the process of installing and configuring the metastore, see
<xref href="impala_install.xml#install"/>.
</p>
- <p audience="hidden">
- Always configure a <b>Hive metastore service</b> rather than connecting directly to the metastore
- database. The Hive metastore service is required to interoperate between possibly different levels of
- metastore APIs used by CDH and Impala, and avoids known issues with connecting directly to the
- metastore database. The Hive metastore service is set up for you by default if you install through
- Cloudera Manager 4.5 or higher.
- </p>
-
+
<p>
Always configure a <b>Hive metastore service</b> rather than connecting directly to the metastore
database. The Hive metastore service is required to interoperate between different levels of
- metastore APIs if this is necessary for your environment, and using it avoids known issues with
+ metastore APIs if this is necessary for your environment, and using it avoids known issues with
connecting directly to the metastore database.
</p>
-
-
-
+
<p>
A summary of the metastore installation process is as follows:
</p>
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_proxy.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_proxy.xml b/docs/topics/impala_proxy.xml
index 3c4aa10..2f3a3f6 100644
--- a/docs/topics/impala_proxy.xml
+++ b/docs/topics/impala_proxy.xml
@@ -161,7 +161,7 @@ under the License.
</concept>
- <concept id="proxy_balancing" rev="CDH-33836 DOCS-349 CDH-39925 CDH-36812" audience="hidden">
+ <concept id="proxy_balancing" rev="" audience="hidden">
<title>Choosing the Load-Balancing Algorithm</title>
<conbody>
<p>
@@ -260,27 +260,19 @@ under the License.
running the <cmdname>impalad</cmdname> daemon.
</li>
-
- <li rev="CDH-40363" audience="hidden">
- For a cluster managed by Cloudera Manager (5.4.2 or higher), fill in the Impala configuration setting
- <uicontrol>Impala Daemons Load Balancer</uicontrol> with the appropriate host:port combination.
- Then restart the Impala service.
- For systems using a recent level of Cloudera Manager, this is all the configuration you need; you can skip the remaining steps in this procedure.
- </li>
-
<li>
Copy the keytab file from the proxy host to all other hosts in the cluster that run the
<cmdname>impalad</cmdname> daemon. (For optimal performance, <cmdname>impalad</cmdname> should be running
on all DataNodes in the cluster.) Put the keytab file in a secure location on each of these other hosts.
</li>
-
+
<li>
Add an entry <codeph>impala/<varname>actual_hostname</varname>@<varname>realm</varname></codeph> to the keytab on each
host running the <cmdname>impalad</cmdname> daemon.
</li>
-
+
<li>
-
+
For each impalad node, merge the existing keytab with the proxy\u2019s keytab using
<cmdname>ktutil</cmdname>, producing a new keytab file. For example:
<codeblock>$ ktutil
@@ -288,11 +280,11 @@ under the License.
ktutil: read_kt impala.keytab
ktutil: write_kt proxy_impala.keytab
ktutil: quit</codeblock>
-
+
</li>
-
+
<li>
-
+
To verify that the keytabs are merged, run the command:
<codeblock>
klist -k <varname>keytabfile</varname>
@@ -300,14 +292,14 @@ klist -k <varname>keytabfile</varname>
which lists the credentials for both <codeph>principal</codeph> and <codeph>be_principal</codeph> on
all nodes.
</li>
-
+
<li>
-
- Make sure that the <codeph>impala</codeph> user has permission to read this merged keytab file.
-
+
+ Make sure that the <codeph>impala</codeph> user has permission to read this merged keytab file.
+
</li>
-
+
<li>
Change the following configuration settings for each host in the cluster that participates
in the load balancing:
@@ -322,98 +314,27 @@ klist -k <varname>keytabfile</varname>
<note>
Every host has different <codeph>--be_principal</codeph> because the actual hostname
is different on each host.
-
+
Specify the fully qualified domain name (FQDN) for the proxy host, not the IP
address. Use the exact FQDN as returned by a reverse DNS lookup for the associated
IP address.
-
+
</note>
</li>
-
+
<li>
Modify the startup options. See <xref href="impala_config_options.xml#config_options"/> for the procedure to modify the startup
options.
</li>
</ul>
</li>
-
-
+
<li>
-
Restart Impala to make the changes take effect. Restart the <cmdname>impalad</cmdname> daemons on all
hosts in the cluster, as well as the <cmdname>statestored</cmdname> and <cmdname>catalogd</cmdname>
daemons.
-
- </li>
-
-
- <li audience="hidden">
- On systems not managed by Cloudera Manager, or systems using Cloudera Manager earlier than 5.4.2:
-
- <ol>
- <li>
- Copy the keytab file from the proxy host to all other hosts in the cluster that run the
- <cmdname>impalad</cmdname> daemon. (For optimal performance, <cmdname>impalad</cmdname> should be running
- on all DataNodes in the cluster.) Put the keytab file in a secure location on each of these other hosts.
- </li>
-
- <li>
- Add an entry <codeph>impala/<varname>actual_hostname</varname>@<varname>realm</varname></codeph> to the keytab on each
- host running the <cmdname>impalad</cmdname> daemon.
- </li>
-
- <li>
- For each impalad node, merge the existing keytab with the proxy\u2019s keytab using
- <cmdname>ktutil</cmdname>, producing a new keytab file. For example:
- <codeblock>$ ktutil
- ktutil: read_kt proxy.keytab
- ktutil: read_kt impala.keytab
- ktutil: write_kt proxy_impala.keytab
- ktutil: quit</codeblock>
- <note>
- On systems managed by Cloudera Manager 5.1.0 and later, the keytab merging happens automatically. To
- verify that Cloudera Manager has merged the keytabs, run the command:
- <codeblock>klist -k <varname>keytabfile</varname></codeblock>
- which lists the credentials for both <codeph>principal</codeph> and <codeph>be_principal</codeph> on
- all nodes.
- </note>
- </li>
-
- <li>
- Make sure that the <codeph>impala</codeph> user has permission to read this merged keytab file.
- </li>
-
- <li>
- Change some configuration settings for each host in the cluster that participates in the load balancing.
- Follow the appropriate steps depending on whether you use Cloudera Manager or not:
- <ul>
- <li> In the <cmdname>impalad</cmdname> option definition, or the advanced
- configuration snippet, add: <codeblock>--principal=impala/<varname>proxy_host</varname>@<varname>realm</varname>
- --be_principal=impala/<varname>actual_host</varname>@<varname>realm</varname>
- --keytab_file=<varname>path_to_merged_keytab</varname></codeblock>
- </li>
-
- <li>
- On a cluster not managed by Cloudera Manager, see
- <xref href="impala_config_options.xml#config_options"/> for the procedure to modify the startup
- options.
- </li>
- </ul>
- </li>
-
- <li>
- Restart Impala to make the changes take effect. Follow the appropriate steps depending on whether you use
- Cloudera Manager or not:
- <ul>
- <li>
- On a cluster not managed by Cloudera Manager, restart the <cmdname>impalad</cmdname> daemons on all
- hosts in the cluster, as well as the <cmdname>statestored</cmdname> and <cmdname>catalogd</cmdname>
- daemons.
- </li>
- </ul>
- </li>
- </ol>
</li>
+
</ol>
<!--
@@ -571,128 +492,6 @@ listen impalajdbc :21051
<note conref="../shared/impala_common.xml#common/proxy_jdbc_caveat"/>
- <p audience="hidden">
- The following example shows extra steps needed for a cluster using Kerberos authentication:
- </p>
-
-<codeblock audience="hidden">$ klist
-$ impala-shell -k
-$ kinit -r 1d -kt /systest/keytabs/hdfs.keytab hdfs
-$ impala-shell -i c2104.hal.cloudera.com:21000
-$ impala-shell -i c2104.hal.cloudera.com:25003
-[root@c2104 alan]# ps -ef |grep impalad
-root 6442 6428 0 12:21 pts/0 00:00:00 grep impalad
-impala 30577 22192 99 Nov14 ? 3-16:42:32 /usr/lib/impala/sbin-debug/impalad --flagfile=/var/run/cloudera-scm-agent/process/10342-impala-IMPALAD/impala-conf/impalad_flags
-[root@c2104 alan]# vi /var/run/cloudera-scm-agent/process/10342-impala-IMPALAD/impala-conf/impalad_flags
-$ klist -k /var/run/cloudera-scm-agent/process/10342-impala-IMPALAD/impala.keytab
-Keytab name: FILE:/var/run/cloudera-scm-agent/process/10342-impala-IMPALAD/impala.keytab
-KVNO Principal
----- --------------------------------------------------------------------------
- 2 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
-$ klist
-Ticket cache: FILE:/tmp/krb5cc_4028
-Default principal: hdfs@HAL17.CLOUDERA.COM
-
-Valid starting Expires Service principal
-11/15/13 12:17:17 11/15/13 12:32:17 krbtgt/HAL17.CLOUDERA.COM@HAL17.CLOUDERA.COM
- renew until 11/16/13 12:17:17
-11/15/13 12:17:21 11/15/13 12:32:17 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
- renew until 11/16/13 12:17:17
-$ kinit -r 1d -kt /systest/keytabs/hdfs.keytab hdfs
-$ kinit -R
-$ impala-shell -k -i c2106.hal.cloudera.com:21000
-Starting Impala Shell using Kerberos authentication
-Using service name 'impala'
-Connected to c2106.hal.cloudera.com:21000
-$ impala-shell -i c2104.hal.cloudera.com:25003
-$ impala-shell -k -i c2104.hal.cloudera.com:25003
-Starting Impala Shell using Kerberos authentication
-Using service name 'impala'
-Connected to c2104.hal.cloudera.com:25003
-[c2104.hal.cloudera.com:25003] > create table alan_tmp(a int);
-Query: create table alan_tmp(a int)
-ERROR: InternalException: Got exception: org.apache.hadoop.ipc.RemoteException User: hive/c2102.hal.cloudera.com@HAL17.CLOUDERA.COM is not allowed to impersonate impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
-$ kdestroy
-$ kinit -r 1d -kt /systest/keytabs/hdfs.keytab hdfs
-$ impala-shell -k -i c2104.hal.cloudera.com:25003
-# klist -k c2104.keytab
-Keytab name: FILE:c2104.keytab
-KVNO Principal
----- --------------------------------------------------------------------------
- 2 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
-$ klist -k -t c2106.keytab
-Keytab name: FILE:c2106.keytab
-KVNO Timestamp Principal
----- ----------------- --------------------------------------------------------
- 2 02/14/13 12:12:22 HTTP/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 02/14/13 12:12:22 HTTP/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 02/14/13 12:12:22 HTTP/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 02/14/13 12:12:22 HTTP/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 02/14/13 12:12:22 impala/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 02/14/13 12:12:22 impala/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 02/14/13 12:12:22 impala/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 02/14/13 12:12:22 impala/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
-$ ktutil
-ktutil: rkt c2104.keytab
-ktutil: rkt c2106.keytab
-ktutil: wkt my_test.keytab
-ktutil: q
-$ klist -k -t my_test.keytab
-Keytab name: FILE:my_test.keytab
-KVNO Timestamp Principal
----- ----------------- --------------------------------------------------------
- 2 11/21/13 16:22:40 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 11/21/13 16:22:40 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 11/21/13 16:22:40 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 11/21/13 16:22:40 impala/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 11/21/13 16:22:40 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 11/21/13 16:22:40 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 11/21/13 16:22:40 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 11/21/13 16:22:40 HTTP/c2104.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 11/21/13 16:22:40 HTTP/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 11/21/13 16:22:41 HTTP/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 11/21/13 16:22:41 HTTP/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 11/21/13 16:22:41 HTTP/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 11/21/13 16:22:41 impala/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 11/21/13 16:22:41 impala/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 11/21/13 16:22:41 impala/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
- 2 11/21/13 16:22:41 impala/c2106.hal.cloudera.com@HAL17.CLOUDERA.COM
-$ kdestroy
-$ kinit -r 1d -kt /systest/keytabs/hdfs.keytab hdfs
-$ vi README
-$ kinit -R
-$ impala-shell -k -i c2104.hal.cloudera.com:25003
-Starting Impala Shell using Kerberos authentication
-Using service name 'impala'
-Connected to c2104.hal.cloudera.com:25003
-<ph conref="../shared/ImpalaVariables.xml#impala_vars/ImpaladBanner"/>
-Welcome to the Impala shell. Press TAB twice to see a list of available commands.
-...
-<ph conref="../shared/ImpalaVariables.xml#impala_vars/ShellBanner"/>
-[c2104.hal.cloudera.com:25003] > show tables;
-Query: show tables
-ERROR: AnalysisException: This Impala daemon is not ready to accept user requests. Status: Waiting for catalog update from the StateStore.
-[c2104.hal.cloudera.com:25003] > quit;</codeblock>
-
- <!--
- At that point in the walkthrough with Alan Choi, we could never get Impala to accept any requests through the catalog server.
- So I have not seen a 100% successful proxy setup process to verify all the details.
- -->
-
</conbody>
</concept>
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_resource_management.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_resource_management.xml b/docs/topics/impala_resource_management.xml
index ade204f..b4174e3 100644
--- a/docs/topics/impala_resource_management.xml
+++ b/docs/topics/impala_resource_management.xml
@@ -61,21 +61,7 @@ under the License.
can be safely run at the same time. Then the Impala daemon enforces the limit by activating the
spill-to-disk mechanism when necessary, or cancelling a query altogether if the limit is exceeded at runtime.
</p>
-
-
- <ul audience="hidden">
- <li>
- If Cloudera Manager Static Partitioning is used, it creates a cgroup in which Impala runs.
- This cgroup limits CPU, network, and IO according to the static partitioning policy.
- </li>
- <li>
- Limits on memory usage are enforced by Impala's process memory limit (the <codeph>MEM_LIMIT</codeph>
- query option setting). The admission control feature checks this setting to decide how many queries
- can be safely run at the same time. Then the Impala daemon enforces the limit by activating the
- spill-to-disk mechanism when necessary, or cancelling a query altogether if the limit is exceeded at runtime.
- </li>
- </ul>
</conbody>
</concept>
@@ -110,11 +96,6 @@ under the License.
cluster.
</p>
- <p>
- For information about setting up the YARN service, see the instructions for
- <xref href="http://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_yarn_service.html" scope="external" format="html">Cloudera
- Manager</xref>.
- </p>
</conbody>
</concept>
@@ -144,7 +125,7 @@ under the License.
<codeph>-rm_always_use_defaults</codeph>: If this Boolean option is enabled, Impala ignores computed
estimates and always obtains the default memory and CPU allocation settings at the start of the
query. These default estimates are approximately 2 CPUs and 4 GB of memory, possibly varying slightly
- depending on cluster size, workload, and so on. <ph rev="upstream">Cloudera</ph> recommends enabling
+ depending on cluster size, workload, and so on. Where practical, enable
<codeph>-rm_always_use_defaults</codeph> whenever resource management is used, and relying on these
default values (that is, leaving out the two following options).
</li>
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_s3.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_s3.xml b/docs/topics/impala_s3.xml
index dd29748..5131796 100644
--- a/docs/topics/impala_s3.xml
+++ b/docs/topics/impala_s3.xml
@@ -108,7 +108,7 @@ under the License.
form of the <codeph>CREATE TABLE</codeph> statement, can copy data from an HDFS table or another S3
table into an S3 table. The <xref href="impala_s3_skip_insert_staging.xml#s3_skip_insert_staging"/>
query option chooses whether or not to use a fast code path for these write operations to S3,
- with the tradeoff of potential inconsistency in the case of a failure during the statement.
+ with the tradeoff of potential inconsistency in the case of a failure during the statement.
</p>
</li>
</ul>
@@ -152,7 +152,7 @@ under the License.
Hive services. (Restarting Hive is required because Impala queries, CREATE TABLE statements, and so on go
through the Hive metastore.)
</p>
-
+
<note type="important">
<!--
<ul>
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_scalability.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_scalability.xml b/docs/topics/impala_scalability.xml
index f3a7d4c..588aa29 100644
--- a/docs/topics/impala_scalability.xml
+++ b/docs/topics/impala_scalability.xml
@@ -294,11 +294,6 @@ Memory Usage: Additional Notes
</dlentry>
</dl>
- <p audience="hidden">
- As of CDH 5.3, not all of these flags are present in the Cloudera Manager user interface. Some must be set
- using the <uicontrol>Advanced Configuration Snippet</uicontrol> fields for the statestore component.
- </p>
-
<p>
If it takes a very long time for a cluster to start up, and <cmdname>impala-shell</cmdname> consistently
displays <codeph>This Impala daemon is not ready to accept user requests</codeph>, the statestore might be
@@ -682,32 +677,6 @@ these tables, hint the plan or disable this behavior via query options to enable
case, you know that any queries that spill will not go overboard with their memory consumption.
</p>
-<!--
- <p>
- <b>Turning off the spill-to-disk feature: (<keyword keyref="impala24_full"/> and lower only)</b>
- </p>
-
- <p>
- Prior to <keyword keyref="impala25_full"/> certain conditions...
- </p>
-
- <p>
- You might turn off the spill-to-disk feature if you are in an environment with constraints on disk space,
- or if you prefer for queries that exceed the memory capacity in your cluster to <q>fail fast</q> so that
- you can tune and retry them.
- </p>
-
- <p>
- To turn off this feature, set the following configuration options for each <cmdname>impalad</cmdname>
- daemon, either through the <cmdname>impalad</cmdname> advanced configuration snippet in Cloudera Manager,
- or during <cmdname>impalad</cmdname> startup on each DataNode on systems not managed by Cloudera Manager:
- </p>
-
-<codeblock>−−enable_partitioned_aggregation=false
-−−enable_partitioned_hash_join=false
-</codeblock>
--->
-
</conbody>
</concept>
@@ -795,21 +764,14 @@ While these authentication requests are being processed, any submitted Impala qu
During this period, the KDC and DNS may be slow to respond to requests from components other than Impala,
so other secure services might be affected temporarily.
</p>
-
+
<p>
To reduce the frequency of the <codeph>kinit</codeph> renewal that initiates
- a new set of authentication requests, increase the <codeph>kerberos_reinit_interval</codeph>
- configuration setting for the <cmdname>impalad</cmdname> daemons. Currently, the default is 60 minutes.
- Consider using a higher value such as 360 (6 hours).
-</p>
-
-<p audience="hidden">
-To reduce the frequency of the <codeph>kinit</codeph> renewal that initiates a new set of
-authentication requests, increase the <codeph>kerberos_reinit_interval</codeph> configuration setting
-for the <cmdname>impalad</cmdname> daemons. Currently, the default for a cluster not managed by
-Cloudera Manager is 60 minutes, while the default under Cloudera Manager is 10 minutes.
-Consider using a higher value such as 360 (6 hours).
+ a new set of authentication requests, increase the <codeph>kerberos_reinit_interval</codeph>
+ configuration setting for the <cmdname>impalad</cmdname> daemons. Currently, the default is 60 minutes.
+ Consider using a higher value such as 360 (6 hours).
</p>
+
</conbody>
</concept>
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_schema_design.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_schema_design.xml b/docs/topics/impala_schema_design.xml
index dbe98a2..711c61b 100644
--- a/docs/topics/impala_schema_design.xml
+++ b/docs/topics/impala_schema_design.xml
@@ -229,12 +229,6 @@ under the License.
<p>
</p>
</section>
-
-<section id="schema_design_cm">
-<title>Use Cloudera Manager to monitor queries and overall performance.</title>
-<p>
-</p>
-</section>
-->
</conbody>
</concept>
http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/0467b0d5/docs/topics/impala_timeouts.xml
----------------------------------------------------------------------
diff --git a/docs/topics/impala_timeouts.xml b/docs/topics/impala_timeouts.xml
index 0857f11..63053a0 100644
--- a/docs/topics/impala_timeouts.xml
+++ b/docs/topics/impala_timeouts.xml
@@ -169,7 +169,7 @@ Trying to re-register with state-store</codeblock>
<title>Cancelling a Query</title>
<conbody>
-
+
<p>
Sometimes, an Impala query might run for an unexpectedly long time, tying up resources
in the cluster. You can cancel the query explicitly, independent of the timeout period,
@@ -177,21 +177,6 @@ Trying to re-register with state-store</codeblock>
default), and using the link on the <codeph>/queries</codeph> tab to cancel the running
query. For example, press <codeph>^C</codeph> in <cmdname>impala-shell</cmdname>.
</p>
-
-
-
- <p audience="hidden">
- Sometimes, an Impala query might run for an unexpectedly long time, tying up resources
- in the cluster. You can cancel the query explicitly, independent of the timeout period,
- by going into the web UI for the <cmdname>impalad</cmdname> host (on port 25000 by
- default), and using the link on the <codeph>/queries</codeph> tab to cancel the running
- query. Various client applications let you interactively cancel queries submitted or
- monitored through those applications. For example, by pressing <codeph>^C</codeph> in
- <cmdname>impala-shell</cmdname>, clicking the <uicontrol>Cancel</uicontrol> button from
- the <uicontrol>Watch</uicontrol> page in Hue, clicking <uicontrol>Actions >
- Cancel</uicontrol> from the <uicontrol>Queries</uicontrol> list in Cloudera Manager, and
- so on.
- </p>
</conbody>