You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by ar...@apache.org on 2019/08/23 01:39:36 UTC
[impala] branch asf-site updated: [DOCS] Impala 3.3 docs to be
published
This is an automated email from the ASF dual-hosted git repository.
arodoni pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/impala.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 1e19488 [DOCS] Impala 3.3 docs to be published
1e19488 is described below
commit 1e19488a16134ed293fab7a100bd77fa009fb604
Author: Alex Rodoni <ar...@cloudera.com>
AuthorDate: Wed Aug 21 10:14:35 2019 -0700
[DOCS] Impala 3.3 docs to be published
- Add Impala docs from branch master,
commit hash 623e923cc6e190e1cbcb991cb3eadc3e349556b4
Change-Id: I34427a8766ab8989f6bcb57e06ee2fc493e00f77
Reviewed-on: http://gerrit.cloudera.org:8080/14115
Tested-by: Alex Rodoni <ar...@cloudera.com>
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
---
docs/build/html/commonltr.css | 2 +-
docs/build/html/commonrtl.css | 2 +-
docs/build/html/index.html | 2 +-
docs/build/html/topics/impala_abort_on_error.html | 11 +-
docs/build/html/topics/impala_adls.html | 49 +-
docs/build/html/topics/impala_admin.html | 4 +-
docs/build/html/topics/impala_admission.html | 141 +-
.../build/html/topics/impala_admission_config.html | 432 +--
.../html/topics/impala_aggregate_functions.html | 20 +-
docs/build/html/topics/impala_aliases.html | 37 +-
.../topics/impala_allow_erasure_coded_files.html | 8 +-
.../topics/impala_allow_unsupported_formats.html | 11 +-
docs/build/html/topics/impala_alter_database.html | 8 +-
docs/build/html/topics/impala_alter_table.html | 76 +-
docs/build/html/topics/impala_alter_view.html | 20 +-
.../html/topics/impala_analytic_functions.html | 50 +-
.../html/topics/impala_appx_count_distinct.html | 11 +-
docs/build/html/topics/impala_appx_median.html | 11 +-
docs/build/html/topics/impala_array.html | 81 +-
docs/build/html/topics/impala_auditing.html | 134 +-
docs/build/html/topics/impala_authentication.html | 27 +-
docs/build/html/topics/impala_authorization.html | 1490 +++++-----
docs/build/html/topics/impala_avg.html | 51 +-
docs/build/html/topics/impala_avro.html | 115 +-
docs/build/html/topics/impala_batch_size.html | 4 +-
docs/build/html/topics/impala_bigint.html | 43 +-
docs/build/html/topics/impala_bit_functions.html | 8 +-
docs/build/html/topics/impala_boolean.html | 22 +-
docs/build/html/topics/impala_breakpad.html | 8 +-
.../html/topics/impala_buffer_pool_limit.html | 4 +-
docs/build/html/topics/impala_char.html | 51 +-
docs/build/html/topics/impala_client.html | 332 +++
docs/build/html/topics/impala_comment.html | 8 +-
docs/build/html/topics/impala_comments.html | 4 +-
docs/build/html/topics/impala_complex_types.html | 68 +-
docs/build/html/topics/impala_components.html | 93 +-
.../html/topics/impala_compression_codec.html | 38 +-
docs/build/html/topics/impala_compute_stats.html | 88 +-
.../impala_compute_stats_min_sample_size.html | 10 +-
docs/build/html/topics/impala_concepts.html | 4 +-
.../html/topics/impala_conditional_functions.html | 92 +-
docs/build/html/topics/impala_config.html | 10 +-
docs/build/html/topics/impala_config_options.html | 235 +-
.../html/topics/impala_config_performance.html | 4 +-
docs/build/html/topics/impala_connecting.html | 96 +-
.../html/topics/impala_conversion_functions.html | 982 ++++++-
docs/build/html/topics/impala_count.html | 49 +-
docs/build/html/topics/impala_create_database.html | 32 +-
docs/build/html/topics/impala_create_function.html | 72 +-
docs/build/html/topics/impala_create_role.html | 6 +-
docs/build/html/topics/impala_create_table.html | 284 +-
docs/build/html/topics/impala_create_view.html | 29 +-
.../build/html/topics/impala_custom_timezones.html | 38 +-
docs/build/html/topics/impala_data_cache.html | 108 +
docs/build/html/topics/impala_databases.html | 11 +-
docs/build/html/topics/impala_datatypes.html | 11 +-
docs/build/html/topics/impala_date.html | 240 ++
.../html/topics/impala_datetime_functions.html | 3022 +++++++++-----------
docs/build/html/topics/impala_ddl.html | 8 +-
docs/build/html/topics/impala_debug_action.html | 4 +-
docs/build/html/topics/impala_decimal.html | 54 +-
docs/build/html/topics/impala_decimal_v2.html | 30 +-
.../html/topics/impala_dedicated_coordinator.html | 41 +-
.../html/topics/impala_default_file_format.html | 8 +-
.../impala_default_hints_insert_statement.html | 86 +
.../impala_default_join_distribution_mode.html | 4 +-
.../impala_default_spillable_buffer_size.html | 11 +-
.../topics/impala_default_transactional_type.html | 96 +
docs/build/html/topics/impala_delegation.html | 69 +-
docs/build/html/topics/impala_delete.html | 21 +-
docs/build/html/topics/impala_describe.html | 58 +-
docs/build/html/topics/impala_development.html | 4 +-
docs/build/html/topics/impala_disable_codegen.html | 7 +-
.../impala_disable_codegen_rows_threshold.html | 10 +-
.../impala_disable_row_runtime_filtering.html | 12 +-
.../impala_disable_streaming_preaggregations.html | 22 +-
.../html/topics/impala_disable_unsafe_spills.html | 11 +-
docs/build/html/topics/impala_disk_space.html | 35 +-
docs/build/html/topics/impala_distinct.html | 14 +-
docs/build/html/topics/impala_dml.html | 4 +-
docs/build/html/topics/impala_double.html | 65 +-
docs/build/html/topics/impala_drop_database.html | 16 +-
docs/build/html/topics/impala_drop_function.html | 28 +-
docs/build/html/topics/impala_drop_role.html | 6 +-
docs/build/html/topics/impala_drop_stats.html | 22 +-
docs/build/html/topics/impala_drop_table.html | 44 +-
docs/build/html/topics/impala_drop_view.html | 13 +-
.../html/topics/impala_enable_expr_rewrites.html | 8 +-
.../impala_exec_single_node_rows_threshold.html | 4 +-
.../html/topics/impala_exec_time_limit_s.html | 15 +-
docs/build/html/topics/impala_explain.html | 35 +-
docs/build/html/topics/impala_explain_level.html | 8 +-
docs/build/html/topics/impala_explain_plan.html | 79 +-
docs/build/html/topics/impala_faq.html | 4 +-
docs/build/html/topics/impala_file_formats.html | 117 +-
docs/build/html/topics/impala_fixed_issues.html | 1487 +++++-----
docs/build/html/topics/impala_float.html | 65 +-
docs/build/html/topics/impala_functions.html | 15 +-
.../html/topics/impala_functions_overview.html | 4 +-
docs/build/html/topics/impala_grant.html | 985 +++++--
docs/build/html/topics/impala_group_by.html | 10 +-
docs/build/html/topics/impala_group_concat.html | 13 +-
docs/build/html/topics/impala_hadoop.html | 4 +-
docs/build/html/topics/impala_having.html | 4 +-
docs/build/html/topics/impala_hbase.html | 48 +-
.../html/topics/impala_hbase_cache_blocks.html | 11 +-
docs/build/html/topics/impala_hbase_caching.html | 4 +-
docs/build/html/topics/impala_hints.html | 47 +-
docs/build/html/topics/impala_identifiers.html | 20 +-
.../html/topics/impala_idle_session_timeout.html | 17 +-
docs/build/html/topics/impala_impala_shell.html | 13 +-
.../html/topics/impala_incompatible_changes.html | 1864 +++++-------
docs/build/html/topics/impala_insert.html | 210 +-
docs/build/html/topics/impala_install.html | 4 +-
docs/build/html/topics/impala_int.html | 37 +-
docs/build/html/topics/impala_intro.html | 80 +-
.../html/topics/impala_invalidate_metadata.html | 63 +-
docs/build/html/topics/impala_isilon.html | 90 +-
docs/build/html/topics/impala_jdbc.html | 370 ++-
docs/build/html/topics/impala_joins.html | 22 +-
docs/build/html/topics/impala_kerberos.html | 298 +-
docs/build/html/topics/impala_known_issues.html | 274 +-
docs/build/html/topics/impala_kudu.html | 277 +-
docs/build/html/topics/impala_kudu_read_mode.html | 11 +-
docs/build/html/topics/impala_langref.html | 7 +-
docs/build/html/topics/impala_langref_sql.html | 8 +-
.../html/topics/impala_langref_unsupported.html | 27 +-
docs/build/html/topics/impala_ldap.html | 75 +-
docs/build/html/topics/impala_limit.html | 36 +-
docs/build/html/topics/impala_lineage.html | 6 +-
docs/build/html/topics/impala_literals.html | 400 +--
docs/build/html/topics/impala_live_progress.html | 49 +-
docs/build/html/topics/impala_live_summary.html | 55 +-
docs/build/html/topics/impala_load_data.html | 108 +-
docs/build/html/topics/impala_logging.html | 309 +-
docs/build/html/topics/impala_map.html | 81 +-
docs/build/html/topics/impala_math_functions.html | 76 +-
docs/build/html/topics/impala_max.html | 60 +-
docs/build/html/topics/impala_max_errors.html | 4 +-
.../impala_max_mem_estimate_for_admission.html | 8 +-
.../topics/impala_max_num_runtime_filters.html | 14 +-
docs/build/html/topics/impala_max_row_size.html | 11 +-
.../html/topics/impala_max_scan_range_length.html | 4 +-
docs/build/html/topics/impala_mem_limit.html | 94 +-
docs/build/html/topics/impala_metadata.html | 401 ++-
docs/build/html/topics/impala_min.html | 60 +-
.../topics/impala_min_spillable_buffer_size.html | 11 +-
docs/build/html/topics/impala_misc_functions.html | 160 +-
docs/build/html/topics/impala_mixed_security.html | 4 +-
docs/build/html/topics/impala_mt_dop.html | 10 +-
docs/build/html/topics/impala_ndv.html | 43 +-
docs/build/html/topics/impala_new_features.html | 525 +++-
docs/build/html/topics/impala_num_nodes.html | 21 +-
.../topics/impala_num_rows_produced_limit.html | 8 +-
.../html/topics/impala_num_scanner_threads.html | 4 +-
docs/build/html/topics/impala_odbc.html | 8 +-
docs/build/html/topics/impala_offset.html | 19 +-
docs/build/html/topics/impala_operators.html | 286 +-
.../impala_optimize_partition_key_scans.html | 59 +-
docs/build/html/topics/impala_orc.html | 82 +-
docs/build/html/topics/impala_order_by.html | 88 +-
docs/build/html/topics/impala_parquet.html | 265 +-
.../impala_parquet_annotate_strings_utf8.html | 11 +-
.../topics/impala_parquet_array_resolution.html | 10 +-
.../topics/impala_parquet_compression_codec.html | 4 +-
.../impala_parquet_dictionary_filtering.html | 8 +-
.../impala_parquet_fallback_schema_resolution.html | 4 +-
.../html/topics/impala_parquet_file_size.html | 31 +-
...ml => impala_parquet_page_row_count_limit.html} | 32 +-
...cs.html => impala_parquet_read_page_index.html} | 65 +-
.../topics/impala_parquet_read_statistics.html | 8 +-
.../topics/impala_parquet_write_page_index.html | 90 +
docs/build/html/topics/impala_partitioning.html | 155 +-
.../html/topics/impala_perf_benchmarking.html | 4 +-
docs/build/html/topics/impala_perf_cookbook.html | 10 +-
.../html/topics/impala_perf_hdfs_caching.html | 33 +-
docs/build/html/topics/impala_perf_joins.html | 25 +-
docs/build/html/topics/impala_perf_resources.html | 4 +-
docs/build/html/topics/impala_perf_skew.html | 4 +-
docs/build/html/topics/impala_perf_stats.html | 98 +-
docs/build/html/topics/impala_perf_testing.html | 22 +-
docs/build/html/topics/impala_performance.html | 22 +-
docs/build/html/topics/impala_planning.html | 4 +-
docs/build/html/topics/impala_porting.html | 18 +-
docs/build/html/topics/impala_ports.html | 243 +-
docs/build/html/topics/impala_prefetch_mode.html | 4 +-
docs/build/html/topics/impala_prereqs.html | 16 +-
docs/build/html/topics/impala_processes.html | 28 +-
docs/build/html/topics/impala_proxy.html | 72 +-
docs/build/html/topics/impala_query_options.html | 23 +-
docs/build/html/topics/impala_query_timeout_s.html | 25 +-
docs/build/html/topics/impala_rcfile.html | 72 +-
docs/build/html/topics/impala_real.html | 4 +-
docs/build/html/topics/impala_refresh.html | 67 +-
.../html/topics/impala_refresh_authorization.html | 8 +-
.../html/topics/impala_refresh_functions.html | 8 +-
docs/build/html/topics/impala_release_notes.html | 4 +-
docs/build/html/topics/impala_relnotes.html | 2 +-
.../html/topics/impala_replica_preference.html | 4 +-
docs/build/html/topics/impala_request_pool.html | 4 +-
docs/build/html/topics/impala_reserved_words.html | 8 +-
.../html/topics/impala_resource_management.html | 4 +-
.../html/topics/impala_resource_trace_ratio.html | 8 +-
docs/build/html/topics/impala_revoke.html | 71 +-
.../topics/impala_runtime_bloom_filter_size.html | 18 +-
.../topics/impala_runtime_filter_max_size.html | 14 +-
.../topics/impala_runtime_filter_min_size.html | 14 +-
.../html/topics/impala_runtime_filter_mode.html | 4 +-
.../topics/impala_runtime_filter_wait_time_ms.html | 8 +-
.../html/topics/impala_runtime_filtering.html | 10 +-
docs/build/html/topics/impala_s3.html | 148 +-
.../html/topics/impala_s3_skip_insert_staging.html | 15 +-
docs/build/html/topics/impala_scalability.html | 263 +-
docs/build/html/topics/impala_scaling_limits.html | 351 +++
.../build/html/topics/impala_scan_bytes_limit.html | 10 +-
.../topics/impala_schedule_random_replica.html | 8 +-
docs/build/html/topics/impala_schema_design.html | 4 +-
docs/build/html/topics/impala_schema_objects.html | 4 +-
docs/build/html/topics/impala_scratch_limit.html | 4 +-
docs/build/html/topics/impala_security.html | 8 +-
docs/build/html/topics/impala_security_files.html | 6 +-
.../html/topics/impala_security_guidelines.html | 4 +-
.../build/html/topics/impala_security_install.html | 4 +-
.../html/topics/impala_security_metastore.html | 4 +-
docs/build/html/topics/impala_security_webui.html | 8 +-
docs/build/html/topics/impala_select.html | 62 +-
docs/build/html/topics/impala_seqfile.html | 70 +-
docs/build/html/topics/impala_set.html | 4 +-
docs/build/html/topics/impala_shell_commands.html | 122 +-
docs/build/html/topics/impala_shell_options.html | 311 +-
.../html/topics/impala_shell_running_commands.html | 8 +-
docs/build/html/topics/impala_show.html | 344 ++-
.../html/topics/impala_shuffle_distinct_exprs.html | 14 +-
docs/build/html/topics/impala_shutdown.html | 40 +-
docs/build/html/topics/impala_smallint.html | 37 +-
docs/build/html/topics/impala_ssl.html | 27 +-
docs/build/html/topics/impala_stddev.html | 11 +-
docs/build/html/topics/impala_string.html | 45 +-
.../build/html/topics/impala_string_functions.html | 500 ++--
docs/build/html/topics/impala_struct.html | 79 +-
docs/build/html/topics/impala_subqueries.html | 28 +-
docs/build/html/topics/impala_sum.html | 51 +-
.../html/topics/impala_support_start_over.html | 8 +-
docs/build/html/topics/impala_sync_ddl.html | 15 +-
docs/build/html/topics/impala_tables.html | 151 +-
docs/build/html/topics/impala_tablesample.html | 4 +-
.../impala_thread_reservation_aggregate_limit.html | 8 +-
.../topics/impala_thread_reservation_limit.html | 10 +-
docs/build/html/topics/impala_timeouts.html | 27 +-
docs/build/html/topics/impala_timestamp.html | 307 +-
docs/build/html/topics/impala_timezone.html | 8 +-
docs/build/html/topics/impala_tinyint.html | 45 +-
.../build/html/topics/impala_topn_bytes_limit.html | 8 +-
docs/build/html/topics/impala_transactions.html | 83 +
docs/build/html/topics/impala_troubleshooting.html | 64 +-
docs/build/html/topics/impala_truncate_table.html | 7 +-
docs/build/html/topics/impala_tutorial.html | 46 +-
docs/build/html/topics/impala_txtfile.html | 163 +-
docs/build/html/topics/impala_udf.html | 120 +-
docs/build/html/topics/impala_union.html | 4 +-
docs/build/html/topics/impala_update.html | 21 +-
docs/build/html/topics/impala_upgrading.html | 131 +-
docs/build/html/topics/impala_upsert.html | 21 +-
docs/build/html/topics/impala_use.html | 4 +-
docs/build/html/topics/impala_varchar.html | 48 +-
docs/build/html/topics/impala_variance.html | 11 +-
docs/build/html/topics/impala_views.html | 70 +-
docs/build/html/topics/impala_webui.html | 1401 +++++++--
docs/build/html/topics/impala_with.html | 14 +-
docs/build/impala-3.3.pdf | Bin 0 -> 4106689 bytes
270 files changed, 15632 insertions(+), 10971 deletions(-)
diff --git a/docs/build/html/commonltr.css b/docs/build/html/commonltr.css
index a7d6996..8205ad9 100644
--- a/docs/build/html/commonltr.css
+++ b/docs/build/html/commonltr.css
@@ -337,7 +337,7 @@ div.imagejustify {
{border-right:solid}
This value creates a very thick border in Firefox (does not match other tables)
- Firefox works with
+ Firefox works with
{border-right:solid 1pt}
but this causes a barely visible line in IE */
.cellrowborder {
diff --git a/docs/build/html/commonrtl.css b/docs/build/html/commonrtl.css
index b1318c9..59baa89 100644
--- a/docs/build/html/commonrtl.css
+++ b/docs/build/html/commonrtl.css
@@ -340,7 +340,7 @@ div.imagejustify {
{border-right:solid}
This value creates a very thick border in Firefox (does not match other tables)
- Firefox works with
+ Firefox works with
{border-right:solid 1pt}
but this causes a barely visible line in IE */
.cellrowborder {
diff --git a/docs/build/html/index.html b/docs/build/html/index.html
index 052867f..2761a54 100644
--- a/docs/build/html/index.html
+++ b/docs/build/html/index.html
@@ -12,4 +12,4 @@
<meta name="DC.Identifier" content="impala" />
<link rel="stylesheet" type="text/css" href="commonltr.css" />
<title>Apache Impala Guide</title></head>
-<body id="impala"><h1 class="title topictitle1">Apache Impala Guide</h1><div><ul class="map"><li class="topicref"><a href="topics/impala_intro.html">Introducing Apache Impala</a></li><li class="topicref"><a href="topics/impala_concepts.html">Concepts and Architecture</a><ul><li class="topicref"><a href="topics/impala_components.html">Components</a></li><li class="topicref"><a href="topics/impala_development.html">Developing Applications</a></li><li class="topicref"><a href="topics/impala [...]
\ No newline at end of file
+<body id="impala"><h1 class="title topictitle1">Apache Impala Guide</h1><div><ul class="map"><li class="topicref"><a href="topics/impala_intro.html">Introducing Apache Impala</a></li><li class="topicref"><a href="topics/impala_concepts.html">Concepts and Architecture</a><ul><li class="topicref"><a href="topics/impala_components.html">Components</a></li><li class="topicref"><a href="topics/impala_development.html">Developing Applications</a></li><li class="topicref"><a href="topics/impala [...]
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_abort_on_error.html b/docs/build/html/topics/impala_abort_on_error.html
index 165ca99..6e4eccb 100644
--- a/docs/build/html/topics/impala_abort_on_error.html
+++ b/docs/build/html/topics/impala_abort_on_error.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="abort_on_error" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -45,12 +45,13 @@
<p class="p">
- <strong class="ph b">Type:</strong> Boolean; recognized values are 1 and 0, or <code class="ph codeph">true</code> and <code class="ph codeph">false</code>;
- any other value interpreted as <code class="ph codeph">false</code>
+ <strong class="ph b">Type:</strong> Boolean; recognized values are 1 and 0, or <code class="ph codeph">true</code> and
+ <code class="ph codeph">false</code>; any other value interpreted as <code class="ph codeph">false</code>
</p>
<p class="p">
- <strong class="ph b">Default:</strong> <code class="ph codeph">false</code> (shown as 0 in output of <code class="ph codeph">SET</code> statement)
+ <strong class="ph b">Default:</strong> <code class="ph codeph">false</code> (shown as 0 in output of <code class="ph codeph">SET</code>
+ statement)
</p>
diff --git a/docs/build/html/topics/impala_adls.html b/docs/build/html/topics/impala_adls.html
index ebe9129..cb08957 100644
--- a/docs/build/html/topics/impala_adls.html
+++ b/docs/build/html/topics/impala_adls.html
@@ -9,7 +9,7 @@
<meta name="DC.Type" content="concept" />
<meta name="DC.Title" content="Using Impala with the Azure Data Lake Store (ADLS)" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="adls" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -221,7 +221,7 @@
<value>https://login.microsoftonline.com/<var class="keyword varname">your_azure_tenant_id</var>/oauth2/token</value>
</property></code></pre>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
<p class="p">
Check if your Hadoop distribution or cluster management tool includes support for
filling in and distributing credentials across the cluster in an automated way.
@@ -264,20 +264,23 @@
<h3 class="title topictitle3" id="ariaid-title6">Using Impala DML Statements for ADLS Data</h3>
<div class="body conbody">
- <p class="p"> In <span class="keyword">Impala 2.9</span> and higher, the Impala DML statements
- (<code class="ph codeph">INSERT</code>, <code class="ph codeph">LOAD DATA</code>, and
- <code class="ph codeph">CREATE TABLE AS SELECT</code>) can write data into a table
- or partition that resides in the Azure Data Lake Store (ADLS). ADLS Gen2
- is supported in <span class="keyword">Impala 3.1</span> and higher.</p>
-<p class="p">In the<code class="ph codeph">CREATE TABLE</code> or
- <code class="ph codeph">ALTER TABLE</code> statements, specify the ADLS location for
- tables and partitions with the <code class="ph codeph">adl://</code> prefix for ADLS
- Gen1 and <code class="ph codeph">abfs://</code> or <code class="ph codeph">abfss://</code> for ADLS
- Gen2 in the <code class="ph codeph">LOCATION</code> attribute.</p>
-<p class="p">If you bring data into ADLS
- using the normal ADLS transfer mechanisms instead of Impala DML
- statements, issue a <code class="ph codeph">REFRESH</code> statement for the table
- before using Impala to query the ADLS data. </p>
+ <p class="p">
+ In <span class="keyword">Impala 2.9</span> and higher, the Impala DML statements
+ (<code class="ph codeph">INSERT</code>, <code class="ph codeph">LOAD DATA</code>, and <code class="ph codeph">CREATE TABLE AS
+ SELECT</code>) can write data into a table or partition that resides in the Azure Data
+ Lake Store (ADLS). ADLS Gen2 is supported in <span class="keyword">Impala 3.1</span> and higher.
+ </p>
+<p class="p">
+ In the<code class="ph codeph">CREATE TABLE</code> or <code class="ph codeph">ALTER TABLE</code> statements, specify
+ the ADLS location for tables and partitions with the <code class="ph codeph">adl://</code> prefix for
+ ADLS Gen1 and <code class="ph codeph">abfs://</code> or <code class="ph codeph">abfss://</code> for ADLS Gen2 in the
+ <code class="ph codeph">LOCATION</code> attribute.
+ </p>
+<p class="p">
+ If you bring data into ADLS using the normal ADLS transfer mechanisms instead of Impala
+ DML statements, issue a <code class="ph codeph">REFRESH</code> statement for the table before using
+ Impala to query the ADLS data.
+ </p>
</div>
@@ -355,7 +358,7 @@
storage account.
</p>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
<p class="p"> By default, TLS is enabled both with <code class="ph codeph">abfs://</code> and
<code class="ph codeph">abfss://</code>. </p>
@@ -724,11 +727,11 @@
<p class="p">
- Because ADLS does not expose the block sizes of data files the way HDFS does,
- any Impala <code class="ph codeph">INSERT</code> or <code class="ph codeph">CREATE TABLE AS SELECT</code> statements
- use the <code class="ph codeph">PARQUET_FILE_SIZE</code> query option setting to define the size of
- Parquet data files. (Using a large block size is more important for Parquet tables than
- for tables that use other file formats.)
+ Because ADLS does not expose the block sizes of data files the way HDFS does, any Impala
+ <code class="ph codeph">INSERT</code> or <code class="ph codeph">CREATE TABLE AS SELECT</code> statements use the
+ <code class="ph codeph">PARQUET_FILE_SIZE</code> query option setting to define the size of Parquet
+ data files. (Using a large block size is more important for Parquet tables than for
+ tables that use other file formats.)
</p>
@@ -826,4 +829,4 @@
</body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_admin.html b/docs/build/html/topics/impala_admin.html
index b4525ed..3c5a786 100644
--- a/docs/build/html/topics/impala_admin.html
+++ b/docs/build/html/topics/impala_admin.html
@@ -13,8 +13,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_disk_space.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="admin" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
diff --git a/docs/build/html/topics/impala_admission.html b/docs/build/html/topics/impala_admission.html
index 30c3da6..da7e057 100644
--- a/docs/build/html/topics/impala_admission.html
+++ b/docs/build/html/topics/impala_admission.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_resource_management.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="admission_control" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -88,7 +88,7 @@
for each dynamic resource pool. </p>
<dl class="dl">
-
+
<dt class="dt dlterm"> Max Running Queries </dt>
<dd class="dd"><p class="p">Maximum number of concurrently running queries in this pool.
@@ -115,25 +115,53 @@
safeguard. If queries exceed either the total estimated memory or
the maximum number of concurrent queries, they are added to the
queue. </p>
+<p class="p">If <span class="ph uicontrol">Max Running Queries
+ Multiple</span> is set, the <span class="ph uicontrol">Max Running
+ Queries</span> setting is ignored.</p>
</dd>
-
- </dl>
-
- <dl class="dl">
-
+
+
+ <dt class="dt dlterm">Max Running Queries Multiple</dt>
+
+ <dd class="dd">This floating point number is multiplied by the current total
+ number of executors at runtime to give the maximum number of
+ concurrently running queries allowed in the pool. The effect of this
+ setting scales with the number of executors in the resource
+ pool.<p class="p">This calculation is rounded up to the nearest integer, so
+ the result will always be at least one. </p>
+<p class="p">If set to zero or a
+ negative number, the setting is ignored.</p>
+</dd>
+
+
+
<dt class="dt dlterm"> Max Queued Queries </dt>
<dd class="dd"> Maximum number of queries that can be queued in this pool. The
default value is 200 for Impala 2.1 or higher and 50 for previous
- versions of Impala. (optional)</dd>
-
-
- </dl>
-
- <dl class="dl">
-
+ versions of Impala. (optional)<p class="p">If <span class="ph uicontrol">Max Queued Queries
+ Multiple</span> is set, the <span class="ph uicontrol">Max Queued
+ Queries</span> setting is ignored.</p>
+</dd>
+
+
+
+ <dt class="dt dlterm">Max Queued Queries Multiple</dt>
+
+ <dd class="dd">This floating point number is multiplied by the current total
+ number of executors at runtime to give the maximum number of queries
+ that can be queued in the pool. The effect of this setting scales
+ with the number of executors in the resource pool.<p class="p">This
+ calculation is rounded up to the nearest integer, so the result
+ will always be at least one. </p>
+<p class="p">If set to zero or a negative
+ number, the setting is ignored.</p>
+</dd>
+
+
+
<dt class="dt dlterm"> Queue Timeout </dt>
<dd class="dd"> The amount of time, in milliseconds, that a query waits in the
@@ -171,7 +199,7 @@ You might also need to increase the value to use Impala with
</dd>
-
+
</dl>
<p class="p"> You can combine these settings with the memory-based approach
@@ -198,7 +226,7 @@ You might also need to increase the value to use Impala with
control.</p>
<dl class="dl">
-
+
<dt class="dt dlterm">Max Memory</dt>
<dd class="dd">
@@ -261,10 +289,24 @@ You might also need to increase the value to use Impala with
</div>
+ <p class="p">If <span class="ph uicontrol">Max Memory Multiple</span> is set, the
+ <span class="ph uicontrol">Max Memory</span> setting is ignored.</p>
+
</dd>
+
+
+ <dt class="dt dlterm">Max Memory Multiple</dt>
+ <dd class="dd"> This number of bytes is multiplied by the current total number of
+ executors at runtime to give the maximum memory available across the
+ cluster for the pool. The effect of this setting scales with the
+ number of executors in the resource pool.<p class="p">If set to zero or a
+ negative number, the setting is ignored.</p>
+</dd>
+
+
<dt class="dt dlterm">Minimum Query Memory Limit and Maximum Query Memory Limit</dt>
<dd class="dd">
@@ -315,8 +357,8 @@ If a user tries to submit a query with the
</dd>
-
-
+
+
<dt class="dt dlterm">Default Query Memory Limit</dt>
<dd class="dd">The default memory limit applied to queries executing in this pool
@@ -329,11 +371,11 @@ If a user tries to submit a query with the
Minimum Query Memory Limit. Do not set this if either Maximum Query
Memory Limit or Minimum Query Memory Limit is set.</dd>
-
+
</dl>
<dl class="dl">
-
+
<dt class="dt dlterm"> Clamp MEM_LIMIT Query Option</dt>
<dd class="dd">If this field is not selected, the <code class="ph codeph">MEM_LIMIT</code>
@@ -344,33 +386,44 @@ If a user tries to submit a query with the
Query Memory Limit</strong> and <strong class="ph b">Maximum Query Memory Limit</strong> are
not set.</dd>
-
+
</dl>
- <div class="p"> For example, consider the
- following scenario: <ul class="ul">
- <li class="li">The cluster is running <code class="ph codeph">impalad</code> daemons on five
- hosts.</li>
+ <div class="p">
+ For example, consider the following scenario:
+ <ul class="ul">
+ <li class="li">
+ The cluster is running <code class="ph codeph">impalad</code> daemons on five hosts.
+ </li>
+
+
+ <li class="li">
+ A dynamic resource pool has Max Memory set to 100 GB.
+ </li>
- <li class="li">A dynamic resource pool has Max Memory set to 100 GB.</li>
- <li class="li">The Maximum Query Memory Limit for the pool is 10 GB and Minimum
- Query Memory Limit is 2 GB. Therefore, any query running in this
- pool could use up to 50 GB of memory (Maximum Query Memory Limit *
- number of Impala nodes).</li>
+ <li class="li">
+ The Maximum Query Memory Limit for the pool is 10 GB and Minimum Query Memory Limit
+ is 2 GB. Therefore, any query running in this pool could use up to 50 GB of memory
+ (Maximum Query Memory Limit * number of Impala nodes).
+ </li>
- <li class="li">Impala will execute varying numbers of queries concurrently
- because queries may be given memory limits anywhere between 2 GB and
- 10 GB, depending on the estimated memory requirements. For example,
- Impala may execute up to 10 small queries with 2 GB memory limits or
- two large queries with 10 GB memory limits because that is what will
- fit in the 100 GB cluster-wide limit when executing on five
- hosts.</li>
- <li class="li">The executing queries may use less memory than the per-host memory
- limit or the Max Memory cluster-wide limit if they do not need that
- much memory. In general this is not a problem so long as you are
- able to execute enough queries concurrently to meet your needs.</li>
+ <li class="li">
+ Impala will execute varying numbers of queries concurrently because queries may be
+ given memory limits anywhere between 2 GB and 10 GB, depending on the estimated
+ memory requirements. For example, Impala may execute up to 10 small queries with 2
+ GB memory limits or two large queries with 10 GB memory limits because that is what
+ will fit in the 100 GB cluster-wide limit when executing on five hosts.
+ </li>
+
+
+ <li class="li">
+ The executing queries may use less memory than the per-host memory limit or the Max
+ Memory cluster-wide limit if they do not need that much memory. In general this is
+ not a problem so long as you are able to execute enough queries concurrently to meet
+ your needs.
+ </li>
</ul>
@@ -619,7 +672,7 @@ If a user tries to submit a query with the
<div class="topic concept nested1" aria-labelledby="ariaid-title9" id="admission_guidelines">
<h2 class="title topictitle2" id="ariaid-title9">Guidelines for Using Admission Control</h2>
-
+
<div class="body conbody">
<p class="p"> The limits imposed by admission control are de-centrally managed
<span class="q">"soft"</span> limits. Each Impala coordinator node makes its own
@@ -681,4 +734,4 @@ If a user tries to submit a query with the
</div>
</body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_admission_config.html b/docs/build/html/topics/impala_admission_config.html
index 3819d8a..9644f97 100644
--- a/docs/build/html/topics/impala_admission_config.html
+++ b/docs/build/html/topics/impala_admission_config.html
@@ -10,7 +10,7 @@
<meta name="DC.Title" content="Configuring Admission Control" />
<meta name="DC.Relation" scheme="URI" content="../topics/impala_resource_management.html" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="admission_control" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -22,118 +22,165 @@
<h1 class="title topictitle1" id="ariaid-title1">Configuring Admission Control</h1>
+
+
<div class="body conbody">
- <p class="p">Impala includes features that balance and maximize resources in your
- <span class="keyword">Apache Hadoop</span> cluster. This topic describes how you
- can improve efficiency of your a <span class="keyword">Apache Hadoop</span> cluster
- using those features.</p>
- <p class="p"> The configuration options for admission control range from the simple (a
- single resource pool with a single set of options) to the complex
- (multiple resource pools with different options, each pool handling
- queries for a different set of users and groups). </p>
+ <p class="p">
+ Impala includes features that balance and maximize resources in your
+ <span class="keyword">Apache Hadoop</span> cluster. This topic describes how you can improve
+ efficiency of your a <span class="keyword">Apache Hadoop</span> cluster using those features.
+ </p>
+
+
+ <p class="p">
+ The configuration options for admission control range from the simple (a single resource
+ pool with a single set of options) to the complex (multiple resource pools with different
+ options, each pool handling queries for a different set of users and groups).
+ </p>
+
</div>
+
<div class="related-links">
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_resource_management.html">Resource Management</a></div>
</div>
</div><div class="topic concept nested1" aria-labelledby="ariaid-title2" id="concept_bz4_vxz_jgb">
+
<h2 class="title topictitle2" id="ariaid-title2">Configuring Admission Control in Command Line Interface</h2>
+
<div class="body conbody">
- <p class="p"> To configure admission control, use a combination of startup options
- for the Impala daemon and edit or create the configuration files
- <span class="ph filepath">fair-scheduler.xml</span> and
- <span class="ph filepath">llama-site.xml</span>. </p>
-
- <p class="p"> For a straightforward configuration using a single resource pool named
- <code class="ph codeph">default</code>, you can specify configuration options on the
- command line and skip the <span class="ph filepath">fair-scheduler.xml</span> and
- <span class="ph filepath">llama-site.xml</span> configuration files. </p>
-
- <div class="p"> For an advanced configuration with multiple resource pools using
- different settings:<ol class="ol">
- <li class="li">Set up the <span class="ph filepath">fair-scheduler.xml</span> and
- <span class="ph filepath">llama-site.xml</span> configuration files
- manually.</li>
-
- <li class="li">Provide the paths to each one using the <span class="keyword cmdname">impalad</span>
- command-line options,
- <code class="ph codeph">--fair_scheduler_allocation_path</code> and
- <code class="ph codeph">--llama_site_path</code> respectively. </li>
+
+ <p class="p">
+ To configure admission control, use a combination of startup options for the Impala
+ daemon and edit or create the configuration files
+ <span class="ph filepath">fair-scheduler.xml</span> and <span class="ph filepath">llama-site.xml</span>.
+ </p>
+
+
+ <p class="p">
+ For a straightforward configuration using a single resource pool named
+ <code class="ph codeph">default</code>, you can specify configuration options on the command line and
+ skip the <span class="ph filepath">fair-scheduler.xml</span> and <span class="ph filepath">llama-site.xml</span>
+ configuration files.
+ </p>
+
+
+ <div class="p">
+ For an advanced configuration with multiple resource pools using different settings:
+ <ol class="ol">
+ <li class="li">
+ Set up the <span class="ph filepath">fair-scheduler.xml</span> and
+ <span class="ph filepath">llama-site.xml</span> configuration files manually.
+ </li>
+
+
+ <li class="li">
+ Provide the paths to each one using the <span class="keyword cmdname">impalad</span> command-line
+ options, <code class="ph codeph">‑‑fair_scheduler_allocation_path</code> and
+ <code class="ph codeph">‑‑llama_site_path</code> respectively.
+ </li>
</ol>
-</div>
- <p class="p"> The Impala admission control feature uses the Fair Scheduler
- configuration settings to determine how to map users and groups to
- different resource pools. For example, you might set up different
- resource pools with separate memory limits, and maximum number of
- concurrent and queued queries, for different categories of users within
- your organization. For details about all the Fair Scheduler
- configuration settings, see the <a class="xref" href="http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/FairScheduler.html#Configuration" target="_blank">Apache wiki</a>. </p>
+ </div>
+
+
+ <p class="p">
+ The Impala admission control feature uses the Fair Scheduler configuration settings to
+ determine how to map users and groups to different resource pools. For example, you
+ might set up different resource pools with separate memory limits, and maximum number of
+ concurrent and queued queries, for different categories of users within your
+ organization. For details about all the Fair Scheduler configuration settings, see the
+ <a class="xref" href="http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/FairScheduler.html#Configuration" target="_blank">Apache
+ wiki</a>.
+ </p>
- <p class="p"> The Impala admission control feature uses a small subset of possible
- settings from the <span class="ph filepath">llama-site.xml</span> configuration
- file: </p>
- <pre class="pre codeblock"><code>llama.am.throttling.maximum.placed.reservations.<var class="keyword varname">queue_name</var>
+ <p class="p">
+ The Impala admission control feature uses a small subset of possible settings from the
+ <span class="ph filepath">llama-site.xml</span> configuration file:
+ </p>
+
+
+<pre class="pre codeblock"><code>llama.am.throttling.maximum.placed.reservations.<var class="keyword varname">queue_name</var>
llama.am.throttling.maximum.queued.reservations.<var class="keyword varname">queue_name</var>
<span class="ph">impala.admission-control.pool-default-query-options.<var class="keyword varname">queue_name</var>
impala.admission-control.pool-queue-timeout-ms.<var class="keyword varname">queue_name</var></span>
</code></pre>
- <p class="p"> The
- <code class="ph codeph">impala.admission-control.pool-queue-timeout-ms</code>
- setting specifies the timeout value for this pool in milliseconds. </p>
-
- <p class="p">The<code class="ph codeph">impala.admission-control.pool-default-query-options</code>
- settings designates the default query options for all queries that run
- in this pool. Its argument value is a comma-delimited string of
- 'key=value' pairs, <code class="ph codeph">'key1=val1,key2=val2, ...'</code>. For
- example, this is where you might set a default memory limit for all
- queries in the pool, using an argument such as
- <code class="ph codeph">MEM_LIMIT=5G</code>. </p>
-
- <p class="p"> The
- <code class="ph codeph">impala.admission-control.*</code> configuration settings are
- available in <span class="keyword">Impala 2.5</span> and higher. </p>
+
+ <p class="p">
+ The <code class="ph codeph">impala.admission-control.pool-queue-timeout-ms</code> setting specifies
+ the timeout value for this pool in milliseconds.
+ </p>
+
+
+ <p class="p">
+ The<code class="ph codeph">impala.admission-control.pool-default-query-options</code> settings
+ designates the default query options for all queries that run in this pool. Its argument
+ value is a comma-delimited string of 'key=value' pairs, <code class="ph codeph">'key1=val1,key2=val2,
+ ...'</code>. For example, this is where you might set a default memory limit for all
+ queries in the pool, using an argument such as <code class="ph codeph">MEM_LIMIT=5G</code>.
+ </p>
+
+
+ <p class="p">
+ The <code class="ph codeph">impala.admission-control.*</code> configuration settings are available in
+ <span class="keyword">Impala 2.5</span> and higher.
+ </p>
+
</div>
+
<div class="topic concept nested2" aria-labelledby="ariaid-title3" id="concept_cz4_vxz_jgb">
+
<h3 class="title topictitle3" id="ariaid-title3">Example of Admission Control Configuration</h3>
+
<div class="body conbody">
- <p class="p"> Here are sample <span class="ph filepath">fair-scheduler.xml</span> and
- <span class="ph filepath">llama-site.xml</span> files that define resource pools
- <code class="ph codeph">root.default</code>, <code class="ph codeph">root.development</code>,
- and <code class="ph codeph">root.production</code>. These files define resource
- pools for Impala admission control and are separate from the similar
- <code class="ph codeph">fair-scheduler.xml</code>that defines resource pools for
- YARN.</p>
+
+ <p class="p">
+ Here are sample <span class="ph filepath">fair-scheduler.xml</span> and
+ <span class="ph filepath">llama-site.xml</span> files that define resource pools
+ <code class="ph codeph">root.default</code>, <code class="ph codeph">root.development</code>, and
+ <code class="ph codeph">root.production</code>. These files define resource pools for Impala
+ admission control and are separate from the similar
+ <code class="ph codeph">fair-scheduler.xml</code>that defines resource pools for YARN.
+ </p>
+
<p class="p">
<strong class="ph b">fair-scheduler.xml:</strong>
</p>
- <p class="p"> Although Impala does not use the <code class="ph codeph">vcores</code> value, you
- must still specify it to satisfy YARN requirements for the file
- contents. </p>
- <p class="p"> Each <code class="ph codeph"><aclSubmitApps></code> tag (other than the one
- for <code class="ph codeph">root</code>) contains a comma-separated list of users,
- then a space, then a comma-separated list of groups; these are the
- users and groups allowed to submit Impala statements to the
- corresponding resource pool. </p>
+ <p class="p">
+ Although Impala does not use the <code class="ph codeph">vcores</code> value, you must still specify
+ it to satisfy YARN requirements for the file contents.
+ </p>
- <p class="p"> If you leave the <code class="ph codeph"><aclSubmitApps></code> element
- empty for a pool, nobody can submit directly to that pool; child pools
- can specify their own <code class="ph codeph"><aclSubmitApps></code> values to
- authorize users and groups to submit to those pools. </p>
- <pre class="pre codeblock"><code><allocations>
+ <p class="p">
+ Each <code class="ph codeph"><aclSubmitApps></code> tag (other than the one for
+ <code class="ph codeph">root</code>) contains a comma-separated list of users, then a space, then a
+ comma-separated list of groups; these are the users and groups allowed to submit
+ Impala statements to the corresponding resource pool.
+ </p>
+
+
+ <p class="p">
+ If you leave the <code class="ph codeph"><aclSubmitApps></code> element empty for a pool,
+ nobody can submit directly to that pool; child pools can specify their own
+ <code class="ph codeph"><aclSubmitApps></code> values to authorize users and groups to submit
+ to those pools.
+ </p>
+
+
+<pre class="pre codeblock"><code><allocations>
<queue name="root">
<aclSubmitApps> </aclSubmitApps>
@@ -157,11 +204,13 @@ impala.admission-control.pool-queue-timeout-ms.<var class="keyword varname">queu
</allocations>
</code></pre>
+
<p class="p">
<strong class="ph b">llama-site.xml:</strong>
</p>
- <pre class="pre codeblock"><code>
+
+<pre class="pre codeblock"><code>
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<property>
@@ -193,227 +242,286 @@ impala.admission-control.pool-queue-timeout-ms.<var class="keyword varname">queu
<value>true</value>
</property>
</code></pre>
+
</div>
+
</div>
+
</div>
+
<div class="topic concept nested1" aria-labelledby="ariaid-title4" id="concept_zy4_vxz_jgb">
+
<h2 class="title topictitle2" id="ariaid-title4">Configuring Cluster-wide Admission Control</h2>
+
+
<div class="body conbody">
- <div class="note important"><span class="importanttitle">Important:</span> These settings only apply if you enable admission
- control but leave dynamic resource pools disabled. In <span class="keyword">Impala 2.5</span> and higher, we recommend that you set up
- dynamic resource pools and customize the settings for each pool as
- described in <a class="xref" href="#concept_bz4_vxz_jgb">Configuring Admission Control in Command Line Interface</a>.</div>
- <p class="p"> The following Impala configuration options let you adjust the settings
- of the admission control feature. When supplying the options on the
- <span class="keyword cmdname">impalad</span> command line, prepend the option name with
- <code class="ph codeph">--</code>. </p>
+ <div class="note important"><span class="importanttitle">Important:</span>
+ These settings only apply if you enable admission control but leave dynamic resource
+ pools disabled. In <span class="keyword">Impala 2.5</span> and higher, we recommend
+ that you set up dynamic resource pools and customize the settings for each pool as
+ described in <a class="xref" href="#concept_bz4_vxz_jgb">Configuring Admission Control in Command Line Interface</a>.
+ </div>
+
+
+ <p class="p">
+ The following Impala configuration options let you adjust the settings of the admission
+ control feature. When supplying the options on the <span class="keyword cmdname">impalad</span> command
+ line, prepend the option name with <code class="ph codeph">--</code>.
+ </p>
+
<dl class="dl">
+
<dt class="dt dlterm">
<code class="ph codeph">queue_wait_timeout_ms</code>
</dt>
+
<dd class="dd">
- <strong class="ph b">Purpose:</strong> Maximum amount of time (in milliseconds) that a
- request waits to be admitted before timing out. <p class="p">
- <strong class="ph b">Type:</strong>
- <code class="ph codeph">int64</code>
+ <strong class="ph b">Purpose:</strong> Maximum amount of time (in milliseconds) that a request waits to be
+ admitted before timing out.
+ <p class="p">
+ <strong class="ph b">Type:</strong> <code class="ph codeph">int64</code>
</p>
+
<p class="p">
- <strong class="ph b">Default:</strong>
- <code class="ph codeph">60000</code>
+ <strong class="ph b">Default:</strong> <code class="ph codeph">60000</code>
</p>
</dd>
+
+
+
<dt class="dt dlterm">
<code class="ph codeph">default_pool_max_requests</code>
</dt>
+
<dd class="dd">
- <strong class="ph b">Purpose:</strong> Maximum number of concurrent outstanding requests
- allowed to run before incoming requests are queued. Because this
- limit applies cluster-wide, but each Impala node makes independent
- decisions to run queries immediately or queue them, it is a soft
- limit; the overall number of concurrent queries might be slightly
- higher during times of heavy load. A negative value indicates no
- limit. Ignored if <code class="ph codeph">fair_scheduler_config_path</code> and
- <code class="ph codeph">llama_site_path</code> are set. <p class="p">
- <strong class="ph b">Type:</strong>
- <code class="ph codeph">int64</code>
+ <strong class="ph b">Purpose:</strong> Maximum number of concurrent outstanding requests allowed to run
+ before incoming requests are queued. Because this limit applies cluster-wide, but
+ each Impala node makes independent decisions to run queries immediately or queue
+ them, it is a soft limit; the overall number of concurrent queries might be slightly
+ higher during times of heavy load. A negative value indicates no limit. Ignored if
+ <code class="ph codeph">fair_scheduler_config_path</code> and <code class="ph codeph">llama_site_path</code> are
+ set.
+ <p class="p">
+ <strong class="ph b">Type:</strong> <code class="ph codeph">int64</code>
</p>
+
<p class="p">
- <strong class="ph b">Default:</strong>
- <span class="ph">-1, meaning unlimited (prior to <span class="keyword">Impala 2.5</span> the default was 200)</span>
+ <strong class="ph b">Default:</strong> <span class="ph">-1, meaning unlimited (prior to
+ <span class="keyword">Impala 2.5</span> the default was 200)</span>
</p>
</dd>
+
+
+
<dt class="dt dlterm">
<code class="ph codeph">default_pool_max_queued</code>
</dt>
+
<dd class="dd">
- <strong class="ph b">Purpose:</strong> Maximum number of requests allowed to be queued
- before rejecting requests. Because this limit applies cluster-wide,
- but each Impala node makes independent decisions to run queries
- immediately or queue them, it is a soft limit; the overall number of
- queued queries might be slightly higher during times of heavy load.
- A negative value or 0 indicates requests are always rejected once
- the maximum concurrent requests are executing. Ignored if
- <code class="ph codeph">fair_scheduler_config_path</code> and
- <code class="ph codeph">llama_site_path</code> are set. <p class="p">
- <strong class="ph b">Type:</strong>
- <code class="ph codeph">int64</code>
+ <strong class="ph b">Purpose:</strong> Maximum number of requests allowed to be queued before rejecting
+ requests. Because this limit applies cluster-wide, but each Impala node makes
+ independent decisions to run queries immediately or queue them, it is a soft limit;
+ the overall number of queued queries might be slightly higher during times of heavy
+ load. A negative value or 0 indicates requests are always rejected once the maximum
+ concurrent requests are executing. Ignored if
+ <code class="ph codeph">fair_scheduler_config_path</code> and <code class="ph codeph">llama_site_path</code> are
+ set.
+ <p class="p">
+ <strong class="ph b">Type:</strong> <code class="ph codeph">int64</code>
</p>
+
<p class="p">
- <strong class="ph b">Default:</strong>
- <span class="ph">unlimited</span>
+ <strong class="ph b">Default:</strong> <span class="ph">unlimited</span>
</p>
</dd>
+
+
+
<dt class="dt dlterm">
<code class="ph codeph">default_pool_mem_limit</code>
</dt>
+
<dd class="dd">
- <strong class="ph b">Purpose:</strong> Maximum amount of memory (across the entire cluster)
- that all outstanding requests in this pool can use before new
- requests to this pool are queued. Specified in bytes, megabytes, or
- gigabytes by a number followed by the suffix <code class="ph codeph">b</code>
- (optional), <code class="ph codeph">m</code>, or <code class="ph codeph">g</code>, either
- uppercase or lowercase. You can specify floating-point values for
- megabytes and gigabytes, to represent fractional numbers such as
- <code class="ph codeph">1.5</code>. You can also specify it as a percentage of
- the physical memory by specifying the suffix <code class="ph codeph">%</code>. 0
- or no setting indicates no limit. Defaults to bytes if no unit is
- given. Because this limit applies cluster-wide, but each Impala node
- makes independent decisions to run queries immediately or queue
- them, it is a soft limit; the overall memory used by concurrent
- queries might be slightly higher during times of heavy load. Ignored
- if <code class="ph codeph">fair_scheduler_config_path</code> and
- <code class="ph codeph">llama_site_path</code> are set. <div class="note note"><span class="notetitle">Note:</span>
- Impala relies on the statistics produced by the <code class="ph codeph">COMPUTE STATS</code> statement to estimate memory
- usage for each query. See <a class="xref" href="../shared/../topics/impala_compute_stats.html#compute_stats">COMPUTE STATS Statement</a> for guidelines
- about how and when to use this statement.
+ <strong class="ph b">Purpose:</strong> Maximum amount of memory (across the entire cluster) that all
+ outstanding requests in this pool can use before new requests to this pool are
+ queued. Specified in bytes, megabytes, or gigabytes by a number followed by the
+ suffix <code class="ph codeph">b</code> (optional), <code class="ph codeph">m</code>, or <code class="ph codeph">g</code>,
+ either uppercase or lowercase. You can specify floating-point values for megabytes
+ and gigabytes, to represent fractional numbers such as <code class="ph codeph">1.5</code>. You can
+ also specify it as a percentage of the physical memory by specifying the suffix
+ <code class="ph codeph">%</code>. 0 or no setting indicates no limit. Defaults to bytes if no unit
+ is given. Because this limit applies cluster-wide, but each Impala node makes
+ independent decisions to run queries immediately or queue them, it is a soft limit;
+ the overall memory used by concurrent queries might be slightly higher during times
+ of heavy load. Ignored if <code class="ph codeph">fair_scheduler_config_path</code> and
+ <code class="ph codeph">llama_site_path</code> are set.
+ <div class="note note"><span class="notetitle">Note:</span>
+ Impala relies on the statistics produced by the <code class="ph codeph">COMPUTE STATS</code> statement
+ to estimate memory usage for each query. See
+ <a class="xref" href="../shared/../topics/impala_compute_stats.html#compute_stats">COMPUTE STATS Statement</a> for guidelines about how
+ and when to use this statement.
</div>
+
<p class="p">
<strong class="ph b">Type:</strong> string
</p>
+
<p class="p">
- <strong class="ph b">Default:</strong>
- <code class="ph codeph">""</code> (empty string, meaning unlimited) </p>
+ <strong class="ph b">Default:</strong> <code class="ph codeph">""</code> (empty string, meaning unlimited)
+ </p>
</dd>
+
+
+
<dt class="dt dlterm">
<code class="ph codeph">disable_pool_max_requests</code>
</dt>
+
<dd class="dd">
- <strong class="ph b">Purpose:</strong> Disables all per-pool limits on the maximum number
- of running requests. <p class="p">
- <strong class="ph b">Type:</strong> Boolean </p>
+ <strong class="ph b">Purpose:</strong> Disables all per-pool limits on the maximum number of running
+ requests.
+ <p class="p">
+ <strong class="ph b">Type:</strong> Boolean
+ </p>
+
<p class="p">
- <strong class="ph b">Default:</strong>
- <code class="ph codeph">false</code>
+ <strong class="ph b">Default:</strong> <code class="ph codeph">false</code>
</p>
</dd>
+
+
+
<dt class="dt dlterm">
<code class="ph codeph">disable_pool_mem_limits</code>
</dt>
+
<dd class="dd">
- <strong class="ph b">Purpose:</strong> Disables all per-pool mem limits. <p class="p">
- <strong class="ph b">Type:</strong> Boolean </p>
+ <strong class="ph b">Purpose:</strong> Disables all per-pool mem limits.
+ <p class="p">
+ <strong class="ph b">Type:</strong> Boolean
+ </p>
+
<p class="p">
- <strong class="ph b">Default:</strong>
- <code class="ph codeph">false</code>
+ <strong class="ph b">Default:</strong> <code class="ph codeph">false</code>
</p>
</dd>
+
+
+
<dt class="dt dlterm">
<code class="ph codeph">fair_scheduler_allocation_path</code>
</dt>
+
<dd class="dd">
<strong class="ph b">Purpose:</strong> Path to the fair scheduler allocation file
- (<code class="ph codeph">fair-scheduler.xml</code>). <p class="p">
+ (<code class="ph codeph">fair-scheduler.xml</code>).
+ <p class="p">
<strong class="ph b">Type:</strong> string
</p>
+
<p class="p">
- <strong class="ph b">Default:</strong>
- <code class="ph codeph">""</code> (empty string) </p>
+ <strong class="ph b">Default:</strong> <code class="ph codeph">""</code> (empty string)
+ </p>
+
<p class="p">
- <strong class="ph b">Usage notes:</strong> Admission control only uses a small subset of
- the settings that can go in this file, as described below. For
- details about all the Fair Scheduler configuration settings, see
- the <a class="xref" href="http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/FairScheduler.html#Configuration" target="_blank">Apache wiki</a>. </p>
+ <strong class="ph b">Usage notes:</strong> Admission control only uses a small subset of the settings
+ that can go in this file, as described below. For details about all the Fair
+ Scheduler configuration settings, see the
+ <a class="xref" href="http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/FairScheduler.html#Configuration" target="_blank">Apache
+ wiki</a>.
+ </p>
</dd>
+
+
+
<dt class="dt dlterm">
<code class="ph codeph">llama_site_path</code>
</dt>
+
<dd class="dd">
- <strong class="ph b">Purpose:</strong> Path to the configuration file used by admission
- control (<code class="ph codeph">llama-site.xml</code>). If set,
- <code class="ph codeph">fair_scheduler_allocation_path</code> must also be set.
- <p class="p">
+ <strong class="ph b">Purpose:</strong> Path to the configuration file used by admission control
+ (<code class="ph codeph">llama-site.xml</code>). If set,
+ <code class="ph codeph">fair_scheduler_allocation_path</code> must also be set.
+ <p class="p">
<strong class="ph b">Type:</strong> string
</p>
+
<p class="p">
- <strong class="ph b">Default:</strong>
- <code class="ph codeph">""</code> (empty string) </p>
+ <strong class="ph b">Default:</strong> <code class="ph codeph">""</code> (empty string)
+ </p>
+
<p class="p">
- <strong class="ph b">Usage notes:</strong> Admission control only uses a few of the
- settings that can go in this file, as described below. </p>
+ <strong class="ph b">Usage notes:</strong> Admission control only uses a few of the settings that can go
+ in this file, as described below.
+ </p>
</dd>
+
</dl>
+
</div>
+
</div>
+
</body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_aggregate_functions.html b/docs/build/html/topics/impala_aggregate_functions.html
index 8f89013..8a6352a 100644
--- a/docs/build/html/topics/impala_aggregate_functions.html
+++ b/docs/build/html/topics/impala_aggregate_functions.html
@@ -21,8 +21,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_variance.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="aggregate_functions" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -39,8 +39,9 @@
<div class="body conbody">
<p class="p">
- Aggregate functions are a special category with different rules. These functions calculate a return value
- across all the items in a result set, so they require a <code class="ph codeph">FROM</code> clause in the query:
+ Aggregate functions are a special category with different rules. These functions
+ calculate a return value across all the items in a result set, so they require a
+ <code class="ph codeph">FROM</code> clause in the query:
</p>
@@ -49,11 +50,12 @@ select max(height), avg(height) from census_data where age > 20;
</code></pre>
<p class="p">
- Aggregate functions also ignore <code class="ph codeph">NULL</code> values rather than returning a <code class="ph codeph">NULL</code>
- result. For example, if some rows have <code class="ph codeph">NULL</code> for a particular column, those rows are
- ignored when computing the <code class="ph codeph">AVG()</code> for that column. Likewise, specifying
- <code class="ph codeph">COUNT(<var class="keyword varname">col_name</var>)</code> in a query counts only those rows where
- <var class="keyword varname">col_name</var> contains a non-<code class="ph codeph">NULL</code> value.
+ Aggregate functions also ignore <code class="ph codeph">NULL</code> values rather than returning a
+ <code class="ph codeph">NULL</code> result. For example, if some rows have <code class="ph codeph">NULL</code> for a
+ particular column, those rows are ignored when computing the <code class="ph codeph">AVG()</code> for
+ that column. Likewise, specifying <code class="ph codeph">COUNT(<var class="keyword varname">col_name</var>)</code> in
+ a query counts only those rows where <var class="keyword varname">col_name</var> contains a
+ non-<code class="ph codeph">NULL</code> value.
</p>
diff --git a/docs/build/html/topics/impala_aliases.html b/docs/build/html/topics/impala_aliases.html
index f230c53..93d511a 100644
--- a/docs/build/html/topics/impala_aliases.html
+++ b/docs/build/html/topics/impala_aliases.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_schema_objects.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="aliases" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -71,10 +71,11 @@ SELECT t1.column_n c1, t2.column_n c2 FROM table1 t1, table2 t2
</p>
<div class="p">
- You can specify column aliases with or without the <code class="ph codeph">AS</code> keyword, and with no quotation
- marks, single quotation marks, or double quotation marks. Some kind of quotation marks are required if the
- column alias contains any spaces or other problematic characters. The alias text is displayed in the
- <span class="keyword cmdname">impala-shell</span> output as all-lowercase. For example:
+ You can specify column aliases with or without the <code class="ph codeph">AS</code> keyword, and with
+ no quotation marks, single quotation marks, or double quotation marks. Some kind of
+ quotation marks are required if the column alias contains any spaces or other
+ problematic characters. The alias text is displayed in the
+ <span class="keyword cmdname">impala-shell</span> output as all-lowercase. For example:
<pre class="pre codeblock"><code>[localhost:21000] > select c1 First_Column from t;
[localhost:21000] > select c1 as First_Column from t;
+--------------+
@@ -95,10 +96,10 @@ SELECT t1.column_n c1, t2.column_n c2 FROM table1 t1, table2 t2
| first column |
+--------------+
...</code></pre>
- From Impala 3.0, the alias substitution logic in the <code class="ph codeph">GROUP BY</code>, <code class="ph codeph">HAVING</code>,
- and <code class="ph codeph">ORDER BY</code> clauses has become more consistent with standard SQL behavior, as follows.
- Aliases are now only legal at the top level, and not in subexpressions. The following statements are
- allowed:
+ From Impala 3.0, the alias substitution logic in the <code class="ph codeph">GROUP BY</code>,
+ <code class="ph codeph">HAVING</code>, and <code class="ph codeph">ORDER BY</code> clauses has become more
+ consistent with standard SQL behavior, as follows. Aliases are now only legal at the top
+ level, and not in subexpressions. The following statements are allowed:
<pre class="pre codeblock"><code>
SELECT int_col / 2 AS x
FROM t
@@ -113,7 +114,7 @@ SELECT t1.column_n c1, t2.column_n c2 FROM table1 t1, table2 t2
GROUP BY nb
HAVING nb;
</code></pre>
- And the following statements are NOT allowed:
+ And the following statements are NOT allowed:
<pre class="pre codeblock"><code>
SELECT int_col / 2 AS x
FROM t
@@ -128,7 +129,7 @@ SELECT t1.column_n c1, t2.column_n c2 FROM table1 t1, table2 t2
GROUP BY x
HAVING x > 3;
</code></pre>
- </div>
+ </div>
<p class="p">
@@ -139,10 +140,10 @@ SELECT t1.column_n c1, t2.column_n c2 FROM table1 t1, table2 t2
<p class="p">
- <span class="ph"> Aliases follow the same rules as identifiers when it comes to case
- insensitivity. Aliases can be longer than identifiers (up to the maximum length of a Java string) and can
- include additional characters such as spaces and dashes when they are quoted using backtick characters.
- </span>
+ <span class="ph"> Aliases follow the same rules as identifiers when it
+ comes to case insensitivity. Aliases can be longer than identifiers (up to the maximum
+ length of a Java string) and can include additional characters such as spaces and dashes
+ when they are quoted using backtick characters. </span>
</p>
@@ -170,8 +171,8 @@ SELECT t1.column_n c1, t2.column_n c2 FROM table1 t1, table2 t2
<p class="p">
- Another way to define different names for the same tables or columns is to create views. See
- <a class="xref" href="../shared/../topics/impala_views.html#views">Overview of Impala Views</a> for details.
+ Another way to define different names for the same tables or columns is to create views.
+ See <a class="xref" href="../shared/../topics/impala_views.html#views">Overview of Impala Views</a> for details.
</p>
</div>
diff --git a/docs/build/html/topics/impala_allow_erasure_coded_files.html b/docs/build/html/topics/impala_allow_erasure_coded_files.html
index 6c5457a..c6a0b7e 100644
--- a/docs/build/html/topics/impala_allow_erasure_coded_files.html
+++ b/docs/build/html/topics/impala_allow_erasure_coded_files.html
@@ -10,7 +10,7 @@
<meta name="DC.Title" content="ALLOW_ERASURE_CODED_FILES Query Option" />
<meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="allow_erasure_coded_files" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -22,9 +22,9 @@
<h1 class="title topictitle1" id="ariaid-title1">ALLOW_ERASURE_CODED_FILES Query Option</h1>
+
-
-
+
<div class="body conbody">
@@ -65,4 +65,4 @@
<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_query_options.html">Query Options for the SET Statement</a></div>
</div>
</div></body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_allow_unsupported_formats.html b/docs/build/html/topics/impala_allow_unsupported_formats.html
index 0cb81c7..2757318 100644
--- a/docs/build/html/topics/impala_allow_unsupported_formats.html
+++ b/docs/build/html/topics/impala_allow_unsupported_formats.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="allow_unsupported_formats" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -32,12 +32,13 @@
<p class="p">
- <strong class="ph b">Type:</strong> Boolean; recognized values are 1 and 0, or <code class="ph codeph">true</code> and <code class="ph codeph">false</code>;
- any other value interpreted as <code class="ph codeph">false</code>
+ <strong class="ph b">Type:</strong> Boolean; recognized values are 1 and 0, or <code class="ph codeph">true</code> and
+ <code class="ph codeph">false</code>; any other value interpreted as <code class="ph codeph">false</code>
</p>
<p class="p">
- <strong class="ph b">Default:</strong> <code class="ph codeph">false</code> (shown as 0 in output of <code class="ph codeph">SET</code> statement)
+ <strong class="ph b">Default:</strong> <code class="ph codeph">false</code> (shown as 0 in output of <code class="ph codeph">SET</code>
+ statement)
</p>
</div>
diff --git a/docs/build/html/topics/impala_alter_database.html b/docs/build/html/topics/impala_alter_database.html
index 5f66e2d..ad4690f 100644
--- a/docs/build/html/topics/impala_alter_database.html
+++ b/docs/build/html/topics/impala_alter_database.html
@@ -10,7 +10,7 @@
<meta name="DC.Title" content="ALTER DATABASE Statement" />
<meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="alter_database" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -22,9 +22,9 @@
<h1 class="title topictitle1" id="ariaid-title1">ALTER DATABASE Statement</h1>
+
-
-
+
<div class="body conbody">
@@ -83,4 +83,4 @@ ALTER DATABASE <var class="keyword varname">database_name</var> SET OWNER ROLE r
<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_langref_sql.html">Impala SQL Statements</a></div>
</div>
</div></body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_alter_table.html b/docs/build/html/topics/impala_alter_table.html
index faaf43f..bed66a2 100644
--- a/docs/build/html/topics/impala_alter_table.html
+++ b/docs/build/html/topics/impala_alter_table.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="alter_table" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -226,7 +226,7 @@ alter table fast_growing_data partition (year = 2016, month in (10,11,12)) set f
</code></pre>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
<p class="p">
The extended syntax involving comparison operators and multiple partitions applies to
the <code class="ph codeph">SET FILEFORMAT</code>, <code class="ph codeph">SET TBLPROPERTIES</code>, <code class="ph codeph">SET
@@ -276,11 +276,13 @@ alter table fast_growing_data partition (year = 2016, month in (10,11,12)) set f
<p class="p">
In <span class="keyword">Impala 2.6</span> and higher, Impala DDL statements such as
- <code class="ph codeph">CREATE DATABASE</code>, <code class="ph codeph">CREATE TABLE</code>, <code class="ph codeph">DROP DATABASE CASCADE</code>,
- <code class="ph codeph">DROP TABLE</code>, and <code class="ph codeph">ALTER TABLE [ADD|DROP] PARTITION</code> can create or remove folders
- as needed in the Amazon S3 system. Prior to <span class="keyword">Impala 2.6</span>, you had to create folders yourself and point
- Impala database, tables, or partitions at them, and manually remove folders when no longer needed.
- See <a class="xref" href="../shared/../topics/impala_s3.html#s3">Using Impala with the Amazon S3 Filesystem</a> for details about reading and writing S3 data with Impala.
+ <code class="ph codeph">CREATE DATABASE</code>, <code class="ph codeph">CREATE TABLE</code>, <code class="ph codeph">DROP DATABASE
+ CASCADE</code>, <code class="ph codeph">DROP TABLE</code>, and <code class="ph codeph">ALTER TABLE [ADD|DROP]
+ PARTITION</code> can create or remove folders as needed in the Amazon S3 system. Prior
+ to <span class="keyword">Impala 2.6</span>, you had to create folders yourself and point
+ Impala database, tables, or partitions at them, and manually remove folders when no
+ longer needed. See <a class="xref" href="../shared/../topics/impala_s3.html#s3">Using Impala with the Amazon S3 Filesystem</a> for details about reading
+ and writing S3 data with Impala.
</p>
@@ -299,21 +301,23 @@ alter table fast_growing_data partition (year = 2016, month in (10,11,12)) set f
<p class="p">
- In <span class="keyword">Impala 2.2</span> and higher, the optional <code class="ph codeph">WITH REPLICATION</code> clause
- for <code class="ph codeph">CREATE TABLE</code> and <code class="ph codeph">ALTER TABLE</code> lets you specify
- a <dfn class="term">replication factor</dfn>, the number of hosts on which to cache the same data blocks.
- When Impala processes a cached data block, where the cache replication factor is greater than 1, Impala randomly
- selects a host that has a cached copy of that data block. This optimization avoids excessive CPU
- usage on a single host when the same cached data block is processed multiple times.
- Where practical, specify a value greater than or equal to the HDFS block replication factor.
+ In <span class="keyword">Impala 2.2</span> and higher, the optional <code class="ph codeph">WITH
+ REPLICATION</code> clause for <code class="ph codeph">CREATE TABLE</code> and <code class="ph codeph">ALTER
+ TABLE</code> lets you specify a <dfn class="term">replication factor</dfn>, the number of hosts
+ on which to cache the same data blocks. When Impala processes a cached data block, where
+ the cache replication factor is greater than 1, Impala randomly selects a host that has
+ a cached copy of that data block. This optimization avoids excessive CPU usage on a
+ single host when the same cached data block is processed multiple times. Where
+ practical, specify a value greater than or equal to the HDFS block replication factor.
</p>
<p class="p">
- If you connect to different Impala nodes within an <span class="keyword cmdname">impala-shell</span> session for
- load-balancing purposes, you can enable the <code class="ph codeph">SYNC_DDL</code> query option to make each DDL
- statement wait before returning, until the new or changed metadata has been received by all the Impala
- nodes. See <a class="xref" href="../shared/../topics/impala_sync_ddl.html#sync_ddl">SYNC_DDL Query Option</a> for details.
+ If you connect to different Impala nodes within an <span class="keyword cmdname">impala-shell</span>
+ session for load-balancing purposes, you can enable the <code class="ph codeph">SYNC_DDL</code> query
+ option to make each DDL statement wait before returning, until the new or changed
+ metadata has been received by all the Impala nodes. See
+ <a class="xref" href="../shared/../topics/impala_sync_ddl.html#sync_ddl">SYNC_DDL Query Option</a> for details.
</p>
@@ -409,11 +413,12 @@ alter table p1 add partition (month=2, day=2);
alter table p1 partition (month=1, day=1) set location '/usr/external_data/new_years_day';
</code></pre>
- <div class="note note"><span class="notetitle">Note:</span>
- If you are creating a partition for the first time and specifying its location, for maximum efficiency, use
- a single <code class="ph codeph">ALTER TABLE</code> statement including both the <code class="ph codeph">ADD PARTITION</code> and
- <code class="ph codeph">LOCATION</code> clauses, rather than separate statements with <code class="ph codeph">ADD PARTITION</code> and
- <code class="ph codeph">SET LOCATION</code> clauses.
+ <div class="note note"><span class="notetitle">Note:</span>
+ If you are creating a partition for the first time and specifying its location, for
+ maximum efficiency, use a single <code class="ph codeph">ALTER TABLE</code> statement including both
+ the <code class="ph codeph">ADD PARTITION</code> and <code class="ph codeph">LOCATION</code> clauses, rather than
+ separate statements with <code class="ph codeph">ADD PARTITION</code> and <code class="ph codeph">SET
+ LOCATION</code> clauses.
</div>
@@ -612,9 +617,9 @@ alter table partitioned_data set tblproperties ('numRows'='1030000', 'STATS_GENE
<div class="p">
You specify a case-insensitive symbolic name for the kind of statistics:
- <code class="ph codeph">numDVs</code>, <code class="ph codeph">numNulls</code>, <code class="ph codeph">avgSize</code>, <code class="ph codeph">maxSize</code>.
- The key names and values are both quoted. This operation applies to an entire table,
- not a specific partition. For example:
+ <code class="ph codeph">numDVs</code>, <code class="ph codeph">numNulls</code>, <code class="ph codeph">avgSize</code>,
+ <code class="ph codeph">maxSize</code>. The key names and values are both quoted. This operation
+ applies to an entire table, not a specific partition. For example:
<pre class="pre codeblock"><code>
create table t1 (x int, s string);
insert into t1 values (1, 'one'), (2, 'two'), (2, 'deux');
@@ -1111,7 +1116,7 @@ alter table part_t add partition (month=3) set fileformat=parquet;
<pre class="pre codeblock"><code>alter table time_data add partition (month=concat('Decem','ber'));
alter table sales_data add partition (zipcode = cast(9021 * 10 as string));</code></pre>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
<p class="p">
An alternative way to reorganize a table and its associated data files is to use
<code class="ph codeph">CREATE TABLE</code> to create a variation of the original table, then use
@@ -1129,17 +1134,18 @@ alter table sales_data add partition (zipcode = cast(9021 * 10 as string));</cod
</p>
- <div class="p">
- You can switch a table from internal to external, or from external to internal, by using the <code class="ph codeph">ALTER
- TABLE</code> statement:
-<pre class="pre codeblock"><code>
+ <div class="p"> You can switch a table from
+ internal to external, or from external to internal, by using the
+ <code class="ph codeph">ALTER TABLE</code> statement:
+ <pre class="pre codeblock"><code>
-- Switch a table from internal to external.
ALTER TABLE <var class="keyword varname">table_name</var> SET TBLPROPERTIES('EXTERNAL'='TRUE');
-- Switch a table from external to internal.
ALTER TABLE <var class="keyword varname">table_name</var> SET TBLPROPERTIES('EXTERNAL'='FALSE');
-</code></pre>
- </div>
+</code></pre>If
+ the Kudu service is integrated with the Hive Metastore, the above
+ operations are not supported.</div>
<p class="p">
@@ -1377,4 +1383,4 @@ desc kt;
<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_langref_sql.html">Impala SQL Statements</a></div>
</div>
</div></body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_alter_view.html b/docs/build/html/topics/impala_alter_view.html
index 8dd2dd3..df2501b 100644
--- a/docs/build/html/topics/impala_alter_view.html
+++ b/docs/build/html/topics/impala_alter_view.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="alter_view" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -122,10 +122,11 @@ ALTER VIEW db1.v1 RENAME TO db2.v1; -- Move the view to a difference database wi
<p class="p">
- If you connect to different Impala nodes within an <span class="keyword cmdname">impala-shell</span> session for
- load-balancing purposes, you can enable the <code class="ph codeph">SYNC_DDL</code> query option to make each DDL
- statement wait before returning, until the new or changed metadata has been received by all the Impala
- nodes. See <a class="xref" href="../shared/../topics/impala_sync_ddl.html#sync_ddl">SYNC_DDL Query Option</a> for details.
+ If you connect to different Impala nodes within an <span class="keyword cmdname">impala-shell</span>
+ session for load-balancing purposes, you can enable the <code class="ph codeph">SYNC_DDL</code> query
+ option to make each DDL statement wait before returning, until the new or changed
+ metadata has been received by all the Impala nodes. See
+ <a class="xref" href="../shared/../topics/impala_sync_ddl.html#sync_ddl">SYNC_DDL Query Option</a> for details.
</p>
@@ -135,9 +136,10 @@ ALTER VIEW db1.v1 RENAME TO db2.v1; -- Move the view to a difference database wi
<p class="p">
- If these statements in your environment contain sensitive literal values such as credit card numbers or tax
- identifiers, Impala can redact this sensitive information when displaying the statements in log files and
- other administrative contexts. See <span class="xref">the documentation for your Apache Hadoop distribution</span> for details.
+ If these statements in your environment contain sensitive literal values such as credit
+ card numbers or tax identifiers, Impala can redact this sensitive information when
+ displaying the statements in log files and other administrative contexts. See
+ <span class="xref">the documentation for your Apache Hadoop distribution</span> for details.
</p>
diff --git a/docs/build/html/topics/impala_analytic_functions.html b/docs/build/html/topics/impala_analytic_functions.html
index 46b248e..0108a8a 100644
--- a/docs/build/html/topics/impala_analytic_functions.html
+++ b/docs/build/html/topics/impala_analytic_functions.html
@@ -28,25 +28,25 @@
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="analytic_functions" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -194,10 +194,14 @@ window_clause: See <a class="xref" href="#window_clause">Window Clause</a>
<p class="p">
- In queries involving both analytic functions and partitioned tables, partition pruning only occurs for columns named in the <code class="ph codeph">PARTITION BY</code>
- clause of the analytic function call. For example, if an analytic function query has a clause such as <code class="ph codeph">WHERE year=2016</code>,
- the way to make the query prune all other <code class="ph codeph">YEAR</code> partitions is to include <code class="ph codeph">PARTITION BY year</code> in the analytic function call;
- for example, <code class="ph codeph">OVER (PARTITION BY year,<var class="keyword varname">other_columns</var> <var class="keyword varname">other_analytic_clauses</var>)</code>.
+ In queries involving both analytic functions and partitioned tables, partition pruning
+ only occurs for columns named in the <code class="ph codeph">PARTITION BY</code> clause of the
+ analytic function call. For example, if an analytic function query has a clause such as
+ <code class="ph codeph">WHERE year=2016</code>, the way to make the query prune all other
+ <code class="ph codeph">YEAR</code> partitions is to include <code class="ph codeph">PARTITION BY year</code> in the
+ analytic function call; for example, <code class="ph codeph">OVER (PARTITION BY
+ year,<var class="keyword varname">other_columns</var>
+ <var class="keyword varname">other_analytic_clauses</var>)</code>.
</p>
diff --git a/docs/build/html/topics/impala_appx_count_distinct.html b/docs/build/html/topics/impala_appx_count_distinct.html
index 9654f38..0c45575 100644
--- a/docs/build/html/topics/impala_appx_count_distinct.html
+++ b/docs/build/html/topics/impala_appx_count_distinct.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="appx_count_distinct" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -41,13 +41,14 @@
<p class="p">
- <strong class="ph b">Type:</strong> Boolean; recognized values are 1 and 0, or <code class="ph codeph">true</code> and <code class="ph codeph">false</code>;
- any other value interpreted as <code class="ph codeph">false</code>
+ <strong class="ph b">Type:</strong> Boolean; recognized values are 1 and 0, or <code class="ph codeph">true</code> and
+ <code class="ph codeph">false</code>; any other value interpreted as <code class="ph codeph">false</code>
</p>
<p class="p">
- <strong class="ph b">Default:</strong> <code class="ph codeph">false</code> (shown as 0 in output of <code class="ph codeph">SET</code> statement)
+ <strong class="ph b">Default:</strong> <code class="ph codeph">false</code> (shown as 0 in output of <code class="ph codeph">SET</code>
+ statement)
</p>
diff --git a/docs/build/html/topics/impala_appx_median.html b/docs/build/html/topics/impala_appx_median.html
index 610c671..d052e31 100644
--- a/docs/build/html/topics/impala_appx_median.html
+++ b/docs/build/html/topics/impala_appx_median.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_aggregate_functions.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="appx_median" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -63,8 +63,8 @@
<p class="p">
- <strong class="ph b">Return type:</strong> Same as the input value, except for <code class="ph codeph">CHAR</code> and <code class="ph codeph">VARCHAR</code>
- arguments which produce a <code class="ph codeph">STRING</code> result
+ <strong class="ph b">Return type:</strong> Same as the input value, except for <code class="ph codeph">CHAR</code> and
+ <code class="ph codeph">VARCHAR</code> arguments which produce a <code class="ph codeph">STRING</code> result
</p>
@@ -82,7 +82,8 @@
<p class="p">
- This function cannot be used in an analytic context. That is, the <code class="ph codeph">OVER()</code> clause is not allowed at all with this function.
+ This function cannot be used in an analytic context. That is, the
+ <code class="ph codeph">OVER()</code> clause is not allowed at all with this function.
</p>
diff --git a/docs/build/html/topics/impala_array.html b/docs/build/html/topics/impala_array.html
index ca2f7dc..30a12c5 100644
--- a/docs/build/html/topics/impala_array.html
+++ b/docs/build/html/topics/impala_array.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_datatypes.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="array" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -53,11 +53,11 @@ type ::= <var class="keyword varname">primitive_type</var> | <var class="keyword
<p class="p">
- Because complex types are often used in combination,
- for example an <code class="ph codeph">ARRAY</code> of <code class="ph codeph">STRUCT</code>
- elements, if you are unfamiliar with the Impala complex types,
- start with <a class="xref" href="../shared/../topics/impala_complex_types.html#complex_types">Complex Types (Impala 2.3 or higher only)</a> for
- background information and usage examples.
+ Because complex types are often used in combination, for example an
+ <code class="ph codeph">ARRAY</code> of <code class="ph codeph">STRUCT</code> elements, if you are unfamiliar with
+ the Impala complex types, start with
+ <a class="xref" href="../shared/../topics/impala_complex_types.html#complex_types">Complex Types (Impala 2.3 or higher only)</a> for background
+ information and usage examples.
</p>
@@ -106,20 +106,18 @@ type ::= <var class="keyword varname">primitive_type</var> | <var class="keyword
<p class="p">
- You can pass a multi-part qualified name to <code class="ph codeph">DESCRIBE</code>
- to specify an <code class="ph codeph">ARRAY</code>, <code class="ph codeph">STRUCT</code>, or <code class="ph codeph">MAP</code>
- column and visualize its structure as if it were a table.
- For example, if table <code class="ph codeph">T1</code> contains an <code class="ph codeph">ARRAY</code> column
- <code class="ph codeph">A1</code>, you could issue the statement <code class="ph codeph">DESCRIBE t1.a1</code>.
- If table <code class="ph codeph">T1</code> contained a <code class="ph codeph">STRUCT</code> column <code class="ph codeph">S1</code>,
- and a field <code class="ph codeph">F1</code> within the <code class="ph codeph">STRUCT</code> was a <code class="ph codeph">MAP</code>,
- you could issue the statement <code class="ph codeph">DESCRIBE t1.s1.f1</code>.
- An <code class="ph codeph">ARRAY</code> is shown as a two-column table, with
- <code class="ph codeph">ITEM</code> and <code class="ph codeph">POS</code> columns.
- A <code class="ph codeph">STRUCT</code> is shown as a table with each field
- representing a column in the table.
- A <code class="ph codeph">MAP</code> is shown as a two-column table, with
- <code class="ph codeph">KEY</code> and <code class="ph codeph">VALUE</code> columns.
+ You can pass a multi-part qualified name to <code class="ph codeph">DESCRIBE</code> to specify an
+ <code class="ph codeph">ARRAY</code>, <code class="ph codeph">STRUCT</code>, or <code class="ph codeph">MAP</code> column and
+ visualize its structure as if it were a table. For example, if table <code class="ph codeph">T1</code>
+ contains an <code class="ph codeph">ARRAY</code> column <code class="ph codeph">A1</code>, you could issue the
+ statement <code class="ph codeph">DESCRIBE t1.a1</code>. If table <code class="ph codeph">T1</code> contained a
+ <code class="ph codeph">STRUCT</code> column <code class="ph codeph">S1</code>, and a field <code class="ph codeph">F1</code>
+ within the <code class="ph codeph">STRUCT</code> was a <code class="ph codeph">MAP</code>, you could issue the
+ statement <code class="ph codeph">DESCRIBE t1.s1.f1</code>. An <code class="ph codeph">ARRAY</code> is shown as a
+ two-column table, with <code class="ph codeph">ITEM</code> and <code class="ph codeph">POS</code> columns. A
+ <code class="ph codeph">STRUCT</code> is shown as a table with each field representing a column in the
+ table. A <code class="ph codeph">MAP</code> is shown as a two-column table, with <code class="ph codeph">KEY</code>
+ and <code class="ph codeph">VALUE</code> columns.
</p>
@@ -136,37 +134,44 @@ type ::= <var class="keyword varname">primitive_type</var> | <var class="keyword
<ul class="ul">
<li class="li">
<p class="p">
- Columns with this data type can only be used in tables or partitions with the Parquet or ORC file format.
+ Columns with this data type can only be used in tables or partitions with the
+ Parquet or ORC file format.
</p>
</li>
+
<li class="li">
<p class="p">
- Columns with this data type cannot be used as partition key columns in a partitioned table.
+ Columns with this data type cannot be used as partition key columns in a partitioned
+ table.
</p>
</li>
+
<li class="li">
<p class="p">
- The <code class="ph codeph">COMPUTE STATS</code> statement does not produce any statistics for columns of this data type.
+ The <code class="ph codeph">COMPUTE STATS</code> statement does not produce any statistics for
+ columns of this data type.
</p>
</li>
+
<li class="li">
- <p class="p" id="array__d6e3269">
- The maximum length of the column definition for any complex type, including declarations for any nested types,
- is 4000 characters.
+ <p class="p" id="array__d6e3282">
+ The maximum length of the column definition for any complex type, including
+ declarations for any nested types, is 4000 characters.
</p>
</li>
+
<li class="li">
<p class="p">
- See <a class="xref" href="../shared/../topics/impala_complex_types.html#complex_types_limits">Limitations and Restrictions for Complex Types</a> for a full list of limitations
- and associated guidelines about complex type columns.
+ See <a class="xref" href="../shared/../topics/impala_complex_types.html#complex_types_limits">Limitations and Restrictions for Complex Types</a> for a
+ full list of limitations and associated guidelines about complex type columns.
</p>
</li>
@@ -180,7 +185,8 @@ type ::= <var class="keyword varname">primitive_type</var> | <var class="keyword
<p class="p">
Currently, the data types <code class="ph codeph">CHAR</code>, <code class="ph codeph">VARCHAR</code>,
- <code class="ph codeph">ARRAY</code>, <code class="ph codeph">MAP</code>, and <code class="ph codeph">STRUCT</code> cannot be used with Kudu tables.
+ <code class="ph codeph">ARRAY</code>, <code class="ph codeph">MAP</code>, and <code class="ph codeph">STRUCT</code> cannot be used
+ with Kudu tables.
</p>
@@ -189,12 +195,11 @@ type ::= <var class="keyword varname">primitive_type</var> | <var class="keyword
</p>
- <div class="note note"><span class="notetitle">Note:</span>
- Many of the complex type examples refer to tables
- such as <code class="ph codeph">CUSTOMER</code> and <code class="ph codeph">REGION</code>
- adapted from the tables used in the TPC-H benchmark.
- See <a class="xref" href="../shared/../topics/impala_complex_types.html#complex_sample_schema">Sample Schema and Data for Experimenting with Impala Complex Types</a>
- for the table definitions.
+ <div class="note note"><span class="notetitle">Note:</span>
+ Many of the complex type examples refer to tables such as <code class="ph codeph">CUSTOMER</code> and
+ <code class="ph codeph">REGION</code> adapted from the tables used in the TPC-H benchmark. See
+ <a class="xref" href="../shared/../topics/impala_complex_types.html#complex_sample_schema">Sample Schema and Data for Experimenting with Impala Complex Types</a> for the table
+ definitions.
</div>
@@ -209,7 +214,7 @@ type ::= <var class="keyword varname">primitive_type</var> | <var class="keyword
</p>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
Practice the <code class="ph codeph">CREATE TABLE</code> and query notation for complex type columns
using empty tables, until you can visualize a complex data structure and construct corresponding SQL statements reliably.
</div>
@@ -377,4 +382,4 @@ WHERE places_lived.place like '%California%';
<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_datatypes.html">Data Types</a></div>
</div>
</div></body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_auditing.html b/docs/build/html/topics/impala_auditing.html
index 778a855..87f94f1 100644
--- a/docs/build/html/topics/impala_auditing.html
+++ b/docs/build/html/topics/impala_auditing.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_security.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="auditing" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -23,53 +23,53 @@
<h1 class="title topictitle1" id="ariaid-title1">Auditing Impala Operations</h1>
+
+
<div class="body conbody">
<p class="p">
- To monitor how Impala data is being used within your organization, ensure
- that your Impala authorization and authentication policies are effective.
- To detect attempts at intrusion or unauthorized access to Impala
- data, you can use the auditing feature in Impala 1.2.1 and higher:
+ To monitor how Impala data is being used within your organization, ensure that your Impala
+ authorization and authentication policies are effective. To detect attempts at intrusion
+ or unauthorized access to Impala data, you can use the auditing feature in Impala 1.2.1
+ and higher:
</p>
<ul class="ul">
<li class="li">
Enable auditing by including the option
- <code class="ph codeph">-audit_event_log_dir=<var class="keyword varname">directory_path</var></code>
- in your <span class="keyword cmdname">impalad</span> startup options.
- The log directory must be a local directory on the
- server, not an HDFS directory.
+ <code class="ph codeph">‑‑audit_event_log_dir=<var class="keyword varname">directory_path</var></code> in
+ your <span class="keyword cmdname">impalad</span> startup options. The log directory must be a local
+ directory on the server, not an HDFS directory.
</li>
<li class="li">
Decide how many queries will be represented in each audit event log file. By default,
- Impala starts a new audit event log file every 5000 queries. To specify a different number,
- <span class="ph">include
- the option <code class="ph codeph">--max_audit_event_log_file_size=<var class="keyword varname">number_of_queries</var></code>
+ Impala starts a new audit event log file every 5000 queries. To specify a different
+ number, <span class="ph">include the option
+ <code class="ph codeph">‑‑max_audit_event_log_file_size=<var class="keyword varname">number_of_queries</var></code>
in the <span class="keyword cmdname">impalad</span> startup options</span>.
</li>
<li class="li">
- In <span class="keyword">Impala 2.9</span> and higher, you can control how many
- audit event log files are kept on each host. Specify the option
- <code class="ph codeph">--max_audit_event_log_files=<var class="keyword varname">number_of_log_files</var></code>
+ In <span class="keyword">Impala 2.9</span> and higher, you can control how many audit event
+ log files are kept on each host. Specify the option
+ <code class="ph codeph">‑‑max_audit_event_log_files=<var class="keyword varname">number_of_log_files</var></code>
in the <span class="keyword cmdname">impalad</span> startup options. Once the limit is reached, older
- files are rotated out using the same mechanism as for other Impala log files.
- The default value for this setting is 0, representing an unlimited number of audit
- event log files.
+ files are rotated out using the same mechanism as for other Impala log files. The
+ default value for this setting is 0, representing an unlimited number of audit event log
+ files.
</li>
<li class="li">
- Use a cluster manager with governance capabilities to filter, visualize,
- and produce reports based on the audit logs collected
- from all the hosts in the cluster.
+ Use a cluster manager with governance capabilities to filter, visualize, and produce
+ reports based on the audit logs collected from all the hosts in the cluster.
</li>
</ul>
@@ -77,6 +77,7 @@
<p class="p toc inpage"></p>
+
</div>
@@ -88,7 +89,8 @@
<h2 class="title topictitle2" id="ariaid-title2">Durability and Performance Considerations for Impala Auditing</h2>
-
+
+
<div class="body conbody">
@@ -101,16 +103,17 @@
Because any Impala host can process a query, enable auditing on all hosts where the
<span class="ph"><span class="keyword cmdname">impalad</span> daemon</span>
runs. Each host stores its own log
- files, in a directory in the local filesystem. The log data is periodically flushed to disk (through an
- <code class="ph codeph">fsync()</code> system call) to avoid loss of audit data in case of a crash.
+ files, in a directory in the local filesystem. The log data is periodically flushed to
+ disk (through an <code class="ph codeph">fsync()</code> system call) to avoid loss of audit data in
+ case of a crash.
</p>
<p class="p">
- The runtime overhead of auditing applies to whichever host serves as the coordinator
- for the query, that is, the host you connect to when you issue the query. This might
- be the same host for all queries, or different applications or users might connect to
- and issue queries through different hosts.
+ The runtime overhead of auditing applies to whichever host serves as the coordinator for
+ the query, that is, the host you connect to when you issue the query. This might be the
+ same host for all queries, or different applications or users might connect to and issue
+ queries through different hosts.
</p>
@@ -123,15 +126,18 @@
<p class="p">
- By default, Impala avoids losing any audit log data in the case of an error during a logging operation
- (such as a disk full error), by immediately shutting down
+ By default, Impala avoids losing any audit log data in the case of an error during a
+ logging operation (such as a disk full error), by immediately shutting down
<span class="keyword cmdname">impalad</span> on the host where the auditing problem occurred.
- <span class="ph">You can override this setting by specifying the option
- <code class="ph codeph">-abort_on_failed_audit_event=false</code> in the <span class="keyword cmdname">impalad</span> startup options.</span>
+ <span class="ph">You can override this setting by specifying the
+ option <code class="ph codeph">‑‑abort_on_failed_audit_event=false</code> in the
+ <span class="keyword cmdname">impalad</span> startup options.</span>
</p>
+
</div>
+
</div>
@@ -139,28 +145,30 @@
<h2 class="title topictitle2" id="ariaid-title3">Format of the Audit Log Files</h2>
-
+
+
<div class="body conbody">
<p class="p">
The audit log files represent the query information in JSON format, one query per line.
- Typically, rather than looking at the log files themselves, you should use cluster-management
- software to consolidate the log data from all Impala hosts and filter and visualize the results
- in useful ways. (If you do examine the raw log data, you might run the files through
- a JSON pretty-printer first.)
- </p>
+ Typically, rather than looking at the log files themselves, you should use
+ cluster-management software to consolidate the log data from all Impala hosts and filter
+ and visualize the results in useful ways. (If you do examine the raw log data, you might
+ run the files through a JSON pretty-printer first.)
+ </p>
<p class="p">
- All the information about schema objects accessed by the query is encoded in a single nested record on the
- same line. For example, the audit log for an <code class="ph codeph">INSERT ... SELECT</code> statement records that a
- select operation occurs on the source table and an insert operation occurs on the destination table. The
- audit log for a query against a view records the base table accessed by the view, or multiple base tables
- in the case of a view that includes a join query. Every Impala operation that corresponds to a SQL
- statement is recorded in the audit logs, whether the operation succeeds or fails. Impala records more
- information for a successful operation than for a failed one, because an unauthorized query is stopped
- immediately, before all the query planning is completed.
+ All the information about schema objects accessed by the query is encoded in a single
+ nested record on the same line. For example, the audit log for an <code class="ph codeph">INSERT ...
+ SELECT</code> statement records that a select operation occurs on the source table and
+ an insert operation occurs on the destination table. The audit log for a query against a
+ view records the base table accessed by the view, or multiple base tables in the case of
+ a view that includes a join query. Every Impala operation that corresponds to a SQL
+ statement is recorded in the audit logs, whether the operation succeeds or fails. Impala
+ records more information for a successful operation than for a failed one, because an
+ unauthorized query is stopped immediately, before all the query planning is completed.
</p>
@@ -236,8 +244,8 @@
<li class="li">
- Privilege - How the object is being used (<code class="ph codeph">SELECT</code>, <code class="ph codeph">INSERT</code>,
- <code class="ph codeph">CREATE</code>, and so on)
+ Privilege - How the object is being used (<code class="ph codeph">SELECT</code>,
+ <code class="ph codeph">INSERT</code>, <code class="ph codeph">CREATE</code>, and so on)
</li>
</ul>
@@ -252,8 +260,10 @@
+
</div>
+
</div>
@@ -276,31 +286,37 @@
<li class="li">
- Queries that Impala can analyze and parse to determine that they are authorized. The audit data is
- recorded immediately after Impala finishes its analysis, before the query is actually executed.
+ Queries that Impala can analyze and parse to determine that they are authorized. The
+ audit data is recorded immediately after Impala finishes its analysis, before the
+ query is actually executed.
</li>
</ul>
<p class="p">
- The audit log does not contain entries for queries that could not be parsed and analyzed. For example, a
- query that fails due to a syntax error is not recorded in the audit log. The audit log also does not
- contain queries that fail due to a reference to a table that does not exist, if you would be authorized to
- access the table if it did exist.
+ The audit log does not contain entries for queries that could not be parsed and
+ analyzed. For example, a query that fails due to a syntax error is not recorded in the
+ audit log. The audit log also does not contain queries that fail due to a reference to a
+ table that does not exist, if you would be authorized to access the table if it did
+ exist.
</p>
<p class="p">
- Certain statements in the <span class="keyword cmdname">impala-shell</span> interpreter, such as <code class="ph codeph">CONNECT</code>,
- <code class="ph codeph">SUMMARY</code>, <code class="ph codeph">PROFILE</code>, <code class="ph codeph">SET</code>, and
- <code class="ph codeph">QUIT</code>, do not correspond to actual SQL queries, and these statements are not reflected in
- the audit log.
+ Certain statements in the <span class="keyword cmdname">impala-shell</span> interpreter, such as
+ <code class="ph codeph">CONNECT</code>, <code class="ph codeph">SUMMARY</code>,
+ <code class="ph codeph">PROFILE</code>, <code class="ph codeph">SET</code>, and <code class="ph codeph">QUIT</code>, do not
+ correspond to actual SQL queries, and these statements are not reflected in the audit
+ log.
</p>
+
</div>
+
</div>
+
</body>
</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_authentication.html b/docs/build/html/topics/impala_authentication.html
index 6cbf162..6425da5 100644
--- a/docs/build/html/topics/impala_authentication.html
+++ b/docs/build/html/topics/impala_authentication.html
@@ -15,8 +15,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_delegation.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="authentication" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -44,13 +44,18 @@
Impala supports authentication using either Kerberos or LDAP.
</p>
+ <p class="p">
+ You can also make proxy connections to Impala through Apache Knox.
+ </p>
+
- <div class="note note"><span class="notetitle">Note:</span>
- Regardless of the authentication mechanism used, Impala always creates HDFS directories and data files
- owned by the same user (typically <code class="ph codeph">impala</code>). To implement user-level access to different
- databases, tables, columns, partitions, and so on, use the Sentry authorization feature, as explained in
- <a class="xref" href="../shared/../topics/impala_authorization.html#authorization">Enabling Sentry Authorization for Impala</a>.
- </div>
+ <div class="note note"><span class="notetitle">Note:</span>
+ Regardless of the authentication mechanism used, Impala always creates HDFS directories
+ and data files owned by the same user (typically <code class="ph codeph">impala</code>). To implement
+ user-level access to different databases, tables, columns, partitions, and so on, use
+ the Sentry authorization feature, as explained in
+ <a class="xref" href="../shared/../topics/impala_authorization.html#authorization">Impala Authorization</a>.
+ </div>
<p class="p toc"></p>
@@ -59,7 +64,7 @@
<p class="p">
Once you are finished setting up authentication, move on to authorization, which involves specifying what
databases, tables, HDFS directories, and so on can be accessed by particular users when they connect through
- Impala. See <a class="xref" href="impala_authorization.html#authorization">Enabling Sentry Authorization for Impala</a> for details.
+ Impala. See <a class="xref" href="impala_authorization.html#authorization">Impala Authorization</a> for details.
</p>
</div>
@@ -72,7 +77,7 @@
</li>
<li class="link ulchildlink"><strong><a href="../topics/impala_mixed_security.html">Using Multiple Authentication Methods with Impala</a></strong><br />
</li>
-<li class="link ulchildlink"><strong><a href="../topics/impala_delegation.html">Configuring Impala Delegation for Hue and BI Tools</a></strong><br />
+<li class="link ulchildlink"><strong><a href="../topics/impala_delegation.html">Configuring Impala Delegation for Clients</a></strong><br />
</li>
</ul>
@@ -80,4 +85,4 @@
<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_security.html">Impala Security</a></div>
</div>
</div></body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_authorization.html b/docs/build/html/topics/impala_authorization.html
index 0f62361..1077433 100644
--- a/docs/build/html/topics/impala_authorization.html
+++ b/docs/build/html/topics/impala_authorization.html
@@ -7,21 +7,21 @@
<meta name="copyright" content="(C) Copyright 2019" />
<meta name="DC.rights.owner" content="(C) Copyright 2019" />
<meta name="DC.Type" content="concept" />
-<meta name="DC.Title" content="Enabling Sentry Authorization for Impala" />
+<meta name="DC.Title" content="Impala Authorization" />
<meta name="DC.Relation" scheme="URI" content="../topics/impala_security.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="authorization" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
-<title>Enabling Sentry Authorization for Impala</title>
+<title>Impala Authorization</title>
</head>
<body id="authorization">
- <h1 class="title topictitle1" id="ariaid-title1">Enabling Sentry Authorization for Impala</h1>
+ <h1 class="title topictitle1" id="ariaid-title1">Impala Authorization</h1>
@@ -30,28 +30,18 @@
<p class="p">
Authorization determines which users are allowed to access which resources, and what
- operations they are allowed to perform. In Impala 1.1 and higher, you use Apache Sentry
- for authorization. Sentry adds a fine-grained authorization framework for Hadoop. By
- default (when authorization is not enabled), Impala does all read and write operations
- with the privileges of the <code class="ph codeph">impala</code> user, which is suitable for a
- development/test environment but not for a secure production environment. When
- authorization is enabled, Impala uses the OS user ID of the user who runs
- <span class="keyword cmdname">impala-shell</span> or other client program, and associates various privileges
- with each user.
+ operations they are allowed to perform. You use Apache Sentry or Apache Ranger for
+ authorization. By default, when authorization is not enabled, Impala does all read and
+ write operations with the privileges of the <code class="ph codeph">impala</code> user, which is
+ suitable for a development/test environment but not for a secure production environment.
+ When authorization is enabled, Impala uses the OS user ID of the user who runs
+ <span class="keyword cmdname">impala-shell</span> or other client programs, and associates various
+ privileges with each user.
</p>
- <div class="note note"><span class="notetitle">Note:</span>
- Sentry is typically used in conjunction with Kerberos authentication, which defines which
- hosts are allowed to connect to each server. Using the combination of Sentry and Kerberos
- prevents malicious users from being able to connect by creating a named account on an
- untrusted machine. See <a class="xref" href="impala_kerberos.html#kerberos">Enabling Kerberos Authentication for Impala</a> for details about
- Kerberos authentication.
- </div>
-
-
<p class="p toc inpage">
- See the following sections for details about using the Impala authorization features:
+ See the following sections for details about using the Impala authorization features.
</p>
@@ -64,7 +54,7 @@
</div>
</div><div class="topic concept nested1" aria-labelledby="ariaid-title2" id="sentry_priv_model">
- <h2 class="title topictitle2" id="ariaid-title2">The Sentry Privilege Model</h2>
+ <h2 class="title topictitle2" id="ariaid-title2">The Privilege Model</h2>
<div class="body conbody">
@@ -101,769 +91,1217 @@
</p>
- <div class="p">The table below lists the minimum level
- of privileges and the scope required to execute SQL statements in
- <span class="keyword">Impala 3.0</span> and higher. The following notations
- are used:<ul class="ul">
- <li class="li"><strong class="ph b">ANY</strong> denotes the <code class="ph codeph">SELECT</code>,
- <code class="ph codeph">INSERT</code>, <code class="ph codeph">CREATE</code>,
- <code class="ph codeph">ALTER</code>, <code class="ph codeph">DROP</code>, <strong class="ph b"><em class="ph i">or</em></strong>
- <code class="ph codeph">REFRESH</code> privilege.</li>
+ <div class="p">
+ The table below lists the minimum level of privileges and the scope required to execute
+ SQL statements in <span class="keyword">Impala 3.0</span> and higher. The following notations
+ are used:
+ <ul class="ul">
+ <li class="li">
+ <strong class="ph b">ANY</strong> denotes the <code class="ph codeph">SELECT</code>, <code class="ph codeph">INSERT</code>,
+ <code class="ph codeph">CREATE</code>, <code class="ph codeph">ALTER</code>, <code class="ph codeph">DROP</code>,
+ <strong class="ph b"><em class="ph i">or</em></strong> <code class="ph codeph">REFRESH</code> privilege.
+ </li>
+
+
+ <li class="li">
+ <strong class="ph b">ALL</strong> privilege denotes the <code class="ph codeph">SELECT</code>, <code class="ph codeph">INSERT</code>,
+ <code class="ph codeph">CREATE</code>, <code class="ph codeph">ALTER</code>, <code class="ph codeph">DROP</code>,
+ <strong class="ph b"><em class="ph i">and</em></strong> <code class="ph codeph">REFRESH</code> privileges.
+ </li>
- <li class="li"><strong class="ph b">ALL</strong> privilege denotes the <code class="ph codeph">SELECT</code>,
- <code class="ph codeph">INSERT</code>, <code class="ph codeph">CREATE</code>,
- <code class="ph codeph">ALTER</code>, <code class="ph codeph">DROP</code>, <strong class="ph b"><em class="ph i">and</em></strong>
- <code class="ph codeph">REFRESH</code> privileges.</li>
- <li class="li">The owner of an object effectively has the ALL privilege on the
- object.</li>
+ <li class="li">
+ The owner of an object effectively has the ALL privilege on the object.
+ </li>
+
- <li class="li">The parent levels of the specified scope are implicitly supported
- where a scope refers to the specific level in the object hierarchy
- that the privilege is granted. For example, if a privilege is listed
- with the <code class="ph codeph">TABLE</code> scope, the same privilege granted on
- <code class="ph codeph">DATABASE</code> and <code class="ph codeph">SERVER</code> will allow
- the user to execute the specified SQL statement.</li>
+ <li class="li">
+ The parent levels of the specified scope are implicitly supported where a scope
+ refers to the specific level in the object hierarchy that the privilege is granted.
+ For example, if a privilege is listed with the <code class="ph codeph">TABLE</code> scope, the
+ same privilege granted on <code class="ph codeph">DATABASE</code> and <code class="ph codeph">SERVER</code> will
+ allow the user to execute the specified SQL statement.
+ </li>
</ul>
-<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="sentry_priv_model__d6e169" class="table" frame="border" border="1" rules="all"><colgroup><col /><col /><col /></colgroup><tbody class="tbody">
+
+<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="sentry_priv_model__d6e172" class="table" frame="border" border="1" rules="all"><colgroup><col /><col /><col /></colgroup><tbody class="tbody">
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;"><strong class="ph b">SQL Statement</strong></td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ <strong class="ph b">SQL Statement</strong>
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;"><strong class="ph b">Privileges</strong></td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ <strong class="ph b">Privileges</strong>
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;"><strong class="ph b">Scope</strong></td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ <strong class="ph b">Object Type / </strong><p class="p"><strong class="ph b">Resource Type</strong></p>
+</td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">SELECT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SELECT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SELECT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SELECT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;"> TABLE</td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">WITH SELECT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ WITH SELECT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SELECT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SELECT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">EXPLAIN SELECT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ EXPLAIN SELECT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SELECT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SELECT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">INSERT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ INSERT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">INSERT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ INSERT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">EXPLAIN INSERT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ EXPLAIN INSERT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">INSERT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ INSERT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">TRUNCATE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TRUNCATE
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">INSERT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ INSERT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">LOAD</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ LOAD
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">INSERT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ INSERT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
<td class="entry cellrowborder" style="vertical-align:top;"> </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALL</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALL
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">URI</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ URI
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">CREATE DATABASE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ CREATE DATABASE
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">CREATE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ CREATE
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SERVER</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SERVER
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">CREATE DATABASE LOCATION</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ CREATE DATABASE LOCATION
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">CREATE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ CREATE
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SERVER</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SERVER
+ </td>
</tr>
<tr class="row">
<td class="entry cellrowborder" style="vertical-align:top;"> </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALL</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALL
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">URI</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ URI
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">CREATE TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ CREATE TABLE
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">CREATE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ CREATE
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">DATABASE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DATABASE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">CREATE TABLE LIKE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ CREATE TABLE LIKE
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">CREATE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ CREATE
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">DATABASE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DATABASE
+ </td>
</tr>
<tr class="row">
<td class="entry cellrowborder" style="vertical-align:top;"> </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">CREATE TABLE AS SELECT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ CREATE TABLE AS SELECT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">CREATE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ CREATE
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">DATABASE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DATABASE
+ </td>
</tr>
<tr class="row">
<td class="entry cellrowborder" style="vertical-align:top;"> </td>
- <td class="entry cellrowborder" style="vertical-align:top;">INSERT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ INSERT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">DATABASE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DATABASE
+ </td>
</tr>
<tr class="row">
<td class="entry cellrowborder" style="vertical-align:top;"> </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SELECT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SELECT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">EXPLAIN CREATE TABLE AS SELECT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ EXPLAIN CREATE TABLE AS SELECT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">CREATE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ CREATE
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">DATABASE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DATABASE
+ </td>
</tr>
<tr class="row">
<td class="entry cellrowborder" style="vertical-align:top;"> </td>
- <td class="entry cellrowborder" style="vertical-align:top;">INSERT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ INSERT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">DATABASE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DATABASE
+ </td>
</tr>
<tr class="row">
<td class="entry cellrowborder" style="vertical-align:top;"> </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SELECT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SELECT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">CREATE TABLE LOCATION</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ CREATE TABLE LOCATION
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">CREATE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ CREATE
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
<td class="entry cellrowborder" style="vertical-align:top;"> </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALL</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALL
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">URI</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ URI
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">CREATE VIEW</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ CREATE VIEW
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">CREATE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ CREATE
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">DATABASE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DATABASE
+ </td>
</tr>
<tr class="row">
<td class="entry cellrowborder" style="vertical-align:top;"> </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SELECT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SELECT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">ALTER DATABASE SET OWNER</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALTER DATABASE SET OWNER
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALL WITH GRANT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALL WITH GRANT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">DATABASE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DATABASE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">ALTER TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALTER TABLE
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALTER</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALTER
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">ALTER TABLE SET LOCATION</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALTER TABLE SET LOCATION
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALTER</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALTER
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
<td class="entry cellrowborder" style="vertical-align:top;"> </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALL</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALL
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">URI</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ URI
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">ALTER TABLE RENAME</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALTER TABLE RENAME
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">CREATE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ CREATE
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">DATABASE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DATABASE
+ </td>
</tr>
<tr class="row">
<td class="entry cellrowborder" style="vertical-align:top;"> </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALL</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALL
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">ALTER TABLE SET OWNER</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALTER TABLE SET OWNER
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALL WITH GRANT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALL WITH GRANT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">ALTER VIEW</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALTER VIEW
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALTER</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALTER
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
<td class="entry cellrowborder" style="vertical-align:top;"> </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SELECT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SELECT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">ALTER VIEW RENAME</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALTER VIEW RENAME
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">CREATE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ CREATE
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">DATABASE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DATABASE
+ </td>
</tr>
<tr class="row">
<td class="entry cellrowborder" style="vertical-align:top;"> </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALL</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALL
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">ALTER VIEW SET OWNER</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALTER VIEW SET OWNER
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALL WITH GRANT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALL WITH GRANT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">VIEW</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ VIEW
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">DROP DATABASE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DROP DATABASE
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">DROP</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DROP
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">DATABASE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DATABASE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">DROP TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DROP TABLE
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">DROP</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DROP
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">DROP VIEW</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DROP VIEW
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">DROP</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DROP
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">CREATE FUNCTION</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ CREATE FUNCTION
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">CREATE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ CREATE
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">DATABASE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DATABASE
+ </td>
</tr>
<tr class="row">
<td class="entry cellrowborder" style="vertical-align:top;"> </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALL</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALL
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">URI</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ URI
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">DROP FUNCTION</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DROP FUNCTION
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">DROP</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DROP
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">DATABASE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DATABASE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">COMPUTE STATS</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ COMPUTE STATS
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALTER and SELECT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALTER and SELECT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">DROP STATS</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DROP STATS
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALTER</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALTER
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">INVALIDATE METADATA</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ INVALIDATE METADATA
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">REFRESH</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ REFRESH
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SERVER</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SERVER
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">INVALIDATE METADATA <table></td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ INVALIDATE METADATA <table>
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">REFRESH</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ REFRESH
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">REFRESH <table></td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ REFRESH <table>
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">REFRESH</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ REFRESH
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">REFRESH AUTHORIZATION</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ REFRESH AUTHORIZATION
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">REFRESH</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ REFRESH
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SERVER</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SERVER
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">REFRESH FUNCTIONS</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ REFRESH FUNCTIONS
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">REFRESH</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ REFRESH
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">DATABASE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DATABASE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">COMMENT ON DATABASE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ COMMENT ON DATABASE
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALTER</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALTER
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">DATABASE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DATABASE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">COMMENT ON TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ COMMENT ON TABLE
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALTER</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALTER
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">COMMENT ON VIEW</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ COMMENT ON VIEW
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALTER</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALTER
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">COMMENT ON COLUMN</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ COMMENT ON COLUMN
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALTER</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALTER
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">DESCRIBE DATABASE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DESCRIBE DATABASE
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">DATABASE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DATABASE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">DESCRIBE <table/view></td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DESCRIBE <table/view>
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">If the user has the SELECT privilege at the COLUMN level,
- only the columns the user has access will show.</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ If the user has the SELECT privilege at the COLUMN level, only the columns the
+ user has access will show.
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SELECT</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SELECT
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">COLUMN</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ COLUMN
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">USE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ USE
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ANY</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ANY
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">SHOW DATABASES</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SHOW DATABASES
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ANY</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ANY
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">SHOW TABLES</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SHOW TABLES
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ANY</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ANY
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">SHOW FUNCTIONS</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SHOW FUNCTIONS
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">DATABASE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DATABASE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">SHOW PARTITIONS</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SHOW PARTITIONS
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">SHOW TABLE STATS</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SHOW TABLE STATS
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">SHOW COLUMN STATS</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SHOW COLUMN STATS
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">SHOW FILES</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SHOW FILES
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">SHOW CREATE TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SHOW CREATE TABLE
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">SHOW CREATE VIEW</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SHOW CREATE VIEW
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">SHOW CREATE FUNCTION</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SHOW CREATE FUNCTION
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">DATABASE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DATABASE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">SHOW RANGE PARTITIONS (Kudu only)</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SHOW RANGE PARTITIONS (Kudu only)
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ SELECT, INSERT, <strong class="ph b"><em class="ph i">or</em></strong> REFRESH
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">UPDATE (Kudu only)</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ UPDATE (Kudu only)
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALL</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALL
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">EXPLAIN UPDATE (Kudu only)</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ EXPLAIN UPDATE (Kudu only)
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALL</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALL
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">UPSERT (Kudu only)</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ UPSERT (Kudu only)
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALL</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALL
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">WITH UPSERT (Kudu only)</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ WITH UPSERT (Kudu only)
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALL</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALL
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">EXPLAIN UPSERT (Kudu only)</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ EXPLAIN UPSERT (Kudu only)
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALL</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALL
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">DELETE (Kudu only)</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ DELETE (Kudu only)
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALL</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALL
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
<tr class="row">
- <td class="entry cellrowborder" style="vertical-align:top;">EXPLAIN DELETE (Kudu only)</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ EXPLAIN DELETE (Kudu only)
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">ALL</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ ALL
+ </td>
- <td class="entry cellrowborder" style="vertical-align:top;">TABLE</td>
+ <td class="entry cellrowborder" style="vertical-align:top;">
+ TABLE
+ </td>
</tr>
</tbody>
</table>
</div>
-</div>
+
+ </div>
<p class="p">
- Originally, privileges were encoded in a policy file, stored in HDFS. This mode of
- operation is still an option, but the emphasis of privilege management is moving towards
- being SQL-based. The mode of operation with <code class="ph codeph">GRANT</code> and
- <code class="ph codeph">REVOKE</code> statements instead of the policy file requires that a special
- Sentry service be enabled; this service stores, retrieves, and manipulates privilege
- information stored inside the metastore database.
+ Privileges are managed via the <code class="ph codeph">GRANT</code> and <code class="ph codeph">REVOKE</code> SQL
+ statements that require the Sentry or Ranger service enabled.
</p>
- <div class="note note"><span class="notetitle">Note:</span>
- <p class="p">
- Although this document refers to the <code class="ph codeph">ALL</code> privilege, currently if you
- use the policy file mode, you do not use the actual keyword <code class="ph codeph">ALL</code> in
- the policy file. When you code role entries in the policy file:
- </p>
+ <p class="p">
+ If you change privileges outside of Impala, e.g. adding a user, removing a user,
+ modifying privileges, you must clear the Impala Catalog server cache by running the
+ <code class="ph codeph">REFRESH AUTHORIZATION</code> statement. <code class="ph codeph">REFRESH AUTHORIZATION</code>
+ is not required if you make the changes to privileges within Impala.
+ </p>
- <ul class="ul">
- <li class="li">
- To specify the <code class="ph codeph">ALL</code> privilege for a server, use a role like
- <code class="ph codeph">server=<var class="keyword varname">server_name</var></code>.
- </li>
+ </div>
- <li class="li">
- To specify the <code class="ph codeph">ALL</code> privilege for a database, use a role like
- <code class="ph codeph">server=<var class="keyword varname">server_name</var>->db=<var class="keyword varname">database_name</var></code>.
- </li>
+ </div>
- <li class="li">
- To specify the <code class="ph codeph">ALL</code> privilege for a table, use a role like
- <code class="ph codeph">server=<var class="keyword varname">server_name</var>->db=<var class="keyword varname">database_name</var>->table=<var class="keyword varname">table_name</var>->action=*</code>.
- </li>
- </ul>
+ <div class="topic concept nested1" aria-labelledby="ariaid-title3" id="object_ownership">
- </div>
+ <h2 class="title topictitle2" id="ariaid-title3">Object Ownership in Sentry</h2>
+
+
+ <div class="body conbody">
+
+ <p class="p">
+ Impala supports the ownership on databases, tables, and views. The
+ <code class="ph codeph">CREATE</code> statements implicitly make the user running the statement the
+ owner of the object. An owner has the <code class="ph codeph">OWNER</code> privilege if enabled in
+ Sentry. For example, if <var class="keyword varname">User A</var> creates a database,
+ <var class="keyword varname">foo</var>, via the <code class="ph codeph">CREATE DATABASE</code> statement,
+ <var class="keyword varname">User A</var> now owns the <var class="keyword varname">foo</var> database and is authorized
+ to perform any operation on the <var class="keyword varname">foo</var> database.
+ </p>
+
+
+ <p class="p">
+ The <code class="ph codeph">OWNER</code> privilege is not a grantable or revokable privilege whereas
+ the <code class="ph codeph">ALL</code> privilege is explicitly granted via the <code class="ph codeph">GRANT</code>
+ statement.
+ </p>
+
+
+ <p class="p">
+ The object ownership feature is controlled by a Sentry configuration. The
+ <code class="ph codeph">OWNER</code> privilege is only granted when the feature is enabled in Sentry.
+ When enabled they get the owner privilege, with or without the <code class="ph codeph">GRANT
+ OPTION</code>, which is also controlled by the Sentry configuration.
+ </p>
<p class="p">
- If you change privileges in Sentry, e.g. adding a user, removing a user, modifying
- privileges, you must clear the Impala Catalog server cache by running the
- <code class="ph codeph">INVALIDATE METADATA</code> statement. <code class="ph codeph">INVALIDATE METADATA</code> is
- not required if you make the changes to privileges within Impala.
+ An ownership can be transferred to another user or role via the <code class="ph codeph">ALTER
+ DATABASE</code>, <code class="ph codeph">ALTER TABLE</code>, or <code class="ph codeph">ALTER VIEW</code> with the
+ <code class="ph codeph">SET OWNER</code> clause.
</p>
@@ -873,111 +1311,123 @@
</div>
- <div class="topic concept nested1" aria-labelledby="ariaid-title3" id="secure_startup">
-
- <h2 class="title topictitle2" id="ariaid-title3">Starting the impalad Daemon with Sentry Authorization Enabled</h2>
+ <div class="topic concept nested1" aria-labelledby="ariaid-title4" id="secure_startup">
+ <h2 class="title topictitle2" id="ariaid-title4">Starting Impala with Sentry Authorization Enabled</h2>
+
<div class="body conbody">
- <p class="p">
- To run the <span class="keyword cmdname">impalad</span> daemon with authorization enabled, you add one or
- more options to the <code class="ph codeph">IMPALA_SERVER_ARGS</code> declaration in the
- <span class="ph filepath">/etc/default/impala</span> configuration file:
- </p>
+ <div class="p">
+ To enable authorization in an Impala cluster using Sentry:
+ <ol class="ol">
+ <li class="li">
+ Add the following options to the <code class="ph codeph">IMPALA_SERVER_ARGS</code> and the
+ <code class="ph codeph">IMPALA_CATALOG_ARGS</code> settings in the
+ <span class="ph filepath">/etc/default/impala</span> configuration file:
+ <ul class="ul">
+ <li class="li">
+ <code class="ph codeph">-server_name</code>: For all <span class="keyword cmdname">impalad</span> nodes and the
+ <code class="ph codeph">catalogd</code> in the cluster, specify the same name set in the
+ <code class="ph codeph">sentry.hive.server</code> property in the
+ <span class="ph filepath">sentry-site.xml</span> configuration file for Hive.
+ </li>
- <ul class="ul">
- <li class="li">
- <code class="ph codeph">-server_name</code>: Turns on Sentry authorization for Impala. The
- authorization rules refer to a symbolic server name, and you specify the same name to
- use as the argument to the <code class="ph codeph">-server_name</code> option for all
- <span class="keyword cmdname">impalad</span> nodes in the cluster.
- <p class="p">
- Starting in Impala 1.4.0 and higher, if you specify just
- <code class="ph codeph">-server_name</code> without <code class="ph codeph">-authorization_policy_file</code>,
- Impala uses the Sentry service for authorization.
- </p>
+ <li class="li">
+ <code class="ph codeph">-sentry_config</code>: Specifies the local path to the
+ <code class="ph codeph">sentry-site.xml</code> configuration file.
+ </li>
- </li>
+ </ul>
+ </li>
- <li class="li">
- <code class="ph codeph">-sentry_config</code>: Specifies the local path to the
- <code class="ph codeph">sentry-site.xml</code> configuration file. This setting is required to
- enable authorization.
- </li>
+ <li class="li">
+ Restart the <code class="ph codeph">catalogd</code> and all <span class="keyword cmdname">impalad</span> daemons.
+ </li>
- <li class="li">
- <code class="ph codeph">-authorization_policy_file</code>: Specifies the HDFS path to the policy
- file that defines the privileges on schema objects. Prior to Impala 1.4.0, or if you
- want to continue storing privilege rules in the policy file, specify the
- <code class="ph codeph">-authorization_policy_file</code> option to make Impala read privilege
- information from a policy file, rather than from the metastore database.
- </li>
+ </ol>
- </ul>
+ </div>
- <p class="p">
- For example, you might adapt your <span class="ph filepath">/etc/default/impala</span> configuration
- to contain lines like the following. To use the Sentry service rather than the policy
- file:
- </p>
+ </div>
-<pre class="pre codeblock"><code>IMPALA_SERVER_ARGS=" \
--server_name=server1 \
-...
-</code></pre>
+ </div>
- <p class="p">
- Or to use the policy file, as in releases prior to Impala 1.4:
- </p>
+ <div class="topic concept nested1" aria-labelledby="ariaid-title5" id="enable_ranger_authz">
-<pre class="pre codeblock"><code>IMPALA_SERVER_ARGS=" \
--authorization_policy_file=/user/hive/warehouse/auth-policy.ini \
--server_name=server1 \
-...
-</code></pre>
+ <h2 class="title topictitle2" id="ariaid-title5">Starting Impala with Ranger Authorization Enabled</h2>
- <p class="p">
- The preceding examples set up a symbolic name of <code class="ph codeph">server1</code> to refer to
- the current instance of Impala. Specify the symbolic name for the
- <code class="ph codeph">sentry.hive.server</code> property in the <span class="ph filepath">sentry-site.xml</span>
- configuration file for Hive, as well as in the <code class="ph codeph">-server_name</code> option for
- <span class="keyword cmdname">impalad</span>.
- </p>
+ <div class="body conbody">
<p class="p">
- Now restart the <span class="keyword cmdname">impalad</span> daemons on all the nodes.
+ To enable authorization in an Impala cluster using Ranger:
</p>
+ <ol class="ol">
+ <li class="li">
+ Add the following options to the <code class="ph codeph">IMPALA_SERVER_ARGS</code> and the
+ <code class="ph codeph">IMPALA_CATALOG_ARGS</code> settings in the
+ <span class="ph filepath">/etc/default/impala</span> configuration file:
+ <ul class="ul">
+ <li class="li">
+ <code class="ph codeph">-server_name</code>: Specify the same name for all
+ <span class="keyword cmdname">impalad</span> nodes and the <code class="ph codeph">catalogd</code> in the cluster.
+ </li>
+
+
+ <li class="li">
+ <code class="ph codeph">-ranger_service_type=hive</code>
+ </li>
+
+
+ <li class="li">
+ <code class="ph codeph">-ranger_app_id</code>: Set it to the Ranger application id.
+ </li>
+
+
+ <li class="li">
+ <code class="ph codeph">-authorization_provider=ranger</code>
+ </li>
+
+ </ul>
+
+ </li>
+
+
+ <li class="li">
+ Restart the <code class="ph codeph">catalogd</code> and all <span class="keyword cmdname">impalad</span> daemons.
+ </li>
+
+ </ol>
+
+
</div>
</div>
- <div class="topic concept nested1" aria-labelledby="ariaid-title4" id="sentry_service">
+ <div class="topic concept nested1" aria-labelledby="ariaid-title6" id="sentry_service">
- <h2 class="title topictitle2" id="ariaid-title4">Using Impala with the Sentry Service</h2>
+ <h2 class="title topictitle2" id="ariaid-title6">Managing Privileges</h2>
<div class="body conbody">
<p class="p">
- When you use the Sentry service, you set up privileges through the
- <code class="ph codeph">GRANT</code> and <code class="ph codeph">REVOKE</code> statements in either Impala or Hive.
- Then both components use those same privileges automatically. (Impala added the
- <code class="ph codeph">GRANT</code> and <code class="ph codeph">REVOKE</code> statements in
- <span class="keyword">Impala 2.0</span>.)
+ You set up privileges through the <code class="ph codeph">GRANT</code> and <code class="ph codeph">REVOKE</code>
+ statements in either Impala or Hive. Then both components use those same privileges
+ automatically.
</p>
@@ -991,17 +1441,17 @@
</div>
- <div class="topic concept nested2" aria-labelledby="ariaid-title5" id="changing_privileges">
+ <div class="topic concept nested2" aria-labelledby="ariaid-title7" id="changing_privileges">
- <h3 class="title topictitle3" id="ariaid-title5">Changing Privileges</h3>
+ <h3 class="title topictitle3" id="ariaid-title7">Changing Privileges from Outside of Impala</h3>
<div class="body conbody">
<p class="p">
- If you make a change to privileges in Sentry from outside of Impala, e.g. adding a
- user, removing a user, modifying privileges, there are two options to propagate the
- change:
+ If you make a change to privileges in Sentry or Ranger from outside of Impala, e.g.
+ adding a user, removing a user, modifying privileges, there are two options to
+ propagate the change:
</p>
@@ -1014,9 +1464,16 @@
<li class="li">
- Run the <code class="ph codeph">INVALIDATE METADATA</code> statement to force a Sentry refresh.
- <code class="ph codeph">INVALIDATE METADATA</code> forces a Sentry refresh regardless of the
- <code class="ph codeph">--sentry_catalog_polling_fequency_s</code> flag.
+ Use the <code class="ph codeph">ranger.plugin.hive.policy.pollIntervalMs</code> property to
+ specify how often to do a Ranger refresh. The property is specified in
+ <code class="ph codeph">ranger-hive-security.xml</code> in the <code class="ph codeph">conf</code> directory
+ under your Impala home directory.
+ </li>
+
+
+ <li class="li">
+ Run the <code class="ph codeph">INVALIDATE METADATA</code> or <code class="ph codeph">REFRESH
+ AUTHORIZATION</code> statement to force a refresh.
</li>
</ul>
@@ -1028,7 +1485,7 @@
</p>
- <div class="note warning"><span class="warningtitle">Warning:</span>
+ <div class="note warning"><span class="warningtitle">Warning:</span>
As <code class="ph codeph">INVALIDATE METADATA</code> is an expensive operation, you should use it
judiciously.
</div>
@@ -1040,9 +1497,9 @@
</div>
- <div class="topic concept nested2" aria-labelledby="ariaid-title6" id="granting_on_uri">
+ <div class="topic concept nested2" aria-labelledby="ariaid-title8" id="granting_on_uri">
- <h3 class="title topictitle3" id="ariaid-title6">Granting Privileges on URI</h3>
+ <h3 class="title topictitle3" id="ariaid-title8">Granting Privileges on URI</h3>
<div class="body conbody">
@@ -1077,7 +1534,7 @@ GRANT ALL ON URI 'hdfs://localhost:20500/tmp';
When defining URIs for HDFS, you must also specify the NameNode. For example:
<pre class="pre codeblock"><code>GRANT ALL ON URI file:///path/to/dir TO <role>
GRANT ALL ON URI hdfs://namenode:port/path/to/dir TO <role></code></pre>
- <div class="note warning"><span class="warningtitle">Warning:</span>
+ <div class="note warning"><span class="warningtitle">Warning:</span>
Because the NameNode host and port must be specified, it is strongly recommended
that you use High Availability (HA). This ensures that the URI will remain constant
even if the NameNode changes. For example:
@@ -1093,9 +1550,9 @@ GRANT ALL ON URI hdfs://namenode:port/path/to/dir TO <role></code></pre>
</div>
- <div class="topic concept nested2" aria-labelledby="ariaid-title7" id="concept_k45_lbm_f2b">
+ <div class="topic concept nested2" aria-labelledby="ariaid-title9" id="concept_k45_lbm_f2b">
- <h3 class="title topictitle3" id="ariaid-title7">Examples of Setting up Authorization for Security Scenarios</h3>
+ <h3 class="title topictitle3" id="ariaid-title9">Examples of Setting up Authorization for Security Scenarios</h3>
<div class="body conbody">
@@ -1108,7 +1565,7 @@ GRANT ALL ON URI hdfs://namenode:port/path/to/dir TO <role></code></pre>
<div class="example"><h4 class="title sectiontitle">A User with No Privileges</h4>
-
+
<p class="p">
If a user has no privileges at all, that user cannot access any schema objects in
@@ -1129,7 +1586,7 @@ GRANT ALL ON URI hdfs://namenode:port/path/to/dir TO <role></code></pre>
<div class="example"><h4 class="title sectiontitle">Examples of Privileges for Administrative Users</h4>
-
+
<p class="p">
In this example, the SQL statements grant the <code class="ph codeph">entire_server</code> role
@@ -1147,7 +1604,7 @@ GRANT ALL ON SERVER server1 TO ROLE entire_server;
<div class="example"><h4 class="title sectiontitle">A User with Privileges for Specific Databases and Tables</h4>
-
+
<p class="p">
If a user has privileges for specific tables in specific databases, the user can
@@ -1181,10 +1638,10 @@ GRANT SELECT ON TABLE db1.training TO ROLE student;</code></pre>
<div class="example"><h4 class="title sectiontitle">Privileges for Working with External Data Files</h4>
-
+
<p class="p">
- When data is being inserted through the <code class="ph codeph">LOAD DATA</code> statement, or is
+ When data is being inserted through the <code class="ph codeph">LOAD DATA</code> statement or is
referenced from an HDFS location outside the normal Impala database directories, the
user also needs appropriate permissions on the URIs corresponding to those HDFS
locations.
@@ -1228,14 +1685,14 @@ GRANT ALL ON URI 'hdfs://127.0.0.1:8020/user/impala-user/external_data' TO ROLE
<div class="example"><h4 class="title sectiontitle">Separating Administrator Responsibility from Read and Write Privileges</h4>
-
+
<p class="p">
To create a database, you need the full privilege on that database while day-to-day
operations on tables within that database can be performed with lower levels of
- privilege on specific table. Thus, you might set up separate roles for each database
- or application: an administrative one that could create or drop the database, and a
- user-level one that can access only the relevant tables.
+ privilege on a specific table. Thus, you might set up separate roles for each
+ database or application: an administrative one that could create or drop the
+ database, and a user-level one that can access only the relevant tables.
</p>
@@ -1293,228 +1750,9 @@ GRANT SELECT ON TABLE training1.course1 TO ROLE student;</code></pre>
</div>
- <div class="topic concept nested1" aria-labelledby="ariaid-title8" id="security_policy_file">
-
- <h2 class="title topictitle2" id="ariaid-title8">Using Impala with the Sentry Policy File</h2>
-
-
- <div class="body conbody">
-
- <p class="p">
- The policy file is a file that you put in a designated location in HDFS, and is read
- during the startup of the <span class="keyword cmdname">impalad</span> daemon when you specify both the
- <code class="ph codeph">-server_name</code> and <code class="ph codeph">-authorization_policy_file</code> startup
- options. It controls which objects (databases, tables, and HDFS directory paths) can be
- accessed by the user who connects to <span class="keyword cmdname">impalad</span>, and what operations
- that user can perform on the objects.
- </p>
-
-
- <div class="note note"><span class="notetitle">Note:</span>
- The policy-file based authorization was deprecated in <span class="keyword">Impala 2.6</span>. We
- recommend managing privileges through SQL statements as described in
- <a class="xref" href="impala_authorization.html#sentry_service">Using Impala with the Sentry Service</a>. If you are still using
- policy files, plan to migrate to the new approach some time in the future.
- </div>
-
-
- <p class="p">
- The location of the policy file is listed in the <span class="ph filepath">auth-site.xml</span>
- configuration file.
- </p>
-
-
- <p class="p">
- When authorization is enabled, Impala uses the policy file as a <em class="ph i">whitelist</em>,
- representing every privilege available to any user on any object. That is, only
- operations specified for the appropriate combination of object, role, group, and user
- are allowed. All other operations are not allowed. If a group or role is defined
- multiple times in the policy file, the last definition takes precedence.
- </p>
-
-
- <p class="p">
- To understand the notion of whitelisting, set up a minimal policy file that does not
- provide any privileges for any object. When you connect to an Impala node where this
- policy file is in effect, you get no results for <code class="ph codeph">SHOW DATABASES</code>, and an
- error when you issue any <code class="ph codeph">SHOW TABLES</code>, <code class="ph codeph">USE
- <var class="keyword varname">database_name</var></code>, <code class="ph codeph">DESCRIBE
- <var class="keyword varname">table_name</var></code>, <code class="ph codeph">SELECT</code>, and or other statements
- that expect to access databases or tables, even if the corresponding databases and
- tables exist.
- </p>
-
-
- <p class="p">
- The contents of the policy file are cached, to avoid a performance penalty for each
- query. The policy file is re-checked by each <span class="keyword cmdname">impalad</span> node every 5
- minutes. When you make a non-time-sensitive change such as adding new privileges or new
- users, you can let the change take effect automatically a few minutes later. If you
- remove or reduce privileges, and want the change to take effect immediately, restart the
- <span class="keyword cmdname">impalad</span> daemon on all nodes, again specifying the
- <code class="ph codeph">-server_name</code> and <code class="ph codeph">-authorization_policy_file</code> options so
- that the rules from the updated policy file are applied.
- </p>
-
-
- </div>
-
-
- <div class="topic concept nested2" aria-labelledby="ariaid-title9" id="security_policy_file_details">
-
- <h3 class="title topictitle3" id="ariaid-title9">Policy File Format</h3>
-
-
- <div class="body conbody">
-
- <p class="p">
- The policy file uses the familiar <code class="ph codeph">.ini</code> format, divided into the major
- sections <code class="ph codeph">[groups]</code> and <code class="ph codeph">[roles]</code>.
- </p>
-
-
- <p class="p">
- There is also an optional <code class="ph codeph">[databases]</code> section, which allows you to
- specify a specific policy file for a particular database, as explained in
- <a class="xref" href="#security_multiple_policy_files">Using Multiple Policy Files for Different Databases</a>.
- </p>
-
-
- <p class="p">
- Another optional section, <code class="ph codeph">[users]</code>, allows you to override the
- OS-level mapping of users to groups; that is an advanced technique primarily for
- testing and debugging, and is beyond the scope of this document.
- </p>
-
-
- <p class="p">
- In the <code class="ph codeph">[groups]</code> section, you define various categories of users and
- select which roles are associated with each category. The group and usernames
- correspond to Linux groups and users on the server where the
- <span class="keyword cmdname">impalad</span> daemon runs.
- </p>
-
-
- <p class="p">
- The group and usernames in the <code class="ph codeph">[groups]</code> section correspond to Hadoop
- groups and users on the server where the <span class="keyword cmdname">impalad</span> daemon runs. When
- you access Impala through the <span class="keyword cmdname">impalad</span> interpreter, for purposes of
- authorization, the user is the logged-in Linux user and the groups are the Linux
- groups that user is a member of. When you access Impala through the ODBC or JDBC
- interfaces, the user and password specified through the connection string are used as
- login credentials for the Linux server, and authorization is based on that username
- and the associated Linux group membership.
- </p>
-
-
- <div class="p">
- In the <code class="ph codeph">[roles]</code> section, you a set of roles. For each role, you
- specify precisely the set of privileges is available. That is, which objects users
- with that role can access, and what operations they can perform on those objects. This
- is the lowest-level category of security information; the other sections in the policy
- file map the privileges to higher-level divisions of groups and users. In the
- <code class="ph codeph">[groups]</code> section, you specify which roles are associated with which
- groups. The group and usernames correspond to Linux groups and users on the server
- where the <span class="keyword cmdname">impalad</span> daemon runs. The privileges are specified using
- patterns like:
-<pre class="pre codeblock"><code>server=<var class="keyword varname">server_name</var>->db=<var class="keyword varname">database_name</var>->table=<var class="keyword varname">table_name</var>->action=SELECT
-server=<var class="keyword varname">server_name</var>->db=<var class="keyword varname">database_name</var>->table=t<var class="keyword varname">able_name</var>->action=CREATE
-server=<var class="keyword varname">server_name</var>->db=<var class="keyword varname">database_name</var>->table=<var class="keyword varname">table_name</var>->action=ALL
-</code></pre>
- For the <var class="keyword varname">server_name</var> value, substitute the same symbolic name you
- specify with the <span class="keyword cmdname">impalad</span> <code class="ph codeph">-server_name</code> option. You
- can use <code class="ph codeph">*</code> wildcard characters at each level of the privilege
- specification to allow access to all such objects. For example:
-<pre class="pre codeblock"><code>server=impala-host.example.com->db=default->table=t1->action=SELECT
-server=impala-host.example.com->db=*->table=*->action=CREATE
-server=impala-host.example.com->db=*->table=audit_log->action=SELECT
-server=impala-host.example.com->db=default->table=t1->action=*
-</code></pre>
- </div>
-
-
- </div>
-
-
- </div>
-
-
- <div class="topic concept nested2" aria-labelledby="ariaid-title10" id="security_multiple_policy_files">
-
- <h3 class="title topictitle3" id="ariaid-title10">Using Multiple Policy Files for Different Databases</h3>
-
-
- <div class="body conbody">
-
- <p class="p">
- For an Impala cluster with many databases being accessed by many users and
- applications, it might be cumbersome to update the security policy file for each
- privilege change or each new database, table, or view. You can allow security to be
- managed separately for individual databases, by setting up a separate policy file for
- each database:
- </p>
-
-
- <ul class="ul">
- <li class="li">
- Add the optional <code class="ph codeph">[databases]</code> section to the main policy file.
- </li>
-
-
- <li class="li">
- Add entries in the <code class="ph codeph">[databases]</code> section for each database that has
- its own policy file.
- </li>
-
-
- <li class="li">
- For each listed database, specify the HDFS path of the appropriate policy file.
- </li>
-
- </ul>
-
-
- <p class="p">
- For example:
- </p>
-
-
-<pre class="pre codeblock"><code>[databases]
-# Defines the location of the per-DB policy files for the 'customers' and 'sales' databases.
-customers = hdfs://ha-nn-uri/etc/access/customers.ini
-sales = hdfs://ha-nn-uri/etc/access/sales.ini
-</code></pre>
-
- <p class="p">
- To enable URIs in per-DB policy files, the Java configuration option
- <code class="ph codeph">sentry.allow.uri.db.policyfile</code> must be set to <code class="ph codeph">true</code>.
- For example:
- </p>
-
-
-<pre class="pre codeblock"><code>JAVA_TOOL_OPTIONS="-Dsentry.allow.uri.db.policyfile=true"
-</code></pre>
-
- <div class="note important"><span class="importanttitle">Important:</span>
- Enabling URIs in per-DB policy files introduces a security risk by allowing the owner
- of the db-level policy file to grant himself/herself load privileges to anything the
- <code class="ph codeph">impala</code> user has read permissions for in HDFS (including data in other
- databases controlled by different db-level policy files).
- </div>
-
-
- </div>
-
-
- </div>
-
-
- </div>
-
-
- <div class="topic concept nested1" aria-labelledby="ariaid-title11" id="security_schema">
+ <div class="topic concept nested1" aria-labelledby="ariaid-title10" id="security_schema">
- <h2 class="title topictitle2" id="ariaid-title11">Setting Up Schema Objects for a Secure Impala Deployment</h2>
+ <h2 class="title topictitle2" id="ariaid-title10">Setting Up Schema Objects for a Secure Impala Deployment</h2>
<div class="body conbody">
@@ -1523,7 +1761,7 @@ sales = hdfs://ha-nn-uri/etc/access/sales.ini
In your role definitions, you must specify privileges at the level of individual
databases and tables, or all databases or all tables within a database. To simplify the
structure of these rules, plan ahead of time how to name your schema objects so that
- data with different authorization requirements is divided into separate databases.
+ data with different authorization requirements are divided into separate databases.
</p>
@@ -1540,43 +1778,9 @@ sales = hdfs://ha-nn-uri/etc/access/sales.ini
</div>
- <div class="topic concept nested1" aria-labelledby="ariaid-title12" id="sentry_debug">
-
- <h2 class="title topictitle2" id="ariaid-title12"><span class="ph">Debugging Failed Sentry Authorization Requests</span></h2>
-
-
- <div class="body conbody">
-
- <div class="p"> Sentry logs all facts that lead up to authorization
- decisions at the debug level. If you do not understand why Sentry is
- denying access, the best way to debug is to temporarily turn on debug
- logging: <ul class="ul">
- <li class="li"> Add <code class="ph codeph">log4j.logger.org.apache.sentry=DEBUG</code> to the
- <span class="ph filepath">log4j.properties</span> file on each host in the
- cluster, in the appropriate configuration directory for each
- service. </li>
-
- </ul>
- Specifically, look for exceptions and messages such as:
- <pre class="pre codeblock"><code>FilePermission server..., RequestPermission server...., result [true|false]</code></pre>
- which indicate each evaluation Sentry makes. The
- <code class="ph codeph">FilePermission</code> is from the policy file, while
- <code class="ph codeph">RequestPermission</code> is the privilege required for the
- query. A <code class="ph codeph">RequestPermission</code> will iterate over all
- appropriate <code class="ph codeph">FilePermission</code> settings until a match is
- found. If no matching privilege is found, Sentry returns
- <code class="ph codeph">false</code> indicating <span class="q">"Access Denied"</span>.</div>
-
-
- </div>
-
-
- </div>
-
-
- <div class="topic concept nested1" aria-labelledby="ariaid-title13" id="sec_ex_default">
+ <div class="topic concept nested1" aria-labelledby="ariaid-title11" id="sec_ex_default">
- <h2 class="title topictitle2" id="ariaid-title13">The DEFAULT Database in a Secure Deployment</h2>
+ <h2 class="title topictitle2" id="ariaid-title11">The DEFAULT Database in a Secure Deployment</h2>
<div class="body conbody">
@@ -1584,10 +1788,10 @@ sales = hdfs://ha-nn-uri/etc/access/sales.ini
<p class="p">
Because of the extra emphasis on granular access controls in a secure deployment, you
should move any important or sensitive information out of the <code class="ph codeph">DEFAULT</code>
- database into a named database whose privileges are specified in the policy file.
- Sometimes you might need to give privileges on the <code class="ph codeph">DEFAULT</code> database for
- administrative reasons; for example, as a place you can reliably specify with a
- <code class="ph codeph">USE</code> statement when preparing to drop a database.
+ database into a named database. Sometimes you might need to give privileges on the
+ <code class="ph codeph">DEFAULT</code> database for administrative reasons, for example, as a place
+ you can reliably specify with a <code class="ph codeph">USE</code> statement when preparing to drop a
+ database.
</p>
@@ -1598,4 +1802,4 @@ sales = hdfs://ha-nn-uri/etc/access/sales.ini
</body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_avg.html b/docs/build/html/topics/impala_avg.html
index 5185bc2..80778a3 100644
--- a/docs/build/html/topics/impala_avg.html
+++ b/docs/build/html/topics/impala_avg.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_aggregate_functions.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="avg" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -64,25 +64,25 @@
<p class="p">
- To access a column with a complex type (<code class="ph codeph">ARRAY</code>, <code class="ph codeph">STRUCT</code>, or <code class="ph codeph">MAP</code>)
- in an aggregation function, you unpack the individual elements using join notation in the query,
- and then apply the function to the final scalar item, field, key, or value at the bottom of any nested type hierarchy in the column.
- See <a class="xref" href="../shared/../topics/impala_complex_types.html#complex_types">Complex Types (Impala 2.3 or higher only)</a> for details about using complex types in Impala.
+ To access a column with a complex type (<code class="ph codeph">ARRAY</code>, <code class="ph codeph">STRUCT</code>,
+ or <code class="ph codeph">MAP</code>) in an aggregation function, you unpack the individual elements
+ using join notation in the query, and then apply the function to the final scalar item,
+ field, key, or value at the bottom of any nested type hierarchy in the column. See
+ <a class="xref" href="../shared/../topics/impala_complex_types.html#complex_types">Complex Types (Impala 2.3 or higher only)</a> for details about using
+ complex types in Impala.
</p>
<div class="p">
-The following example demonstrates calls to several aggregation functions
-using values from a column containing nested complex types
-(an <code class="ph codeph">ARRAY</code> of <code class="ph codeph">STRUCT</code> items).
-The array is unpacked inside the query using join notation.
-The array elements are referenced using the <code class="ph codeph">ITEM</code>
-pseudocolumn, and the structure fields inside the array elements
-are referenced using dot notation.
-Numeric values such as <code class="ph codeph">SUM()</code> and <code class="ph codeph">AVG()</code>
-are computed using the numeric <code class="ph codeph">R_NATIONKEY</code> field, and
-the general-purpose <code class="ph codeph">MAX()</code> and <code class="ph codeph">MIN()</code>
-values are computed from the string <code class="ph codeph">N_NAME</code> field.
+ The following example demonstrates calls to several aggregation functions using values
+ from a column containing nested complex types (an <code class="ph codeph">ARRAY</code> of
+ <code class="ph codeph">STRUCT</code> items). The array is unpacked inside the query using join
+ notation. The array elements are referenced using the <code class="ph codeph">ITEM</code>
+ pseudocolumn, and the structure fields inside the array elements are referenced using
+ dot notation. Numeric values such as <code class="ph codeph">SUM()</code> and <code class="ph codeph">AVG()</code>
+ are computed using the numeric <code class="ph codeph">R_NATIONKEY</code> field, and the
+ general-purpose <code class="ph codeph">MAX()</code> and <code class="ph codeph">MIN()</code> values are computed
+ from the string <code class="ph codeph">N_NAME</code> field.
<pre class="pre codeblock"><code>describe region;
+-------------+-------------------------+---------+
| name | type | comment |
@@ -152,7 +152,7 @@ order by r_name;
| MIDDLE EAST | 5 | 58 | 11.6 | EGYPT | SAUDI ARABIA | 5 |
+-------------+-------+-----+------+-----------+----------------+---------------+
</code></pre>
-</div>
+ </div>
<p class="p">
@@ -327,13 +327,14 @@ ERROR: AnalysisException: RANGE is only supported with both the lower and upper
<p class="p">
- Due to the way arithmetic on <code class="ph codeph">FLOAT</code> and <code class="ph codeph">DOUBLE</code> columns uses
- high-performance hardware instructions, and distributed queries can perform these operations in different
- order for each query, results can vary slightly for aggregate function calls such as <code class="ph codeph">SUM()</code>
- and <code class="ph codeph">AVG()</code> for <code class="ph codeph">FLOAT</code> and <code class="ph codeph">DOUBLE</code> columns, particularly on
- large data sets where millions or billions of values are summed or averaged. For perfect consistency and
- repeatability, use the <code class="ph codeph">DECIMAL</code> data type for such operations instead of
- <code class="ph codeph">FLOAT</code> or <code class="ph codeph">DOUBLE</code>.
+ Due to the way arithmetic on <code class="ph codeph">FLOAT</code> and <code class="ph codeph">DOUBLE</code> columns
+ uses high-performance hardware instructions, and distributed queries can perform these
+ operations in different order for each query, results can vary slightly for aggregate
+ function calls such as <code class="ph codeph">SUM()</code> and <code class="ph codeph">AVG()</code> for
+ <code class="ph codeph">FLOAT</code> and <code class="ph codeph">DOUBLE</code> columns, particularly on large data
+ sets where millions or billions of values are summed or averaged. For perfect
+ consistency and repeatability, use the <code class="ph codeph">DECIMAL</code> data type for such
+ operations instead of <code class="ph codeph">FLOAT</code> or <code class="ph codeph">DOUBLE</code>.
</p>
diff --git a/docs/build/html/topics/impala_avro.html b/docs/build/html/topics/impala_avro.html
index ef7bb67..386e862 100644
--- a/docs/build/html/topics/impala_avro.html
+++ b/docs/build/html/topics/impala_avro.html
@@ -20,17 +20,17 @@
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="avro" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -52,26 +52,26 @@
insert operations, use Hive, then switch back to Impala to run queries. </p>
-
+
<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" class="table" frame="border" border="1" rules="all"><caption><span class="tablecap"><span class="table--title-label">Table 1. </span>Avro Format Support in Impala</span></caption><colgroup><col style="width:10%" /><col style="width:10%" /><col style="width:20%" /><col style="width:30%" /><col style="width:30%" /></colgroup><thead class="thead" style="text-align:left;">
<tr class="row">
- <th class="entry nocellnorowborder" style="vertical-align:top;" id="d25123e170">
+ <th class="entry nocellnorowborder" style="vertical-align:top;" id="d24841e170">
File Type
</th>
- <th class="entry nocellnorowborder" style="vertical-align:top;" id="d25123e173">
+ <th class="entry nocellnorowborder" style="vertical-align:top;" id="d24841e173">
Format
</th>
- <th class="entry nocellnorowborder" style="vertical-align:top;" id="d25123e176">
+ <th class="entry nocellnorowborder" style="vertical-align:top;" id="d24841e176">
Compression Codecs
</th>
- <th class="entry nocellnorowborder" style="vertical-align:top;" id="d25123e179">
+ <th class="entry nocellnorowborder" style="vertical-align:top;" id="d24841e179">
Impala Can CREATE?
</th>
- <th class="entry cell-norowborder" style="vertical-align:top;" id="d25123e182">
+ <th class="entry cell-norowborder" style="vertical-align:top;" id="d24841e182">
Impala Can INSERT?
</th>
@@ -80,23 +80,23 @@
</thead>
<tbody class="tbody">
<tr class="row">
- <td class="entry row-nocellborder" style="vertical-align:top;" headers="d25123e170 ">
+ <td class="entry row-nocellborder" style="vertical-align:top;" headers="d24841e170 ">
<a class="xref" href="impala_avro.html#avro">Avro</a>
</td>
- <td class="entry row-nocellborder" style="vertical-align:top;" headers="d25123e173 ">
+ <td class="entry row-nocellborder" style="vertical-align:top;" headers="d24841e173 ">
Structured
</td>
- <td class="entry row-nocellborder" style="vertical-align:top;" headers="d25123e176 ">
+ <td class="entry row-nocellborder" style="vertical-align:top;" headers="d24841e176 ">
Snappy, gzip, deflate
</td>
- <td class="entry row-nocellborder" style="vertical-align:top;" headers="d25123e179 ">
+ <td class="entry row-nocellborder" style="vertical-align:top;" headers="d24841e179 ">
Yes, in Impala 1.4.0 and higher. In lower versions, create the table using Hive.
</td>
- <td class="entry cellrowborder" style="vertical-align:top;" headers="d25123e182 ">
+ <td class="entry cellrowborder" style="vertical-align:top;" headers="d24841e182 ">
No. Import data by using <code class="ph codeph">LOAD DATA</code> on data files already in the
right format, or use <code class="ph codeph">INSERT</code> in Hive followed by <code class="ph codeph">REFRESH
<var class="keyword varname">table_name</var></code> in Impala.
@@ -145,13 +145,14 @@
</p>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
<p class="p">
- Currently, Avro tables cannot contain <code class="ph codeph">TIMESTAMP</code> columns. If you need to store date and
- time values in Avro tables, as a workaround you can use a <code class="ph codeph">STRING</code> representation of the
- values, convert the values to <code class="ph codeph">BIGINT</code> with the <code class="ph codeph">UNIX_TIMESTAMP()</code> function,
- or create separate numeric columns for individual date and time fields using the <code class="ph codeph">EXTRACT()</code>
- function.
+ Currently, Avro tables cannot contain <code class="ph codeph">TIMESTAMP</code> columns. If you need to
+ store date and time values in Avro tables, as a workaround you can use a
+ <code class="ph codeph">STRING</code> representation of the values, convert the values to
+ <code class="ph codeph">BIGINT</code> with the <code class="ph codeph">UNIX_TIMESTAMP()</code> function, or create
+ separate numeric columns for individual date and time fields using the
+ <code class="ph codeph">EXTRACT()</code> function.
</p>
</div>
@@ -243,7 +244,7 @@ hive> CREATE TABLE hive_avro_table
</p>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
For nullable Avro columns, make sure to put the <code class="ph codeph">"null"</code> entry before the actual type name.
In Impala, all columns are nullable; Impala currently does not have a <code class="ph codeph">NOT NULL</code> clause. Any
non-nullable property is only enforced on the Avro side.
@@ -313,15 +314,13 @@ hive> CREATE TABLE hive_avro_table
<p class="p">
- <strong class="ph b">Complex type considerations:</strong>
- Although you can create tables in this file format using
- the complex types (<code class="ph codeph">ARRAY</code>, <code class="ph codeph">STRUCT</code>,
- and <code class="ph codeph">MAP</code>) available in <span class="keyword">Impala 2.3</span> and higher,
- currently, Impala can query these types only in Parquet tables.
- <span class="ph">
- The one exception to the preceding rule is <code class="ph codeph">COUNT(*)</code> queries on RCFile tables that include complex types.
- Such queries are allowed in <span class="keyword">Impala 2.6</span> and higher.
- </span>
+ <strong class="ph b">Complex type considerations:</strong> Although you can create tables in this file format
+ using the complex types (<code class="ph codeph">ARRAY</code>, <code class="ph codeph">STRUCT</code>, and
+ <code class="ph codeph">MAP</code>) available in <span class="keyword">Impala 2.3</span> and higher,
+ currently, Impala can query these types only in Parquet tables. <span class="ph">
+ The one exception to the preceding rule is <code class="ph codeph">COUNT(*)</code> queries on RCFile
+ tables that include complex types. Such queries are allowed in
+ <span class="keyword">Impala 2.6</span> and higher. </span>
</p>
@@ -638,13 +637,13 @@ TIMESTAMP
</code></pre>
<p class="p">
- The Avro specification allows string values up to 2**64 bytes in length.
- Impala queries for Avro tables use 32-bit integers to hold string lengths.
- In <span class="keyword">Impala 2.5</span> and higher, Impala truncates <code class="ph codeph">CHAR</code>
- and <code class="ph codeph">VARCHAR</code> values in Avro tables to (2**31)-1 bytes.
- If a query encounters a <code class="ph codeph">STRING</code> value longer than (2**31)-1
- bytes in an Avro table, the query fails. In earlier releases,
- encountering such long values in an Avro table could cause a crash.
+ The Avro specification allows string values up to 2**64 bytes in length. Impala queries
+ for Avro tables use 32-bit integers to hold string lengths. In
+ <span class="keyword">Impala 2.5</span> and higher, Impala truncates <code class="ph codeph">CHAR</code> and
+ <code class="ph codeph">VARCHAR</code> values in Avro tables to (2**31)-1 bytes. If a query encounters
+ a <code class="ph codeph">STRING</code> value longer than (2**31)-1 bytes in an Avro table, the query
+ fails. In earlier releases, encountering such long values in an Avro table could cause a
+ crash.
</p>
@@ -670,18 +669,18 @@ TIMESTAMP
<p class="p">
- In <span class="keyword">Impala 2.6</span> and higher, Impala queries are optimized for files stored in Amazon S3.
- For Impala tables that use the file formats Parquet, ORC, RCFile, SequenceFile,
- Avro, and uncompressed text, the setting <code class="ph codeph">fs.s3a.block.size</code>
- in the <span class="ph filepath">core-site.xml</span> configuration file determines
- how Impala divides the I/O work of reading the data files. This configuration
- setting is specified in bytes. By default, this
- value is 33554432 (32 MB), meaning that Impala parallelizes S3 read operations on the files
- as if they were made up of 32 MB blocks. For example, if your S3 queries primarily access
- Parquet files written by MapReduce or Hive, increase <code class="ph codeph">fs.s3a.block.size</code>
- to 134217728 (128 MB) to match the row group size of those files. If most S3 queries involve
- Parquet files written by Impala, increase <code class="ph codeph">fs.s3a.block.size</code>
- to 268435456 (256 MB) to match the row group size produced by Impala.
+ In <span class="keyword">Impala 2.6</span> and higher, Impala queries are optimized for files
+ stored in Amazon S3. For Impala tables that use the file formats Parquet, ORC, RCFile,
+ SequenceFile, Avro, and uncompressed text, the setting
+ <code class="ph codeph">fs.s3a.block.size</code> in the <span class="ph filepath">core-site.xml</span>
+ configuration file determines how Impala divides the I/O work of reading the data files.
+ This configuration setting is specified in bytes. By default, this value is 33554432 (32
+ MB), meaning that Impala parallelizes S3 read operations on the files as if they were
+ made up of 32 MB blocks. For example, if your S3 queries primarily access Parquet files
+ written by MapReduce or Hive, increase <code class="ph codeph">fs.s3a.block.size</code> to 134217728
+ (128 MB) to match the row group size of those files. If most S3 queries involve Parquet
+ files written by Impala, increase <code class="ph codeph">fs.s3a.block.size</code> to 268435456 (256
+ MB) to match the row group size produced by Impala.
</p>
@@ -691,4 +690,4 @@ TIMESTAMP
</body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_batch_size.html b/docs/build/html/topics/impala_batch_size.html
index a9358ba..fbbd290 100644
--- a/docs/build/html/topics/impala_batch_size.html
+++ b/docs/build/html/topics/impala_batch_size.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="batch_size" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
diff --git a/docs/build/html/topics/impala_bigint.html b/docs/build/html/topics/impala_bigint.html
index 5c48869..3c66710 100644
--- a/docs/build/html/topics/impala_bigint.html
+++ b/docs/build/html/topics/impala_bigint.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_datatypes.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="bigint" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -56,13 +56,13 @@
(<code class="ph codeph">FLOAT</code> or <code class="ph codeph">DOUBLE</code>) automatically. Use
<code class="ph codeph">CAST()</code> to convert to <code class="ph codeph">TINYINT</code>,
<code class="ph codeph">SMALLINT</code>, <code class="ph codeph">INT</code>,
- <code class="ph codeph">STRING</code>, or <code class="ph codeph">TIMESTAMP</code>. <span class="ph">
- Casting an integer or floating-point value <code class="ph codeph">N</code> to
- <code class="ph codeph">TIMESTAMP</code> produces a value that is <code class="ph codeph">N</code> seconds past the start of the epoch
- date (January 1, 1970). By default, the result value represents a date and time in the UTC time zone.
- If the setting <code class="ph codeph">--use_local_tz_for_unix_timestamp_conversions=true</code> is in effect,
- the resulting <code class="ph codeph">TIMESTAMP</code> represents a date and time in the local time zone.
- </span>
+ <code class="ph codeph">STRING</code>, or <code class="ph codeph">TIMESTAMP</code>. <span class="ph"> Casting an integer or floating-point value
+ <code class="ph codeph">N</code> to <code class="ph codeph">TIMESTAMP</code> produces a value that is
+ <code class="ph codeph">N</code> seconds past the start of the epoch date (January 1, 1970). By
+ default, the result value represents a date and time in the UTC time zone. If the
+ setting <code class="ph codeph">‑‑use_local_tz_for_unix_timestamp_conversions=true</code>
+ is in effect, the resulting <code class="ph codeph">TIMESTAMP</code> represents a date and time in the
+ local time zone. </span>
</p>
@@ -103,14 +103,15 @@ SELECT CAST(1000 AS BIGINT);
<p class="p">
- <strong class="ph b">NULL considerations:</strong> Casting any non-numeric value to this type produces a <code class="ph codeph">NULL</code>
- value.
+ <strong class="ph b">NULL considerations:</strong> Casting any non-numeric value to this type produces a
+ <code class="ph codeph">NULL</code> value.
</p>
<p class="p">
- <strong class="ph b">Partitioning:</strong> Prefer to use this type for a partition key column. Impala can process the numeric
- type more efficiently than a <code class="ph codeph">STRING</code> representation of the value.
+ <strong class="ph b">Partitioning:</strong> Prefer to use this type for a partition key column. Impala can
+ process the numeric type more efficiently than a <code class="ph codeph">STRING</code> representation
+ of the value.
</p>
@@ -122,8 +123,8 @@ SELECT CAST(1000 AS BIGINT);
<p class="p">
- <strong class="ph b">Text table considerations:</strong> Values of this type are potentially larger in text tables than in tables
- using Parquet or other binary formats.
+ <strong class="ph b">Text table considerations:</strong> Values of this type are potentially larger in text
+ tables than in tables using Parquet or other binary formats.
</p>
@@ -140,15 +141,15 @@ SELECT CAST(1000 AS BIGINT);
<p class="p">
- <strong class="ph b">Column statistics considerations:</strong> Because this type has a fixed size, the maximum and average size
- fields are always filled in for column statistics, even before you run the <code class="ph codeph">COMPUTE STATS</code>
- statement.
+ <strong class="ph b">Column statistics considerations:</strong> Because this type has a fixed size, the maximum
+ and average size fields are always filled in for column statistics, even before you run
+ the <code class="ph codeph">COMPUTE STATS</code> statement.
</p>
+
-
-
+
<p class="p">
<strong class="ph b">Related information:</strong>
@@ -169,4 +170,4 @@ SELECT CAST(1000 AS BIGINT);
<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_datatypes.html">Data Types</a></div>
</div>
</div></body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_bit_functions.html b/docs/build/html/topics/impala_bit_functions.html
index 5d4617b..4688ad9 100644
--- a/docs/build/html/topics/impala_bit_functions.html
+++ b/docs/build/html/topics/impala_bit_functions.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_functions.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="bit_functions" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -1049,7 +1049,7 @@ select shiftright(-1,5); /* 11111111 -> 00000111 */
</dd>
-
+
</dl>
@@ -1061,4 +1061,4 @@ select shiftright(-1,5); /* 11111111 -> 00000111 */
<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_functions.html">Impala Built-In Functions</a></div>
</div>
</div></body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_boolean.html b/docs/build/html/topics/impala_boolean.html
index de85f05..d2aa53a 100644
--- a/docs/build/html/topics/impala_boolean.html
+++ b/docs/build/html/topics/impala_boolean.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_datatypes.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="boolean" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -120,8 +120,8 @@
<p class="p">
- <strong class="ph b">NULL considerations:</strong> An expression of this type produces a <code class="ph codeph">NULL</code> value if any
- argument of the expression is <code class="ph codeph">NULL</code>.
+ <strong class="ph b">NULL considerations:</strong> An expression of this type produces a <code class="ph codeph">NULL</code>
+ value if any argument of the expression is <code class="ph codeph">NULL</code>.
</p>
@@ -169,8 +169,8 @@ SELECT claim FROM assertions WHERE really = TRUE;
<p class="p">
- <strong class="ph b">Text table considerations:</strong> Values of this type are potentially larger in text tables than in tables
- using Parquet or other binary formats.
+ <strong class="ph b">Text table considerations:</strong> Values of this type are potentially larger in text
+ tables than in tables using Parquet or other binary formats.
</p>
@@ -181,9 +181,9 @@ SELECT claim FROM assertions WHERE really = TRUE;
<p class="p">
- <strong class="ph b">Column statistics considerations:</strong> Because this type has a fixed size, the maximum and average size
- fields are always filled in for column statistics, even before you run the <code class="ph codeph">COMPUTE STATS</code>
- statement.
+ <strong class="ph b">Column statistics considerations:</strong> Because this type has a fixed size, the maximum
+ and average size fields are always filled in for column statistics, even before you run
+ the <code class="ph codeph">COMPUTE STATS</code> statement.
</p>
@@ -194,8 +194,8 @@ SELECT claim FROM assertions WHERE really = TRUE;
</p>
<p class="p">
- Currently, the data types <code class="ph codeph">BOOLEAN</code>, <code class="ph codeph">FLOAT</code>,
- and <code class="ph codeph">DOUBLE</code> cannot be used for primary key columns in Kudu tables.
+ Currently, the data types <code class="ph codeph">BOOLEAN</code>, <code class="ph codeph">FLOAT</code>, and
+ <code class="ph codeph">DOUBLE</code> cannot be used for primary key columns in Kudu tables.
</p>
diff --git a/docs/build/html/topics/impala_breakpad.html b/docs/build/html/topics/impala_breakpad.html
index 39b250a..3600775 100644
--- a/docs/build/html/topics/impala_breakpad.html
+++ b/docs/build/html/topics/impala_breakpad.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_troubleshooting.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="breakpad" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -39,7 +39,7 @@
</p>
- <div class="note important"><span class="importanttitle">Important:</span>
+ <div class="note important"><span class="importanttitle">Important:</span>
Because of the internal mechanisms involving Impala memory allocation and Linux
signalling for out-of-memory (OOM) errors, if an Impala-related daemon experiences a
crash due to an OOM condition, it does <em class="ph i">not</em> generate a minidump for that error.
@@ -302,4 +302,4 @@ Wrote minidump to /var/log/impala-minidumps/impalad/0980da2d-a905-01e1-25ff883a-
</body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_buffer_pool_limit.html b/docs/build/html/topics/impala_buffer_pool_limit.html
index 039fbb2..37a5b47 100644
--- a/docs/build/html/topics/impala_buffer_pool_limit.html
+++ b/docs/build/html/topics/impala_buffer_pool_limit.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="buffer_pool_limit" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
diff --git a/docs/build/html/topics/impala_char.html b/docs/build/html/topics/impala_char.html
index c80e0fc..27cff29 100644
--- a/docs/build/html/topics/impala_char.html
+++ b/docs/build/html/topics/impala_char.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_datatypes.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="char" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -125,10 +125,10 @@
<p class="p">
- <strong class="ph b">Partitioning:</strong> This type can be used for partition key columns. Because of the efficiency advantage
- of numeric values over character-based values, if the partition key is a string representation of a number,
- prefer to use an integer type with sufficient range (<code class="ph codeph">INT</code>, <code class="ph codeph">BIGINT</code>, and so
- on) where practical.
+ <strong class="ph b">Partitioning:</strong> This type can be used for partition key columns. Because of the
+ efficiency advantage of numeric values over character-based values, if the partition key
+ is a string representation of a number, prefer to use an integer type with sufficient
+ range (<code class="ph codeph">INT</code>, <code class="ph codeph">BIGINT</code>, and so on) where practical.
</p>
@@ -196,13 +196,13 @@
<p class="p">
- The Avro specification allows string values up to 2**64 bytes in length.
- Impala queries for Avro tables use 32-bit integers to hold string lengths.
- In <span class="keyword">Impala 2.5</span> and higher, Impala truncates <code class="ph codeph">CHAR</code>
- and <code class="ph codeph">VARCHAR</code> values in Avro tables to (2**31)-1 bytes.
- If a query encounters a <code class="ph codeph">STRING</code> value longer than (2**31)-1
- bytes in an Avro table, the query fails. In earlier releases,
- encountering such long values in an Avro table could cause a crash.
+ The Avro specification allows string values up to 2**64 bytes in length. Impala queries
+ for Avro tables use 32-bit integers to hold string lengths. In
+ <span class="keyword">Impala 2.5</span> and higher, Impala truncates <code class="ph codeph">CHAR</code> and
+ <code class="ph codeph">VARCHAR</code> values in Avro tables to (2**31)-1 bytes. If a query encounters
+ a <code class="ph codeph">STRING</code> value longer than (2**31)-1 bytes in an Avro table, the query
+ fails. In earlier releases, encountering such long values in an Avro table could cause a
+ crash.
</p>
@@ -225,9 +225,9 @@
<p class="p">
- <strong class="ph b">Internal details:</strong> Represented in memory as a byte array with the same size as the length
- specification. Values that are shorter than the specified length are padded on the right with trailing
- spaces.
+ <strong class="ph b">Internal details:</strong> Represented in memory as a byte array with the same size as the
+ length specification. Values that are shorter than the specified length are padded on
+ the right with trailing spaces.
</p>
@@ -237,15 +237,15 @@
<p class="p">
- <strong class="ph b">Column statistics considerations:</strong> Because this type has a fixed size, the maximum and average size
- fields are always filled in for column statistics, even before you run the <code class="ph codeph">COMPUTE STATS</code>
- statement.
+ <strong class="ph b">Column statistics considerations:</strong> Because this type has a fixed size, the maximum
+ and average size fields are always filled in for column statistics, even before you run
+ the <code class="ph codeph">COMPUTE STATS</code> statement.
</p>
<p class="p">
- <strong class="ph b">UDF considerations:</strong> This type cannot be used for the argument or return type of a user-defined
- function (UDF) or user-defined aggregate function (UDA).
+ <strong class="ph b">UDF considerations:</strong> This type cannot be used for the argument or return type of a
+ user-defined function (UDF) or user-defined aggregate function (UDA).
</p>
@@ -256,7 +256,8 @@
<p class="p">
Currently, the data types <code class="ph codeph">CHAR</code>, <code class="ph codeph">VARCHAR</code>,
- <code class="ph codeph">ARRAY</code>, <code class="ph codeph">MAP</code>, and <code class="ph codeph">STRUCT</code> cannot be used with Kudu tables.
+ <code class="ph codeph">ARRAY</code>, <code class="ph codeph">MAP</code>, and <code class="ph codeph">STRUCT</code> cannot be used
+ with Kudu tables.
</p>
@@ -286,9 +287,9 @@
<p class="p">
- All data in <code class="ph codeph">CHAR</code> and <code class="ph codeph">VARCHAR</code> columns must be in a character encoding that
- is compatible with UTF-8. If you have binary data from another database system (that is, a BLOB type), use
- a <code class="ph codeph">STRING</code> column to hold it.
+ All data in <code class="ph codeph">CHAR</code> and <code class="ph codeph">VARCHAR</code> columns must be in a
+ character encoding that is compatible with UTF-8. If you have binary data from another
+ database system (that is, a BLOB type), use a <code class="ph codeph">STRING</code> column to hold it.
</p>
diff --git a/docs/build/html/topics/impala_client.html b/docs/build/html/topics/impala_client.html
new file mode 100644
index 0000000..82f8a77
--- /dev/null
+++ b/docs/build/html/topics/impala_client.html
@@ -0,0 +1,332 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE html
+ PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+
+<meta name="copyright" content="(C) Copyright 2019" />
+<meta name="DC.rights.owner" content="(C) Copyright 2019" />
+<meta name="DC.Type" content="concept" />
+<meta name="DC.Title" content="Impala Client Access" />
+<meta name="DC.Relation" scheme="URI" content="../topics/impala_impala_shell.html" />
+<meta name="DC.Relation" scheme="URI" content="../topics/impala_odbc.html" />
+<meta name="DC.Relation" scheme="URI" content="../topics/impala_jdbc.html" />
+<meta name="prodname" content="Impala" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="DC.Format" content="XHTML" />
+<meta name="DC.Identifier" content="intro_client" />
+<link rel="stylesheet" type="text/css" href="../commonltr.css" />
+<title>Impala Client Access</title>
+</head>
+<body id="intro_client">
+
+
+ <h1 class="title topictitle1" id="ariaid-title1">Impala Client Access</h1>
+
+
+ <div class="body conbody">
+
+ <p class="p">
+ Application developers have a number of options to interface with Impala. The core
+ development language with Impala is SQL. You can also use Java or other languages to
+ interact with Impala through the standard JDBC and ODBC interfaces used by many business
+ intelligence tools. For specialized kinds of analysis, you can supplement the Impala
+ built-in functions by writing user-defined functions in C++ or Java.
+ </p>
+
+
+ <p class="p">
+ You can connect and submit requests to the Impala through:
+ </p>
+
+
+ <ul class="ul">
+ <li class="li">
+ The impala-shell interactive command interpreter
+ </li>
+
+
+ <li class="li">
+ The Hue web-based user interface
+ </li>
+
+
+ <li class="li">
+ JDBC
+ </li>
+
+
+ <li class="li">
+ ODBC
+ </li>
+
+ </ul>
+
+
+ <p class="p">
+ Impala clients can connect to the Coordinator Impala Daemon (<code class="ph codeph">impalad</code>) via
+ HiveServer2 over HTTP or over the TCP binary. Both HTTP and binary support the Kerberos
+ SPNEGO and LDAP for authentication to Impala. See below for the default ports and the
+ Impala flags to change the ports.
+ </p>
+
+
+ <div class="p">
+ <table cellpadding="4" cellspacing="0" summary="" id="intro_client__simpletable_tr2_gnt_43b" border="1" class="simpletable"><col style="width:22.675736961451246%" /><col style="width:23.356009070294785%" /><col style="width:53.96825396825397%" /><thead></thead><tbody><tr class="strow">
+
+ <td style="vertical-align:top;" class="stentry"><strong class="ph b">Protocol</strong>
+
+ </td>
+
+
+ <td style="vertical-align:top;" class="stentry"><strong class="ph b">Default Port</strong>
+
+ </td>
+
+
+ <td style="vertical-align:top;" class="stentry"><strong class="ph b">Flag to Specify an Alternate Port</strong>
+
+ </td>
+
+
+ </tr>
+<tr class="strow">
+
+ <td style="vertical-align:top;" class="stentry">HTTP</td>
+
+
+ <td style="vertical-align:top;" class="stentry">28000</td>
+
+
+ <td style="vertical-align:top;" class="stentry"><code class="ph codeph">‑‑hs2_http_port</code>
+
+ </td>
+
+
+ </tr>
+<tr class="strow">
+
+ <td style="vertical-align:top;" class="stentry">Binary TCP</td>
+
+
+ <td style="vertical-align:top;" class="stentry">21050</td>
+
+
+ <td style="vertical-align:top;" class="stentry"><code class="ph codeph">‑‑hs2_port</code>
+
+ </td>
+
+
+ </tr>
+</tbody></table>
+
+ </div>
+
+
+ <p class="p">
+ Each <code class="ph codeph">impalad</code> daemon process, running on separate nodes in a cluster,
+ listens to <a class="xref" href="impala_ports.html#ports">several ports</a> for incoming requests:
+ </p>
+
+
+ <ul class="ul">
+ <li class="li">
+ Requests from <code class="ph codeph">impala-shell</code> and Hue are routed to the
+ <code class="ph codeph">impalad</code> daemons through the same port.
+ </li>
+
+
+ <li class="li">
+ The <code class="ph codeph">impalad</code> daemons listen on separate ports for JDBC and ODBC
+ requests.
+ </li>
+
+ </ul>
+
+
+ <div class="section" id="intro_client__section_egg_wjt_f3b"><h2 class="title sectiontitle">Impala Startup Options for Client Connections</h2>
+
+
+
+ <p class="p">
+ Use the following flags when starting Impala Daemon coordinator to control client
+ connections to Impala.
+ </p>
+
+
+ <dl class="dl">
+
+
+ <dt class="dt dlterm">
+ --accepted_client_cnxn_timeout
+ </dt>
+
+
+ <dd class="dd">
+ Controls how Impala treats new connection requests if it has run out of the number
+ of threads configured by <code class="ph codeph">--fe_service_threads</code>.
+ <p class="p">
+ If <code class="ph codeph">--accepted_client_cnxn_timeout > 0</code>, new connection requests
+ are rejected if Impala can't get a server thread within the specified (in seconds)
+ timeout.
+ </p>
+
+
+ <p class="p">
+ If <code class="ph codeph">--accepted_client_cnxn_timeout=0</code>, i.e. no timeout, clients
+ wait indefinitely to open the new session until more threads are available.
+ </p>
+
+
+ <p class="p">
+ The default timeout is 5 minutes.
+ </p>
+
+
+ <p class="p">
+ The timeout applies only to client facing thrift servers, i.e., HS2 and Beeswax
+ servers.
+ </p>
+
+ </dd>
+
+
+
+
+
+
+ <dt class="dt dlterm">
+ --disconnected_session_timeout
+ </dt>
+
+
+ <dd class="dd">
+ When a HiveServer2 session has had no open connections for longer than this value,
+ the session will be closed, and any associated queries will be unregistered.
+ <p class="p">
+ Specify the value in hours.
+ </p>
+
+
+ <p class="p">
+ The default value is 1 hour.
+ </p>
+
+
+ <p class="p">
+ This flag does not apply to Beeswax clients. When a Beeswax client connection is
+ closed, Impala closes the session associated with that connection.
+ </p>
+
+ </dd>
+
+
+
+
+
+
+ <dt class="dt dlterm">
+ --fe_service_threads
+ </dt>
+
+
+ <dd class="dd">
+ Specifies the maximum number of concurrent client connections allowed. The default
+ value is 64 with which 64 queries can run simultaneously.
+ <p class="p">
+ If you have more clients trying to connect to Impala than the value of this
+ setting, the later arriving clients have to wait for the duration specified by
+ <code class="ph codeph">--accepted_client_cnxn_timeout</code>. You can increase this value to
+ allow more client connections. However, a large value means more threads to be
+ maintained even if most of the connections are idle, and it could negatively
+ impact query latency. Client applications should use the connection pool to avoid
+ need for large number of sessions.
+ </p>
+
+ </dd>
+
+
+
+
+
+
+ <dt class="dt dlterm">
+ --idle_client_poll_time_s
+ </dt>
+
+
+ <dd class="dd">
+ The value of this setting specifies how frequently Impala polls to check if a client
+ connection is idle and closes it if the connection is idle. A client connection is
+ idle if all sessions associated with the client connection are idle.
+ <p class="p">
+ By default, <code class="ph codeph">--idle_client_poll_time_s</code> is set to 30 seconds.
+ </p>
+
+
+ <p class="p">
+ If <code class="ph codeph">--idle_client_poll_time_s</code> is set to 0, idle client connections
+ stay open until explicitly closed by the clients.
+ </p>
+
+
+ <p class="p">
+ The connection will only be closed if all the associated sessions are idle or
+ closed. Sessions cannot be idle unless either the flag
+ <code class="ph codeph">--idle_session_timeout</code> or the
+ <code class="ph codeph">IDLE_SESSION_TIMEOUT</code> query option is set to greater than 0. If
+ idle session timeout is not configured, a session cannot become idle by
+ definition, and therefore its connection stays open until the client explicitly
+ closes it.
+ </p>
+
+ </dd>
+
+
+
+
+
+
+ <dt class="dt dlterm">
+ --hs2_http_port
+ </dt>
+
+
+ <dd class="dd">
+ Specifies the port for clients to connect to Impala server over HTTP.
+ <p class="p">
+ You can disable the HTTP end point for clients by setting the flag to
+ <code class="ph codeph">0</code>.
+ </p>
+
+
+ <p class="p">
+ To enable TLS/SSL for HiveServer2 HTTP endpoint use
+ <code class="ph codeph">--ssl_server_certificate</code> and <code class="ph codeph">--ssl_private_key</code>.
+ See <a class="xref" href="impala_ssl.html#ssl">Configuring TLS/SSL for Impala</a> for detail.
+ </p>
+
+ </dd>
+
+
+
+ </dl>
+
+
+ </div>
+
+
+ </div>
+
+
+<div class="related-links">
+<ul class="ullinks">
+<li class="link ulchildlink"><strong><a href="../topics/impala_impala_shell.html">Using the Impala Shell (impala-shell Command)</a></strong><br />
+</li>
+<li class="link ulchildlink"><strong><a href="../topics/impala_odbc.html">Configuring Impala to Work with ODBC</a></strong><br />
+</li>
+<li class="link ulchildlink"><strong><a href="../topics/impala_jdbc.html">Configuring Impala to Work with JDBC</a></strong><br />
+</li>
+</ul>
+</div></body>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_comment.html b/docs/build/html/topics/impala_comment.html
index a127b0b..94196af 100644
--- a/docs/build/html/topics/impala_comment.html
+++ b/docs/build/html/topics/impala_comment.html
@@ -10,7 +10,7 @@
<meta name="DC.Title" content="COMMENT Statement" />
<meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="comment" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -22,9 +22,9 @@
<h1 class="title topictitle1" id="ariaid-title1">COMMENT Statement</h1>
+
-
-
+
<div class="body conbody">
@@ -114,4 +114,4 @@
<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_langref_sql.html">Impala SQL Statements</a></div>
</div>
</div></body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_comments.html b/docs/build/html/topics/impala_comments.html
index f5cab8c..89602f7 100644
--- a/docs/build/html/topics/impala_comments.html
+++ b/docs/build/html/topics/impala_comments.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_langref.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="comments" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
diff --git a/docs/build/html/topics/impala_complex_types.html b/docs/build/html/topics/impala_complex_types.html
index e272c3a..93d46db 100644
--- a/docs/build/html/topics/impala_complex_types.html
+++ b/docs/build/html/topics/impala_complex_types.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_datatypes.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="complex_types" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -195,7 +195,7 @@
</p>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
<p class="p">
Although Impala can query complex types that are present in Parquet files, Impala currently cannot create new Parquet files
containing complex types. Therefore, the discussion and examples presume that you are working with existing Parquet data produced
@@ -681,8 +681,8 @@
<p class="p">
- The maximum length of the column definition for any complex type, including declarations for any nested types,
- is 4000 characters.
+ The maximum length of the column definition for any complex type, including
+ declarations for any nested types, is 4000 characters.
</p>
@@ -716,12 +716,12 @@
<p class="p">
- Impala currently cannot write new data files containing complex type columns.
- Therefore, although the <code class="ph codeph">SELECT</code> statement works for queries
- involving complex type columns, you cannot use a statement form that writes
- data to complex type columns, such as <code class="ph codeph">CREATE TABLE AS SELECT</code> or <code class="ph codeph">INSERT ... SELECT</code>.
- To create data files containing complex type data, use the Hive <code class="ph codeph">INSERT</code> statement, or another
- ETL mechanism such as MapReduce jobs, Spark jobs, Pig, and so on.
+ Impala currently cannot write new data files containing complex type columns. Therefore,
+ although the <code class="ph codeph">SELECT</code> statement works for queries involving complex type
+ columns, you cannot use a statement form that writes data to complex type columns, such
+ as <code class="ph codeph">CREATE TABLE AS SELECT</code> or <code class="ph codeph">INSERT ... SELECT</code>. To
+ create data files containing complex type data, use the Hive <code class="ph codeph">INSERT</code>
+ statement, or another ETL mechanism such as MapReduce jobs, Spark jobs, Pig, and so on.
</p>
@@ -809,8 +809,8 @@
<p class="p">
- Partitioned tables can contain complex type columns.
- All the partition key columns must be scalar types.
+ Partitioned tables can contain complex type columns. All the partition key columns must
+ be scalar types.
</p>
@@ -830,7 +830,7 @@
</p>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
<p class="p">
Once you have a table set up with complex type columns, use the <code class="ph codeph">DESCRIBE</code> and <code class="ph codeph">SHOW CREATE TABLE</code>
statements to see the correct notation with <code class="ph codeph"><</code> and <code class="ph codeph">></code> delimiters and comma and colon
@@ -1058,12 +1058,12 @@ CREATE TABLE contacts_detailed_phones
<p class="p">
- Impala currently cannot write new data files containing complex type columns.
- Therefore, although the <code class="ph codeph">SELECT</code> statement works for queries
- involving complex type columns, you cannot use a statement form that writes
- data to complex type columns, such as <code class="ph codeph">CREATE TABLE AS SELECT</code> or <code class="ph codeph">INSERT ... SELECT</code>.
- To create data files containing complex type data, use the Hive <code class="ph codeph">INSERT</code> statement, or another
- ETL mechanism such as MapReduce jobs, Spark jobs, Pig, and so on.
+ Impala currently cannot write new data files containing complex type columns. Therefore,
+ although the <code class="ph codeph">SELECT</code> statement works for queries involving complex type
+ columns, you cannot use a statement form that writes data to complex type columns, such
+ as <code class="ph codeph">CREATE TABLE AS SELECT</code> or <code class="ph codeph">INSERT ... SELECT</code>. To
+ create data files containing complex type data, use the Hive <code class="ph codeph">INSERT</code>
+ statement, or another ETL mechanism such as MapReduce jobs, Spark jobs, Pig, and so on.
</p>
@@ -1156,12 +1156,11 @@ STORED AS PARQUET;
</p>
- <div class="note note"><span class="notetitle">Note:</span>
- Many of the complex type examples refer to tables
- such as <code class="ph codeph">CUSTOMER</code> and <code class="ph codeph">REGION</code>
- adapted from the tables used in the TPC-H benchmark.
- See <a class="xref" href="../shared/../topics/impala_complex_types.html#complex_sample_schema">Sample Schema and Data for Experimenting with Impala Complex Types</a>
- for the table definitions.
+ <div class="note note"><span class="notetitle">Note:</span>
+ Many of the complex type examples refer to tables such as <code class="ph codeph">CUSTOMER</code> and
+ <code class="ph codeph">REGION</code> adapted from the tables used in the TPC-H benchmark. See
+ <a class="xref" href="../shared/../topics/impala_complex_types.html#complex_sample_schema">Sample Schema and Data for Experimenting with Impala Complex Types</a> for the table
+ definitions.
</div>
@@ -1319,12 +1318,11 @@ LIMIT 10;
- <div class="note note"><span class="notetitle">Note:</span>
- Many of the complex type examples refer to tables
- such as <code class="ph codeph">CUSTOMER</code> and <code class="ph codeph">REGION</code>
- adapted from the tables used in the TPC-H benchmark.
- See <a class="xref" href="../shared/../topics/impala_complex_types.html#complex_sample_schema">Sample Schema and Data for Experimenting with Impala Complex Types</a>
- for the table definitions.
+ <div class="note note"><span class="notetitle">Note:</span>
+ Many of the complex type examples refer to tables such as <code class="ph codeph">CUSTOMER</code> and
+ <code class="ph codeph">REGION</code> adapted from the tables used in the TPC-H benchmark. See
+ <a class="xref" href="../shared/../topics/impala_complex_types.html#complex_sample_schema">Sample Schema and Data for Experimenting with Impala Complex Types</a> for the table
+ definitions.
</div>
@@ -1793,7 +1791,7 @@ WHERE associates.item LIKE '% MacGuffin';
</p>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
If you find that the single-item nature of the <code class="ph codeph">VALUE</code> makes it difficult to model your data accurately, the
solution is typically to add some nesting to the complex type. For example, to have several sets of key-value pairs, make the
column an <code class="ph codeph">ARRAY</code> whose elements are <code class="ph codeph">MAP</code>. To make a set of key-value pairs that holds more
@@ -1920,7 +1918,7 @@ WHERE
<code class="ph codeph">CREATE TABLE</code> statement. </p>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
Because Hive currently cannot construct individual rows using complex types through the <code class="ph codeph">INSERT ... VALUES</code> syntax,
you prepare the data in flat form in a separate table, then copy it to the table with complex columns using <code class="ph codeph">INSERT ...
SELECT</code> and the complex type constructors. See <a class="xref" href="impala_complex_types.html#complex_types_ex_hive_etl">Constructing Parquet/ORC Files with Complex Columns Using Hive</a> for
@@ -2901,4 +2899,4 @@ describe customer_ctlp;
</body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_components.html b/docs/build/html/topics/impala_components.html
index 6bdceb2..01c6566 100644
--- a/docs/build/html/topics/impala_components.html
+++ b/docs/build/html/topics/impala_components.html
@@ -14,11 +14,11 @@
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="intro_components" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -149,13 +149,14 @@
<p class="p">
- Most considerations for load balancing and high availability apply to the <span class="keyword cmdname">impalad</span> daemon.
- The <span class="keyword cmdname">statestored</span> and <span class="keyword cmdname">catalogd</span> daemons do not have special
- requirements for high availability, because problems with those daemons do not result in data loss.
- If those daemons become unavailable due to an outage on a particular
- host, you can stop the Impala service, delete the <span class="ph uicontrol">Impala StateStore</span> and
- <span class="ph uicontrol">Impala Catalog Server</span> roles, add the roles on a different host, and restart the
- Impala service.
+ Most considerations for load balancing and high availability apply to the
+ <span class="keyword cmdname">impalad</span> daemon. The <span class="keyword cmdname">statestored</span> and
+ <span class="keyword cmdname">catalogd</span> daemons do not have special requirements for high
+ availability, because problems with those daemons do not result in data loss. If those
+ daemons become unavailable due to an outage on a particular host, you can stop the
+ Impala service, delete the <span class="ph uicontrol">Impala StateStore</span> and
+ <span class="ph uicontrol">Impala Catalog Server</span> roles, add the roles on a different host,
+ and restart the Impala service.
</p>
@@ -232,37 +233,35 @@
<div class="p">
- Use <code class="ph codeph">--load_catalog_in_background</code> option to control when
- the metadata of a table is loaded.
+ Use <code class="ph codeph">‑‑load_catalog_in_background</code> option to control when the
+ metadata of a table is loaded.
<ul class="ul">
<li class="li">
- If set to <code class="ph codeph">false</code>, the metadata of a table is
- loaded when it is referenced for the first time. This means that the
- first run of a particular query can be slower than subsequent runs.
- Starting in Impala 2.2, the default for
- <code class="ph codeph">load_catalog_in_background</code> is
- <code class="ph codeph">false</code>.
+ If set to <code class="ph codeph">false</code>, the metadata of a table is loaded when it is
+ referenced for the first time. This means that the first run of a particular query
+ can be slower than subsequent runs. Starting in Impala 2.2, the default for
+ <code class="ph codeph">‑‑load_catalog_in_background</code> is <code class="ph codeph">false</code>.
</li>
+
<li class="li">
- If set to <code class="ph codeph">true</code>, the catalog service attempts to
- load metadata for a table even if no query needed that metadata. So
- metadata will possibly be already loaded when the first query that
- would need it is run. However, for the following reasons, we
- recommend not to set the option to <code class="ph codeph">true</code>.
+ If set to <code class="ph codeph">true</code>, the catalog service attempts to load metadata for a
+ table even if no query needed that metadata. So metadata will possibly be already
+ loaded when the first query that would need it is run. However, for the following
+ reasons, we recommend not to set the option to <code class="ph codeph">true</code>.
<ul class="ul">
<li class="li">
- Background load can interfere with query-specific metadata
- loading. This can happen on startup or after invalidating
- metadata, with a duration depending on the amount of metadata,
- and can lead to a seemingly random long running queries that are
- difficult to diagnose.
+ Background load can interfere with query-specific metadata loading. This can
+ happen on startup or after invalidating metadata, with a duration depending on
+ the amount of metadata, and can lead to a seemingly random long running queries
+ that are difficult to diagnose.
</li>
+
<li class="li">
- Impala may load metadata for tables that are possibly never
- used, potentially increasing catalog size and consequently memory
- usage for both catalog service and Impala Daemon.
+ Impala may load metadata for tables that are possibly never used, potentially
+ increasing catalog size and consequently memory usage for both catalog service
+ and Impala Daemon.
</li>
</ul>
@@ -275,22 +274,24 @@
<p class="p">
- Most considerations for load balancing and high availability apply to the <span class="keyword cmdname">impalad</span> daemon.
- The <span class="keyword cmdname">statestored</span> and <span class="keyword cmdname">catalogd</span> daemons do not have special
- requirements for high availability, because problems with those daemons do not result in data loss.
- If those daemons become unavailable due to an outage on a particular
- host, you can stop the Impala service, delete the <span class="ph uicontrol">Impala StateStore</span> and
- <span class="ph uicontrol">Impala Catalog Server</span> roles, add the roles on a different host, and restart the
- Impala service.
+ Most considerations for load balancing and high availability apply to the
+ <span class="keyword cmdname">impalad</span> daemon. The <span class="keyword cmdname">statestored</span> and
+ <span class="keyword cmdname">catalogd</span> daemons do not have special requirements for high
+ availability, because problems with those daemons do not result in data loss. If those
+ daemons become unavailable due to an outage on a particular host, you can stop the
+ Impala service, delete the <span class="ph uicontrol">Impala StateStore</span> and
+ <span class="ph uicontrol">Impala Catalog Server</span> roles, add the roles on a different host,
+ and restart the Impala service.
</p>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
<p class="p">
- In Impala 1.2.4 and higher, you can specify a table name with <code class="ph codeph">INVALIDATE METADATA</code> after
- the table is created in Hive, allowing you to make individual tables visible to Impala without doing a full
- reload of the catalog metadata. Impala 1.2.4 also includes other changes to make the metadata broadcast
- mechanism faster and more responsive, especially during Impala startup. See
+ In Impala 1.2.4 and higher, you can specify a table name with <code class="ph codeph">INVALIDATE
+ METADATA</code> after the table is created in Hive, allowing you to make individual
+ tables visible to Impala without doing a full reload of the catalog metadata. Impala
+ 1.2.4 also includes other changes to make the metadata broadcast mechanism faster and
+ more responsive, especially during Impala startup. See
<a class="xref" href="../shared/../topics/impala_new_features.html#new_features_124">New Features in Impala 1.2.4</a> for details.
</p>
@@ -307,4 +308,4 @@
</div>
</body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_compression_codec.html b/docs/build/html/topics/impala_compression_codec.html
index ac658c7..3ee4467 100644
--- a/docs/build/html/topics/impala_compression_codec.html
+++ b/docs/build/html/topics/impala_compression_codec.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="compression_codec" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -39,7 +39,7 @@
</p>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
Prior to Impala 2.0, this option was named <code class="ph codeph">PARQUET_COMPRESSION_CODEC</code>. In Impala 2.0 and
later, the <code class="ph codeph">PARQUET_COMPRESSION_CODEC</code> name is not recognized. Use the more general name
<code class="ph codeph">COMPRESSION_CODEC</code> for new code.
@@ -51,15 +51,25 @@
</p>
-<pre class="pre codeblock"><code>SET COMPRESSION_CODEC=<var class="keyword varname">codec_name</var>;</code></pre>
+<pre class="pre codeblock"><code>SET COMPRESSION_CODEC=<var class="keyword varname">codec_name</var>; // Supported for all codecs.
+SET COMPRESSION_CODEC=<var class="keyword varname">codec_name</var>:<var class="keyword varname">compression_level</var>; // Only supported for ZSTD.
+</code></pre>
<p class="p">
The allowed values for this query option are <code class="ph codeph">SNAPPY</code> (the default), <code class="ph codeph">GZIP</code>,
- and <code class="ph codeph">NONE</code>.
+ <code class="ph codeph">ZSTD</code>, <code class="ph codeph">LZ4</code>, and <code class="ph codeph">NONE</code>.
+ </p>
+
+
+ <p class="p">
+ <code class="ph codeph">ZSTD</code> also supports setting a compression level. The lower the level, the faster the speed at
+ the cost of compression ratio. Compression levels from 1 up to 22 are supported for <code class="ph codeph">ZSTD</code>.
+ The default compression level 3 is used, if one is not passed using the <code class="ph codeph">compression_codec</code>
+ query option.
</p>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
A Parquet file created with <code class="ph codeph">COMPRESSION_CODEC=NONE</code> is still typically smaller than the
original data, due to encoding schemes such as run-length encoding and dictionary encoding that are applied
separately from compression.
@@ -97,7 +107,17 @@
</p>
-<pre class="pre codeblock"><code>set compression_codec=gzip;
+<pre class="pre codeblock"><code>
+set compression_codec=lz4;
+insert into parquet_table_lz4_compressed select * from t1;
+
+set compression_codec=zstd; // Default compression level 3.
+insert into parquet_table_zstd_default_compressed select * from t1;
+
+set compression_codec=zstd:12; // Compression level 12.
+insert into parquet_table_zstd_highly_compressed select * from t1;
+
+set compression_codec=gzip;
insert into parquet_table_highly_compressed select * from t1;
set compression_codec=snappy;
@@ -118,7 +138,7 @@ ERROR: Invalid compression codec: foo
<p class="p">
For information about how compressing Parquet data files affects query performance, see
- <a class="xref" href="impala_parquet.html#parquet_compression">Snappy and GZip Compression for Parquet Data Files</a>.
+ <a class="xref" href="impala_parquet.html#parquet_compression">Compressions for Parquet Data Files</a>.
</p>
</div>
@@ -128,4 +148,4 @@ ERROR: Invalid compression codec: foo
<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_query_options.html">Query Options for the SET Statement</a></div>
</div>
</div></body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_compute_stats.html b/docs/build/html/topics/impala_compute_stats.html
index 0af6f63..5ae804c 100644
--- a/docs/build/html/topics/impala_compute_stats.html
+++ b/docs/build/html/topics/impala_compute_stats.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="compute_stats" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -60,12 +60,14 @@ COMPUTE INCREMENTAL STATS [<var class="keyword varname">db_name</var>.]<var clas
</code></pre>
<p class="p">
- The <code class="ph codeph">PARTITION</code> clause is only allowed in combination with the <code class="ph codeph">INCREMENTAL</code>
- clause. It is optional for <code class="ph codeph">COMPUTE INCREMENTAL STATS</code>, and required for <code class="ph codeph">DROP
- INCREMENTAL STATS</code>. Whenever you specify partitions through the <code class="ph codeph">PARTITION
- (<var class="keyword varname">partition_spec</var>)</code> clause in a <code class="ph codeph">COMPUTE INCREMENTAL STATS</code> or
- <code class="ph codeph">DROP INCREMENTAL STATS</code> statement, you must include all the partitioning columns in the
- specification, and specify constant values for all the partition key columns.
+ The <code class="ph codeph">PARTITION</code> clause is only allowed in combination with the
+ <code class="ph codeph">INCREMENTAL</code> clause. It is optional for <code class="ph codeph">COMPUTE INCREMENTAL
+ STATS</code>, and required for <code class="ph codeph">DROP INCREMENTAL STATS</code>. Whenever you
+ specify partitions through the <code class="ph codeph">PARTITION
+ (<var class="keyword varname">partition_spec</var>)</code> clause in a <code class="ph codeph">COMPUTE INCREMENTAL
+ STATS</code> or <code class="ph codeph">DROP INCREMENTAL STATS</code> statement, you must include
+ all the partitioning columns in the specification, and specify constant values for all
+ the partition key columns.
</p>
@@ -138,33 +140,32 @@ COMPUTE INCREMENTAL STATS [<var class="keyword varname">db_name</var>.]<var clas
</p>
- <div class="note important"><span class="importanttitle">Important:</span>
+ <div class="note important"><span class="importanttitle">Important:</span>
<p class="p">
- For a particular table, use either <code class="ph codeph">COMPUTE STATS</code> or
- <code class="ph codeph">COMPUTE INCREMENTAL STATS</code>, but never combine the two or
- alternate between them. If you switch from <code class="ph codeph">COMPUTE STATS</code> to
- <code class="ph codeph">COMPUTE INCREMENTAL STATS</code> during the lifetime of a table, or
- vice versa, drop all statistics by running <code class="ph codeph">DROP STATS</code> before
- making the switch.
+ For a particular table, use either <code class="ph codeph">COMPUTE STATS</code> or <code class="ph codeph">COMPUTE
+ INCREMENTAL STATS</code>, but never combine the two or alternate between them. If you
+ switch from <code class="ph codeph">COMPUTE STATS</code> to <code class="ph codeph">COMPUTE INCREMENTAL STATS</code>
+ during the lifetime of a table, or vice versa, drop all statistics by running
+ <code class="ph codeph">DROP STATS</code> before making the switch.
</p>
<p class="p">
When you run <code class="ph codeph">COMPUTE INCREMENTAL STATS</code> on a table for the first time,
the statistics are computed again from scratch regardless of whether the table already
- has statistics. Therefore, expect a one-time resource-intensive operation
- for scanning the entire table when running <code class="ph codeph">COMPUTE INCREMENTAL STATS</code>
- for the first time on a given table.
+ has statistics. Therefore, expect a one-time resource-intensive operation for scanning
+ the entire table when running <code class="ph codeph">COMPUTE INCREMENTAL STATS</code> for the first
+ time on a given table.
</p>
- <p class="p"> In Impala 3.0 and lower, approximately
- 400 bytes of metadata per column per partition are needed for caching.
- Tables with a big number of partitions and many columns can add up to a
- significant memory overhead as the metadata must be cached on the
- <span class="keyword cmdname">catalogd</span> host and on every
- <span class="keyword cmdname">impalad</span> host that is eligible to be a coordinator.
- If this metadata for all tables exceeds 2 GB, you might experience
- service downtime. In Impala 3.1 and higher, the issue was alleviated
- with an improved handling of incremental stats.</p>
+ <p class="p">
+ In Impala 3.0 and lower, approximately 400 bytes of metadata per column per partition
+ are needed for caching. Tables with a big number of partitions and many columns can add
+ up to a significant memory overhead as the metadata must be cached on the
+ <span class="keyword cmdname">catalogd</span> host and on every <span class="keyword cmdname">impalad</span> host that is
+ eligible to be a coordinator. If this metadata for all tables exceeds 2 GB, you might
+ experience service downtime. In Impala 3.1 and higher, the issue was alleviated with an
+ improved handling of incremental stats.
+ </p>
</div>
@@ -176,7 +177,7 @@ COMPUTE INCREMENTAL STATS [<var class="keyword varname">db_name</var>.]<var clas
<code class="ph codeph">Incremental stats</code> column of the <code class="ph codeph">SHOW TABLE STATS</code> output.
</p>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
<div class="p">
Because many of the most performance-critical and resource-intensive
operations rely on table and column statistics to construct accurate and
@@ -625,13 +626,14 @@ show table stats item_partitioned;
<p class="p">
- <strong class="ph b">Cancellation:</strong> Certain multi-stage statements (<code class="ph codeph">CREATE TABLE AS SELECT</code> and
- <code class="ph codeph">COMPUTE STATS</code>) can be cancelled during some stages, when running <code class="ph codeph">INSERT</code>
- or <code class="ph codeph">SELECT</code> operations internally. To cancel this statement, use Ctrl-C from the
- <span class="keyword cmdname">impala-shell</span> interpreter, the <span class="ph uicontrol">Cancel</span> button from the
- <span class="ph uicontrol">Watch</span> page in Hue, or <span class="ph uicontrol">Cancel</span> from the list of
- in-flight queries (for a particular node) on the <span class="ph uicontrol">Queries</span> tab in the Impala web UI
- (port 25000).
+ <strong class="ph b">Cancellation:</strong> Certain multi-stage statements (<code class="ph codeph">CREATE TABLE AS
+ SELECT</code> and <code class="ph codeph">COMPUTE STATS</code>) can be cancelled during some stages,
+ when running <code class="ph codeph">INSERT</code> or <code class="ph codeph">SELECT</code> operations internally.
+ To cancel this statement, use Ctrl-C from the <span class="keyword cmdname">impala-shell</span>
+ interpreter, the <span class="ph uicontrol">Cancel</span> button from the
+ <span class="ph uicontrol">Watch</span> page in Hue, or <span class="ph uicontrol">Cancel</span> from the list
+ of in-flight queries (for a particular node) on the <span class="ph uicontrol">Queries</span> tab
+ in the Impala web UI (port 25000).
</p>
@@ -640,13 +642,13 @@ show table stats item_partitioned;
</p>
- <div class="note note"><span class="notetitle">Note:</span> Prior to Impala 1.4.0,
- <code class="ph codeph">COMPUTE STATS</code> counted the number of
- <code class="ph codeph">NULL</code> values in each column and recorded that figure
- in the metastore database. Because Impala does not currently use the
- <code class="ph codeph">NULL</code> count during query planning, Impala 1.4.0 and
- higher speeds up the <code class="ph codeph">COMPUTE STATS</code> statement by
- skipping this <code class="ph codeph">NULL</code> counting. </div>
+ <div class="note note"><span class="notetitle">Note:</span>
+ Prior to Impala 1.4.0, <code class="ph codeph">COMPUTE STATS</code> counted the number of
+ <code class="ph codeph">NULL</code> values in each column and recorded that figure in the metastore
+ database. Because Impala does not currently use the <code class="ph codeph">NULL</code> count during
+ query planning, Impala 1.4.0 and higher speeds up the <code class="ph codeph">COMPUTE STATS</code>
+ statement by skipping this <code class="ph codeph">NULL</code> counting.
+ </div>
<p class="p">
@@ -720,4 +722,4 @@ show table stats item_partitioned;
<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_langref_sql.html">Impala SQL Statements</a></div>
</div>
</div></body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_compute_stats_min_sample_size.html b/docs/build/html/topics/impala_compute_stats_min_sample_size.html
index 89c651d..18ae008 100644
--- a/docs/build/html/topics/impala_compute_stats_min_sample_size.html
+++ b/docs/build/html/topics/impala_compute_stats_min_sample_size.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_query_options.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="compute_stats_sample_min_sample_size" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -22,8 +22,8 @@
<h1 class="title topictitle1" id="ariaid-title1">COMPUTE_STATS_MIN_SAMPLE_SIZE Query Option</h1>
-
-
+
+
<div class="body conbody">
<p class="p">The <code class="ph codeph">COMPUTE_STATS_MIN_SAMPLE_SIZE</code> query option specifies
the minimum number of bytes that will be scanned in <code class="ph codeph">COMPUTE STATS
@@ -51,4 +51,4 @@
<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_query_options.html">Query Options for the SET Statement</a></div>
</div>
</div></body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_concepts.html b/docs/build/html/topics/impala_concepts.html
index 51f833c..497309a 100644
--- a/docs/build/html/topics/impala_concepts.html
+++ b/docs/build/html/topics/impala_concepts.html
@@ -13,8 +13,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_hadoop.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="concepts" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
diff --git a/docs/build/html/topics/impala_conditional_functions.html b/docs/build/html/topics/impala_conditional_functions.html
index 6fa5966..a4dade7 100644
--- a/docs/build/html/topics/impala_conditional_functions.html
+++ b/docs/build/html/topics/impala_conditional_functions.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_functions.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="conditional_functions" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -141,9 +141,10 @@
<strong class="ph b">Purpose:</strong> Compares an expression to one or more possible values, and returns a
corresponding result when a match is found.
<p class="p">
- <strong class="ph b">Return type:</strong> same as the initial argument value, except that integer values are promoted to
- <code class="ph codeph">BIGINT</code> and floating-point values are promoted to <code class="ph codeph">DOUBLE</code>; use
- <code class="ph codeph">CAST()</code> when inserting into a smaller numeric column
+ <strong class="ph b">Return type:</strong> same as the initial argument value, except that integer values are
+ promoted to <code class="ph codeph">BIGINT</code> and floating-point values are promoted to
+ <code class="ph codeph">DOUBLE</code>; use <code class="ph codeph">CAST()</code> when inserting into a smaller
+ numeric column
</p>
@@ -199,9 +200,10 @@
<code class="ph codeph">TRUE</code>, and returns a corresponding result for the first true
expression.
<p class="p">
- <strong class="ph b">Return type:</strong> same as the initial argument value, except that integer values are promoted to
- <code class="ph codeph">BIGINT</code> and floating-point values are promoted to <code class="ph codeph">DOUBLE</code>; use
- <code class="ph codeph">CAST()</code> when inserting into a smaller numeric column
+ <strong class="ph b">Return type:</strong> same as the initial argument value, except that integer values are
+ promoted to <code class="ph codeph">BIGINT</code> and floating-point values are promoted to
+ <code class="ph codeph">DOUBLE</code>; use <code class="ph codeph">CAST()</code> when inserting into a smaller
+ numeric column
</p>
@@ -289,9 +291,10 @@
<code class="ph codeph">NULL</code>, or <code class="ph codeph">NULL</code> if all arguments are
<code class="ph codeph">NULL</code>.
<p class="p">
- <strong class="ph b">Return type:</strong> same as the initial argument value, except that integer values are promoted to
- <code class="ph codeph">BIGINT</code> and floating-point values are promoted to <code class="ph codeph">DOUBLE</code>; use
- <code class="ph codeph">CAST()</code> when inserting into a smaller numeric column
+ <strong class="ph b">Return type:</strong> same as the initial argument value, except that integer values are
+ promoted to <code class="ph codeph">BIGINT</code> and floating-point values are promoted to
+ <code class="ph codeph">DOUBLE</code>; use <code class="ph codeph">CAST()</code> when inserting into a smaller
+ numeric column
</p>
</dd>
@@ -487,12 +490,12 @@
</p>
- <p class="p"> In
- <span class="keyword">Impala 2.11</span> and higher, you can use the
- operators <code class="ph codeph">IS [NOT] TRUE</code> and <code class="ph codeph">IS [NOT]
- FALSE</code> as equivalents for the built-in functions
- <code class="ph codeph">ISTRUE()</code>, <code class="ph codeph">ISNOTTRUE()</code>,
- <code class="ph codeph">ISFALSE()</code>, and <code class="ph codeph">ISNOTFALSE()</code>. </p>
+ <p class="p">
+ In <span class="keyword">Impala 2.11</span> and higher, you can use the operators <code class="ph codeph">IS
+ [NOT] TRUE</code> and <code class="ph codeph">IS [NOT] FALSE</code> as equivalents for the built-in
+ functions <code class="ph codeph">ISTRUE()</code>, <code class="ph codeph">ISNOTTRUE()</code>,
+ <code class="ph codeph">ISFALSE()</code>, and <code class="ph codeph">ISNOTFALSE()</code>.
+ </p>
</dd>
@@ -527,7 +530,8 @@
<p class="p">
- <strong class="ph b">Usage notes:</strong> Primarily for compatibility with code containing industry extensions to SQL.
+ <strong class="ph b">Usage notes:</strong> Primarily for compatibility with code containing industry extensions
+ to SQL.
</p>
@@ -541,12 +545,12 @@
</p>
- <p class="p"> In
- <span class="keyword">Impala 2.11</span> and higher, you can use the
- operators <code class="ph codeph">IS [NOT] TRUE</code> and <code class="ph codeph">IS [NOT]
- FALSE</code> as equivalents for the built-in functions
- <code class="ph codeph">ISTRUE()</code>, <code class="ph codeph">ISNOTTRUE()</code>,
- <code class="ph codeph">ISFALSE()</code>, and <code class="ph codeph">ISNOTFALSE()</code>. </p>
+ <p class="p">
+ In <span class="keyword">Impala 2.11</span> and higher, you can use the operators <code class="ph codeph">IS
+ [NOT] TRUE</code> and <code class="ph codeph">IS [NOT] FALSE</code> as equivalents for the built-in
+ functions <code class="ph codeph">ISTRUE()</code>, <code class="ph codeph">ISNOTTRUE()</code>,
+ <code class="ph codeph">ISFALSE()</code>, and <code class="ph codeph">ISNOTFALSE()</code>.
+ </p>
</dd>
@@ -590,12 +594,12 @@
</p>
- <p class="p"> In
- <span class="keyword">Impala 2.11</span> and higher, you can use the
- operators <code class="ph codeph">IS [NOT] TRUE</code> and <code class="ph codeph">IS [NOT]
- FALSE</code> as equivalents for the built-in functions
- <code class="ph codeph">ISTRUE()</code>, <code class="ph codeph">ISNOTTRUE()</code>,
- <code class="ph codeph">ISFALSE()</code>, and <code class="ph codeph">ISNOTFALSE()</code>. </p>
+ <p class="p">
+ In <span class="keyword">Impala 2.11</span> and higher, you can use the operators <code class="ph codeph">IS
+ [NOT] TRUE</code> and <code class="ph codeph">IS [NOT] FALSE</code> as equivalents for the built-in
+ functions <code class="ph codeph">ISTRUE()</code>, <code class="ph codeph">ISNOTTRUE()</code>,
+ <code class="ph codeph">ISFALSE()</code>, and <code class="ph codeph">ISNOTFALSE()</code>.
+ </p>
</dd>
@@ -657,7 +661,8 @@
<p class="p">
- <strong class="ph b">Usage notes:</strong> Primarily for compatibility with code containing industry extensions to SQL.
+ <strong class="ph b">Usage notes:</strong> Primarily for compatibility with code containing industry extensions
+ to SQL.
</p>
@@ -671,12 +676,12 @@
</p>
- <p class="p"> In
- <span class="keyword">Impala 2.11</span> and higher, you can use the
- operators <code class="ph codeph">IS [NOT] TRUE</code> and <code class="ph codeph">IS [NOT]
- FALSE</code> as equivalents for the built-in functions
- <code class="ph codeph">ISTRUE()</code>, <code class="ph codeph">ISNOTTRUE()</code>,
- <code class="ph codeph">ISFALSE()</code>, and <code class="ph codeph">ISNOTFALSE()</code>. </p>
+ <p class="p">
+ In <span class="keyword">Impala 2.11</span> and higher, you can use the operators <code class="ph codeph">IS
+ [NOT] TRUE</code> and <code class="ph codeph">IS [NOT] FALSE</code> as equivalents for the built-in
+ functions <code class="ph codeph">ISTRUE()</code>, <code class="ph codeph">ISNOTTRUE()</code>,
+ <code class="ph codeph">ISFALSE()</code>, and <code class="ph codeph">ISNOTFALSE()</code>.
+ </p>
</dd>
@@ -709,7 +714,8 @@
<p class="p">
- <strong class="ph b">Usage notes:</strong> Primarily for compatibility with code containing industry extensions to SQL.
+ <strong class="ph b">Usage notes:</strong> Primarily for compatibility with code containing industry extensions
+ to SQL.
</p>
@@ -760,9 +766,10 @@ END</code></pre>
<p class="p">
- <strong class="ph b">Return type:</strong> same as the initial argument value, except that integer values are promoted to
- <code class="ph codeph">BIGINT</code> and floating-point values are promoted to <code class="ph codeph">DOUBLE</code>; use
- <code class="ph codeph">CAST()</code> when inserting into a smaller numeric column
+ <strong class="ph b">Return type:</strong> same as the initial argument value, except that integer values are
+ promoted to <code class="ph codeph">BIGINT</code> and floating-point values are promoted to
+ <code class="ph codeph">DOUBLE</code>; use <code class="ph codeph">CAST()</code> when inserting into a smaller
+ numeric column
</p>
@@ -832,7 +839,8 @@ END</code></pre>
<p class="p">
- <strong class="ph b">Usage notes:</strong> Primarily for compatibility with code containing industry extensions to SQL.
+ <strong class="ph b">Usage notes:</strong> Primarily for compatibility with code containing industry extensions
+ to SQL.
</p>
diff --git a/docs/build/html/topics/impala_config.html b/docs/build/html/topics/impala_config.html
index 53af12b..9f97bb3 100644
--- a/docs/build/html/topics/impala_config.html
+++ b/docs/build/html/topics/impala_config.html
@@ -9,12 +9,10 @@
<meta name="DC.Type" content="concept" />
<meta name="DC.Title" content="Managing Impala" />
<meta name="DC.Relation" scheme="URI" content="../topics/impala_config_performance.html" />
-<meta name="DC.Relation" scheme="URI" content="../topics/impala_odbc.html" />
-<meta name="DC.Relation" scheme="URI" content="../topics/impala_jdbc.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="config" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -82,10 +80,6 @@
<ul class="ullinks">
<li class="link ulchildlink"><strong><a href="../topics/impala_config_performance.html">Post-Installation Configuration for Impala</a></strong><br />
</li>
-<li class="link ulchildlink"><strong><a href="../topics/impala_odbc.html">Configuring Impala to Work with ODBC</a></strong><br />
-</li>
-<li class="link ulchildlink"><strong><a href="../topics/impala_jdbc.html">Configuring Impala to Work with JDBC</a></strong><br />
-</li>
</ul>
</div></body>
</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_config_options.html b/docs/build/html/topics/impala_config_options.html
index ff3c892..d6a5b82 100644
--- a/docs/build/html/topics/impala_config_options.html
+++ b/docs/build/html/topics/impala_config_options.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_processes.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="config_options" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -29,10 +29,9 @@
<div class="body conbody">
<p class="p">
- The configuration options for the Impala daemons let you choose which
- hosts and ports to use for the services that run on a single host, specify
- directories for logging, control resource usage and security, and specify
- other aspects of the Impala software.
+ The configuration options for the Impala daemons let you choose which hosts and ports to
+ use for the services that run on a single host, specify directories for logging, control
+ resource usage and security, and specify other aspects of the Impala software.
</p>
@@ -54,8 +53,8 @@
<div class="body conbody">
<p class="p">
- The Impala server, statestore, and catalog services start up using values provided in a
- defaults file, <span class="ph filepath">/etc/default/impala</span>.
+ The Impala server, <code class="ph codeph">statestore</code>, and catalog services start up using
+ values provided in a defaults file, <span class="ph filepath">/etc/default/impala</span>.
</p>
@@ -63,8 +62,8 @@
This file includes information about many resources used by Impala. Most of the defaults
included in this file should be effective in most cases. For example, typically you
would not change the definition of the <code class="ph codeph">CLASSPATH</code> variable, but you
- would always set the address used by the statestore server. Some of the content you
- might modify includes:
+ would always set the address used by the <code class="ph codeph">statestore</code> server. Some of the
+ content you might modify includes:
</p>
@@ -99,7 +98,7 @@ Stopping Impala Server: [ OK ]
Starting Impala Server: [ OK ]</code></pre>
<p class="p">
- Restart the Impala statestore using the following commands:
+ Restart the Impala StateStore using the following commands:
</p>
@@ -108,7 +107,7 @@ Stopping Impala State Store Server: [ OK ]
Starting Impala State Store Server: [ OK ]</code></pre>
<p class="p">
- Restart the Impala catalog service using the following commands:
+ Restart the Impala Catalog Service using the following commands:
</p>
@@ -124,10 +123,11 @@ Starting Impala Catalog Server: [ OK ]</code></pre>
<ul class="ul">
<li class="li">
<p class="p">
- Statestore address. Where practical, put the statestore on a separate host not
- running the <span class="keyword cmdname">impalad</span> daemon. In that recommended configuration,
- the <span class="keyword cmdname">impalad</span> daemon cannot refer to the statestore server using
- the loopback address. If the statestore is hosted on a machine with an IP address of
+ StateStore address. Where practical, put the <code class="ph codeph">statestored</code> on a
+ separate host not running the <span class="keyword cmdname">impalad</span> daemon. In that recommended
+ configuration, the <span class="keyword cmdname">impalad</span> daemon cannot refer to the
+ <code class="ph codeph">statestored</code> server using the loopback address. If the
+ <code class="ph codeph">statestored</code> is hosted on a machine with an IP address of
192.168.0.27, change:
</p>
@@ -143,8 +143,8 @@ Starting Impala Catalog Server: [ OK ]</code></pre>
<li class="li">
<p class="p">
Catalog server address (including both the hostname and the port number). Update the
- value of the <code class="ph codeph">IMPALA_CATALOG_SERVICE_HOST</code> variable. Where
- practical, run the catalog server on the same host as the statestore. In that
+ value of the <code class="ph codeph">IMPALA_CATALOG_SERVICE_HOST</code> variable. Where practical,
+ run the catalog server on the same host as the <code class="ph codeph">statestore</code>. In that
recommended configuration, the <span class="keyword cmdname">impalad</span> daemon cannot refer to the
catalog server using the loopback address. If the catalog service is hosted on a
machine with an IP address of 192.168.0.27, add the following line:
@@ -156,9 +156,9 @@ Starting Impala Catalog Server: [ OK ]</code></pre>
an <code class="ph codeph">IMPALA_CATALOG_ARGS</code> environment variable, but if you add one it
will be recognized by the service startup/shutdown script. Add a definition for this
variable to <span class="ph filepath">/etc/default/impala</span> and add the option
- <code class="ph codeph">-catalog_service_host=<var class="keyword varname">hostname</var></code>. If the port is
- different than the default 26000, also add the option
- <code class="ph codeph">-catalog_service_port=<var class="keyword varname">port</var></code>.
+ <code class="ph codeph">‑‑catalog_service_host=<var class="keyword varname">hostname</var></code>. If
+ the port is different than the default 26000, also add the option
+ <code class="ph codeph">‑‑catalog_service_port=<var class="keyword varname">port</var></code>.
</p>
</li>
@@ -191,7 +191,7 @@ Starting Impala Catalog Server: [ OK ]</code></pre>
</p>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
Queries that exceed the specified memory limit are aborted. Percentage limits are
based on the physical memory of the machine and do not consider cgroups.
</div>
@@ -200,7 +200,9 @@ Starting Impala Catalog Server: [ OK ]</code></pre>
<li class="li">
- <p class="p"> Core dump enablement. To enable core dumps, change: </p>
+ <p class="p">
+ Core dump enablement. To enable core dumps, change:
+ </p>
<pre class="pre codeblock"><code>export ENABLE_CORE_DUMPS=${ENABLE_COREDUMPS:-false}</code></pre>
<p class="p">
@@ -208,24 +210,26 @@ Starting Impala Catalog Server: [ OK ]</code></pre>
</p>
<pre class="pre codeblock"><code>export ENABLE_CORE_DUMPS=${ENABLE_COREDUMPS:-true}</code></pre>
+ <div class="note note"><span class="notetitle">Note:</span>
+ <ul class="ul">
+ <li class="li">
+ <p class="p">
+ The location of core dump files may vary according to your operating system
+ configuration.
+ </p>
- <div class="note note"><span class="notetitle">Note:</span>
- <ul class="ul">
- <li class="li">
- <p class="p">
- The location of core dump files may vary according to your operating system configuration.
- </p>
+ </li>
- </li>
- <li class="li">
- <p class="p">
- Other security settings may prevent Impala from writing core dumps even when this option is enabled.
- </p>
+ <li class="li">
+ <p class="p">
+ Other security settings may prevent Impala from writing core dumps even when this
+ option is enabled.
+ </p>
- </li>
+ </li>
- </ul>
+ </ul>
</div>
@@ -234,12 +238,11 @@ Starting Impala Catalog Server: [ OK ]</code></pre>
<li class="li">
<p class="p">
- Authorization using the open source Sentry plugin. Specify the
- <code class="ph codeph">-server_name</code> and <code class="ph codeph">-authorization_policy_file</code>
- options as part of the <code class="ph codeph">IMPALA_SERVER_ARGS</code> and
- <code class="ph codeph">IMPALA_STATE_STORE_ARGS</code> settings to enable the core Impala support
- for authentication. See <a class="xref" href="impala_authorization.html#secure_startup">Starting the impalad Daemon with Sentry Authorization Enabled</a> for
- details.
+ Authorization. Specify the
+ <code class="ph codeph">‑‑server_name</code> option as part of the
+ <code class="ph codeph">IMPALA_SERVER_ARGS</code> and
+ <code class="ph codeph">IMPALA_CATALOG_ARGS</code> settings to enable the core
+ Impala support for authorization. See <a class="xref" href="impala_authorization.html#secure_startup">Starting Impala with Sentry Authorization Enabled</a> for details.
</p>
</li>
@@ -248,12 +251,14 @@ Starting Impala Catalog Server: [ OK ]</code></pre>
<li class="li">
<p class="p">
Auditing for successful or blocked Impala queries, another aspect of security.
- Specify the <code class="ph codeph">-audit_event_log_dir=<var class="keyword varname">directory_path</var></code>
+ Specify the
+ <code class="ph codeph">‑‑audit_event_log_dir=<var class="keyword varname">directory_path</var></code>
option and optionally the
- <code class="ph codeph">-max_audit_event_log_file_size=<var class="keyword varname">number_of_queries</var></code>
- and <code class="ph codeph">-abort_on_failed_audit_event</code> options as part of the
- <code class="ph codeph">IMPALA_SERVER_ARGS</code> settings, for each Impala node, to enable and
- customize auditing. See <a class="xref" href="impala_auditing.html#auditing">Auditing Impala Operations</a> for details.
+ <code class="ph codeph">‑‑max_audit_event_log_file_size=<var class="keyword varname">number_of_queries</var></code>
+ and <code class="ph codeph">‑‑abort_on_failed_audit_event</code> options as part of
+ the <code class="ph codeph">IMPALA_SERVER_ARGS</code> settings, for each Impala node, to enable
+ and customize auditing. See
+ <a class="xref" href="impala_auditing.html#auditing">Auditing Impala Operations</a> for details.
</p>
</li>
@@ -263,11 +268,12 @@ Starting Impala Catalog Server: [ OK ]</code></pre>
<p class="p">
Password protection for the Impala web UI, which listens on port 25000 by default.
This feature involves adding some or all of the
- <code class="ph codeph">--webserver_password_file</code>,
- <code class="ph codeph">--webserver_authentication_domain</code>, and
- <code class="ph codeph">--webserver_certificate_file</code> options to the
+ <code class="ph codeph">‑‑webserver_password_file</code>,
+ <code class="ph codeph">‑‑webserver_authentication_domain</code>, and
+ <code class="ph codeph">‑‑webserver_certificate_file</code> options to the
<code class="ph codeph">IMPALA_SERVER_ARGS</code> and <code class="ph codeph">IMPALA_STATE_STORE_ARGS</code>
- settings. See <a class="xref" href="impala_security_guidelines.html#security_guidelines">Security Guidelines for Impala</a> for
+ settings. See
+ <a class="xref" href="impala_security_guidelines.html#security_guidelines">Security Guidelines for Impala</a> for
details.
</p>
@@ -278,11 +284,12 @@ Starting Impala Catalog Server: [ OK ]</code></pre>
<div class="p">
Another setting you might add to <code class="ph codeph">IMPALA_SERVER_ARGS</code> is a
comma-separated list of query options and values:
-<pre class="pre codeblock"><code>-default_query_options='<var class="keyword varname">option</var>=<var class="keyword varname">value</var>,<var class="keyword varname">option</var>=<var class="keyword varname">value</var>,...'
+<pre class="pre codeblock"><code>‑‑default_query_options='<var class="keyword varname">option</var>=<var class="keyword varname">value</var>,<var class="keyword varname">option</var>=<var class="keyword varname">value</var>,...'
</code></pre>
These options control the behavior of queries performed by this
<span class="keyword cmdname">impalad</span> instance. The option values you specify here override the
- default values for <a class="xref" href="impala_query_options.html#query_options">Impala query
+ default values for
+ <a class="xref" href="impala_query_options.html#query_options">Impala query
options</a>, as shown by the <code class="ph codeph">SET</code> statement in
<span class="keyword cmdname">impala-shell</span>.
</div>
@@ -292,9 +299,9 @@ Starting Impala Catalog Server: [ OK ]</code></pre>
<li class="li">
<p class="p">
- During troubleshooting, <span class="keyword">the appropriate support channel</span> might direct you to change other values,
- particularly for <code class="ph codeph">IMPALA_SERVER_ARGS</code>, to work around issues or
- gather debugging information.
+ During troubleshooting, <span class="keyword">the appropriate support channel</span> might direct you to change
+ other values, particularly for <code class="ph codeph">IMPALA_SERVER_ARGS</code>, to work around
+ issues or gather debugging information.
</p>
</li>
@@ -302,7 +309,7 @@ Starting Impala Catalog Server: [ OK ]</code></pre>
</ul>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
<p class="p">
These startup options for the <span class="keyword cmdname">impalad</span> daemon are different from the
command-line options for the <span class="keyword cmdname">impala-shell</span> command. For the
@@ -347,110 +354,50 @@ Starting Impala Catalog Server: [ OK ]</code></pre>
</div>
- <div class="topic concept nested1" aria-labelledby="ariaid-title4" id="config_options_impalad">
+ <div class="topic concept nested1" aria-labelledby="ariaid-title4" id="config_options_catalogd">
- <h2 class="title topictitle2" id="ariaid-title4">Startup Options for impalad Daemon</h2>
+ <h2 class="title topictitle2" id="ariaid-title4">Startup Options for catalogd Daemon</h2>
<div class="body conbody">
- <div class="p">
- The <code class="ph codeph">impalad</code> daemon implements the main Impala
- service, which performs query processing and reads from and writes to the data
- files. Some of the noteworthy options are:
- <ul class="ul">
- <li class="li">
- The <code class="ph codeph">fe_service_threads</code> option specifies the
- maximum number of concurrent client connections allowed. The
- default value is 64 with which 64 queries can run
- simultaneously.
- <p class="p">
- If you have more clients trying to connect to
- Impala than the value of this setting, the later arriving clients
- have to wait until previous clients disconnect. You can increase
- this value to allow more client connections. However, a large value
- means more threads to be maintained even if most of the connections
- are idle, and it could negatively impact query latency. Client
- applications should use the connection pool to avoid need for
- large number of sessions.
- </p>
-
- </li>
-
- </ul>
-</div>
-
-
- </div>
-
-
- </div>
-
-
- <div class="topic concept nested1" aria-labelledby="ariaid-title5" id="config_options_statestored">
-
- <h2 class="title topictitle2" id="ariaid-title5">Startup Options for statestored Daemon</h2>
-
-
- <div class="body conbody">
-
- <p class="p"> The <span class="keyword cmdname">statestored</span> daemon implements the Impala
- StateStore service, which monitors the availability of Impala services
- across the cluster, and handles situations such as nodes becoming
- unavailable or becoming available again. </p>
-
-
- </div>
-
-
- </div>
-
-
- <div class="topic concept nested1" aria-labelledby="ariaid-title6" id="config_options_catalogd">
-
- <h2 class="title topictitle2" id="ariaid-title6">Startup Options for catalogd Daemon</h2>
-
-
- <div class="body conbody">
-
- <p class="p"> The <span class="keyword cmdname">catalogd</span> daemon implements the Impala Catalog
- service, which broadcasts metadata changes to all the Impala nodes when
- Impala creates a table, inserts data, or performs other kinds of DDL and
- DML operations. </p>
+ <p class="p">
+ The <span class="keyword cmdname">catalogd</span> daemon implements the Impala Catalog service, which
+ broadcasts metadata changes to all the Impala nodes when Impala creates a table, inserts
+ data, or performs other kinds of DDL and DML operations.
+ </p>
<div class="p">
- Use <code class="ph codeph">--load_catalog_in_background</code> option to control when
- the metadata of a table is loaded.
+ Use <code class="ph codeph">‑‑load_catalog_in_background</code> option to control when the
+ metadata of a table is loaded.
<ul class="ul">
<li class="li">
- If set to <code class="ph codeph">false</code>, the metadata of a table is
- loaded when it is referenced for the first time. This means that the
- first run of a particular query can be slower than subsequent runs.
- Starting in Impala 2.2, the default for
- <code class="ph codeph">load_catalog_in_background</code> is
- <code class="ph codeph">false</code>.
+ If set to <code class="ph codeph">false</code>, the metadata of a table is loaded when it is
+ referenced for the first time. This means that the first run of a particular query
+ can be slower than subsequent runs. Starting in Impala 2.2, the default for
+ <code class="ph codeph">‑‑load_catalog_in_background</code> is <code class="ph codeph">false</code>.
</li>
+
<li class="li">
- If set to <code class="ph codeph">true</code>, the catalog service attempts to
- load metadata for a table even if no query needed that metadata. So
- metadata will possibly be already loaded when the first query that
- would need it is run. However, for the following reasons, we
- recommend not to set the option to <code class="ph codeph">true</code>.
+ If set to <code class="ph codeph">true</code>, the catalog service attempts to load metadata for a
+ table even if no query needed that metadata. So metadata will possibly be already
+ loaded when the first query that would need it is run. However, for the following
+ reasons, we recommend not to set the option to <code class="ph codeph">true</code>.
<ul class="ul">
<li class="li">
- Background load can interfere with query-specific metadata
- loading. This can happen on startup or after invalidating
- metadata, with a duration depending on the amount of metadata,
- and can lead to a seemingly random long running queries that are
- difficult to diagnose.
+ Background load can interfere with query-specific metadata loading. This can
+ happen on startup or after invalidating metadata, with a duration depending on
+ the amount of metadata, and can lead to a seemingly random long running queries
+ that are difficult to diagnose.
</li>
+
<li class="li">
- Impala may load metadata for tables that are possibly never
- used, potentially increasing catalog size and consequently memory
- usage for both catalog service and Impala Daemon.
+ Impala may load metadata for tables that are possibly never used, potentially
+ increasing catalog size and consequently memory usage for both catalog service
+ and Impala Daemon.
</li>
</ul>
@@ -469,4 +416,4 @@ Starting Impala Catalog Server: [ OK ]</code></pre>
</body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_config_performance.html b/docs/build/html/topics/impala_config_performance.html
index 696c3a6..3d4795f 100644
--- a/docs/build/html/topics/impala_config_performance.html
+++ b/docs/build/html/topics/impala_config_performance.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_config.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="config_performance" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
diff --git a/docs/build/html/topics/impala_connecting.html b/docs/build/html/topics/impala_connecting.html
index 8445a2d..46595e8 100644
--- a/docs/build/html/topics/impala_connecting.html
+++ b/docs/build/html/topics/impala_connecting.html
@@ -7,49 +7,45 @@
<meta name="copyright" content="(C) Copyright 2019" />
<meta name="DC.rights.owner" content="(C) Copyright 2019" />
<meta name="DC.Type" content="concept" />
-<meta name="DC.Title" content="Connecting to impalad through impala-shell" />
+<meta name="DC.Title" content="Connecting to Impala Daemon from impala-shell" />
<meta name="DC.Relation" scheme="URI" content="../topics/impala_impala_shell.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="connecting" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
-<title>Connecting to impalad through impala-shell</title>
+<title>Connecting to Impala Daemon from impala-shell</title>
</head>
<body id="connecting">
- <h1 class="title topictitle1" id="ariaid-title1">Connecting to impalad through impala-shell</h1>
+ <h1 class="title topictitle1" id="ariaid-title1">Connecting to Impala Daemon from impala-shell</h1>
<div class="body conbody">
+ <div class="p"> Within an <span class="keyword cmdname">impala-shell</span> session, you can only issue
+ queries while connected to an instance of the <span class="keyword cmdname">impalad</span>
+ daemon. You can specify the connection information: <ul class="ul">
+ <li class="li"> Through command-line options when you run the
+ <span class="keyword cmdname">impala-shell</span> command. </li>
+ <li class="li"> Through a configuration file that is read when you run the
+ <span class="keyword cmdname">impala-shell</span> command. </li>
- <div class="p">
- Within an <span class="keyword cmdname">impala-shell</span> session, you can only issue queries while connected to an instance
- of the <span class="keyword cmdname">impalad</span> daemon. You can specify the connection information:
- <ul class="ul">
- <li class="li">
- Through command-line options when you run the <span class="keyword cmdname">impala-shell</span> command.
- </li>
-
- <li class="li">
- Through a configuration file that is read when you run the <span class="keyword cmdname">impala-shell</span> command.
- </li>
-
- <li class="li">
- During an <span class="keyword cmdname">impala-shell</span> session, by issuing a <code class="ph codeph">CONNECT</code> command.
- </li>
+ <li class="li"> During an <span class="keyword cmdname">impala-shell</span> session, by issuing a
+ <code class="ph codeph">CONNECT</code> command. </li>
</ul>
-
- See <a class="xref" href="impala_shell_options.html">impala-shell Configuration Options</a> for the command-line and configuration file options you can use.
- </div>
+<div class="note note"><span class="notetitle">Note:</span> You cannot connect to the 3.2 or earlier versions of Impala
+ using the <code class="ph codeph">'hs2'</code> or <code class="ph codeph">'hs2-http'</code> protocol
+ (<code class="ph codeph">--protocol</code> option).</div>
+ See <a class="xref" href="impala_shell_options.html">impala-shell Configuration Options</a> for the command-line and configuration
+ file options you can use. </div>
<p class="p"> You can connect to any Impala daemon (<span class="keyword cmdname">impalad</span>), and
@@ -69,7 +65,7 @@
<p class="p">
- <strong class="ph b">To connect the Impala shell during shell startup:</strong>
+ <strong class="ph b">To connect to an Impala during shell startup:</strong>
</p>
@@ -81,25 +77,18 @@
also. </li>
- <li class="li">
- Use the <code class="ph codeph">-i</code> option to the
- <span class="keyword cmdname">impala-shell</span> interpreter to specify the connection information for
- that instance of <span class="keyword cmdname">impalad</span>:
-<pre class="pre codeblock"><code># When you are logged into the same machine running impalad.
+ <li class="li"> Use the <code class="ph codeph">-i</code> option to the
+ <span class="keyword cmdname">impala-shell</span> interpreter to specify the connection
+ information for that instance of <span class="keyword cmdname">impalad</span>:
+ <pre class="pre codeblock"><code># When you are connecting to an impalad running on the same machine.
# The prompt will reflect the current hostname.
$ impala-shell
-# When you are logged into the same machine running impalad.
-# The host will reflect the hostname 'localhost'.
-$ impala-shell -i localhost
-
-# When you are logged onto a different host, perhaps a client machine
-# outside the Hadoop cluster.
-$ impala-shell -i <var class="keyword varname">some.other.hostname</var>
-
-# When you are logged onto a different host, and impalad is listening
-# on a non-default port. Perhaps a load balancer is forwarding requests
-# to a different host/port combination behind the scenes.
+# When you are connecting to an impalad running on a remote machine, and impalad is listening
+# on a non-default port over the HTTP HiveServer2 protocol.
+$ impala-shell -i <var class="keyword varname">some.other.hostname</var>:<var class="keyword varname">port_number</var> --protocol='hs2-http'
+# When you are connecting to an impalad running on a remote machine, and impalad is listening
+# on a non-default port.
$ impala-shell -i <var class="keyword varname">some.other.hostname</var>:<var class="keyword varname">port_number</var>
</code></pre>
</li>
@@ -108,25 +97,14 @@ $ impala-shell -i <var class="keyword varname">some.other.hostname</var>:<var cl
<p class="p">
- <strong class="ph b">To connect the Impala shell after shell startup:</strong>
+ <strong class="ph b">To connect to an Impala in the<span class="keyword cmdname">impala-shell</span>
+ session:</strong>
</p>
<ol class="ol">
- <li class="li">
- Start the Impala shell with no connection:
-<pre class="pre codeblock"><code>$ impala-shell</code></pre>
- <p class="p">
- You should see a prompt like the following:
- </p>
-
-<pre class="pre codeblock"><code>Welcome to the Impala shell. Press TAB twice to see a list of available commands.
-...
-<span class="ph">(Shell
- build version: Impala Shell v3.2.x (<var class="keyword varname">hash</var>) built on
- <var class="keyword varname">date</var>)</span>
-[Not connected] > </code></pre>
- </li>
+ <li class="li"> Start the Impala shell with no connection:
+ <pre class="pre codeblock"><code>$ impala-shell</code></pre></li>
<li class="li"> Locate the hostname that is running the <span class="keyword cmdname">impalad</span>
@@ -136,10 +114,10 @@ $ impala-shell -i <var class="keyword varname">some.other.hostname</var>:<var cl
<li class="li"> Use the <code class="ph codeph">connect</code> command to connect to an Impala
- instance. Enter a command of the form: <pre class="pre codeblock"><code>[Not connected] > connect <var class="keyword varname">impalad-host</var>
-[<var class="keyword varname">impalad-host</var>:21000] ></code></pre>
- <div class="note note"><span class="notetitle">Note:</span> Replace <var class="keyword varname">impalad-host</var> with the hostname you
- have configured to run Impala in your environment. The changed prompt
+ instance. Enter a command of the form:
+ <pre class="pre codeblock"><code>[Not connected] > connect <var class="keyword varname">impalad-host</var></code></pre><div class="note note"><span class="notetitle">Note:</span>
+ Replace <var class="keyword varname">impalad-host</var> with the hostname you have
+ configured to run Impala in your environment. The changed prompt
indicates a successful connection. </div>
</li>
diff --git a/docs/build/html/topics/impala_conversion_functions.html b/docs/build/html/topics/impala_conversion_functions.html
index c159947..d9ae415 100644
--- a/docs/build/html/topics/impala_conversion_functions.html
+++ b/docs/build/html/topics/impala_conversion_functions.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_functions.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="conversion_functions" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -31,24 +31,16 @@
<div class="body conbody">
<p class="p">
- Conversion functions are usually used in combination with other functions, to explicitly
+ Conversion functions are typically used in combination with other functions to explicitly
pass the expected data types. Impala has strict rules regarding data types for function
parameters. For example, Impala does not automatically convert a <code class="ph codeph">DOUBLE</code>
value to <code class="ph codeph">FLOAT</code>, a <code class="ph codeph">BIGINT</code> value to <code class="ph codeph">INT</code>,
or other conversion where precision could be lost or overflow could occur. Also, for
- reporting or dealing with loosely defined schemas in big data contexts, you might
- frequently need to convert values to or from the <code class="ph codeph">STRING</code> type.
+ reporting or dealing with loosely defined schemas in big data contexts, you might need to
+ convert values to or from the <code class="ph codeph">STRING</code> type.
</p>
- <div class="note note"><span class="notetitle">Note:</span>
- Although in <span class="keyword">Impala 2.3</span>, the <code class="ph codeph">SHOW FUNCTIONS</code> output
- for database <code class="ph codeph">_IMPALA_BUILTINS</code> contains some function signatures matching
- the pattern <code class="ph codeph">castto*</code>, these functions are not intended for public use and
- are expected to be hidden in future.
- </div>
-
-
<p class="p">
<strong class="ph b">Function reference:</strong>
</p>
@@ -73,65 +65,68 @@
<dl class="dl">
-
+
<dt class="dt dlterm" id="conversion_functions__cast">
- CAST(expr AS type)
+ CAST(expression AS type)
</dt>
<dd class="dd">
- <strong class="ph b">Purpose:</strong> Converts the value of an expression to any other type. If the
- expression value is of a type that cannot be converted to the target type, the result
- is <code class="ph codeph">NULL</code>.
+ <strong class="ph b">Purpose:</strong> Returns <var class="keyword varname">expression</var> converted to the
+ <var class="keyword varname">type</var> data type.
+ <div class="p">
+ If the <var class="keyword varname">expression</var> value is of a type that cannot be converted to
+ the target <var class="keyword varname">type</var>:
+ <ul class="ul">
+ <li class="li">
+ Of <code class="ph codeph">DECIMAL</code>, <code class="ph codeph">DATE</code>, and
+ <code class="ph codeph">BOOLEAN</code>, the function returns an error.
+ </li>
+
+
+ <li class="li">
+ Of all other types, the function returns <code class="ph codeph">NULL</code>.
+ </li>
+
+ </ul>
+
+ </div>
+
+
<p class="p">
- <strong class="ph b">Usage notes:</strong> Use <code class="ph codeph">CAST</code> when passing a column value or literal
- to a function that expects a parameter with a different type. Frequently used in SQL
- operations such as <code class="ph codeph">CREATE TABLE AS SELECT</code> and <code class="ph codeph">INSERT ...
- VALUES</code> to ensure that values from various sources are of the appropriate
- type for the destination columns. Where practical, do a one-time
- <code class="ph codeph">CAST()</code> operation during the ingestion process to make each column
- into the appropriate type, rather than using many <code class="ph codeph">CAST()</code> operations
- in each query; doing type conversions for each row during each query can be
- expensive for tables with millions or billions of rows.
+ <strong class="ph b">Usage notes:</strong>
</p>
<p class="p">
- The way this function deals with time zones when converting to or from <code class="ph codeph">TIMESTAMP</code>
- values is affected by the <code class="ph codeph">--use_local_tz_for_unix_timestamp_conversions</code> startup flag for the
- <span class="keyword cmdname">impalad</span> daemon. See <a class="xref" href="../shared/../topics/impala_timestamp.html#timestamp">TIMESTAMP Data Type</a> for details about
- how Impala handles time zone considerations for the <code class="ph codeph">TIMESTAMP</code> data type.
- </p>
+ Use <code class="ph codeph">CAST</code> when passing a column value or literal to a function that
+ expects a parameter with a different type. Frequently used in SQL operations such as
+ <code class="ph codeph">CREATE TABLE AS SELECT</code> and <code class="ph codeph">INSERT ... VALUES</code> to
+ ensure that values from various sources are of the appropriate type for the
+ destination columns.
+ </p>
<p class="p">
- <strong class="ph b">Examples:</strong>
- </p>
+ Where practical, do a one-time <code class="ph codeph">CAST()</code> operation during the
+ ingestion process to make each column into the appropriate type, rather than using
+ many <code class="ph codeph">CAST()</code> operations in each query; doing type conversions for
+ each row during each query can be expensive for tables with millions or billions of
+ rows.
+ </p>
+
-<pre class="pre codeblock"><code>SELECT CONCAT('Here are the first ',10,' results.'); -- Fails
-SELECT CONCAT('Here are the first ',CAST(10 AS STRING),' results.'); -- Succeeds
-</code></pre>
<p class="p">
- The following example starts with a text table where every column has a type of
- <code class="ph codeph">STRING</code>, which might be how you ingest data of unknown schema until
- you can verify the cleanliness of the underlying values. Then it uses
- <code class="ph codeph">CAST()</code> to create a new Parquet table with the same data, but using
- specific numeric data types for the columns with numeric data. Using numeric types
- of appropriate sizes can result in substantial space savings on disk and in memory,
- and performance improvements in queries, over using strings or larger-than-necessary
- numeric types.
- </p>
+ The way this function deals with time zones when converting to or from
+ <code class="ph codeph">TIMESTAMP</code> values is affected by the
+ <code class="ph codeph">‑‑use_local_tz_for_unix_timestamp_conversions</code> startup flag
+ for the <span class="keyword cmdname">impalad</span> daemon. See
+ <a class="xref" href="../shared/../topics/impala_timestamp.html#timestamp">TIMESTAMP Data Type</a> for details about how
+ Impala handles time zone considerations for the <code class="ph codeph">TIMESTAMP</code> data type.
+ </p>
-<pre class="pre codeblock"><code>CREATE TABLE t1 (name STRING, x STRING, y STRING, z STRING);
-CREATE TABLE t2 STORED AS PARQUET
-AS SELECT
- name,
- CAST(x AS BIGINT) x,
- CAST(y AS TIMESTAMP) y,
- CAST(z AS SMALLINT) z
-FROM t1;</code></pre>
<p class="p">
<strong class="ph b">Related information:</strong>
</p>
@@ -145,10 +140,10 @@ FROM t1;</code></pre>
<a class="xref" href="impala_bigint.html#bigint">BIGINT Data Type</a>,
<a class="xref" href="impala_float.html#float">FLOAT Data Type</a>,
<a class="xref" href="impala_double.html#double">DOUBLE Data Type</a>,
- <a class="xref" href="impala_decimal.html#decimal">DECIMAL Data Type (Impala 3.0 or higher only)</a>,
+ <a class="xref" href="impala_decimal.html#decimal">DECIMAL Data Type</a>,
<a class="xref" href="impala_string.html#string">STRING Data Type</a>,
- <a class="xref" href="impala_char.html#char">CHAR Data Type (Impala 2.0 or higher only)</a>,
- <a class="xref" href="impala_varchar.html#varchar">VARCHAR Data Type (Impala 2.0 or higher only)</a>,
+ <a class="xref" href="impala_char.html#char">CHAR Data Type</a>,
+ <a class="xref" href="impala_varchar.html#varchar">VARCHAR Data Type</a>,
<a class="xref" href="impala_timestamp.html#timestamp">TIMESTAMP Data Type</a>,
<a class="xref" href="impala_boolean.html#boolean">BOOLEAN Data Type</a>
</p>
@@ -156,21 +151,860 @@ FROM t1;</code></pre>
</dd>
+
+
+
+
+ <dt class="dt dlterm" id="conversion_functions__cast_format">
+ CAST(expression AS type FORMAT pattern)
+ </dt>
+
+
+ <dd class="dd">
+ <strong class="ph b">Purpose:</strong> Returns <var class="keyword varname">expression</var> converted to the
+ <var class="keyword varname">type</var> data type based on the <var class="keyword varname">pattern</var> format
+ string. This signature of <code class="ph codeph">CAST()</code> with the <code class="ph codeph">FORMAT</code>
+ clause is supported only for casts between <code class="ph codeph">STRING</code> /
+ <code class="ph codeph">CHAR</code> / <code class="ph codeph">VARCHAR</code> types and <code class="ph codeph">TIMESTAMP</code>
+ / <code class="ph codeph">DATE</code> types.
+ <div class="note note"><span class="notetitle">Note:</span>
+ The patterns allowed in the <code class="ph codeph">FORMAT</code> clause support ISO SQL:2016
+ standard patterns. Those patterns are not the same format patterns used with the
+ other Impala conversion functions, e.g. <code class="ph codeph">TO_TIMESTAMP()</code> and
+ <code class="ph codeph">FROM_TIMESTAMP()</code>.
+ </div>
+
+
+ <div class="p">
+ The following rules apply to <var class="keyword varname">pattern</var>. Any exceptions to these
+ rules are noted in the Details column of the table below.
+ <ul class="ul">
+ <li class="li">
+ <var class="keyword varname">pattern</var> is a case-insensitive <code class="ph codeph">STRING</code>.
+ </li>
+
+
+ <li class="li">
+ If <var class="keyword varname">pattern</var> is <code class="ph codeph">NULL</code>, an empty string, or a
+ number, an error returns.
+ </li>
+
+
+ <li class="li">
+ A fewer digits in <var class="keyword varname">expression</var> than specified in the
+ <var class="keyword varname">pattern</var> is accepted if a separator is correctly specified in
+ the <var class="keyword varname">pattern</var>. For example, <code class="ph codeph">CAST('5-01-2017' AS DATE
+ FORMAT 'MM-dd-yyyy')</code> returns <code class="ph codeph">DATE'2017-05-01'</code>.
+ </li>
+
+
+ <li class="li">
+ If fewer number of digits are in <var class="keyword varname">expression</var> than specified in
+ the <var class="keyword varname">pattern</var>, the current date is used to complete the year
+ pattern. For example, <code class="ph codeph">CAST('19/05' AS DATE FORMAT 'YYYY/MM')</code>
+ will return <code class="ph codeph">DATE'2019-05-01'</code> when executed on August 8, 2019.
+ </li>
+
+ </ul>
+
+ </div>
+
+
+ <div class="p">
+ The following format patterns are supported in the <code class="ph codeph">FORMAT</code> clause.
+
+<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="conversion_functions__table_gbt_5ym_r3b" class="table" frame="border" border="1" rules="all"><colgroup><col style="width:10.695187165775401%" /><col style="width:18.71657754010695%" /><col style="width:70.58823529411764%" /></colgroup><thead class="thead" style="text-align:left;">
+ <tr class="row">
+ <th class="entry cellrowborder" style="text-align:left;vertical-align:top;" id="d39549e394">
+ Pattern
+ </th>
+
+ <th class="entry cellrowborder" style="text-align:left;vertical-align:top;" id="d39549e397">
+ Description
+ </th>
+
+ <th class="entry cellrowborder" style="text-align:left;vertical-align:top;" id="d39549e400">
+ Details
+ </th>
+
+ </tr>
+
+ </thead>
+<tbody class="tbody">
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e394 ">
+ <code class="ph codeph">YYYY</code>
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e397 ">
+ 4-digit year.
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e400 "> </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e394 ">
+ <code class="ph codeph">YYY</code>
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e397 ">
+ Last 3 digits of a year.
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e400 "> </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e394 ">
+ <code class="ph codeph">YY</code>
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e397 ">
+ Last 2 digits of a year.
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e400 "> </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e394 ">
+ <code class="ph codeph">Y</code>
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e397 ">
+ Last digit of a year
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e400 "> </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e394 ">
+ <code class="ph codeph">RRRR</code>
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e397 ">
+ 4-digit round year
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e400 ">
+ <p class="p">
+ If 1,3 or 4-digit year values are provided in
+ <var class="keyword varname">expression</var>, treated as <code class="ph codeph">YYYY</code>.
+ </p>
+
+
+
+
+ <p class="p">
+ If 2-digit years are provided in <var class="keyword varname">expression</var>, treated
+ as <code class="ph codeph">RR</code>.
+ </p>
+
+
+
+
+ <p class="p">
+ For datetime to string conversions, treated as <code class="ph codeph">YYYY</code>.
+ </p>
+
+
+
+
+ <p class="p">
+ If <code class="ph codeph">YYYY</code>, <code class="ph codeph">YYY</code>, <code class="ph codeph">YY</code>,
+ <code class="ph codeph">Y</code>, or <code class="ph codeph">RR</code> is given in the same
+ <var class="keyword varname">pattern</var> for a string to datetime conversion, an error
+ returns.
+ </p>
+
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e394 ">
+ <code class="ph codeph">RR</code>
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e397 ">
+ 2-digit round year.
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e400 ">
+ <div class="p">
+ <ul class="ul">
+ <li class="li">
+ For datetime to string conversion, same as <code class="ph codeph">YY</code>.
+ </li>
+
+
+ <li class="li">
+ For string to datetime conversions, the first 2 digits of the year
+ in the return value depends on the specified two-digit year and the
+ last two digits of the current year as follows:
+ <ul class="ul">
+ <li class="li">
+ <p class="p">
+ If the specified 2-digit year is 00 to 49:
+ </p>
+
+ <ul class="ul">
+ <li class="li">
+ <p class="p">
+ If the last 2 digits of the current year are 00 to 49, the
+ returned year has the same first 2 digits as the current
+ year.
+ </p>
+
+ </li>
+
+
+ <li class="li">
+ <p class="p">
+ If the last 2 digits of the current year are 50 to 99, the
+ first 2 digits of the returned year are 1 greater than the
+ first 2 digits of the current year.
+ </p>
+
+ </li>
+
+ </ul>
+
+ </li>
+
+
+ <li class="li">
+ <p class="p">
+ If the specified 2-digit year is 50 to 99:
+ </p>
+
+ <ul class="ul">
+ <li class="li">
+ <p class="p">
+ If the last 2 digits of the current year are 00 to 49, the
+ first 2 digits of the returned year are 1 less than the
+ first 2 digits of the current year.
+ </p>
+
+ </li>
+
+
+ <li class="li">
+ <p class="p">
+ If the last 2 digits of the current year are 50 to 99, the
+ returned year has the same first 2 digits as the current
+ year.
+ </p>
+
+ </li>
+
+ </ul>
+
+ </li>
+
+ </ul>
+
+ </li>
+
+ </ul>
+
+ </div>
+
+
+
+
+ <p class="p">
+ If <code class="ph codeph">YYYY</code>, <code class="ph codeph">YYY</code>, <code class="ph codeph">YY</code>,
+ <code class="ph codeph">Y</code>, or <code class="ph codeph">RR</code> is given in the same
+ <var class="keyword varname">pattern</var> for a string to datetime conversion, an error
+ returns.
+ </p>
+
+
+
+
+ <p class="p">
+ If 1-digit year values are provided in <var class="keyword varname">expression</var>, it
+ is treated as <code class="ph codeph">YYYY</code>.
+ </p>
+
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e394 ">
+ <code class="ph codeph">MM</code>
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e397 ">
+ Month
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e400 ">
+ <p class="p">
+ In datetime to string conversions, 1-digit month is prefixed with a
+ zero.
+ </p>
+
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e394 ">
+ <code class="ph codeph">DD</code>
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e397 ">
+ Day of month (1-31)
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e400 ">
+ <p class="p">
+ In datetime to string conversions, one digit day is prefixed with a
+ zero.
+ </p>
+
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e394 ">
+ <code class="ph codeph">DDD</code>
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e397 ">
+ Day of year (1-366)
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e400 ">
+ <p class="p">
+ In string to datetime conversions, providing <code class="ph codeph">MM</code> and
+ <code class="ph codeph">DD</code> along with <code class="ph codeph">DDD</code> results an error,
+ e.g. <code class="ph codeph">YYYY-MM-DDD</code>.
+ </p>
+
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e394 ">
+ <code class="ph codeph">HH</code>
+
+ <p class="p">
+ <code class="ph codeph">HH12</code>
+ </p>
+
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e397 ">
+ Hour of day (1-12)
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e400 ">
+ <p class="p">
+ In datetime to string conversions, 1-digit hours are prefixed with a
+ zero.
+ </p>
+
+
+
+
+ <p class="p">
+ If provided hour in <var class="keyword varname">expression</var> is not between 1 and
+ 12, returns an error.
+ </p>
+
+
+
+
+ <p class="p">
+ If no AM/PM is provided in <var class="keyword varname">expression</var>, the default is
+ <code class="ph codeph">AM</code>.
+ </p>
+
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e394 ">
+ <code class="ph codeph">HH24</code>
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e397 ">
+ Hour of day (0-23)
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e400 ">
+ <p class="p">
+ In string to datetime conversions, if <code class="ph codeph">HH12</code>,
+ <code class="ph codeph">AM</code>, <code class="ph codeph">PM</code> are given in the same
+ <var class="keyword varname">pattern</var>, an error returns.
+ </p>
+
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e394 ">
+ <code class="ph codeph">MI</code>
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e397 ">
+ Minute of hour (0-59)
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e400 ">
+ <p class="p">
+ In datetime to string conversions, 1-digit minutes are prefixed with a
+ zero.
+ </p>
+
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e394 ">
+ <code class="ph codeph">SS</code>
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e397 ">
+ Second of minute (0-59)
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e400 ">
+ <p class="p">
+ In datetime to string conversions, 1-digit seconds are prefixed with a
+ zero.
+ </p>
+
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e394 ">
+ <code class="ph codeph">SSSSS</code>
+ </td>
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e397 ">
+ Second of Day (0-86399)
+ </td>
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e400 ">
+ In string to timestamp conversions, if <code class="ph codeph">SS</code>,
+ <code class="ph codeph">HH</code>, <code class="ph codeph">HH12</code>, <code class="ph codeph">HH24</code>,
+ <code class="ph codeph">MI</code>, <code class="ph codeph">AM</code>/<code class="ph codeph">PM</code> are given in
+ the same <var class="keyword varname">pattern</var>, an error returns.
+ </td>
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e394 ">
+ <code class="ph codeph">FF</code>
+
+ <p class="p">
+ <code class="ph codeph">FF1</code>, ..., <code class="ph codeph">FF9</code>
+ </p>
+
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e397 ">
+ Fractional second
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e400 ">
+ <p class="p">
+ A number, 1 to 9, can be used to indicate the number of digits.
+ </p>
+
+
+
+
+ <p class="p">
+ <code class="ph codeph">FF</code> specifies a 9 digits fractional second.
+ </p>
+
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e394 ">
+ <code class="ph codeph">AM</code>
+
+ <p class="p">
+ <code class="ph codeph">PM</code>
+ </p>
+
+
+
+
+ <p class="p">
+ <code class="ph codeph">A.M.</code>
+ </p>
+
+
+
+
+ <p class="p">
+ <code class="ph codeph">P.M.</code>
+ </p>
+
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e397 ">
+ Meridiem indicator
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e400 ">
+ <p class="p">
+ For datetime to string conversions, <code class="ph codeph">AM</code> and
+ <code class="ph codeph">PM</code> are treated as synonyms. For example, casting
+ <code class="ph codeph">'2019-01-01 11:00 am'</code> to <code class="ph codeph">TIMESTAMP</code>
+ with the <code class="ph codeph">'YYYY-MM-DD HH12:MI PM'</code> pattern returns
+ <code class="ph codeph">01-JAN-19 11.00.00.000000 AM</code>.
+ </p>
+
+
+
+
+ <p class="p">
+ For string to datetime conversion, <code class="ph codeph">HH24</code> in the same
+ <var class="keyword varname">pattern</var> returns an error.
+ </p>
+
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e394 ">
+ <code class="ph codeph">TZH</code>
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e397 ">
+ Timezone offset hour
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e400 ">
+ <p class="p">
+ An optional sign, + or -, and 2 digits for the value of signed numbers
+ are allowed for the source <var class="keyword varname">expression</var>, e.g.
+ <code class="ph codeph">“+10”</code>, <code class="ph codeph">“-05”</code>,
+ <code class="ph codeph">"04"</code>.
+ </p>
+
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e394 ">
+ <code class="ph codeph">TZM</code>
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e397 ">
+ Timezone offset minute
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e400 ">
+ <p class="p">
+ Unsigned numbers are allowed for the source
+ <var class="keyword varname">expression</var>.
+ </p>
+
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e394 ">
+ <code class="ph codeph">-</code>
+
+ <p class="p">
+ <code class="ph codeph">.</code>
+ </p>
+
+
+
+
+ <p class="p">
+ <code class="ph codeph">/</code>
+ </p>
+
+
+
+
+ <p class="p">
+ <code class="ph codeph">,</code>
+ </p>
+
+
+
+
+ <p class="p">
+ <code class="ph codeph">'</code>
+ </p>
+
+
+
+
+ <p class="p">
+ <code class="ph codeph">;</code>
+ </p>
+
+
+
+
+ <p class="p">
+ <code class="ph codeph">:</code>
+ </p>
+
+
+
+
+ <p class="p">
+ <space>
+ </p>
+
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e397 ">
+ Separator
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e400 ">
+ <p class="p">
+ For string to datetime conversions, any separator character in the
+ <var class="keyword varname">pattern</var> string would match any separator character in
+ the input <var class="keyword varname">expression</var>.
+ </p>
+
+
+
+
+ <p class="p">
+ For example, <code class="ph codeph">CAST(“20191010” AS DATE FORMAT
+ “YYYY-MM-DD”)</code> returns an error, but <code class="ph codeph">CAST("2019-.;10
+ 10" AS DATE FORMAT "YYYY-MM-DD")</code> succeeds.
+ </p>
+
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e394 ">
+ <code class="ph codeph">T</code>
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e397 ">
+ Separates the date from the time.
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e400 ">
+ This pattern is used for accepting ISO 8601 datetime formats.
+
+ <p class="p">
+ Example: <code class="ph codeph">YYYY-MM-DD<strong class="ph b">T</strong>HH24:MI:SS.FF9Z</code>
+ </p>
+
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e394 ">
+ <code class="ph codeph">Z</code>
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e397 ">
+ Indicates the zero hour offset from UTC.
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e400 ">
+ This pattern is used for accepting ISO 8601 datetime formats.
+ </td>
+
+ </tr>
+
+ </tbody>
+</table>
+</div>
+
+ </div>
+
+
+ <p class="p">
+ <strong class="ph b">Examples:</strong>
+ </p>
+
+
+<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="conversion_functions__table_qqg_txn_r3b" class="table" frame="border" border="1" rules="all"><colgroup><col style="width:50%" /><col style="width:50%" /></colgroup><thead class="thead" style="text-align:left;">
+ <tr class="row">
+ <th class="entry cellrowborder" style="text-align:left;vertical-align:top;" id="d39549e1126">
+ Input
+ </th>
+
+ <th class="entry cellrowborder" style="text-align:left;vertical-align:top;" id="d39549e1129">
+ Output
+ </th>
+
+ </tr>
+
+ </thead>
+<tbody class="tbody">
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e1126 ">
+ <code class="ph codeph">CAST("02-11-2014" AS DATE FORMAT 'dd-mm-yyyy')</code>
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e1129 ">
+ 2014-11-02
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e1126 ">
+ <code class="ph codeph">CAST("365 2014" AS DATE FORMAT 'DDD-YYYY')</code>
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e1129 ">
+ 2014-12-31
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e1126 ">
+ <code class="ph codeph">CAST("5-01-26" AS DATE FORMAT 'YY-MM-DD')</code>
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e1129 ">
+ Executed at 2019-01-01 11:11:11:
+
+ <p class="p">
+ 2015-01-26
+ </p>
+
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e1126 ">
+ <code class="ph codeph">CAST('2018-11-10T15:11:04Z' AS TIMESTAMP</code>
+
+ <p class="p">
+ <code class="ph codeph">FORMAT 'YYYY-MM-DDTHH24:MI:SSZ')</code>
+ </p>
+
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e1129 ">
+ 2018-11-10 15:11:04
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e1126 ">
+ <code class="ph codeph">CAST("95-01-28" AS DATE FORMAT 'YYY-MM-DD')</code>
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e1129 ">
+ Executed at 2019-01-01 11:11:11:
+
+ <p class="p">
+ 2095-01-28
+ </p>
+
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e1126 ">
+ <code class="ph codeph">CAST("49-01-15" AS DATE FORMAT 'RR-MM-DD')</code>
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e1129 ">
+ Round year when last 2 digits of current year is greater than 49.
+
+ <p class="p">
+ Executed at 2099-01-01 11:11:11:
+ </p>
+
+
+
+
+ <p class="p">
+ 2149-01-15
+ </p>
+
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e1126 ">
+ <code class="ph codeph">CAST('2019.10.10 13:30:40.123456 +01:30' </code>
+
+ <p class="p">
+ <code class="ph codeph">AS TIMESTAMP</code>
+ </p>
+
+
+
+
+ <p class="p">
+ <code class="ph codeph">FORMAT 'YYYY-MM-DD HH24:MI:SS.FF9 TZH:TZM')</code>
+ </p>
+
+ </td>
+
+ <td class="entry cellrowborder" style="text-align:left;vertical-align:top;" headers="d39549e1129 ">
+ 2019-10-10 13:30:40.123456000
+ </td>
+
+ </tr>
+
+ </tbody>
+</table>
+</div>
+
+ </dd>
+
+
+
+
+
<dt class="dt dlterm" id="conversion_functions__typeof">
- TYPEOF(type value)
+ TYPEOF(expression)
</dt>
<dd class="dd">
- <strong class="ph b">Purpose:</strong> Returns the name of the data type corresponding to an expression. For
- types with extra attributes, such as length for <code class="ph codeph">CHAR</code> and
- <code class="ph codeph">VARCHAR</code>, or precision and scale for <code class="ph codeph">DECIMAL</code>,
- includes the full specification of the type.
-
+ <strong class="ph b">Purpose:</strong> Returns the name of the data type corresponding to
+ <var class="keyword varname">expression</var>. For types with extra attributes, such as length for
+ <code class="ph codeph">CHAR</code> and <code class="ph codeph">VARCHAR</code>, or precision and scale for
+ <code class="ph codeph">DECIMAL</code>, includes the full specification of the type.
<p class="p">
<strong class="ph b">Return type:</strong> <code class="ph codeph">STRING</code>
</p>
@@ -188,25 +1022,23 @@ FROM t1;</code></pre>
<p class="p">
- <strong class="ph b">Added in:</strong> <span class="keyword">Impala 2.3.0</span>
+ <strong class="ph b">Examples:</strong>
</p>
<p class="p">
- <strong class="ph b">Examples:</strong>
- </p>
+ <code class="ph codeph">TYPEOF(2)</code> returns <code class="ph codeph">TINYINT</code>.
+ </p>
-<pre class="pre codeblock"><code>SELECT TYPEOF(2), TYPEOF(2+2);
-+-----------+---------------+
-| typeof(2) | typeof(2 + 2) |
-+-----------+---------------+
-| TINYINT | SMALLINT |
-+-----------+---------------+
-</code></pre>
- </dd>
+ <p class="p">
+ <code class="ph codeph">TYPEOF(NOW())</code> returns <code class="ph codeph">TIMESTAMP</code>.
+ </p>
+
+ </dd>
+
</dl>
@@ -218,4 +1050,4 @@ FROM t1;</code></pre>
<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_functions.html">Impala Built-In Functions</a></div>
</div>
</div></body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_count.html b/docs/build/html/topics/impala_count.html
index 5564bb01..a18d9de 100644
--- a/docs/build/html/topics/impala_count.html
+++ b/docs/build/html/topics/impala_count.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_aggregate_functions.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="count" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -83,12 +83,13 @@
<p class="p">
- If you frequently run aggregate functions such as <code class="ph codeph">MIN()</code>, <code class="ph codeph">MAX()</code>, and
- <code class="ph codeph">COUNT(DISTINCT)</code> on partition key columns, consider enabling the <code class="ph codeph">OPTIMIZE_PARTITION_KEY_SCANS</code>
- query option, which optimizes such queries. This feature is available in <span class="keyword">Impala 2.5</span> and higher.
- See <a class="xref" href="../shared/../topics/impala_optimize_partition_key_scans.html">OPTIMIZE_PARTITION_KEY_SCANS Query Option (Impala 2.5 or higher only)</a>
- for the kinds of queries that this option applies to, and slight differences in how partitions are
- evaluated when this query option is enabled.
+ If you frequently run aggregate functions such as <code class="ph codeph">MIN()</code>,
+ <code class="ph codeph">MAX()</code>, and <code class="ph codeph">COUNT(DISTINCT)</code> on partition key columns,
+ consider enabling the <code class="ph codeph">OPTIMIZE_PARTITION_KEY_SCANS</code> query option, which
+ optimizes such queries. This feature is available in <span class="keyword">Impala 2.5</span>
+ and higher. See <a class="xref" href="../shared/../topics/impala_optimize_partition_key_scans.html">OPTIMIZE_PARTITION_KEY_SCANS Query Option (Impala 2.5 or higher only)</a> for the
+ kinds of queries that this option applies to, and slight differences in how partitions
+ are evaluated when this query option is enabled.
</p>
@@ -98,25 +99,25 @@
<p class="p">
- To access a column with a complex type (<code class="ph codeph">ARRAY</code>, <code class="ph codeph">STRUCT</code>, or <code class="ph codeph">MAP</code>)
- in an aggregation function, you unpack the individual elements using join notation in the query,
- and then apply the function to the final scalar item, field, key, or value at the bottom of any nested type hierarchy in the column.
- See <a class="xref" href="../shared/../topics/impala_complex_types.html#complex_types">Complex Types (Impala 2.3 or higher only)</a> for details about using complex types in Impala.
+ To access a column with a complex type (<code class="ph codeph">ARRAY</code>, <code class="ph codeph">STRUCT</code>,
+ or <code class="ph codeph">MAP</code>) in an aggregation function, you unpack the individual elements
+ using join notation in the query, and then apply the function to the final scalar item,
+ field, key, or value at the bottom of any nested type hierarchy in the column. See
+ <a class="xref" href="../shared/../topics/impala_complex_types.html#complex_types">Complex Types (Impala 2.3 or higher only)</a> for details about using
+ complex types in Impala.
</p>
<div class="p">
-The following example demonstrates calls to several aggregation functions
-using values from a column containing nested complex types
-(an <code class="ph codeph">ARRAY</code> of <code class="ph codeph">STRUCT</code> items).
-The array is unpacked inside the query using join notation.
-The array elements are referenced using the <code class="ph codeph">ITEM</code>
-pseudocolumn, and the structure fields inside the array elements
-are referenced using dot notation.
-Numeric values such as <code class="ph codeph">SUM()</code> and <code class="ph codeph">AVG()</code>
-are computed using the numeric <code class="ph codeph">R_NATIONKEY</code> field, and
-the general-purpose <code class="ph codeph">MAX()</code> and <code class="ph codeph">MIN()</code>
-values are computed from the string <code class="ph codeph">N_NAME</code> field.
+ The following example demonstrates calls to several aggregation functions using values
+ from a column containing nested complex types (an <code class="ph codeph">ARRAY</code> of
+ <code class="ph codeph">STRUCT</code> items). The array is unpacked inside the query using join
+ notation. The array elements are referenced using the <code class="ph codeph">ITEM</code>
+ pseudocolumn, and the structure fields inside the array elements are referenced using
+ dot notation. Numeric values such as <code class="ph codeph">SUM()</code> and <code class="ph codeph">AVG()</code>
+ are computed using the numeric <code class="ph codeph">R_NATIONKEY</code> field, and the
+ general-purpose <code class="ph codeph">MAX()</code> and <code class="ph codeph">MIN()</code> values are computed
+ from the string <code class="ph codeph">N_NAME</code> field.
<pre class="pre codeblock"><code>describe region;
+-------------+-------------------------+---------+
| name | type | comment |
@@ -186,7 +187,7 @@ order by r_name;
| MIDDLE EAST | 5 | 58 | 11.6 | EGYPT | SAUDI ARABIA | 5 |
+-------------+-------+-----+------+-----------+----------------+---------------+
</code></pre>
-</div>
+ </div>
<p class="p">
diff --git a/docs/build/html/topics/impala_create_database.html b/docs/build/html/topics/impala_create_database.html
index 7bcb329..684ee4d 100644
--- a/docs/build/html/topics/impala_create_database.html
+++ b/docs/build/html/topics/impala_create_database.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="create_database" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -95,9 +95,10 @@
<div class="p">
- Impala includes another predefined database, <code class="ph codeph">_impala_builtins</code>, that serves as the location
- for the <a class="xref" href="../shared/../topics/impala_functions.html#builtins">built-in functions</a>. To see the built-in
- functions, use a statement like the following:
+ Impala includes another predefined database, <code class="ph codeph">_impala_builtins</code>, that
+ serves as the location for the
+ <a class="xref" href="../shared/../topics/impala_functions.html#builtins">built-in functions</a>. To see
+ the built-in functions, use a statement like the following:
<pre class="pre codeblock"><code>show functions in _impala_builtins;
show functions in _impala_builtins like '*<var class="keyword varname">substring</var>*';
</code></pre>
@@ -120,10 +121,11 @@ show functions in _impala_builtins like '*<var class="keyword varname">substring
<p class="p">
- If you connect to different Impala nodes within an <span class="keyword cmdname">impala-shell</span> session for
- load-balancing purposes, you can enable the <code class="ph codeph">SYNC_DDL</code> query option to make each DDL
- statement wait before returning, until the new or changed metadata has been received by all the Impala
- nodes. See <a class="xref" href="../shared/../topics/impala_sync_ddl.html#sync_ddl">SYNC_DDL Query Option</a> for details.
+ If you connect to different Impala nodes within an <span class="keyword cmdname">impala-shell</span>
+ session for load-balancing purposes, you can enable the <code class="ph codeph">SYNC_DDL</code> query
+ option to make each DDL statement wait before returning, until the new or changed
+ metadata has been received by all the Impala nodes. See
+ <a class="xref" href="../shared/../topics/impala_sync_ddl.html#sync_ddl">SYNC_DDL Query Option</a> for details.
</p>
@@ -163,11 +165,13 @@ show functions in _impala_builtins like '*<var class="keyword varname">substring
<p class="p">
In <span class="keyword">Impala 2.6</span> and higher, Impala DDL statements such as
- <code class="ph codeph">CREATE DATABASE</code>, <code class="ph codeph">CREATE TABLE</code>, <code class="ph codeph">DROP DATABASE CASCADE</code>,
- <code class="ph codeph">DROP TABLE</code>, and <code class="ph codeph">ALTER TABLE [ADD|DROP] PARTITION</code> can create or remove folders
- as needed in the Amazon S3 system. Prior to <span class="keyword">Impala 2.6</span>, you had to create folders yourself and point
- Impala database, tables, or partitions at them, and manually remove folders when no longer needed.
- See <a class="xref" href="../shared/../topics/impala_s3.html#s3">Using Impala with the Amazon S3 Filesystem</a> for details about reading and writing S3 data with Impala.
+ <code class="ph codeph">CREATE DATABASE</code>, <code class="ph codeph">CREATE TABLE</code>, <code class="ph codeph">DROP DATABASE
+ CASCADE</code>, <code class="ph codeph">DROP TABLE</code>, and <code class="ph codeph">ALTER TABLE [ADD|DROP]
+ PARTITION</code> can create or remove folders as needed in the Amazon S3 system. Prior
+ to <span class="keyword">Impala 2.6</span>, you had to create folders yourself and point
+ Impala database, tables, or partitions at them, and manually remove folders when no
+ longer needed. See <a class="xref" href="../shared/../topics/impala_s3.html#s3">Using Impala with the Amazon S3 Filesystem</a> for details about reading
+ and writing S3 data with Impala.
</p>
diff --git a/docs/build/html/topics/impala_create_function.html b/docs/build/html/topics/impala_create_function.html
index 2e80dfe..8d47c55 100644
--- a/docs/build/html/topics/impala_create_function.html
+++ b/docs/build/html/topics/impala_create_function.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="create_function" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -28,11 +28,9 @@
<div class="body conbody">
- <p class="p">
-
- Creates a user-defined function (UDF), which you can use to implement custom logic during
- <code class="ph codeph">SELECT</code> or <code class="ph codeph">INSERT</code> operations.
- </p>
+ <p class="p"> Creates a user-defined function (UDF), which you can use to implement
+ custom logic during <code class="ph codeph">SELECT</code> or <code class="ph codeph">INSERT</code>
+ operations. </p>
<p class="p">
@@ -73,8 +71,6 @@
</div>
-
-
<p class="p">
To create a persistent UDA, which must be written in C++, issue a <code class="ph codeph">CREATE AGGREGATE FUNCTION</code> statement:
</p>
@@ -82,6 +78,7 @@
<pre class="pre codeblock"><code>CREATE [AGGREGATE] FUNCTION [IF NOT EXISTS] [<var class="keyword varname">db_name</var>.]<var class="keyword varname">function_name</var>([<var class="keyword varname">arg_type</var>[, <var class="keyword varname">arg_type</var>...])
RETURNS <var class="keyword varname">return_type</var>
+ <span class="ph">[INTERMEDIATE <var class="keyword varname">type_spec</var>]</span>
LOCATION '<var class="keyword varname">hdfs_path</var>'
[INIT_FN='<var class="keyword varname">function</var>]
UPDATE_FN='<var class="keyword varname">function</var>
@@ -89,8 +86,7 @@
[PREPARE_FN='<var class="keyword varname">function</var>]
[CLOSEFN='<var class="keyword varname">function</var>]
<span class="ph">[SERIALIZE_FN='<var class="keyword varname">function</var>]</span>
- [FINALIZE_FN='<var class="keyword varname">function</var>]
- <span class="ph">[INTERMEDIATE <var class="keyword varname">type_spec</var>]</span></code></pre>
+ [FINALIZE_FN='<var class="keyword varname">function</var>]</code></pre>
<p class="p">
<strong class="ph b">Statement type:</strong> DDL
@@ -102,7 +98,7 @@
</p>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
<p class="p">
Variable-length argument lists are supported for C++ UDFs, but currently not for Java UDFs.
</p>
@@ -230,8 +226,6 @@
</p>
-
-
<p class="p">
For end-to-end examples of UDAs, see <a class="xref" href="impala_udf.html#udfs">User-Defined Functions (UDFs)</a>.
</p>
@@ -243,8 +237,8 @@
<p class="p">
- Currently, Impala UDFs cannot accept arguments or return values of the Impala complex types
- (<code class="ph codeph">STRUCT</code>, <code class="ph codeph">ARRAY</code>, or <code class="ph codeph">MAP</code>).
+ Currently, Impala UDFs cannot accept arguments or return values of the Impala complex
+ types (<code class="ph codeph">STRUCT</code>, <code class="ph codeph">ARRAY</code>, or <code class="ph codeph">MAP</code>).
</p>
@@ -254,6 +248,17 @@
<ul class="ul">
+ <li class="li"> When authorization is enabled, the <code class="ph codeph">CREATE FUNCTION</code>
+ statement requires:<ul class="ul">
+ <li class="li">The <code class="ph codeph">CREATE</code> privilege on the database.</li>
+
+ <li class="li">The <code class="ph codeph">ALL</code> privilege on URI where URI is the value
+ you specified for the <code class="ph codeph">LOCATION</code> in the
+ <code class="ph codeph">CREATE FUNCTION</code> statement. </li>
+
+ </ul>
+</li>
+
<li class="li">
You can write Impala UDFs in either C++ or Java. C++ UDFs are new to Impala, and are the recommended format
for high performance utilizing native code. Java-based UDFs are compatible between Impala and Hive, and are
@@ -347,10 +352,11 @@
<p class="p">
- If you connect to different Impala nodes within an <span class="keyword cmdname">impala-shell</span> session for
- load-balancing purposes, you can enable the <code class="ph codeph">SYNC_DDL</code> query option to make each DDL
- statement wait before returning, until the new or changed metadata has been received by all the Impala
- nodes. See <a class="xref" href="../shared/../topics/impala_sync_ddl.html#sync_ddl">SYNC_DDL Query Option</a> for details.
+ If you connect to different Impala nodes within an <span class="keyword cmdname">impala-shell</span>
+ session for load-balancing purposes, you can enable the <code class="ph codeph">SYNC_DDL</code> query
+ option to make each DDL statement wait before returning, until the new or changed
+ metadata has been received by all the Impala nodes. See
+ <a class="xref" href="../shared/../topics/impala_sync_ddl.html#sync_ddl">SYNC_DDL Query Option</a> for details.
</p>
@@ -367,8 +373,8 @@
<p class="p">
- The Hive <code class="ph codeph">current_user()</code> function cannot be
- called from a Java UDF through Impala.
+ The Hive <code class="ph codeph">current_user()</code> function cannot be called from a Java UDF
+ through Impala.
</p>
@@ -376,15 +382,17 @@
<p class="p">
- In <span class="keyword">Impala 2.5</span> and higher, Impala UDFs and UDAs written in C++ are persisted in the metastore database.
- Java UDFs are also persisted, if they were created with the new <code class="ph codeph">CREATE FUNCTION</code> syntax for Java UDFs,
- where the Java function argument and return types are omitted.
- Java-based UDFs created with the old <code class="ph codeph">CREATE FUNCTION</code> syntax do not persist across restarts
- because they are held in the memory of the <span class="keyword cmdname">catalogd</span> daemon.
- Until you re-create such Java UDFs using the new <code class="ph codeph">CREATE FUNCTION</code> syntax,
- you must reload those Java-based UDFs by running the original <code class="ph codeph">CREATE FUNCTION</code> statements again each time
- you restart the <span class="keyword cmdname">catalogd</span> daemon.
- Prior to <span class="keyword">Impala 2.5</span> the requirement to reload functions after a restart applied to both C++ and Java functions.
+ In <span class="keyword">Impala 2.5</span> and higher, Impala UDFs and UDAs written in C++ are
+ persisted in the metastore database. Java UDFs are also persisted, if they were created
+ with the new <code class="ph codeph">CREATE FUNCTION</code> syntax for Java UDFs, where the Java
+ function argument and return types are omitted. Java-based UDFs created with the old
+ <code class="ph codeph">CREATE FUNCTION</code> syntax do not persist across restarts because they are
+ held in the memory of the <span class="keyword cmdname">catalogd</span> daemon. Until you re-create such
+ Java UDFs using the new <code class="ph codeph">CREATE FUNCTION</code> syntax, you must reload those
+ Java-based UDFs by running the original <code class="ph codeph">CREATE FUNCTION</code> statements
+ again each time you restart the <span class="keyword cmdname">catalogd</span> daemon. Prior to
+ <span class="keyword">Impala 2.5</span> the requirement to reload functions after a restart
+ applied to both C++ and Java functions.
</p>
@@ -590,4 +598,4 @@ show functions;
<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_langref_sql.html">Impala SQL Statements</a></div>
</div>
</div></body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_create_role.html b/docs/build/html/topics/impala_create_role.html
index fab21e5..858af49 100644
--- a/docs/build/html/topics/impala_create_role.html
+++ b/docs/build/html/topics/impala_create_role.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="create_role" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -91,7 +91,7 @@
<p class="p">
- <a class="xref" href="impala_authorization.html#authorization">Enabling Sentry Authorization for Impala</a>, <a class="xref" href="impala_grant.html#grant">GRANT Statement (Impala 2.0 or higher only)</a>,
+ <a class="xref" href="impala_authorization.html#authorization">Impala Authorization</a>, <a class="xref" href="impala_grant.html#grant">GRANT Statement (Impala 2.0 or higher only)</a>,
<a class="xref" href="impala_revoke.html#revoke">REVOKE Statement (Impala 2.0 or higher only)</a>, <a class="xref" href="impala_drop_role.html#drop_role">DROP ROLE Statement (Impala 2.0 or higher only)</a>,
<a class="xref" href="impala_show.html#show">SHOW Statement</a>
</p>
diff --git a/docs/build/html/topics/impala_create_table.html b/docs/build/html/topics/impala_create_table.html
index dd9c9f1..26c60b5 100644
--- a/docs/build/html/topics/impala_create_table.html
+++ b/docs/build/html/topics/impala_create_table.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="create_table" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -31,7 +31,6 @@
<div class="body conbody">
<p class="p">
-
Creates a new table and specifies its characteristics. While creating a table, you
optionally specify aspects such as:
</p>
@@ -211,7 +210,7 @@ data_type:
<div class="p">
<strong class="ph b">kudu_partition_clause:</strong>
- <pre class="pre codeblock"><code>
+<pre class="pre codeblock"><code>
kudu_partition_clause ::= [ <var class="keyword varname">hash_clause</var> [, ...]] [, <var class="keyword varname">range_clause</var> ]
hash_clause ::=
@@ -322,8 +321,8 @@ AS
<p class="p">
- Partitioned tables can contain complex type columns.
- All the partition key columns must be scalar types.
+ Partitioned tables can contain complex type columns. All the partition key columns must
+ be scalar types.
</p>
@@ -380,20 +379,21 @@ AS
</p>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
<p class="p">
All Kudu tables require partitioning, which involves different syntax than non-Kudu
tables. See the <code class="ph codeph">PARTITION BY</code> clause, rather than <code class="ph codeph">PARTITIONED
BY</code>, for Kudu tables.
</p>
+
<p class="p">
In <span class="keyword">Impala 2.10</span> and higher, the <code class="ph codeph">PARTITION BY</code>
clause is optional for Kudu tables. If the clause is omitted, Impala automatically
- constructs a single partition that is not connected to any column. Because such a
- table cannot take advantage of Kudu features for parallelized queries and
- query optimizations, omitting the <code class="ph codeph">PARTITION BY</code> clause is only
- appropriate for small lookup tables.
+ constructs a single partition that is not connected to any column. Because such a table
+ cannot take advantage of Kudu features for parallelized queries and query optimizations,
+ omitting the <code class="ph codeph">PARTITION BY</code> clause is only appropriate for small lookup
+ tables.
</p>
</div>
@@ -415,54 +415,54 @@ AS
<p class="p">
- The optional <code class="ph codeph">SORT BY</code> clause lets you specify zero or more columns
- that are sorted in the data files created by each Impala <code class="ph codeph">INSERT</code> or
- <code class="ph codeph">CREATE TABLE AS SELECT</code> operation. Creating data files that are
- sorted is most useful for Parquet tables, where the metadata stored inside each file includes
- the minimum and maximum values for each column in the file. (The statistics apply to each row group
- within the file; for simplicity, Impala writes a single row group in each file.) Grouping
- data values together in relatively narrow ranges within each data file makes it possible
- for Impala to quickly skip over data files that do not contain value ranges indicated in
- the <code class="ph codeph">WHERE</code> clause of a query, and can improve the effectiveness
- of Parquet encoding and compression.
+ The optional <code class="ph codeph">SORT BY</code> clause lets you specify zero or more columns that
+ are sorted in the data files created by each Impala <code class="ph codeph">INSERT</code> or
+ <code class="ph codeph">CREATE TABLE AS SELECT</code> operation. Creating data files that are sorted is
+ most useful for Parquet tables, where the metadata stored inside each file includes the
+ minimum and maximum values for each column in the file. (The statistics apply to each row
+ group within the file; for simplicity, Impala writes a single row group in each file.)
+ Grouping data values together in relatively narrow ranges within each data file makes it
+ possible for Impala to quickly skip over data files that do not contain value ranges
+ indicated in the <code class="ph codeph">WHERE</code> clause of a query, and can improve the
+ effectiveness of Parquet encoding and compression.
</p>
<p class="p">
- This clause is not applicable for Kudu tables or HBase tables. Although it works
- for other HDFS file formats besides Parquet, the more efficient layout is most
- evident with Parquet tables, because each Parquet data file includes statistics
- about the data values in that file.
+ This clause is not applicable for Kudu tables or HBase tables. Although it works for other
+ HDFS file formats besides Parquet, the more efficient layout is most evident with Parquet
+ tables, because each Parquet data file includes statistics about the data values in that
+ file.
</p>
<p class="p">
- The <code class="ph codeph">SORT BY</code> columns cannot include any partition key columns
- for a partitioned table, because those column values are not represented in
- the underlying data files.
+ The <code class="ph codeph">SORT BY</code> columns cannot include any partition key columns for a
+ partitioned table, because those column values are not represented in the underlying data
+ files.
</p>
<p class="p">
- Because data files can arrive in Impala tables by mechanisms that do not respect
- the <code class="ph codeph">SORT BY</code> clause, such as <code class="ph codeph">LOAD DATA</code> or ETL
- tools that create HDFS files, Impala does not guarantee or rely on the data being
- sorted. The sorting aspect is only used to create a more efficient layout for
- Parquet files generated by Impala, which helps to optimize the processing of
- those Parquet files during Impala queries. During an <code class="ph codeph">INSERT</code>
- or <code class="ph codeph">CREATE TABLE AS SELECT</code> operation, the sorting occurs
- when the <code class="ph codeph">SORT BY</code> clause applies to the destination table
- for the data, regardless of whether the source table has a <code class="ph codeph">SORT BY</code>
- clause.
+ Because data files can arrive in Impala tables by mechanisms that do not respect the
+ <code class="ph codeph">SORT BY</code> clause, such as <code class="ph codeph">LOAD DATA</code> or ETL tools that
+ create HDFS files, Impala does not guarantee or rely on the data being sorted. The sorting
+ aspect is only used to create a more efficient layout for Parquet files generated by
+ Impala, which helps to optimize the processing of those Parquet files during Impala
+ queries. During an <code class="ph codeph">INSERT</code> or <code class="ph codeph">CREATE TABLE AS SELECT</code>
+ operation, the sorting occurs when the <code class="ph codeph">SORT BY</code> clause applies to the
+ destination table for the data, regardless of whether the source table has a <code class="ph codeph">SORT
+ BY</code> clause.
</p>
<p class="p">
- For example, when creating a table intended to contain census data, you might define
- sort columns such as last name and state. If a data file in this table contains a
- narrow range of last names, for example from <code class="ph codeph">Smith</code> to <code class="ph codeph">Smythe</code>,
- Impala can quickly detect that this data file contains no matches for a <code class="ph codeph">WHERE</code>
- clause such as <code class="ph codeph">WHERE last_name = 'Jones'</code> and avoid reading the entire file.
+ For example, when creating a table intended to contain census data, you might define sort
+ columns such as last name and state. If a data file in this table contains a narrow range
+ of last names, for example from <code class="ph codeph">Smith</code> to <code class="ph codeph">Smythe</code>, Impala
+ can quickly detect that this data file contains no matches for a <code class="ph codeph">WHERE</code>
+ clause such as <code class="ph codeph">WHERE last_name = 'Jones'</code> and avoid reading the entire
+ file.
</p>
@@ -473,9 +473,9 @@ AS
<p class="p">
Likewise, if an existing table contains data without any sort order, you can reorganize
- the data in a more efficient way by using <code class="ph codeph">INSERT</code> or
- <code class="ph codeph">CREATE TABLE AS SELECT</code> to copy that data into a new table with a
- <code class="ph codeph">SORT BY</code> clause:
+ the data in a more efficient way by using <code class="ph codeph">INSERT</code> or <code class="ph codeph">CREATE TABLE
+ AS SELECT</code> to copy that data into a new table with a <code class="ph codeph">SORT BY</code>
+ clause:
</p>
@@ -487,10 +487,42 @@ AS
</code></pre>
<p class="p">
- The metadata for the <code class="ph codeph">SORT BY</code> clause is stored in the <code class="ph codeph">TBLPROPERTIES</code>
- fields for the table. Other SQL engines that can interoperate with Impala tables, such as Hive
- and Spark SQL, do not recognize this property when inserting into a table that has a <code class="ph codeph">SORT BY</code>
- clause.
+ The metadata for the <code class="ph codeph">SORT BY</code> clause is stored in the
+ <code class="ph codeph">TBLPROPERTIES</code> fields for the table. Other SQL engines that can
+ interoperate with Impala tables, such as Hive and Spark SQL, do not recognize this
+ property when inserting into a table that has a <code class="ph codeph">SORT BY</code> clause.
+ </p>
+
+
+ <p class="p">
+ <strong class="ph b">Transactional tables:</strong>
+ </p>
+
+
+ <p class="p">
+ In the version 3.3 and higher, when integrated with Hive 3, Impala can create, read, and
+ insert into transactional tables.
+ </p>
+
+
+ <div class="p">
+ To create a table that supports transactions, use the <code class="ph codeph">TBLPROPERTIES</code>
+ clause and set the <code class="ph codeph">'transactional'</code> and
+ <code class="ph codeph">'transactional_properties'</code> as below. Currently, Impala only supports
+ insert-only transactional tables.
+<pre class="pre codeblock"><code>TBLPROPERTIES('transactional'='true', 'transactional_properties'='insert_only')</code></pre>
+ </div>
+
+
+ <p class="p">
+ When integrated with Hive3 and the <code class="ph codeph">DEFAULT_TRANSACTIONAL_TYPE</code> query
+ option is set to <code class="ph codeph">INSERT_ONLY</code>, tables are created as insert-only
+ transactional table by default.
+ </p>
+
+
+ <p class="p">
+ Transactional tables are not supported for Kudu and HBase.
</p>
@@ -504,9 +536,10 @@ AS
partitioning mechanisms, the syntax associated with the <code class="ph codeph">STORED AS KUDU</code>
clause is shown separately in the above syntax descriptions. Kudu tables have their own
syntax for <code class="ph codeph">CREATE TABLE</code>, <code class="ph codeph">CREATE EXTERNAL TABLE</code>, and
- <code class="ph codeph">CREATE TABLE AS SELECT</code>. <span class="ph">Prior to <span class="keyword">Impala 2.10</span>,
- all internal Kudu tables require a <code class="ph codeph">PARTITION BY</code> clause, different than
- the <code class="ph codeph">PARTITIONED BY</code> clause for HDFS-backed tables.</span>
+ <code class="ph codeph">CREATE TABLE AS SELECT</code>. <span class="ph">Prior to
+ <span class="keyword">Impala 2.10</span>, all internal Kudu tables require a <code class="ph codeph">PARTITION
+ BY</code> clause, different than the <code class="ph codeph">PARTITIONED BY</code> clause for
+ HDFS-backed tables.</span>
</p>
@@ -616,7 +649,7 @@ CREATE TABLE ctas_t1
<p class="p">
For more on creating a Kudu table with a specific replication factor, see
- <a class="xref" href="impala_kudu.html#kudu_replication_factor">Kudu Replication Factor</a>.
+ <a class="xref" href="impala_kudu.html#kudu_replication_factor">Kudu Replication Factor</a>.
</p>
@@ -798,7 +831,7 @@ create table combined_t2 (x bigint, s string, s2 string, primary key (x, s))
</p>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
In Impala 1.4.0 and higher, Impala can create Avro tables, which formerly required doing
the <code class="ph codeph">CREATE TABLE</code> statement in Hive. See
<a class="xref" href="impala_avro.html#avro">Using the Avro File Format with Impala Tables</a> for details and examples.
@@ -837,15 +870,17 @@ create table combined_t2 (x bigint, s string, s2 string, primary key (x, s))
</p>
- <div class="note note"><span class="notetitle">Note:</span>
- The <code class="ph codeph">CREATE TABLE</code> clauses <code class="ph codeph">FIELDS TERMINATED BY</code>, <code class="ph codeph">ESCAPED
- BY</code>, and <code class="ph codeph">LINES TERMINATED BY</code> have special rules for the string literal used for
- their argument, because they all require a single character. You can use a regular character surrounded by
- single or double quotation marks, an octal sequence such as <code class="ph codeph">'\054'</code> (representing a comma),
- or an integer in the range '-127'..'128' (with quotation marks but no backslash), which is interpreted as a
- single-byte ASCII character. Negative values are subtracted from 256; for example, <code class="ph codeph">FIELDS
- TERMINATED BY '-2'</code> sets the field delimiter to ASCII code 254, the <span class="q">"Icelandic Thorn"</span>
- character used as a delimiter by some data formats.
+ <div class="note note"><span class="notetitle">Note:</span>
+ The <code class="ph codeph">CREATE TABLE</code> clauses <code class="ph codeph">FIELDS TERMINATED BY</code>,
+ <code class="ph codeph">ESCAPED BY</code>, and <code class="ph codeph">LINES TERMINATED BY</code> have special rules
+ for the string literal used for their argument, because they all require a single
+ character. You can use a regular character surrounded by single or double quotation
+ marks, an octal sequence such as <code class="ph codeph">'\054'</code> (representing a comma), or an
+ integer in the range '-127'..'128' (with quotation marks but no backslash), which is
+ interpreted as a single-byte ASCII character. Negative values are subtracted from 256;
+ for example, <code class="ph codeph">FIELDS TERMINATED BY '-2'</code> sets the field delimiter to
+ ASCII code 254, the <span class="q">"Icelandic Thorn"</span> character used as a delimiter by some data
+ formats.
</div>
@@ -868,7 +903,7 @@ create table combined_t2 (x bigint, s string, s2 string, primary key (x, s))
[STORED AS <var class="keyword varname">file_format</var>]
[LOCATION '<var class="keyword varname">hdfs_path</var>']</code></pre>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
<p class="p">
To clone the structure of a table and transfer data into it in a single operation, use
the <code class="ph codeph">CREATE TABLE AS SELECT</code> syntax described in the next subsection.
@@ -912,13 +947,14 @@ create table combined_t2 (x bigint, s string, s2 string, primary key (x, s))
<p class="p">
Prior to Impala 1.4.0, it was not possible to use the <code class="ph codeph">CREATE TABLE LIKE
- <var class="keyword varname">view_name</var></code> syntax. In Impala 1.4.0 and higher, you can create a table with the
- same column definitions as a view using the <code class="ph codeph">CREATE TABLE LIKE</code> technique. Although
- <code class="ph codeph">CREATE TABLE LIKE</code> normally inherits the file format of the original table, a view has no
- underlying file format, so <code class="ph codeph">CREATE TABLE LIKE <var class="keyword varname">view_name</var></code> produces a text
- table by default. To specify a different file format, include a <code class="ph codeph">STORED AS
- <var class="keyword varname">file_format</var></code> clause at the end of the <code class="ph codeph">CREATE TABLE LIKE</code>
- statement.
+ <var class="keyword varname">view_name</var></code> syntax. In Impala 1.4.0 and higher, you can create
+ a table with the same column definitions as a view using the <code class="ph codeph">CREATE TABLE
+ LIKE</code> technique. Although <code class="ph codeph">CREATE TABLE LIKE</code> normally inherits
+ the file format of the original table, a view has no underlying file format, so
+ <code class="ph codeph">CREATE TABLE LIKE <var class="keyword varname">view_name</var></code> produces a text table by
+ default. To specify a different file format, include a <code class="ph codeph">STORED AS
+ <var class="keyword varname">file_format</var></code> clause at the end of the <code class="ph codeph">CREATE TABLE
+ LIKE</code> statement.
</p>
@@ -1024,7 +1060,7 @@ SELECT * FROM t5;
</p>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
When using the <code class="ph codeph">STORED AS</code> clause with a <code class="ph codeph">CREATE TABLE AS
SELECT</code> statement, the destination table must be a file format that Impala can
write to: currently, text or Parquet. You cannot specify an Avro, SequenceFile, or RCFile
@@ -1035,16 +1071,16 @@ SELECT * FROM t5;
<p class="p">
Prior to <span class="keyword">Impala 2.5</span> you could use a partitioned table as the source
and copy data from it, but could not specify any partitioning clauses for the new table.
- In <span class="keyword">Impala 2.5</span> and higher, you can now use the <code class="ph codeph">PARTITIONED
- BY</code> clause with a <code class="ph codeph">CREATE TABLE AS SELECT</code> statement. The following
- example demonstrates how you can copy data from an unpartitioned table in a <code class="ph codeph">CREATE
- TABLE AS SELECT</code> operation, creating a new partitioned table in the process. The
- main syntax consideration is the column order in the <code class="ph codeph">PARTITIONED BY</code>
- clause and the select list: the partition key columns must be listed last in the select
- list, in the same order as in the <code class="ph codeph">PARTITIONED BY</code> clause. Therefore, in
- this case, the column order in the destination table is different from the source table.
- You also only specify the column names in the <code class="ph codeph">PARTITIONED BY</code> clause, not
- the data types or column comments.
+ In <span class="keyword">Impala 2.5</span> and higher, you can now use the
+ <code class="ph codeph">PARTITIONED BY</code> clause with a <code class="ph codeph">CREATE TABLE AS SELECT</code>
+ statement. The following example demonstrates how you can copy data from an unpartitioned
+ table in a <code class="ph codeph">CREATE TABLE AS SELECT</code> operation, creating a new partitioned
+ table in the process. The main syntax consideration is the column order in the
+ <code class="ph codeph">PARTITIONED BY</code> clause and the select list: the partition key columns must
+ be listed last in the select list, in the same order as in the <code class="ph codeph">PARTITIONED
+ BY</code> clause. Therefore, in this case, the column order in the destination table is
+ different from the source table. You also only specify the column names in the
+ <code class="ph codeph">PARTITIONED BY</code> clause, not the data types or column comments.
</p>
@@ -1137,11 +1173,12 @@ ERROR: AnalysisException: Partition column name mismatch: year != month
<p class="p">
- <strong class="ph b">Sorting considerations:</strong> Although you can specify an <code class="ph codeph">ORDER BY</code> clause in an
- <code class="ph codeph">INSERT ... SELECT</code> statement, any <code class="ph codeph">ORDER BY</code> clause is ignored and the
- results are not necessarily sorted. An <code class="ph codeph">INSERT ... SELECT</code> operation potentially creates
- many different data files, prepared by different executor Impala daemons, and therefore the notion of the data being
- stored in sorted order is impractical.
+ <strong class="ph b">Sorting considerations:</strong> Although you can specify an <code class="ph codeph">ORDER BY</code>
+ clause in an <code class="ph codeph">INSERT ... SELECT</code> statement, any <code class="ph codeph">ORDER BY</code>
+ clause is ignored and the results are not necessarily sorted. An <code class="ph codeph">INSERT ...
+ SELECT</code> operation potentially creates many different data files, prepared by
+ different executor Impala daemons, and therefore the notion of the data being stored in
+ sorted order is impractical.
</p>
@@ -1287,19 +1324,20 @@ ERROR: AnalysisException: Partition column name mismatch: year != month
If you specify the <code class="ph codeph">CACHED IN</code> clause, any existing or future data files in
the table directory or the partition subdirectories are designated to be loaded into
memory with the HDFS caching mechanism. See
- <a class="xref" href="impala_perf_hdfs_caching.html#hdfs_caching">Using HDFS Caching with Impala (Impala 2.1 or higher only)</a> for details about using the HDFS
- caching feature.
+ <a class="xref" href="impala_perf_hdfs_caching.html#hdfs_caching">Using HDFS Caching with Impala (Impala 2.1 or higher only)</a> for details about using
+ the HDFS caching feature.
</p>
<p class="p">
- In <span class="keyword">Impala 2.2</span> and higher, the optional <code class="ph codeph">WITH REPLICATION</code> clause
- for <code class="ph codeph">CREATE TABLE</code> and <code class="ph codeph">ALTER TABLE</code> lets you specify
- a <dfn class="term">replication factor</dfn>, the number of hosts on which to cache the same data blocks.
- When Impala processes a cached data block, where the cache replication factor is greater than 1, Impala randomly
- selects a host that has a cached copy of that data block. This optimization avoids excessive CPU
- usage on a single host when the same cached data block is processed multiple times.
- Where practical, specify a value greater than or equal to the HDFS block replication factor.
+ In <span class="keyword">Impala 2.2</span> and higher, the optional <code class="ph codeph">WITH
+ REPLICATION</code> clause for <code class="ph codeph">CREATE TABLE</code> and <code class="ph codeph">ALTER
+ TABLE</code> lets you specify a <dfn class="term">replication factor</dfn>, the number of hosts
+ on which to cache the same data blocks. When Impala processes a cached data block, where
+ the cache replication factor is greater than 1, Impala randomly selects a host that has
+ a cached copy of that data block. This optimization avoids excessive CPU usage on a
+ single host when the same cached data block is processed multiple times. Where
+ practical, specify a value greater than or equal to the HDFS block replication factor.
</p>
@@ -1382,7 +1420,7 @@ ERROR: AnalysisException: Partition column name mismatch: year != month
</p>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
<p class="p">
The Impala <code class="ph codeph">CREATE TABLE</code> statement cannot create an HBase table, because
it currently does not support the <code class="ph codeph">STORED BY</code> clause needed for HBase
@@ -1414,20 +1452,23 @@ ERROR: AnalysisException: Partition column name mismatch: year != month
<p class="p">
In <span class="keyword">Impala 2.6</span> and higher, Impala DDL statements such as
- <code class="ph codeph">CREATE DATABASE</code>, <code class="ph codeph">CREATE TABLE</code>, <code class="ph codeph">DROP DATABASE CASCADE</code>,
- <code class="ph codeph">DROP TABLE</code>, and <code class="ph codeph">ALTER TABLE [ADD|DROP] PARTITION</code> can create or remove folders
- as needed in the Amazon S3 system. Prior to <span class="keyword">Impala 2.6</span>, you had to create folders yourself and point
- Impala database, tables, or partitions at them, and manually remove folders when no longer needed.
- See <a class="xref" href="../shared/../topics/impala_s3.html#s3">Using Impala with the Amazon S3 Filesystem</a> for details about reading and writing S3 data with Impala.
+ <code class="ph codeph">CREATE DATABASE</code>, <code class="ph codeph">CREATE TABLE</code>, <code class="ph codeph">DROP DATABASE
+ CASCADE</code>, <code class="ph codeph">DROP TABLE</code>, and <code class="ph codeph">ALTER TABLE [ADD|DROP]
+ PARTITION</code> can create or remove folders as needed in the Amazon S3 system. Prior
+ to <span class="keyword">Impala 2.6</span>, you had to create folders yourself and point
+ Impala database, tables, or partitions at them, and manually remove folders when no
+ longer needed. See <a class="xref" href="../shared/../topics/impala_s3.html#s3">Using Impala with the Amazon S3 Filesystem</a> for details about reading
+ and writing S3 data with Impala.
</p>
<p class="p">
- <strong class="ph b">Sorting considerations:</strong> Although you can specify an <code class="ph codeph">ORDER BY</code> clause in an
- <code class="ph codeph">INSERT ... SELECT</code> statement, any <code class="ph codeph">ORDER BY</code> clause is ignored and the
- results are not necessarily sorted. An <code class="ph codeph">INSERT ... SELECT</code> operation potentially creates
- many different data files, prepared by different executor Impala daemons, and therefore the notion of the data being
- stored in sorted order is impractical.
+ <strong class="ph b">Sorting considerations:</strong> Although you can specify an <code class="ph codeph">ORDER BY</code>
+ clause in an <code class="ph codeph">INSERT ... SELECT</code> statement, any <code class="ph codeph">ORDER BY</code>
+ clause is ignored and the results are not necessarily sorted. An <code class="ph codeph">INSERT ...
+ SELECT</code> operation potentially creates many different data files, prepared by
+ different executor Impala daemons, and therefore the notion of the data being stored in
+ sorted order is impractical.
</p>
@@ -1473,20 +1514,22 @@ ERROR: AnalysisException: Partition column name mismatch: year != month
<p class="p">
- If these statements in your environment contain sensitive literal values such as credit card numbers or tax
- identifiers, Impala can redact this sensitive information when displaying the statements in log files and
- other administrative contexts. See <span class="xref">the documentation for your Apache Hadoop distribution</span> for details.
+ If these statements in your environment contain sensitive literal values such as credit
+ card numbers or tax identifiers, Impala can redact this sensitive information when
+ displaying the statements in log files and other administrative contexts. See
+ <span class="xref">the documentation for your Apache Hadoop distribution</span> for details.
</p>
<p class="p">
- <strong class="ph b">Cancellation:</strong> Certain multi-stage statements (<code class="ph codeph">CREATE TABLE AS SELECT</code> and
- <code class="ph codeph">COMPUTE STATS</code>) can be cancelled during some stages, when running <code class="ph codeph">INSERT</code>
- or <code class="ph codeph">SELECT</code> operations internally. To cancel this statement, use Ctrl-C from the
- <span class="keyword cmdname">impala-shell</span> interpreter, the <span class="ph uicontrol">Cancel</span> button from the
- <span class="ph uicontrol">Watch</span> page in Hue, or <span class="ph uicontrol">Cancel</span> from the list of
- in-flight queries (for a particular node) on the <span class="ph uicontrol">Queries</span> tab in the Impala web UI
- (port 25000).
+ <strong class="ph b">Cancellation:</strong> Certain multi-stage statements (<code class="ph codeph">CREATE TABLE AS
+ SELECT</code> and <code class="ph codeph">COMPUTE STATS</code>) can be cancelled during some stages,
+ when running <code class="ph codeph">INSERT</code> or <code class="ph codeph">SELECT</code> operations internally.
+ To cancel this statement, use Ctrl-C from the <span class="keyword cmdname">impala-shell</span>
+ interpreter, the <span class="ph uicontrol">Cancel</span> button from the
+ <span class="ph uicontrol">Watch</span> page in Hue, or <span class="ph uicontrol">Cancel</span> from the list
+ of in-flight queries (for a particular node) on the <span class="ph uicontrol">Queries</span> tab
+ in the Impala web UI (port 25000).
</p>
@@ -1503,7 +1546,8 @@ ERROR: AnalysisException: Partition column name mismatch: year != month
<a class="xref" href="impala_tables.html#internal_tables">Internal Tables</a>,
<a class="xref" href="impala_tables.html#external_tables">External Tables</a>,
<a class="xref" href="impala_compute_stats.html#compute_stats">COMPUTE STATS Statement</a>,
- <a class="xref" href="impala_sync_ddl.html#sync_ddl">SYNC_DDL Query Option</a>, <a class="xref" href="impala_show.html#show_tables">SHOW TABLES Statement</a>,
+ <a class="xref" href="impala_sync_ddl.html#sync_ddl">SYNC_DDL Query Option</a>,
+ <a class="xref" href="impala_show.html#show_tables">SHOW TABLES Statement</a>,
<a class="xref" href="impala_show.html#show_create_table">SHOW CREATE TABLE Statement</a>,
<a class="xref" href="impala_describe.html#describe">DESCRIBE Statement</a>
</p>
@@ -1517,4 +1561,4 @@ ERROR: AnalysisException: Partition column name mismatch: year != month
<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_langref_sql.html">Impala SQL Statements</a></div>
</div>
</div></body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_create_view.html b/docs/build/html/topics/impala_create_view.html
index c2fae86..73ec9fe 100644
--- a/docs/build/html/topics/impala_create_view.html
+++ b/docs/build/html/topics/impala_create_view.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_langref_sql.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="create_view" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -135,11 +135,10 @@ select * from <var class="keyword varname">view_name</var> order by c1 desc limi
<p class="p">
For tables containing complex type columns (<code class="ph codeph">ARRAY</code>,
- <code class="ph codeph">STRUCT</code>, or <code class="ph codeph">MAP</code>), you typically use
- join queries to refer to the complex values. You can use views to
- hide the join notation, making such tables seem like traditional denormalized
- tables, and making those tables queryable by business intelligence tools
- that do not have built-in support for those complex types.
+ <code class="ph codeph">STRUCT</code>, or <code class="ph codeph">MAP</code>), you typically use join queries to
+ refer to the complex values. You can use views to hide the join notation, making such
+ tables seem like traditional denormalized tables, and making those tables queryable by
+ business intelligence tools that do not have built-in support for those complex types.
See <a class="xref" href="../shared/../topics/impala_complex_types.html#complex_types_views">Accessing Complex Type Data in Flattened Form Using Views</a> for details.
</p>
@@ -152,10 +151,11 @@ select * from <var class="keyword varname">view_name</var> order by c1 desc limi
<p class="p">
- If you connect to different Impala nodes within an <span class="keyword cmdname">impala-shell</span> session for
- load-balancing purposes, you can enable the <code class="ph codeph">SYNC_DDL</code> query option to make each DDL
- statement wait before returning, until the new or changed metadata has been received by all the Impala
- nodes. See <a class="xref" href="../shared/../topics/impala_sync_ddl.html#sync_ddl">SYNC_DDL Query Option</a> for details.
+ If you connect to different Impala nodes within an <span class="keyword cmdname">impala-shell</span>
+ session for load-balancing purposes, you can enable the <code class="ph codeph">SYNC_DDL</code> query
+ option to make each DDL statement wait before returning, until the new or changed
+ metadata has been received by all the Impala nodes. See
+ <a class="xref" href="../shared/../topics/impala_sync_ddl.html#sync_ddl">SYNC_DDL Query Option</a> for details.
</p>
@@ -165,9 +165,10 @@ select * from <var class="keyword varname">view_name</var> order by c1 desc limi
<p class="p">
- If these statements in your environment contain sensitive literal values such as credit card numbers or tax
- identifiers, Impala can redact this sensitive information when displaying the statements in log files and
- other administrative contexts. See <span class="xref">the documentation for your Apache Hadoop distribution</span> for details.
+ If these statements in your environment contain sensitive literal values such as credit
+ card numbers or tax identifiers, Impala can redact this sensitive information when
+ displaying the statements in log files and other administrative contexts. See
+ <span class="xref">the documentation for your Apache Hadoop distribution</span> for details.
</p>
diff --git a/docs/build/html/topics/impala_custom_timezones.html b/docs/build/html/topics/impala_custom_timezones.html
index 1de62a8..d5250ca 100644
--- a/docs/build/html/topics/impala_custom_timezones.html
+++ b/docs/build/html/topics/impala_custom_timezones.html
@@ -10,7 +10,7 @@
<meta name="DC.Title" content="Customizing Time Zones" />
<meta name="DC.Relation" scheme="URI" content="../topics/impala_timestamp.html" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="custom_timezone" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -22,9 +22,9 @@
<h1 class="title topictitle1" id="ariaid-title1">Customizing Time Zones</h1>
+
-
-
+
<div class="body conbody">
@@ -50,24 +50,24 @@
<ul class="ul">
<li class="li">
- <code class="ph codeph">--hdfs_zone_info_zip</code>: This flag allows Impala administrators to
- specify a custom timezone database. The flag should be set to a shared (not
- necessarily HDFS) path that points to a zip archive of a custom IANA timezone
- database. The timezone database is expected to be in a compiled binary format. If
- the startup flag is set, Impala will use the specified timezone database instead
- of the default <code class="ph codeph">/usr/share/zoneinfo </code>database. The timezone db
- upgrade process is described in detail below.
+ <code class="ph codeph">‑‑hdfs_zone_info_zip</code>: This flag allows Impala
+ administrators to specify a custom timezone database. The flag should be set to a
+ shared (not necessarily HDFS) path that points to a zip archive of a custom IANA
+ timezone database. The timezone database is expected to be in a compiled binary
+ format. If the startup flag is set, Impala will use the specified timezone
+ database instead of the default <code class="ph codeph">/usr/share/zoneinfo </code>database. The
+ timezone db upgrade process is described in detail below.
</li>
<li class="li">
<p class="p">
- <code class="ph codeph">--hdfs_zone_alias_conf</code>: This flag allows Impala administrators
- to specify definitions for custom timezone aliases. The flag should be set to a
- shared (not necessarily HDFS) path that specifies a config file containing
- custom timezone alias definitions. This config file can be used as a workaround
- for users who want to keep using their legacy timezone names. Configuring custom
- aliases is described in detail below.
+ <code class="ph codeph">‑‑hdfs_zone_alias_conf</code>: This flag allows Impala
+ administrators to specify definitions for custom timezone aliases. The flag
+ should be set to a shared (not necessarily HDFS) path that specifies a config
+ file containing custom timezone alias definitions. This config file can be used
+ as a workaround for users who want to keep using their legacy timezone names.
+ Configuring custom aliases is described in detail below.
</p>
</li>
@@ -124,7 +124,7 @@ hdfs dfs -copyFromLocal ./tzdata/etc/zoneinfo.zip /tzdb/latest</code></pre>
<li class="li">
- Set the <code class="ph codeph">--hdfs_zone_info_zip</code> startup flag to
+ Set the <code class="ph codeph">‑‑hdfs_zone_info_zip</code> startup flag to
<code class="ph codeph">/tzdb/latest/zoneinfo.zip</code> as an <code class="ph codeph">impalad </code>safety
valve.
</li>
@@ -172,7 +172,7 @@ hdfs dfs -copyFromLocal tzalias.conf /tzdb</code></pre>
<li class="li">
- Set the <code class="ph codeph">--hdfs_zone_alias_conf</code> startup flag to
+ Set the <code class="ph codeph">‑‑hdfs_zone_alias_conf</code> startup flag to
<code class="ph codeph">/tzdb/tzalias.conf</code> as an <code class="ph codeph">impalad </code>safety valve.
</li>
@@ -199,4 +199,4 @@ hdfs dfs -copyFromLocal tzalias.conf /tzdb</code></pre>
<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_timestamp.html">TIMESTAMP Data Type</a></div>
</div>
</div></body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_data_cache.html b/docs/build/html/topics/impala_data_cache.html
new file mode 100644
index 0000000..a7c98bf
--- /dev/null
+++ b/docs/build/html/topics/impala_data_cache.html
@@ -0,0 +1,108 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE html
+ PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+
+<meta name="copyright" content="(C) Copyright 2019" />
+<meta name="DC.rights.owner" content="(C) Copyright 2019" />
+<meta name="DC.Type" content="concept" />
+<meta name="DC.Title" content="Data Cache for Remote Reads" />
+<meta name="DC.Relation" scheme="URI" content="../topics/impala_performance.html" />
+<meta name="prodname" content="Impala" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="DC.Format" content="XHTML" />
+<meta name="DC.Identifier" content="data_cache" />
+<link rel="stylesheet" type="text/css" href="../commonltr.css" />
+<title>Data Cache for Remote Reads</title>
+</head>
+<body id="data_cache">
+
+
+ <h1 class="title topictitle1" id="ariaid-title1">Data Cache for Remote Reads</h1>
+
+
+ <div class="body conbody">
+
+ <p class="p">
+ When Impala compute nodes and its storage are not co-located, the network bandwidth
+ requirement goes up as the network traffic includes the data fetch as well as the
+ shuffling exchange traffic of intermediate results.
+ </p>
+
+
+ <div class="note note"><span class="notetitle">Note:</span>
+ This is an experimental feature in Impala 3.3 and is not generally supported.
+ </div>
+
+
+ <p class="p">
+ To mitigate the pressure on the network, you can enable the compute nodes to cache the
+ working set read from remote filesystems, such as, remote HDFS data node, S3, ABFS, ADLS.
+ </p>
+
+
+ <p class="p">
+ To enable remote data cache, set the <code class="ph codeph">--data_cache</code> Impala Daemon start-up
+ flag as below:
+ </p>
+
+
+<pre class="pre codeblock"><code>--data_cache=<var class="keyword varname">dir1</var>,<var class="keyword varname">dir2</var>,<var class="keyword varname">dir3</var>,...:<var class="keyword varname">quota</var></code></pre>
+
+ <p class="p">
+ The flag is set to a list of directories, separated by <code class="ph codeph">,</code>, followed by a
+ <code class="ph codeph">:</code>, and a capacity <code class="ph codeph"><var class="keyword varname">quota</var></code> per
+ directory.
+ </p>
+
+
+ <p class="p">
+ If set to an empty string, data caching is disabled.
+ </p>
+
+
+ <p class="p">
+ Cached data is stored in the specified directories.
+ </p>
+
+
+ <p class="p">
+ The specified directories must exist in the local filesystem of each Impala Daemon, or
+ Impala will fail to start.
+ </p>
+
+
+ <p class="p">
+ In addition, the filesystem which the directory resides in must support hole punching.
+ </p>
+
+
+ <p class="p">
+ The cache can consume up to the <code class="ph codeph">quota</code> bytes for each of the directories
+ specified.
+ </p>
+
+
+ <p class="p">
+ The default setting for <code class="ph codeph">--data_cache</code> is an empty string.
+ </p>
+
+
+ <p class="p">
+ For example, with the following setting, the data cache may use up to 1 TB, with 500 GB
+ max in <code class="ph codeph">/data/0</code> and <code class="ph codeph">/data/1</code> respectively.
+ </p>
+
+
+<pre class="pre codeblock"><code>--data_cache=/data/0,/data/1:500GB</code></pre>
+
+ </div>
+
+
+<div class="related-links">
+<div class="familylinks">
+<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_performance.html">Tuning Impala for Performance</a></div>
+</div>
+</div></body>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_databases.html b/docs/build/html/topics/impala_databases.html
index 7369948..68c491d 100644
--- a/docs/build/html/topics/impala_databases.html
+++ b/docs/build/html/topics/impala_databases.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_schema_objects.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="databases" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -68,9 +68,10 @@
<div class="p">
- Impala includes another predefined database, <code class="ph codeph">_impala_builtins</code>, that serves as the location
- for the <a class="xref" href="../shared/../topics/impala_functions.html#builtins">built-in functions</a>. To see the built-in
- functions, use a statement like the following:
+ Impala includes another predefined database, <code class="ph codeph">_impala_builtins</code>, that
+ serves as the location for the
+ <a class="xref" href="../shared/../topics/impala_functions.html#builtins">built-in functions</a>. To see
+ the built-in functions, use a statement like the following:
<pre class="pre codeblock"><code>show functions in _impala_builtins;
show functions in _impala_builtins like '*<var class="keyword varname">substring</var>*';
</code></pre>
diff --git a/docs/build/html/topics/impala_datatypes.html b/docs/build/html/topics/impala_datatypes.html
index a289f5b..6dd18f2 100644
--- a/docs/build/html/topics/impala_datatypes.html
+++ b/docs/build/html/topics/impala_datatypes.html
@@ -13,6 +13,7 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_bigint.html" />
<meta name="DC.Relation" scheme="URI" content="../topics/impala_boolean.html" />
<meta name="DC.Relation" scheme="URI" content="../topics/impala_char.html" />
+<meta name="DC.Relation" scheme="URI" content="../topics/impala_date.html" />
<meta name="DC.Relation" scheme="URI" content="../topics/impala_decimal.html" />
<meta name="DC.Relation" scheme="URI" content="../topics/impala_double.html" />
<meta name="DC.Relation" scheme="URI" content="../topics/impala_float.html" />
@@ -28,8 +29,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_complex_types.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="datatypes" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -51,7 +52,7 @@
</p>
- <div class="note note"><span class="notetitle">Note:</span>
+ <div class="note note"><span class="notetitle">Note:</span>
Currently, Impala supports only scalar types, not composite or nested types. Accessing a table containing any
columns with unsupported types causes an error.
</div>
@@ -83,6 +84,8 @@
</li>
<li class="link ulchildlink"><strong><a href="../topics/impala_char.html">CHAR Data Type (Impala 2.0 or higher only)</a></strong><br />
</li>
+<li class="link ulchildlink"><strong><a href="../topics/impala_date.html">DATE Data Type</a></strong><br />
+</li>
<li class="link ulchildlink"><strong><a href="../topics/impala_decimal.html">DECIMAL Data Type (Impala 3.0 or higher only)</a></strong><br />
</li>
<li class="link ulchildlink"><strong><a href="../topics/impala_double.html">DOUBLE Data Type</a></strong><br />
@@ -115,4 +118,4 @@
<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_langref.html">Impala SQL Language Reference</a></div>
</div>
</div></body>
-</html>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_date.html b/docs/build/html/topics/impala_date.html
new file mode 100644
index 0000000..eeee8b6
--- /dev/null
+++ b/docs/build/html/topics/impala_date.html
@@ -0,0 +1,240 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE html
+ PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
+
+<meta name="copyright" content="(C) Copyright 2019" />
+<meta name="DC.rights.owner" content="(C) Copyright 2019" />
+<meta name="DC.Type" content="concept" />
+<meta name="DC.Title" content="DATE Data Type" />
+<meta name="DC.Relation" scheme="URI" content="../topics/impala_datatypes.html" />
+<meta name="prodname" content="Impala" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="DC.Format" content="XHTML" />
+<meta name="DC.Identifier" content="date" />
+<link rel="stylesheet" type="text/css" href="../commonltr.css" />
+<title>DATE Data Type</title>
+</head>
+<body id="date">
+
+
+ <h1 class="title topictitle1" id="ariaid-title1">DATE Data Type</h1>
+
+
+
+
+
+
+ <div class="body conbody">
+
+ <p class="p">
+ Use the <code class="ph codeph">DATE</code> data type to store date values. The <code class="ph codeph">DATE</code>
+ type is supported for HBase, Text, and Parquet.
+ </p>
+
+
+ <p class="p">
+ <strong class="ph b">Range:</strong>
+ </p>
+
+
+ <p class="p">
+ 0000-01-01 to 9999-12-31
+ </p>
+
+
+ <p class="p">
+ <strong class="ph b">Literals and expressions:</strong>
+ </p>
+
+
+ <p class="p">
+ The <code class="ph codeph">DATE</code> literals are in the form of <code class="ph codeph">DATE'YYYY-MM-DD'</code>.
+ For example, <code class="ph codeph">DATE '2013-01-01'</code>
+ </p>
+
+
+ <p class="p">
+ <strong class="ph b">Parquet considerations:</strong>
+ </p>
+
+
+ <p class="p">
+ Parquet uses <code class="ph codeph">DATE</code> logical type for dates. The <code class="ph codeph">DATE</code>
+ logical type annotates an <code class="ph codeph">INT32</code> that stores the number of days from the
+ Unix epoch, January 1, 1970. This representation introduces a parquet interoperability
+ issue between Impala and older versions of Hive:
+ </p>
+
+
+ <p class="p">
+ If Hive versions lower than 3.1 wrote dates earlier than 1582-10-15 to a parquet table,
+ those dates will be read back incorrectly by Impala and vice versa. In Hive 3.1 and
+ higher, this is no longer an issue.
+ </p>
+
+
+ <p class="p">
+ <strong class="ph b">Explicit casting between DATE and other data types:</strong>
+ </p>
+
+
+ <p class="p">
+ <code class="ph codeph">DATE</code> type can only be converted to/from <code class="ph codeph">DATE</code>,
+ <code class="ph codeph">TIMESTAMP</code>, or <code class="ph codeph">STRING</code> types as described below.
+ </p>
+
+
+
+<div class="tablenoborder"><table cellpadding="4" cellspacing="0" summary="" id="date__table_ovw_zt1_p3b" class="table" frame="border" border="1" rules="all"><colgroup><col style="width:18.01801801801802%" /><col style="width:19.639639639639643%" /><col style="width:62.34234234234234%" /></colgroup><thead class="thead" style="text-align:left;">
+ <tr class="row">
+ <th class="entry nocellnorowborder" style="vertical-align:top;" id="d48173e142">
+ Cast from
+ </th>
+
+ <th class="entry nocellnorowborder" style="vertical-align:top;" id="d48173e145">
+ Cast to
+ </th>
+
+ <th class="entry cell-norowborder" style="vertical-align:top;" id="d48173e148">
+ Result
+ </th>
+
+ </tr>
+
+ </thead>
+<tbody class="tbody">
+ <tr class="row">
+ <td class="entry nocellnorowborder" style="vertical-align:top;" headers="d48173e142 ">
+ <code class="ph codeph">TIMESTAMP</code>
+ </td>
+
+ <td class="entry nocellnorowborder" style="vertical-align:top;" headers="d48173e145 ">
+ <code class="ph codeph">DATE</code>
+ </td>
+
+ <td class="entry cell-norowborder" style="vertical-align:top;" headers="d48173e148 ">
+ The date component of the <code class="ph codeph">TIMESTAMP</code> is returned, and the time of
+ the day component of the <code class="ph codeph">TIMESTAMP</code> is ignored.
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry nocellnorowborder" style="vertical-align:top;" headers="d48173e142 ">
+ <code class="ph codeph">STRING</code>
+ </td>
+
+ <td class="entry nocellnorowborder" style="vertical-align:top;" headers="d48173e145 ">
+ <code class="ph codeph">DATE</code>
+ </td>
+
+ <td class="entry cell-norowborder" style="vertical-align:top;" headers="d48173e148 ">
+ The <code class="ph codeph">DATE</code> value of <code class="ph codeph">yyyy-MM-dd</code> is returned.
+
+ <p class="p">
+ The <code class="ph codeph">STRING</code> value must be in the <code class="ph codeph">yyyy-MM-dd</code> or
+ <code class="ph codeph">yyyy-MM-dd HH:mm:ss.SSSSSSSSS</code> pattern.
+ </p>
+
+
+
+
+ <p class="p">
+ If the time component is present in <code class="ph codeph">STRING</code>, it is silently
+ ignored.
+ </p>
+
+
+
+
+ <p class="p">
+ If the <code class="ph codeph">STRING</code> value does not match the above formats, an error
+ is returned.
+ </p>
+
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry nocellnorowborder" style="vertical-align:top;" headers="d48173e142 ">
+ <code class="ph codeph">DATE</code>
+ </td>
+
+ <td class="entry nocellnorowborder" style="vertical-align:top;" headers="d48173e145 ">
+ <code class="ph codeph">TIMESTAMP</code>
+ </td>
+
+ <td class="entry cell-norowborder" style="vertical-align:top;" headers="d48173e148 ">
+ The year, month, and day of the <code class="ph codeph">DATE</code> is returned along with the
+ time of day component set to <code class="ph codeph">00:00:00</code>.
+ </td>
+
+ </tr>
+
+ <tr class="row">
+ <td class="entry row-nocellborder" style="vertical-align:top;" headers="d48173e142 ">
+ <code class="ph codeph">DATE</code>
+ </td>
+
+ <td class="entry row-nocellborder" style="vertical-align:top;" headers="d48173e145 ">
+ <code class="ph codeph">STRING</code>
+ </td>
+
+ <td class="entry cellrowborder" style="vertical-align:top;" headers="d48173e148 ">
+ The <code class="ph codeph">STRING</code> value, <code class="ph codeph">'yyyy-MM-dd'</code>, is returned.
+ </td>
+
+ </tr>
+
+ </tbody>
+</table>
+</div>
+
+
+ <p class="p">
+ <strong class="ph b">Implicit casting between DATE and other types: </strong>
+ </p>
+
+
+ <p class="p">
+ Implicit casting is supported:
+ </p>
+
+
+ <ul class="ul">
+ <li class="li">
+ From <code class="ph codeph">STRING</code> to <code class="ph codeph">DATE</code> if the source
+ <code class="ph codeph">STRING</code> value is in the <code class="ph codeph">yyyy-MM-dd</code> or
+ <code class="ph codeph">yyyy-MM-dd HH:mm:ss.SSSSSSSSS</code> pattern.
+ </li>
+
+
+ <li class="li">
+ From <code class="ph codeph">DATE</code> to <code class="ph codeph">TIMESTAMP</code>.
+ </li>
+
+ </ul>
+
+
+ <p class="p">
+ <strong class="ph b">Added in:</strong>
+ </p>
+
+
+ <p class="p">
+ The <code class="ph codeph">DATE</code> type is available in Impala 3.3 and higher.
+ </p>
+
+
+ </div>
+
+
+<div class="related-links">
+<div class="familylinks">
+<div class="parentlink"><strong>Parent topic:</strong> <a class="link" href="../topics/impala_datatypes.html">Data Types</a></div>
+</div>
+</div></body>
+</html>
\ No newline at end of file
diff --git a/docs/build/html/topics/impala_datetime_functions.html b/docs/build/html/topics/impala_datetime_functions.html
index f654358..e7967eb 100644
--- a/docs/build/html/topics/impala_datetime_functions.html
+++ b/docs/build/html/topics/impala_datetime_functions.html
@@ -11,8 +11,8 @@
<meta name="DC.Relation" scheme="URI" content="../topics/impala_functions.html" />
<meta name="prodname" content="Impala" />
<meta name="prodname" content="Impala" />
-<meta name="version" content="Impala 3.2.x" />
-<meta name="version" content="Impala 3.2.x" />
+<meta name="version" content="Impala 3.3.x" />
+<meta name="version" content="Impala 3.3.x" />
<meta name="DC.Format" content="XHTML" />
<meta name="DC.Identifier" content="datetime_functions" />
<link rel="stylesheet" type="text/css" href="../commonltr.css" />
@@ -31,36 +31,35 @@
<div class="body conbody">
<p class="p">
- The underlying Impala data type for date and time data is
- <code class="ph codeph"><a class="xref" href="impala_timestamp.html#timestamp">TIMESTAMP</a></code>, which has
- both a date and a time portion. Functions that extract a single field, such as
- <code class="ph codeph">hour()</code> or <code class="ph codeph">minute()</code>, typically return an integer value.
- Functions that format the date portion, such as <code class="ph codeph">date_add()</code> or
- <code class="ph codeph">to_date()</code>, typically return a string value.
+ The underlying Impala data types for date and time data are <code class="ph codeph">TIMESTAMP</code> and
+ <code class="ph codeph">DATE</code>.
</p>
- <p class="p">
- You can also adjust a <code class="ph codeph">TIMESTAMP</code> value by adding or subtracting an
- <code class="ph codeph">INTERVAL</code> expression. See <a class="xref" href="impala_timestamp.html#timestamp">TIMESTAMP Data Type</a>
- for details. <code class="ph codeph">INTERVAL</code> expressions are also allowed as the second argument
- for the <code class="ph codeph">date_add()</code> and <code class="ph codeph">date_sub()</code> functions, rather than
- integers.
- </p>
+ <div class="p">
+ Some of the date/time functions are affected by the setting of the
+ <code class="ph codeph">‑‑use_local_tz_for_unix_timestamp_conversions</code> startup flag
+ for the <span class="keyword cmdname">impalad</span> daemon:
+ <ul class="ul">
+ <li class="li">
+ The setting is off by default, meaning that functions such as
+ <code class="ph codeph">FROM_UNIXTIME()</code> and <code class="ph codeph">UNIX_TIMESTAMP()</code> consider the
+ input values to always represent the UTC time zone.
+ </li>
- <p class="p">
- Some of these functions are affected by the setting of the
- <code class="ph codeph">--use_local_tz_for_unix_timestamp_conversions</code> startup flag for the
- <span class="keyword cmdname">impalad</span> daemon. This setting is off by default, meaning that functions
- such as <code class="ph codeph">from_unixtime()</code> and <code class="ph codeph">unix_timestamp()</code> consider
- the input values to always represent the UTC time zone. This setting also applies when you
- <code class="ph codeph">CAST()</code> a <code class="ph codeph">BIGINT</code> value to <code class="ph codeph">TIMESTAMP</code>, or
- a <code class="ph codeph">TIMESTAMP</code> value to <code class="ph codeph">BIGINT</code>. When this setting is
- enabled, these functions and operations convert to and from values representing the local
- time zone. See <a class="xref" href="impala_timestamp.html#timestamp">TIMESTAMP Data Type</a> for details about how Impala
- handles time zone considerations for the <code class="ph codeph">TIMESTAMP</code> data type.
- </p>
+ <li class="li">
+ The setting also applies when you <code class="ph codeph">CAST()</code> a <code class="ph codeph">BIGINT</code>
+ value to <code class="ph codeph">TIMESTAMP</code>, or a <code class="ph codeph">TIMESTAMP</code> value to
+ <code class="ph codeph">BIGINT</code>. When this setting is enabled, these functions and operations
+ convert to and from values representing the local time zone. See
+ <a class="xref" href="impala_timestamp.html#timestamp">TIMESTAMP Data Type</a> for details about how Impala
+ handles time zone considerations for the <code class="ph codeph">TIMESTAMP</code> data type.
+ </li>
+
+ </ul>
+
+ </div>
<p class="p">
@@ -69,7 +68,7 @@
<p class="p">
- Impala supports the following data and time functions:
+ Impala supports the following date and time functions:
</p>
@@ -85,6 +84,11 @@
<li class="li">
+ <a class="xref" href="#datetime_functions__current_date">CURRENT_DATE</a>
+ </li>
+
+
+ <li class="li">
<a class="xref" href="#datetime_functions__current_timestamp">CURRENT_TIMESTAMP</a>
</li>
@@ -370,15 +374,31 @@
<dt class="dt dlterm" id="datetime_functions__add_months">
- ADD_MONTHS(TIMESTAMP date, INT months), ADD_MONTHS(TIMESTAMP date, BIGINT months)
+ ADD_MONTHS(TIMESTAMP / DATE date, INT months), ADD_MONTHS(TIMESTAMP / DATE date,
+ BIGINT months)
</dt>
<dd class="dd">
- <strong class="ph b">Purpose:</strong> Returns the specified date and time plus some number of months.
- <p class="p">
- <strong class="ph b">Return type:</strong> <code class="ph codeph">TIMESTAMP</code>
- </p>
+ <strong class="ph b">Purpose:</strong> Adds <var class="keyword varname">months</var> to <var class="keyword varname">date</var> and returns
+ the new date value.
+ <div class="p">
+ <strong class="ph b">Return type:</strong>
+ <ul class="ul">
+ <li class="li">
+ If <var class="keyword varname">date</var> is <code class="ph codeph">TIMESTAMP</code>, returns
+ <code class="ph codeph">TIMESTAMP</code>.
+ </li>
+
+
+ <li class="li">
+ If <var class="keyword varname">date</var> is <code class="ph codeph">DATE</code>, returns
+ <code class="ph codeph">DATE</code>.
+ </li>
+
+ </ul>
+
+ </div>
<p class="p">
@@ -393,41 +413,45 @@
extensions.
</p>
+ </dd>
- <p class="p">
- <strong class="ph b">Examples:</strong>
- </p>
+
+
+
+
+
+ <dt class="dt dlterm" id="datetime_functions__adddate">
+ ADDDATE(TIMESTAMP / DATE date, INT / BIGINT days)
+ </dt>
+ <dd class="dd">
+ <strong class="ph b">Purpose:</strong> Adds <var class="keyword varname">days</var> to <var class="keyword varname">date</var> and returns
+ the new date value.
<p class="p">
- The following examples demonstrate adding months to construct the same day of the
- month in a different month; how if the current day of the month does not exist in
- the target month, the last day of that month is substituted; and how a negative
- argument produces a return value from a previous month.
+ The <var class="keyword varname">days</var> value can be negative, which gives the same result as
+ the <code class="ph codeph">SUBDATE()</code> function.
</p>
-<pre class="pre codeblock"><code>
-select now(), add_months(now(), 2);
-+-------------------------------+-------------------------------+
-| now() | add_months(now(), 2) |
-+-------------------------------+-------------------------------+
-| 2016-05-31 10:47:00.429109000 | 2016-07-31 10:47:00.429109000 |
-+-------------------------------+-------------------------------+
-select now(), add_months(now(), 1);
-+-------------------------------+-------------------------------+
-| now() | add_months(now(), 1) |
-+-------------------------------+-------------------------------+
-| 2016-05-31 10:47:14.540226000 | 2016-06-30 10:47:14.540226000 |
-+-------------------------------+-------------------------------+
+ <div class="p">
+ <strong class="ph b">Return type:</strong>
+ <ul class="ul">
+ <li class="li">
+ If <var class="keyword varname">date</var> is <code class="ph codeph">TIMESTAMP</code>, returns
+ <code class="ph codeph">TIMESTAMP</code>.
+ </li>
+
+
+ <li class="li">
+ If <var class="keyword varname">date</var> is <code class="ph codeph">DATE</code>, returns
+ <code class="ph codeph">DATE</code>.
+ </li>
+
+ </ul>
+
+ </div>
-select now(), add_months(now(), -1);
-+-------------------------------+-------------------------------+
-| now() | add_months(now(), -1) |
-+-------------------------------+-------------------------------+
-| 2016-05-31 10:47:31.732298000 | 2016-04-30 10:47:31.732298000 |
-+-------------------------------+-------------------------------+
-</code></pre>
</dd>
@@ -435,47 +459,23 @@ select now(), add_months(now(), -1);
- <dt class="dt dlterm" id="datetime_functions__adddate">
- ADDDATE(TIMESTAMP startdate, INT days), ADDDATE(TIMESTAMP startdate, BIGINT days)
+ <dt class="dt dlterm" id="datetime_functions__current_date">
+ CURRENT_DATE()
</dt>
<dd class="dd">
- <strong class="ph b">Purpose:</strong> Adds a specified number of days to a <code class="ph codeph">TIMESTAMP</code> value.
- Similar to <code class="ph codeph">DATE_ADD()</code>, but starts with an actual
- <code class="ph codeph">TIMESTAMP</code> value instead of a string that is converted to a
- <code class="ph codeph">TIMESTAMP</code>.
+ <strong class="ph b">Purpose:</strong> Returns the current date.
<p class="p">
- <strong class="ph b">Return type:</strong> <code class="ph codeph">TIMESTAMP</code>
+ Any references to the <code class="ph codeph">CURRENT_DATE()</code> function are evaluated at the
+ start of a query. All calls to <code class="ph codeph">CURRENT_DATE()</code> within the same query
+ return the same value, and the value does not depend on how long the query takes.
</p>
-
- <p class="p">
- <strong class="ph b">Examples:</strong>
- </p>
-
-
<p class="p">
- The following examples show how to add a number of days to a
- <code class="ph codeph">TIMESTAMP</code>. The number of days can also be negative, which gives the
- same effect as the <code class="ph codeph">subdate()</code> function.
+ <strong class="ph b">Return type:</strong> <code class="ph codeph">DATE</code>
</p>
-<pre class="pre codeblock"><code>
-select now() as right_now, adddate(now(), 30) as now_plus_30;
-+-------------------------------+-------------------------------+
-| right_now | now_plus_30 |
-+-------------------------------+-------------------------------+
-| 2016-05-20 10:23:08.640111000 | 2016-06-19 10:23:08.640111000 |
-+-------------------------------+-------------------------------+
-
-select now() as right_now, adddate(now(), -15) as now_minus_15;
-+-------------------------------+-------------------------------+
-| right_now | now_minus_15 |
-+-------------------------------+-------------------------------+
-| 2016-05-20 10:23:38.214064000 | 2016-05-05 10:23:38.214064000 |
-+-------------------------------+-------------------------------+
-</code></pre>
</dd>
@@ -494,27 +494,6 @@ select now() as right_now, adddate(now(), -15) as now_minus_15;
<strong class="ph b">Return type:</strong> <code class="ph codeph">TIMESTAMP</code>
</p>
-
- <p class="p">
- <strong class="ph b">Examples:</strong>
- </p>
-
-<pre class="pre codeblock"><code>
-select now(), current_timestamp();
-+-------------------------------+-------------------------------+
-| now() | current_timestamp() |
-+-------------------------------+-------------------------------+
-| 2016-05-19 16:10:14.237849000 | 2016-05-19 16:10:14.237849000 |
-+-------------------------------+-------------------------------+
-
-select current_timestamp() as right_now,
- current_timestamp() + interval 3 hours as in_three_hours;
-+-------------------------------+-------------------------------+
-| right_now | in_three_hours |
-+-------------------------------+-------------------------------+
-| 2016-05-19 16:13:20.017117000 | 2016-05-19 19:13:20.017117000 |
-+-------------------------------+-------------------------------+
-</code></pre>
</dd>
@@ -523,40 +502,40 @@ select current_timestamp() as right_now,
<dt class="dt dlterm" id="datetime_functions__date_add">
- DATE_ADD(TIMESTAMP startdate, INT days), DATE_ADD(TIMESTAMP startdate,
+ DATE_ADD(TIMESTAMP / DATE date, INT / BIGINT days), DATE_ADD(TIMESTAMP / DATE date,
interval_expression)
</dt>
<dd class="dd">
- <strong class="ph b">Purpose:</strong> Adds a specified number of days to a <code class="ph codeph">TIMESTAMP</code> value.
+ <strong class="ph b">Purpose:</strong> Adds a specified number of days to the <var class="keyword varname">date</var>
+ argument.
+ </dd>
+
+ <dd class="dd ddexpand">
With an <code class="ph codeph">INTERVAL</code> expression as the second argument, you can calculate
a delta value using other units such as weeks, years, hours, seconds, and so on; see
<a class="xref" href="impala_timestamp.html#timestamp">TIMESTAMP Data Type</a> for details.
- <p class="p">
- <strong class="ph b">Return type:</strong> <code class="ph codeph">TIMESTAMP</code>
- </p>
+ <div class="p">
+ <strong class="ph b">Return type:</strong>
+ <ul class="ul">
+ <li class="li">
+ If <var class="keyword varname">date</var> is <code class="ph codeph">TIMESTAMP</code>, returns
+ <code class="ph codeph">TIMESTAMP</code>.
+ </li>
- <p class="p">
- <strong class="ph b">Examples:</strong>
- </p>
+ <li class="li">
+ If <var class="keyword varname">date</var> is <code class="ph codeph">DATE</code>, returns
+ <code class="ph codeph">DATE</code>.
+ </li>
+ </ul>
+
+ </div>
- <p class="p">
- The following example shows the simplest usage, of adding a specified number of days
- to a <code class="ph codeph">TIMESTAMP</code> value:
- </p>
-<pre class="pre codeblock"><code>
-select now() as right_now, date_add(now(), 7) as next_week;
-+-------------------------------+-------------------------------+
-| right_now | next_week |
-+-------------------------------+-------------------------------+
-| 2016-05-20 11:03:48.687055000 | 2016-05-27 11:03:48.687055000 |
-+-------------------------------+-------------------------------+
-</code></pre>
<p class="p">
The following examples show the shorthand notation of an <code class="ph codeph">INTERVAL</code>
expression, instead of specifying the precise number of days. The
@@ -601,8 +580,50 @@ select date_add(cast('2016-01-31' as timestamp), interval 3 months) as 'april_31
+ <dt class="dt dlterm">
+ DATE_CMP(DATE date1, DATE date2)
+ </dt>
+
+
+ <dd class="dd">
+ <strong class="ph b">Purpose:</strong> Compares <var class="keyword varname">date1</var> and <var class="keyword varname">date2</var> and
+ returns:
+ <ul class="ul">
+ <li class="li">
+ <code class="ph codeph">0</code> if the dates are identical.
+ </li>
+
+
+ <li class="li">
+ 1 if <var class="keyword varname">date1</var> > <var class="keyword varname">date2</var>.
+ </li>
+
+
+ <li class="li">
+ -1 if <var class="keyword varname">date1</var> < <var class="keyword varname">date2</var>.
+ </li>
+
+
+ <li class="li">
+ <code class="ph codeph">NULL</code> if <var class="keyword varname">date1</var> or <var class="keyword varname">date2</var> is
+ <code class="ph codeph">NULL</code>.
+ </li>
+
+ </ul>
+
+ <p class="p">
+ <strong class="ph b">Return type:</strong> <code class="ph codeph">INT</code>
+ </p>
+
+ </dd>
+
+
+
+
+
+
<dt class="dt dlterm" id="datetime_functions__date_part">
- DATE_PART(STRING part, TIMESTAMP date)
+ DATE_PART(STRING part, TIMESTAMP / DATE date)
</dt>
@@ -616,26 +637,6 @@ select date_add(cast('2016-01-31' as timestamp), interval 3 months) as 'april_31
<strong class="ph b">Return type:</strong> <code class="ph codeph">BIGINT</code>
</p>
-
- <p class="p">
- <strong class="ph b">Examples:</strong>
- </p>
-
-<pre class="pre codeblock"><code>
-select date_part('year',now()) as current_year;
-+--------------+
-| current_year |
-+--------------+
-| 2016 |
-+--------------+
-
-select date_part('hour',now()) as hour_of_day;
-+-------------+
-| hour_of_day |
-+-------------+
-| 11 |
-+-------------+
-</code></pre>
</dd>
@@ -651,14 +652,26 @@ select date_part('hour',now()) as hour_of_day;
<dd class="dd">
<strong class="ph b">Purpose:</strong> Subtracts a specified number of days from a <code class="ph codeph">TIMESTAMP</code>
- value.
+ value. With an <code class="ph codeph">INTERVAL</code> expression as the second argument, you can
+ calculate a delta value using other units such as weeks, years, hours, seconds, and so
+ on; see <a class="xref" href="impala_timestamp.html#timestamp">TIMESTAMP Data Type</a> for details.
+ <div class="p">
+ <strong class="ph b">Return type:</strong>
+ <ul class="ul">
+ <li class="li">
+ If <var class="keyword varname">date</var> is <code class="ph codeph">TIMESTAMP</code>, returns
+ <code class="ph codeph">TIMESTAMP</code>.
+ </li>
- With an <code class="ph codeph">INTERVAL</code> expression as the second argument, you can calculate
- a delta value using other units such as weeks, years, hours, seconds, and so on; see
- <a class="xref" href="impala_timestamp.html#timestamp">TIMESTAMP Data Type</a> for details.
- <p class="p">
- <strong class="ph b">Return type:</strong> <code class="ph codeph">TIMESTAMP</code>
- </p>
+
+ <li class="li">
+ If <var class="keyword varname">date</var> is <code class="ph codeph">DATE</code>, returns
+ <code class="ph codeph">DATE</code>.
+ </li>
+
+ </ul>
+
+ </div>
<p class="p">
@@ -724,96 +737,220 @@ select date_sub(cast('2016-05-31' as timestamp), interval 1 months) as 'april_31
<dt class="dt dlterm" id="datetime_functions__date_trunc">
- DATE_TRUNC(STRING unit, TIMESTAMP ts)
+ DATE_TRUNC(STRING unit, TIMESTAMP / DATE ts)
</dt>
<dd class="dd">
- <strong class="ph b">Purpose:</strong> Truncates a <code class="ph codeph">TIMESTAMP</code> value to the specified
- precision.
+ <strong class="ph b">Purpose:</strong> Returns the <var class="keyword varname">ts</var> value truncated to the specified
+ <var class="keyword varname">unit</var>.
<p class="p">
- <strong class="ph b">Unit argument:</strong> The <code class="ph codeph">unit</code> argument value for truncating
- <code class="ph codeph">TIMESTAMP</code> values is not case-sensitive. This argument string can be
- one of:
+ <strong class="ph b">Argument:</strong> The <code class="ph codeph">unit</code> argument is not case-sensitive. This
... 37607 lines suppressed ...