You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kudu.apache.org by gr...@apache.org on 2019/06/21 17:35:20 UTC
[kudu] 02/02: [docs] Add 1.10.0 release notes
This is an automated email from the ASF dual-hosted git repository.
granthenke pushed a commit to branch branch-1.10.x
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit 95463d1958a86378ef0985f5fc2477bf3c2752e2
Author: Grant Henke <gr...@apache.org>
AuthorDate: Mon Jun 17 11:33:55 2019 -0500
[docs] Add 1.10.0 release notes
Change-Id: I040df9debdf78f1bd932915164a1b6ba490be135
Reviewed-on: http://gerrit.cloudera.org:8080/13669
Reviewed-by: Alexey Serbin <as...@cloudera.com>
Tested-by: Grant Henke <gr...@apache.org>
Reviewed-by: Greg Solovyev <gs...@cloudera.com>
---
docs/release_notes.adoc | 211 +++++++++++++++++++++++++++++++++++++++++++++---
1 file changed, 201 insertions(+), 10 deletions(-)
diff --git a/docs/release_notes.adoc b/docs/release_notes.adoc
index 8b57faf..c6fd699 100644
--- a/docs/release_notes.adoc
+++ b/docs/release_notes.adoc
@@ -31,13 +31,9 @@
[[rn_1.10.0_upgrade_notes]]
== Upgrade Notes
-* Flume 1.8+ requires Java 8 at runtime even though the Kudu Flume integration
- is Java 7 compatible. Flume 1.9 is the default dependency version as of
- Kudu 1.9.0.
-
-* Hadoop 3.0+ requires Java 8 at runtime even though the Kudu Hadoop integration
- is Java 7 compatible. Hadoop 3.2 is the default dependency version as of
- Kudu 1.9.0.
+* The default tablet history retention time has been raised from 15 minutes to
+ 7 days to better support touchless incremental backups
+ (see link:https://issues.apache.org/jira/browse/KUDU-2677[KUDU-2677]).
[[rn_1.10.0_obsoletions]]
== Obsoletions
@@ -46,20 +42,201 @@
[[rn_1.10.0_deprecations]]
== Deprecations
-* Support for Java 7 has been deprecated since Kudu 1.5.0 and may be removed in
- the next major release.
[[rn_1.10.0_new_features]]
== New features
+* Kudu now supports both full and incremental table backups via a job
+ implemented using Apache Spark. Additionally it supports restoring
+ tables from full and incremental backups via a restore job implemented using
+ Apache Spark. See the backup documentation for more details.
+
+* Kudu can now synchronize its internal catalog with the Apache Hive Metastore,
+ automatically updating Hive Metastore table entries upon table creation,
+ deletion, and alterations in Kudu. See the HMS synchronization documentation
+ for more details.
+
+* Kudu now supports native fine-grained authorization via integration with
+ Apache Sentry. Kudu may now enforce access control policies defined for Kudu
+ tables and columns, as well as policies defined on Hive servers and databases
+ that may store Kudu tables. See the authorization documentation for more
+ details.
+
+* Kudu’s web UI now supports SPNEGO, a protocol for securing HTTP requests with
+ Kerberos by passing negotiation through HTTP headers. To enable, set the
+ `--webserver_require_spnego` command line flag.
+
+* Column comments can now be stored in Kudu tables, and can be updated using
+ the AlterTable API
+ (see link:https://issues.apache.org/jira/browse/KUDU-1711[KUDU-1711]).
+
+* The Java scan token builder can now create multiple tokens per tablet.
+ To use this functionality, call `setSplitSizeBytes()` to specify how many bytes
+ of data each token should scan. The same API is also available in Kudu’s
+ Spark integration, where it can be used to spawn multiple Spark tasks per
+ scanned tablet
+ (see link:https://issues.apache.org/jira/browse/KUDU-2670[KUDU-2670]).
+
+* Experimental Kudu Docker images are now published on
+ link:https://hub.docker.com/r/apache/kudu[Docker Hub].
+
+* Kudu now has an experimental Kubernetes StatefulSet manifest and Helm chart,
+ which can be used to define and provision Kudu clusters using Kubernetes
+ (see link:https://issues.apache.org/jira/browse/KUDU-2398[KUDU-2398]).
+
+* The Kudu CLI now has rudimentary YAML-based configuration file support,
+ which can be used to provide cluster connection information via cluster name
+ instead of keying in comma-separated lists of master addresses.
+ See the link:https://kudu.apache.org/docs/administration.html#using_cluster_names_in_kudu_tool[cluster name documentation]
+ for more details.
+
+* `kudu perf table_scan` scans a table and displays a table’s row count as well
+ as the time it took to run the scan.
+
+* `kudu table copy` copies data from one table to another, within the same
+ cluster or across clusters. Note, this implementation leverages a single client,
+ therefore it may not be suitable for large tables.
+
+* Tablet history retention time can now be configured on a table-by-table basis.
+ (see link:https://issues.apache.org/jira/browse/KUDU-2514[KUDU-2514]).
[[rn_1.10.0_improvements]]
== Optimizations and improvements
+* The performance of mutations (i.e. UPDATE, DELETE, and re-INSERT) to
+ not-yet-flushed Kudu data has been significantly optimized
+ (see link:https://issues.apache.org/jira/browse/KUDU-2826[KUDU-2826] and
+ link:https://github.com/apache/kudu/commit/f9f9526d3[f9f9526d3]).
+
+* Predicate performance for primitive columns has been optimized
+ (see link:https://issues.apache.org/jira/browse/KUDU-2846[KUDU-2846]).
+
+* IS NULL and IS NOT NULL predicate performance has been optimized
+ (see link:https://issues.apache.org/jira/browse/KUDU-2846[KUDU-2846]).
+
+* Optimized the performance of fetching tablet locations from the master for
+ tables with large numbers of partitions. This can improve the performance of
+ short-running Spark or Impala queries as well as user applications which make
+ use of short-lived client instances
+ (see link:https://issues.apache.org/jira/browse/KUDU-2711[KUDU-2711]).
+
+* The `tableExists()` (Java) and `TableExists()` (C++) APIs are now more performant
+ (see link:https://issues.apache.org/jira/browse/KUDU-2802[KUDU-2802]).
+
+* Fault tolerant scans are now much more performant and consume far less memory
+ (see link:https://issues.apache.org/jira/browse/KUDU-2466[KUDU-2466]).
+
+* `kudu cluster ksck` now sends more requests in parallel, which should result
+ in a speed-up when running against clusters with many tables or when there’s
+ high latency between the node running the CLI and the cluster nodes.
+
+* Kudu’s block manager now deletes spent block containers when needed instead of
+ just at server startup. This should reduce server startup times somewhat
+ (see link:https://issues.apache.org/jira/browse/KUDU-2636[KUDU-2636]).
+
+* DNS resolutions are now cached by Kudu masters, tablet servers, and
+ Kudu C++ clients. The TTL for a resolved DNS entry in the cache is 15 seconds
+ by default
+ (see link:https://issues.apache.org/jira/browse/KUDU-2791[KUDU-2791]).
+
+* Tables created in Kudu 1.10.0 or later will show their creation time as well
+ as their last alteration time in the web UI
+ (see link:https://issues.apache.org/jira/browse/KUDU-2750[KUDU-2750]).
+
+* The Kudu CLI and C++ client now support overriding the local username using
+ the ‘KUDU_USER_NAME’ environment variable. This allows operating against a
+ Kudu cluster using an identity which differs from the local Unix user on the
+ client. Note that this has no effect on secure clusters, where client identity
+ is determined by Kerberos authentication
+ (see link:https://issues.apache.org/jira/browse/KUDU-2717[KUDU-2717]).
+
+* Kudu C++ client now performs stricter verification on the input data of
+ INSERT and UPSERT operations w.r.t. table schema constraints. This helps
+ spotting schema violations before sending the data to a tablet server.
+
+* The `KuduScanner` in the Java client is now iterable. Additionally the
+ `KuduScannerIterator` will automatically make scanner keep alive calls to
+ ensure scanners do not time out while iterating.
+
+* A `KuduPartitioner` API was added to the Java client. The `KuduPartitioner`
+ API allows a client to determine which partition a row falls into without
+ actually writing that row. For example, the `KuduPartitioner` is used in the
+ Spark integration to optionally repartition and pre-sort the data before
+ writing to Kudu
+ (see link:https://issues.apache.org/jira/browse/KUDU-2674[KUDU-2674] and
+ link:https://issues.apache.org/jira/browse/KUDU-2672[KUDU-2672]).
+
+* The `PartialRow` and `RowResult` Java API have new methods that accept and return
+ Java Objects. These methods are useful when you don't care about autoboxing
+ and your existing type handling logic is based on Java types. See the javadoc
+ for more details.
+
+* The Kudu Java client now logs RPC trace summaries instead of full RPC traces when
+ the log level is `INFO` or higher. This reduces log noise and makes RPC issues
+ more visible in a more compact format
+ (see link:https://issues.apache.org/jira/browse/KUDU-2830[KUDU-2830]).
+
+* Kudu servers now display the time at which they were started in their web UIs.
+
+* Kudu tablet servers now display a table’s total column count in the web UI.
+
+* The `/metrics` web UI endpoint now supports filtering on entity types,
+ entity IDs, entity attributes, and metric names. This can be used to more
+ efficiently collect important metrics when there is a large number of tablets
+ on a tablet server.
+
+* The Kudu rebalancer now accepts the `--ignored_tservers` command line
+ argument, which can be used to ignore the health status of specific tablet
+ servers (i.e. if they are down) when deciding whether or not it’s safe to
+ rebalance the cluster.
+
+* `kudu master list` now displays the Raft consensus role of each master in the
+ cluster (i.e. LEADER or FOLLOWER)
+ (see link:https://issues.apache.org/jira/browse/KUDU-2825[KUDU-2825]).
+
+* `kudu table scan` no longer interleaves its output, and now projects all
+ columns without having to manually list the column names.
+
+* `kudu perf loadgen` now supports creating empty tables. The semantics of the
+ special value of 0 for `--num_rows_per_thread` flag has changed. A value of 0
+ now indicates that no rows should be generated, and a value of -1 indicates
+ there should be no limit to the number of rows generated.
+
+* Running `make install` after building Kudu from source will now install the
+ Kudu binaries into appropriate locations.
+ (see link:https://issues.apache.org/jira/browse/KUDU-1344[KUDU-1344]).
[[rn_1.10.0_fixed_issues]]
== Fixed Issues
+* Fixed an issue where the Java client would fail scans that took a very long
+ time to return a single block of rows, such as highly selective scans over a
+ large amount of data
+ (see link:https://issues.apache.org/jira/browse/KUDU-1868[KUDU-1868]).
+
+* Fixed the handling of SERVICE_UNAVAILABLE errors that caused the Java client
+ to do unnecessary master lookups.
+
+* Kudu scan tokens now work correctly when the target table is renamed between
+ when the scan token is created and when it is rehydrated into a scanner.
+
+* Kudu’s “NTP synchronization wait” behavior at startup now works properly when
+ Kudu is run in a containerized environment.
+
+* Fixed a crash when a flush or compaction overlapped with another compaction
+ (see link:https://issues.apache.org/jira/browse/KUDU-2807[KUDU-2807]).
+
+* Fixed a rare race at startup where the leader master would fruitlessly try to
+ tablet copy to a healthy follower master, causing the cluster to operate as if
+ it had two masters until master leadership changed
+ (see link:https://issues.apache.org/jira/browse/KUDU-2748[KUDU-2748]).
+
+* Under rare circumstances, it was possible for Kudu to crash in libkrb5 when
+ negotiating multiple TLS connections concurrently. This crash has been fixed
+ (see link:https://issues.apache.org/jira/browse/KUDU-2706[KUDU-2706]).
+
+* Kudu no longer crashes at startup on machines with disabled CPUs
+ (see link:https://issues.apache.org/jira/browse/KUDU-2721[KUDU-2721]).
[[rn_1.10.0_wire_compatibility]]
== Wire Protocol compatibility
@@ -85,6 +262,9 @@ on wire compatibility between Kudu 1.10 and versions earlier than 1.3:
[[rn_1.10.0_incompatible_changes]]
== Incompatible Changes in Kudu 1.10.0
+* Support for building and running with Java 7 has been dropped in this release.
+ It had been deprecated since Kudu 1.5.0.
+ (see link:https://issues.apache.org/jira/browse/KUDU-2099[KUDU-2099]).
[[rn_1.10.0_client_compatibility]]
=== Client Library Compatibility
@@ -93,7 +273,7 @@ on wire compatibility between Kudu 1.10 and versions earlier than 1.3:
written against Kudu 1.9 will compile and run against the Kudu 1.10 client library and
vice-versa.
-* The Kudu 1.9 {cpp} client is API- and ABI-forward-compatible with Kudu 1.9.
+* The Kudu 1.10 {cpp} client is API- and ABI-forward-compatible with Kudu 1.9.
Applications written and compiled against the Kudu 1.9 client library will run without
modification against the Kudu 1.10 client library. Applications written and compiled
against the Kudu 1.10 client library will run without modification against the Kudu 1.9
@@ -112,6 +292,17 @@ documentation.
[[rn_1.10.0_contributors]]
== Contributors
+Kudu 1.10 includes contributions from 27 people, including 6 first-time contributors:
+
+* Csaba Fulop
+* Florentino Sainz
+* Guangchao Deng
+* Jia Hongchao
+* Ye Yuqiang
+* Yifan Zhang
+
+Thank you for your help in making Kudu even better!
+
[[resources_and_next_steps]]
== Resources