You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kudu.apache.org by gr...@apache.org on 2019/07/09 17:03:30 UTC
[kudu] 02/02: release notes: add prior notes for 1.10.0
This is an automated email from the ASF dual-hosted git repository.
granthenke pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit 43f112ec3e9c2285988fb56abe09fe8e3287eecf
Author: Grant Henke <gr...@apache.org>
AuthorDate: Tue Jul 9 10:41:54 2019 -0500
release notes: add prior notes for 1.10.0
Change-Id: If783f945d7b739dd2df24f58d43b4a0c89ac3e71
Reviewed-on: http://gerrit.cloudera.org:8080/13825
Tested-by: Kudu Jenkins
Reviewed-by: Andrew Wong <aw...@cloudera.com>
---
docs/prior_release_notes.adoc | 281 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 281 insertions(+)
diff --git a/docs/prior_release_notes.adoc b/docs/prior_release_notes.adoc
index 74b5011..7818953 100644
--- a/docs/prior_release_notes.adoc
+++ b/docs/prior_release_notes.adoc
@@ -37,6 +37,287 @@ reproduced on this page. Please consult the
link:http://kudu.apache.org/releases/[documentation of the appropriate release]
for a list of known issues and limitations.
+[[rn_1.10.0]]
+== Release Notes Specific to 1.10.0
+
+[[rn_1.10.0_upgrade_notes]]
+== Upgrade Notes
+
+* The default tablet history retention time has been raised from 15 minutes to
+ 7 days to better support touchless incremental backups
+ (see link:https://issues.apache.org/jira/browse/KUDU-2677[KUDU-2677]).
+
+[[rn_1.10.0_obsoletions]]
+== Obsoletions
+
+
+[[rn_1.10.0_deprecations]]
+== Deprecations
+
+
+[[rn_1.10.0_new_features]]
+== New features
+
+* Kudu now supports both full and incremental table backups via a job
+ implemented using Apache Spark. Additionally it supports restoring
+ tables from full and incremental backups via a restore job implemented using
+ Apache Spark. See the link:administration.html#backup[backup documentation]
+ for more details.
+
+* Kudu can now synchronize its internal catalog with the Apache Hive Metastore,
+ automatically updating Hive Metastore table entries upon table creation,
+ deletion, and alterations in Kudu. See the
+ link:hive_metastore.html#metadata_sync[HMS synchronization documentation]
+ for more details.
+
+* Kudu now supports native fine-grained authorization via integration with
+ Apache Sentry. Kudu may now enforce access control policies defined for Kudu
+ tables and columns, as well as policies defined on Hive servers and databases
+ that may store Kudu tables. See the
+ link:security.html#fine_grained_authz[authorization documentation] for more
+ details.
+
+* Kudu’s web UI now supports SPNEGO, a protocol for securing HTTP requests with
+ Kerberos by passing negotiation through HTTP headers. To enable, set the
+ `--webserver_require_spnego` command line flag.
+
+* Column comments can now be stored in Kudu tables, and can be updated using
+ the AlterTable API
+ (see link:https://issues.apache.org/jira/browse/KUDU-1711[KUDU-1711]).
+
+* The Java scan token builder can now create multiple tokens per tablet.
+ To use this functionality, call `setSplitSizeBytes()` to specify how many bytes
+ of data each token should scan. The same API is also available in Kudu’s
+ Spark integration, where it can be used to spawn multiple Spark tasks per
+ scanned tablet
+ (see link:https://issues.apache.org/jira/browse/KUDU-2670[KUDU-2670]).
+
+* Experimental Kudu Docker images are now published on
+ link:https://hub.docker.com/r/apache/kudu[Docker Hub].
+
+* Kudu now has an experimental Kubernetes StatefulSet manifest and Helm chart,
+ which can be used to define and provision Kudu clusters using Kubernetes
+ (see link:https://issues.apache.org/jira/browse/KUDU-2398[KUDU-2398]).
+
+* The Kudu CLI now has rudimentary YAML-based configuration file support,
+ which can be used to provide cluster connection information via cluster name
+ instead of keying in comma-separated lists of master addresses.
+ See the link:https://kudu.apache.org/docs/administration.html#using_cluster_names_in_kudu_tool[cluster name documentation]
+ for more details.
+
+* `kudu perf table_scan` scans a table and displays a table’s row count as well
+ as the time it took to run the scan.
+
+* `kudu table copy` copies data from one table to another, within the same
+ cluster or across clusters. Note, this implementation leverages a single client,
+ therefore it may not be suitable for large tables.
+
+* Tablet history retention time can now be configured on a table-by-table basis.
+ (see link:https://issues.apache.org/jira/browse/KUDU-2514[KUDU-2514]).
+
+[[rn_1.10.0_improvements]]
+== Optimizations and improvements
+
+* The performance of mutations (i.e. UPDATE, DELETE, and re-INSERT) to
+ not-yet-flushed Kudu data has been significantly optimized
+ (see link:https://issues.apache.org/jira/browse/KUDU-2826[KUDU-2826] and
+ link:https://github.com/apache/kudu/commit/f9f9526d3[f9f9526d3]).
+
+* Predicate performance for primitive columns has been optimized
+ (see link:https://issues.apache.org/jira/browse/KUDU-2846[KUDU-2846]).
+
+* IS NULL and IS NOT NULL predicate performance has been optimized
+ (see link:https://issues.apache.org/jira/browse/KUDU-2846[KUDU-2846]).
+
+* Optimized the performance of fetching tablet locations from the master for
+ tables with large numbers of partitions. This can improve the performance of
+ short-running Spark or Impala queries as well as user applications which make
+ use of short-lived client instances
+ (see link:https://issues.apache.org/jira/browse/KUDU-2711[KUDU-2711]).
+
+* The `tableExists()` (Java) and `TableExists()` (C++) APIs are now more performant
+ (see link:https://issues.apache.org/jira/browse/KUDU-2802[KUDU-2802]).
+
+* Fault tolerant scans are now much more performant and consume far less memory
+ (see link:https://issues.apache.org/jira/browse/KUDU-2466[KUDU-2466]).
+
+* `kudu cluster ksck` now sends more requests in parallel, which should result
+ in a speed-up when running against clusters with many tables or when there’s
+ high latency between the node running the CLI and the cluster nodes.
+
+* Kudu’s block manager now deletes spent block containers when needed instead of
+ just at server startup. This should reduce server startup times somewhat
+ (see link:https://issues.apache.org/jira/browse/KUDU-2636[KUDU-2636]).
+
+* DNS resolutions are now cached by Kudu masters, tablet servers, and
+ Kudu C++ clients. The TTL for a resolved DNS entry in the cache is 15 seconds
+ by default
+ (see link:https://issues.apache.org/jira/browse/KUDU-2791[KUDU-2791]).
+
+* Tables created in Kudu 1.10.0 or later will show their creation time as well
+ as their last alteration time in the web UI
+ (see link:https://issues.apache.org/jira/browse/KUDU-2750[KUDU-2750]).
+
+* The Kudu CLI and C++ client now support overriding the local username using
+ the ‘KUDU_USER_NAME’ environment variable. This allows operating against a
+ Kudu cluster using an identity which differs from the local Unix user on the
+ client. Note that this has no effect on secure clusters, where client identity
+ is determined by Kerberos authentication
+ (see link:https://issues.apache.org/jira/browse/KUDU-2717[KUDU-2717]).
+
+* Kudu C++ client now performs stricter verification on the input data of
+ INSERT and UPSERT operations w.r.t. table schema constraints. This helps
+ spotting schema violations before sending the data to a tablet server.
+
+* The `KuduScanner` in the Java client is now iterable. Additionally the
+ `KuduScannerIterator` will automatically make scanner keep alive calls to
+ ensure scanners do not time out while iterating.
+
+* A `KuduPartitioner` API was added to the Java client. The `KuduPartitioner`
+ API allows a client to determine which partition a row falls into without
+ actually writing that row. For example, the `KuduPartitioner` is used in the
+ Spark integration to optionally repartition and pre-sort the data before
+ writing to Kudu
+ (see link:https://issues.apache.org/jira/browse/KUDU-2674[KUDU-2674] and
+ link:https://issues.apache.org/jira/browse/KUDU-2672[KUDU-2672]).
+
+* The `PartialRow` and `RowResult` Java API have new methods that accept and return
+ Java Objects. These methods are useful when you don't care about autoboxing
+ and your existing type handling logic is based on Java types. See the javadoc
+ for more details.
+
+* The Kudu Java client now logs RPC trace summaries instead of full RPC traces when
+ the log level is `INFO` or higher. This reduces log noise and makes RPC issues
+ more visible in a more compact format
+ (see link:https://issues.apache.org/jira/browse/KUDU-2830[KUDU-2830]).
+
+* Kudu servers now display the time at which they were started in their web UIs.
+
+* Kudu tablet servers now display a table’s total column count in the web UI.
+
+* The `/metrics` web UI endpoint now supports filtering on entity types,
+ entity IDs, entity attributes, and metric names. This can be used to more
+ efficiently collect important metrics when there is a large number of tablets
+ on a tablet server.
+
+* The Kudu rebalancer now accepts the `--ignored_tservers` command line
+ argument, which can be used to ignore the health status of specific tablet
+ servers (i.e. if they are down) when deciding whether or not it’s safe to
+ rebalance the cluster.
+
+* `kudu master list` now displays the Raft consensus role of each master in the
+ cluster (i.e. LEADER or FOLLOWER)
+ (see link:https://issues.apache.org/jira/browse/KUDU-2825[KUDU-2825]).
+
+* `kudu table scan` no longer interleaves its output, and now projects all
+ columns without having to manually list the column names.
+
+* `kudu perf loadgen` now supports creating empty tables. The semantics of the
+ special value of 0 for `--num_rows_per_thread` flag has changed. A value of 0
+ now indicates that no rows should be generated, and a value of -1 indicates
+ there should be no limit to the number of rows generated.
+
+* Running `make install` after building Kudu from source will now install the
+ Kudu binaries into appropriate locations.
+ (see link:https://issues.apache.org/jira/browse/KUDU-1344[KUDU-1344]).
+
+[[rn_1.10.0_fixed_issues]]
+== Fixed Issues
+
+* Fixed an issue where the Java client would fail scans that took a very long
+ time to return a single block of rows, such as highly selective scans over a
+ large amount of data
+ (see link:https://issues.apache.org/jira/browse/KUDU-1868[KUDU-1868]).
+
+* Fixed the handling of SERVICE_UNAVAILABLE errors that caused the Java client
+ to do unnecessary master lookups.
+
+* Kudu scan tokens now work correctly when the target table is renamed between
+ when the scan token is created and when it is rehydrated into a scanner.
+
+* Kudu’s “NTP synchronization wait” behavior at startup now works properly when
+ Kudu is run in a containerized environment.
+
+* Fixed a crash when a flush or compaction overlapped with another compaction
+ (see link:https://issues.apache.org/jira/browse/KUDU-2807[KUDU-2807]).
+
+* Fixed a rare race at startup where the leader master would fruitlessly try to
+ tablet copy to a healthy follower master, causing the cluster to operate as if
+ it had two masters until master leadership changed
+ (see link:https://issues.apache.org/jira/browse/KUDU-2748[KUDU-2748]).
+
+* Under rare circumstances, it was possible for Kudu to crash in libkrb5 when
+ negotiating multiple TLS connections concurrently. This crash has been fixed
+ (see link:https://issues.apache.org/jira/browse/KUDU-2706[KUDU-2706]).
+
+* Kudu no longer crashes at startup on machines with disabled CPUs
+ (see link:https://issues.apache.org/jira/browse/KUDU-2721[KUDU-2721]).
+
+[[rn_1.10.0_wire_compatibility]]
+== Wire Protocol compatibility
+
+Kudu 1.10.0 is wire-compatible with previous versions of Kudu:
+
+* Kudu 1.10 clients may connect to servers running Kudu 1.0 or later. If the client uses
+ features that are not available on the target server, an error will be returned.
+* Rolling upgrade between Kudu 1.9 and Kudu 1.10 servers is believed to be possible
+ though has not been sufficiently tested. Users are encouraged to shut down all nodes
+ in the cluster, upgrade the software, and then restart the daemons on the new version.
+* Kudu 1.0 clients may connect to servers running Kudu 1.10 with the exception of the
+ below-mentioned restrictions regarding secure clusters.
+
+The authentication features introduced in Kudu 1.3 place the following limitations
+on wire compatibility between Kudu 1.10 and versions earlier than 1.3:
+
+* If a Kudu 1.10 cluster is configured with authentication or encryption set to "required",
+ clients older than Kudu 1.3 will be unable to connect.
+* If a Kudu 1.10 cluster is configured with authentication and encryption set to "optional"
+ or "disabled", older clients will still be able to connect.
+
+[[rn_1.10.0_incompatible_changes]]
+== Incompatible Changes in Kudu 1.10.0
+
+* Support for building and running with Java 7 has been dropped in this release.
+ It had been deprecated since Kudu 1.5.0.
+ (see link:https://issues.apache.org/jira/browse/KUDU-2099[KUDU-2099]).
+
+[[rn_1.10.0_client_compatibility]]
+=== Client Library Compatibility
+
+* The Kudu 1.10 Java client library is API- and ABI-compatible with Kudu 1.9. Applications
+ written against Kudu 1.9 will compile and run against the Kudu 1.10 client library and
+ vice-versa.
+
+* The Kudu 1.10 {cpp} client is API- and ABI-forward-compatible with Kudu 1.9.
+ Applications written and compiled against the Kudu 1.9 client library will run without
+ modification against the Kudu 1.10 client library. Applications written and compiled
+ against the Kudu 1.10 client library will run without modification against the Kudu 1.9
+ client library.
+
+* The Kudu 1.10 Python client is API-compatible with Kudu 1.9. Applications
+ written against Kudu 1.9 will continue to run against the Kudu 1.10 client
+ and vice-versa.
+
+[[rn_1.10.0_known_issues]]
+== Known Issues and Limitations
+
+Please refer to the link:known_issues.html[Known Issues and Limitations] section of the
+documentation.
+
+[[rn_1.10.0_contributors]]
+== Contributors
+
+Kudu 1.10 includes contributions from 27 people, including 6 first-time contributors:
+
+* Csaba Fulop
+* Florentino Sainz
+* Guangchao Deng
+* Jia Hongchao
+* Ye Yuqiang
+* Yifan Zhang
+
+Thank you for your help in making Kudu even better!
+
[[rn_1.9.0]]
== Release Notes Specific to 1.9.0