You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kudu.apache.org by gr...@apache.org on 2019/07/09 17:03:28 UTC
[kudu] branch master updated (1a1e405 -> 43f112e)
This is an automated email from the ASF dual-hosted git repository.
granthenke pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git.
from 1a1e405 [docs] update Hive Metastore integration and Impala integration docs
new b612181 [docs] Pin the bundler install to 1.17.3
new 43f112e release notes: add prior notes for 1.10.0
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs/prior_release_notes.adoc | 281 ++++++++++++++++++++++++++++++++++++++
docs/support/scripts/make_docs.sh | 5 +-
2 files changed, 285 insertions(+), 1 deletion(-)
[kudu] 02/02: release notes: add prior notes for 1.10.0
Posted by gr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
granthenke pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit 43f112ec3e9c2285988fb56abe09fe8e3287eecf
Author: Grant Henke <gr...@apache.org>
AuthorDate: Tue Jul 9 10:41:54 2019 -0500
release notes: add prior notes for 1.10.0
Change-Id: If783f945d7b739dd2df24f58d43b4a0c89ac3e71
Reviewed-on: http://gerrit.cloudera.org:8080/13825
Tested-by: Kudu Jenkins
Reviewed-by: Andrew Wong <aw...@cloudera.com>
---
docs/prior_release_notes.adoc | 281 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 281 insertions(+)
diff --git a/docs/prior_release_notes.adoc b/docs/prior_release_notes.adoc
index 74b5011..7818953 100644
--- a/docs/prior_release_notes.adoc
+++ b/docs/prior_release_notes.adoc
@@ -37,6 +37,287 @@ reproduced on this page. Please consult the
link:http://kudu.apache.org/releases/[documentation of the appropriate release]
for a list of known issues and limitations.
+[[rn_1.10.0]]
+== Release Notes Specific to 1.10.0
+
+[[rn_1.10.0_upgrade_notes]]
+== Upgrade Notes
+
+* The default tablet history retention time has been raised from 15 minutes to
+ 7 days to better support touchless incremental backups
+ (see link:https://issues.apache.org/jira/browse/KUDU-2677[KUDU-2677]).
+
+[[rn_1.10.0_obsoletions]]
+== Obsoletions
+
+
+[[rn_1.10.0_deprecations]]
+== Deprecations
+
+
+[[rn_1.10.0_new_features]]
+== New features
+
+* Kudu now supports both full and incremental table backups via a job
+ implemented using Apache Spark. Additionally it supports restoring
+ tables from full and incremental backups via a restore job implemented using
+ Apache Spark. See the link:administration.html#backup[backup documentation]
+ for more details.
+
+* Kudu can now synchronize its internal catalog with the Apache Hive Metastore,
+ automatically updating Hive Metastore table entries upon table creation,
+ deletion, and alterations in Kudu. See the
+ link:hive_metastore.html#metadata_sync[HMS synchronization documentation]
+ for more details.
+
+* Kudu now supports native fine-grained authorization via integration with
+ Apache Sentry. Kudu may now enforce access control policies defined for Kudu
+ tables and columns, as well as policies defined on Hive servers and databases
+ that may store Kudu tables. See the
+ link:security.html#fine_grained_authz[authorization documentation] for more
+ details.
+
+* Kudu’s web UI now supports SPNEGO, a protocol for securing HTTP requests with
+ Kerberos by passing negotiation through HTTP headers. To enable, set the
+ `--webserver_require_spnego` command line flag.
+
+* Column comments can now be stored in Kudu tables, and can be updated using
+ the AlterTable API
+ (see link:https://issues.apache.org/jira/browse/KUDU-1711[KUDU-1711]).
+
+* The Java scan token builder can now create multiple tokens per tablet.
+ To use this functionality, call `setSplitSizeBytes()` to specify how many bytes
+ of data each token should scan. The same API is also available in Kudu’s
+ Spark integration, where it can be used to spawn multiple Spark tasks per
+ scanned tablet
+ (see link:https://issues.apache.org/jira/browse/KUDU-2670[KUDU-2670]).
+
+* Experimental Kudu Docker images are now published on
+ link:https://hub.docker.com/r/apache/kudu[Docker Hub].
+
+* Kudu now has an experimental Kubernetes StatefulSet manifest and Helm chart,
+ which can be used to define and provision Kudu clusters using Kubernetes
+ (see link:https://issues.apache.org/jira/browse/KUDU-2398[KUDU-2398]).
+
+* The Kudu CLI now has rudimentary YAML-based configuration file support,
+ which can be used to provide cluster connection information via cluster name
+ instead of keying in comma-separated lists of master addresses.
+ See the link:https://kudu.apache.org/docs/administration.html#using_cluster_names_in_kudu_tool[cluster name documentation]
+ for more details.
+
+* `kudu perf table_scan` scans a table and displays a table’s row count as well
+ as the time it took to run the scan.
+
+* `kudu table copy` copies data from one table to another, within the same
+ cluster or across clusters. Note, this implementation leverages a single client,
+ therefore it may not be suitable for large tables.
+
+* Tablet history retention time can now be configured on a table-by-table basis.
+ (see link:https://issues.apache.org/jira/browse/KUDU-2514[KUDU-2514]).
+
+[[rn_1.10.0_improvements]]
+== Optimizations and improvements
+
+* The performance of mutations (i.e. UPDATE, DELETE, and re-INSERT) to
+ not-yet-flushed Kudu data has been significantly optimized
+ (see link:https://issues.apache.org/jira/browse/KUDU-2826[KUDU-2826] and
+ link:https://github.com/apache/kudu/commit/f9f9526d3[f9f9526d3]).
+
+* Predicate performance for primitive columns has been optimized
+ (see link:https://issues.apache.org/jira/browse/KUDU-2846[KUDU-2846]).
+
+* IS NULL and IS NOT NULL predicate performance has been optimized
+ (see link:https://issues.apache.org/jira/browse/KUDU-2846[KUDU-2846]).
+
+* Optimized the performance of fetching tablet locations from the master for
+ tables with large numbers of partitions. This can improve the performance of
+ short-running Spark or Impala queries as well as user applications which make
+ use of short-lived client instances
+ (see link:https://issues.apache.org/jira/browse/KUDU-2711[KUDU-2711]).
+
+* The `tableExists()` (Java) and `TableExists()` (C++) APIs are now more performant
+ (see link:https://issues.apache.org/jira/browse/KUDU-2802[KUDU-2802]).
+
+* Fault tolerant scans are now much more performant and consume far less memory
+ (see link:https://issues.apache.org/jira/browse/KUDU-2466[KUDU-2466]).
+
+* `kudu cluster ksck` now sends more requests in parallel, which should result
+ in a speed-up when running against clusters with many tables or when there’s
+ high latency between the node running the CLI and the cluster nodes.
+
+* Kudu’s block manager now deletes spent block containers when needed instead of
+ just at server startup. This should reduce server startup times somewhat
+ (see link:https://issues.apache.org/jira/browse/KUDU-2636[KUDU-2636]).
+
+* DNS resolutions are now cached by Kudu masters, tablet servers, and
+ Kudu C++ clients. The TTL for a resolved DNS entry in the cache is 15 seconds
+ by default
+ (see link:https://issues.apache.org/jira/browse/KUDU-2791[KUDU-2791]).
+
+* Tables created in Kudu 1.10.0 or later will show their creation time as well
+ as their last alteration time in the web UI
+ (see link:https://issues.apache.org/jira/browse/KUDU-2750[KUDU-2750]).
+
+* The Kudu CLI and C++ client now support overriding the local username using
+ the ‘KUDU_USER_NAME’ environment variable. This allows operating against a
+ Kudu cluster using an identity which differs from the local Unix user on the
+ client. Note that this has no effect on secure clusters, where client identity
+ is determined by Kerberos authentication
+ (see link:https://issues.apache.org/jira/browse/KUDU-2717[KUDU-2717]).
+
+* Kudu C++ client now performs stricter verification on the input data of
+ INSERT and UPSERT operations w.r.t. table schema constraints. This helps
+ spotting schema violations before sending the data to a tablet server.
+
+* The `KuduScanner` in the Java client is now iterable. Additionally the
+ `KuduScannerIterator` will automatically make scanner keep alive calls to
+ ensure scanners do not time out while iterating.
+
+* A `KuduPartitioner` API was added to the Java client. The `KuduPartitioner`
+ API allows a client to determine which partition a row falls into without
+ actually writing that row. For example, the `KuduPartitioner` is used in the
+ Spark integration to optionally repartition and pre-sort the data before
+ writing to Kudu
+ (see link:https://issues.apache.org/jira/browse/KUDU-2674[KUDU-2674] and
+ link:https://issues.apache.org/jira/browse/KUDU-2672[KUDU-2672]).
+
+* The `PartialRow` and `RowResult` Java API have new methods that accept and return
+ Java Objects. These methods are useful when you don't care about autoboxing
+ and your existing type handling logic is based on Java types. See the javadoc
+ for more details.
+
+* The Kudu Java client now logs RPC trace summaries instead of full RPC traces when
+ the log level is `INFO` or higher. This reduces log noise and makes RPC issues
+ more visible in a more compact format
+ (see link:https://issues.apache.org/jira/browse/KUDU-2830[KUDU-2830]).
+
+* Kudu servers now display the time at which they were started in their web UIs.
+
+* Kudu tablet servers now display a table’s total column count in the web UI.
+
+* The `/metrics` web UI endpoint now supports filtering on entity types,
+ entity IDs, entity attributes, and metric names. This can be used to more
+ efficiently collect important metrics when there is a large number of tablets
+ on a tablet server.
+
+* The Kudu rebalancer now accepts the `--ignored_tservers` command line
+ argument, which can be used to ignore the health status of specific tablet
+ servers (i.e. if they are down) when deciding whether or not it’s safe to
+ rebalance the cluster.
+
+* `kudu master list` now displays the Raft consensus role of each master in the
+ cluster (i.e. LEADER or FOLLOWER)
+ (see link:https://issues.apache.org/jira/browse/KUDU-2825[KUDU-2825]).
+
+* `kudu table scan` no longer interleaves its output, and now projects all
+ columns without having to manually list the column names.
+
+* `kudu perf loadgen` now supports creating empty tables. The semantics of the
+ special value of 0 for `--num_rows_per_thread` flag has changed. A value of 0
+ now indicates that no rows should be generated, and a value of -1 indicates
+ there should be no limit to the number of rows generated.
+
+* Running `make install` after building Kudu from source will now install the
+ Kudu binaries into appropriate locations.
+ (see link:https://issues.apache.org/jira/browse/KUDU-1344[KUDU-1344]).
+
+[[rn_1.10.0_fixed_issues]]
+== Fixed Issues
+
+* Fixed an issue where the Java client would fail scans that took a very long
+ time to return a single block of rows, such as highly selective scans over a
+ large amount of data
+ (see link:https://issues.apache.org/jira/browse/KUDU-1868[KUDU-1868]).
+
+* Fixed the handling of SERVICE_UNAVAILABLE errors that caused the Java client
+ to do unnecessary master lookups.
+
+* Kudu scan tokens now work correctly when the target table is renamed between
+ when the scan token is created and when it is rehydrated into a scanner.
+
+* Kudu’s “NTP synchronization wait” behavior at startup now works properly when
+ Kudu is run in a containerized environment.
+
+* Fixed a crash when a flush or compaction overlapped with another compaction
+ (see link:https://issues.apache.org/jira/browse/KUDU-2807[KUDU-2807]).
+
+* Fixed a rare race at startup where the leader master would fruitlessly try to
+ tablet copy to a healthy follower master, causing the cluster to operate as if
+ it had two masters until master leadership changed
+ (see link:https://issues.apache.org/jira/browse/KUDU-2748[KUDU-2748]).
+
+* Under rare circumstances, it was possible for Kudu to crash in libkrb5 when
+ negotiating multiple TLS connections concurrently. This crash has been fixed
+ (see link:https://issues.apache.org/jira/browse/KUDU-2706[KUDU-2706]).
+
+* Kudu no longer crashes at startup on machines with disabled CPUs
+ (see link:https://issues.apache.org/jira/browse/KUDU-2721[KUDU-2721]).
+
+[[rn_1.10.0_wire_compatibility]]
+== Wire Protocol compatibility
+
+Kudu 1.10.0 is wire-compatible with previous versions of Kudu:
+
+* Kudu 1.10 clients may connect to servers running Kudu 1.0 or later. If the client uses
+ features that are not available on the target server, an error will be returned.
+* Rolling upgrade between Kudu 1.9 and Kudu 1.10 servers is believed to be possible
+ though has not been sufficiently tested. Users are encouraged to shut down all nodes
+ in the cluster, upgrade the software, and then restart the daemons on the new version.
+* Kudu 1.0 clients may connect to servers running Kudu 1.10 with the exception of the
+ below-mentioned restrictions regarding secure clusters.
+
+The authentication features introduced in Kudu 1.3 place the following limitations
+on wire compatibility between Kudu 1.10 and versions earlier than 1.3:
+
+* If a Kudu 1.10 cluster is configured with authentication or encryption set to "required",
+ clients older than Kudu 1.3 will be unable to connect.
+* If a Kudu 1.10 cluster is configured with authentication and encryption set to "optional"
+ or "disabled", older clients will still be able to connect.
+
+[[rn_1.10.0_incompatible_changes]]
+== Incompatible Changes in Kudu 1.10.0
+
+* Support for building and running with Java 7 has been dropped in this release.
+ It had been deprecated since Kudu 1.5.0.
+ (see link:https://issues.apache.org/jira/browse/KUDU-2099[KUDU-2099]).
+
+[[rn_1.10.0_client_compatibility]]
+=== Client Library Compatibility
+
+* The Kudu 1.10 Java client library is API- and ABI-compatible with Kudu 1.9. Applications
+ written against Kudu 1.9 will compile and run against the Kudu 1.10 client library and
+ vice-versa.
+
+* The Kudu 1.10 {cpp} client is API- and ABI-forward-compatible with Kudu 1.9.
+ Applications written and compiled against the Kudu 1.9 client library will run without
+ modification against the Kudu 1.10 client library. Applications written and compiled
+ against the Kudu 1.10 client library will run without modification against the Kudu 1.9
+ client library.
+
+* The Kudu 1.10 Python client is API-compatible with Kudu 1.9. Applications
+ written against Kudu 1.9 will continue to run against the Kudu 1.10 client
+ and vice-versa.
+
+[[rn_1.10.0_known_issues]]
+== Known Issues and Limitations
+
+Please refer to the link:known_issues.html[Known Issues and Limitations] section of the
+documentation.
+
+[[rn_1.10.0_contributors]]
+== Contributors
+
+Kudu 1.10 includes contributions from 27 people, including 6 first-time contributors:
+
+* Csaba Fulop
+* Florentino Sainz
+* Guangchao Deng
+* Jia Hongchao
+* Ye Yuqiang
+* Yifan Zhang
+
+Thank you for your help in making Kudu even better!
+
[[rn_1.9.0]]
== Release Notes Specific to 1.9.0
[kudu] 01/02: [docs] Pin the bundler install to 1.17.3
Posted by gr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
granthenke pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kudu.git
commit b612181916be426b18e5cc5490ac1a2d23355936
Author: Grant Henke <gr...@apache.org>
AuthorDate: Tue Jul 9 10:38:05 2019 -0500
[docs] Pin the bundler install to 1.17.3
The bundler install is pinned to an explicit version because
versions after 2.0.0 require ruby >= 2.3.0 which is not
available on many supported operating systems.
Change-Id: Iafbcd31c28a9c8b2444bf83b99dc97882f8c2b5d
Reviewed-on: http://gerrit.cloudera.org:8080/13824
Tested-by: Kudu Jenkins
Reviewed-by: Andrew Wong <aw...@cloudera.com>
---
docs/support/scripts/make_docs.sh | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/docs/support/scripts/make_docs.sh b/docs/support/scripts/make_docs.sh
index 1033c5f..b9837ba 100755
--- a/docs/support/scripts/make_docs.sh
+++ b/docs/support/scripts/make_docs.sh
@@ -103,7 +103,10 @@ BUNDLE="$GEM_PATH/bin/bundle"
echo "Locally installing ruby gems needed to build docs."
if [ ! -x "$BUNDLE" ]; then
set -x
- gem install --no-ri --no-rdoc -q --install-dir "$GEM_PATH" bundler
+ # The bundler install is pinned to an explicit version because versions after
+ # 2.0.0 require ruby >= 2.3.0 which is not available on many supported
+ # operating systems.
+ gem install --no-ri --no-rdoc -q --install-dir "$GEM_PATH" bundler -v '1.17.3'
set +x
fi