You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kudu.apache.org by gr...@apache.org on 2018/03/16 01:25:45 UTC
[1/2] kudu git commit: Bump RAT version to latest 0.12
Repository: kudu
Updated Branches:
refs/heads/master 4e30d6cce -> 4a6bb1ac2
Bump RAT version to latest 0.12
Change-Id: I558e73f07a9ce1ad3e527bfefea057c285b795fd
Reviewed-on: http://gerrit.cloudera.org:8080/9668
Reviewed-by: Dan Burkert <da...@apache.org>
Tested-by: Grant Henke <gr...@gmail.com>
Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/e35bba0f
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/e35bba0f
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/e35bba0f
Branch: refs/heads/master
Commit: e35bba0fee8ca0f8d396edbd58b621f29e507f91
Parents: 4e30d6c
Author: Grant Henke <gr...@apache.org>
Authored: Thu Mar 15 15:47:55 2018 -0500
Committer: Grant Henke <gr...@gmail.com>
Committed: Thu Mar 15 21:14:04 2018 +0000
----------------------------------------------------------------------
build-support/build_source_release.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/kudu/blob/e35bba0f/build-support/build_source_release.py
----------------------------------------------------------------------
diff --git a/build-support/build_source_release.py b/build-support/build_source_release.py
index cc29d23..47d2459 100755
--- a/build-support/build_source_release.py
+++ b/build-support/build_source_release.py
@@ -144,7 +144,7 @@ def run_rat(tarball_path):
return
# TODO: Cache and call the jar from the maven repo?
- rat_url = "http://central.maven.org/maven2/org/apache/rat/apache-rat/0.11/apache-rat-0.11.jar"
+ rat_url = "http://central.maven.org/maven2/org/apache/rat/apache-rat/0.12/apache-rat-0.12.jar"
tmpdir_path = tempfile.mkdtemp()
rat_report_result = ''
[2/2] kudu git commit: docs: update release notes and scaling guide
with new thread information
Posted by gr...@apache.org.
docs: update release notes and scaling guide with new thread information
Change-Id: I6f6a39faa572e16805e1961105140d823bd82f40
Reviewed-on: http://gerrit.cloudera.org:8080/9552
Reviewed-by: David Ribeiro Alves <da...@gmail.com>
Tested-by: Adar Dembo <ad...@cloudera.com>
Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/4a6bb1ac
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/4a6bb1ac
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/4a6bb1ac
Branch: refs/heads/master
Commit: 4a6bb1ac21de56fd75f981fdb820742df18c62cb
Parents: e35bba0
Author: Adar Dembo <ad...@cloudera.com>
Authored: Wed Mar 7 18:42:56 2018 -0800
Committer: Adar Dembo <ad...@cloudera.com>
Committed: Thu Mar 15 23:38:13 2018 +0000
----------------------------------------------------------------------
docs/release_notes.adoc | 8 ++++++++
docs/scaling_guide.adoc | 22 ++++++++--------------
2 files changed, 16 insertions(+), 14 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/kudu/blob/4a6bb1ac/docs/release_notes.adoc
----------------------------------------------------------------------
diff --git a/docs/release_notes.adoc b/docs/release_notes.adoc
index 522f868..5467d7f 100644
--- a/docs/release_notes.adoc
+++ b/docs/release_notes.adoc
@@ -89,6 +89,7 @@
future releases.
== Optimizations and improvements
+
* Reporting for the `kudu cluster ksck` tool has been updated so tablets and
tables with on-going tablet copies are shown as "recovering". Additional
reporting changes have been made to make various common scenarios,
@@ -104,6 +105,13 @@
scheme, set the `--raft_prepare_replacement_before_eviction` run-time flag to
`false` for *all* tablet servers and masters in Kudu 1.7 cluster.
+* KUDU-1913: the number of OS threads used in the steady state and during bursts
+ of activity (such as in Raft leader elections triggered by a node failure)
+ has been drastically reduced and should no longer exceed the value of
+ `ulimit -u`. As such, it should no longer be necessary to increase the value
+ of `ulimit -u` (or of /proc/sys/kernel/threads-max) in order to run a Kudu
+ tablet server in most cases.
+
[[rn_1.7.0_fixed_issues]]
== Fixed Issues
http://git-wip-us.apache.org/repos/asf/kudu/blob/4a6bb1ac/docs/scaling_guide.adoc
----------------------------------------------------------------------
diff --git a/docs/scaling_guide.adoc b/docs/scaling_guide.adoc
index 965d1ac..f7c7f33 100644
--- a/docs/scaling_guide.adoc
+++ b/docs/scaling_guide.adoc
@@ -183,17 +183,11 @@ If a Kudu tablet server's thread count exceeds the OS limit, it will crash, usua
in the logs like "pthread_create failed: Resource temporarily unavailable". If the system thread
count limit is exceeded, other processes on the same node may also crash.
-The table below summarizes thread usage by a Kudu tablet server:
-
-.Tablet Server Thread Usage
-|===
-| Consumer | Multiplier
-
-| Hot replica | 5 threads per hot replica
-| Cold replica | 2 threads per cold replica
-| Replica at startup | 5 threads per replica
-|===
-
-As indicated in the table, all replicas may be considered "hot" when the tablet server starts, so,
-for our example load, thread usage should peak around 5 threads / replica * (200 hot replicas + 1600
-cold replicas) = 18000 threads at startup time.
+Threads and threadpools are used all over Kudu for various purposes, but the number of threads found
+in nearly all of these does not scale with load or data/tablet size; instead, the number of threads
+is either a hardcoded constant, a constant defined by a configuration parameter, or based on a
+static dimension (such as the number of CPU cores).
+
+The only exception to this is the WAL append thread, one of which exists for every "hot" replica.
+Note that all replicas may be considered hot at startup, so tablet servers' thread usage will
+generally peak when started and settle down thereafter.