You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kudu.apache.org by gr...@apache.org on 2018/03/16 01:25:45 UTC

[1/2] kudu git commit: Bump RAT version to latest 0.12

Repository: kudu
Updated Branches:
  refs/heads/master 4e30d6cce -> 4a6bb1ac2


Bump RAT version to latest 0.12

Change-Id: I558e73f07a9ce1ad3e527bfefea057c285b795fd
Reviewed-on: http://gerrit.cloudera.org:8080/9668
Reviewed-by: Dan Burkert <da...@apache.org>
Tested-by: Grant Henke <gr...@gmail.com>


Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/e35bba0f
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/e35bba0f
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/e35bba0f

Branch: refs/heads/master
Commit: e35bba0fee8ca0f8d396edbd58b621f29e507f91
Parents: 4e30d6c
Author: Grant Henke <gr...@apache.org>
Authored: Thu Mar 15 15:47:55 2018 -0500
Committer: Grant Henke <gr...@gmail.com>
Committed: Thu Mar 15 21:14:04 2018 +0000

----------------------------------------------------------------------
 build-support/build_source_release.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kudu/blob/e35bba0f/build-support/build_source_release.py
----------------------------------------------------------------------
diff --git a/build-support/build_source_release.py b/build-support/build_source_release.py
index cc29d23..47d2459 100755
--- a/build-support/build_source_release.py
+++ b/build-support/build_source_release.py
@@ -144,7 +144,7 @@ def run_rat(tarball_path):
     return
 
   # TODO: Cache and call the jar from the maven repo?
-  rat_url = "http://central.maven.org/maven2/org/apache/rat/apache-rat/0.11/apache-rat-0.11.jar"
+  rat_url = "http://central.maven.org/maven2/org/apache/rat/apache-rat/0.12/apache-rat-0.12.jar"
 
   tmpdir_path = tempfile.mkdtemp()
   rat_report_result = ''


[2/2] kudu git commit: docs: update release notes and scaling guide with new thread information

Posted by gr...@apache.org.
docs: update release notes and scaling guide with new thread information

Change-Id: I6f6a39faa572e16805e1961105140d823bd82f40
Reviewed-on: http://gerrit.cloudera.org:8080/9552
Reviewed-by: David Ribeiro Alves <da...@gmail.com>
Tested-by: Adar Dembo <ad...@cloudera.com>


Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/4a6bb1ac
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/4a6bb1ac
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/4a6bb1ac

Branch: refs/heads/master
Commit: 4a6bb1ac21de56fd75f981fdb820742df18c62cb
Parents: e35bba0
Author: Adar Dembo <ad...@cloudera.com>
Authored: Wed Mar 7 18:42:56 2018 -0800
Committer: Adar Dembo <ad...@cloudera.com>
Committed: Thu Mar 15 23:38:13 2018 +0000

----------------------------------------------------------------------
 docs/release_notes.adoc |  8 ++++++++
 docs/scaling_guide.adoc | 22 ++++++++--------------
 2 files changed, 16 insertions(+), 14 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kudu/blob/4a6bb1ac/docs/release_notes.adoc
----------------------------------------------------------------------
diff --git a/docs/release_notes.adoc b/docs/release_notes.adoc
index 522f868..5467d7f 100644
--- a/docs/release_notes.adoc
+++ b/docs/release_notes.adoc
@@ -89,6 +89,7 @@
   future releases.
 
 == Optimizations and improvements
+
 * Reporting for the `kudu cluster ksck` tool has been updated so tablets and
   tables with on-going tablet copies are shown as "recovering". Additional
   reporting changes have been made to make various common scenarios,
@@ -104,6 +105,13 @@
   scheme, set the `--raft_prepare_replacement_before_eviction` run-time flag to
   `false` for *all* tablet servers and masters in Kudu 1.7 cluster.
 
+* KUDU-1913: the number of OS threads used in the steady state and during bursts
+  of activity (such as in Raft leader elections triggered by a node failure)
+  has been drastically reduced and should no longer exceed the value of
+  `ulimit -u`. As such, it should no longer be necessary to increase the value
+  of `ulimit -u` (or of /proc/sys/kernel/threads-max) in order to run a Kudu
+  tablet server in most cases.
+
 [[rn_1.7.0_fixed_issues]]
 == Fixed Issues
 

http://git-wip-us.apache.org/repos/asf/kudu/blob/4a6bb1ac/docs/scaling_guide.adoc
----------------------------------------------------------------------
diff --git a/docs/scaling_guide.adoc b/docs/scaling_guide.adoc
index 965d1ac..f7c7f33 100644
--- a/docs/scaling_guide.adoc
+++ b/docs/scaling_guide.adoc
@@ -183,17 +183,11 @@ If a Kudu tablet server's thread count exceeds the OS limit, it will crash, usua
 in the logs like "pthread_create failed: Resource temporarily unavailable". If the system thread
 count limit is exceeded, other processes on the same node may also crash.
 
-The table below summarizes thread usage by a Kudu tablet server:
-
-.Tablet Server Thread Usage
-|===
-| Consumer | Multiplier
-
-| Hot replica | 5 threads per hot replica
-| Cold replica | 2 threads per cold replica
-| Replica at startup | 5 threads per replica
-|===
-
-As indicated in the table, all replicas may be considered "hot" when the tablet server starts, so,
-for our example load, thread usage should peak around 5 threads / replica * (200 hot replicas + 1600
-cold replicas) = 18000 threads at startup time.
+Threads and threadpools are used all over Kudu for various purposes, but the number of threads found
+in nearly all of these does not scale with load or data/tablet size; instead, the number of threads
+is either a hardcoded constant, a constant defined by a configuration parameter, or based on a
+static dimension (such as the number of CPU cores).
+
+The only exception to this is the WAL append thread, one of which exists for every "hot" replica.
+Note that all replicas may be considered hot at startup, so tablet servers' thread usage will
+generally peak when started and settle down thereafter.