You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kudu.apache.org by ab...@apache.org on 2018/10/05 18:41:09 UTC
kudu git commit: KUDU-1592 Make warnings about FBM more ominous
Repository: kudu
Updated Branches:
refs/heads/master b848488d9 -> f1f192ca3
KUDU-1592 Make warnings about FBM more ominous
The documentation warned about scalability issues with file block
manager, but it was rather lightly worded and so was the warning message
of a hole punch test failure.
This commit removes the actual flag necessary to enable FBM from the
hole punching tests' warning to force users look it up in the
documentation instead of blindly applying it, adds a note about it being
unsuitable for production use to the flag's description and expands the
warning in the troubleshooting section.
Change-Id: I15adf42dd7e6376a85cdb178ad460f239a4f87a1
Reviewed-on: http://gerrit.cloudera.org:8080/11579
Reviewed-by: Adar Dembo <ad...@cloudera.com>
Reviewed-by: Andrew Wong <aw...@cloudera.com>
Tested-by: Attila Bukor <ab...@apache.org>
Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/f1f192ca
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/f1f192ca
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/f1f192ca
Branch: refs/heads/master
Commit: f1f192ca3abead4055e7ab6e6bba84d3d6e9866b
Parents: b848488
Author: Attila Bukor <ab...@apache.org>
Authored: Fri Oct 5 19:53:38 2018 +0200
Committer: Attila Bukor <ab...@apache.org>
Committed: Fri Oct 5 18:40:27 2018 +0000
----------------------------------------------------------------------
docs/troubleshooting.adoc | 26 ++++++++++++++++++++------
src/kudu/fs/data_dirs.cc | 8 ++++++--
src/kudu/fs/fs_manager.cc | 3 ++-
3 files changed, 28 insertions(+), 9 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/kudu/blob/f1f192ca/docs/troubleshooting.adoc
----------------------------------------------------------------------
diff --git a/docs/troubleshooting.adoc b/docs/troubleshooting.adoc
index 1bebe07..c3f8fbf 100644
--- a/docs/troubleshooting.adoc
+++ b/docs/troubleshooting.adoc
@@ -48,8 +48,12 @@ Error during hole punch test. The log block manager requires a
filesystem with hole punching support such as ext4 or xfs. On el6,
kernel version 2.6.32-358 or newer is required. To run without hole
punching (at the cost of some efficiency and scalability), reconfigure
-Kudu with --block_manager=file. Refer to the Kudu documentation for more
-details. Raw error message follows.
+Kudu to use the file block manager. Refer to the Kudu documentation for
+more details. WARNING: the file block manager is not suitable for
+production use and should be used only for small-scale evaluation and
+development on systems where hole-punching is not available. It's
+impossible to switch between block managers after data is written to the
+server. Raw error message follows
----
[NOTE]
@@ -70,8 +74,18 @@ adding the `--block_manager=file` flag to the commands you use to start the mast
and tablet servers. The file block manager does not scale as well as the log block
manager.
-WARNING: The file block manager is known to scale and perform poorly, and should
-only be used for small-scale evaluation and development.
+[WARNING]
+====
+The file block manager is known to scale and perform poorly, and should
+only be used for small-scale evaluation and development, and only on systems
+where hole punching is unavailable.
+
+The file block manager uses one file per block. As multiple blocks are written
+for each rowset, the number of blocks can be very high, especially for actively
+written tablets. This can cause performance issues compared to the log block
+manager even with a small amount of data and it's *impossible to switch between
+block managers* without wiping and reinitializing the tablet servers.
+====
[[disk_issues]]
=== Already present: FS layout already exists
@@ -224,7 +238,7 @@ jitter, while `lpeers` lists dispersion.
[NOTE]
-****
+====
.Using `chrony` for time synchronization
Some operating systems offer `chrony` as an alternative to `ntpd` for network time
@@ -233,7 +247,7 @@ synchronization. Kudu has been tested most thoroughly using `ntpd` and use of
In order to use `chrony` for synchronization, `chrony.conf` must be configured
with the `rtcsync` option.
-****
+====
==== NTP Configuration Best Practices
http://git-wip-us.apache.org/repos/asf/kudu/blob/f1f192ca/src/kudu/fs/data_dirs.cc
----------------------------------------------------------------------
diff --git a/src/kudu/fs/data_dirs.cc b/src/kudu/fs/data_dirs.cc
index 49139ea..af0cbec 100644
--- a/src/kudu/fs/data_dirs.cc
+++ b/src/kudu/fs/data_dirs.cc
@@ -136,8 +136,12 @@ const char kHolePunchErrorMsg[] =
"filesystem with hole punching support such as ext4 or xfs. On el6, "
"kernel version 2.6.32-358 or newer is required. To run without hole "
"punching (at the cost of some efficiency and scalability), reconfigure "
- "Kudu with --block_manager=file. Refer to the Kudu documentation for more "
- "details. Raw error message follows";
+ "Kudu to use the file block manager. Refer to the Kudu documentation for "
+ "more details. WARNING: the file block manager is not suitable for "
+ "production use and should be used only for small-scale evaluation and "
+ "development on systems where hole-punching is not available. It's "
+ "impossible to switch between block managers after data is written to the "
+ "server. Raw error message follows";
Status CheckHolePunch(Env* env, const string& path) {
// Arbitrary constants.
http://git-wip-us.apache.org/repos/asf/kudu/blob/f1f192ca/src/kudu/fs/fs_manager.cc
----------------------------------------------------------------------
diff --git a/src/kudu/fs/fs_manager.cc b/src/kudu/fs/fs_manager.cc
index a9b954d..1255888 100644
--- a/src/kudu/fs/fs_manager.cc
+++ b/src/kudu/fs/fs_manager.cc
@@ -67,7 +67,8 @@ TAG_FLAG(enable_data_block_fsync, unsafe);
#if defined(__linux__)
DEFINE_string(block_manager, "log", "Which block manager to use for storage. "
- "Valid options are 'file' and 'log'.");
+ "Valid options are 'file' and 'log'. The file block manager is not suitable for "
+ "production use due to scaling limitations.");
#else
DEFINE_string(block_manager, "file", "Which block manager to use for storage. "
"Only the file block manager is supported for non-Linux systems.");