hbase git commit: HBASE-14071 Document troubleshooting unexpected filesystem usage by snapshots and WALs

[mi...@apache.org]

Repository: hbase
Updated Branches:
  refs/heads/master 0daae4334 -> b29c731a5


HBASE-14071 Document troubleshooting unexpected filesystem usage by snapshots and WALs


Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/b29c731a
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/b29c731a
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/b29c731a

Branch: refs/heads/master
Commit: b29c731a57e2f37c4dd4401833ee2f29d0689ae3
Parents: 0daae43
Author: Misty Stanley-Jones <ms...@cloudera.com>
Authored: Mon Aug 10 09:04:48 2015 +1000
Committer: Misty Stanley-Jones <ms...@cloudera.com>
Committed: Mon Aug 10 14:07:03 2015 +1000

----------------------------------------------------------------------
 .../asciidoc/_chapters/troubleshooting.adoc     | 25 ++++++++++++++++++++
 1 file changed, 25 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hbase/blob/b29c731a/src/main/asciidoc/_chapters/troubleshooting.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/troubleshooting.adoc b/src/main/asciidoc/_chapters/troubleshooting.adoc
index a2c9247..8ae61b4 100644
--- a/src/main/asciidoc/_chapters/troubleshooting.adoc
+++ b/src/main/asciidoc/_chapters/troubleshooting.adoc
@@ -842,6 +842,31 @@ Two common use-cases for querying HDFS for HBase objects is research the degree
 If there are a large number of StoreFiles for each ColumnFamily it could indicate the need for a major compaction.
 Additionally, after a major compaction if the resulting StoreFile is "small" it could indicate the need for a reduction of ColumnFamilies for the table.
 
+=== Unexpected Filesystem Growth
+
+If you see an unexpected spike in filesystem usage by HBase, two possible culprits
+are snapshots and WALs.
+
+Snapshots::
+  When you create a snapshot, HBase retains everything it needs to recreate the table's
+  state at that time of tne snapshot. This includes deleted cells or expired versions.
+  For this reason, your snapshot usage pattern should be well-planned, and you should
+  prune snapshots that you no longer need. Snapshots are stored in `/hbase/.snapshots`,
+  and archives needed to restore snapshots are stored in
+  `/hbase/.archive/<_tablename_>/<_region_>/<_column_family_>/`.
+
+  *Do not* manage snapshots or archives manually via HDFS. HBase provides APIs and
+  HBase Shell commands for managing them. For more information, see <<ops.snapshots>>.
+
+WAL::
+  Write-ahead logs (WALs) are stored in subdirectories of `/hbase/.logs/`, depending
+  on their status. Already-processed WALs are stored in `/hbase/.logs/oldWALs/` and
+  corrupt WALs are stored in `/hbase/.logs/.corrupt/` for examination.
+  If the size of any subdirectory of `/hbase/.logs/` is growing, examine the HBase
+  server logs to find the root cause for why WALs are not being processed correctly.
+
+*Do not* manage WALs manually via HDFS.
+
 [[trouble.network]]
 == Network