You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hbase.apache.org by en...@apache.org on 2015/04/18 07:30:18 UTC
[1/2] hbase git commit: Updated CHANGES.txt for 1.0.1RC2
Repository: hbase
Updated Branches:
refs/heads/branch-1.0 a2035f0dc -> 66a93c09d
Updated CHANGES.txt for 1.0.1RC2
Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/4f5d9bb2
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/4f5d9bb2
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/4f5d9bb2
Branch: refs/heads/branch-1.0
Commit: 4f5d9bb22efb0009c6d6b42911964becb0cec76f
Parents: a2035f0
Author: Enis Soztutar <en...@apache.org>
Authored: Fri Apr 17 22:03:09 2015 -0700
Committer: Enis Soztutar <en...@apache.org>
Committed: Fri Apr 17 22:03:09 2015 -0700
----------------------------------------------------------------------
CHANGES.txt | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hbase/blob/4f5d9bb2/CHANGES.txt
----------------------------------------------------------------------
diff --git a/CHANGES.txt b/CHANGES.txt
index e8f3200..400c026 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,6 +1,6 @@
HBase Change Log
-Release Notes - HBase - Version 1.0.1 04/16/2015
+Release Notes - HBase - Version 1.0.1 04/25/2015
** Sub-task
* [HBASE-13006] - Document visibility label support for groups
@@ -10,9 +10,13 @@ Release Notes - HBase - Version 1.0.1 04/16/2015
* [HBASE-13332] - Fix the usage of doAs/runAs in Visibility Controller tests.
* [HBASE-13335] - Update ClientSmallScanner and ClientSmallReversedScanner
* [HBASE-13386] - Backport HBASE-12601 to all active branches other than master
+ * [HBASE-13479] - [branch-1.0] Master should not bind to region server ports
+ * [HBASE-13481] - Master should respect master (old) DNS/bind related configurations
** Bug
* [HBASE-10728] - get_counter value is never used.
+ * [HBASE-11542] - Unit Test KeyStoreTestUtil.java compilation failure in IBM JDK
+ * [HBASE-12006] - [JDK 8] KeyStoreTestUtil#generateCertificate fails due to "subject class type invalid"
* [HBASE-12102] - Duplicate keys in HBase.RegionServer metrics JSON
* [HBASE-12908] - Typos in MemStoreFlusher javadocs
* [HBASE-12931] - The existing KeyValues in memstore are not removed completely after inserting cell into memStore
@@ -72,6 +76,7 @@ Release Notes - HBase - Version 1.0.1 04/16/2015
* [HBASE-13294] - Fix the critical ancient loopholes in security testing infrastructure.
* [HBASE-13296] - Fix the deletion of acl notify nodes for namespace.
* [HBASE-13298] - Clarify if Table.{set|get}WriteBufferSize() is deprecated or not
+ * [HBASE-13301] - Possible memory leak in BucketCache
* [HBASE-13305] - Get(Get get) is not copying the row key
* [HBASE-13309] - Some tests do not reset EnvironmentEdgeManager
* [HBASE-13314] - Fix NPE in HMaster.getClusterStatus()
@@ -85,7 +90,11 @@ Release Notes - HBase - Version 1.0.1 04/16/2015
* [HBASE-13409] - Add categories to uncategorized tests
* [HBASE-13410] - Bug in KeyValueUtil.oswrite() for non Keyvalue cases
* [HBASE-13414] - TestHCM no longer needs to test for JRE 6.
+ * [HBASE-13423] - Remove duplicate entry for hbase.regionserver.regionSplitLimit in hbase-default.xml
* [HBASE-13433] - Backward compatibility in 1.0.z for shell broken for get_counter
+ * [HBASE-13457] - SnapshotExistsException doesn't honor the DoNotRetry
+ * [HBASE-13473] - deleted cells come back alive after the stripe compaction
+ * [HBASE-13491] - Issue in FuzzyRowFilter#getNextForFuzzyRule
** Improvement
* [HBASE-13002] - Make encryption cipher configurable
@@ -111,6 +120,8 @@ Release Notes - HBase - Version 1.0.1 04/16/2015
* [HBASE-13362] - Set max result size from client only (like scanner caching).
* [HBASE-13370] - PE tool could give option for using Explicit Column Tracker which leads to seeks
* [HBASE-13381] - Expand TestSizeFailures to include small scans
+ * [HBASE-13419] - Thrift gateway should propagate text from exception causes.
+ * [HBASE-13436] - Include user name in ADE for scans
** New Feature
* [HBASE-12869] - Add a REST API implementation of the ClusterManager interface
[2/2] hbase git commit: Updated book from master
Posted by en...@apache.org.
Updated book from master
Project: http://git-wip-us.apache.org/repos/asf/hbase/repo
Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/66a93c09
Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/66a93c09
Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/66a93c09
Branch: refs/heads/branch-1.0
Commit: 66a93c09df3b12ff7b86c39bc8475c60e15af82d
Parents: 4f5d9bb
Author: Enis Soztutar <en...@apache.org>
Authored: Fri Apr 17 22:06:38 2015 -0700
Committer: Enis Soztutar <en...@apache.org>
Committed: Fri Apr 17 22:06:38 2015 -0700
----------------------------------------------------------------------
src/main/asciidoc/_chapters/datamodel.adoc | 2 +-
src/main/asciidoc/_chapters/schema_design.adoc | 96 ++++++++++++++++++++-
2 files changed, 96 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hbase/blob/66a93c09/src/main/asciidoc/_chapters/datamodel.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/datamodel.adoc b/src/main/asciidoc/_chapters/datamodel.adoc
index 74238ca..b76adc8 100644
--- a/src/main/asciidoc/_chapters/datamodel.adoc
+++ b/src/main/asciidoc/_chapters/datamodel.adoc
@@ -495,7 +495,7 @@ For an informative discussion on how deletes and versioning interact, see the th
Also see <<keyvalue,keyvalue>> for more information on the internal KeyValue format.
-Delete markers are purged during the next major compaction of the store, unless the `KEEP_DELETED_CELLS` option is set in the column family.
+Delete markers are purged during the next major compaction of the store, unless the `KEEP_DELETED_CELLS` option is set in the column family (See <<cf.keep.deleted>>).
To keep the deletes for a configurable amount of time, you can set the delete TTL via the +hbase.hstore.time.to.purge.deletes+ property in _hbase-site.xml_.
If `hbase.hstore.time.to.purge.deletes` is not set, or set to 0, all delete markers, including those with timestamps in the future, are purged during the next major compaction.
Otherwise, a delete marker with a timestamp in the future is kept until the major compaction which occurs after the time represented by the marker's timestamp plus the value of `hbase.hstore.time.to.purge.deletes`, in milliseconds.
http://git-wip-us.apache.org/repos/asf/hbase/blob/66a93c09/src/main/asciidoc/_chapters/schema_design.adoc
----------------------------------------------------------------------
diff --git a/src/main/asciidoc/_chapters/schema_design.adoc b/src/main/asciidoc/_chapters/schema_design.adoc
index 28f28a5..9319c65 100644
--- a/src/main/asciidoc/_chapters/schema_design.adoc
+++ b/src/main/asciidoc/_chapters/schema_design.adoc
@@ -461,7 +461,101 @@ HColumnDescriptor.setKeepDeletedCells(true);
----
====
-See the API documentation for link:http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/HColumnDescriptor.html#KEEP_DELETED_CELLS[KEEP_DELETED_CELLS] for more information.
+Let us illustrate the basic effect of setting the `KEEP_DELETED_CELLS` attribute on a table.
+
+First, without:
+[source]
+----
+create 'test', {NAME=>'e', VERSIONS=>2147483647}
+put 'test', 'r1', 'e:c1', 'value', 10
+put 'test', 'r1', 'e:c1', 'value', 12
+put 'test', 'r1', 'e:c1', 'value', 14
+delete 'test', 'r1', 'e:c1', 11
+
+hbase(main):017:0> scan 'test', {RAW=>true, VERSIONS=>1000}
+ROW COLUMN+CELL
+ r1 column=e:c1, timestamp=14, value=value
+ r1 column=e:c1, timestamp=12, value=value
+ r1 column=e:c1, timestamp=11, type=DeleteColumn
+ r1 column=e:c1, timestamp=10, value=value
+1 row(s) in 0.0120 seconds
+
+hbase(main):018:0> flush 'test'
+0 row(s) in 0.0350 seconds
+
+hbase(main):019:0> scan 'test', {RAW=>true, VERSIONS=>1000}
+ROW COLUMN+CELL
+ r1 column=e:c1, timestamp=14, value=value
+ r1 column=e:c1, timestamp=12, value=value
+ r1 column=e:c1, timestamp=11, type=DeleteColumn
+1 row(s) in 0.0120 seconds
+
+hbase(main):020:0> major_compact 'test'
+0 row(s) in 0.0260 seconds
+
+hbase(main):021:0> scan 'test', {RAW=>true, VERSIONS=>1000}
+ROW COLUMN+CELL
+ r1 column=e:c1, timestamp=14, value=value
+ r1 column=e:c1, timestamp=12, value=value
+1 row(s) in 0.0120 seconds
+----
+
+Notice how delete cells are let go.
+
+Now lets run the same test only with `KEEP_DELETED_CELLS` set on the table (you can do table or per-column-family):
+
+[source]
+----
+hbase(main):005:0> create 'test', {NAME=>'e', VERSIONS=>2147483647, KEEP_DELETED_CELLS => true}
+0 row(s) in 0.2160 seconds
+
+=> Hbase::Table - test
+hbase(main):006:0> put 'test', 'r1', 'e:c1', 'value', 10
+0 row(s) in 0.1070 seconds
+
+hbase(main):007:0> put 'test', 'r1', 'e:c1', 'value', 12
+0 row(s) in 0.0140 seconds
+
+hbase(main):008:0> put 'test', 'r1', 'e:c1', 'value', 14
+0 row(s) in 0.0160 seconds
+
+hbase(main):009:0> delete 'test', 'r1', 'e:c1', 11
+0 row(s) in 0.0290 seconds
+
+hbase(main):010:0> scan 'test', {RAW=>true, VERSIONS=>1000}
+ROW COLUMN+CELL
+ r1 column=e:c1, timestamp=14, value=value
+ r1 column=e:c1, timestamp=12, value=value
+ r1 column=e:c1, timestamp=11, type=DeleteColumn
+ r1 column=e:c1, timestamp=10, value=value
+1 row(s) in 0.0550 seconds
+
+hbase(main):011:0> flush 'test'
+0 row(s) in 0.2780 seconds
+
+hbase(main):012:0> scan 'test', {RAW=>true, VERSIONS=>1000}
+ROW COLUMN+CELL
+ r1 column=e:c1, timestamp=14, value=value
+ r1 column=e:c1, timestamp=12, value=value
+ r1 column=e:c1, timestamp=11, type=DeleteColumn
+ r1 column=e:c1, timestamp=10, value=value
+1 row(s) in 0.0620 seconds
+
+hbase(main):013:0> major_compact 'test'
+0 row(s) in 0.0530 seconds
+
+hbase(main):014:0> scan 'test', {RAW=>true, VERSIONS=>1000}
+ROW COLUMN+CELL
+ r1 column=e:c1, timestamp=14, value=value
+ r1 column=e:c1, timestamp=12, value=value
+ r1 column=e:c1, timestamp=11, type=DeleteColumn
+ r1 column=e:c1, timestamp=10, value=value
+1 row(s) in 0.0650 seconds
+----
+
+KEEP_DELETED_CELLS is to avoid removing Cells from HBase when the _only_ reason to remove them is the delete marker.
+So with KEEP_DELETED_CELLS enabled deleted cells would get removed if either you write more versions than the configured max, or you have a TTL and Cells are in excess of the configured timeout, etc.
+
[[secondary.indexes]]
== Secondary Indexes and Alternate Query Paths