You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@accumulo.apache.org by mw...@apache.org on 2017/12/18 21:29:22 UTC
[accumulo-website] branch asf-site updated: Jekyll build from
master:8da2b18
This is an automated email from the ASF dual-hosted git repository.
mwalch pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/accumulo-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 7ff6ffe Jekyll build from master:8da2b18
7ff6ffe is described below
commit 7ff6ffebb7d7dbcfbe5a2d8ffe8a34c8d1ae1815
Author: Mike Walch <mw...@apache.org>
AuthorDate: Mon Dec 18 16:29:06 2017 -0500
Jekyll build from master:8da2b18
Improved design documentation of tablet server (#49)
---
docs/2.0/administration/caching.html | 23 ++++++++++++-----------
docs/2.0/getting-started/design.html | 16 ++++++++++------
feed.xml | 4 ++--
images/docs/tablet_server.png | Bin 0 -> 51783 bytes
4 files changed, 24 insertions(+), 19 deletions(-)
diff --git a/docs/2.0/administration/caching.html b/docs/2.0/administration/caching.html
index adf483b..d7613e0 100644
--- a/docs/2.0/administration/caching.html
+++ b/docs/2.0/administration/caching.html
@@ -334,7 +334,7 @@
<div class="col-md-2"><a class="pull-right" style="margin-top: 25px;" href="https://github.com/apache/accumulo-website/edit/master/_docs-2-0/administration/caching.md" role="button"><i class="glyphicon glyphicon-pencil"></i> <small>Edit this page</small></a></div>
</div>
- <p>Accumulo tablet servers have a <strong>block cache</strong> that buffers data in memory to limit reads from disk.
+ <p>Accumulo <a href="/docs/2.0/getting-started/design#tablet-server-1">tablet servers</a> have <strong>block caches</strong> that buffer data in memory to limit reads from disk.
This caching has the following benefits:</p>
<ul>
@@ -342,22 +342,23 @@ This caching has the following benefits:</p>
<li>helps alleviate hotspots in tables</li>
</ul>
-<p>The block cache stores index and data blocks. A typical Accumulo read operation will perform a binary search
-over several index blocks followed by a linear scan of one or more data blocks. Each tablet server
-has its own block cache that is shared by all hosted tablets. Therefore, block caches are only enabled
+<p>Each tablet server has an index and data block cache that is shared by all hosted tablets (see the <a href="/docs/2.0/getting-started/design#tablet-server-1">tablet server diagram</a>
+to learn more). A typical Accumulo read operation will perform a binary search over several index blocks followed by a linear scan
+of one or more data blocks. If these blocks are not in a cache, they will need to be retrieved from <a href="/docs/2.0/getting-started/design#rfile">RFiles</a> in HDFS. While the index
+block cache is enabled for all tables, the data block cache has to be enabled for a table by the user. It is typically only enabled
for tables where read performance is critical.</p>
<h2 id="configuration">Configuration</h2>
-<p>While the block cache is enabled by default for the Accumulo metadata tables, it must be enabled
-for all other tables by setting the following table properties to <code class="highlighter-rouge">true</code>:</p>
+<p>The index and data block caches are configured for tables by the following properties:</p>
<ul>
- <li><a href="/docs/2.0/administration/properties#table_cache_block_enable">table.cache.block.enable</a> - enables data block cache on the table</li>
- <li><a href="/docs/2.0/administration/properties#table_cache_index_enable">table.cache.index.enable</a> - enables index block cache on the table</li>
+ <li><a href="/docs/2.0/administration/properties#table_cache_block_enable">table.cache.block.enable</a> - enables data block cache on the table (default is <code class="highlighter-rouge">false</code>)</li>
+ <li><a href="/docs/2.0/administration/properties#table_cache_index_enable">table.cache.index.enable</a> - enables index block cache on the table (default is <code class="highlighter-rouge">true</code>)</li>
</ul>
-<p>These properties can be set in the Accumulo shell using the following command:</p>
+<p>While the index block cache is enabled by default for all Accumulo tables, users must enable the data block cache by
+settting <a href="/docs/2.0/administration/properties#table_cache_block_enable">table.cache.block.enable</a> to <code class="highlighter-rouge">true</code> in the shell:</p>
<div class="highlighter-rouge"><pre class="highlight"><code>config -t mytable -s table.cache.block.enable=true
</code></pre>
@@ -369,8 +370,8 @@ for all other tables by setting the following table properties to <code class="h
</code></pre>
</div>
-<p>The sizes of the index and data block caches can be changed from their defaults by setting
-the following properties:</p>
+<p>The size of the index and data block caches (which are shared by all tablets of tablet server) can be changed from
+their defaults by setting the following properties:</p>
<ul>
<li><a href="/docs/2.0/administration/properties#tserver_cache_data_size">tserver.cache.data.size</a></li>
diff --git a/docs/2.0/getting-started/design.html b/docs/2.0/getting-started/design.html
index 3145a93..f3a03cf 100644
--- a/docs/2.0/getting-started/design.html
+++ b/docs/2.0/getting-started/design.html
@@ -366,7 +366,7 @@ one Master server and many Clients.</p>
<p>The TabletServer manages some subset of all the tablets (partitions of tables). This includes receiving writes from clients, persisting writes to a
write-ahead log, sorting new key-value pairs in memory, periodically
flushing sorted key-value pairs to new files in HDFS, and responding
-to reads from clients, forming a merge-sorted view of all keys and
+to reads from clients, forming a sorted merge view of all keys and
values from all the files it has created and the sorted in-memory
store.</p>
@@ -432,7 +432,7 @@ ingest and query load is balanced across the cluster.</p>
<p><img src="https://accumulo.apache.org/images/docs/data_distribution.png" alt="data distribution" /></p>
-<h2 id="tablet-service">Tablet Service</h2>
+<h2 id="tablet-server-1">Tablet Server</h2>
<p>When a write arrives at a TabletServer it is written to a Write-Ahead Log and
then inserted into a sorted data structure in memory called a MemTable. When the
@@ -442,10 +442,14 @@ called a minor compaction. A new MemTable is then created and the fact of the
compaction is recorded in the Write-Ahead Log.</p>
<p>When a request to read data arrives at a TabletServer, the TabletServer does a
-binary search across the MemTable as well as the in-memory indexes associated
-with each RFile to find the relevant values. If clients are performing a scan,
-several key-value pairs are returned to the client in order from the MemTable
-and the set of RFiles by performing a merge-sort as they are read.</p>
+binary search across the MemTable as well as the index blocks associated with each RFile
+to find the relevant values. If clients are performing a scan, several key-value pairs
+are returned to the client in order from the MemTable and data blocks of RFiles by performing
+a sorted merge as they are read. If <a href="/docs/2.0/administration/caching">caching</a> is enabled for the table, any index or data
+block is stored in the block cache to speed up future scans.</p>
+
+<p><img src="https://accumulo.apache.org/images/docs/tablet_server.png" alt="tablet server diagram" />
+<!-- Source at https://docs.google.com/presentation/d/1yEBNM044FxrzksVfxU35WDbxcVWUYUMy3tgRP75dzus/edit?usp=sharing --></p>
<h2 id="rfile">RFile</h2>
diff --git a/feed.xml b/feed.xml
index fd900e3..d32b7f4 100644
--- a/feed.xml
+++ b/feed.xml
@@ -6,8 +6,8 @@
</description>
<link>https://accumulo.apache.org/</link>
<atom:link href="https://accumulo.apache.org/feed.xml" rel="self" type="application/rss+xml"/>
- <pubDate>Wed, 13 Dec 2017 12:04:54 -0500</pubDate>
- <lastBuildDate>Wed, 13 Dec 2017 12:04:54 -0500</lastBuildDate>
+ <pubDate>Mon, 18 Dec 2017 16:28:52 -0500</pubDate>
+ <lastBuildDate>Mon, 18 Dec 2017 16:28:52 -0500</lastBuildDate>
<generator>Jekyll v3.5.2</generator>
diff --git a/images/docs/tablet_server.png b/images/docs/tablet_server.png
new file mode 100644
index 0000000..2581dd0
Binary files /dev/null and b/images/docs/tablet_server.png differ
--
To stop receiving notification emails like this one, please contact
['"commits@accumulo.apache.org" <co...@accumulo.apache.org>'].