You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@accumulo.apache.org by kt...@apache.org on 2018/08/02 17:40:00 UTC
[accumulo-website] branch asf-site updated: Jekyll build from
master:7b25602
This is an automated email from the ASF dual-hosted git repository.
kturner pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/accumulo-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 5545c5d Jekyll build from master:7b25602
5545c5d is described below
commit 5545c5d2d3da047c91f5dc36e43fc0a5151fedaf
Author: Keith Turner <kt...@apache.org>
AuthorDate: Thu Aug 2 13:39:29 2018 -0400
Jekyll build from master:7b25602
Add docs for scan execution hints apache/accumulo#571 (#100)
---
docs/2.0/administration/scan-executors.html | 37 ++++++++++++++++++++++++++---
feed.xml | 4 ++--
redirects.json | 2 +-
search_data.json | 2 +-
4 files changed, 38 insertions(+), 7 deletions(-)
diff --git a/docs/2.0/administration/scan-executors.html b/docs/2.0/administration/scan-executors.html
index 36a9a58..99756b2 100644
--- a/docs/2.0/administration/scan-executors.html
+++ b/docs/2.0/administration/scan-executors.html
@@ -374,7 +374,7 @@ tserver.scan.executors.<name>.prioritizer.opts.<key>=<value>
<p>After creating an executor, configure <a href="/docs/2.0/administration/properties#table_scan_dispatcher">table.scan.dispatcher</a> to use it. A
dispatcher is Java subclass of <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-core/1.9.2/org/apache/accumulo/core/spi/scan/ScanDispatcher.html">ScanDispatcher</a>
-that decides which scan executor should service a table. Set the following table
+that decides which scan executor should service a table. Set the following table
property to configure a dispatcher.</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>table.scan.dispatcher=<class name>
@@ -413,12 +413,15 @@ config -s tserver.scan.executors.high.threads=8
<p>Tablet servers should be restarted after configuring scan executors, then tables can be configured.</p>
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>config -t LOW1 -s table.scan.dispatcher.opts.executor=low
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>config -t LOW1 -s table.scan.dispatcher=org.apache.accumulo.core.spi.scan.SimpleScanDispatcher
+config -t LOW1 -s table.scan.dispatcher.opts.executor=low
+config -t LOW2 -s table.scan.dispatcher=org.apache.accumulo.core.spi.scan.SimpleScanDispatcher
config -t LOW2 -s table.scan.dispatcher.opts.executor=low
+config -t HIGH -s table.scan.dispatcher=org.apache.accumulo.core.spi.scan.SimpleScanDispatcher
config -t HIGH -s table.scan.dispatcher.opts.executor=high
</code></pre></div></div>
-<p>While not necessary because its the default, it would be safer to also set
+<p>While not necessary because its the default, it is safer to also set
<code class="highlighter-rouge">table.scan.dispatcher=org.apache.accumulo.core.spi.scan.SimpleScanDispatcher</code>
for each table. This ensures things work as expected in the case where
<code class="highlighter-rouge">table.scan.dispatcher</code> was set at the system or namespace level.</p>
@@ -441,6 +444,34 @@ executor.</p>
threads repeatedly doing small random lookups made a significant difference.
In this test the average lookup time for the 5 threads went from 250ms to 5 ms.</p>
+<h3 id="providing-hints-from-the-client-side">Providing hints from the client side.</h3>
+
+<p>Scanners can provide hints to ScanDispatchers and ScanPriotizers by calling
+<a href="https://static.javadoc.io/org.apache.accumulo/accumulo-core/1.9.2/org/apache/accumulo/core/client/ScannerBase.html#setExecutionHints-java.util.Map-">setExecutionHints</a> on the Scanner. What, if anything, is done with these
+hints depends on what is configured for the table and system. Accumulo’s
+default configuration ignores hints. The following shell commands make it
+possible to choose an executor and set priorities from a scanner for the
+table <code class="highlighter-rouge">tex</code>.</p>
+
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>config -s tserver.scan.executors.special.threads=8
+config -s tserver.scan.executors.special.prioritizer=org.apache.accumulo.core.spi.scan.HintScanPrioritizer
+createtable tex
+config -t tex -s table.scan.dispatcher=org.apache.accumulo.core.spi.scan.SimpleScanDispatcher
+config -t tex -s table.scan.dispatcher.opts.heed_hints=true
+</code></pre></div></div>
+
+<p>The <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-core/1.9.2/org/apache/accumulo/core/spi/scan/HintScanPrioritizer.html">HintScanPrioritizer</a> honors
+hints of the form <code class="highlighter-rouge">priority=<integer></code> to prioritize scans, with lower integers
+resulting in a higher priority. The <code class="highlighter-rouge">SimpleScanDispatcher</code>, which is the
+default dispatcher, supports the <code class="highlighter-rouge">heed_hints</code> option. By default the
+<code class="highlighter-rouge">SimpleScanDispatcher</code> ignores hints, but when <code class="highlighter-rouge">heed_hints</code> is set to <code class="highlighter-rouge">true</code> it
+will honor hints of the form <code class="highlighter-rouge">executor=<executor name></code> when choosing an
+executor. After restarting tservers, the following command will start a scan
+that uses the executor <code class="highlighter-rouge">special</code> with a priority of 3.</p>
+
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>scan -t tex --execution-hints priority=3,executor=special
+</code></pre></div></div>
+
<div class="row" style="margin-top: 20px;">
diff --git a/feed.xml b/feed.xml
index 7e4737f..936cb97 100644
--- a/feed.xml
+++ b/feed.xml
@@ -6,8 +6,8 @@
</description>
<link>https://accumulo.apache.org/</link>
<atom:link href="https://accumulo.apache.org/feed.xml" rel="self" type="application/rss+xml"/>
- <pubDate>Wed, 25 Jul 2018 09:24:49 -0400</pubDate>
- <lastBuildDate>Wed, 25 Jul 2018 09:24:49 -0400</lastBuildDate>
+ <pubDate>Thu, 02 Aug 2018 13:39:23 -0400</pubDate>
+ <lastBuildDate>Thu, 02 Aug 2018 13:39:23 -0400</lastBuildDate>
<generator>Jekyll v3.7.3</generator>
diff --git a/redirects.json b/redirects.json
index 2ec52c0..580f60e 100644
--- a/redirects.json
+++ b/redirects.json
@@ -1 +1 @@
-{"/release_notes/1.5.1.html":"https://accumulo.apache.org/release/accumulo-1.5.1/","/release_notes/1.6.0.html":"https://accumulo.apache.org/release/accumulo-1.6.0/","/release_notes/1.6.1.html":"https://accumulo.apache.org/release/accumulo-1.6.1/","/release_notes/1.6.2.html":"https://accumulo.apache.org/release/accumulo-1.6.2/","/release_notes/1.7.0.html":"https://accumulo.apache.org/release/accumulo-1.7.0/","/release_notes/1.5.3.html":"https://accumulo.apache.org/release/accumulo-1.5.3/" [...]
\ No newline at end of file
+{"/release_notes/1.5.1.html":"https://accumulo.apache.org/release/accumulo-1.5.1/","/release_notes/1.6.0.html":"https://accumulo.apache.org/release/accumulo-1.6.0/","/release_notes/1.6.1.html":"https://accumulo.apache.org/release/accumulo-1.6.1/","/release_notes/1.6.2.html":"https://accumulo.apache.org/release/accumulo-1.6.2/","/release_notes/1.7.0.html":"https://accumulo.apache.org/release/accumulo-1.7.0/","/release_notes/1.5.3.html":"https://accumulo.apache.org/release/accumulo-1.5.3/" [...]
\ No newline at end of file
diff --git a/search_data.json b/search_data.json
index 588cb58..b844660 100644
--- a/search_data.json
+++ b/search_data.json
@@ -65,7 +65,7 @@
"docs-2-0-administration-scan-executors": {
"title": "Scan Executors",
- "content" : "Accumulo scans operate by repeatedly fetching batches of data from a tabletserver. On the tablet server side, a thread pool fetches batches.In Java threads pools are called executors. By default, a single executor pertablet server handles all scans in FIFO order. For some workloads, the singleFIFO executor is suboptimal. For example, consider many unimportant scansreading lots of data mixed with a few important scans reading small amounts ofdata. The long scans n [...]
+ "content" : "Accumulo scans operate by repeatedly fetching batches of data from a tabletserver. On the tablet server side, a thread pool fetches batches.In Java threads pools are called executors. By default, a single executor pertablet server handles all scans in FIFO order. For some workloads, the singleFIFO executor is suboptimal. For example, consider many unimportant scansreading lots of data mixed with a few important scans reading small amounts ofdata. The long scans n [...]
"url": " /docs/2.0/administration/scan-executors",
"categories": "administration"
},