You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@fluo.apache.org by kt...@apache.org on 2016/10/10 19:33:59 UTC
[3/4] incubator-fluo-website git commit: Jekyll build from gh-pages:
b359233
http://git-wip-us.apache.org/repos/asf/incubator-fluo-website/blob/76022205/tour/exercise-1/index.html
----------------------------------------------------------------------
diff --git a/tour/exercise-1/index.html b/tour/exercise-1/index.html
new file mode 100644
index 0000000..d4dc119
--- /dev/null
+++ b/tour/exercise-1/index.html
@@ -0,0 +1,545 @@
+<!DOCTYPE html>
+<html>
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link rel="stylesheet" href="/stylesheets/fluo.css">
+ <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Montserrat:700,400">
+ <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Merriweather">
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css" >
+ <link rel="canonical" href="http://fluo.io/tour/exercise-1/">
+ <link rel="icon" type="image/png" href="/resources/favicon.png">
+ <title>Word count Exercise | Apache Fluo</title>
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ </head>
+ <body>
+ <div id="fluo-nav" class="navbar navbar-default navbar-static-top">
+ <div class="container">
+ <div class="navbar-header">
+ <div class="navbar-toggle-wrapper visible-xs">
+ <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".js-navbar-collapse">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ </div>
+ <a href="/" class="navbar-brand"><img src="/resources/fluo-logo.png" alt="Apache Fluo"></a>
+ </div>
+ <div class="collapse navbar-collapse js-navbar-collapse" style="margin-top: 20px">
+ <ul class="navbar-nav nav">
+ <li><a href="/docs/">Documentation</a></li>
+ <li><a href="/download/">Download</a></li>
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="/getinvolved/">Get Involved</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ <li><a href="/people/">People</a></li>
+ <li><a href="/related-projects/">Related Projects</a></li>
+ <li><a href="/poweredby/">Powered By</a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Contributing<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="/how-to-contribute/">How To Contribute</a></li>
+ <li><a href="/release-process/">Release Process</a></li>
+ </ul>
+ </li>
+ </ul>
+ <ul class="navbar-nav nav navbar-right">
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="https://www.apache.org">Apache Homepage</a></li>
+ <li><a href="https://www.apache.org/licenses/LICENSE-2.0">License</a></li>
+ <li><a href="https://www.apache.org/foundation/sponsorship">Sponsorship</i></a></li>
+ <li><a href="https://www.apache.org/security">Security</a></li>
+ <li><a href="https://www.apache.org/foundation/thanks">Thanks</a></li>
+ <li><a href="https://www.apache.org/foundation/policies/conduct">Code of Conduct</a></li>
+ </ul>
+ </li>
+ </ul>
+ </div>
+ </div>
+ </div>
+ <div class="container">
+ <div class="row">
+ <div class="col-sm-12">
+
+
+
+<div id="tour-header">
+ <h2><a href="/tour/">Fluo Tour</a>: Word count Exercise</h2>
+ <p class="text-muted">Tour page 19 of 26</p>
+</div>
+<div id="tour-content">
+ <p>This excercise gives you an opportunity to use everything you have learned so
+far to attempt writing a simple Fluo application. A bare minimum of code,
+along with a conceptual sketch of a solution, is provided to get you started.</p>
+
+<p>The application should compute word counts for unique documents. This
+application should do the following.</p>
+
+<ul>
+ <li>Deduplicate content based on hash</li>
+ <li>Count how many URIs reference content</li>
+ <li>For the unique words in content, update global word counts.</li>
+ <li>When new content is added increment the global counts.</li>
+ <li>When content is no longer referenced by any URIs, decrement the global word counts and delete
+that content.</li>
+ <li>Partition different types of data using row prefixes. Use <em>u:</em> for URIs, use <em>d:</em> for document
+content, and use <em>w:</em> for word counts.</li>
+</ul>
+
+<h2 id="part-1--loading-data">Part 1 : Loading data.</h2>
+
+<p>The class below is a simple POJO for documents.</p>
+
+<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="kn">package</span> <span class="n">ft</span><span class="o">;</span>
+
+<span class="kn">import</span> <span class="nn">com.google.common.hash.Hashing</span><span class="o">;</span>
+
+<span class="kd">public</span> <span class="kd">class</span> <span class="nc">Document</span> <span class="o">{</span>
+ <span class="kd">public</span> <span class="kd">final</span> <span class="n">String</span> <span class="n">uri</span><span class="o">;</span>
+ <span class="kd">public</span> <span class="kd">final</span> <span class="n">String</span> <span class="n">content</span><span class="o">;</span>
+
+ <span class="kd">public</span> <span class="nf">Document</span><span class="o">(</span><span class="n">String</span> <span class="n">uri</span><span class="o">,</span> <span class="n">String</span> <span class="n">content</span><span class="o">)</span> <span class="o">{</span>
+ <span class="k">this</span><span class="o">.</span><span class="na">uri</span> <span class="o">=</span> <span class="n">uri</span><span class="o">;</span>
+ <span class="k">this</span><span class="o">.</span><span class="na">content</span> <span class="o">=</span> <span class="n">content</span><span class="o">;</span>
+ <span class="o">}</span>
+
+ <span class="kd">public</span> <span class="n">String</span> <span class="nf">hash</span><span class="o">()</span> <span class="o">{</span>
+ <span class="c1">//use short prefix of hash for example</span>
+ <span class="k">return</span> <span class="n">Hashing</span><span class="o">.</span><span class="na">sha1</span><span class="o">().</span><span class="na">hashString</span><span class="o">(</span><span class="n">content</span><span class="o">).</span><span class="na">toString</span><span class="o">().</span><span class="na">substring</span><span class="o">(</span><span class="mi">0</span><span class="o">,</span> <span class="mi">7</span><span class="o">);</span>
+ <span class="o">}</span>
+<span class="o">}</span>
+</code></pre>
+</div>
+
+<p>The following code loads documents into Fluo. It should do the following :</p>
+
+<ul>
+ <li>Keep track of the current hash associated with a URI.</li>
+ <li>Deduplicate content based on hash</li>
+ <li>Reference count how many URIs point to content. Track this information in a column named
+<em>doc:refc</em> with a row based on the hash.</li>
+ <li>Track the status of whether content is referenced or unreferenced in a column named <em>doc:refs</em>.
+Note <em>refs</em> is short for reference status. When the reference count for content is 0 this
+columns value should be <em>unreferenced</em>. When the reference count is greater than 0, the
+<em>doc:refs</em> columns value should be <em>referenced</em>. In a later example, an Observer will watch this
+column.</li>
+ <li>Track the content associated with a hash using the <em>doc:content</em> column.</li>
+</ul>
+
+<p>Some of this is implemented below, but not all. The parts that are not done have TODOs.</p>
+
+<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="kn">package</span> <span class="n">ft</span><span class="o">;</span>
+
+<span class="kn">import</span> <span class="nn">org.apache.fluo.api.client.Loader</span><span class="o">;</span>
+<span class="kn">import</span> <span class="nn">org.apache.fluo.api.client.TransactionBase</span><span class="o">;</span>
+<span class="kn">import</span> <span class="nn">org.apache.fluo.api.data.Column</span><span class="o">;</span>
+
+<span class="kd">public</span> <span class="kd">class</span> <span class="nc">DocLoader</span> <span class="kd">implements</span> <span class="n">Loader</span> <span class="o">{</span>
+
+ <span class="kd">private</span> <span class="kd">final</span> <span class="n">Document</span> <span class="n">doc</span><span class="o">;</span>
+
+ <span class="kd">public</span> <span class="kd">static</span> <span class="kd">final</span> <span class="n">Column</span> <span class="n">HASH_COL</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Column</span><span class="o">(</span><span class="s">"uri"</span><span class="o">,</span> <span class="s">"hash"</span><span class="o">);</span>
+ <span class="kd">public</span> <span class="kd">static</span> <span class="kd">final</span> <span class="n">Column</span> <span class="n">REF_COUNT_COL</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Column</span><span class="o">(</span><span class="s">"doc"</span><span class="o">,</span> <span class="s">"refc"</span><span class="o">);</span>
+ <span class="kd">public</span> <span class="kd">static</span> <span class="kd">final</span> <span class="n">Column</span> <span class="n">REF_STATUS_COL</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Column</span><span class="o">(</span><span class="s">"doc"</span><span class="o">,</span> <span class="s">"refs"</span><span class="o">);</span>
+ <span class="kd">public</span> <span class="kd">static</span> <span class="kd">final</span> <span class="n">Column</span> <span class="n">CONTENT_COL</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Column</span><span class="o">(</span><span class="s">"doc"</span><span class="o">,</span> <span class="s">"content"</span><span class="o">);</span>
+
+ <span class="kd">public</span> <span class="nf">DocLoader</span><span class="o">(</span><span class="n">Document</span> <span class="n">doc</span><span class="o">)</span> <span class="o">{</span>
+ <span class="k">this</span><span class="o">.</span><span class="na">doc</span> <span class="o">=</span> <span class="n">doc</span><span class="o">;</span>
+ <span class="o">}</span>
+
+ <span class="nd">@Override</span>
+ <span class="kd">public</span> <span class="kt">void</span> <span class="nf">load</span><span class="o">(</span><span class="n">TransactionBase</span> <span class="n">tx</span><span class="o">,</span> <span class="n">Context</span> <span class="n">context</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
+ <span class="n">String</span> <span class="n">newHash</span> <span class="o">=</span> <span class="n">doc</span><span class="o">.</span><span class="na">hash</span><span class="o">();</span>
+ <span class="n">String</span> <span class="n">oldHash</span> <span class="o">=</span> <span class="n">tx</span><span class="o">.</span><span class="na">gets</span><span class="o">(</span><span class="s">"u:"</span> <span class="o">+</span> <span class="n">doc</span><span class="o">.</span><span class="na">uri</span><span class="o">,</span> <span class="n">HASH_COL</span><span class="o">);</span>
+
+ <span class="c1">// TODO check if uri already has the same content hash. If so, then nothing to do.</span>
+
+ <span class="c1">// TODO set the new hash associated with the URI</span>
+
+ <span class="k">if</span> <span class="o">(</span><span class="n">oldHash</span> <span class="o">!=</span> <span class="kc">null</span><span class="o">)</span> <span class="o">{</span>
+ <span class="c1">// TODO decrement the reference count at row "d:"+oldHash</span>
+ <span class="c1">// TODO set REF_STATUS_COL to "unreferenced" when the reference count goes from 1 to 0. Do</span>
+ <span class="c1">// this for row "d:"+oldHash</span>
+ <span class="o">}</span>
+
+ <span class="c1">// TODO increment the reference count for the newHash content.</span>
+ <span class="c1">// TODO add the new content when the reference count does not exists</span>
+ <span class="c1">// TODO set REF_STATUS_COL to "referenced" when the reference count for the new content goes</span>
+ <span class="c1">// from 0 to 1. Do this for row "d:"+newHash</span>
+ <span class="o">}</span>
+<span class="o">}</span>
+</code></pre>
+</div>
+
+<p>Add the following to the ft.Main class.</p>
+
+<div class="language-java highlighter-rouge"><pre class="highlight"><code> <span class="c1">// some test data</span>
+ <span class="kd">private</span> <span class="kd">static</span> <span class="n">Document</span><span class="o">[]</span> <span class="n">docs1</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Document</span><span class="o">[]</span> <span class="o">{</span>
+ <span class="k">new</span> <span class="nf">Document</span><span class="o">(</span><span class="s">"http://news.com/a23"</span><span class="o">,</span>
+ <span class="s">"Jebediah orbits Mun for 35 days. No power, forgot solar panels."</span><span class="o">),</span>
+ <span class="k">new</span> <span class="nf">Document</span><span class="o">(</span><span class="s">"http://news.com/a24"</span><span class="o">,</span>
+ <span class="s">"Bill plans to rescue Jebediah after taking tourist to Minimus."</span><span class="o">)};</span>
+
+ <span class="kd">private</span> <span class="kd">static</span> <span class="n">Document</span><span class="o">[]</span> <span class="n">docs2</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Document</span><span class="o">[]</span> <span class="o">{</span><span class="k">new</span> <span class="n">Document</span><span class="o">(</span><span class="s">"http://oldnews.com/a23"</span><span class="o">,</span>
+ <span class="s">"Jebediah orbits Mun for 35 days. No power, forgot solar panels."</span><span class="o">)};</span>
+
+ <span class="kd">private</span> <span class="kd">static</span> <span class="n">Document</span><span class="o">[]</span> <span class="n">docs3</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Document</span><span class="o">[]</span> <span class="o">{</span>
+ <span class="k">new</span> <span class="nf">Document</span><span class="o">(</span><span class="s">"http://news.com/a23"</span><span class="o">,</span>
+ <span class="s">"Jebediah orbits Mun for 38 days. No power, forgot solar panels."</span><span class="o">),</span>
+ <span class="k">new</span> <span class="nf">Document</span><span class="o">(</span><span class="s">"http://news.com/a24"</span><span class="o">,</span>
+ <span class="s">"Crisis at KSC. Tourist stuck at Minimus. Bill forgot solar panels."</span><span class="o">)};</span>
+
+ <span class="cm">/**
+ * Utility method for loading documents and printing out Fluo table after load completes.
+ */</span>
+ <span class="kd">private</span> <span class="kd">static</span> <span class="kt">void</span> <span class="nf">loadAndPrint</span><span class="o">(</span><span class="n">MiniFluo</span> <span class="n">mini</span><span class="o">,</span> <span class="n">FluoClient</span> <span class="n">client</span><span class="o">,</span> <span class="n">Document</span><span class="o">[]</span> <span class="n">docs</span><span class="o">)</span> <span class="o">{</span>
+
+ <span class="k">try</span> <span class="o">(</span><span class="n">LoaderExecutor</span> <span class="n">loaderExecutor</span> <span class="o">=</span> <span class="n">client</span><span class="o">.</span><span class="na">newLoaderExecutor</span><span class="o">())</span> <span class="o">{</span>
+ <span class="k">for</span> <span class="o">(</span><span class="n">Document</span> <span class="n">document</span> <span class="o">:</span> <span class="n">docs</span><span class="o">)</span> <span class="o">{</span>
+ <span class="n">loaderExecutor</span><span class="o">.</span><span class="na">execute</span><span class="o">(</span><span class="k">new</span> <span class="n">DocLoader</span><span class="o">(</span><span class="n">document</span><span class="o">));</span>
+ <span class="o">}</span>
+ <span class="o">}</span> <span class="c1">// this will close loaderExecutor and wait for all load transactions to complete</span>
+
+ <span class="c1">//This line is not needed in this step of the excercise. However the next step will need this</span>
+ <span class="c1">//line.</span>
+ <span class="n">mini</span><span class="o">.</span><span class="na">waitForObservers</span><span class="o">();</span>
+
+ <span class="n">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"**** begin table dump ****"</span><span class="o">);</span>
+ <span class="k">try</span> <span class="o">(</span><span class="n">Snapshot</span> <span class="n">snap</span> <span class="o">=</span> <span class="n">client</span><span class="o">.</span><span class="na">newSnapshot</span><span class="o">())</span> <span class="o">{</span>
+ <span class="n">snap</span><span class="o">.</span><span class="na">scanner</span><span class="o">().</span><span class="na">build</span><span class="o">().</span><span class="na">forEach</span><span class="o">(</span><span class="n">rcv</span> <span class="o">-></span> <span class="n">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">" "</span> <span class="o">+</span> <span class="n">rcv</span><span class="o">));</span>
+ <span class="o">}</span>
+ <span class="n">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"**** end table dump ****\n"</span><span class="o">);</span>
+ <span class="o">}</span>
+
+ <span class="kd">private</span> <span class="kd">static</span> <span class="kt">void</span> <span class="nf">excercise</span><span class="o">(</span><span class="n">MiniFluo</span> <span class="n">mini</span><span class="o">,</span> <span class="n">FluoClient</span> <span class="n">client</span><span class="o">)</span> <span class="o">{</span>
+ <span class="n">loadAndPrint</span><span class="o">(</span><span class="n">mini</span><span class="o">,</span> <span class="n">client</span><span class="o">,</span> <span class="n">docs1</span><span class="o">);</span>
+ <span class="n">loadAndPrint</span><span class="o">(</span><span class="n">mini</span><span class="o">,</span> <span class="n">client</span><span class="o">,</span> <span class="n">docs2</span><span class="o">);</span>
+ <span class="n">loadAndPrint</span><span class="o">(</span><span class="n">mini</span><span class="o">,</span> <span class="n">client</span><span class="o">,</span> <span class="n">docs3</span><span class="o">);</span>
+ <span class="o">}</span>
+</code></pre>
+</div>
+
+<p>Once the TODOs in the DocLoader class are implemented, running Main should print out the following.</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>**** begin table dump ****
+ d:a6c4d1f doc content Jebediah orbits Mun for 35 days. No power, forgot solar panels.
+ d:a6c4d1f doc refc 1
+ d:a6c4d1f doc refs referenced
+ d:cf8ddc0 doc content Bill plans to rescue Jebediah after taking tourist to Minimus.
+ d:cf8ddc0 doc refc 1
+ d:cf8ddc0 doc refs referenced
+ u:http://news.com/a23 uri hash a6c4d1f
+ u:http://news.com/a24 uri hash cf8ddc0
+**** end table dump ****
+
+**** begin table dump ****
+ d:a6c4d1f doc content Jebediah orbits Mun for 35 days. No power, forgot solar panels.
+ d:a6c4d1f doc refc 2
+ d:a6c4d1f doc refs referenced
+ d:cf8ddc0 doc content Bill plans to rescue Jebediah after taking tourist to Minimus.
+ d:cf8ddc0 doc refc 1
+ d:cf8ddc0 doc refs referenced
+ u:http://news.com/a23 uri hash a6c4d1f
+ u:http://news.com/a24 uri hash cf8ddc0
+ u:http://oldnews.com/a23 uri hash a6c4d1f
+**** end table dump ****
+
+**** begin table dump ****
+ d:2732ebc doc content Crisis at KSC. Tourist stuck at Minimus. Bill forgot solar panels.
+ d:2732ebc doc refc 1
+ d:2732ebc doc refs referenced
+ d:6658252 doc content Jebediah orbits Mun for 38 days. No power, forgot solar panels.
+ d:6658252 doc refc 1
+ d:6658252 doc refs referenced
+ d:a6c4d1f doc content Jebediah orbits Mun for 35 days. No power, forgot solar panels.
+ d:a6c4d1f doc refc 1
+ d:a6c4d1f doc refs referenced
+ d:cf8ddc0 doc content Bill plans to rescue Jebediah after taking tourist to Minimus.
+ d:cf8ddc0 doc refc 0
+ d:cf8ddc0 doc refs unreferenced
+ u:http://news.com/a23 uri hash 6658252
+ u:http://news.com/a24 uri hash 2732ebc
+ u:http://oldnews.com/a23 uri hash a6c4d1f
+**** end table dump ****
+</code></pre>
+</div>
+
+<h2 id="part-2--computing-word-counts">Part 2 : Computing word counts.</h2>
+
+<p>Now that you have data loading, create an observer that watches the reference
+status column. This observer should increment word counts when new content is
+referenced and decrement word counts when content is dereferenced. The
+observer should also delete the content when its dereferenced.</p>
+
+<p>Make sure you handle the following scenario correctly.</p>
+
+<ul>
+ <li>content A becomes referenced</li>
+ <li>content A becomes unreferenced</li>
+ <li>an observer runs on content A</li>
+</ul>
+
+<p>In this situation, word counts were never incremented for content A so there is no need to decrement
+the word counts. One way to handle this is to have a column that tracks if word counts were
+incremented.</p>
+
+<p>Below is a skeleton for an observer to compute word counts.</p>
+
+<div class="language-java highlighter-rouge"><pre class="highlight"><code><span class="kn">package</span> <span class="n">ft</span><span class="o">;</span>
+
+<span class="kn">import</span> <span class="nn">java.util.Arrays</span><span class="o">;</span>
+<span class="kn">import</span> <span class="nn">java.util.Collection</span><span class="o">;</span>
+<span class="kn">import</span> <span class="nn">java.util.HashSet</span><span class="o">;</span>
+<span class="kn">import</span> <span class="nn">java.util.Map</span><span class="o">;</span>
+<span class="kn">import</span> <span class="nn">java.util.Set</span><span class="o">;</span>
+
+<span class="kn">import</span> <span class="nn">com.google.common.collect.ImmutableSet</span><span class="o">;</span>
+<span class="kn">import</span> <span class="nn">org.apache.fluo.api.client.TransactionBase</span><span class="o">;</span>
+<span class="kn">import</span> <span class="nn">org.apache.fluo.api.data.Bytes</span><span class="o">;</span>
+<span class="kn">import</span> <span class="nn">org.apache.fluo.api.data.Column</span><span class="o">;</span>
+<span class="kn">import</span> <span class="nn">org.apache.fluo.api.data.RowColumn</span><span class="o">;</span>
+<span class="kn">import</span> <span class="nn">org.apache.fluo.api.observer.AbstractObserver</span><span class="o">;</span>
+
+<span class="kd">public</span> <span class="kd">class</span> <span class="nc">ContentObserver</span> <span class="kd">extends</span> <span class="n">AbstractObserver</span> <span class="o">{</span>
+
+ <span class="kd">public</span> <span class="kd">static</span> <span class="kd">final</span> <span class="n">Column</span> <span class="n">PROCESSED_COL</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Column</span><span class="o">(</span><span class="s">"doc"</span><span class="o">,</span> <span class="s">"processed"</span><span class="o">);</span>
+ <span class="kd">public</span> <span class="kd">static</span> <span class="kd">final</span> <span class="n">Column</span> <span class="n">WORD_COUNT</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Column</span><span class="o">(</span><span class="s">"word"</span><span class="o">,</span><span class="s">"docCount"</span><span class="o">);</span>
+
+ <span class="cm">/**
+ * Utility method to tokenize the content of a document into unique words.
+ */</span>
+ <span class="kd">private</span> <span class="n">Set</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="nf">tokenize</span><span class="o">(</span><span class="n">String</span> <span class="n">content</span><span class="o">)</span> <span class="o">{</span>
+ <span class="k">return</span> <span class="k">new</span> <span class="n">HashSet</span><span class="o"><</span><span class="n">String</span><span class="o">>(</span><span class="n">Arrays</span><span class="o">.</span><span class="na">asList</span><span class="o">(</span><span class="n">content</span><span class="o">.</span><span class="na">split</span><span class="o">(</span><span class="s">"[ .!,]+"</span><span class="o">)));</span>
+ <span class="o">}</span>
+
+ <span class="cm">/**
+ * Adds the passed to delta to the values for each word.
+ */</span>
+ <span class="kd">private</span> <span class="kt">void</span> <span class="nf">adjustCounts</span><span class="o">(</span><span class="n">TransactionBase</span> <span class="n">tx</span><span class="o">,</span> <span class="kt">int</span> <span class="n">delta</span><span class="o">,</span> <span class="n">Set</span><span class="o"><</span><span class="n">String</span><span class="o">></span> <span class="n">words</span><span class="o">)</span> <span class="o">{</span>
+ <span class="c1">// TODO make a single call to get all of the current word counts. Could use</span>
+ <span class="c1">//tx.gets(Collection<RowColumn>)</span>
+
+ <span class="c1">// TODO for each word, add delta to the current value and set the new value</span>
+ <span class="o">}</span>
+
+
+ <span class="nd">@Override</span>
+ <span class="kd">public</span> <span class="kt">void</span> <span class="nf">process</span><span class="o">(</span><span class="n">TransactionBase</span> <span class="n">tx</span><span class="o">,</span> <span class="n">Bytes</span> <span class="n">brow</span><span class="o">,</span> <span class="n">Column</span> <span class="n">col</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
+
+ <span class="n">String</span> <span class="n">row</span> <span class="o">=</span> <span class="n">brow</span><span class="o">.</span><span class="na">toString</span><span class="o">();</span>
+
+ <span class="n">Map</span><span class="o"><</span><span class="n">Column</span><span class="o">,</span> <span class="n">String</span><span class="o">></span> <span class="n">colVals</span> <span class="o">=</span>
+ <span class="n">tx</span><span class="o">.</span><span class="na">gets</span><span class="o">(</span><span class="n">row</span><span class="o">,</span> <span class="n">DocLoader</span><span class="o">.</span><span class="na">CONTENT_COL</span><span class="o">,</span> <span class="n">DocLoader</span><span class="o">.</span><span class="na">REF_STATUS_COL</span><span class="o">,</span> <span class="n">PROCESSED_COL</span><span class="o">);</span>
+
+ <span class="n">String</span> <span class="n">content</span> <span class="o">=</span> <span class="n">colVals</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="n">DocLoader</span><span class="o">.</span><span class="na">CONTENT_COL</span><span class="o">);</span>
+ <span class="n">String</span> <span class="n">status</span> <span class="o">=</span> <span class="n">colVals</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="n">DocLoader</span><span class="o">.</span><span class="na">REF_STATUS_COL</span><span class="o">);</span>
+ <span class="n">String</span> <span class="n">processed</span> <span class="o">=</span> <span class="n">colVals</span><span class="o">.</span><span class="na">getOrDefault</span><span class="o">(</span><span class="n">PROCESSED_COL</span><span class="o">,</span> <span class="s">"false"</span><span class="o">);</span>
+
+ <span class="c1">// TODO if status is referenced and not already processed the adjustCounts by +1 and set</span>
+ <span class="c1">// PROCESSED_COL to true</span>
+
+ <span class="c1">// TODO is status is unreferenced then delete all columns for content</span>
+ <span class="c1">// TODO if status is unreferenced and document was processed, then adjust counts by -1</span>
+ <span class="o">}</span>
+
+
+ <span class="nd">@Override</span>
+ <span class="kd">public</span> <span class="n">ObservedColumn</span> <span class="nf">getObservedColumn</span><span class="o">()</span> <span class="o">{</span>
+ <span class="k">return</span> <span class="k">new</span> <span class="nf">ObservedColumn</span><span class="o">(</span><span class="n">DocLoader</span><span class="o">.</span><span class="na">REF_STATUS_COL</span><span class="o">,</span> <span class="n">NotificationType</span><span class="o">.</span><span class="na">STRONG</span><span class="o">);</span>
+ <span class="o">}</span>
+<span class="o">}</span>
+</code></pre>
+</div>
+
+<p>Something to think about: why observe the reference status column instead of the reference count
+column?</p>
+
+<p>When you are ready to run the observer, modify the <code class="highlighter-rouge">preInit()</code> method in <code class="highlighter-rouge">ft.Main</code> to configure the
+observer as follows.</p>
+
+<div class="language-java highlighter-rouge"><pre class="highlight"><code> <span class="kd">private</span> <span class="kd">static</span> <span class="kt">void</span> <span class="nf">preInit</span><span class="o">(</span><span class="n">FluoConfiguration</span> <span class="n">fluoConfig</span><span class="o">)</span> <span class="o">{</span>
+ <span class="n">fluoConfig</span><span class="o">.</span><span class="na">addObserver</span><span class="o">(</span><span class="k">new</span> <span class="n">ObserverSpecification</span><span class="o">(</span><span class="n">ContentObserver</span><span class="o">.</span><span class="na">class</span><span class="o">.</span><span class="na">getName</span><span class="o">()));</span>
+ <span class="o">}</span>
+</code></pre>
+</div>
+
+<p>After implementing the Observer, the output of the program should look like the following.</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>**** begin table dump ****
+ d:a6c4d1f doc content Jebediah orbits Mun for 35 days. No power, forgot solar panels.
+ d:a6c4d1f doc processed true
+ d:a6c4d1f doc refc 1
+ d:a6c4d1f doc refs referenced
+ d:cf8ddc0 doc content Bill plans to rescue Jebediah after taking tourist to Minimus.
+ d:cf8ddc0 doc processed true
+ d:cf8ddc0 doc refc 1
+ d:cf8ddc0 doc refs referenced
+ u:http://news.com/a23 uri hash a6c4d1f
+ u:http://news.com/a24 uri hash cf8ddc0
+ w:35 word docCount 1
+ w:Bill word docCount 1
+ w:Jebediah word docCount 2
+ w:Minimus word docCount 1
+ w:Mun word docCount 1
+ w:No word docCount 1
+ w:after word docCount 1
+ w:days word docCount 1
+ w:for word docCount 1
+ w:forgot word docCount 1
+ w:orbits word docCount 1
+ w:panels word docCount 1
+ w:plans word docCount 1
+ w:power word docCount 1
+ w:rescue word docCount 1
+ w:solar word docCount 1
+ w:taking word docCount 1
+ w:to word docCount 1
+ w:tourist word docCount 1
+**** end table dump ****
+
+**** begin table dump ****
+ d:a6c4d1f doc content Jebediah orbits Mun for 35 days. No power, forgot solar panels.
+ d:a6c4d1f doc processed true
+ d:a6c4d1f doc refc 2
+ d:a6c4d1f doc refs referenced
+ d:cf8ddc0 doc content Bill plans to rescue Jebediah after taking tourist to Minimus.
+ d:cf8ddc0 doc processed true
+ d:cf8ddc0 doc refc 1
+ d:cf8ddc0 doc refs referenced
+ u:http://news.com/a23 uri hash a6c4d1f
+ u:http://news.com/a24 uri hash cf8ddc0
+ u:http://oldnews.com/a23 uri hash a6c4d1f
+ w:35 word docCount 1
+ w:Bill word docCount 1
+ w:Jebediah word docCount 2
+ w:Minimus word docCount 1
+ w:Mun word docCount 1
+ w:No word docCount 1
+ w:after word docCount 1
+ w:days word docCount 1
+ w:for word docCount 1
+ w:forgot word docCount 1
+ w:orbits word docCount 1
+ w:panels word docCount 1
+ w:plans word docCount 1
+ w:power word docCount 1
+ w:rescue word docCount 1
+ w:solar word docCount 1
+ w:taking word docCount 1
+ w:to word docCount 1
+ w:tourist word docCount 1
+**** end table dump ****
+
+**** begin table dump ****
+ d:2732ebc doc content Crisis at KSC. Tourist stuck at Minimus. Bill forgot solar panels.
+ d:2732ebc doc processed true
+ d:2732ebc doc refc 1
+ d:2732ebc doc refs referenced
+ d:6658252 doc content Jebediah orbits Mun for 38 days. No power, forgot solar panels.
+ d:6658252 doc processed true
+ d:6658252 doc refc 1
+ d:6658252 doc refs referenced
+ d:a6c4d1f doc content Jebediah orbits Mun for 35 days. No power, forgot solar panels.
+ d:a6c4d1f doc processed true
+ d:a6c4d1f doc refc 1
+ d:a6c4d1f doc refs referenced
+ u:http://news.com/a23 uri hash 6658252
+ u:http://news.com/a24 uri hash 2732ebc
+ u:http://oldnews.com/a23 uri hash a6c4d1f
+ w:35 word docCount 1
+ w:38 word docCount 1
+ w:Bill word docCount 1
+ w:Crisis word docCount 1
+ w:Jebediah word docCount 2
+ w:KSC word docCount 1
+ w:Minimus word docCount 1
+ w:Mun word docCount 2
+ w:No word docCount 2
+ w:Tourist word docCount 1
+ w:at word docCount 1
+ w:days word docCount 2
+ w:for word docCount 2
+ w:forgot word docCount 3
+ w:orbits word docCount 2
+ w:panels word docCount 3
+ w:power word docCount 2
+ w:solar word docCount 3
+ w:stuck word docCount 1
+**** end table dump ****
+</code></pre>
+</div>
+
+<h2 id="part-3--using-fluo-recipes">Part 3 : Using Fluo Recipes</h2>
+
+<p>The way to compute word counts above is very prone to transactional collisions. One way to avoid
+these collisions is to use the CollisionFreeMap provided in Fluo Recipes. Currently Fluo Recipes is
+not released, this section will be updated with more information once it is.</p>
+
+</div>
+
+<script>
+document.body.onkeyup = function(e){
+
+if (e.keyCode == '37') { window.location = '/tour/observer_example/'; }
+
+
+
+if (e.keyCode == '39') { window.location = '/tour/row-locking/'; }
+
+};
+</script>
+
+<div class="text-center">
+
+ <h2>
+
+ <a href="/tour/observer_example/"><</a>
+
+
+ 19 / 26
+
+ <a href="/tour/row-locking/">></a>
+
+ </h2>
+</div>
+
+ </div>
+ </div>
+ <hr>
+ <div class="row footer">
+ <div class="col-sm-12 text-center">
+ <div class="center-block">
+ <a href="https://apache.org"><img src="/resources/feather.png" alt="Apache"></a>
+ Copyright © 2016 The Apache Software Foundation. Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>
+ </div>
+ </div>
+ </div>
+ </div>
+ <script src="/javascripts/jquery.min.js"></script>
+ <script src="/javascripts/bootstrap.min.js"></script>
+ <!-- Place your <script> tags here. -->
+
+<!-- Google Analytics -->
+<script>
+ (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+ m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+ })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
+
+ ga('create', 'UA-55360307-1', 'auto');
+ ga('send', 'pageview');
+
+</script>
+
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-fluo-website/blob/76022205/tour/index.html
----------------------------------------------------------------------
diff --git a/tour/index.html b/tour/index.html
index a3065b6..0ef1818 100644
--- a/tour/index.html
+++ b/tour/index.html
@@ -10,7 +10,7 @@
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css" >
<link rel="canonical" href="http://fluo.io/tour/">
<link rel="icon" type="image/png" href="/resources/favicon.png">
- <title>Apache Fluo Tour | Apache Fluo</title>
+ <title>Fluo Tour | Apache Fluo</title>
<!--[if lt IE 9]>
<script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
<script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
@@ -73,11 +73,100 @@
<div class="page">
<header class="post-header">
- <h2 class="post-title">Apache Fluo Tour</h2>
+ <h2 class="post-title">Fluo Tour</h2>
</header>
<article id="page-content">
- <p>Shortly after Fluo 1.0.0-incubating is released, a step by step introduction with hands on examples will be placed here. For now this page is a place holder so that the documentation that will ship with Fluo can be modified to point here before release.</p>
+
+<p>Welcome to the Fluo tour! The tour starts by introducing Fluo\u2019s <a href="/tour/data-model/">Data Model</a>.</p>
+
+<p>We recommend following the tour in order. However, all pages are listed below for review. When on a
+tour page, the left and right keys on the keyboard can be used to navigate. If you have any
+questions or suggestions while going through the tour, please let us know. There are multiple
+options for getting in touch : <a href="/getinvolved/">mailing list, IRC</a>, and <a href="https://github.com/apache/incubator-fluo-website/issues">Github Issues</a>. Any
+thoughts, solutions, etc related to this tour can also be tweeted using the hashtag
+<a href="https://twitter.com/hashtag/apachefluotour">#apachefluotour</a>.</p>
+
+<ol>
+ <li>
+ <p><a href="/tour/data-model/">Data Model</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/architecture/">Architecture</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/writing-code/">Writing and Running Fluo code</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/basic-read-write/">Read and Write Data</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/data-pojos/">Basic Data Types</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/snapshot-isolation/">Snapshot Isolation</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/snapshot-isolation-code/">Snapshot Isolation Code</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/collisions/">Collisions</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/collision-code/">Collision code</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/tx-logging/">Transaction Logging</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/write-skew/">Write Skew</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/write-skew-code/">Write Skew Code</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/scanning/">Scanning</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/scanning-code/">Scanning Code</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/multi-get/">Fetching multiple cells</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/loader-executer/">Loader Executor</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/observers/">Observer Concepts</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/observer_example/">Observer Example</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/exercise-1/">Word count Exercise</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/row-locking/">Row Locking</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/weak-notifications/">Weak Notification Exercise</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/weak-code/">Weak Notification Code</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/application-configuration/">Application Configuration</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/mem-self-ntfy/">Memory limits and self notify</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/mem-self-ntfy-code/">Memory limits and self notify code</a></p>
+ </li>
+ <li>
+ <p><a href="/tour/recipes/">Fluo Recipes</a></p>
+ </li>
+</ol>
</article>
http://git-wip-us.apache.org/repos/asf/incubator-fluo-website/blob/76022205/tour/loader-executer/index.html
----------------------------------------------------------------------
diff --git a/tour/loader-executer/index.html b/tour/loader-executer/index.html
new file mode 100644
index 0000000..fedb4ba
--- /dev/null
+++ b/tour/loader-executer/index.html
@@ -0,0 +1,154 @@
+<!DOCTYPE html>
+<html>
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link rel="stylesheet" href="/stylesheets/fluo.css">
+ <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Montserrat:700,400">
+ <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Merriweather">
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css" >
+ <link rel="canonical" href="http://fluo.io/tour/loader-executer/">
+ <link rel="icon" type="image/png" href="/resources/favicon.png">
+ <title>Loader Executor | Apache Fluo</title>
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ </head>
+ <body>
+ <div id="fluo-nav" class="navbar navbar-default navbar-static-top">
+ <div class="container">
+ <div class="navbar-header">
+ <div class="navbar-toggle-wrapper visible-xs">
+ <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".js-navbar-collapse">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ </div>
+ <a href="/" class="navbar-brand"><img src="/resources/fluo-logo.png" alt="Apache Fluo"></a>
+ </div>
+ <div class="collapse navbar-collapse js-navbar-collapse" style="margin-top: 20px">
+ <ul class="navbar-nav nav">
+ <li><a href="/docs/">Documentation</a></li>
+ <li><a href="/download/">Download</a></li>
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="/getinvolved/">Get Involved</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ <li><a href="/people/">People</a></li>
+ <li><a href="/related-projects/">Related Projects</a></li>
+ <li><a href="/poweredby/">Powered By</a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Contributing<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="/how-to-contribute/">How To Contribute</a></li>
+ <li><a href="/release-process/">Release Process</a></li>
+ </ul>
+ </li>
+ </ul>
+ <ul class="navbar-nav nav navbar-right">
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="https://www.apache.org">Apache Homepage</a></li>
+ <li><a href="https://www.apache.org/licenses/LICENSE-2.0">License</a></li>
+ <li><a href="https://www.apache.org/foundation/sponsorship">Sponsorship</i></a></li>
+ <li><a href="https://www.apache.org/security">Security</a></li>
+ <li><a href="https://www.apache.org/foundation/thanks">Thanks</a></li>
+ <li><a href="https://www.apache.org/foundation/policies/conduct">Code of Conduct</a></li>
+ </ul>
+ </li>
+ </ul>
+ </div>
+ </div>
+ </div>
+ <div class="container">
+ <div class="row">
+ <div class="col-sm-12">
+
+
+
+<div id="tour-header">
+ <h2><a href="/tour/">Fluo Tour</a>: Loader Executor</h2>
+ <p class="text-muted">Tour page 16 of 26</p>
+</div>
+<div id="tour-content">
+ <p>Fluo provides a simple mechanism to help load data called the <a href="/apidocs/fluo/1.0.0-beta-2/org/apache/fluo/api/client/LoaderExecutor.html">LoaderExecutor</a>. Loading data
+into Fluo requires a transaction. The LoaderExecutor manages creating, committing, and retrying
+transactions when collisions occur. It also runs transactions in multiple threads and batches
+commit processing of separate transactions for efficiency. FluoConfiguration provides two methods
+for configuring LoaderExecutors <a href="/apidocs/fluo/1.0.0-beta-2/org/apache/fluo/api/config/FluoConfiguration.html#setLoaderQueueSize-int-">setLoaderQueueSize()</a> and <a href="/apidocs/fluo/1.0.0-beta-2/org/apache/fluo/api/config/FluoConfiguration.html#setLoaderThreads-int-">setLoaderThreads()</a>.</p>
+
+<p>Objects that implement <a href="/apidocs/fluo/1.0.0-beta-2/org/apache/fluo/api/client/Loader.html">Loader</a> are given to a LoaderExecutor. The <a href="/apidocs/fluo/1.0.0-beta-2/org/apache/fluo/api/client/Loader.html#load-org.apache.fluo.api.client.TransactionBase-org.apache.fluo.api.client.Loader.Context-">load()</a> method will
+eventually be called on these objects at which point the passed in transactions can be used to load
+data. When <code class="highlighter-rouge">close()</code> is called on a LoaderExecutor, it waits for all running and queued work to
+finish.</p>
+
+<p>There is no stand alone exercise for the LoaderExecutor. Hands on experience with it can be
+obtained by completing the <a href="/tour/exercise-1/">word count exercise</a> that is a few pages later in
+the tour.</p>
+
+
+</div>
+
+<script>
+document.body.onkeyup = function(e){
+
+if (e.keyCode == '37') { window.location = '/tour/multi-get/'; }
+
+
+
+if (e.keyCode == '39') { window.location = '/tour/observers/'; }
+
+};
+</script>
+
+<div class="text-center">
+
+ <h2>
+
+ <a href="/tour/multi-get/"><</a>
+
+
+ 16 / 26
+
+ <a href="/tour/observers/">></a>
+
+ </h2>
+</div>
+
+ </div>
+ </div>
+ <hr>
+ <div class="row footer">
+ <div class="col-sm-12 text-center">
+ <div class="center-block">
+ <a href="https://apache.org"><img src="/resources/feather.png" alt="Apache"></a>
+ Copyright © 2016 The Apache Software Foundation. Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>
+ </div>
+ </div>
+ </div>
+ </div>
+ <script src="/javascripts/jquery.min.js"></script>
+ <script src="/javascripts/bootstrap.min.js"></script>
+ <!-- Place your <script> tags here. -->
+
+<!-- Google Analytics -->
+<script>
+ (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+ m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+ })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
+
+ ga('create', 'UA-55360307-1', 'auto');
+ ga('send', 'pageview');
+
+</script>
+
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-fluo-website/blob/76022205/tour/mem-self-ntfy-code/index.html
----------------------------------------------------------------------
diff --git a/tour/mem-self-ntfy-code/index.html b/tour/mem-self-ntfy-code/index.html
new file mode 100644
index 0000000..bb56968
--- /dev/null
+++ b/tour/mem-self-ntfy-code/index.html
@@ -0,0 +1,252 @@
+<!DOCTYPE html>
+<html>
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link rel="stylesheet" href="/stylesheets/fluo.css">
+ <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Montserrat:700,400">
+ <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Merriweather">
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css" >
+ <link rel="canonical" href="http://fluo.io/tour/mem-self-ntfy-code/">
+ <link rel="icon" type="image/png" href="/resources/favicon.png">
+ <title>Memory limits and self notify code | Apache Fluo</title>
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ </head>
+ <body>
+ <div id="fluo-nav" class="navbar navbar-default navbar-static-top">
+ <div class="container">
+ <div class="navbar-header">
+ <div class="navbar-toggle-wrapper visible-xs">
+ <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".js-navbar-collapse">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ </div>
+ <a href="/" class="navbar-brand"><img src="/resources/fluo-logo.png" alt="Apache Fluo"></a>
+ </div>
+ <div class="collapse navbar-collapse js-navbar-collapse" style="margin-top: 20px">
+ <ul class="navbar-nav nav">
+ <li><a href="/docs/">Documentation</a></li>
+ <li><a href="/download/">Download</a></li>
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="/getinvolved/">Get Involved</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ <li><a href="/people/">People</a></li>
+ <li><a href="/related-projects/">Related Projects</a></li>
+ <li><a href="/poweredby/">Powered By</a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Contributing<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="/how-to-contribute/">How To Contribute</a></li>
+ <li><a href="/release-process/">Release Process</a></li>
+ </ul>
+ </li>
+ </ul>
+ <ul class="navbar-nav nav navbar-right">
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="https://www.apache.org">Apache Homepage</a></li>
+ <li><a href="https://www.apache.org/licenses/LICENSE-2.0">License</a></li>
+ <li><a href="https://www.apache.org/foundation/sponsorship">Sponsorship</i></a></li>
+ <li><a href="https://www.apache.org/security">Security</a></li>
+ <li><a href="https://www.apache.org/foundation/thanks">Thanks</a></li>
+ <li><a href="https://www.apache.org/foundation/policies/conduct">Code of Conduct</a></li>
+ </ul>
+ </li>
+ </ul>
+ </div>
+ </div>
+ </div>
+ <div class="container">
+ <div class="row">
+ <div class="col-sm-12">
+
+
+
+<div id="tour-header">
+ <h2><a href="/tour/">Fluo Tour</a>: Memory limits and self notify code</h2>
+ <p class="text-muted">Tour page 25 of 26</p>
+</div>
+<div id="tour-content">
+ <div class="language-java highlighter-rouge"><pre class="highlight"><code> <span class="kd">static</span> <span class="n">Column</span> <span class="n">NC</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Column</span><span class="o">(</span><span class="s">"ntyf"</span><span class="o">,</span> <span class="s">"sum"</span><span class="o">);</span>
+ <span class="kd">static</span> <span class="n">Column</span> <span class="n">TOTAL_COL</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Column</span><span class="o">(</span><span class="s">"sum"</span><span class="o">,</span> <span class="s">"total"</span><span class="o">);</span>
+ <span class="kd">static</span> <span class="n">Column</span> <span class="n">UPDATE_COL</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Column</span><span class="o">(</span><span class="s">"sum"</span><span class="o">,</span> <span class="s">"update"</span><span class="o">);</span>
+ <span class="kd">static</span> <span class="n">Column</span> <span class="n">CONTINUE_COL</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Column</span><span class="o">(</span><span class="s">"sum"</span><span class="o">,</span> <span class="s">"continue"</span><span class="o">);</span>
+
+ <span class="kd">public</span> <span class="kd">static</span> <span class="kd">class</span> <span class="nc">SummingObserver</span> <span class="kd">extends</span> <span class="n">AbstractObserver</span> <span class="o">{</span>
+
+ <span class="kd">private</span> <span class="kt">int</span> <span class="n">maxToProcess</span><span class="o">;</span>
+
+ <span class="nd">@Override</span>
+ <span class="kd">public</span> <span class="kt">void</span> <span class="nf">init</span><span class="o">(</span><span class="n">Context</span> <span class="n">context</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
+ <span class="c1">//made the max amount to process in a single transaction configurable</span>
+ <span class="n">maxToProcess</span> <span class="o">=</span> <span class="n">context</span><span class="o">.</span><span class="na">getObserverConfiguration</span><span class="o">().</span><span class="na">getInt</span><span class="o">(</span><span class="s">"maxToProcess"</span><span class="o">,</span> <span class="mi">100</span><span class="o">);</span>
+ <span class="o">}</span>
+
+ <span class="nd">@Override</span>
+ <span class="kd">public</span> <span class="n">ObservedColumn</span> <span class="nf">getObservedColumn</span><span class="o">()</span> <span class="o">{</span>
+ <span class="k">return</span> <span class="k">new</span> <span class="nf">ObservedColumn</span><span class="o">(</span><span class="n">NC</span><span class="o">,</span> <span class="n">NotificationType</span><span class="o">.</span><span class="na">WEAK</span><span class="o">);</span>
+ <span class="o">}</span>
+
+ <span class="nd">@Override</span>
+ <span class="kd">public</span> <span class="kt">void</span> <span class="nf">process</span><span class="o">(</span><span class="n">TransactionBase</span> <span class="n">tx</span><span class="o">,</span> <span class="n">Bytes</span> <span class="n">brow</span><span class="o">,</span> <span class="n">Column</span> <span class="n">col</span><span class="o">)</span> <span class="kd">throws</span> <span class="n">Exception</span> <span class="o">{</span>
+
+ <span class="n">String</span> <span class="n">row</span> <span class="o">=</span> <span class="n">brow</span><span class="o">.</span><span class="na">toString</span><span class="o">();</span>
+
+ <span class="n">Map</span><span class="o"><</span><span class="n">Column</span><span class="o">,</span> <span class="n">String</span><span class="o">></span> <span class="n">colVals</span> <span class="o">=</span> <span class="n">tx</span><span class="o">.</span><span class="na">gets</span><span class="o">(</span><span class="n">row</span><span class="o">,</span> <span class="n">TOTAL_COL</span><span class="o">,</span> <span class="n">CONTINUE_COL</span><span class="o">);</span>
+
+ <span class="kt">int</span> <span class="n">sum</span> <span class="o">=</span> <span class="n">Integer</span><span class="o">.</span><span class="na">parseInt</span><span class="o">(</span><span class="n">colVals</span><span class="o">.</span><span class="na">getOrDefault</span><span class="o">(</span><span class="n">TOTAL_COL</span><span class="o">,</span> <span class="s">"0"</span><span class="o">));</span>
+
+ <span class="c1">// construct a scan range that uses the continue row</span>
+ <span class="n">String</span> <span class="n">startRow</span> <span class="o">=</span> <span class="n">colVals</span><span class="o">.</span><span class="na">getOrDefault</span><span class="o">(</span><span class="n">CONTINUE_COL</span><span class="o">,</span> <span class="n">row</span> <span class="o">+</span> <span class="s">"/"</span><span class="o">);</span>
+ <span class="n">String</span> <span class="n">endRow</span> <span class="o">=</span> <span class="n">row</span> <span class="o">+</span> <span class="s">"/:"</span><span class="o">;</span> <span class="c1">// after the character '9' comes ':'</span>
+ <span class="n">CellScanner</span> <span class="n">scanner</span> <span class="o">=</span> <span class="n">tx</span><span class="o">.</span><span class="na">scanner</span><span class="o">().</span><span class="na">over</span><span class="o">(</span><span class="k">new</span> <span class="n">Span</span><span class="o">(</span><span class="n">startRow</span><span class="o">,</span> <span class="kc">true</span><span class="o">,</span> <span class="n">endRow</span><span class="o">,</span> <span class="kc">false</span><span class="o">)).</span><span class="na">build</span><span class="o">();</span>
+
+ <span class="kt">int</span> <span class="n">processed</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span>
+
+ <span class="k">for</span> <span class="o">(</span><span class="n">RowColumnValue</span> <span class="n">rcv</span> <span class="o">:</span> <span class="n">scanner</span><span class="o">)</span> <span class="o">{</span>
+ <span class="k">if</span> <span class="o">(</span><span class="n">processed</span> <span class="o">>=</span> <span class="n">maxToProcess</span><span class="o">)</span> <span class="o">{</span>
+ <span class="c1">// stop processing and set the continue row</span>
+ <span class="n">tx</span><span class="o">.</span><span class="na">set</span><span class="o">(</span><span class="n">row</span><span class="o">,</span> <span class="n">CONTINUE_COL</span><span class="o">,</span> <span class="n">rcv</span><span class="o">.</span><span class="na">getsRow</span><span class="o">());</span>
+ <span class="n">tx</span><span class="o">.</span><span class="na">setWeakNotification</span><span class="o">(</span><span class="n">brow</span><span class="o">,</span> <span class="n">col</span><span class="o">);</span>
+ <span class="k">break</span><span class="o">;</span>
+ <span class="o">}</span>
+ <span class="n">sum</span> <span class="o">+=</span> <span class="n">Integer</span><span class="o">.</span><span class="na">parseInt</span><span class="o">(</span><span class="n">rcv</span><span class="o">.</span><span class="na">getsValue</span><span class="o">());</span>
+ <span class="n">tx</span><span class="o">.</span><span class="na">delete</span><span class="o">(</span><span class="n">rcv</span><span class="o">.</span><span class="na">getRow</span><span class="o">(),</span> <span class="n">rcv</span><span class="o">.</span><span class="na">getColumn</span><span class="o">());</span>
+ <span class="n">processed</span><span class="o">++;</span>
+ <span class="o">}</span>
+
+ <span class="n">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"sum : "</span> <span class="o">+</span> <span class="n">sum</span> <span class="o">+</span> <span class="s">" start: "</span> <span class="o">+</span> <span class="n">startRow</span> <span class="o">+</span> <span class="s">" processed: "</span> <span class="o">+</span> <span class="n">processed</span><span class="o">);</span>
+
+ <span class="n">tx</span><span class="o">.</span><span class="na">set</span><span class="o">(</span><span class="n">row</span><span class="o">,</span> <span class="n">TOTAL_COL</span><span class="o">,</span> <span class="s">""</span> <span class="o">+</span> <span class="n">sum</span><span class="o">);</span>
+
+ <span class="c1">// if did not set the continue column and it exists, then delete it</span>
+ <span class="k">if</span> <span class="o">(</span><span class="n">processed</span> <span class="o"><</span> <span class="n">maxToProcess</span> <span class="o">&&</span> <span class="n">colVals</span><span class="o">.</span><span class="na">containsKey</span><span class="o">(</span><span class="n">CONTINUE_COL</span><span class="o">))</span> <span class="o">{</span>
+ <span class="n">tx</span><span class="o">.</span><span class="na">delete</span><span class="o">(</span><span class="n">row</span><span class="o">,</span> <span class="n">CONTINUE_COL</span><span class="o">);</span>
+ <span class="c1">// need to start over at the beginning and see if there is new data before the continue</span>
+ <span class="c1">// column</span>
+ <span class="n">tx</span><span class="o">.</span><span class="na">setWeakNotification</span><span class="o">(</span><span class="n">brow</span><span class="o">,</span> <span class="n">col</span><span class="o">);</span>
+ <span class="o">}</span>
+ <span class="o">}</span>
+ <span class="o">}</span>
+
+ <span class="kd">private</span> <span class="kd">static</span> <span class="kt">void</span> <span class="nf">preInit</span><span class="o">(</span><span class="n">FluoConfiguration</span> <span class="n">fluoConfig</span><span class="o">)</span> <span class="o">{</span>
+ <span class="n">ObserverSpecification</span> <span class="n">ospec</span> <span class="o">=</span> <span class="k">new</span> <span class="n">ObserverSpecification</span><span class="o">(</span><span class="n">SummingObserver</span><span class="o">.</span><span class="na">class</span><span class="o">.</span><span class="na">getName</span><span class="o">());</span>
+ <span class="n">ospec</span><span class="o">.</span><span class="na">getConfiguration</span><span class="o">().</span><span class="na">setProperty</span><span class="o">(</span><span class="s">"maxToProcess"</span><span class="o">,</span> <span class="mi">500</span><span class="o">);</span>
+ <span class="n">fluoConfig</span><span class="o">.</span><span class="na">addObserver</span><span class="o">(</span><span class="n">ospec</span><span class="o">);</span>
+ <span class="o">}</span>
+
+ <span class="kd">private</span> <span class="kd">static</span> <span class="kt">void</span> <span class="nf">excercise</span><span class="o">(</span><span class="n">MiniFluo</span> <span class="n">mini</span><span class="o">,</span> <span class="n">FluoClient</span> <span class="n">client</span><span class="o">)</span> <span class="o">{</span>
+ <span class="k">try</span> <span class="o">(</span><span class="n">LoaderExecutor</span> <span class="n">le</span> <span class="o">=</span> <span class="n">client</span><span class="o">.</span><span class="na">newLoaderExecutor</span><span class="o">())</span> <span class="o">{</span>
+ <span class="n">Random</span> <span class="n">r</span> <span class="o">=</span> <span class="k">new</span> <span class="n">Random</span><span class="o">(</span><span class="mi">42</span><span class="o">);</span>
+ <span class="k">for</span> <span class="o">(</span><span class="kt">int</span> <span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="o">;</span> <span class="n">i</span> <span class="o"><</span> <span class="mi">5000</span><span class="o">;</span> <span class="n">i</span><span class="o">++)</span> <span class="o">{</span>
+ <span class="c1">// The Loader interface only has one function and can therefore be written as a lambda</span>
+ <span class="c1">// below.</span>
+ <span class="n">le</span><span class="o">.</span><span class="na">execute</span><span class="o">((</span><span class="n">tx</span><span class="o">,</span> <span class="n">ctx</span><span class="o">)</span> <span class="o">-></span> <span class="o">{</span>
+ <span class="n">String</span> <span class="n">row</span> <span class="o">=</span> <span class="s">"counter001/"</span> <span class="o">+</span> <span class="n">String</span><span class="o">.</span><span class="na">format</span><span class="o">(</span><span class="s">"%07d"</span><span class="o">,</span> <span class="n">r</span><span class="o">.</span><span class="na">nextInt</span><span class="o">(</span><span class="mi">10000000</span><span class="o">));</span>
+ <span class="kt">int</span> <span class="n">curVal</span> <span class="o">=</span> <span class="n">Integer</span><span class="o">.</span><span class="na">parseInt</span><span class="o">(</span><span class="n">tx</span><span class="o">.</span><span class="na">gets</span><span class="o">(</span><span class="n">row</span><span class="o">,</span> <span class="n">UPDATE_COL</span><span class="o">,</span> <span class="s">"0"</span><span class="o">));</span>
+ <span class="n">tx</span><span class="o">.</span><span class="na">set</span><span class="o">(</span><span class="n">row</span><span class="o">,</span> <span class="n">UPDATE_COL</span><span class="o">,</span> <span class="n">curVal</span> <span class="o">+</span> <span class="mi">1</span> <span class="o">+</span> <span class="s">""</span><span class="o">);</span>
+ <span class="n">tx</span><span class="o">.</span><span class="na">setWeakNotification</span><span class="o">(</span><span class="s">"counter001"</span><span class="o">,</span> <span class="n">NC</span><span class="o">);</span>
+ <span class="o">});</span>
+ <span class="o">}</span>
+ <span class="o">}</span>
+
+ <span class="n">mini</span><span class="o">.</span><span class="na">waitForObservers</span><span class="o">();</span>
+
+ <span class="k">try</span> <span class="o">(</span><span class="n">Snapshot</span> <span class="n">snap</span> <span class="o">=</span> <span class="n">client</span><span class="o">.</span><span class="na">newSnapshot</span><span class="o">())</span> <span class="o">{</span>
+ <span class="n">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="s">"final sum : "</span> <span class="o">+</span> <span class="n">snap</span><span class="o">.</span><span class="na">gets</span><span class="o">(</span><span class="s">"counter001"</span><span class="o">,</span> <span class="n">TOTAL_COL</span><span class="o">));</span>
+ <span class="o">}</span>
+ <span class="o">}</span>
+</code></pre>
+</div>
+
+<p>The code above will print something like the following.</p>
+
+<div class="highlighter-rouge"><pre class="highlight"><code>$ mvn -q clean compile exec:java
+Starting MiniFluo ... started.
+sum : 500 start: counter001/ processed: 500
+sum : 891 start: counter001/7945963 processed: 390
+sum : 1391 start: counter001/ processed: 500
+sum : 1891 start: counter001/2938489 processed: 500
+sum : 2391 start: counter001/5210523 processed: 500
+sum : 2892 start: counter001/6912090 processed: 500
+sum : 3392 start: counter001/8410312 processed: 500
+sum : 3398 start: counter001/9991522 processed: 6
+sum : 3898 start: counter001/ processed: 500
+sum : 4398 start: counter001/1824962 processed: 500
+sum : 4899 start: counter001/4076664 processed: 500
+sum : 5000 start: counter001/6993690 processed: 101
+sum : 5000 start: counter001/ processed: 0
+final sum : 5000
+</code></pre>
+</div>
+
+
+</div>
+
+<script>
+document.body.onkeyup = function(e){
+
+if (e.keyCode == '37') { window.location = '/tour/mem-self-ntfy/'; }
+
+
+
+if (e.keyCode == '39') { window.location = '/tour/recipes/'; }
+
+};
+</script>
+
+<div class="text-center">
+
+ <h2>
+
+ <a href="/tour/mem-self-ntfy/"><</a>
+
+
+ 25 / 26
+
+ <a href="/tour/recipes/">></a>
+
+ </h2>
+</div>
+
+ </div>
+ </div>
+ <hr>
+ <div class="row footer">
+ <div class="col-sm-12 text-center">
+ <div class="center-block">
+ <a href="https://apache.org"><img src="/resources/feather.png" alt="Apache"></a>
+ Copyright © 2016 The Apache Software Foundation. Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>
+ </div>
+ </div>
+ </div>
+ </div>
+ <script src="/javascripts/jquery.min.js"></script>
+ <script src="/javascripts/bootstrap.min.js"></script>
+ <!-- Place your <script> tags here. -->
+
+<!-- Google Analytics -->
+<script>
+ (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+ m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+ })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
+
+ ga('create', 'UA-55360307-1', 'auto');
+ ga('send', 'pageview');
+
+</script>
+
+ </body>
+</html>
http://git-wip-us.apache.org/repos/asf/incubator-fluo-website/blob/76022205/tour/mem-self-ntfy/index.html
----------------------------------------------------------------------
diff --git a/tour/mem-self-ntfy/index.html b/tour/mem-self-ntfy/index.html
new file mode 100644
index 0000000..b19103a
--- /dev/null
+++ b/tour/mem-self-ntfy/index.html
@@ -0,0 +1,159 @@
+<!DOCTYPE html>
+<html>
+ <head>
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link rel="stylesheet" href="/stylesheets/fluo.css">
+ <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Montserrat:700,400">
+ <link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Merriweather">
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css" >
+ <link rel="canonical" href="http://fluo.io/tour/mem-self-ntfy/">
+ <link rel="icon" type="image/png" href="/resources/favicon.png">
+ <title>Memory limits and self notify | Apache Fluo</title>
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script>
+ <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ </head>
+ <body>
+ <div id="fluo-nav" class="navbar navbar-default navbar-static-top">
+ <div class="container">
+ <div class="navbar-header">
+ <div class="navbar-toggle-wrapper visible-xs">
+ <button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".js-navbar-collapse">
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ </div>
+ <a href="/" class="navbar-brand"><img src="/resources/fluo-logo.png" alt="Apache Fluo"></a>
+ </div>
+ <div class="collapse navbar-collapse js-navbar-collapse" style="margin-top: 20px">
+ <ul class="navbar-nav nav">
+ <li><a href="/docs/">Documentation</a></li>
+ <li><a href="/download/">Download</a></li>
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Community<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="/getinvolved/">Get Involved</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ <li><a href="/people/">People</a></li>
+ <li><a href="/related-projects/">Related Projects</a></li>
+ <li><a href="/poweredby/">Powered By</a></li>
+ </ul>
+ </li>
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Contributing<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="/how-to-contribute/">How To Contribute</a></li>
+ <li><a href="/release-process/">Release Process</a></li>
+ </ul>
+ </li>
+ </ul>
+ <ul class="navbar-nav nav navbar-right">
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation<span class="caret"></span></a>
+ <ul class="dropdown-menu">
+ <li><a href="https://www.apache.org">Apache Homepage</a></li>
+ <li><a href="https://www.apache.org/licenses/LICENSE-2.0">License</a></li>
+ <li><a href="https://www.apache.org/foundation/sponsorship">Sponsorship</i></a></li>
+ <li><a href="https://www.apache.org/security">Security</a></li>
+ <li><a href="https://www.apache.org/foundation/thanks">Thanks</a></li>
+ <li><a href="https://www.apache.org/foundation/policies/conduct">Code of Conduct</a></li>
+ </ul>
+ </li>
+ </ul>
+ </div>
+ </div>
+ </div>
+ <div class="container">
+ <div class="row">
+ <div class="col-sm-12">
+
+
+
+<div id="tour-header">
+ <h2><a href="/tour/">Fluo Tour</a>: Memory limits and self notify</h2>
+ <p class="text-muted">Tour page 24 of 26</p>
+</div>
+<div id="tour-content">
+ <p>All modifications made as part of a transaction must fit into memory because the sets and deletes
+are buffered in memory until commit. If there is more data to process that will fit in memory, one
+way to handle this is to process some data and self notify.</p>
+
+<p>As an exercise try modifying the <a href="/tour/weak-notifications/">weak notification exercise</a> and
+making it self notify. Modify the observer such that it does the following :</p>
+
+<ul>
+ <li>Processes a maximum number of updates. Could make this configurable using per observer
+configuration.</li>
+ <li>When the max is reached :
+ <ul>
+ <li>Stop processing.</li>
+ <li>Records the stop row.</li>
+ <li>Notify self.</li>
+ </ul>
+ </li>
+ <li>Use the row where it previously stopped when creating scan range.</li>
+ <li>Delete the stop row, if it exists, after scanning to the end of the range.</li>
+</ul>
+
+</div>
+
+<script>
+document.body.onkeyup = function(e){
+
+if (e.keyCode == '37') { window.location = '/tour/application-configuration/'; }
+
+
+
+if (e.keyCode == '39') { window.location = '/tour/mem-self-ntfy-code/'; }
+
+};
+</script>
+
+<div class="text-center">
+
+ <h2>
+
+ <a href="/tour/application-configuration/"><</a>
+
+
+ 24 / 26
+
+ <a href="/tour/mem-self-ntfy-code/">></a>
+
+ </h2>
+</div>
+
+ </div>
+ </div>
+ <hr>
+ <div class="row footer">
+ <div class="col-sm-12 text-center">
+ <div class="center-block">
+ <a href="https://apache.org"><img src="/resources/feather.png" alt="Apache"></a>
+ Copyright © 2016 The Apache Software Foundation. Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>
+ </div>
+ </div>
+ </div>
+ </div>
+ <script src="/javascripts/jquery.min.js"></script>
+ <script src="/javascripts/bootstrap.min.js"></script>
+ <!-- Place your <script> tags here. -->
+
+<!-- Google Analytics -->
+<script>
+ (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
+ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
+ m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
+ })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
+
+ ga('create', 'UA-55360307-1', 'auto');
+ ga('send', 'pageview');
+
+</script>
+
+ </body>
+</html>