You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@accumulo.apache.org by vi...@apache.org on 2011/12/19 17:21:40 UTC
svn commit: r800522 [2/2] - in /websites/production/accumulo: ./
content/accumulo/ content/accumulo/css/ content/accumulo/downloads/
content/accumulo/downloads/1.3.5/ content/accumulo/governance/
content/accumulo/images/ content/accumulo/user_manual_1....
Modified: websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Analytics.html
==============================================================================
--- websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Analytics.html (original)
+++ websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Analytics.html Mon Dec 19 16:21:39 2011
@@ -37,19 +37,15 @@
</head>
<body>
- <div id="banner">
- <img id="logo" alt="Apache accumulo (Incubating)" src="/accumulo/images/accumulo-logo.png"/>
- <div id="bannertext">
-
- </div><br />
+ <div id="banner">
</div>
-
+
<div id="navigation">
<h1 id="project">Project</h1>
<ul>
<li><a href="/accumulo">Home</a></li>
-<li><a href="http://incubator.apache.org/projects/accumulo.html">Incubator page</a>
-<!-- - Download --></li>
+<li><a href="http://incubator.apache.org/projects/accumulo.html">Incubator page</a></li>
+<li><a href="/accumulo/downloads/downloads.html">Downloads</a></li>
<li><a href="/accumulo/notable_features.html">Features</a></li>
<li><a href="http://www.apache.org/licenses/LICENSE-2.0">License</a></li>
</ul>
@@ -90,6 +86,10 @@
</ul>
</div>
+ <div id="bannertext">
+ <img id="logo" alt="Apache Accumulo (Incubating)" src="/accumulo/images/accumulo-logo.png"/>
+ </div>
+
<div id="content">
<h1 class="title">Accumulo User Manual: Analytics</h1>
<p><strong> Next:</strong> <a href="Security.html">Security</a> <strong> Up:</strong> <a href="accumulo_user_manual.html">Accumulo User Manual Version 1.4</a> <strong> Previous:</strong> <a href="High_Speed_Ingest.html">High-Speed Ingest</a> <strong> <a href="Contents.html">Contents</a></strong> <br />
@@ -97,21 +97,21 @@
<p><a id=CHILD_LINKS></a><strong>Subsections</strong></p>
<ul>
<li><a href="Analytics.html#MapReduce">MapReduce</a></li>
-<li><a href="Analytics.html#Aggregating_Iterators">Aggregating Iterators</a></li>
+<li><a href="Analytics.html#Combiners">Combiners</a></li>
<li><a href="Analytics.html#Statistical_Modeling">Statistical Modeling</a></li>
</ul>
<hr />
<h2 id="a_idanalyticsa_analytics"><a id=Analytics></a> Analytics</h2>
<p>Accumulo supports more advanced data processing than simply keeping keys sorted and performing efficient lookups. Analytics can be developed by using MapReduce and Iterators in conjunction with Accumulo tables. </p>
<h2 id="a_idmapreducea_mapreduce"><a id=MapReduce></a> MapReduce</h2>
-<p>Accumulo tables can be used as the source and destination of MapReduce jobs. To use a Accumulo table with a MapReduce job (specifically with the new Hadoop API as of version 0.20), configure the job parameters to use the AccumuloInputFormat and AccumuloOutputFormat. Accumulo specific parameters can be set via these two format classes to do the following: </p>
+<p>Accumulo tables can be used as the source and destination of MapReduce jobs. To use an Accumulo table with a MapReduce job (specifically with the new Hadoop API as of version 0.20), configure the job parameters to use the AccumuloInputFormat and AccumuloOutputFormat. Accumulo specific parameters can be set via these two format classes to do the following: </p>
<ul>
<li>Authenticate and provide user credentials for the input </li>
<li>Restrict the scan to a range of rows </li>
<li>Restrict the input to a subset of available columns </li>
</ul>
<h3 id="a_idmapper_and_reducer_classesa_mapper_and_reducer_classes"><a id=Mapper_and_Reducer_classes></a> Mapper and Reducer classes</h3>
-<p>To read from a Accumulo table create a Mapper with the following class parameterization and be sure to configure the AccumuloInputFormat. </p>
+<p>To read from an Accumulo table create a Mapper with the following class parameterization and be sure to configure the AccumuloInputFormat. </p>
<div class="codehilite"><pre><span class="n">class</span> <span class="n">MyMapper</span> <span class="n">extends</span> <span class="n">Mapper</span><span class="sr"><Key,Value,WritableComparable,Writable></span> <span class="p">{</span>
<span class="n">public</span> <span class="n">void</span> <span class="nb">map</span><span class="p">(</span><span class="n">Key</span> <span class="n">k</span><span class="p">,</span> <span class="n">Value</span> <span class="n">v</span><span class="p">,</span> <span class="n">Context</span> <span class="n">c</span><span class="p">)</span> <span class="p">{</span>
<span class="sr">//</span> <span class="n">transform</span> <span class="n">key</span> <span class="ow">and</span> <span class="n">value</span> <span class="n">data</span> <span class="n">here</span>
@@ -120,7 +120,7 @@
</pre></div>
-<p>To write to a Accumulo table, create a Reducer with the following class parameterization and be sure to configure the AccumuloOutputFormat. The key emitted from the Reducer identifies the table to which the mutation is sent. This allows a single Reducer to write to more than one table if desired. A default table can be configured using the AccumuloOutputFormat, in which case the output table name does not have to be passed to the Context object within the Reducer. </p>
+<p>To write to an Accumulo table, create a Reducer with the following class parameterization and be sure to configure the AccumuloOutputFormat. The key emitted from the Reducer identifies the table to which the mutation is sent. This allows a single Reducer to write to more than one table if desired. A default table can be configured using the AccumuloOutputFormat, in which case the output table name does not have to be passed to the Context object within the Reducer. </p>
<div class="codehilite"><pre><span class="n">class</span> <span class="n">MyReducer</span> <span class="n">extends</span> <span class="n">Reducer</span><span class="o"><</span><span class="n">WritableComparable</span><span class="p">,</span> <span class="n">Writable</span><span class="p">,</span> <span class="n">Text</span><span class="p">,</span> <span class="n">Mutation</span><span class="o">></span> <span class="p">{</span>
<span class="n">public</span> <span class="n">void</span> <span class="n">reduce</span><span class="p">(</span><span class="n">WritableComparable</span> <span class="n">key</span><span class="p">,</span> <span class="n">Iterator</span><span class="sr"><Text></span> <span class="nb">values</span><span class="p">,</span> <span class="n">Context</span> <span class="n">c</span><span class="p">)</span> <span class="p">{</span>
@@ -192,14 +192,14 @@
<p>An example of using MapReduce with Accumulo can be found at <br />
accumulo/docs/examples/README.mapred </p>
-<h2 id="a_idaggregating_iteratorsa_aggregating_iterators"><a id=Aggregating_Iterators></a> Aggregating Iterators</h2>
-<p>Many applications can benefit from the ability to aggregate values across common keys. This can be done via aggregating iterators and is similar to the Reduce step in MapReduce. This provides the ability to define online, incrementally updated analytics without the overhead or latency associated with batch-oriented MapReduce jobs. </p>
-<p>All that is needed to aggregate values of a table is to identify the fields over which values will be grouped, insert mutations with those fields as the key, and configure the table with an aggregating iterator that supports the summarization operation desired. </p>
-<p>The only restriction on an aggregating iterator is that the aggregator developer should not assume that all values for a given key have been seen, since new mutations can be inserted at anytime. This precludes using the total number of values in the aggregation such as when calculating an average, for example. </p>
+<h2 id="a_idcombinersa_combiners"><a id=Combiners></a> Combiners</h2>
+<p>Many applications can benefit from the ability to aggregate values across common keys. This can be done via Combiner iterators and is similar to the Reduce step in MapReduce. This provides the ability to define online, incrementally updated analytics without the overhead or latency associated with batch-oriented MapReduce jobs. </p>
+<p>All that is needed to aggregate values of a table is to identify the fields over which values will be grouped, insert mutations with those fields as the key, and configure the table with a combining iterator that supports the summarizing operation desired. </p>
+<p>The only restriction on an combining iterator is that the combiner developer should not assume that all values for a given key have been seen, since new mutations can be inserted at anytime. This precludes using the total number of values in the aggregation such as when calculating an average, for example. </p>
<h3 id="a_idfeature_vectorsa_feature_vectors"><a id=Feature_Vectors></a> Feature Vectors</h3>
-<p>An interesting use of aggregating iterators within a Accumulo table is to store feature vectors for use in machine learning algorithms. For example, many algorithms such as k-means clustering, support vector machines, anomaly detection, etc. use the concept of a feature vector and the calculation of distance metrics to learn a particular model. The columns in a Accumulo table can be used to efficiently store sparse features and their weights to be incrementally updated via the use of an aggregating iterator. </p>
+<p>An interesting use of combining iterators within an Accumulo table is to store feature vectors for use in machine learning algorithms. For example, many algorithms such as k-means clustering, support vector machines, anomaly detection, etc. use the concept of a feature vector and the calculation of distance metrics to learn a particular model. The columns in an Accumulo table can be used to efficiently store sparse features and their weights to be incrementally updated via the use of an combining iterator. </p>
<h2 id="a_idstatistical_modelinga_statistical_modeling"><a id=Statistical_Modeling></a> Statistical Modeling</h2>
-<p>Statistical models that need to be updated by many machines in parallel could be similarly stored within a Accumulo table. For example, a MapReduce job that is iteratively updating a global statistical model could have each map or reduce worker reference the parts of the model to be read and updated through an embedded Accumulo client. </p>
+<p>Statistical models that need to be updated by many machines in parallel could be similarly stored within an Accumulo table. For example, a MapReduce job that is iteratively updating a global statistical model could have each map or reduce worker reference the parts of the model to be read and updated through an embedded Accumulo client. </p>
<p>Using Accumulo this way enables efficient and fast lookups and updates of small pieces of information in a random access pattern, which is complementary to MapReduce's sequential access model. </p>
<hr />
<p><strong> Next:</strong> <a href="Security.html">Security</a> <strong> Up:</strong> <a href="accumulo_user_manual.html">Accumulo User Manual Version 1.4</a> <strong> Previous:</strong> <a href="High_Speed_Ingest.html">High-Speed Ingest</a> <strong> <a href="Contents.html">Contents</a></strong></p>
Modified: websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Contents.html
==============================================================================
--- websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Contents.html (original)
+++ websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Contents.html Mon Dec 19 16:21:39 2011
@@ -37,19 +37,15 @@
</head>
<body>
- <div id="banner">
- <img id="logo" alt="Apache accumulo (Incubating)" src="/accumulo/images/accumulo-logo.png"/>
- <div id="bannertext">
-
- </div><br />
+ <div id="banner">
</div>
-
+
<div id="navigation">
<h1 id="project">Project</h1>
<ul>
<li><a href="/accumulo">Home</a></li>
-<li><a href="http://incubator.apache.org/projects/accumulo.html">Incubator page</a>
-<!-- - Download --></li>
+<li><a href="http://incubator.apache.org/projects/accumulo.html">Incubator page</a></li>
+<li><a href="/accumulo/downloads/downloads.html">Downloads</a></li>
<li><a href="/accumulo/notable_features.html">Features</a></li>
<li><a href="http://www.apache.org/licenses/LICENSE-2.0">License</a></li>
</ul>
@@ -90,6 +86,10 @@
</ul>
</div>
+ <div id="bannertext">
+ <img id="logo" alt="Apache Accumulo (Incubating)" src="/accumulo/images/accumulo-logo.png"/>
+ </div>
+
<div id="content">
<h1 class="title">Accumulo User Manual: Contents</h1>
<p><strong> Next:</strong> <a href="Introduction.html">Introduction</a> <strong> Up:</strong> <a href="accumulo_user_manual.html">Accumulo User Manual Version 1.4</a> <strong> Previous:</strong> <a href="accumulo_user_manual.html">Accumulo User Manual Version 1.4</a> <br />
@@ -168,7 +168,7 @@
<li><a href="Table_Configuration.html#Setting_Iterators_Programmatically">Setting Iterators Programmatically</a></li>
<li><a href="Table_Configuration.html#Versioning_Iterators_and_Timestamps">Versioning Iterators and Timestamps</a></li>
<li><a href="Table_Configuration.html#Filters">Filters</a></li>
-<li><a href="Table_Configuration.html#Aggregating_Iterators">Aggregating Iterators</a></li>
+<li><a href="Table_Configuration.html#Combiners">Combiners</a></li>
</ul>
</li>
<li>
@@ -213,7 +213,7 @@
</ul>
</li>
<li>
-<p><a href="Analytics.html#Aggregating_Iterators">Aggregating Iterators</a></p>
+<p><a href="Analytics.html#Combiners">Combiners</a></p>
<ul>
<li><a href="Analytics.html#Feature_Vectors">Feature Vectors</a></li>
</ul>
Modified: websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/High_Speed_Ingest.html
==============================================================================
--- websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/High_Speed_Ingest.html (original)
+++ websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/High_Speed_Ingest.html Mon Dec 19 16:21:39 2011
@@ -37,19 +37,15 @@
</head>
<body>
- <div id="banner">
- <img id="logo" alt="Apache accumulo (Incubating)" src="/accumulo/images/accumulo-logo.png"/>
- <div id="bannertext">
-
- </div><br />
+ <div id="banner">
</div>
-
+
<div id="navigation">
<h1 id="project">Project</h1>
<ul>
<li><a href="/accumulo">Home</a></li>
-<li><a href="http://incubator.apache.org/projects/accumulo.html">Incubator page</a>
-<!-- - Download --></li>
+<li><a href="http://incubator.apache.org/projects/accumulo.html">Incubator page</a></li>
+<li><a href="/accumulo/downloads/downloads.html">Downloads</a></li>
<li><a href="/accumulo/notable_features.html">Features</a></li>
<li><a href="http://www.apache.org/licenses/LICENSE-2.0">License</a></li>
</ul>
@@ -90,6 +86,10 @@
</ul>
</div>
+ <div id="bannertext">
+ <img id="logo" alt="Apache Accumulo (Incubating)" src="/accumulo/images/accumulo-logo.png"/>
+ </div>
+
<div id="content">
<h1 class="title">Accumulo User Manual: High Speed Ingest</h1>
<p><strong> Next:</strong> <a href="Analytics.html">Analytics</a> <strong> Up:</strong> <a href="accumulo_user_manual.html">Accumulo User Manual Version 1.4</a> <strong> Previous:</strong> <a href="Table_Design.html">Table Design</a> <strong> <a href="Contents.html">Contents</a></strong> <br />
@@ -140,7 +140,7 @@
accumulo/docs/examples/README.bulkIngest </p>
<h2 id="a_idlogical_time_for_bulk_ingesta_logical_time_for_bulk_ingest"><a id=Logical_Time_for_Bulk_Ingest></a> Logical Time for Bulk Ingest</h2>
<p>Logical time is important for bulk imported data, for which the client code may be choosing a timestamp. At bulk import time, the user can choose to enable logical time for the set of files being imported. When its enabled, Accumulo uses a specialized system iterator to lazily set times in a bulk imported file. This mechanism guarantees that times set by unsynchronized multi-node applications (such as those running on MapReduce) will maintain some semblance of causal ordering. This mitigates the problem of the time being wrong on the system that created the file for bulk import. These times are not set when the file is imported, but whenever it is read by scans or compactions. At import, a time is obtained and always used by the specialized system iterator to set that time. </p>
-<p>The timestamp asigned by accumulo will be the same for every key in the file. This could cause problems if the file contains multiple keys that are identical except for the timestamp. In this case, the sort order of the keys will be undefined. This could occur if an insert and an update were in the same bulk import file. </p>
+<p>The timestamp assigned by accumulo will be the same for every key in the file. This could cause problems if the file contains multiple keys that are identical except for the timestamp. In this case, the sort order of the keys will be undefined. This could occur if an insert and an update were in the same bulk import file. </p>
<h2 id="a_idmapreduce_ingesta_mapreduce_ingest"><a id=MapReduce_Ingest></a> MapReduce Ingest</h2>
<p>It is possible to efficiently write many mutations to Accumulo in parallel via a MapReduce job. In this scenario the MapReduce is written to process data that lives in HDFS and write mutations to Accumulo using the AccumuloOutputFormat. See the MapReduce section under Analytics for details. </p>
<p>An example of using MapReduce can be found under <br />
Modified: websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Introduction.html
==============================================================================
--- websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Introduction.html (original)
+++ websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Introduction.html Mon Dec 19 16:21:39 2011
@@ -37,19 +37,15 @@
</head>
<body>
- <div id="banner">
- <img id="logo" alt="Apache accumulo (Incubating)" src="/accumulo/images/accumulo-logo.png"/>
- <div id="bannertext">
-
- </div><br />
+ <div id="banner">
</div>
-
+
<div id="navigation">
<h1 id="project">Project</h1>
<ul>
<li><a href="/accumulo">Home</a></li>
-<li><a href="http://incubator.apache.org/projects/accumulo.html">Incubator page</a>
-<!-- - Download --></li>
+<li><a href="http://incubator.apache.org/projects/accumulo.html">Incubator page</a></li>
+<li><a href="/accumulo/downloads/downloads.html">Downloads</a></li>
<li><a href="/accumulo/notable_features.html">Features</a></li>
<li><a href="http://www.apache.org/licenses/LICENSE-2.0">License</a></li>
</ul>
@@ -90,6 +86,10 @@
</ul>
</div>
+ <div id="bannertext">
+ <img id="logo" alt="Apache Accumulo (Incubating)" src="/accumulo/images/accumulo-logo.png"/>
+ </div>
+
<div id="content">
<h1 class="title">Accumulo User Manual: Introduction</h1>
<p><strong> Next:</strong> <a href="Accumulo_Design.html">Accumulo Design</a> <strong> Up:</strong> <a href="accumulo_user_manual.html">Accumulo User Manual Version 1.4</a> <strong> Previous:</strong> <a href="Contents.html">Contents</a> <strong> <a href="Contents.html">Contents</a></strong> <br />
Modified: websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Security.html
==============================================================================
--- websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Security.html (original)
+++ websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Security.html Mon Dec 19 16:21:39 2011
@@ -37,19 +37,15 @@
</head>
<body>
- <div id="banner">
- <img id="logo" alt="Apache accumulo (Incubating)" src="/accumulo/images/accumulo-logo.png"/>
- <div id="bannertext">
-
- </div><br />
+ <div id="banner">
</div>
-
+
<div id="navigation">
<h1 id="project">Project</h1>
<ul>
<li><a href="/accumulo">Home</a></li>
-<li><a href="http://incubator.apache.org/projects/accumulo.html">Incubator page</a>
-<!-- - Download --></li>
+<li><a href="http://incubator.apache.org/projects/accumulo.html">Incubator page</a></li>
+<li><a href="/accumulo/downloads/downloads.html">Downloads</a></li>
<li><a href="/accumulo/notable_features.html">Features</a></li>
<li><a href="http://www.apache.org/licenses/LICENSE-2.0">License</a></li>
</ul>
@@ -90,6 +86,10 @@
</ul>
</div>
+ <div id="bannertext">
+ <img id="logo" alt="Apache Accumulo (Incubating)" src="/accumulo/images/accumulo-logo.png"/>
+ </div>
+
<div id="content">
<h1 class="title">Accumulo User Manual: Security</h1>
<p><strong> Next:</strong> <a href="Administration.html">Administration</a> <strong> Up:</strong> <a href="accumulo_user_manual.html">Accumulo User Manual Version 1.4</a> <strong> Previous:</strong> <a href="Analytics.html">Analytics</a> <strong> <a href="Contents.html">Contents</a></strong> <br />
@@ -164,9 +164,9 @@
</pre></div>
-<p>Any user with the alter table permission can add or remove this constraint. This constraint is not applied to bulk imported data, if this a concern then disable the bulk import pesmission. </p>
+<p>Any user with the alter table permission can add or remove this constraint. This constraint is not applied to bulk imported data, if this a concern then disable the bulk import permission. </p>
<h2 id="a_idsecure_authorizations_handlinga_secure_authorizations_handling"><a id=Secure_Authorizations_Handling></a> Secure Authorizations Handling</h2>
-<p>For applications serving many users, it is not expected that a accumulo user will be created for each application user. In this case a accumulo user with all authorizations needed by any of the applications users must be created. To service queries, the application should create a scanner with the application users authorizations. These authorizations could be obtined from a trusted 3rd party. </p>
+<p>For applications serving many users, it is not expected that an accumulo user will be created for each application user. In this case an accumulo user with all authorizations needed by any of the applications users must be created. To service queries, the application should create a scanner with the application users authorizations. These authorizations could be obtained from a trusted 3rd party. </p>
<p>Often production systems will integrate with Public-Key Infrastructure (PKI) and designate client code within the query layer to negotiate with PKI servers in order to authenticate users and retrieve their authorization tokens (credentials). This requires users to specify only the information necessary to authenticate themselves to the system. Once user identity is established, their credentials can be accessed by the client code and passed to Accumulo outside of the reach of the user. </p>
<h2 id="a_idquery_services_layera_query_services_layer"><a id=Query_Services_Layer></a> Query Services Layer</h2>
<p>Since the primary method of interaction with Accumulo is through the Java API, production environments often call for the implementation of a Query layer. This can be done using web services in containers such as Apache Tomcat, but is not a requirement. The Query Services Layer provides a mechanism for providing a platform on which user facing applications can be built. This allows the application designers to isolate potentially complex query logic, and enables a convenient point at which to perform essential security functions. </p>
Modified: websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Shell_Commands.html
==============================================================================
--- websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Shell_Commands.html (original)
+++ websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Shell_Commands.html Mon Dec 19 16:21:39 2011
@@ -37,19 +37,15 @@
</head>
<body>
- <div id="banner">
- <img id="logo" alt="Apache accumulo (Incubating)" src="/accumulo/images/accumulo-logo.png"/>
- <div id="bannertext">
-
- </div><br />
+ <div id="banner">
</div>
-
+
<div id="navigation">
<h1 id="project">Project</h1>
<ul>
<li><a href="/accumulo">Home</a></li>
-<li><a href="http://incubator.apache.org/projects/accumulo.html">Incubator page</a>
-<!-- - Download --></li>
+<li><a href="http://incubator.apache.org/projects/accumulo.html">Incubator page</a></li>
+<li><a href="/accumulo/downloads/downloads.html">Downloads</a></li>
<li><a href="/accumulo/notable_features.html">Features</a></li>
<li><a href="http://www.apache.org/licenses/LICENSE-2.0">License</a></li>
</ul>
@@ -90,6 +86,10 @@
</ul>
</div>
+ <div id="bannertext">
+ <img id="logo" alt="Apache Accumulo (Incubating)" src="/accumulo/images/accumulo-logo.png"/>
+ </div>
+
<div id="content">
<h1 class="title">Accumulo User Manual: Shell Commands</h1>
<p><strong> Up:</strong> <a href="accumulo_user_manual.html">Accumulo User Manual Version 1.4</a> <strong> Previous:</strong> <a href="Administration.html">Administration</a> <strong> <a href="Contents.html">Contents</a></strong> <br />
@@ -373,19 +373,23 @@
</p>
<div class="codehilite"><pre><span class="err">usage:</span> <span class="err">egrep</span> <span class="err"><regex></span> <span class="err"><regex></span> <span class="err">[-?]</span> <span class="err">[-b</span> <span class="err"><start-row>]</span> <span class="err">[-c</span>
<span class="err"><<columnfamily>[:<columnqualifier>]>]</span> <span class="err">[-e</span> <span class="err"><end-row>]</span> <span class="err">[-f</span> <span class="err"><int>]</span>
- <span class="err">[-np]</span> <span class="err">[-s</span> <span class="err"><comma-separated-authorizations>]</span> <span class="err">[-st]</span> <span class="err">[-t</span> <span class="err"><arg>]</span>
+ <span class="err">[-np]</span> <span class="err">[-nt</span> <span class="err"><arg>]</span> <span class="err">[-s</span> <span class="err"><comma-separated-authorizations>]</span> <span class="err">[-st]</span> <span class="err">[-t</span>
+ <span class="err"><table>]</span>
<span class="err">description:</span> <span class="err">searches</span> <span class="err">each</span> <span class="err">row,</span> <span class="err">column</span> <span class="err">family,</span> <span class="err">column</span> <span class="err">qualifier</span> <span class="err">and</span> <span class="err">value,</span> <span class="err">in</span>
- <span class="err">parallel,</span> <span class="err">on</span> <span class="err">the</span> <span class="err">server</span> <span class="err">side</span> <span class="err">(using</span> <span class="err">a</span> <span class="err">java</span> <span class="err">regular</span> <span class="err">expression)</span>
+ <span class="err">parallel,</span> <span class="err">on</span> <span class="err">the</span> <span class="err">server</span> <span class="err">side</span> <span class="err">(using</span> <span class="err">a</span> <span class="err">java</span> <span class="err">Matcher,</span> <span class="err">so</span> <span class="err">put</span> <span class="err">.*</span>
+ <span class="err">before</span> <span class="err">and</span> <span class="err">after</span> <span class="err">your</span> <span class="err">term</span> <span class="err">if</span> <span class="err">you're</span> <span class="err">not</span> <span class="err">matching</span> <span class="err">the</span> <span class="err">whole</span>
+ <span class="err">element)</span>
<span class="err">-?,-help</span> <span class="err">display</span> <span class="err">this</span> <span class="err">help</span>
<span class="err">-b,-begin-row</span> <span class="err"><start-row></span> <span class="err">begin</span> <span class="err">row</span> <span class="err">(inclusive)</span>
<span class="err">-c,-columns</span> <span class="err"><<columnfamily>[:<columnqualifier>]></span> <span class="err">comma-separated</span> <span class="err">columns</span>
<span class="err">-e,-end-row</span> <span class="err"><end-row></span> <span class="err">end</span> <span class="err">row</span> <span class="err">(inclusive)</span>
<span class="err">-f,-show</span> <span class="err">few</span> <span class="err"><int></span> <span class="err">Only</span> <span class="err">shows</span> <span class="err">certain</span> <span class="err">amount</span> <span class="err">of</span> <span class="err">characters</span>
<span class="err">-np,-no-pagination</span> <span class="err">disables</span> <span class="err">pagination</span> <span class="err">of</span> <span class="err">output</span>
+ <span class="err">-nt,-num-threads</span> <span class="err"><arg></span> <span class="err">num</span> <span class="err">threads</span>
<span class="err">-s,-scan-authorizations</span> <span class="err"><comma-separated-authorizations></span> <span class="err">scan</span> <span class="err">authorizations</span>
<span class="err">(all</span> <span class="err">user</span> <span class="err">auths</span> <span class="err">are</span> <span class="err">used</span> <span class="err">if</span> <span class="err">this</span> <span class="err">argument</span> <span class="err">is</span> <span class="err">not</span> <span class="err">specified)</span>
<span class="err">-st,-show-timestamps</span> <span class="err">enables</span> <span class="err">displaying</span> <span class="err">timestamps</span>
- <span class="err">-t,-num-threads</span> <span class="err"><arg></span> <span class="err">num</span> <span class="err">threads</span>
+ <span class="err">-t,-tableName</span> <span class="err"><table></span> <span class="err">table</span> <span class="err">to</span> <span class="err">grep</span> <span class="err">through</span>
</pre></div>
@@ -477,7 +481,8 @@
</p>
<div class="codehilite"><pre><span class="err">usage:</span> <span class="err">grep</span> <span class="err"><term></span> <span class="err"><term></span> <span class="err">[-?]</span> <span class="err">[-b</span> <span class="err"><start-row>]</span> <span class="err">[-c</span>
<span class="err"><<columnfamily>[:<columnqualifier>]>]</span> <span class="err">[-e</span> <span class="err"><end-row>]</span> <span class="err">[-f</span> <span class="err"><int>]</span>
- <span class="err">[-np]</span> <span class="err">[-s</span> <span class="err"><comma-separated-authorizations>]</span> <span class="err">[-st]</span> <span class="err">[-t</span> <span class="err"><arg>]</span>
+ <span class="err">[-np]</span> <span class="err">[-nt</span> <span class="err"><arg>]</span> <span class="err">[-s</span> <span class="err"><comma-separated-authorizations>]</span> <span class="err">[-st]</span> <span class="err">[-t</span>
+ <span class="err"><table>]</span>
<span class="err">description:</span> <span class="err">searches</span> <span class="err">each</span> <span class="err">row,</span> <span class="err">column</span> <span class="err">family,</span> <span class="err">column</span> <span class="err">qualifier</span> <span class="err">and</span> <span class="err">value</span> <span class="err">in</span> <span class="err">a</span>
<span class="err">table</span> <span class="err">for</span> <span class="err">a</span> <span class="err">substring</span> <span class="err">(not</span> <span class="err">a</span> <span class="err">regular</span> <span class="err">expression),</span> <span class="err">in</span> <span class="err">parallel,</span> <span class="err">on</span>
<span class="err">the</span> <span class="err">server</span> <span class="err">side</span>
@@ -487,10 +492,11 @@
<span class="err">-e,-end-row</span> <span class="err"><end-row></span> <span class="err">end</span> <span class="err">row</span> <span class="err">(inclusive)</span>
<span class="err">-f,-show</span> <span class="err">few</span> <span class="err"><int></span> <span class="err">Only</span> <span class="err">shows</span> <span class="err">certain</span> <span class="err">amount</span> <span class="err">of</span> <span class="err">characters</span>
<span class="err">-np,-no-pagination</span> <span class="err">disables</span> <span class="err">pagination</span> <span class="err">of</span> <span class="err">output</span>
+ <span class="err">-nt,-num-threads</span> <span class="err"><arg></span> <span class="err">num</span> <span class="err">threads</span>
<span class="err">-s,-scan-authorizations</span> <span class="err"><comma-separated-authorizations></span> <span class="err">scan</span> <span class="err">authorizations</span>
<span class="err">(all</span> <span class="err">user</span> <span class="err">auths</span> <span class="err">are</span> <span class="err">used</span> <span class="err">if</span> <span class="err">this</span> <span class="err">argument</span> <span class="err">is</span> <span class="err">not</span> <span class="err">specified)</span>
<span class="err">-st,-show-timestamps</span> <span class="err">enables</span> <span class="err">displaying</span> <span class="err">timestamps</span>
- <span class="err">-t,-num-threads</span> <span class="err"><arg></span> <span class="err">num</span> <span class="err">threads</span>
+ <span class="err">-t,-tableName</span> <span class="err"><table></span> <span class="err">table</span> <span class="err">to</span> <span class="err">grep</span> <span class="err">through</span>
</pre></div>
@@ -545,7 +551,7 @@
</p>
<div class="codehilite"><pre><span class="err">usage:</span> <span class="err">listscans</span> <span class="err">[-?]</span> <span class="err">[-np]</span> <span class="err">[-ts</span> <span class="err"><tablet</span> <span class="err">server>]</span>
<span class="err">description:</span> <span class="err">list</span> <span class="err">what</span> <span class="err">scans</span> <span class="err">are</span> <span class="err">currently</span> <span class="err">running</span> <span class="err">in</span> <span class="err">accumulo.</span> <span class="err">See</span> <span class="err">the</span>
- <span class="err">org.apache.accumulo.core.client.admin.ActiveScan</span> <span class="err">javadoc</span> <span class="err">for</span> <span class="err">more</span> <span class="err">information</span>
+ <span class="err">accumulo.core.client.admin.ActiveScan</span> <span class="err">javadoc</span> <span class="err">for</span> <span class="err">more</span> <span class="err">information</span>
<span class="err">about</span> <span class="err">columns.</span>
<span class="err">-?,-help</span> <span class="err">display</span> <span class="err">this</span> <span class="err">help</span>
<span class="err">-np,-no-pagination</span> <span class="err">disables</span> <span class="err">pagination</span> <span class="err">of</span> <span class="err">output</span>
@@ -727,8 +733,8 @@
<p><strong>setiter</strong> <br />
</p>
-<div class="codehilite"><pre><span class="err">usage:</span> <span class="err">setiter</span> <span class="err">[-?]</span> <span class="err">-ageoff</span> <span class="err">|</span> <span class="err">-agg</span> <span class="err">|</span> <span class="err">-</span><span class="kd">class</span> <span class="p"><</span><span class="err">name</span><span class="p">></span> <span class="err">|</span> <span class="err">-nolabel</span> <span class="err">|</span> <span class="err">-regex</span> <span class="err">|</span> <span class="err">-vers</span>
- <span class="err">[-majc]</span> <span class="err">[-minc]</span> <span class="err">[-n</span> <span class="err"><itername>]</span> <span class="err">-p</span> <span class="err"><pri></span> <span class="err">[-scan]</span> <span class="err">[-t</span> <span class="err"><table>]</span>
+<div class="codehilite"><pre><span class="err">usage:</span> <span class="err">setiter</span> <span class="err">[-?]</span> <span class="err">-ageoff</span> <span class="err">|</span> <span class="err">-agg</span> <span class="err">|</span> <span class="err">-</span><span class="kd">class</span> <span class="p"><</span><span class="err">name</span><span class="p">></span> <span class="err">|</span> <span class="err">-regex</span> <span class="err">|</span> <span class="err">-reqvis</span> <span class="err">|</span> <span class="err">-vers</span>
+ <span class="err">[-majc]</span> <span class="err">[-minc]</span> <span class="err">[-n</span> <span class="err"><itername>]</span> <span class="err">-p</span> <span class="err"><pri></span> <span class="err">[-scan]</span> <span class="err">[-t</span> <span class="err"><table>]</span>
<span class="err">description:</span> <span class="err">sets</span> <span class="err">a</span> <span class="err">table-specific</span> <span class="err">iterator</span>
<span class="err">-?,-help</span> <span class="err">display</span> <span class="err">this</span> <span class="err">help</span>
<span class="err">-ageoff,-ageoff</span> <span class="err">an</span> <span class="err">aging</span> <span class="err">off</span> <span class="err">type</span>
@@ -737,9 +743,9 @@
<span class="err">-majc,-major-compaction</span> <span class="err">applied</span> <span class="err">at</span> <span class="err">major</span> <span class="err">compaction</span>
<span class="err">-minc,-minor-compaction</span> <span class="err">applied</span> <span class="err">at</span> <span class="err">minor</span> <span class="err">compaction</span>
<span class="err">-n,-name</span> <span class="err"><itername></span> <span class="err">iterator</span> <span class="err">to</span> <span class="err">set</span>
- <span class="err">-nolabel,-no-label</span> <span class="err">a</span> <span class="err">no-labeling</span> <span class="err">type</span>
<span class="err">-p,-priority</span> <span class="err"><pri></span> <span class="err">the</span> <span class="err">order</span> <span class="err">in</span> <span class="err">which</span> <span class="err">the</span> <span class="err">iterator</span> <span class="err">is</span> <span class="err">applied</span>
<span class="err">-regex,-regular-expression</span> <span class="err">a</span> <span class="err">regex</span> <span class="err">matching</span> <span class="err">type</span>
+ <span class="err">-reqvis,-require-visibility</span> <span class="err">a</span> <span class="err">type</span> <span class="err">that</span> <span class="err">omits</span> <span class="err">entries</span> <span class="err">with</span> <span class="err">empty</span> <span class="err">visibilities</span>
<span class="err">-scan,-scan-time</span> <span class="err">applied</span> <span class="err">at</span> <span class="err">scan</span> <span class="err">time</span>
<span class="err">-t,-table</span> <span class="err"><table></span> <span class="err">tableName</span>
<span class="err">-vers,-version</span> <span class="err">a</span> <span class="err">versioning</span> <span class="err">type</span>
@@ -748,17 +754,17 @@
<p><strong>setscaniter</strong> <br />
</p>
-<div class="codehilite"><pre><span class="err">usage:</span> <span class="err">setscaniter</span> <span class="err">[-?]</span> <span class="err">-ageoff</span> <span class="err">|</span> <span class="err">-agg</span> <span class="err">|</span> <span class="err">-</span><span class="kd">class</span> <span class="p"><</span><span class="err">name</span><span class="p">></span> <span class="err">|</span> <span class="err">-nolabel</span> <span class="err">|</span> <span class="err">-regex</span> <span class="err">|</span>
- <span class="err">-vers</span> <span class="err">[-n</span> <span class="err"><itername>]</span> <span class="err">-p</span> <span class="err"><pri></span> <span class="err">[-t</span> <span class="err"><table>]</span>
+<div class="codehilite"><pre><span class="err">usage:</span> <span class="err">setscaniter</span> <span class="err">[-?]</span> <span class="err">-ageoff</span> <span class="err">|</span> <span class="err">-agg</span> <span class="err">|</span> <span class="err">-</span><span class="kd">class</span> <span class="p"><</span><span class="err">name</span><span class="p">></span> <span class="err">|</span> <span class="err">-regex</span> <span class="err">|</span> <span class="err">-reqvis</span> <span class="err">|</span> <span class="err">-vers</span>
+ <span class="err">[-n</span> <span class="err"><itername>]</span> <span class="err">-p</span> <span class="err"><pri></span> <span class="err">[-t</span> <span class="err"><table>]</span>
<span class="err">description:</span> <span class="err">sets</span> <span class="err">a</span> <span class="err">table-specific</span> <span class="err">scan</span> <span class="err">iterator</span> <span class="err">for</span> <span class="err">this</span> <span class="err">shell</span> <span class="err">session</span>
<span class="err">-?,-help</span> <span class="err">display</span> <span class="err">this</span> <span class="err">help</span>
<span class="err">-ageoff,-ageoff</span> <span class="err">an</span> <span class="err">aging</span> <span class="err">off</span> <span class="err">type</span>
<span class="err">-agg,-aggregator</span> <span class="err">an</span> <span class="err">aggregating</span> <span class="err">type</span>
<span class="err">-class,-class-name</span> <span class="err"><name></span> <span class="err">a</span> <span class="err">java</span> <span class="err">class</span> <span class="err">type</span>
<span class="err">-n,-name</span> <span class="err"><itername></span> <span class="err">iterator</span> <span class="err">to</span> <span class="err">set</span>
- <span class="err">-nolabel,-no-label</span> <span class="err">a</span> <span class="err">no-labeling</span> <span class="err">type</span>
<span class="err">-p,-priority</span> <span class="err"><pri></span> <span class="err">the</span> <span class="err">order</span> <span class="err">in</span> <span class="err">which</span> <span class="err">the</span> <span class="err">iterator</span> <span class="err">is</span> <span class="err">applied</span>
<span class="err">-regex,-regular-expression</span> <span class="err">a</span> <span class="err">regex</span> <span class="err">matching</span> <span class="err">type</span>
+ <span class="err">-reqvis,-require-visibility</span> <span class="err">a</span> <span class="err">type</span> <span class="err">that</span> <span class="err">omits</span> <span class="err">entries</span> <span class="err">with</span> <span class="err">empty</span> <span class="err">visibilities</span>
<span class="err">-t,-table</span> <span class="err"><table></span> <span class="err">tableName</span>
<span class="err">-vers,-version</span> <span class="err">a</span> <span class="err">versioning</span> <span class="err">type</span>
</pre></div>
Modified: websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Table_Configuration.html
==============================================================================
--- websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Table_Configuration.html (original)
+++ websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Table_Configuration.html Mon Dec 19 16:21:39 2011
@@ -37,19 +37,15 @@
</head>
<body>
- <div id="banner">
- <img id="logo" alt="Apache accumulo (Incubating)" src="/accumulo/images/accumulo-logo.png"/>
- <div id="bannertext">
-
- </div><br />
+ <div id="banner">
</div>
-
+
<div id="navigation">
<h1 id="project">Project</h1>
<ul>
<li><a href="/accumulo">Home</a></li>
-<li><a href="http://incubator.apache.org/projects/accumulo.html">Incubator page</a>
-<!-- - Download --></li>
+<li><a href="http://incubator.apache.org/projects/accumulo.html">Incubator page</a></li>
+<li><a href="/accumulo/downloads/downloads.html">Downloads</a></li>
<li><a href="/accumulo/notable_features.html">Features</a></li>
<li><a href="http://www.apache.org/licenses/LICENSE-2.0">License</a></li>
</ul>
@@ -90,6 +86,10 @@
</ul>
</div>
+ <div id="bannertext">
+ <img id="logo" alt="Apache Accumulo (Incubating)" src="/accumulo/images/accumulo-logo.png"/>
+ </div>
+
<div id="content">
<h1 class="title">Accumulo User Manual: Table Configuration</h1>
<p><strong> Next:</strong> <a href="Table_Design.html">Table Design</a> <strong> Up:</strong> <a href="accumulo_user_manual.html">Accumulo User Manual Version 1.4</a> <strong> Previous:</strong> <a href="Writing_Accumulo_Clients.html">Writing Accumulo Clients</a> <strong> <a href="Contents.html">Contents</a></strong> <br />
@@ -175,7 +175,7 @@
accumulo/docs/examples/README.constraints with corresponding code under <br />
accumulo/src/examples/main/java/accumulo/examples/constraints . </p>
<h2 id="a_idbloom_filtersa_bloom_filters"><a id=Bloom_Filters></a> Bloom Filters</h2>
-<p>As mutations are applied to a Accumulo table, several files are created per tablet. If bloom filters are enabled, Accumulo will create and load a small data structure into memory to determine whether a file contains a given key before opening the file. This can speed up lookups considerably. </p>
+<p>As mutations are applied to an Accumulo table, several files are created per tablet. If bloom filters are enabled, Accumulo will create and load a small data structure into memory to determine whether a file contains a given key before opening the file. This can speed up lookups considerably. </p>
<p>To enable bloom filters, enter the following command in the Shell: </p>
<div class="codehilite"><pre><span class="n">user</span><span class="nv">@myinstance</span><span class="o">></span> <span class="n">config</span> <span class="o">-</span><span class="n">t</span> <span class="n">mytable</span> <span class="o">-</span><span class="n">s</span> <span class="n">table</span><span class="o">.</span><span class="n">bloom</span><span class="o">.</span><span class="n">enabled</span><span class="o">=</span><span class="n">true</span>
</pre></div>
@@ -186,9 +186,9 @@ accumulo/docs/examples/README.bloom . </
<h2 id="a_iditeratorsa_iterators"><a id=Iterators></a> Iterators</h2>
<p>Iterators provide a modular mechanism for adding functionality to be executed by TabletServers when scanning or compacting data. This allows users to efficiently summarize, filter, and aggregate data. In fact, the built-in features of cell-level security and column fetching are implemented using Iterators. Some useful Iterators are provided with Accumulo and can be found in the org.apache.accumulo.core.iterators.user package. </p>
<h3 id="a_idsetting_iterators_via_the_shella_setting_iterators_via_the_shell"><a id=Setting_Iterators_via_the_Shell></a> Setting Iterators via the Shell</h3>
-<div class="codehilite"><pre><span class="err">usage:</span> <span class="err">setiter</span> <span class="err">[-?]</span> <span class="err">-agg</span> <span class="err">|</span> <span class="err">-</span><span class="kd">class</span> <span class="p"><</span><span class="err">name</span><span class="p">></span> <span class="err">|</span> <span class="err">-filter</span> <span class="err">|</span> <span class="err">-nolabel</span> <span class="err">|</span>
-<span class="err">-regex</span> <span class="err">|</span> <span class="err">-vers</span> <span class="err">[-majc]</span> <span class="err">[-minc]</span> <span class="err">[-n</span> <span class="err"><itername>]</span> <span class="err">-p</span> <span class="err"><pri></span> <span class="err">[-scan]</span>
-<span class="err">[-t</span> <span class="err"><table>]</span>
+<div class="codehilite"><pre><span class="err">usage:</span> <span class="err">setiter</span> <span class="err">[-?]</span> <span class="err">-ageoff</span> <span class="err">|</span> <span class="err">-agg</span> <span class="err">|</span> <span class="err">-</span><span class="kd">class</span> <span class="p"><</span><span class="err">name</span><span class="p">></span> <span class="err">|</span> <span class="err">-regex</span> <span class="err">|</span>
+<span class="err">-reqvis</span> <span class="err">|</span> <span class="err">-vers</span> <span class="err">[-majc]</span> <span class="err">[-minc]</span> <span class="err">[-n</span> <span class="err"><itername>]</span> <span class="err">-p</span> <span class="err"><pri></span>
+<span class="err">[-scan]</span> <span class="err">[-t</span> <span class="err"><table>]</span>
<span class="err">user@myinstance</span> <span class="err">mytable></span> <span class="err">setiter</span> <span class="err">-t</span> <span class="err">mytable</span> <span class="err">-scan</span> <span class="err">-p</span> <span class="err">10</span> <span class="err">-n</span> <span class="err">myiter</span>
</pre></div>
@@ -277,9 +277,9 @@ org.apache.accumulo.core.iterators.Filte
</pre></div>
-<h3 id="a_idaggregating_iteratorsa_aggregating_iterators"><a id=Aggregating_Iterators></a> Aggregating Iterators</h3>
-<p>Accumulo allows aggregating iterators to be configured on tables and column families. When an aggregating iterator is set, the iterator is applied across the values associated with any keys that share rowID, column family, and column qualifier. This is similar to the reduce step in MapReduce, which applied some function to all the values associated with a particular key. </p>
-<p>For example, if an aggregating iterator were configured on a table and the following mutations were inserted: </p>
+<h3 id="a_idcombinersa_combiners"><a id=Combiners></a> Combiners</h3>
+<p>Accumulo allows Combiners to be configured on tables and column families. When a Combiner is set it is applied across the values associated with any keys that share rowID, column family, and column qualifier. This is similar to the reduce step in MapReduce, which applied some function to all the values associated with a particular key. </p>
+<p>For example, if a summing combiner were configured on a table and the following mutations were inserted: </p>
<div class="codehilite"><pre><span class="n">Row</span> <span class="n">Family</span> <span class="n">Qualifier</span> <span class="n">Timestamp</span> <span class="n">Value</span>
<span class="n">rowID1</span> <span class="n">colfA</span> <span class="n">colqA</span> <span class="mi">20100101</span> <span class="mi">1</span>
<span class="n">rowID1</span> <span class="n">colfA</span> <span class="n">colqA</span> <span class="mi">20100102</span> <span class="mi">1</span>
@@ -291,34 +291,34 @@ org.apache.accumulo.core.iterators.Filte
</pre></div>
-<p>Aggregating iterators can be enabled for a table as follows: </p>
-<div class="codehilite"><pre><span class="n">user</span><span class="nv">@myinstance</span><span class="o">></span> <span class="n">createtable</span> <span class="n">perDayCounts</span> <span class="o">-</span><span class="n">a</span>
-<span class="n">day</span><span class="o">=</span><span class="n">org</span><span class="o">.</span><span class="n">apache</span><span class="o">.</span><span class="n">accumulo</span><span class="o">.</span><span class="n">core</span><span class="o">.</span><span class="n">iterators</span><span class="o">.</span><span class="n">aggregation</span><span class="o">.</span><span class="n">StringSummation</span>
-
-<span class="n">user</span><span class="nv">@myinstance</span> <span class="n">perDayCounts</span><span class="o">></span> <span class="n">insert</span> <span class="n">row1</span> <span class="n">day</span> <span class="mi">20080101</span> <span class="mi">1</span>
-<span class="n">user</span><span class="nv">@myinstance</span> <span class="n">perDayCounts</span><span class="o">></span> <span class="n">insert</span> <span class="n">row1</span> <span class="n">day</span> <span class="mi">20080101</span> <span class="mi">1</span>
-<span class="n">user</span><span class="nv">@myinstance</span> <span class="n">perDayCounts</span><span class="o">></span> <span class="n">insert</span> <span class="n">row1</span> <span class="n">day</span> <span class="mi">20080103</span> <span class="mi">1</span>
-<span class="n">user</span><span class="nv">@myinstance</span> <span class="n">perDayCounts</span><span class="o">></span> <span class="n">insert</span> <span class="n">row2</span> <span class="n">day</span> <span class="mi">20080101</span> <span class="mi">1</span>
-<span class="n">user</span><span class="nv">@myinstance</span> <span class="n">perDayCounts</span><span class="o">></span> <span class="n">insert</span> <span class="n">row3</span> <span class="n">day</span> <span class="mi">20080101</span> <span class="mi">1</span>
-
-<span class="n">user</span><span class="nv">@myinstance</span> <span class="n">perDayCounts</span><span class="o">></span> <span class="n">scan</span>
-<span class="n">row1</span> <span class="n">day:20080101</span> <span class="o">[]</span> <span class="mi">2</span>
-<span class="n">row1</span> <span class="n">day:20080103</span> <span class="o">[]</span> <span class="mi">1</span>
-<span class="n">row2</span> <span class="n">day:20080101</span> <span class="o">[]</span> <span class="mi">2</span>
+<p>Combiners can be enabled for a table using the setiter command in the shell. Below is an example. </p>
+<div class="codehilite"><pre><span class="n">root</span><span class="nv">@a14</span> <span class="n">perDayCounts</span><span class="o">></span> <span class="n">setiter</span> <span class="o">-</span><span class="n">t</span> <span class="n">perDayCounts</span> <span class="o">-</span><span class="n">p</span> <span class="mi">10</span> <span class="o">-</span><span class="n">scan</span> <span class="o">-</span><span class="n">minc</span> <span class="o">-</span><span class="n">majc</span> <span class="o">-</span><span class="n">n</span> <span class="n">daycount</span>
+ <span class="o">-</span><span class="n">class</span> <span class="n">org</span><span class="o">.</span><span class="n">apache</span><span class="o">.</span><span class="n">accumulo</span><span class="o">.</span><span class="n">core</span><span class="o">.</span><span class="n">iterators</span><span class="o">.</span><span class="n">user</span><span class="o">.</span><span class="n">SummingCombiner</span>
+<span class="n">TypedValueCombiner</span> <span class="n">can</span> <span class="n">interpret</span> <span class="n">Values</span> <span class="n">as</span> <span class="n">a</span> <span class="n">variety</span> <span class="n">of</span> <span class="n">number</span> <span class="n">encodings</span>
+ <span class="p">(</span><span class="n">VLong</span><span class="p">,</span> <span class="n">Long</span><span class="p">,</span> <span class="ow">or</span> <span class="n">String</span><span class="p">)</span> <span class="n">before</span> <span class="n">combining</span>
+<span class="o">----------></span> <span class="n">set</span> <span class="n">SummingCombiner</span> <span class="n">parameter</span> <span class="n">columns</span><span class="p">,</span>
+ <span class="o"><</span><span class="n">col</span> <span class="n">fam</span><span class="o">></span><span class="p">[:</span><span class="o"><</span><span class="n">col</span> <span class="n">qual</span><span class="o">></span><span class="p">]{,</span><span class="o"><</span><span class="n">col</span> <span class="n">fam</span><span class="o">></span><span class="p">[:</span><span class="o"><</span><span class="n">col</span> <span class="n">qual</span><span class="o">></span><span class="p">]}</span> <span class="p">:</span> <span class="n">day</span>
+<span class="o">----------></span> <span class="n">set</span> <span class="n">SummingCombiner</span> <span class="n">parameter</span> <span class="n">type</span><span class="p">,</span> <span class="sr"><VARNUM|LONG|STRING></span><span class="p">:</span> <span class="n">STRING</span>
+
+<span class="n">root</span><span class="nv">@a14</span> <span class="n">perDayCounts</span><span class="o">></span> <span class="n">insert</span> <span class="n">foo</span> <span class="n">day</span> <span class="mi">20080101</span> <span class="mi">1</span>
+<span class="n">root</span><span class="nv">@a14</span> <span class="n">perDayCounts</span><span class="o">></span> <span class="n">insert</span> <span class="n">foo</span> <span class="n">day</span> <span class="mi">20080101</span> <span class="mi">1</span>
+<span class="n">root</span><span class="nv">@a14</span> <span class="n">perDayCounts</span><span class="o">></span> <span class="n">insert</span> <span class="n">foo</span> <span class="n">day</span> <span class="mi">20080103</span> <span class="mi">1</span>
+<span class="n">root</span><span class="nv">@a14</span> <span class="n">perDayCounts</span><span class="o">></span> <span class="n">insert</span> <span class="n">bar</span> <span class="n">day</span> <span class="mi">20080101</span> <span class="mi">1</span>
+<span class="n">root</span><span class="nv">@a14</span> <span class="n">perDayCounts</span><span class="o">></span> <span class="n">insert</span> <span class="n">bar</span> <span class="n">day</span> <span class="mi">20080101</span> <span class="mi">1</span>
+
+<span class="n">root</span><span class="nv">@a14</span> <span class="n">perDayCounts</span><span class="o">></span> <span class="n">scan</span>
+<span class="n">bar</span> <span class="n">day:20080101</span> <span class="o">[]</span> <span class="mi">2</span>
+<span class="n">foo</span> <span class="n">day:20080101</span> <span class="o">[]</span> <span class="mi">2</span>
+<span class="n">foo</span> <span class="n">day:20080103</span> <span class="o">[]</span> <span class="mi">1</span>
</pre></div>
-<p>Accumulo includes the following aggregators: </p>
-<ul>
-<li><strong>LongSummation</strong>: expects values of type long and adds them. </li>
-<li><strong>StringSummation</strong>: expects numbers represented as strings and adds them. </li>
-<li><strong>StringMax</strong>: expects numbers as strings and retains the maximum number inserted. </li>
-<li><strong>StringMin</strong>: expects numbers as strings and retains the minimum number inserted. </li>
-</ul>
-<p>Additional Aggregators can be added by creating a Java class that implements <br />
-<strong>org.apache.accumulo.core.iterators.aggregation.Aggregator</strong> and adding a jar containing that class to Accumulo's lib directory. </p>
-<p>An example of an aggregator can be found under <br />
-accumulo/src/examples/main/java/org/apache/accumulo/examples/aggregation/SortedSetAggregator.java </p>
+<p>Accumulo includes some useful Combiners out of the box. To find these look in the <br />
+<strong>org.apache.accumulo.core.iterators.user</strong> package. </p>
+<p>Additional Combiners can be added by creating a Java class that extends <br />
+<strong>org.apache.accumulo.core.iterators.Combiner</strong> and adding a jar containing that class to Accumulo's lib/ext directory. </p>
+<p>An example of a Combiner can be found under <br />
+accumulo/src/examples/main/java/org/apache/accumulo/examples/combiner/StatsCombiner.java </p>
<h2 id="a_idblock_cachea_block_cache"><a id=Block_Cache></a> Block Cache</h2>
<p>In order to increase throughput of commonly accessed entries, Accumulo employs a block cache. This block cache buffers data in memory so that it doesn't have to be read off of disk. The RFile format that Accumulo prefers is a mix of index blocks and data blocks, where the index blocks are used to find the appropriate data blocks. Typical queries to Accumulo result in a binary search over several index blocks followed by a linear scan of one or more data blocks. </p>
<p>The block cache can be configured on a per-table basis, and all tablets hosted on a tablet server share a single resource pool. To configure the size of the tablet server's block cache, set the following properties: </p>
@@ -359,7 +359,7 @@ accumulo/src/examples/main/java/org/apac
</pre></div>
-<p>When a tablet reaches this number of files and needs to flush its in-memory data to disk, it will choose to do a merging minor compaction. A merging minor compaction will merge the tablet's smallest file with the data in memory at minor compaction time. Therefore the number of files will not grow beyond this limit. This will make minor compactions take longer, which will cause ingest performance to decrease. This can cause ingest to slow down until major compactions have enough time to catch up. When adjusting this property, also consider adjusting the compaction ratio. Ideally, merging minor compactions never need to occur and major compactions will keep up. It is possible to configure the file max and compaction ratio such that only merging minor compactions occur and major compactions never occur. This should be avoided because doing only merging minor compactions causes <img alt="$O(N^2)$" src="img2.png" /> work to be done. The amount of work done by major compactions
is <img alt="$O(N*klzzwxh:0051og_R(N))$" src="img3.png" /> where <em>R</em> is the compaction ratio. </p>
+<p>When a tablet reaches this number of files and needs to flush its in-memory data to disk, it will choose to do a merging minor compaction. A merging minor compaction will merge the tablet's smallest file with the data in memory at minor compaction time. Therefore the number of files will not grow beyond this limit. This will make minor compactions take longer, which will cause ingest performance to decrease. This can cause ingest to slow down until major compactions have enough time to catch up. When adjusting this property, also consider adjusting the compaction ratio. Ideally, merging minor compactions never need to occur and major compactions will keep up. It is possible to configure the file max and compaction ratio such that only merging minor compactions occur and major compactions never occur. This should be avoided because doing only merging minor compactions causes <img alt="$O(N^2)$" src="img2.png" /> work to be done. The amount of work done by major compactions
is <img alt="$O(N*klzzwxh:0053og_R(N))$" src="img3.png" /> where <em>R</em> is the compaction ratio. </p>
<p>Compactions can be initiated manually for a table. To initiate a minor compaction, use the flush command in the shell. To initiate a major compaction, use the compact command in the shell. The compact command will compact all tablets in a table to one file. Even tablets with one file are compacted. This is useful for the case where a major compaction filter is configured for a table. In 1.4 the ability to compact a range of a table was added. To use this feature specify start and stop rows for the compact command. This will only compact tablets that overlap the given row range. </p>
<h2 id="a_idpre-splitting_tablesa_pre-splitting_tables"><a id=Pre-splitting_tables></a> Pre-splitting tables</h2>
<p>Accumulo will balance and distribute tables accross servers. Before a table gets large, it will be maintained as a single tablet on a single server. This limits the speed at which data can be added or queried to the speed of a single node. To improve performance when the a table is new, or small, you can add split points and generate new tablets. </p>
Modified: websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Table_Design.html
==============================================================================
--- websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Table_Design.html (original)
+++ websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Table_Design.html Mon Dec 19 16:21:39 2011
@@ -37,19 +37,15 @@
</head>
<body>
- <div id="banner">
- <img id="logo" alt="Apache accumulo (Incubating)" src="/accumulo/images/accumulo-logo.png"/>
- <div id="bannertext">
-
- </div><br />
+ <div id="banner">
</div>
-
+
<div id="navigation">
<h1 id="project">Project</h1>
<ul>
<li><a href="/accumulo">Home</a></li>
-<li><a href="http://incubator.apache.org/projects/accumulo.html">Incubator page</a>
-<!-- - Download --></li>
+<li><a href="http://incubator.apache.org/projects/accumulo.html">Incubator page</a></li>
+<li><a href="/accumulo/downloads/downloads.html">Downloads</a></li>
<li><a href="/accumulo/notable_features.html">Features</a></li>
<li><a href="http://www.apache.org/licenses/LICENSE-2.0">License</a></li>
</ul>
@@ -90,6 +86,10 @@
</ul>
</div>
+ <div id="bannertext">
+ <img id="logo" alt="Apache Accumulo (Incubating)" src="/accumulo/images/accumulo-logo.png"/>
+ </div>
+
<div id="content">
<h1 class="title">Accumulo User Manual: Table Design</h1>
<p><strong> Next:</strong> <a href="High_Speed_Ingest.html">High-Speed Ingest</a> <strong> Up:</strong> <a href="accumulo_user_manual.html">Accumulo User Manual Version 1.4</a> <strong> Previous:</strong> <a href="Table_Configuration.html">Table Configuration</a> <strong> <a href="Contents.html">Contents</a></strong> <br />
Modified: websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Writing_Accumulo_Clients.html
==============================================================================
--- websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Writing_Accumulo_Clients.html (original)
+++ websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/Writing_Accumulo_Clients.html Mon Dec 19 16:21:39 2011
@@ -37,19 +37,15 @@
</head>
<body>
- <div id="banner">
- <img id="logo" alt="Apache accumulo (Incubating)" src="/accumulo/images/accumulo-logo.png"/>
- <div id="bannertext">
-
- </div><br />
+ <div id="banner">
</div>
-
+
<div id="navigation">
<h1 id="project">Project</h1>
<ul>
<li><a href="/accumulo">Home</a></li>
-<li><a href="http://incubator.apache.org/projects/accumulo.html">Incubator page</a>
-<!-- - Download --></li>
+<li><a href="http://incubator.apache.org/projects/accumulo.html">Incubator page</a></li>
+<li><a href="/accumulo/downloads/downloads.html">Downloads</a></li>
<li><a href="/accumulo/notable_features.html">Features</a></li>
<li><a href="http://www.apache.org/licenses/LICENSE-2.0">License</a></li>
</ul>
@@ -90,6 +86,10 @@
</ul>
</div>
+ <div id="bannertext">
+ <img id="logo" alt="Apache Accumulo (Incubating)" src="/accumulo/images/accumulo-logo.png"/>
+ </div>
+
<div id="content">
<h1 class="title">Accumulo User Manual: Writing Accumulo Clients</h1>
<p><strong> Next:</strong> <a href="Table_Configuration.html">Table Configuration</a> <strong> Up:</strong> <a href="accumulo_user_manual.html">Accumulo User Manual Version 1.4</a> <strong> Previous:</strong> <a href="Accumulo_Shell.html">Accumulo Shell</a> <strong> <a href="Contents.html">Contents</a></strong> <br />
@@ -171,7 +171,7 @@ accumulo/docs/examples/README.batch </p>
<li>iterators executed as part of a minor or major compaction </li>
<li>bulk import of new files </li>
</ul>
-<p>Isolation guarantees that either all or none of the changes made by these operations on a row are seen. Use the IsolatedScanner to obtain an isolated view of a accumulo table. When using the regular scanner it is possible to see a non isolated view of a row. For example if a mutation modifies three columns, it is possible that you will only see two of those modifications. With the isolated scanner either all three of the changes are seen or none. </p>
+<p>Isolation guarantees that either all or none of the changes made by these operations on a row are seen. Use the IsolatedScanner to obtain an isolated view of an accumulo table. When using the regular scanner it is possible to see a non isolated view of a row. For example if a mutation modifies three columns, it is possible that you will only see two of those modifications. With the isolated scanner either all three of the changes are seen or none. </p>
<p>The IsolatedScanner buffers rows on the client side so a large row will not crash a tablet server. By default rows are buffered in memory, but the user can easily supply their own buffer if they wish to buffer to disk when rows are large. </p>
<p>For an example, look at the following <br />
src/examples/src/main/java/org/apache/accumulo/examples/isolation/InterferenceTest.java</p>
Modified: websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/accumulo_user_manual.html
==============================================================================
--- websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/accumulo_user_manual.html (original)
+++ websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/accumulo_user_manual.html Mon Dec 19 16:21:39 2011
@@ -37,19 +37,15 @@
</head>
<body>
- <div id="banner">
- <img id="logo" alt="Apache accumulo (Incubating)" src="/accumulo/images/accumulo-logo.png"/>
- <div id="bannertext">
-
- </div><br />
+ <div id="banner">
</div>
-
+
<div id="navigation">
<h1 id="project">Project</h1>
<ul>
<li><a href="/accumulo">Home</a></li>
-<li><a href="http://incubator.apache.org/projects/accumulo.html">Incubator page</a>
-<!-- - Download --></li>
+<li><a href="http://incubator.apache.org/projects/accumulo.html">Incubator page</a></li>
+<li><a href="/accumulo/downloads/downloads.html">Downloads</a></li>
<li><a href="/accumulo/notable_features.html">Features</a></li>
<li><a href="http://www.apache.org/licenses/LICENSE-2.0">License</a></li>
</ul>
@@ -90,6 +86,10 @@
</ul>
</div>
+ <div id="bannertext">
+ <img id="logo" alt="Apache Accumulo (Incubating)" src="/accumulo/images/accumulo-logo.png"/>
+ </div>
+
<div id="content">
<h1 class="title">Accumulo User Manual: index</h1>
<p><strong> Next:</strong> <a href="Contents.html">Contents</a> <strong> <a href="Contents.html">Contents</a></strong> <br />
Modified: websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/img1.png
==============================================================================
Binary files - no diff available.
Modified: websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/img4.png
==============================================================================
Binary files - no diff available.
Modified: websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/img5.png
==============================================================================
Binary files - no diff available.
Modified: websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/img6.png
==============================================================================
Binary files - no diff available.
Modified: websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/img7.png
==============================================================================
Binary files - no diff available.
Modified: websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/index.html
==============================================================================
--- websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/index.html (original)
+++ websites/production/accumulo/content/accumulo/user_manual_1.4-incubating/index.html Mon Dec 19 16:21:39 2011
@@ -37,19 +37,15 @@
</head>
<body>
- <div id="banner">
- <img id="logo" alt="Apache accumulo (Incubating)" src="/accumulo/images/accumulo-logo.png"/>
- <div id="bannertext">
-
- </div><br />
+ <div id="banner">
</div>
-
+
<div id="navigation">
<h1 id="project">Project</h1>
<ul>
<li><a href="/accumulo">Home</a></li>
-<li><a href="http://incubator.apache.org/projects/accumulo.html">Incubator page</a>
-<!-- - Download --></li>
+<li><a href="http://incubator.apache.org/projects/accumulo.html">Incubator page</a></li>
+<li><a href="/accumulo/downloads/downloads.html">Downloads</a></li>
<li><a href="/accumulo/notable_features.html">Features</a></li>
<li><a href="http://www.apache.org/licenses/LICENSE-2.0">License</a></li>
</ul>
@@ -90,6 +86,10 @@
</ul>
</div>
+ <div id="bannertext">
+ <img id="logo" alt="Apache Accumulo (Incubating)" src="/accumulo/images/accumulo-logo.png"/>
+ </div>
+
<div id="content">
<h1 class="title">Accumulo User Manual: index</h1>
<p><strong> Next:</strong> <a href="Contents.html">Contents</a> <strong> <a href="Contents.html">Contents</a></strong> <br />