You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@accumulo.apache.org by ec...@apache.org on 2011/11/08 15:01:48 UTC
svn commit: r1199246 - in /incubator/accumulo/trunk: ./ docs/
docs/src/user_manual/appendices/ docs/src/user_manual/chapters/
src/assemble/ src/core/src/main/java/org/apache/accumulo/core/conf/
Author: ecn
Date: Tue Nov 8 14:01:47 2011
New Revision: 1199246
URL: http://svn.apache.org/viewvc?rev=1199246&view=rev
Log:
ACCUMULO-125 merge minor issues found in 1.3 release to trunk
Modified:
incubator/accumulo/trunk/ (props changed)
incubator/accumulo/trunk/docs/aggregation.html
incubator/accumulo/trunk/docs/config.html
incubator/accumulo/trunk/docs/constraints.html
incubator/accumulo/trunk/docs/isolation.html
incubator/accumulo/trunk/docs/lgroups.html
incubator/accumulo/trunk/docs/src/user_manual/appendices/shell_commands.tex
incubator/accumulo/trunk/docs/src/user_manual/chapters/table_configuration.tex
incubator/accumulo/trunk/src/assemble/dist.xml
incubator/accumulo/trunk/src/core/src/main/java/org/apache/accumulo/core/conf/Property.java
Propchange: incubator/accumulo/trunk/
------------------------------------------------------------------------------
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Nov 8 14:01:47 2011
@@ -1 +1 @@
-/incubator/accumulo/branches/1.3:1190280,1190413,1190420,1190427,1190500,1195622,1195625,1195629,1195635,1196044,1196054,1196057,1196071-1196072,1196106,1197066
+/incubator/accumulo/branches/1.3:1190280,1190413,1190420,1190427,1190500,1195622,1195625,1195629,1195635,1196044,1196054,1196057,1196071-1196072,1196106,1197066,1198935
Modified: incubator/accumulo/trunk/docs/aggregation.html
URL: http://svn.apache.org/viewvc/incubator/accumulo/trunk/docs/aggregation.html?rev=1199246&r1=1199245&r2=1199246&view=diff
==============================================================================
--- incubator/accumulo/trunk/docs/aggregation.html (original)
+++ incubator/accumulo/trunk/docs/aggregation.html Tue Nov 8 14:01:47 2011
@@ -53,7 +53,7 @@ user@instance:9999 perDayCounts>
</pre>
-<p>Implementing a new aggregation function is a snap. Simply write some Java code that implements <a href='apidocs/accumulo/core/iterators/aggregation/Aggregator.html'>org.apache.accumulo.core.iterators.aggregation.Aggregator</a>. A good example to look at is <a href='apidocs/accumulo/core/iterators/aggregation/StringSummation.html'>StringSummation</a> which sums numbers encoded as ascii strings. However, one could easily write a much more efficient summation aggregator that operates on numbers encoded in twos complement.
+<p>Implementing a new aggregation function is a snap. Simply write some Java code that implements <a href='apidocs/org/apache/accumulo/core/iterators/aggregation/Aggregator.html'>org.apache.accumulo.core.iterators.aggregation.Aggregator</a>. A good example to look at is <a href='apidocs/org/apache/accumulo/core/iterators/aggregation/StringSummation.html'>StringSummation</a> which sums numbers encoded as ascii strings. However, one could easily write a much more efficient summation aggregator that operates on numbers encoded in twos complement.
<p>To deploy a new aggregator, jar it up and put the jar in accumulo/lib. To see an example look at <a href='examples/README.aggregation'>README.aggregation</a>
Modified: incubator/accumulo/trunk/docs/config.html
URL: http://svn.apache.org/viewvc/incubator/accumulo/trunk/docs/config.html?rev=1199246&r1=1199245&r2=1199246&view=diff
==============================================================================
--- incubator/accumulo/trunk/docs/config.html (original)
+++ incubator/accumulo/trunk/docs/config.html Tue Nov 8 14:01:47 2011
@@ -669,7 +669,7 @@ $HADOOP_HOME/lib/[^.].*.jar,
<td>table.balancer</td>
<td><b><a href='#STRING'>string</a></b></td>
<td>yes</td>
- <td><pre>accumulo.server.master.balancer.DefaultLoadBalancer</pre></td>
+ <td><pre>org.apache.accumulo.server.master.balancer.DefaultLoadBalancer</pre></td>
<td>This property can be set to allow the LoadBalanceByTable load balancer to change the called Load Balancer for this table</td>
</tr>
<tr >
@@ -698,7 +698,7 @@ $HADOOP_HOME/lib/[^.].*.jar,
<td><b><a href='#CLASSNAME'>java class</a></b></td>
<td>yes</td>
<td><pre>org.apache.accumulo.core.file.keyfunctor.RowFunctor</pre></td>
- <td>A function that can transform the key prior to insertion and check of bloom filter. accumulo.core.file.keyfunctor.RowFunctor,,accumulo.core.file.keyfunctor.ColumnFamilyFunctor, and accumulo.core.file.keyfunctor.ColumnQualifierFunctor are allowable values. One can extend any of the above mentioned classes to perform specialized parsing of the key. </td>
+ <td>A function that can transform the key prior to insertion and check of bloom filter. org.apache.accumulo.core.file.keyfunctor.RowFunctor,,org.apache.accumulo.core.file.keyfunctor.ColumnFamilyFunctor, and org.apache.accumulo.core.file.keyfunctor.ColumnQualifierFunctor are allowable values. One can extend any of the above mentioned classes to perform specialized parsing of the key. </td>
</tr>
<tr >
<td>table.bloom.load.threshold</td>
@@ -848,9 +848,9 @@ $HADOOP_HOME/lib/[^.].*.jar,
<td>Use the write-ahead log to prevent the loss of data.</td>
</tr>
<tr><td colspan='5'><a name='TABLE_CONSTRAINT_PREFIX' class='large'>table.constraint.*</a></td></tr>
- <tr><td colspan='5'><i>Properties in this category are per-table properties that add constraints to a table. These properties start with the category prefix, followed by a number, and their values correspond to a fully qualified Java class that implements the Constraint interface.<br />For example, table.constraint.1 = accumulo.core.constraints.MyCustomConstraint and table.constraint.2 = my.package.constraints.MySecondConstraint</i></td></tr>
+ <tr><td colspan='5'><i>Properties in this category are per-table properties that add constraints to a table. These properties start with the category prefix, followed by a number, and their values correspond to a fully qualified Java class that implements the Constraint interface.<br />For example, table.constraint.1 = org.apache.accumulo.core.constraints.MyCustomConstraint and table.constraint.2 = my.package.constraints.MySecondConstraint</i></td></tr>
<tr><td colspan='5'><a name='TABLE_ITERATOR_PREFIX' class='large'>table.iterator.*</a></td></tr>
- <tr><td colspan='5'><i>Properties in this category specify iterators that are applied at various stages (scopes) of interaction with a table. These properties start with the category prefix, followed by a scope (minc, majc, scan, etc.), followed by a period, followed by a name, as in table.iterator.scan.vers, or table.iterator.scan.custom. The values for these properties are a number indicating the ordering in which it is applied, and a class name such as table.iterator.scan.vers = 10,accumulo.core.iterators.VersioningIterator<br /> These iterators can take options if additional properties are set that look like this property, but are suffixed with a period, followed by 'opt' followed by another period, and a property name.<br />For example, table.iterator.minc.vers.opt.maxVersions = 3</i></td></tr>
+ <tr><td colspan='5'><i>Properties in this category specify iterators that are applied at various stages (scopes) of interaction with a table. These properties start with the category prefix, followed by a scope (minc, majc, scan, etc.), followed by a period, followed by a name, as in table.iterator.scan.vers, or table.iterator.scan.custom. The values for these properties are a number indicating the ordering in which it is applied, and a class name such as table.iterator.scan.vers = 10,org.apache.accumulo.core.iterators.VersioningIterator<br /> These iterators can take options if additional properties are set that look like this property, but are suffixed with a period, followed by 'opt' followed by another period, and a property name.<br />For example, table.iterator.minc.vers.opt.maxVersions = 3</i></td></tr>
<tr><td colspan='5'><a name='TABLE_LOCALITY_GROUP_PREFIX' class='large'>table.group.*</a></td></tr>
<tr><td colspan='5'><i>Properties in this category are per-table properties that define locality groups in a table. These properties start with the category prefix, followed by a name, followed by a period, and followed by a property for that group.<br />For example table.group.group1=x,y,z sets the column families for a group called group1. Once configured, group1 can be enabled by adding it to the list of groups in the table.groups.enabled property.<br />Additional group options may be specified for a named group by setting table.group.<name>.opt.<key>=<value>.</i></td></tr>
</table>
Modified: incubator/accumulo/trunk/docs/constraints.html
URL: http://svn.apache.org/viewvc/incubator/accumulo/trunk/docs/constraints.html?rev=1199246&r1=1199245&r2=1199246&view=diff
==============================================================================
--- incubator/accumulo/trunk/docs/constraints.html (original)
+++ incubator/accumulo/trunk/docs/constraints.html Tue Nov 8 14:01:47 2011
@@ -25,7 +25,7 @@
Accumulo supports constraints. Constraints are applied to mutations at ingest time.
-<p>Implementing a new constraint is a snap. Simply write some Java code that implements <a href='apidocs/accumulo/core/constraints/Constraint.html'>org.apache.accumulo.core.constraints.Constraint</a>.
+<p>Implementing a new constraint is a snap. Simply write some Java code that implements <a href='apidocs/org/apache/accumulo/core/constraints/Constraint.html'>org.apache.accumulo.core.constraints.Constraint</a>.
<p>To deploy a new constraint, jar it up and put the jar in accumulo/lib/ext.
Modified: incubator/accumulo/trunk/docs/isolation.html
URL: http://svn.apache.org/viewvc/incubator/accumulo/trunk/docs/isolation.html?rev=1199246&r1=1199245&r2=1199246&view=diff
==============================================================================
--- incubator/accumulo/trunk/docs/isolation.html (original)
+++ incubator/accumulo/trunk/docs/isolation.html Tue Nov 8 14:01:47 2011
@@ -31,9 +31,9 @@
<li>iterators executed as part of a minor or major compaction
<li>bulk import of new files
</ul>
-Isolation garuantees that either all or none of the changes made by these operations on a row are seen. Use the <a href='apidocs/accumulo/core/client/IsolatedScanner.html'>IsolatedScanner</a> to obtain an isolated view of a accumulo table. When using the regular scanner it is possible to see a non isolated view of a row. For example if a mutation modifies three columns, it is possible that you will only see two of those modifications. With the isolated scanner either all three of the changes are seen or none. For an example of this try running the <a href='apidocs/accumulo/examples/isolation/InterferenceTest.html'>InterferenceTest</a> example.
+Isolation garuantees that either all or none of the changes made by these operations on a row are seen. Use the <a href='apidocs/org/apache/accumulo/core/client/IsolatedScanner.html'>IsolatedScanner</a> to obtain an isolated view of a accumulo table. When using the regular scanner it is possible to see a non isolated view of a row. For example if a mutation modifies three columns, it is possible that you will only see two of those modifications. With the isolated scanner either all three of the changes are seen or none. For an example of this try running the <a href='apidocs/org/apache/accumulo/examples/isolation/InterferenceTest.html'>InterferenceTest</a> example.
-<p>At this time there is no client side isolation support for the <a href='apidocs/accumulo/core/client/BatchScanner.html'>BatchScanner</a>. You may consider using the <a href='apidocs/accumulo/core/iterators/WholeRowIterator.html'>WholeRowIterator</a> with the <a href='apidocs/accumulo/core/client/BatchScanner.html'>BatchScanner</a> to achieve isolation though. This drawback of doing this is that entire rows are read into memory on the server side. If a row is too big, it may crash a tablet server. The <a href='apidocs/accumulo/core/client/IsolatedScanner.html'>IsolatedScanner</a> buffers rows on the client side so a large row will not crash a tablet server.
+<p>At this time there is no client side isolation support for the <a href='apidocs/org/apache/accumulo/core/client/BatchScanner.html'>BatchScanner</a>. You may consider using the <a href='apidocs/org/apache/accumulo/core/iterators/WholeRowIterator.html'>WholeRowIterator</a> with the <a href='apidocs/org/apache/accumulo/core/client/BatchScanner.html'>BatchScanner</a> to achieve isolation though. This drawback of doing this is that entire rows are read into memory on the server side. If a row is too big, it may crash a tablet server. The <a href='apidocs/org/apache/accumulo/core/client/IsolatedScanner.html'>IsolatedScanner</a> buffers rows on the client side so a large row will not crash a tablet server.
<h3>Iterators</h3>
<p>When writing server side iterators for accumulo isolation is something to be aware of. A scan time iterator in accumulo reads from a set of data sources. While an iterator is reading data it has an isolated view. However, after it returns a key/value it is possible that accumulo may switch data sources and re-seek the iterator. This is done so that resources may be reclaimed. When the user does not request isolation this can occur after any key is returned. When a user request isolation this will only occur after a new row is returned, in which case it will reseek to the very beginning of the next possible row.
Modified: incubator/accumulo/trunk/docs/lgroups.html
URL: http://svn.apache.org/viewvc/incubator/accumulo/trunk/docs/lgroups.html?rev=1199246&r1=1199245&r2=1199246&view=diff
==============================================================================
--- incubator/accumulo/trunk/docs/lgroups.html (original)
+++ incubator/accumulo/trunk/docs/lgroups.html Tue Nov 8 14:01:47 2011
@@ -34,9 +34,9 @@
<P>When the locality group configuration for a table is changed it has no effect on existing data. All minor and major compactions that occur after the change will organize data into the new locality group structure. As data is written into a table, it will cause minor and major compactions to occur. Over time this will result in all data being organized according to the new locality groups. If all data must be reorganized into the new locality groups immediately, this can be accomplished by forcing a full major compaction of the table. Use the compact command in the shell to accomplish this.
-<P>There are two ways to manipulate locality groups, via the shell or through the Java API. From the shell use the getgroups and setgroups commands. Through the API, <a href='apidocs/accumulo/core/client/admin/TableOperations.html'>TableOperations</a> has the methods setLocalityGroups() and getLocalityGroups().
+<P>There are two ways to manipulate locality groups, via the shell or through the Java API. From the shell use the getgroups and setgroups commands. Through the API, <a href='apidocs/org/apache/accumulo/core/client/admin/TableOperations.html'>TableOperations</a> has the methods setLocalityGroups() and getLocalityGroups().
-<P>To limit scans to a set of locality groups, use the fetchColumnFamily() function on <a href='apidocs/accumulo/core/client/Scanner.html'>Scanner</a> or <a href='apidocs/accumulo/core/client/BatchScanner.html'>BatchScanner</a>. From the shell use scan with the -c option.
+<P>To limit scans to a set of locality groups, use the fetchColumnFamily() function on <a href='apidocs/org/apache/accumulo/core/client/Scanner.html'>Scanner</a> or <a href='apidocs/org/apache/accumulo/core/client/BatchScanner.html'>BatchScanner</a>. From the shell use scan with the -c option.
</body>
</html>
Modified: incubator/accumulo/trunk/docs/src/user_manual/appendices/shell_commands.tex
URL: http://svn.apache.org/viewvc/incubator/accumulo/trunk/docs/src/user_manual/appendices/shell_commands.tex?rev=1199246&r1=1199245&r2=1199246&view=diff
==============================================================================
--- incubator/accumulo/trunk/docs/src/user_manual/appendices/shell_commands.tex (original)
+++ incubator/accumulo/trunk/docs/src/user_manual/appendices/shell_commands.tex Tue Nov 8 14:01:47 2011
@@ -390,7 +390,7 @@
usage: listscans [-?] [-np] [-ts <tablet server>]
description: list what scans are currently running in accumulo. See the
- accumulo.core.client.admin.ActiveScan javadoc for more information
+ org.apache.accumulo.core.client.admin.ActiveScan javadoc for more information
about columns.
-?,--help display this help
-np,--no-pagination disables pagination of output
Modified: incubator/accumulo/trunk/docs/src/user_manual/chapters/table_configuration.tex
URL: http://svn.apache.org/viewvc/incubator/accumulo/trunk/docs/src/user_manual/chapters/table_configuration.tex?rev=1199246&r1=1199245&r2=1199246&view=diff
==============================================================================
--- incubator/accumulo/trunk/docs/src/user_manual/chapters/table_configuration.tex (original)
+++ incubator/accumulo/trunk/docs/src/user_manual/chapters/table_configuration.tex Tue Nov 8 14:01:47 2011
@@ -353,7 +353,7 @@ Additional Aggregators can be added by c
class to Accumulo's lib directory.
An example of an aggregator can be found under\\
-accumulo/src/examples/main/java/accumulo/examples/aggregation/SortedSetAggregator.java
+accumulo/src/examples/main/java/org/apache/accumulo/examples/aggregation/SortedSetAggregator.java
\section{Block Cache}
Modified: incubator/accumulo/trunk/src/assemble/dist.xml
URL: http://svn.apache.org/viewvc/incubator/accumulo/trunk/src/assemble/dist.xml?rev=1199246&r1=1199245&r2=1199246&view=diff
==============================================================================
--- incubator/accumulo/trunk/src/assemble/dist.xml (original)
+++ incubator/accumulo/trunk/src/assemble/dist.xml Tue Nov 8 14:01:47 2011
@@ -81,6 +81,7 @@
<exclude>**/randomwalk/conf/logger.xml</exclude>
<exclude>**/randomwalk/conf/randomwalk.conf</exclude>
<exclude>**/randomwalk/conf/walkers</exclude>
+ <exclude>src/**/*.so</exclude>
</excludes>
<fileMode>0644</fileMode>
<directoryMode>0755</directoryMode>
Modified: incubator/accumulo/trunk/src/core/src/main/java/org/apache/accumulo/core/conf/Property.java
URL: http://svn.apache.org/viewvc/incubator/accumulo/trunk/src/core/src/main/java/org/apache/accumulo/core/conf/Property.java?rev=1199246&r1=1199245&r2=1199246&view=diff
==============================================================================
--- incubator/accumulo/trunk/src/core/src/main/java/org/apache/accumulo/core/conf/Property.java (original)
+++ incubator/accumulo/trunk/src/core/src/main/java/org/apache/accumulo/core/conf/Property.java Tue Nov 8 14:01:47 2011
@@ -215,7 +215,7 @@ public enum Property {
"The maximum amount of memory that will be used to cache results of a client query/scan. "
+ "Once this limit is reached, the buffered data is sent to the client."),
TABLE_FILE_TYPE("table.file.type", RFile.EXTENSION, PropertyType.STRING, "Change the type of file a table writes"),
- TABLE_LOAD_BALANCER("table.balancer", "accumulo.server.master.balancer.DefaultLoadBalancer", PropertyType.STRING,
+ TABLE_LOAD_BALANCER("table.balancer", "org.apache.accumulo.server.master.balancer.DefaultLoadBalancer", PropertyType.STRING,
"This property can be set to allow the LoadBalanceByTable load balancer to change the called Load Balancer for this table"),
TABLE_FILE_COMPRESSION_TYPE("table.file.compress.type", "gz", PropertyType.STRING, "One of gz,lzo,none"),
TABLE_FILE_COMPRESSED_BLOCK_SIZE("table.file.compress.blocksize", "100K", PropertyType.MEMORY,
@@ -240,8 +240,8 @@ public enum Property {
TABLE_BLOOM_SIZE("table.bloom.size", "1048576", PropertyType.COUNT, "Bloom filter size, as number of keys."),
TABLE_BLOOM_ERRORRATE("table.bloom.error.rate", "0.5%", PropertyType.FRACTION, "Bloom filter error rate."),
TABLE_BLOOM_KEY_FUNCTOR("table.bloom.key.functor", "org.apache.accumulo.core.file.keyfunctor.RowFunctor", PropertyType.CLASSNAME,
- "A function that can transform the key prior to insertion and check of bloom filter. accumulo.core.file.keyfunctor.RowFunctor,"
- + ",accumulo.core.file.keyfunctor.ColumnFamilyFunctor, and accumulo.core.file.keyfunctor.ColumnQualifierFunctor are allowable values."
+ "A function that can transform the key prior to insertion and check of bloom filter. org.apache.accumulo.core.file.keyfunctor.RowFunctor,"
+ + ",org.apache.accumulo.core.file.keyfunctor.ColumnFamilyFunctor, and org.apache.accumulo.core.file.keyfunctor.ColumnQualifierFunctor are allowable values."
+ " One can extend any of the above mentioned classes to perform specialized parsing of the key. "),
TABLE_BLOOM_HASHTYPE("table.bloom.hash.type", "murmur", PropertyType.STRING, "The bloom filter hash type"),
TABLE_FAILURES_IGNORE("table.failures.ignore", "false", PropertyType.BOOLEAN,
@@ -260,7 +260,7 @@ public enum Property {
"Properties in this category are per-table properties that add constraints to a table. "
+ "These properties start with the category prefix, followed by a number, and their values "
+ "correspond to a fully qualified Java class that implements the Constraint interface.<br />"
- + "For example, table.constraint.1 = accumulo.core.constraints.MyCustomConstraint "
+ + "For example, table.constraint.1 = org.apache.accumulo.core.constraints.MyCustomConstraint "
+ "and table.constraint.2 = my.package.constraints.MySecondConstraint"),
TABLE_INDEXCACHE_ENABLED("table.cache.index.enable", "true", PropertyType.BOOLEAN, "Determines whether index cache is enabled."),
TABLE_BLOCKCACHE_ENABLED("table.cache.block.enable", "false", PropertyType.BOOLEAN, "Determines whether file block cache is enabled."),
@@ -269,7 +269,7 @@ public enum Property {
+ "with a table. These properties start with the category prefix, followed by a scope (minc, majc, scan, etc.), "
+ "followed by a period, followed by a name, as in table.iterator.scan.vers, or table.iterator.scan.custom. "
+ "The values for these properties are a number indicating the ordering in which it is applied, and a class name "
- + "such as table.iterator.scan.vers = 10,accumulo.core.iterators.VersioningIterator<br /> "
+ + "such as table.iterator.scan.vers = 10,org.apache.accumulo.core.iterators.VersioningIterator<br /> "
+ "These iterators can take options if additional properties are set that look like this property, "
+ "but are suffixed with a period, followed by 'opt' followed by another period, and a property name.<br />"
+ "For example, table.iterator.minc.vers.opt.maxVersions = 3"),