You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@accumulo.apache.org by el...@apache.org on 2013/07/19 00:30:08 UTC
svn commit: r1504698 - in /accumulo/site/branches/git: ./
content/notable_features.mdtext content/people.mdtext content/source.mdtext
Author: elserj
Date: Thu Jul 18 22:30:08 2013
New Revision: 1504698
URL: http://svn.apache.org/r1504698
Log:
Merge the prod site into the git fork and remove the now useless svn info
Modified:
accumulo/site/branches/git/ (props changed)
accumulo/site/branches/git/content/notable_features.mdtext
accumulo/site/branches/git/content/people.mdtext
accumulo/site/branches/git/content/source.mdtext
Propchange: accumulo/site/branches/git/
------------------------------------------------------------------------------
svn:mergeinfo = /accumulo/site/trunk:1490086-1504677
Modified: accumulo/site/branches/git/content/notable_features.mdtext
URL: http://svn.apache.org/viewvc/accumulo/site/branches/git/content/notable_features.mdtext?rev=1504698&r1=1504697&r2=1504698&view=diff
==============================================================================
--- accumulo/site/branches/git/content/notable_features.mdtext (original)
+++ accumulo/site/branches/git/content/notable_features.mdtext Thu Jul 18 22:30:08 2013
@@ -74,11 +74,11 @@ and Zookeeper (see [FATE](#fate)).
### Write ahead log
-Tablet servers send mutations to loggers, not HDFS. Each mutation is written
-to multiple loggers, before being committed. In normal configurations, loggers
+In version 1.4, tablet servers send mutations to loggers which write to the local file system, not HDFS.
+Each mutation is written to multiple loggers, before being committed. In normal configurations, loggers
are run on every server that runs a tablet server. When a log is needed for
recovery, it is copied into HDFS. For efficient recovery, the log is sorted as
-it is copied.
+it is copied. In version 1.5, write ahead logs are written to HDFS, and loggers are no longer needed.
### Logical time
@@ -136,7 +136,8 @@ Scans will not see data inserted into a
If consecutive keys have identical portions (row, colf, colq, or colvis), there
is a flag to indicate that a portion is the same as that of the previous key.
This is applied when keys are stored on disk and when transferred over the
-network.
+network. Starting with 1.5, prefix erasure is supported. When its cost
+effective, prefixes repeated in subsequent key fields are not repeated.
### Native In-Memory Map
@@ -170,6 +171,16 @@ written. When an index block exceeds the
written out between data blocks. The size of index blocks is configurable on a
per table basis.
+### Binary search in RFile blocks (1.5)
+
+RFile uses its index to locate a block of key values. Once it reaches a block
+it performs a linear scan to find a key on interest. Starting with 1.5, Accumulo
+will generate indexes of cached blocks in an adaptive manner. Accumulo indexes
+the blocks that are read most frequently. When a block is read a few times, a
+small index is generated. As a block is read more, larger indexes are generated
+making future seeks faster. This strategy allows Accumulo to dynamically respond
+to read patterns without precomputing block indexes when RFiles are written.
+
## Testing <a id="testing"></a>
### Mock
@@ -177,6 +188,13 @@ per table basis.
The Accumulo client API has a mock implementation that is useful writing unit
test against Accumulo. Mock Accumulo is in memory and in process.
+### Mini Accumulo Cluster (1.5 & 1.4.4)
+
+Mini Accumulo cluster is a set of utility code that makes it easy to spin up
+a local Accumulo instance running against the local filesystem. Mini Accumulo
+is slower than Mock Accumulo, but its behavior mirrors a real Accumulo
+instance more closely.
+
### Functional Test
Small, system-level tests of basic Accumulo features run in a test harness,
@@ -236,6 +254,13 @@ could be different from the Accumulo nod
Accumulo can be a source and/or sink for map reduce jobs.
+### Thrift Proxy (1.5 & 1.4.4)
+
+The Accumulo client code contains a lot of complexity. For example, the
+client code locates tablets, retries in the case of failures, and supports
+concurrent reading and writing. All of this is written in Java. The thrift
+proxy wraps the Accumulo client API with thrift, making this API easily
+available to other languages like Python, Ruby, C++, etc.
## Extensible Behaviors <a id="behaviors"></a>
@@ -327,6 +352,12 @@ was growing. Without this feature, inge
constant rate, even as scan performance decreases because tablets have too many
files.
+### Loading jars using VFS (1.5)
+
+User written iterators are a useful way to manipulate data in data in Accumulo.
+Before 1.5., users had to copy their iterators to each tablet server. Starting
+with 1.5 Accumulo can load iterators from HDFS using Apache commons VFS.
+
## On-demand Data Management <a id="ondemand_dm"></a>
### Compactions
@@ -335,7 +366,8 @@ Ability to force tablets to compact to o
compacted. This is useful for improving query performance, permanently
applying iterators, or using a new locality group configuration. One example
of using iterators is applying a filtering iterator to remove data from a
-table.
+table. As of 1.5, users can initiate a compaction with iterators only applied to
+that compaction event.
### Split points
@@ -346,7 +378,8 @@ used to accommodate new data patterns in
### Tablet Merging (1.4)
Tablet merging is a new feature. Merging of tablets can be requested in the
-shell; Accumulo does not merge tablets automatically.
+shell; Accumulo does not merge tablets automatically. In 1.5, the METADATA tablets
+can be merged.
### Table Cloning (1.4)
@@ -356,6 +389,11 @@ mutated independently. Testing was the m
feature. For example to test a new filtering iterator, clone the table, add the
filter to the clone, and force a major compaction.
+### Import/Export Table (1.5)
+
+An offline tables metadata and files can easily be copied to another cluster and
+imported.
+
### Compact Range (1.4)
Compact each tablet that falls within a row range down to a single file.
@@ -369,4 +407,3 @@ beginning and end of the range are split
[2]: /1.4/user_manual/Security.html
[4]: /1.4/user_manual/Writing_Accumulo_Clients.html
[6]: /1.4/user_manual/High_Speed_Ingest.html#Bulk_Ingest
-
Modified: accumulo/site/branches/git/content/people.mdtext
URL: http://svn.apache.org/viewvc/accumulo/site/branches/git/content/people.mdtext?rev=1504698&r1=1504697&r2=1504698&view=diff
==============================================================================
--- accumulo/site/branches/git/content/people.mdtext (original)
+++ accumulo/site/branches/git/content/people.mdtext Thu Jul 18 22:30:08 2013
@@ -48,12 +48,13 @@ PMC and Committers
<tr><th>username</th><th>name</th><th>organization</th><th>timezone</th></tr>
<tr><td>acordova</td><td>Aaron Cordova</td><td></td><td></td></tr>
<tr><td>afuchs</td><td>Adam Fuchs</td><td><a href=http://sqrrl.com/>sqrrl</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
- <tr><td>bimargulies</td><td>Benson Margulies</td><td></td><td></td></tr>
+ <tr><td>bimargulies</td><td>Benson Margulies</td><td><a href="http://www.basistech.com">Basis Technology Corp.</a></td>
+<td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
<tr><td>billie</td><td>Billie Rinaldi</td><td><a href=http://hortonworks.com/>Hortonworks</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
<tr><td>brianloss</td><td>Brian Loss</td><td><a href=http://www.praxiseng.com/>Praxis Engineering</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
<tr><td>cawaring</td><td>Chris Waring</td><td></td><td></td></tr>
<tr><td><a href="http://people.apache.org/~ctubbsii/">ctubbsii</a></td><td>Christopher Tubbs</td><td><a href="http://www.nsa.gov/">NSA</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
- <tr><td>cjnolet</td><td>Corey J. Nolet</td><td><a href=http://www.texeltek.com/>Texeltek, Inc.</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
+ <tr><td>cjnolet</td><td>Corey J. Nolet</td><td><a href=http://www.objectivesolutions.com/>Objective Solutions, Inc.</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
<tr><td>dlmarion</td><td>Dave Marion</td><td><a href=http://www.objectivesolutions.com>Objective Solutions, Inc.</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
<tr><td>medined</td><td>David Medinets</td><td></td><td></td></tr>
<tr><td>drew</td><td>Drew Farris</td><td><a href=http://www.boozallen.com/>Booz Allen Hamilton</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
@@ -62,6 +63,7 @@ PMC and Committers
<tr><td>vines</td><td>John Vines</td><td><a href=http://sqrrl.com/>sqrrl</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
<tr><td>elserj</td><td>Josh Elser</td><td><a href="http://sra.com">SRA International, Inc</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
<tr><td>kturner</td><td>Keith Turner</td><td><a href=http://www.ptech-llc.com/>Peterson Technologies</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
+ <tr><td>mdrob</td><td>Mike Drob</td><td></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
<tr><td>ujustgotbilld</td><td>William Slacum</td><td><a href="http://www.atg-hq.com">Agile Technology Group, Inc</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
</table>
@@ -75,6 +77,7 @@ Contributors
<tr><td></td><td>Chris McCubbin</td><td><a href=http://sqrrl.com/>sqrrl</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
<tr><td></td><td>Christian Rohling</td><td><a href=http://endgame.com/>Endgame</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
<tr><td></td><td>Damon Brown</td><td><a href="http://www.tetraconcepts.com">Tetra Concepts LLC</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
+ <tr><td></td><td>David M. Lyle</td><td></td><td></td></tr>
<tr><td></td><td>Dennis Patrone</td><td><a href=http://www.jhuapl.edu/>The Johns Hopkins University<br>Applied Physics Laboratory</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
<tr><td></td><td>Ed Kohlwey</td><td><a href=http://www.boozallen.com/>Booz Allen Hamilton</a></td><td></td></tr>
<tr><td>edwardyoon</td><td>Edward Yoon</td><td></td><td></td></tr>
@@ -83,6 +86,7 @@ Contributors
<tr><td></td><td>Jesse Yates</td><td></td><td></td></tr>
<tr><td></td><td>Jim Klucar</td><td><a href=http://www.praxiseng.com/>Praxis Engineering</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
<tr><td></td><td>Joe Skora</td><td></td><td></td></tr>
+ <tr><td>jmhsieh</td><td>Jonathan M. Hsieh</td><td><a href="http://www.cloudera.com">Cloudera</a></td><td>PT (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/pst.html>-8</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/pdt.html>-7</a>)</td></tr>
<tr><td></td><td>Kevin Faro</td><td><a href="http://www.tetraconcepts.com">Tetra Concepts LLC</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
<tr><td></td><td>Laura Peaslee</td><td><a href=http://www.objectivesolutions.com>Objective Solutions, Inc.</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
<tr><td></td><td>Luke Brassard</td><td><a href=http://sqrrl.com/>sqrrl</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
@@ -90,7 +94,7 @@ Contributors
<tr><td></td><td>Michael Allen</td><td><a href=http://sqrrl.com/>sqrrl</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
<tr><td></td><td>Michael Berman</td><td></td><td></td></tr>
<tr><td></td><td>Michael Wall</td><td></td><td></td></tr>
- <tr><td></td><td>Mike Drob</td><td></td><td></td></tr>
+ <tr><td></td><td>Miguel Pereira</td><td><a href="http://sra.com">SRA International, Inc</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
<tr><td></td><td>Oren Falkowitz</td><td><a href=http://sqrrl.com/>sqrrl</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
<tr><td></td><td>Phil Eberhardt</td><td><a href=http://sqrrl.com/>sqrrl</a></td><td>ET (<a href=http://www.timeanddate.com/library/abbreviations/timezones/na/est.html>-5</a> / <a href=http://www.timeanddate.com/library/abbreviations/timezones/na/edt.html>-4</a>)</td></tr>
<tr><td></td><td>Philip Young</td><td></td><td></td></tr>
@@ -99,6 +103,6 @@ Contributors
<tr><td></td><td>Scott Kuehn</td><td></td><td></td></tr>
<tr><td>supun</td><td>Supun Kamburugamuva</td><td></td><td></td></tr>
<tr><td></td><td>Tim Halloran</td><td></td><td></td></tr>
- <tr><td></td><td>Tim Reardon</td><td><a href="http://www.texeltek.com/">TexelTek</a></td><td></td></tr>
+ <tr><td></td><td>Tim Reardon</td><td></td><td></td></tr>
<tr><td></td><td>Travis Pinney</td><td></td><td></td></tr>
</table>
\ No newline at end of file
Modified: accumulo/site/branches/git/content/source.mdtext
URL: http://svn.apache.org/viewvc/accumulo/site/branches/git/content/source.mdtext?rev=1504698&r1=1504697&r2=1504698&view=diff
==============================================================================
--- accumulo/site/branches/git/content/source.mdtext (original)
+++ accumulo/site/branches/git/content/source.mdtext Thu Jul 18 22:30:08 2013
@@ -18,12 +18,12 @@ Notice: Licensed to the Apache Softwa
## Source Code
-Apache Accumulo™ source code is maintained using [Apache Subversion][subversion] version control
-([browse][viewvc]|[checkout][svnloc]). It builds with [Apache Maven][maven].
+Apache Accumulo™ source code is maintained using [Git][git] version control
+([browse][cgit]|[checkout][anongit]). It builds with [Apache Maven][maven].
## Website
-Accumulo's web site is also maintained in Subversion using Apache's [Content Management System][cms].
+Accumulo's web site is maintained in [Subversion][subversion] using Apache's [Content Management System][cms].
Committers may edit the site by following [these instructions][cmsusage]. Non-committers should follow
[this FAQ entry][cmsanon].
@@ -39,12 +39,12 @@ will wait for you to enter a newline to
'thrift' command is in your path. Watch out for THRIFT-1367; you may need to configure Thrift with
--without-ruby.
-#### Checking out from svn
+#### Checking out from Git
To check out the code:
- svn co https://svn.apache.org/repos/asf/accumulo/trunk/
+ git clone http://git-wip-us.apache.org/repos/asf/accumulo.git
#### Building
@@ -73,28 +73,6 @@ Accumulo uses [Jenkins][jenkins] for aut
Accumulo [tracks issues][jiraloc] with [JIRA][jira]. Every commit should reference a JIRA ticket of the form ACCUMULO-#.
-### Merging Practices
-
-Changes should be merged from earlier branches of Accumulo to later branches. Currently, all changes are being merged from the 1.4 branch and later, but not all changes are merged from the 1.3 branch. When you make a change in branch 1.x, use the following process to merge to subsequent versions 1.y and 1.z:
-
- svn merge -r 1:HEAD 1.x_svn_url 1.y_local_dir
-
- examine merged files and conflicts
-
- svn ci 1.y_local_dir
-
- make sure the commit includes the svn mergeinfo
-
- svn merge -r 1:HEAD 1.y_svn_url 1.z_local_dir
-
- examine merged files and conflicts
-
- svn ci 1.z_local_dir
-
- make sure the commit includes the svn mergeinfo
-
-An additional step is now required to merge from 1.4 to 1.5 since the directory structure changed. Ask the dev list for instructions.
-
### Coding Practices
* License Header: Always add the current ASF license header as described in [ASF Source Header][srcheaders].
@@ -122,8 +100,6 @@ Accumulo's release guide can be found [h
[subversion]: http://subversion.apache.org/
[maven]: http://maven.apache.org/
-[viewvc]: http://svn.apache.org/viewvc/accumulo/
-[svnloc]: https://svn.apache.org/repos/asf/accumulo/trunk/
[srcheaders]: http://www.apache.org/legal/src-headers.html
[styles]: https://svn.apache.org/repos/asf/accumulo/trunk/contrib/
[jenkins]: http://jenkins-ci.org/
@@ -133,7 +109,10 @@ Accumulo's release guide can be found [h
[13build]: https://builds.apache.org/job/Accumulo-1.3.x/
[jiraloc]: https://issues.apache.org/jira/browse/accumulo
[jira]: http://www.atlassian.com/software/jira/overview
-[release]: /governance/releasing.html
+[release]: governance/releasing.html
[cms]: http://www.apache.org/dev/cms.html
[cmsusage]: http://www.apache.org/dev/cms.html#usage
[cmsanon]: http://www.apache.org/dev/cmsref.html#non-committer
+[git]: http://git-scm.com/
+[cgit]: https://git-wip-us.apache.org/repos/asf?p=accumulo.git;a=summary
+[anongit]: http://git-wip-us.apache.org/repos/asf/accumulo.git