You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@accumulo.apache.org by ec...@apache.org on 2014/01/07 00:33:54 UTC
[1/8] git commit: [maven-release-plugin] copy for tag 1.5.0-RC5
Updated Branches:
refs/heads/1.5.0 [created] 5266a71be
refs/heads/1.5.1-SNAPSHOT f43ba3bb7 -> 65aaddd17
refs/heads/1.6.0-SNAPSHOT d1f3dfe8a -> 4a8738349
[maven-release-plugin] copy for tag 1.5.0-RC5
git-svn-id: https://svn.apache.org/repos/asf/accumulo/tags/1.5.0-RC5@1484984 13f79535-47bb-0310-9956-ffa450edef68
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/c96d4c59
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/c96d4c59
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/c96d4c59
Branch: refs/heads/1.5.0
Commit: c96d4c59c9686c038ba6118e89350104a02c3110
Parents: 3854c8d
Author: Christopher Tubbs <ct...@apache.org>
Authored: Tue May 21 21:51:31 2013 +0000
Committer: Christopher Tubbs <ct...@apache.org>
Committed: Tue May 21 21:51:31 2013 +0000
----------------------------------------------------------------------
----------------------------------------------------------------------
[4/8] git commit: ACCUMULO-2141 found a typo
Posted by ec...@apache.org.
ACCUMULO-2141 found a typo
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/a18e469e
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/a18e469e
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/a18e469e
Branch: refs/heads/1.6.0-SNAPSHOT
Commit: a18e469e922f937716aa0d69e0570219614a2220
Parents: 4c9a666
Author: Eric Newton <er...@gmail.com>
Authored: Mon Jan 6 18:33:35 2014 -0500
Committer: Eric Newton <er...@gmail.com>
Committed: Mon Jan 6 18:33:35 2014 -0500
----------------------------------------------------------------------
docs/examples/README.dirlist | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/accumulo/blob/a18e469e/docs/examples/README.dirlist
----------------------------------------------------------------------
diff --git a/docs/examples/README.dirlist b/docs/examples/README.dirlist
index b60650f..e505cf9 100644
--- a/docs/examples/README.dirlist
+++ b/docs/examples/README.dirlist
@@ -56,7 +56,7 @@ To perform searches on file or directory names, also use QueryUtil.java. Search
$ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.QueryUtil -i instance -z zookeepers -u username -p password -t indexTable --auths exampleVis --path filename --search
$ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.QueryUtil -i instance -z zookeepers -u username -p password -t indexTable --auths exampleVis --path 'filename*' --search
$ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.QueryUtil -i instance -z zookeepers -u username -p password -t indexTable --auths exampleVis --path '*jar' --search
- $ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.QueryUtil -i instance -z zookeepers -u username -p password -t indexTable --auths exampleVis iipath 'filename*jar' --search
+ $ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.QueryUtil -i instance -z zookeepers -u username -p password -t indexTable --auths exampleVis --path 'filename*jar' --search
To count the number of direct children (directories and files) and descendants (children and children's descendants, directories and files), run the FileCount over the dirTable table.
The results are written back to the same table. FileCount reads from and writes to Accumulo. This requires scan authorizations for the read and a visibility for the data written.
[2/8] git commit: ACCUMULO-970 Rename 1.5.0-RC5 tag to 1.5.0
Posted by ec...@apache.org.
ACCUMULO-970 Rename 1.5.0-RC5 tag to 1.5.0
git-svn-id: https://svn.apache.org/repos/asf/accumulo/tags/1.5.0@1486259 13f79535-47bb-0310-9956-ffa450edef68
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/5266a71b
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/5266a71b
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/5266a71b
Branch: refs/heads/1.5.0
Commit: 5266a71be09772d58a3cce223fdc61192097c339
Parents: c96d4c5
Author: Christopher Tubbs <ct...@apache.org>
Authored: Sat May 25 00:25:00 2013 +0000
Committer: Christopher Tubbs <ct...@apache.org>
Committed: Sat May 25 00:25:00 2013 +0000
----------------------------------------------------------------------
----------------------------------------------------------------------
[8/8] git commit: Merge branch '1.5.1-SNAPSHOT' of
https://git-wip-us.apache.org/repos/asf/accumulo into 1.5.1-SNAPSHOT
Posted by ec...@apache.org.
Merge branch '1.5.1-SNAPSHOT' of https://git-wip-us.apache.org/repos/asf/accumulo into 1.5.1-SNAPSHOT
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/65aaddd1
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/65aaddd1
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/65aaddd1
Branch: refs/heads/1.5.1-SNAPSHOT
Commit: 65aaddd1703f170bacec3b3f35f4d3ff59ff2c7d
Parents: a18e469 f43ba3b
Author: Eric Newton <er...@gmail.com>
Authored: Mon Jan 6 18:34:05 2014 -0500
Committer: Eric Newton <er...@gmail.com>
Committed: Mon Jan 6 18:34:05 2014 -0500
----------------------------------------------------------------------
.../core/conf/AccumuloConfiguration.java | 35 +++++++++-----
.../core/conf/AccumuloConfigurationTest.java | 48 ++++++++++++++++++++
2 files changed, 72 insertions(+), 11 deletions(-)
----------------------------------------------------------------------
[7/8] git commit: Merge branch '1.6.0-SNAPSHOT' of
https://git-wip-us.apache.org/repos/asf/accumulo into 1.6.0-SNAPSHOT
Posted by ec...@apache.org.
Merge branch '1.6.0-SNAPSHOT' of https://git-wip-us.apache.org/repos/asf/accumulo into 1.6.0-SNAPSHOT
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/4a873834
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/4a873834
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/4a873834
Branch: refs/heads/1.6.0-SNAPSHOT
Commit: 4a873834945d285b73205442181844a3adc568a8
Parents: 67496e0 d1f3dfe
Author: Eric Newton <er...@gmail.com>
Authored: Mon Jan 6 18:33:54 2014 -0500
Committer: Eric Newton <er...@gmail.com>
Committed: Mon Jan 6 18:33:54 2014 -0500
----------------------------------------------------------------------
.../core/conf/AccumuloConfiguration.java | 35 +++++++++-----
.../core/conf/AccumuloConfigurationTest.java | 48 ++++++++++++++++++++
2 files changed, 72 insertions(+), 11 deletions(-)
----------------------------------------------------------------------
[3/8] git commit: ACCUMULO-2141 updating docs to 1.5 option parsing
Posted by ec...@apache.org.
ACCUMULO-2141 updating docs to 1.5 option parsing
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/4c9a6662
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/4c9a6662
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/4c9a6662
Branch: refs/heads/1.5.1-SNAPSHOT
Commit: 4c9a6662b22a1fa7bd4b3559f3e07802cb611233
Parents: f624d40
Author: Eric Newton <er...@gmail.com>
Authored: Mon Jan 6 18:20:37 2014 -0500
Committer: Eric Newton <er...@gmail.com>
Committed: Mon Jan 6 18:20:37 2014 -0500
----------------------------------------------------------------------
docs/examples/README.dirlist | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/accumulo/blob/4c9a6662/docs/examples/README.dirlist
----------------------------------------------------------------------
diff --git a/docs/examples/README.dirlist b/docs/examples/README.dirlist
index 60d3233..b60650f 100644
--- a/docs/examples/README.dirlist
+++ b/docs/examples/README.dirlist
@@ -62,7 +62,7 @@ To count the number of direct children (directories and files) and descendants (
The results are written back to the same table. FileCount reads from and writes to Accumulo. This requires scan authorizations for the read and a visibility for the data written.
In this example, the authorizations and visibility are set to the same value, exampleVis. See README.visibility for more information on visibility and authorizations.
- $ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.FileCount instance zookeepers username password dirTable exampleVis exampleVis
+ $ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.FileCount -i instance -z zookeepers -u username -p password -t dirTable --auths exampleVis
## Directory Table
[5/8] git commit: ACCUMULO-2141 found a typo
Posted by ec...@apache.org.
ACCUMULO-2141 found a typo
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/a18e469e
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/a18e469e
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/a18e469e
Branch: refs/heads/1.5.1-SNAPSHOT
Commit: a18e469e922f937716aa0d69e0570219614a2220
Parents: 4c9a666
Author: Eric Newton <er...@gmail.com>
Authored: Mon Jan 6 18:33:35 2014 -0500
Committer: Eric Newton <er...@gmail.com>
Committed: Mon Jan 6 18:33:35 2014 -0500
----------------------------------------------------------------------
docs/examples/README.dirlist | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/accumulo/blob/a18e469e/docs/examples/README.dirlist
----------------------------------------------------------------------
diff --git a/docs/examples/README.dirlist b/docs/examples/README.dirlist
index b60650f..e505cf9 100644
--- a/docs/examples/README.dirlist
+++ b/docs/examples/README.dirlist
@@ -56,7 +56,7 @@ To perform searches on file or directory names, also use QueryUtil.java. Search
$ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.QueryUtil -i instance -z zookeepers -u username -p password -t indexTable --auths exampleVis --path filename --search
$ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.QueryUtil -i instance -z zookeepers -u username -p password -t indexTable --auths exampleVis --path 'filename*' --search
$ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.QueryUtil -i instance -z zookeepers -u username -p password -t indexTable --auths exampleVis --path '*jar' --search
- $ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.QueryUtil -i instance -z zookeepers -u username -p password -t indexTable --auths exampleVis iipath 'filename*jar' --search
+ $ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.QueryUtil -i instance -z zookeepers -u username -p password -t indexTable --auths exampleVis --path 'filename*jar' --search
To count the number of direct children (directories and files) and descendants (children and children's descendants, directories and files), run the FileCount over the dirTable table.
The results are written back to the same table. FileCount reads from and writes to Accumulo. This requires scan authorizations for the read and a visibility for the data written.
[6/8] git commit: Merge branch '1.5.1-SNAPSHOT' into 1.6.0-SNAPSHOT
Posted by ec...@apache.org.
Merge branch '1.5.1-SNAPSHOT' into 1.6.0-SNAPSHOT
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/67496e0c
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/67496e0c
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/67496e0c
Branch: refs/heads/1.6.0-SNAPSHOT
Commit: 67496e0c47d0561ac03b82e4a6d9efa95ef149d4
Parents: 2781349 a18e469
Author: Eric Newton <er...@gmail.com>
Authored: Mon Jan 6 18:33:45 2014 -0500
Committer: Eric Newton <er...@gmail.com>
Committed: Mon Jan 6 18:33:45 2014 -0500
----------------------------------------------------------------------
server/monitor/src/main/resources/docs/examples/README.dirlist | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/accumulo/blob/67496e0c/server/monitor/src/main/resources/docs/examples/README.dirlist
----------------------------------------------------------------------
diff --cc server/monitor/src/main/resources/docs/examples/README.dirlist
index b60650f,0000000..e505cf9
mode 100644,000000..100644
--- a/server/monitor/src/main/resources/docs/examples/README.dirlist
+++ b/server/monitor/src/main/resources/docs/examples/README.dirlist
@@@ -1,114 -1,0 +1,114 @@@
+Title: Apache Accumulo File System Archive
+Notice: Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+ .
+ http://www.apache.org/licenses/LICENSE-2.0
+ .
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+This example stores filesystem information in accumulo. The example stores the information in the following three tables. More information about the table structures can be found at the end of README.dirlist.
+
+ * directory table : This table stores information about the filesystem directory structure.
+ * index table : This table stores a file name index. It can be used to quickly find files with given name, suffix, or prefix.
+ * data table : This table stores the file data. File with duplicate data are only stored once.
+
+This example shows how to use Accumulo to store a file system history. It has the following classes:
+
+ * Ingest.java - Recursively lists the files and directories under a given path, ingests their names and file info into one Accumulo table, indexes the file names in a separate table, and the file data into a third table.
+ * QueryUtil.java - Provides utility methods for getting the info for a file, listing the contents of a directory, and performing single wild card searches on file or directory names.
+ * Viewer.java - Provides a GUI for browsing the file system information stored in Accumulo.
+ * FileCount.java - Computes recursive counts over file system information and stores them back into the same Accumulo table.
+
+To begin, ingest some data with Ingest.java.
+
+ $ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.Ingest -i instance -z zookeepers -u username -p password --vis exampleVis --chunkSize 100000 /local/username/workspace
+
+This may take some time if there are large files in the /local/username/workspace directory. If you use 0 instead of 100000 on the command line, the ingest will run much faster, but it will not put any file data into Accumulo (the dataTable will be empty).
+Note that running this example will create tables dirTable, indexTable, and dataTable in Accumulo that you should delete when you have completed the example.
+If you modify a file or add new files in the directory ingested (e.g. /local/username/workspace), you can run Ingest again to add new information into the Accumulo tables.
+
+To browse the data ingested, use Viewer.java. Be sure to give the "username" user the authorizations to see the data (in this case, run
+
+ $ ./bin/accumulo shell -u root -e 'setauths -u username -s exampleVis'
+
+then run the Viewer:
+
+ $ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.Viewer -i instance -z zookeepers -u username -p password -t dirTable --dataTable dataTable --auths exampleVis --path /local/username/workspace
+
+To list the contents of specific directories, use QueryUtil.java.
+
+ $ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.QueryUtil -i instance -z zookeepers -u username -p password -t dirTable --auths exampleVis --path /local/username
+ $ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.QueryUtil -i instance -z zookeepers -u username -p password -t dirTable --auths exampleVis --path /local/username/workspace
+
+To perform searches on file or directory names, also use QueryUtil.java. Search terms must contain no more than one wild card and cannot contain "/".
+*Note* these queries run on the _indexTable_ table instead of the dirTable table.
+
+ $ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.QueryUtil -i instance -z zookeepers -u username -p password -t indexTable --auths exampleVis --path filename --search
+ $ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.QueryUtil -i instance -z zookeepers -u username -p password -t indexTable --auths exampleVis --path 'filename*' --search
+ $ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.QueryUtil -i instance -z zookeepers -u username -p password -t indexTable --auths exampleVis --path '*jar' --search
- $ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.QueryUtil -i instance -z zookeepers -u username -p password -t indexTable --auths exampleVis iipath 'filename*jar' --search
++ $ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.QueryUtil -i instance -z zookeepers -u username -p password -t indexTable --auths exampleVis --path 'filename*jar' --search
+
+To count the number of direct children (directories and files) and descendants (children and children's descendants, directories and files), run the FileCount over the dirTable table.
+The results are written back to the same table. FileCount reads from and writes to Accumulo. This requires scan authorizations for the read and a visibility for the data written.
+In this example, the authorizations and visibility are set to the same value, exampleVis. See README.visibility for more information on visibility and authorizations.
+
+ $ ./bin/accumulo org.apache.accumulo.examples.simple.dirlist.FileCount -i instance -z zookeepers -u username -p password -t dirTable --auths exampleVis
+
+## Directory Table
+
+Here is a illustration of what data looks like in the directory table:
+
+ row colf:colq [vis] value
+ 000 dir:exec [exampleVis] true
+ 000 dir:hidden [exampleVis] false
+ 000 dir:lastmod [exampleVis] 1291996886000
+ 000 dir:length [exampleVis] 1666
+ 001/local dir:exec [exampleVis] true
+ 001/local dir:hidden [exampleVis] false
+ 001/local dir:lastmod [exampleVis] 1304945270000
+ 001/local dir:length [exampleVis] 272
+ 002/local/Accumulo.README \x7F\xFF\xFE\xCFH\xA1\x82\x97:exec [exampleVis] false
+ 002/local/Accumulo.README \x7F\xFF\xFE\xCFH\xA1\x82\x97:hidden [exampleVis] false
+ 002/local/Accumulo.README \x7F\xFF\xFE\xCFH\xA1\x82\x97:lastmod [exampleVis] 1308746481000
+ 002/local/Accumulo.README \x7F\xFF\xFE\xCFH\xA1\x82\x97:length [exampleVis] 9192
+ 002/local/Accumulo.README \x7F\xFF\xFE\xCFH\xA1\x82\x97:md5 [exampleVis] 274af6419a3c4c4a259260ac7017cbf1
+
+The rows are of the form depth + path, where depth is the number of slashes ("/") in the path padded to 3 digits. This is so that all the children of a directory appear as consecutive keys in Accumulo; without the depth, you would for example see all the subdirectories of /local before you saw /usr.
+For directories the column family is "dir". For files the column family is Long.MAX_VALUE - lastModified in bytes rather than string format so that newer versions sort earlier.
+
+## Index Table
+
+Here is an illustration of what data looks like in the index table:
+
+ row colf:colq [vis]
+ fAccumulo.README i:002/local/Accumulo.README [exampleVis]
+ flocal i:001/local [exampleVis]
+ rEMDAER.olumuccA i:002/local/Accumulo.README [exampleVis]
+ rlacol i:001/local [exampleVis]
+
+The values of the index table are null. The rows are of the form "f" + filename or "r" + reverse file name. This is to enable searches with wildcards at the beginning, middle, or end.
+
+## Data Table
+
+Here is an illustration of what data looks like in the data table:
+
+ row colf:colq [vis] value
+ 274af6419a3c4c4a259260ac7017cbf1 refs:e77276a2b56e5c15b540eaae32b12c69\x00filext [exampleVis] README
+ 274af6419a3c4c4a259260ac7017cbf1 refs:e77276a2b56e5c15b540eaae32b12c69\x00name [exampleVis] /local/Accumulo.README
+ 274af6419a3c4c4a259260ac7017cbf1 ~chunk:\x00\x0FB@\x00\x00\x00\x00 [exampleVis] *******************************************************************************\x0A1. Building\x0A\x0AIn the normal tarball or RPM release of accumulo, [truncated]
+ 274af6419a3c4c4a259260ac7017cbf1 ~chunk:\x00\x0FB@\x00\x00\x00\x01 [exampleVis]
+
+The rows are the md5 hash of the file. Some column family : column qualifier pairs are "refs" : hash of file name + null byte + property name, in which case the value is property value. There can be multiple references to the same file which are distinguished by the hash of the file name.
+Other column family : column qualifier pairs are "~chunk" : chunk size in bytes + chunk number in bytes, in which case the value is the bytes for that chunk of the file. There is an end of file data marker whose chunk number is the number of chunks for the file and whose value is empty.
+
+There may exist multiple copies of the same file (with the same md5 hash) with different chunk sizes or different visibilities. There is an iterator that can be set on the data table that combines these copies into a single copy with a visibility taken from the visibilities of the file references, e.g. (vis from ref1)|(vis from ref2).