You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@accumulo.apache.org by mm...@apache.org on 2019/10/31 17:15:27 UTC
[accumulo-website] branch master updated: Update Tour for 2.0.
Fixes #153 (#201)
This is an automated email from the ASF dual-hosted git repository.
mmiller pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/accumulo-website.git
The following commit(s) were added to refs/heads/master by this push:
new b49d061 Update Tour for 2.0. Fixes #153 (#201)
b49d061 is described below
commit b49d0617e68e0773ef9b05a0c9f97bdbc3ba9c9d
Author: Mike Miller <mm...@apache.org>
AuthorDate: Thu Oct 31 13:15:18 2019 -0400
Update Tour for 2.0. Fixes #153 (#201)
* Update modules to use client instead of connector
* Add a module for using client
* Use new createBatchWriter method
Co-Authored-By: Keith Turner <kt...@apache.org>
---
_data/tour.yml | 1 +
tour/authorizations-code.md | 41 +++++++++++++++++++++--------------------
tour/authorizations.md | 19 +++++++++++++------
tour/basic-read-write.md | 17 ++++++-----------
tour/batch-scanner-code.md | 11 +++++------
tour/batch-scanner.md | 9 ++++-----
tour/client.md | 38 ++++++++++++++++++++++++++++++++++++++
tour/conditional-writer-code.md | 4 ++--
tour/conditional-writer.md | 29 ++++++++++++++---------------
tour/data-model-code.md | 11 +++++------
tour/getting-started.md | 11 ++++++-----
tour/index.md | 8 ++------
12 files changed, 117 insertions(+), 82 deletions(-)
diff --git a/_data/tour.yml b/_data/tour.yml
index 9bd0287..8f4d81f 100644
--- a/_data/tour.yml
+++ b/_data/tour.yml
@@ -1,5 +1,6 @@
docs:
- getting-started
+ - client
- basic-read-write
- data-model
- data-model-code
diff --git a/tour/authorizations-code.md b/tour/authorizations-code.md
index e14710b..45893ac 100644
--- a/tour/authorizations-code.md
+++ b/tour/authorizations-code.md
@@ -5,10 +5,9 @@ title: Authorizations Code
Below is a solution for the exercise.
```java
-static void exercise(MiniAccumuloCluster mac) throws Exception {
- // Connect to Mini Accumulo as the root user and create a table called "GothamPD".
- Connector conn = mac.getConnector("root", "tourguide");
- conn.tableOperations().create("GothamPD");
+static void exercise(AccumuloClient client) throws Exception {
+ // create a table called "GothamPD".
+ client.tableOperations().create("GothamPD");
// Create a "secretId" authorization & visibility
final String secretId = "secretId";
@@ -16,35 +15,37 @@ static void exercise(MiniAccumuloCluster mac) throws Exception {
ColumnVisibility colVis = new ColumnVisibility(secretId);
// Create a user with the "secretId" authorization and grant him read permissions on our table
- conn.securityOperations().createLocalUser("commissioner", new PasswordToken("gordonrocks"));
- conn.securityOperations().changeUserAuthorizations("commissioner", auths);
- conn.securityOperations().grantTablePermission("commissioner", "GothamPD", TablePermission.READ);
+ client.securityOperations().createLocalUser("commissioner", new PasswordToken("gordonrocks"));
+ client.securityOperations().changeUserAuthorizations("commissioner", auths);
+ client.securityOperations().grantTablePermission("commissioner", "GothamPD", TablePermission.READ);
// Create 3 Mutation objects, securing the proper columns.
Mutation mutation1 = new Mutation("id0001");
- mutation1.put("hero","alias", "Batman");
- mutation1.put("hero","name", colVis, "Bruce Wayne");
- mutation1.put("hero","wearsCape?", "true");
+ mutation1.put("hero", "alias", "Batman");
+ mutation1.put("hero", "name", colVis, "Bruce Wayne");
+ mutation1.put("hero", "wearsCape?", "true");
Mutation mutation2 = new Mutation("id0002");
- mutation2.put("hero","alias", "Robin");
- mutation2.put("hero","name", colVis,"Dick Grayson");
- mutation2.put("hero","wearsCape?", "true");
+ mutation2.put("hero", "alias", "Robin");
+ mutation2.put("hero", "name", colVis, "Dick Grayson");
+ mutation2.put("hero", "wearsCape?", "true");
Mutation mutation3 = new Mutation("id0003");
- mutation3.put("villain","alias", "Joker");
- mutation3.put("villain","name", "Unknown");
- mutation3.put("villain","wearsCape?", "false");
+ mutation3.put("villain", "alias", "Joker");
+ mutation3.put("villain", "name", "Unknown");
+ mutation3.put("villain", "wearsCape?", "false");
// Create a BatchWriter to the GothamPD table and add your mutations to it.
// Once the BatchWriter is closed by the try w/ resources, data will be available to scans.
- try (BatchWriter writer = conn.createBatchWriter("GothamPD", new BatchWriterConfig())) {
+ try (BatchWriter writer = client.createBatchWriter("GothamPD")) {
writer.addMutation(mutation1);
writer.addMutation(mutation2);
writer.addMutation(mutation3);
}
- // Read and print all rows of the commissioner can see. Pass Scanner proper authorizations
- Connector commishConn = mac.getConnector("commissioner", "gordonrocks");
- try (Scanner scan = commishConn.createScanner("GothamPD", auths)) {
+ // Create a second client for the commissioner. Then print all the rows visibile to the
+ // commissioner. Make sure to pass the proper authorizations to the Scanner
+ try (AccumuloClient commishClient = Accumulo.newClient().from(client.properties())
+ .as("commissioner", "gordonrocks").build();
+ Scanner scan = commishClient.createScanner("GothamPD", auths)) {
System.out.println("Gotham Police Department Persons of Interest:");
for (Map.Entry<Key, Value> entry : scan) {
System.out.printf("Key : %-60s Value : %s\n", entry.getKey(), entry.getValue());
diff --git a/tour/authorizations.md b/tour/authorizations.md
index 4c18c8f..dd25d2f 100644
--- a/tour/authorizations.md
+++ b/tour/authorizations.md
@@ -16,7 +16,7 @@ For example:
We now want to secure our secret identities of the heroes so that only users with the proper authorizations can read their names.
-1. Using the code from the previous exercise, add the following to the beginning of the _exercise_ method (after we get the Connector).
+1. Using the code from the previous exercise, add the following to the beginning of the _exercise_ method.
```java
// Create a "secretId" authorization & visibility
final String secretId = "secretId";
@@ -24,9 +24,9 @@ We now want to secure our secret identities of the heroes so that only users wit
ColumnVisibility colVis = new ColumnVisibility(secretId);
// Create a user with the "secretId" authorization and grant him read permissions on our table
- conn.securityOperations().createLocalUser("commissioner", new PasswordToken("gordonrocks"));
- conn.securityOperations().changeUserAuthorizations("commissioner", auths);
- conn.securityOperations().grantTablePermission("commissioner", "GothamPD", TablePermission.READ);
+ client.securityOperations().createLocalUser("commissioner", new PasswordToken("gordonrocks"));
+ client.securityOperations().changeUserAuthorizations("commissioner", auths);
+ client.securityOperations().grantTablePermission("commissioner", "GothamPD", TablePermission.READ);
```
2. The [Mutation] API allows you to set the `secretId` visibility on a column. Find the proper method for setting a column visibility in
@@ -38,9 +38,15 @@ the Mutation API and modify the code so the `colVis` variable created above secu
* Replace the `Authorizations.EMPTY` in the Scanner with the `auths` variable created above and run it again.
* This should result in an error since the root user doesn't have the authorizations we tried to pass to the Scanner.
-4. Get a connector for the "commissioner" and from it create a Scanner with the authorizations needed to view the secret identities.
+4. Use the following to create a client for the "commissioner" using the [Accumulo] entry point.
+```java
+try (AccumuloClient commishClient = Accumulo.newClient().from(client.properties())
+ .as("commissioner", "gordonrocks").build();
+```
+
+5. Using the commissioner client, create a Scanner with the authorizations needed to view the secret identities.
-5. Build and run. You should see all the rows in the GothamPD table printed, including these secured key/value pairs:
+6. Build and run. You should see all the rows in the GothamPD table printed, including these secured key/value pairs:
```commandline
Key : id0001 hero:name [secretId] 1511900180231 false Value : Bruce Wayne
Key : id0002 hero:name [secretId] 1511900180231 false Value : Dick Grayson
@@ -49,3 +55,4 @@ Key : id0002 hero:name [secretId] 1511900180231 false Value : Dick Grays
[Authorizations]: {% jurl org.apache.accumulo.core.security.Authorizations %}
[ColumnVisibility]: {% jurl org.apache.accumulo.core.security.ColumnVisibility %}
[Mutation]: {% jurl org.apache.accumulo.core.data.Mutation %}
+[Accumulo]: {% jurl org.apache.accumulo.core.client.Accumulo %}
diff --git a/tour/basic-read-write.md b/tour/basic-read-write.md
index b6446f4..e7e4a8f 100644
--- a/tour/basic-read-write.md
+++ b/tour/basic-read-write.md
@@ -6,10 +6,9 @@ data in tables and rows. Each row in an Accumulo table can hold many key/value
write and read from a table.
```java
-static void exercise(MiniAccumuloCluster mac) {
- // Connect to Mini Accumulo as the root user and create a table called "GothamPD".
- Connector conn = mac.getConnector("root", "tourguide");
- conn.tableOperations().create("GothamPD");
+static void exercise(AccumuloClient client) throws Exception {
+ // create a table called "GothamPD".
+ client.tableOperations().create("GothamPD");
// Create a Mutation object to hold all changes to a row in a table. Each row has a unique row ID.
Mutation mutation = new Mutation("id0001");
@@ -20,12 +19,12 @@ static void exercise(MiniAccumuloCluster mac) {
mutation.put("hero","wearsCape?", "true");
// Create a BatchWriter to the GothamPD table and add your mutation to it. Try w/ resources will close for us.
- try (BatchWriter writer = conn.createBatchWriter("GothamPD", new BatchWriterConfig())) {
+ try (BatchWriter writer = client.createBatchWriter("GothamPD")) {
writer.addMutation(mutation);
}
// Read and print all rows of the "GothamPD" table. Try w/ resources will close for us.
- try (Scanner scan = conn.createScanner("GothamPD", Authorizations.EMPTY)) {
+ try (Scanner scan = client.createScanner("GothamPD", Authorizations.EMPTY)) {
System.out.println("Gotham Police Department Persons of Interest:");
// A Scanner is an extension of java.lang.Iterable so behaves just like one.
for (Map.Entry<Key, Value> entry : scan) {
@@ -35,11 +34,7 @@ static void exercise(MiniAccumuloCluster mac) {
}
```
-Copy this code into your `exercise` method and run it using the command below.
-
-```commandline
-mvn -q clean compile exec:java
-```
+Copy this code into your `exercise` method then compile and run.
Good job! That is all it takes to write and read from Accumulo.
diff --git a/tour/batch-scanner-code.md b/tour/batch-scanner-code.md
index bc7722c..8e4de7a 100644
--- a/tour/batch-scanner-code.md
+++ b/tour/batch-scanner-code.md
@@ -5,13 +5,12 @@ title: Batch Scanner Code
Below is a solution to the exercise.
```java
-static void exercise(MiniAccumuloCluster mac) throws Exception {
- // Connect to Mini Accumulo as the root user and create a table called "GothamPD".
- Connector conn = mac.getConnector("root", "tourguide");
- conn.tableOperations().create("GothamPD");
+static void exercise(AccumuloClient client) throws Exception {
+ // create a table called "GothamPD".
+ client.tableOperations().create("GothamPD");
// Generate 10,000 rows of henchman data
- try(BatchWriter writer = conn.createBatchWriter("GothamPD", new BatchWriterConfig())) {
+ try(BatchWriter writer = client.createBatchWriter("GothamPD")) {
for(int i = 0; i < 10_000; i++) {
Mutation m = new Mutation(String.format("id%04d", i));
m.put("villain", "alias", "henchman" + i);
@@ -22,7 +21,7 @@ static void exercise(MiniAccumuloCluster mac) throws Exception {
}
// 1. Create a BatchScanner with 5 query threads
- try(BatchScanner batchScanner = conn.createBatchScanner("GothamPD", Authorizations.EMPTY, 5)) {
+ try(BatchScanner batchScanner = client.createBatchScanner("GothamPD", Authorizations.EMPTY, 5)) {
// 2. Create a collection of 2 sample ranges and set it to the batchScanner
List ranges = new ArrayList<Range>();
ranges.add(new Range("id1000", "id1999"));
diff --git a/tour/batch-scanner.md b/tour/batch-scanner.md
index 2d237dc..f80433e 100644
--- a/tour/batch-scanner.md
+++ b/tour/batch-scanner.md
@@ -6,13 +6,12 @@ will retrieve multiple Ranges of data using multiple threads. A BatchScanner ca
For this exercise, we need to generate a bunch of data to test BatchScanner. Copy the code below into your `exercise` method.
```java
-static void exercise(MiniAccumuloCluster mac) throws Exception {
- // Connect to Mini Accumulo as the root user and create a table called "GothamPD".
- Connector conn = mac.getConnector("root", "tourguide");
- conn.tableOperations().create("GothamPD");
+static void exercise(AccumuloClient client) throws Exception {
+ // create a table called "GothamPD".
+ client.tableOperations().create("GothamPD");
// Generate 10,000 rows of henchman data, each with a different number yearsOfService
- try (BatchWriter writer = conn.createBatchWriter("GothamPD", new BatchWriterConfig())) {
+ try (BatchWriter writer = client.createBatchWriter("GothamPD")) {
for (int i = 0; i < 10_000; i++) {
Mutation m = new Mutation(String.format("id%04d", i));
m.put("villain", "alias", "henchman" + i);
diff --git a/tour/client.md b/tour/client.md
new file mode 100644
index 0000000..739a818
--- /dev/null
+++ b/tour/client.md
@@ -0,0 +1,38 @@
+---
+title: Connecting to Accumulo
+---
+
+Connecting to a live instance of Accumulo is done through the [AccumuloClient] object. This object contains a live connection
+to Accumulo and will remain open until closed. All client operations can be accessed from this one object.
+
+The [Accumulo] entry point is used to create a client by calling ```Accumulo.newClient()```. The client can
+be created from properties by using one of the ```from()``` methods or using the ```to()``` and ```as()``` methods
+to specify the connection information directly.
+
+For the tour, the client is passed to each exercise method and then closed before stopping the cluster.
+The properties used to create the client can be seen in ```accumulo-client.properties``` under ```target/mac######/conf```
+Notice the client can be wrapped in a Java try-with-resources since it is AutoCloseable.
+
+Start by using table operations to list the default tables and instance operations to get the instance ID.
+```java
+static void exercise(AccumuloClient client) throws Exception {
+ for (String t : client.tableOperations().list())
+ System.out.println("Table: " + t);
+
+ System.out.println("Instance ID: " + client.instanceOperations().getInstanceID());
+}
+```
+
+Different types of operations are accessed by their respective method on the client:
+```java
+client.tableOperations();
+client.namespaceOperations();
+client.securityOperations();
+client.instanceOperations();
+client.replicationOperations();
+```
+
+The client is also used directly to create Scanners and perform batch operations. These will be explored later.
+
+[AccumuloClient]: {% jurl org.apache.accumulo.core.client.AccumuloClient %}
+[Accumulo]: {% jurl org.apache.accumulo.core.client.Accumulo %}
\ No newline at end of file
diff --git a/tour/conditional-writer-code.md b/tour/conditional-writer-code.md
index e825976..10f8469 100644
--- a/tour/conditional-writer-code.md
+++ b/tour/conditional-writer-code.md
@@ -5,8 +5,8 @@ title: Conditional Writer Code
Below is a solution to the exercise.
```java
- static boolean setAddress(Connector conn, String id, String expectedAddr, String newAddr) {
- try (ConditionalWriter writer = conn.createConditionalWriter("GothamPD", new ConditionalWriterConfig())) {
+ static boolean setAddress(AccumuloClient client, String id, String expectedAddr, String newAddr) {
+ try (ConditionalWriter writer = client.createConditionalWriter("GothamPD", new ConditionalWriterConfig())) {
Condition condition = new Condition("location", "home");
if(expectedAddr != null) {
condition.setValue(expectedAddr);
diff --git a/tour/conditional-writer.md b/tour/conditional-writer.md
index 618caf0..2dc8aa4 100644
--- a/tour/conditional-writer.md
+++ b/tour/conditional-writer.md
@@ -36,9 +36,9 @@ is the batch writer always makes the update, even when the value has
changed since it was read.
```java
- static String getAddress(Connector conn, String id) {
+ static String getAddress(AccumuloClient client, String id) {
// The IsolatedScanner ensures partial changes to a row are not seen
- try (Scanner scanner = new IsolatedScanner(conn.createScanner("GothamPD", Authorizations.EMPTY))) {
+ try (Scanner scanner = new IsolatedScanner(client.createScanner("GothamPD", Authorizations.EMPTY))) {
scanner.setRange(Range.exact(id, "location", "home"));
for (Entry<Key,Value> entry : scanner) {
return entry.getValue().toString();
@@ -49,8 +49,8 @@ changed since it was read.
}
}
- static boolean setAddress(Connector conn, String id, String expectedAddr, String newAddr) {
- try (BatchWriter writer = conn.createBatchWriter("GothamPD", new BatchWriterConfig())) {
+ static boolean setAddress(AccumuloClient client, String id, String expectedAddr, String newAddr) {
+ try (BatchWriter writer = client.createBatchWriter("GothamPD")) {
Mutation mutation = new Mutation(id);
mutation.put("location", "home", newAddr);
writer.addMutation(mutation);
@@ -60,34 +60,33 @@ changed since it was read.
}
}
- public static Future<Void> modifyAddress(Connector conn, String id, Function<String,String> modifier) {
+ public static Future<Void> modifyAddress(AccumuloClient client, String id, Function<String,String> modifier) {
return CompletableFuture.runAsync(() -> {
String currAddr, newAddr;
do {
- currAddr = getAddress(conn, id);
+ currAddr = getAddress(client, id);
newAddr = modifier.apply(currAddr);
System.out.printf("Thread %3d attempting change %20s -> %-20s\n",
Thread.currentThread().getId(), "'"+currAddr+"'", "'"+newAddr+"'");
- } while (!setAddress(conn, id, currAddr, newAddr));
+ } while (!setAddress(client, id, currAddr, newAddr));
});
}
- static void exercise(MiniAccumuloCluster mac) throws Exception {
- Connector conn = mac.getConnector("root", "tourguide");
- conn.tableOperations().create("GothamPD");
+ static void exercise(AccumuloClient client) throws Exception {
+ client.tableOperations().create("GothamPD");
String id = "id0001";
- setAddress(conn, id, null, " 1007 Mountain Drive, Gotham, New York ");
+ setAddress(client, id, null, " 1007 Mountain Drive, Gotham, New York ");
// create async operation to trim whitespace
- Future<Void> future1 = modifyAddress(conn, id, String::trim);
+ Future<Void> future1 = modifyAddress(client, id, String::trim);
// create async operation to replace Dr with Drive
- Future<Void> future2 = modifyAddress(conn, id, addr -> addr.replace("Drive", "Dr"));
+ Future<Void> future2 = modifyAddress(client, id, addr -> addr.replace("Drive", "Dr"));
// create async operation to replace New York with NY
- Future<Void> future3 = modifyAddress(conn, id, addr -> addr.replace("New York", "NY"));
+ Future<Void> future3 = modifyAddress(client, id, addr -> addr.replace("New York", "NY"));
// wait for async operations to complete
future1.get();
@@ -95,7 +94,7 @@ changed since it was read.
future3.get();
// print the address stored in Accumulo
- System.out.println("Final address : '"+getAddress(conn, id)+"'");
+ System.out.println("Final address : '"+getAddress(client, id)+"'");
}
```
diff --git a/tour/data-model-code.md b/tour/data-model-code.md
index 31c9da5..e76edee 100644
--- a/tour/data-model-code.md
+++ b/tour/data-model-code.md
@@ -5,10 +5,9 @@ title: Data Model Code
Below is the solution for the exercise.
```java
-static void exercise(MiniAccumuloCluster mac) {
- // Connect to Mini Accumulo as the root user and create a table called "GothamPD".
- Connector conn = mac.getConnector("root", "tourguide");
- conn.tableOperations().create("GothamPD");
+static void exercise(AccumuloClient client) throws Exception {
+ // create a table called "GothamPD".
+ client.tableOperations().create("GothamPD");
// Create a row for Batman
Mutation mutation1 = new Mutation("id0001");
@@ -30,14 +29,14 @@ static void exercise(MiniAccumuloCluster mac) {
// Create a BatchWriter to the GothamPD table and add your mutations to it.
// Once the BatchWriter is closed by the try w/ resources, data will be available to scans.
- try (BatchWriter writer = conn.createBatchWriter("GothamPD", new BatchWriterConfig())) {
+ try (BatchWriter writer = client.createBatchWriter("GothamPD")) {
writer.addMutation(mutation1);
writer.addMutation(mutation2);
writer.addMutation(mutation3);
}
// Read and print all rows of the "GothamPD" table. Try w/ resources will close for us.
- try (Scanner scan = conn.createScanner("GothamPD", Authorizations.EMPTY)) {
+ try (Scanner scan = client.createScanner("GothamPD", Authorizations.EMPTY)) {
System.out.println("Gotham Police Department Persons of Interest:");
// A Scanner is an extension of java.lang.Iterable so behaves just like one.
for (Map.Entry<Key, Value> entry : scan) {
diff --git a/tour/getting-started.md b/tour/getting-started.md
index b443cb4..30be62f 100644
--- a/tour/getting-started.md
+++ b/tour/getting-started.md
@@ -13,20 +13,21 @@ cd tour
```commandline
vim ./src/main/java/tour/Main.java
```
-Notice the main method creates a MiniAccumuloCluster with a root password of "tourguide". MiniAccumuloCluster is a mini
+Notice the main method creates a [MiniAccumuloCluster] with a root password of "tourpass". [MiniAccumuloCluster] is a mini
version of Accumulo that runs on your local filesystem. It should only be used for development purposes but will work
-great here on the tour. Files and logs used by MiniAccumuloCluster can be seen in the `target/mac######` directory.
+great here on the tour. Files and logs used by [MiniAccumuloCluster] can be seen in the `target/mac######` directory.
3. Modify the _exercise_ method to print a hello message. You will put your code in this method for each lesson.
```java
- static void exercise(MiniAccumuloCluster mac) {
+ static void exercise(AccumuloClient client) {
// start writing your code here
System.out.println("Hello world");
}
```
-4. Build and run to make sure everything is cool.
+4. Use the following Maven command to build and run the tour.
```commandline
-mvn -q clean compile exec:java
+mvn -q clean compile exec:exec
```
[Main.java]: https://github.com/apache/accumulo-website/blob/tour/src/main/java/tour/Main.java
+[MiniAccumuloCluster]: {% jurl org.apache.accumulo.minicluster.MiniAccumuloCluster %}
diff --git a/tour/index.md b/tour/index.md
index 66e76c2..987dad3 100644
--- a/tour/index.md
+++ b/tour/index.md
@@ -11,17 +11,13 @@ skiph1fortitle: true
Welcome to the Accumulo tour! The tour offers a hands on introduction to the [Accumulo Java API](/api), broken down into
independent steps and exercises. The exercises give you a chance to apply what you have learned by writing code on your
-own. The answers to an exercise are typically provided in the next step. The tour starts with a
-[{{ first_page.title }}]({{ first_url }}) page that will help you get set up.
+own. The answers to an exercise are typically provided in the next step. The tour begins at the
+[{{ first_page.title }}]({{ first_url }}) page.
When on a tour page, the left and right keys on the keyboard can be used to navigate. If you have any questions
or suggestions while going through the tour, please send an email to our [mailing list][mlist]
or [create an issue][issue].
-Tour excercises are run using MiniAccumulo which does not work with Java 11.
-This problem will be fixed when 1.9.3 and 2.0.0 are released. See {% ghi 942
-%}.
-
{% for p in tour_pages %}
{% assign doc_url = p | prepend: '/tour/' | append: '/' %}
{% assign link_to_page = site.pages | where:'url',doc_url | first %}