You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by "jmark99 (via GitHub)" <gi...@apache.org> on 2023/05/05 13:31:51 UTC
[GitHub] [accumulo] jmark99 opened a new pull request, #3379: Allow setting of initial hosting goal with NewTableConfiguration
jmark99 opened a new pull request, #3379:
URL: https://github.com/apache/accumulo/pull/3379
Modified NewTableConfiguration to support users setting initial hosting goals for tables at time of table creation.
Updates allow for setting goals via API and via the Accumulo shell.
CreateTable fate operation was updated to set initial goal upon creation of tables. If splits are set at table creation, each tablet will have the appropriate hosting goal set for the tablet.
Unit and IT tests were created to verify the changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] jmark99 commented on a diff in pull request #3379: Allow setting of initial hosting goal with NewTableConfiguration
Posted by "jmark99 (via GitHub)" <gi...@apache.org>.
jmark99 commented on code in PR #3379:
URL: https://github.com/apache/accumulo/pull/3379#discussion_r1186204839
##########
test/src/main/java/org/apache/accumulo/test/NewTableConfigurationIT.java:
##########
@@ -93,6 +100,85 @@ public void testSetPropertiesOverwriteOlderProperties()
}
}
+ @Test
+ public void testCreateTableWithInitialHostingGoal() throws AccumuloException,
+ AccumuloSecurityException, TableNotFoundException, TableExistsException {
+ try (AccumuloClient client = Accumulo.newClient().from(getClientProps()).build()) {
+
+ String[] tableNames = getUniqueNames(8);
+
+ // use a default NewTableConfiguration
+ verifyNtcWithGoal(client, tableNames[0], null, null);
+ // set initial goals for tables upon creation, without splits
+ verifyNtcWithGoal(client, tableNames[1], TabletHostingGoal.ONDEMAND, null);
+ verifyNtcWithGoal(client, tableNames[2], TabletHostingGoal.ALWAYS, null);
+ verifyNtcWithGoal(client, tableNames[3], TabletHostingGoal.NEVER, null);
+
+ SortedSet<Text> splits = new TreeSet<>();
+ splits.add(new Text("d"));
+ splits.add(new Text("h"));
+ splits.add(new Text("m"));
+ splits.add(new Text("r"));
+ splits.add(new Text("w"));
+
+ // Use NTC to set initial splits. Verify each tablet has hosting goal set.
+ // Should work with no goal explicitly supplied as well as each of the accepted goals
+ verifyNtcWithGoal(client, tableNames[4], null, splits);
+ verifyNtcWithGoal(client, tableNames[5], TabletHostingGoal.ONDEMAND, splits);
+ verifyNtcWithGoal(client, tableNames[6], TabletHostingGoal.ALWAYS, splits);
+ verifyNtcWithGoal(client, tableNames[7], TabletHostingGoal.NEVER, splits);
+ }
+ }
+
+ // Verify that NewTableConfiguration correctly sets the initial hosting goal on a table, both with
+ // and without initial splits being set.
+ private void verifyNtcWithGoal(AccumuloClient client, String tableName, TabletHostingGoal goal,
+ SortedSet<Text> splits) throws TableNotFoundException, AccumuloException,
+ AccumuloSecurityException, TableExistsException {
+
+ NewTableConfiguration ntc = new NewTableConfiguration();
+
+ // If goal not supplied via NewTableConfiguration, expect ONDEMAND as default
+ String expectedGoal = TabletHostingGoal.ONDEMAND.toString();
+ if (goal != null) {
+ expectedGoal = goal.toString();
+ ntc.withInitialHostingGoal(goal);
+ }
+
+ // Set expected number of tablets if no splits are provided
+ int expectedTabletCount = 1;
+ if (splits != null) {
+ expectedTabletCount = splits.size() + 1;
+ ntc.withSplits(splits);
+ }
+ client.tableOperations().create(tableName, ntc);
+
+ String tableId = getTableId(tableName);
+ Text beginRow;
+ Text endRow = new Text(tableId + "<");
+ if (expectedTabletCount == 1) {
+ beginRow = endRow;
+ } else {
+ beginRow = new Text(tableId + ";" + splits.first());
+ }
+ Range range = new Range(beginRow, endRow);
+ try (Scanner scanner = client.createScanner("accumulo.metadata")) {
+ scanner.fetchColumn(new Text("hosting"), new Text("goal"));
Review Comment:
Thanks @dlmarion. I made the change and added the required import in `359811a0cd6c9`. Re-tested to verify nothing broke.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] dlmarion commented on a diff in pull request #3379: Allow setting of initial hosting goal with NewTableConfiguration
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on code in PR #3379:
URL: https://github.com/apache/accumulo/pull/3379#discussion_r1186139049
##########
test/src/main/java/org/apache/accumulo/test/NewTableConfigurationIT.java:
##########
@@ -93,6 +100,85 @@ public void testSetPropertiesOverwriteOlderProperties()
}
}
+ @Test
+ public void testCreateTableWithInitialHostingGoal() throws AccumuloException,
+ AccumuloSecurityException, TableNotFoundException, TableExistsException {
+ try (AccumuloClient client = Accumulo.newClient().from(getClientProps()).build()) {
+
+ String[] tableNames = getUniqueNames(8);
+
+ // use a default NewTableConfiguration
+ verifyNtcWithGoal(client, tableNames[0], null, null);
+ // set initial goals for tables upon creation, without splits
+ verifyNtcWithGoal(client, tableNames[1], TabletHostingGoal.ONDEMAND, null);
+ verifyNtcWithGoal(client, tableNames[2], TabletHostingGoal.ALWAYS, null);
+ verifyNtcWithGoal(client, tableNames[3], TabletHostingGoal.NEVER, null);
+
+ SortedSet<Text> splits = new TreeSet<>();
+ splits.add(new Text("d"));
+ splits.add(new Text("h"));
+ splits.add(new Text("m"));
+ splits.add(new Text("r"));
+ splits.add(new Text("w"));
+
+ // Use NTC to set initial splits. Verify each tablet has hosting goal set.
+ // Should work with no goal explicitly supplied as well as each of the accepted goals
+ verifyNtcWithGoal(client, tableNames[4], null, splits);
+ verifyNtcWithGoal(client, tableNames[5], TabletHostingGoal.ONDEMAND, splits);
+ verifyNtcWithGoal(client, tableNames[6], TabletHostingGoal.ALWAYS, splits);
+ verifyNtcWithGoal(client, tableNames[7], TabletHostingGoal.NEVER, splits);
+ }
+ }
+
+ // Verify that NewTableConfiguration correctly sets the initial hosting goal on a table, both with
+ // and without initial splits being set.
+ private void verifyNtcWithGoal(AccumuloClient client, String tableName, TabletHostingGoal goal,
+ SortedSet<Text> splits) throws TableNotFoundException, AccumuloException,
+ AccumuloSecurityException, TableExistsException {
+
+ NewTableConfiguration ntc = new NewTableConfiguration();
+
+ // If goal not supplied via NewTableConfiguration, expect ONDEMAND as default
+ String expectedGoal = TabletHostingGoal.ONDEMAND.toString();
+ if (goal != null) {
+ expectedGoal = goal.toString();
+ ntc.withInitialHostingGoal(goal);
+ }
+
+ // Set expected number of tablets if no splits are provided
+ int expectedTabletCount = 1;
+ if (splits != null) {
+ expectedTabletCount = splits.size() + 1;
+ ntc.withSplits(splits);
+ }
+ client.tableOperations().create(tableName, ntc);
+
+ String tableId = getTableId(tableName);
+ Text beginRow;
+ Text endRow = new Text(tableId + "<");
+ if (expectedTabletCount == 1) {
+ beginRow = endRow;
+ } else {
+ beginRow = new Text(tableId + ";" + splits.first());
+ }
+ Range range = new Range(beginRow, endRow);
+ try (Scanner scanner = client.createScanner("accumulo.metadata")) {
+ scanner.fetchColumn(new Text("hosting"), new Text("goal"));
Review Comment:
```suggestion
HostingColumnFamily.GOAL_COLUMN.fetch(scanner);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] jmark99 merged pull request #3379: Allow setting of initial hosting goal with NewTableConfiguration
Posted by "jmark99 (via GitHub)" <gi...@apache.org>.
jmark99 merged PR #3379:
URL: https://github.com/apache/accumulo/pull/3379
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org