You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by "ArbaazKhan1 (via GitHub)" <gi...@apache.org> on 2024/07/24 14:17:22 UTC
[PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
ArbaazKhan1 opened a new pull request, #4754:
URL: https://github.com/apache/accumulo/pull/4754
Closes [issue #4705 ](https://github.com/apache/accumulo/issues/4705)
Added in a new test into ComprehensiveIT, that check the behavior of eventual scan. Structure of the test is as follows:
1. Add data to table
2. Run eventual scan
3. Add new data to table
4. Run 2nd eventual scan
5. Should not see new data
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "kevinrr888 (via GitHub)" <gi...@apache.org>.
kevinrr888 commented on code in PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#discussion_r2198650654
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT_SimpleSuite.java:
##########
@@ -19,20 +19,34 @@
package org.apache.accumulo.test;
import static org.apache.accumulo.harness.AccumuloITBase.SUNNY_DAY;
+import static org.apache.accumulo.harness.SharedMiniClusterBase.getClientProps;
+import static org.apache.accumulo.test.ComprehensiveITBase.AUTHORIZATIONS;
import org.apache.accumulo.core.client.Accumulo;
import org.apache.accumulo.core.client.AccumuloClient;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.harness.MiniClusterConfigurationCallback;
import org.apache.accumulo.harness.SharedMiniClusterBase;
+import org.apache.accumulo.miniclusterImpl.MiniAccumuloConfigImpl;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Tag;
@Tag(SUNNY_DAY)
public class ComprehensiveIT_SimpleSuite extends ComprehensiveITBase {
Review Comment:
```suggestion
public class ComprehensiveIT extends ComprehensiveITBase {
```
The setup for this test changed from starting MAC without any custom config to starting with a custom config, so it should no longer be part of the suite.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "ArbaazKhan1 (via GitHub)" <gi...@apache.org>.
ArbaazKhan1 commented on code in PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#discussion_r1743983207
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +151,58 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ Properties props = new Properties();
+ props.putAll(getClientProps());
+ props.put(ClientProperty.SCAN_SERVER_SELECTOR.getKey(),
+ ConfigurableScanServerSelector.class.getName());
+ props.put(ClientProperty.SCAN_SERVER_SELECTOR_OPTS_PREFIX.getKey() + "profiles",
+ "[{'isDefault':true,'timeToWaitForScanServers' : '10d','maxBusyTimeout':'5m','busyTimeoutMultiplier':4,'attemptPlans':[{\"servers\":\"100%\", \"busyTimeout\":\"3ms\"}]}]");
+
+ try (AccumuloClient client = Accumulo.newClient().from(props).build()) {
+
+ String table = getUniqueNames(1)[0];
+ client.tableOperations().create(table);
+
+ // Start Scan Server
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
+
+ // generate 100 rows of data
+ write(client, table, generateMutations(0, 100, tr -> true));
+
+ // need flush table so that eventual scan can see it.
+ getCluster().getClusterControl().stop(ServerType.SCAN_SERVER);
Review Comment:
I took a look at setting `Property.SSERV_CACHED_TABLET_METADATA_EXPIRATION` to a larger value but even if increase the time and remove the start/stop scan server the tests fail. The most consistent way to ensure that data is present on scan servers is by either using `Thread.sleep` for a long enough time or starting/stops servers, and the discussion was that start/stop gives us the most reliable consistency.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "ArbaazKhan1 (via GitHub)" <gi...@apache.org>.
ArbaazKhan1 commented on code in PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#discussion_r1734843144
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +151,58 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ Properties props = new Properties();
+ props.putAll(getClientProps());
+ props.put(ClientProperty.SCAN_SERVER_SELECTOR.getKey(),
+ ConfigurableScanServerSelector.class.getName());
+ props.put(ClientProperty.SCAN_SERVER_SELECTOR_OPTS_PREFIX.getKey() + "profiles",
+ "[{'isDefault':true,'timeToWaitForScanServers' : '10d','maxBusyTimeout':'5m','busyTimeoutMultiplier':4,'attemptPlans':[{\"servers\":\"100%\", \"busyTimeout\":\"3ms\"}]}]");
+
+ try (AccumuloClient client = Accumulo.newClient().from(props).build()) {
+
+ String table = getUniqueNames(1)[0];
+ client.tableOperations().create(table);
+
+ // Start Scan Server
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
+
+ // generate 100 rows of data
+ write(client, table, generateMutations(0, 100, tr -> true));
+
+ // need flush table so that eventual scan can see it.
+ getCluster().getClusterControl().stop(ServerType.SCAN_SERVER);
Review Comment:
As I understand the starting and stopping of the scan server is to make sure that the data is present after a flush.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "kevinrr888 (via GitHub)" <gi...@apache.org>.
kevinrr888 commented on code in PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#discussion_r1691863203
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -119,9 +124,29 @@ public class ComprehensiveIT extends SharedMiniClusterBase {
private static final Logger log = LoggerFactory.getLogger(ComprehensiveIT.class);
+ private static class ComprehensiveITConfiguration implements MiniClusterConfigurationCallback {
+
+ @Override
+ public void configureMiniCluster(MiniAccumuloConfigImpl cfg,
+ org.apache.hadoop.conf.Configuration coreSite) {
+ cfg.setNumScanServers(1);
+
+ // Timeout scan sessions after being idle for 3 seconds
+ cfg.setProperty(Property.TSERV_SESSION_MAXIDLE, "3s");
+
+ // Configure the scan server to only have 1 scan executor thread. This means
+ // that the scan server will run scans serially, not concurrently.
+ cfg.setProperty(Property.SSERV_SCAN_EXECUTORS_DEFAULT_THREADS, "1");
+ }
+ }
+
@BeforeAll
public static void setup() throws Exception {
Review Comment:
Are we sure we want this new config to be used for all ComprehensiveIT tests? If this config is only necessary for testEventualScan(), should probably only be set for that test in case these conflict with future tests
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -119,9 +124,29 @@ public class ComprehensiveIT extends SharedMiniClusterBase {
private static final Logger log = LoggerFactory.getLogger(ComprehensiveIT.class);
+ private static class ComprehensiveITConfiguration implements MiniClusterConfigurationCallback {
+
+ @Override
+ public void configureMiniCluster(MiniAccumuloConfigImpl cfg,
+ org.apache.hadoop.conf.Configuration coreSite) {
+ cfg.setNumScanServers(1);
+
+ // Timeout scan sessions after being idle for 3 seconds
+ cfg.setProperty(Property.TSERV_SESSION_MAXIDLE, "3s");
+
+ // Configure the scan server to only have 1 scan executor thread. This means
+ // that the scan server will run scans serially, not concurrently.
+ cfg.setProperty(Property.SSERV_SCAN_EXECUTORS_DEFAULT_THREADS, "1");
+ }
+ }
+
@BeforeAll
public static void setup() throws Exception {
- SharedMiniClusterBase.startMiniCluster();
+ // SharedMiniClusterBase.startMiniCluster();
+ ComprehensiveITConfiguration c = new ComprehensiveITConfiguration();
+ SharedMiniClusterBase.startMiniClusterWithConfig(c);
+ SharedMiniClusterBase.getCluster().getClusterControl().start(ServerType.SCAN_SERVER,
+ "localhost");
Review Comment:
MiniAccumuloClusterControl doesn't use the 'hostname' param, so "localhost" can be dropped. Think the idea is to get rid of this hostname param (https://github.com/apache/accumulo/issues/4055), so not including it here would make this change easier in the future
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#discussion_r1717276846
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +152,49 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ try (AccumuloClient client = Accumulo.newClient().from(getClientProps()).build()) {
+
+ String table = getUniqueNames(1)[0];
+ client.tableOperations().create(table);
+
+ // Start Scan Server
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
+
+ // generate 100 rows of data
+ write(client, table, generateMutations(0, 100, tr -> true));
+
+ // need flush table so that eventual scan can see it.
+ client.tableOperations().flush(table, null, null, true);
+ Thread.sleep(5000);
+
+ // should see all data that was flushed in eventual scan
+ verifyData(client, table, AUTHORIZATIONS, generateKeys(0, 100),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ // should not see data with col vis set
+ verifyData(client, table, Authorizations.EMPTY, generateKeys(0, 100, tr -> tr.vis.isEmpty()),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ // write some more rows after 100 and verfy those are not seen by eventual scan until table is
+ // flushed.
+ write(client, table, generateMutations(100, 200, tr -> true));
+ verifyData(client, table, AUTHORIZATIONS, generateKeys(0, 100),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ // need flush table so that eventual scan can see it.
+ client.tableOperations().flush(table, null, null, true);
Review Comment:
Same wait remaining block could be used here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "ArbaazKhan1 (via GitHub)" <gi...@apache.org>.
ArbaazKhan1 commented on PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#issuecomment-3234098297
To fix the tests failing in the flakyITs I added a large wait time in `eventualScanTest` to compensate for the the fact that the flakyITs are slowing things down on an already slow test, but the byproduct of this is the test now runs a but longer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#discussion_r1723713286
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +152,61 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ try (AccumuloClient client = Accumulo.newClient().from(getClientProps()).build()) {
Review Comment:
I opened #4814 to change the default behavior to no fall back. If that change is merged in then no changes would be needed in this test.
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +152,61 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ try (AccumuloClient client = Accumulo.newClient().from(getClientProps()).build()) {
Review Comment:
I opened #4814 to change the default behavior to not fall back. If that change is merged in then no changes would be needed in this test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on code in PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#discussion_r2195597669
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveITBase.java:
##########
@@ -121,6 +126,56 @@ public abstract class ComprehensiveITBase extends SharedMiniClusterBase {
private static final Logger log = LoggerFactory.getLogger(ComprehensiveIT_SimpleSuite.class);
+ @Test
+ public void testEventualScan() throws Exception {
+ Properties props = new Properties();
+ props.putAll(getClientProps());
+ props.put(ClientProperty.SCAN_SERVER_SELECTOR.getKey(),
+ ConfigurableScanServerSelector.class.getName());
+ props.put(ClientProperty.SCAN_SERVER_SELECTOR_OPTS_PREFIX.getKey() + "profiles",
+ "[{'isDefault':true,'timeToWaitForScanServers' : '10d','maxBusyTimeout':'5m','busyTimeoutMultiplier':4,'attemptPlans':[{\"servers\":\"100%\", \"busyTimeout\":\"3ms\"}]}]");
+
+ try (AccumuloClient client = Accumulo.newClient().from(props).build()) {
+ String table = getUniqueNames(1)[0];
+ client.tableOperations().create(table);
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
+ Wait.waitFor(
+ () -> !client.instanceOperations().getServers(ServerId.Type.SCAN_SERVER).isEmpty());
+
+ write(client, table, generateMutations(0, 100, tr -> true));
+ client.tableOperations().flush(table, null, null, true);
Review Comment:
I think this looks good. Can you add two things before this flush call?
1. Scan without setting consistencyLevel to ensure 1-100 are returned
2. Scan with setting consistencyLevel to EVENTUAL and ensure nothing is returned.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "ArbaazKhan1 (via GitHub)" <gi...@apache.org>.
ArbaazKhan1 commented on PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#issuecomment-3234409215
Thanks Dan! Setting the metadata expiration time in the flakyITs did resolve the the issue and the test is much faster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "ArbaazKhan1 (via GitHub)" <gi...@apache.org>.
ArbaazKhan1 commented on PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#issuecomment-3234311558
Sorry I was testing timeout durations and forgot to set it back to 5 mins, but I still think is a bit too long for the test. I think I'll take a look at chaining the metadata expiration property and see if that could resolve the test failure without making it run forever
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#discussion_r1723713286
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +152,61 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ try (AccumuloClient client = Accumulo.newClient().from(getClientProps()).build()) {
Review Comment:
I opened #4814 to change the default behavior to not fall back. If that change is merged in then no changes would be needed in this test for scan server selector config.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on code in PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#discussion_r1734935278
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +151,58 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ Properties props = new Properties();
+ props.putAll(getClientProps());
+ props.put(ClientProperty.SCAN_SERVER_SELECTOR.getKey(),
+ ConfigurableScanServerSelector.class.getName());
+ props.put(ClientProperty.SCAN_SERVER_SELECTOR_OPTS_PREFIX.getKey() + "profiles",
+ "[{'isDefault':true,'timeToWaitForScanServers' : '10d','maxBusyTimeout':'5m','busyTimeoutMultiplier':4,'attemptPlans':[{\"servers\":\"100%\", \"busyTimeout\":\"3ms\"}]}]");
+
+ try (AccumuloClient client = Accumulo.newClient().from(props).build()) {
+
+ String table = getUniqueNames(1)[0];
+ client.tableOperations().create(table);
+
+ // Start Scan Server
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
+
+ // generate 100 rows of data
+ write(client, table, generateMutations(0, 100, tr -> true));
+
+ // need flush table so that eventual scan can see it.
+ getCluster().getClusterControl().stop(ServerType.SCAN_SERVER);
+ client.tableOperations().flush(table, null, null, true);
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
+
+ // should see all data that was flushed in eventual scan
+ verifyData(client, table, AUTHORIZATIONS, generateKeys(0, 100),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ // should not see data with col vis set
+ verifyData(client, table, Authorizations.EMPTY, generateKeys(0, 100, tr -> tr.vis.isEmpty()),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ // write some more rows after 100 and verify those are not seen by eventual scan until table
+ // is flushed.
+ write(client, table, generateMutations(100, 200, tr -> true));
+ verifyData(client, table, AUTHORIZATIONS, generateKeys(0, 100),
Review Comment:
ok, I see it now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on code in PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#discussion_r1734851482
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +151,58 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ Properties props = new Properties();
+ props.putAll(getClientProps());
+ props.put(ClientProperty.SCAN_SERVER_SELECTOR.getKey(),
+ ConfigurableScanServerSelector.class.getName());
+ props.put(ClientProperty.SCAN_SERVER_SELECTOR_OPTS_PREFIX.getKey() + "profiles",
+ "[{'isDefault':true,'timeToWaitForScanServers' : '10d','maxBusyTimeout':'5m','busyTimeoutMultiplier':4,'attemptPlans':[{\"servers\":\"100%\", \"busyTimeout\":\"3ms\"}]}]");
+
+ try (AccumuloClient client = Accumulo.newClient().from(props).build()) {
+
+ String table = getUniqueNames(1)[0];
+ client.tableOperations().create(table);
+
+ // Start Scan Server
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
+
+ // generate 100 rows of data
+ write(client, table, generateMutations(0, 100, tr -> true));
+
+ // need flush table so that eventual scan can see it.
+ getCluster().getClusterControl().stop(ServerType.SCAN_SERVER);
Review Comment:
`Property.SSERV_CACHED_TABLET_METADATA_EXPIRATION` is being set to "5s" above, so it should be the case that after that expiration the ScanServer refreshes its view of the metadata for the tablets you are scanning over. Because you called `client.tableOperations().flush()` a new file was created on the filesystem and attached to that tablet. I think you just need to wait longer than 5s (maybe wait 10s) to pick up the new version of the tablet metadata. I would suggest trying that instead of stopping/starting the ScanServers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on code in PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#discussion_r1717785229
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +152,49 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ try (AccumuloClient client = Accumulo.newClient().from(getClientProps()).build()) {
+
+ String table = getUniqueNames(1)[0];
+ client.tableOperations().create(table);
+
+ // Start Scan Server
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
+
+ // generate 100 rows of data
+ write(client, table, generateMutations(0, 100, tr -> true));
+
+ // need flush table so that eventual scan can see it.
+ client.tableOperations().flush(table, null, null, true);
Review Comment:
I'm very confused by this approach, and I don't think it's correct. First, it uses `System.currentTimeMillis()` to compute an elapsed time, which we should *never* be doing unless we're estimating elapsed time between two different processes and have no other more authoritative source for time than the system clock.
But more importantly, the time elapsed prior to the conclusion of the flush command doesn't count at all, and there's no point in tracking it. The only time that counts is the time immediately following the conclusion of the flush, because that's the earliest possible time we are guaranteed that the data is written and the flushed file now appears in the metadata for the scan server to see. It does not matter how long the time has elapsed during the flush call... only how much time has elapsed after.
Let's say that the scan server refreshed *immediately* prior to the flush finishing. The test has to wait for the scan server to complete it's *full* 5 second cycle to be guaranteed that the scan server will pick up the new file. But, if the scan server was busy having just finished up a refresh, but its refresh clock hasn't been reset yet, it might take a little bit longer. So, the concept of starting the clock in this test prior to the flush, and only waiting a "remaining" amount doesn't make any sense. You have to fully complete the flush (using wait), and *then* you have to wait at least the full cache refresh time for the scan server, and then maybe a little bit more, before you can guarantee the scan server will pick up the changes.
The only way I can think of shortening the time is to try to verify the data right away, and if it fails and it's less than 5 seconds, wait a tiny bit and then loop to try again. If it takes much longer than the 5 seconds, you can time out and assume it's never going to pick up the new file. But that's probably a bit overkill since you're just keeping the system unnecessarily busy for 5 seconds in the hopes that it will refresh sooner. If you're going to do that, it'd be better to just restart the scan server, to guarantee that it has refreshed data.
I suggest one of the following:
1. flush(wait = true); then sleep 6 seconds (5+20%) to nearly guarantee that it is refreshed; verify data, OR
2. stop scan server; flush(wait = true); restart scan server to absolutely guarantee that it is refreshed; verify data
With option 1, the test may be flaky on extremely rare occassions, but good enough. Option 2 is my preferred approach, assuming it's not substantially slower than option 1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#discussion_r1693623692
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +153,51 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ try (AccumuloClient client = Accumulo.newClient().from(getClientProps()).build()) {
+ String tableName = getUniqueNames(1)[0];
+ client.tableOperations().create(tableName);
+
Review Comment:
My suggestion has a bug. If you run eventual scan before flush, then run eventual scan after flush will probably still see no data as the scan server caches tablet files for a bit. So would either need to drop that scan before flush, or set the scan server cache timeout low and wait for longer than that timeout before scanning after flush.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#discussion_r1693463244
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +153,51 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ try (AccumuloClient client = Accumulo.newClient().from(getClientProps()).build()) {
+ String tableName = getUniqueNames(1)[0];
+ client.tableOperations().create(tableName);
+
Review Comment:
This test has some utils that are use to read and write data. Using those will test authorizations, scanner, and batch scanner. Also would make the code shorter in addition to testing more. The test functions allow passing a lambda to configure the scanner can use that to set eventual consistency.
Could use them in this test like the following to create a basic verification of eventual scans.
```java
// TODO start scan server
// generate 300 rows of data
write(client, table, generateMutations(0, 300, tr -> true));
// eventual scan should not see anything when table is not flushed
verifyData(client, table, AUTHORIZATIONS, Map.of(),
scanner -> scanner.setConsistencyLevel(EVENTUAL));
// TODO need flush table so that eventual scan can see it.
// should see all data that was flushed in eventual scan
verifyData(client, table, AUTHORIZATIONS, generateKeys(0, 300),
scanner -> scanner.setConsistencyLevel(EVENTUAL));
// should not see data with col vis set
verifyData(client, table, Authorizations.EMPTY, generateKeys(0, 300, tr -> tr.vis.isEmpty()),
scanner -> scanner.setConsistencyLevel(EVENTUAL));
// TODO write some more rows after 300 and verfy those are not seen by eventual scan until table is flushed.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#discussion_r1717275812
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +152,49 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ try (AccumuloClient client = Accumulo.newClient().from(getClientProps()).build()) {
+
+ String table = getUniqueNames(1)[0];
+ client.tableOperations().create(table);
+
+ // Start Scan Server
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
+
+ // generate 100 rows of data
+ write(client, table, generateMutations(0, 100, tr -> true));
+
+ // need flush table so that eventual scan can see it.
+ client.tableOperations().flush(table, null, null, true);
Review Comment:
The Scan Servers are set to only refresh data after 5 seconds.
You could reference that as a min wait time and only wait the remainder vs the full 5 seconds.
```suggestion
long minWait = 5_000;
var startTime = System.currentTimeMillis();
client.tableOperations().flush(table, null, null, true);
var duration = System.currentTimeMillis() - startTime;
// if duration is less than minWait, wait the remainder + 0.5 seconds.
if (minWait > duration) {
Thread.sleep( (minWait - duration) + 500);
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "ArbaazKhan1 (via GitHub)" <gi...@apache.org>.
ArbaazKhan1 commented on PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#issuecomment-2312610785
I added in keith's suggestions on how not fall back to tservers and the tests are now working as expected when stoping and restarting scan servers
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#discussion_r1707956179
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +152,50 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ try (AccumuloClient client = Accumulo.newClient().from(getClientProps()).build()) {
+
+ String table = getUniqueNames(1)[0];
+ client.tableOperations().create(table);
+
+ // Start Scan Server
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
+
+ // generate 100 rows of data
+ write(client, table, generateMutations(0, 100, tr -> true));
+
+ // need flush table so that eventual scan can see it.
+ client.tableOperations().flush(table);
+
+ Thread.sleep(10000);
Review Comment:
This flush call starts the flush but does not wait for it. There is another flush call where you can optionally wait. Could call that and set wait to true and drop the sleep.
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +152,50 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ try (AccumuloClient client = Accumulo.newClient().from(getClientProps()).build()) {
+
+ String table = getUniqueNames(1)[0];
+ client.tableOperations().create(table);
+
+ // Start Scan Server
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
+
+ // generate 100 rows of data
+ write(client, table, generateMutations(0, 100, tr -> true));
+
+ // need flush table so that eventual scan can see it.
+ client.tableOperations().flush(table);
+
+ Thread.sleep(10000);
+
+ // should see all data that was flushed in eventual scan
+ verifyData(client, table, AUTHORIZATIONS, generateKeys(0, 100),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ // should not see data with col vis set
+ verifyData(client, table, Authorizations.EMPTY, generateKeys(0, 100, tr -> tr.vis.isEmpty()),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ // write some more rows after 100 and verfy those are not seen by eventual scan until table is
+ // flushed.
+ write(client, table, generateMutations(100, 200, tr -> true));
+ verifyData(client, table, AUTHORIZATIONS, generateKeys(0, 100),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ // need flush table so that eventual scan can see it.
+ client.tableOperations().flush(table);
+
+ Thread.sleep(10000);
+
+ verifyData(client, table, AUTHORIZATIONS, generateKeys(0, 200),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ getCluster().getClusterControl().stop(ServerType.SCAN_SERVER);
Review Comment:
Could put this in a finally block adding it to the existing try.
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +152,50 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ try (AccumuloClient client = Accumulo.newClient().from(getClientProps()).build()) {
+
+ String table = getUniqueNames(1)[0];
+ client.tableOperations().create(table);
+
+ // Start Scan Server
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
+
+ // generate 100 rows of data
+ write(client, table, generateMutations(0, 100, tr -> true));
+
+ // need flush table so that eventual scan can see it.
+ client.tableOperations().flush(table);
+
+ Thread.sleep(10000);
+
+ // should see all data that was flushed in eventual scan
+ verifyData(client, table, AUTHORIZATIONS, generateKeys(0, 100),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ // should not see data with col vis set
+ verifyData(client, table, Authorizations.EMPTY, generateKeys(0, 100, tr -> tr.vis.isEmpty()),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ // write some more rows after 100 and verfy those are not seen by eventual scan until table is
+ // flushed.
+ write(client, table, generateMutations(100, 200, tr -> true));
+ verifyData(client, table, AUTHORIZATIONS, generateKeys(0, 100),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ // need flush table so that eventual scan can see it.
+ client.tableOperations().flush(table);
+
+ Thread.sleep(10000);
Review Comment:
Can also call the flush with wait here and drop the sleep.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#discussion_r1744535780
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +151,58 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ Properties props = new Properties();
+ props.putAll(getClientProps());
+ props.put(ClientProperty.SCAN_SERVER_SELECTOR.getKey(),
+ ConfigurableScanServerSelector.class.getName());
+ props.put(ClientProperty.SCAN_SERVER_SELECTOR_OPTS_PREFIX.getKey() + "profiles",
+ "[{'isDefault':true,'timeToWaitForScanServers' : '10d','maxBusyTimeout':'5m','busyTimeoutMultiplier':4,'attemptPlans':[{\"servers\":\"100%\", \"busyTimeout\":\"3ms\"}]}]");
+
+ try (AccumuloClient client = Accumulo.newClient().from(props).build()) {
+
+ String table = getUniqueNames(1)[0];
+ client.tableOperations().create(table);
+
+ // Start Scan Server
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
+
+ // generate 100 rows of data
+ write(client, table, generateMutations(0, 100, tr -> true));
+
+ // need flush table so that eventual scan can see it.
+ getCluster().getClusterControl().stop(ServerType.SCAN_SERVER);
+ client.tableOperations().flush(table, null, null, true);
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
+
+ // should see all data that was flushed in eventual scan
+ verifyData(client, table, AUTHORIZATIONS, generateKeys(0, 100),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ // should not see data with col vis set
+ verifyData(client, table, Authorizations.EMPTY, generateKeys(0, 100, tr -> tr.vis.isEmpty()),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ // write some more rows after 100 and verify those are not seen by eventual scan until table
+ // is flushed.
+ write(client, table, generateMutations(100, 200, tr -> true));
+ verifyData(client, table, AUTHORIZATIONS, generateKeys(0, 100),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ // need flush table so that eventual scan can see it.
+ getCluster().getClusterControl().stop(ServerType.SCAN_SERVER);
+ client.tableOperations().flush(table, null, null, true);
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
Review Comment:
If we add some code to wait for the new data to be present on the scan server then do not need to restart.
```suggestion
client.tableOperations().flush(table, null, null, true);
// wait for the eventual scan to see the new data
final int initialSize = generateKeys(0, 100).size();
Wait.waitFor(() -> {
try (var scanner = client.createScanner(table, AUTHORIZATIONS)) {
scanner.setConsistencyLevel(EVENTUAL);
return scan(scanner).size() > initialSize;
}
});
```
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +151,58 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ Properties props = new Properties();
+ props.putAll(getClientProps());
+ props.put(ClientProperty.SCAN_SERVER_SELECTOR.getKey(),
+ ConfigurableScanServerSelector.class.getName());
+ props.put(ClientProperty.SCAN_SERVER_SELECTOR_OPTS_PREFIX.getKey() + "profiles",
+ "[{'isDefault':true,'timeToWaitForScanServers' : '10d','maxBusyTimeout':'5m','busyTimeoutMultiplier':4,'attemptPlans':[{\"servers\":\"100%\", \"busyTimeout\":\"3ms\"}]}]");
+
+ try (AccumuloClient client = Accumulo.newClient().from(props).build()) {
+
+ String table = getUniqueNames(1)[0];
+ client.tableOperations().create(table);
+
+ // Start Scan Server
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
+
+ // generate 100 rows of data
+ write(client, table, generateMutations(0, 100, tr -> true));
+
+ // need flush table so that eventual scan can see it.
+ getCluster().getClusterControl().stop(ServerType.SCAN_SERVER);
+ client.tableOperations().flush(table, null, null, true);
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
Review Comment:
Should not need to start and stop the server here. At this point the tablet has not been accessed via the scan server so the first time the tablet is read on the scan server it will see the files from this flush.
```suggestion
client.tableOperations().flush(table, null, null, true);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on code in PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#discussion_r1734823803
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +151,58 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ Properties props = new Properties();
+ props.putAll(getClientProps());
+ props.put(ClientProperty.SCAN_SERVER_SELECTOR.getKey(),
+ ConfigurableScanServerSelector.class.getName());
+ props.put(ClientProperty.SCAN_SERVER_SELECTOR_OPTS_PREFIX.getKey() + "profiles",
+ "[{'isDefault':true,'timeToWaitForScanServers' : '10d','maxBusyTimeout':'5m','busyTimeoutMultiplier':4,'attemptPlans':[{\"servers\":\"100%\", \"busyTimeout\":\"3ms\"}]}]");
+
+ try (AccumuloClient client = Accumulo.newClient().from(props).build()) {
+
+ String table = getUniqueNames(1)[0];
+ client.tableOperations().create(table);
+
+ // Start Scan Server
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
+
+ // generate 100 rows of data
+ write(client, table, generateMutations(0, 100, tr -> true));
+
+ // need flush table so that eventual scan can see it.
+ getCluster().getClusterControl().stop(ServerType.SCAN_SERVER);
Review Comment:
I'm not sure why you are starting and stopping the scan server multiple times. You should be able to start it at the beginning of the test and shut it down at the end. Am I missing something?
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +151,58 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ Properties props = new Properties();
+ props.putAll(getClientProps());
+ props.put(ClientProperty.SCAN_SERVER_SELECTOR.getKey(),
+ ConfigurableScanServerSelector.class.getName());
+ props.put(ClientProperty.SCAN_SERVER_SELECTOR_OPTS_PREFIX.getKey() + "profiles",
+ "[{'isDefault':true,'timeToWaitForScanServers' : '10d','maxBusyTimeout':'5m','busyTimeoutMultiplier':4,'attemptPlans':[{\"servers\":\"100%\", \"busyTimeout\":\"3ms\"}]}]");
+
+ try (AccumuloClient client = Accumulo.newClient().from(props).build()) {
+
+ String table = getUniqueNames(1)[0];
+ client.tableOperations().create(table);
+
+ // Start Scan Server
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
+
+ // generate 100 rows of data
+ write(client, table, generateMutations(0, 100, tr -> true));
+
+ // need flush table so that eventual scan can see it.
+ getCluster().getClusterControl().stop(ServerType.SCAN_SERVER);
+ client.tableOperations().flush(table, null, null, true);
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
+
+ // should see all data that was flushed in eventual scan
+ verifyData(client, table, AUTHORIZATIONS, generateKeys(0, 100),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ // should not see data with col vis set
+ verifyData(client, table, Authorizations.EMPTY, generateKeys(0, 100, tr -> tr.vis.isEmpty()),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ // write some more rows after 100 and verify those are not seen by eventual scan until table
+ // is flushed.
+ write(client, table, generateMutations(100, 200, tr -> true));
+ verifyData(client, table, AUTHORIZATIONS, generateKeys(0, 100),
Review Comment:
I don't think this is verifying that the scanner can't see rows 101-200, it's only verifying that it can see rows 0 - 100, right? My guess is that you want to run this with 0-200, like you are doing below, but assert that an error is thrown for missing data.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#discussion_r1723682568
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +152,61 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ try (AccumuloClient client = Accumulo.newClient().from(getClientProps()).build()) {
+
+ String table = getUniqueNames(1)[0];
+ client.tableOperations().create(table);
+
+ // Start Scan Server
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
+
+ // generate 100 rows of data
+ write(client, table, generateMutations(0, 100, tr -> true));
+
+ // need flush table so that eventual scan can see it.
+
+ long minWait = 5_000;
+ var startTime = System.currentTimeMillis();
+ client.tableOperations().flush(table, null, null, true);
+ var duration = System.currentTimeMillis() - startTime;
+ // if duration is less than minWait, wait the remainder + 0.5 seconds.
+ if (minWait > duration) {
+ Thread.sleep((minWait - duration) + 500);
+ }
Review Comment:
There is no need to sleep here if we assume nothing else is using the scan servers, which is the case for this test. The first scan will cause the scan server to read and cache the tablets files.
```suggestion
client.tableOperations().flush(table, null, null, true);
```
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +152,61 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ try (AccumuloClient client = Accumulo.newClient().from(getClientProps()).build()) {
Review Comment:
Eventual scans may fall back to tablet servers if no scan servers are present. This code starts the scans server and then continues the test, so the scan server may still be starting up while the test is running. To avoid this we could update the client config using the changes from #4671 to avoid falling back to a tablet server.
```suggestion
Properties props = new Properties();
props.putAll(getClientProps());
// TODO configure scan server selector to wait for scan servers by adding to props
try (AccumuloClient client = Accumulo.newClient().from(props).build()) {
```
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +152,61 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ try (AccumuloClient client = Accumulo.newClient().from(getClientProps()).build()) {
+
+ String table = getUniqueNames(1)[0];
+ client.tableOperations().create(table);
+
+ // Start Scan Server
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
+
+ // generate 100 rows of data
+ write(client, table, generateMutations(0, 100, tr -> true));
+
+ // need flush table so that eventual scan can see it.
+
+ long minWait = 5_000;
+ var startTime = System.currentTimeMillis();
+ client.tableOperations().flush(table, null, null, true);
+ var duration = System.currentTimeMillis() - startTime;
+ // if duration is less than minWait, wait the remainder + 0.5 seconds.
+ if (minWait > duration) {
+ Thread.sleep((minWait - duration) + 500);
+ }
+
+ // should see all data that was flushed in eventual scan
+ verifyData(client, table, AUTHORIZATIONS, generateKeys(0, 100),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ // should not see data with col vis set
+ verifyData(client, table, Authorizations.EMPTY, generateKeys(0, 100, tr -> tr.vis.isEmpty()),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ // write some more rows after 100 and verfy those are not seen by eventual scan until table is
+ // flushed.
+ write(client, table, generateMutations(100, 200, tr -> true));
+ verifyData(client, table, AUTHORIZATIONS, generateKeys(0, 100),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ // need flush table so that eventual scan can see it.
+ startTime = System.currentTimeMillis();
+ client.tableOperations().flush(table, null, null, true);
+ duration = System.currentTimeMillis() - startTime;
+ // if duration is less than minWait, wait the remainder + 0.5 seconds.
+ if (minWait > duration) {
+ Thread.sleep((minWait - duration) + 500);
+ }
Review Comment:
Instead of sleeping after the flush could instead wait for the data to be present like the following.
```suggestion
// wait for the eventual scan to see the new data
final int initialSize = generateKeys(0, 100).size();
Wait.waitFor(()-> {
try(var scanner = client.createScanner(table, AUTHORIZATIONS)) {
scanner.setConsistencyLevel(EVENTUAL);
return scan(scanner).size() > initialSize;
}
});
```
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +152,61 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ try (AccumuloClient client = Accumulo.newClient().from(getClientProps()).build()) {
+
+ String table = getUniqueNames(1)[0];
+ client.tableOperations().create(table);
+
+ // Start Scan Server
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
+
+ // generate 100 rows of data
+ write(client, table, generateMutations(0, 100, tr -> true));
+
+ // need flush table so that eventual scan can see it.
+
+ long minWait = 5_000;
+ var startTime = System.currentTimeMillis();
+ client.tableOperations().flush(table, null, null, true);
+ var duration = System.currentTimeMillis() - startTime;
+ // if duration is less than minWait, wait the remainder + 0.5 seconds.
+ if (minWait > duration) {
+ Thread.sleep((minWait - duration) + 500);
+ }
+
+ // should see all data that was flushed in eventual scan
+ verifyData(client, table, AUTHORIZATIONS, generateKeys(0, 100),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ // should not see data with col vis set
+ verifyData(client, table, Authorizations.EMPTY, generateKeys(0, 100, tr -> tr.vis.isEmpty()),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ // write some more rows after 100 and verfy those are not seen by eventual scan until table is
+ // flushed.
+ write(client, table, generateMutations(100, 200, tr -> true));
+ verifyData(client, table, AUTHORIZATIONS, generateKeys(0, 100),
Review Comment:
This could fail if the scan server has not started yet and the code falls back to the tablet server.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "kevinrr888 (via GitHub)" <gi...@apache.org>.
kevinrr888 merged PR #4754:
URL: https://github.com/apache/accumulo/pull/4754
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#issuecomment-3234250861
> The wait was adjusted to 600000 ms = 10 min. Is this intended? This seems excessive
The default value for `sserver.cache.metadata.expiration` is 5min which is likely why a timeout of 10min is being used.
The metadata expiration property value should be changed in the FlakyITs so the scan servers refresh faster.
The `5s` value used in ComprehensiveIT should be sufficient for reducing this timeout.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on code in PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#discussion_r1692106112
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +153,51 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ try (AccumuloClient client = Accumulo.newClient().from(getClientProps()).build()) {
+ String tableName = getUniqueNames(1)[0];
+ client.tableOperations().create(tableName);
+
+ try (Scanner scanner = client.createScanner(tableName, Authorizations.EMPTY)) {
+
+ // Add data to the table
+ try (BatchWriter writer = client.createBatchWriter(tableName, new BatchWriterConfig())) {
+ Mutation m = new Mutation("row1");
+ m.put("cf", "cq", "value1");
+ writer.addMutation(m);
+ }
+
+ // Run initial scan with eventual consistency
Review Comment:
I would not think that this eventual scan would consistently find the data as you are not flushing the table after inserting the data into it. If you have not seen them already, take a look at `ScanServerIT.testScan` and `ScanServerIT.testBatchScan`. It might be sufficient to copy them here or invoke them from here if that actually works.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "ArbaazKhan1 (via GitHub)" <gi...@apache.org>.
ArbaazKhan1 commented on code in PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#discussion_r1734860957
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +151,58 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ Properties props = new Properties();
+ props.putAll(getClientProps());
+ props.put(ClientProperty.SCAN_SERVER_SELECTOR.getKey(),
+ ConfigurableScanServerSelector.class.getName());
+ props.put(ClientProperty.SCAN_SERVER_SELECTOR_OPTS_PREFIX.getKey() + "profiles",
+ "[{'isDefault':true,'timeToWaitForScanServers' : '10d','maxBusyTimeout':'5m','busyTimeoutMultiplier':4,'attemptPlans':[{\"servers\":\"100%\", \"busyTimeout\":\"3ms\"}]}]");
+
+ try (AccumuloClient client = Accumulo.newClient().from(props).build()) {
+
+ String table = getUniqueNames(1)[0];
+ client.tableOperations().create(table);
+
+ // Start Scan Server
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
+
+ // generate 100 rows of data
+ write(client, table, generateMutations(0, 100, tr -> true));
+
+ // need flush table so that eventual scan can see it.
+ getCluster().getClusterControl().stop(ServerType.SCAN_SERVER);
+ client.tableOperations().flush(table, null, null, true);
+ getCluster().getClusterControl().start(ServerType.SCAN_SERVER);
+
+ // should see all data that was flushed in eventual scan
+ verifyData(client, table, AUTHORIZATIONS, generateKeys(0, 100),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ // should not see data with col vis set
+ verifyData(client, table, Authorizations.EMPTY, generateKeys(0, 100, tr -> tr.vis.isEmpty()),
+ scanner -> scanner.setConsistencyLevel(EVENTUAL));
+
+ // write some more rows after 100 and verify those are not seen by eventual scan until table
+ // is flushed.
+ write(client, table, generateMutations(100, 200, tr -> true));
+ verifyData(client, table, AUTHORIZATIONS, generateKeys(0, 100),
Review Comment:
VerifyData is checking the full range of the scanner. So by checking rows 0-100 after writing rows 101-200 is to make sure data is not present until the flush.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#discussion_r1723680133
##########
test/src/main/java/org/apache/accumulo/test/ComprehensiveIT.java:
##########
@@ -133,6 +152,61 @@ public static void teardown() {
SharedMiniClusterBase.stopMiniCluster();
}
+ @Test
+ public void testEventualScan() throws Exception {
+ try (AccumuloClient client = Accumulo.newClient().from(getClientProps()).build()) {
Review Comment:
Eventual scans may fall back to tablet servers if no scan servers are present. This code starts the scans server and then continues the test, so the scan server may still be starting up while the test is running. To avoid this we could update the client config using the changes from #4671 to avoid falling back to a tablet server.
```suggestion
Properties props = new Properties();
props.putAll(getClientProps());
props.put(ClientProperty.SCAN_SERVER_SELECTOR.getKey(), ConfigurableScanServerSelector.class.getName());
props.put(ClientProperty.SCAN_SERVER_SELECTOR_OPTS_PREFIX.getKey()+"profiles", "[{'isDefault':true,'timeToWaitForScanServers' : '10d','maxBusyTimeout':'5m','busyTimeoutMultiplier':4,'attemptPlans':[{\"servers\":\"100%\", \"busyTimeout\":\"3ms\"}]}]");
try (AccumuloClient client = Accumulo.newClient().from(props).build()) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Added Eventual Scan test in ComprehensiveIT [accumulo]
Posted by "kevinrr888 (via GitHub)" <gi...@apache.org>.
kevinrr888 commented on PR #4754:
URL: https://github.com/apache/accumulo/pull/4754#issuecomment-3234168040
The wait was adjusted to 600000 ms = 10 min. Is this intended? This seems excessive
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org