You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/08/11 09:27:57 UTC
[GitHub] [pulsar] vroyer opened a new pull request #11638: [Issue #11628][ES-Sink] Fix flaky tests
vroyer opened a new pull request #11638:
URL: https://github.com/apache/pulsar/pull/11638
Fixes #11628
* Use random index names and delete it at the end of each test
* Try to fix the flaky test testBulkBlocking
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on a change in pull request #11638: [Issue #11628][ES-Sink] Fix flaky tests
Posted by GitBox <gi...@apache.org>.
eolivelli commented on a change in pull request #11638:
URL: https://github.com/apache/pulsar/pull/11638#discussion_r686740178
##########
File path: pulsar-io/elastic-search/src/test/java/org/apache/pulsar/io/elasticsearch/ElasticSearchClientTests.java
##########
@@ -212,39 +229,49 @@ public void testBulkBlocking() throws Exception {
.setRetryBackoffInMs(100)
.setBulkFlushIntervalInMs(10000);
ElasticSearchClient client = new ElasticSearchClient(config);
- client.createIndexIfNeeded(index);
-
- MockRecord<GenericObject> mockRecord = new MockRecord<>();
- for(int i = 1; i <= 5; i++) {
- client.bulkIndex(mockRecord, Pair.of(Integer.toString(i), "{\"a\":"+i+"}"));
- }
- // wait bulk flush interval
- Awaitility.await().untilAsserted( () -> {
- assertEquals(mockRecord.acked, 5);
+ assertTrue(client.createIndexIfNeeded(index));
+
+ try {
+ MockRecord<GenericObject> mockRecord = new MockRecord<>();
+ for (int i = 1; i <= 5; i++) {
+ client.bulkIndex(mockRecord, Pair.of(Integer.toString(i), "{\"a\":" + i + "}"));
+ }
+
+ Awaitility.await().untilAsserted(() -> {
+ assertThat("acked record", mockRecord.acked, greaterThanOrEqualTo(4));
+ assertEquals(mockRecord.failed, 0);
+ assertThat("totalHits", client.totalHits(index), greaterThanOrEqualTo(4L));
+ });
+ client.flush();
+ Awaitility.await().untilAsserted(() -> {
+ assertEquals(mockRecord.acked, 5);
+ assertEquals(mockRecord.failed, 0);
+ assertEquals(client.totalHits(index), 5);
+ });
+
+ ChaosContainer<?> chaosContainer = new ChaosContainer<>(container.getContainerName(), "30s");
+ chaosContainer.start();
+ Thread.sleep(1000L);
+
+ // 11th bulkIndex is blocking because we have 2 pending requests, and the 3rd request is blocked.
+ long start = System.currentTimeMillis();
+ for (int i = 6; i <= 15; i++) {
+ client.bulkIndex(mockRecord, Pair.of(Integer.toString(i), "{\"a\":" + i + "}"));
+ log.info("{} index [}", System.currentTimeMillis(), i);
+ }
+ long elapsed = System.currentTimeMillis() - start;
+ log.info("elapsed = {}", elapsed);
+ assertTrue(elapsed > 29000); // bulkIndex was blocking while elasticsearch was down or busy
Review comment:
so this test is taking more than 30 seconds?
if we cannot do better then I am fine with it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli merged pull request #11638: [Issue #11628][ES-Sink] Fix flaky tests
Posted by GitBox <gi...@apache.org>.
eolivelli merged pull request #11638:
URL: https://github.com/apache/pulsar/pull/11638
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org