You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "gargvishesh (via GitHub)" <gi...@apache.org> on 2024/09/11 08:45:28 UTC

[PR] Allow MSQ engine for compaction only when run as a supervisor on the overlord (druid)

gargvishesh opened a new pull request, #17033:
URL: https://github.com/apache/druid/pull/17033

   ### Description
   
   https://github.com/apache/druid/pull/16768 added the functionality to run compaction as a supervisor on the overlord. This PR builds on top of that to restrict MSQ engine to compaction in the supervisor-mode only. With these changes, users can no longer add MSQ engine as part of datasource compaction config, or as the default cluster-level compaction engine, on the Coordinator. 
   
   The PR also adds an Overlord runtime property `druid.supervisor.compaction.defaultEngine=<msq/native>` to specify the default engine for compaction supervisors.
   
   Since these updates require major changes to existing MSQ compaction integration tests, this PR disables MSQ-specific compaction integration tests -- they will be taken up in a follow-up PR.
   
   ##### Key changed/added classes in this PR
   * CompactionSupervisor
   * CompactionSupervisorSpec
   * CoordinatorCompactionConfigsResource
   * OverlordCompactionScheduler
   
   
   This PR has:
   
   - [x] been self-reviewed.
      - [ ] using the [concurrency checklist](https://github.com/apache/druid/blob/master/dev/code-review/concurrency.md) (Remove this item if the PR doesn't have any relation to concurrency.)
   - [ ] added documentation for new or modified features or behaviors.
   - [ ] a release note entry in the PR description.
   - [ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
   - [ ] added or updated version, license, or notice information in [licenses.yaml](https://github.com/apache/druid/blob/master/dev/license.md)
   - [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
   - [x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for [code coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md) is met.
   - [ ] added integration tests.
   - [x] been tested in a test Druid cluster.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [PR] Allow MSQ engine only for compaction supervisors (druid)

Posted by "gargvishesh (via GitHub)" <gi...@apache.org>.
gargvishesh commented on code in PR #17033:
URL: https://github.com/apache/druid/pull/17033#discussion_r1771301164


##########
server/src/main/java/org/apache/druid/client/indexing/ClientCompactionRunnerInfo.java:
##########
@@ -135,9 +135,8 @@ public static CompactionConfigValidationResult validatePartitionsSpecForMSQ(Part
     if (!(partitionsSpec instanceof DimensionRangePartitionsSpec
           || partitionsSpec instanceof DynamicPartitionsSpec)) {
       return CompactionConfigValidationResult.failure(
-          "MSQ: Invalid partitioning type[%s]. Must be either 'dynamic' or 'range'",
+          "MSQ: Invalid partitioning type[%s]. Must be either dynamic or range",

Review Comment:
   Reverted to quotes. When the text was passed as part of exception, single-quotes weren't being rendered in the web-console as `&apos;` -- which is not the case when passed as part of `AutoCompactionSnapshot`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [PR] Allow MSQ engine only for compaction supervisors (druid)

Posted by "gargvishesh (via GitHub)" <gi...@apache.org>.
gargvishesh commented on code in PR #17033:
URL: https://github.com/apache/druid/pull/17033#discussion_r1771298987


##########
indexing-service/src/main/java/org/apache/druid/indexing/compact/CompactionSupervisorSpec.java:
##########
@@ -77,6 +73,11 @@ public String getId()
     return ID_PREFIX + spec.getDataSource();
   }
 
+  public CompactionConfigValidationResult getValidationResult()

Review Comment:
   Confirmed that it doesn't.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [PR] Allow MSQ engine only for compaction supervisors (druid)

Posted by "gargvishesh (via GitHub)" <gi...@apache.org>.
gargvishesh commented on code in PR #17033:
URL: https://github.com/apache/druid/pull/17033#discussion_r1771301482


##########
server/src/main/java/org/apache/druid/server/coordinator/CompactionSupervisorConfig.java:
##########
@@ -33,10 +34,12 @@
  */
 public class CompactionSupervisorConfig
 {
-  private static final CompactionSupervisorConfig DEFAULT = new CompactionSupervisorConfig(null);
+  private static final CompactionSupervisorConfig DEFAULT = new CompactionSupervisorConfig(null, null);
 
   @JsonProperty
   private final boolean enabled;
+  @JsonProperty
+  private final CompactionEngine defaultEngine;

Review Comment:
   Updated to `engine`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [PR] Allow MSQ engine only for compaction supervisors (druid)

Posted by "gargvishesh (via GitHub)" <gi...@apache.org>.
gargvishesh commented on code in PR #17033:
URL: https://github.com/apache/druid/pull/17033#discussion_r1772742024


##########
indexing-service/src/test/java/org/apache/druid/indexing/compact/OverlordCompactionSchedulerTest.java:
##########
@@ -158,7 +158,10 @@ public void testCompactionSupervisorConfigSerde() throws JsonProcessingException
             OBJECT_MAPPER.writeValueAsString(ImmutableMap.of("enabled", enabled, "engine", defaultEngine)),
             CompactionSupervisorConfig.class
         );
-    Assert.assertEquals(new CompactionSupervisorConfig(enabled, defaultEngine), compactionSupervisorConfig);
+    CompactionSupervisorConfig expectedConfig = new CompactionSupervisorConfig(enabled, defaultEngine);
+    Assert.assertEquals(expectedConfig, compactionSupervisorConfig);
+    // Superfluous test to meet min branch coverage for CompactionSupervisorSpec.
+    Assert.assertNotEquals(expectedConfig, "enabled");

Review Comment:
   Fixed!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [PR] Allow MSQ engine only for compaction supervisors (druid)

Posted by "kfaraz (via GitHub)" <gi...@apache.org>.
kfaraz commented on code in PR #17033:
URL: https://github.com/apache/druid/pull/17033#discussion_r1772732705


##########
indexing-service/src/test/java/org/apache/druid/indexing/compact/OverlordCompactionSchedulerTest.java:
##########
@@ -158,7 +158,10 @@ public void testCompactionSupervisorConfigSerde() throws JsonProcessingException
             OBJECT_MAPPER.writeValueAsString(ImmutableMap.of("enabled", enabled, "engine", defaultEngine)),
             CompactionSupervisorConfig.class
         );
-    Assert.assertEquals(new CompactionSupervisorConfig(enabled, defaultEngine), compactionSupervisorConfig);
+    CompactionSupervisorConfig expectedConfig = new CompactionSupervisorConfig(enabled, defaultEngine);
+    Assert.assertEquals(expectedConfig, compactionSupervisorConfig);
+    // Superfluous test to meet min branch coverage for CompactionSupervisorSpec.
+    Assert.assertNotEquals(expectedConfig, "enabled");

Review Comment:
   Nit: Best to have this test in a separate method.
   Let's not call it superfluous as it does verify some code after all 😛 
   
   ```suggestion
       Assert.assertNotEquals(expectedConfig, new Object());
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [PR] Allow MSQ engine only for compaction supervisors (druid)

Posted by "kfaraz (via GitHub)" <gi...@apache.org>.
kfaraz merged PR #17033:
URL: https://github.com/apache/druid/pull/17033


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [PR] Allow MSQ engine only for compaction supervisors (druid)

Posted by "gargvishesh (via GitHub)" <gi...@apache.org>.
gargvishesh commented on code in PR #17033:
URL: https://github.com/apache/druid/pull/17033#discussion_r1771298639


##########
indexing-service/src/main/java/org/apache/druid/indexing/compact/CompactionSupervisor.java:
##########
@@ -76,6 +84,11 @@ public SupervisorReport<AutoCompactionSnapshot> getStatus()
       snapshot = AutoCompactionSnapshot.builder(dataSource)
                                        .withStatus(AutoCompactionSnapshot.AutoCompactionScheduleStatus.NOT_ENABLED)
                                        .build();
+    } else if (!supervisorSpec.getValidationResult().isValid()) {
+      throw InvalidInput.exception(
+          "Compaction supervisor spec is invalid. Reason[%s].",
+          supervisorSpec.getValidationResult().getReason()
+      );

Review Comment:
   Introduced an additional field in `AutoCompactionSnapshot`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [PR] Allow MSQ engine only for compaction supervisors (druid)

Posted by "gargvishesh (via GitHub)" <gi...@apache.org>.
gargvishesh commented on code in PR #17033:
URL: https://github.com/apache/druid/pull/17033#discussion_r1771301901


##########
indexing-service/src/test/java/org/apache/druid/indexing/compact/OverlordCompactionSchedulerTest.java:
##########
@@ -146,10 +148,23 @@ private void initScheduler()
     );
   }
 
+  @Test
+  public void testCompactionSupervisorConfigSerde() throws JsonProcessingException
+  {
+    boolean enabled = true;
+    CompactionEngine defaultEngine = CompactionEngine.MSQ;

Review Comment:
   Done



##########
indexing-service/src/test/java/org/apache/druid/indexing/compact/CompactionSupervisorSpecTest.java:
##########
@@ -82,20 +82,32 @@ public void testSerdeOfSuspendedSpec()
   }
 
   @Test
-  public void testInvalidSpecThrowsException()
+  public void testSupervisorWithInvalidSpecThrowsExceptionForStatus()
   {
     Mockito.when(scheduler.validateCompactionConfig(ArgumentMatchers.any()))
            .thenReturn(CompactionConfigValidationResult.failure("bad spec"));
     final DruidException exception = Assert.assertThrows(
         DruidException.class,
-        () -> new CompactionSupervisorSpec(null, false, scheduler)
+        () -> new CompactionSupervisorSpec(
+            new DataSourceCompactionConfig.Builder().forDataSource("datasource").build(),
+            false,
+            scheduler
+        ).createSupervisor().getStatus()
     );
     Assert.assertEquals(
-        "Compaction supervisor 'spec' is invalid. Reason[bad spec].",
+        "Compaction supervisor spec is invalid. Reason[bad spec].",
         exception.getMessage()
     );
   }
 
+  @Test
+  public void testInvalidSpecReturnsInvalid()

Review Comment:
   Done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [PR] Allow MSQ engine only for compaction supervisors (druid)

Posted by "kfaraz (via GitHub)" <gi...@apache.org>.
kfaraz commented on code in PR #17033:
URL: https://github.com/apache/druid/pull/17033#discussion_r1772732705


##########
indexing-service/src/test/java/org/apache/druid/indexing/compact/OverlordCompactionSchedulerTest.java:
##########
@@ -158,7 +158,10 @@ public void testCompactionSupervisorConfigSerde() throws JsonProcessingException
             OBJECT_MAPPER.writeValueAsString(ImmutableMap.of("enabled", enabled, "engine", defaultEngine)),
             CompactionSupervisorConfig.class
         );
-    Assert.assertEquals(new CompactionSupervisorConfig(enabled, defaultEngine), compactionSupervisorConfig);
+    CompactionSupervisorConfig expectedConfig = new CompactionSupervisorConfig(enabled, defaultEngine);
+    Assert.assertEquals(expectedConfig, compactionSupervisorConfig);
+    // Superfluous test to meet min branch coverage for CompactionSupervisorSpec.
+    Assert.assertNotEquals(expectedConfig, "enabled");

Review Comment:
   Nit: Best to have this test in a separate method.
   Let's not call it superfluous as it does verify some code after all 😛 
   
   ```suggestion
       Assert.assertNotEquals(new CompactionSupervisorConfig(null, null), new Object());
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [PR] Allow MSQ engine only for compaction supervisors (druid)

Posted by "kfaraz (via GitHub)" <gi...@apache.org>.
kfaraz commented on code in PR #17033:
URL: https://github.com/apache/druid/pull/17033#discussion_r1770732466


##########
indexing-service/src/main/java/org/apache/druid/indexing/compact/CompactionSupervisor.java:
##########
@@ -55,6 +56,13 @@ public void start()
     if (supervisorSpec.isSuspended()) {
       log.info("Suspending compaction for dataSource[%s].", dataSource);
       scheduler.stopCompaction(dataSource);
+    } else if (!supervisorSpec.getValidationResult().isValid()) {
+      log.error(
+          "Failed to start compaction supervisor for datasource[%s] due to invalid compaction supervisor spec. "

Review Comment:
   ```suggestion
             "Cannot start compaction supervisor for datasource[%s] since the compaction supervisor spec is invalid. "
   ```



##########
indexing-service/src/main/java/org/apache/druid/indexing/compact/CompactionSupervisor.java:
##########
@@ -76,6 +84,11 @@ public SupervisorReport<AutoCompactionSnapshot> getStatus()
       snapshot = AutoCompactionSnapshot.builder(dataSource)
                                        .withStatus(AutoCompactionSnapshot.AutoCompactionScheduleStatus.NOT_ENABLED)
                                        .build();
+    } else if (!supervisorSpec.getValidationResult().isValid()) {
+      throw InvalidInput.exception(
+          "Compaction supervisor spec is invalid. Reason[%s].",
+          supervisorSpec.getValidationResult().getReason()
+      );

Review Comment:
   Let's not throw an exception here. We can either add new fields to `AutoCompactionSnapshot` or add a new class that is returned by this method.



##########
indexing-service/src/main/java/org/apache/druid/indexing/compact/CompactionSupervisor.java:
##########
@@ -55,6 +56,13 @@ public void start()
     if (supervisorSpec.isSuspended()) {
       log.info("Suspending compaction for dataSource[%s].", dataSource);
       scheduler.stopCompaction(dataSource);
+    } else if (!supervisorSpec.getValidationResult().isValid()) {
+      log.error(

Review Comment:
   ```suggestion
         log.warn(
   ```



##########
server/src/main/java/org/apache/druid/server/http/CoordinatorCompactionConfigsResource.java:
##########
@@ -161,12 +135,11 @@ public Response addOrUpdateDatasourceCompactionConfig(
   )
   {
     UnaryOperator<DruidCompactionConfig> callable = current -> {
-      CompactionConfigValidationResult validationResult =
-          ClientCompactionRunnerInfo.validateCompactionConfig(newConfig, current.getEngine());
-      if (validationResult.isValid()) {
-        return current.withDatasourceConfig(newConfig);
+      if (newConfig.getEngine() == CompactionEngine.MSQ) {
+        throw InvalidInput.exception(
+            "MSQ engine in compaction config only supported with supervisor-based compaction on the Overlord.");

Review Comment:
   ```suggestion
               "MSQ-based compaction is supported only with Compaction Supervisors.");
   ```



##########
server/src/main/java/org/apache/druid/server/coordinator/CompactionSupervisorConfig.java:
##########
@@ -33,10 +34,12 @@
  */
 public class CompactionSupervisorConfig
 {
-  private static final CompactionSupervisorConfig DEFAULT = new CompactionSupervisorConfig(null);
+  private static final CompactionSupervisorConfig DEFAULT = new CompactionSupervisorConfig(null, null);
 
   @JsonProperty
   private final boolean enabled;
+  @JsonProperty
+  private final CompactionEngine defaultEngine;

Review Comment:
   Nit: Maybe we should call this property just `engine`.



##########
indexing-service/src/test/java/org/apache/druid/indexing/compact/OverlordCompactionSchedulerTest.java:
##########
@@ -146,10 +148,23 @@ private void initScheduler()
     );
   }
 
+  @Test
+  public void testCompactionSupervisorConfigSerde() throws JsonProcessingException
+  {
+    boolean enabled = true;
+    CompactionEngine defaultEngine = CompactionEngine.MSQ;

Review Comment:
   Nit: Can be final.



##########
indexing-service/src/main/java/org/apache/druid/indexing/compact/CompactionSupervisorSpec.java:
##########
@@ -77,6 +73,11 @@ public String getId()
     return ID_PREFIX + spec.getDataSource();
   }
 
+  public CompactionConfigValidationResult getValidationResult()

Review Comment:
   Please confirm that this doesn't get serialized out. If it does, let's add a `@JsonIgnore`.



##########
server/src/main/java/org/apache/druid/client/indexing/ClientCompactionRunnerInfo.java:
##########
@@ -135,9 +135,8 @@ public static CompactionConfigValidationResult validatePartitionsSpecForMSQ(Part
     if (!(partitionsSpec instanceof DimensionRangePartitionsSpec
           || partitionsSpec instanceof DynamicPartitionsSpec)) {
       return CompactionConfigValidationResult.failure(
-          "MSQ: Invalid partitioning type[%s]. Must be either 'dynamic' or 'range'",
+          "MSQ: Invalid partitioning type[%s]. Must be either dynamic or range",

Review Comment:
   I think having the quotes was better. You can even use square brackets instead of quotes.



##########
indexing-service/src/test/java/org/apache/druid/indexing/compact/CompactionSupervisorSpecTest.java:
##########
@@ -82,20 +82,32 @@ public void testSerdeOfSuspendedSpec()
   }
 
   @Test
-  public void testInvalidSpecThrowsException()
+  public void testSupervisorWithInvalidSpecThrowsExceptionForStatus()
   {
     Mockito.when(scheduler.validateCompactionConfig(ArgumentMatchers.any()))
            .thenReturn(CompactionConfigValidationResult.failure("bad spec"));
     final DruidException exception = Assert.assertThrows(
         DruidException.class,
-        () -> new CompactionSupervisorSpec(null, false, scheduler)
+        () -> new CompactionSupervisorSpec(
+            new DataSourceCompactionConfig.Builder().forDataSource("datasource").build(),
+            false,
+            scheduler
+        ).createSupervisor().getStatus()
     );
     Assert.assertEquals(
-        "Compaction supervisor 'spec' is invalid. Reason[bad spec].",
+        "Compaction supervisor spec is invalid. Reason[bad spec].",
         exception.getMessage()
     );
   }
 
+  @Test
+  public void testInvalidSpecReturnsInvalid()
+  {
+    Mockito.when(scheduler.validateCompactionConfig(ArgumentMatchers.any()))
+           .thenReturn(CompactionConfigValidationResult.failure("bad spec"));
+    Assert.assertFalse(new CompactionSupervisorSpec(null, false, scheduler).getValidationResult().isValid());

Review Comment:
   Rather than just checking `isValid()`, verify the whole validation result.



##########
indexing-service/src/test/java/org/apache/druid/indexing/compact/CompactionSupervisorSpecTest.java:
##########
@@ -82,20 +82,32 @@ public void testSerdeOfSuspendedSpec()
   }
 
   @Test
-  public void testInvalidSpecThrowsException()
+  public void testSupervisorWithInvalidSpecThrowsExceptionForStatus()
   {
     Mockito.when(scheduler.validateCompactionConfig(ArgumentMatchers.any()))
            .thenReturn(CompactionConfigValidationResult.failure("bad spec"));
     final DruidException exception = Assert.assertThrows(
         DruidException.class,
-        () -> new CompactionSupervisorSpec(null, false, scheduler)
+        () -> new CompactionSupervisorSpec(
+            new DataSourceCompactionConfig.Builder().forDataSource("datasource").build(),
+            false,
+            scheduler
+        ).createSupervisor().getStatus()
     );
     Assert.assertEquals(
-        "Compaction supervisor 'spec' is invalid. Reason[bad spec].",
+        "Compaction supervisor spec is invalid. Reason[bad spec].",
         exception.getMessage()
     );
   }
 
+  @Test
+  public void testInvalidSpecReturnsInvalid()

Review Comment:
   ```suggestion
     public void testGetValidationResultForInvalidSpec()
   ```



##########
indexing-service/src/test/java/org/apache/druid/indexing/compact/CompactionSupervisorSpecTest.java:
##########
@@ -82,20 +82,32 @@ public void testSerdeOfSuspendedSpec()
   }
 
   @Test
-  public void testInvalidSpecThrowsException()
+  public void testSupervisorWithInvalidSpecThrowsExceptionForStatus()

Review Comment:
   This test might have to change a little as we shouldn't throw an exception while getting state/status even if the spec is invalid.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [PR] Allow MSQ engine only for compaction supervisors (druid)

Posted by "kfaraz (via GitHub)" <gi...@apache.org>.
kfaraz commented on code in PR #17033:
URL: https://github.com/apache/druid/pull/17033#discussion_r1772732705


##########
indexing-service/src/test/java/org/apache/druid/indexing/compact/OverlordCompactionSchedulerTest.java:
##########
@@ -158,7 +158,10 @@ public void testCompactionSupervisorConfigSerde() throws JsonProcessingException
             OBJECT_MAPPER.writeValueAsString(ImmutableMap.of("enabled", enabled, "engine", defaultEngine)),
             CompactionSupervisorConfig.class
         );
-    Assert.assertEquals(new CompactionSupervisorConfig(enabled, defaultEngine), compactionSupervisorConfig);
+    CompactionSupervisorConfig expectedConfig = new CompactionSupervisorConfig(enabled, defaultEngine);
+    Assert.assertEquals(expectedConfig, compactionSupervisorConfig);
+    // Superfluous test to meet min branch coverage for CompactionSupervisorSpec.
+    Assert.assertNotEquals(expectedConfig, "enabled");

Review Comment:
   Nit: Best to have this test in a separate method.
   Let's not call it superfluous as it does verify some code after all 😛 
   
   ```suggestion
       Assert.assertNotEquals(expectedConfig, "enabled");
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [PR] Allow MSQ engine only for compaction supervisors (druid)

Posted by "gargvishesh (via GitHub)" <gi...@apache.org>.
gargvishesh commented on code in PR #17033:
URL: https://github.com/apache/druid/pull/17033#discussion_r1771302337


##########
indexing-service/src/test/java/org/apache/druid/indexing/compact/CompactionSupervisorSpecTest.java:
##########
@@ -82,20 +82,32 @@ public void testSerdeOfSuspendedSpec()
   }
 
   @Test
-  public void testInvalidSpecThrowsException()
+  public void testSupervisorWithInvalidSpecThrowsExceptionForStatus()
   {
     Mockito.when(scheduler.validateCompactionConfig(ArgumentMatchers.any()))
            .thenReturn(CompactionConfigValidationResult.failure("bad spec"));
     final DruidException exception = Assert.assertThrows(
         DruidException.class,
-        () -> new CompactionSupervisorSpec(null, false, scheduler)
+        () -> new CompactionSupervisorSpec(
+            new DataSourceCompactionConfig.Builder().forDataSource("datasource").build(),
+            false,
+            scheduler
+        ).createSupervisor().getStatus()
     );
     Assert.assertEquals(
-        "Compaction supervisor 'spec' is invalid. Reason[bad spec].",
+        "Compaction supervisor spec is invalid. Reason[bad spec].",
         exception.getMessage()
     );
   }
 
+  @Test
+  public void testInvalidSpecReturnsInvalid()
+  {
+    Mockito.when(scheduler.validateCompactionConfig(ArgumentMatchers.any()))
+           .thenReturn(CompactionConfigValidationResult.failure("bad spec"));
+    Assert.assertFalse(new CompactionSupervisorSpec(null, false, scheduler).getValidationResult().isValid());

Review Comment:
   Added the message check.



##########
indexing-service/src/test/java/org/apache/druid/indexing/compact/CompactionSupervisorSpecTest.java:
##########
@@ -82,20 +82,32 @@ public void testSerdeOfSuspendedSpec()
   }
 
   @Test
-  public void testInvalidSpecThrowsException()
+  public void testSupervisorWithInvalidSpecThrowsExceptionForStatus()

Review Comment:
   Updated.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org