You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "gongxuanzhang (via GitHub)" <gi...@apache.org> on 2024/06/15 07:01:45 UTC

[PR] KAFKA-16921: Migrate test of connect module to Junit5 (Runtime direct) [kafka]

gongxuanzhang opened a new pull request, #16351:
URL: https://github.com/apache/kafka/pull/16351

   This PR is part of a task submodule, with the purpose of migrating from JUnit 4 to JUnit 5.
   see 
   * https://github.com/apache/kafka/pull/16253
   * https://github.com/apache/kafka/pull/16330
   * https://github.com/apache/kafka/pull/16328
   
   In this PR,AbstractWorkerSourceTaskTest  test case has error.maybe other modules are involved.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16921: Migrate test of connect module to Junit5 (Runtime direct) [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 merged PR #16351:
URL: https://github.com/apache/kafka/pull/16351


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16921: Migrate test of connect module to Junit5 (Runtime direct) [kafka]

Posted by "gongxuanzhang (via GitHub)" <gi...@apache.org>.
gongxuanzhang commented on PR #16351:
URL: https://github.com/apache/kafka/pull/16351#issuecomment-2174753090

   I update it. plz take a look @chia7712 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16921: Migrate test of connect module to Junit5 (Runtime direct) [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #16351:
URL: https://github.com/apache/kafka/pull/16351#discussion_r1642956855


##########
connect/runtime/src/test/java/org/apache/kafka/connect/runtime/WorkerSinkTaskThreadedTest.java:
##########
@@ -643,7 +647,11 @@ public Object answer(InvocationOnMock invocation) {
                 }
             }
         }).when(sinkTask).preCommit(anyMap());
-
+        //  if the only command has error, consumer don't commit, so we don't need to mock commitAsync.

Review Comment:
   Could we have two methods? 
   
   ```java
       private void expectPreCommit(ExpectOffsetCommitCommand... commands) {
           doAnswer(new Answer<Object>() {
               int index = 0;
   
               @Override
               public Object answer(InvocationOnMock invocation) {
                   ExpectOffsetCommitCommand commitCommand = commands[index++];
                   // All assigned partitions will have offsets committed, but we've only processed messages/updated offsets for one
                   final Map<TopicPartition, OffsetAndMetadata> offsetsToCommit = offsetsToCommitFn.apply(commitCommand.expectedMessages);
   
                   if (commitCommand.error != null) {
                       throw commitCommand.error;
                   } else {
                       return offsetsToCommit;
                   }
               }
           }).when(sinkTask).preCommit(anyMap());
       }
   
       private void expectOffsetCommit(ExpectOffsetCommitCommand... commands) {
           expectPreCommit(commands);
           ...
      }
   ```
   
   with that changes, each test case can call specific "expect"



##########
connect/runtime/src/test/java/org/apache/kafka/connect/runtime/WorkerSourceTaskTest.java:
##########
@@ -190,19 +187,10 @@ public class WorkerSourceTaskTest {
             new SourceRecord(PARTITION, OFFSET, "topic", null, KEY_SCHEMA, KEY, RECORD_SCHEMA, RECORD)
     );
 
-    private final boolean enableTopicCreation;
-
-    @ParameterizedTest.Parameters
-    public static Collection<Boolean> parameters() {
-        return Arrays.asList(false, true);
-    }
+    private boolean enableTopicCreation;

Review Comment:
   we should remove this, and each test case should call `expectTopicCreation` when `enableTopicCreation=true`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16921: Migrate test of connect module to Junit5 (Runtime direct) [kafka]

Posted by "gongxuanzhang (via GitHub)" <gi...@apache.org>.
gongxuanzhang commented on PR #16351:
URL: https://github.com/apache/kafka/pull/16351#issuecomment-2171052932

   @chia7712  I update it 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16921: Migrate test of connect module to Junit5 (Runtime direct) [kafka]

Posted by "gongxuanzhang (via GitHub)" <gi...@apache.org>.
gongxuanzhang commented on PR #16351:
URL: https://github.com/apache/kafka/pull/16351#issuecomment-2172045859

   plz take a look @chia7712 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16921: Migrate test of connect module to Junit5 (Runtime direct) [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #16351:
URL: https://github.com/apache/kafka/pull/16351#discussion_r1641593418


##########
connect/runtime/src/test/java/org/apache/kafka/connect/runtime/ExactlyOnceWorkerSourceTaskTest.java:
##########
@@ -293,8 +287,10 @@ private void createWorkerTask(TargetState initialState, Converter keyConverter,
                 sourceConfig, Runnable::run, preProducerCheck, postProducerCheck, Collections::emptyList);
     }
 
-    @Test
-    public void testRemoveMetrics() {
+    @ParameterizedTest
+    @MethodSource("parameters")

Review Comment:
   Could you please use `@ValueSource(booleans = {true, false})` instead? that is more readable to me



##########
connect/runtime/src/test/java/org/apache/kafka/connect/runtime/ExactlyOnceWorkerSourceTaskTest.java:
##########
@@ -333,8 +331,10 @@ public void testStartPaused() throws Exception {
         assertPollMetrics(0);
     }
 
-    @Test
-    public void testPause() throws Exception {
+    @ParameterizedTest
+    @MethodSource("parameters")

Review Comment:
   ditto



##########
connect/runtime/src/test/java/org/apache/kafka/connect/runtime/ExactlyOnceWorkerSourceTaskTest.java:
##########
@@ -310,8 +306,10 @@ private Set<MetricName> filterToTaskMetrics(Set<MetricName> metricNames) {
                 .collect(Collectors.toSet());
     }
 
-    @Test
-    public void testStartPaused() throws Exception {
+    @ParameterizedTest
+    @MethodSource("parameters")

Review Comment:
   ditto



##########
connect/runtime/src/test/java/org/apache/kafka/connect/runtime/ExactlyOnceWorkerSourceTaskTest.java:
##########
@@ -372,8 +372,10 @@ public void testPause() throws Exception {
         verifyCleanShutdown();
     }
 
-    @Test
-    public void testFailureInPreProducerCheck() throws Exception {
+    @ParameterizedTest
+    @MethodSource("parameters")

Review Comment:
   ditto



##########
connect/runtime/src/test/java/org/apache/kafka/connect/runtime/ExactlyOnceWorkerSourceTaskTest.java:
##########
@@ -386,8 +388,10 @@ public void testFailureInPreProducerCheck() throws Exception {
         verifyShutdown(true, false);
     }
 
-    @Test
-    public void testFailureInProducerInitialization() throws Exception {
+    @ParameterizedTest

Review Comment:
   ditto



##########
connect/runtime/src/test/java/org/apache/kafka/connect/runtime/WorkerSourceTaskTest.java:
##########
@@ -355,8 +353,10 @@ public void testPollsInBackground() throws Exception {
         verifyClose();
     }
 
-    @Test
-    public void testFailureInPoll() throws Exception {
+    @ParameterizedTest
+    @MethodSource("parameters")

Review Comment:
   ditto



##########
connect/runtime/src/test/java/org/apache/kafka/connect/runtime/WorkerConnectorTest.java:
##########
@@ -429,8 +446,10 @@ public void testShutdownFailure() {
         verifyShutdown(false, true);
     }
 
-    @Test
-    public void testTransitionStartedToStarted() {
+    @ParameterizedTest
+    @MethodSource("parameters")

Review Comment:
   Could you please use `@EnumSource(value = ConnectorType.class, names = {"SOURCE", "SINK"})` instead? it is more readable



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16921: Migrate test of connect module to Junit5 (Runtime direct) [kafka]

Posted by "gongxuanzhang (via GitHub)" <gi...@apache.org>.
gongxuanzhang commented on PR #16351:
URL: https://github.com/apache/kafka/pull/16351#issuecomment-2172997846

   I have fixed all the test cases that failed due to the changes.
   @chia7712 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16921: Migrate test of connect module to Junit5 (Runtime direct) [kafka]

Posted by "gongxuanzhang (via GitHub)" <gi...@apache.org>.
gongxuanzhang commented on PR #16351:
URL: https://github.com/apache/kafka/pull/16351#issuecomment-2171121243

   @chia7712  I update it 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16921: Migrate test of connect module to Junit5 (Runtime direct) [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #16351:
URL: https://github.com/apache/kafka/pull/16351#discussion_r1642052004


##########
connect/runtime/src/test/java/org/apache/kafka/connect/runtime/ErrorHandlingTaskTest.java:
##########
@@ -179,19 +176,15 @@ public class ErrorHandlingTaskTest {
 
     private ErrorHandlingMetrics errorHandlingMetrics;
 
-    private final boolean enableTopicCreation;
+    private boolean enableTopicCreation;
 
-    @Parameterized.Parameters
     public static Collection<Boolean> parameters() {

Review Comment:
   this is unused now, right?



##########
connect/runtime/src/test/java/org/apache/kafka/connect/runtime/ExactlyOnceWorkerSourceTaskTest.java:
##########
@@ -174,23 +171,20 @@ public class ExactlyOnceWorkerSourceTaskTest {
     private final AtomicReference<CountDownLatch> pollLatch = new AtomicReference<>(new CountDownLatch(0));
     private final AtomicReference<List<SourceRecord>> pollRecords = new AtomicReference<>(RECORDS);
 
-    private final boolean enableTopicCreation;
+    private boolean enableTopicCreation;
 
     private boolean taskStarted;
     private Future<?> workerTaskFuture;
 
-    @ParameterizedTest.Parameters
-    public static Collection<Boolean> parameters() {
-        return Arrays.asList(false, true);
+    
+    public static Stream<Boolean> parameters() {

Review Comment:
   ditto



##########
connect/runtime/src/test/java/org/apache/kafka/connect/runtime/WorkerConnectorTest.java:
##########
@@ -77,25 +75,22 @@ public class WorkerConnectorTest {
     public ConnectorConfig connectorConfig;
     public MockConnectMetrics metrics;
 
-    @Rule
-    public MockitoRule rule = MockitoJUnit.rule().strictness(Strictness.STRICT_STUBS);
 
     @Mock private Plugins plugins;
     @Mock private CloseableConnectorContext ctx;
     @Mock private ConnectorStatus.Listener listener;
     @Mock private ClassLoader classLoader;
 
-    private final ConnectorType connectorType;
-    private final Connector connector;
-    private final CloseableOffsetStorageReader offsetStorageReader;
-    private final ConnectorOffsetBackingStore offsetStore;
+    private ConnectorType connectorType;
+    private Connector connector;
+    private CloseableOffsetStorageReader offsetStorageReader;
+    private ConnectorOffsetBackingStore offsetStore;
 
-    @Parameterized.Parameters
     public static Collection<ConnectorType> parameters() {

Review Comment:
   ditto



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-16921: Migrate test of connect module to Junit5 (Runtime direct) [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on PR #16351:
URL: https://github.com/apache/kafka/pull/16351#issuecomment-2172601878

   @gongxuanzhang please fix those failed tests


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org