You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "muralibasani (via GitHub)" <gi...@apache.org> on 2026/05/10 05:41:05 UTC

[PR] KAFKA-20553 : Eliminate the dependencies on sourceSets.test.output for metadata [kafka]

muralibasani opened a new pull request, #22247:
URL: https://github.com/apache/kafka/pull/22247

   - Move ControllerRequestContextUtil.java, MockAclMutator.java and RecordTestUtils.java to testFixtures
   - Had to extract StandardAuthorizerPropertyTest.java from StandardAuthorizerTest.java as that's an innner class
   - updated build.gradle testFixtures(project(':metadata'))


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20553 : Eliminate the dependencies on sourceSets.test.output for metadata [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #22247:
URL: https://github.com/apache/kafka/pull/22247#discussion_r3220111581


##########
metadata/src/testFixtures/java/org/apache/kafka/image/ClusterImageFixtures.java:
##########
@@ -0,0 +1,228 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.image;
+
+import org.apache.kafka.common.Endpoint;
+import org.apache.kafka.common.Uuid;
+import org.apache.kafka.common.metadata.BrokerRegistrationChangeRecord;
+import org.apache.kafka.common.metadata.FenceBrokerRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpoint;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpointCollection;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerFeature;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerFeatureCollection;
+import org.apache.kafka.common.metadata.RegisterControllerRecord;
+import org.apache.kafka.common.metadata.RegisterControllerRecord.ControllerEndpoint;
+import org.apache.kafka.common.metadata.RegisterControllerRecord.ControllerEndpointCollection;
+import org.apache.kafka.common.metadata.UnfenceBrokerRecord;
+import org.apache.kafka.common.metadata.UnregisterBrokerRecord;
+import org.apache.kafka.common.security.auth.SecurityProtocol;
+import org.apache.kafka.metadata.BrokerRegistration;
+import org.apache.kafka.metadata.BrokerRegistrationInControlledShutdownChange;
+import org.apache.kafka.metadata.ControllerRegistration;
+import org.apache.kafka.metadata.RecordTestUtils;
+import org.apache.kafka.metadata.VersionRange;
+import org.apache.kafka.server.common.ApiMessageAndVersion;
+import org.apache.kafka.server.common.MetadataVersion;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+
+import static org.apache.kafka.common.metadata.MetadataRecordType.BROKER_REGISTRATION_CHANGE_RECORD;
+import static org.apache.kafka.common.metadata.MetadataRecordType.FENCE_BROKER_RECORD;
+import static org.apache.kafka.common.metadata.MetadataRecordType.REGISTER_BROKER_RECORD;
+import static org.apache.kafka.common.metadata.MetadataRecordType.UNFENCE_BROKER_RECORD;
+
+public final class ClusterImageFixtures {
+    public static final ClusterImage IMAGE1;
+    public static final List<ApiMessageAndVersion> DELTA1_RECORDS;
+    public static final ClusterDelta DELTA1;
+    public static final ClusterImage IMAGE2;
+    public static final List<ApiMessageAndVersion> DELTA2_RECORDS;
+    public static final ClusterDelta DELTA2;
+    public static final ClusterImage IMAGE3;
+
+    static {

Review Comment:
   Could you please eliminate the static block?
   ```java
   public final class ClusterImageFixtures {
   
       public static final ClusterImage IMAGE1 = new ClusterImage(
               Map.of(
                       0, new BrokerRegistration.Builder().
                               setId(0).
                               setEpoch(1000).
                               setIncarnationId(Uuid.fromString("vZKYST0pSA2HO5x_6hoO2Q")).
                               setListeners(List.of(new Endpoint("PLAINTEXT", SecurityProtocol.PLAINTEXT, "localhost", 9092))).
                               setSupportedFeatures(Map.of("foo", VersionRange.of((short) 1, (short) 3))).
                               setRack(Optional.empty()).
                               setFenced(true).
                               setInControlledShutdown(false).build(),
                       1, new BrokerRegistration.Builder().
                               setId(1).
                               setEpoch(1001).
                               setIncarnationId(Uuid.fromString("U52uRe20RsGI0RvpcTx33Q")).
                               setListeners(List.of(new Endpoint("PLAINTEXT", SecurityProtocol.PLAINTEXT, "localhost", 9093))).
                               setSupportedFeatures(Map.of("foo", VersionRange.of((short) 1, (short) 3))).
                               setRack(Optional.empty()).
                               setFenced(false).
                               setInControlledShutdown(false).build(),
                       2, new BrokerRegistration.Builder().
                               setId(2).
                               setEpoch(123).
                               setIncarnationId(Uuid.fromString("hr4TVh3YQiu3p16Awkka6w")).
                               setListeners(List.of(new Endpoint("PLAINTEXT", SecurityProtocol.PLAINTEXT, "localhost", 9094))).
                               setSupportedFeatures(Map.of()).
                               setRack(Optional.of("arack")).
                               setFenced(false).
                               setInControlledShutdown(false).build()
               ),
               Map.of(
                       1000, new ControllerRegistration.Builder().
                               setId(1000).
                               setIncarnationId(Uuid.fromString("9ABu6HEgRuS-hjHLgC4cHw")).
                               setZkMigrationReady(false).
                               setListeners(Map.of("PLAINTEXT", new Endpoint("PLAINTEXT", SecurityProtocol.PLAINTEXT, "localhost", 19092))).
                               setSupportedFeatures(Map.of()).build()
               )
       );
   
       public static final List<ApiMessageAndVersion> DELTA1_RECORDS = List.of(
               new ApiMessageAndVersion(new UnfenceBrokerRecord().setId(0).setEpoch(1000), UNFENCE_BROKER_RECORD.highestSupportedVersion()),
               new ApiMessageAndVersion(new FenceBrokerRecord().setId(1).setEpoch(1001), FENCE_BROKER_RECORD.highestSupportedVersion()),
               new ApiMessageAndVersion(new BrokerRegistrationChangeRecord().setBrokerId(0).setBrokerEpoch(1000)
                       .setInControlledShutdown(BrokerRegistrationInControlledShutdownChange.IN_CONTROLLED_SHUTDOWN.value()),
                       BROKER_REGISTRATION_CHANGE_RECORD.highestSupportedVersion()),
               new ApiMessageAndVersion(new UnregisterBrokerRecord().setBrokerId(2).setBrokerEpoch(123), (short) 0),
               new ApiMessageAndVersion(new RegisterControllerRecord().setControllerId(1001).setIncarnationId(Uuid.fromString("FdEHF-IqScKfYyjZ1CjfNQ"))
                       .setZkMigrationReady(true).setEndPoints(new ControllerEndpointCollection(
                               List.of(new ControllerEndpoint().setHost("localhost").setName("PLAINTEXT").setPort(19093).setSecurityProtocol(SecurityProtocol.PLAINTEXT.id)).iterator()
                       )), (short) 0)
       );
   
       public static final ClusterDelta DELTA1 = RecordTestUtils.replayAll(new ClusterDelta(IMAGE1), DELTA1_RECORDS);
   
       public static final ClusterImage IMAGE2 = new ClusterImage(
               Map.of(
                       0, new BrokerRegistration.Builder().
                               setId(0).
                               setEpoch(1000).
                               setIncarnationId(Uuid.fromString("vZKYST0pSA2HO5x_6hoO2Q")).
                               setListeners(List.of(new Endpoint("PLAINTEXT", SecurityProtocol.PLAINTEXT, "localhost", 9092))).
                               setSupportedFeatures(Map.of("foo", VersionRange.of((short) 1, (short) 3))).
                               setRack(Optional.empty()).
                               setFenced(false).
                               setInControlledShutdown(true).build(),
                       1, new BrokerRegistration.Builder().
                               setId(1).
                               setEpoch(1001).
                               setIncarnationId(Uuid.fromString("U52uRe20RsGI0RvpcTx33Q")).
                               setListeners(List.of(new Endpoint("PLAINTEXT", SecurityProtocol.PLAINTEXT, "localhost", 9093))).
                               setSupportedFeatures(Map.of("foo", VersionRange.of((short) 1, (short) 3))).
                               setRack(Optional.empty()).
                               setFenced(true).
                               setInControlledShutdown(false).build()
               ),
               Map.of(
                       1000, IMAGE1.controllers().get(1000),
                       1001, new ControllerRegistration.Builder().
                               setId(1001).
                               setIncarnationId(Uuid.fromString("FdEHF-IqScKfYyjZ1CjfNQ")).
                               setZkMigrationReady(true).
                               setListeners(Map.of("PLAINTEXT", new Endpoint("PLAINTEXT", SecurityProtocol.PLAINTEXT, "localhost", 19093))).
                               setSupportedFeatures(Map.of()).build()
               )
       );
   
       public static final List<ApiMessageAndVersion> DELTA2_RECORDS = Stream.concat(
               DELTA1_RECORDS.stream(),
               Stream.of(
                       new ApiMessageAndVersion(new FenceBrokerRecord().setId(0).setEpoch(1000), FENCE_BROKER_RECORD.highestSupportedVersion()),
                       new ApiMessageAndVersion(new UnfenceBrokerRecord().setId(1).setEpoch(1001), UNFENCE_BROKER_RECORD.highestSupportedVersion()),
                       new ApiMessageAndVersion(new BrokerRegistrationChangeRecord().setBrokerId(0).setBrokerEpoch(1000)
                               .setInControlledShutdown(BrokerRegistrationInControlledShutdownChange.NONE.value()),
                               BROKER_REGISTRATION_CHANGE_RECORD.highestSupportedVersion()),
                       new ApiMessageAndVersion(new RegisterBrokerRecord().setBrokerId(2).setIsMigratingZkBroker(true)
                               .setIncarnationId(Uuid.fromString("Am5Yse7GQxaw0b2alM74bP")).setBrokerEpoch(1002)
                               .setEndPoints(new BrokerEndpointCollection(
                                       List.of(new BrokerEndpoint().setName("PLAINTEXT").setHost("localhost").setPort(9094).setSecurityProtocol((short) 0)).iterator()
                               ))
                               .setFeatures(new BrokerFeatureCollection(
                                       Set.of(new BrokerFeature().setName(MetadataVersion.FEATURE_NAME)
                                               .setMinSupportedVersion(MetadataVersion.MINIMUM_VERSION.featureLevel())
                                               .setMaxSupportedVersion(MetadataVersion.IBP_3_6_IV0.featureLevel())).iterator()
                               )).setRack("rack3"), REGISTER_BROKER_RECORD.highestSupportedVersion())
               )
       ).toList();
   
       public static final ClusterDelta DELTA2 = RecordTestUtils.replayAll(new ClusterDelta(IMAGE2), DELTA2_RECORDS);
   
       public static final ClusterImage IMAGE3 = new ClusterImage(
               Map.of(
                       0, new BrokerRegistration.Builder().
                               setId(0).
                               setEpoch(1000).
                               setIncarnationId(Uuid.fromString("vZKYST0pSA2HO5x_6hoO2Q")).
                               setListeners(List.of(new Endpoint("PLAINTEXT", SecurityProtocol.PLAINTEXT, "localhost", 9092))).
                               setSupportedFeatures(Map.of("foo", VersionRange.of((short) 1, (short) 3))).
                               setRack(Optional.empty()).
                               setFenced(true).
                               setInControlledShutdown(true).build(),
                       1, new BrokerRegistration.Builder().
                               setId(1).
                               setEpoch(1001).
                               setIncarnationId(Uuid.fromString("U52uRe20RsGI0RvpcTx33Q")).
                               setListeners(List.of(new Endpoint("PLAINTEXT", SecurityProtocol.PLAINTEXT, "localhost", 9093))).
                               setSupportedFeatures(Map.of("foo", VersionRange.of((short) 1, (short) 3))).
                               setRack(Optional.empty()).
                               setFenced(false).
                               setInControlledShutdown(false).build(),
                       2, new BrokerRegistration.Builder().
                               setId(2).
                               setEpoch(1002).
                               setIncarnationId(Uuid.fromString("Am5Yse7GQxaw0b2alM74bP")).
                               setListeners(List.of(new Endpoint("PLAINTEXT", SecurityProtocol.PLAINTEXT, "localhost", 9094))).
                               setSupportedFeatures(Map.of("metadata.version",
                                       VersionRange.of(MetadataVersion.MINIMUM_VERSION.featureLevel(), MetadataVersion.IBP_3_6_IV0.featureLevel()))).
                               setRack(Optional.of("rack3")).
                               setFenced(true).
                               setIsMigratingZkBroker(true).build()
               ),
               IMAGE2.controllers()
       );
   
       private ClusterImageFixtures() {
       }
   }
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20553 : Eliminate the dependencies on sourceSets.test.output for metadata [kafka]

Posted by "muralibasani (via GitHub)" <gi...@apache.org>.
muralibasani commented on code in PR #22247:
URL: https://github.com/apache/kafka/pull/22247#discussion_r3220402709


##########
metadata/src/testFixtures/java/org/apache/kafka/image/ClusterImageFixtures.java:
##########
@@ -0,0 +1,228 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.image;
+
+import org.apache.kafka.common.Endpoint;
+import org.apache.kafka.common.Uuid;
+import org.apache.kafka.common.metadata.BrokerRegistrationChangeRecord;
+import org.apache.kafka.common.metadata.FenceBrokerRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpoint;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerEndpointCollection;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerFeature;
+import org.apache.kafka.common.metadata.RegisterBrokerRecord.BrokerFeatureCollection;
+import org.apache.kafka.common.metadata.RegisterControllerRecord;
+import org.apache.kafka.common.metadata.RegisterControllerRecord.ControllerEndpoint;
+import org.apache.kafka.common.metadata.RegisterControllerRecord.ControllerEndpointCollection;
+import org.apache.kafka.common.metadata.UnfenceBrokerRecord;
+import org.apache.kafka.common.metadata.UnregisterBrokerRecord;
+import org.apache.kafka.common.security.auth.SecurityProtocol;
+import org.apache.kafka.metadata.BrokerRegistration;
+import org.apache.kafka.metadata.BrokerRegistrationInControlledShutdownChange;
+import org.apache.kafka.metadata.ControllerRegistration;
+import org.apache.kafka.metadata.RecordTestUtils;
+import org.apache.kafka.metadata.VersionRange;
+import org.apache.kafka.server.common.ApiMessageAndVersion;
+import org.apache.kafka.server.common.MetadataVersion;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+
+import static org.apache.kafka.common.metadata.MetadataRecordType.BROKER_REGISTRATION_CHANGE_RECORD;
+import static org.apache.kafka.common.metadata.MetadataRecordType.FENCE_BROKER_RECORD;
+import static org.apache.kafka.common.metadata.MetadataRecordType.REGISTER_BROKER_RECORD;
+import static org.apache.kafka.common.metadata.MetadataRecordType.UNFENCE_BROKER_RECORD;
+
+public final class ClusterImageFixtures {
+    public static final ClusterImage IMAGE1;
+    public static final List<ApiMessageAndVersion> DELTA1_RECORDS;
+    public static final ClusterDelta DELTA1;
+    public static final ClusterImage IMAGE2;
+    public static final List<ApiMessageAndVersion> DELTA2_RECORDS;
+    public static final ClusterDelta DELTA2;
+    public static final ClusterImage IMAGE3;
+
+    static {

Review Comment:
   > Could you please eliminate the static block?
   > 
   > ```java
   > public final class ClusterImageFixtures {
   > 
   >     public static final ClusterImage IMAGE1 = new ClusterImage(
   >             Map.of(
   >                     0, new BrokerRegistration.Builder().
   >                             setId(0).
   >                             setEpoch(1000).
   >                             setIncarnationId(Uuid.fromString("vZKYST0pSA2HO5x_6hoO2Q")).
   >                             setListeners(List.of(new Endpoint("PLAINTEXT", SecurityProtocol.PLAINTEXT, "localhost", 9092))).
   >                             setSupportedFeatures(Map.of("foo", VersionRange.of((short) 1, (short) 3))).
   >                             setRack(Optional.empty()).
   >                             setFenced(true).
   >                             setInControlledShutdown(false).build(),
   >                     1, new BrokerRegistration.Builder().
   >                             setId(1).
   >                             setEpoch(1001).
   >                             setIncarnationId(Uuid.fromString("U52uRe20RsGI0RvpcTx33Q")).
   >                             setListeners(List.of(new Endpoint("PLAINTEXT", SecurityProtocol.PLAINTEXT, "localhost", 9093))).
   >                             setSupportedFeatures(Map.of("foo", VersionRange.of((short) 1, (short) 3))).
   >                             setRack(Optional.empty()).
   >                             setFenced(false).
   >                             setInControlledShutdown(false).build(),
   >                     2, new BrokerRegistration.Builder().
   >                             setId(2).
   >                             setEpoch(123).
   >                             setIncarnationId(Uuid.fromString("hr4TVh3YQiu3p16Awkka6w")).
   >                             setListeners(List.of(new Endpoint("PLAINTEXT", SecurityProtocol.PLAINTEXT, "localhost", 9094))).
   >                             setSupportedFeatures(Map.of()).
   >                             setRack(Optional.of("arack")).
   >                             setFenced(false).
   >                             setInControlledShutdown(false).build()
   >             ),
   >             Map.of(
   >                     1000, new ControllerRegistration.Builder().
   >                             setId(1000).
   >                             setIncarnationId(Uuid.fromString("9ABu6HEgRuS-hjHLgC4cHw")).
   >                             setZkMigrationReady(false).
   >                             setListeners(Map.of("PLAINTEXT", new Endpoint("PLAINTEXT", SecurityProtocol.PLAINTEXT, "localhost", 19092))).
   >                             setSupportedFeatures(Map.of()).build()
   >             )
   >     );
   > 
   >     public static final List<ApiMessageAndVersion> DELTA1_RECORDS = List.of(
   >             new ApiMessageAndVersion(new UnfenceBrokerRecord().setId(0).setEpoch(1000), UNFENCE_BROKER_RECORD.highestSupportedVersion()),
   >             new ApiMessageAndVersion(new FenceBrokerRecord().setId(1).setEpoch(1001), FENCE_BROKER_RECORD.highestSupportedVersion()),
   >             new ApiMessageAndVersion(new BrokerRegistrationChangeRecord().setBrokerId(0).setBrokerEpoch(1000)
   >                     .setInControlledShutdown(BrokerRegistrationInControlledShutdownChange.IN_CONTROLLED_SHUTDOWN.value()),
   >                     BROKER_REGISTRATION_CHANGE_RECORD.highestSupportedVersion()),
   >             new ApiMessageAndVersion(new UnregisterBrokerRecord().setBrokerId(2).setBrokerEpoch(123), (short) 0),
   >             new ApiMessageAndVersion(new RegisterControllerRecord().setControllerId(1001).setIncarnationId(Uuid.fromString("FdEHF-IqScKfYyjZ1CjfNQ"))
   >                     .setZkMigrationReady(true).setEndPoints(new ControllerEndpointCollection(
   >                             List.of(new ControllerEndpoint().setHost("localhost").setName("PLAINTEXT").setPort(19093).setSecurityProtocol(SecurityProtocol.PLAINTEXT.id)).iterator()
   >                     )), (short) 0)
   >     );
   > 
   >     public static final ClusterDelta DELTA1 = RecordTestUtils.replayAll(new ClusterDelta(IMAGE1), DELTA1_RECORDS);
   > 
   >     public static final ClusterImage IMAGE2 = new ClusterImage(
   >             Map.of(
   >                     0, new BrokerRegistration.Builder().
   >                             setId(0).
   >                             setEpoch(1000).
   >                             setIncarnationId(Uuid.fromString("vZKYST0pSA2HO5x_6hoO2Q")).
   >                             setListeners(List.of(new Endpoint("PLAINTEXT", SecurityProtocol.PLAINTEXT, "localhost", 9092))).
   >                             setSupportedFeatures(Map.of("foo", VersionRange.of((short) 1, (short) 3))).
   >                             setRack(Optional.empty()).
   >                             setFenced(false).
   >                             setInControlledShutdown(true).build(),
   >                     1, new BrokerRegistration.Builder().
   >                             setId(1).
   >                             setEpoch(1001).
   >                             setIncarnationId(Uuid.fromString("U52uRe20RsGI0RvpcTx33Q")).
   >                             setListeners(List.of(new Endpoint("PLAINTEXT", SecurityProtocol.PLAINTEXT, "localhost", 9093))).
   >                             setSupportedFeatures(Map.of("foo", VersionRange.of((short) 1, (short) 3))).
   >                             setRack(Optional.empty()).
   >                             setFenced(true).
   >                             setInControlledShutdown(false).build()
   >             ),
   >             Map.of(
   >                     1000, IMAGE1.controllers().get(1000),
   >                     1001, new ControllerRegistration.Builder().
   >                             setId(1001).
   >                             setIncarnationId(Uuid.fromString("FdEHF-IqScKfYyjZ1CjfNQ")).
   >                             setZkMigrationReady(true).
   >                             setListeners(Map.of("PLAINTEXT", new Endpoint("PLAINTEXT", SecurityProtocol.PLAINTEXT, "localhost", 19093))).
   >                             setSupportedFeatures(Map.of()).build()
   >             )
   >     );
   > 
   >     public static final List<ApiMessageAndVersion> DELTA2_RECORDS = Stream.concat(
   >             DELTA1_RECORDS.stream(),
   >             Stream.of(
   >                     new ApiMessageAndVersion(new FenceBrokerRecord().setId(0).setEpoch(1000), FENCE_BROKER_RECORD.highestSupportedVersion()),
   >                     new ApiMessageAndVersion(new UnfenceBrokerRecord().setId(1).setEpoch(1001), UNFENCE_BROKER_RECORD.highestSupportedVersion()),
   >                     new ApiMessageAndVersion(new BrokerRegistrationChangeRecord().setBrokerId(0).setBrokerEpoch(1000)
   >                             .setInControlledShutdown(BrokerRegistrationInControlledShutdownChange.NONE.value()),
   >                             BROKER_REGISTRATION_CHANGE_RECORD.highestSupportedVersion()),
   >                     new ApiMessageAndVersion(new RegisterBrokerRecord().setBrokerId(2).setIsMigratingZkBroker(true)
   >                             .setIncarnationId(Uuid.fromString("Am5Yse7GQxaw0b2alM74bP")).setBrokerEpoch(1002)
   >                             .setEndPoints(new BrokerEndpointCollection(
   >                                     List.of(new BrokerEndpoint().setName("PLAINTEXT").setHost("localhost").setPort(9094).setSecurityProtocol((short) 0)).iterator()
   >                             ))
   >                             .setFeatures(new BrokerFeatureCollection(
   >                                     Set.of(new BrokerFeature().setName(MetadataVersion.FEATURE_NAME)
   >                                             .setMinSupportedVersion(MetadataVersion.MINIMUM_VERSION.featureLevel())
   >                                             .setMaxSupportedVersion(MetadataVersion.IBP_3_6_IV0.featureLevel())).iterator()
   >                             )).setRack("rack3"), REGISTER_BROKER_RECORD.highestSupportedVersion())
   >             )
   >     ).toList();
   > 
   >     public static final ClusterDelta DELTA2 = RecordTestUtils.replayAll(new ClusterDelta(IMAGE2), DELTA2_RECORDS);
   > 
   >     public static final ClusterImage IMAGE3 = new ClusterImage(
   >             Map.of(
   >                     0, new BrokerRegistration.Builder().
   >                             setId(0).
   >                             setEpoch(1000).
   >                             setIncarnationId(Uuid.fromString("vZKYST0pSA2HO5x_6hoO2Q")).
   >                             setListeners(List.of(new Endpoint("PLAINTEXT", SecurityProtocol.PLAINTEXT, "localhost", 9092))).
   >                             setSupportedFeatures(Map.of("foo", VersionRange.of((short) 1, (short) 3))).
   >                             setRack(Optional.empty()).
   >                             setFenced(true).
   >                             setInControlledShutdown(true).build(),
   >                     1, new BrokerRegistration.Builder().
   >                             setId(1).
   >                             setEpoch(1001).
   >                             setIncarnationId(Uuid.fromString("U52uRe20RsGI0RvpcTx33Q")).
   >                             setListeners(List.of(new Endpoint("PLAINTEXT", SecurityProtocol.PLAINTEXT, "localhost", 9093))).
   >                             setSupportedFeatures(Map.of("foo", VersionRange.of((short) 1, (short) 3))).
   >                             setRack(Optional.empty()).
   >                             setFenced(false).
   >                             setInControlledShutdown(false).build(),
   >                     2, new BrokerRegistration.Builder().
   >                             setId(2).
   >                             setEpoch(1002).
   >                             setIncarnationId(Uuid.fromString("Am5Yse7GQxaw0b2alM74bP")).
   >                             setListeners(List.of(new Endpoint("PLAINTEXT", SecurityProtocol.PLAINTEXT, "localhost", 9094))).
   >                             setSupportedFeatures(Map.of("metadata.version",
   >                                     VersionRange.of(MetadataVersion.MINIMUM_VERSION.featureLevel(), MetadataVersion.IBP_3_6_IV0.featureLevel()))).
   >                             setRack(Optional.of("rack3")).
   >                             setFenced(true).
   >                             setIsMigratingZkBroker(true).build()
   >             ),
   >             IMAGE2.controllers()
   >     );
   > 
   >     private ClusterImageFixtures() {
   >     }
   > }
   > ```
   
   Removed. It's like a code smell, with less readability.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20553 : Eliminate the dependencies on sourceSets.test.output for metadata [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #22247:
URL: https://github.com/apache/kafka/pull/22247#discussion_r3215258117


##########
metadata/src/testFixtures/java/org/apache/kafka/image/MetadataImageTestFixtures.java:
##########
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.image;
+
+import org.apache.kafka.metadata.RecordTestUtils;
+
+public final class MetadataImageTestFixtures {

Review Comment:
   this name `MetadataCacheTestFixtures` is a sort of verbose. how about "MetadataCacheFixtures"?



##########
metadata/src/testFixtures/java/org/apache/kafka/metadata/authorizer/StandardAclTestFixtures.java:
##########
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata.authorizer;
+
+import org.apache.kafka.common.acl.AclOperation;
+import org.apache.kafka.common.acl.AclPermissionType;
+import org.apache.kafka.common.resource.PatternType;
+import org.apache.kafka.common.resource.Resource;
+import org.apache.kafka.common.resource.ResourceType;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import static org.apache.kafka.metadata.authorizer.StandardAuthorizerData.WILDCARD;
+import static org.apache.kafka.metadata.authorizer.StandardAuthorizerData.WILDCARD_PRINCIPAL;
+
+public final class StandardAclTestFixtures {
+    public static final List<StandardAcl> TEST_ACLS = new ArrayList<>();

Review Comment:
   ```java
   public static final List<StandardAcl> TEST_ACLS = List.of(
       new StandardAcl(
           ResourceType.CLUSTER,
           Resource.CLUSTER_NAME,
           PatternType.LITERAL,
           WILDCARD_PRINCIPAL,
           WILDCARD,
           AclOperation.ALTER,
           AclPermissionType.ALLOW
       ),
       new StandardAcl(
           ResourceType.TOPIC,
           "foo_",
           PatternType.PREFIXED,
           WILDCARD_PRINCIPAL,
           WILDCARD,
           AclOperation.READ,
           AclPermissionType.ALLOW
       ),
       new StandardAcl(
           ResourceType.GROUP,
           "mygroup",
           PatternType.LITERAL,
           "User:foo",
           WILDCARD,
           AclOperation.READ,
           AclPermissionType.DENY
       ),
       new StandardAcl(
           ResourceType.GROUP,
           "mygroup",
           PatternType.PREFIXED,
           "User:foo",
           WILDCARD,
           AclOperation.READ,
           AclPermissionType.DENY
       ),
       new StandardAcl(
           ResourceType.GROUP,
           "foo",
           PatternType.PREFIXED,
           "User:foo",
           WILDCARD,
           AclOperation.READ,
           AclPermissionType.DENY
       )
   );
   ```



##########
metadata/src/testFixtures/java/org/apache/kafka/image/FeaturesImageTestFixtures.java:
##########
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.image;
+
+import org.apache.kafka.common.metadata.FeatureLevelRecord;
+import org.apache.kafka.metadata.RecordTestUtils;
+import org.apache.kafka.server.common.ApiMessageAndVersion;
+import org.apache.kafka.server.common.MetadataVersion;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+public final class FeaturesImageTestFixtures {

Review Comment:
   ```java
   public final class FeaturesImageTestFixtures {
   
       public static final FeaturesImage IMAGE1 = new FeaturesImage(
               Map.of("foo", (short) 2, "bar", (short) 1),
               MetadataVersion.latestTesting()
       );
   
       public static final List<ApiMessageAndVersion> DELTA1_RECORDS = List.of(
               new ApiMessageAndVersion(new FeatureLevelRecord().setName("foo").setFeatureLevel((short) 3), (short) 0),
               new ApiMessageAndVersion(new FeatureLevelRecord().setName("bar").setFeatureLevel((short) 0), (short) 0),
               new ApiMessageAndVersion(new FeatureLevelRecord().setName("baz").setFeatureLevel((short) 8), (short) 0)
       );
   
       public static final FeaturesDelta DELTA1 = RecordTestUtils.replayAll(
               new FeaturesDelta(IMAGE1),
               DELTA1_RECORDS
       );
   
       public static final FeaturesImage IMAGE2 = new FeaturesImage(
               Map.of("foo", (short) 3, "baz", (short) 8),
               MetadataVersion.latestTesting()
       );
   
       public static final List<ApiMessageAndVersion> DELTA2_RECORDS = List.of(
               new ApiMessageAndVersion(new FeatureLevelRecord().setName("foo").setFeatureLevel((short) 0), (short) 0),
               new ApiMessageAndVersion(new FeatureLevelRecord().setName("baz").setFeatureLevel((short) 0), (short) 0),
               new ApiMessageAndVersion(new FeatureLevelRecord().setName("bar").setFeatureLevel((short) 1), (short) 0)
       );
   
       public static final FeaturesDelta DELTA2 = RecordTestUtils.replayAll(
               new FeaturesDelta(IMAGE2),
               DELTA2_RECORDS
       );
   
       public static final FeaturesImage IMAGE3 = new FeaturesImage(
               Map.of("bar", (short) 1),
               MetadataVersion.latestTesting()
       );
   
       private FeaturesImageTestFixtures() {
       }
   }
   ```



##########
core/src/test/scala/unit/kafka/server/KafkaApisTest.scala:
##########
@@ -86,7 +86,7 @@ import org.apache.kafka.coordinator.group.streams.StreamsGroupHeartbeatResult
 import org.apache.kafka.coordinator.share.{ShareCoordinator, ShareCoordinatorTestConfig}
 import org.apache.kafka.coordinator.transaction.TransactionLogConfig
 import org.apache.kafka.image.{MetadataDelta, MetadataImage, MetadataProvenance}
-import org.apache.kafka.metadata.{ConfigRepository, KRaftMetadataCache, MetadataCache, MetadataCacheTest, MockConfigRepository}
+import org.apache.kafka.metadata.{ConfigRepository, KRaftMetadataCache, MetadataCache, MetadataCacheTestFixtures, MockConfigRepository}

Review Comment:
   `SimpleConfigRepository` in the storage module could be replaced by `MockConfigRepository`



##########
metadata/src/testFixtures/java/org/apache/kafka/image/ProducerIdsImageTestFixtures.java:
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.image;
+
+import org.apache.kafka.common.metadata.ProducerIdsRecord;
+import org.apache.kafka.metadata.RecordTestUtils;
+import org.apache.kafka.server.common.ApiMessageAndVersion;
+
+import java.util.ArrayList;
+import java.util.List;
+
+public final class ProducerIdsImageTestFixtures {

Review Comment:
   ```java
   public final class ProducerIdsImageTestFixtures {
   
       public static final ProducerIdsImage IMAGE1 = new ProducerIdsImage(123);
   
       public static final List<ApiMessageAndVersion> DELTA1_RECORDS = List.of(
               new ApiMessageAndVersion(new ProducerIdsRecord().setBrokerId(2).setBrokerEpoch(100).setNextProducerId(456), (short) 0),
               new ApiMessageAndVersion(new ProducerIdsRecord().setBrokerId(3).setBrokerEpoch(100).setNextProducerId(780), (short) 0),
               new ApiMessageAndVersion(new ProducerIdsRecord().setBrokerId(3).setBrokerEpoch(100).setNextProducerId(785), (short) 0),
               new ApiMessageAndVersion(new ProducerIdsRecord().setBrokerId(2).setBrokerEpoch(100).setNextProducerId(800), (short) 0)
       );
   
       public static final ProducerIdsDelta DELTA1 = RecordTestUtils.replayAll(
               new ProducerIdsDelta(IMAGE1),
               DELTA1_RECORDS
       );
   
       public static final ProducerIdsImage IMAGE2 = new ProducerIdsImage(800);
   
       private ProducerIdsImageTestFixtures() {
       }
   }
   ```



##########
metadata/src/testFixtures/java/org/apache/kafka/image/ConfigurationsImageTestFixtures.java:
##########
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.image;
+
+import org.apache.kafka.common.config.ConfigResource;
+import org.apache.kafka.common.metadata.ConfigRecord;
+import org.apache.kafka.metadata.RecordTestUtils;
+import org.apache.kafka.metadata.SupportedConfigChecker;
+import org.apache.kafka.server.common.ApiMessageAndVersion;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.kafka.common.config.ConfigResource.Type.BROKER;
+import static org.apache.kafka.common.metadata.MetadataRecordType.CONFIG_RECORD;
+
+public final class ConfigurationsImageTestFixtures {

Review Comment:
   ```java
   public final class ConfigurationsImageTestFixtures {
   
       public static final ConfigurationsImage IMAGE1 = new ConfigurationsImage(Map.of(
               new ConfigResource(BROKER, "0"), new ConfigurationImage(new ConfigResource(BROKER, "0"), Map.of("foo", "bar", "baz", "quux")),
               new ConfigResource(BROKER, "1"), new ConfigurationImage(new ConfigResource(BROKER, "1"), Map.of("foobar", "foobaz"))
       ));
   
       public static final List<ApiMessageAndVersion> DELTA1_RECORDS = List.of(
               new ApiMessageAndVersion(new ConfigRecord().setResourceType(BROKER.id())
                       .setResourceName("0").setName("foo").setValue(null), CONFIG_RECORD.highestSupportedVersion()),
               new ApiMessageAndVersion(new ConfigRecord().setResourceType(BROKER.id())
                       .setResourceName("0").setName("baz").setValue(null), CONFIG_RECORD.highestSupportedVersion()),
               new ApiMessageAndVersion(new ConfigRecord().setResourceType(BROKER.id())
                       .setResourceName("1").setName("foobar").setValue(null), CONFIG_RECORD.highestSupportedVersion()),
               new ApiMessageAndVersion(new ConfigRecord().setResourceType(BROKER.id())
                       .setResourceName("1").setName("barfoo").setValue("bazfoo"), CONFIG_RECORD.highestSupportedVersion()),
               new ApiMessageAndVersion(new ConfigRecord().setResourceType(BROKER.id())
                       .setResourceName("2").setName("foo").setValue("bar"), CONFIG_RECORD.highestSupportedVersion())
       );
   
       public static final ConfigurationsDelta DELTA1 = RecordTestUtils.replayAll(new ConfigurationsDelta(IMAGE1, SupportedConfigChecker.TRUE), DELTA1_RECORDS);
   
       public static final ConfigurationsImage IMAGE2 = new ConfigurationsImage(Map.of(
               new ConfigResource(BROKER, "1"), new ConfigurationImage(new ConfigResource(BROKER, "1"), Map.of("barfoo", "bazfoo")),
               new ConfigResource(BROKER, "2"), new ConfigurationImage(new ConfigResource(BROKER, "2"), Map.of("foo", "bar"))
       ));
   
       private ConfigurationsImageTestFixtures() {
       }
   }
   ```



##########
metadata/src/testFixtures/java/org/apache/kafka/metadata/authorizer/StandardAclWithIdTestFixtures.java:
##########
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.metadata.authorizer;
+
+import org.apache.kafka.common.Uuid;
+
+import java.util.ArrayList;
+import java.util.List;
+
+public final class StandardAclWithIdTestFixtures {

Review Comment:
   ```java
   public final class StandardAclWithIdTestFixtures {
   
       public static final List<StandardAclWithId> TEST_ACLS = List.of(
               new StandardAclWithId(Uuid.fromString("QZDDv-R7SyaPgetDPGd0Mw"), StandardAclTestFixtures.TEST_ACLS.get(0)),
               new StandardAclWithId(Uuid.fromString("SdDjEdlbRmy2__WFKe3RMg"), StandardAclTestFixtures.TEST_ACLS.get(1)),
               new StandardAclWithId(Uuid.fromString("wQzt5gkSTwuQNXZF5gIw7A"), StandardAclTestFixtures.TEST_ACLS.get(2)),
               new StandardAclWithId(Uuid.fromString("ab_5xjJXSbS1o5jGfhgQXg"), StandardAclTestFixtures.TEST_ACLS.get(3)),
               new StandardAclWithId(Uuid.fromString("wP_cCK0LTEGSX9oDRInJHQ"), StandardAclTestFixtures.TEST_ACLS.get(4))
       );
   
       private StandardAclWithIdTestFixtures() {
       }
   }
   ```



##########
metadata/src/testFixtures/java/org/apache/kafka/image/ConfigurationsImageTestFixtures.java:
##########
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.image;
+
+import org.apache.kafka.common.config.ConfigResource;
+import org.apache.kafka.common.metadata.ConfigRecord;
+import org.apache.kafka.metadata.RecordTestUtils;
+import org.apache.kafka.metadata.SupportedConfigChecker;
+import org.apache.kafka.server.common.ApiMessageAndVersion;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.kafka.common.config.ConfigResource.Type.BROKER;
+import static org.apache.kafka.common.metadata.MetadataRecordType.CONFIG_RECORD;
+
+public final class ConfigurationsImageTestFixtures {

Review Comment:
   We need to make `RecordTestUtils.replayAll` return the input target to make the above code work



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20553 : Eliminate the dependencies on sourceSets.test.output for metadata [kafka]

Posted by "muralibasani (via GitHub)" <gi...@apache.org>.
muralibasani commented on code in PR #22247:
URL: https://github.com/apache/kafka/pull/22247#discussion_r3215240615


##########
metadata/src/test/java/org/apache/kafka/image/ScramImageTest.java:
##########
@@ -17,116 +17,31 @@
 
 package org.apache.kafka.image;
 
-import org.apache.kafka.clients.admin.ScramMechanism;
-import org.apache.kafka.common.metadata.RemoveUserScramCredentialRecord;
-import org.apache.kafka.common.metadata.UserScramCredentialRecord;
 import org.apache.kafka.image.writer.ImageWriterOptions;
 import org.apache.kafka.image.writer.RecordListWriter;
 import org.apache.kafka.metadata.RecordTestUtils;
-import org.apache.kafka.metadata.ScramCredentialData;
 import org.apache.kafka.server.common.ApiMessageAndVersion;
 import org.apache.kafka.server.common.MetadataVersion;
-import org.apache.kafka.server.util.MockRandom;
 
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.api.Timeout;
 
-import java.util.ArrayList;
-import java.util.HashMap;
 import java.util.List;
-import java.util.Map;
 import java.util.Optional;
-import java.util.Random;
 
-import static org.apache.kafka.clients.admin.ScramMechanism.SCRAM_SHA_256;
-import static org.apache.kafka.clients.admin.ScramMechanism.SCRAM_SHA_512;
 import static org.junit.jupiter.api.Assertions.assertEquals;
 import static org.junit.jupiter.api.Assertions.assertThrows;
 
 
 @Timeout(value = 40)
 public class ScramImageTest {
-    public static final ScramImage IMAGE1;
+    public static final ScramImage IMAGE1 = ScramImageTestFixtures.IMAGE1;

Review Comment:
   ScramImageTestFixtures is shared by MetadataImageTestFixtures( MetadataImageTestFixtures is in testfixtures because BrokerMetadataPublisherTest.scala from core references MetadataImageTest)
   
   So it is a shared one.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20553 : Eliminate the dependencies on sourceSets.test.output for metadata [kafka]

Posted by "muralibasani (via GitHub)" <gi...@apache.org>.
muralibasani commented on PR #22247:
URL: https://github.com/apache/kafka/pull/22247#issuecomment-4417835466

   > @muralibasani thanks for this patch. I've left a couple of comments
   
   @chia7712 thanks for the review. Updated pr based on the comments except one related to storage module.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20553 : Eliminate the dependencies on sourceSets.test.output for metadata [kafka]

Posted by "muralibasani (via GitHub)" <gi...@apache.org>.
muralibasani commented on code in PR #22247:
URL: https://github.com/apache/kafka/pull/22247#discussion_r3223701385


##########
core/src/test/scala/unit/kafka/security/authorizer/AuthorizerTest.scala:
##########
@@ -34,8 +34,7 @@ import org.apache.kafka.common.resource.{PatternType, ResourcePattern, ResourceP
 import org.apache.kafka.common.security.auth.{KafkaPrincipal, SecurityProtocol}
 import org.apache.kafka.common.utils.internals.{SecurityUtils => JSecurityUtils}
 import org.apache.kafka.controller.MockAclMutator

Review Comment:
   Agree, sure, will do.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20553 : Eliminate the dependencies on sourceSets.test.output for metadata [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #22247:
URL: https://github.com/apache/kafka/pull/22247#discussion_r3215208296


##########
metadata/src/test/java/org/apache/kafka/image/ScramImageTest.java:
##########
@@ -17,116 +17,31 @@
 
 package org.apache.kafka.image;
 
-import org.apache.kafka.clients.admin.ScramMechanism;
-import org.apache.kafka.common.metadata.RemoveUserScramCredentialRecord;
-import org.apache.kafka.common.metadata.UserScramCredentialRecord;
 import org.apache.kafka.image.writer.ImageWriterOptions;
 import org.apache.kafka.image.writer.RecordListWriter;
 import org.apache.kafka.metadata.RecordTestUtils;
-import org.apache.kafka.metadata.ScramCredentialData;
 import org.apache.kafka.server.common.ApiMessageAndVersion;
 import org.apache.kafka.server.common.MetadataVersion;
-import org.apache.kafka.server.util.MockRandom;
 
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.api.Timeout;
 
-import java.util.ArrayList;
-import java.util.HashMap;
 import java.util.List;
-import java.util.Map;
 import java.util.Optional;
-import java.util.Random;
 
-import static org.apache.kafka.clients.admin.ScramMechanism.SCRAM_SHA_256;
-import static org.apache.kafka.clients.admin.ScramMechanism.SCRAM_SHA_512;
 import static org.junit.jupiter.api.Assertions.assertEquals;
 import static org.junit.jupiter.api.Assertions.assertThrows;
 
 
 @Timeout(value = 40)
 public class ScramImageTest {
-    public static final ScramImage IMAGE1;
+    public static final ScramImage IMAGE1 = ScramImageTestFixtures.IMAGE1;

Review Comment:
   If `ScramImageTestFixtures` is used by `ScramImageTest` only, we could keep it as is. Otherwise, the testfixtures will contain many classes which are not shared actually



##########
metadata/src/test/java/org/apache/kafka/image/ScramImageTest.java:
##########
@@ -17,116 +17,31 @@
 
 package org.apache.kafka.image;
 
-import org.apache.kafka.clients.admin.ScramMechanism;
-import org.apache.kafka.common.metadata.RemoveUserScramCredentialRecord;
-import org.apache.kafka.common.metadata.UserScramCredentialRecord;
 import org.apache.kafka.image.writer.ImageWriterOptions;
 import org.apache.kafka.image.writer.RecordListWriter;
 import org.apache.kafka.metadata.RecordTestUtils;
-import org.apache.kafka.metadata.ScramCredentialData;
 import org.apache.kafka.server.common.ApiMessageAndVersion;
 import org.apache.kafka.server.common.MetadataVersion;
-import org.apache.kafka.server.util.MockRandom;
 
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.api.Timeout;
 
-import java.util.ArrayList;
-import java.util.HashMap;
 import java.util.List;
-import java.util.Map;
 import java.util.Optional;
-import java.util.Random;
 
-import static org.apache.kafka.clients.admin.ScramMechanism.SCRAM_SHA_256;
-import static org.apache.kafka.clients.admin.ScramMechanism.SCRAM_SHA_512;
 import static org.junit.jupiter.api.Assertions.assertEquals;
 import static org.junit.jupiter.api.Assertions.assertThrows;
 
 
 @Timeout(value = 40)
 public class ScramImageTest {
-    public static final ScramImage IMAGE1;
+    public static final ScramImage IMAGE1 = ScramImageTestFixtures.IMAGE1;

Review Comment:
   If `ScramImageTestFixtures` is used by `ScramImageTest` only, we should keep it as is. Otherwise, test-fixtures will end up containing many classes that aren't actually shared.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20553 : Eliminate the dependencies on sourceSets.test.output for metadata [kafka]

Posted by "muralibasani (via GitHub)" <gi...@apache.org>.
muralibasani commented on code in PR #22247:
URL: https://github.com/apache/kafka/pull/22247#discussion_r3215369578


##########
core/src/test/scala/unit/kafka/server/KafkaApisTest.scala:
##########
@@ -86,7 +86,7 @@ import org.apache.kafka.coordinator.group.streams.StreamsGroupHeartbeatResult
 import org.apache.kafka.coordinator.share.{ShareCoordinator, ShareCoordinatorTestConfig}
 import org.apache.kafka.coordinator.transaction.TransactionLogConfig
 import org.apache.kafka.image.{MetadataDelta, MetadataImage, MetadataProvenance}
-import org.apache.kafka.metadata.{ConfigRepository, KRaftMetadataCache, MetadataCache, MetadataCacheTest, MockConfigRepository}
+import org.apache.kafka.metadata.{ConfigRepository, KRaftMetadataCache, MetadataCache, MetadataCacheTestFixtures, MockConfigRepository}

Review Comment:
   Agree. To make this pr specific to metadata, I shall update this in a follow up pr. Hope it's ok.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20553 : Eliminate the dependencies on sourceSets.test.output for metadata [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #22247:
URL: https://github.com/apache/kafka/pull/22247#discussion_r3222505060


##########
metadata/src/testFixtures/java/org/apache/kafka/image/AclsImageFixtures.java:
##########
@@ -0,0 +1,65 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.image;
+
+import org.apache.kafka.common.Uuid;
+import org.apache.kafka.common.metadata.RemoveAccessControlEntryRecord;
+import org.apache.kafka.metadata.RecordTestUtils;
+import org.apache.kafka.metadata.authorizer.StandardAcl;
+import org.apache.kafka.metadata.authorizer.StandardAclWithId;
+import org.apache.kafka.metadata.authorizer.StandardAclWithIdFixtures;
+import org.apache.kafka.server.common.ApiMessageAndVersion;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+public final class AclsImageFixtures {
+    public static final AclsImage IMAGE1;
+    public static final List<ApiMessageAndVersion> DELTA1_RECORDS;
+    public static final AclsDelta DELTA1;
+    public static final AclsImage IMAGE2;
+
+    static {

Review Comment:
   ```java
   public final class AclsImageFixtures {
   
       public static final AclsImage IMAGE1 = new AclsImage(createAclMap(0, 4));
   
       public static final List<ApiMessageAndVersion> DELTA1_RECORDS = List.of(
               new ApiMessageAndVersion(new RemoveAccessControlEntryRecord().
                       setId(Uuid.fromString("QZDDv-R7SyaPgetDPGd0Mw")), (short) 0),
               new ApiMessageAndVersion(StandardAclWithIdFixtures.TEST_ACLS.get(4).toRecord(), (short) 0)
       );
   
       public static final AclsDelta DELTA1 = RecordTestUtils.replayAll(new AclsDelta(IMAGE1), DELTA1_RECORDS);
   
       public static final AclsImage IMAGE2 = new AclsImage(createAclMap(1, 5));
   
       private static Map<Uuid, StandardAcl> createAclMap(int startInclusive, int endExclusive) {
           return IntStream.range(startInclusive, endExclusive)
                   .mapToObj(StandardAclWithIdFixtures.TEST_ACLS::get)
                   .collect(Collectors.toUnmodifiableMap(StandardAclWithId::id, StandardAclWithId::acl));
       }
   
       private AclsImageFixtures() {
       }
   }
   ```



##########
metadata/src/testFixtures/java/org/apache/kafka/image/ClientQuotasImageFixtures.java:
##########
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.image;
+
+import org.apache.kafka.common.metadata.ClientQuotaRecord;
+import org.apache.kafka.common.metadata.ClientQuotaRecord.EntityData;
+import org.apache.kafka.common.quota.ClientQuotaEntity;
+import org.apache.kafka.metadata.RecordTestUtils;
+import org.apache.kafka.server.common.ApiMessageAndVersion;
+import org.apache.kafka.server.config.QuotaConfig;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.kafka.common.metadata.MetadataRecordType.CLIENT_QUOTA_RECORD;
+
+public final class ClientQuotasImageFixtures {
+    public static final ClientQuotasImage IMAGE1;
+    public static final List<ApiMessageAndVersion> DELTA1_RECORDS;
+    public static final ClientQuotasDelta DELTA1;
+    public static final ClientQuotasImage IMAGE2;
+
+    static {

Review Comment:
   ```java
   public final class ClientQuotasImageFixtures {
   
       public static final ClientQuotasImage IMAGE1 = new ClientQuotasImage(Map.of(
               new ClientQuotaEntity(Map.of(ClientQuotaEntity.USER, "foo")),
               new ClientQuotaImage(Map.of(QuotaConfig.PRODUCER_BYTE_RATE_OVERRIDE_CONFIG, 123.0)),
               new ClientQuotaEntity(Map.of(ClientQuotaEntity.USER, "bar", ClientQuotaEntity.IP, "127.0.0.1")),
               new ClientQuotaImage(Map.of(QuotaConfig.CONSUMER_BYTE_RATE_OVERRIDE_CONFIG, 456.0))
       ));
   
       public static final List<ApiMessageAndVersion> DELTA1_RECORDS = List.of(
               // remove quota
               new ApiMessageAndVersion(new ClientQuotaRecord().
                       setEntity(List.of(
                               new EntityData().setEntityType(ClientQuotaEntity.USER).setEntityName("bar"),
                               new EntityData().setEntityType(ClientQuotaEntity.IP).setEntityName("127.0.0.1"))).
                       setKey(QuotaConfig.CONSUMER_BYTE_RATE_OVERRIDE_CONFIG).
                       setRemove(true), CLIENT_QUOTA_RECORD.highestSupportedVersion()),
               // alter quota
               new ApiMessageAndVersion(new ClientQuotaRecord().
                       setEntity(List.of(
                               new EntityData().setEntityType(ClientQuotaEntity.USER).setEntityName("foo"))).
                       setKey(QuotaConfig.PRODUCER_BYTE_RATE_OVERRIDE_CONFIG).
                       setValue(234.0), CLIENT_QUOTA_RECORD.highestSupportedVersion()),
               // add quota to entity with existing quota
               new ApiMessageAndVersion(new ClientQuotaRecord().
                       setEntity(List.of(
                               new EntityData().setEntityType(ClientQuotaEntity.USER).setEntityName("foo"))).
                       setKey(QuotaConfig.CONSUMER_BYTE_RATE_OVERRIDE_CONFIG).
                       setValue(999.0), CLIENT_QUOTA_RECORD.highestSupportedVersion())
       );
   
       public static final ClientQuotasDelta DELTA1 = RecordTestUtils.replayAll(
               new ClientQuotasDelta(IMAGE1),
               DELTA1_RECORDS
       );
   
       public static final ClientQuotasImage IMAGE2 = new ClientQuotasImage(Map.of(
               new ClientQuotaEntity(Map.of(ClientQuotaEntity.USER, "foo")),
               new ClientQuotaImage(Map.of(
                       QuotaConfig.PRODUCER_BYTE_RATE_OVERRIDE_CONFIG, 234.0,
                       QuotaConfig.CONSUMER_BYTE_RATE_OVERRIDE_CONFIG, 999.0
               ))
       ));
   
       private ClientQuotasImageFixtures() {
       }
   }
   ```



##########
core/src/test/scala/unit/kafka/security/authorizer/AuthorizerTest.scala:
##########
@@ -34,8 +34,7 @@ import org.apache.kafka.common.resource.{PatternType, ResourcePattern, ResourceP
 import org.apache.kafka.common.security.auth.{KafkaPrincipal, SecurityProtocol}
 import org.apache.kafka.common.utils.internals.{SecurityUtils => JSecurityUtils}
 import org.apache.kafka.controller.MockAclMutator

Review Comment:
   It seems `AuthorizerTest` is the only class using `MockAclMutator`. Would you mind opening a ticket to move `MockAclMutator` alongside `AuthorizerTest` when the latter is moved out of the core module?



##########
metadata/src/testFixtures/java/org/apache/kafka/image/DelegationTokenImageFixtures.java:
##########
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.image;
+
+import org.apache.kafka.common.metadata.DelegationTokenRecord;
+import org.apache.kafka.common.metadata.RemoveDelegationTokenRecord;
+import org.apache.kafka.common.security.auth.KafkaPrincipal;
+import org.apache.kafka.common.security.token.delegation.TokenInformation;
+import org.apache.kafka.common.utils.internals.SecurityUtils;
+import org.apache.kafka.metadata.DelegationTokenData;
+import org.apache.kafka.metadata.RecordTestUtils;
+import org.apache.kafka.server.common.ApiMessageAndVersion;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+public final class DelegationTokenImageFixtures {

Review Comment:
   ```java
   public final class DelegationTokenImageFixtures {
   
       private static DelegationTokenData randomDelegationTokenData(String tokenId, long expireTimestamp) {
           TokenInformation ti = new TokenInformation(
                   tokenId,
                   SecurityUtils.parseKafkaPrincipal(KafkaPrincipal.USER_TYPE + ":" + "fred"),
                   SecurityUtils.parseKafkaPrincipal(KafkaPrincipal.USER_TYPE + ":" + "fred"),
                   List.of(),
                   0,
                   1000,
                   expireTimestamp);
           return new DelegationTokenData(ti);
       }
   
       public static final DelegationTokenImage IMAGE1 = new DelegationTokenImage(Map.of(
               "somerandomuuid1", randomDelegationTokenData("somerandomuuid1", 100),
               "somerandomuuid2", randomDelegationTokenData("somerandomuuid2", 100),
               "somerandomuuid3", randomDelegationTokenData("somerandomuuid3", 100)
       ));
   
       public static final List<ApiMessageAndVersion> DELTA1_RECORDS = List.of(
               new ApiMessageAndVersion(new DelegationTokenRecord().
                       setOwner(KafkaPrincipal.USER_TYPE + ":" + "fred").
                       setRequester(KafkaPrincipal.USER_TYPE + ":" + "fred").
                       setIssueTimestamp(0).
                       setMaxTimestamp(1000).
                       setExpirationTimestamp(200).
                       setTokenId("somerandomuuid1"), (short) 0),
               new ApiMessageAndVersion(new RemoveDelegationTokenRecord().
                       setTokenId("somerandomuuid3"), (short) 0)
       );
   
       public static final DelegationTokenDelta DELTA1 = RecordTestUtils.replayAll(
               new DelegationTokenDelta(IMAGE1),
               DELTA1_RECORDS
       );
   
       public static final DelegationTokenImage IMAGE2 = new DelegationTokenImage(Map.of(
               "somerandomuuid1", randomDelegationTokenData("somerandomuuid1", 200),
               "somerandomuuid2", randomDelegationTokenData("somerandomuuid2", 100)
       ));
   
       private DelegationTokenImageFixtures() {
       }
   }
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20553 : Eliminate the dependencies on sourceSets.test.output for metadata [kafka]

Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 merged PR #22247:
URL: https://github.com/apache/kafka/pull/22247


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-20553 : Eliminate the dependencies on sourceSets.test.output for metadata [kafka]

Posted by "muralibasani (via GitHub)" <gi...@apache.org>.
muralibasani commented on code in PR #22247:
URL: https://github.com/apache/kafka/pull/22247#discussion_r3237222068


##########
core/src/test/scala/unit/kafka/security/authorizer/AuthorizerTest.scala:
##########
@@ -34,8 +34,7 @@ import org.apache.kafka.common.resource.{PatternType, ResourcePattern, ResourceP
 import org.apache.kafka.common.security.auth.{KafkaPrincipal, SecurityProtocol}
 import org.apache.kafka.common.utils.internals.{SecurityUtils => JSecurityUtils}
 import org.apache.kafka.controller.MockAclMutator

Review Comment:
   @chia7712  https://issues.apache.org/jira/browse/KAFKA-20581 have this ticket opened for moving MockAclMutator



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org