You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "RaidenE1 (via GitHub)" <gi...@apache.org> on 2025/08/08 03:32:56 UTC
[PR] add error cache for auto topic creation failure [kafka]
RaidenE1 opened a new pull request, #20325:
URL: https://github.com/apache/kafka/pull/20325
# Summary
This PR implements error caching for internal topic creation failures in Kafka Streams, allowing errors to be
surfaced to users via the Streams group heartbeat status instead of only appearing in broker logs.
# Problem
Currently, when internal topic creation fails during Streams group heartbeat processing, the error messages are
only logged in the broker logs and not exposed to users. As mentioned in the code comments around
KafkaApis.scala:2857-2893, the result of the create topic call forwarded to the controller is not awaited, so if
an internal topic fails to be created, users cannot see the specific reason for the failure.
# Solution
1. Error Caching in AutoTopicCreationManager
- Added CachedTopicCreationError case class to store error messages with timestamps
- Implemented getTopicCreationErrors() method with:
- Lazy cleanup: Expired entries are removed during access based on configurable TTL
- Size limits: Cache is limited to 1000 entries, with oldest entries removed when exceeded
- TTL based on existing config: Uses 3 × request.timeout.ms (default: 90 seconds) for cache expiration
- Enhanced ControllerRequestCompletionHandler.onComplete() to parse CreateTopicsResponse and cache errors for
failed topics only
- Added proper resource cleanup in close() method
2. Integration with KafkaApis
- Enhanced Streams group heartbeat processing in KafkaApis.scala
- When MISSING_INTERNAL_TOPICS status is detected, query cached errors and append to status details
- Only query cache when Group Coordinator has already reported missing topics
3. Lifecycle Management
- Added autoTopicCreationManager.close() call in BrokerServer.shutdown() to ensure proper cleanup
# Key Features
- Thread-safe: Uses ConcurrentHashMap for concurrent access
- Memory efficient: TTL-based expiration and size limits prevent memory leaks
- Configurable TTL: Based on existing request.timeout.ms configuration (3× multiplier)
- Lazy cleanup: No background threads needed - cleanup happens during normal operation
- Selective caching: Only caches actual failures (errorCode != NONE), successful creations are ignored
- Comprehensive error handling: Handles authentication failures, version mismatches, and topic-specific errors
- Backward compatible: No changes to existing APIs or behavior
# Configuration
The error cache TTL is automatically calculated as 3 × request.timeout.ms:
- Default: 90 seconds (3 × 30s default request timeout)
- Configurable: Adjusts automatically when request.timeout.ms is modified
- Cache size limit: 1000 entries (hardcoded)
# Testing
- Added comprehensive unit tests for error caching, TTL cleanup, and size limit management
- Added integration test for KafkaApis to verify end-to-end functionality
- Updated test cases to use realistic TTL values based on test configuration
- All existing tests pass without modification
# Code Changes
- AutoTopicCreationManager.scala: Added error caching functionality (~70 lines)
- KafkaApis.scala: Enhanced Streams heartbeat processing (~15 lines)
- BrokerServer.scala: Added cleanup call in shutdown (~2 lines)
- Test files: Added comprehensive test coverage (~170 lines)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "RaidenE1 (via GitHub)" <gi...@apache.org>.
RaidenE1 commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2271333309
##########
core/src/main/scala/kafka/server/KafkaApis.scala:
##########
@@ -2889,6 +2889,24 @@ class KafkaApis(val requestChannel: RequestChannel,
}
} else {
autoTopicCreationManager.createStreamsInternalTopics(topicsToCreate, requestContext);
+
+ // Check for cached topic creation errors only if there's already a MISSING_INTERNAL_TOPICS status
+ val hasMissingInternalTopicsStatus = responseData.status() != null &&
+ responseData.status().stream().anyMatch(s => s.statusCode() == StreamsGroupHeartbeatResponse.Status.MISSING_INTERNAL_TOPICS.code())
Review Comment:
Because this ticket is about when source topic is missing, we check the cache, if we found it failed to be created, then we inform user. If it's not missing then there is no error?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "aliehsaeedii (via GitHub)" <gi...@apache.org>.
aliehsaeedii commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2273282296
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -135,10 +190,17 @@ class DefaultAutoTopicCreationManager(
clearInflightRequests(creatableTopics)
if (response.authenticationException() != null) {
warn(s"Auto topic creation failed for ${creatableTopics.keys} with authentication exception")
+ cacheTopicCreationErrors(creatableTopics.keys.toSet, "Authentication failed")
} else if (response.versionMismatch() != null) {
warn(s"Auto topic creation failed for ${creatableTopics.keys} with invalid version exception")
+ cacheTopicCreationErrors(creatableTopics.keys.toSet, "Version mismatch")
} else {
- debug(s"Auto topic creation completed for ${creatableTopics.keys} with response ${response.responseBody}.")
+ response.responseBody() match {
+ case createTopicsResponse: CreateTopicsResponse =>
+ cacheTopicCreationErrorsFromResponse(createTopicsResponse)
Review Comment:
I also think no need to add the success case to the error cache
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "aliehsaeedii (via GitHub)" <gi...@apache.org>.
aliehsaeedii commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2273257251
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -112,6 +128,45 @@ class DefaultAutoTopicCreationManager(
}
}
+ override def getTopicCreationErrors(
+ topicNames: Set[String]
+ ): Map[String, String] = {
+ val currentTime = System.currentTimeMillis()
+ val errors = mutable.Map.empty[String, String]
+ val expiredKeys = mutable.Set.empty[String]
+
+ // Check requested topics and collect expired keys
+ topicNames.foreach { topicName =>
+ Option(topicCreationErrorCache.get(topicName)) match {
+ case Some(cachedError) if (currentTime - cachedError.timestamp) <= errorCacheTtlMs =>
+ errors.put(topicName, cachedError.errorMessage)
+ case Some(_) =>
+ expiredKeys += topicName
Review Comment:
Being ttl passed in enough I assume.
@RaidenE1 please address this comment as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "lucasbru (via GitHub)" <gi...@apache.org>.
lucasbru commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2336984764
##########
core/src/test/scala/unit/kafka/server/AutoTopicCreationManagerTest.scala:
##########
@@ -356,4 +371,226 @@ class AutoTopicCreationManagerTest {
.setNumPartitions(numPartitions)
.setReplicationFactor(replicationFactor)
}
+
+ @Test
+ def testTopicCreationErrorCaching(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator,
+ mockTime)
+
+ val topics = Map(
+ "test-topic-1" -> new CreatableTopic().setName("test-topic-1").setNumPartitions(1).setReplicationFactor(1)
+ )
+ val requestContext = initializeRequestContextWithUserPrincipal()
+
+ autoTopicCreationManager.createStreamsInternalTopics(topics, requestContext, config.groupCoordinatorConfig.streamsGroupHeartbeatIntervalMs() * 2)
+
+ val argumentCaptor = ArgumentCaptor.forClass(classOf[ControllerRequestCompletionHandler])
+ Mockito.verify(brokerToController).sendRequest(
+ any(classOf[AbstractRequest.Builder[_ <: AbstractRequest]]),
+ argumentCaptor.capture())
+
+ // Simulate a CreateTopicsResponse with errors
+ val createTopicsResponseData = new org.apache.kafka.common.message.CreateTopicsResponseData()
+ val topicResult = new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("test-topic-1")
+ .setErrorCode(Errors.TOPIC_ALREADY_EXISTS.code())
+ .setErrorMessage("Topic 'test-topic-1' already exists.")
+ createTopicsResponseData.topics().add(topicResult)
+
+ val createTopicsResponse = new CreateTopicsResponse(createTopicsResponseData)
+ val header = new RequestHeader(ApiKeys.CREATE_TOPICS, 0, "client", 1)
+ val clientResponse = new ClientResponse(header, null, null,
+ 0, 0, false, null, null, createTopicsResponse)
+
+ // Trigger the completion handler
+ argumentCaptor.getValue.asInstanceOf[ControllerRequestCompletionHandler].onComplete(clientResponse)
+
+ // Verify that the error was cached
+ val cachedErrors = autoTopicCreationManager.getStreamsInternalTopicCreationErrors(Set("test-topic-1"), mockTime.milliseconds())
+ assertEquals(1, cachedErrors.size)
+ assertTrue(cachedErrors.contains("test-topic-1"))
+ assertEquals("Topic 'test-topic-1' already exists.", cachedErrors("test-topic-1"))
+ }
+
+ @Test
+ def testGetTopicCreationErrorsWithMultipleTopics(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator,
+ mockTime)
+
+ val topics = Map(
+ "success-topic" -> new CreatableTopic().setName("success-topic").setNumPartitions(1).setReplicationFactor(1),
+ "failed-topic" -> new CreatableTopic().setName("failed-topic").setNumPartitions(1).setReplicationFactor(1)
+ )
+ val requestContext = initializeRequestContextWithUserPrincipal()
+ autoTopicCreationManager.createStreamsInternalTopics(topics, requestContext, config.groupCoordinatorConfig.streamsGroupHeartbeatIntervalMs() * 2)
+
+ val argumentCaptor = ArgumentCaptor.forClass(classOf[ControllerRequestCompletionHandler])
+ Mockito.verify(brokerToController).sendRequest(
+ any(classOf[AbstractRequest.Builder[_ <: AbstractRequest]]),
+ argumentCaptor.capture())
+
+ // Simulate mixed response - one success, one failure
+ val createTopicsResponseData = new org.apache.kafka.common.message.CreateTopicsResponseData()
+ createTopicsResponseData.topics().add(
+ new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("success-topic")
+ .setErrorCode(Errors.NONE.code())
+ )
+ createTopicsResponseData.topics().add(
+ new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("failed-topic")
+ .setErrorCode(Errors.POLICY_VIOLATION.code())
+ .setErrorMessage("Policy violation")
+ )
+
+ val createTopicsResponse = new CreateTopicsResponse(createTopicsResponseData)
+ val header = new RequestHeader(ApiKeys.CREATE_TOPICS, 0, "client", 1)
+ val clientResponse = new ClientResponse(header, null, null,
+ 0, 0, false, null, null, createTopicsResponse)
+
+ argumentCaptor.getValue.asInstanceOf[ControllerRequestCompletionHandler].onComplete(clientResponse)
+
+ // Only the failed topic should be cached
+ val cachedErrors = autoTopicCreationManager.getStreamsInternalTopicCreationErrors(Set("success-topic", "failed-topic", "nonexistent-topic"), mockTime.milliseconds())
+ assertEquals(1, cachedErrors.size)
+ assertTrue(cachedErrors.contains("failed-topic"))
+ assertEquals("Policy violation", cachedErrors("failed-topic"))
+ }
+
+ @Test
+ def testErrorCacheTTL(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator,
+ mockTime)
+
+
+ // First cache an error by simulating topic creation failure
+ val topics = Map(
+ "test-topic" -> new CreatableTopic().setName("test-topic").setNumPartitions(1).setReplicationFactor(1)
+ )
+ val requestContext = initializeRequestContextWithUserPrincipal()
+ val shortTtlMs = 1000L // Use 1 second TTL for faster testing
+ autoTopicCreationManager.createStreamsInternalTopics(topics, requestContext, shortTtlMs)
+
+ val argumentCaptor = ArgumentCaptor.forClass(classOf[ControllerRequestCompletionHandler])
+ Mockito.verify(brokerToController).sendRequest(
+ any(classOf[AbstractRequest.Builder[_ <: AbstractRequest]]),
+ argumentCaptor.capture())
+
+ // Simulate a CreateTopicsResponse with error
+ val createTopicsResponseData = new org.apache.kafka.common.message.CreateTopicsResponseData()
+ val topicResult = new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("test-topic")
+ .setErrorCode(Errors.INVALID_REPLICATION_FACTOR.code())
+ .setErrorMessage("Invalid replication factor")
+ createTopicsResponseData.topics().add(topicResult)
+
+ val createTopicsResponse = new CreateTopicsResponse(createTopicsResponseData)
+ val header = new RequestHeader(ApiKeys.CREATE_TOPICS, 0, "client", 1)
+ val clientResponse = new ClientResponse(header, null, null,
+ 0, 0, false, null, null, createTopicsResponse)
+
+ // Cache the error at T0
+ argumentCaptor.getValue.asInstanceOf[ControllerRequestCompletionHandler].onComplete(clientResponse)
+
+ // Verify error is cached and accessible within TTL
+ val cachedErrors = autoTopicCreationManager.getStreamsInternalTopicCreationErrors(Set("test-topic"), mockTime.milliseconds())
+ assertEquals(1, cachedErrors.size)
+ assertEquals("Invalid replication factor", cachedErrors("test-topic"))
+
+ // Advance time beyond TTL
+ mockTime.sleep(shortTtlMs + 100) // T0 + 1.1 seconds
+
+ // Verify error is now expired and proactively cleaned up
+ val expiredErrors = autoTopicCreationManager.getStreamsInternalTopicCreationErrors(Set("test-topic"), mockTime.milliseconds())
+ assertTrue(expiredErrors.isEmpty, "Expired errors should be proactively cleaned up")
+ }
+
+ @Test
+ def testErrorCacheLRUEviction(): Unit = {
+ // Create a config with a small cache size for testing
+ val props = TestUtils.createBrokerConfig(1)
+ props.setProperty(ServerConfigs.REQUEST_TIMEOUT_MS_CONFIG, requestTimeout.toString)
+ props.setProperty(ServerConfigs.MAX_INCREMENTAL_FETCH_SESSION_CACHE_SLOTS_CONFIG, "3") // Small cache size for testing
Review Comment:
I don't understand why we are doing this
##########
core/src/test/scala/unit/kafka/server/AutoTopicCreationManagerTest.scala:
##########
@@ -356,4 +371,226 @@ class AutoTopicCreationManagerTest {
.setNumPartitions(numPartitions)
.setReplicationFactor(replicationFactor)
}
+
+ @Test
+ def testTopicCreationErrorCaching(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator,
+ mockTime)
+
+ val topics = Map(
+ "test-topic-1" -> new CreatableTopic().setName("test-topic-1").setNumPartitions(1).setReplicationFactor(1)
+ )
+ val requestContext = initializeRequestContextWithUserPrincipal()
+
+ autoTopicCreationManager.createStreamsInternalTopics(topics, requestContext, config.groupCoordinatorConfig.streamsGroupHeartbeatIntervalMs() * 2)
+
+ val argumentCaptor = ArgumentCaptor.forClass(classOf[ControllerRequestCompletionHandler])
+ Mockito.verify(brokerToController).sendRequest(
+ any(classOf[AbstractRequest.Builder[_ <: AbstractRequest]]),
+ argumentCaptor.capture())
+
+ // Simulate a CreateTopicsResponse with errors
+ val createTopicsResponseData = new org.apache.kafka.common.message.CreateTopicsResponseData()
+ val topicResult = new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("test-topic-1")
+ .setErrorCode(Errors.TOPIC_ALREADY_EXISTS.code())
+ .setErrorMessage("Topic 'test-topic-1' already exists.")
+ createTopicsResponseData.topics().add(topicResult)
+
+ val createTopicsResponse = new CreateTopicsResponse(createTopicsResponseData)
+ val header = new RequestHeader(ApiKeys.CREATE_TOPICS, 0, "client", 1)
+ val clientResponse = new ClientResponse(header, null, null,
+ 0, 0, false, null, null, createTopicsResponse)
+
+ // Trigger the completion handler
+ argumentCaptor.getValue.asInstanceOf[ControllerRequestCompletionHandler].onComplete(clientResponse)
+
+ // Verify that the error was cached
+ val cachedErrors = autoTopicCreationManager.getStreamsInternalTopicCreationErrors(Set("test-topic-1"), mockTime.milliseconds())
+ assertEquals(1, cachedErrors.size)
+ assertTrue(cachedErrors.contains("test-topic-1"))
+ assertEquals("Topic 'test-topic-1' already exists.", cachedErrors("test-topic-1"))
+ }
+
+ @Test
+ def testGetTopicCreationErrorsWithMultipleTopics(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator,
+ mockTime)
+
+ val topics = Map(
+ "success-topic" -> new CreatableTopic().setName("success-topic").setNumPartitions(1).setReplicationFactor(1),
+ "failed-topic" -> new CreatableTopic().setName("failed-topic").setNumPartitions(1).setReplicationFactor(1)
+ )
+ val requestContext = initializeRequestContextWithUserPrincipal()
+ autoTopicCreationManager.createStreamsInternalTopics(topics, requestContext, config.groupCoordinatorConfig.streamsGroupHeartbeatIntervalMs() * 2)
+
+ val argumentCaptor = ArgumentCaptor.forClass(classOf[ControllerRequestCompletionHandler])
+ Mockito.verify(brokerToController).sendRequest(
+ any(classOf[AbstractRequest.Builder[_ <: AbstractRequest]]),
+ argumentCaptor.capture())
+
+ // Simulate mixed response - one success, one failure
+ val createTopicsResponseData = new org.apache.kafka.common.message.CreateTopicsResponseData()
+ createTopicsResponseData.topics().add(
+ new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("success-topic")
+ .setErrorCode(Errors.NONE.code())
+ )
+ createTopicsResponseData.topics().add(
+ new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("failed-topic")
+ .setErrorCode(Errors.POLICY_VIOLATION.code())
+ .setErrorMessage("Policy violation")
+ )
+
+ val createTopicsResponse = new CreateTopicsResponse(createTopicsResponseData)
+ val header = new RequestHeader(ApiKeys.CREATE_TOPICS, 0, "client", 1)
+ val clientResponse = new ClientResponse(header, null, null,
+ 0, 0, false, null, null, createTopicsResponse)
+
+ argumentCaptor.getValue.asInstanceOf[ControllerRequestCompletionHandler].onComplete(clientResponse)
+
+ // Only the failed topic should be cached
+ val cachedErrors = autoTopicCreationManager.getStreamsInternalTopicCreationErrors(Set("success-topic", "failed-topic", "nonexistent-topic"), mockTime.milliseconds())
+ assertEquals(1, cachedErrors.size)
+ assertTrue(cachedErrors.contains("failed-topic"))
+ assertEquals("Policy violation", cachedErrors("failed-topic"))
+ }
+
+ @Test
+ def testErrorCacheTTL(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator,
+ mockTime)
+
+
+ // First cache an error by simulating topic creation failure
+ val topics = Map(
+ "test-topic" -> new CreatableTopic().setName("test-topic").setNumPartitions(1).setReplicationFactor(1)
+ )
+ val requestContext = initializeRequestContextWithUserPrincipal()
+ val shortTtlMs = 1000L // Use 1 second TTL for faster testing
+ autoTopicCreationManager.createStreamsInternalTopics(topics, requestContext, shortTtlMs)
+
+ val argumentCaptor = ArgumentCaptor.forClass(classOf[ControllerRequestCompletionHandler])
+ Mockito.verify(brokerToController).sendRequest(
+ any(classOf[AbstractRequest.Builder[_ <: AbstractRequest]]),
+ argumentCaptor.capture())
+
+ // Simulate a CreateTopicsResponse with error
+ val createTopicsResponseData = new org.apache.kafka.common.message.CreateTopicsResponseData()
+ val topicResult = new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("test-topic")
+ .setErrorCode(Errors.INVALID_REPLICATION_FACTOR.code())
+ .setErrorMessage("Invalid replication factor")
+ createTopicsResponseData.topics().add(topicResult)
+
+ val createTopicsResponse = new CreateTopicsResponse(createTopicsResponseData)
+ val header = new RequestHeader(ApiKeys.CREATE_TOPICS, 0, "client", 1)
+ val clientResponse = new ClientResponse(header, null, null,
+ 0, 0, false, null, null, createTopicsResponse)
+
+ // Cache the error at T0
+ argumentCaptor.getValue.asInstanceOf[ControllerRequestCompletionHandler].onComplete(clientResponse)
+
+ // Verify error is cached and accessible within TTL
+ val cachedErrors = autoTopicCreationManager.getStreamsInternalTopicCreationErrors(Set("test-topic"), mockTime.milliseconds())
+ assertEquals(1, cachedErrors.size)
+ assertEquals("Invalid replication factor", cachedErrors("test-topic"))
+
+ // Advance time beyond TTL
+ mockTime.sleep(shortTtlMs + 100) // T0 + 1.1 seconds
+
+ // Verify error is now expired and proactively cleaned up
+ val expiredErrors = autoTopicCreationManager.getStreamsInternalTopicCreationErrors(Set("test-topic"), mockTime.milliseconds())
+ assertTrue(expiredErrors.isEmpty, "Expired errors should be proactively cleaned up")
+ }
+
+ @Test
+ def testErrorCacheLRUEviction(): Unit = {
+ // Create a config with a small cache size for testing
+ val props = TestUtils.createBrokerConfig(1)
+ props.setProperty(ServerConfigs.REQUEST_TIMEOUT_MS_CONFIG, requestTimeout.toString)
+ props.setProperty(ServerConfigs.MAX_INCREMENTAL_FETCH_SESSION_CACHE_SLOTS_CONFIG, "3") // Small cache size for testing
+
+ props.setProperty(GroupCoordinatorConfig.OFFSETS_TOPIC_REPLICATION_FACTOR_CONFIG, internalTopicPartitions.toString)
+ props.setProperty(TransactionLogConfig.TRANSACTIONS_TOPIC_REPLICATION_FACTOR_CONFIG, internalTopicPartitions.toString)
+ props.setProperty(ShareCoordinatorConfig.STATE_TOPIC_REPLICATION_FACTOR_CONFIG , internalTopicPartitions.toString)
+
+ props.setProperty(GroupCoordinatorConfig.OFFSETS_TOPIC_PARTITIONS_CONFIG, internalTopicReplicationFactor.toString)
+ props.setProperty(TransactionLogConfig.TRANSACTIONS_TOPIC_PARTITIONS_CONFIG, internalTopicReplicationFactor.toString)
+ props.setProperty(ShareCoordinatorConfig.STATE_TOPIC_NUM_PARTITIONS_CONFIG, internalTopicReplicationFactor.toString)
+
+ val smallCacheConfig = KafkaConfig.fromProps(props)
+
+ // Verify the configuration was properly set
+ assertEquals(3, smallCacheConfig.maxIncrementalFetchSessionCacheSlots, "Cache size configuration should be 3")
Review Comment:
Why are we doing this
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -50,21 +52,111 @@ trait AutoTopicCreationManager {
def createStreamsInternalTopics(
topics: Map[String, CreatableTopic],
- requestContext: RequestContext
+ requestContext: RequestContext,
+ timeoutMs: Long
): Unit
+ def getStreamsInternalTopicCreationErrors(
+ topicNames: Set[String],
+ currentTimeMs: Long
+ ): Map[String, String]
+
+ def close(): Unit = {}
+
+}
+
+/**
+ * Thread-safe cache that stores topic creation errors with per-entry expiration.
+ * - Expiration: maintained by a min-heap (priority queue) on expiration time
+ * - Capacity: enforced by insertion-order removal (keeps the most recently inserted entries)
+ */
+private[server] class ExpiringErrorCache(maxSize: Int, time: Time) {
+
+ private case class Entry(topicName: String, errorMessage: String, expirationTimeMs: Long)
+
+ private val byTopic = new ConcurrentHashMap[String, Entry]()
+ private val expiryQueue = new java.util.PriorityQueue[Entry](11, new java.util.Comparator[Entry] {
+ override def compare(a: Entry, b: Entry): Int = java.lang.Long.compare(a.expirationTimeMs, b.expirationTimeMs)
+ })
+ private val lock = new ReentrantLock()
+
+ def put(topicName: String, errorMessage: String, ttlMs: Long): Unit = {
+ lock.lock()
+ try {
+ val existing = byTopic.get(topicName)
+ if (existing != null) {
+ // Remove old instance from structures
+ expiryQueue.remove(existing)
Review Comment:
This remove is a linear time operation, right? I think we should avoid that. I think it may be fine to just leave it in the expiryQueue, since once it expired, we will no deletethe key from the map if the new value was replaced.
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -50,21 +52,111 @@ trait AutoTopicCreationManager {
def createStreamsInternalTopics(
topics: Map[String, CreatableTopic],
- requestContext: RequestContext
+ requestContext: RequestContext,
+ timeoutMs: Long
): Unit
+ def getStreamsInternalTopicCreationErrors(
+ topicNames: Set[String],
+ currentTimeMs: Long
+ ): Map[String, String]
+
+ def close(): Unit = {}
+
+}
+
+/**
+ * Thread-safe cache that stores topic creation errors with per-entry expiration.
+ * - Expiration: maintained by a min-heap (priority queue) on expiration time
+ * - Capacity: enforced by insertion-order removal (keeps the most recently inserted entries)
+ */
+private[server] class ExpiringErrorCache(maxSize: Int, time: Time) {
+
+ private case class Entry(topicName: String, errorMessage: String, expirationTimeMs: Long)
+
+ private val byTopic = new ConcurrentHashMap[String, Entry]()
+ private val expiryQueue = new java.util.PriorityQueue[Entry](11, new java.util.Comparator[Entry] {
+ override def compare(a: Entry, b: Entry): Int = java.lang.Long.compare(a.expirationTimeMs, b.expirationTimeMs)
+ })
+ private val lock = new ReentrantLock()
+
+ def put(topicName: String, errorMessage: String, ttlMs: Long): Unit = {
+ lock.lock()
+ try {
+ val existing = byTopic.get(topicName)
+ if (existing != null) {
+ // Remove old instance from structures
+ expiryQueue.remove(existing)
+ }
+
+ val currentTimeMs = time.milliseconds()
+ val expirationTimeMs = currentTimeMs + ttlMs
+ val entry = Entry(topicName, errorMessage, expirationTimeMs)
+ byTopic.put(topicName, entry)
+ expiryQueue.add(entry)
+
+ // Clean up expired entries
+ while (!expiryQueue.isEmpty && expiryQueue.peek().expirationTimeMs <= currentTimeMs) {
+ val expired = expiryQueue.poll()
+ val current = byTopic.get(expired.topicName)
+ if (current != null && (current eq expired)) {
+ byTopic.remove(expired.topicName)
+ }
+ }
+
+ // Enforce capacity by removing entries with earliest expiration time first
+ while (byTopic.size() > maxSize && !expiryQueue.isEmpty) {
Review Comment:
Could you merge this loop into the loop above by just checking the condition
`!expiryQueue.isEmpty && (expiryQueue.peek().expirationTimeMs <= currentTimeMs || byTopic.size() > maxSize)`
in the while loop?
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -50,21 +52,111 @@ trait AutoTopicCreationManager {
def createStreamsInternalTopics(
topics: Map[String, CreatableTopic],
- requestContext: RequestContext
+ requestContext: RequestContext,
+ timeoutMs: Long
): Unit
+ def getStreamsInternalTopicCreationErrors(
+ topicNames: Set[String],
+ currentTimeMs: Long
+ ): Map[String, String]
+
+ def close(): Unit = {}
+
+}
+
+/**
+ * Thread-safe cache that stores topic creation errors with per-entry expiration.
+ * - Expiration: maintained by a min-heap (priority queue) on expiration time
+ * - Capacity: enforced by insertion-order removal (keeps the most recently inserted entries)
+ */
+private[server] class ExpiringErrorCache(maxSize: Int, time: Time) {
+
+ private case class Entry(topicName: String, errorMessage: String, expirationTimeMs: Long)
+
+ private val byTopic = new ConcurrentHashMap[String, Entry]()
+ private val expiryQueue = new java.util.PriorityQueue[Entry](11, new java.util.Comparator[Entry] {
+ override def compare(a: Entry, b: Entry): Int = java.lang.Long.compare(a.expirationTimeMs, b.expirationTimeMs)
+ })
+ private val lock = new ReentrantLock()
+
+ def put(topicName: String, errorMessage: String, ttlMs: Long): Unit = {
+ lock.lock()
+ try {
+ val existing = byTopic.get(topicName)
+ if (existing != null) {
+ // Remove old instance from structures
+ expiryQueue.remove(existing)
+ }
+
+ val currentTimeMs = time.milliseconds()
+ val expirationTimeMs = currentTimeMs + ttlMs
+ val entry = Entry(topicName, errorMessage, expirationTimeMs)
+ byTopic.put(topicName, entry)
+ expiryQueue.add(entry)
+
+ // Clean up expired entries
+ while (!expiryQueue.isEmpty && expiryQueue.peek().expirationTimeMs <= currentTimeMs) {
+ val expired = expiryQueue.poll()
+ val current = byTopic.get(expired.topicName)
+ if (current != null && (current eq expired)) {
Review Comment:
Is `eq` doing a deep comparison here? Maybe it would be enough to compare the timestamps, the deep comparison is expensive
##########
core/src/main/scala/kafka/server/KafkaApis.scala:
##########
@@ -2888,10 +2888,35 @@ class KafkaApis(val requestChannel: RequestChannel,
)
}
} else {
- autoTopicCreationManager.createStreamsInternalTopics(topicsToCreate, requestContext);
+ // Compute group-specific timeout for caching errors (2 * heartbeat interval)
+ val heartbeatIntervalMs = Option(groupConfigManager.groupConfig(streamsGroupHeartbeatRequest.data.groupId).orElse(null))
Review Comment:
The TTL is calculated as 2 × heartbeat interval, but the PR description mentions 3 × request.timeout.ms. This inconsistency could be confusing. Can you please fix the PR description? I would keep it much shorter and less AI generated so that it is easier to keep up-to-date.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "RaidenE1 (via GitHub)" <gi...@apache.org>.
RaidenE1 commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2270909118
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -112,6 +128,45 @@ class DefaultAutoTopicCreationManager(
}
}
+ override def getTopicCreationErrors(
+ topicNames: Set[String]
+ ): Map[String, String] = {
+ val currentTime = System.currentTimeMillis()
+ val errors = mutable.Map.empty[String, String]
+ val expiredKeys = mutable.Set.empty[String]
+
+ // Check requested topics and collect expired keys
+ topicNames.foreach { topicName =>
+ Option(topicCreationErrorCache.get(topicName)) match {
+ case Some(cachedError) if (currentTime - cachedError.timestamp) <= errorCacheTtlMs =>
+ errors.put(topicName, cachedError.errorMessage)
+ case Some(_) =>
Review Comment:
@mjsax The case Some(_) will indeed be executed. In Scala pattern matching:
1. case Some(cachedError) if (condition) only matches when the guard condition is true
2. If the condition is false (TTL expired), it falls through to case Some(_)
3. So we hit the second case when we have a cached error that's expired
The pattern matching is sequential with guard conditions, not a "catch all" for the
first Some(...).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2315358917
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -53,17 +55,47 @@ trait AutoTopicCreationManager {
requestContext: RequestContext
): Unit
+ def getTopicCreationErrors(
+ topicNames: Set[String],
+ errorCacheTtlMs: Long
+ ): Map[String, String]
+
+ def close(): Unit = {}
+
}
+case class CachedTopicCreationError(
+ errorMessage: String,
+ time: Time
+) {
+ val timestamp: Long = time.milliseconds()
Review Comment:
The timestamp should be captured when the error is created, not when accessed. The current implementation captures the timestamp on object initialization, but since `time.milliseconds()` is called every time the case class is instantiated, this could lead to inconsistent timestamps if the Time instance is mutable or if multiple instances share the same Time object.
```suggestion
errorMessage: String
) {
val timestamp: Long = System.currentTimeMillis()
```
##########
core/src/main/scala/kafka/server/KafkaApis.scala:
##########
@@ -2889,9 +2889,31 @@ class KafkaApis(val requestChannel: RequestChannel,
}
} else {
autoTopicCreationManager.createStreamsInternalTopics(topicsToCreate, requestContext);
+
+ // Check for cached topic creation errors only if there's already a MISSING_INTERNAL_TOPICS status
+ val hasMissingInternalTopicsStatus = responseData.status() != null &&
+ responseData.status().stream().anyMatch(s => s.statusCode() == StreamsGroupHeartbeatResponse.Status.MISSING_INTERNAL_TOPICS.code())
+
+ if (hasMissingInternalTopicsStatus) {
+ // Calculate group-specific error cache TTL
+ val errorCacheTtlMs = Option(groupConfigManager.groupConfig(streamsGroupHeartbeatRequest.data.groupId).orElse(null))
+ .map(_.streamsSessionTimeoutMs().toLong)
+ .getOrElse(config.groupCoordinatorConfig.streamsGroupSessionTimeoutMs().toLong)
+
+ val cachedErrors = autoTopicCreationManager.getTopicCreationErrors(topicsToCreate.keys.toSet, errorCacheTtlMs)
+ if (cachedErrors.nonEmpty) {
+ val missingInternalTopicStatus =
+ responseData.status().stream().filter(x => x.statusCode() == StreamsGroupHeartbeatResponse.Status.MISSING_INTERNAL_TOPICS.code()).findFirst()
+ val creationErrorDetails = cachedErrors.map { case (topic, error) => s"$topic ($error)" }.mkString(", ")
+ if (missingInternalTopicStatus.isPresent) {
+ missingInternalTopicStatus.get().setStatusDetail(
+ missingInternalTopicStatus.get().statusDetail() + s"; Creation failed: $creationErrorDetails."
+ )
Review Comment:
Potential null pointer exception if `statusDetail()` returns null. The concatenation should handle the case where the existing status detail is null.
##########
core/src/test/scala/unit/kafka/server/AutoTopicCreationManagerTest.scala:
##########
@@ -356,4 +365,230 @@ class AutoTopicCreationManagerTest {
.setNumPartitions(numPartitions)
.setReplicationFactor(replicationFactor)
}
+
+ @Test
+ def testTopicCreationErrorCaching(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator,
+ mockTime)
+
+ val topics = Map(
+ "test-topic-1" -> new CreatableTopic().setName("test-topic-1").setNumPartitions(1).setReplicationFactor(1)
+ )
+ val requestContext = initializeRequestContextWithUserPrincipal()
+
+ autoTopicCreationManager.createStreamsInternalTopics(topics, requestContext)
+
+ val argumentCaptor = ArgumentCaptor.forClass(classOf[ControllerRequestCompletionHandler])
+ Mockito.verify(brokerToController).sendRequest(
+ any(classOf[AbstractRequest.Builder[_ <: AbstractRequest]]),
+ argumentCaptor.capture())
+
+ // Simulate a CreateTopicsResponse with errors
+ val createTopicsResponseData = new org.apache.kafka.common.message.CreateTopicsResponseData()
+ val topicResult = new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("test-topic-1")
+ .setErrorCode(Errors.TOPIC_ALREADY_EXISTS.code())
+ .setErrorMessage("Topic 'test-topic-1' already exists.")
+ createTopicsResponseData.topics().add(topicResult)
+
+ val createTopicsResponse = new CreateTopicsResponse(createTopicsResponseData)
+ val header = new RequestHeader(ApiKeys.CREATE_TOPICS, 0, "client", 1)
+ val clientResponse = new ClientResponse(header, null, null,
+ 0, 0, false, null, null, createTopicsResponse)
+
+ // Trigger the completion handler
+ argumentCaptor.getValue.asInstanceOf[ControllerRequestCompletionHandler].onComplete(clientResponse)
+
+ // Verify that the error was cached
+ val defaultTtlMs = config.groupCoordinatorConfig.streamsGroupSessionTimeoutMs()
+ val cachedErrors = autoTopicCreationManager.getTopicCreationErrors(Set("test-topic-1"), defaultTtlMs)
+ assertEquals(1, cachedErrors.size)
+ assertTrue(cachedErrors.contains("test-topic-1"))
+ assertEquals("Topic 'test-topic-1' already exists.", cachedErrors("test-topic-1"))
+ }
+
+ @Test
+ def testGetTopicCreationErrorsWithMultipleTopics(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator,
+ mockTime)
+
+ val topics = Map(
+ "success-topic" -> new CreatableTopic().setName("success-topic").setNumPartitions(1).setReplicationFactor(1),
+ "failed-topic" -> new CreatableTopic().setName("failed-topic").setNumPartitions(1).setReplicationFactor(1)
+ )
+ val requestContext = initializeRequestContextWithUserPrincipal()
+ autoTopicCreationManager.createStreamsInternalTopics(topics, requestContext)
+
+ val argumentCaptor = ArgumentCaptor.forClass(classOf[ControllerRequestCompletionHandler])
+ Mockito.verify(brokerToController).sendRequest(
+ any(classOf[AbstractRequest.Builder[_ <: AbstractRequest]]),
+ argumentCaptor.capture())
+
+ // Simulate mixed response - one success, one failure
+ val createTopicsResponseData = new org.apache.kafka.common.message.CreateTopicsResponseData()
+ createTopicsResponseData.topics().add(
+ new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("success-topic")
+ .setErrorCode(Errors.NONE.code())
+ )
+ createTopicsResponseData.topics().add(
+ new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("failed-topic")
+ .setErrorCode(Errors.POLICY_VIOLATION.code())
+ .setErrorMessage("Policy violation")
+ )
+
+ val createTopicsResponse = new CreateTopicsResponse(createTopicsResponseData)
+ val header = new RequestHeader(ApiKeys.CREATE_TOPICS, 0, "client", 1)
+ val clientResponse = new ClientResponse(header, null, null,
+ 0, 0, false, null, null, createTopicsResponse)
+
+ argumentCaptor.getValue.asInstanceOf[ControllerRequestCompletionHandler].onComplete(clientResponse)
+
+ // Only the failed topic should be cached
+ val defaultTtlMs = config.groupCoordinatorConfig.streamsGroupSessionTimeoutMs()
+ val cachedErrors = autoTopicCreationManager.getTopicCreationErrors(Set("success-topic", "failed-topic", "nonexistent-topic"), defaultTtlMs)
+ assertEquals(1, cachedErrors.size)
+ assertTrue(cachedErrors.contains("failed-topic"))
+ assertEquals("Policy violation", cachedErrors("failed-topic"))
+ }
+
+ @Test
+ def testErrorCacheTTL(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator,
+ mockTime)
+
+
+ // First cache an error by simulating topic creation failure
+ val topics = Map(
+ "test-topic" -> new CreatableTopic().setName("test-topic").setNumPartitions(1).setReplicationFactor(1)
+ )
+ val requestContext = initializeRequestContextWithUserPrincipal()
+ autoTopicCreationManager.createStreamsInternalTopics(topics, requestContext)
+
+ val argumentCaptor = ArgumentCaptor.forClass(classOf[ControllerRequestCompletionHandler])
+ Mockito.verify(brokerToController).sendRequest(
+ any(classOf[AbstractRequest.Builder[_ <: AbstractRequest]]),
+ argumentCaptor.capture())
+
+ // Simulate a CreateTopicsResponse with error
+ val createTopicsResponseData = new org.apache.kafka.common.message.CreateTopicsResponseData()
+ val topicResult = new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("test-topic")
+ .setErrorCode(Errors.INVALID_REPLICATION_FACTOR.code())
+ .setErrorMessage("Invalid replication factor")
+ createTopicsResponseData.topics().add(topicResult)
+
+ val createTopicsResponse = new CreateTopicsResponse(createTopicsResponseData)
+ val header = new RequestHeader(ApiKeys.CREATE_TOPICS, 0, "client", 1)
+ val clientResponse = new ClientResponse(header, null, null,
+ 0, 0, false, null, null, createTopicsResponse)
+
+ // Cache the error at T0
+ argumentCaptor.getValue.asInstanceOf[ControllerRequestCompletionHandler].onComplete(clientResponse)
+
+ val shortTtlMs = 1000L // Use 1 second TTL for faster testing
+
+ // Verify error is cached and accessible within TTL
+ val cachedErrors = autoTopicCreationManager.getTopicCreationErrors(Set("test-topic"), shortTtlMs)
+ assertEquals(1, cachedErrors.size)
+ assertEquals("Invalid replication factor", cachedErrors("test-topic"))
+
+ // Advance time beyond TTL
+ mockTime.sleep(shortTtlMs + 100) // T0 + 1.1 seconds
+
+ // Verify error is now expired and proactively cleaned up
+ val expiredErrors = autoTopicCreationManager.getTopicCreationErrors(Set("test-topic"), shortTtlMs)
+ assertTrue(expiredErrors.isEmpty, "Expired errors should be proactively cleaned up")
+ }
+
+ @Test
+ def testErrorCacheLRUEviction(): Unit = {
+ // Create a config with a small cache size for testing
+ val props = TestUtils.createBrokerConfig(1)
+ props.setProperty(ServerConfigs.REQUEST_TIMEOUT_MS_CONFIG, requestTimeout.toString)
+ props.setProperty(ServerConfigs.MAX_INCREMENTAL_FETCH_SESSION_CACHE_SLOTS_CONFIG, "3") // Small cache size for testing
+
+ props.setProperty(GroupCoordinatorConfig.OFFSETS_TOPIC_REPLICATION_FACTOR_CONFIG, internalTopicPartitions.toString)
+ props.setProperty(TransactionLogConfig.TRANSACTIONS_TOPIC_REPLICATION_FACTOR_CONFIG, internalTopicPartitions.toString)
+ props.setProperty(ShareCoordinatorConfig.STATE_TOPIC_REPLICATION_FACTOR_CONFIG , internalTopicPartitions.toString)
+
+ props.setProperty(GroupCoordinatorConfig.OFFSETS_TOPIC_PARTITIONS_CONFIG, internalTopicReplicationFactor.toString)
+ props.setProperty(TransactionLogConfig.TRANSACTIONS_TOPIC_PARTITIONS_CONFIG, internalTopicReplicationFactor.toString)
+ props.setProperty(ShareCoordinatorConfig.STATE_TOPIC_NUM_PARTITIONS_CONFIG, internalTopicReplicationFactor.toString)
+
+ val smallCacheConfig = KafkaConfig.fromProps(props)
+
+ // Verify the configuration was properly set
+ assertEquals(3, smallCacheConfig.maxIncrementalFetchSessionCacheSlots, "Cache size configuration should be 3")
+
+ // Replace the test class's config with our smallCacheConfig
+ // so that initializeRequestContext will use the correct config
+ config = smallCacheConfig
Review Comment:
Modifying the test class's config field directly could affect other tests if they run in the same instance. Consider creating a separate AutoTopicCreationManager instance with the small cache config instead of modifying the shared config field.
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -96,7 +128,56 @@ class DefaultAutoTopicCreationManager(
requestContext: RequestContext
): Unit = {
if (topics.nonEmpty) {
- sendCreateTopicRequest(topics, Some(requestContext))
+ sendCreateTopicRequestWithErrorCaching(topics, Some(requestContext))
+ }
+ }
+
+ override def getTopicCreationErrors(
+ topicNames: Set[String],
+ errorCacheTtlMs: Long
+ ): Map[String, String] = {
+ // Proactively expire old entries using the provided TTL
+ expireOldEntries(errorCacheTtlMs)
+
+ val errors = mutable.Map.empty[String, String]
+
+ // Check requested topics
+ topicNames.foreach { topicName =>
+ Option(topicCreationErrorCache.get(topicName)) match {
+ case Some(error) =>
+ errors.put(topicName, error.errorMessage)
+ case None =>
+ }
+ }
+
+ errors.toMap
+ }
+
+ /**
+ * Remove expired entries from the cache using the provided TTL.
+ * Since we use LinkedHashMap with insertion order, we only need to check
+ * entries from the beginning until we find a non-expired entry.
+ */
+ private def expireOldEntries(ttlMs: Long): Unit = {
+ val currentTime = time.milliseconds()
+
+ // Iterate and remove expired entries
+ val iterator = topicCreationErrorCache.entrySet().iterator()
+
+ breakable {
+ while (iterator.hasNext) {
+ val entry = iterator.next()
+ val cachedError = entry.getValue
+
+ if (currentTime - cachedError.timestamp > ttlMs) {
+ iterator.remove()
+ debug(s"Removed expired topic creation error cache entry for ${entry.getKey}")
+ } else {
+ // Since entries are in insertion order, if this entry is not expired,
+ // all following entries are also not expired
+ break()
+ }
Review Comment:
The assumption that entries are in insertion order and that if one entry is not expired, all following entries are also not expired is incorrect. The LinkedHashMap is configured with `accessOrder = false` (line 94), meaning it maintains insertion order, but entries can have different timestamps based on when they were inserted. Breaking early could leave expired entries in the cache.
```suggestion
while (iterator.hasNext) {
val entry = iterator.next()
val cachedError = entry.getValue
if (currentTime - cachedError.timestamp > ttlMs) {
iterator.remove()
debug(s"Removed expired topic creation error cache entry for ${entry.getKey}")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "RaidenE1 (via GitHub)" <gi...@apache.org>.
RaidenE1 commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2347859336
##########
core/src/test/scala/unit/kafka/server/AutoTopicCreationManagerTest.scala:
##########
@@ -356,4 +371,226 @@ class AutoTopicCreationManagerTest {
.setNumPartitions(numPartitions)
.setReplicationFactor(replicationFactor)
}
+
+ @Test
+ def testTopicCreationErrorCaching(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator,
+ mockTime)
+
+ val topics = Map(
+ "test-topic-1" -> new CreatableTopic().setName("test-topic-1").setNumPartitions(1).setReplicationFactor(1)
+ )
+ val requestContext = initializeRequestContextWithUserPrincipal()
+
+ autoTopicCreationManager.createStreamsInternalTopics(topics, requestContext, config.groupCoordinatorConfig.streamsGroupHeartbeatIntervalMs() * 2)
+
+ val argumentCaptor = ArgumentCaptor.forClass(classOf[ControllerRequestCompletionHandler])
+ Mockito.verify(brokerToController).sendRequest(
+ any(classOf[AbstractRequest.Builder[_ <: AbstractRequest]]),
+ argumentCaptor.capture())
+
+ // Simulate a CreateTopicsResponse with errors
+ val createTopicsResponseData = new org.apache.kafka.common.message.CreateTopicsResponseData()
+ val topicResult = new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("test-topic-1")
+ .setErrorCode(Errors.TOPIC_ALREADY_EXISTS.code())
+ .setErrorMessage("Topic 'test-topic-1' already exists.")
+ createTopicsResponseData.topics().add(topicResult)
+
+ val createTopicsResponse = new CreateTopicsResponse(createTopicsResponseData)
+ val header = new RequestHeader(ApiKeys.CREATE_TOPICS, 0, "client", 1)
+ val clientResponse = new ClientResponse(header, null, null,
+ 0, 0, false, null, null, createTopicsResponse)
+
+ // Trigger the completion handler
+ argumentCaptor.getValue.asInstanceOf[ControllerRequestCompletionHandler].onComplete(clientResponse)
+
+ // Verify that the error was cached
+ val cachedErrors = autoTopicCreationManager.getStreamsInternalTopicCreationErrors(Set("test-topic-1"), mockTime.milliseconds())
+ assertEquals(1, cachedErrors.size)
+ assertTrue(cachedErrors.contains("test-topic-1"))
+ assertEquals("Topic 'test-topic-1' already exists.", cachedErrors("test-topic-1"))
+ }
+
+ @Test
+ def testGetTopicCreationErrorsWithMultipleTopics(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator,
+ mockTime)
+
+ val topics = Map(
+ "success-topic" -> new CreatableTopic().setName("success-topic").setNumPartitions(1).setReplicationFactor(1),
+ "failed-topic" -> new CreatableTopic().setName("failed-topic").setNumPartitions(1).setReplicationFactor(1)
+ )
+ val requestContext = initializeRequestContextWithUserPrincipal()
+ autoTopicCreationManager.createStreamsInternalTopics(topics, requestContext, config.groupCoordinatorConfig.streamsGroupHeartbeatIntervalMs() * 2)
+
+ val argumentCaptor = ArgumentCaptor.forClass(classOf[ControllerRequestCompletionHandler])
+ Mockito.verify(brokerToController).sendRequest(
+ any(classOf[AbstractRequest.Builder[_ <: AbstractRequest]]),
+ argumentCaptor.capture())
+
+ // Simulate mixed response - one success, one failure
+ val createTopicsResponseData = new org.apache.kafka.common.message.CreateTopicsResponseData()
+ createTopicsResponseData.topics().add(
+ new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("success-topic")
+ .setErrorCode(Errors.NONE.code())
+ )
+ createTopicsResponseData.topics().add(
+ new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("failed-topic")
+ .setErrorCode(Errors.POLICY_VIOLATION.code())
+ .setErrorMessage("Policy violation")
+ )
+
+ val createTopicsResponse = new CreateTopicsResponse(createTopicsResponseData)
+ val header = new RequestHeader(ApiKeys.CREATE_TOPICS, 0, "client", 1)
+ val clientResponse = new ClientResponse(header, null, null,
+ 0, 0, false, null, null, createTopicsResponse)
+
+ argumentCaptor.getValue.asInstanceOf[ControllerRequestCompletionHandler].onComplete(clientResponse)
+
+ // Only the failed topic should be cached
+ val cachedErrors = autoTopicCreationManager.getStreamsInternalTopicCreationErrors(Set("success-topic", "failed-topic", "nonexistent-topic"), mockTime.milliseconds())
+ assertEquals(1, cachedErrors.size)
+ assertTrue(cachedErrors.contains("failed-topic"))
+ assertEquals("Policy violation", cachedErrors("failed-topic"))
+ }
+
+ @Test
+ def testErrorCacheTTL(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator,
+ mockTime)
+
+
+ // First cache an error by simulating topic creation failure
+ val topics = Map(
+ "test-topic" -> new CreatableTopic().setName("test-topic").setNumPartitions(1).setReplicationFactor(1)
+ )
+ val requestContext = initializeRequestContextWithUserPrincipal()
+ val shortTtlMs = 1000L // Use 1 second TTL for faster testing
+ autoTopicCreationManager.createStreamsInternalTopics(topics, requestContext, shortTtlMs)
+
+ val argumentCaptor = ArgumentCaptor.forClass(classOf[ControllerRequestCompletionHandler])
+ Mockito.verify(brokerToController).sendRequest(
+ any(classOf[AbstractRequest.Builder[_ <: AbstractRequest]]),
+ argumentCaptor.capture())
+
+ // Simulate a CreateTopicsResponse with error
+ val createTopicsResponseData = new org.apache.kafka.common.message.CreateTopicsResponseData()
+ val topicResult = new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("test-topic")
+ .setErrorCode(Errors.INVALID_REPLICATION_FACTOR.code())
+ .setErrorMessage("Invalid replication factor")
+ createTopicsResponseData.topics().add(topicResult)
+
+ val createTopicsResponse = new CreateTopicsResponse(createTopicsResponseData)
+ val header = new RequestHeader(ApiKeys.CREATE_TOPICS, 0, "client", 1)
+ val clientResponse = new ClientResponse(header, null, null,
+ 0, 0, false, null, null, createTopicsResponse)
+
+ // Cache the error at T0
+ argumentCaptor.getValue.asInstanceOf[ControllerRequestCompletionHandler].onComplete(clientResponse)
+
+ // Verify error is cached and accessible within TTL
+ val cachedErrors = autoTopicCreationManager.getStreamsInternalTopicCreationErrors(Set("test-topic"), mockTime.milliseconds())
+ assertEquals(1, cachedErrors.size)
+ assertEquals("Invalid replication factor", cachedErrors("test-topic"))
+
+ // Advance time beyond TTL
+ mockTime.sleep(shortTtlMs + 100) // T0 + 1.1 seconds
+
+ // Verify error is now expired and proactively cleaned up
+ val expiredErrors = autoTopicCreationManager.getStreamsInternalTopicCreationErrors(Set("test-topic"), mockTime.milliseconds())
+ assertTrue(expiredErrors.isEmpty, "Expired errors should be proactively cleaned up")
+ }
+
+ @Test
+ def testErrorCacheLRUEviction(): Unit = {
+ // Create a config with a small cache size for testing
+ val props = TestUtils.createBrokerConfig(1)
+ props.setProperty(ServerConfigs.REQUEST_TIMEOUT_MS_CONFIG, requestTimeout.toString)
+ props.setProperty(ServerConfigs.MAX_INCREMENTAL_FETCH_SESSION_CACHE_SLOTS_CONFIG, "3") // Small cache size for testing
Review Comment:
I think the cache size it set to 1000 by default, if we want to test its limit, then we need to create 1000 exception in the test?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2275159146
##########
core/src/main/scala/kafka/server/KafkaApis.scala:
##########
@@ -2889,6 +2889,24 @@ class KafkaApis(val requestChannel: RequestChannel,
}
} else {
autoTopicCreationManager.createStreamsInternalTopics(topicsToCreate, requestContext);
+
+ // Check for cached topic creation errors only if there's already a MISSING_INTERNAL_TOPICS status
+ val hasMissingInternalTopicsStatus = responseData.status() != null &&
+ responseData.status().stream().anyMatch(s => s.statusCode() == StreamsGroupHeartbeatResponse.Status.MISSING_INTERNAL_TOPICS.code())
Review Comment:
> If it's not missing then there is no error?
Yes, maybe, but it seems the check is redundant? If there is any error in the cache, we know the topic is missing and we report the error. If the cache is empty, there is nothing to be reported (but we don't need to check the error code to find out that the cache is empty, we just look at the cache)?
Of course, if a retry to create a topic is successful, we need to clear any previous error for this topic from the cache to not incorrectly report an older error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "RaidenE1 (via GitHub)" <gi...@apache.org>.
RaidenE1 commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2270818122
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -64,6 +77,9 @@ class DefaultAutoTopicCreationManager(
) extends AutoTopicCreationManager with Logging {
private val inflightTopics = Collections.newSetFromMap(new ConcurrentHashMap[String, java.lang.Boolean]())
+ private val topicCreationErrorCache = new ConcurrentHashMap[String, CachedTopicCreationError]()
+ private val errorCacheTtlMs = config.requestTimeoutMs.toLong * 3 // 3x request timeout
Review Comment:
1. changed
2. I think the topic is globally used, so if it's missing then it's missing for all groups
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "RaidenE1 (via GitHub)" <gi...@apache.org>.
RaidenE1 commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2270921287
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -112,6 +128,45 @@ class DefaultAutoTopicCreationManager(
}
}
+ override def getTopicCreationErrors(
+ topicNames: Set[String]
+ ): Map[String, String] = {
+ val currentTime = System.currentTimeMillis()
+ val errors = mutable.Map.empty[String, String]
+ val expiredKeys = mutable.Set.empty[String]
+
+ // Check requested topics and collect expired keys
+ topicNames.foreach { topicName =>
+ Option(topicCreationErrorCache.get(topicName)) match {
+ case Some(cachedError) if (currentTime - cachedError.timestamp) <= errorCacheTtlMs =>
Review Comment:
I think it makes sense
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "lucasbru (via GitHub)" <gi...@apache.org>.
lucasbru commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2353370452
##########
core/src/test/scala/unit/kafka/server/KafkaApisTest.scala:
##########
@@ -10949,6 +10950,61 @@ class KafkaApisTest extends Logging {
)
}
+ @Test
+ def testStreamsGroupHeartbeatRequestWithCachedTopicCreationErrors(): Unit = {
Review Comment:
That's using mocks. But I suppose we can test it later on.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "lucasbru (via GitHub)" <gi...@apache.org>.
lucasbru commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2316984834
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -122,9 +232,11 @@ class DefaultAutoTopicCreationManager(
override def onComplete(response: ClientResponse): Unit = {
clearInflightRequests(creatableTopics)
if (response.authenticationException() != null) {
- warn(s"Auto topic creation failed for ${creatableTopics.keys} with authentication exception")
+ val authException = response.authenticationException()
Review Comment:
nit: lets not change this function at all, it's not related to what we are doing.
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -50,21 +52,121 @@ trait AutoTopicCreationManager {
def createStreamsInternalTopics(
topics: Map[String, CreatableTopic],
- requestContext: RequestContext
+ requestContext: RequestContext,
+ expirationTimeMs: Long
): Unit
+ def getStreamsInternalTopicCreationErrors(
+ topicNames: Set[String],
+ currentTimeMs: Long
+ ): Map[String, String]
+
+ def close(): Unit = {}
+
+}
+
+/**
+ * Thread-safe cache that stores topic creation errors with per-entry expiration.
+ * - Expiration: maintained by a min-heap (priority queue) on expiration time
+ * - Capacity: enforced by insertion-order removal (keeps the most recently inserted entries)
+ */
+class ExpiringErrorCache(maxSize: Int) {
+
+ private case class Entry(topicName: String, errorMessage: String, expirationTimeMs: Long)
+
+ private val byTopic = new java.util.HashMap[String, Entry]()
+ private val expiryQueue = new java.util.PriorityQueue[Entry](11, new java.util.Comparator[Entry] {
+ override def compare(a: Entry, b: Entry): Int = java.lang.Long.compare(a.expirationTimeMs, b.expirationTimeMs)
+ })
+ private val lock = new ReentrantLock()
+
+ def put(topicName: String, errorMessage: String, expirationTimeMs: Long): Unit = {
+ lock.lock()
+ try {
+ val existing = byTopic.get(topicName)
+ if (existing != null) {
+ // Remove old instance from structures
+ expiryQueue.remove(existing)
+ }
+
+ val entry = Entry(topicName, errorMessage, expirationTimeMs)
+ byTopic.put(topicName, entry)
+ expiryQueue.add(entry)
+
+ // Enforce capacity by removing entries with earliest expiration time first
+ while (byTopic.size() > maxSize && !expiryQueue.isEmpty) {
+ val evicted = expiryQueue.poll()
+ if (evicted != null) {
+ val current = byTopic.get(evicted.topicName)
+ if (current != null && (current eq evicted)) {
+ byTopic.remove(evicted.topicName)
+ }
+ }
+ }
+ } finally {
+ lock.unlock()
+ }
+ }
+
+ def getErrorsForTopics(topicNames: Set[String], currentTimeMs: Long): Map[String, String] = {
+ lock.lock()
+ try {
+ cleanupExpired(currentTimeMs)
+ val result = mutable.Map.empty[String, String]
+ topicNames.foreach { topicName =>
+ val entry = byTopic.get(topicName)
+ if (entry != null) {
+ result.put(topicName, entry.errorMessage)
+ }
+ }
+ result.toMap
+ } finally {
+ lock.unlock()
+ }
+ }
+
+ def cleanupExpired(currentTimeMs: Long): Unit = {
+ lock.lock()
+ try {
+ while (!expiryQueue.isEmpty && expiryQueue.peek().expirationTimeMs <= currentTimeMs) {
+ val expired = expiryQueue.poll()
+ val current = byTopic.get(expired.topicName)
+ if (current != null && (current eq expired)) {
+ byTopic.remove(expired.topicName)
+ }
+ }
+ } finally {
+ lock.unlock()
+ }
+ }
+
+ def clear(): Unit = {
+ lock.lock()
+ try {
+ byTopic.clear()
+ expiryQueue.clear()
+ } finally {
+ lock.unlock()
+ }
+ }
}
+
class DefaultAutoTopicCreationManager(
config: KafkaConfig,
channelManager: NodeToControllerChannelManager,
groupCoordinator: GroupCoordinator,
txnCoordinator: TransactionCoordinator,
- shareCoordinator: ShareCoordinator
+ shareCoordinator: ShareCoordinator,
+ time: Time,
+ cacheCapacity: Int = 1000
Review Comment:
nit: topicErrorCacheCapacity
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -50,21 +52,121 @@ trait AutoTopicCreationManager {
def createStreamsInternalTopics(
topics: Map[String, CreatableTopic],
- requestContext: RequestContext
+ requestContext: RequestContext,
+ expirationTimeMs: Long
): Unit
+ def getStreamsInternalTopicCreationErrors(
+ topicNames: Set[String],
+ currentTimeMs: Long
+ ): Map[String, String]
+
+ def close(): Unit = {}
+
+}
+
+/**
+ * Thread-safe cache that stores topic creation errors with per-entry expiration.
+ * - Expiration: maintained by a min-heap (priority queue) on expiration time
+ * - Capacity: enforced by insertion-order removal (keeps the most recently inserted entries)
+ */
+class ExpiringErrorCache(maxSize: Int) {
+
+ private case class Entry(topicName: String, errorMessage: String, expirationTimeMs: Long)
+
+ private val byTopic = new java.util.HashMap[String, Entry]()
+ private val expiryQueue = new java.util.PriorityQueue[Entry](11, new java.util.Comparator[Entry] {
+ override def compare(a: Entry, b: Entry): Int = java.lang.Long.compare(a.expirationTimeMs, b.expirationTimeMs)
+ })
+ private val lock = new ReentrantLock()
+
+ def put(topicName: String, errorMessage: String, expirationTimeMs: Long): Unit = {
+ lock.lock()
+ try {
+ val existing = byTopic.get(topicName)
+ if (existing != null) {
+ // Remove old instance from structures
+ expiryQueue.remove(existing)
+ }
+
+ val entry = Entry(topicName, errorMessage, expirationTimeMs)
+ byTopic.put(topicName, entry)
+ expiryQueue.add(entry)
+
+ // Enforce capacity by removing entries with earliest expiration time first
+ while (byTopic.size() > maxSize && !expiryQueue.isEmpty) {
+ val evicted = expiryQueue.poll()
+ if (evicted != null) {
+ val current = byTopic.get(evicted.topicName)
+ if (current != null && (current eq evicted)) {
+ byTopic.remove(evicted.topicName)
+ }
+ }
+ }
+ } finally {
+ lock.unlock()
+ }
+ }
+
+ def getErrorsForTopics(topicNames: Set[String], currentTimeMs: Long): Map[String, String] = {
+ lock.lock()
+ try {
+ cleanupExpired(currentTimeMs)
+ val result = mutable.Map.empty[String, String]
+ topicNames.foreach { topicName =>
+ val entry = byTopic.get(topicName)
+ if (entry != null) {
+ result.put(topicName, entry.errorMessage)
+ }
+ }
+ result.toMap
+ } finally {
+ lock.unlock()
+ }
+ }
+
+ def cleanupExpired(currentTimeMs: Long): Unit = {
+ lock.lock()
Review Comment:
Why is this not private? Isn't this called from places where we already own the lock?
##########
core/src/main/scala/kafka/server/KafkaApis.scala:
##########
@@ -2888,10 +2888,34 @@ class KafkaApis(val requestChannel: RequestChannel,
)
}
} else {
- autoTopicCreationManager.createStreamsInternalTopics(topicsToCreate, requestContext);
+ // Compute group-specific expiration time for freshly cached errors
+ val sessionTimeoutMs = Option(groupConfigManager.groupConfig(streamsGroupHeartbeatRequest.data.groupId).orElse(null))
+ .map(_.streamsSessionTimeoutMs().toLong)
+ .getOrElse(config.groupCoordinatorConfig.streamsGroupSessionTimeoutMs().toLong)
+ val expirationTimeMs = time.milliseconds() + sessionTimeoutMs
Review Comment:
Two things about this
- I believe we said that we want to use `heartbeatIntervalMs * 2` for the timeout, to make sure a client sees the topic creation errors.
- I would pass an timeoutMs (a duration), not the expirationTimeMs (a time). We want to start the timeout from the point we receive the response, not here.
##########
core/src/main/scala/kafka/server/KafkaApis.scala:
##########
@@ -2889,9 +2889,31 @@ class KafkaApis(val requestChannel: RequestChannel,
}
} else {
autoTopicCreationManager.createStreamsInternalTopics(topicsToCreate, requestContext);
+
+ // Check for cached topic creation errors only if there's already a MISSING_INTERNAL_TOPICS status
+ val hasMissingInternalTopicsStatus = responseData.status() != null &&
+ responseData.status().stream().anyMatch(s => s.statusCode() == StreamsGroupHeartbeatResponse.Status.MISSING_INTERNAL_TOPICS.code())
+
+ if (hasMissingInternalTopicsStatus) {
+ // Calculate group-specific error cache TTL
+ val errorCacheTtlMs = Option(groupConfigManager.groupConfig(streamsGroupHeartbeatRequest.data.groupId).orElse(null))
+ .map(_.streamsSessionTimeoutMs().toLong)
+ .getOrElse(config.groupCoordinatorConfig.streamsGroupSessionTimeoutMs().toLong)
+
+ val cachedErrors = autoTopicCreationManager.getTopicCreationErrors(topicsToCreate.keys.toSet, errorCacheTtlMs)
+ if (cachedErrors.nonEmpty) {
+ val missingInternalTopicStatus =
+ responseData.status().stream().filter(x => x.statusCode() == StreamsGroupHeartbeatResponse.Status.MISSING_INTERNAL_TOPICS.code()).findFirst()
+ val creationErrorDetails = cachedErrors.map { case (topic, error) => s"$topic ($error)" }.mkString(", ")
+ if (missingInternalTopicStatus.isPresent) {
+ missingInternalTopicStatus.get().setStatusDetail(
+ missingInternalTopicStatus.get().statusDetail() + s"; Creation failed: $creationErrorDetails."
+ )
Review Comment:
Please check if this can happen or handle this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "RaidenE1 (via GitHub)" <gi...@apache.org>.
RaidenE1 commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2352725485
##########
core/src/test/scala/unit/kafka/server/KafkaApisTest.scala:
##########
@@ -10949,6 +10950,61 @@ class KafkaApisTest extends Logging {
)
}
+ @Test
+ def testStreamsGroupHeartbeatRequestWithCachedTopicCreationErrors(): Unit = {
Review Comment:
@lucasbru Hey Lucas, does this test count as an end-to-end test or we need more?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2379548348
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -264,4 +349,101 @@ class DefaultAutoTopicCreationManager(
(creatableTopics, uncreatableTopics)
}
+
+ private def sendCreateTopicRequestWithErrorCaching(
+ creatableTopics: Map[String, CreatableTopic],
+ requestContext: Option[RequestContext],
Review Comment:
Excuse me, it seems there is no case where `none` is passed to `requestContext`. Is it intended to support more use case in the future? If not, removing the `Option` could simplify the code introduced in #20569
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -264,4 +349,101 @@ class DefaultAutoTopicCreationManager(
(creatableTopics, uncreatableTopics)
}
+
+ private def sendCreateTopicRequestWithErrorCaching(
+ creatableTopics: Map[String, CreatableTopic],
+ requestContext: Option[RequestContext],
Review Comment:
Excuse me, it seems there is no case where `none` is passed to `requestContext`. Is it intended to support more use case in the future? If not, removing the `Option` could simplify the code introduced in #20569
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2379550411
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -264,4 +349,101 @@ class DefaultAutoTopicCreationManager(
(creatableTopics, uncreatableTopics)
}
+
+ private def sendCreateTopicRequestWithErrorCaching(
+ creatableTopics: Map[String, CreatableTopic],
+ requestContext: Option[RequestContext],
Review Comment:
Excuse me, it seems there is no case where `none` is passed to `requestContext`. Is it intended to support more use case in the future? If not, removing the `Option` could simplify the code introduced in #20569
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "RaidenE1 (via GitHub)" <gi...@apache.org>.
RaidenE1 commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2270897569
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -112,6 +128,45 @@ class DefaultAutoTopicCreationManager(
}
}
+ override def getTopicCreationErrors(
+ topicNames: Set[String]
+ ): Map[String, String] = {
+ val currentTime = System.currentTimeMillis()
+ val errors = mutable.Map.empty[String, String]
+ val expiredKeys = mutable.Set.empty[String]
+
+ // Check requested topics and collect expired keys
+ topicNames.foreach { topicName =>
+ Option(topicCreationErrorCache.get(topicName)) match {
+ case Some(cachedError) if (currentTime - cachedError.timestamp) <= errorCacheTtlMs =>
Review Comment:
@mjsax This is functionally equivalent to a cleanup process!
The TTL serves two purposes:
1. Errors from too long ago become meaningless since topics can be recreated/issues
resolved
2. Lazy cleanup during queries is simpler than maintaining a separate background
cleanup thread
Both approaches achieve the same result - just trading off code complexity vs cleanup
timing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "RaidenE1 (via GitHub)" <gi...@apache.org>.
RaidenE1 commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2271200963
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -135,10 +190,17 @@ class DefaultAutoTopicCreationManager(
clearInflightRequests(creatableTopics)
if (response.authenticationException() != null) {
warn(s"Auto topic creation failed for ${creatableTopics.keys} with authentication exception")
+ cacheTopicCreationErrors(creatableTopics.keys.toSet, "Authentication failed")
} else if (response.versionMismatch() != null) {
warn(s"Auto topic creation failed for ${creatableTopics.keys} with invalid version exception")
+ cacheTopicCreationErrors(creatableTopics.keys.toSet, "Version mismatch")
} else {
- debug(s"Auto topic creation completed for ${creatableTopics.keys} with response ${response.responseBody}.")
+ response.responseBody() match {
+ case createTopicsResponse: CreateTopicsResponse =>
+ cacheTopicCreationErrorsFromResponse(createTopicsResponse)
Review Comment:
In that method I check the error code, only cache if the code is not 0. I think we can definitely moved it out
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "aliehsaeedii (via GitHub)" <gi...@apache.org>.
aliehsaeedii commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2273440201
##########
core/src/main/scala/kafka/server/KafkaApis.scala:
##########
@@ -2889,6 +2889,24 @@ class KafkaApis(val requestChannel: RequestChannel,
}
} else {
autoTopicCreationManager.createStreamsInternalTopics(topicsToCreate, requestContext);
+
+ // Check for cached topic creation errors only if there's already a MISSING_INTERNAL_TOPICS status
+ val hasMissingInternalTopicsStatus = responseData.status() != null &&
+ responseData.status().stream().anyMatch(s => s.statusCode() == StreamsGroupHeartbeatResponse.Status.MISSING_INTERNAL_TOPICS.code())
Review Comment:
What do we have in the cache in case of version mismatch or auth error? Is the `MISSING_INTERNAL_TOPICS` present there? Have you tested all these cases?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "aliehsaeedii (via GitHub)" <gi...@apache.org>.
aliehsaeedii commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2273217176
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -53,17 +55,34 @@ trait AutoTopicCreationManager {
requestContext: RequestContext
): Unit
+ def getTopicCreationErrors(
+ topicNames: Set[String]
+ ): Map[String, String]
+
+ def close(): Unit = {}
+
+}
+
+case class CachedTopicCreationError(
+ errorMessage: String,
+ time: Time
+) {
+ val timestamp: Long = time.milliseconds()
}
class DefaultAutoTopicCreationManager(
config: KafkaConfig,
channelManager: NodeToControllerChannelManager,
groupCoordinator: GroupCoordinator,
txnCoordinator: TransactionCoordinator,
- shareCoordinator: ShareCoordinator
+ shareCoordinator: ShareCoordinator,
+ time: Time
) extends AutoTopicCreationManager with Logging {
private val inflightTopics = Collections.newSetFromMap(new ConcurrentHashMap[String, java.lang.Boolean]())
+ private val topicCreationErrorCache = new ConcurrentHashMap[String, CachedTopicCreationError]()
+ // Use session timeout instead of request timeout for better semantic alignment with client lifecycle
Review Comment:
Do we need to add `instead of...`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "aliehsaeedii (via GitHub)" <gi...@apache.org>.
aliehsaeedii commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2273380363
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -134,11 +180,20 @@ class DefaultAutoTopicCreationManager(
override def onComplete(response: ClientResponse): Unit = {
clearInflightRequests(creatableTopics)
if (response.authenticationException() != null) {
- warn(s"Auto topic creation failed for ${creatableTopics.keys} with authentication exception")
+ val authException = response.authenticationException()
+ warn(s"Auto topic creation failed for ${creatableTopics.keys} with authentication exception: ${authException.getMessage}")
+ cacheTopicCreationErrors(creatableTopics.keys.toSet, authException.getMessage)
} else if (response.versionMismatch() != null) {
- warn(s"Auto topic creation failed for ${creatableTopics.keys} with invalid version exception")
+ val versionException = response.versionMismatch()
+ warn(s"Auto topic creation failed for ${creatableTopics.keys} with version mismatch exception: ${versionException.getMessage}")
+ cacheTopicCreationErrors(creatableTopics.keys.toSet, versionException.getMessage)
} else {
- debug(s"Auto topic creation completed for ${creatableTopics.keys} with response ${response.responseBody}.")
+ response.responseBody() match {
+ case createTopicsResponse: CreateTopicsResponse =>
+ cacheTopicCreationErrorsFromResponse(createTopicsResponse)
+ case _ =>
Review Comment:
If it is not `CreateTopicsResponse`, why the log message is about creating topic?!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "aliehsaeedii (via GitHub)" <gi...@apache.org>.
aliehsaeedii commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2273419281
##########
core/src/test/scala/unit/kafka/server/AutoTopicCreationManagerTest.scala:
##########
@@ -393,4 +403,119 @@ class AutoTopicCreationManagerTest {
.setNumPartitions(numPartitions)
.setReplicationFactor(replicationFactor)
}
+
+ @Test
+ def testTopicCreationErrorCaching(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator,
+ mockTime)
+
+ val topics = Map(
+ "test-topic-1" -> new CreatableTopic().setName("test-topic-1").setNumPartitions(1).setReplicationFactor(1)
+ )
+ val requestContext = initializeRequestContextWithUserPrincipal()
+
+ autoTopicCreationManager.createStreamsInternalTopics(topics, requestContext)
+
+ val argumentCaptor = ArgumentCaptor.forClass(classOf[ControllerRequestCompletionHandler])
+ Mockito.verify(brokerToController).sendRequest(
+ any(classOf[AbstractRequest.Builder[_ <: AbstractRequest]]),
+ argumentCaptor.capture())
+
+ // Simulate a CreateTopicsResponse with errors
+ val createTopicsResponseData = new org.apache.kafka.common.message.CreateTopicsResponseData()
+ val topicResult = new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("test-topic-1")
+ .setErrorCode(Errors.TOPIC_ALREADY_EXISTS.code())
+ .setErrorMessage("Topic 'test-topic-1' already exists.")
+ createTopicsResponseData.topics().add(topicResult)
+
+ val createTopicsResponse = new CreateTopicsResponse(createTopicsResponseData)
+ val header = new RequestHeader(ApiKeys.CREATE_TOPICS, 0, "client", 1)
+ val clientResponse = new ClientResponse(header, null, null,
+ 0, 0, false, null, null, createTopicsResponse)
+
+ // Trigger the completion handler
+ argumentCaptor.getValue.asInstanceOf[ControllerRequestCompletionHandler].onComplete(clientResponse)
+
+ // Verify that the error was cached
+ val cachedErrors = autoTopicCreationManager.getTopicCreationErrors(Set("test-topic-1"))
+ assertEquals(1, cachedErrors.size)
+ assertTrue(cachedErrors.contains("test-topic-1"))
+ assertEquals("Topic 'test-topic-1' already exists.", cachedErrors("test-topic-1"))
+ }
+
+ @Test
+ def testGetTopicCreationErrorsWithMultipleTopics(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator,
+ mockTime)
+
+ val topics = Map(
+ "success-topic" -> new CreatableTopic().setName("success-topic").setNumPartitions(1).setReplicationFactor(1),
+ "failed-topic" -> new CreatableTopic().setName("failed-topic").setNumPartitions(1).setReplicationFactor(1)
+ )
+ val requestContext = initializeRequestContextWithUserPrincipal()
+ autoTopicCreationManager.createStreamsInternalTopics(topics, requestContext)
+
+ val argumentCaptor = ArgumentCaptor.forClass(classOf[ControllerRequestCompletionHandler])
+ Mockito.verify(brokerToController).sendRequest(
+ any(classOf[AbstractRequest.Builder[_ <: AbstractRequest]]),
+ argumentCaptor.capture())
+
+ // Simulate mixed response - one success, one failure
+ val createTopicsResponseData = new org.apache.kafka.common.message.CreateTopicsResponseData()
+ createTopicsResponseData.topics().add(
+ new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("success-topic")
+ .setErrorCode(Errors.NONE.code())
+ )
+ createTopicsResponseData.topics().add(
+ new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("failed-topic")
+ .setErrorCode(Errors.POLICY_VIOLATION.code())
+ .setErrorMessage("Policy violation")
+ )
+
+ val createTopicsResponse = new CreateTopicsResponse(createTopicsResponseData)
+ val header = new RequestHeader(ApiKeys.CREATE_TOPICS, 0, "client", 1)
+ val clientResponse = new ClientResponse(header, null, null,
+ 0, 0, false, null, null, createTopicsResponse)
+
+ argumentCaptor.getValue.asInstanceOf[ControllerRequestCompletionHandler].onComplete(clientResponse)
+
+ // Only the failed topic should be cached
+ val cachedErrors = autoTopicCreationManager.getTopicCreationErrors(Set("success-topic", "failed-topic", "nonexistent-topic"))
+ assertEquals(1, cachedErrors.size)
+ assertTrue(cachedErrors.contains("failed-topic"))
+ assertEquals("Policy violation", cachedErrors("failed-topic"))
+ }
+
+ @Test
+ def testErrorCacheTTL(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator,
+ mockTime)
+
+ // 测试getTopicCreationErrors在没有缓存时返回空
+ val initialResult = autoTopicCreationManager.getTopicCreationErrors(Set("nonexistent-topic"))
+ assertTrue(initialResult.isEmpty)
+
+ // 让时间前进,再次查询确保仍然为空
Review Comment:
Wrong chars! The same as above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2275137991
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -64,6 +77,9 @@ class DefaultAutoTopicCreationManager(
) extends AutoTopicCreationManager with Logging {
private val inflightTopics = Collections.newSetFromMap(new ConcurrentHashMap[String, java.lang.Boolean]())
+ private val topicCreationErrorCache = new ConcurrentHashMap[String, CachedTopicCreationError]()
+ private val errorCacheTtlMs = config.requestTimeoutMs.toLong * 3 // 3x request timeout
Review Comment:
> I think the topic is globally used, so if it's missing then it's missing for all groups
My understanding was, that `AutoTopicCreationManager.scala` is used to create internal topics for KS apps. Or course, the manager is used for all groups, but it does create topics which are individual to a specific group.
So if there is two KS applications, and there is an issue creating a topic for application A, we need to ensure to report this error back to application A (and only to application A, but not also to application B).
And we would also buffer error messages for application A with the application A specific `session.timeout.ms` (and not use `session.timeout.ms` of application B, for application A errors)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "lucasbru (via GitHub)" <gi...@apache.org>.
lucasbru commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2348882422
##########
core/src/test/scala/unit/kafka/server/AutoTopicCreationManagerTest.scala:
##########
@@ -356,4 +371,226 @@ class AutoTopicCreationManagerTest {
.setNumPartitions(numPartitions)
.setReplicationFactor(replicationFactor)
}
+
+ @Test
+ def testTopicCreationErrorCaching(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator,
+ mockTime)
+
+ val topics = Map(
+ "test-topic-1" -> new CreatableTopic().setName("test-topic-1").setNumPartitions(1).setReplicationFactor(1)
+ )
+ val requestContext = initializeRequestContextWithUserPrincipal()
+
+ autoTopicCreationManager.createStreamsInternalTopics(topics, requestContext, config.groupCoordinatorConfig.streamsGroupHeartbeatIntervalMs() * 2)
+
+ val argumentCaptor = ArgumentCaptor.forClass(classOf[ControllerRequestCompletionHandler])
+ Mockito.verify(brokerToController).sendRequest(
+ any(classOf[AbstractRequest.Builder[_ <: AbstractRequest]]),
+ argumentCaptor.capture())
+
+ // Simulate a CreateTopicsResponse with errors
+ val createTopicsResponseData = new org.apache.kafka.common.message.CreateTopicsResponseData()
+ val topicResult = new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("test-topic-1")
+ .setErrorCode(Errors.TOPIC_ALREADY_EXISTS.code())
+ .setErrorMessage("Topic 'test-topic-1' already exists.")
+ createTopicsResponseData.topics().add(topicResult)
+
+ val createTopicsResponse = new CreateTopicsResponse(createTopicsResponseData)
+ val header = new RequestHeader(ApiKeys.CREATE_TOPICS, 0, "client", 1)
+ val clientResponse = new ClientResponse(header, null, null,
+ 0, 0, false, null, null, createTopicsResponse)
+
+ // Trigger the completion handler
+ argumentCaptor.getValue.asInstanceOf[ControllerRequestCompletionHandler].onComplete(clientResponse)
+
+ // Verify that the error was cached
+ val cachedErrors = autoTopicCreationManager.getStreamsInternalTopicCreationErrors(Set("test-topic-1"), mockTime.milliseconds())
+ assertEquals(1, cachedErrors.size)
+ assertTrue(cachedErrors.contains("test-topic-1"))
+ assertEquals("Topic 'test-topic-1' already exists.", cachedErrors("test-topic-1"))
+ }
+
+ @Test
+ def testGetTopicCreationErrorsWithMultipleTopics(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator,
+ mockTime)
+
+ val topics = Map(
+ "success-topic" -> new CreatableTopic().setName("success-topic").setNumPartitions(1).setReplicationFactor(1),
+ "failed-topic" -> new CreatableTopic().setName("failed-topic").setNumPartitions(1).setReplicationFactor(1)
+ )
+ val requestContext = initializeRequestContextWithUserPrincipal()
+ autoTopicCreationManager.createStreamsInternalTopics(topics, requestContext, config.groupCoordinatorConfig.streamsGroupHeartbeatIntervalMs() * 2)
+
+ val argumentCaptor = ArgumentCaptor.forClass(classOf[ControllerRequestCompletionHandler])
+ Mockito.verify(brokerToController).sendRequest(
+ any(classOf[AbstractRequest.Builder[_ <: AbstractRequest]]),
+ argumentCaptor.capture())
+
+ // Simulate mixed response - one success, one failure
+ val createTopicsResponseData = new org.apache.kafka.common.message.CreateTopicsResponseData()
+ createTopicsResponseData.topics().add(
+ new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("success-topic")
+ .setErrorCode(Errors.NONE.code())
+ )
+ createTopicsResponseData.topics().add(
+ new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("failed-topic")
+ .setErrorCode(Errors.POLICY_VIOLATION.code())
+ .setErrorMessage("Policy violation")
+ )
+
+ val createTopicsResponse = new CreateTopicsResponse(createTopicsResponseData)
+ val header = new RequestHeader(ApiKeys.CREATE_TOPICS, 0, "client", 1)
+ val clientResponse = new ClientResponse(header, null, null,
+ 0, 0, false, null, null, createTopicsResponse)
+
+ argumentCaptor.getValue.asInstanceOf[ControllerRequestCompletionHandler].onComplete(clientResponse)
+
+ // Only the failed topic should be cached
+ val cachedErrors = autoTopicCreationManager.getStreamsInternalTopicCreationErrors(Set("success-topic", "failed-topic", "nonexistent-topic"), mockTime.milliseconds())
+ assertEquals(1, cachedErrors.size)
+ assertTrue(cachedErrors.contains("failed-topic"))
+ assertEquals("Policy violation", cachedErrors("failed-topic"))
+ }
+
+ @Test
+ def testErrorCacheTTL(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator,
+ mockTime)
+
+
+ // First cache an error by simulating topic creation failure
+ val topics = Map(
+ "test-topic" -> new CreatableTopic().setName("test-topic").setNumPartitions(1).setReplicationFactor(1)
+ )
+ val requestContext = initializeRequestContextWithUserPrincipal()
+ val shortTtlMs = 1000L // Use 1 second TTL for faster testing
+ autoTopicCreationManager.createStreamsInternalTopics(topics, requestContext, shortTtlMs)
+
+ val argumentCaptor = ArgumentCaptor.forClass(classOf[ControllerRequestCompletionHandler])
+ Mockito.verify(brokerToController).sendRequest(
+ any(classOf[AbstractRequest.Builder[_ <: AbstractRequest]]),
+ argumentCaptor.capture())
+
+ // Simulate a CreateTopicsResponse with error
+ val createTopicsResponseData = new org.apache.kafka.common.message.CreateTopicsResponseData()
+ val topicResult = new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("test-topic")
+ .setErrorCode(Errors.INVALID_REPLICATION_FACTOR.code())
+ .setErrorMessage("Invalid replication factor")
+ createTopicsResponseData.topics().add(topicResult)
+
+ val createTopicsResponse = new CreateTopicsResponse(createTopicsResponseData)
+ val header = new RequestHeader(ApiKeys.CREATE_TOPICS, 0, "client", 1)
+ val clientResponse = new ClientResponse(header, null, null,
+ 0, 0, false, null, null, createTopicsResponse)
+
+ // Cache the error at T0
+ argumentCaptor.getValue.asInstanceOf[ControllerRequestCompletionHandler].onComplete(clientResponse)
+
+ // Verify error is cached and accessible within TTL
+ val cachedErrors = autoTopicCreationManager.getStreamsInternalTopicCreationErrors(Set("test-topic"), mockTime.milliseconds())
+ assertEquals(1, cachedErrors.size)
+ assertEquals("Invalid replication factor", cachedErrors("test-topic"))
+
+ // Advance time beyond TTL
+ mockTime.sleep(shortTtlMs + 100) // T0 + 1.1 seconds
+
+ // Verify error is now expired and proactively cleaned up
+ val expiredErrors = autoTopicCreationManager.getStreamsInternalTopicCreationErrors(Set("test-topic"), mockTime.milliseconds())
+ assertTrue(expiredErrors.isEmpty, "Expired errors should be proactively cleaned up")
+ }
+
+ @Test
+ def testErrorCacheLRUEviction(): Unit = {
+ // Create a config with a small cache size for testing
+ val props = TestUtils.createBrokerConfig(1)
+ props.setProperty(ServerConfigs.REQUEST_TIMEOUT_MS_CONFIG, requestTimeout.toString)
+ props.setProperty(ServerConfigs.MAX_INCREMENTAL_FETCH_SESSION_CACHE_SLOTS_CONFIG, "3") // Small cache size for testing
Review Comment:
But this constant is not used anywhere in the implementation??
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "lucasbru (via GitHub)" <gi...@apache.org>.
lucasbru commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2324653887
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -50,21 +52,122 @@ trait AutoTopicCreationManager {
def createStreamsInternalTopics(
topics: Map[String, CreatableTopic],
- requestContext: RequestContext
+ requestContext: RequestContext,
+ timeoutMs: Long
): Unit
+ def getStreamsInternalTopicCreationErrors(
+ topicNames: Set[String],
+ currentTimeMs: Long
+ ): Map[String, String]
+
+ def close(): Unit = {}
+
+}
+
+/**
+ * Thread-safe cache that stores topic creation errors with per-entry expiration.
+ * - Expiration: maintained by a min-heap (priority queue) on expiration time
+ * - Capacity: enforced by insertion-order removal (keeps the most recently inserted entries)
+ */
+private[server] class ExpiringErrorCache(maxSize: Int, time: Time) {
+
+ private case class Entry(topicName: String, errorMessage: String, expirationTimeMs: Long)
+
+ private val byTopic = new java.util.HashMap[String, Entry]()
+ private val expiryQueue = new java.util.PriorityQueue[Entry](11, new java.util.Comparator[Entry] {
+ override def compare(a: Entry, b: Entry): Int = java.lang.Long.compare(a.expirationTimeMs, b.expirationTimeMs)
+ })
+ private val lock = new ReentrantLock()
Review Comment:
Can we make `byTopic` a ConcurrentHashMap and use this lock as a write lock only?
That is, make the read path lock contention free? That would mean we can only expire on the put path, which should be fine. However, then we may read expired entries when getting from the map, so in `get` you need to check if the returned entry is expired before returning it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2379547587
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -50,21 +52,96 @@ trait AutoTopicCreationManager {
def createStreamsInternalTopics(
topics: Map[String, CreatableTopic],
- requestContext: RequestContext
+ requestContext: RequestContext,
+ timeoutMs: Long
): Unit
+ def getStreamsInternalTopicCreationErrors(
+ topicNames: Set[String],
+ currentTimeMs: Long
+ ): Map[String, String]
+
+ def close(): Unit = {}
+
}
+/**
+ * Thread-safe cache that stores topic creation errors with per-entry expiration.
+ * - Expiration: maintained by a min-heap (priority queue) on expiration time
+ * - Capacity: enforced by evicting entries with earliest expiration time (not LRU)
+ * - Updates: old entries remain in queue but are ignored via reference equality check
+ */
+private[server] class ExpiringErrorCache(maxSize: Int, time: Time) {
+
+ private case class Entry(topicName: String, errorMessage: String, expirationTimeMs: Long)
+
+ private val byTopic = new ConcurrentHashMap[String, Entry]()
+ private val expiryQueue = new java.util.PriorityQueue[Entry](11, new java.util.Comparator[Entry] {
Review Comment:
`Comparator` could be implemented by lambda.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "aliehsaeedii (via GitHub)" <gi...@apache.org>.
aliehsaeedii commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2273282296
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -135,10 +190,17 @@ class DefaultAutoTopicCreationManager(
clearInflightRequests(creatableTopics)
if (response.authenticationException() != null) {
warn(s"Auto topic creation failed for ${creatableTopics.keys} with authentication exception")
+ cacheTopicCreationErrors(creatableTopics.keys.toSet, "Authentication failed")
} else if (response.versionMismatch() != null) {
warn(s"Auto topic creation failed for ${creatableTopics.keys} with invalid version exception")
+ cacheTopicCreationErrors(creatableTopics.keys.toSet, "Version mismatch")
} else {
- debug(s"Auto topic creation completed for ${creatableTopics.keys} with response ${response.responseBody}.")
+ response.responseBody() match {
+ case createTopicsResponse: CreateTopicsResponse =>
+ cacheTopicCreationErrorsFromResponse(createTopicsResponse)
Review Comment:
I also think no need to add the success case to the error cache. Not optimized + misleading code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "aliehsaeedii (via GitHub)" <gi...@apache.org>.
aliehsaeedii commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2273417888
##########
core/src/test/scala/unit/kafka/server/AutoTopicCreationManagerTest.scala:
##########
@@ -393,4 +403,119 @@ class AutoTopicCreationManagerTest {
.setNumPartitions(numPartitions)
.setReplicationFactor(replicationFactor)
}
+
+ @Test
+ def testTopicCreationErrorCaching(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator,
+ mockTime)
+
+ val topics = Map(
+ "test-topic-1" -> new CreatableTopic().setName("test-topic-1").setNumPartitions(1).setReplicationFactor(1)
+ )
+ val requestContext = initializeRequestContextWithUserPrincipal()
+
+ autoTopicCreationManager.createStreamsInternalTopics(topics, requestContext)
+
+ val argumentCaptor = ArgumentCaptor.forClass(classOf[ControllerRequestCompletionHandler])
+ Mockito.verify(brokerToController).sendRequest(
+ any(classOf[AbstractRequest.Builder[_ <: AbstractRequest]]),
+ argumentCaptor.capture())
+
+ // Simulate a CreateTopicsResponse with errors
+ val createTopicsResponseData = new org.apache.kafka.common.message.CreateTopicsResponseData()
+ val topicResult = new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("test-topic-1")
+ .setErrorCode(Errors.TOPIC_ALREADY_EXISTS.code())
+ .setErrorMessage("Topic 'test-topic-1' already exists.")
+ createTopicsResponseData.topics().add(topicResult)
+
+ val createTopicsResponse = new CreateTopicsResponse(createTopicsResponseData)
+ val header = new RequestHeader(ApiKeys.CREATE_TOPICS, 0, "client", 1)
+ val clientResponse = new ClientResponse(header, null, null,
+ 0, 0, false, null, null, createTopicsResponse)
+
+ // Trigger the completion handler
+ argumentCaptor.getValue.asInstanceOf[ControllerRequestCompletionHandler].onComplete(clientResponse)
+
+ // Verify that the error was cached
+ val cachedErrors = autoTopicCreationManager.getTopicCreationErrors(Set("test-topic-1"))
+ assertEquals(1, cachedErrors.size)
+ assertTrue(cachedErrors.contains("test-topic-1"))
+ assertEquals("Topic 'test-topic-1' already exists.", cachedErrors("test-topic-1"))
+ }
+
+ @Test
+ def testGetTopicCreationErrorsWithMultipleTopics(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator,
+ mockTime)
+
+ val topics = Map(
+ "success-topic" -> new CreatableTopic().setName("success-topic").setNumPartitions(1).setReplicationFactor(1),
+ "failed-topic" -> new CreatableTopic().setName("failed-topic").setNumPartitions(1).setReplicationFactor(1)
+ )
+ val requestContext = initializeRequestContextWithUserPrincipal()
+ autoTopicCreationManager.createStreamsInternalTopics(topics, requestContext)
+
+ val argumentCaptor = ArgumentCaptor.forClass(classOf[ControllerRequestCompletionHandler])
+ Mockito.verify(brokerToController).sendRequest(
+ any(classOf[AbstractRequest.Builder[_ <: AbstractRequest]]),
+ argumentCaptor.capture())
+
+ // Simulate mixed response - one success, one failure
+ val createTopicsResponseData = new org.apache.kafka.common.message.CreateTopicsResponseData()
+ createTopicsResponseData.topics().add(
+ new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("success-topic")
+ .setErrorCode(Errors.NONE.code())
+ )
+ createTopicsResponseData.topics().add(
+ new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("failed-topic")
+ .setErrorCode(Errors.POLICY_VIOLATION.code())
+ .setErrorMessage("Policy violation")
+ )
+
+ val createTopicsResponse = new CreateTopicsResponse(createTopicsResponseData)
+ val header = new RequestHeader(ApiKeys.CREATE_TOPICS, 0, "client", 1)
+ val clientResponse = new ClientResponse(header, null, null,
+ 0, 0, false, null, null, createTopicsResponse)
+
+ argumentCaptor.getValue.asInstanceOf[ControllerRequestCompletionHandler].onComplete(clientResponse)
+
+ // Only the failed topic should be cached
+ val cachedErrors = autoTopicCreationManager.getTopicCreationErrors(Set("success-topic", "failed-topic", "nonexistent-topic"))
+ assertEquals(1, cachedErrors.size)
+ assertTrue(cachedErrors.contains("failed-topic"))
+ assertEquals("Policy violation", cachedErrors("failed-topic"))
+ }
+
+ @Test
+ def testErrorCacheTTL(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator,
+ mockTime)
+
+ // 测试getTopicCreationErrors在没有缓存时返回空
Review Comment:
?!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2275149725
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -112,6 +128,45 @@ class DefaultAutoTopicCreationManager(
}
}
+ override def getTopicCreationErrors(
+ topicNames: Set[String]
+ ): Map[String, String] = {
+ val currentTime = System.currentTimeMillis()
+ val errors = mutable.Map.empty[String, String]
+ val expiredKeys = mutable.Set.empty[String]
+
+ // Check requested topics and collect expired keys
+ topicNames.foreach { topicName =>
+ Option(topicCreationErrorCache.get(topicName)) match {
+ case Some(cachedError) if (currentTime - cachedError.timestamp) <= errorCacheTtlMs =>
+ errors.put(topicName, cachedError.errorMessage)
+ case Some(_) =>
Review Comment:
Oh I see -- `Some(cachedError) if` is one statement in Scala. Interesting. Thanks for clarifying.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "aliehsaeedii (via GitHub)" <gi...@apache.org>.
aliehsaeedii commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2273302083
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -135,10 +190,17 @@ class DefaultAutoTopicCreationManager(
clearInflightRequests(creatableTopics)
if (response.authenticationException() != null) {
warn(s"Auto topic creation failed for ${creatableTopics.keys} with authentication exception")
+ cacheTopicCreationErrors(creatableTopics.keys.toSet, "Authentication failed")
} else if (response.versionMismatch() != null) {
warn(s"Auto topic creation failed for ${creatableTopics.keys} with invalid version exception")
+ cacheTopicCreationErrors(creatableTopics.keys.toSet, "Version mismatch")
} else {
- debug(s"Auto topic creation completed for ${creatableTopics.keys} with response ${response.responseBody}.")
+ response.responseBody() match {
+ case createTopicsResponse: CreateTopicsResponse =>
+ cacheTopicCreationErrorsFromResponse(createTopicsResponse)
Review Comment:
Oh, I see you check the error before putting it in cache. Mmm, hard to say. As the above if/else blocks may not cover the entire error cases, let's keep it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2275144231
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -64,6 +77,9 @@ class DefaultAutoTopicCreationManager(
) extends AutoTopicCreationManager with Logging {
private val inflightTopics = Collections.newSetFromMap(new ConcurrentHashMap[String, java.lang.Boolean]())
+ private val topicCreationErrorCache = new ConcurrentHashMap[String, CachedTopicCreationError]()
+ private val errorCacheTtlMs = config.requestTimeoutMs.toLong * 3 // 3x request timeout
+ private val maxCacheSize = 1000
Review Comment:
Not sure if we would need a KIP (we would need one, if we would make it configurable). But I would expect error rate to be low. If there is some issue at some point, we can still limit it in the future, and with a TTL we would expires old entries anyway.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "aliehsaeedii (via GitHub)" <gi...@apache.org>.
aliehsaeedii commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2273386686
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -263,4 +318,30 @@ class DefaultAutoTopicCreationManager(
(creatableTopics, uncreatableTopics)
}
+
+ private def cacheTopicCreationErrors(topicNames: Set[String], errorMessage: String): Unit = {
+ topicNames.foreach { topicName =>
+ topicCreationErrorCache.put(topicName, CachedTopicCreationError(errorMessage, time))
+ }
+ }
+
+ private def cacheTopicCreationErrorsFromResponse(response: CreateTopicsResponse): Unit = {
+ response.data().topics().forEach { topicResult =>
+ if (topicResult.errorCode() != Errors.NONE.code()) {
+ val errorMessage = Option(topicResult.errorMessage())
+ .filter(_.nonEmpty)
+ .getOrElse(Errors.forCode(topicResult.errorCode()).message())
+ topicCreationErrorCache.put(
+ topicResult.name(),
+ CachedTopicCreationError(errorMessage, time)
+ )
+ debug(s"Cached topic creation error for ${topicResult.name()}: $errorMessage")
+ }
+
Review Comment:
Any log message here? `topic creates successfully...`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "RaidenE1 (via GitHub)" <gi...@apache.org>.
RaidenE1 commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2347933926
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -50,21 +52,111 @@ trait AutoTopicCreationManager {
def createStreamsInternalTopics(
topics: Map[String, CreatableTopic],
- requestContext: RequestContext
+ requestContext: RequestContext,
+ timeoutMs: Long
): Unit
+ def getStreamsInternalTopicCreationErrors(
+ topicNames: Set[String],
+ currentTimeMs: Long
+ ): Map[String, String]
+
+ def close(): Unit = {}
+
+}
+
+/**
+ * Thread-safe cache that stores topic creation errors with per-entry expiration.
+ * - Expiration: maintained by a min-heap (priority queue) on expiration time
+ * - Capacity: enforced by insertion-order removal (keeps the most recently inserted entries)
+ */
+private[server] class ExpiringErrorCache(maxSize: Int, time: Time) {
+
+ private case class Entry(topicName: String, errorMessage: String, expirationTimeMs: Long)
+
+ private val byTopic = new ConcurrentHashMap[String, Entry]()
+ private val expiryQueue = new java.util.PriorityQueue[Entry](11, new java.util.Comparator[Entry] {
+ override def compare(a: Entry, b: Entry): Int = java.lang.Long.compare(a.expirationTimeMs, b.expirationTimeMs)
+ })
+ private val lock = new ReentrantLock()
+
+ def put(topicName: String, errorMessage: String, ttlMs: Long): Unit = {
+ lock.lock()
+ try {
+ val existing = byTopic.get(topicName)
+ if (existing != null) {
+ // Remove old instance from structures
+ expiryQueue.remove(existing)
+ }
+
+ val currentTimeMs = time.milliseconds()
+ val expirationTimeMs = currentTimeMs + ttlMs
+ val entry = Entry(topicName, errorMessage, expirationTimeMs)
+ byTopic.put(topicName, entry)
+ expiryQueue.add(entry)
+
+ // Clean up expired entries
+ while (!expiryQueue.isEmpty && expiryQueue.peek().expirationTimeMs <= currentTimeMs) {
+ val expired = expiryQueue.poll()
+ val current = byTopic.get(expired.topicName)
+ if (current != null && (current eq expired)) {
Review Comment:
I think not, it just checks if same object instance, which is O(1)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "RaidenE1 (via GitHub)" <gi...@apache.org>.
RaidenE1 commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2270821034
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -64,6 +77,9 @@ class DefaultAutoTopicCreationManager(
) extends AutoTopicCreationManager with Logging {
private val inflightTopics = Collections.newSetFromMap(new ConcurrentHashMap[String, java.lang.Boolean]())
+ private val topicCreationErrorCache = new ConcurrentHashMap[String, CachedTopicCreationError]()
+ private val errorCacheTtlMs = config.requestTimeoutMs.toLong * 3 // 3x request timeout
+ private val maxCacheSize = 1000
Review Comment:
Just want to make sure not caching too much errors and take up too much memory, but I want to remove it since it might requires a KIP?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "RaidenE1 (via GitHub)" <gi...@apache.org>.
RaidenE1 commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2270897569
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -112,6 +128,45 @@ class DefaultAutoTopicCreationManager(
}
}
+ override def getTopicCreationErrors(
+ topicNames: Set[String]
+ ): Map[String, String] = {
+ val currentTime = System.currentTimeMillis()
+ val errors = mutable.Map.empty[String, String]
+ val expiredKeys = mutable.Set.empty[String]
+
+ // Check requested topics and collect expired keys
+ topicNames.foreach { topicName =>
+ Option(topicCreationErrorCache.get(topicName)) match {
+ case Some(cachedError) if (currentTime - cachedError.timestamp) <= errorCacheTtlMs =>
Review Comment:
@mjsax This is functionally equivalent to a cleanup process!
The TTL serves two purposes:
1. Errors from too long ago become meaningless since topics can be recreated/issues
resolved
2. Lazy cleanup during queries is simpler than maintaining a separate background
cleanup thread
Both approaches achieve the same result - just trading off code complexity vs cleanup
timing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "RaidenE1 (via GitHub)" <gi...@apache.org>.
RaidenE1 commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2271540373
##########
core/src/test/scala/unit/kafka/server/KafkaApisTest.scala:
##########
@@ -10949,6 +10950,59 @@ class KafkaApisTest extends Logging {
)
}
+ @Test
+ def testStreamsGroupHeartbeatRequestWithCachedTopicCreationErrors(): Unit = {
Review Comment:
it tests that:
1. Cached topic creation errors are retrieved and appended to the response
2. The final status detail contains both the original "missing topics" message and the cached error details
3. The format is: "Internal topics are missing: [test-topic]; Creation failed: test-topic (INVALID_REPLICATION_FACTOR)."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "lucasbru (via GitHub)" <gi...@apache.org>.
lucasbru commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2303883693
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -112,6 +131,33 @@ class DefaultAutoTopicCreationManager(
}
}
+ override def getTopicCreationErrors(
+ topicNames: Set[String]
+ ): Map[String, String] = {
+ val currentTime = time.milliseconds()
+ val errors = mutable.Map.empty[String, String]
+ val expiredKeys = mutable.Set.empty[String]
+
+ // Check requested topics and collect expired keys
+ topicNames.foreach { topicName =>
+ Option(topicCreationErrorCache.get(topicName)) match {
+ case Some(cachedError) if (currentTime - cachedError.timestamp) <= errorCacheTtlMs =>
+ errors.put(topicName, cachedError.errorMessage)
+ case Some(_) =>
+ expiredKeys += topicName
+ case None =>
+ }
+ }
+
+ // Remove expired entries
+ expiredKeys.foreach { key =>
+ topicCreationErrorCache.remove(key)
+ debug(s"Removed expired topic creation error cache entry for $key")
+ }
+
+ errors.toMap
+ }
+
private def sendCreateTopicRequest(
Review Comment:
`sendCreateTopicRequest` is used by both streams and other internal topics. I would suggest making a separate copy for this method to not affect the code path for other internal topics (since the error message will not be used on those code paths.)
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -112,6 +131,33 @@ class DefaultAutoTopicCreationManager(
}
}
+ override def getTopicCreationErrors(
+ topicNames: Set[String]
+ ): Map[String, String] = {
+ val currentTime = time.milliseconds()
+ val errors = mutable.Map.empty[String, String]
+ val expiredKeys = mutable.Set.empty[String]
+
+ // Check requested topics and collect expired keys
+ topicNames.foreach { topicName =>
+ Option(topicCreationErrorCache.get(topicName)) match {
+ case Some(cachedError) if (currentTime - cachedError.timestamp) <= errorCacheTtlMs =>
+ errors.put(topicName, cachedError.errorMessage)
+ case Some(_) =>
+ expiredKeys += topicName
+ case None =>
+ }
+ }
+
+ // Remove expired entries
+ expiredKeys.foreach { key =>
+ topicCreationErrorCache.remove(key)
+ debug(s"Removed expired topic creation error cache entry for $key")
+ }
+
Review Comment:
I think we still need to make sure that eventually, errors are evicted even if we do not receive another topic creation request. I wonder if we shouldn't build an expiring cache for that implements `org.apache.kafka.common.cache.Cache`. It could use a LinkedHashMap similar LRUCache and every time we insert or get from the cache, we try to expire the last elements of the LinkedHashMap. WDYT?
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -53,17 +55,34 @@ trait AutoTopicCreationManager {
requestContext: RequestContext
): Unit
+ def getTopicCreationErrors(
+ topicNames: Set[String]
+ ): Map[String, String]
+
+ def close(): Unit = {}
+
+}
+
+case class CachedTopicCreationError(
+ errorMessage: String,
+ time: Time
+) {
+ val timestamp: Long = time.milliseconds()
}
class DefaultAutoTopicCreationManager(
config: KafkaConfig,
channelManager: NodeToControllerChannelManager,
groupCoordinator: GroupCoordinator,
txnCoordinator: TransactionCoordinator,
- shareCoordinator: ShareCoordinator
+ shareCoordinator: ShareCoordinator,
+ time: Time
) extends AutoTopicCreationManager with Logging {
private val inflightTopics = Collections.newSetFromMap(new ConcurrentHashMap[String, java.lang.Boolean]())
+ private val topicCreationErrorCache = new ConcurrentHashMap[String, CachedTopicCreationError]()
+ // Use session timeout for better semantic alignment with client lifecycle
+ private val errorCacheTtlMs = config.groupCoordinatorConfig.classicGroupMaxSessionTimeoutMs.toLong
Review Comment:
We'd want to pass this in from the outside. Session timeouts can be overwritten for a specific group, so we need to use `groupConfigManager` which is present in `KafkaApis`.
```
Optional<GroupConfig> groupConfig = groupConfigManager.groupConfig(groupId);
return groupConfig.map(GroupConfig::streamsSessionTimeoutMs)
.orElse(config.groupCoordinatorConfig.streamsGroupSessionTimeoutMs());
```
I would also re-fetch this value every time we create an error message, since the session timeout may change over time. So better pass this into `createStreamsInternalTopics`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "lucasbru (via GitHub)" <gi...@apache.org>.
lucasbru merged PR #20325:
URL: https://github.com/apache/kafka/pull/20325
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "lucasbru (via GitHub)" <gi...@apache.org>.
lucasbru commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2315376555
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -53,17 +55,47 @@ trait AutoTopicCreationManager {
requestContext: RequestContext
): Unit
+ def getTopicCreationErrors(
+ topicNames: Set[String],
+ errorCacheTtlMs: Long
+ ): Map[String, String]
+
+ def close(): Unit = {}
+
}
+case class CachedTopicCreationError(
+ errorMessage: String,
+ time: Time
+) {
+ val timestamp: Long = time.milliseconds()
+}
+
+
class DefaultAutoTopicCreationManager(
config: KafkaConfig,
channelManager: NodeToControllerChannelManager,
groupCoordinator: GroupCoordinator,
txnCoordinator: TransactionCoordinator,
- shareCoordinator: ShareCoordinator
+ shareCoordinator: ShareCoordinator,
+ time: Time
) extends AutoTopicCreationManager with Logging {
private val inflightTopics = Collections.newSetFromMap(new ConcurrentHashMap[String, java.lang.Boolean]())
+
+ // Use MAX_INCREMENTAL_FETCH_SESSION_CACHE_SLOTS_CONFIG as the size limit for the error cache
+ // This provides a reasonable bound (default 1000) to prevent unbounded growth
+ private val maxCacheSize = config.maxIncrementalFetchSessionCacheSlots
Review Comment:
Using that config seems a bit random. I would just hard-code it.
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -122,23 +203,12 @@ class DefaultAutoTopicCreationManager(
override def onComplete(response: ClientResponse): Unit = {
clearInflightRequests(creatableTopics)
if (response.authenticationException() != null) {
- warn(s"Auto topic creation failed for ${creatableTopics.keys} with authentication exception")
+ val authException = response.authenticationException()
+ warn(s"Auto topic creation failed for ${creatableTopics.keys} with authentication exception: ${authException.getMessage}")
} else if (response.versionMismatch() != null) {
- warn(s"Auto topic creation failed for ${creatableTopics.keys} with invalid version exception")
+ val versionException = response.versionMismatch()
+ warn(s"Auto topic creation failed for ${creatableTopics.keys} with version mismatch exception: ${versionException.getMessage}")
} else {
- if (response.hasResponse) {
Review Comment:
why did you remove this code? I would revert all changes in `sendCreateTopicRequest`
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -53,17 +55,47 @@ trait AutoTopicCreationManager {
requestContext: RequestContext
): Unit
+ def getTopicCreationErrors(
Review Comment:
`getStreamsInternalTopicCreationErrors`?
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -53,17 +55,47 @@ trait AutoTopicCreationManager {
requestContext: RequestContext
): Unit
+ def getTopicCreationErrors(
+ topicNames: Set[String],
+ errorCacheTtlMs: Long
+ ): Map[String, String]
+
+ def close(): Unit = {}
+
}
+case class CachedTopicCreationError(
+ errorMessage: String,
+ time: Time
+) {
+ val timestamp: Long = time.milliseconds()
+}
+
+
class DefaultAutoTopicCreationManager(
config: KafkaConfig,
channelManager: NodeToControllerChannelManager,
groupCoordinator: GroupCoordinator,
txnCoordinator: TransactionCoordinator,
- shareCoordinator: ShareCoordinator
+ shareCoordinator: ShareCoordinator,
+ time: Time
) extends AutoTopicCreationManager with Logging {
private val inflightTopics = Collections.newSetFromMap(new ConcurrentHashMap[String, java.lang.Boolean]())
+
+ // Use MAX_INCREMENTAL_FETCH_SESSION_CACHE_SLOTS_CONFIG as the size limit for the error cache
+ // This provides a reasonable bound (default 1000) to prevent unbounded growth
+ private val maxCacheSize = config.maxIncrementalFetchSessionCacheSlots
+ info(s"AutoTopicCreationManager initialized with error cache size limit: $maxCacheSize")
Review Comment:
Remove the extra logging
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -53,17 +55,47 @@ trait AutoTopicCreationManager {
requestContext: RequestContext
): Unit
+ def getTopicCreationErrors(
+ topicNames: Set[String],
+ errorCacheTtlMs: Long
+ ): Map[String, String]
+
+ def close(): Unit = {}
+
}
+case class CachedTopicCreationError(
+ errorMessage: String,
+ time: Time
+) {
+ val timestamp: Long = time.milliseconds()
+}
+
+
class DefaultAutoTopicCreationManager(
config: KafkaConfig,
channelManager: NodeToControllerChannelManager,
groupCoordinator: GroupCoordinator,
txnCoordinator: TransactionCoordinator,
- shareCoordinator: ShareCoordinator
+ shareCoordinator: ShareCoordinator,
+ time: Time
) extends AutoTopicCreationManager with Logging {
private val inflightTopics = Collections.newSetFromMap(new ConcurrentHashMap[String, java.lang.Boolean]())
+
+ // Use MAX_INCREMENTAL_FETCH_SESSION_CACHE_SLOTS_CONFIG as the size limit for the error cache
+ // This provides a reasonable bound (default 1000) to prevent unbounded growth
+ private val maxCacheSize = config.maxIncrementalFetchSessionCacheSlots
+ info(s"AutoTopicCreationManager initialized with error cache size limit: $maxCacheSize")
+
+ // LRU cache with size limit to prevent unbounded memory growth
+ private val topicCreationErrorCache = Collections.synchronizedMap(
+ new java.util.LinkedHashMap[String, CachedTopicCreationError](16, 0.75f, false) {
+ override def removeEldestEntry(eldest: java.util.Map.Entry[String, CachedTopicCreationError]): Boolean = {
+ size() > maxCacheSize
Review Comment:
I kind of expected you to implement a new subclass of `Cache` for this. I think we can also do it this way.
Could we also remove the eldest entry if it is expired?
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -96,7 +128,56 @@ class DefaultAutoTopicCreationManager(
requestContext: RequestContext
): Unit = {
if (topics.nonEmpty) {
- sendCreateTopicRequest(topics, Some(requestContext))
+ sendCreateTopicRequestWithErrorCaching(topics, Some(requestContext))
+ }
+ }
+
+ override def getTopicCreationErrors(
+ topicNames: Set[String],
+ errorCacheTtlMs: Long
+ ): Map[String, String] = {
+ // Proactively expire old entries using the provided TTL
+ expireOldEntries(errorCacheTtlMs)
+
+ val errors = mutable.Map.empty[String, String]
+
+ // Check requested topics
+ topicNames.foreach { topicName =>
+ Option(topicCreationErrorCache.get(topicName)) match {
+ case Some(error) =>
+ errors.put(topicName, error.errorMessage)
+ case None =>
+ }
+ }
+
+ errors.toMap
+ }
+
+ /**
+ * Remove expired entries from the cache using the provided TTL.
+ * Since we use LinkedHashMap with insertion order, we only need to check
+ * entries from the beginning until we find a non-expired entry.
+ */
+ private def expireOldEntries(ttlMs: Long): Unit = {
Review Comment:
I think the expiration won't work if we have different TTLs for different groups, right?
Since the entries will not expire in insertion order.
I think we may need a priority queue and hashmap to solve this correctly. We should probably put this into a little helper class that is synchronized.
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -53,17 +55,47 @@ trait AutoTopicCreationManager {
requestContext: RequestContext
): Unit
+ def getTopicCreationErrors(
+ topicNames: Set[String],
+ errorCacheTtlMs: Long
+ ): Map[String, String]
+
+ def close(): Unit = {}
+
}
+case class CachedTopicCreationError(
Review Comment:
Is this public or private? Can we make this a purely internal thing inside the topic creation manager, since it does not appear in the interface?
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -96,7 +128,56 @@ class DefaultAutoTopicCreationManager(
requestContext: RequestContext
): Unit = {
if (topics.nonEmpty) {
- sendCreateTopicRequest(topics, Some(requestContext))
+ sendCreateTopicRequestWithErrorCaching(topics, Some(requestContext))
+ }
+ }
+
+ override def getTopicCreationErrors(
+ topicNames: Set[String],
+ errorCacheTtlMs: Long
Review Comment:
Passing the TTL here is incorrect. The TTL is defined group-specific, so you are expiring topic creation errors for one group with the TTL for a different group.
I think we need to pass the errorCacheTTL into `createStreamsInternalTopics`, and store the expiry time instead of the error receival timestamp in `CachedTopicCreationError`.
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -264,4 +334,103 @@ class DefaultAutoTopicCreationManager(
(creatableTopics, uncreatableTopics)
}
+
+ private def sendCreateTopicRequestWithErrorCaching(
+ creatableTopics: Map[String, CreatableTopic],
+ requestContext: Option[RequestContext]
+ ): Seq[MetadataResponseTopic] = {
+ val topicsToCreate = new CreateTopicsRequestData.CreatableTopicCollection(creatableTopics.size)
+ topicsToCreate.addAll(creatableTopics.values.asJavaCollection)
+
+ val createTopicsRequest = new CreateTopicsRequest.Builder(
+ new CreateTopicsRequestData()
+ .setTimeoutMs(config.requestTimeoutMs)
+ .setTopics(topicsToCreate)
+ )
+
+ val requestCompletionHandler = new ControllerRequestCompletionHandler {
+ override def onTimeout(): Unit = {
+ clearInflightRequests(creatableTopics)
+ debug(s"Auto topic creation timed out for ${creatableTopics.keys}.")
Review Comment:
Shouldn't we add a `cacheTopicCreationErrors` in this case as well?
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -264,4 +334,103 @@ class DefaultAutoTopicCreationManager(
(creatableTopics, uncreatableTopics)
}
+
+ private def sendCreateTopicRequestWithErrorCaching(
+ creatableTopics: Map[String, CreatableTopic],
+ requestContext: Option[RequestContext]
+ ): Seq[MetadataResponseTopic] = {
+ val topicsToCreate = new CreateTopicsRequestData.CreatableTopicCollection(creatableTopics.size)
+ topicsToCreate.addAll(creatableTopics.values.asJavaCollection)
+
+ val createTopicsRequest = new CreateTopicsRequest.Builder(
+ new CreateTopicsRequestData()
+ .setTimeoutMs(config.requestTimeoutMs)
+ .setTopics(topicsToCreate)
+ )
+
+ val requestCompletionHandler = new ControllerRequestCompletionHandler {
+ override def onTimeout(): Unit = {
+ clearInflightRequests(creatableTopics)
+ debug(s"Auto topic creation timed out for ${creatableTopics.keys}.")
+ }
+
+ override def onComplete(response: ClientResponse): Unit = {
+ clearInflightRequests(creatableTopics)
+ if (response.authenticationException() != null) {
+ val authException = response.authenticationException()
+ warn(s"Auto topic creation failed for ${creatableTopics.keys} with authentication exception: ${authException.getMessage}")
+ cacheTopicCreationErrors(creatableTopics.keys.toSet, authException.getMessage)
+ } else if (response.versionMismatch() != null) {
+ val versionException = response.versionMismatch()
+ warn(s"Auto topic creation failed for ${creatableTopics.keys} with version mismatch exception: ${versionException.getMessage}")
+ cacheTopicCreationErrors(creatableTopics.keys.toSet, versionException.getMessage)
+ } else {
+ response.responseBody() match {
+ case createTopicsResponse: CreateTopicsResponse =>
+ cacheTopicCreationErrorsFromResponse(createTopicsResponse)
+ case _ =>
+ debug(s"Auto topic creation completed for ${creatableTopics.keys} with response ${response.responseBody}.")
+ }
+ }
+ }
+ }
+
+ val request = requestContext.map { context =>
+ val requestVersion =
+ channelManager.controllerApiVersions.toScala match {
+ case None =>
+ // We will rely on the Metadata request to be retried in the case
+ // that the latest version is not usable by the controller.
+ ApiKeys.CREATE_TOPICS.latestVersion()
+ case Some(nodeApiVersions) =>
+ nodeApiVersions.latestUsableVersion(ApiKeys.CREATE_TOPICS)
+ }
+
+ // Borrow client information such as client id and correlation id from the original request,
+ // in order to correlate the create request with the original metadata request.
+ val requestHeader = new RequestHeader(ApiKeys.CREATE_TOPICS,
+ requestVersion,
+ context.clientId,
+ context.correlationId)
+ ForwardingManager.buildEnvelopeRequest(context,
+ createTopicsRequest.build(requestVersion).serializeWithHeader(requestHeader))
+ }.getOrElse(createTopicsRequest)
+
+ channelManager.sendRequest(request, requestCompletionHandler)
+
+ val creatableTopicResponses = creatableTopics.keySet.toSeq.map { topic =>
+ new MetadataResponseTopic()
+ .setErrorCode(Errors.UNKNOWN_TOPIC_OR_PARTITION.code)
+ .setName(topic)
+ .setIsInternal(Topic.isInternal(topic))
+ }
+
+ info(s"Sent auto-creation request for ${creatableTopics.keys} to the active controller.")
Review Comment:
Remove this info log, or set it to debug level. We need to be careful to not create to many log messages that will spam the kafka logs with not so relevant information.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2268282851
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -64,6 +77,9 @@ class DefaultAutoTopicCreationManager(
) extends AutoTopicCreationManager with Logging {
private val inflightTopics = Collections.newSetFromMap(new ConcurrentHashMap[String, java.lang.Boolean]())
+ private val topicCreationErrorCache = new ConcurrentHashMap[String, CachedTopicCreationError]()
+ private val errorCacheTtlMs = config.requestTimeoutMs.toLong * 3 // 3x request timeout
Review Comment:
Not sure if we should couple this to request timeout? -- Might be better to couple it to session timeout? If a client does not heartbeat within session timeout, we would remove it from the group.
Side question: would we need to track error per streams group, and use a group specific ttl, given that each group could set an individual session timeout?
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -112,6 +128,45 @@ class DefaultAutoTopicCreationManager(
}
}
+ override def getTopicCreationErrors(
+ topicNames: Set[String]
+ ): Map[String, String] = {
+ val currentTime = System.currentTimeMillis()
+ val errors = mutable.Map.empty[String, String]
+ val expiredKeys = mutable.Set.empty[String]
+
+ // Check requested topics and collect expired keys
+ topicNames.foreach { topicName =>
+ Option(topicCreationErrorCache.get(topicName)) match {
+ case Some(cachedError) if (currentTime - cachedError.timestamp) <= errorCacheTtlMs =>
Review Comment:
Why would we exclude the error message if we still have it? -- I thought the ttl would apply for the case, that we never returned an error, and want to drop it on the floor, via some cleanup process?
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -135,10 +190,17 @@ class DefaultAutoTopicCreationManager(
clearInflightRequests(creatableTopics)
if (response.authenticationException() != null) {
warn(s"Auto topic creation failed for ${creatableTopics.keys} with authentication exception")
+ cacheTopicCreationErrors(creatableTopics.keys.toSet, "Authentication failed")
Review Comment:
Should we use the error message from `response.authenticationException()` instead of a manually created error message? Also wondering, why we don't log the actual exception error message in the `warn` above?
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -263,4 +325,31 @@ class DefaultAutoTopicCreationManager(
(creatableTopics, uncreatableTopics)
}
+
+ private def cacheTopicCreationErrors(topicNames: Set[String], errorMessage: String): Unit = {
+ val timestamp = System.currentTimeMillis()
+ topicNames.foreach { topicName =>
+ topicCreationErrorCache.put(topicName, CachedTopicCreationError(errorMessage, timestamp))
+ }
+ }
+
+ private def cacheTopicCreationErrorsFromResponse(response: CreateTopicsResponse): Unit = {
+ val timestamp = System.currentTimeMillis()
Review Comment:
One more.
##########
core/src/test/scala/unit/kafka/server/AutoTopicCreationManagerTest.scala:
##########
@@ -393,4 +393,176 @@ class AutoTopicCreationManagerTest {
.setNumPartitions(numPartitions)
.setReplicationFactor(replicationFactor)
}
+
+ @Test
+ def testTopicCreationErrorCaching(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator)
+
+ val topics = Map(
+ "test-topic-1" -> new CreatableTopic().setName("test-topic-1").setNumPartitions(1).setReplicationFactor(1)
+ )
+ val requestContext = initializeRequestContextWithUserPrincipal()
+
+ autoTopicCreationManager.createStreamsInternalTopics(topics, requestContext)
+
+ val argumentCaptor = ArgumentCaptor.forClass(classOf[ControllerRequestCompletionHandler])
+ Mockito.verify(brokerToController).sendRequest(
+ any(classOf[AbstractRequest.Builder[_ <: AbstractRequest]]),
+ argumentCaptor.capture())
+
+ // Simulate a CreateTopicsResponse with errors
+ val createTopicsResponseData = new org.apache.kafka.common.message.CreateTopicsResponseData()
+ val topicResult = new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("test-topic-1")
+ .setErrorCode(Errors.TOPIC_ALREADY_EXISTS.code())
+ .setErrorMessage("Topic 'test-topic-1' already exists.")
+ createTopicsResponseData.topics().add(topicResult)
+
+ val createTopicsResponse = new CreateTopicsResponse(createTopicsResponseData)
+ val header = new RequestHeader(ApiKeys.CREATE_TOPICS, 0, "client", 1)
+ val clientResponse = new ClientResponse(header, null, null,
+ 0, 0, false, null, null, createTopicsResponse)
+
+ // Trigger the completion handler
+ argumentCaptor.getValue.asInstanceOf[ControllerRequestCompletionHandler].onComplete(clientResponse)
+
+ // Verify that the error was cached
+ val cachedErrors = autoTopicCreationManager.getTopicCreationErrors(Set("test-topic-1"))
+ assertEquals(1, cachedErrors.size)
+ assertTrue(cachedErrors.contains("test-topic-1"))
+ assertEquals("Topic 'test-topic-1' already exists.", cachedErrors("test-topic-1"))
+ }
+
+ @Test
+ def testGetTopicCreationErrorsWithMultipleTopics(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator)
+
+ val topics = Map(
+ "success-topic" -> new CreatableTopic().setName("success-topic").setNumPartitions(1).setReplicationFactor(1),
+ "failed-topic" -> new CreatableTopic().setName("failed-topic").setNumPartitions(1).setReplicationFactor(1)
+ )
+ val requestContext = initializeRequestContextWithUserPrincipal()
+ autoTopicCreationManager.createStreamsInternalTopics(topics, requestContext)
+
+ val argumentCaptor = ArgumentCaptor.forClass(classOf[ControllerRequestCompletionHandler])
+ Mockito.verify(brokerToController).sendRequest(
+ any(classOf[AbstractRequest.Builder[_ <: AbstractRequest]]),
+ argumentCaptor.capture())
+
+ // Simulate mixed response - one success, one failure
+ val createTopicsResponseData = new org.apache.kafka.common.message.CreateTopicsResponseData()
+ createTopicsResponseData.topics().add(
+ new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("success-topic")
+ .setErrorCode(Errors.NONE.code())
+ )
+ createTopicsResponseData.topics().add(
+ new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("failed-topic")
+ .setErrorCode(Errors.POLICY_VIOLATION.code())
+ .setErrorMessage("Policy violation")
+ )
+
+ val createTopicsResponse = new CreateTopicsResponse(createTopicsResponseData)
+ val header = new RequestHeader(ApiKeys.CREATE_TOPICS, 0, "client", 1)
+ val clientResponse = new ClientResponse(header, null, null,
+ 0, 0, false, null, null, createTopicsResponse)
+
+ argumentCaptor.getValue.asInstanceOf[ControllerRequestCompletionHandler].onComplete(clientResponse)
+
+ // Only the failed topic should be cached
+ val cachedErrors = autoTopicCreationManager.getTopicCreationErrors(Set("success-topic", "failed-topic", "nonexistent-topic"))
+ assertEquals(1, cachedErrors.size)
+ assertTrue(cachedErrors.contains("failed-topic"))
+ assertEquals("Policy violation", cachedErrors("failed-topic"))
+ }
+
+ @Test
+ def testLazyCleanupOfExpiredCacheEntries(): Unit = {
+ // Test will verify that expired entries are cleaned up when accessed
+ // We'll simulate the passage of time by directly manipulating the cache
+
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator)
+
+ // Manually add an expired entry to the cache using reflection
Review Comment:
Seems you need to use reflection as you cannot modify the time? If we use `MockTime` in the test, we should be able to avoid reflection.
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -64,6 +77,9 @@ class DefaultAutoTopicCreationManager(
) extends AutoTopicCreationManager with Logging {
private val inflightTopics = Collections.newSetFromMap(new ConcurrentHashMap[String, java.lang.Boolean]())
+ private val topicCreationErrorCache = new ConcurrentHashMap[String, CachedTopicCreationError]()
+ private val errorCacheTtlMs = config.requestTimeoutMs.toLong * 3 // 3x request timeout
+ private val maxCacheSize = 1000
Review Comment:
Wondering why we would need to bound the cache size? What is the reasoning for this?
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -112,6 +128,45 @@ class DefaultAutoTopicCreationManager(
}
}
+ override def getTopicCreationErrors(
+ topicNames: Set[String]
+ ): Map[String, String] = {
+ val currentTime = System.currentTimeMillis()
+ val errors = mutable.Map.empty[String, String]
+ val expiredKeys = mutable.Set.empty[String]
+
+ // Check requested topics and collect expired keys
+ topicNames.foreach { topicName =>
+ Option(topicCreationErrorCache.get(topicName)) match {
+ case Some(cachedError) if (currentTime - cachedError.timestamp) <= errorCacheTtlMs =>
+ errors.put(topicName, cachedError.errorMessage)
+ case Some(_) =>
+ expiredKeys += topicName
Review Comment:
Not sure if I understand this logic? I though we would expire an entry, if we never returned it to the client, and if TTL passed?
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -112,6 +128,45 @@ class DefaultAutoTopicCreationManager(
}
}
+ override def getTopicCreationErrors(
+ topicNames: Set[String]
+ ): Map[String, String] = {
+ val currentTime = System.currentTimeMillis()
Review Comment:
We should never call `System.currentTimeMillis()` directly, but use a `Time` object -- as the `AutoTopicCreationManager` does not have one yet, we need to add it to the constructor and make available, and trace back where `AutoTopicCreationManager` is setup to find the `Time` object there.
Using `Time` interface allows us to mock time in unit tests.
##########
core/src/test/scala/unit/kafka/server/KafkaApisTest.scala:
##########
@@ -10949,6 +10950,59 @@ class KafkaApisTest extends Logging {
)
}
+ @Test
+ def testStreamsGroupHeartbeatRequestWithCachedTopicCreationErrors(): Unit = {
Review Comment:
It's a little unclear to me, what this method is supposed to actually verify?
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -135,10 +190,17 @@ class DefaultAutoTopicCreationManager(
clearInflightRequests(creatableTopics)
if (response.authenticationException() != null) {
warn(s"Auto topic creation failed for ${creatableTopics.keys} with authentication exception")
+ cacheTopicCreationErrors(creatableTopics.keys.toSet, "Authentication failed")
} else if (response.versionMismatch() != null) {
warn(s"Auto topic creation failed for ${creatableTopics.keys} with invalid version exception")
+ cacheTopicCreationErrors(creatableTopics.keys.toSet, "Version mismatch")
Review Comment:
As above
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -112,6 +128,45 @@ class DefaultAutoTopicCreationManager(
}
}
+ override def getTopicCreationErrors(
+ topicNames: Set[String]
+ ): Map[String, String] = {
+ val currentTime = System.currentTimeMillis()
+ val errors = mutable.Map.empty[String, String]
+ val expiredKeys = mutable.Set.empty[String]
+
+ // Check requested topics and collect expired keys
+ topicNames.foreach { topicName =>
+ Option(topicCreationErrorCache.get(topicName)) match {
+ case Some(cachedError) if (currentTime - cachedError.timestamp) <= errorCacheTtlMs =>
+ errors.put(topicName, cachedError.errorMessage)
+ case Some(_) =>
Review Comment:
If we are using `Some(cacheError)` above, would this case actually every be executed (I am not a Scala person, but my understanding is, that `Same(cacheError)` would be a "catch all"?
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -263,4 +325,31 @@ class DefaultAutoTopicCreationManager(
(creatableTopics, uncreatableTopics)
}
+
+ private def cacheTopicCreationErrors(topicNames: Set[String], errorMessage: String): Unit = {
+ val timestamp = System.currentTimeMillis()
Review Comment:
As above
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -263,4 +325,31 @@ class DefaultAutoTopicCreationManager(
(creatableTopics, uncreatableTopics)
}
+
+ private def cacheTopicCreationErrors(topicNames: Set[String], errorMessage: String): Unit = {
+ val timestamp = System.currentTimeMillis()
+ topicNames.foreach { topicName =>
+ topicCreationErrorCache.put(topicName, CachedTopicCreationError(errorMessage, timestamp))
Review Comment:
Should we change `CachedTopicCreationError` constructor to set timestamp automatically, instead of passing in it? If we just get current ms ts anyway, we can move the code into the constructor. Or would we ever need to pass in something different?
##########
core/src/main/scala/kafka/server/KafkaApis.scala:
##########
@@ -2889,6 +2889,24 @@ class KafkaApis(val requestChannel: RequestChannel,
}
} else {
autoTopicCreationManager.createStreamsInternalTopics(topicsToCreate, requestContext);
+
+ // Check for cached topic creation errors only if there's already a MISSING_INTERNAL_TOPICS status
+ val hasMissingInternalTopicsStatus = responseData.status() != null &&
+ responseData.status().stream().anyMatch(s => s.statusCode() == StreamsGroupHeartbeatResponse.Status.MISSING_INTERNAL_TOPICS.code())
Review Comment:
Not sure if I can follow? Why would we need to check `MISSING_INTERNAL_TOPICS` status?
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -135,10 +190,17 @@ class DefaultAutoTopicCreationManager(
clearInflightRequests(creatableTopics)
if (response.authenticationException() != null) {
warn(s"Auto topic creation failed for ${creatableTopics.keys} with authentication exception")
+ cacheTopicCreationErrors(creatableTopics.keys.toSet, "Authentication failed")
} else if (response.versionMismatch() != null) {
warn(s"Auto topic creation failed for ${creatableTopics.keys} with invalid version exception")
+ cacheTopicCreationErrors(creatableTopics.keys.toSet, "Version mismatch")
} else {
- debug(s"Auto topic creation completed for ${creatableTopics.keys} with response ${response.responseBody}.")
+ response.responseBody() match {
+ case createTopicsResponse: CreateTopicsResponse =>
+ cacheTopicCreationErrorsFromResponse(createTopicsResponse)
Review Comment:
I thought, for this `else` branch, the request was successful and no error would be returned?
##########
core/src/test/scala/unit/kafka/server/KafkaApisTest.scala:
##########
@@ -10949,6 +10950,59 @@ class KafkaApisTest extends Logging {
)
}
+ @Test
+ def testStreamsGroupHeartbeatRequestWithCachedTopicCreationErrors(): Unit = {
+ val features = mock(classOf[FinalizedFeatures])
+ when(features.finalizedFeatures()).thenReturn(util.Map.of(StreamsVersion.FEATURE_NAME, 1.toShort))
+
+ metadataCache = mock(classOf[KRaftMetadataCache])
+ when(metadataCache.features()).thenReturn(features)
+
+ val streamsGroupHeartbeatRequest = new StreamsGroupHeartbeatRequestData().setGroupId("group")
+ val requestChannelRequest = buildRequest(new StreamsGroupHeartbeatRequest.Builder(streamsGroupHeartbeatRequest, true).build())
+
+ val future = new CompletableFuture[StreamsGroupHeartbeatResult]()
+ when(groupCoordinator.streamsGroupHeartbeat(
+ requestChannelRequest.context,
+ streamsGroupHeartbeatRequest
+ )).thenReturn(future)
+
+ // Mock AutoTopicCreationManager to return cached errors
+ val mockAutoTopicCreationManager = mock(classOf[AutoTopicCreationManager])
+ when(mockAutoTopicCreationManager.getTopicCreationErrors(Set("test-topic")))
+ .thenReturn(Map("test-topic" -> "INVALID_REPLICATION_FACTOR"))
+
+ kafkaApis = createKafkaApis(autoTopicCreationManager = Some(mockAutoTopicCreationManager))
+ kafkaApis.handle(requestChannelRequest, RequestLocal.noCaching)
+
+ // Group coordinator returns MISSING_INTERNAL_TOPICS status and topics to create
+ val missingTopics = util.Map.of("test-topic", new CreatableTopic())
+ val streamsGroupHeartbeatResponse = new StreamsGroupHeartbeatResponseData()
+ .setMemberId("member")
+ .setStatus(util.List.of(
+ new StreamsGroupHeartbeatResponseData.Status()
+ .setStatusCode(StreamsGroupHeartbeatResponse.Status.MISSING_INTERNAL_TOPICS.code())
+ .setStatusDetail("Internal topics are missing: [test-topic]")
+ ))
+
+ future.complete(new StreamsGroupHeartbeatResult(streamsGroupHeartbeatResponse, missingTopics))
+ val response = verifyNoThrottling[StreamsGroupHeartbeatResponse](requestChannelRequest)
+
+ assertEquals(Errors.NONE.code, response.data.errorCode())
+ assertEquals(null, response.data.errorMessage())
+
+ // Verify that the cached error was appended to the existing status detail
Review Comment:
Are we really verifying this? It seem our test code, does assemble the `StreamsGroupHeartbeatResponseData`, so we don't really execute prod code? So are we only verifying that our test code does setup the right response? For this case, it seems the test would not actually test anything?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "aliehsaeedii (via GitHub)" <gi...@apache.org>.
aliehsaeedii commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2273377362
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -134,11 +180,20 @@ class DefaultAutoTopicCreationManager(
override def onComplete(response: ClientResponse): Unit = {
clearInflightRequests(creatableTopics)
if (response.authenticationException() != null) {
- warn(s"Auto topic creation failed for ${creatableTopics.keys} with authentication exception")
+ val authException = response.authenticationException()
+ warn(s"Auto topic creation failed for ${creatableTopics.keys} with authentication exception: ${authException.getMessage}")
+ cacheTopicCreationErrors(creatableTopics.keys.toSet, authException.getMessage)
} else if (response.versionMismatch() != null) {
- warn(s"Auto topic creation failed for ${creatableTopics.keys} with invalid version exception")
+ val versionException = response.versionMismatch()
+ warn(s"Auto topic creation failed for ${creatableTopics.keys} with version mismatch exception: ${versionException.getMessage}")
+ cacheTopicCreationErrors(creatableTopics.keys.toSet, versionException.getMessage)
} else {
- debug(s"Auto topic creation completed for ${creatableTopics.keys} with response ${response.responseBody}.")
+ response.responseBody() match {
+ case createTopicsResponse: CreateTopicsResponse =>
+ cacheTopicCreationErrorsFromResponse(createTopicsResponse)
+ case _ =>
Review Comment:
What sort of response could it be?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "aliehsaeedii (via GitHub)" <gi...@apache.org>.
aliehsaeedii commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2273440201
##########
core/src/main/scala/kafka/server/KafkaApis.scala:
##########
@@ -2889,6 +2889,24 @@ class KafkaApis(val requestChannel: RequestChannel,
}
} else {
autoTopicCreationManager.createStreamsInternalTopics(topicsToCreate, requestContext);
+
+ // Check for cached topic creation errors only if there's already a MISSING_INTERNAL_TOPICS status
+ val hasMissingInternalTopicsStatus = responseData.status() != null &&
+ responseData.status().stream().anyMatch(s => s.statusCode() == StreamsGroupHeartbeatResponse.Status.MISSING_INTERNAL_TOPICS.code())
Review Comment:
What do we have in the cache in case of version mismatch or auth error? Is the `MISSING_INTERNAL_TOPICS` present there?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "RaidenE1 (via GitHub)" <gi...@apache.org>.
RaidenE1 commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2270947839
##########
core/src/main/scala/kafka/server/AutoTopicCreationManager.scala:
##########
@@ -135,10 +190,17 @@ class DefaultAutoTopicCreationManager(
clearInflightRequests(creatableTopics)
if (response.authenticationException() != null) {
warn(s"Auto topic creation failed for ${creatableTopics.keys} with authentication exception")
+ cacheTopicCreationErrors(creatableTopics.keys.toSet, "Authentication failed")
Review Comment:
Makes sense, fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "RaidenE1 (via GitHub)" <gi...@apache.org>.
RaidenE1 commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2271540985
##########
core/src/test/scala/unit/kafka/server/KafkaApisTest.scala:
##########
@@ -10949,6 +10950,59 @@ class KafkaApisTest extends Logging {
)
}
+ @Test
+ def testStreamsGroupHeartbeatRequestWithCachedTopicCreationErrors(): Unit = {
+ val features = mock(classOf[FinalizedFeatures])
+ when(features.finalizedFeatures()).thenReturn(util.Map.of(StreamsVersion.FEATURE_NAME, 1.toShort))
+
+ metadataCache = mock(classOf[KRaftMetadataCache])
+ when(metadataCache.features()).thenReturn(features)
+
+ val streamsGroupHeartbeatRequest = new StreamsGroupHeartbeatRequestData().setGroupId("group")
+ val requestChannelRequest = buildRequest(new StreamsGroupHeartbeatRequest.Builder(streamsGroupHeartbeatRequest, true).build())
+
+ val future = new CompletableFuture[StreamsGroupHeartbeatResult]()
+ when(groupCoordinator.streamsGroupHeartbeat(
+ requestChannelRequest.context,
+ streamsGroupHeartbeatRequest
+ )).thenReturn(future)
+
+ // Mock AutoTopicCreationManager to return cached errors
+ val mockAutoTopicCreationManager = mock(classOf[AutoTopicCreationManager])
+ when(mockAutoTopicCreationManager.getTopicCreationErrors(Set("test-topic")))
+ .thenReturn(Map("test-topic" -> "INVALID_REPLICATION_FACTOR"))
+
+ kafkaApis = createKafkaApis(autoTopicCreationManager = Some(mockAutoTopicCreationManager))
+ kafkaApis.handle(requestChannelRequest, RequestLocal.noCaching)
+
+ // Group coordinator returns MISSING_INTERNAL_TOPICS status and topics to create
+ val missingTopics = util.Map.of("test-topic", new CreatableTopic())
+ val streamsGroupHeartbeatResponse = new StreamsGroupHeartbeatResponseData()
+ .setMemberId("member")
+ .setStatus(util.List.of(
+ new StreamsGroupHeartbeatResponseData.Status()
+ .setStatusCode(StreamsGroupHeartbeatResponse.Status.MISSING_INTERNAL_TOPICS.code())
+ .setStatusDetail("Internal topics are missing: [test-topic]")
+ ))
+
+ future.complete(new StreamsGroupHeartbeatResult(streamsGroupHeartbeatResponse, missingTopics))
+ val response = verifyNoThrottling[StreamsGroupHeartbeatResponse](requestChannelRequest)
+
+ assertEquals(Errors.NONE.code, response.data.errorCode())
+ assertEquals(null, response.data.errorMessage())
+
+ // Verify that the cached error was appended to the existing status detail
Review Comment:
added mock create topic
##########
core/src/test/scala/unit/kafka/server/AutoTopicCreationManagerTest.scala:
##########
@@ -393,4 +393,176 @@ class AutoTopicCreationManagerTest {
.setNumPartitions(numPartitions)
.setReplicationFactor(replicationFactor)
}
+
+ @Test
+ def testTopicCreationErrorCaching(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator)
+
+ val topics = Map(
+ "test-topic-1" -> new CreatableTopic().setName("test-topic-1").setNumPartitions(1).setReplicationFactor(1)
+ )
+ val requestContext = initializeRequestContextWithUserPrincipal()
+
+ autoTopicCreationManager.createStreamsInternalTopics(topics, requestContext)
+
+ val argumentCaptor = ArgumentCaptor.forClass(classOf[ControllerRequestCompletionHandler])
+ Mockito.verify(brokerToController).sendRequest(
+ any(classOf[AbstractRequest.Builder[_ <: AbstractRequest]]),
+ argumentCaptor.capture())
+
+ // Simulate a CreateTopicsResponse with errors
+ val createTopicsResponseData = new org.apache.kafka.common.message.CreateTopicsResponseData()
+ val topicResult = new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("test-topic-1")
+ .setErrorCode(Errors.TOPIC_ALREADY_EXISTS.code())
+ .setErrorMessage("Topic 'test-topic-1' already exists.")
+ createTopicsResponseData.topics().add(topicResult)
+
+ val createTopicsResponse = new CreateTopicsResponse(createTopicsResponseData)
+ val header = new RequestHeader(ApiKeys.CREATE_TOPICS, 0, "client", 1)
+ val clientResponse = new ClientResponse(header, null, null,
+ 0, 0, false, null, null, createTopicsResponse)
+
+ // Trigger the completion handler
+ argumentCaptor.getValue.asInstanceOf[ControllerRequestCompletionHandler].onComplete(clientResponse)
+
+ // Verify that the error was cached
+ val cachedErrors = autoTopicCreationManager.getTopicCreationErrors(Set("test-topic-1"))
+ assertEquals(1, cachedErrors.size)
+ assertTrue(cachedErrors.contains("test-topic-1"))
+ assertEquals("Topic 'test-topic-1' already exists.", cachedErrors("test-topic-1"))
+ }
+
+ @Test
+ def testGetTopicCreationErrorsWithMultipleTopics(): Unit = {
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator)
+
+ val topics = Map(
+ "success-topic" -> new CreatableTopic().setName("success-topic").setNumPartitions(1).setReplicationFactor(1),
+ "failed-topic" -> new CreatableTopic().setName("failed-topic").setNumPartitions(1).setReplicationFactor(1)
+ )
+ val requestContext = initializeRequestContextWithUserPrincipal()
+ autoTopicCreationManager.createStreamsInternalTopics(topics, requestContext)
+
+ val argumentCaptor = ArgumentCaptor.forClass(classOf[ControllerRequestCompletionHandler])
+ Mockito.verify(brokerToController).sendRequest(
+ any(classOf[AbstractRequest.Builder[_ <: AbstractRequest]]),
+ argumentCaptor.capture())
+
+ // Simulate mixed response - one success, one failure
+ val createTopicsResponseData = new org.apache.kafka.common.message.CreateTopicsResponseData()
+ createTopicsResponseData.topics().add(
+ new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("success-topic")
+ .setErrorCode(Errors.NONE.code())
+ )
+ createTopicsResponseData.topics().add(
+ new org.apache.kafka.common.message.CreateTopicsResponseData.CreatableTopicResult()
+ .setName("failed-topic")
+ .setErrorCode(Errors.POLICY_VIOLATION.code())
+ .setErrorMessage("Policy violation")
+ )
+
+ val createTopicsResponse = new CreateTopicsResponse(createTopicsResponseData)
+ val header = new RequestHeader(ApiKeys.CREATE_TOPICS, 0, "client", 1)
+ val clientResponse = new ClientResponse(header, null, null,
+ 0, 0, false, null, null, createTopicsResponse)
+
+ argumentCaptor.getValue.asInstanceOf[ControllerRequestCompletionHandler].onComplete(clientResponse)
+
+ // Only the failed topic should be cached
+ val cachedErrors = autoTopicCreationManager.getTopicCreationErrors(Set("success-topic", "failed-topic", "nonexistent-topic"))
+ assertEquals(1, cachedErrors.size)
+ assertTrue(cachedErrors.contains("failed-topic"))
+ assertEquals("Policy violation", cachedErrors("failed-topic"))
+ }
+
+ @Test
+ def testLazyCleanupOfExpiredCacheEntries(): Unit = {
+ // Test will verify that expired entries are cleaned up when accessed
+ // We'll simulate the passage of time by directly manipulating the cache
+
+ autoTopicCreationManager = new DefaultAutoTopicCreationManager(
+ config,
+ brokerToController,
+ groupCoordinator,
+ transactionCoordinator,
+ shareCoordinator)
+
+ // Manually add an expired entry to the cache using reflection
Review Comment:
use mock time and remove reflection
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] MINOR: Expose internal topic creation errors to the user [kafka]
Posted by "lucasbru (via GitHub)" <gi...@apache.org>.
lucasbru commented on code in PR #20325:
URL: https://github.com/apache/kafka/pull/20325#discussion_r2301479085
##########
core/src/main/scala/kafka/server/KafkaApis.scala:
##########
@@ -2889,6 +2889,24 @@ class KafkaApis(val requestChannel: RequestChannel,
}
} else {
autoTopicCreationManager.createStreamsInternalTopics(topicsToCreate, requestContext);
+
+ // Check for cached topic creation errors only if there's already a MISSING_INTERNAL_TOPICS status
+ val hasMissingInternalTopicsStatus = responseData.status() != null &&
+ responseData.status().stream().anyMatch(s => s.statusCode() == StreamsGroupHeartbeatResponse.Status.MISSING_INTERNAL_TOPICS.code())
Review Comment:
It's still fine to have this as a defensive measure, since we definitely do not want to break our heartbeat response because for whatever the error wasn't cleared
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org