[GitHub] [kafka] lucasbru commented on a diff in pull request #13179: KAFKA-10575: Add onRestoreSuspsnded to StateRestoreListener

["lucasbru (via GitHub)" <gi...@apache.org>]

lucasbru commented on code in PR #13179:
URL: https://github.com/apache/kafka/pull/13179#discussion_r1097157421


##########
streams/src/main/java/org/apache/kafka/streams/processor/StateRestoreListener.java:
##########
@@ -37,6 +40,11 @@
  * These two interfaces serve different restoration purposes and users should not try to implement both of them in a single
  * class during state store registration.
  *
+ * <p>
+ * Also note that standby tasks restoration process are not monitored via this interface, since a standby task keep

Review Comment:
   ```suggestion
    * Also note that the restoration process of standby tasks is not monitored via this interface, since a standby task keeps
   ```



##########
streams/src/main/java/org/apache/kafka/streams/processor/StateRestoreListener.java:
##########
@@ -85,4 +93,17 @@ void onRestoreEnd(final TopicPartition topicPartition,
                       final String storeName,
                       final long totalRestored);
 
+    /**
+     * Method called when restoring the {@link StateStore} is suspended due to the task being migrated out of the host.
+     * If the migrated task is recycled or re-assigned back to the current host, another
+     * {@link #onRestoreStart(TopicPartition, String, long, long)} would be called.
+     *
+     * @param topicPartition the TopicPartition containing the values to restore

Review Comment:
   ```suggestion
        * @param topicPartition the {@link TopicPartition} containing the values to restore
   ```



##########
streams/src/main/java/org/apache/kafka/streams/processor/internals/StoreChangelogReader.java:
##########
@@ -986,8 +986,23 @@ public void unregister(final Collection<TopicPartition> revokedChangelogs) {
         for (final TopicPartition partition : revokedChangelogs) {
             final ChangelogMetadata changelogMetadata = changelogs.remove(partition);
             if (changelogMetadata != null) {
+                // if the changelog is still in REGISTERED, it means it has not initialized and started
+                // restoring yet, and hence we should not try to remove the changelog partition
                 if (!changelogMetadata.state().equals(ChangelogState.REGISTERED)) {
                     revokedInitializedChangelogs.add(partition);
+
+                    // if the changelog is not in RESTORING, it means
+                    // the corresponding onRestoreStart was not called; in this case
+                    // we should not call onRestoreSuspended either
+                    if (changelogMetadata.stateManager.taskType() == Task.TaskType.ACTIVE &&
+                        changelogMetadata.state().equals(ChangelogState.RESTORING)) {
+                        try {
+                            final String storeName = changelogMetadata.storeMetadata.store().name();
+                            stateRestoreListener.onRestoreSuspended(partition, storeName, changelogMetadata.totalRestored);
+                        } catch (final Exception e) {
+                            throw new StreamsException("State restore listener failed on restore paused", e);

Review Comment:
   Why wrap the exception?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org