[jira] [Created] (KAFKA-8987) Allow IQ on restored state stores while others are still restoring

["Sophie Blee-Goldman (Jira)" <ji...@apache.org>]

Sophie Blee-Goldman created KAFKA-8987:
------------------------------------------

             Summary: Allow IQ on restored state stores while others are still restoring
                 Key: KAFKA-8987
                 URL: https://issues.apache.org/jira/browse/KAFKA-8987
             Project: Kafka
          Issue Type: Improvement
          Components: streams
            Reporter: Sophie Blee-Goldman


Currently we block IQ while we are not in RUNNING state, meaning throughout the rebalancing and restoring phase. With KIP-429 we should be able to reduce the unavailability gap to just the restoring phase, which should be only those stores which had to be migrated to balance the cluster.

This means we will often have a large number of tasks that are completely restored and ready to start processing once we finish restoring some smaller subset of tasks. For strict correctness (see KAFKA-6144) we should continue to block IQ on those restoring tasks, but there is no reason to prevent the stores that are all caught up from serving queries.

We should refactor the IQ code a bit following the work of 429 to reduce the unavailability gap even further and block IQ only on those tasks that are not yet restored



--
This message was sent by Atlassian Jira
(v8.3.4#803005)