[jira] [Created] (KAFKA-13096) QueryableStoreProvider is not updated when threads are added/removed/replaced rendering IQ impossible

["A. Sophie Blee-Goldman (Jira)" <ji...@apache.org>]

A. Sophie Blee-Goldman created KAFKA-13096:
----------------------------------------------

             Summary: QueryableStoreProvider is not updated when threads are added/removed/replaced rendering IQ impossible
                 Key: KAFKA-13096
                 URL: https://issues.apache.org/jira/browse/KAFKA-13096
             Project: Kafka
          Issue Type: Bug
          Components: streams
    Affects Versions: 2.8.0
            Reporter: A. Sophie Blee-Goldman
             Fix For: 3.0.0, 2.8.1


The QueryableStoreProviders class is used to route queries to the correct state store on the owning StreamThread, making it a critical piece of IQ. It gets instantiated when you create a new KafkaStreams, and is passed in a list of StreamThreadStateStoreProviders which it then copies and stores. Because it only stores a copy it only ever contains a provider for the StreamThreads that were created during the app's startup, and unfortunately is never updated during an add/remove/replace thread event. 

This means that IQ can’t get a handle on any stores that belong to a thread that wasn’t in the original set. If the app is starting up new threads through the #addStreamThread API or following a REPLACE_THREAD event, none of the data in any of the stores owned by that new thread will be accessible by IQ. If a user is removing threads through #removeStreamThread, or threads die and get replaced, you can fall into an endless loop of {{InvalidStateStoreException}} from doing a lookup into stores that have been closed since the thread was removed/died.

If over time all of the original threads are removed or replaced, then IQ won’t work at all.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)