You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@samza.apache.org by "Martin Kleppmann (JIRA)" <ji...@apache.org> on 2014/04/05 00:41:20 UTC

[jira] [Created] (SAMZA-223) Add a cache for stream metadata

Martin Kleppmann created SAMZA-223:
--------------------------------------

             Summary: Add a cache for stream metadata
                 Key: SAMZA-223
                 URL: https://issues.apache.org/jira/browse/SAMZA-223
             Project: Samza
          Issue Type: Improvement
            Reporter: Martin Kleppmann
            Assignee: Martin Kleppmann


As part of SAMZA-179 I wrote a cache to reduce the number of calls to SystemAdmin.getSystemStreamMetadata (which go to the broker). It's currently unclear whether we still want to use the stream metadata to determine whether a task is caught up, so I've split the StreamMetadataCache out of SAMZA-179, and made it a standalone patch. This patch doesn't contain any of the job shutdown stuff, only the metadata caching.

Admittedly there's less of a need for this cache if we're not requesting stream metadata within tasks, so perhaps we should just throw this code away. But it may also be useful, and certainly shouldn't do any harm. What do you think?



--
This message was sent by Atlassian JIRA
(v6.2#6252)