[jira] [Updated] (SAMZA-223) Add a cache for stream metadata

["Martin Kleppmann (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SAMZA-223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Martin Kleppmann updated SAMZA-223:
-----------------------------------

    Attachment: SAMZA-223.1.patch

RB: https://reviews.apache.org/r/20056/ and patch attached. This patch incorporates Chris' and Jakob's comments on https://reviews.apache.org/r/19384/ (insofar they apply to the metadata caching code).

> Add a cache for stream metadata
> -------------------------------
>
>                 Key: SAMZA-223
>                 URL: https://issues.apache.org/jira/browse/SAMZA-223
>             Project: Samza
>          Issue Type: Improvement
>            Reporter: Martin Kleppmann
>            Assignee: Martin Kleppmann
>         Attachments: SAMZA-223.1.patch
>
>
> As part of SAMZA-179 I wrote a cache to reduce the number of calls to SystemAdmin.getSystemStreamMetadata (which go to the broker). It's currently unclear whether we still want to use the stream metadata to determine whether a task is caught up, so I've split the StreamMetadataCache out of SAMZA-179, and made it a standalone patch. This patch doesn't contain any of the job shutdown stuff, only the metadata caching.
> Admittedly there's less of a need for this cache if we're not requesting stream metadata within tasks, so perhaps we should just throw this code away. But it may also be useful, and certainly shouldn't do any harm. What do you think?



--
This message was sent by Atlassian JIRA
(v6.2#6252)