[jira] [Created] (DL-101) Improve session expire handling on fetching log segments for readers

["Sijie Guo (JIRA)" <ji...@apache.org>]

Sijie Guo created DL-101:
----------------------------

             Summary: Improve session expire handling on fetching log segments for readers
                 Key: DL-101
                 URL: https://issues.apache.org/jira/browse/DL-101
             Project: DistributedLog
          Issue Type: Improvement
          Components: distributedlog-core
            Reporter: Sijie Guo
            Assignee: Sijie Guo


    The log segment management in DL is now done by 3 parts.

    - a LogSegmentMetadataStore (one per namespace instance): it is used for fetching the log segments from log segment metadata store (ZooKeeper). it doesn't do any caching.
    - a LogSegmentMetadataCache (one per namespace instance): it is a guava cache based metadata cache. it maintains a mapping between log segment metadata path and the log segment metadata. it manages the cache for the log segments that will be accessed in this namespace instance. it doesn't manage the sequence of the log segments for streams.
    - a PerStreamLogSegmentCache for each BKLogHandler. the log segment cache is per stream. it maintains the sequence of the log segments.

    BKLogWriteHandler doesn't watch the log segment changes. It fetches minimal number of log segments when it is created and fetches the full list of log segments for truncations. New log segments will be added to the per stream log segment cache with log segment rolling.

    BKLogReadHandler watch the log segments changes and only notify when the list of log segments is changed. the session handling which is specific to the metadata store is hidden to the implementations of LogSegmentMetadataStore.

    The change tries to cleanup bunch of unused methods in BKLog{Read,Write}Handler too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)