You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by ka...@apache.org on 2023/11/15 13:11:28 UTC

(druid) branch master updated: Document Nuances in SegmentMetadataCache Behaviour (#15367)

This is an automated email from the ASF dual-hosted git repository.

karan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git


The following commit(s) were added to refs/heads/master by this push:
     new 03a092f4ece Document Nuances in SegmentMetadataCache Behaviour (#15367)
03a092f4ece is described below

commit 03a092f4ecedf1950321b12046c7f35fa52d91a9
Author: Rishabh Singh <65...@users.noreply.github.com>
AuthorDate: Wed Nov 15 18:41:20 2023 +0530

    Document Nuances in SegmentMetadataCache Behaviour (#15367)
    
    * Document segment metadata cache behaviour
    
    * Fix typo
    
    * Minor update
    
    * Minor change
---
 .../segment/metadata/AbstractSegmentMetadataCache.java    | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/server/src/main/java/org/apache/druid/segment/metadata/AbstractSegmentMetadataCache.java b/server/src/main/java/org/apache/druid/segment/metadata/AbstractSegmentMetadataCache.java
index bcc08f54274..512f247c4fc 100644
--- a/server/src/main/java/org/apache/druid/segment/metadata/AbstractSegmentMetadataCache.java
+++ b/server/src/main/java/org/apache/druid/segment/metadata/AbstractSegmentMetadataCache.java
@@ -90,9 +90,18 @@ import java.util.stream.StreamSupport;
  * An abstract class that listens for segment change events and caches segment metadata. It periodically refreshes
  * the segments, by fetching their metadata which includes schema information from sources like
  * data nodes, tasks, metadata database and builds table schema.
- *
- * <p>This class has an abstract method {@link #refresh(Set, Set)} which the child class must override
- * with the logic to build and cache table schema.</p>
+ * <p>
+ * At startup, the cache awaits the initialization of the timeline.
+ * If the cache employs a segment metadata query to retrieve segment schema, it attempts to refresh a maximum
+ * of {@code MAX_SEGMENTS_PER_QUERY} segments for each datasource in each refresh cycle.
+ * Once all datasources have undergone this process, the initial schema of each datasource is constructed,
+ * and the cache is marked as initialized.
+ * Subsequently, the cache continues to periodically refresh segments and update the datasource schema.
+ * It is also important to note that a failure in segment refresh results in pausing the refresh work,
+ * and the process is resumed in the next refresh cycle.
+ * <p>
+ * This class has an abstract method {@link #refresh(Set, Set)} which the child class must override
+ * with the logic to build and cache table schema.
  *
  * @param <T> The type of information associated with the data source, which must extend {@link DataSourceInformation}.
  */


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org