You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by ka...@apache.org on 2023/11/15 13:11:28 UTC
(druid) branch master updated: Document Nuances in SegmentMetadataCache Behaviour (#15367)
This is an automated email from the ASF dual-hosted git repository.
karan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/druid.git
The following commit(s) were added to refs/heads/master by this push:
new 03a092f4ece Document Nuances in SegmentMetadataCache Behaviour (#15367)
03a092f4ece is described below
commit 03a092f4ecedf1950321b12046c7f35fa52d91a9
Author: Rishabh Singh <65...@users.noreply.github.com>
AuthorDate: Wed Nov 15 18:41:20 2023 +0530
Document Nuances in SegmentMetadataCache Behaviour (#15367)
* Document segment metadata cache behaviour
* Fix typo
* Minor update
* Minor change
---
.../segment/metadata/AbstractSegmentMetadataCache.java | 15 ++++++++++++---
1 file changed, 12 insertions(+), 3 deletions(-)
diff --git a/server/src/main/java/org/apache/druid/segment/metadata/AbstractSegmentMetadataCache.java b/server/src/main/java/org/apache/druid/segment/metadata/AbstractSegmentMetadataCache.java
index bcc08f54274..512f247c4fc 100644
--- a/server/src/main/java/org/apache/druid/segment/metadata/AbstractSegmentMetadataCache.java
+++ b/server/src/main/java/org/apache/druid/segment/metadata/AbstractSegmentMetadataCache.java
@@ -90,9 +90,18 @@ import java.util.stream.StreamSupport;
* An abstract class that listens for segment change events and caches segment metadata. It periodically refreshes
* the segments, by fetching their metadata which includes schema information from sources like
* data nodes, tasks, metadata database and builds table schema.
- *
- * <p>This class has an abstract method {@link #refresh(Set, Set)} which the child class must override
- * with the logic to build and cache table schema.</p>
+ * <p>
+ * At startup, the cache awaits the initialization of the timeline.
+ * If the cache employs a segment metadata query to retrieve segment schema, it attempts to refresh a maximum
+ * of {@code MAX_SEGMENTS_PER_QUERY} segments for each datasource in each refresh cycle.
+ * Once all datasources have undergone this process, the initial schema of each datasource is constructed,
+ * and the cache is marked as initialized.
+ * Subsequently, the cache continues to periodically refresh segments and update the datasource schema.
+ * It is also important to note that a failure in segment refresh results in pausing the refresh work,
+ * and the process is resumed in the next refresh cycle.
+ * <p>
+ * This class has an abstract method {@link #refresh(Set, Set)} which the child class must override
+ * with the logic to build and cache table schema.
*
* @param <T> The type of information associated with the data source, which must extend {@link DataSourceInformation}.
*/
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org