You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "lhotari (via GitHub)" <gi...@apache.org> on 2026/10/13 14:26:28 UTC
[PR] [improve][offload] Share fetched offloaded data between the read handles and deduplicate concurrent fetches [pulsar]
lhotari opened a new pull request, #26885:
URL: https://github.com/apache/pulsar/pull/26885
### Motivation
Each offloaded ledger's read handle reads its data object through one read buffer, the size of `managedLedgerOffloadReadBufferSizeInBytes` (1 MiB by default). Every cursor that reads the ledger shares that buffer. When cursors read the same offloaded ledger at different positions, as subscriptions that fall behind by different amounts do, each read replaces the buffer that the previous read filled, and the same ranges are fetched from the blob store again and again. A read also fetched ranges that it only skips, and it fetched on the ledger's ordered read thread, one range at a time.
The offloading scenario of the Pulsar Performance Testing Framework (#26884) shows it. Six subscriptions read a backlog of 100,000 1 KiB messages from tiered storage, from positions 10,240 messages apart, with 10 ms of latency to an S3-compatible store. On master, each 1 MiB range is fetched 3.8 to 5.3 times with `dispatcherMaxReadBatchSize=500`, and about 18 times with 100, the default before #26744.
### Modifications
- `OffloadedDataChunkCache` keeps the chunks that the read handles fetch. It is shared by the read handles of all offloaders created by the jcloud offloader factory:
- Chunks are aligned to the read buffer size and keyed by bucket, object key and range.
- They are kept within a size budget, with least recently used eviction, and dropped once no read has used them for the expiry time.
- Chunks are reference counted: the cache, a fetch completing the chunk, and each reader hold a reference, and the buffer is released with the last one.
- Concurrent fetches of the same range are deduplicated. A read of a chunk that is being fetched joins that fetch instead of fetching it again. The fetch takes a reference for each read that joined it before it completes, so that the chunk can't be released between the fetch's completion and those reads. A fetch that fails fails the reads that joined it, and the next read fetches again.
- When a read moves on to the next chunk, the chunks after it are fetched in the background into the cache. Fetches run on a small dedicated thread pool instead of the ledger's ordered read thread.
- `BlobStoreBackedInputStreamImpl` reads from a referenced chunk by absolute index, and `skip()` moves the position without fetching the skipped bytes. A stream created without a cache fetches its chunks for itself, as before.
- New broker settings, in `ServiceConfiguration` and `broker.conf`:
| Setting | Default |
|---|---:|
| `managedLedgerOffloadReadCacheSizeInMB` | 64 |
| `managedLedgerOffloadReadCacheExpirySeconds` | 60 |
| `managedLedgerOffloadReadPrefetchChunks` | 1 |
| `managedLedgerOffloadReadFetchThreads` | 4 |
The cache size bounds the cached chunks; chunks that reads are using and fetches in progress come on top of it.
- The settings reach the offloader factory through a new `OffloadedReadCacheConfig` record and a new `LedgerOffloaderFactory.create(...)` overload that takes it. The overload's default implementation calls the existing method, so other offloader implementations are unaffected. The jcloud factory creates its cache from the settings of the first offloader it creates, since they are the broker's.
### Results
The offloading scenario from #26884 on master (`0be9e38b8e0a`) and on this branch, two runs of each, alternating, on one workstation. The storage's request log counted the reads.
| Build | `dispatcherMaxReadBatchSize` | The last subscription received its backlog after | GETs | Read from the storage | Reads of each 1 MiB range |
|---|---:|---:|---:|---:|---:|
| master | 500 (default) | 98.7 s, 108.2 s | 571, 633 | 548, 610 MB | 3.8, 5.3 |
| this branch | 500 | 36.6 s, 35.7 s | 238, 239 | 224, 225 MB | 1.5, 1.6 |
| master | 100 | 513.0 s, 509.8 s | 2,441, 2,441 | 2,385, 2,385 MB | 18.2, 18.4 |
| this branch | 100 | 37.2 s, 36.9 s | 242, 240 | 228, 226 MB | 1.6, 1.6 |
With the cache, the subscriptions receive their backlog about 2.8 times faster with the default `dispatcherMaxReadBatchSize`, and the read batch size no longer matters. The ranges that are still read more than once are mostly chunks that the cache's 64 MB budget dropped before the trailing subscriptions reached them, since the offloaded backlog is about 100 MB.


### Verifying this change
- [x] Make sure that the change passes the CI checks.
This change added tests and can be verified as follows:
- `OffloadedDataChunkCacheTest`:
- concurrent reads of a chunk sharing one fetch;
- cache hits and the size budget, and expiry;
- an evicted chunk staying readable until its last reader releases it;
- no budget;
- failed fetches and fetching again;
- an interrupted read releasing its reference;
- closing the cache failing the reads that wait for a fetch;
- prefetching, and not prefetching chunks larger than the budget;
- two streams reading an object at alternating positions fetching each chunk once;
- skipping without fetching.
- The existing tiered storage tests, `BlobStoreBackedInputStreamTest` and `BlobStoreBackedReadHandleImplTest` among them, pass unchanged, apart from the buffer-filling test's use of the new helper.
- The offloading scenario runs above, which also check that every subscription received every message, in order.
### Does this pull request potentially affect one of the following parts:
<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
*If the box was checked, please highlight the changes*
- [ ] Dependencies (add or upgrade a dependency)
- [ ] The public API
- [ ] The schema
- [ ] The default values of configurations
- [x] The threading model
- [ ] The binary protocol
- [ ] The REST endpoints
- [ ] The admin CLI options
- [ ] The metrics
- [ ] Anything that affects deployment
The change adds broker settings and an `@Evolving` `LedgerOffloaderFactory` method with a default implementation. Threading model: reads of offloaded ledgers fetch on a new thread pool, `offloader-chunk-fetch`, and wait for the fetch on the ledger's ordered read thread, and use up to `managedLedgerOffloadReadCacheSizeInMB` of direct memory for the cache.
### Documentation
- [x] `doc-not-needed` — the new settings are documented in `ServiceConfiguration` and `broker.conf`, from which the reference documentation is generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [improve][offload] Share fetched offloaded data between the read handles and deduplicate concurrent fetches [pulsar]
Posted by "void-ptr974 (via GitHub)" <gi...@apache.org>.
void-ptr974 commented on code in PR #26885:
URL: https://github.com/apache/pulsar/pull/26885#discussion_r4321916146
##########
tiered-storage/jcloud/src/main/java/org/apache/bookkeeper/mledger/offload/jcloud/impl/OffloadedDataChunkCache.java:
##########
@@ -0,0 +1,422 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.bookkeeper.mledger.offload.jcloud.impl;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.cache.Cache;
+import com.google.common.cache.CacheBuilder;
+import com.google.common.cache.RemovalListener;
+import com.google.common.util.concurrent.ThreadFactoryBuilder;
+import io.netty.buffer.ByteBuf;
+import java.io.IOException;
+import java.io.InterruptedIOException;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.RejectedExecutionException;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.LongAdder;
+import lombok.CustomLog;
+import org.apache.bookkeeper.mledger.OffloadedReadCacheConfig;
+
+/**
+ * The ranges of offloaded data objects that the read handles have fetched from the blob store, shared by every read
+ * handle of the offloader factory: chunks of an object, aligned to the read buffer size, kept within a size budget and
+ * dropped when no read has used them for a while. Cursors that read a ledger at different positions each keep their
+ * chunk instead of replacing each other's, and a read of a chunk that is being fetched waits for that fetch instead of
+ * fetching it again. A read fetches the chunk that it needs itself, on its thread. A reader that moves on to the next
+ * chunk has the chunks after it fetched in the background, on the prefetch threads, at most as many at a time as there
+ * are threads, so that a prefetch never waits in a queue.
+ *
+ * <p>Chunks are reference counted: the cache, the fetch that is completing it, and each reader using it hold a
+ * reference, and the chunk's buffer is released when the last one is released.
+ */
+@CustomLog
+public class OffloadedDataChunkCache implements AutoCloseable {
+ /** A chunk of a data object: the bytes from {@code start}, {@code length} of them. */
+ record Key(String bucket, String key, long start, int length) {
+ }
+
+ /** Fetches a chunk's bytes from the blob store into a buffer that the caller then owns. */
+ @FunctionalInterface
+ interface Fetcher {
+ ByteBuf fetch(Key key) throws IOException;
+ }
+
+ /** A fetched chunk; its buffer is released when the last reference is released. */
+ static final class Chunk {
+ private final ByteBuf buffer;
+ private final AtomicInteger references = new AtomicInteger(1);
+ // Runs when the buffer is released; null for none
+ private final Runnable onRelease;
+
+ Chunk(ByteBuf buffer) {
+ this(buffer, null);
+ }
+
+ Chunk(ByteBuf buffer, Runnable onRelease) {
+ this.buffer = buffer;
+ this.onRelease = onRelease;
+ }
+
+ ByteBuf buffer() {
+ return buffer;
+ }
+
+ /** Takes a reference unless the chunk has already been released. */
+ boolean tryRetain() {
+ for (;;) {
+ int current = references.get();
+ if (current == 0) {
+ return false;
+ }
+ if (references.compareAndSet(current, current + 1)) {
+ return true;
+ }
+ }
+ }
+
+ void retain(int count) {
+ if (count > 0) {
+ references.addAndGet(count);
+ }
+ }
+
+ void release() {
+ if (references.decrementAndGet() == 0) {
+ buffer.release();
+ if (onRelease != null) {
+ onRelease.run();
+ }
+ }
+ }
+
+ @VisibleForTesting
+ int references() {
+ return references.get();
+ }
+ }
+
+ /**
+ * A fetch in progress, which later reads of the chunk join. Before it completes, the fetch takes a reference for
+ * each read that joined it, so that the chunk can't be released between the fetch's completion and the reads.
+ */
+ private static final class PendingFetch {
+ private final CompletableFuture<Chunk> future = new CompletableFuture<>();
+ // Guarded by this
+ private int waiters;
+ private boolean done;
+ private Chunk chunk;
+
+ /**
+ * @param waiters 1 for a fetch that a read starts, which is the fetch's first waiter from the start, so that
+ * it can't miss the chunk; 0 for a prefetch
+ */
+ PendingFetch(int waiters) {
+ this.waiters = waiters;
+ }
+ }
+
+ private final long maxSizeBytes;
+ private final int prefetchChunks;
+ private final Cache<Key, Chunk> chunks;
+ private final ConcurrentHashMap<Key, PendingFetch> pendingFetches = new ConcurrentHashMap<>();
+ private final ExecutorService prefetchExecutor;
+ private final int prefetchThreads;
+ // The prefetches queued or running, at most prefetchThreads
+ private final AtomicInteger prefetchesInProgress = new AtomicInteger();
+ private final ScheduledExecutorService cleanupExecutor;
+ // Guarded by this
+ private boolean closed;
+ private static final String CLOSED_MESSAGE = "The offloaded data chunk cache is closed";
+ private final LongAdder hits = new LongAdder();
+ private final LongAdder joinedFetches = new LongAdder();
+ private final LongAdder fetches = new LongAdder();
+ // The fetched chunks whose buffers haven't been released yet
+ private final LongAdder liveChunks = new LongAdder();
+
+ public OffloadedDataChunkCache(OffloadedReadCacheConfig config) {
+ this(config.cacheSizeBytes(), config.cacheExpirySeconds(), config.prefetchChunks(), config.prefetchThreads());
+ }
+
+ @VisibleForTesting
+ OffloadedDataChunkCache(long maxSizeBytes, int ttlSeconds, int prefetchChunks, int prefetchThreads) {
+ this.maxSizeBytes = Math.max(0, maxSizeBytes);
+ // Prefetched chunks are only kept by the cache
+ this.prefetchChunks = this.maxSizeBytes > 0 ? Math.max(0, prefetchChunks) : 0;
+ this.chunks = CacheBuilder.newBuilder()
+ // One segment, so that the budget and the least recently used order are the cache's, not each
+ // segment's; the cache is used once per chunk read, which keeps contention low
+ .concurrencyLevel(1)
+ .maximumWeight(this.maxSizeBytes)
+ .<Key, Chunk>weigher((key, chunk) -> key.length())
+ .expireAfterAccess(Math.max(1, ttlSeconds), TimeUnit.SECONDS)
+ .removalListener((RemovalListener<Key, Chunk>) notification -> notification.getValue().release())
+ .build();
+ this.prefetchThreads = Math.max(1, prefetchThreads);
+ this.prefetchExecutor = Executors.newFixedThreadPool(this.prefetchThreads,
+ new ThreadFactoryBuilder().setNameFormat("offloader-chunk-prefetch-%d").setDaemon(true).build());
+ this.cleanupExecutor = Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder().setNameFormat("offloader-chunk-cache-cleanup").setDaemon(true).build());
+ int period = Math.max(1, ttlSeconds / 2);
+ // Expired chunks are otherwise only removed, and their buffers released, by later cache operations
+ cleanupExecutor.scheduleAtFixedRate(chunks::cleanUp, period, period, TimeUnit.SECONDS);
+ }
+
+ /**
+ * Returns the chunk with a reference that the caller releases: from the cache, from a fetch of it in progress, or
+ * from a fetch that this call makes on the calling thread.
+ */
+ Chunk acquire(Key key, Fetcher fetcher) throws IOException {
+ for (;;) {
+ Chunk cached = chunks.getIfPresent(key);
+ if (cached != null && cached.tryRetain()) {
+ hits.increment();
+ return cached;
+ }
+ PendingFetch pending = pendingFetches.get(key);
+ if (pending == null) {
+ // A fetch that completed in between has put its chunk into the cache before it stopped being pending
+ cached = chunks.getIfPresent(key);
+ if (cached != null && cached.tryRetain()) {
+ hits.increment();
+ return cached;
+ }
+ PendingFetch created = new PendingFetch(1);
+ pending = pendingFetches.putIfAbsent(key, created);
+ if (pending == null) {
+ // This read is the fetch's first waiter already
+ fetchOrUseCached(key, fetcher, created);
+ return await(created);
+ }
+ }
+ joinedFetches.increment();
+ synchronized (pending) {
+ if (pending.done) {
+ if (pending.chunk != null && pending.chunk.tryRetain()) {
+ return pending.chunk;
+ }
+ if (pending.chunk != null) {
+ // Evicted since its fetch completed: fetch it again
+ continue;
+ }
+ } else {
+ pending.waiters++;
+ }
+ }
+ return await(pending);
Review Comment:
A read that joins an in-flight fetch inherits its failure. With the default prefetch of 1 chunk, a transient 503 on the background GET also fails a read that joined it. The dispatcher retries after `readFailureBackoff` (15 s by default), so it's a delay rather than a client error, but the read waits out the backoff for a fetch it didn't ask for.
Maybe we can retry the read's own request when the prefetch it joined fails.
##########
tiered-storage/jcloud/src/main/java/org/apache/bookkeeper/mledger/offload/jcloud/impl/OffloadedDataChunkCache.java:
##########
@@ -0,0 +1,422 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.bookkeeper.mledger.offload.jcloud.impl;
+
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.cache.Cache;
+import com.google.common.cache.CacheBuilder;
+import com.google.common.cache.RemovalListener;
+import com.google.common.util.concurrent.ThreadFactoryBuilder;
+import io.netty.buffer.ByteBuf;
+import java.io.IOException;
+import java.io.InterruptedIOException;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.RejectedExecutionException;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.LongAdder;
+import lombok.CustomLog;
+import org.apache.bookkeeper.mledger.OffloadedReadCacheConfig;
+
+/**
+ * The ranges of offloaded data objects that the read handles have fetched from the blob store, shared by every read
+ * handle of the offloader factory: chunks of an object, aligned to the read buffer size, kept within a size budget and
+ * dropped when no read has used them for a while. Cursors that read a ledger at different positions each keep their
+ * chunk instead of replacing each other's, and a read of a chunk that is being fetched waits for that fetch instead of
+ * fetching it again. A read fetches the chunk that it needs itself, on its thread. A reader that moves on to the next
+ * chunk has the chunks after it fetched in the background, on the prefetch threads, at most as many at a time as there
+ * are threads, so that a prefetch never waits in a queue.
+ *
+ * <p>Chunks are reference counted: the cache, the fetch that is completing it, and each reader using it hold a
+ * reference, and the chunk's buffer is released when the last one is released.
+ */
+@CustomLog
+public class OffloadedDataChunkCache implements AutoCloseable {
+ /** A chunk of a data object: the bytes from {@code start}, {@code length} of them. */
+ record Key(String bucket, String key, long start, int length) {
+ }
+
+ /** Fetches a chunk's bytes from the blob store into a buffer that the caller then owns. */
+ @FunctionalInterface
+ interface Fetcher {
+ ByteBuf fetch(Key key) throws IOException;
+ }
+
+ /** A fetched chunk; its buffer is released when the last reference is released. */
+ static final class Chunk {
+ private final ByteBuf buffer;
+ private final AtomicInteger references = new AtomicInteger(1);
+ // Runs when the buffer is released; null for none
+ private final Runnable onRelease;
+
+ Chunk(ByteBuf buffer) {
+ this(buffer, null);
+ }
+
+ Chunk(ByteBuf buffer, Runnable onRelease) {
+ this.buffer = buffer;
+ this.onRelease = onRelease;
+ }
+
+ ByteBuf buffer() {
+ return buffer;
+ }
+
+ /** Takes a reference unless the chunk has already been released. */
+ boolean tryRetain() {
+ for (;;) {
+ int current = references.get();
+ if (current == 0) {
+ return false;
+ }
+ if (references.compareAndSet(current, current + 1)) {
+ return true;
+ }
+ }
+ }
+
+ void retain(int count) {
+ if (count > 0) {
+ references.addAndGet(count);
+ }
+ }
+
+ void release() {
+ if (references.decrementAndGet() == 0) {
+ buffer.release();
+ if (onRelease != null) {
+ onRelease.run();
+ }
+ }
+ }
+
+ @VisibleForTesting
+ int references() {
+ return references.get();
+ }
+ }
+
+ /**
+ * A fetch in progress, which later reads of the chunk join. Before it completes, the fetch takes a reference for
+ * each read that joined it, so that the chunk can't be released between the fetch's completion and the reads.
+ */
+ private static final class PendingFetch {
+ private final CompletableFuture<Chunk> future = new CompletableFuture<>();
+ // Guarded by this
+ private int waiters;
+ private boolean done;
+ private Chunk chunk;
+
+ /**
+ * @param waiters 1 for a fetch that a read starts, which is the fetch's first waiter from the start, so that
+ * it can't miss the chunk; 0 for a prefetch
+ */
+ PendingFetch(int waiters) {
+ this.waiters = waiters;
+ }
+ }
+
+ private final long maxSizeBytes;
+ private final int prefetchChunks;
+ private final Cache<Key, Chunk> chunks;
+ private final ConcurrentHashMap<Key, PendingFetch> pendingFetches = new ConcurrentHashMap<>();
+ private final ExecutorService prefetchExecutor;
+ private final int prefetchThreads;
+ // The prefetches queued or running, at most prefetchThreads
+ private final AtomicInteger prefetchesInProgress = new AtomicInteger();
+ private final ScheduledExecutorService cleanupExecutor;
+ // Guarded by this
+ private boolean closed;
+ private static final String CLOSED_MESSAGE = "The offloaded data chunk cache is closed";
+ private final LongAdder hits = new LongAdder();
+ private final LongAdder joinedFetches = new LongAdder();
+ private final LongAdder fetches = new LongAdder();
+ // The fetched chunks whose buffers haven't been released yet
+ private final LongAdder liveChunks = new LongAdder();
+
+ public OffloadedDataChunkCache(OffloadedReadCacheConfig config) {
+ this(config.cacheSizeBytes(), config.cacheExpirySeconds(), config.prefetchChunks(), config.prefetchThreads());
+ }
+
+ @VisibleForTesting
+ OffloadedDataChunkCache(long maxSizeBytes, int ttlSeconds, int prefetchChunks, int prefetchThreads) {
+ this.maxSizeBytes = Math.max(0, maxSizeBytes);
+ // Prefetched chunks are only kept by the cache
+ this.prefetchChunks = this.maxSizeBytes > 0 ? Math.max(0, prefetchChunks) : 0;
+ this.chunks = CacheBuilder.newBuilder()
+ // One segment, so that the budget and the least recently used order are the cache's, not each
+ // segment's; the cache is used once per chunk read, which keeps contention low
+ .concurrencyLevel(1)
+ .maximumWeight(this.maxSizeBytes)
+ .<Key, Chunk>weigher((key, chunk) -> key.length())
+ .expireAfterAccess(Math.max(1, ttlSeconds), TimeUnit.SECONDS)
+ .removalListener((RemovalListener<Key, Chunk>) notification -> notification.getValue().release())
+ .build();
+ this.prefetchThreads = Math.max(1, prefetchThreads);
+ this.prefetchExecutor = Executors.newFixedThreadPool(this.prefetchThreads,
+ new ThreadFactoryBuilder().setNameFormat("offloader-chunk-prefetch-%d").setDaemon(true).build());
+ this.cleanupExecutor = Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder().setNameFormat("offloader-chunk-cache-cleanup").setDaemon(true).build());
+ int period = Math.max(1, ttlSeconds / 2);
+ // Expired chunks are otherwise only removed, and their buffers released, by later cache operations
+ cleanupExecutor.scheduleAtFixedRate(chunks::cleanUp, period, period, TimeUnit.SECONDS);
+ }
+
+ /**
+ * Returns the chunk with a reference that the caller releases: from the cache, from a fetch of it in progress, or
+ * from a fetch that this call makes on the calling thread.
+ */
+ Chunk acquire(Key key, Fetcher fetcher) throws IOException {
+ for (;;) {
+ Chunk cached = chunks.getIfPresent(key);
+ if (cached != null && cached.tryRetain()) {
+ hits.increment();
+ return cached;
+ }
+ PendingFetch pending = pendingFetches.get(key);
+ if (pending == null) {
+ // A fetch that completed in between has put its chunk into the cache before it stopped being pending
+ cached = chunks.getIfPresent(key);
+ if (cached != null && cached.tryRetain()) {
+ hits.increment();
+ return cached;
+ }
+ PendingFetch created = new PendingFetch(1);
+ pending = pendingFetches.putIfAbsent(key, created);
+ if (pending == null) {
+ // This read is the fetch's first waiter already
+ fetchOrUseCached(key, fetcher, created);
+ return await(created);
+ }
+ }
+ joinedFetches.increment();
+ synchronized (pending) {
+ if (pending.done) {
+ if (pending.chunk != null && pending.chunk.tryRetain()) {
+ return pending.chunk;
+ }
+ if (pending.chunk != null) {
+ // Evicted since its fetch completed: fetch it again
+ continue;
+ }
+ } else {
+ pending.waiters++;
+ }
+ }
+ return await(pending);
+ }
+ }
+
+ /**
+ * Fetches the chunk in the background unless it is cached or being fetched, or as many prefetches as there are
+ * prefetch threads are in progress.
+ */
+ void prefetch(Key key, Fetcher fetcher) {
+ // A chunk larger than the budget would be dropped at once, before a read uses it
+ if (prefetchChunks == 0 || key.length() > maxSizeBytes || chunks.getIfPresent(key) != null
+ || pendingFetches.containsKey(key)) {
+ return;
+ }
+ // Queued and running together, so that a prefetch, which a read may join, never waits in a queue
+ if (prefetchesInProgress.incrementAndGet() > prefetchThreads) {
+ prefetchesInProgress.decrementAndGet();
+ return;
+ }
+ PendingFetch created = new PendingFetch(0);
+ if (pendingFetches.putIfAbsent(key, created) != null) {
+ prefetchesInProgress.decrementAndGet();
+ return;
+ }
+ try {
+ prefetchExecutor.execute(() -> {
Review Comment:
`execute()` only handles `RejectedExecutionException`. If it throws anything else (e.g. `OutOfMemoryError` from `Thread.start()`), the task never runs, but the pending entry and the prefetch permit are left behind. A read that joins that entry then blocks in `future.get()` with no timeout, occupying an `offloader-read` thread until the cache is closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org