You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "TanYuxin-tyx (via GitHub)" <gi...@apache.org> on 2023/04/05 04:49:02 UTC
[GitHub] [flink] TanYuxin-tyx opened a new pull request, #22352: [FLINK-31639][network] Introduce tiered store memory manager
TanYuxin-tyx opened a new pull request, #22352:
URL: https://github.com/apache/flink/pull/22352
<!--
*Thank you very much for contributing to Apache Flink - we are happy that you want to help us improve Flink. To help the community review your contribution in the best possible way, please go through the checklist below, which will get the contribution into a shape in which it can be best reviewed.*
*Please understand that we do not do this to make contributions to Flink a hassle. In order to uphold a high standard of quality for code contributions, while at the same time managing a large number of contributions, we need contributors to prepare the contributions well, and give reviewers enough contextual information for the review. Please also understand that contributions that do not follow this guide will take longer to review and thus typically be picked up with lower priority by the community.*
## Contribution Checklist
- Make sure that the pull request corresponds to a [JIRA issue](https://issues.apache.org/jira/projects/FLINK/issues). Exceptions are made for typos in JavaDoc or documentation files, which need no JIRA issue.
- Name the pull request in the form "[FLINK-XXXX] [component] Title of the pull request", where *FLINK-XXXX* should be replaced by the actual issue number. Skip *component* if you are unsure about which is the best component.
Typo fixes that have no associated JIRA issue should be named following this pattern: `[hotfix] [docs] Fix typo in event time introduction` or `[hotfix] [javadocs] Expand JavaDoc for PuncuatedWatermarkGenerator`.
- Fill out the template below to describe the changes contributed by the pull request. That will give reviewers the context they need to do the review.
- Make sure that the change passes the automated tests, i.e., `mvn clean verify` passes. You can set up Azure Pipelines CI to do that following [this guide](https://cwiki.apache.org/confluence/display/FLINK/Azure+Pipelines#AzurePipelines-Tutorial:SettingupAzurePipelinesforaforkoftheFlinkrepository).
- Each pull request should address only one issue, not mix up code from multiple issues.
- Each commit in the pull request has a meaningful commit message (including the JIRA id)
- Once all items of the checklist are addressed, remove the above text and this checklist, leaving only the filled out template below.
**(The sections below can be removed for hotfixes of typos)**
-->
## What is the purpose of the change
*We should introduce a new manager to manage the memory of the tiered store. The change introduces `TieredStoreMemoryManager` and `CacheFlushManager` and implements the upstream tiered store memory manager.*
## Brief change log
- *Introduce the API of `TieredStoreMemoryManager` and `CacheFlushManager`.*
- *Implement the cache flush manager.*
- *Implement the upstream tiered store memory manager.*
## Verifying this change
This change added tests:
- *Added tests `UpstreamTieredStoreMemoryManagerTest` and `CacheFlushManagerTest`.*
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yno)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not documented)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193821258
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManager.java:
##########
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+
+import java.util.List;
+
+/**
+ * The {@link TieredStorageMemoryManager} is to request or recycle buffers from {@link
+ * LocalBufferPool} for different memory owners, for example, the tiers, the buffer accumulator,
+ * etc. Note that the logic for requesting and recycling buffers is consistent for these owners.
+ *
+ * <p>The memory managed by {@link TieredStorageMemoryManager} is categorized into two types:
+ * long-term occupied memory which cannot be immediately released and short-term occupied memory
+ * which can be reclaimed quickly and safely. Long-term occupied memory usage necessitates waiting
+ * for other operations to complete before releasing it, such as downstream consumption. On the
+ * other hand, short-term occupied memory can be freed up at any time, enabling rapid memory
+ * recycling for tasks such as flushing memory to disk or remote storage.
+ *
+ * <p>This {@link TieredStorageMemoryManager} aim to streamline and harmonize memory management
+ * across various layers. Instead of tracking the number of buffers utilized by individual users, it
+ * dynamically calculates a user's maximum guaranteed amount based on the current status of the
+ * manager and the local buffer pool. Specifically, if a user is a long-term occupied memory user,
+ * the {@link TieredStorageMemoryManager} does not limit the user's memory usage, while if a user is
+ * a short-term occupied memory user, the current guaranteed buffers of the user is the left buffers
+ * in the buffer pool - guaranteed amount of other users (excluding the current user).
Review Comment:
I rewrite the doc to clarify the non-reclaimable vs reclaimable buffers. And remove the sentence from "Specifically..."
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManager.java:
##########
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+
+import java.util.List;
+
+/**
+ * The {@link TieredStorageMemoryManager} is to request or recycle buffers from {@link
+ * LocalBufferPool} for different memory owners, for example, the tiers, the buffer accumulator,
+ * etc. Note that the logic for requesting and recycling buffers is consistent for these owners.
+ *
+ * <p>The memory managed by {@link TieredStorageMemoryManager} is categorized into two types:
+ * long-term occupied memory which cannot be immediately released and short-term occupied memory
+ * which can be reclaimed quickly and safely. Long-term occupied memory usage necessitates waiting
+ * for other operations to complete before releasing it, such as downstream consumption. On the
+ * other hand, short-term occupied memory can be freed up at any time, enabling rapid memory
+ * recycling for tasks such as flushing memory to disk or remote storage.
+ *
+ * <p>This {@link TieredStorageMemoryManager} aim to streamline and harmonize memory management
+ * across various layers. Instead of tracking the number of buffers utilized by individual users, it
+ * dynamically calculates a user's maximum guaranteed amount based on the current status of the
+ * manager and the local buffer pool. Specifically, if a user is a long-term occupied memory user,
+ * the {@link TieredStorageMemoryManager} does not limit the user's memory usage, while if a user is
+ * a short-term occupied memory user, the current guaranteed buffers of the user is the left buffers
+ * in the buffer pool - guaranteed amount of other users (excluding the current user).
+ */
+public interface TieredStorageMemoryManager {
+
+ /**
+ * Setup the {@link TieredStorageMemoryManager}. When setting up the manager, the {@link
+ * TieredStorageMemorySpec}s for different tiered storages should be ready to indicate each
+ * tiered storage's memory requirement specs.
+ *
+ * @param bufferPool the local buffer pool
+ * @param storageMemorySpecs the memory specs for different tiered storages
+ */
+ void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs);
+
+ /**
+ * Register a buffer flush call back to flush and recycle all the memory in the user when
+ * needed.
+ *
+ * <p>When the left buffers in the {@link BufferPool} are not enough, {@link
+ * TieredStorageMemoryManager} will flush the buffers of the user with the registered callback.
+ *
+ * @param userBufferFlushCallback the buffer flush call back of the memory user
+ */
+ void registerBufferFlushCallback(Runnable userBufferFlushCallback);
Review Comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on PR #22352:
URL: https://github.com/apache/flink/pull/22352#issuecomment-1524658552
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1188316799
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManager.java:
##########
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+
+/**
+ * The {@link TieredStorageMemoryManager} is to request or recycle buffer from {@link
+ * LocalBufferPool} for different tiers or {@link BufferAccumulator}.
+ *
+ * <p>The buffers of {@link BufferAccumulator} is also managed in the {@link
+ * TieredStorageMemoryManager}. When {@link BufferAccumulator} requests or recycles buffers, it uses
+ * separate methods that are different from tiers.
+ */
+public interface TieredStorageMemoryManager {
+
+ /** Setup the {@link TieredStorageMemoryManager}. */
+ void setup(BufferPool bufferPool);
+
+ /**
+ * Request a {@link MemorySegment} instance from {@link LocalBufferPool} for a specific tier.
+ *
+ * @param tierIndex the tier index
+ * @return the requested buffer
+ */
+ MemorySegment requestBufferBlocking(int tierIndex);
+
+ /**
+ * Recycle a {@link MemorySegment} buffer to {@link LocalBufferPool} for a specific tier.
+ *
+ * @param memorySegment the buffer to be recycled
+ * @param tierIndex the specific tier index
+ */
+ void recycleBuffer(MemorySegment memorySegment, int tierIndex);
+
+ /** Request a {@link MemorySegment} buffer for the buffer accumulator. */
+ MemorySegment requestBufferInAccumulator();
+
+ /** Recycle a {@link MemorySegment} buffer for the buffer accumulator. */
+ void recycleBufferInAccumulator(MemorySegment memorySegment);
Review Comment:
I have updated the code. I unified the logic of requesting and recycling buffers of any memory consumer owners.
Removed the different interfaces for tiers and the accumulator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1188320931
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/tier/TierMemorySpec.java:
##########
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.tier;
+
+/**
+ * The memory specs for a tier, including the tier index, the number of exclusive buffers of the
+ * tier, whether the tier can use the shared buffers in the memory manager, etc.
+ */
+public class TierMemorySpec {
+
+ private final int tierIndex;
+
+ private final int numExclusiveBuffers;
+
+ private final boolean canUseSharedBuffers;
+
+ public TierMemorySpec(int tierIndex, int numExclusiveBuffers, boolean canUseSharedBuffers) {
+ this.tierIndex = tierIndex;
+ this.numExclusiveBuffers = numExclusiveBuffers;
+ this.canUseSharedBuffers = canUseSharedBuffers;
+ }
+
+ public int getTierIndex() {
+ return tierIndex;
+ }
+
+ public int getNumExclusiveBuffers() {
+ return numExclusiveBuffers;
+ }
+
+ public boolean canUseShareBuffers() {
+ return canUseSharedBuffers;
Review Comment:
Rename and refactor it to `TieredStorageMemorySpec`.
And `canUseSharedBuffers is renamed` to `isMemoryReleasable`.
Some tiers(for example, the memory tier) should depend on the downstream consumption to release the memory. For these tiers, the `isMemoryReleasable` is `false`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193122499
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/CacheFlushManager.java:
##########
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.runtime.io.network.partition.hybrid.tiered.common.TieredStorageUtils;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+
+/**
+ * A manager to check whether the cached buffers need to be flushed.
+ *
+ * <p>Cached buffers refer to the buffers stored in a tier that can be released by flushing them to
+ * the disk or remote storage. If the requested buffers from the buffer pool reach the ratio limit,
+ * {@link CacheFlushManager} will trigger a process wherein all these buffers are flushed to the
+ * disk or remote storage to recycling these buffers.
+ *
+ * <p>Note that when initializing a tier with cached buffers, the tier should register a {@link
+ * CacheBufferFlushTrigger}, as a listener to flush cached buffers.
+ */
+public class CacheFlushManager {
+ private final float numBuffersTriggerFlushRatio;
+
+ private final List<CacheBufferFlushTrigger> flushTriggers;
+
+ private TieredStorageMemoryManager storageMemoryManager;
+
+ private final ScheduledExecutorService executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("cache flush trigger")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+
+ public CacheFlushManager(float numBuffersTriggerFlushRatio) {
+ this.numBuffersTriggerFlushRatio = numBuffersTriggerFlushRatio;
+ this.flushTriggers = new ArrayList<>();
+
+ executor.scheduleWithFixedDelay(
+ this::checkNeedTriggerFlushCachedBuffers, 10, 50, TimeUnit.MILLISECONDS);
+ }
+
+ public void setup(TieredStorageMemoryManager storageMemoryManager) {
+ this.storageMemoryManager = storageMemoryManager;
+ }
+
+ public void registerCacheBufferFlushTrigger(CacheBufferFlushTrigger cacheBufferFlushTrigger) {
+ flushTriggers.add(cacheBufferFlushTrigger);
+ }
+
+ public void triggerFlushCachedBuffers() {
Review Comment:
Fixed it, and I renamed it to `checkShouldFlushCachedBuffers`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1188318141
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManager.java:
##########
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+
+/**
+ * The {@link TieredStorageMemoryManager} is to request or recycle buffer from {@link
+ * LocalBufferPool} for different tiers or {@link BufferAccumulator}.
+ *
+ * <p>The buffers of {@link BufferAccumulator} is also managed in the {@link
+ * TieredStorageMemoryManager}. When {@link BufferAccumulator} requests or recycles buffers, it uses
+ * separate methods that are different from tiers.
+ */
+public interface TieredStorageMemoryManager {
+
+ /** Setup the {@link TieredStorageMemoryManager}. */
+ void setup(BufferPool bufferPool);
+
+ /**
+ * Request a {@link MemorySegment} instance from {@link LocalBufferPool} for a specific tier.
+ *
+ * @param tierIndex the tier index
+ * @return the requested buffer
+ */
+ MemorySegment requestBufferBlocking(int tierIndex);
+
+ /**
+ * Recycle a {@link MemorySegment} buffer to {@link LocalBufferPool} for a specific tier.
+ *
+ * @param memorySegment the buffer to be recycled
+ * @param tierIndex the specific tier index
+ */
+ void recycleBuffer(MemorySegment memorySegment, int tierIndex);
+
+ /** Request a {@link MemorySegment} buffer for the buffer accumulator. */
+ MemorySegment requestBufferInAccumulator();
+
+ /** Recycle a {@link MemorySegment} buffer for the buffer accumulator. */
+ void recycleBufferInAccumulator(MemorySegment memorySegment);
+
+ /**
+ * Increase the number of requested buffers for a tier.
+ *
+ * @param tierIndex the specific tier index
+ */
+ void incNumRequestedBuffer(int tierIndex);
+
+ /**
+ * Decrease the number of requested buffers for a tier.
+ *
+ * @param tierIndex the specific tier index
+ */
+ void decNumRequestedBuffer(int tierIndex);
+
+ /** Increase the number of requested buffers for buffer accumulator. */
+ void incNumRequestedBufferInAccumulator();
+
+ /** Decrease the number of requested buffers for buffer accumulator. */
+ void decNumRequestedBufferInAccumulator();
Review Comment:
Removed them. Only `numAvailableBuffers`, `numTotalBuffers`, `numRequestedBuffers` are left.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193832953
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,224 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking flush. */
+ public static final int DEFAULT_CHECK_FLUSH_INITIAL_DELAY_MS = 10;
+
+ /** Check flush period. */
+ public static final int DEFAULT_CHECK_FLUSH_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** The registered callbacks to flush the buffers in the registered tiered storages. */
+ private final List<Runnable> bufferFlushCallbacks;
+
+ /** The buffer pool usage ratio of triggering the registered storages to flush buffers. */
+ private final float numBuffersTriggerFlushRatio;
+
+ /**
+ * Indicate whether to start the buffer flush checker thread. If the memory manager is used in
+ * downstream, the field will be false because no buffer flush checker is needed.
+ */
+ private final boolean shouldStartBufferFlushChecker;
+
+ /** The number of requested buffers from {@link BufferPool}. */
+ private final AtomicInteger numRequestedBuffers;
+
+ /** A thread to check whether to flush buffers in each tiered storage. */
+ private ScheduledExecutorService executor;
+
+ /** The total number of guaranteed buffers for all tiered storages. */
+ private int numTotalGuaranteedBuffers;
+
+ /** The buffer pool where the buffer is requested or recyceld. */
+ private BufferPool bufferPool;
+
+ /**
+ * Indicate whether the {@link TieredStorageMemoryManagerImpl} is in running state. Before
+ * setting up, this field is false.
+ *
+ * <p>Note that before requesting buffers or getting the maximum allowed buffers, this running
+ * state should be checked.
+ */
+ private boolean isRunning;
+
+ /**
+ * The constructor of the {@link TieredStorageMemoryManagerImpl}.
+ *
+ * @param numBuffersTriggerFlushRatio the buffer pool usage ratio of triggering each tiered
+ * storage to flush buffers
+ * @param shouldStartBufferFlushChecker indicate whether to start the buffer flushing checker
+ * thread
+ */
+ public TieredStorageMemoryManagerImpl(
+ float numBuffersTriggerFlushRatio, boolean shouldStartBufferFlushChecker) {
+ this.numBuffersTriggerFlushRatio = numBuffersTriggerFlushRatio;
+ this.shouldStartBufferFlushChecker = shouldStartBufferFlushChecker;
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ this.bufferFlushCallbacks = new ArrayList<>();
+ this.isRunning = false;
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs) {
+ this.bufferPool = bufferPool;
+ for (TieredStorageMemorySpec memorySpec : storageMemorySpecs) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalGuaranteedBuffers += memorySpec.getNumGuaranteedBuffers();
+ }
+
+ if (shouldStartBufferFlushChecker) {
+ this.executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("buffer flush checker")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+ this.executor.scheduleWithFixedDelay(
+ this::checkShouldFlushCachedBuffers,
+ DEFAULT_CHECK_FLUSH_INITIAL_DELAY_MS,
+ DEFAULT_CHECK_FLUSH_PERIOD_DURATION_MS,
+ TimeUnit.MILLISECONDS);
+ }
+
+ this.isRunning = true;
+ }
+
+ @Override
+ public void registerBufferFlushCallback(Runnable userBufferFlushCallBack) {
+ bufferFlushCallbacks.add(userBufferFlushCallBack);
+ }
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link BufferPool} for a specific owner. The
+ * {@link TieredStorageMemoryManagerImpl} will not check whether a buffer can be requested and
+ * only record the total number of requested buffers. If the buffers in the {@link BufferPool}
+ * is not enough, this will trigger each tiered storage to flush buffers as much as possible.
+ *
+ * @return the requested buffer
+ */
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ checkIsRunning();
+
+ MemorySegment requestedBuffer = null;
+ try {
+ requestedBuffer = bufferPool.requestMemorySegmentBlocking();
+ } catch (Throwable throwable) {
+ ExceptionUtils.rethrow(throwable, "Failed to request memory segments.");
+ }
+ numRequestedBuffers.incrementAndGet();
+ checkShouldFlushCachedBuffers();
Review Comment:
Move this before requesting. But that may not make sure the buffer reclaim occurs, so we need a checking thread.
Note that it is **not** possible to remove this, as doing so could result in the task becoming stuck in the buffer request. As the number of buffers in the buffer pool can vary at any given time, the stuck may occur if the thread is removed.
For instance, if the memory usage of the {@link BufferPool} has been checked and {@link TieredStorageMemoryManagerImpl} determined that buffer reclamation is unnecessary, but then the buffer pool size is suddenly reduced to a very small size, the buffer request will become stuck and the task will never be able to call for buffer reclamation if this thread is removed, then a task stuck occurs.
Java doc also added these descriptions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1200334551
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,279 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_INITIAL_DELAY_MS = 10;
+
+ /** The period of checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** Listeners used to listen the requests for reclaiming buffer in different tiered storage. */
+ private final List<Runnable> bufferReclaimRequestListeners;
+
+ /** The buffer pool usage ratio of triggering the registered storages to reclaim buffers. */
+ private final float numTriggerReclaimBuffersRatio;
+
+ /**
+ * Indicate whether it is necessary to start a periodically checking buffer reclaim thread. If
+ * the memory manager is used in downstream, the field will be false because periodical buffer
+ * reclaim checker is needed.
+ */
+ private final boolean needPeriodicalCheckReclaimBuffer;
+
+ /**
+ * The number of requested buffers from {@link BufferPool}. Only this field can be touched both
+ * by the task thread and the netty thread, so it is an atomic type.
+ */
+ private final AtomicInteger numRequestedBuffers;
+
+ /**
+ * A thread to check whether to reclaim buffers from each tiered storage.
+ *
+ * <p>Note that it is not possible to remove this, as doing so could result in the task becoming
+ * stuck in the buffer request. As the number of buffers in the buffer pool can vary at any
+ * given time, the stuck may occur if the thread is removed.
+ *
+ * <p>For instance, if the memory usage of the {@link BufferPool} has been checked and {@link
+ * TieredStorageMemoryManagerImpl} determined that buffer reclamation is unnecessary, but then
+ * the buffer pool size is suddenly reduced to a very small size, the buffer request will become
+ * stuck and the task will never be able to call for buffer reclamation if this thread is
+ * removed, then a task stuck occurs.
+ */
+ private ScheduledExecutorService executor;
+
+ /** The total number of guaranteed buffers for all tiered storages. */
+ private int numTotalGuaranteedBuffers;
+
+ /** The buffer pool where the buffer is requested or recycled. */
+ private BufferPool bufferPool;
+
+ /**
+ * Indicate whether the {@link TieredStorageMemoryManagerImpl} is initialized. Before setting
+ * up, this field is false.
+ *
+ * <p>Note that before requesting buffers or getting the maximum allowed buffers, this
+ * initialized state should be checked.
+ */
+ private boolean isInitialized;
+
+ /**
+ * The constructor of the {@link TieredStorageMemoryManagerImpl}.
+ *
+ * @param numTriggerReclaimBuffersRatio the buffer pool usage ratio of requesting each tiered
+ * storage to reclaim buffers
+ * @param needPeriodicalCheckReclaimBuffer indicate whether it is necessary to start a
+ * periodically checking buffer reclaim thread
+ */
+ public TieredStorageMemoryManagerImpl(
+ float numTriggerReclaimBuffersRatio, boolean needPeriodicalCheckReclaimBuffer) {
+ this.numTriggerReclaimBuffersRatio = numTriggerReclaimBuffersRatio;
+ this.needPeriodicalCheckReclaimBuffer = needPeriodicalCheckReclaimBuffer;
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ this.bufferReclaimRequestListeners = new ArrayList<>();
+ this.isInitialized = false;
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs) {
+ this.bufferPool = bufferPool;
+ for (TieredStorageMemorySpec memorySpec : storageMemorySpecs) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalGuaranteedBuffers += memorySpec.getNumGuaranteedBuffers();
+ }
+
+ if (needPeriodicalCheckReclaimBuffer) {
+ this.executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("buffer reclaim checker")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+ }
+
+ this.isInitialized = true;
+ }
+
+ @Override
+ public void listenBufferReclaimRequest(Runnable onBufferReclaimRequest) {
+ bufferReclaimRequestListeners.add(onBufferReclaimRequest);
+ }
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link BufferPool} for a specific owner. The
+ * {@link TieredStorageMemoryManagerImpl} will not check whether a buffer can be requested and
+ * only record the total number of requested buffers. If the buffers in the {@link BufferPool}
+ * is not enough, this will request each tiered storage to reclaim the buffers as much as
+ * possible.
+ *
+ * <p>Note that synchronization is not necessary for this method, as it can only be called by
+ * the task thread.
+ *
+ * @return the requested buffer
+ */
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ checkIsInitialized();
+
+ tryReclaimBuffers();
+ CompletableFuture<MemorySegment> requestedBuffer =
+ CompletableFuture.supplyAsync(
+ () -> {
+ try {
+ return bufferPool.requestMemorySegmentBlocking();
+ } catch (InterruptedException e) {
+ throw new RuntimeException(e);
+ }
+ });
+
+ ScheduledFuture<?> scheduledFuture = getPeriodicalCheckReclaimBufferFuture();
+
+ MemorySegment memorySegment = null;
+ try {
+ memorySegment = requestedBuffer.get();
+ } catch (InterruptedException | ExecutionException e) {
+ ExceptionUtils.rethrow(e);
+ }
+
+ if (scheduledFuture != null) {
+ scheduledFuture.cancel(false);
Review Comment:
Fixed by a `CompletableFuture<Void>`. And these two lines are removed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] xintongsong closed pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "xintongsong (via GitHub)" <gi...@apache.org>.
xintongsong closed pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
URL: https://github.com/apache/flink/pull/22352
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193834438
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,224 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking flush. */
+ public static final int DEFAULT_CHECK_FLUSH_INITIAL_DELAY_MS = 10;
+
+ /** Check flush period. */
+ public static final int DEFAULT_CHECK_FLUSH_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** The registered callbacks to flush the buffers in the registered tiered storages. */
+ private final List<Runnable> bufferFlushCallbacks;
+
+ /** The buffer pool usage ratio of triggering the registered storages to flush buffers. */
+ private final float numBuffersTriggerFlushRatio;
+
+ /**
+ * Indicate whether to start the buffer flush checker thread. If the memory manager is used in
+ * downstream, the field will be false because no buffer flush checker is needed.
+ */
+ private final boolean shouldStartBufferFlushChecker;
+
+ /** The number of requested buffers from {@link BufferPool}. */
+ private final AtomicInteger numRequestedBuffers;
+
+ /** A thread to check whether to flush buffers in each tiered storage. */
+ private ScheduledExecutorService executor;
+
+ /** The total number of guaranteed buffers for all tiered storages. */
+ private int numTotalGuaranteedBuffers;
+
+ /** The buffer pool where the buffer is requested or recyceld. */
+ private BufferPool bufferPool;
+
+ /**
+ * Indicate whether the {@link TieredStorageMemoryManagerImpl} is in running state. Before
+ * setting up, this field is false.
+ *
+ * <p>Note that before requesting buffers or getting the maximum allowed buffers, this running
+ * state should be checked.
+ */
+ private boolean isRunning;
+
+ /**
+ * The constructor of the {@link TieredStorageMemoryManagerImpl}.
+ *
+ * @param numBuffersTriggerFlushRatio the buffer pool usage ratio of triggering each tiered
+ * storage to flush buffers
+ * @param shouldStartBufferFlushChecker indicate whether to start the buffer flushing checker
+ * thread
+ */
+ public TieredStorageMemoryManagerImpl(
+ float numBuffersTriggerFlushRatio, boolean shouldStartBufferFlushChecker) {
+ this.numBuffersTriggerFlushRatio = numBuffersTriggerFlushRatio;
+ this.shouldStartBufferFlushChecker = shouldStartBufferFlushChecker;
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ this.bufferFlushCallbacks = new ArrayList<>();
+ this.isRunning = false;
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs) {
+ this.bufferPool = bufferPool;
+ for (TieredStorageMemorySpec memorySpec : storageMemorySpecs) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalGuaranteedBuffers += memorySpec.getNumGuaranteedBuffers();
+ }
+
+ if (shouldStartBufferFlushChecker) {
+ this.executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("buffer flush checker")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+ this.executor.scheduleWithFixedDelay(
+ this::checkShouldFlushCachedBuffers,
+ DEFAULT_CHECK_FLUSH_INITIAL_DELAY_MS,
+ DEFAULT_CHECK_FLUSH_PERIOD_DURATION_MS,
+ TimeUnit.MILLISECONDS);
+ }
+
+ this.isRunning = true;
+ }
+
+ @Override
+ public void registerBufferFlushCallback(Runnable userBufferFlushCallBack) {
+ bufferFlushCallbacks.add(userBufferFlushCallBack);
+ }
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link BufferPool} for a specific owner. The
+ * {@link TieredStorageMemoryManagerImpl} will not check whether a buffer can be requested and
+ * only record the total number of requested buffers. If the buffers in the {@link BufferPool}
+ * is not enough, this will trigger each tiered storage to flush buffers as much as possible.
+ *
+ * @return the requested buffer
+ */
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ checkIsRunning();
+
+ MemorySegment requestedBuffer = null;
+ try {
+ requestedBuffer = bufferPool.requestMemorySegmentBlocking();
+ } catch (Throwable throwable) {
+ ExceptionUtils.rethrow(throwable, "Failed to request memory segments.");
+ }
+ numRequestedBuffers.incrementAndGet();
+ checkShouldFlushCachedBuffers();
+ return new BufferBuilder(checkNotNull(requestedBuffer), this::recycleBuffer);
+ }
+
+ @Override
+ public int getMaxAllowedBuffers(Object owner) {
+ checkIsRunning();
+
+ TieredStorageMemorySpec ownerMemorySpec = checkNotNull(tieredMemorySpecs.get(owner));
+ if (ownerMemorySpec.isMemoryReleasable()) {
+ return Integer.MAX_VALUE;
+ } else {
+ int ownerGuaranteedBuffers = ownerMemorySpec.getNumGuaranteedBuffers();
+ return bufferPool.getNumBuffers()
+ - numRequestedBuffers.get()
+ - numTotalGuaranteedBuffers
+ + ownerGuaranteedBuffers;
+ }
+ }
+
+ @Override
+ public void release() {
+ checkState(numRequestedBuffers.get() == 0, "Leaking buffers.");
+ if (executor != null) {
+ executor.shutdown();
+ try {
+ if (!executor.awaitTermination(5L, TimeUnit.MINUTES)) {
+ throw new TimeoutException(
+ "Timeout for shutting down the cache flush executor.");
+ }
+ } catch (Exception e) {
+ ExceptionUtils.rethrow(e);
+ }
+ }
+ }
+
+ private void checkShouldFlushCachedBuffers() {
+ if (shouldFlushBuffers()) {
+ bufferFlushCallbacks.forEach(Runnable::run);
+ }
+ }
+
+ private boolean shouldFlushBuffers() {
+ synchronized (this) {
+ int numTotal = bufferPool.getNumBuffers();
+ int numRequested = numRequestedBuffers.get();
+ return numRequested >= numTotal
+ || (numRequested * 1.0 / numTotal) >= numBuffersTriggerFlushRatio;
+ }
Review Comment:
FIxed.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,224 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking flush. */
+ public static final int DEFAULT_CHECK_FLUSH_INITIAL_DELAY_MS = 10;
+
+ /** Check flush period. */
+ public static final int DEFAULT_CHECK_FLUSH_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** The registered callbacks to flush the buffers in the registered tiered storages. */
+ private final List<Runnable> bufferFlushCallbacks;
+
+ /** The buffer pool usage ratio of triggering the registered storages to flush buffers. */
+ private final float numBuffersTriggerFlushRatio;
+
+ /**
+ * Indicate whether to start the buffer flush checker thread. If the memory manager is used in
+ * downstream, the field will be false because no buffer flush checker is needed.
+ */
+ private final boolean shouldStartBufferFlushChecker;
+
+ /** The number of requested buffers from {@link BufferPool}. */
+ private final AtomicInteger numRequestedBuffers;
+
+ /** A thread to check whether to flush buffers in each tiered storage. */
+ private ScheduledExecutorService executor;
+
+ /** The total number of guaranteed buffers for all tiered storages. */
+ private int numTotalGuaranteedBuffers;
+
+ /** The buffer pool where the buffer is requested or recyceld. */
+ private BufferPool bufferPool;
+
+ /**
+ * Indicate whether the {@link TieredStorageMemoryManagerImpl} is in running state. Before
+ * setting up, this field is false.
+ *
+ * <p>Note that before requesting buffers or getting the maximum allowed buffers, this running
+ * state should be checked.
+ */
+ private boolean isRunning;
+
+ /**
+ * The constructor of the {@link TieredStorageMemoryManagerImpl}.
+ *
+ * @param numBuffersTriggerFlushRatio the buffer pool usage ratio of triggering each tiered
+ * storage to flush buffers
+ * @param shouldStartBufferFlushChecker indicate whether to start the buffer flushing checker
+ * thread
+ */
+ public TieredStorageMemoryManagerImpl(
+ float numBuffersTriggerFlushRatio, boolean shouldStartBufferFlushChecker) {
+ this.numBuffersTriggerFlushRatio = numBuffersTriggerFlushRatio;
+ this.shouldStartBufferFlushChecker = shouldStartBufferFlushChecker;
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ this.bufferFlushCallbacks = new ArrayList<>();
+ this.isRunning = false;
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs) {
+ this.bufferPool = bufferPool;
+ for (TieredStorageMemorySpec memorySpec : storageMemorySpecs) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalGuaranteedBuffers += memorySpec.getNumGuaranteedBuffers();
+ }
+
+ if (shouldStartBufferFlushChecker) {
+ this.executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("buffer flush checker")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+ this.executor.scheduleWithFixedDelay(
+ this::checkShouldFlushCachedBuffers,
+ DEFAULT_CHECK_FLUSH_INITIAL_DELAY_MS,
+ DEFAULT_CHECK_FLUSH_PERIOD_DURATION_MS,
+ TimeUnit.MILLISECONDS);
+ }
+
+ this.isRunning = true;
+ }
+
+ @Override
+ public void registerBufferFlushCallback(Runnable userBufferFlushCallBack) {
+ bufferFlushCallbacks.add(userBufferFlushCallBack);
+ }
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link BufferPool} for a specific owner. The
+ * {@link TieredStorageMemoryManagerImpl} will not check whether a buffer can be requested and
+ * only record the total number of requested buffers. If the buffers in the {@link BufferPool}
+ * is not enough, this will trigger each tiered storage to flush buffers as much as possible.
+ *
+ * @return the requested buffer
+ */
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ checkIsRunning();
+
+ MemorySegment requestedBuffer = null;
+ try {
+ requestedBuffer = bufferPool.requestMemorySegmentBlocking();
+ } catch (Throwable throwable) {
+ ExceptionUtils.rethrow(throwable, "Failed to request memory segments.");
+ }
+ numRequestedBuffers.incrementAndGet();
+ checkShouldFlushCachedBuffers();
+ return new BufferBuilder(checkNotNull(requestedBuffer), this::recycleBuffer);
+ }
+
+ @Override
+ public int getMaxAllowedBuffers(Object owner) {
+ checkIsRunning();
+
+ TieredStorageMemorySpec ownerMemorySpec = checkNotNull(tieredMemorySpecs.get(owner));
+ if (ownerMemorySpec.isMemoryReleasable()) {
+ return Integer.MAX_VALUE;
+ } else {
Review Comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193834734
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,224 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking flush. */
+ public static final int DEFAULT_CHECK_FLUSH_INITIAL_DELAY_MS = 10;
+
+ /** Check flush period. */
+ public static final int DEFAULT_CHECK_FLUSH_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** The registered callbacks to flush the buffers in the registered tiered storages. */
+ private final List<Runnable> bufferFlushCallbacks;
+
+ /** The buffer pool usage ratio of triggering the registered storages to flush buffers. */
+ private final float numBuffersTriggerFlushRatio;
+
+ /**
+ * Indicate whether to start the buffer flush checker thread. If the memory manager is used in
+ * downstream, the field will be false because no buffer flush checker is needed.
+ */
+ private final boolean shouldStartBufferFlushChecker;
+
+ /** The number of requested buffers from {@link BufferPool}. */
+ private final AtomicInteger numRequestedBuffers;
+
+ /** A thread to check whether to flush buffers in each tiered storage. */
+ private ScheduledExecutorService executor;
+
+ /** The total number of guaranteed buffers for all tiered storages. */
+ private int numTotalGuaranteedBuffers;
+
+ /** The buffer pool where the buffer is requested or recyceld. */
+ private BufferPool bufferPool;
+
+ /**
+ * Indicate whether the {@link TieredStorageMemoryManagerImpl} is in running state. Before
+ * setting up, this field is false.
+ *
+ * <p>Note that before requesting buffers or getting the maximum allowed buffers, this running
+ * state should be checked.
+ */
+ private boolean isRunning;
Review Comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193122008
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemorySpec.java:
##########
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+/**
+ * The memory specs for a memory owner, including the owner itself, the number of exclusive buffers
+ * of the owner, whether the owner's memory can be released when not consumed, etc.
+ */
+public class TieredStorageMemorySpec {
+
+ private final Object owner;
+
+ private final int numExclusiveBuffers;
+
+ private final boolean isMemoryReleasable;
Review Comment:
Added a specific java doc to indicate the meaning of the field.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193121723
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ private int numTotalExclusiveBuffers;
+
+ private BufferPool bufferPool;
+
+ private final AtomicInteger numRequestedBuffers;
+
+ public TieredStorageMemoryManagerImpl() {
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool) {
+ this.bufferPool = bufferPool;
+ }
+
+ @Override
+ public void registerMemorySpec(TieredStorageMemorySpec memorySpec) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec registration.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalExclusiveBuffers += memorySpec.getNumExclusiveBuffers();
+ }
+
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ MemorySegment requestedBuffer = null;
+ try {
+ requestedBuffer = bufferPool.requestMemorySegmentBlocking();
+ } catch (Throwable throwable) {
+ ExceptionUtils.rethrow(throwable, "Failed to request memory segments.");
+ }
+ numRequestedBuffers.incrementAndGet();
+ return new BufferBuilder(checkNotNull(requestedBuffer), this::recycleBuffer);
+ }
Review Comment:
I have added the java docs to describe the overall responsibility and protocol.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1188332319
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/CacheFlushManager.java:
##########
@@ -0,0 +1,100 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.runtime.io.network.partition.hybrid.tiered.common.TieredStorageUtils;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+
+/**
+ * A manager to check whether the cached buffers need to be flushed.
+ *
+ * <p>Note that when initializing a tier with cached buffers, the tier should register a {@link
+ * CacheBufferFlushTrigger}, as a listener to flush cached buffers.
+ */
+public class CacheFlushManager {
Review Comment:
This class is to trigger to flush the cached buffers in the tiers(The `isMemoryReleasable` of the tiers is true).
In order to avoid blocking and waiting when requesting buffers from `LocalBufferPool`, we should trigger flushing (for example, to the disk or the remote storage) to release buffers when the use ratio reaches a certain ratio.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193821816
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManager.java:
##########
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+
+import java.util.List;
+
+/**
+ * The {@link TieredStorageMemoryManager} is to request or recycle buffers from {@link
+ * LocalBufferPool} for different memory owners, for example, the tiers, the buffer accumulator,
+ * etc. Note that the logic for requesting and recycling buffers is consistent for these owners.
+ *
+ * <p>The memory managed by {@link TieredStorageMemoryManager} is categorized into two types:
+ * long-term occupied memory which cannot be immediately released and short-term occupied memory
+ * which can be reclaimed quickly and safely. Long-term occupied memory usage necessitates waiting
+ * for other operations to complete before releasing it, such as downstream consumption. On the
+ * other hand, short-term occupied memory can be freed up at any time, enabling rapid memory
+ * recycling for tasks such as flushing memory to disk or remote storage.
+ *
+ * <p>This {@link TieredStorageMemoryManager} aim to streamline and harmonize memory management
+ * across various layers. Instead of tracking the number of buffers utilized by individual users, it
+ * dynamically calculates a user's maximum guaranteed amount based on the current status of the
+ * manager and the local buffer pool. Specifically, if a user is a long-term occupied memory user,
+ * the {@link TieredStorageMemoryManager} does not limit the user's memory usage, while if a user is
+ * a short-term occupied memory user, the current guaranteed buffers of the user is the left buffers
+ * in the buffer pool - guaranteed amount of other users (excluding the current user).
+ */
+public interface TieredStorageMemoryManager {
+
+ /**
+ * Setup the {@link TieredStorageMemoryManager}. When setting up the manager, the {@link
+ * TieredStorageMemorySpec}s for different tiered storages should be ready to indicate each
+ * tiered storage's memory requirement specs.
+ *
+ * @param bufferPool the local buffer pool
+ * @param storageMemorySpecs the memory specs for different tiered storages
+ */
+ void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs);
+
+ /**
+ * Register a buffer flush call back to flush and recycle all the memory in the user when
+ * needed.
+ *
+ * <p>When the left buffers in the {@link BufferPool} are not enough, {@link
+ * TieredStorageMemoryManager} will flush the buffers of the user with the registered callback.
+ *
+ * @param userBufferFlushCallback the buffer flush call back of the memory user
+ */
+ void registerBufferFlushCallback(Runnable userBufferFlushCallback);
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link LocalBufferPool} for a specific owner.
+ *
+ * @return the requested buffer
+ */
+ BufferBuilder requestBufferBlocking();
+
+ /**
+ * Return the available buffers for the owner.
+ *
+ * <p>Note that the available buffers are calculated dynamically based on some conditions, for
+ * example, the state of the {@link BufferPool}, the {@link TieredStorageMemorySpec} of the
+ * owner, etc.
+ *
+ * <p>If a user is a long-term occupied memory user, the {@link TieredStorageMemoryManager} does
+ * not limit the user's memory usage, while if a user is a short-term occupied memory user, the
+ * max allowed buffers of the user is the left buffers in the buffer pool - guaranteed amount of
+ * other users (excluding the current user).
+ */
+ int getMaxAllowedBuffers(Object owner);
Review Comment:
Fixed and rewrite the docs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193828115
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,224 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking flush. */
+ public static final int DEFAULT_CHECK_FLUSH_INITIAL_DELAY_MS = 10;
+
+ /** Check flush period. */
+ public static final int DEFAULT_CHECK_FLUSH_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** The registered callbacks to flush the buffers in the registered tiered storages. */
+ private final List<Runnable> bufferFlushCallbacks;
+
+ /** The buffer pool usage ratio of triggering the registered storages to flush buffers. */
+ private final float numBuffersTriggerFlushRatio;
+
+ /**
+ * Indicate whether to start the buffer flush checker thread. If the memory manager is used in
+ * downstream, the field will be false because no buffer flush checker is needed.
+ */
+ private final boolean shouldStartBufferFlushChecker;
Review Comment:
I believe the thread belongs to this class. But the thread name and the switch name are not that right. I renamed the `boolean` to `needPeriodicalCheckReclaimBuffer` and renamed `numTriggerReclaimBuffersRatio`.
The accuracy of the memory usage ratio may be compromised due to the varying buffer pool sizes. However, this only impacts a single iteration of the buffer usage check. Upon the next iteration, the buffer reclaim will eventually be triggered.
The Java doc also added the above reason descriptions of adding the thread.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on PR #22352:
URL: https://github.com/apache/flink/pull/22352#issuecomment-1524499360
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193118533
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ private int numTotalExclusiveBuffers;
+
+ private BufferPool bufferPool;
+
+ private final AtomicInteger numRequestedBuffers;
Review Comment:
`numTotalExclusiveBuffers` is initialized in the `setup` method and it will not be changed any more once initialized.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193122680
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/CacheFlushManager.java:
##########
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.runtime.io.network.partition.hybrid.tiered.common.TieredStorageUtils;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+
+/**
+ * A manager to check whether the cached buffers need to be flushed.
+ *
+ * <p>Cached buffers refer to the buffers stored in a tier that can be released by flushing them to
+ * the disk or remote storage. If the requested buffers from the buffer pool reach the ratio limit,
+ * {@link CacheFlushManager} will trigger a process wherein all these buffers are flushed to the
+ * disk or remote storage to recycling these buffers.
+ *
+ * <p>Note that when initializing a tier with cached buffers, the tier should register a {@link
+ * CacheBufferFlushTrigger}, as a listener to flush cached buffers.
+ */
+public class CacheFlushManager {
+ private final float numBuffersTriggerFlushRatio;
+
+ private final List<CacheBufferFlushTrigger> flushTriggers;
+
+ private TieredStorageMemoryManager storageMemoryManager;
+
+ private final ScheduledExecutorService executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("cache flush trigger")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+
+ public CacheFlushManager(float numBuffersTriggerFlushRatio) {
+ this.numBuffersTriggerFlushRatio = numBuffersTriggerFlushRatio;
+ this.flushTriggers = new ArrayList<>();
+
+ executor.scheduleWithFixedDelay(
+ this::checkNeedTriggerFlushCachedBuffers, 10, 50, TimeUnit.MILLISECONDS);
+ }
+
+ public void setup(TieredStorageMemoryManager storageMemoryManager) {
+ this.storageMemoryManager = storageMemoryManager;
+ }
+
+ public void registerCacheBufferFlushTrigger(CacheBufferFlushTrigger cacheBufferFlushTrigger) {
+ flushTriggers.add(cacheBufferFlushTrigger);
+ }
+
+ public void triggerFlushCachedBuffers() {
+ flushTriggers.forEach(CacheBufferFlushTrigger::notifyFlushCachedBuffers);
+ }
+
+ public void checkNeedTriggerFlushCachedBuffers() {
+ if (storageMemoryManager == null) {
+ return;
+ }
+
+ if (TieredStorageUtils.needFlushCacheBuffers(
+ storageMemoryManager, numBuffersTriggerFlushRatio)) {
+ triggerFlushCachedBuffers();
+ }
+ }
+
+ public int numFlushTriggers() {
Review Comment:
Fixed.
We have removed the two Apis of `numTotalBuffers ` and `numRequestedBuffers` because we have merged `TieredStorageMemoryManager` and `CacheFlushManager`. So we do not need expose the two args any more.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] xintongsong commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "xintongsong (via GitHub)" <gi...@apache.org>.
xintongsong commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193288010
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManager.java:
##########
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+
+import java.util.List;
+
+/**
+ * The {@link TieredStorageMemoryManager} is to request or recycle buffers from {@link
+ * LocalBufferPool} for different memory owners, for example, the tiers, the buffer accumulator,
+ * etc. Note that the logic for requesting and recycling buffers is consistent for these owners.
+ *
+ * <p>The memory managed by {@link TieredStorageMemoryManager} is categorized into two types:
+ * long-term occupied memory which cannot be immediately released and short-term occupied memory
+ * which can be reclaimed quickly and safely. Long-term occupied memory usage necessitates waiting
+ * for other operations to complete before releasing it, such as downstream consumption. On the
+ * other hand, short-term occupied memory can be freed up at any time, enabling rapid memory
+ * recycling for tasks such as flushing memory to disk or remote storage.
+ *
+ * <p>This {@link TieredStorageMemoryManager} aim to streamline and harmonize memory management
+ * across various layers. Instead of tracking the number of buffers utilized by individual users, it
+ * dynamically calculates a user's maximum guaranteed amount based on the current status of the
+ * manager and the local buffer pool. Specifically, if a user is a long-term occupied memory user,
+ * the {@link TieredStorageMemoryManager} does not limit the user's memory usage, while if a user is
+ * a short-term occupied memory user, the current guaranteed buffers of the user is the left buffers
+ * in the buffer pool - guaranteed amount of other users (excluding the current user).
Review Comment:
This is a bit hard to understand. I think the memory manager never limit users' memory usage. It may not need to understand the long-term and short-term differences.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManager.java:
##########
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+
+import java.util.List;
+
+/**
+ * The {@link TieredStorageMemoryManager} is to request or recycle buffers from {@link
+ * LocalBufferPool} for different memory owners, for example, the tiers, the buffer accumulator,
+ * etc. Note that the logic for requesting and recycling buffers is consistent for these owners.
+ *
+ * <p>The memory managed by {@link TieredStorageMemoryManager} is categorized into two types:
+ * long-term occupied memory which cannot be immediately released and short-term occupied memory
+ * which can be reclaimed quickly and safely. Long-term occupied memory usage necessitates waiting
+ * for other operations to complete before releasing it, such as downstream consumption. On the
+ * other hand, short-term occupied memory can be freed up at any time, enabling rapid memory
+ * recycling for tasks such as flushing memory to disk or remote storage.
+ *
+ * <p>This {@link TieredStorageMemoryManager} aim to streamline and harmonize memory management
+ * across various layers. Instead of tracking the number of buffers utilized by individual users, it
+ * dynamically calculates a user's maximum guaranteed amount based on the current status of the
+ * manager and the local buffer pool. Specifically, if a user is a long-term occupied memory user,
+ * the {@link TieredStorageMemoryManager} does not limit the user's memory usage, while if a user is
+ * a short-term occupied memory user, the current guaranteed buffers of the user is the left buffers
+ * in the buffer pool - guaranteed amount of other users (excluding the current user).
+ */
+public interface TieredStorageMemoryManager {
+
+ /**
+ * Setup the {@link TieredStorageMemoryManager}. When setting up the manager, the {@link
+ * TieredStorageMemorySpec}s for different tiered storages should be ready to indicate each
+ * tiered storage's memory requirement specs.
+ *
+ * @param bufferPool the local buffer pool
+ * @param storageMemorySpecs the memory specs for different tiered storages
+ */
+ void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs);
+
+ /**
+ * Register a buffer flush call back to flush and recycle all the memory in the user when
+ * needed.
+ *
+ * <p>When the left buffers in the {@link BufferPool} are not enough, {@link
+ * TieredStorageMemoryManager} will flush the buffers of the user with the registered callback.
+ *
+ * @param userBufferFlushCallback the buffer flush call back of the memory user
+ */
+ void registerBufferFlushCallback(Runnable userBufferFlushCallback);
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link LocalBufferPool} for a specific owner.
+ *
+ * @return the requested buffer
+ */
+ BufferBuilder requestBufferBlocking();
+
+ /**
+ * Return the available buffers for the owner.
+ *
+ * <p>Note that the available buffers are calculated dynamically based on some conditions, for
+ * example, the state of the {@link BufferPool}, the {@link TieredStorageMemorySpec} of the
+ * owner, etc.
+ *
+ * <p>If a user is a long-term occupied memory user, the {@link TieredStorageMemoryManager} does
+ * not limit the user's memory usage, while if a user is a short-term occupied memory user, the
+ * max allowed buffers of the user is the left buffers in the buffer pool - guaranteed amount of
+ * other users (excluding the current user).
Review Comment:
These are internal.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,224 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking flush. */
+ public static final int DEFAULT_CHECK_FLUSH_INITIAL_DELAY_MS = 10;
+
+ /** Check flush period. */
+ public static final int DEFAULT_CHECK_FLUSH_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** The registered callbacks to flush the buffers in the registered tiered storages. */
+ private final List<Runnable> bufferFlushCallbacks;
+
+ /** The buffer pool usage ratio of triggering the registered storages to flush buffers. */
+ private final float numBuffersTriggerFlushRatio;
+
+ /**
+ * Indicate whether to start the buffer flush checker thread. If the memory manager is used in
+ * downstream, the field will be false because no buffer flush checker is needed.
+ */
+ private final boolean shouldStartBufferFlushChecker;
Review Comment:
Why do we need periodical flush while we already have checks on requesting buffers?
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManager.java:
##########
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+
+import java.util.List;
+
+/**
+ * The {@link TieredStorageMemoryManager} is to request or recycle buffers from {@link
+ * LocalBufferPool} for different memory owners, for example, the tiers, the buffer accumulator,
+ * etc. Note that the logic for requesting and recycling buffers is consistent for these owners.
+ *
+ * <p>The memory managed by {@link TieredStorageMemoryManager} is categorized into two types:
+ * long-term occupied memory which cannot be immediately released and short-term occupied memory
+ * which can be reclaimed quickly and safely. Long-term occupied memory usage necessitates waiting
+ * for other operations to complete before releasing it, such as downstream consumption. On the
+ * other hand, short-term occupied memory can be freed up at any time, enabling rapid memory
+ * recycling for tasks such as flushing memory to disk or remote storage.
+ *
+ * <p>This {@link TieredStorageMemoryManager} aim to streamline and harmonize memory management
+ * across various layers. Instead of tracking the number of buffers utilized by individual users, it
+ * dynamically calculates a user's maximum guaranteed amount based on the current status of the
+ * manager and the local buffer pool. Specifically, if a user is a long-term occupied memory user,
+ * the {@link TieredStorageMemoryManager} does not limit the user's memory usage, while if a user is
+ * a short-term occupied memory user, the current guaranteed buffers of the user is the left buffers
+ * in the buffer pool - guaranteed amount of other users (excluding the current user).
+ */
+public interface TieredStorageMemoryManager {
+
+ /**
+ * Setup the {@link TieredStorageMemoryManager}. When setting up the manager, the {@link
+ * TieredStorageMemorySpec}s for different tiered storages should be ready to indicate each
+ * tiered storage's memory requirement specs.
+ *
+ * @param bufferPool the local buffer pool
+ * @param storageMemorySpecs the memory specs for different tiered storages
+ */
+ void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs);
+
+ /**
+ * Register a buffer flush call back to flush and recycle all the memory in the user when
+ * needed.
+ *
+ * <p>When the left buffers in the {@link BufferPool} are not enough, {@link
+ * TieredStorageMemoryManager} will flush the buffers of the user with the registered callback.
+ *
+ * @param userBufferFlushCallback the buffer flush call back of the memory user
+ */
+ void registerBufferFlushCallback(Runnable userBufferFlushCallback);
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link LocalBufferPool} for a specific owner.
+ *
+ * @return the requested buffer
+ */
+ BufferBuilder requestBufferBlocking();
+
+ /**
+ * Return the available buffers for the owner.
+ *
+ * <p>Note that the available buffers are calculated dynamically based on some conditions, for
+ * example, the state of the {@link BufferPool}, the {@link TieredStorageMemorySpec} of the
+ * owner, etc.
+ *
+ * <p>If a user is a long-term occupied memory user, the {@link TieredStorageMemoryManager} does
+ * not limit the user's memory usage, while if a user is a short-term occupied memory user, the
+ * max allowed buffers of the user is the left buffers in the buffer pool - guaranteed amount of
+ * other users (excluding the current user).
+ */
+ int getMaxAllowedBuffers(Object owner);
Review Comment:
```suggestion
int getMaxNonReclaimableBuffers(Object owner);
```
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManager.java:
##########
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+
+import java.util.List;
+
+/**
+ * The {@link TieredStorageMemoryManager} is to request or recycle buffers from {@link
+ * LocalBufferPool} for different memory owners, for example, the tiers, the buffer accumulator,
+ * etc. Note that the logic for requesting and recycling buffers is consistent for these owners.
+ *
+ * <p>The memory managed by {@link TieredStorageMemoryManager} is categorized into two types:
+ * long-term occupied memory which cannot be immediately released and short-term occupied memory
+ * which can be reclaimed quickly and safely. Long-term occupied memory usage necessitates waiting
+ * for other operations to complete before releasing it, such as downstream consumption. On the
+ * other hand, short-term occupied memory can be freed up at any time, enabling rapid memory
+ * recycling for tasks such as flushing memory to disk or remote storage.
+ *
+ * <p>This {@link TieredStorageMemoryManager} aim to streamline and harmonize memory management
+ * across various layers. Instead of tracking the number of buffers utilized by individual users, it
+ * dynamically calculates a user's maximum guaranteed amount based on the current status of the
+ * manager and the local buffer pool. Specifically, if a user is a long-term occupied memory user,
+ * the {@link TieredStorageMemoryManager} does not limit the user's memory usage, while if a user is
+ * a short-term occupied memory user, the current guaranteed buffers of the user is the left buffers
+ * in the buffer pool - guaranteed amount of other users (excluding the current user).
+ */
+public interface TieredStorageMemoryManager {
+
+ /**
+ * Setup the {@link TieredStorageMemoryManager}. When setting up the manager, the {@link
+ * TieredStorageMemorySpec}s for different tiered storages should be ready to indicate each
+ * tiered storage's memory requirement specs.
+ *
+ * @param bufferPool the local buffer pool
+ * @param storageMemorySpecs the memory specs for different tiered storages
+ */
+ void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs);
+
+ /**
+ * Register a buffer flush call back to flush and recycle all the memory in the user when
+ * needed.
+ *
+ * <p>When the left buffers in the {@link BufferPool} are not enough, {@link
+ * TieredStorageMemoryManager} will flush the buffers of the user with the registered callback.
+ *
+ * @param userBufferFlushCallback the buffer flush call back of the memory user
+ */
+ void registerBufferFlushCallback(Runnable userBufferFlushCallback);
Review Comment:
```suggestion
void listenBufferReclaimRequest(Runnable onBufferReclaimRequest);
```
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManager.java:
##########
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+
+import java.util.List;
+
+/**
+ * The {@link TieredStorageMemoryManager} is to request or recycle buffers from {@link
+ * LocalBufferPool} for different memory owners, for example, the tiers, the buffer accumulator,
+ * etc. Note that the logic for requesting and recycling buffers is consistent for these owners.
+ *
+ * <p>The memory managed by {@link TieredStorageMemoryManager} is categorized into two types:
+ * long-term occupied memory which cannot be immediately released and short-term occupied memory
+ * which can be reclaimed quickly and safely. Long-term occupied memory usage necessitates waiting
+ * for other operations to complete before releasing it, such as downstream consumption. On the
+ * other hand, short-term occupied memory can be freed up at any time, enabling rapid memory
+ * recycling for tasks such as flushing memory to disk or remote storage.
+ *
+ * <p>This {@link TieredStorageMemoryManager} aim to streamline and harmonize memory management
+ * across various layers. Instead of tracking the number of buffers utilized by individual users, it
+ * dynamically calculates a user's maximum guaranteed amount based on the current status of the
+ * manager and the local buffer pool. Specifically, if a user is a long-term occupied memory user,
+ * the {@link TieredStorageMemoryManager} does not limit the user's memory usage, while if a user is
+ * a short-term occupied memory user, the current guaranteed buffers of the user is the left buffers
+ * in the buffer pool - guaranteed amount of other users (excluding the current user).
+ */
+public interface TieredStorageMemoryManager {
+
+ /**
+ * Setup the {@link TieredStorageMemoryManager}. When setting up the manager, the {@link
+ * TieredStorageMemorySpec}s for different tiered storages should be ready to indicate each
+ * tiered storage's memory requirement specs.
+ *
+ * @param bufferPool the local buffer pool
+ * @param storageMemorySpecs the memory specs for different tiered storages
+ */
+ void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs);
+
+ /**
+ * Register a buffer flush call back to flush and recycle all the memory in the user when
+ * needed.
+ *
+ * <p>When the left buffers in the {@link BufferPool} are not enough, {@link
+ * TieredStorageMemoryManager} will flush the buffers of the user with the registered callback.
+ *
+ * @param userBufferFlushCallback the buffer flush call back of the memory user
+ */
+ void registerBufferFlushCallback(Runnable userBufferFlushCallback);
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link LocalBufferPool} for a specific owner.
+ *
+ * @return the requested buffer
+ */
+ BufferBuilder requestBufferBlocking();
+
+ /**
+ * Return the available buffers for the owner.
+ *
+ * <p>Note that the available buffers are calculated dynamically based on some conditions, for
+ * example, the state of the {@link BufferPool}, the {@link TieredStorageMemorySpec} of the
+ * owner, etc.
Review Comment:
So the caller should always check before requesting non-reclaimable buffers.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,224 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking flush. */
+ public static final int DEFAULT_CHECK_FLUSH_INITIAL_DELAY_MS = 10;
+
+ /** Check flush period. */
+ public static final int DEFAULT_CHECK_FLUSH_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** The registered callbacks to flush the buffers in the registered tiered storages. */
+ private final List<Runnable> bufferFlushCallbacks;
+
+ /** The buffer pool usage ratio of triggering the registered storages to flush buffers. */
+ private final float numBuffersTriggerFlushRatio;
+
+ /**
+ * Indicate whether to start the buffer flush checker thread. If the memory manager is used in
+ * downstream, the field will be false because no buffer flush checker is needed.
+ */
+ private final boolean shouldStartBufferFlushChecker;
+
+ /** The number of requested buffers from {@link BufferPool}. */
+ private final AtomicInteger numRequestedBuffers;
+
+ /** A thread to check whether to flush buffers in each tiered storage. */
+ private ScheduledExecutorService executor;
+
+ /** The total number of guaranteed buffers for all tiered storages. */
+ private int numTotalGuaranteedBuffers;
+
+ /** The buffer pool where the buffer is requested or recyceld. */
+ private BufferPool bufferPool;
+
+ /**
+ * Indicate whether the {@link TieredStorageMemoryManagerImpl} is in running state. Before
+ * setting up, this field is false.
+ *
+ * <p>Note that before requesting buffers or getting the maximum allowed buffers, this running
+ * state should be checked.
+ */
+ private boolean isRunning;
+
+ /**
+ * The constructor of the {@link TieredStorageMemoryManagerImpl}.
+ *
+ * @param numBuffersTriggerFlushRatio the buffer pool usage ratio of triggering each tiered
+ * storage to flush buffers
+ * @param shouldStartBufferFlushChecker indicate whether to start the buffer flushing checker
+ * thread
+ */
+ public TieredStorageMemoryManagerImpl(
+ float numBuffersTriggerFlushRatio, boolean shouldStartBufferFlushChecker) {
+ this.numBuffersTriggerFlushRatio = numBuffersTriggerFlushRatio;
+ this.shouldStartBufferFlushChecker = shouldStartBufferFlushChecker;
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ this.bufferFlushCallbacks = new ArrayList<>();
+ this.isRunning = false;
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs) {
+ this.bufferPool = bufferPool;
+ for (TieredStorageMemorySpec memorySpec : storageMemorySpecs) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalGuaranteedBuffers += memorySpec.getNumGuaranteedBuffers();
+ }
+
+ if (shouldStartBufferFlushChecker) {
+ this.executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("buffer flush checker")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+ this.executor.scheduleWithFixedDelay(
+ this::checkShouldFlushCachedBuffers,
+ DEFAULT_CHECK_FLUSH_INITIAL_DELAY_MS,
+ DEFAULT_CHECK_FLUSH_PERIOD_DURATION_MS,
+ TimeUnit.MILLISECONDS);
+ }
+
+ this.isRunning = true;
+ }
+
+ @Override
+ public void registerBufferFlushCallback(Runnable userBufferFlushCallBack) {
+ bufferFlushCallbacks.add(userBufferFlushCallBack);
+ }
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link BufferPool} for a specific owner. The
+ * {@link TieredStorageMemoryManagerImpl} will not check whether a buffer can be requested and
+ * only record the total number of requested buffers. If the buffers in the {@link BufferPool}
+ * is not enough, this will trigger each tiered storage to flush buffers as much as possible.
+ *
+ * @return the requested buffer
+ */
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ checkIsRunning();
+
+ MemorySegment requestedBuffer = null;
+ try {
+ requestedBuffer = bufferPool.requestMemorySegmentBlocking();
+ } catch (Throwable throwable) {
+ ExceptionUtils.rethrow(throwable, "Failed to request memory segments.");
+ }
+ numRequestedBuffers.incrementAndGet();
+ checkShouldFlushCachedBuffers();
Review Comment:
Shouldn't this happen before requesting?
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,224 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking flush. */
+ public static final int DEFAULT_CHECK_FLUSH_INITIAL_DELAY_MS = 10;
+
+ /** Check flush period. */
+ public static final int DEFAULT_CHECK_FLUSH_PERIOD_DURATION_MS = 50;
Review Comment:
private
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,224 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking flush. */
+ public static final int DEFAULT_CHECK_FLUSH_INITIAL_DELAY_MS = 10;
+
+ /** Check flush period. */
+ public static final int DEFAULT_CHECK_FLUSH_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** The registered callbacks to flush the buffers in the registered tiered storages. */
+ private final List<Runnable> bufferFlushCallbacks;
+
+ /** The buffer pool usage ratio of triggering the registered storages to flush buffers. */
+ private final float numBuffersTriggerFlushRatio;
+
+ /**
+ * Indicate whether to start the buffer flush checker thread. If the memory manager is used in
+ * downstream, the field will be false because no buffer flush checker is needed.
+ */
+ private final boolean shouldStartBufferFlushChecker;
+
+ /** The number of requested buffers from {@link BufferPool}. */
+ private final AtomicInteger numRequestedBuffers;
+
+ /** A thread to check whether to flush buffers in each tiered storage. */
+ private ScheduledExecutorService executor;
+
+ /** The total number of guaranteed buffers for all tiered storages. */
+ private int numTotalGuaranteedBuffers;
+
+ /** The buffer pool where the buffer is requested or recyceld. */
+ private BufferPool bufferPool;
+
+ /**
+ * Indicate whether the {@link TieredStorageMemoryManagerImpl} is in running state. Before
+ * setting up, this field is false.
+ *
+ * <p>Note that before requesting buffers or getting the maximum allowed buffers, this running
+ * state should be checked.
+ */
+ private boolean isRunning;
+
+ /**
+ * The constructor of the {@link TieredStorageMemoryManagerImpl}.
+ *
+ * @param numBuffersTriggerFlushRatio the buffer pool usage ratio of triggering each tiered
+ * storage to flush buffers
+ * @param shouldStartBufferFlushChecker indicate whether to start the buffer flushing checker
+ * thread
+ */
+ public TieredStorageMemoryManagerImpl(
+ float numBuffersTriggerFlushRatio, boolean shouldStartBufferFlushChecker) {
+ this.numBuffersTriggerFlushRatio = numBuffersTriggerFlushRatio;
+ this.shouldStartBufferFlushChecker = shouldStartBufferFlushChecker;
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ this.bufferFlushCallbacks = new ArrayList<>();
+ this.isRunning = false;
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs) {
+ this.bufferPool = bufferPool;
+ for (TieredStorageMemorySpec memorySpec : storageMemorySpecs) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalGuaranteedBuffers += memorySpec.getNumGuaranteedBuffers();
+ }
+
+ if (shouldStartBufferFlushChecker) {
+ this.executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("buffer flush checker")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+ this.executor.scheduleWithFixedDelay(
+ this::checkShouldFlushCachedBuffers,
+ DEFAULT_CHECK_FLUSH_INITIAL_DELAY_MS,
+ DEFAULT_CHECK_FLUSH_PERIOD_DURATION_MS,
+ TimeUnit.MILLISECONDS);
+ }
+
+ this.isRunning = true;
+ }
+
+ @Override
+ public void registerBufferFlushCallback(Runnable userBufferFlushCallBack) {
+ bufferFlushCallbacks.add(userBufferFlushCallBack);
+ }
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link BufferPool} for a specific owner. The
+ * {@link TieredStorageMemoryManagerImpl} will not check whether a buffer can be requested and
+ * only record the total number of requested buffers. If the buffers in the {@link BufferPool}
+ * is not enough, this will trigger each tiered storage to flush buffers as much as possible.
+ *
+ * @return the requested buffer
+ */
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ checkIsRunning();
+
+ MemorySegment requestedBuffer = null;
+ try {
+ requestedBuffer = bufferPool.requestMemorySegmentBlocking();
+ } catch (Throwable throwable) {
+ ExceptionUtils.rethrow(throwable, "Failed to request memory segments.");
+ }
+ numRequestedBuffers.incrementAndGet();
+ checkShouldFlushCachedBuffers();
+ return new BufferBuilder(checkNotNull(requestedBuffer), this::recycleBuffer);
+ }
+
+ @Override
+ public int getMaxAllowedBuffers(Object owner) {
+ checkIsRunning();
+
+ TieredStorageMemorySpec ownerMemorySpec = checkNotNull(tieredMemorySpecs.get(owner));
+ if (ownerMemorySpec.isMemoryReleasable()) {
+ return Integer.MAX_VALUE;
+ } else {
Review Comment:
We may not need the releasable / unreleasable at all.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,224 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking flush. */
+ public static final int DEFAULT_CHECK_FLUSH_INITIAL_DELAY_MS = 10;
+
+ /** Check flush period. */
+ public static final int DEFAULT_CHECK_FLUSH_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** The registered callbacks to flush the buffers in the registered tiered storages. */
+ private final List<Runnable> bufferFlushCallbacks;
+
+ /** The buffer pool usage ratio of triggering the registered storages to flush buffers. */
+ private final float numBuffersTriggerFlushRatio;
+
+ /**
+ * Indicate whether to start the buffer flush checker thread. If the memory manager is used in
+ * downstream, the field will be false because no buffer flush checker is needed.
+ */
+ private final boolean shouldStartBufferFlushChecker;
Review Comment:
It seems the periodical flush is not part of the memory management.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,224 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking flush. */
+ public static final int DEFAULT_CHECK_FLUSH_INITIAL_DELAY_MS = 10;
+
+ /** Check flush period. */
+ public static final int DEFAULT_CHECK_FLUSH_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** The registered callbacks to flush the buffers in the registered tiered storages. */
+ private final List<Runnable> bufferFlushCallbacks;
+
+ /** The buffer pool usage ratio of triggering the registered storages to flush buffers. */
+ private final float numBuffersTriggerFlushRatio;
+
+ /**
+ * Indicate whether to start the buffer flush checker thread. If the memory manager is used in
+ * downstream, the field will be false because no buffer flush checker is needed.
+ */
+ private final boolean shouldStartBufferFlushChecker;
+
+ /** The number of requested buffers from {@link BufferPool}. */
+ private final AtomicInteger numRequestedBuffers;
+
+ /** A thread to check whether to flush buffers in each tiered storage. */
+ private ScheduledExecutorService executor;
+
+ /** The total number of guaranteed buffers for all tiered storages. */
+ private int numTotalGuaranteedBuffers;
+
+ /** The buffer pool where the buffer is requested or recyceld. */
+ private BufferPool bufferPool;
+
+ /**
+ * Indicate whether the {@link TieredStorageMemoryManagerImpl} is in running state. Before
+ * setting up, this field is false.
+ *
+ * <p>Note that before requesting buffers or getting the maximum allowed buffers, this running
+ * state should be checked.
+ */
+ private boolean isRunning;
Review Comment:
For `isRunning`, we should set it to `false` in `release()`. Or we can rename it to `initialized`.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,224 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking flush. */
+ public static final int DEFAULT_CHECK_FLUSH_INITIAL_DELAY_MS = 10;
+
+ /** Check flush period. */
+ public static final int DEFAULT_CHECK_FLUSH_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** The registered callbacks to flush the buffers in the registered tiered storages. */
+ private final List<Runnable> bufferFlushCallbacks;
+
+ /** The buffer pool usage ratio of triggering the registered storages to flush buffers. */
+ private final float numBuffersTriggerFlushRatio;
+
+ /**
+ * Indicate whether to start the buffer flush checker thread. If the memory manager is used in
+ * downstream, the field will be false because no buffer flush checker is needed.
+ */
+ private final boolean shouldStartBufferFlushChecker;
+
+ /** The number of requested buffers from {@link BufferPool}. */
+ private final AtomicInteger numRequestedBuffers;
+
+ /** A thread to check whether to flush buffers in each tiered storage. */
+ private ScheduledExecutorService executor;
+
+ /** The total number of guaranteed buffers for all tiered storages. */
+ private int numTotalGuaranteedBuffers;
+
+ /** The buffer pool where the buffer is requested or recyceld. */
+ private BufferPool bufferPool;
+
+ /**
+ * Indicate whether the {@link TieredStorageMemoryManagerImpl} is in running state. Before
+ * setting up, this field is false.
+ *
+ * <p>Note that before requesting buffers or getting the maximum allowed buffers, this running
+ * state should be checked.
+ */
+ private boolean isRunning;
+
+ /**
+ * The constructor of the {@link TieredStorageMemoryManagerImpl}.
+ *
+ * @param numBuffersTriggerFlushRatio the buffer pool usage ratio of triggering each tiered
+ * storage to flush buffers
+ * @param shouldStartBufferFlushChecker indicate whether to start the buffer flushing checker
+ * thread
+ */
+ public TieredStorageMemoryManagerImpl(
+ float numBuffersTriggerFlushRatio, boolean shouldStartBufferFlushChecker) {
+ this.numBuffersTriggerFlushRatio = numBuffersTriggerFlushRatio;
+ this.shouldStartBufferFlushChecker = shouldStartBufferFlushChecker;
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ this.bufferFlushCallbacks = new ArrayList<>();
+ this.isRunning = false;
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs) {
+ this.bufferPool = bufferPool;
+ for (TieredStorageMemorySpec memorySpec : storageMemorySpecs) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalGuaranteedBuffers += memorySpec.getNumGuaranteedBuffers();
+ }
+
+ if (shouldStartBufferFlushChecker) {
+ this.executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("buffer flush checker")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+ this.executor.scheduleWithFixedDelay(
+ this::checkShouldFlushCachedBuffers,
+ DEFAULT_CHECK_FLUSH_INITIAL_DELAY_MS,
+ DEFAULT_CHECK_FLUSH_PERIOD_DURATION_MS,
+ TimeUnit.MILLISECONDS);
+ }
+
+ this.isRunning = true;
+ }
+
+ @Override
+ public void registerBufferFlushCallback(Runnable userBufferFlushCallBack) {
+ bufferFlushCallbacks.add(userBufferFlushCallBack);
+ }
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link BufferPool} for a specific owner. The
+ * {@link TieredStorageMemoryManagerImpl} will not check whether a buffer can be requested and
+ * only record the total number of requested buffers. If the buffers in the {@link BufferPool}
+ * is not enough, this will trigger each tiered storage to flush buffers as much as possible.
+ *
+ * @return the requested buffer
+ */
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ checkIsRunning();
+
+ MemorySegment requestedBuffer = null;
+ try {
+ requestedBuffer = bufferPool.requestMemorySegmentBlocking();
+ } catch (Throwable throwable) {
+ ExceptionUtils.rethrow(throwable, "Failed to request memory segments.");
+ }
+ numRequestedBuffers.incrementAndGet();
+ checkShouldFlushCachedBuffers();
+ return new BufferBuilder(checkNotNull(requestedBuffer), this::recycleBuffer);
+ }
+
+ @Override
+ public int getMaxAllowedBuffers(Object owner) {
+ checkIsRunning();
+
+ TieredStorageMemorySpec ownerMemorySpec = checkNotNull(tieredMemorySpecs.get(owner));
+ if (ownerMemorySpec.isMemoryReleasable()) {
+ return Integer.MAX_VALUE;
+ } else {
+ int ownerGuaranteedBuffers = ownerMemorySpec.getNumGuaranteedBuffers();
+ return bufferPool.getNumBuffers()
+ - numRequestedBuffers.get()
+ - numTotalGuaranteedBuffers
+ + ownerGuaranteedBuffers;
+ }
+ }
+
+ @Override
+ public void release() {
+ checkState(numRequestedBuffers.get() == 0, "Leaking buffers.");
+ if (executor != null) {
+ executor.shutdown();
+ try {
+ if (!executor.awaitTermination(5L, TimeUnit.MINUTES)) {
+ throw new TimeoutException(
+ "Timeout for shutting down the cache flush executor.");
+ }
+ } catch (Exception e) {
+ ExceptionUtils.rethrow(e);
+ }
+ }
+ }
+
+ private void checkShouldFlushCachedBuffers() {
+ if (shouldFlushBuffers()) {
+ bufferFlushCallbacks.forEach(Runnable::run);
+ }
+ }
+
+ private boolean shouldFlushBuffers() {
+ synchronized (this) {
+ int numTotal = bufferPool.getNumBuffers();
+ int numRequested = numRequestedBuffers.get();
+ return numRequested >= numTotal
+ || (numRequested * 1.0 / numTotal) >= numBuffersTriggerFlushRatio;
+ }
Review Comment:
This does not guarantees the atomicity, unless all access to `bufferPool` and `numRequestedBuffers` are guarded by `this`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] xintongsong commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "xintongsong (via GitHub)" <gi...@apache.org>.
xintongsong commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1181474271
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/CacheFlushManager.java:
##########
@@ -0,0 +1,100 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.runtime.io.network.partition.hybrid.tiered.common.TieredStorageUtils;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+
+/**
+ * A manager to check whether the cached buffers need to be flushed.
+ *
+ * <p>Note that when initializing a tier with cached buffers, the tier should register a {@link
+ * CacheBufferFlushTrigger}, as a listener to flush cached buffers.
+ */
+public class CacheFlushManager {
Review Comment:
I see what this class does from the implementation. But it is unclear to me why do we need this? Why would we need to flush everything when a certain ratio of buffers in the pool are used?
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManager.java:
##########
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+
+/**
+ * The {@link TieredStorageMemoryManager} is to request or recycle buffer from {@link
+ * LocalBufferPool} for different tiers or {@link BufferAccumulator}.
+ *
+ * <p>The buffers of {@link BufferAccumulator} is also managed in the {@link
+ * TieredStorageMemoryManager}. When {@link BufferAccumulator} requests or recycles buffers, it uses
+ * separate methods that are different from tiers.
+ */
+public interface TieredStorageMemoryManager {
+
+ /** Setup the {@link TieredStorageMemoryManager}. */
+ void setup(BufferPool bufferPool);
+
+ /**
+ * Request a {@link MemorySegment} instance from {@link LocalBufferPool} for a specific tier.
+ *
+ * @param tierIndex the tier index
+ * @return the requested buffer
+ */
+ MemorySegment requestBufferBlocking(int tierIndex);
+
+ /**
+ * Recycle a {@link MemorySegment} buffer to {@link LocalBufferPool} for a specific tier.
+ *
+ * @param memorySegment the buffer to be recycled
+ * @param tierIndex the specific tier index
+ */
+ void recycleBuffer(MemorySegment memorySegment, int tierIndex);
+
+ /** Request a {@link MemorySegment} buffer for the buffer accumulator. */
+ MemorySegment requestBufferInAccumulator();
+
+ /** Recycle a {@link MemorySegment} buffer for the buffer accumulator. */
+ void recycleBufferInAccumulator(MemorySegment memorySegment);
+
+ /**
+ * Increase the number of requested buffers for a tier.
+ *
+ * @param tierIndex the specific tier index
+ */
+ void incNumRequestedBuffer(int tierIndex);
+
+ /**
+ * Decrease the number of requested buffers for a tier.
+ *
+ * @param tierIndex the specific tier index
+ */
+ void decNumRequestedBuffer(int tierIndex);
+
+ /** Increase the number of requested buffers for buffer accumulator. */
+ void incNumRequestedBufferInAccumulator();
+
+ /** Decrease the number of requested buffers for buffer accumulator. */
+ void decNumRequestedBufferInAccumulator();
Review Comment:
These seems to be internal methods and should not be exposed via the interface.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManager.java:
##########
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+
+/**
+ * The {@link TieredStorageMemoryManager} is to request or recycle buffer from {@link
+ * LocalBufferPool} for different tiers or {@link BufferAccumulator}.
+ *
+ * <p>The buffers of {@link BufferAccumulator} is also managed in the {@link
+ * TieredStorageMemoryManager}. When {@link BufferAccumulator} requests or recycles buffers, it uses
+ * separate methods that are different from tiers.
+ */
+public interface TieredStorageMemoryManager {
+
+ /** Setup the {@link TieredStorageMemoryManager}. */
+ void setup(BufferPool bufferPool);
+
+ /**
+ * Request a {@link MemorySegment} instance from {@link LocalBufferPool} for a specific tier.
+ *
+ * @param tierIndex the tier index
+ * @return the requested buffer
+ */
+ MemorySegment requestBufferBlocking(int tierIndex);
+
+ /**
+ * Recycle a {@link MemorySegment} buffer to {@link LocalBufferPool} for a specific tier.
+ *
+ * @param memorySegment the buffer to be recycled
+ * @param tierIndex the specific tier index
+ */
+ void recycleBuffer(MemorySegment memorySegment, int tierIndex);
+
+ /** Request a {@link MemorySegment} buffer for the buffer accumulator. */
+ MemorySegment requestBufferInAccumulator();
+
+ /** Recycle a {@link MemorySegment} buffer for the buffer accumulator. */
+ void recycleBufferInAccumulator(MemorySegment memorySegment);
Review Comment:
Why do we have different interfaces for tiers and accumulator? How are they different as memory consumers?
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,178 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.runtime.io.network.partition.hybrid.tiered.tier.TierMemorySpec;
+import org.apache.flink.util.ExceptionUtils;
+
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different tiers or {@link BufferAccumulator}.
+ *
+ * <p>The buffers in a buffer pool are mainly divided into types, the exclusive buffers for a tier
+ * and the shared buffers for all tiers. Each tier decides the number of exclusive buffers and
+ * whether it can use the shared buffers by {@link TierMemorySpec}.
+ *
+ * <p>The buffers of {@link BufferAccumulator} is also managed in the {@link
+ * TieredStorageMemoryManager}. When {@link BufferAccumulator} requests or recycles buffers, it uses
+ * separate methods that are different from tiers.
+ *
+ * <p>Note that when initializing the {@link TieredStorageMemoryManagerImpl}, the {@link
+ * TierMemorySpec} of each tier should be ready. These tier memory specs mainly decide how many
+ * buffers can be used for every tier.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
Review Comment:
How does this memory manager guarantees the exclusive buffers? I see this class bookkeeps the number of buffers each tier and accumulator uses. However, I don't see any logic for reversing exclusive buffers for a tier, or limiting a tier from requesting too many buffers.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManager.java:
##########
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+
+/**
+ * The {@link TieredStorageMemoryManager} is to request or recycle buffer from {@link
+ * LocalBufferPool} for different tiers or {@link BufferAccumulator}.
+ *
+ * <p>The buffers of {@link BufferAccumulator} is also managed in the {@link
+ * TieredStorageMemoryManager}. When {@link BufferAccumulator} requests or recycles buffers, it uses
+ * separate methods that are different from tiers.
+ */
+public interface TieredStorageMemoryManager {
+
+ /** Setup the {@link TieredStorageMemoryManager}. */
+ void setup(BufferPool bufferPool);
+
+ /**
+ * Request a {@link MemorySegment} instance from {@link LocalBufferPool} for a specific tier.
+ *
+ * @param tierIndex the tier index
+ * @return the requested buffer
+ */
+ MemorySegment requestBufferBlocking(int tierIndex);
+
+ /**
+ * Recycle a {@link MemorySegment} buffer to {@link LocalBufferPool} for a specific tier.
+ *
+ * @param memorySegment the buffer to be recycled
+ * @param tierIndex the specific tier index
+ */
+ void recycleBuffer(MemorySegment memorySegment, int tierIndex);
Review Comment:
Why not provide `Buffer` directly? So that we won't need a `recycle` interface at all, because the recycler can be set in the returned buffer.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/tier/TierMemorySpec.java:
##########
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.tier;
+
+/**
+ * The memory specs for a tier, including the tier index, the number of exclusive buffers of the
+ * tier, whether the tier can use the shared buffers in the memory manager, etc.
+ */
+public class TierMemorySpec {
+
+ private final int tierIndex;
+
+ private final int numExclusiveBuffers;
+
+ private final boolean canUseSharedBuffers;
+
+ public TierMemorySpec(int tierIndex, int numExclusiveBuffers, boolean canUseSharedBuffers) {
+ this.tierIndex = tierIndex;
+ this.numExclusiveBuffers = numExclusiveBuffers;
+ this.canUseSharedBuffers = canUseSharedBuffers;
+ }
+
+ public int getTierIndex() {
+ return tierIndex;
+ }
+
+ public int getNumExclusiveBuffers() {
+ return numExclusiveBuffers;
+ }
+
+ public boolean canUseShareBuffers() {
+ return canUseSharedBuffers;
Review Comment:
Under what condition should this be `false`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] xintongsong commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "xintongsong (via GitHub)" <gi...@apache.org>.
xintongsong commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1199686953
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,279 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_INITIAL_DELAY_MS = 10;
+
+ /** The period of checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** Listeners used to listen the requests for reclaiming buffer in different tiered storage. */
+ private final List<Runnable> bufferReclaimRequestListeners;
+
+ /** The buffer pool usage ratio of triggering the registered storages to reclaim buffers. */
+ private final float numTriggerReclaimBuffersRatio;
+
+ /**
+ * Indicate whether it is necessary to start a periodically checking buffer reclaim thread. If
+ * the memory manager is used in downstream, the field will be false because periodical buffer
+ * reclaim checker is needed.
+ */
+ private final boolean needPeriodicalCheckReclaimBuffer;
+
+ /**
+ * The number of requested buffers from {@link BufferPool}. Only this field can be touched both
+ * by the task thread and the netty thread, so it is an atomic type.
+ */
+ private final AtomicInteger numRequestedBuffers;
+
+ /**
+ * A thread to check whether to reclaim buffers from each tiered storage.
+ *
+ * <p>Note that it is not possible to remove this, as doing so could result in the task becoming
+ * stuck in the buffer request. As the number of buffers in the buffer pool can vary at any
+ * given time, the stuck may occur if the thread is removed.
+ *
+ * <p>For instance, if the memory usage of the {@link BufferPool} has been checked and {@link
+ * TieredStorageMemoryManagerImpl} determined that buffer reclamation is unnecessary, but then
+ * the buffer pool size is suddenly reduced to a very small size, the buffer request will become
+ * stuck and the task will never be able to call for buffer reclamation if this thread is
+ * removed, then a task stuck occurs.
+ */
+ private ScheduledExecutorService executor;
+
+ /** The total number of guaranteed buffers for all tiered storages. */
+ private int numTotalGuaranteedBuffers;
+
+ /** The buffer pool where the buffer is requested or recycled. */
+ private BufferPool bufferPool;
+
+ /**
+ * Indicate whether the {@link TieredStorageMemoryManagerImpl} is initialized. Before setting
+ * up, this field is false.
+ *
+ * <p>Note that before requesting buffers or getting the maximum allowed buffers, this
+ * initialized state should be checked.
+ */
+ private boolean isInitialized;
+
+ /**
+ * The constructor of the {@link TieredStorageMemoryManagerImpl}.
+ *
+ * @param numTriggerReclaimBuffersRatio the buffer pool usage ratio of requesting each tiered
+ * storage to reclaim buffers
+ * @param needPeriodicalCheckReclaimBuffer indicate whether it is necessary to start a
+ * periodically checking buffer reclaim thread
+ */
+ public TieredStorageMemoryManagerImpl(
+ float numTriggerReclaimBuffersRatio, boolean needPeriodicalCheckReclaimBuffer) {
+ this.numTriggerReclaimBuffersRatio = numTriggerReclaimBuffersRatio;
+ this.needPeriodicalCheckReclaimBuffer = needPeriodicalCheckReclaimBuffer;
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ this.bufferReclaimRequestListeners = new ArrayList<>();
+ this.isInitialized = false;
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs) {
+ this.bufferPool = bufferPool;
+ for (TieredStorageMemorySpec memorySpec : storageMemorySpecs) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalGuaranteedBuffers += memorySpec.getNumGuaranteedBuffers();
+ }
+
+ if (needPeriodicalCheckReclaimBuffer) {
+ this.executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("buffer reclaim checker")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+ }
+
+ this.isInitialized = true;
+ }
+
+ @Override
+ public void listenBufferReclaimRequest(Runnable onBufferReclaimRequest) {
+ bufferReclaimRequestListeners.add(onBufferReclaimRequest);
+ }
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link BufferPool} for a specific owner. The
+ * {@link TieredStorageMemoryManagerImpl} will not check whether a buffer can be requested and
+ * only record the total number of requested buffers. If the buffers in the {@link BufferPool}
+ * is not enough, this will request each tiered storage to reclaim the buffers as much as
+ * possible.
+ *
+ * <p>Note that synchronization is not necessary for this method, as it can only be called by
+ * the task thread.
+ *
+ * @return the requested buffer
+ */
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ checkIsInitialized();
+
+ tryReclaimBuffers();
+ CompletableFuture<MemorySegment> requestedBuffer =
+ CompletableFuture.supplyAsync(
+ () -> {
+ try {
+ return bufferPool.requestMemorySegmentBlocking();
+ } catch (InterruptedException e) {
+ throw new RuntimeException(e);
+ }
+ });
+
+ ScheduledFuture<?> scheduledFuture = getPeriodicalCheckReclaimBufferFuture();
+
+ MemorySegment memorySegment = null;
+ try {
+ memorySegment = requestedBuffer.get();
+ } catch (InterruptedException | ExecutionException e) {
+ ExceptionUtils.rethrow(e);
+ }
+
+ if (scheduledFuture != null) {
+ scheduledFuture.cancel(false);
+ }
+
+ numRequestedBuffers.incrementAndGet();
+ return new BufferBuilder(checkNotNull(memorySegment), this::recycleBuffer);
+ }
+
+ /**
+ * The synchronization is not necessary for this method, as it can only be called by the task
+ * thread.
+ *
+ * <p>The accuracy of the memory usage ratio may be compromised due to the varying buffer pool
+ * sizes. However, this only impacts a single iteration of the buffer usage check. Upon the next
+ * iteration, the buffer reclaim will eventually be triggered.
+ */
+ @Override
+ public int getMaxNonReclaimableBuffers(Object owner) {
+ checkIsInitialized();
+
+ TieredStorageMemorySpec ownerMemorySpec = checkNotNull(tieredMemorySpecs.get(owner));
+ int ownerGuaranteedBuffers = ownerMemorySpec.getNumGuaranteedBuffers();
+ return bufferPool.getNumBuffers()
+ - numRequestedBuffers.get()
+ - numTotalGuaranteedBuffers
+ + ownerGuaranteedBuffers;
+ }
+
+ @Override
+ public void release() {
+ checkState(numRequestedBuffers.get() == 0, "Leaking buffers.");
+ if (executor != null) {
+ executor.shutdown();
+ try {
+ if (!executor.awaitTermination(5L, TimeUnit.MINUTES)) {
+ throw new TimeoutException(
+ "Timeout for shutting down the buffer reclaim checker executor.");
+ }
+ } catch (Exception e) {
+ ExceptionUtils.rethrow(e);
+ }
+ }
+ }
+
+ private void tryReclaimBuffers() {
+ if (shouldReclaimBuffers()) {
+ bufferReclaimRequestListeners.forEach(Runnable::run);
+ }
+ }
+
+ /**
+ * The accuracy of the memory usage ratio may be compromised due to the varying buffer pool
+ * sizes. However, this only impacts a single iteration of the buffer usage check. Upon the next
+ * iteration, the buffer reclaim will eventually be triggered.
+ */
Review Comment:
Explanation for internal implementations should be inline comments.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,279 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_INITIAL_DELAY_MS = 10;
+
+ /** The period of checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** Listeners used to listen the requests for reclaiming buffer in different tiered storage. */
+ private final List<Runnable> bufferReclaimRequestListeners;
+
+ /** The buffer pool usage ratio of triggering the registered storages to reclaim buffers. */
+ private final float numTriggerReclaimBuffersRatio;
+
+ /**
+ * Indicate whether it is necessary to start a periodically checking buffer reclaim thread. If
+ * the memory manager is used in downstream, the field will be false because periodical buffer
+ * reclaim checker is needed.
+ */
+ private final boolean needPeriodicalCheckReclaimBuffer;
+
+ /**
+ * The number of requested buffers from {@link BufferPool}. Only this field can be touched both
+ * by the task thread and the netty thread, so it is an atomic type.
+ */
+ private final AtomicInteger numRequestedBuffers;
+
+ /**
+ * A thread to check whether to reclaim buffers from each tiered storage.
+ *
+ * <p>Note that it is not possible to remove this, as doing so could result in the task becoming
+ * stuck in the buffer request. As the number of buffers in the buffer pool can vary at any
+ * given time, the stuck may occur if the thread is removed.
+ *
+ * <p>For instance, if the memory usage of the {@link BufferPool} has been checked and {@link
+ * TieredStorageMemoryManagerImpl} determined that buffer reclamation is unnecessary, but then
+ * the buffer pool size is suddenly reduced to a very small size, the buffer request will become
+ * stuck and the task will never be able to call for buffer reclamation if this thread is
+ * removed, then a task stuck occurs.
+ */
+ private ScheduledExecutorService executor;
+
+ /** The total number of guaranteed buffers for all tiered storages. */
+ private int numTotalGuaranteedBuffers;
+
+ /** The buffer pool where the buffer is requested or recycled. */
+ private BufferPool bufferPool;
+
+ /**
+ * Indicate whether the {@link TieredStorageMemoryManagerImpl} is initialized. Before setting
+ * up, this field is false.
+ *
+ * <p>Note that before requesting buffers or getting the maximum allowed buffers, this
+ * initialized state should be checked.
+ */
+ private boolean isInitialized;
+
+ /**
+ * The constructor of the {@link TieredStorageMemoryManagerImpl}.
+ *
+ * @param numTriggerReclaimBuffersRatio the buffer pool usage ratio of requesting each tiered
+ * storage to reclaim buffers
+ * @param needPeriodicalCheckReclaimBuffer indicate whether it is necessary to start a
+ * periodically checking buffer reclaim thread
+ */
+ public TieredStorageMemoryManagerImpl(
+ float numTriggerReclaimBuffersRatio, boolean needPeriodicalCheckReclaimBuffer) {
+ this.numTriggerReclaimBuffersRatio = numTriggerReclaimBuffersRatio;
+ this.needPeriodicalCheckReclaimBuffer = needPeriodicalCheckReclaimBuffer;
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ this.bufferReclaimRequestListeners = new ArrayList<>();
+ this.isInitialized = false;
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs) {
+ this.bufferPool = bufferPool;
+ for (TieredStorageMemorySpec memorySpec : storageMemorySpecs) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalGuaranteedBuffers += memorySpec.getNumGuaranteedBuffers();
+ }
+
+ if (needPeriodicalCheckReclaimBuffer) {
+ this.executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("buffer reclaim checker")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+ }
+
+ this.isInitialized = true;
+ }
+
+ @Override
+ public void listenBufferReclaimRequest(Runnable onBufferReclaimRequest) {
+ bufferReclaimRequestListeners.add(onBufferReclaimRequest);
+ }
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link BufferPool} for a specific owner. The
+ * {@link TieredStorageMemoryManagerImpl} will not check whether a buffer can be requested and
+ * only record the total number of requested buffers. If the buffers in the {@link BufferPool}
+ * is not enough, this will request each tiered storage to reclaim the buffers as much as
+ * possible.
+ *
+ * <p>Note that synchronization is not necessary for this method, as it can only be called by
+ * the task thread.
+ *
+ * @return the requested buffer
+ */
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ checkIsInitialized();
+
+ tryReclaimBuffers();
+ CompletableFuture<MemorySegment> requestedBuffer =
+ CompletableFuture.supplyAsync(
+ () -> {
+ try {
+ return bufferPool.requestMemorySegmentBlocking();
+ } catch (InterruptedException e) {
+ throw new RuntimeException(e);
+ }
+ });
+
+ ScheduledFuture<?> scheduledFuture = getPeriodicalCheckReclaimBufferFuture();
+
+ MemorySegment memorySegment = null;
+ try {
+ memorySegment = requestedBuffer.get();
+ } catch (InterruptedException | ExecutionException e) {
+ ExceptionUtils.rethrow(e);
+ }
+
+ if (scheduledFuture != null) {
+ scheduledFuture.cancel(false);
Review Comment:
Instead of canceling the scheduled future, I'd suggest to check whether the request buffer future is completed in the periodical task.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,279 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_INITIAL_DELAY_MS = 10;
+
+ /** The period of checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** Listeners used to listen the requests for reclaiming buffer in different tiered storage. */
+ private final List<Runnable> bufferReclaimRequestListeners;
+
+ /** The buffer pool usage ratio of triggering the registered storages to reclaim buffers. */
+ private final float numTriggerReclaimBuffersRatio;
+
+ /**
+ * Indicate whether it is necessary to start a periodically checking buffer reclaim thread. If
+ * the memory manager is used in downstream, the field will be false because periodical buffer
+ * reclaim checker is needed.
+ */
+ private final boolean needPeriodicalCheckReclaimBuffer;
+
+ /**
+ * The number of requested buffers from {@link BufferPool}. Only this field can be touched both
+ * by the task thread and the netty thread, so it is an atomic type.
+ */
+ private final AtomicInteger numRequestedBuffers;
+
+ /**
+ * A thread to check whether to reclaim buffers from each tiered storage.
+ *
+ * <p>Note that it is not possible to remove this, as doing so could result in the task becoming
+ * stuck in the buffer request. As the number of buffers in the buffer pool can vary at any
+ * given time, the stuck may occur if the thread is removed.
+ *
+ * <p>For instance, if the memory usage of the {@link BufferPool} has been checked and {@link
+ * TieredStorageMemoryManagerImpl} determined that buffer reclamation is unnecessary, but then
+ * the buffer pool size is suddenly reduced to a very small size, the buffer request will become
+ * stuck and the task will never be able to call for buffer reclamation if this thread is
+ * removed, then a task stuck occurs.
+ */
+ private ScheduledExecutorService executor;
+
+ /** The total number of guaranteed buffers for all tiered storages. */
+ private int numTotalGuaranteedBuffers;
+
+ /** The buffer pool where the buffer is requested or recycled. */
+ private BufferPool bufferPool;
+
+ /**
+ * Indicate whether the {@link TieredStorageMemoryManagerImpl} is initialized. Before setting
+ * up, this field is false.
+ *
+ * <p>Note that before requesting buffers or getting the maximum allowed buffers, this
+ * initialized state should be checked.
+ */
+ private boolean isInitialized;
+
+ /**
+ * The constructor of the {@link TieredStorageMemoryManagerImpl}.
+ *
+ * @param numTriggerReclaimBuffersRatio the buffer pool usage ratio of requesting each tiered
+ * storage to reclaim buffers
+ * @param needPeriodicalCheckReclaimBuffer indicate whether it is necessary to start a
+ * periodically checking buffer reclaim thread
+ */
+ public TieredStorageMemoryManagerImpl(
+ float numTriggerReclaimBuffersRatio, boolean needPeriodicalCheckReclaimBuffer) {
+ this.numTriggerReclaimBuffersRatio = numTriggerReclaimBuffersRatio;
+ this.needPeriodicalCheckReclaimBuffer = needPeriodicalCheckReclaimBuffer;
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ this.bufferReclaimRequestListeners = new ArrayList<>();
+ this.isInitialized = false;
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs) {
+ this.bufferPool = bufferPool;
+ for (TieredStorageMemorySpec memorySpec : storageMemorySpecs) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalGuaranteedBuffers += memorySpec.getNumGuaranteedBuffers();
+ }
+
+ if (needPeriodicalCheckReclaimBuffer) {
+ this.executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("buffer reclaim checker")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+ }
+
+ this.isInitialized = true;
+ }
+
+ @Override
+ public void listenBufferReclaimRequest(Runnable onBufferReclaimRequest) {
+ bufferReclaimRequestListeners.add(onBufferReclaimRequest);
+ }
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link BufferPool} for a specific owner. The
+ * {@link TieredStorageMemoryManagerImpl} will not check whether a buffer can be requested and
+ * only record the total number of requested buffers. If the buffers in the {@link BufferPool}
+ * is not enough, this will request each tiered storage to reclaim the buffers as much as
+ * possible.
+ *
+ * <p>Note that synchronization is not necessary for this method, as it can only be called by
+ * the task thread.
+ *
+ * @return the requested buffer
+ */
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ checkIsInitialized();
+
+ tryReclaimBuffers();
+ CompletableFuture<MemorySegment> requestedBuffer =
+ CompletableFuture.supplyAsync(
Review Comment:
This involves an unnecessary thread. Creating an future and completes it after `bufferPool.requestMemorySegmentBlocking()` should be good enough.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,279 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_INITIAL_DELAY_MS = 10;
+
+ /** The period of checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** Listeners used to listen the requests for reclaiming buffer in different tiered storage. */
+ private final List<Runnable> bufferReclaimRequestListeners;
+
+ /** The buffer pool usage ratio of triggering the registered storages to reclaim buffers. */
+ private final float numTriggerReclaimBuffersRatio;
+
+ /**
+ * Indicate whether it is necessary to start a periodically checking buffer reclaim thread. If
+ * the memory manager is used in downstream, the field will be false because periodical buffer
+ * reclaim checker is needed.
+ */
+ private final boolean needPeriodicalCheckReclaimBuffer;
+
+ /**
+ * The number of requested buffers from {@link BufferPool}. Only this field can be touched both
+ * by the task thread and the netty thread, so it is an atomic type.
+ */
+ private final AtomicInteger numRequestedBuffers;
+
+ /**
+ * A thread to check whether to reclaim buffers from each tiered storage.
+ *
+ * <p>Note that it is not possible to remove this, as doing so could result in the task becoming
+ * stuck in the buffer request. As the number of buffers in the buffer pool can vary at any
+ * given time, the stuck may occur if the thread is removed.
+ *
+ * <p>For instance, if the memory usage of the {@link BufferPool} has been checked and {@link
+ * TieredStorageMemoryManagerImpl} determined that buffer reclamation is unnecessary, but then
+ * the buffer pool size is suddenly reduced to a very small size, the buffer request will become
+ * stuck and the task will never be able to call for buffer reclamation if this thread is
+ * removed, then a task stuck occurs.
+ */
+ private ScheduledExecutorService executor;
+
+ /** The total number of guaranteed buffers for all tiered storages. */
+ private int numTotalGuaranteedBuffers;
+
+ /** The buffer pool where the buffer is requested or recycled. */
+ private BufferPool bufferPool;
+
+ /**
+ * Indicate whether the {@link TieredStorageMemoryManagerImpl} is initialized. Before setting
+ * up, this field is false.
+ *
+ * <p>Note that before requesting buffers or getting the maximum allowed buffers, this
+ * initialized state should be checked.
+ */
+ private boolean isInitialized;
+
+ /**
+ * The constructor of the {@link TieredStorageMemoryManagerImpl}.
+ *
+ * @param numTriggerReclaimBuffersRatio the buffer pool usage ratio of requesting each tiered
+ * storage to reclaim buffers
+ * @param needPeriodicalCheckReclaimBuffer indicate whether it is necessary to start a
+ * periodically checking buffer reclaim thread
+ */
+ public TieredStorageMemoryManagerImpl(
+ float numTriggerReclaimBuffersRatio, boolean needPeriodicalCheckReclaimBuffer) {
+ this.numTriggerReclaimBuffersRatio = numTriggerReclaimBuffersRatio;
+ this.needPeriodicalCheckReclaimBuffer = needPeriodicalCheckReclaimBuffer;
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ this.bufferReclaimRequestListeners = new ArrayList<>();
+ this.isInitialized = false;
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs) {
+ this.bufferPool = bufferPool;
+ for (TieredStorageMemorySpec memorySpec : storageMemorySpecs) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalGuaranteedBuffers += memorySpec.getNumGuaranteedBuffers();
+ }
+
+ if (needPeriodicalCheckReclaimBuffer) {
+ this.executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("buffer reclaim checker")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+ }
+
+ this.isInitialized = true;
+ }
+
+ @Override
+ public void listenBufferReclaimRequest(Runnable onBufferReclaimRequest) {
+ bufferReclaimRequestListeners.add(onBufferReclaimRequest);
+ }
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link BufferPool} for a specific owner. The
+ * {@link TieredStorageMemoryManagerImpl} will not check whether a buffer can be requested and
+ * only record the total number of requested buffers. If the buffers in the {@link BufferPool}
+ * is not enough, this will request each tiered storage to reclaim the buffers as much as
+ * possible.
+ *
+ * <p>Note that synchronization is not necessary for this method, as it can only be called by
+ * the task thread.
+ *
+ * @return the requested buffer
+ */
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ checkIsInitialized();
+
+ tryReclaimBuffers();
+ CompletableFuture<MemorySegment> requestedBuffer =
+ CompletableFuture.supplyAsync(
+ () -> {
+ try {
+ return bufferPool.requestMemorySegmentBlocking();
+ } catch (InterruptedException e) {
+ throw new RuntimeException(e);
+ }
+ });
+
+ ScheduledFuture<?> scheduledFuture = getPeriodicalCheckReclaimBufferFuture();
+
+ MemorySegment memorySegment = null;
+ try {
+ memorySegment = requestedBuffer.get();
+ } catch (InterruptedException | ExecutionException e) {
+ ExceptionUtils.rethrow(e);
+ }
+
+ if (scheduledFuture != null) {
+ scheduledFuture.cancel(false);
+ }
+
+ numRequestedBuffers.incrementAndGet();
+ return new BufferBuilder(checkNotNull(memorySegment), this::recycleBuffer);
+ }
+
+ /**
+ * The synchronization is not necessary for this method, as it can only be called by the task
+ * thread.
+ *
+ * <p>The accuracy of the memory usage ratio may be compromised due to the varying buffer pool
+ * sizes. However, this only impacts a single iteration of the buffer usage check. Upon the next
+ * iteration, the buffer reclaim will eventually be triggered.
Review Comment:
Should be inline comments.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,279 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_INITIAL_DELAY_MS = 10;
+
+ /** The period of checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** Listeners used to listen the requests for reclaiming buffer in different tiered storage. */
+ private final List<Runnable> bufferReclaimRequestListeners;
+
+ /** The buffer pool usage ratio of triggering the registered storages to reclaim buffers. */
+ private final float numTriggerReclaimBuffersRatio;
+
+ /**
+ * Indicate whether it is necessary to start a periodically checking buffer reclaim thread. If
+ * the memory manager is used in downstream, the field will be false because periodical buffer
+ * reclaim checker is needed.
+ */
+ private final boolean needPeriodicalCheckReclaimBuffer;
+
+ /**
+ * The number of requested buffers from {@link BufferPool}. Only this field can be touched both
+ * by the task thread and the netty thread, so it is an atomic type.
+ */
+ private final AtomicInteger numRequestedBuffers;
+
+ /**
+ * A thread to check whether to reclaim buffers from each tiered storage.
+ *
+ * <p>Note that it is not possible to remove this, as doing so could result in the task becoming
+ * stuck in the buffer request. As the number of buffers in the buffer pool can vary at any
+ * given time, the stuck may occur if the thread is removed.
+ *
+ * <p>For instance, if the memory usage of the {@link BufferPool} has been checked and {@link
+ * TieredStorageMemoryManagerImpl} determined that buffer reclamation is unnecessary, but then
+ * the buffer pool size is suddenly reduced to a very small size, the buffer request will become
+ * stuck and the task will never be able to call for buffer reclamation if this thread is
+ * removed, then a task stuck occurs.
+ */
+ private ScheduledExecutorService executor;
+
+ /** The total number of guaranteed buffers for all tiered storages. */
+ private int numTotalGuaranteedBuffers;
+
+ /** The buffer pool where the buffer is requested or recycled. */
+ private BufferPool bufferPool;
+
+ /**
+ * Indicate whether the {@link TieredStorageMemoryManagerImpl} is initialized. Before setting
+ * up, this field is false.
+ *
+ * <p>Note that before requesting buffers or getting the maximum allowed buffers, this
+ * initialized state should be checked.
+ */
+ private boolean isInitialized;
+
+ /**
+ * The constructor of the {@link TieredStorageMemoryManagerImpl}.
+ *
+ * @param numTriggerReclaimBuffersRatio the buffer pool usage ratio of requesting each tiered
+ * storage to reclaim buffers
+ * @param needPeriodicalCheckReclaimBuffer indicate whether it is necessary to start a
+ * periodically checking buffer reclaim thread
+ */
+ public TieredStorageMemoryManagerImpl(
+ float numTriggerReclaimBuffersRatio, boolean needPeriodicalCheckReclaimBuffer) {
+ this.numTriggerReclaimBuffersRatio = numTriggerReclaimBuffersRatio;
+ this.needPeriodicalCheckReclaimBuffer = needPeriodicalCheckReclaimBuffer;
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ this.bufferReclaimRequestListeners = new ArrayList<>();
+ this.isInitialized = false;
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs) {
+ this.bufferPool = bufferPool;
+ for (TieredStorageMemorySpec memorySpec : storageMemorySpecs) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalGuaranteedBuffers += memorySpec.getNumGuaranteedBuffers();
+ }
+
+ if (needPeriodicalCheckReclaimBuffer) {
+ this.executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("buffer reclaim checker")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+ }
+
+ this.isInitialized = true;
+ }
+
+ @Override
+ public void listenBufferReclaimRequest(Runnable onBufferReclaimRequest) {
+ bufferReclaimRequestListeners.add(onBufferReclaimRequest);
+ }
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link BufferPool} for a specific owner. The
+ * {@link TieredStorageMemoryManagerImpl} will not check whether a buffer can be requested and
+ * only record the total number of requested buffers. If the buffers in the {@link BufferPool}
+ * is not enough, this will request each tiered storage to reclaim the buffers as much as
+ * possible.
+ *
+ * <p>Note that synchronization is not necessary for this method, as it can only be called by
+ * the task thread.
+ *
+ * @return the requested buffer
+ */
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ checkIsInitialized();
+
+ tryReclaimBuffers();
+ CompletableFuture<MemorySegment> requestedBuffer =
+ CompletableFuture.supplyAsync(
+ () -> {
+ try {
+ return bufferPool.requestMemorySegmentBlocking();
+ } catch (InterruptedException e) {
+ throw new RuntimeException(e);
+ }
+ });
+
+ ScheduledFuture<?> scheduledFuture = getPeriodicalCheckReclaimBufferFuture();
+
+ MemorySegment memorySegment = null;
+ try {
+ memorySegment = requestedBuffer.get();
+ } catch (InterruptedException | ExecutionException e) {
+ ExceptionUtils.rethrow(e);
+ }
+
+ if (scheduledFuture != null) {
+ scheduledFuture.cancel(false);
+ }
+
+ numRequestedBuffers.incrementAndGet();
+ return new BufferBuilder(checkNotNull(memorySegment), this::recycleBuffer);
+ }
+
+ /**
+ * The synchronization is not necessary for this method, as it can only be called by the task
+ * thread.
+ *
+ * <p>The accuracy of the memory usage ratio may be compromised due to the varying buffer pool
+ * sizes. However, this only impacts a single iteration of the buffer usage check. Upon the next
+ * iteration, the buffer reclaim will eventually be triggered.
+ */
+ @Override
+ public int getMaxNonReclaimableBuffers(Object owner) {
+ checkIsInitialized();
+
+ TieredStorageMemorySpec ownerMemorySpec = checkNotNull(tieredMemorySpecs.get(owner));
+ int ownerGuaranteedBuffers = ownerMemorySpec.getNumGuaranteedBuffers();
+ return bufferPool.getNumBuffers()
+ - numRequestedBuffers.get()
+ - numTotalGuaranteedBuffers
+ + ownerGuaranteedBuffers;
+ }
+
+ @Override
+ public void release() {
+ checkState(numRequestedBuffers.get() == 0, "Leaking buffers.");
+ if (executor != null) {
+ executor.shutdown();
+ try {
+ if (!executor.awaitTermination(5L, TimeUnit.MINUTES)) {
+ throw new TimeoutException(
+ "Timeout for shutting down the buffer reclaim checker executor.");
+ }
+ } catch (Exception e) {
+ ExceptionUtils.rethrow(e);
+ }
+ }
+ }
+
+ private void tryReclaimBuffers() {
+ if (shouldReclaimBuffers()) {
+ bufferReclaimRequestListeners.forEach(Runnable::run);
+ }
+ }
+
+ /**
+ * The accuracy of the memory usage ratio may be compromised due to the varying buffer pool
+ * sizes. However, this only impacts a single iteration of the buffer usage check. Upon the next
+ * iteration, the buffer reclaim will eventually be triggered.
+ */
+ private boolean shouldReclaimBuffers() {
+ int numTotal = bufferPool.getNumBuffers();
+ int numRequested = numRequestedBuffers.get();
+ return numRequested >= numTotal
+ // Because we do the checking before requesting buffers, we need add additional one
+ // buffer when calculating the usage ratio.
Review Comment:
Then let's name the method `shouldReclaimBuffersBeforeRequesting`
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,279 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_INITIAL_DELAY_MS = 10;
+
+ /** The period of checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** Listeners used to listen the requests for reclaiming buffer in different tiered storage. */
+ private final List<Runnable> bufferReclaimRequestListeners;
+
+ /** The buffer pool usage ratio of triggering the registered storages to reclaim buffers. */
+ private final float numTriggerReclaimBuffersRatio;
+
+ /**
+ * Indicate whether it is necessary to start a periodically checking buffer reclaim thread. If
+ * the memory manager is used in downstream, the field will be false because periodical buffer
+ * reclaim checker is needed.
+ */
+ private final boolean needPeriodicalCheckReclaimBuffer;
+
+ /**
+ * The number of requested buffers from {@link BufferPool}. Only this field can be touched both
+ * by the task thread and the netty thread, so it is an atomic type.
+ */
+ private final AtomicInteger numRequestedBuffers;
+
+ /**
+ * A thread to check whether to reclaim buffers from each tiered storage.
+ *
+ * <p>Note that it is not possible to remove this, as doing so could result in the task becoming
+ * stuck in the buffer request. As the number of buffers in the buffer pool can vary at any
+ * given time, the stuck may occur if the thread is removed.
+ *
+ * <p>For instance, if the memory usage of the {@link BufferPool} has been checked and {@link
+ * TieredStorageMemoryManagerImpl} determined that buffer reclamation is unnecessary, but then
+ * the buffer pool size is suddenly reduced to a very small size, the buffer request will become
+ * stuck and the task will never be able to call for buffer reclamation if this thread is
+ * removed, then a task stuck occurs.
+ */
+ private ScheduledExecutorService executor;
+
+ /** The total number of guaranteed buffers for all tiered storages. */
+ private int numTotalGuaranteedBuffers;
+
+ /** The buffer pool where the buffer is requested or recycled. */
+ private BufferPool bufferPool;
+
+ /**
+ * Indicate whether the {@link TieredStorageMemoryManagerImpl} is initialized. Before setting
+ * up, this field is false.
+ *
+ * <p>Note that before requesting buffers or getting the maximum allowed buffers, this
+ * initialized state should be checked.
+ */
+ private boolean isInitialized;
+
+ /**
+ * The constructor of the {@link TieredStorageMemoryManagerImpl}.
+ *
+ * @param numTriggerReclaimBuffersRatio the buffer pool usage ratio of requesting each tiered
+ * storage to reclaim buffers
+ * @param needPeriodicalCheckReclaimBuffer indicate whether it is necessary to start a
+ * periodically checking buffer reclaim thread
+ */
+ public TieredStorageMemoryManagerImpl(
+ float numTriggerReclaimBuffersRatio, boolean needPeriodicalCheckReclaimBuffer) {
+ this.numTriggerReclaimBuffersRatio = numTriggerReclaimBuffersRatio;
+ this.needPeriodicalCheckReclaimBuffer = needPeriodicalCheckReclaimBuffer;
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ this.bufferReclaimRequestListeners = new ArrayList<>();
+ this.isInitialized = false;
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs) {
+ this.bufferPool = bufferPool;
+ for (TieredStorageMemorySpec memorySpec : storageMemorySpecs) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalGuaranteedBuffers += memorySpec.getNumGuaranteedBuffers();
+ }
+
+ if (needPeriodicalCheckReclaimBuffer) {
+ this.executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("buffer reclaim checker")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+ }
+
+ this.isInitialized = true;
+ }
+
+ @Override
+ public void listenBufferReclaimRequest(Runnable onBufferReclaimRequest) {
+ bufferReclaimRequestListeners.add(onBufferReclaimRequest);
+ }
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link BufferPool} for a specific owner. The
+ * {@link TieredStorageMemoryManagerImpl} will not check whether a buffer can be requested and
+ * only record the total number of requested buffers. If the buffers in the {@link BufferPool}
+ * is not enough, this will request each tiered storage to reclaim the buffers as much as
+ * possible.
+ *
+ * <p>Note that synchronization is not necessary for this method, as it can only be called by
+ * the task thread.
+ *
+ * @return the requested buffer
+ */
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ checkIsInitialized();
+
+ tryReclaimBuffers();
+ CompletableFuture<MemorySegment> requestedBuffer =
+ CompletableFuture.supplyAsync(
+ () -> {
+ try {
+ return bufferPool.requestMemorySegmentBlocking();
+ } catch (InterruptedException e) {
+ throw new RuntimeException(e);
+ }
+ });
+
+ ScheduledFuture<?> scheduledFuture = getPeriodicalCheckReclaimBufferFuture();
+
+ MemorySegment memorySegment = null;
+ try {
+ memorySegment = requestedBuffer.get();
+ } catch (InterruptedException | ExecutionException e) {
+ ExceptionUtils.rethrow(e);
+ }
+
+ if (scheduledFuture != null) {
+ scheduledFuture.cancel(false);
+ }
+
+ numRequestedBuffers.incrementAndGet();
+ return new BufferBuilder(checkNotNull(memorySegment), this::recycleBuffer);
+ }
+
+ /**
+ * The synchronization is not necessary for this method, as it can only be called by the task
+ * thread.
+ *
+ * <p>The accuracy of the memory usage ratio may be compromised due to the varying buffer pool
+ * sizes. However, this only impacts a single iteration of the buffer usage check. Upon the next
+ * iteration, the buffer reclaim will eventually be triggered.
+ */
+ @Override
+ public int getMaxNonReclaimableBuffers(Object owner) {
+ checkIsInitialized();
+
+ TieredStorageMemorySpec ownerMemorySpec = checkNotNull(tieredMemorySpecs.get(owner));
+ int ownerGuaranteedBuffers = ownerMemorySpec.getNumGuaranteedBuffers();
+ return bufferPool.getNumBuffers()
+ - numRequestedBuffers.get()
+ - numTotalGuaranteedBuffers
+ + ownerGuaranteedBuffers;
+ }
+
+ @Override
+ public void release() {
+ checkState(numRequestedBuffers.get() == 0, "Leaking buffers.");
+ if (executor != null) {
+ executor.shutdown();
+ try {
+ if (!executor.awaitTermination(5L, TimeUnit.MINUTES)) {
+ throw new TimeoutException(
+ "Timeout for shutting down the buffer reclaim checker executor.");
+ }
+ } catch (Exception e) {
+ ExceptionUtils.rethrow(e);
+ }
+ }
+ }
+
+ private void tryReclaimBuffers() {
Review Comment:
```suggestion
private void reclaimBuffersIfNeeded() {
```
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,279 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_INITIAL_DELAY_MS = 10;
+
+ /** The period of checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** Listeners used to listen the requests for reclaiming buffer in different tiered storage. */
+ private final List<Runnable> bufferReclaimRequestListeners;
+
+ /** The buffer pool usage ratio of triggering the registered storages to reclaim buffers. */
+ private final float numTriggerReclaimBuffersRatio;
+
+ /**
+ * Indicate whether it is necessary to start a periodically checking buffer reclaim thread. If
+ * the memory manager is used in downstream, the field will be false because periodical buffer
+ * reclaim checker is needed.
+ */
+ private final boolean needPeriodicalCheckReclaimBuffer;
+
+ /**
+ * The number of requested buffers from {@link BufferPool}. Only this field can be touched both
+ * by the task thread and the netty thread, so it is an atomic type.
+ */
+ private final AtomicInteger numRequestedBuffers;
+
+ /**
+ * A thread to check whether to reclaim buffers from each tiered storage.
+ *
+ * <p>Note that it is not possible to remove this, as doing so could result in the task becoming
+ * stuck in the buffer request. As the number of buffers in the buffer pool can vary at any
+ * given time, the stuck may occur if the thread is removed.
+ *
+ * <p>For instance, if the memory usage of the {@link BufferPool} has been checked and {@link
+ * TieredStorageMemoryManagerImpl} determined that buffer reclamation is unnecessary, but then
+ * the buffer pool size is suddenly reduced to a very small size, the buffer request will become
+ * stuck and the task will never be able to call for buffer reclamation if this thread is
+ * removed, then a task stuck occurs.
+ */
+ private ScheduledExecutorService executor;
+
+ /** The total number of guaranteed buffers for all tiered storages. */
+ private int numTotalGuaranteedBuffers;
+
+ /** The buffer pool where the buffer is requested or recycled. */
+ private BufferPool bufferPool;
+
+ /**
+ * Indicate whether the {@link TieredStorageMemoryManagerImpl} is initialized. Before setting
+ * up, this field is false.
+ *
+ * <p>Note that before requesting buffers or getting the maximum allowed buffers, this
+ * initialized state should be checked.
+ */
+ private boolean isInitialized;
+
+ /**
+ * The constructor of the {@link TieredStorageMemoryManagerImpl}.
+ *
+ * @param numTriggerReclaimBuffersRatio the buffer pool usage ratio of requesting each tiered
+ * storage to reclaim buffers
+ * @param needPeriodicalCheckReclaimBuffer indicate whether it is necessary to start a
+ * periodically checking buffer reclaim thread
+ */
+ public TieredStorageMemoryManagerImpl(
+ float numTriggerReclaimBuffersRatio, boolean needPeriodicalCheckReclaimBuffer) {
+ this.numTriggerReclaimBuffersRatio = numTriggerReclaimBuffersRatio;
+ this.needPeriodicalCheckReclaimBuffer = needPeriodicalCheckReclaimBuffer;
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ this.bufferReclaimRequestListeners = new ArrayList<>();
+ this.isInitialized = false;
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs) {
+ this.bufferPool = bufferPool;
+ for (TieredStorageMemorySpec memorySpec : storageMemorySpecs) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalGuaranteedBuffers += memorySpec.getNumGuaranteedBuffers();
+ }
+
+ if (needPeriodicalCheckReclaimBuffer) {
+ this.executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("buffer reclaim checker")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+ }
+
+ this.isInitialized = true;
+ }
+
+ @Override
+ public void listenBufferReclaimRequest(Runnable onBufferReclaimRequest) {
+ bufferReclaimRequestListeners.add(onBufferReclaimRequest);
+ }
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link BufferPool} for a specific owner. The
+ * {@link TieredStorageMemoryManagerImpl} will not check whether a buffer can be requested and
+ * only record the total number of requested buffers. If the buffers in the {@link BufferPool}
+ * is not enough, this will request each tiered storage to reclaim the buffers as much as
+ * possible.
+ *
+ * <p>Note that synchronization is not necessary for this method, as it can only be called by
+ * the task thread.
+ *
+ * @return the requested buffer
+ */
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ checkIsInitialized();
+
+ tryReclaimBuffers();
+ CompletableFuture<MemorySegment> requestedBuffer =
+ CompletableFuture.supplyAsync(
+ () -> {
+ try {
+ return bufferPool.requestMemorySegmentBlocking();
+ } catch (InterruptedException e) {
+ throw new RuntimeException(e);
+ }
+ });
+
+ ScheduledFuture<?> scheduledFuture = getPeriodicalCheckReclaimBufferFuture();
+
+ MemorySegment memorySegment = null;
+ try {
+ memorySegment = requestedBuffer.get();
+ } catch (InterruptedException | ExecutionException e) {
+ ExceptionUtils.rethrow(e);
+ }
+
+ if (scheduledFuture != null) {
+ scheduledFuture.cancel(false);
+ }
+
+ numRequestedBuffers.incrementAndGet();
+ return new BufferBuilder(checkNotNull(memorySegment), this::recycleBuffer);
+ }
+
+ /**
+ * The synchronization is not necessary for this method, as it can only be called by the task
+ * thread.
+ *
+ * <p>The accuracy of the memory usage ratio may be compromised due to the varying buffer pool
+ * sizes. However, this only impacts a single iteration of the buffer usage check. Upon the next
+ * iteration, the buffer reclaim will eventually be triggered.
+ */
+ @Override
+ public int getMaxNonReclaimableBuffers(Object owner) {
+ checkIsInitialized();
+
+ TieredStorageMemorySpec ownerMemorySpec = checkNotNull(tieredMemorySpecs.get(owner));
+ int ownerGuaranteedBuffers = ownerMemorySpec.getNumGuaranteedBuffers();
+ return bufferPool.getNumBuffers()
+ - numRequestedBuffers.get()
+ - numTotalGuaranteedBuffers
+ + ownerGuaranteedBuffers;
Review Comment:
The calculation doesn't seem right.
- This implies all the requested buffers are non-guaranteed.
- The calculation result can be less than the owner's guaranteed buffer, or even negative.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193122816
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/CacheFlushManager.java:
##########
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.runtime.io.network.partition.hybrid.tiered.common.TieredStorageUtils;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+
+/**
+ * A manager to check whether the cached buffers need to be flushed.
+ *
+ * <p>Cached buffers refer to the buffers stored in a tier that can be released by flushing them to
+ * the disk or remote storage. If the requested buffers from the buffer pool reach the ratio limit,
+ * {@link CacheFlushManager} will trigger a process wherein all these buffers are flushed to the
+ * disk or remote storage to recycling these buffers.
+ *
+ * <p>Note that when initializing a tier with cached buffers, the tier should register a {@link
+ * CacheBufferFlushTrigger}, as a listener to flush cached buffers.
+ */
+public class CacheFlushManager {
+ private final float numBuffersTriggerFlushRatio;
+
+ private final List<CacheBufferFlushTrigger> flushTriggers;
+
+ private TieredStorageMemoryManager storageMemoryManager;
+
+ private final ScheduledExecutorService executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("cache flush trigger")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+
+ public CacheFlushManager(float numBuffersTriggerFlushRatio) {
+ this.numBuffersTriggerFlushRatio = numBuffersTriggerFlushRatio;
+ this.flushTriggers = new ArrayList<>();
+
+ executor.scheduleWithFixedDelay(
+ this::checkNeedTriggerFlushCachedBuffers, 10, 50, TimeUnit.MILLISECONDS);
+ }
+
+ public void setup(TieredStorageMemoryManager storageMemoryManager) {
+ this.storageMemoryManager = storageMemoryManager;
+ }
+
+ public void registerCacheBufferFlushTrigger(CacheBufferFlushTrigger cacheBufferFlushTrigger) {
+ flushTriggers.add(cacheBufferFlushTrigger);
+ }
+
+ public void triggerFlushCachedBuffers() {
+ flushTriggers.forEach(CacheBufferFlushTrigger::notifyFlushCachedBuffers);
+ }
+
+ public void checkNeedTriggerFlushCachedBuffers() {
Review Comment:
Fixed it. and I renamed it to `checkShouldFlushCachedBuffers`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193122290
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/common/TieredStorageUtils.java:
##########
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.common;
+
+import org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage.TieredStorageMemoryManager;
+
+/** Utils for reading or writing to tiered store. */
+public class TieredStorageUtils {
+ public static boolean needFlushCacheBuffers(
+ TieredStorageMemoryManager storageMemoryManager, float numBuffersTriggerFlushRatio) {
+ int numTotal = storageMemoryManager.numTotalBuffers();
+ int numRequested = storageMemoryManager.numRequestedBuffers();
Review Comment:
We have removed the two Apis of `numTotalBuffers ` and `numRequestedBuffers` because we have merged `TieredStorageMemoryManager` and `CacheFlushManager`. So we do not need expose the two args any more.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193822668
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManager.java:
##########
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+
+import java.util.List;
+
+/**
+ * The {@link TieredStorageMemoryManager} is to request or recycle buffers from {@link
+ * LocalBufferPool} for different memory owners, for example, the tiers, the buffer accumulator,
+ * etc. Note that the logic for requesting and recycling buffers is consistent for these owners.
+ *
+ * <p>The memory managed by {@link TieredStorageMemoryManager} is categorized into two types:
+ * long-term occupied memory which cannot be immediately released and short-term occupied memory
+ * which can be reclaimed quickly and safely. Long-term occupied memory usage necessitates waiting
+ * for other operations to complete before releasing it, such as downstream consumption. On the
+ * other hand, short-term occupied memory can be freed up at any time, enabling rapid memory
+ * recycling for tasks such as flushing memory to disk or remote storage.
+ *
+ * <p>This {@link TieredStorageMemoryManager} aim to streamline and harmonize memory management
+ * across various layers. Instead of tracking the number of buffers utilized by individual users, it
+ * dynamically calculates a user's maximum guaranteed amount based on the current status of the
+ * manager and the local buffer pool. Specifically, if a user is a long-term occupied memory user,
+ * the {@link TieredStorageMemoryManager} does not limit the user's memory usage, while if a user is
+ * a short-term occupied memory user, the current guaranteed buffers of the user is the left buffers
+ * in the buffer pool - guaranteed amount of other users (excluding the current user).
+ */
+public interface TieredStorageMemoryManager {
+
+ /**
+ * Setup the {@link TieredStorageMemoryManager}. When setting up the manager, the {@link
+ * TieredStorageMemorySpec}s for different tiered storages should be ready to indicate each
+ * tiered storage's memory requirement specs.
+ *
+ * @param bufferPool the local buffer pool
+ * @param storageMemorySpecs the memory specs for different tiered storages
+ */
+ void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs);
+
+ /**
+ * Register a buffer flush call back to flush and recycle all the memory in the user when
+ * needed.
+ *
+ * <p>When the left buffers in the {@link BufferPool} are not enough, {@link
+ * TieredStorageMemoryManager} will flush the buffers of the user with the registered callback.
+ *
+ * @param userBufferFlushCallback the buffer flush call back of the memory user
+ */
+ void registerBufferFlushCallback(Runnable userBufferFlushCallback);
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link LocalBufferPool} for a specific owner.
+ *
+ * @return the requested buffer
+ */
+ BufferBuilder requestBufferBlocking();
+
+ /**
+ * Return the available buffers for the owner.
+ *
+ * <p>Note that the available buffers are calculated dynamically based on some conditions, for
+ * example, the state of the {@link BufferPool}, the {@link TieredStorageMemorySpec} of the
+ * owner, etc.
+ *
+ * <p>If a user is a long-term occupied memory user, the {@link TieredStorageMemoryManager} does
+ * not limit the user's memory usage, while if a user is a short-term occupied memory user, the
+ * max allowed buffers of the user is the left buffers in the buffer pool - guaranteed amount of
+ * other users (excluding the current user).
Review Comment:
Removed the sentence and rewrite the doc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193832953
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,224 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking flush. */
+ public static final int DEFAULT_CHECK_FLUSH_INITIAL_DELAY_MS = 10;
+
+ /** Check flush period. */
+ public static final int DEFAULT_CHECK_FLUSH_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** The registered callbacks to flush the buffers in the registered tiered storages. */
+ private final List<Runnable> bufferFlushCallbacks;
+
+ /** The buffer pool usage ratio of triggering the registered storages to flush buffers. */
+ private final float numBuffersTriggerFlushRatio;
+
+ /**
+ * Indicate whether to start the buffer flush checker thread. If the memory manager is used in
+ * downstream, the field will be false because no buffer flush checker is needed.
+ */
+ private final boolean shouldStartBufferFlushChecker;
+
+ /** The number of requested buffers from {@link BufferPool}. */
+ private final AtomicInteger numRequestedBuffers;
+
+ /** A thread to check whether to flush buffers in each tiered storage. */
+ private ScheduledExecutorService executor;
+
+ /** The total number of guaranteed buffers for all tiered storages. */
+ private int numTotalGuaranteedBuffers;
+
+ /** The buffer pool where the buffer is requested or recyceld. */
+ private BufferPool bufferPool;
+
+ /**
+ * Indicate whether the {@link TieredStorageMemoryManagerImpl} is in running state. Before
+ * setting up, this field is false.
+ *
+ * <p>Note that before requesting buffers or getting the maximum allowed buffers, this running
+ * state should be checked.
+ */
+ private boolean isRunning;
+
+ /**
+ * The constructor of the {@link TieredStorageMemoryManagerImpl}.
+ *
+ * @param numBuffersTriggerFlushRatio the buffer pool usage ratio of triggering each tiered
+ * storage to flush buffers
+ * @param shouldStartBufferFlushChecker indicate whether to start the buffer flushing checker
+ * thread
+ */
+ public TieredStorageMemoryManagerImpl(
+ float numBuffersTriggerFlushRatio, boolean shouldStartBufferFlushChecker) {
+ this.numBuffersTriggerFlushRatio = numBuffersTriggerFlushRatio;
+ this.shouldStartBufferFlushChecker = shouldStartBufferFlushChecker;
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ this.bufferFlushCallbacks = new ArrayList<>();
+ this.isRunning = false;
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs) {
+ this.bufferPool = bufferPool;
+ for (TieredStorageMemorySpec memorySpec : storageMemorySpecs) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalGuaranteedBuffers += memorySpec.getNumGuaranteedBuffers();
+ }
+
+ if (shouldStartBufferFlushChecker) {
+ this.executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("buffer flush checker")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+ this.executor.scheduleWithFixedDelay(
+ this::checkShouldFlushCachedBuffers,
+ DEFAULT_CHECK_FLUSH_INITIAL_DELAY_MS,
+ DEFAULT_CHECK_FLUSH_PERIOD_DURATION_MS,
+ TimeUnit.MILLISECONDS);
+ }
+
+ this.isRunning = true;
+ }
+
+ @Override
+ public void registerBufferFlushCallback(Runnable userBufferFlushCallBack) {
+ bufferFlushCallbacks.add(userBufferFlushCallBack);
+ }
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link BufferPool} for a specific owner. The
+ * {@link TieredStorageMemoryManagerImpl} will not check whether a buffer can be requested and
+ * only record the total number of requested buffers. If the buffers in the {@link BufferPool}
+ * is not enough, this will trigger each tiered storage to flush buffers as much as possible.
+ *
+ * @return the requested buffer
+ */
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ checkIsRunning();
+
+ MemorySegment requestedBuffer = null;
+ try {
+ requestedBuffer = bufferPool.requestMemorySegmentBlocking();
+ } catch (Throwable throwable) {
+ ExceptionUtils.rethrow(throwable, "Failed to request memory segments.");
+ }
+ numRequestedBuffers.incrementAndGet();
+ checkShouldFlushCachedBuffers();
Review Comment:
Move this before requesting. But that may not make sure the buffer reclaim occurs, so we need a checking thread.
Note that it is **not** possible to remove the checking thread, as doing so could result in the task becoming stuck in the buffer request. As the number of buffers in the buffer pool can vary at any given time, the stuck may occur if the thread is removed.
For instance, if the memory usage of the {@link BufferPool} has been checked and {@link TieredStorageMemoryManagerImpl} determined that buffer reclamation is unnecessary, but then the buffer pool size is suddenly reduced to a very small size, the buffer request will become stuck and the task will never be able to call for buffer reclamation if this thread is removed, then a task stuck occurs.
Java doc also added these descriptions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1200337852
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,279 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_INITIAL_DELAY_MS = 10;
+
+ /** The period of checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** Listeners used to listen the requests for reclaiming buffer in different tiered storage. */
+ private final List<Runnable> bufferReclaimRequestListeners;
+
+ /** The buffer pool usage ratio of triggering the registered storages to reclaim buffers. */
+ private final float numTriggerReclaimBuffersRatio;
+
+ /**
+ * Indicate whether it is necessary to start a periodically checking buffer reclaim thread. If
+ * the memory manager is used in downstream, the field will be false because periodical buffer
+ * reclaim checker is needed.
+ */
+ private final boolean needPeriodicalCheckReclaimBuffer;
+
+ /**
+ * The number of requested buffers from {@link BufferPool}. Only this field can be touched both
+ * by the task thread and the netty thread, so it is an atomic type.
+ */
+ private final AtomicInteger numRequestedBuffers;
+
+ /**
+ * A thread to check whether to reclaim buffers from each tiered storage.
+ *
+ * <p>Note that it is not possible to remove this, as doing so could result in the task becoming
+ * stuck in the buffer request. As the number of buffers in the buffer pool can vary at any
+ * given time, the stuck may occur if the thread is removed.
+ *
+ * <p>For instance, if the memory usage of the {@link BufferPool} has been checked and {@link
+ * TieredStorageMemoryManagerImpl} determined that buffer reclamation is unnecessary, but then
+ * the buffer pool size is suddenly reduced to a very small size, the buffer request will become
+ * stuck and the task will never be able to call for buffer reclamation if this thread is
+ * removed, then a task stuck occurs.
+ */
+ private ScheduledExecutorService executor;
+
+ /** The total number of guaranteed buffers for all tiered storages. */
+ private int numTotalGuaranteedBuffers;
+
+ /** The buffer pool where the buffer is requested or recycled. */
+ private BufferPool bufferPool;
+
+ /**
+ * Indicate whether the {@link TieredStorageMemoryManagerImpl} is initialized. Before setting
+ * up, this field is false.
+ *
+ * <p>Note that before requesting buffers or getting the maximum allowed buffers, this
+ * initialized state should be checked.
+ */
+ private boolean isInitialized;
+
+ /**
+ * The constructor of the {@link TieredStorageMemoryManagerImpl}.
+ *
+ * @param numTriggerReclaimBuffersRatio the buffer pool usage ratio of requesting each tiered
+ * storage to reclaim buffers
+ * @param needPeriodicalCheckReclaimBuffer indicate whether it is necessary to start a
+ * periodically checking buffer reclaim thread
+ */
+ public TieredStorageMemoryManagerImpl(
+ float numTriggerReclaimBuffersRatio, boolean needPeriodicalCheckReclaimBuffer) {
+ this.numTriggerReclaimBuffersRatio = numTriggerReclaimBuffersRatio;
+ this.needPeriodicalCheckReclaimBuffer = needPeriodicalCheckReclaimBuffer;
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ this.bufferReclaimRequestListeners = new ArrayList<>();
+ this.isInitialized = false;
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs) {
+ this.bufferPool = bufferPool;
+ for (TieredStorageMemorySpec memorySpec : storageMemorySpecs) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalGuaranteedBuffers += memorySpec.getNumGuaranteedBuffers();
+ }
+
+ if (needPeriodicalCheckReclaimBuffer) {
+ this.executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("buffer reclaim checker")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+ }
+
+ this.isInitialized = true;
+ }
+
+ @Override
+ public void listenBufferReclaimRequest(Runnable onBufferReclaimRequest) {
+ bufferReclaimRequestListeners.add(onBufferReclaimRequest);
+ }
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link BufferPool} for a specific owner. The
+ * {@link TieredStorageMemoryManagerImpl} will not check whether a buffer can be requested and
+ * only record the total number of requested buffers. If the buffers in the {@link BufferPool}
+ * is not enough, this will request each tiered storage to reclaim the buffers as much as
+ * possible.
+ *
+ * <p>Note that synchronization is not necessary for this method, as it can only be called by
+ * the task thread.
+ *
+ * @return the requested buffer
+ */
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ checkIsInitialized();
+
+ tryReclaimBuffers();
+ CompletableFuture<MemorySegment> requestedBuffer =
+ CompletableFuture.supplyAsync(
+ () -> {
+ try {
+ return bufferPool.requestMemorySegmentBlocking();
+ } catch (InterruptedException e) {
+ throw new RuntimeException(e);
+ }
+ });
+
+ ScheduledFuture<?> scheduledFuture = getPeriodicalCheckReclaimBufferFuture();
+
+ MemorySegment memorySegment = null;
+ try {
+ memorySegment = requestedBuffer.get();
+ } catch (InterruptedException | ExecutionException e) {
+ ExceptionUtils.rethrow(e);
+ }
+
+ if (scheduledFuture != null) {
+ scheduledFuture.cancel(false);
+ }
+
+ numRequestedBuffers.incrementAndGet();
+ return new BufferBuilder(checkNotNull(memorySegment), this::recycleBuffer);
+ }
+
+ /**
+ * The synchronization is not necessary for this method, as it can only be called by the task
+ * thread.
+ *
+ * <p>The accuracy of the memory usage ratio may be compromised due to the varying buffer pool
+ * sizes. However, this only impacts a single iteration of the buffer usage check. Upon the next
+ * iteration, the buffer reclaim will eventually be triggered.
+ */
+ @Override
+ public int getMaxNonReclaimableBuffers(Object owner) {
+ checkIsInitialized();
+
+ TieredStorageMemorySpec ownerMemorySpec = checkNotNull(tieredMemorySpecs.get(owner));
+ int ownerGuaranteedBuffers = ownerMemorySpec.getNumGuaranteedBuffers();
+ return bufferPool.getNumBuffers()
+ - numRequestedBuffers.get()
+ - numTotalGuaranteedBuffers
+ + ownerGuaranteedBuffers;
Review Comment:
Refactoring it by returning `bufferPool.getNumBuffers() - numBuffersOfOtherOwners`.
The buffers of other owner (except for the request owner itself) is calculated by `Math.max(numGuaranteed, numRequested)`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193822924
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,224 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking flush. */
+ public static final int DEFAULT_CHECK_FLUSH_INITIAL_DELAY_MS = 10;
+
+ /** Check flush period. */
+ public static final int DEFAULT_CHECK_FLUSH_PERIOD_DURATION_MS = 50;
Review Comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1200333899
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,279 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_INITIAL_DELAY_MS = 10;
+
+ /** The period of checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** Listeners used to listen the requests for reclaiming buffer in different tiered storage. */
+ private final List<Runnable> bufferReclaimRequestListeners;
+
+ /** The buffer pool usage ratio of triggering the registered storages to reclaim buffers. */
+ private final float numTriggerReclaimBuffersRatio;
+
+ /**
+ * Indicate whether it is necessary to start a periodically checking buffer reclaim thread. If
+ * the memory manager is used in downstream, the field will be false because periodical buffer
+ * reclaim checker is needed.
+ */
+ private final boolean needPeriodicalCheckReclaimBuffer;
+
+ /**
+ * The number of requested buffers from {@link BufferPool}. Only this field can be touched both
+ * by the task thread and the netty thread, so it is an atomic type.
+ */
+ private final AtomicInteger numRequestedBuffers;
+
+ /**
+ * A thread to check whether to reclaim buffers from each tiered storage.
+ *
+ * <p>Note that it is not possible to remove this, as doing so could result in the task becoming
+ * stuck in the buffer request. As the number of buffers in the buffer pool can vary at any
+ * given time, the stuck may occur if the thread is removed.
+ *
+ * <p>For instance, if the memory usage of the {@link BufferPool} has been checked and {@link
+ * TieredStorageMemoryManagerImpl} determined that buffer reclamation is unnecessary, but then
+ * the buffer pool size is suddenly reduced to a very small size, the buffer request will become
+ * stuck and the task will never be able to call for buffer reclamation if this thread is
+ * removed, then a task stuck occurs.
+ */
+ private ScheduledExecutorService executor;
+
+ /** The total number of guaranteed buffers for all tiered storages. */
+ private int numTotalGuaranteedBuffers;
+
+ /** The buffer pool where the buffer is requested or recycled. */
+ private BufferPool bufferPool;
+
+ /**
+ * Indicate whether the {@link TieredStorageMemoryManagerImpl} is initialized. Before setting
+ * up, this field is false.
+ *
+ * <p>Note that before requesting buffers or getting the maximum allowed buffers, this
+ * initialized state should be checked.
+ */
+ private boolean isInitialized;
+
+ /**
+ * The constructor of the {@link TieredStorageMemoryManagerImpl}.
+ *
+ * @param numTriggerReclaimBuffersRatio the buffer pool usage ratio of requesting each tiered
+ * storage to reclaim buffers
+ * @param needPeriodicalCheckReclaimBuffer indicate whether it is necessary to start a
+ * periodically checking buffer reclaim thread
+ */
+ public TieredStorageMemoryManagerImpl(
+ float numTriggerReclaimBuffersRatio, boolean needPeriodicalCheckReclaimBuffer) {
+ this.numTriggerReclaimBuffersRatio = numTriggerReclaimBuffersRatio;
+ this.needPeriodicalCheckReclaimBuffer = needPeriodicalCheckReclaimBuffer;
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ this.bufferReclaimRequestListeners = new ArrayList<>();
+ this.isInitialized = false;
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs) {
+ this.bufferPool = bufferPool;
+ for (TieredStorageMemorySpec memorySpec : storageMemorySpecs) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalGuaranteedBuffers += memorySpec.getNumGuaranteedBuffers();
+ }
+
+ if (needPeriodicalCheckReclaimBuffer) {
+ this.executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("buffer reclaim checker")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+ }
+
+ this.isInitialized = true;
+ }
+
+ @Override
+ public void listenBufferReclaimRequest(Runnable onBufferReclaimRequest) {
+ bufferReclaimRequestListeners.add(onBufferReclaimRequest);
+ }
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link BufferPool} for a specific owner. The
+ * {@link TieredStorageMemoryManagerImpl} will not check whether a buffer can be requested and
+ * only record the total number of requested buffers. If the buffers in the {@link BufferPool}
+ * is not enough, this will request each tiered storage to reclaim the buffers as much as
+ * possible.
+ *
+ * <p>Note that synchronization is not necessary for this method, as it can only be called by
+ * the task thread.
+ *
+ * @return the requested buffer
+ */
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ checkIsInitialized();
+
+ tryReclaimBuffers();
+ CompletableFuture<MemorySegment> requestedBuffer =
+ CompletableFuture.supplyAsync(
Review Comment:
Good catch. Fixed by a `CompletableFuture<Void>`.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,279 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_INITIAL_DELAY_MS = 10;
+
+ /** The period of checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** Listeners used to listen the requests for reclaiming buffer in different tiered storage. */
+ private final List<Runnable> bufferReclaimRequestListeners;
+
+ /** The buffer pool usage ratio of triggering the registered storages to reclaim buffers. */
+ private final float numTriggerReclaimBuffersRatio;
+
+ /**
+ * Indicate whether it is necessary to start a periodically checking buffer reclaim thread. If
+ * the memory manager is used in downstream, the field will be false because periodical buffer
+ * reclaim checker is needed.
+ */
+ private final boolean needPeriodicalCheckReclaimBuffer;
+
+ /**
+ * The number of requested buffers from {@link BufferPool}. Only this field can be touched both
+ * by the task thread and the netty thread, so it is an atomic type.
+ */
+ private final AtomicInteger numRequestedBuffers;
+
+ /**
+ * A thread to check whether to reclaim buffers from each tiered storage.
+ *
+ * <p>Note that it is not possible to remove this, as doing so could result in the task becoming
+ * stuck in the buffer request. As the number of buffers in the buffer pool can vary at any
+ * given time, the stuck may occur if the thread is removed.
+ *
+ * <p>For instance, if the memory usage of the {@link BufferPool} has been checked and {@link
+ * TieredStorageMemoryManagerImpl} determined that buffer reclamation is unnecessary, but then
+ * the buffer pool size is suddenly reduced to a very small size, the buffer request will become
+ * stuck and the task will never be able to call for buffer reclamation if this thread is
+ * removed, then a task stuck occurs.
+ */
+ private ScheduledExecutorService executor;
+
+ /** The total number of guaranteed buffers for all tiered storages. */
+ private int numTotalGuaranteedBuffers;
+
+ /** The buffer pool where the buffer is requested or recycled. */
+ private BufferPool bufferPool;
+
+ /**
+ * Indicate whether the {@link TieredStorageMemoryManagerImpl} is initialized. Before setting
+ * up, this field is false.
+ *
+ * <p>Note that before requesting buffers or getting the maximum allowed buffers, this
+ * initialized state should be checked.
+ */
+ private boolean isInitialized;
+
+ /**
+ * The constructor of the {@link TieredStorageMemoryManagerImpl}.
+ *
+ * @param numTriggerReclaimBuffersRatio the buffer pool usage ratio of requesting each tiered
+ * storage to reclaim buffers
+ * @param needPeriodicalCheckReclaimBuffer indicate whether it is necessary to start a
+ * periodically checking buffer reclaim thread
+ */
+ public TieredStorageMemoryManagerImpl(
+ float numTriggerReclaimBuffersRatio, boolean needPeriodicalCheckReclaimBuffer) {
+ this.numTriggerReclaimBuffersRatio = numTriggerReclaimBuffersRatio;
+ this.needPeriodicalCheckReclaimBuffer = needPeriodicalCheckReclaimBuffer;
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ this.bufferReclaimRequestListeners = new ArrayList<>();
+ this.isInitialized = false;
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs) {
+ this.bufferPool = bufferPool;
+ for (TieredStorageMemorySpec memorySpec : storageMemorySpecs) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalGuaranteedBuffers += memorySpec.getNumGuaranteedBuffers();
+ }
+
+ if (needPeriodicalCheckReclaimBuffer) {
+ this.executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("buffer reclaim checker")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+ }
+
+ this.isInitialized = true;
+ }
+
+ @Override
+ public void listenBufferReclaimRequest(Runnable onBufferReclaimRequest) {
+ bufferReclaimRequestListeners.add(onBufferReclaimRequest);
+ }
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link BufferPool} for a specific owner. The
+ * {@link TieredStorageMemoryManagerImpl} will not check whether a buffer can be requested and
+ * only record the total number of requested buffers. If the buffers in the {@link BufferPool}
+ * is not enough, this will request each tiered storage to reclaim the buffers as much as
+ * possible.
+ *
+ * <p>Note that synchronization is not necessary for this method, as it can only be called by
+ * the task thread.
+ *
+ * @return the requested buffer
+ */
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ checkIsInitialized();
+
+ tryReclaimBuffers();
+ CompletableFuture<MemorySegment> requestedBuffer =
+ CompletableFuture.supplyAsync(
+ () -> {
+ try {
+ return bufferPool.requestMemorySegmentBlocking();
+ } catch (InterruptedException e) {
+ throw new RuntimeException(e);
+ }
+ });
+
+ ScheduledFuture<?> scheduledFuture = getPeriodicalCheckReclaimBufferFuture();
+
+ MemorySegment memorySegment = null;
+ try {
+ memorySegment = requestedBuffer.get();
+ } catch (InterruptedException | ExecutionException e) {
+ ExceptionUtils.rethrow(e);
+ }
+
+ if (scheduledFuture != null) {
+ scheduledFuture.cancel(false);
+ }
+
+ numRequestedBuffers.incrementAndGet();
+ return new BufferBuilder(checkNotNull(memorySegment), this::recycleBuffer);
+ }
+
+ /**
+ * The synchronization is not necessary for this method, as it can only be called by the task
+ * thread.
+ *
+ * <p>The accuracy of the memory usage ratio may be compromised due to the varying buffer pool
+ * sizes. However, this only impacts a single iteration of the buffer usage check. Upon the next
+ * iteration, the buffer reclaim will eventually be triggered.
+ */
+ @Override
+ public int getMaxNonReclaimableBuffers(Object owner) {
+ checkIsInitialized();
+
+ TieredStorageMemorySpec ownerMemorySpec = checkNotNull(tieredMemorySpecs.get(owner));
+ int ownerGuaranteedBuffers = ownerMemorySpec.getNumGuaranteedBuffers();
+ return bufferPool.getNumBuffers()
+ - numRequestedBuffers.get()
+ - numTotalGuaranteedBuffers
+ + ownerGuaranteedBuffers;
+ }
+
+ @Override
+ public void release() {
+ checkState(numRequestedBuffers.get() == 0, "Leaking buffers.");
+ if (executor != null) {
+ executor.shutdown();
+ try {
+ if (!executor.awaitTermination(5L, TimeUnit.MINUTES)) {
+ throw new TimeoutException(
+ "Timeout for shutting down the buffer reclaim checker executor.");
+ }
+ } catch (Exception e) {
+ ExceptionUtils.rethrow(e);
+ }
+ }
+ }
+
+ private void tryReclaimBuffers() {
Review Comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1200337852
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,279 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_INITIAL_DELAY_MS = 10;
+
+ /** The period of checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** Listeners used to listen the requests for reclaiming buffer in different tiered storage. */
+ private final List<Runnable> bufferReclaimRequestListeners;
+
+ /** The buffer pool usage ratio of triggering the registered storages to reclaim buffers. */
+ private final float numTriggerReclaimBuffersRatio;
+
+ /**
+ * Indicate whether it is necessary to start a periodically checking buffer reclaim thread. If
+ * the memory manager is used in downstream, the field will be false because periodical buffer
+ * reclaim checker is needed.
+ */
+ private final boolean needPeriodicalCheckReclaimBuffer;
+
+ /**
+ * The number of requested buffers from {@link BufferPool}. Only this field can be touched both
+ * by the task thread and the netty thread, so it is an atomic type.
+ */
+ private final AtomicInteger numRequestedBuffers;
+
+ /**
+ * A thread to check whether to reclaim buffers from each tiered storage.
+ *
+ * <p>Note that it is not possible to remove this, as doing so could result in the task becoming
+ * stuck in the buffer request. As the number of buffers in the buffer pool can vary at any
+ * given time, the stuck may occur if the thread is removed.
+ *
+ * <p>For instance, if the memory usage of the {@link BufferPool} has been checked and {@link
+ * TieredStorageMemoryManagerImpl} determined that buffer reclamation is unnecessary, but then
+ * the buffer pool size is suddenly reduced to a very small size, the buffer request will become
+ * stuck and the task will never be able to call for buffer reclamation if this thread is
+ * removed, then a task stuck occurs.
+ */
+ private ScheduledExecutorService executor;
+
+ /** The total number of guaranteed buffers for all tiered storages. */
+ private int numTotalGuaranteedBuffers;
+
+ /** The buffer pool where the buffer is requested or recycled. */
+ private BufferPool bufferPool;
+
+ /**
+ * Indicate whether the {@link TieredStorageMemoryManagerImpl} is initialized. Before setting
+ * up, this field is false.
+ *
+ * <p>Note that before requesting buffers or getting the maximum allowed buffers, this
+ * initialized state should be checked.
+ */
+ private boolean isInitialized;
+
+ /**
+ * The constructor of the {@link TieredStorageMemoryManagerImpl}.
+ *
+ * @param numTriggerReclaimBuffersRatio the buffer pool usage ratio of requesting each tiered
+ * storage to reclaim buffers
+ * @param needPeriodicalCheckReclaimBuffer indicate whether it is necessary to start a
+ * periodically checking buffer reclaim thread
+ */
+ public TieredStorageMemoryManagerImpl(
+ float numTriggerReclaimBuffersRatio, boolean needPeriodicalCheckReclaimBuffer) {
+ this.numTriggerReclaimBuffersRatio = numTriggerReclaimBuffersRatio;
+ this.needPeriodicalCheckReclaimBuffer = needPeriodicalCheckReclaimBuffer;
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ this.bufferReclaimRequestListeners = new ArrayList<>();
+ this.isInitialized = false;
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs) {
+ this.bufferPool = bufferPool;
+ for (TieredStorageMemorySpec memorySpec : storageMemorySpecs) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalGuaranteedBuffers += memorySpec.getNumGuaranteedBuffers();
+ }
+
+ if (needPeriodicalCheckReclaimBuffer) {
+ this.executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("buffer reclaim checker")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+ }
+
+ this.isInitialized = true;
+ }
+
+ @Override
+ public void listenBufferReclaimRequest(Runnable onBufferReclaimRequest) {
+ bufferReclaimRequestListeners.add(onBufferReclaimRequest);
+ }
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link BufferPool} for a specific owner. The
+ * {@link TieredStorageMemoryManagerImpl} will not check whether a buffer can be requested and
+ * only record the total number of requested buffers. If the buffers in the {@link BufferPool}
+ * is not enough, this will request each tiered storage to reclaim the buffers as much as
+ * possible.
+ *
+ * <p>Note that synchronization is not necessary for this method, as it can only be called by
+ * the task thread.
+ *
+ * @return the requested buffer
+ */
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ checkIsInitialized();
+
+ tryReclaimBuffers();
+ CompletableFuture<MemorySegment> requestedBuffer =
+ CompletableFuture.supplyAsync(
+ () -> {
+ try {
+ return bufferPool.requestMemorySegmentBlocking();
+ } catch (InterruptedException e) {
+ throw new RuntimeException(e);
+ }
+ });
+
+ ScheduledFuture<?> scheduledFuture = getPeriodicalCheckReclaimBufferFuture();
+
+ MemorySegment memorySegment = null;
+ try {
+ memorySegment = requestedBuffer.get();
+ } catch (InterruptedException | ExecutionException e) {
+ ExceptionUtils.rethrow(e);
+ }
+
+ if (scheduledFuture != null) {
+ scheduledFuture.cancel(false);
+ }
+
+ numRequestedBuffers.incrementAndGet();
+ return new BufferBuilder(checkNotNull(memorySegment), this::recycleBuffer);
+ }
+
+ /**
+ * The synchronization is not necessary for this method, as it can only be called by the task
+ * thread.
+ *
+ * <p>The accuracy of the memory usage ratio may be compromised due to the varying buffer pool
+ * sizes. However, this only impacts a single iteration of the buffer usage check. Upon the next
+ * iteration, the buffer reclaim will eventually be triggered.
+ */
+ @Override
+ public int getMaxNonReclaimableBuffers(Object owner) {
+ checkIsInitialized();
+
+ TieredStorageMemorySpec ownerMemorySpec = checkNotNull(tieredMemorySpecs.get(owner));
+ int ownerGuaranteedBuffers = ownerMemorySpec.getNumGuaranteedBuffers();
+ return bufferPool.getNumBuffers()
+ - numRequestedBuffers.get()
+ - numTotalGuaranteedBuffers
+ + ownerGuaranteedBuffers;
Review Comment:
Refactoring it by returning `bufferPool.getNumBuffers() - numBuffersOfOtherOwners`.
The buffers of another owner (except for the request owner itself) is calculated by `Math.max(numGuaranteed, numRequested)`.
And I added some comments on why it may return a negative value. Added some comments on why an owner may occupy others' guaranteed buffers occassionly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1200340333
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,279 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_INITIAL_DELAY_MS = 10;
+
+ /** The period of checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** Listeners used to listen the requests for reclaiming buffer in different tiered storage. */
+ private final List<Runnable> bufferReclaimRequestListeners;
+
+ /** The buffer pool usage ratio of triggering the registered storages to reclaim buffers. */
+ private final float numTriggerReclaimBuffersRatio;
+
+ /**
+ * Indicate whether it is necessary to start a periodically checking buffer reclaim thread. If
+ * the memory manager is used in downstream, the field will be false because periodical buffer
+ * reclaim checker is needed.
+ */
+ private final boolean needPeriodicalCheckReclaimBuffer;
+
+ /**
+ * The number of requested buffers from {@link BufferPool}. Only this field can be touched both
+ * by the task thread and the netty thread, so it is an atomic type.
+ */
+ private final AtomicInteger numRequestedBuffers;
+
+ /**
+ * A thread to check whether to reclaim buffers from each tiered storage.
+ *
+ * <p>Note that it is not possible to remove this, as doing so could result in the task becoming
+ * stuck in the buffer request. As the number of buffers in the buffer pool can vary at any
+ * given time, the stuck may occur if the thread is removed.
+ *
+ * <p>For instance, if the memory usage of the {@link BufferPool} has been checked and {@link
+ * TieredStorageMemoryManagerImpl} determined that buffer reclamation is unnecessary, but then
+ * the buffer pool size is suddenly reduced to a very small size, the buffer request will become
+ * stuck and the task will never be able to call for buffer reclamation if this thread is
+ * removed, then a task stuck occurs.
+ */
+ private ScheduledExecutorService executor;
+
+ /** The total number of guaranteed buffers for all tiered storages. */
+ private int numTotalGuaranteedBuffers;
+
+ /** The buffer pool where the buffer is requested or recycled. */
+ private BufferPool bufferPool;
+
+ /**
+ * Indicate whether the {@link TieredStorageMemoryManagerImpl} is initialized. Before setting
+ * up, this field is false.
+ *
+ * <p>Note that before requesting buffers or getting the maximum allowed buffers, this
+ * initialized state should be checked.
+ */
+ private boolean isInitialized;
+
+ /**
+ * The constructor of the {@link TieredStorageMemoryManagerImpl}.
+ *
+ * @param numTriggerReclaimBuffersRatio the buffer pool usage ratio of requesting each tiered
+ * storage to reclaim buffers
+ * @param needPeriodicalCheckReclaimBuffer indicate whether it is necessary to start a
+ * periodically checking buffer reclaim thread
+ */
+ public TieredStorageMemoryManagerImpl(
+ float numTriggerReclaimBuffersRatio, boolean needPeriodicalCheckReclaimBuffer) {
+ this.numTriggerReclaimBuffersRatio = numTriggerReclaimBuffersRatio;
+ this.needPeriodicalCheckReclaimBuffer = needPeriodicalCheckReclaimBuffer;
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ this.bufferReclaimRequestListeners = new ArrayList<>();
+ this.isInitialized = false;
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs) {
+ this.bufferPool = bufferPool;
+ for (TieredStorageMemorySpec memorySpec : storageMemorySpecs) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalGuaranteedBuffers += memorySpec.getNumGuaranteedBuffers();
+ }
+
+ if (needPeriodicalCheckReclaimBuffer) {
+ this.executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("buffer reclaim checker")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+ }
+
+ this.isInitialized = true;
+ }
+
+ @Override
+ public void listenBufferReclaimRequest(Runnable onBufferReclaimRequest) {
+ bufferReclaimRequestListeners.add(onBufferReclaimRequest);
+ }
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link BufferPool} for a specific owner. The
+ * {@link TieredStorageMemoryManagerImpl} will not check whether a buffer can be requested and
+ * only record the total number of requested buffers. If the buffers in the {@link BufferPool}
+ * is not enough, this will request each tiered storage to reclaim the buffers as much as
+ * possible.
+ *
+ * <p>Note that synchronization is not necessary for this method, as it can only be called by
+ * the task thread.
+ *
+ * @return the requested buffer
+ */
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ checkIsInitialized();
+
+ tryReclaimBuffers();
+ CompletableFuture<MemorySegment> requestedBuffer =
+ CompletableFuture.supplyAsync(
+ () -> {
+ try {
+ return bufferPool.requestMemorySegmentBlocking();
+ } catch (InterruptedException e) {
+ throw new RuntimeException(e);
+ }
+ });
+
+ ScheduledFuture<?> scheduledFuture = getPeriodicalCheckReclaimBufferFuture();
+
+ MemorySegment memorySegment = null;
+ try {
+ memorySegment = requestedBuffer.get();
+ } catch (InterruptedException | ExecutionException e) {
+ ExceptionUtils.rethrow(e);
+ }
+
+ if (scheduledFuture != null) {
+ scheduledFuture.cancel(false);
+ }
+
+ numRequestedBuffers.incrementAndGet();
+ return new BufferBuilder(checkNotNull(memorySegment), this::recycleBuffer);
+ }
+
+ /**
+ * The synchronization is not necessary for this method, as it can only be called by the task
+ * thread.
+ *
+ * <p>The accuracy of the memory usage ratio may be compromised due to the varying buffer pool
+ * sizes. However, this only impacts a single iteration of the buffer usage check. Upon the next
+ * iteration, the buffer reclaim will eventually be triggered.
Review Comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on PR #22352:
URL: https://github.com/apache/flink/pull/22352#issuecomment-1556994939
@xintongsong Thanks for reviewing, added a fixup commit to address the comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1200333268
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,279 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.ScheduledFuture;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_INITIAL_DELAY_MS = 10;
+
+ /** The period of checking buffer reclaim. */
+ private static final int DEFAULT_CHECK_BUFFER_RECLAIM_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** Listeners used to listen the requests for reclaiming buffer in different tiered storage. */
+ private final List<Runnable> bufferReclaimRequestListeners;
+
+ /** The buffer pool usage ratio of triggering the registered storages to reclaim buffers. */
+ private final float numTriggerReclaimBuffersRatio;
+
+ /**
+ * Indicate whether it is necessary to start a periodically checking buffer reclaim thread. If
+ * the memory manager is used in downstream, the field will be false because periodical buffer
+ * reclaim checker is needed.
+ */
+ private final boolean needPeriodicalCheckReclaimBuffer;
+
+ /**
+ * The number of requested buffers from {@link BufferPool}. Only this field can be touched both
+ * by the task thread and the netty thread, so it is an atomic type.
+ */
+ private final AtomicInteger numRequestedBuffers;
+
+ /**
+ * A thread to check whether to reclaim buffers from each tiered storage.
+ *
+ * <p>Note that it is not possible to remove this, as doing so could result in the task becoming
+ * stuck in the buffer request. As the number of buffers in the buffer pool can vary at any
+ * given time, the stuck may occur if the thread is removed.
+ *
+ * <p>For instance, if the memory usage of the {@link BufferPool} has been checked and {@link
+ * TieredStorageMemoryManagerImpl} determined that buffer reclamation is unnecessary, but then
+ * the buffer pool size is suddenly reduced to a very small size, the buffer request will become
+ * stuck and the task will never be able to call for buffer reclamation if this thread is
+ * removed, then a task stuck occurs.
+ */
+ private ScheduledExecutorService executor;
+
+ /** The total number of guaranteed buffers for all tiered storages. */
+ private int numTotalGuaranteedBuffers;
+
+ /** The buffer pool where the buffer is requested or recycled. */
+ private BufferPool bufferPool;
+
+ /**
+ * Indicate whether the {@link TieredStorageMemoryManagerImpl} is initialized. Before setting
+ * up, this field is false.
+ *
+ * <p>Note that before requesting buffers or getting the maximum allowed buffers, this
+ * initialized state should be checked.
+ */
+ private boolean isInitialized;
+
+ /**
+ * The constructor of the {@link TieredStorageMemoryManagerImpl}.
+ *
+ * @param numTriggerReclaimBuffersRatio the buffer pool usage ratio of requesting each tiered
+ * storage to reclaim buffers
+ * @param needPeriodicalCheckReclaimBuffer indicate whether it is necessary to start a
+ * periodically checking buffer reclaim thread
+ */
+ public TieredStorageMemoryManagerImpl(
+ float numTriggerReclaimBuffersRatio, boolean needPeriodicalCheckReclaimBuffer) {
+ this.numTriggerReclaimBuffersRatio = numTriggerReclaimBuffersRatio;
+ this.needPeriodicalCheckReclaimBuffer = needPeriodicalCheckReclaimBuffer;
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ this.bufferReclaimRequestListeners = new ArrayList<>();
+ this.isInitialized = false;
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs) {
+ this.bufferPool = bufferPool;
+ for (TieredStorageMemorySpec memorySpec : storageMemorySpecs) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalGuaranteedBuffers += memorySpec.getNumGuaranteedBuffers();
+ }
+
+ if (needPeriodicalCheckReclaimBuffer) {
+ this.executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("buffer reclaim checker")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+ }
+
+ this.isInitialized = true;
+ }
+
+ @Override
+ public void listenBufferReclaimRequest(Runnable onBufferReclaimRequest) {
+ bufferReclaimRequestListeners.add(onBufferReclaimRequest);
+ }
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link BufferPool} for a specific owner. The
+ * {@link TieredStorageMemoryManagerImpl} will not check whether a buffer can be requested and
+ * only record the total number of requested buffers. If the buffers in the {@link BufferPool}
+ * is not enough, this will request each tiered storage to reclaim the buffers as much as
+ * possible.
+ *
+ * <p>Note that synchronization is not necessary for this method, as it can only be called by
+ * the task thread.
+ *
+ * @return the requested buffer
+ */
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ checkIsInitialized();
+
+ tryReclaimBuffers();
+ CompletableFuture<MemorySegment> requestedBuffer =
+ CompletableFuture.supplyAsync(
+ () -> {
+ try {
+ return bufferPool.requestMemorySegmentBlocking();
+ } catch (InterruptedException e) {
+ throw new RuntimeException(e);
+ }
+ });
+
+ ScheduledFuture<?> scheduledFuture = getPeriodicalCheckReclaimBufferFuture();
+
+ MemorySegment memorySegment = null;
+ try {
+ memorySegment = requestedBuffer.get();
+ } catch (InterruptedException | ExecutionException e) {
+ ExceptionUtils.rethrow(e);
+ }
+
+ if (scheduledFuture != null) {
+ scheduledFuture.cancel(false);
+ }
+
+ numRequestedBuffers.incrementAndGet();
+ return new BufferBuilder(checkNotNull(memorySegment), this::recycleBuffer);
+ }
+
+ /**
+ * The synchronization is not necessary for this method, as it can only be called by the task
+ * thread.
+ *
+ * <p>The accuracy of the memory usage ratio may be compromised due to the varying buffer pool
+ * sizes. However, this only impacts a single iteration of the buffer usage check. Upon the next
+ * iteration, the buffer reclaim will eventually be triggered.
+ */
+ @Override
+ public int getMaxNonReclaimableBuffers(Object owner) {
+ checkIsInitialized();
+
+ TieredStorageMemorySpec ownerMemorySpec = checkNotNull(tieredMemorySpecs.get(owner));
+ int ownerGuaranteedBuffers = ownerMemorySpec.getNumGuaranteedBuffers();
+ return bufferPool.getNumBuffers()
+ - numRequestedBuffers.get()
+ - numTotalGuaranteedBuffers
+ + ownerGuaranteedBuffers;
+ }
+
+ @Override
+ public void release() {
+ checkState(numRequestedBuffers.get() == 0, "Leaking buffers.");
+ if (executor != null) {
+ executor.shutdown();
+ try {
+ if (!executor.awaitTermination(5L, TimeUnit.MINUTES)) {
+ throw new TimeoutException(
+ "Timeout for shutting down the buffer reclaim checker executor.");
+ }
+ } catch (Exception e) {
+ ExceptionUtils.rethrow(e);
+ }
+ }
+ }
+
+ private void tryReclaimBuffers() {
+ if (shouldReclaimBuffers()) {
+ bufferReclaimRequestListeners.forEach(Runnable::run);
+ }
+ }
+
+ /**
+ * The accuracy of the memory usage ratio may be compromised due to the varying buffer pool
+ * sizes. However, this only impacts a single iteration of the buffer usage check. Upon the next
+ * iteration, the buffer reclaim will eventually be triggered.
+ */
+ private boolean shouldReclaimBuffers() {
+ int numTotal = bufferPool.getNumBuffers();
+ int numRequested = numRequestedBuffers.get();
+ return numRequested >= numTotal
+ // Because we do the checking before requesting buffers, we need add additional one
+ // buffer when calculating the usage ratio.
Review Comment:
Renamed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1188324829
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,178 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.runtime.io.network.partition.hybrid.tiered.tier.TierMemorySpec;
+import org.apache.flink.util.ExceptionUtils;
+
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different tiers or {@link BufferAccumulator}.
+ *
+ * <p>The buffers in a buffer pool are mainly divided into types, the exclusive buffers for a tier
+ * and the shared buffers for all tiers. Each tier decides the number of exclusive buffers and
+ * whether it can use the shared buffers by {@link TierMemorySpec}.
+ *
+ * <p>The buffers of {@link BufferAccumulator} is also managed in the {@link
+ * TieredStorageMemoryManager}. When {@link BufferAccumulator} requests or recycles buffers, it uses
+ * separate methods that are different from tiers.
+ *
+ * <p>Note that when initializing the {@link TieredStorageMemoryManagerImpl}, the {@link
+ * TierMemorySpec} of each tier should be ready. These tier memory specs mainly decide how many
+ * buffers can be used for every tier.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
Review Comment:
I have updated the code according to the offline discussion. The main method of available buffers of a owner is `numAvailableBuffers`, which mainly to limiting a tier (`isMemoryReleasable` of the tier is `false`) from requesting too many buffers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193121929
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ private int numTotalExclusiveBuffers;
+
+ private BufferPool bufferPool;
+
+ private final AtomicInteger numRequestedBuffers;
+
+ public TieredStorageMemoryManagerImpl() {
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool) {
+ this.bufferPool = bufferPool;
+ }
+
+ @Override
+ public void registerMemorySpec(TieredStorageMemorySpec memorySpec) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec registration.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalExclusiveBuffers += memorySpec.getNumExclusiveBuffers();
+ }
Review Comment:
I have removed the `registerMemorySpec`. When setting up the memory manager, the `TieredStorageMemorySpec`s of different storages should be ready.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193122906
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/CacheFlushManager.java:
##########
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.runtime.io.network.partition.hybrid.tiered.common.TieredStorageUtils;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+
+/**
+ * A manager to check whether the cached buffers need to be flushed.
+ *
+ * <p>Cached buffers refer to the buffers stored in a tier that can be released by flushing them to
+ * the disk or remote storage. If the requested buffers from the buffer pool reach the ratio limit,
+ * {@link CacheFlushManager} will trigger a process wherein all these buffers are flushed to the
+ * disk or remote storage to recycling these buffers.
+ *
+ * <p>Note that when initializing a tier with cached buffers, the tier should register a {@link
+ * CacheBufferFlushTrigger}, as a listener to flush cached buffers.
+ */
+public class CacheFlushManager {
Review Comment:
I have merged `TieredStorageMemoryManager` and `CacheFlushManager` into one class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193122339
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/CacheBufferFlushTrigger.java:
##########
@@ -0,0 +1,26 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+/** Notify the specific tier to try flushing the cached buffers. */
+public interface CacheBufferFlushTrigger {
Review Comment:
Removed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193122840
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/CacheFlushManager.java:
##########
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.runtime.io.network.partition.hybrid.tiered.common.TieredStorageUtils;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+
+/**
+ * A manager to check whether the cached buffers need to be flushed.
+ *
+ * <p>Cached buffers refer to the buffers stored in a tier that can be released by flushing them to
+ * the disk or remote storage. If the requested buffers from the buffer pool reach the ratio limit,
+ * {@link CacheFlushManager} will trigger a process wherein all these buffers are flushed to the
+ * disk or remote storage to recycling these buffers.
+ *
+ * <p>Note that when initializing a tier with cached buffers, the tier should register a {@link
+ * CacheBufferFlushTrigger}, as a listener to flush cached buffers.
+ */
+public class CacheFlushManager {
+ private final float numBuffersTriggerFlushRatio;
+
+ private final List<CacheBufferFlushTrigger> flushTriggers;
+
+ private TieredStorageMemoryManager storageMemoryManager;
+
+ private final ScheduledExecutorService executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("cache flush trigger")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+
+ public CacheFlushManager(float numBuffersTriggerFlushRatio) {
+ this.numBuffersTriggerFlushRatio = numBuffersTriggerFlushRatio;
+ this.flushTriggers = new ArrayList<>();
+
+ executor.scheduleWithFixedDelay(
+ this::checkNeedTriggerFlushCachedBuffers, 10, 50, TimeUnit.MILLISECONDS);
Review Comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on PR #22352:
URL: https://github.com/apache/flink/pull/22352#issuecomment-1525027465
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193122339
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/CacheBufferFlushTrigger.java:
##########
@@ -0,0 +1,26 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+/** Notify the specific tier to try flushing the cached buffers. */
+public interface CacheBufferFlushTrigger {
Review Comment:
Removed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #22352:
URL: https://github.com/apache/flink/pull/22352#issuecomment-1496914217
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "45012e9dc367a09fcc21ce668265c0f80af2c35a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "45012e9dc367a09fcc21ce668265c0f80af2c35a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 45012e9dc367a09fcc21ce668265c0f80af2c35a UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on PR #22352:
URL: https://github.com/apache/flink/pull/22352#issuecomment-1523266865
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193822060
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManager.java:
##########
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+
+import java.util.List;
+
+/**
+ * The {@link TieredStorageMemoryManager} is to request or recycle buffers from {@link
+ * LocalBufferPool} for different memory owners, for example, the tiers, the buffer accumulator,
+ * etc. Note that the logic for requesting and recycling buffers is consistent for these owners.
+ *
+ * <p>The memory managed by {@link TieredStorageMemoryManager} is categorized into two types:
+ * long-term occupied memory which cannot be immediately released and short-term occupied memory
+ * which can be reclaimed quickly and safely. Long-term occupied memory usage necessitates waiting
+ * for other operations to complete before releasing it, such as downstream consumption. On the
+ * other hand, short-term occupied memory can be freed up at any time, enabling rapid memory
+ * recycling for tasks such as flushing memory to disk or remote storage.
+ *
+ * <p>This {@link TieredStorageMemoryManager} aim to streamline and harmonize memory management
+ * across various layers. Instead of tracking the number of buffers utilized by individual users, it
+ * dynamically calculates a user's maximum guaranteed amount based on the current status of the
+ * manager and the local buffer pool. Specifically, if a user is a long-term occupied memory user,
+ * the {@link TieredStorageMemoryManager} does not limit the user's memory usage, while if a user is
+ * a short-term occupied memory user, the current guaranteed buffers of the user is the left buffers
+ * in the buffer pool - guaranteed amount of other users (excluding the current user).
+ */
+public interface TieredStorageMemoryManager {
+
+ /**
+ * Setup the {@link TieredStorageMemoryManager}. When setting up the manager, the {@link
+ * TieredStorageMemorySpec}s for different tiered storages should be ready to indicate each
+ * tiered storage's memory requirement specs.
+ *
+ * @param bufferPool the local buffer pool
+ * @param storageMemorySpecs the memory specs for different tiered storages
+ */
+ void setup(BufferPool bufferPool, List<TieredStorageMemorySpec> storageMemorySpecs);
+
+ /**
+ * Register a buffer flush call back to flush and recycle all the memory in the user when
+ * needed.
+ *
+ * <p>When the left buffers in the {@link BufferPool} are not enough, {@link
+ * TieredStorageMemoryManager} will flush the buffers of the user with the registered callback.
+ *
+ * @param userBufferFlushCallback the buffer flush call back of the memory user
+ */
+ void registerBufferFlushCallback(Runnable userBufferFlushCallback);
+
+ /**
+ * Request a {@link BufferBuilder} instance from {@link LocalBufferPool} for a specific owner.
+ *
+ * @return the requested buffer
+ */
+ BufferBuilder requestBufferBlocking();
+
+ /**
+ * Return the available buffers for the owner.
+ *
+ * <p>Note that the available buffers are calculated dynamically based on some conditions, for
+ * example, the state of the {@link BufferPool}, the {@link TieredStorageMemorySpec} of the
+ * owner, etc.
Review Comment:
Fixed and rewrite the docs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193122840
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/CacheFlushManager.java:
##########
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.runtime.io.network.partition.hybrid.tiered.common.TieredStorageUtils;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+
+/**
+ * A manager to check whether the cached buffers need to be flushed.
+ *
+ * <p>Cached buffers refer to the buffers stored in a tier that can be released by flushing them to
+ * the disk or remote storage. If the requested buffers from the buffer pool reach the ratio limit,
+ * {@link CacheFlushManager} will trigger a process wherein all these buffers are flushed to the
+ * disk or remote storage to recycling these buffers.
+ *
+ * <p>Note that when initializing a tier with cached buffers, the tier should register a {@link
+ * CacheBufferFlushTrigger}, as a listener to flush cached buffers.
+ */
+public class CacheFlushManager {
+ private final float numBuffersTriggerFlushRatio;
+
+ private final List<CacheBufferFlushTrigger> flushTriggers;
+
+ private TieredStorageMemoryManager storageMemoryManager;
+
+ private final ScheduledExecutorService executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("cache flush trigger")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+
+ public CacheFlushManager(float numBuffersTriggerFlushRatio) {
+ this.numBuffersTriggerFlushRatio = numBuffersTriggerFlushRatio;
+ this.flushTriggers = new ArrayList<>();
+
+ executor.scheduleWithFixedDelay(
+ this::checkNeedTriggerFlushCachedBuffers, 10, 50, TimeUnit.MILLISECONDS);
Review Comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193122008
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemorySpec.java:
##########
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+/**
+ * The memory specs for a memory owner, including the owner itself, the number of exclusive buffers
+ * of the owner, whether the owner's memory can be released when not consumed, etc.
+ */
+public class TieredStorageMemorySpec {
+
+ private final Object owner;
+
+ private final int numExclusiveBuffers;
+
+ private final boolean isMemoryReleasable;
Review Comment:
Added a specific java doc to indicate the meaning of the field.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on PR #22352:
URL: https://github.com/apache/flink/pull/22352#issuecomment-1549222155
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1193828115
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,224 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ /** Initial delay before checking flush. */
+ public static final int DEFAULT_CHECK_FLUSH_INITIAL_DELAY_MS = 10;
+
+ /** Check flush period. */
+ public static final int DEFAULT_CHECK_FLUSH_PERIOD_DURATION_MS = 50;
+
+ /** The tiered storage memory specs of each memory user owner. */
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ /** The registered callbacks to flush the buffers in the registered tiered storages. */
+ private final List<Runnable> bufferFlushCallbacks;
+
+ /** The buffer pool usage ratio of triggering the registered storages to flush buffers. */
+ private final float numBuffersTriggerFlushRatio;
+
+ /**
+ * Indicate whether to start the buffer flush checker thread. If the memory manager is used in
+ * downstream, the field will be false because no buffer flush checker is needed.
+ */
+ private final boolean shouldStartBufferFlushChecker;
Review Comment:
I believe the thread belongs to this class, because it is responsible for checking and reclaiming the buffers from different tiered storage. I think it is natural that a memory manager reclaims buffers from users.
But the thread name and the switch name are not that right. I renamed the `boolean` to `needPeriodicalCheckReclaimBuffer` and renamed `numTriggerReclaimBuffersRatio`.
The accuracy of the memory usage ratio may be compromised due to the varying buffer pool sizes. However, this only impacts a single iteration of the buffer usage check. Upon the next iteration, the buffer reclaim will eventually be triggered.
The Java doc also added the above reason descriptions of adding the thread.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on PR #22352:
URL: https://github.com/apache/flink/pull/22352#issuecomment-1547859489
Thanks for helping review @xintongsong . I updated the pr with a new fixup commit.
Note that it is not possible to remove the checking thread, as doing so could result in the task becoming stuck in the buffer request. As the number of buffers in the buffer pool can vary at any given time, the stuck may occur if the thread is removed.
For instance, if the memory usage of the {https://github.com/link BufferPool} has been checked and {https://github.com/link TieredStorageMemoryManagerImpl} determined that buffer reclamation is unnecessary, but then the buffer pool size is suddenly reduced to a very small size, the buffer request will become stuck and the task will never be able to call for buffer reclamation if this thread is removed, then a task stuck occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on PR #22352:
URL: https://github.com/apache/flink/pull/22352#issuecomment-1559133540
@xintongsong Thanks for helping review and giving valuable ideas & suggestions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] TanYuxin-tyx commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "TanYuxin-tyx (via GitHub)" <gi...@apache.org>.
TanYuxin-tyx commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1188359934
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManager.java:
##########
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+
+/**
+ * The {@link TieredStorageMemoryManager} is to request or recycle buffer from {@link
+ * LocalBufferPool} for different tiers or {@link BufferAccumulator}.
+ *
+ * <p>The buffers of {@link BufferAccumulator} is also managed in the {@link
+ * TieredStorageMemoryManager}. When {@link BufferAccumulator} requests or recycles buffers, it uses
+ * separate methods that are different from tiers.
+ */
+public interface TieredStorageMemoryManager {
+
+ /** Setup the {@link TieredStorageMemoryManager}. */
+ void setup(BufferPool bufferPool);
+
+ /**
+ * Request a {@link MemorySegment} instance from {@link LocalBufferPool} for a specific tier.
+ *
+ * @param tierIndex the tier index
+ * @return the requested buffer
+ */
+ MemorySegment requestBufferBlocking(int tierIndex);
+
+ /**
+ * Recycle a {@link MemorySegment} buffer to {@link LocalBufferPool} for a specific tier.
+ *
+ * @param memorySegment the buffer to be recycled
+ * @param tierIndex the specific tier index
+ */
+ void recycleBuffer(MemorySegment memorySegment, int tierIndex);
Review Comment:
I changed the return value of `requestBufferBlocking` to `BufferBuilder`. The recycler has been included, so the method `recycleBuffer ` is removed.
The reason why returning a `BufferBuilder` (not a `Buffer`) is that we should use a `BufferBuilder` to append a record (ByteBuffer) in `SubpartitionCachedBuffer#writeRecord` directly.
This return value is like `HsMemoryDataManager#requestBufferFromPool`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] xintongsong commented on a diff in pull request #22352: [FLINK-31639][network] Introduce tiered store memory manager
Posted by "xintongsong (via GitHub)" <gi...@apache.org>.
xintongsong commented on code in PR #22352:
URL: https://github.com/apache/flink/pull/22352#discussion_r1191854430
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ private int numTotalExclusiveBuffers;
+
+ private BufferPool bufferPool;
+
+ private final AtomicInteger numRequestedBuffers;
+
+ public TieredStorageMemoryManagerImpl() {
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool) {
+ this.bufferPool = bufferPool;
+ }
+
+ @Override
+ public void registerMemorySpec(TieredStorageMemorySpec memorySpec) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec registration.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalExclusiveBuffers += memorySpec.getNumExclusiveBuffers();
+ }
+
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ MemorySegment requestedBuffer = null;
+ try {
+ requestedBuffer = bufferPool.requestMemorySegmentBlocking();
+ } catch (Throwable throwable) {
+ ExceptionUtils.rethrow(throwable, "Failed to request memory segments.");
+ }
+ numRequestedBuffers.incrementAndGet();
+ return new BufferBuilder(checkNotNull(requestedBuffer), this::recycleBuffer);
+ }
Review Comment:
We are not checking whether a buffer can be requested. This should be explained.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/CacheFlushManager.java:
##########
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.runtime.io.network.partition.hybrid.tiered.common.TieredStorageUtils;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+
+/**
+ * A manager to check whether the cached buffers need to be flushed.
+ *
+ * <p>Cached buffers refer to the buffers stored in a tier that can be released by flushing them to
+ * the disk or remote storage. If the requested buffers from the buffer pool reach the ratio limit,
+ * {@link CacheFlushManager} will trigger a process wherein all these buffers are flushed to the
+ * disk or remote storage to recycling these buffers.
+ *
+ * <p>Note that when initializing a tier with cached buffers, the tier should register a {@link
+ * CacheBufferFlushTrigger}, as a listener to flush cached buffers.
+ */
+public class CacheFlushManager {
+ private final float numBuffersTriggerFlushRatio;
+
+ private final List<CacheBufferFlushTrigger> flushTriggers;
+
+ private TieredStorageMemoryManager storageMemoryManager;
+
+ private final ScheduledExecutorService executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("cache flush trigger")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+
+ public CacheFlushManager(float numBuffersTriggerFlushRatio) {
+ this.numBuffersTriggerFlushRatio = numBuffersTriggerFlushRatio;
+ this.flushTriggers = new ArrayList<>();
+
+ executor.scheduleWithFixedDelay(
+ this::checkNeedTriggerFlushCachedBuffers, 10, 50, TimeUnit.MILLISECONDS);
+ }
+
+ public void setup(TieredStorageMemoryManager storageMemoryManager) {
+ this.storageMemoryManager = storageMemoryManager;
+ }
+
+ public void registerCacheBufferFlushTrigger(CacheBufferFlushTrigger cacheBufferFlushTrigger) {
+ flushTriggers.add(cacheBufferFlushTrigger);
+ }
+
+ public void triggerFlushCachedBuffers() {
Review Comment:
Should be private.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/CacheFlushManager.java:
##########
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.runtime.io.network.partition.hybrid.tiered.common.TieredStorageUtils;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+
+/**
+ * A manager to check whether the cached buffers need to be flushed.
+ *
+ * <p>Cached buffers refer to the buffers stored in a tier that can be released by flushing them to
+ * the disk or remote storage. If the requested buffers from the buffer pool reach the ratio limit,
+ * {@link CacheFlushManager} will trigger a process wherein all these buffers are flushed to the
+ * disk or remote storage to recycling these buffers.
+ *
+ * <p>Note that when initializing a tier with cached buffers, the tier should register a {@link
+ * CacheBufferFlushTrigger}, as a listener to flush cached buffers.
+ */
+public class CacheFlushManager {
+ private final float numBuffersTriggerFlushRatio;
+
+ private final List<CacheBufferFlushTrigger> flushTriggers;
+
+ private TieredStorageMemoryManager storageMemoryManager;
+
+ private final ScheduledExecutorService executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("cache flush trigger")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+
+ public CacheFlushManager(float numBuffersTriggerFlushRatio) {
+ this.numBuffersTriggerFlushRatio = numBuffersTriggerFlushRatio;
+ this.flushTriggers = new ArrayList<>();
+
+ executor.scheduleWithFixedDelay(
+ this::checkNeedTriggerFlushCachedBuffers, 10, 50, TimeUnit.MILLISECONDS);
+ }
+
+ public void setup(TieredStorageMemoryManager storageMemoryManager) {
+ this.storageMemoryManager = storageMemoryManager;
+ }
+
+ public void registerCacheBufferFlushTrigger(CacheBufferFlushTrigger cacheBufferFlushTrigger) {
+ flushTriggers.add(cacheBufferFlushTrigger);
+ }
+
+ public void triggerFlushCachedBuffers() {
+ flushTriggers.forEach(CacheBufferFlushTrigger::notifyFlushCachedBuffers);
+ }
+
+ public void checkNeedTriggerFlushCachedBuffers() {
+ if (storageMemoryManager == null) {
+ return;
+ }
+
+ if (TieredStorageUtils.needFlushCacheBuffers(
+ storageMemoryManager, numBuffersTriggerFlushRatio)) {
+ triggerFlushCachedBuffers();
+ }
+ }
+
+ public int numFlushTriggers() {
Review Comment:
Why do we need this?
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ private int numTotalExclusiveBuffers;
+
+ private BufferPool bufferPool;
+
+ private final AtomicInteger numRequestedBuffers;
+
+ public TieredStorageMemoryManagerImpl() {
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool) {
+ this.bufferPool = bufferPool;
+ }
+
+ @Override
+ public void registerMemorySpec(TieredStorageMemorySpec memorySpec) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec registration.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalExclusiveBuffers += memorySpec.getNumExclusiveBuffers();
+ }
Review Comment:
Do we allow adding new specs after setting up? How do we guarantee the exclusive buffers for the new spec?
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemorySpec.java:
##########
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+/**
+ * The memory specs for a memory owner, including the owner itself, the number of exclusive buffers
+ * of the owner, whether the owner's memory can be released when not consumed, etc.
+ */
+public class TieredStorageMemorySpec {
+
+ private final Object owner;
+
+ private final int numExclusiveBuffers;
Review Comment:
Maybe "guaranteed" or "required" rather than "exclusive".
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemorySpec.java:
##########
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+/**
+ * The memory specs for a memory owner, including the owner itself, the number of exclusive buffers
+ * of the owner, whether the owner's memory can be released when not consumed, etc.
+ */
+public class TieredStorageMemorySpec {
+
+ private final Object owner;
+
+ private final int numExclusiveBuffers;
+
+ private final boolean isMemoryReleasable;
Review Comment:
The name is a bit ambiguous. However, I cannot think of a better one. Maybe we should explain this with javadocs.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ private int numTotalExclusiveBuffers;
+
+ private BufferPool bufferPool;
+
+ private final AtomicInteger numRequestedBuffers;
+
+ public TieredStorageMemoryManagerImpl() {
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool) {
+ this.bufferPool = bufferPool;
+ }
+
+ @Override
+ public void registerMemorySpec(TieredStorageMemorySpec memorySpec) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec registration.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalExclusiveBuffers += memorySpec.getNumExclusiveBuffers();
+ }
+
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ MemorySegment requestedBuffer = null;
+ try {
+ requestedBuffer = bufferPool.requestMemorySegmentBlocking();
+ } catch (Throwable throwable) {
+ ExceptionUtils.rethrow(throwable, "Failed to request memory segments.");
+ }
+ numRequestedBuffers.incrementAndGet();
+ return new BufferBuilder(checkNotNull(requestedBuffer), this::recycleBuffer);
+ }
+
+ @Override
+ public int numAvailableBuffers(Object owner) {
Review Comment:
I'd suggest the name `getMaxAllowedBuffers`.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ private int numTotalExclusiveBuffers;
+
+ private BufferPool bufferPool;
+
+ private final AtomicInteger numRequestedBuffers;
Review Comment:
Is this class thread-safe or not? Why `numRequestedBuffers` is atomic while `numTotalExclusiveBuffers` is not?
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/common/TieredStorageUtils.java:
##########
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.common;
+
+import org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage.TieredStorageMemoryManager;
+
+/** Utils for reading or writing to tiered store. */
+public class TieredStorageUtils {
+ public static boolean needFlushCacheBuffers(
+ TieredStorageMemoryManager storageMemoryManager, float numBuffersTriggerFlushRatio) {
+ int numTotal = storageMemoryManager.numTotalBuffers();
+ int numRequested = storageMemoryManager.numRequestedBuffers();
Review Comment:
These are non-atomic now. Why not add a `getUtilization` for the memory manager.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/TieredStorageMemoryManagerImpl.java:
##########
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.core.memory.MemorySegment;
+import org.apache.flink.runtime.io.network.buffer.BufferBuilder;
+import org.apache.flink.runtime.io.network.buffer.BufferPool;
+import org.apache.flink.runtime.io.network.buffer.LocalBufferPool;
+import org.apache.flink.util.ExceptionUtils;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import static org.apache.flink.util.Preconditions.checkNotNull;
+import static org.apache.flink.util.Preconditions.checkState;
+
+/**
+ * The implementation for {@link TieredStorageMemoryManager}. This is to request or recycle buffers
+ * from {@link LocalBufferPool} for different memory owners, for example, the tiers, the buffer
+ * accumulator, etc.
+ *
+ * <p>Note that the memory owner should register its {@link TieredStorageMemorySpec} firstly before
+ * requesting buffers.
+ */
+public class TieredStorageMemoryManagerImpl implements TieredStorageMemoryManager {
+
+ private final Map<Object, TieredStorageMemorySpec> tieredMemorySpecs;
+
+ private int numTotalExclusiveBuffers;
+
+ private BufferPool bufferPool;
+
+ private final AtomicInteger numRequestedBuffers;
+
+ public TieredStorageMemoryManagerImpl() {
+ this.tieredMemorySpecs = new HashMap<>();
+ this.numRequestedBuffers = new AtomicInteger(0);
+ }
+
+ @Override
+ public void setup(BufferPool bufferPool) {
+ this.bufferPool = bufferPool;
+ }
+
+ @Override
+ public void registerMemorySpec(TieredStorageMemorySpec memorySpec) {
+ checkState(
+ !tieredMemorySpecs.containsKey(memorySpec.getOwner()),
+ "Duplicated memory spec registration.");
+ tieredMemorySpecs.put(memorySpec.getOwner(), memorySpec);
+ numTotalExclusiveBuffers += memorySpec.getNumExclusiveBuffers();
+ }
+
+ @Override
+ public BufferBuilder requestBufferBlocking() {
+ MemorySegment requestedBuffer = null;
+ try {
+ requestedBuffer = bufferPool.requestMemorySegmentBlocking();
+ } catch (Throwable throwable) {
+ ExceptionUtils.rethrow(throwable, "Failed to request memory segments.");
+ }
+ numRequestedBuffers.incrementAndGet();
+ return new BufferBuilder(checkNotNull(requestedBuffer), this::recycleBuffer);
+ }
Review Comment:
I think we should describe the overall responsibility and protocol of the memory manager in the javadoc.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/CacheFlushManager.java:
##########
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.runtime.io.network.partition.hybrid.tiered.common.TieredStorageUtils;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+
+/**
+ * A manager to check whether the cached buffers need to be flushed.
+ *
+ * <p>Cached buffers refer to the buffers stored in a tier that can be released by flushing them to
+ * the disk or remote storage. If the requested buffers from the buffer pool reach the ratio limit,
+ * {@link CacheFlushManager} will trigger a process wherein all these buffers are flushed to the
+ * disk or remote storage to recycling these buffers.
+ *
+ * <p>Note that when initializing a tier with cached buffers, the tier should register a {@link
+ * CacheBufferFlushTrigger}, as a listener to flush cached buffers.
+ */
+public class CacheFlushManager {
+ private final float numBuffersTriggerFlushRatio;
+
+ private final List<CacheBufferFlushTrigger> flushTriggers;
+
+ private TieredStorageMemoryManager storageMemoryManager;
+
+ private final ScheduledExecutorService executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("cache flush trigger")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+
+ public CacheFlushManager(float numBuffersTriggerFlushRatio) {
+ this.numBuffersTriggerFlushRatio = numBuffersTriggerFlushRatio;
+ this.flushTriggers = new ArrayList<>();
+
+ executor.scheduleWithFixedDelay(
+ this::checkNeedTriggerFlushCachedBuffers, 10, 50, TimeUnit.MILLISECONDS);
+ }
+
+ public void setup(TieredStorageMemoryManager storageMemoryManager) {
+ this.storageMemoryManager = storageMemoryManager;
+ }
+
+ public void registerCacheBufferFlushTrigger(CacheBufferFlushTrigger cacheBufferFlushTrigger) {
+ flushTriggers.add(cacheBufferFlushTrigger);
+ }
+
+ public void triggerFlushCachedBuffers() {
+ flushTriggers.forEach(CacheBufferFlushTrigger::notifyFlushCachedBuffers);
+ }
+
+ public void checkNeedTriggerFlushCachedBuffers() {
Review Comment:
Do we need this to be public? Do we need to manually call the check in addition to the scheduled one?
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/common/TieredStorageUtils.java:
##########
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.common;
+
+import org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage.TieredStorageMemoryManager;
+
+/** Utils for reading or writing to tiered store. */
+public class TieredStorageUtils {
+ public static boolean needFlushCacheBuffers(
+ TieredStorageMemoryManager storageMemoryManager, float numBuffersTriggerFlushRatio) {
+ int numTotal = storageMemoryManager.numTotalBuffers();
+ int numRequested = storageMemoryManager.numRequestedBuffers();
Review Comment:
What if the requested buffers are not completed yet?
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/CacheFlushManager.java:
##########
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.runtime.io.network.partition.hybrid.tiered.common.TieredStorageUtils;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+
+/**
+ * A manager to check whether the cached buffers need to be flushed.
+ *
+ * <p>Cached buffers refer to the buffers stored in a tier that can be released by flushing them to
+ * the disk or remote storage. If the requested buffers from the buffer pool reach the ratio limit,
+ * {@link CacheFlushManager} will trigger a process wherein all these buffers are flushed to the
+ * disk or remote storage to recycling these buffers.
+ *
+ * <p>Note that when initializing a tier with cached buffers, the tier should register a {@link
+ * CacheBufferFlushTrigger}, as a listener to flush cached buffers.
+ */
+public class CacheFlushManager {
+ private final float numBuffersTriggerFlushRatio;
+
+ private final List<CacheBufferFlushTrigger> flushTriggers;
+
+ private TieredStorageMemoryManager storageMemoryManager;
+
+ private final ScheduledExecutorService executor =
+ Executors.newSingleThreadScheduledExecutor(
+ new ThreadFactoryBuilder()
+ .setNameFormat("cache flush trigger")
+ .setUncaughtExceptionHandler(FatalExitExceptionHandler.INSTANCE)
+ .build());
+
+ public CacheFlushManager(float numBuffersTriggerFlushRatio) {
+ this.numBuffersTriggerFlushRatio = numBuffersTriggerFlushRatio;
+ this.flushTriggers = new ArrayList<>();
+
+ executor.scheduleWithFixedDelay(
+ this::checkNeedTriggerFlushCachedBuffers, 10, 50, TimeUnit.MILLISECONDS);
Review Comment:
Hardcoded.
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/CacheBufferFlushTrigger.java:
##########
@@ -0,0 +1,26 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+/** Notify the specific tier to try flushing the cached buffers. */
+public interface CacheBufferFlushTrigger {
Review Comment:
This seems unnecessary. Why not just use `Runnable`?
##########
flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/hybrid/tiered/storage/CacheFlushManager.java:
##########
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.runtime.io.network.partition.hybrid.tiered.storage;
+
+import org.apache.flink.runtime.io.network.partition.hybrid.tiered.common.TieredStorageUtils;
+import org.apache.flink.util.ExceptionUtils;
+import org.apache.flink.util.FatalExitExceptionHandler;
+
+import org.apache.flink.shaded.guava30.com.google.common.util.concurrent.ThreadFactoryBuilder;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.Executors;
+import java.util.concurrent.ScheduledExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+
+/**
+ * A manager to check whether the cached buffers need to be flushed.
+ *
+ * <p>Cached buffers refer to the buffers stored in a tier that can be released by flushing them to
+ * the disk or remote storage. If the requested buffers from the buffer pool reach the ratio limit,
+ * {@link CacheFlushManager} will trigger a process wherein all these buffers are flushed to the
+ * disk or remote storage to recycling these buffers.
+ *
+ * <p>Note that when initializing a tier with cached buffers, the tier should register a {@link
+ * CacheBufferFlushTrigger}, as a listener to flush cached buffers.
+ */
+public class CacheFlushManager {
Review Comment:
It seems this is closely related to memory manager. I wonder if it makes sense to combine them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org