Re: [PR] [server] add support of `KvPreWriteBuffer` backpressure [fluss]

["zuston (via GitHub)" <gi...@apache.org>]

zuston commented on code in PR #3751:
URL: https://github.com/apache/fluss/pull/3751#discussion_r3717803625


##########
fluss-common/src/main/java/org/apache/fluss/exception/InsufficientKvPreWriteBufferException.java:
##########
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.fluss.exception;
+
+import org.apache.fluss.annotation.PublicEvolving;
+
+/**
+ * The tablet server rejected a KV write because its global pre-write buffer high watermark was
+ * reached.
+ *
+ * <p>The client may retry the write after a backoff. The rejected write has not been appended to

Review Comment:
   Thanks, agreed. I reviewed #3463, and I agree that its cooperative server-client throttling should be the primary backpressure mechanism.
   
   This PR is intended as a defensive, last-resort safeguard rather than a replacement for #3463. While #3463 derives pressure per KvTablet from RocksDB L0/flush conditions, this PR limits the TabletServer-wide aggregate heap retained by all KvPreWriteBuffers. It protects the server from OOM in exceptional cases—for example, when replication stalls, clients do not react quickly enough, or memory accumulates across many tablets.
   
   Now that #3463 has landed, I’ll rebase this PR and reuse its cooperative path: expose pre-write-buffer memory usage as an additional pressure source, combine it with the existing RocksDB pressure, and use StorageBackpressureException for the final hard-rejection guard. The global high watermark will therefore act only as the last line of defense, while normal traffic regulation remains handled by #3463’s client-side throttling mechanism.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org