You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Zakelly (via GitHub)" <gi...@apache.org> on 2024/12/07 04:52:19 UTC

[PR] [FLINK-36860][State/v2] Make internal list state inherit the internal merging state [flink]

Zakelly opened a new pull request, #25760:
URL: https://github.com/apache/flink/pull/25760

   ## What is the purpose of the change
   
   There is a internal state interface hierarchy for state v2, which is pretty much like the one for state v1. Currently the `InternalListState` does not extend from `InternalMergingState` for convenience of class type parameter signature. However, the table implementation depends on the inheritance, so this PR adds it back.
   
   ## Brief change log
   
    - Extract a `InternalStateAccessible` interface from `InternalAppendingState`
    - Make `InternalListState` inherit `InternalMergingState`
    - Implement necessary methods from `InternalMergingState` for existing List States.
   
    - Also, a hotfix fixing the state adaptor
   
   ## Verifying this change
   
   This change added tests `ForStListStateTest`.
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): no
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
     - The serializers: no
     - The runtime per-record code paths (performance sensitive): yes
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
     - The S3 file system connector: no
   
   ## Documentation
   
     - Does this pull request introduce a new feature? no
     - If yes, how is the feature documented? not applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-36860][State/v2] Make internal list state inherit the internal merging state [flink]

Posted by "fredia (via GitHub)" <gi...@apache.org>.
fredia commented on code in PR #25760:
URL: https://github.com/apache/flink/pull/25760#discussion_r1875309207


##########
flink-runtime/src/main/java/org/apache/flink/runtime/asyncprocessing/AsyncExecutionController.java:
##########
@@ -291,13 +291,21 @@ public <IN, OUT> InternalStateFuture<OUT> handleRequest(
     public <IN, OUT> OUT handleRequestSync(
             State state, StateRequestType type, @Nullable IN payload) {
         InternalStateFuture<OUT> stateFuture = handleRequest(state, type, payload);
-        while (!stateFuture.isDone()) {
-            try {
-                mailboxExecutor.yield();
-            } catch (InterruptedException e) {
-                LOG.warn("Error while waiting for state future to complete.", e);
-                throw new RuntimeException("Error while waiting for state future to complete.", e);
+        triggerIfNeeded(true);
+        try {
+            while (!stateFuture.isDone()) {
+                if (!mailboxExecutor.tryYield()) {
+                    // We force trigger the buffer if targetNum == 0 (for draining) or the state

Review Comment:
   nit: We force trigger the buffer if the state executor is not fully loaded.
   ```suggestion
                       We force trigger the buffer if the state executor is not fully loaded.
   ```



##########
flink-state-backends/flink-statebackend-forst/src/main/java/org/apache/flink/state/forst/ForStDBMultiRawMergePutRequest.java:
##########
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.state.forst;
+
+import org.apache.flink.core.state.InternalStateFuture;
+
+import org.forstdb.RocksDB;
+import org.forstdb.RocksDBException;
+
+import java.io.IOException;
+import java.util.Collection;
+
+/**
+ * The Put access request for ForStDB.
+ *
+ * @param <K> The type of key in put access request.
+ * @param <N> The type of namespace in put access request.
+ * @param <V> The type of value in put access request.
+ */
+public class ForStDBMultiRawMergePutRequest<K, N, V> extends ForStDBPutRequest<K, N, V> {

Review Comment:
   How about extracting `isMerge` from `ForStDBPutRequest`, and combining `isMerge` and `RawMerge` together?
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-36860][State/v2] Make internal list state inherit the internal merging state [flink]

Posted by "Zakelly (via GitHub)" <gi...@apache.org>.
Zakelly merged PR #25760:
URL: https://github.com/apache/flink/pull/25760


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-36860][State/v2] Make internal list state inherit the internal merging state [flink]

Posted by "Zakelly (via GitHub)" <gi...@apache.org>.
Zakelly commented on code in PR #25760:
URL: https://github.com/apache/flink/pull/25760#discussion_r1875622405


##########
flink-state-backends/flink-statebackend-forst/src/main/java/org/apache/flink/state/forst/ForStDBMultiRawMergePutRequest.java:
##########
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.state.forst;
+
+import org.apache.flink.core.state.InternalStateFuture;
+
+import org.forstdb.RocksDB;
+import org.forstdb.RocksDBException;
+
+import java.io.IOException;
+import java.util.Collection;
+
+/**
+ * The Put access request for ForStDB.
+ *
+ * @param <K> The type of key in put access request.
+ * @param <N> The type of namespace in put access request.
+ * @param <V> The type of value in put access request.
+ */
+public class ForStDBMultiRawMergePutRequest<K, N, V> extends ForStDBPutRequest<K, N, V> {

Review Comment:
   I wish so but the `ForStDBMultiRawMergePutRequest` and `ForStDBPutRequest` have different parameter type in constructor(List<byte[]> vs. V), resulting in different signature.
   
   I'd prefer keeping it.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-36860][State/v2] Make internal list state inherit the internal merging state [flink]

Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #25760:
URL: https://github.com/apache/flink/pull/25760#issuecomment-2524903027

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "f56b9fbfef3f2b975f65049626e1f0730e679ce2",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "f56b9fbfef3f2b975f65049626e1f0730e679ce2",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * f56b9fbfef3f2b975f65049626e1f0730e679ce2 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org