You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Matthias Pohl (Jira)" <ji...@apache.org> on 2023/04/03 09:26:00 UTC
[jira] [Comment Edited] (FLINK-27204) FileSystemJobResultStore should operator on the ioExecutor
[ https://issues.apache.org/jira/browse/FLINK-27204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707883#comment-17707883 ]
Matthias Pohl edited comment on FLINK-27204 at 4/3/23 9:25 AM:
---------------------------------------------------------------
Can you clarify why {{JobResultStore}} is a public interface? It's annotated as {{@Internal}} in [JobResultStore:39|https://github.com/apache/flink/blob/5c39a7ea2c538d0aa81533f708fb8b5e7fb77dd4/flink-runtime/src/main/java/org/apache/flink/runtime/highavailability/JobResultStore.java#L39]. May you elaborate more on where you see that changes to the interface would have significant impact on other components?
was (Author: mapohl):
Why are you saying that the JobResultStore is a public interface? It's annotated as {{@Internal}} in [JobResultStore:39|https://github.com/apache/flink/blob/5c39a7ea2c538d0aa81533f708fb8b5e7fb77dd4/flink-runtime/src/main/java/org/apache/flink/runtime/highavailability/JobResultStore.java#L39]. May you elaborate more on where you see that changes to the interface would have significant impact on other components?
> FileSystemJobResultStore should operator on the ioExecutor
> ----------------------------------------------------------
>
> Key: FLINK-27204
> URL: https://issues.apache.org/jira/browse/FLINK-27204
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Affects Versions: 1.16.0
> Reporter: Matthias Pohl
> Assignee: Wencong Liu
> Priority: Major
>
> The {{JobResultStore}} interface is synchronous currently. For the {{FileSystemJobResultStore}} this means that (possibly) IO-heavy operations have to be explicitly called moved to the ioExecutor within the Dispatcher.
> Instead, we could change the {{JobResultStore}} interface in a way that it returns {{CompletableFuture}} instances instead. That would enable us to run the {{FileSystemJobResultStore}} operations in the ioExecutor which would be set when initializing the {{{}FileSystemJobResultStore{}}}. This would move the responsibility of where to run the operation from the {{Dispatcher}} into the {{JobResultStore.}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)