[jira] [Commented] (FLINK-14793) Add SPI interface to discovery state backend and establish well-known names

["Seth Wiesman (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-14793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16974353#comment-16974353 ] 

Seth Wiesman commented on FLINK-14793:
--------------------------------------

Wondering if you’ve seen StateBackendFactory and if that works for your needs. 

https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/state/StateBackendFactory.java

> Add SPI interface to discovery state backend and establish well-known names
> ---------------------------------------------------------------------------
>
>                 Key: FLINK-14793
>                 URL: https://issues.apache.org/jira/browse/FLINK-14793
>             Project: Flink
>          Issue Type: New Feature
>          Components: Runtime / State Backends
>            Reporter: Kezhu Wang
>            Priority: Major
>
> Currently, {{StateBackendLoader.loadStateBackendFromConfig}} uses hybrid of hardcoded state backend name and class path to construct {{StateBackendFactory}}.
> I think we can provide a SPI interface, say \{{StateBackendProvider}}, to accomplish this.
> {code:title=StateBackendProvider.java}
> /**
>  * SPI interface for {@link StateBackend} discovery and naming.
>  *
>  * @param <T> StateBackend
>  */
> @PublicEvolving
> public interface StateBackendProvider<T extends StateBackend> extends StateBackendFactory<T> {
> 	/**
> 	 * Name of provided {@link StateBackend}.
> 	 *
> 	 * @return name of provided {@link StateBackend}
> 	 */
> 	@Nonnull
> 	String name();
> 	/**
> 	 * Aliases for provided {@link StateBackend}. Implementations can use this for deprecated names.
> 	 *
> 	 * @return aliases for provided {@link StateBackend}
> 	 */
> 	default Set<String> aliases() {
> 		return Collections.emptySet();
> 	}
> }
> {code}
> With above spi, we gain several benefits over current hardcode and class path approach:
>  * Explicit and established state backend naming and discovery instead of hardcoded. We can know exactly which state backends are avaiable and whether they are provided by user jars or not.
>  * In state backend configuration, we can simply use established state backend name while not tied to fragile class path which is easily refactored out without great caution. These established state backend names could be flink community level, such as `jobmanager`, `filesystem` and `rocksdb`, and corporation level, such as `corporation-a.product.state-backend-basename`. Together with FLINK-14790, we can switch among state backends without knowledge of concrete implementation of target state backends in source code level.
>  * More compatible with java module shipped in java 9 and above.
> Discovery of state backend can be restricted in {{StateBackendLoader}}, and we can fallback to class path approach if no matching state backend found in discovered state backends.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)