[jira] [Commented] (FLINK-11691) Introduce ClassLoader in the methods of StateBackendFactory and ConfigurableStateBackend

["Stefan Richter (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-11691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773123#comment-16773123 ] 

Stefan Richter commented on FLINK-11691:
----------------------------------------

I think this is a useful change for what ou are planning, so that custom implementations of a service class like compression or rocksdb options factory can be specified in the flink-conf. I was discussing this with [~StephanEwen] for {{ConfigurableStateBackend}} and we agreed that the interface could be changed if needed. I think the same applies to {{StateBackendFactory}} as well.

> Introduce ClassLoader in the methods of StateBackendFactory and ConfigurableStateBackend
> ----------------------------------------------------------------------------------------
>
>                 Key: FLINK-11691
>                 URL: https://issues.apache.org/jira/browse/FLINK-11691
>             Project: Flink
>          Issue Type: Improvement
>            Reporter: Yun Tang
>            Assignee: Yun Tang
>            Priority: Major
>             Fix For: 1.8.0
>
>
> Currently, interface {{ConfigurableStateBackend}} has method definition below:
> {code:java}
> StateBackend configure(Configuration config) throws IllegalConfigurationException;
> {code}
> Actually, we need the class loader for anything needed to instantiate through reflection when calling {{StateBackend#configure}}. This is really helpful if we want something to be configurable. Take an example, by providing a class name via flink-conf.yaml, user could configure anything they want, such as the {{OptionsFactory}} in {{RocksDBStateBackend}} (FLINK-10912) or the compression decorator in {{StateBackend}} (FLINK-11313)
> On the other hand, interface {{StateBackendFactory}} also has method definition below, which would then call {{StateBackend#configure}}.
> {code:java}
> T createFromConfig(Configuration config) throws IllegalConfigurationException, IOException;
> {code}
> They were all called mainly in {{StateBackendLoader#loadStateBackendFromConfig}} and {{StateBackendLoader#fromApplicationOrConfigOrDefault}}. Those two methods already have {{ClassLoader}} in their parameters, and it would be easy to pass the class loader to the {{configure}} methods mentioned above.
> {{ConfigurableStateBackend}} has no annotation, and from its usage, it should be an annotation of {{'Internal'}}. On the other hand, {{StateBackendFactory}} has an annotation of '{{PublicEvolving'}}, which means we could change the interface if needed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)