You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Dawid Wysakowicz (Jira)" <ji...@apache.org> on 2020/06/11 12:45:00 UTC
[jira] [Comment Edited] (FLINK-18241) Custom OptionsFactory in user
code not working when configured via flink-conf.yaml
[ https://issues.apache.org/jira/browse/FLINK-18241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17133229#comment-17133229 ]
Dawid Wysakowicz edited comment on FLINK-18241 at 6/11/20, 12:44 PM:
---------------------------------------------------------------------
The problem is that the StreamPlanEnvironment/OptimizerPlanEnvironment does not set user class loader correctly. Those are used when submitting a job through web ui and in per job mode in 1.10.
In 1.11 they are not used any longer, but we use a ContextEnvironment which sets the classloader correctly and thus it works. (they are still used for {{flink info}} which I think will also fail in 1.11)
In 1.9 it works because there is no logic that would read it when constructing the environment. It uses a completely different path which uses the correct classloader on the cluster side.
As a side note all versions work correctly if we use the cli (flink run)
I will prepare patches for 1.10/1.11/master for setting the correct classloader.
was (Author: dawidwys):
The problem is that the StreamPlanEnvironment/OptimizerPlanEnvironment does not set user class loader correctly. Those are used when submitting a job through web ui and in per job mode in 1.10.
In 1.11 they are not used any longer, but we use a ContextEnvironment which sets the classloader correctly and thus it works.
In 1.9 it works because there is no logic that would read it when constructing the environment. It uses a completely different path which uses the correct classloader on the cluster side.
As a side note all versions work correctly if we use the cli (flink run)
I will prepare patches for 1.10/1.11/master for setting the correct classloader.
> Custom OptionsFactory in user code not working when configured via flink-conf.yaml
> ----------------------------------------------------------------------------------
>
> Key: FLINK-18241
> URL: https://issues.apache.org/jira/browse/FLINK-18241
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Configuration, Runtime / State Backends
> Affects Versions: 1.10.0, 1.10.1
> Reporter: Nico Kruber
> Assignee: Dawid Wysakowicz
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.0, 1.10.2
>
> Attachments: DefaultConfigurableOptionsFactoryWithLog.java
>
>
> It seems like Flink 1.10 broke custom {{OptionsFactory}} definitions via the {{state.backend.rocksdb.options-factory}} configuration if the implementation resides in the user-code jar file. This is particularly bad to debug RocksDB issues since we disabled its (ever-growing) LOG file in FLINK-15068.
> If you look at the stack trace from the error below, you will notice, that {{StreamExecutionEnvironment}} is not provided with a user-code classloader and will us the one of its own class which is the parent loader that does not know about our {{OptionsFactory}}. This exact same code was working with Flink 1.9.3.
> (I believe putting the custom {{OptionsFactory}} into a separate jar file inside Flink's lib folder may be a workaround but that should ideally not be needed).
> {code:java}
> 2020-06-09 16:18:59,409 ERROR org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Could not start cluster entrypoint StandaloneJobClusterEntryPoint.
> org.apache.flink.runtime.entrypoint.ClusterEntrypointException: Failed to initialize the cluster entrypoint StandaloneJobClusterEntryPoint.
> at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:192) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:525) [flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.container.entrypoint.StandaloneJobClusterEntryPoint.main(StandaloneJobClusterEntryPoint.java:116) [flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> Caused by: org.apache.flink.util.FlinkException: Could not create the DispatcherResourceManagerComponent.
> at org.apache.flink.runtime.entrypoint.component.DefaultDispatcherResourceManagerComponentFactory.create(DefaultDispatcherResourceManagerComponentFactory.java:261) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:220) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:174) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:173) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> ... 2 more
> Caused by: org.apache.flink.util.FlinkRuntimeException: Could not retrieve the JobGraph.
> at org.apache.flink.runtime.dispatcher.runner.JobDispatcherLeaderProcessFactoryFactory.createFactory(JobDispatcherLeaderProcessFactoryFactory.java:57) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.dispatcher.runner.DefaultDispatcherRunnerFactory.createDispatcherRunner(DefaultDispatcherRunnerFactory.java:51) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.component.DefaultDispatcherResourceManagerComponentFactory.create(DefaultDispatcherResourceManagerComponentFactory.java:196) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:220) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:174) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:173) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> ... 2 more
> Caused by: org.apache.flink.util.FlinkException: Could not create the JobGraph from the provided user code jar.
> at org.apache.flink.container.entrypoint.ClassPathJobGraphRetriever.retrieveJobGraph(ClassPathJobGraphRetriever.java:114) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.dispatcher.runner.JobDispatcherLeaderProcessFactoryFactory.createFactory(JobDispatcherLeaderProcessFactoryFactory.java:55) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.dispatcher.runner.DefaultDispatcherRunnerFactory.createDispatcherRunner(DefaultDispatcherRunnerFactory.java:51) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.component.DefaultDispatcherResourceManagerComponentFactory.create(DefaultDispatcherResourceManagerComponentFactory.java:196) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:220) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:174) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:173) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> ... 2 more
> Caused by: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: org.apache.flink.util.DynamicCodeLoadingException: Cannot find configured options factory class: com.ververica.DefaultConfigurableOptionsFactoryWithLog
> at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.client.program.OptimizerPlanEnvironment.getPipeline(OptimizerPlanEnvironment.java:80) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.client.program.PackagedProgramUtils.getPipelineFromProgram(PackagedProgramUtils.java:108) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:58) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.container.entrypoint.ClassPathJobGraphRetriever.retrieveJobGraph(ClassPathJobGraphRetriever.java:104) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.dispatcher.runner.JobDispatcherLeaderProcessFactoryFactory.createFactory(JobDispatcherLeaderProcessFactoryFactory.java:55) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.dispatcher.runner.DefaultDispatcherRunnerFactory.createDispatcherRunner(DefaultDispatcherRunnerFactory.java:51) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.component.DefaultDispatcherResourceManagerComponentFactory.create(DefaultDispatcherResourceManagerComponentFactory.java:196) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:220) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:174) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:173) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> ... 2 more
> Caused by: org.apache.flink.util.FlinkRuntimeException: org.apache.flink.util.DynamicCodeLoadingException: Cannot find configured options factory class: com.ververica.DefaultConfigurableOptionsFactoryWithLog
> at org.apache.flink.contrib.streaming.state.RocksDBStateBackend.<init>(RocksDBStateBackend.java:376) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.contrib.streaming.state.RocksDBStateBackend.configure(RocksDBStateBackend.java:394) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:47) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:32) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.state.StateBackendLoader.loadStateBackendFromConfig(StateBackendLoader.java:154) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.loadStateBackend(StreamExecutionEnvironment.java:767) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.configure(StreamExecutionEnvironment.java:750) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.<init>(StreamExecutionEnvironment.java:218) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.<init>(StreamExecutionEnvironment.java:190) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.streaming.api.environment.StreamPlanEnvironment.<init>(StreamPlanEnvironment.java:38) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.createStreamExecutionEnvironment(StreamExecutionEnvironment.java:1871) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at java.util.Optional.orElseGet(Optional.java:267) ~[?:1.8.0_252]
> at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getExecutionEnvironment(StreamExecutionEnvironment.java:1859) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at com.ververica.flinktraining.exercises.troubleshoot.TroubledStreamingJobUtils.createConfiguredEnvironment(TroubledStreamingJobUtils.java:40) ~[?:?]
> at com.ververica.flinktraining.solutions.troubleshoot.TroubledStreamingJobSolution43.main(TroubledStreamingJobSolution43.java:40) ~[?:?]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_252]
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_252]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_252]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_252]
> at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.client.program.OptimizerPlanEnvironment.getPipeline(OptimizerPlanEnvironment.java:80) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.client.program.PackagedProgramUtils.getPipelineFromProgram(PackagedProgramUtils.java:108) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:58) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.container.entrypoint.ClassPathJobGraphRetriever.retrieveJobGraph(ClassPathJobGraphRetriever.java:104) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.dispatcher.runner.JobDispatcherLeaderProcessFactoryFactory.createFactory(JobDispatcherLeaderProcessFactoryFactory.java:55) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.dispatcher.runner.DefaultDispatcherRunnerFactory.createDispatcherRunner(DefaultDispatcherRunnerFactory.java:51) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.component.DefaultDispatcherResourceManagerComponentFactory.create(DefaultDispatcherResourceManagerComponentFactory.java:196) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:220) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:174) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:173) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> ... 2 more
> Caused by: org.apache.flink.util.DynamicCodeLoadingException: Cannot find configured options factory class: com.ververica.DefaultConfigurableOptionsFactoryWithLog
> at org.apache.flink.contrib.streaming.state.RocksDBStateBackend.configureOptionsFactory(RocksDBStateBackend.java:605) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.contrib.streaming.state.RocksDBStateBackend.<init>(RocksDBStateBackend.java:370) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.contrib.streaming.state.RocksDBStateBackend.configure(RocksDBStateBackend.java:394) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:47) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:32) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.state.StateBackendLoader.loadStateBackendFromConfig(StateBackendLoader.java:154) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.loadStateBackend(StreamExecutionEnvironment.java:767) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.configure(StreamExecutionEnvironment.java:750) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.<init>(StreamExecutionEnvironment.java:218) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.<init>(StreamExecutionEnvironment.java:190) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.streaming.api.environment.StreamPlanEnvironment.<init>(StreamPlanEnvironment.java:38) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.createStreamExecutionEnvironment(StreamExecutionEnvironment.java:1871) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at java.util.Optional.orElseGet(Optional.java:267) ~[?:1.8.0_252]
> at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getExecutionEnvironment(StreamExecutionEnvironment.java:1859) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at com.ververica.flinktraining.exercises.troubleshoot.TroubledStreamingJobUtils.createConfiguredEnvironment(TroubledStreamingJobUtils.java:40) ~[?:?]
> at com.ververica.flinktraining.solutions.troubleshoot.TroubledStreamingJobSolution43.main(TroubledStreamingJobSolution43.java:40) ~[?:?]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_252]
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_252]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_252]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_252]
> at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.client.program.OptimizerPlanEnvironment.getPipeline(OptimizerPlanEnvironment.java:80) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.client.program.PackagedProgramUtils.getPipelineFromProgram(PackagedProgramUtils.java:108) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:58) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.container.entrypoint.ClassPathJobGraphRetriever.retrieveJobGraph(ClassPathJobGraphRetriever.java:104) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.dispatcher.runner.JobDispatcherLeaderProcessFactoryFactory.createFactory(JobDispatcherLeaderProcessFactoryFactory.java:55) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.dispatcher.runner.DefaultDispatcherRunnerFactory.createDispatcherRunner(DefaultDispatcherRunnerFactory.java:51) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.component.DefaultDispatcherResourceManagerComponentFactory.create(DefaultDispatcherResourceManagerComponentFactory.java:196) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:220) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:174) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:173) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> ... 2 more
> Caused by: java.lang.ClassNotFoundException: com.ververica.DefaultConfigurableOptionsFactoryWithLog
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:1.8.0_252]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418) ~[?:1.8.0_252]
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) ~[?:1.8.0_252]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ~[?:1.8.0_252]
> at java.lang.Class.forName0(Native Method) ~[?:1.8.0_252]
> at java.lang.Class.forName(Class.java:348) ~[?:1.8.0_252]
> at org.apache.flink.contrib.streaming.state.RocksDBStateBackend.configureOptionsFactory(RocksDBStateBackend.java:594) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.contrib.streaming.state.RocksDBStateBackend.<init>(RocksDBStateBackend.java:370) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.contrib.streaming.state.RocksDBStateBackend.configure(RocksDBStateBackend.java:394) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:47) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:32) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.state.StateBackendLoader.loadStateBackendFromConfig(StateBackendLoader.java:154) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.loadStateBackend(StreamExecutionEnvironment.java:767) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.configure(StreamExecutionEnvironment.java:750) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.<init>(StreamExecutionEnvironment.java:218) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.<init>(StreamExecutionEnvironment.java:190) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.streaming.api.environment.StreamPlanEnvironment.<init>(StreamPlanEnvironment.java:38) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.createStreamExecutionEnvironment(StreamExecutionEnvironment.java:1871) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at java.util.Optional.orElseGet(Optional.java:267) ~[?:1.8.0_252]
> at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getExecutionEnvironment(StreamExecutionEnvironment.java:1859) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at com.ververica.flinktraining.exercises.troubleshoot.TroubledStreamingJobUtils.createConfiguredEnvironment(TroubledStreamingJobUtils.java:40) ~[?:?]
> at com.ververica.flinktraining.solutions.troubleshoot.TroubledStreamingJobSolution43.main(TroubledStreamingJobSolution43.java:40) ~[?:?]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_252]
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_252]
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_252]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_252]
> at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.client.program.OptimizerPlanEnvironment.getPipeline(OptimizerPlanEnvironment.java:80) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.client.program.PackagedProgramUtils.getPipelineFromProgram(PackagedProgramUtils.java:108) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:58) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.container.entrypoint.ClassPathJobGraphRetriever.retrieveJobGraph(ClassPathJobGraphRetriever.java:104) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.dispatcher.runner.JobDispatcherLeaderProcessFactoryFactory.createFactory(JobDispatcherLeaderProcessFactoryFactory.java:55) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.dispatcher.runner.DefaultDispatcherRunnerFactory.createDispatcherRunner(DefaultDispatcherRunnerFactory.java:51) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.component.DefaultDispatcherResourceManagerComponentFactory.create(DefaultDispatcherResourceManagerComponentFactory.java:196) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:220) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$0(ClusterEntrypoint.java:174) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]
> at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:173) ~[flink-dist_2.12-1.10.1-stream1.jar:1.10.1-stream1]{code}
> I attached the {{OptionsFactory}} that I used to show the error. (just put it into the user code jar and configure
> {code:java}
> state.backend.rocksdb.options-factory: com.ververica.DefaultConfigurableOptionsFactoryWithLog {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)