You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Canbin Zheng <fe...@gmail.com> on 2020/04/01 09:25:43 UTC

Question about the ReadableConfigToConfigurationAdapter

Hi everyone,

Recently I failed to run a Flink job when enabling RocksDBStateBackend on
the branch master and 1.10.

The exception stack trace is:

The program finished with the following
exception:org.apache.flink.client.program.ProgramInvocationException:
The main method caused an error: The adapter does not support this
method
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335)
	at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
	at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:143)
	at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:659)
	at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:210)
	at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:890)
	at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:963)
	at org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
	at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:963)
Caused by: java.lang.UnsupportedOperationException: The adapter does
not support this method
	at org.apache.flink.configuration.ReadableConfigToConfigurationAdapter.getEnum(ReadableConfigToConfigurationAdapter.java:258)
	at org.apache.flink.contrib.streaming.state.RocksDBStateBackend.<init>(RocksDBStateBackend.java:336)
	at org.apache.flink.contrib.streaming.state.RocksDBStateBackend.configure(RocksDBStateBackend.java:394)
	at org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:47)
	at org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:32)
	at org.apache.flink.runtime.state.StateBackendLoader.loadStateBackendFromConfig(StateBackendLoader.java:154)
	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.loadStateBackend(StreamExecutionEnvironment.java:792)
	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.configure(StreamExecutionEnvironment.java:761)
	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.<init>(StreamExecutionEnvironment.java:217)
	at org.apache.flink.client.program.StreamContextEnvironment.<init>(StreamContextEnvironment.java:53)
	at org.apache.flink.client.program.StreamContextEnvironment.lambda$setAsContext$2(StreamContextEnvironment.java:103)
	at java.util.Optional.map(Optional.java:215)
	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getExecutionEnvironment(StreamExecutionEnvironment.java:1882)
	at org.apache.flink.streaming.examples.socket.SocketWindowWordCount.main(SocketWindowWordCount.java:62)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
	... 8 more

It seems that this is a BUG. Does anyone encounter such a problem?

I am wondering why we introduce ReadableConfigToConfigurationAdapter
to wrap the Configuration but leave many of the methods in it to throw
UnsupportedOperationException that causes problems.

Regards,

Canbin Zheng

Re: Question about the ReadableConfigToConfigurationAdapter

Posted by Canbin Zheng <fe...@gmail.com>.
Thanks for your feedback, Till & Dawid.

Dawid Wysakowicz <dw...@apache.org> 于2020年4月2日周四 下午11:11写道：

> Hi,
>
> Yes this is a bug that is tracked here:
> https://issues.apache.org/jira/browse/FLINK-16913. I am working on it
> right now.
>
> You should expect a fix very soon.
>
> Best,
>
> Dawid
> On 02/04/2020 17:07, Till Rohrmann wrote:
>
> Hi Canbin,
>
> this looks indeed like a bug to me. I'm pulling in Dawid who worked on
> this part and might be able to tell us more about it.
>
> If he agrees, then I would suggest to open a JIRA issue and to fix it.
>
> Cheers,
> Till
>
> On Wed, Apr 1, 2020 at 11:26 AM Canbin Zheng <fe...@gmail.com>
> wrote:
>
>> Hi everyone,
>>
>> Recently I failed to run a Flink job when enabling RocksDBStateBackend on
>> the branch master and 1.10.
>>
>> The exception stack trace is:
>>
>> The program finished with the following
>> exception:org.apache.flink.client.program.ProgramInvocationException:
>> The main method caused an error: The adapter does not support this
>> method
>>         at
>> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335)
>>         at
>> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
>>         at
>> org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:143)
>>         at
>> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:659)
>>         at
>> org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:210)
>>         at
>> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:890)
>>         at
>> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:963)
>>         at
>> org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
>>         at
>> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:963)
>> Caused by: java.lang.UnsupportedOperationException: The adapter does
>> not support this method
>>         at
>> org.apache.flink.configuration.ReadableConfigToConfigurationAdapter.getEnum(ReadableConfigToConfigurationAdapter.java:258)
>>         at
>> org.apache.flink.contrib.streaming.state.RocksDBStateBackend.<init>(RocksDBStateBackend.java:336)
>>         at
>> org.apache.flink.contrib.streaming.state.RocksDBStateBackend.configure(RocksDBStateBackend.java:394)
>>         at
>> org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:47)
>>         at
>> org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:32)
>>         at
>> org.apache.flink.runtime.state.StateBackendLoader.loadStateBackendFromConfig(StateBackendLoader.java:154)
>>         at
>> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.loadStateBackend(StreamExecutionEnvironment.java:792)
>>         at
>> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.configure(StreamExecutionEnvironment.java:761)
>>         at
>> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.<init>(StreamExecutionEnvironment.java:217)
>>         at
>> org.apache.flink.client.program.StreamContextEnvironment.<init>(StreamContextEnvironment.java:53)
>>         at
>> org.apache.flink.client.program.StreamContextEnvironment.lambda$setAsContext$2(StreamContextEnvironment.java:103)
>>         at java.util.Optional.map(Optional.java:215)
>>         at
>> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getExecutionEnvironment(StreamExecutionEnvironment.java:1882)
>>         at
>> org.apache.flink.streaming.examples.socket.SocketWindowWordCount.main(SocketWindowWordCount.java:62)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>         at java.lang.reflect.Method.invoke(Method.java:498)
>>         at
>> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
>>         ... 8 more
>>
>> It seems that this is a BUG. Does anyone encounter such a problem?
>>
>> I am wondering why we introduce ReadableConfigToConfigurationAdapter
>> to wrap the Configuration but leave many of the methods in it to throw
>> UnsupportedOperationException that causes problems.
>>
>> Regards,
>>
>> Canbin Zheng
>>
>

Re: Question about the ReadableConfigToConfigurationAdapter

Posted by Dawid Wysakowicz <dw...@apache.org>.
Hi,

Yes this is a bug that is tracked here:
https://issues.apache.org/jira/browse/FLINK-16913. I am working on it
right now.

You should expect a fix very soon.

Best,

Dawid

On 02/04/2020 17:07, Till Rohrmann wrote:
> Hi Canbin,
>
> this looks indeed like a bug to me. I'm pulling in Dawid who worked on
> this part and might be able to tell us more about it.
>
> If he agrees, then I would suggest to open a JIRA issue and to fix it.
>
> Cheers,
> Till
>
> On Wed, Apr 1, 2020 at 11:26 AM Canbin Zheng <felixzhengcb@gmail.com
> <ma...@gmail.com>> wrote:
>
>     Hi everyone,
>
>     Recently I failed to run a Flink job when enabling
>     RocksDBStateBackend on
>     the branch master and 1.10.
>
>     The exception stack trace is:
>
>     The program finished with the following
>     exception:org.apache.flink.client.program.ProgramInvocationException:
>     The main method caused an error: The adapter does not support this
>     method
>             at
>     org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335)
>             at
>     org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
>             at
>     org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:143)
>             at
>     org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:659)
>             at
>     org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:210)
>             at
>     org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:890)
>             at
>     org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:963)
>             at
>     org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
>             at
>     org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:963)
>     Caused by: java.lang.UnsupportedOperationException: The adapter does
>     not support this method
>             at
>     org.apache.flink.configuration.ReadableConfigToConfigurationAdapter.getEnum(ReadableConfigToConfigurationAdapter.java:258)
>             at
>     org.apache.flink.contrib.streaming.state.RocksDBStateBackend.<init>(RocksDBStateBackend.java:336)
>             at
>     org.apache.flink.contrib.streaming.state.RocksDBStateBackend.configure(RocksDBStateBackend.java:394)
>             at
>     org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:47)
>             at
>     org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:32)
>             at
>     org.apache.flink.runtime.state.StateBackendLoader.loadStateBackendFromConfig(StateBackendLoader.java:154)
>             at
>     org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.loadStateBackend(StreamExecutionEnvironment.java:792)
>             at
>     org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.configure(StreamExecutionEnvironment.java:761)
>             at
>     org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.<init>(StreamExecutionEnvironment.java:217)
>             at
>     org.apache.flink.client.program.StreamContextEnvironment.<init>(StreamContextEnvironment.java:53)
>             at
>     org.apache.flink.client.program.StreamContextEnvironment.lambda$setAsContext$2(StreamContextEnvironment.java:103)
>             at java.util.Optional.map(Optional.java:215)
>             at
>     org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getExecutionEnvironment(StreamExecutionEnvironment.java:1882)
>             at
>     org.apache.flink.streaming.examples.socket.SocketWindowWordCount.main(SocketWindowWordCount.java:62)
>             at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>             at
>     sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>             at
>     sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>             at java.lang.reflect.Method.invoke(Method.java:498)
>             at
>     org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
>             ... 8 more
>
>     It seems that this is a BUG. Does anyone encounter such a problem?
>
>     I am wondering why we introduce ReadableConfigToConfigurationAdapter
>     to wrap the Configuration but leave many of the methods in it to throw
>     UnsupportedOperationException that causes problems.
>
>     Regards,
>
>     Canbin Zheng
>

Re: Question about the ReadableConfigToConfigurationAdapter

Posted by Till Rohrmann <tr...@apache.org>.
Hi Canbin,

this looks indeed like a bug to me. I'm pulling in Dawid who worked on this
part and might be able to tell us more about it.

If he agrees, then I would suggest to open a JIRA issue and to fix it.

Cheers,
Till

On Wed, Apr 1, 2020 at 11:26 AM Canbin Zheng <fe...@gmail.com> wrote:

> Hi everyone,
>
> Recently I failed to run a Flink job when enabling RocksDBStateBackend on
> the branch master and 1.10.
>
> The exception stack trace is:
>
> The program finished with the following
> exception:org.apache.flink.client.program.ProgramInvocationException:
> The main method caused an error: The adapter does not support this
> method
>         at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335)
>         at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
>         at
> org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:143)
>         at
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:659)
>         at
> org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:210)
>         at
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:890)
>         at
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:963)
>         at
> org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
>         at
> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:963)
> Caused by: java.lang.UnsupportedOperationException: The adapter does
> not support this method
>         at
> org.apache.flink.configuration.ReadableConfigToConfigurationAdapter.getEnum(ReadableConfigToConfigurationAdapter.java:258)
>         at
> org.apache.flink.contrib.streaming.state.RocksDBStateBackend.<init>(RocksDBStateBackend.java:336)
>         at
> org.apache.flink.contrib.streaming.state.RocksDBStateBackend.configure(RocksDBStateBackend.java:394)
>         at
> org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:47)
>         at
> org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:32)
>         at
> org.apache.flink.runtime.state.StateBackendLoader.loadStateBackendFromConfig(StateBackendLoader.java:154)
>         at
> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.loadStateBackend(StreamExecutionEnvironment.java:792)
>         at
> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.configure(StreamExecutionEnvironment.java:761)
>         at
> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.<init>(StreamExecutionEnvironment.java:217)
>         at
> org.apache.flink.client.program.StreamContextEnvironment.<init>(StreamContextEnvironment.java:53)
>         at
> org.apache.flink.client.program.StreamContextEnvironment.lambda$setAsContext$2(StreamContextEnvironment.java:103)
>         at java.util.Optional.map(Optional.java:215)
>         at
> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getExecutionEnvironment(StreamExecutionEnvironment.java:1882)
>         at
> org.apache.flink.streaming.examples.socket.SocketWindowWordCount.main(SocketWindowWordCount.java:62)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
>         ... 8 more
>
> It seems that this is a BUG. Does anyone encounter such a problem?
>
> I am wondering why we introduce ReadableConfigToConfigurationAdapter
> to wrap the Configuration but leave many of the methods in it to throw
> UnsupportedOperationException that causes problems.
>
> Regards,
>
> Canbin Zheng
>