You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by "WangYu (Jira)" <ji...@apache.org> on 2021/09/16 08:51:00 UTC
[jira] [Created] (FLINK-24309) NoClassDefFoundError occurs when
using spring-data-redis for event notification in a flink cluster
environment
WangYu created FLINK-24309:
------------------------------
Summary: NoClassDefFoundError occurs when using spring-data-redis for event notification in a flink cluster environment
Key: FLINK-24309
URL: https://issues.apache.org/jira/browse/FLINK-24309
Project: Flink
Issue Type: Bug
Components: API / DataStream
Affects Versions: 1.13.0
Environment: Flink 1.13.0 zookeeper cluster
1 JobManager
3 TaskManagers
Every TaskManager has 8 slots
And parallelism of every subtask is 6
Reporter: WangYu
I used spring-data-redis in my Flink application to implement notifications for subscribing to refresh events. The code is as follows:
@Bean
public RedisMessageListenerContainer redisContainer(RedisConnectionFactory redisConnectionFactory, MessageListenerAdapter commonListenerAdapter) {
RedisMessageListenerContainer container = new RedisMessageListenerContainer();
container.setConnectionFactory(redisConnectionFactory);
container.addMessageListener(commonListenerAdapter, new ChannelTopic(ApplicationConstants.REDIS_TOPIC_NAME));
return container;
}
@Bean
MessageListenerAdapter commonListenerAdapter(RedisReceiver redisReceiver) {
MessageListenerAdapter messageListenerAdapter = new MessageListenerAdapter(redisReceiver, "onMessage");
messageListenerAdapter.setSerializer(jacksonSerializer());
return messageListenerAdapter;
}
It works very well in the local environment, application can receive notifications of refresh events through redis.
But when I deployed the application to the Flink cluster, I got an error message:
2021-09-16 16:22:05,197 WARN io.netty.channel.AbstractChannelHandlerContext [] - An exception 'java.lang.NoClassDefFoundError: io/netty/util/internal/logging/InternalLogLevel' [enable DEBUG level for full stacktrace] was thrown by a user handler's exceptionCaught() method while handling the following exception:2021-09-16 16:22:05,197 WARN io.netty.channel.AbstractChannelHandlerContext [] - An exception 'java.lang.NoClassDefFoundError: io/netty/util/internal/logging/InternalLogLevel' [enable DEBUG level for full stacktrace] was thrown by a user handler's exceptionCaught() method while handling the following exception:java.lang.NoClassDefFoundError: org/springframework/data/redis/connection/DefaultMessage at org.springframework.data.redis.connection.lettuce.LettuceMessageListener.message(LettuceMessageListener.java:43) ~[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?] at org.springframework.data.redis.connection.lettuce.LettuceMessageListener.message(LettuceMessageListener.java:29) ~[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?] at io.lettuce.core.pubsub.PubSubEndpoint.notifyListeners(PubSubEndpoint.java:191) ~[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?] at io.lettuce.core.pubsub.PubSubEndpoint.notifyMessage(PubSubEndpoint.java:180) ~[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?] at io.lettuce.core.pubsub.PubSubCommandHandler.doNotifyMessage(PubSubCommandHandler.java:217) ~[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?] at io.lettuce.core.pubsub.PubSubCommandHandler.decode(PubSubCommandHandler.java:123) ~[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?] at io.lettuce.core.protocol.CommandHandler.channelRead(CommandHandler.java:591) ~[blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) [blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?] at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650) [blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576) [blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [blob_p-e4b9f89a7dac271b36297079ae0306cc0273f659-a280a62d8bc228a1b65e682dcf2f8472:?] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_271]Caused by: java.lang.ClassNotFoundException: org.springframework.data.redis.connection.DefaultMessage at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:1.8.0_271] at java.lang.ClassLoader.loadClass(ClassLoader.java:418) ~[?:1.8.0_271] at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64) ~[flink-dist_2.11-1.13.0.jar:1.13.0] at org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:74) ~[flink-dist_2.11-1.13.0.jar:1.13.0] at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48) ~[flink-dist_2.11-1.13.0.jar:1.13.0] at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ~[?:1.8.0_271] ... 23 more2021-09-16 16:22:05,215 INFO com.cccar.vdm.mpp.client.listener.ModelInstanceRefreshListener [] - ============= refresh model instance =============
I checked the jar package of the application and confirmed that the rg/springframework/data/redis/connection/DefaultMessage class file is in the classpath.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)