You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user-zh@flink.apache.org by "casel.chen" <ca...@126.com> on 2021/11/05 15:12:56 UTC
提交flink作业抛 java.lang.LinkageError
我在公司实时计算平台上提交了一个streaming api写的作业，结果抛如下异常。因为我们的实时计算平台是以flink sql为主的，上面已经集成了flink-kafka-connector。而我提交的作业也是需要从kafka消费，所以将相同版本的flink kafka connector也打进了作业jar包内。请问是什么原因造成的，需要如何修复？谢谢！
2021-11-05 16:38:58 - [submit-session-executor-6] ERROR c.h.s.launcher.AbstractJobExecutor - -----start job failed-----
org.apache.flink.client.program.ProgramInvocationException: The program caused an error:
Classpath: [file:/opt/streamsql/jobs/aml-aml-aml/aml-datasync/TEST/aml-datasync-1.0-SNAPSHOT_zwb3274543418822102949.jar]
System.out: (none)
System.err: (none)
at org.apache.flink.client.program.PackagedProgramUtils.generateException(PackagedProgramUtils.java:264)
at org.apache.flink.client.program.PackagedProgramUtils.getPipelineFromProgram(PackagedProgramUtils.java:172)
at com.huifu.streamsql.launcher.AbstractJobExecutor.createJobGraph(AbstractJobExecutor.java:205)
at com.huifu.streamsql.launcher.standalone.RemoteExecutor.doStart(RemoteExecutor.java:31)
at com.huifu.streamsql.launcher.AbstractJobExecutor.start(AbstractJobExecutor.java:51)
at com.huifu.streamsql.launcher.JobCommand$1.execute(JobCommand.java:15)
at com.huifu.streamsql.service.StreamSqlServiceImpl.submitJob(StreamSqlServiceImpl.java:443)
at com.huifu.kunpeng.service.DeploymentServiceImpl.submitJob(DeploymentServiceImpl.java:1662)
at com.huifu.kunpeng.service.DeploymentServiceImpl.launchDeployment(DeploymentServiceImpl.java:1623)
at com.huifu.kunpeng.service.DeploymentServiceImpl$$FastClassBySpringCGLIB$$855501cb.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at org.springframework.retry.annotation.AnnotationAwareRetryOperationsInterceptor.invoke(AnnotationAwareRetryOperationsInterceptor.java:156)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
at com.huifu.kunpeng.service.DeploymentServiceImpl$$EnhancerBySpringCGLIB$$9aed5b42.launchDeployment(<generated>)
at com.huifu.kunpeng.runner.SubmitQueueApplicationRunner.lambda$run$0(SubmitQueueApplicationRunner.java:63)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.LinkageError: loader constraint violation: loader (instance of org/apache/flink/util/ChildFirstClassLoader) previously initiated loading for a different type with name "org/apache/kafka/clients/consumer/ConsumerRecord"
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
Re:回复: Re: 提交flink作业抛 java.lang.LinkageError
Posted by "casel.chen" <ca...@126.com>.
试过设置成provided，作业运行会抛ClassNotFoundError: org/apache/kafka/clients/consumer/ConsumerRecord
在 2021-11-09 09:54:59，"WuKong" <wu...@foxmail.com> 写道：
>Hi :
> 看报错日志，还是类加载问题 提示的报错信息 是说已经由不同类加载器已经加装了改依赖。如果生产环境上已经由了相关依赖包，建议将依赖设置为provided
>
>Caused by: java.lang.LinkageError: loader constraint violation: loader
>>> (instance of org/apache/flink/util/ChildFirstClassLoader) previously
>>> initiated loading for a different type with name
>>> "org/apache/kafka/clients/consumer/ConsumerRecord"
>
>
>
>---
>Best,
>WuKong
>
>发件人： casel.chen
>发送时间： 2021-11-08 14:38
>收件人： user-zh
>主题： Re:Re: 提交flink作业抛 java.lang.LinkageError
>版本是一致的，都是1.12.5版本
>
>
>
>
>在 2021-11-08 11:11:35，"Shuiqiang Chen" <ac...@gmail.com> 写道：
>>Hi,
>>
>>能检查下作业jar里 kafka client的版本和平台上的是否一致吗？
>>
>>casel.chen <ca...@126.com> 于2021年11月5日周五 下午11:25写道：
>>
>>> 我在公司实时计算平台上提交了一个streaming api写的作业，结果抛如下异常。因为我们的实时计算平台是以flink
>>> sql为主的，上面已经集成了flink-kafka-connector。而我提交的作业也是需要从kafka消费，所以将相同版本的flink kafka
>>> connector也打进了作业jar包内。请问是什么原因造成的，需要如何修复？谢谢！
>>>
>>>
>>> 2021-11-05 16:38:58 - [submit-session-executor-6] ERROR
>>> c.h.s.launcher.AbstractJobExecutor - -----start job failed-----
>>>
>>>
>>> org.apache.flink.client.program.ProgramInvocationException: The program
>>> caused an error:
>>>
>>>
>>>
>>>
>>>
>>>
>>> Classpath:
>>> [file:/opt/streamsql/jobs/aml-aml-aml/aml-datasync/TEST/aml-datasync-1.0-SNAPSHOT_zwb3274543418822102949.jar]
>>>
>>>
>>>
>>>
>>>
>>>
>>> System.out: (none)
>>>
>>>
>>>
>>>
>>>
>>>
>>> System.err: (none)
>>>
>>>
>>> at
>>> org.apache.flink.client.program.PackagedProgramUtils.generateException(PackagedProgramUtils.java:264)
>>>
>>>
>>> at
>>> org.apache.flink.client.program.PackagedProgramUtils.getPipelineFromProgram(PackagedProgramUtils.java:172)
>>>
>>>
>>> at
>>> com.huifu.streamsql.launcher.AbstractJobExecutor.createJobGraph(AbstractJobExecutor.java:205)
>>>
>>>
>>> at
>>> com.huifu.streamsql.launcher.standalone.RemoteExecutor.doStart(RemoteExecutor.java:31)
>>>
>>>
>>> at
>>> com.huifu.streamsql.launcher.AbstractJobExecutor.start(AbstractJobExecutor.java:51)
>>>
>>>
>>> at com.huifu.streamsql.launcher.JobCommand$1.execute(JobCommand.java:15)
>>>
>>>
>>> at
>>> com.huifu.streamsql.service.StreamSqlServiceImpl.submitJob(StreamSqlServiceImpl.java:443)
>>>
>>>
>>> at
>>> com.huifu.kunpeng.service.DeploymentServiceImpl.submitJob(DeploymentServiceImpl.java:1662)
>>>
>>>
>>> at
>>> com.huifu.kunpeng.service.DeploymentServiceImpl.launchDeployment(DeploymentServiceImpl.java:1623)
>>>
>>>
>>> at
>>> com.huifu.kunpeng.service.DeploymentServiceImpl$$FastClassBySpringCGLIB$$855501cb.invoke(<generated>)
>>>
>>>
>>> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
>>>
>>>
>>> at
>>> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
>>>
>>>
>>> at
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
>>>
>>>
>>> at
>>> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
>>>
>>>
>>> at
>>> org.springframework.retry.annotation.AnnotationAwareRetryOperationsInterceptor.invoke(AnnotationAwareRetryOperationsInterceptor.java:156)
>>>
>>>
>>> at
>>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
>>>
>>>
>>> at
>>> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
>>>
>>>
>>> at
>>> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
>>>
>>>
>>> at
>>> com.huifu.kunpeng.service.DeploymentServiceImpl$$EnhancerBySpringCGLIB$$9aed5b42.launchDeployment(<generated>)
>>>
>>>
>>> at
>>> com.huifu.kunpeng.runner.SubmitQueueApplicationRunner.lambda$run$0(SubmitQueueApplicationRunner.java:63)
>>>
>>>
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>>
>>>
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>>
>>>
>>> at java.lang.Thread.run(Thread.java:748)
>>>
>>>
>>> Caused by: java.lang.LinkageError: loader constraint violation: loader
>>> (instance of org/apache/flink/util/ChildFirstClassLoader) previously
>>> initiated loading for a different type with name
>>> "org/apache/kafka/clients/consumer/ConsumerRecord"
>>>
>>>
>>> at java.lang.ClassLoader.defineClass1(Native Method)
>>>
>>>
>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
>>>
>>>
>>> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>>>
>>>
>>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
>>>
>>>
>>> at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
>>>
>>>
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
>>>
>>>
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
>>>
>>>
>>> at java.security.AccessController.doPrivileged(Native Method)
回复: Re: 提交flink作业抛 java.lang.LinkageError
Posted by WuKong <wu...@foxmail.com>.
Hi :
看报错日志，还是类加载问题 提示的报错信息 是说已经由不同类加载器已经加装了改依赖。如果生产环境上已经由了相关依赖包，建议将依赖设置为provided
Caused by: java.lang.LinkageError: loader constraint violation: loader
>> (instance of org/apache/flink/util/ChildFirstClassLoader) previously
>> initiated loading for a different type with name
>> "org/apache/kafka/clients/consumer/ConsumerRecord"
---
Best,
WuKong
发件人： casel.chen
发送时间： 2021-11-08 14:38
收件人： user-zh
主题： Re:Re: 提交flink作业抛 java.lang.LinkageError
版本是一致的，都是1.12.5版本
在 2021-11-08 11:11:35，"Shuiqiang Chen" <ac...@gmail.com> 写道：
>Hi,
>
>能检查下作业jar里 kafka client的版本和平台上的是否一致吗？
>
>casel.chen <ca...@126.com> 于2021年11月5日周五 下午11:25写道：
>
>> 我在公司实时计算平台上提交了一个streaming api写的作业，结果抛如下异常。因为我们的实时计算平台是以flink
>> sql为主的，上面已经集成了flink-kafka-connector。而我提交的作业也是需要从kafka消费，所以将相同版本的flink kafka
>> connector也打进了作业jar包内。请问是什么原因造成的，需要如何修复？谢谢！
>>
>>
>> 2021-11-05 16:38:58 - [submit-session-executor-6] ERROR
>> c.h.s.launcher.AbstractJobExecutor - -----start job failed-----
>>
>>
>> org.apache.flink.client.program.ProgramInvocationException: The program
>> caused an error:
>>
>>
>>
>>
>>
>>
>> Classpath:
>> [file:/opt/streamsql/jobs/aml-aml-aml/aml-datasync/TEST/aml-datasync-1.0-SNAPSHOT_zwb3274543418822102949.jar]
>>
>>
>>
>>
>>
>>
>> System.out: (none)
>>
>>
>>
>>
>>
>>
>> System.err: (none)
>>
>>
>> at
>> org.apache.flink.client.program.PackagedProgramUtils.generateException(PackagedProgramUtils.java:264)
>>
>>
>> at
>> org.apache.flink.client.program.PackagedProgramUtils.getPipelineFromProgram(PackagedProgramUtils.java:172)
>>
>>
>> at
>> com.huifu.streamsql.launcher.AbstractJobExecutor.createJobGraph(AbstractJobExecutor.java:205)
>>
>>
>> at
>> com.huifu.streamsql.launcher.standalone.RemoteExecutor.doStart(RemoteExecutor.java:31)
>>
>>
>> at
>> com.huifu.streamsql.launcher.AbstractJobExecutor.start(AbstractJobExecutor.java:51)
>>
>>
>> at com.huifu.streamsql.launcher.JobCommand$1.execute(JobCommand.java:15)
>>
>>
>> at
>> com.huifu.streamsql.service.StreamSqlServiceImpl.submitJob(StreamSqlServiceImpl.java:443)
>>
>>
>> at
>> com.huifu.kunpeng.service.DeploymentServiceImpl.submitJob(DeploymentServiceImpl.java:1662)
>>
>>
>> at
>> com.huifu.kunpeng.service.DeploymentServiceImpl.launchDeployment(DeploymentServiceImpl.java:1623)
>>
>>
>> at
>> com.huifu.kunpeng.service.DeploymentServiceImpl$$FastClassBySpringCGLIB$$855501cb.invoke(<generated>)
>>
>>
>> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
>>
>>
>> at
>> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
>>
>>
>> at
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
>>
>>
>> at
>> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
>>
>>
>> at
>> org.springframework.retry.annotation.AnnotationAwareRetryOperationsInterceptor.invoke(AnnotationAwareRetryOperationsInterceptor.java:156)
>>
>>
>> at
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
>>
>>
>> at
>> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
>>
>>
>> at
>> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
>>
>>
>> at
>> com.huifu.kunpeng.service.DeploymentServiceImpl$$EnhancerBySpringCGLIB$$9aed5b42.launchDeployment(<generated>)
>>
>>
>> at
>> com.huifu.kunpeng.runner.SubmitQueueApplicationRunner.lambda$run$0(SubmitQueueApplicationRunner.java:63)
>>
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>
>>
>> at java.lang.Thread.run(Thread.java:748)
>>
>>
>> Caused by: java.lang.LinkageError: loader constraint violation: loader
>> (instance of org/apache/flink/util/ChildFirstClassLoader) previously
>> initiated loading for a different type with name
>> "org/apache/kafka/clients/consumer/ConsumerRecord"
>>
>>
>> at java.lang.ClassLoader.defineClass1(Native Method)
>>
>>
>> at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
>>
>>
>> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>>
>>
>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
>>
>>
>> at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
>>
>>
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
>>
>>
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
>>
>>
>> at java.security.AccessController.doPrivileged(Native Method)
回复: Re: 提交flink作业抛 java.lang.LinkageError
Posted by WuKong <wu...@foxmail.com>.
Hi :
看报错日志，还是类加载问题 提示的报错信息 是说已经由不同类加载器已经加装了改依赖。如果生产环境上已经由了相关依赖包，建议将依赖设置为provided
Caused by: java.lang.LinkageError: loader constraint violation: loader
>> (instance of org/apache/flink/util/ChildFirstClassLoader) previously
>> initiated loading for a different type with name
>> "org/apache/kafka/clients/consumer/ConsumerRecord"
---
Best,
WuKong
发件人： casel.chen
发送时间： 2021-11-08 14:38
收件人： user-zh
主题： Re:Re: 提交flink作业抛 java.lang.LinkageError
版本是一致的，都是1.12.5版本
在 2021-11-08 11:11:35，"Shuiqiang Chen" <ac...@gmail.com> 写道：
>Hi,
>
>能检查下作业jar里 kafka client的版本和平台上的是否一致吗？
>
>casel.chen <ca...@126.com> 于2021年11月5日周五 下午11:25写道：
>
>> 我在公司实时计算平台上提交了一个streaming api写的作业，结果抛如下异常。因为我们的实时计算平台是以flink
>> sql为主的，上面已经集成了flink-kafka-connector。而我提交的作业也是需要从kafka消费，所以将相同版本的flink kafka
>> connector也打进了作业jar包内。请问是什么原因造成的，需要如何修复？谢谢！
>>
>>
>> 2021-11-05 16:38:58 - [submit-session-executor-6] ERROR
>> c.h.s.launcher.AbstractJobExecutor - -----start job failed-----
>>
>>
>> org.apache.flink.client.program.ProgramInvocationException: The program
>> caused an error:
>>
>>
>>
>>
>>
>>
>> Classpath:
>> [file:/opt/streamsql/jobs/aml-aml-aml/aml-datasync/TEST/aml-datasync-1.0-SNAPSHOT_zwb3274543418822102949.jar]
>>
>>
>>
>>
>>
>>
>> System.out: (none)
>>
>>
>>
>>
>>
>>
>> System.err: (none)
>>
>>
>> at
>> org.apache.flink.client.program.PackagedProgramUtils.generateException(PackagedProgramUtils.java:264)
>>
>>
>> at
>> org.apache.flink.client.program.PackagedProgramUtils.getPipelineFromProgram(PackagedProgramUtils.java:172)
>>
>>
>> at
>> com.huifu.streamsql.launcher.AbstractJobExecutor.createJobGraph(AbstractJobExecutor.java:205)
>>
>>
>> at
>> com.huifu.streamsql.launcher.standalone.RemoteExecutor.doStart(RemoteExecutor.java:31)
>>
>>
>> at
>> com.huifu.streamsql.launcher.AbstractJobExecutor.start(AbstractJobExecutor.java:51)
>>
>>
>> at com.huifu.streamsql.launcher.JobCommand$1.execute(JobCommand.java:15)
>>
>>
>> at
>> com.huifu.streamsql.service.StreamSqlServiceImpl.submitJob(StreamSqlServiceImpl.java:443)
>>
>>
>> at
>> com.huifu.kunpeng.service.DeploymentServiceImpl.submitJob(DeploymentServiceImpl.java:1662)
>>
>>
>> at
>> com.huifu.kunpeng.service.DeploymentServiceImpl.launchDeployment(DeploymentServiceImpl.java:1623)
>>
>>
>> at
>> com.huifu.kunpeng.service.DeploymentServiceImpl$$FastClassBySpringCGLIB$$855501cb.invoke(<generated>)
>>
>>
>> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
>>
>>
>> at
>> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
>>
>>
>> at
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
>>
>>
>> at
>> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
>>
>>
>> at
>> org.springframework.retry.annotation.AnnotationAwareRetryOperationsInterceptor.invoke(AnnotationAwareRetryOperationsInterceptor.java:156)
>>
>>
>> at
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
>>
>>
>> at
>> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
>>
>>
>> at
>> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
>>
>>
>> at
>> com.huifu.kunpeng.service.DeploymentServiceImpl$$EnhancerBySpringCGLIB$$9aed5b42.launchDeployment(<generated>)
>>
>>
>> at
>> com.huifu.kunpeng.runner.SubmitQueueApplicationRunner.lambda$run$0(SubmitQueueApplicationRunner.java:63)
>>
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>
>>
>> at java.lang.Thread.run(Thread.java:748)
>>
>>
>> Caused by: java.lang.LinkageError: loader constraint violation: loader
>> (instance of org/apache/flink/util/ChildFirstClassLoader) previously
>> initiated loading for a different type with name
>> "org/apache/kafka/clients/consumer/ConsumerRecord"
>>
>>
>> at java.lang.ClassLoader.defineClass1(Native Method)
>>
>>
>> at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
>>
>>
>> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>>
>>
>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
>>
>>
>> at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
>>
>>
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
>>
>>
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
>>
>>
>> at java.security.AccessController.doPrivileged(Native Method)
回复: Re: 提交flink作业抛 java.lang.LinkageError
Posted by WuKong <wu...@foxmail.com>.
Hi :
看报错日志，还是类加载问题 提示的报错信息 是说已经由不同类加载器已经加装了改依赖。如果生产环境上已经由了相关依赖包，建议将依赖设置为provided
Caused by: java.lang.LinkageError: loader constraint violation: loader
>> (instance of org/apache/flink/util/ChildFirstClassLoader) previously
>> initiated loading for a different type with name
>> "org/apache/kafka/clients/consumer/ConsumerRecord"
---
Best,
WuKong
发件人： casel.chen
发送时间： 2021-11-08 14:38
收件人： user-zh
主题： Re:Re: 提交flink作业抛 java.lang.LinkageError
版本是一致的，都是1.12.5版本
在 2021-11-08 11:11:35，"Shuiqiang Chen" <ac...@gmail.com> 写道：
>Hi,
>
>能检查下作业jar里 kafka client的版本和平台上的是否一致吗？
>
>casel.chen <ca...@126.com> 于2021年11月5日周五 下午11:25写道：
>
>> 我在公司实时计算平台上提交了一个streaming api写的作业，结果抛如下异常。因为我们的实时计算平台是以flink
>> sql为主的，上面已经集成了flink-kafka-connector。而我提交的作业也是需要从kafka消费，所以将相同版本的flink kafka
>> connector也打进了作业jar包内。请问是什么原因造成的，需要如何修复？谢谢！
>>
>>
>> 2021-11-05 16:38:58 - [submit-session-executor-6] ERROR
>> c.h.s.launcher.AbstractJobExecutor - -----start job failed-----
>>
>>
>> org.apache.flink.client.program.ProgramInvocationException: The program
>> caused an error:
>>
>>
>>
>>
>>
>>
>> Classpath:
>> [file:/opt/streamsql/jobs/aml-aml-aml/aml-datasync/TEST/aml-datasync-1.0-SNAPSHOT_zwb3274543418822102949.jar]
>>
>>
>>
>>
>>
>>
>> System.out: (none)
>>
>>
>>
>>
>>
>>
>> System.err: (none)
>>
>>
>> at
>> org.apache.flink.client.program.PackagedProgramUtils.generateException(PackagedProgramUtils.java:264)
>>
>>
>> at
>> org.apache.flink.client.program.PackagedProgramUtils.getPipelineFromProgram(PackagedProgramUtils.java:172)
>>
>>
>> at
>> com.huifu.streamsql.launcher.AbstractJobExecutor.createJobGraph(AbstractJobExecutor.java:205)
>>
>>
>> at
>> com.huifu.streamsql.launcher.standalone.RemoteExecutor.doStart(RemoteExecutor.java:31)
>>
>>
>> at
>> com.huifu.streamsql.launcher.AbstractJobExecutor.start(AbstractJobExecutor.java:51)
>>
>>
>> at com.huifu.streamsql.launcher.JobCommand$1.execute(JobCommand.java:15)
>>
>>
>> at
>> com.huifu.streamsql.service.StreamSqlServiceImpl.submitJob(StreamSqlServiceImpl.java:443)
>>
>>
>> at
>> com.huifu.kunpeng.service.DeploymentServiceImpl.submitJob(DeploymentServiceImpl.java:1662)
>>
>>
>> at
>> com.huifu.kunpeng.service.DeploymentServiceImpl.launchDeployment(DeploymentServiceImpl.java:1623)
>>
>>
>> at
>> com.huifu.kunpeng.service.DeploymentServiceImpl$$FastClassBySpringCGLIB$$855501cb.invoke(<generated>)
>>
>>
>> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
>>
>>
>> at
>> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
>>
>>
>> at
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
>>
>>
>> at
>> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
>>
>>
>> at
>> org.springframework.retry.annotation.AnnotationAwareRetryOperationsInterceptor.invoke(AnnotationAwareRetryOperationsInterceptor.java:156)
>>
>>
>> at
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
>>
>>
>> at
>> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
>>
>>
>> at
>> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
>>
>>
>> at
>> com.huifu.kunpeng.service.DeploymentServiceImpl$$EnhancerBySpringCGLIB$$9aed5b42.launchDeployment(<generated>)
>>
>>
>> at
>> com.huifu.kunpeng.runner.SubmitQueueApplicationRunner.lambda$run$0(SubmitQueueApplicationRunner.java:63)
>>
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>
>>
>> at java.lang.Thread.run(Thread.java:748)
>>
>>
>> Caused by: java.lang.LinkageError: loader constraint violation: loader
>> (instance of org/apache/flink/util/ChildFirstClassLoader) previously
>> initiated loading for a different type with name
>> "org/apache/kafka/clients/consumer/ConsumerRecord"
>>
>>
>> at java.lang.ClassLoader.defineClass1(Native Method)
>>
>>
>> at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
>>
>>
>> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>>
>>
>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
>>
>>
>> at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
>>
>>
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
>>
>>
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
>>
>>
>> at java.security.AccessController.doPrivileged(Native Method)
Re:Re: 提交flink作业抛 java.lang.LinkageError
Posted by "casel.chen" <ca...@126.com>.
版本是一致的，都是1.12.5版本
在 2021-11-08 11:11:35，"Shuiqiang Chen" <ac...@gmail.com> 写道：
>Hi,
>
>能检查下作业jar里 kafka client的版本和平台上的是否一致吗？
>
>casel.chen <ca...@126.com> 于2021年11月5日周五 下午11:25写道：
>
>> 我在公司实时计算平台上提交了一个streaming api写的作业，结果抛如下异常。因为我们的实时计算平台是以flink
>> sql为主的，上面已经集成了flink-kafka-connector。而我提交的作业也是需要从kafka消费，所以将相同版本的flink kafka
>> connector也打进了作业jar包内。请问是什么原因造成的，需要如何修复？谢谢！
>>
>>
>> 2021-11-05 16:38:58 - [submit-session-executor-6] ERROR
>> c.h.s.launcher.AbstractJobExecutor - -----start job failed-----
>>
>>
>> org.apache.flink.client.program.ProgramInvocationException: The program
>> caused an error:
>>
>>
>>
>>
>>
>>
>> Classpath:
>> [file:/opt/streamsql/jobs/aml-aml-aml/aml-datasync/TEST/aml-datasync-1.0-SNAPSHOT_zwb3274543418822102949.jar]
>>
>>
>>
>>
>>
>>
>> System.out: (none)
>>
>>
>>
>>
>>
>>
>> System.err: (none)
>>
>>
>> at
>> org.apache.flink.client.program.PackagedProgramUtils.generateException(PackagedProgramUtils.java:264)
>>
>>
>> at
>> org.apache.flink.client.program.PackagedProgramUtils.getPipelineFromProgram(PackagedProgramUtils.java:172)
>>
>>
>> at
>> com.huifu.streamsql.launcher.AbstractJobExecutor.createJobGraph(AbstractJobExecutor.java:205)
>>
>>
>> at
>> com.huifu.streamsql.launcher.standalone.RemoteExecutor.doStart(RemoteExecutor.java:31)
>>
>>
>> at
>> com.huifu.streamsql.launcher.AbstractJobExecutor.start(AbstractJobExecutor.java:51)
>>
>>
>> at com.huifu.streamsql.launcher.JobCommand$1.execute(JobCommand.java:15)
>>
>>
>> at
>> com.huifu.streamsql.service.StreamSqlServiceImpl.submitJob(StreamSqlServiceImpl.java:443)
>>
>>
>> at
>> com.huifu.kunpeng.service.DeploymentServiceImpl.submitJob(DeploymentServiceImpl.java:1662)
>>
>>
>> at
>> com.huifu.kunpeng.service.DeploymentServiceImpl.launchDeployment(DeploymentServiceImpl.java:1623)
>>
>>
>> at
>> com.huifu.kunpeng.service.DeploymentServiceImpl$$FastClassBySpringCGLIB$$855501cb.invoke(<generated>)
>>
>>
>> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
>>
>>
>> at
>> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
>>
>>
>> at
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
>>
>>
>> at
>> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
>>
>>
>> at
>> org.springframework.retry.annotation.AnnotationAwareRetryOperationsInterceptor.invoke(AnnotationAwareRetryOperationsInterceptor.java:156)
>>
>>
>> at
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
>>
>>
>> at
>> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
>>
>>
>> at
>> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
>>
>>
>> at
>> com.huifu.kunpeng.service.DeploymentServiceImpl$$EnhancerBySpringCGLIB$$9aed5b42.launchDeployment(<generated>)
>>
>>
>> at
>> com.huifu.kunpeng.runner.SubmitQueueApplicationRunner.lambda$run$0(SubmitQueueApplicationRunner.java:63)
>>
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>
>>
>> at java.lang.Thread.run(Thread.java:748)
>>
>>
>> Caused by: java.lang.LinkageError: loader constraint violation: loader
>> (instance of org/apache/flink/util/ChildFirstClassLoader) previously
>> initiated loading for a different type with name
>> "org/apache/kafka/clients/consumer/ConsumerRecord"
>>
>>
>> at java.lang.ClassLoader.defineClass1(Native Method)
>>
>>
>> at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
>>
>>
>> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>>
>>
>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
>>
>>
>> at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
>>
>>
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
>>
>>
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
>>
>>
>> at java.security.AccessController.doPrivileged(Native Method)
Re: 提交flink作业抛 java.lang.LinkageError
Posted by Shuiqiang Chen <ac...@gmail.com>.
Hi,
能检查下作业jar里 kafka client的版本和平台上的是否一致吗？
casel.chen <ca...@126.com> 于2021年11月5日周五 下午11:25写道：
> 我在公司实时计算平台上提交了一个streaming api写的作业，结果抛如下异常。因为我们的实时计算平台是以flink
> sql为主的，上面已经集成了flink-kafka-connector。而我提交的作业也是需要从kafka消费，所以将相同版本的flink kafka
> connector也打进了作业jar包内。请问是什么原因造成的，需要如何修复？谢谢！
>
>
> 2021-11-05 16:38:58 - [submit-session-executor-6] ERROR
> c.h.s.launcher.AbstractJobExecutor - -----start job failed-----
>
>
> org.apache.flink.client.program.ProgramInvocationException: The program
> caused an error:
>
>
>
>
>
>
> Classpath:
> [file:/opt/streamsql/jobs/aml-aml-aml/aml-datasync/TEST/aml-datasync-1.0-SNAPSHOT_zwb3274543418822102949.jar]
>
>
>
>
>
>
> System.out: (none)
>
>
>
>
>
>
> System.err: (none)
>
>
> at
> org.apache.flink.client.program.PackagedProgramUtils.generateException(PackagedProgramUtils.java:264)
>
>
> at
> org.apache.flink.client.program.PackagedProgramUtils.getPipelineFromProgram(PackagedProgramUtils.java:172)
>
>
> at
> com.huifu.streamsql.launcher.AbstractJobExecutor.createJobGraph(AbstractJobExecutor.java:205)
>
>
> at
> com.huifu.streamsql.launcher.standalone.RemoteExecutor.doStart(RemoteExecutor.java:31)
>
>
> at
> com.huifu.streamsql.launcher.AbstractJobExecutor.start(AbstractJobExecutor.java:51)
>
>
> at com.huifu.streamsql.launcher.JobCommand$1.execute(JobCommand.java:15)
>
>
> at
> com.huifu.streamsql.service.StreamSqlServiceImpl.submitJob(StreamSqlServiceImpl.java:443)
>
>
> at
> com.huifu.kunpeng.service.DeploymentServiceImpl.submitJob(DeploymentServiceImpl.java:1662)
>
>
> at
> com.huifu.kunpeng.service.DeploymentServiceImpl.launchDeployment(DeploymentServiceImpl.java:1623)
>
>
> at
> com.huifu.kunpeng.service.DeploymentServiceImpl$$FastClassBySpringCGLIB$$855501cb.invoke(<generated>)
>
>
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
>
>
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
>
>
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
>
>
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
>
>
> at
> org.springframework.retry.annotation.AnnotationAwareRetryOperationsInterceptor.invoke(AnnotationAwareRetryOperationsInterceptor.java:156)
>
>
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
>
>
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
>
>
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
>
>
> at
> com.huifu.kunpeng.service.DeploymentServiceImpl$$EnhancerBySpringCGLIB$$9aed5b42.launchDeployment(<generated>)
>
>
> at
> com.huifu.kunpeng.runner.SubmitQueueApplicationRunner.lambda$run$0(SubmitQueueApplicationRunner.java:63)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>
>
> at java.lang.Thread.run(Thread.java:748)
>
>
> Caused by: java.lang.LinkageError: loader constraint violation: loader
> (instance of org/apache/flink/util/ChildFirstClassLoader) previously
> initiated loading for a different type with name
> "org/apache/kafka/clients/consumer/ConsumerRecord"
>
>
> at java.lang.ClassLoader.defineClass1(Native Method)
>
>
> at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
>
>
> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>
>
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
>
>
> at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
>
>
> at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
>
>
> at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
>
>
> at java.security.AccessController.doPrivileged(Native Method)