You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Dulce Morim <Du...@inetum.com> on 2023/11/26 21:51:44 UTC
Error flink 1.18 not found ExecutionConfig
Hello,
I'm using Flink 1.18 and have this error, any idea? Works in IDE.
2023-11-24 16:04:47.062 [DEBUG] [jobmanager-io-thread-1] DefaultJobMasterServiceProcess - Initialization of the JobMasterService for job abbad350a6a161919104f8e2fe2670b8 under leader id b5a63392-d3e4-49d8-85d1-52cd3b71bf89 failed.
org.apache.flink.runtime.client.JobInitializationException: Could not start the JobMaster.
at deployment.teste.ear.teste.war/org.apache.flink.runtime.jobmaster.DefaultJobMasterServiceProcess.lambda$new$0(DefaultJobMasterServiceProcess.java:97) ~[flink-runtime-1.18.0.jar:1.18.0]
at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) [?:?]
at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) [?:?]
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) [?:?]
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1773) [?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.base/java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: java.util.concurrent.CompletionException: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.flink.api.common.ExecutionConfig
at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320) ~[?:?]
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1770) ~[?:?]
... 3 more
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.flink.api.common.ExecutionConfig
at deployment.teste.ear.teste.war/org.apache.flink.util.ExceptionUtils.rethrow(ExceptionUtils.java:321) ~[flink-core-1.18.0.jar:1.18.0]
at deployment.teste.ear.teste.war/org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:114) ~[flink-core-1.18.0.jar:1.18.0]
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768) ~[?:?]
... 3 more
Caused by: java.lang.ClassNotFoundException: org.apache.flink.api.common.ExecutionConfig
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641) ~[?:?]
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188) ~[?:?]
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520) ~[?:?]
at java.base/java.lang.Class.forName0(Native Method) ~[?:?]
at java.base/java.lang.Class.forName(Class.java:467) ~[?:?]
at deployment.teste.ear.teste.war/org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:78) ~[flink-core-1.18.0.jar:1.18.0]
at java.base/java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:2043) ~[?:?]
at java.base/java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1907) ~[?:?]
at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2209) ~[?:?]
at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1742) ~[?:?]
at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:514) ~[?:?]
at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:472) ~[?:?]
at deployment.teste.ear.teste.war/org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:539) ~[flink-core-1.18.0.jar:1.18.0]
at deployment.teste.ear.teste.war/org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:527) ~[flink-core-1.18.0.jar:1.18.0]
at deployment.teste.ear.teste.war/org.apache.flink.util.SerializedValue.deserializeValue(SerializedValue.java:67) ~[flink-core-1.18.0.jar:1.18.0]
at deployment.teste.ear.teste.war/org.apache.flink.runtime.scheduler.adaptivebatch.AdaptiveBatchSchedulerFactory.createInstance(AdaptiveBatchSchedulerFactory.java:149) ~[flink-runtime-1.18.0.jar:1.18.0]
at deployment.teste.ear.teste.war/org.apache.flink.runtime.jobmaster.DefaultSlotPoolServiceSchedulerFactory.createScheduler(DefaultSlotPoolServiceSchedulerFactory.java:122) ~[flink-runtime-1.18.0.jar:1.18.0]
at deployment.teste.ear.teste.war/org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:379) ~[flink-runtime-1.18.0.jar:1.18.0]
at deployment.teste.ear.teste.war/org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:356) ~[flink-runtime-1.18.0.jar:1.18.0]
at deployment.teste.ear.teste.war/org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.internalCreateJobMasterService(DefaultJobMasterServiceFactory.java:128) ~[flink-runtime-1.18.0.jar:1.18.0]
at deployment.teste.ear.teste.war/org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.lambda$createJobMasterService$0(DefaultJobMasterServiceFactory.java:100) ~[flink-runtime-1.18.0.jar:1.18.0]
at deployment.teste.ear.teste.war/org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:112) ~[flink-core-1.18.0.jar:1.18.0]
at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768) ~[?:?]
... 3 more
Thanks,
Dulce Morim
Flink Slow Execution
Posted by Dulce Morim <Du...@inetum.com>.
Hello,
In a single JVM, I'm running multiple flink batch jobs locally using the MiniCluster (Java 17 and Flink 1.18).
At the beginning of the process, the Mini Cluster starts pretty much instantly. However, I'm running into an issue where the more jobs I execute the longer the MiniCluster takes to start.
Here's an example:
2024-01-17 17:07:26.989 [INFO ] MiniCluster - Starting Flink Mini Cluster
2024-01-17 17:07:27.165 [INFO ] MiniCluster - Starting Metrics Registry
2024-01-17 17:07:33.801 [INFO ] MetricRegistryImpl - No metrics reporter configured, no metrics will be exposed/reported.
2024-01-17 17:07:33.801 [INFO ] MiniCluster - Starting RPC Service(s)
2024-01-17 17:07:34.646 [INFO ] MiniCluster - Flink Mini Cluster started successfully
Has anyone faced this issue?
Thanks.
Re: Error flink 1.18 not found ExecutionConfig
Posted by Yunfeng Zhou <fl...@gmail.com>.
Hi Dulce,
Thanks for your clarification. From my experience it is usually a
dependency problem, and you may verify this by checking whether you
have correctly included the dependencies in your jar (like `jar -tf
xxx.jar | grep ExecutionConfig`), and whether the dependencies in your
local environment for Flink job submission and the environment of the
Flink cluster are the same.
Best,
Yunfeng Zhou
On Mon, Nov 27, 2023 at 8:39 PM Dulce Morim <Du...@inetum.com> wrote:
>
> Hi,
>
> In my IDE it works. I'm trying to run it in the EAR with all dependencies jars, but it gives an error. Could it be problems with dependencies or execution options?
>
> Thanks,
> Dulce Morim
>
> -----Original Message-----
> From: Yunfeng Zhou <fl...@gmail.com>
> Sent: 27 de novembro de 2023 01:26
> To: Dulce Morim <Du...@inetum.com>
> Subject: Re: Error flink 1.18 not found ExecutionConfig
>
> CAUTION: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
>
>
> Hi Dulce,
>
> You may need to select the "Include dependencies with 'Provided'
> scope" configuration before running your program in IntelliJ IDEA, if this is the IDE you are using.
>
> Best regards,
> Yunfeng Zhou
>
> On Mon, Nov 27, 2023 at 5:51 AM Dulce Morim <Du...@inetum.com> wrote:
> >
> > Hello,
> >
> >
> >
> > I’m using Flink 1.18 and have this error, any idea? Works in IDE.
> >
> >
> >
> > 2023-11-24 16:04:47.062 [DEBUG] [jobmanager-io-thread-1] DefaultJobMasterServiceProcess - Initialization of the JobMasterService for job abbad350a6a161919104f8e2fe2670b8 under leader id b5a63392-d3e4-49d8-85d1-52cd3b71bf89 failed.
> >
> > org.apache.flink.runtime.client.JobInitializationException: Could not start the JobMaster.
> >
> > at
> > deployment.teste.ear.teste.war/org.apache.flink.runtime.jobmaster.Defa
> > ultJobMasterServiceProcess.lambda$new$0(DefaultJobMasterServiceProcess
> > .java:97) ~[flink-runtime-1.18.0.jar:1.18.0]
> >
> > at
> > java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(Compl
> > etableFuture.java:863) [?:?]
> >
> > at
> > java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFi
> > re(CompletableFuture.java:841) [?:?]
> >
> > at
> > java.base/java.util.concurrent.CompletableFuture.postComplete(Completa
> > bleFuture.java:510) [?:?]
> >
> > at
> > java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(Compl
> > etableFuture.java:1773) [?:?]
> >
> > at
> > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> > Executor.java:1136) [?:?]
> >
> > at
> > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> > lExecutor.java:635) [?:?]
> >
> > at java.base/java.lang.Thread.run(Thread.java:833) [?:?]
> >
> > Caused by: java.util.concurrent.CompletionException:
> > java.lang.RuntimeException: java.lang.ClassNotFoundException:
> > org.apache.flink.api.common.ExecutionConfig
> >
> > at
> > java.base/java.util.concurrent.CompletableFuture.encodeThrowable(Compl
> > etableFuture.java:315) ~[?:?]
> >
> > at
> > java.base/java.util.concurrent.CompletableFuture.completeThrowable(Com
> > pletableFuture.java:320) ~[?:?]
> >
> > at
> > java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(Compl
> > etableFuture.java:1770) ~[?:?]
> >
> > ... 3 more
> >
> > Caused by: java.lang.RuntimeException:
> > java.lang.ClassNotFoundException:
> > org.apache.flink.api.common.ExecutionConfig
> >
> > at
> > deployment.teste.ear.teste.war/org.apache.flink.util.ExceptionUtils.re
> > throw(ExceptionUtils.java:321) ~[flink-core-1.18.0.jar:1.18.0]
> >
> > at
> > deployment.teste.ear.teste.war/org.apache.flink.util.function.Function
> > Utils.lambda$uncheckedSupplier$4(FunctionUtils.java:114)
> > ~[flink-core-1.18.0.jar:1.18.0]
> >
> > at
> > java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(Compl
> > etableFuture.java:1768) ~[?:?]
> >
> > ... 3 more
> >
> > Caused by: java.lang.ClassNotFoundException:
> > org.apache.flink.api.common.ExecutionConfig
> >
> > at
> > java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClas
> > sLoader.java:641) ~[?:?]
> >
> > at
> > java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Cl
> > assLoaders.java:188) ~[?:?]
> >
> > at
> > java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520) ~[?:?]
> >
> > at java.base/java.lang.Class.forName0(Native Method)
> > ~[?:?]
> >
> > at java.base/java.lang.Class.forName(Class.java:467)
> > ~[?:?]
> >
> > at
> > deployment.teste.ear.teste.war/org.apache.flink.util.InstantiationUtil
> > $ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:78)
> > ~[flink-core-1.18.0.jar:1.18.0]
> >
> > at
> > java.base/java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream
> > .java:2043) ~[?:?]
> >
> > at
> > java.base/java.io.ObjectInputStream.readClassDesc(ObjectInputStream.ja
> > va:1907) ~[?:?]
> >
> > at
> > java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStre
> > am.java:2209) ~[?:?]
> >
> > at
> > java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java
> > :1742) ~[?:?]
> >
> > at
> > java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:
> > 514) ~[?:?]
> >
> > at
> > java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:
> > 472) ~[?:?]
> >
> > at
> > deployment.teste.ear.teste.war/org.apache.flink.util.InstantiationUtil
> > .deserializeObject(InstantiationUtil.java:539)
> > ~[flink-core-1.18.0.jar:1.18.0]
> >
> > at
> > deployment.teste.ear.teste.war/org.apache.flink.util.InstantiationUtil
> > .deserializeObject(InstantiationUtil.java:527)
> > ~[flink-core-1.18.0.jar:1.18.0]
> >
> > at
> > deployment.teste.ear.teste.war/org.apache.flink.util.SerializedValue.d
> > eserializeValue(SerializedValue.java:67)
> > ~[flink-core-1.18.0.jar:1.18.0]
> >
> > at
> > deployment.teste.ear.teste.war/org.apache.flink.runtime.scheduler.adap
> > tivebatch.AdaptiveBatchSchedulerFactory.createInstance(AdaptiveBatchSc
> > hedulerFactory.java:149) ~[flink-runtime-1.18.0.jar:1.18.0]
> >
> > at
> > deployment.teste.ear.teste.war/org.apache.flink.runtime.jobmaster.Defa
> > ultSlotPoolServiceSchedulerFactory.createScheduler(DefaultSlotPoolServ
> > iceSchedulerFactory.java:122) ~[flink-runtime-1.18.0.jar:1.18.0]
> >
> > at
> > deployment.teste.ear.teste.war/org.apache.flink.runtime.jobmaster.JobM
> > aster.createScheduler(JobMaster.java:379)
> > ~[flink-runtime-1.18.0.jar:1.18.0]
> >
> > at
> > deployment.teste.ear.teste.war/org.apache.flink.runtime.jobmaster.JobM
> > aster.<init>(JobMaster.java:356) ~[flink-runtime-1.18.0.jar:1.18.0]
> >
> > at
> > deployment.teste.ear.teste.war/org.apache.flink.runtime.jobmaster.fact
> > ories.DefaultJobMasterServiceFactory.internalCreateJobMasterService(De
> > faultJobMasterServiceFactory.java:128)
> > ~[flink-runtime-1.18.0.jar:1.18.0]
> >
> > at
> > deployment.teste.ear.teste.war/org.apache.flink.runtime.jobmaster.fact
> > ories.DefaultJobMasterServiceFactory.lambda$createJobMasterService$0(D
> > efaultJobMasterServiceFactory.java:100)
> > ~[flink-runtime-1.18.0.jar:1.18.0]
> >
> > at
> > deployment.teste.ear.teste.war/org.apache.flink.util.function.Function
> > Utils.lambda$uncheckedSupplier$4(FunctionUtils.java:112)
> > ~[flink-core-1.18.0.jar:1.18.0]
> >
> > at
> > java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(Compl
> > etableFuture.java:1768) ~[?:?]
> >
> > ... 3 more
> >
> >
> >
> > Thanks,
> >
> > Dulce Morim