You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@linkis.apache.org by GitBox <gi...@apache.org> on 2022/08/14 06:16:28 UTC
[GitHub] [incubator-linkis] jasonmrliu opened a new issue, #2762: linkis1.1.1启动成功后提交任务给Spark引擎执行出错问题[Bug]
jasonmrliu opened a new issue, #2762:
URL: https://github.com/apache/incubator-linkis/issues/2762
### Search before asking
- [X] I searched the [issues](https://github.com/apache/incubator-linkis/issues) and found no similar issues.
### Linkis Component
linkis-engineconnn-plugin
### Steps to reproduce
1. 所有linkis安装完成并成功启动
2. 运行命令：```sh linkis-cli -submitUser hadoop -engineType spark-2.2.0.cloudera1 -codeType sql -code "show tables"```，测试spark引擎状态，报错，控制台错误如下：
```
=====Java Start Command=====
exec /usr/java/default//bin/java -server -Xms32m -Xmx2048m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/linkis/logs/linkis-cli -XX:ErrorFile=/opt/linkis/logs/linkis-cli/ps_err_pid%p.log -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=80 -XX:+DisableExplicitGC -classpath /opt/linkis/conf/linkis-cli:/opt/linkis/lib/linkis-computation-governance/linkis-client/linkis-cli/*:/opt/linkis/lib/linkis-commons/public-module/*: -Dconf.root=/opt/linkis/conf/linkis-cli -Dconf.file=linkis-cli.properties -Dlog.path=/opt/linkis/logs/linkis-cli -Dlog.file=linkis-client.hadoop.log.20220812201515831679477 org.apache.linkis.cli.application.LinkisClientApplication '-submitUser hadoop -engineType spark-2.2.0.cloudera1 -codeType sql -code show tables'
[INFO] LogFile path: /opt/linkis/logs/linkis-cli/linkis-client.hadoop.log.20220812201515831679477
[INFO] User does not provide usr-configuration file. Will use default config
[INFO] user does not specify proxy-user, will use current submit-user "hadoop" by default.
[INFO] connecting to linkis gateway:http://127.0.0.1:9001
JobId:33
TaskId:33
ExecId:exec_id018019linkis-cg-entrance172.18.192.103:9104LINKISCLI_hadoop_spark_0
[INFO] Job is successfully submitted!
2022-08-12 20:15:19.015 INFO Program is substituting variables for you
2022-08-12 20:15:19.015 INFO Variables substitution ended successfully
2022-08-12 20:15:20.015 WARN The code you submit will not be limited by the limit
Job with jobId : LINKISCLI_hadoop_spark_0 and execID : LINKISCLI_hadoop_spark_0 submitted
2022-08-12 20:15:21.015 INFO You have submitted a new job, script code (after variable substitution) is
************************************SCRIPT CODE************************************
show tables
************************************SCRIPT CODE************************************
2022-08-12 20:15:21.015 INFO Your job is accepted, jobID is LINKISCLI_hadoop_spark_0 and jobReqId is 33 in ServiceInstance(linkis-cg-entrance, 172.18.192.103:9104). Please wait it to be scheduled
job is scheduled.
2022-08-12 20:15:21.015 INFO Your job is Scheduled. Please wait it to run.
Your job is being scheduled by orchestrator.
2022-08-12 20:15:21.015 INFO job is running.
2022-08-12 20:15:21.015 INFO Your job is Running now. Please wait it to complete.
2022-08-12 20:15:21.015 INFO Job with jobGroupId : 33 and subJobId : 32 was submitted to Orchestrator.
2022-08-12 20:15:21.015 INFO Background is starting a new engine for you,execId astJob_0_codeExec_0 mark id is mark_0, it may take several seconds, please wait
2022-08-12 20:15:42.015 ERROR Job with execId-LINKISCLI_hadoop_spark_0 + subJobId : 32 execute failed,21304, Task is Failed,errorMsg: errCode: 12003 ,desc: 172.18.192.103:9101_0 Failed to async get EngineNode AMErrorException: errCode: 30002 ,desc: ServiceInstance(linkis-cg-engineconn, 172.18.192.103:16786) ticketID:543ec8fe-9406-446a-88f8-351d56e34d2c 初始化引擎失败,原因: Failed to start EngineConn, reason: ECMErrorException: errCode: 11102 ,desc: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=250m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/appcom/tmp/engineConnPublickDir/315bd4d0-f2c4-49a5-ae3c-835618c55fed/v000003/lib/log4j-slf4j-impl-2.17.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
20:15:40.608 ERROR org.springframework.boot.SpringApplication 834 reportFailure - Application run failed org.springframework.context.ApplicationContextException: Failed to start bean 'webServerStartStop'; nested exception is org.springframework.boot.web.server.WebServerException: Unable to start embedded Jetty server
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:185) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:53) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:360) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:158) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:122) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:895) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:554) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:143) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:755) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:402) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:312) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.apache.linkis.DataWorkCloudApplication.main(DataWorkCloudApplication.java:121) ~[linkis-module-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.callback.hook.CallbackEngineConnHook.beforeCreateEngineConn(CallbackEngineConnHook.scala:54) ~[linkis-callback-service-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.launch.EngineConnServer$$anonfun$main$5.apply(EngineConnServer.scala:61) ~[linkis-engineconn-core-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.launch.EngineConnServer$$anonfun$main$5.apply(EngineConnServer.scala:61) ~[linkis-engineconn-core-1.1.1.jar:1.1.1]
at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) ~[scala-library-2.11.8.jar:?]
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186) ~[scala-library-2.11.8.jar:?]
at org.apache.linkis.engineconn.launch.EngineConnServer$.main(EngineConnServer.scala:61) ~[linkis-engineconn-core-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.launch.EngineConnServer.main(EngineConnServer.scala) ~[linkis-engineconn-core-1.1.1.jar:1.1.1]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_141]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_141]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_141]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_141]
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:755) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
Caused by: org.springframework.boot.web.server.WebServerException: Unable to start embedded Jetty server
at org.springframework.boot.web.embedded.jetty.JettyWebServer.start(JettyWebServer.java:194) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.web.servlet.context.WebServerStartStopLifecycle.start(WebServerStartStopLifecycle.java:43) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:182) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
... 28 more
Caused by: java.lang.IllegalStateException
at org.eclipse.jetty.servlet.ServletHolder.setClassFrom(ServletHolder.java:300) ~[jetty-servlet-9.4.42.v20210604.jar:9.4.42.v20210604]
at org.eclipse.jetty.servlet.ServletHolder.doStart(ServletHolder.java:347) ~[jetty-servlet-9.4.42.v20210604.jar:9.4.42.v20210604]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) ~[jetty-util-9.4.42.v20210604.jar:9.4.42.v20210604]
at org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:730) ~[jetty-servlet-9.4.42.v20210604.jar:9.4.42.v20210604]
at java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:352) ~[?:1.8.0_141]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0_141]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0_141]
at java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:312) ~[?:1.8.0_141]
at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:743) ~[?:1.8.0_141]
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580) ~[?:1.8.0_141]
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:755) ~[jetty-servlet-9.4.42.v20210604.jar:9.4.42.v20210604]
at org.springframework.boot.web.embedded.jetty.JettyEmbeddedWebAppContext$JettyEmbeddedServletHandler.deferredInitialize(JettyEmbeddedWebAppContext.java:46) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.web.embedded.jetty.JettyEmbeddedWebAppContext.deferredInitialize(JettyEmbeddedWebAppContext.java:36) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.web.embedded.jetty.JettyWebServer.handleDeferredInitialize(JettyWebServer.java:233) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.web.embedded.jetty.JettyWebServer.start(JettyWebServer.java:169) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.web.servlet.context.WebServerStartStopLifecycle.start(WebServerStartStopLifecycle.java:43) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:182) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
... 28 more
20:15:40.618 ERROR org.apache.linkis.common.utils.Logging$class 58 error - EngineConnServer Start Failed. org.springframework.context.ApplicationContextException: Failed to start bean 'webServerStartStop'; nested exception is org.springframework.boot.web.server.WebServerException: Unable to start embedded Jetty server
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:185) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:53) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:360) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:158) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:122) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:895) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:554) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:143) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:755) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:402) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:312) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.apache.linkis.DataWorkCloudApplication.main(DataWorkCloudApplication.java:121) ~[linkis-module-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.callback.hook.CallbackEngineConnHook.beforeCreateEngineConn(CallbackEngineConnHook.scala:54) ~[linkis-callback-service-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.launch.EngineConnServer$$anonfun$main$5.apply(EngineConnServer.scala:61) ~[linkis-engineconn-core-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.launch.EngineConnServer$$anonfun$main$5.apply(EngineConnServer.scala:61) ~[linkis-engineconn-core-1.1.1.jar:1.1.1]
at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) ~[scala-library-2.11.8.jar:?]
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186) ~[scala-library-2.11.8.jar:?]
at org.apache.linkis.engineconn.launch.EngineConnServer$.main(EngineConnServer.scala:61) ~[linkis-engineconn-core-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.launch.EngineConnServer.main(EngineConnServer.scala) ~[linkis-engineconn-core-1.1.1.jar:1.1.1]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_141]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_141]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_141]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_141]
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:755) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
Caused by: org.springframework.boot.web.server.WebServerException: Unable to start embedded Jetty server
at org.springframework.boot.web.embedded.jetty.JettyWebServer.start(JettyWebServer.java:194) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.web.servlet.context.WebServerStartStopLifecycle.start(WebServerStartStopLifecycle.java:43) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:182) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
... 28 more
Caused by: java.lang.IllegalStateException
at org.eclipse.jetty.servlet.ServletHolder.setClassFrom(ServletHolder.java:300) ~[jetty-servlet-9.4.42.v20210604.jar:9.4.42.v20210604]
at org.eclipse.jetty.servlet.ServletHolder.doStart(ServletHolder.java:347) ~[jetty-servlet-9.4.42.v20210604.jar:9.4.42.v20210604]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) ~[jetty-util-9.4.42.v20210604.jar:9.4.42.v20210604]
at org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:730) ~[jetty-servlet-9.4.42.v20210604.jar:9.4.42.v20210604]
at java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:352) ~[?:1.8.0_141]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0_141]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0_141]
at java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:312) ~[?:1.8.0_141]
at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:743) ~[?:1.8.0_141]
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580) ~[?:1.8.0_141]
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:755) ~[jetty-servlet-9.4.42.v20210604.jar:9.4.42.v20210604]
at org.springframework.boot.web.embedded.jetty.JettyEmbeddedWebAppContext$JettyEmbeddedServletHandler.deferredInitialize(JettyEmbeddedWebAppContext.java:46) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.web.embedded.jetty.JettyEmbeddedWebAppContext.deferredInitialize(JettyEmbeddedWebAppContext.java:36) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.web.embedded.jetty.JettyWebServer.handleDeferredInitialize(JettyWebServer.java:233) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.web.embedded.jetty.JettyWebServer.start(JettyWebServer.java:169) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.web.servlet.context.WebServerStartStopLifecycle.start(WebServerStartStopLifecycle.java:43) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:182) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
... 28 more
20:15:40.641 ERROR org.apache.linkis.engineconn.callback.service.AbstractEngineConnStartUpCallback 46 callback - protocol will send to em: EngineConnStatusCallback(ServiceInstance(linkis-cg-engineconn, 172.18.192.103:16786),543ec8fe-9406-446a-88f8-351d56e34d2c,Failed,ServiceInstance(linkis-cg-engineconn, 172.18.192.103:16786): log dir: /appcom/tmp/hadoop/20220812/spark/543ec8fe-9406-446a-88f8-351d56e34d2c/logs,IllegalStateException: )
20:15:40.682 ERROR org.apache.linkis.common.utils.Utils$$anonfun$tryAndError$1 107 apply - java.lang.IllegalStateException: org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext@7b80af04 has not been refreshed yet
at org.springframework.context.support.AbstractApplicationContext.assertBeanFactoryActive(AbstractApplicationContext.java:1096) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.getBeansOfType(AbstractApplicationContext.java:1242) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.apache.linkis.rpc.RPCSpringBeanCache$.getRPCInterceptors(RPCSpringBeanCache.scala:68) ~[linkis-rpc-1.1.1.jar:1.1.1]
at org.apache.linkis.rpc.sender.SpringMVCRPCSender.getRPCInterceptors(SpringMVCRPCSender.scala:43) ~[linkis-rpc-1.1.1.jar:1.1.1]
at org.apache.linkis.rpc.BaseRPCSender.execute(BaseRPCSender.scala:77) ~[linkis-rpc-1.1.1.jar:1.1.1]
at org.apache.linkis.rpc.BaseRPCSender.sendIt(BaseRPCSender.scala:98) ~[linkis-rpc-1.1.1.jar:1.1.1]
at org.apache.linkis.rpc.BaseRPCSender.send(BaseRPCSender.scala:107) ~[linkis-rpc-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.callback.service.AbstractEngineConnStartUpCallback.callback(EngineConnCallback.scala:53) ~[linkis-callback-service-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.callback.hook.CallbackEngineConnHook$$anonfun$afterEngineServerStartFailed$1.apply$mcV$sp(CallbackEngineConnHook.scala:68) ~[linkis-callback-service-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.callback.hook.CallbackEngineConnHook$$anonfun$afterEngineServerStartFailed$1.apply(CallbackEngineConnHook.scala:68) ~[linkis-callback-service-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.callback.hook.CallbackEngineConnHook$$anonfun$afterEngineServerStartFailed$1.apply(CallbackEngineConnHook.scala:68) ~[linkis-callback-service-1.1.1.jar:1.1.1]
at org.apache.linkis.common.utils.Utils$.tryCatch(Utils.scala:40) ~[linkis-common-1.1.1.jar:1.1.1]
at org.apache.linkis.common.utils.Utils$.tryAndError(Utils.scala:97) ~[linkis-common-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.callback.hook.CallbackEngineConnHook.afterEngineServerStartFailed(CallbackEngineConnHook.scala:68) ~[linkis-callback-service-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.launch.EngineConnServer$$anonfun$main$16.apply(EngineConnServer.scala:81) ~[linkis-engineconn-core-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.launch.EngineConnServer$$anonfun$main$16.apply(EngineConnServer.scala:81) ~[linkis-engineconn-core-1.1.1.jar:1.1.1]
at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) ~[scala-library-2.11.8.jar:?]
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186) ~[scala-library-2.11.8.jar:?]
at org.apache.linkis.engineconn.launch.EngineConnServer$.main(EngineConnServer.scala:81) ~[linkis-engineconn-core-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.launch.EngineConnServer.main(EngineConnServer.scala) ~[linkis-engineconn-core-1.1.1.jar:1.1.1]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_141]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_141]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_141]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_141]
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:755) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
20:15:40.684 ERROR org.apache.linkis.common.utils.Logging$class 58 error - EngineConnSever start failed! now exit. org.springframework.context.ApplicationContextException: Failed to start bean 'webServerStartStop'; nested exception is org.springframework.boot.web.server.WebServerException: Unable to start embedded Jetty server
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:185) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:53) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:360) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:158) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:122) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:895) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:554) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:143) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:755) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:402) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:312) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.apache.linkis.DataWorkCloudApplication.main(DataWorkCloudApplication.java:121) ~[linkis-module-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.callback.hook.CallbackEngineConnHook.beforeCreateEngineConn(CallbackEngineConnHook.scala:54) ~[linkis-callback-service-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.launch.EngineConnServer$$anonfun$main$5.apply(EngineConnServer.scala:61) ~[linkis-engineconn-core-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.launch.EngineConnServer$$anonfun$main$5.apply(EngineConnServer.scala:61) ~[linkis-engineconn-core-1.1.1.jar:1.1.1]
at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) ~[scala-library-2.11.8.jar:?]
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186) ~[scala-library-2.11.8.jar:?]
at org.apache.linkis.engineconn.launch.EngineConnServer$.main(EngineConnServer.scala:61) ~[linkis-engineconn-core-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.launch.EngineConnServer.main(EngineConnServer.scala) ~[linkis-engineconn-core-1.1.1.jar:1.1.1]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_141]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_141]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_141]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_141]
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:755) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
Caused by: org.springframework.boot.web.server.WebServerException: Unable to start embedded Jetty server
at org.springframework.boot.web.embedded.jetty.JettyWebServer.start(JettyWebServer.java:194) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.web.servlet.context.WebServerStartStopLifecycle.start(WebServerStartStopLifecycle.java:43) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:182) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
... 28 more
Caused by: java.lang.IllegalStateException
at org.eclipse.jetty.servlet.ServletHolder.setClassFrom(ServletHolder.java:300) ~[jetty-servlet-9.4.42.v20210604.jar:9.4.42.v20210604]
at org.eclipse.jetty.servlet.ServletHolder.doStart(ServletHolder.java:347) ~[jetty-servlet-9.4.42.v20210604.jar:9.4.42.v20210604]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) ~[jetty-util-9.4.42.v20210604.jar:9.4.42.v20210604]
at org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:730) ~[jetty-servlet-9.4.42.v20210604.jar:9.4.42.v20210604]
at java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:352) ~[?:1.8.0_141]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0_141]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0_141]
at java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:312) ~[?:1.8.0_141]
at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:743) ~[?:1.8.0_141]
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580) ~[?:1.8.0_141]
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:755) ~[jetty-servlet-9.4.42.v20210604.jar:9.4.42.v20210604]
at org.springframework.boot.web.embedded.jetty.JettyEmbeddedWebAppContext$JettyEmbeddedServletHandler.deferredInitialize(JettyEmbeddedWebAppContext.java:46) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.web.embedded.jetty.JettyEmbeddedWebAppContext.deferredInitialize(JettyEmbeddedWebAppContext.java:36) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.web.embedded.jetty.JettyWebServer.handleDeferredInitialize(JettyWebServer.java:233) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.web.embedded.jetty.JettyWebServer.start(JettyWebServer.java:169) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.web.servlet.context.WebServerStartStopLifecycle.start(WebServerStartStopLifecycle.java:43) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:182) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
... 28 more
20:15:40.686 ERROR org.apache.linkis.common.utils.Logging$class 58 error - process exit reason: org.springframework.context.ApplicationContextException: Failed to start bean 'webServerStartStop'; nested exception is org.springframework.boot.web.server.WebServerException: Unable to start embedded Jetty server
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:185) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:53) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:360) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:158) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:122) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:895) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:554) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:143) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:755) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:402) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:312) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.apache.linkis.DataWorkCloudApplication.main(DataWorkCloudApplication.java:121) ~[linkis-module-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.callback.hook.CallbackEngineConnHook.beforeCreateEngineConn(CallbackEngineConnHook.scala:54) ~[linkis-callback-service-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.launch.EngineConnServer$$anonfun$main$5.apply(EngineConnServer.scala:61) ~[linkis-engineconn-core-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.launch.EngineConnServer$$anonfun$main$5.apply(EngineConnServer.scala:61) ~[linkis-engineconn-core-1.1.1.jar:1.1.1]
at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) ~[scala-library-2.11.8.jar:?]
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186) ~[scala-library-2.11.8.jar:?]
at org.apache.linkis.engineconn.launch.EngineConnServer$.main(EngineConnServer.scala:61) ~[linkis-engineconn-core-1.1.1.jar:1.1.1]
at org.apache.linkis.engineconn.launch.EngineConnServer.main(EngineConnServer.scala) ~[linkis-engineconn-core-1.1.1.jar:1.1.1]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_141]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_141]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_141]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_141]
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:755) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) ~[spark-core_2.11-2.2.0.cloudera1.jar:2.2.0.cloudera1]
Caused by: org.springframework.boot.web.server.WebServerException: Unable to start embedded Jetty server
at org.springframework.boot.web.embedded.jetty.JettyWebServer.start(JettyWebServer.java:194) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.web.servlet.context.WebServerStartStopLifecycle.start(WebServerStartStopLifecycle.java:43) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:182) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
... 28 more
Caused by: java.lang.IllegalStateException
at org.eclipse.jetty.servlet.ServletHolder.setClassFrom(ServletHolder.java:300) ~[jetty-servlet-9.4.42.v20210604.jar:9.4.42.v20210604]
at org.eclipse.jetty.servlet.ServletHolder.doStart(ServletHolder.java:347) ~[jetty-servlet-9.4.42.v20210604.jar:9.4.42.v20210604]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) ~[jetty-util-9.4.42.v20210604.jar:9.4.42.v20210604]
at org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:730) ~[jetty-servlet-9.4.42.v20210604.jar:9.4.42.v20210604]
at java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:352) ~[?:1.8.0_141]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0_141]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0_141]
at java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:312) ~[?:1.8.0_141]
at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:743) ~[?:1.8.0_141]
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580) ~[?:1.8.0_141]
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:755) ~[jetty-servlet-9.4.42.v20210604.jar:9.4.42.v20210604]
at org.springframework.boot.web.embedded.jetty.JettyEmbeddedWebAppContext$JettyEmbeddedServletHandler.deferredInitialize(JettyEmbeddedWebAppContext.java:46) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.web.embedded.jetty.JettyEmbeddedWebAppContext.deferredInitialize(JettyEmbeddedWebAppContext.java:36) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.web.embedded.jetty.JettyWebServer.handleDeferredInitialize(JettyWebServer.java:233) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.web.embedded.jetty.JettyWebServer.start(JettyWebServer.java:169) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.boot.web.servlet.context.WebServerStartStopLifecycle.start(WebServerStartStopLifecycle.java:43) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:182) ~[spring-context-5.2.15.RELEASE.jar:5.2.15.RELEASE]
... 28 more
,ip: 172.18.192.103 ,port: 9102 ,serviceKind: linkis-cg-engineconnmanager
You can go to this path(/appcom/tmp/hadoop/20220812/spark/543ec8fe-9406-446a-88f8-351d56e34d2c/logs) to find the reason or ask the administrator for help ,ip: 172.18.192.103 ,port: 9101 ,serviceKind: linkis-cg-linkismanager ,ip: 172.18.192.103 ,port: 9104 ,serviceKind: linkis-cg-entrance
2022-08-12 20:15:43.015 INFO job is completed.
2022-08-12 20:15:43.015 INFO Task creation time(任务创建时间): 2022-08-12 20:15:18, Task scheduling time(任务调度时间): 2022-08-12 20:15:21, Task start time(任务开始时间): 2022-08-12 20:15:21, Mission end time(任务结束时间): 2022-08-12 20:15:43
2022-08-12 20:15:43.015 INFO Your mission(您的任务) 33 The total time spent is(总耗时时间为): 24.3 秒
2022-08-12 20:15:43.015 INFO Sorry. Your job completed with a status Failed. You can view logs for the reason.
[INFO] Job status is not success but 'FAILED'. Will not try to retrieve any Result
[INFO] Job failed! Will not try get execute result.
============Result:================
TaskId:33
ExecId: exec_id018019linkis-cg-entrance172.18.192.103:9104LINKISCLI_hadoop_spark_0
User:hadoop
Current job status:FAILED
extraMsg:
errDesc: 21304, Task is Failed,errorMsg: errCode: 12003 ,desc: 172.18.192.103:9101_0 Failed to async get EngineNode AMErrorException: errCode: 30002 ,desc: ServiceInstance(linkis-cg-engineconn, 172.18.192.103:16786) ticketID:543ec8fe-9406-446a-88f8-351d56e34d2c 初始
[INFO] JobStatus is not 'success'. Will not retrieve result-set.
############Execute Error!!!########
```
其他错误请见错误日志文件：
[linkis-cg-engineconnmanager.log](https://github.com/apache/incubator-linkis/files/9332795/linkis-cg-engineconnmanager.log)
[linkis-cg-entrance.log](https://github.com/apache/incubator-linkis/files/9332796/linkis-cg-entrance.log)
[stdout.log](https://github.com/apache/incubator-linkis/files/9332798/stdout.log)
### Expected behavior
启动spark的engineconn时生成的脚本如下：
```
#!/bin/bash
ulimit -c 0
export HADOOP_CONF_DIR="/etc/hadoop/conf"
export HIVE_CONF_DIR="/etc/hive/conf"
export JAVA_HOME="/usr/java/default/"
export LOG_DIRS="/appcom/tmp/hadoop/20220812/spark/543ec8fe-9406-446a-88f8-351d56e34d2c/logs"
export ECM_PORT="9102"
export EUREKA_PREFER_IP="true"
export USER="hadoop"
export ECM_HOST="bda01node03-adm.oracle.cn"
export ENGINECONN_ENVKEYS=""
export TEMP_DIRS="/appcom/tmp/hadoop/20220812/spark/543ec8fe-9406-446a-88f8-351d56e34d2c/tmp"
export HADOOP_HOME="/opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/lib/hadoop"
export ECM_HOME="/opt/linkis/conf/"
export UDF_JARS=""
export PWD="/appcom/tmp/hadoop/20220812/spark/543ec8fe-9406-446a-88f8-351d56e34d2c"
export RANDOM_PORT="37767"
export CLASSPATH="${PWD}/conf:/opt/linkis/conf:${PWD}/lib/*:${PWD}"
ln -sf "/appcom/tmp/engineConnPublickDir/315bd4d0-f2c4-49a5-ae3c-835618c55fed/v000003/lib" "/appcom/tmp/hadoop/20220812/spark/543ec8fe-9406-446a-88f8-351d56e34d2c/lib"
linkis_engineconn_errorcode=$?
if [ $linkis_engineconn_errorcode -ne 0 ]
then
cat ${LOG_DIRS}/stderr
exit $linkis_engineconn_errorcode
fi
ln -sf "/appcom/tmp/engineConnPublickDir/bf92d541-56e4-4cb1-aa7d-e9df95b3b1c1/v000002/conf" "/appcom/tmp/hadoop/20220812/spark/543ec8fe-9406-446a-88f8-351d56e34d2c/conf"
linkis_engineconn_errorcode=$?
if [ $linkis_engineconn_errorcode -ne 0 ]
then
cat ${LOG_DIRS}/stderr
exit $linkis_engineconn_errorcode
fi
spark-submit --master yarn --deploy-mode client --name Linkis-EngineConn-Spark_LINKISCLI --jars ,${UDF_JARS} --conf spark.driver.extraJavaOptions=" -server -XX:+UseG1GC -XX:MaxPermSize=250m -XX:PermSize=128m -Xloggc:${LOG_DIRS}/gc.log -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -Dwds.linkis.server.conf=linkis-engineconn.properties -Dwds.linkis.gateway.url=http://172.18.192.103:9001 -Dlogging.file=log4j2.xml -DTICKET_ID=543ec8fe-9406-446a-88f8-351d56e34d2c -Djava.io.tmpdir=${TEMP_DIRS} -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=${RANDOM_PORT}" --conf spark.tispark.pd.addresses="pd0:2379" --conf spark.tispark.tidb.port="4000" --conf spark.tispark.tidb.user="root" --conf spark.tispark.tidb.addr="tidb" --driver-memory 1g --driver-class-path :${CLASSPATH} --driver-cores 1 --executor-memory 1g --executor-cores 1 --num-executors 1 --queue default --class org.apache.linkis.engineconn.launch.EngineConnServer 1> ${LOG_DIRS}/stdout 2>> ${LOG_DIRS}/st
derr ${PWD}/lib/linkis-engineplugin-spark-1.1.1.jar --engineconn-conf wds.linkis.rm.instance=10 --engineconn-conf label.userCreator=hadoop-LINKISCLI --engineconn-conf ticketId=543ec8fe-9406-446a-88f8-351d56e34d2c --engineconn-conf wds.linkis.rm.yarnqueue.memory.max=300G --engineconn-conf spark.python.version=python2 --engineconn-conf label.engineType=spark-2.2.0.cloudera1 --engineconn-conf wds.linkis.rm.yarnqueue.instance.max=30 --engineconn-conf wds.linkis.rm.client.memory.max=20G --engineconn-conf wds.linkis.engineconn.max.free.time=1h --engineconn-conf wds.linkis.rm.client.core.max=10 --engineconn-conf wds.linkis.rm.yarnqueue.cores.max=150 --engineconn-conf user=hadoop --engineconn-conf wds.linkis.rm.yarnqueue=default --spring-conf eureka.client.serviceUrl.defaultZone=http://172.18.192.103:20303/eureka/ --spring-conf logging.config=classpath:log4j2.xml --spring-conf eureka.instance.prefer-ip-address=true --spring-conf spring.profiles.active=engineconn --spring-conf eureka.instan
ce.instance-id=\${spring.cloud.client.ip-address}:\${spring.application.name}:\${server.port} --spring-conf server.port=16786 --spring-conf spring.application.name=linkis-cg-engineconn
linkis_engineconn_errorcode=$?
if [ $linkis_engineconn_errorcode -ne 0 ]
then
cat ${LOG_DIRS}/stderr
exit $linkis_engineconn_errorcode
fi
````
我怀疑是jetty包冲突所致，但是我不确定也不知道该怎么解决这个问题，还请高人指点一二
### Your environment
<!-- Please describe the linkis version you are using and basic environment information -->
<!-- 请描述您使用的linkis版本和基本环境信息 -->
- Linkis version used: 1.1.2
- Environment name and version:
- cdh-5.12.1
- hdp-2.6.0-cdh5.12.1
- hive-1.1.0-cdh5.12.1
- spark-2.2.0.cloudera1
- scala-2.11.12
- jdk 1.8.0_121
- OS：Red Hat Enterprise Linux Server release 6.9
### Anything else
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [incubator-linkis] casionone commented on issue #2762: linkis1.1.1启动成功后提交任务给Spark引擎执行出错问题[Bug]
Posted by GitBox <gi...@apache.org>.
casionone commented on issue #2762:
URL: https://github.com/apache/incubator-linkis/issues/2762#issuecomment-1215118124
this is my spark engine lib list
```
annotations-2.0.1.jar
antlr-2.7.7.jar
antlr-runtime-3.4.jar
aopalliance-1.0.jar
aopalliance-repackaged-2.4.0-b34.jar
archaius-core-0.7.7.jar
asm-analysis-9.3.jar
asm-commons-9.3.jar
asm-tree-9.3.jar
aspectjweaver-1.9.6.jar
bcpkix-jdk15on-1.64.jar
bcprov-jdk15on-1.64.jar
byte-buddy-1.10.22.jar
classgraph-4.1.7.jar
classmate-1.5.1.jar
commons-beanutils-1.9.4.jar
commons-codec-1.14.jar
commons-collections-3.2.2.jar
commons-configuration-1.10.jar
commons-dbcp-1.4.jar
commons-exec-1.3.jar
commons-fileupload-1.4.jar
commons-io-2.11.0.jar
commons-jxpath-1.3.jar
commons-lang-2.6.jar
commons-lang3-3.12.0.jar
commons-math-2.2.jar
commons-net-3.1.jar
commons-pool-1.6.jar
eureka-client-1.10.14.jar
eureka-core-1.10.14.jar
evictor-1.0.0.jar
feign-core-10.12.jar
feign-form-3.8.0.jar
feign-form-spring-3.8.0.jar
feign-hystrix-10.12.jar
feign-slf4j-10.12.jar
gson-2.8.9.jar
guice-4.1.0.jar
HdrHistogram-2.1.12.jar
hk2-api-2.4.0-b34.jar
hk2-locator-2.4.0-b34.jar
hk2-utils-2.4.0-b34.jar
httpclient-4.5.13.jar
httpcore-4.4.14.jar
httpmime-4.5.13.jar
hystrix-core-1.5.18.jar
jackson-annotations-2.13.2.jar
jackson-core-2.13.2.jar
jackson-core-asl-1.9.2.jar
jackson-databind-2.13.2.1.jar
jackson-datatype-jdk8-2.13.2.jar
jackson-datatype-jsr310-2.13.2.jar
jackson-jaxrs-1.9.2.jar
jackson-mapper-asl-1.9.13.jar
jackson-module-parameter-names-2.13.2.jar
jackson-module-scala_2.11-2.13.2.jar
jackson-xc-1.9.2.jar
jakarta.annotation-api-1.3.5.jar
jakarta.el-3.0.3.jar
jakarta.servlet-api-4.0.4.jar
jakarta.websocket-api-1.1.2.jar
javassist-3.27.0-GA.jar
javax.activation-api-1.2.0.jar
javax.annotation-api-1.3.2.jar
javax.inject-1.jar
javax.inject-2.4.0-b34.jar
javax-websocket-client-impl-9.4.48.v20220622.jar
javax-websocket-server-impl-9.4.48.v20220622.jar
javax.ws.rs-api-2.0.1.jar
jaxb-api-2.3.1.jar
jaxb-impl-2.2.3-1.jar
jersey-apache-client4-1.19.4.jar
jersey-client-1.19.4.jar
jersey-client-2.23.1.jar
jersey-common-2.23.1.jar
jersey-container-servlet-2.23.1.jar
jersey-container-servlet-core-2.23.1.jar
jersey-core-1.19.4.jar
jersey-guava-2.23.1.jar
jersey-json-1.19.4.jar
jersey-media-jaxb-2.23.1.jar
jersey-server-1.19.4.jar
jersey-server-2.23.1.jar
jersey-servlet-1.19.4.jar
jettison-1.1.jar
jetty-annotations-9.4.48.v20220622.jar
jetty-client-9.4.48.v20220622.jar
jetty-continuation-9.4.48.v20220622.jar
jetty-http-9.4.48.v20220622.jar
jetty-io-9.4.48.v20220622.jar
jetty-plus-9.4.48.v20220622.jar
jetty-security-9.4.48.v20220622.jar
jetty-server-9.4.48.v20220622.jar
jetty-servlet-9.4.48.v20220622.jar
jetty-servlets-9.4.48.v20220622.jar
jetty-util-9.4.48.v20220622.jar
jetty-util-ajax-9.4.48.v20220622.jar
jetty-webapp-9.4.48.v20220622.jar
jetty-xml-9.4.48.v20220622.jar
joda-time-2.9.3.jar
jul-to-slf4j-1.7.30.jar
knife4j-annotations-2.0.9.jar
knife4j-core-2.0.9.jar
knife4j-spring-2.0.9.jar
knife4j-spring-boot-autoconfigure-2.0.9.jar
knife4j-spring-boot-starter-2.0.9.jar
knife4j-spring-ui-2.0.9.jar
LatencyUtils-2.0.3.jar
linkis-accessible-executor-1.2.0.jar
linkis-bml-client-1.2.0.jar
linkis-bml-common-1.2.0.jar
linkis-common-1.2.0.jar
linkis-computation-engineconn-1.2.0.jar
linkis-computation-governance-common-1.2.0.jar
linkis-cs-client-1.2.0.jar
linkis-cs-common-1.2.0.jar
linkis-cs-listener-1.2.0.jar
linkis-engineconn-common-1.2.0.jar
linkis-engineconn-core-1.2.0.jar
linkis-engineconn-plugin-core-1.2.0.jar
linkis-engineplugin-spark-1.2.0.jar
linkis-executor-core-1.2.0.jar
linkis-gateway-httpclient-support-1.2.0.jar
linkis-hadoop-common-1.2.0.jar
linkis-httpclient-1.2.0.jar
linkis-label-common-1.2.0.jar
linkis-manager-common-1.2.0.jar
linkis-module-1.2.0.jar
linkis-protocol-1.2.0.jar
linkis-rpc-1.2.0.jar
linkis-scheduler-1.2.0.jar
linkis-storage-1.2.0.jar
linkis-udf-client-1.2.0.jar
linkis-udf-common-1.2.0.jar
log4j-1.2-api-2.17.2.jar
log4j-api-2.17.2.jar
log4j-core-2.17.2.jar
log4j-jcl-2.17.2.jar
log4j-jul-2.17.2.jar
log4j-slf4j-impl-2.17.2.jar
mapstruct-1.3.1.Final.jar
micrometer-core-1.5.14.jar
micrometer-registry-prometheus-1.5.14.jar
netflix-commons-util-0.3.0.jar
netflix-eventbus-0.3.0.jar
netflix-infix-0.3.0.jar
netflix-statistics-0.1.1.jar
osgi-resource-locator-1.0.1.jar
out.txt
paranamer-2.8.jar
protobuf-java-3.15.8.jar
protostuff-api-1.6.2.jar
protostuff-collectionschema-1.6.2.jar
protostuff-core-1.6.2.jar
protostuff-runtime-1.6.2.jar
reactive-streams-1.0.3.jar
reactor-core-3.3.17.RELEASE.jar
reactor-extra-3.3.6.RELEASE.jar
reflections-0.9.10.jar
ribbon-2.3.0.jar
ribbon-core-2.3.0.jar
ribbon-eureka-2.3.0.jar
ribbon-httpclient-2.3.0.jar
ribbon-loadbalancer-2.3.0.jar
ribbon-transport-2.3.0.jar
rxjava-1.3.8.jar
rxnetty-0.4.9.jar
rxnetty-contexts-0.4.9.jar
rxnetty-servo-0.4.9.jar
scalap-2.11.12.jar
servo-core-0.12.21.jar
simpleclient-0.8.1.jar
simpleclient_common-0.8.1.jar
slf4j-api-1.7.30.jar
snakeyaml-1.26.jar
spring-aop-5.2.22.RELEASE.jar
spring-beans-5.2.22.RELEASE.jar
spring-boot-2.3.12.RELEASE.jar
spring-boot-actuator-2.3.12.RELEASE.jar
spring-boot-actuator-autoconfigure-2.3.12.RELEASE.jar
spring-boot-autoconfigure-2.3.12.RELEASE.jar
spring-boot-starter-2.3.12.RELEASE.jar
spring-boot-starter-actuator-2.3.12.RELEASE.jar
spring-boot-starter-aop-2.3.12.RELEASE.jar
spring-boot-starter-cache-2.3.12.RELEASE.jar
spring-boot-starter-jetty-2.3.12.RELEASE.jar
spring-boot-starter-json-2.3.12.RELEASE.jar
spring-boot-starter-log4j2-2.3.12.RELEASE.jar
spring-boot-starter-validation-2.3.12.RELEASE.jar
spring-boot-starter-web-2.3.12.RELEASE.jar
spring-cloud-commons-2.2.9.RELEASE.jar
spring-cloud-config-client-2.2.8.RELEASE.jar
spring-cloud-context-2.2.9.RELEASE.jar
spring-cloud-loadbalancer-2.2.9.RELEASE.jar
spring-cloud-netflix-archaius-2.2.9.RELEASE.jar
spring-cloud-netflix-eureka-client-2.2.9.RELEASE.jar
spring-cloud-netflix-hystrix-2.2.9.RELEASE.jar
spring-cloud-netflix-ribbon-2.2.9.RELEASE.jar
spring-cloud-openfeign-core-2.2.9.RELEASE.jar
spring-cloud-starter-2.2.9.RELEASE.jar
spring-cloud-starter-config-2.2.8.RELEASE.jar
spring-cloud-starter-loadbalancer-2.2.9.RELEASE.jar
spring-cloud-starter-netflix-archaius-2.2.9.RELEASE.jar
spring-cloud-starter-netflix-eureka-client-2.2.9.RELEASE.jar
spring-cloud-starter-netflix-ribbon-2.2.9.RELEASE.jar
spring-cloud-starter-openfeign-2.2.9.RELEASE.jar
spring-context-5.2.22.RELEASE.jar
spring-context-support-5.2.22.RELEASE.jar
spring-core-5.2.22.RELEASE.jar
spring-expression-5.2.22.RELEASE.jar
springfox-bean-validators-2.10.5.jar
springfox-core-2.10.5.jar
springfox-schema-2.10.5.jar
springfox-spi-2.10.5.jar
springfox-spring-web-2.10.5.jar
springfox-spring-webmvc-2.10.5.jar
springfox-swagger2-2.10.5.jar
springfox-swagger-common-2.10.5.jar
spring-jcl-5.2.22.RELEASE.jar
spring-plugin-core-2.0.0.RELEASE.jar
spring-plugin-metadata-2.0.0.RELEASE.jar
spring-security-crypto-5.3.9.RELEASE.jar
spring-security-rsa-1.0.9.RELEASE.jar
spring-web-5.2.22.RELEASE.jar
spring-webmvc-5.2.22.RELEASE.jar
stax2-api-4.2.1.jar
stringtemplate-3.2.1.jar
swagger-annotations-1.5.22.jar
swagger-models-1.5.22.jar
validation-api-2.0.1.Final.jar
websocket-api-9.4.48.v20220622.jar
websocket-client-9.4.48.v20220622.jar
websocket-common-9.4.48.v20220622.jar
websocket-server-9.4.48.v20220622.jar
websocket-servlet-9.4.48.v20220622.jar
woodstox-core-6.2.1.jar
xstream-1.4.19.jar
```
You can try to compare to see what dependencies are missing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [incubator-linkis] Yiutto commented on issue #2762: linkis1.1.1启动成功后提交任务给Spark引擎执行出错问题[Bug]
Posted by GitBox <gi...@apache.org>.
Yiutto commented on issue #2762:
URL: https://github.com/apache/incubator-linkis/issues/2762#issuecomment-1239370730
@casionone 后续这个问题有解决吗，编译linkis1.1.3和1.2.0在cdh上执行spark-sql都出现这个问题，请问如何解决
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [incubator-linkis] github-actions[bot] commented on issue #2762: linkis1.1.1启动成功后提交任务给Spark引擎执行出错问题[Bug]
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #2762:
URL: https://github.com/apache/incubator-linkis/issues/2762#issuecomment-1214293656
:blush: Welcome to the Apache Linkis (incubating) community!!
We are glad that you are contributing by opening this issue.
Please make sure to include all the relevant context.
We will be here shortly.
If you are interested in contributing to our website project, please let us know!
You can check out our contributing guide on
:point_right: [How to Participate in Project Contribution](https://linkis.apache.org/community/how-to-contribute).
WeChat Group:
Mailing Lists:
|name|description|Subscribe|Unsubscribe|archive|
|:-----|:--------|:------|:-------|:-----|
| [dev@linkis.apache.org](mailto:dev@linkis.apache.org) | community activity information | [subscribe](mailto:dev-subscribe@linkis.apache.org) | [unsubscribe](mailto:dev-unsubscribe@linkis.apache.org) | [archive](http://mail-archives.apache.org/mod_mbox/linkis-dev) |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [incubator-linkis] jasonmrliu commented on issue #2762: linkis1.1.1启动成功后提交任务给Spark引擎执行出错问题[Bug]
Posted by GitBox <gi...@apache.org>.
jasonmrliu commented on issue #2762:
URL: https://github.com/apache/incubator-linkis/issues/2762#issuecomment-1214537733
> Check the version information of the jetty jar in the $LINKIS_HOME/lib directory to see if there are multiple versions find ./lib -name "jetty-*.jar"
>
> 查看下 $LINKIS_HOME/lib目录下 jetty 包版本信息 是否存在多个版本
下面是我的整个lib目录下jetty相关的jar包，不同的引擎有相同的jar应该是没问题的吧
./lib/linkis-spring-cloud-services/linkis-mg-gateway/jetty-util-9.4.42.v20210604.jar
./lib/linkis-spring-cloud-services/linkis-mg-gateway/jetty-io-9.4.42.v20210604.jar
./lib/linkis-spring-cloud-services/linkis-mg-gateway/jetty-client-9.4.42.v20210604.jar
./lib/linkis-spring-cloud-services/linkis-mg-gateway/jetty-annotations-9.4.42.v20210604.jar
./lib/linkis-spring-cloud-services/linkis-mg-gateway/jetty-plus-9.4.42.v20210604.jar
./lib/linkis-spring-cloud-services/linkis-mg-gateway/jetty-servlets-9.4.42.v20210604.jar
./lib/linkis-spring-cloud-services/linkis-mg-gateway/jetty-continuation-9.4.42.v20210604.jar
./lib/linkis-spring-cloud-services/linkis-mg-gateway/jetty-http-9.4.42.v20210604.jar
./lib/linkis-engineconn-plugins/spark/dist/v2.2.0.cloudera1/lib/jetty-util-9.4.42.v20210604.jar
./lib/linkis-engineconn-plugins/spark/dist/v2.2.0.cloudera1/lib/jetty-util-ajax-9.4.42.v20210604.jar
./lib/linkis-engineconn-plugins/spark/dist/v2.2.0.cloudera1/lib/jetty-io-9.4.42.v20210604.jar
./lib/linkis-engineconn-plugins/spark/dist/v2.2.0.cloudera1/lib/jetty-client-9.4.42.v20210604.jar
./lib/linkis-engineconn-plugins/spark/dist/v2.2.0.cloudera1/lib/jetty-annotations-9.4.42.v20210604.jar
./lib/linkis-engineconn-plugins/spark/dist/v2.2.0.cloudera1/lib/jetty-server-9.4.42.v20210604.jar
./lib/linkis-engineconn-plugins/spark/dist/v2.2.0.cloudera1/lib/jetty-plus-9.4.42.v20210604.jar
./lib/linkis-engineconn-plugins/spark/dist/v2.2.0.cloudera1/lib/jetty-webapp-9.4.42.v20210604.jar
./lib/linkis-engineconn-plugins/spark/dist/v2.2.0.cloudera1/lib/jetty-servlets-9.4.42.v20210604.jar
./lib/linkis-engineconn-plugins/spark/dist/v2.2.0.cloudera1/lib/jetty-servlet-9.4.42.v20210604.jar
./lib/linkis-engineconn-plugins/spark/dist/v2.2.0.cloudera1/lib/jetty-continuation-9.4.42.v20210604.jar
./lib/linkis-engineconn-plugins/spark/dist/v2.2.0.cloudera1/lib/jetty-xml-9.4.42.v20210604.jar
./lib/linkis-engineconn-plugins/spark/dist/v2.2.0.cloudera1/lib/jetty-http-9.4.42.v20210604.jar
./lib/linkis-engineconn-plugins/spark/dist/v2.2.0.cloudera1/lib/jetty-security-9.4.42.v20210604.jar
./lib/linkis-commons/public-module/jetty-util-9.4.42.v20210604.jar
./lib/linkis-commons/public-module/jetty-util-ajax-9.4.42.v20210604.jar
./lib/linkis-commons/public-module/jetty-io-9.4.42.v20210604.jar
./lib/linkis-commons/public-module/jetty-client-9.4.42.v20210604.jar
./lib/linkis-commons/public-module/jetty-annotations-9.4.42.v20210604.jar
./lib/linkis-commons/public-module/jetty-server-9.4.42.v20210604.jar
./lib/linkis-commons/public-module/jetty-plus-9.4.42.v20210604.jar
./lib/linkis-commons/public-module/jetty-webapp-9.4.42.v20210604.jar
./lib/linkis-commons/public-module/jetty-servlets-9.4.42.v20210604.jar
./lib/linkis-commons/public-module/jetty-servlet-9.4.42.v20210604.jar
./lib/linkis-commons/public-module/jetty-continuation-9.4.42.v20210604.jar
./lib/linkis-commons/public-module/jetty-xml-9.4.42.v20210604.jar
./lib/linkis-commons/public-module/jetty-http-9.4.42.v20210604.jar
./lib/linkis-commons/public-module/jetty-security-9.4.42.v20210604.jar
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [incubator-linkis] Yiutto commented on issue #2762: linkis1.1.1启动成功后提交任务给Spark引擎执行出错问题[Bug]
Posted by GitBox <gi...@apache.org>.
Yiutto commented on issue #2762:
URL: https://github.com/apache/incubator-linkis/issues/2762#issuecomment-1240365970
### ==最后解决方法==
cdh5.12.1
hive-1.1.0_cdh5.12.1
hadoop-2.6.0_cdh5.12.1
python3
spark呢我还是装原生的Apache spark2.4.3
这里变动一下。
### /opt/spark-2.4.3-bin-hadoop2.6/conf 里面 spark-evn.sh内容置空
```
cp spark-env.sh.template spark-env.sh
```
### 把hive-site.xml添加至conf
在/etc/hive/conf里面复制进去
### 把yarn-site.xml添加至conf
这里面有个尴尬的事情，就是之前的机器没有添加到yarn集群，导致提交spark-sql一直等待。
后来从其他机器里面/etc/hadoop/conf/yarn-site.xml拷贝进去。
### log4j冲突
将linkis的engine的jar删掉
lib/linkis-engineconn-plugins/spark/dist/v2.4.3/lib/log4j-slf4j-impl-2.17.2.jar.bak
### 配置环境变量
https://github.com/apache/incubator-linkis/discussions/2745
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [incubator-linkis] casionone commented on issue #2762: linkis1.1.1启动成功后提交任务给Spark引擎执行出错问题[Bug]
Posted by GitBox <gi...@apache.org>.
casionone commented on issue #2762:
URL: https://github.com/apache/incubator-linkis/issues/2762#issuecomment-1214399791
Check the version information of the jetty jar in the $LINKIS_HOME/lib directory to see if there are multiple versions
find ./lib -name "jetty-*.jar"
查看下 $LINKIS_HOME/lib目录下 jetty 包版本信息 是否存在多个版本
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [incubator-linkis] casionone closed issue #2762: linkis1.1.1启动成功后提交任务给Spark引擎执行出错问题[Bug]
Posted by GitBox <gi...@apache.org>.
casionone closed issue #2762: linkis1.1.1启动成功后提交任务给Spark引擎执行出错问题[Bug]
URL: https://github.com/apache/incubator-linkis/issues/2762
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org
[GitHub] [incubator-linkis] Yiutto commented on issue #2762: linkis1.1.1启动成功后提交任务给Spark引擎执行出错问题[Bug]
Posted by GitBox <gi...@apache.org>.
Yiutto commented on issue #2762:
URL: https://github.com/apache/incubator-linkis/issues/2762#issuecomment-1235383892
我跟楼主的问题感觉是类似的，后来对比发现少了好多jar，请问有影响吗
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@linkis.apache.org
For additional commands, e-mail: notifications-help@linkis.apache.org