You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2020/12/08 09:32:03 UTC
[GitHub] [incubator-dolphinscheduler] share23 opened a new issue #4181: [Bug][condition] Update workflow definition error after use condition moudle and save workflow
share23 opened a new issue #4181:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4181
I just want to test whether the condition moudule function well, but after deploy the condition moudle(save the moudle is well) and save the workflow ,It show me that "Update workflow definition error"
Here are my steps
step 1: create shell moudle "node_a " , echo hello
step 2: create condition moudle "node_condition" just like this
step 3: create shell moudle "node_b" ,"node_c" , echo success or failed
step 4: save the workflow
The error message like this after i click save workflow button:
The version of Dolphin Scheduler is 1.3.2 download from GitHub, I have modify the pom.xml and compile the DS, It work well in shell and hive noudle
modify pom.xml like this :
before
`<hadoop.version>2.7.3</hadoop.version>
<hive.jdbc.version>2.1.0</hive.jdbc.version>`
after
`<hadoop.version>2.6.0</hadoop.version>
<hive.jdbc.version>1.1.0</hive.jdbc.version>`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] BoYiZhang commented on issue #4181: [Bug][condition] Update workflow definition error after use condition moudle and save workflow
Posted by GitBox <gi...@apache.org>.
BoYiZhang commented on issue #4181:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4181#issuecomment-741502269
Check the getResourceIds method of the ProcessDefinitionServiceImpl class
There is something wrong with this method.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] share23 commented on issue #4181: [Bug][condition] Update workflow definition error after use condition moudle and save workflow
Posted by GitBox <gi...@apache.org>.
share23 commented on issue #4181:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4181#issuecomment-741428758
> According to the log provided, there is no way to locate the exception code
>
> More exception log information is needed
>
> [ dolphinscheduler-api- server.log ]
[INFO] 2020-12-08 18:59:43.319 org.apache.dolphinscheduler.api.controller.ProcessDefinitionController:[213] - verify process definition name unique, user:test_user, project name:project_test, process definition name:aaa
[INFO] 2020-12-08 18:59:43.630 org.apache.dolphinscheduler.api.controller.ProcessDefinitionController:[125] - login user test_user, create process definition, project name: project_test, process definition name: aaa, process_definition_json: {"globalParams":[],"tasks":[{"type":"SHELL","id":"tasks-4889","name":"node_a","params":{"resourceList":[],"localParams":[],"rawScript":"#!/bin/bash\necho \"hello\""},"description":"","runFlag":"NORMAL","conditionResult":{"successNode":[""],"failedNode":[""]},"dependence":{},"maxRetryTimes":"0","retryInterval":"1","delayTime":"0","timeout":{"strategy":"","interval":null,"enable":false},"waitStartTimeout":{},"taskInstancePriority":"MEDIUM","workerGroup":"default","preTasks":[]},{"type":"CONDITIONS","id":"tasks-50478","name":"node_condition","params":{},"description":"","runFlag":"NORMAL","conditionResult":{"successNode":["node_b"],"failedNode":["node_c"]},"dependence":{"relation":"AND","dependTaskList":[{"relation":"AND","dependItemList":[{"dep
Tasks":"node_a","status":"SUCCESS"}]}]},"maxRetryTimes":1,"retryInterval":"1","delayTime":"0","timeout":{"strategy":"","interval":null,"enable":false},"waitStartTimeout":{},"taskInstancePriority":"MEDIUM","workerGroup":"default","preTasks":["node_a"]},{"type":"SHELL","id":"tasks-53103","name":"node_b","params":{"resourceList":[],"localParams":[],"rawScript":"echo \"happy\""},"description":"","runFlag":"NORMAL","conditionResult":{"successNode":[""],"failedNode":[""]},"dependence":{},"maxRetryTimes":"0","retryInterval":"1","delayTime":"0","timeout":{"strategy":"","interval":null,"enable":false},"waitStartTimeout":{},"taskInstancePriority":"MEDIUM","workerGroup":"default","preTasks":["node_condition"]},{"type":"SHELL","id":"tasks-64379","name":"node_c","params":{"resourceList":[],"localParams":[],"rawScript":"echo \"sad\""},"description":"","runFlag":"NORMAL","conditionResult":{"successNode":[""],"failedNode":[""]},"dependence":{},"maxRetryTimes":"0","retryInterval":"1","delayTime":"0"
,"timeout":{"strategy":"","interval":null,"enable":false},"waitStartTimeout":{},"taskInstancePriority":"MEDIUM","workerGroup":"default","preTasks":["node_condition"]}],"tenantId":6,"timeout":0}, desc: qqq locations:{"tasks-4889":{"name":"node_a","targetarr":"","nodenumber":"1","x":96,"y":186},"tasks-50478":{"name":"node_condition","targetarr":"tasks-4889","nodenumber":"3","x":351,"y":198},"tasks-53103":{"name":"node_b","targetarr":"tasks-50478","nodenumber":"0","x":655,"y":101},"tasks-64379":{"name":"node_c","targetarr":"tasks-50478","nodenumber":"0","x":659,"y":289}}, connects:[{"endPointSourceId":"tasks-4889","endPointTargetId":"tasks-50478","label":""},{"endPointSourceId":"tasks-50478","endPointTargetId":"tasks-53103","label":""},{"endPointSourceId":"tasks-50478","endPointTargetId":"tasks-64379","label":""}]
[ERROR] 2020-12-08 18:59:43.644 org.apache.dolphinscheduler.api.exceptions.ApiExceptionHandler:[46] - 创建工作流错误
java.lang.NullPointerException: null
at org.apache.dolphinscheduler.api.service.impl.ProcessDefinitionServiceImpl.lambda$getResourceIds$1(ProcessDefinitionServiceImpl.java:242)
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:267)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.apache.dolphinscheduler.api.service.impl.ProcessDefinitionServiceImpl.getResourceIds(ProcessDefinitionServiceImpl.java:246)
at org.apache.dolphinscheduler.api.service.impl.ProcessDefinitionServiceImpl.createProcessDefinition(ProcessDefinitionServiceImpl.java:200)
at org.apache.dolphinscheduler.api.service.impl.ProcessDefinitionServiceImpl$$FastClassBySpringCGLIB$$e8e34ed9.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:687)
at org.apache.dolphinscheduler.api.service.impl.ProcessDefinitionServiceImpl$$EnhancerBySpringCGLIB$$bf8f7b2b.createProcessDefinition(<generated>)
at org.apache.dolphinscheduler.api.controller.ProcessDefinitionController.createProcessDefinition(ProcessDefinitionController.java:128)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:892)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:797)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651)
at com.github.xiaoymin.swaggerbootstrapui.filter.SecurityBasicAuthFilter.doFilter(SecurityBasicAuthFilter.java:84)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638)
at com.github.xiaoymin.swaggerbootstrapui.filter.ProductionSecurityFilter.doFilter(ProductionSecurityFilter.java:53)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638)
at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:94)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:501)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
at java.lang.Thread.run(Thread.java:745)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] BoYiZhang commented on issue #4181: [Bug][condition] Update workflow definition error after use condition moudle and save workflow
Posted by GitBox <gi...@apache.org>.
BoYiZhang commented on issue #4181:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4181#issuecomment-741401648
According to the log provided, there is no way to locate the exception code
More exception log information is needed
[ dolphinscheduler-api- server.log ]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] share23 commented on issue #4181: [Bug][condition] Update workflow definition error after use condition moudle and save workflow
Posted by GitBox <gi...@apache.org>.
share23 commented on issue #4181:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4181#issuecomment-741488055
> Your code should not be version 1.3.2
>
> The 1.3.2 release does not have a processdefinitionserviceimpl class
>
> What version of code do you use?
I have download the source code (version 1.3.2) from GitHub on 2020.10.23
then import in IDEA and modify the pom.xml to match the enviroment (cdh 5.13.1)
compile the project and install DS by cluster mode
modify pom.xml like this
before
`<hadoop.version>2.7.3</hadoop.version>`
`<hive.jdbc.version>2.1.0</hive.jdbc.version>`
after
`<hadoop.version>2.6.0</hadoop.version>`
`<hive.jdbc.version>1.1.0</hive.jdbc.version>`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] BoYiZhang commented on issue #4181: [Bug][condition] Update workflow definition error after use condition moudle and save workflow
Posted by GitBox <gi...@apache.org>.
BoYiZhang commented on issue #4181:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/4181#issuecomment-741450345
Your code should not be version 1.3.2
The 1.3.2 release does not have a processdefinitionserviceimpl class
What version of code do you use?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org