You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/05/16 01:51:46 UTC
[GitHub] [dolphinscheduler] syncwt opened a new issue, #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query
syncwt opened a new issue, #10037:
URL: https://github.com/apache/dolphinscheduler/issues/10037
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
### What happened
I have upgraded 1.3.5 to 2.0.5 successfully.When I login to the console,I found the process defined list and task instance list are empty.And the HTTP API response some error as list.
```
process-definition list api
req:
/dolphinscheduler/projects/3/process-definition?
resp:
pageSize=10&pageNo=1&searchVal=&userId=&_t=0.20745839771660646
{"code":10122,"msg":"分页查询工作流定义列表错误","data":null,"failed":true,"success":false}
task-instance list api
req:
/dolphinscheduler/projects/3/task-instances?pageSize=10&pageNo=1&searchVal=&processInstanceId=&host=&stateType=&startDate=&endDate=&executorName=&processInstanceName=&_t=0.5995929372135493
resp:
{"code":0,"msg":"成功","data":{"totalList":[],"total":0,"totalPage":1,"pageSize":10,"currentPage":1,"start":0},"failed":false,"success":true}
```
I got error from dolphinscheduler-api.log as list:
```log
[INFO] 2022-05-11 15:29:52.021 org.apache.dolphinscheduler.api.aspect.AccessLogAspect:[90] - REQUEST TRACE_ID:3e42bbe6-8b16-4a37-9db4-db9178833873, LOGIN_US
ER:wentao, URI:/dolphinscheduler/projects/2/process-definition, METHOD:GET, HANDLER:org.apache.dolphinscheduler.api.controller.ProcessDefinitionController.q
ueryProcessDefinitionListPaging, ARGS:{searchVal=, projectCode=2, pageNo=1, pageSize=10, userId=0}
[ERROR] 2022-05-11 15:29:52.090 org.apache.dolphinscheduler.api.exceptions.ApiExceptionHandler:[46] - 分页查询工作流定义列表错误
java.lang.NullPointerException: null
at org.apache.dolphinscheduler.api.service.impl.ProcessDefinitionServiceImpl.queryProcessDefinitionListPaging(ProcessDefinitionServiceImpl.java:421)
at org.apache.dolphinscheduler.api.service.impl.ProcessDefinitionServiceImpl$$FastClassBySpringCGLIB$$e8e34ed9.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)
at org.apache.dolphinscheduler.api.service.impl.ProcessDefinitionServiceImpl$$EnhancerBySpringCGLIB$$c60e4701.queryProcessDefinitionListPaging(<gene
rated>)
at org.apache.dolphinscheduler.api.controller.ProcessDefinitionController.queryProcessDefinitionListPaging(ProcessDefinitionController.java:493)
at org.apache.dolphinscheduler.api.controller.ProcessDefinitionController$$FastClassBySpringCGLIB$$dc9bf5db.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
at org.apache.dolphinscheduler.api.aspect.AccessLogAspect.doAround(AccessLogAspect.java:101)
at sun.reflect.GeneratedMethodAccessor116.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698)
at org.apache.dolphinscheduler.api.controller.ProcessDefinitionController$$EnhancerBySpringCGLIB$$6683953e.queryProcessDefinitionListPaging(<generat
ed>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1067)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:497)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:584)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631)
at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:91)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at com.github.xiaoymin.swaggerbootstrapui.filter.SecurityBasicAuthFilter.doFilter(SecurityBasicAuthFilter.java:84)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at com.github.xiaoymin.swaggerbootstrapui.filter.ProductionSecurityFilter.doFilter(ProductionSecurityFilter.java:53)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:763)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:400)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:645)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:392)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.lang.Thread.run(Thread.java:750)
```
I found the table `t_ds_process_definition_log` is empty after upgraded,then cause NPE in the code .
```java
ProcessDefinitionLog processDefinitionLog = processDefinitionLogMapper.queryByDefinitionCodeAndVersion(pd.getCode(), pd.getVersion());
User user = userMapper.selectById(processDefinitionLog.getOperator());
```
### What you expected to happen
Normal query list
### How to reproduce
Click query list button on console.
### Anything else
_No response_
### Version
2.0.5
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] syncwt commented on issue #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query process defined list
Posted by GitBox <gi...@apache.org>.
syncwt commented on issue #10037:
URL: https://github.com/apache/dolphinscheduler/issues/10037#issuecomment-1128294919
> Hi, @syncwt .From the log, it is because the dependent node is illegal. Can you provide the JSON of the dependent node
Thanks! But how to find the illegal dependent node ? I saw the code , it's foreach the process defined to split dependent node but not print the error node .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on issue #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query process defined list
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on issue #10037:
URL: https://github.com/apache/dolphinscheduler/issues/10037#issuecomment-1127441570
@JinyLeeChina Could you take a look at this issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on issue #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query process defined list
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on issue #10037:
URL: https://github.com/apache/dolphinscheduler/issues/10037#issuecomment-1127301312
> There is missed about version 1.3.7. Is the 1.3.8 is ok for upgrade from 1.3.5. https://dolphinscheduler.apache.org/zh-cn/download/download.html @SbloodyS
You can give it a try. Or you can try upgrading directly to 1.3.9.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] syncwt commented on issue #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query process defined list
Posted by GitBox <gi...@apache.org>.
syncwt commented on issue #10037:
URL: https://github.com/apache/dolphinscheduler/issues/10037#issuecomment-1127299601
There is missed about version 1.3.7. Is the 1.3.8 is ok for upgrade from 1.3.5.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #10037:
URL: https://github.com/apache/dolphinscheduler/issues/10037#issuecomment-1127133515
Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can [join our slack](https://join.slack.com/t/asf-dolphinscheduler/shared_invite/zt-omtdhuio-_JISsxYhiVsltmC5h38yfw) and send your question to channel `#troubleshooting`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] syncwt commented on issue #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query process defined list
Posted by GitBox <gi...@apache.org>.
syncwt commented on issue #10037:
URL: https://github.com/apache/dolphinscheduler/issues/10037#issuecomment-1127137914
> Which database you are using? mysql or postgres?
Mysql 5.7.30
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on issue #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on issue #10037:
URL: https://github.com/apache/dolphinscheduler/issues/10037#issuecomment-1127136498
Which database you are using? mysql or postgres?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] syncwt commented on issue #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query process defined list
Posted by GitBox <gi...@apache.org>.
syncwt commented on issue #10037:
URL: https://github.com/apache/dolphinscheduler/issues/10037#issuecomment-1127295066
>
I will have a try. But... is't not elegant. Thanks a lot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] JunjianS commented on issue #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query process defined list
Posted by GitBox <gi...@apache.org>.
JunjianS commented on issue #10037:
URL: https://github.com/apache/dolphinscheduler/issues/10037#issuecomment-1168576942
how to fix the dependent task by manual? @s
> > ```java
> > for (int i = 0; i < dependTaskList.size(); i++) {
> > ObjectNode dependTask = (ObjectNode) dependTaskList.path(i);
> > ArrayNode dependItemList = JSONUtils.parseArray(JSONUtils.toJsonString(dependTask.get("dependItemList")));
> > for (int j = 0; j < dependItemList.size(); j++) {
> > ObjectNode dependItem = (ObjectNode) dependItemList.path(j);
> > dependItem.put("projectCode", projectIdCodeMap.get(dependItem.get("projectId").asInt()));
> > int definitionId = dependItem.get("definitionId").asInt();
> > Map<Long, Map<String, Long>> processCodeTaskNameCodeMap = processTaskMap.get(definitionId);
> > if (processCodeTaskNameCodeMap == null) {
> > logger.warn("We can't find processDefinition [{}], please check it is not exist, remove this dependence", definitionId);
> > dependItemList.remove(j);
> > continue;
> > }
> > Optional<Map.Entry<Long, Map<String, Long>>> mapEntry = processCodeTaskNameCodeMap.entrySet().stream().findFirst();
> > if (mapEntry.isPresent()) {
> > Map.Entry<Long, Map<String, Long>> processCodeTaskNameCodeEntry = mapEntry.get();
> > dependItem.put("definitionCode", processCodeTaskNameCodeEntry.getKey());
> > String depTasks = dependItem.get("depTasks").asText();
> > long taskCode = "ALL".equals(depTasks) || processCodeTaskNameCodeEntry.getValue() == null ? 0L : processCodeTaskNameCodeEntry.getValue().get(depTasks); // exception code
> > dependItem.put("depTaskCode", taskCode);
> > }
> > dependItem.remove("projectId");
> > dependItem.remove("definitionId");
> > dependItem.remove("depTasks");
> > dependItemList.set(j, dependItem);
> > }
> > dependTask.put("dependItemList", dependItemList);
> > dependTaskList.set(i, dependTask);
> > }
> > ```
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > look at the exception code. `processCodeTaskNameCodeEntry` shouldn't be null. And when `processCodeTaskNameCodeEntry.getValue()` is null ,it should be return `0L`. I can't figure how to happened. My branch‘s version is 2.0.5-release and I have checked the code is same with above from the compiled jar . Could you have a look at here, please. @JinyLeeChina
>
> I found the reason! Firstly, the NPE cause by `long` can't be null. Maybe it should replaced by `Long`. When the task A was depended by a sub process 's task B, task A rename to A1, or be deleted. And the task B 's depend item list wouldn't update, then cause the result that can't find the `depTasks`. It would be ok with fix the dependent task by manual.
how to fix the dependent task by manual?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on issue #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query process defined list
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on issue #10037:
URL: https://github.com/apache/dolphinscheduler/issues/10037#issuecomment-1127146635
@syncwt Could you describe the steps of your upgrade?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] syncwt commented on issue #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query process defined list
Posted by GitBox <gi...@apache.org>.
syncwt commented on issue #10037:
URL: https://github.com/apache/dolphinscheduler/issues/10037#issuecomment-1127287590
Thanks for your tips.I found some error log from upgraded log.
Only 1.3.7 to 2.0.0 print some error infos.
```
2022-05-11 16:20:09.805 INFO 24158 --- [ main] o.a.d.d.upgrade.DolphinSchedulerManager : upgrade DolphinScheduler metadata version from 1.3.7 to 2.0.0
```
... some normal logs are omitted
```
2022-05-11 16:34:11.096 INFO 24158 --- [ main] o.a.d.dao.upgrade.UpgradeDao : sqlSQLFilePathclass path resource [sql/upgrade/2.0.0_schema/mysql/dolphinscheduler_dml.sql]
2022-05-11 16:34:12.955 ERROR 24158 --- [ main] o.a.d.dao.upgrade.UpgradeDao : json split error
java.lang.NullPointerException: null
at org.apache.dolphinscheduler.dao.upgrade.UpgradeDao.convertDependence(UpgradeDao.java:602) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
at org.apache.dolphinscheduler.dao.upgrade.UpgradeDao.processDefinitionJsonSplit(UpgradeDao.java:406) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
at org.apache.dolphinscheduler.dao.upgrade.UpgradeDao.upgradeDolphinSchedulerTo200(UpgradeDao.java:149) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
at org.apache.dolphinscheduler.dao.upgrade.DolphinSchedulerManager.upgradeDolphinScheduler(DolphinSchedulerManager.java:111) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
at org.apache.dolphinscheduler.dao.upgrade.shell.UpgradeDolphinScheduler$UpgradeRunner.run(UpgradeDolphinScheduler.java:55) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:791) [spring-boot-2.5.6.jar:2.5.6]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:775) [spring-boot-2.5.6.jar:2.5.6]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:345) [spring-boot-2.5.6.jar:2.5.6]
at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:143) [spring-boot-2.5.6.jar:2.5.6]
at org.apache.dolphinscheduler.dao.upgrade.shell.UpgradeDolphinScheduler.main(UpgradeDolphinScheduler.java:39) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #10037:
URL: https://github.com/apache/dolphinscheduler/issues/10037#issuecomment-1127133412
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
### What happened
I have upgraded 1.3.5 to 2.0.5 successfully.When I login to the console,I found the process defined list and task instance list are empty.And the HTTP API response some error as list.
```
process-definition list api
req:
/dolphinscheduler/projects/3/process-definition?
resp:
pageSize=10&pageNo=1&searchVal=&userId=&_t=0.20745839771660646
{"code":10122,"msg":"分页查询工作流定义列表错误","data":null,"failed":true,"success":false}
task-instance list api
req:
/dolphinscheduler/projects/3/task-instances?pageSize=10&pageNo=1&searchVal=&processInstanceId=&host=&stateType=&startDate=&endDate=&executorName=&processInstanceName=&_t=0.5995929372135493
resp:
{"code":0,"msg":"成功","data":{"totalList":[],"total":0,"totalPage":1,"pageSize":10,"currentPage":1,"start":0},"failed":false,"success":true}
```
I got error from dolphinscheduler-api.log as list:
```log
[INFO] 2022-05-11 15:29:52.021 org.apache.dolphinscheduler.api.aspect.AccessLogAspect:[90] - REQUEST TRACE_ID:3e42bbe6-8b16-4a37-9db4-db9178833873, LOGIN_US
ER:wentao, URI:/dolphinscheduler/projects/2/process-definition, METHOD:GET, HANDLER:org.apache.dolphinscheduler.api.controller.ProcessDefinitionController.q
ueryProcessDefinitionListPaging, ARGS:{searchVal=, projectCode=2, pageNo=1, pageSize=10, userId=0}
[ERROR] 2022-05-11 15:29:52.090 org.apache.dolphinscheduler.api.exceptions.ApiExceptionHandler:[46] - 分页查询工作流定义列表错误
java.lang.NullPointerException: null
at org.apache.dolphinscheduler.api.service.impl.ProcessDefinitionServiceImpl.queryProcessDefinitionListPaging(ProcessDefinitionServiceImpl.java:421)
at org.apache.dolphinscheduler.api.service.impl.ProcessDefinitionServiceImpl$$FastClassBySpringCGLIB$$e8e34ed9.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)
at org.apache.dolphinscheduler.api.service.impl.ProcessDefinitionServiceImpl$$EnhancerBySpringCGLIB$$c60e4701.queryProcessDefinitionListPaging(<gene
rated>)
at org.apache.dolphinscheduler.api.controller.ProcessDefinitionController.queryProcessDefinitionListPaging(ProcessDefinitionController.java:493)
at org.apache.dolphinscheduler.api.controller.ProcessDefinitionController$$FastClassBySpringCGLIB$$dc9bf5db.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:783)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
at org.apache.dolphinscheduler.api.aspect.AccessLogAspect.doAround(AccessLogAspect.java:101)
at sun.reflect.GeneratedMethodAccessor116.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698)
at org.apache.dolphinscheduler.api.controller.ProcessDefinitionController$$EnhancerBySpringCGLIB$$6683953e.queryProcessDefinitionListPaging(<generat
ed>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1067)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:497)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:584)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1631)
at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:91)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at com.github.xiaoymin.swaggerbootstrapui.filter.SecurityBasicAuthFilter.doFilter(SecurityBasicAuthFilter.java:84)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at com.github.xiaoymin.swaggerbootstrapui.filter.ProductionSecurityFilter.doFilter(ProductionSecurityFilter.java:53)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:763)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:400)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:645)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:392)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.lang.Thread.run(Thread.java:750)
```
I found the table `t_ds_process_definition_log` is empty after upgraded,then cause NPE in the code .
```java
ProcessDefinitionLog processDefinitionLog = processDefinitionLogMapper.queryByDefinitionCodeAndVersion(pd.getCode(), pd.getVersion());
User user = userMapper.selectById(processDefinitionLog.getOperator());
```
### What you expected to happen
Normal query list
### How to reproduce
Click query list button on console.
### Anything else
_No response_
### Version
2.0.5
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] JinyLeeChina commented on issue #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query process defined list
Posted by GitBox <gi...@apache.org>.
JinyLeeChina commented on issue #10037:
URL: https://github.com/apache/dolphinscheduler/issues/10037#issuecomment-1127679111
Hi, @syncwt .From the log, it is because the dependent node is illegal. Can you provide the JSON of the dependent node
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] syncwt closed issue #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query process defined list
Posted by GitBox <gi...@apache.org>.
syncwt closed issue #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query process defined list
URL: https://github.com/apache/dolphinscheduler/issues/10037
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on issue #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query process defined list
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on issue #10037:
URL: https://github.com/apache/dolphinscheduler/issues/10037#issuecomment-1127280849
I also upgraded from 1.3.5 to 2.0.1 six months ago. And then from 2.0.1 to 2.0.5. Everything works fine.
Is there any error logs when executing ```script/upgrade-dolphinscheduler.sh```?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] syncwt commented on issue #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query process defined list
Posted by GitBox <gi...@apache.org>.
syncwt commented on issue #10037:
URL: https://github.com/apache/dolphinscheduler/issues/10037#issuecomment-1127424805
>
I have tried to upgrade 1.3.5 to 1.3.9 ,and then 2.0.5. successful from 1.3.5 to 1.3.9, but the same error from 1.3.9 to 2.0.5.
```
2022-05-16 16:07:31.829 INFO 27457 --- [ main] o.a.d.d.upgrade.DolphinSchedulerManager : upgrade DolphinScheduler metadata version from 1.3.9 to 2.0.0
```
```
2022-05-16 16:21:41.902 ERROR 27457 --- [ main] o.a.d.dao.upgrade.UpgradeDao : json split error
java.lang.NullPointerException: null
at org.apache.dolphinscheduler.dao.upgrade.UpgradeDao.convertDependence(UpgradeDao.java:602) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
at org.apache.dolphinscheduler.dao.upgrade.UpgradeDao.processDefinitionJsonSplit(UpgradeDao.java:406) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
at org.apache.dolphinscheduler.dao.upgrade.UpgradeDao.upgradeDolphinSchedulerTo200(UpgradeDao.java:149) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
at org.apache.dolphinscheduler.dao.upgrade.DolphinSchedulerManager.upgradeDolphinScheduler(DolphinSchedulerManager.java:111) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
at org.apache.dolphinscheduler.dao.upgrade.shell.UpgradeDolphinScheduler$UpgradeRunner.run(UpgradeDolphinScheduler.java:55) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:791) [spring-boot-2.5.6.jar:2.5.6]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:775) [spring-boot-2.5.6.jar:2.5.6]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:345) [spring-boot-2.5.6.jar:2.5.6]
at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:143) [spring-boot-2.5.6.jar:2.5.6]
at org.apache.dolphinscheduler.dao.upgrade.shell.UpgradeDolphinScheduler.main(UpgradeDolphinScheduler.java:39) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
```
Maybe some wrong data in my database in old version?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] kendn1993 commented on issue #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query process defined list
Posted by "kendn1993 (via GitHub)" <gi...@apache.org>.
kendn1993 commented on issue #10037:
URL: https://github.com/apache/dolphinscheduler/issues/10037#issuecomment-1573458526
how to fix the dependent task by manual? @s
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] syncwt commented on issue #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query process defined list
Posted by GitBox <gi...@apache.org>.
syncwt commented on issue #10037:
URL: https://github.com/apache/dolphinscheduler/issues/10037#issuecomment-1127149467
> @syncwt Could you describe the steps of your upgrade?
I upgraded as official doc. The simple step as:
1. download bin tar package(Total Binary Distribution) from https://dolphinscheduler.apache.org/zh-cn/download/download.html
2. edit param about db and path
3. sh script/upgrade-dolphinscheduler.sh
4. upgrade log say successfully
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on issue #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query process defined list
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on issue #10037:
URL: https://github.com/apache/dolphinscheduler/issues/10037#issuecomment-1127291845
> Thanks for your tips.I found some error log from upgraded log. Only 1.3.7 to 2.0.0 print some error infos.
>
> ```
> 2022-05-11 16:20:09.805 INFO 24158 --- [ main] o.a.d.d.upgrade.DolphinSchedulerManager : upgrade DolphinScheduler metadata version from 1.3.7 to 2.0.0
> ```
>
> ... some normal logs are omitted
>
> ```
> 2022-05-11 16:34:11.096 INFO 24158 --- [ main] o.a.d.dao.upgrade.UpgradeDao : sqlSQLFilePathclass path resource [sql/upgrade/2.0.0_schema/mysql/dolphinscheduler_dml.sql]
> 2022-05-11 16:34:12.955 ERROR 24158 --- [ main] o.a.d.dao.upgrade.UpgradeDao : json split error
>
> java.lang.NullPointerException: null
> at org.apache.dolphinscheduler.dao.upgrade.UpgradeDao.convertDependence(UpgradeDao.java:602) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
> at org.apache.dolphinscheduler.dao.upgrade.UpgradeDao.processDefinitionJsonSplit(UpgradeDao.java:406) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
> at org.apache.dolphinscheduler.dao.upgrade.UpgradeDao.upgradeDolphinSchedulerTo200(UpgradeDao.java:149) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
> at org.apache.dolphinscheduler.dao.upgrade.DolphinSchedulerManager.upgradeDolphinScheduler(DolphinSchedulerManager.java:111) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
> at org.apache.dolphinscheduler.dao.upgrade.shell.UpgradeDolphinScheduler$UpgradeRunner.run(UpgradeDolphinScheduler.java:55) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
> at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:791) [spring-boot-2.5.6.jar:2.5.6]
> at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:775) [spring-boot-2.5.6.jar:2.5.6]
> at org.springframework.boot.SpringApplication.run(SpringApplication.java:345) [spring-boot-2.5.6.jar:2.5.6]
> at org.springframework.boot.builder.SpringApplicationBuilder.run(SpringApplicationBuilder.java:143) [spring-boot-2.5.6.jar:2.5.6]
> at org.apache.dolphinscheduler.dao.upgrade.shell.UpgradeDolphinScheduler.main(UpgradeDolphinScheduler.java:39) [dolphinscheduler-dao-2.0.5.jar:2.0.5]
> ```
It should not upgrading from 1.3.7 to 2.0.0. It should be 1.3.9. There may be some incompatible with the old versions.
You can try to upgrade to 1.3.7 from 1.3.5 firstly. And then upgrade from 1.3.7 to 1.3.9. And then upgrade from 1.3.9 to 2.0.5. I think these upgrade steps should work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] syncwt commented on issue #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query process defined list
Posted by GitBox <gi...@apache.org>.
syncwt commented on issue #10037:
URL: https://github.com/apache/dolphinscheduler/issues/10037#issuecomment-1128551361
```java
for (int i = 0; i < dependTaskList.size(); i++) {
ObjectNode dependTask = (ObjectNode) dependTaskList.path(i);
ArrayNode dependItemList = JSONUtils.parseArray(JSONUtils.toJsonString(dependTask.get("dependItemList")));
for (int j = 0; j < dependItemList.size(); j++) {
ObjectNode dependItem = (ObjectNode) dependItemList.path(j);
dependItem.put("projectCode", projectIdCodeMap.get(dependItem.get("projectId").asInt()));
int definitionId = dependItem.get("definitionId").asInt();
Map<Long, Map<String, Long>> processCodeTaskNameCodeMap = processTaskMap.get(definitionId);
if (processCodeTaskNameCodeMap == null) {
logger.warn("We can't find processDefinition [{}], please check it is not exist, remove this dependence", definitionId);
dependItemList.remove(j);
continue;
}
Optional<Map.Entry<Long, Map<String, Long>>> mapEntry = processCodeTaskNameCodeMap.entrySet().stream().findFirst();
if (mapEntry.isPresent()) {
Map.Entry<Long, Map<String, Long>> processCodeTaskNameCodeEntry = mapEntry.get();
dependItem.put("definitionCode", processCodeTaskNameCodeEntry.getKey());
String depTasks = dependItem.get("depTasks").asText();
long taskCode = "ALL".equals(depTasks) || processCodeTaskNameCodeEntry.getValue() == null ? 0L : processCodeTaskNameCodeEntry.getValue().get(depTasks); // exception code
dependItem.put("depTaskCode", taskCode);
}
dependItem.remove("projectId");
dependItem.remove("definitionId");
dependItem.remove("depTasks");
dependItemList.set(j, dependItem);
}
dependTask.put("dependItemList", dependItemList);
dependTaskList.set(i, dependTask);
}
```
look at the exception code. `processCodeTaskNameCodeEntry` shouldn't be null. And when `processCodeTaskNameCodeEntry.getValue()` is null ,it should be return `0L`. I can't figure how to happened. My branch version is 2.0.5-release. Could you have a look at here, please. @JinyLeeChina
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] syncwt commented on issue #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query process defined list
Posted by GitBox <gi...@apache.org>.
syncwt commented on issue #10037:
URL: https://github.com/apache/dolphinscheduler/issues/10037#issuecomment-1136785101
> ```java
> for (int i = 0; i < dependTaskList.size(); i++) {
> ObjectNode dependTask = (ObjectNode) dependTaskList.path(i);
> ArrayNode dependItemList = JSONUtils.parseArray(JSONUtils.toJsonString(dependTask.get("dependItemList")));
> for (int j = 0; j < dependItemList.size(); j++) {
> ObjectNode dependItem = (ObjectNode) dependItemList.path(j);
> dependItem.put("projectCode", projectIdCodeMap.get(dependItem.get("projectId").asInt()));
> int definitionId = dependItem.get("definitionId").asInt();
> Map<Long, Map<String, Long>> processCodeTaskNameCodeMap = processTaskMap.get(definitionId);
> if (processCodeTaskNameCodeMap == null) {
> logger.warn("We can't find processDefinition [{}], please check it is not exist, remove this dependence", definitionId);
> dependItemList.remove(j);
> continue;
> }
> Optional<Map.Entry<Long, Map<String, Long>>> mapEntry = processCodeTaskNameCodeMap.entrySet().stream().findFirst();
> if (mapEntry.isPresent()) {
> Map.Entry<Long, Map<String, Long>> processCodeTaskNameCodeEntry = mapEntry.get();
> dependItem.put("definitionCode", processCodeTaskNameCodeEntry.getKey());
> String depTasks = dependItem.get("depTasks").asText();
> long taskCode = "ALL".equals(depTasks) || processCodeTaskNameCodeEntry.getValue() == null ? 0L : processCodeTaskNameCodeEntry.getValue().get(depTasks); // exception code
> dependItem.put("depTaskCode", taskCode);
> }
> dependItem.remove("projectId");
> dependItem.remove("definitionId");
> dependItem.remove("depTasks");
> dependItemList.set(j, dependItem);
> }
> dependTask.put("dependItemList", dependItemList);
> dependTaskList.set(i, dependTask);
> }
> ```
>
> look at the exception code. `processCodeTaskNameCodeEntry` shouldn't be null. And when `processCodeTaskNameCodeEntry.getValue()` is null ,it should be return `0L`. I can't figure how to happened. My branch‘s version is 2.0.5-release and I have checked the code is same with above from the compiled jar . Could you have a look at here, please. @JinyLeeChina
I found the reason! Firstly, the NPE cause by `long` can't be null. Maybe it should replaced by `Long`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on issue #10037: [Bug] [Upgrade] Upgrade 1.3.5 to 2.0.5 failed to query process defined list
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on issue #10037:
URL: https://github.com/apache/dolphinscheduler/issues/10037#issuecomment-1139433061
Glad to here that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org