You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2021/07/22 06:01:06 UTC
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #5876: [Bug][SQL] Suggest to increase the max length of the field resource_ids in the table t_ds_task_definition.
github-actions[bot] commented on issue #5876:
URL: https://github.com/apache/dolphinscheduler/issues/5876#issuecomment-884674063
**Describe the bug**
When I saved a shell task that used many resource files ,the service threw PSQLException.
**To Reproduce**
Steps to reproduce the behavior, for example:
1. Create a shell task and select many resource files.
2. Save the workflow, you would see that again. If you don't , maybe you would do well to pick more resource files.
**Expected behavior**
Expect to increase the max length of the field resource_ids in the table t_ds_task_definition.
**Screenshots**
**Which version of Dolphin Scheduler:**
-[dev]
**Additional context**
Exception stack trace:
```
[INFO] 2021-07-22 11:51:07.658 org.apache.dolphinscheduler.api.aspect.AccessLogAspect:[75] - REQUEST TRANCE_ID:0702204b-58c0-4f0f-8b4b-dbfef2be63cc, LOGIN_USER:admin, URI:/dolphinscheduler/users/list-paging, METHOD:GET, HANDLER:org.apache.dolphinscheduler.api.controller.UsersController.queryUserList, ARGS:{searchVal=jiang.hua, pageNo=1, pageSize=10}
[INFO] 2021-07-22 11:51:07.682 org.apache.dolphinscheduler.api.aspect.AccessLogAspect:[75] - REQUEST TRANCE_ID:55caab3c-e003-4925-88c9-172dd2318d7c, LOGIN_USER:admin, URI:/dolphinscheduler/projects/feature-works/process/update, METHOD:POST, HANDLER:org.apache.dolphinscheduler.api.controller.ProcessDefinitionController.updateProcessDefinition, ARGS:{connects=[], name=jianghua_test, description=test, locations={"tasks-24242":{"name":"restore_from_checkpoint_test123","targetarr":"","nodenumber":"0","x":-800,"y":-900,"color":"#1297DC"}}, id=8, releaseState=OFFLINE, projectName=feature-works, processDefinitionJson={"tasks":[{"id":"tasks-24242","code":491450867713,"version":27,"name":"restore_from_checkpoint_test123","desc":null,"type":"SHELL","runFlag":"NORMAL","loc":null,"maxRetryTimes":3,"retryInterval":1,"params":{"resourceList":[{"id":157},{"id":158},{"id":162},{"id":164},{"id":166},{"id":168},{"id":160},{"id":161},{"id":159},{"id":163},{"id":165},{"id":167},{"id":95},{"id":97},{"i
d":98},{"id":99},{"id":100},{"id":104},{"id":105},{"id":106},{"id":107},{"id":116},{"id":94},{"id":96},{"id":101},{"id":102},{"id":103},{"id":108},{"id":109},{"id":110},{"id":112},{"id":113},{"id":114},{"id":115},{"id":117},{"id":170},{"id":171},{"id":172},{"id":120},{"id":121},{"id":123},{"id":125},{"id":127},{"id":129},{"id":131},{"id":133},{"id":135},{"id":137},{"id":139},{"id":141},{"id":147},{"id":148},{"id":149},{"id":150},{"id":151},{"id":152},{"id":134},{"id":136},{"id":118},{"id":119},{"id":122},{"id":124},{"id":126},{"id":128},{"id":130},{"id":132},{"id":138},{"id":140},{"id":142},{"id":143},{"id":144},{"id":145},{"id":146},{"id":169},{"id":76},{"id":77},{"id":78}],"localParams":[{"prop":"task.name","direct":"IN","type":"VARCHAR","value":"restore_from_checkpoint_test.sql","deleteScript":"-n"},{"prop":"task.script.file","direct":"IN","type":"VARCHAR","value":"Projects/feature-works/tasks/stream/jianghua_test/restore_from_checkpoint_test.sql","deleteScript":"-f"},{"prop":"ta
sk.execution.parallel","direct":"IN","type":"VARCHAR","value":"3","deleteScript":"-p"},{"prop":"task.execution.slots","direct":"IN","type":"VARCHAR","value":"10","deleteScript":"-s"},{"prop":"task.execution.yjm","direct":"IN","type":"VARCHAR","value":"4096m","deleteScript":"-j"},{"prop":"task.execution.ytm","direct":"IN","type":"VARCHAR","value":"4096m","deleteScript":"-w"},{"prop":"task.instance.id","direct":"IN","type":"VARCHAR","value":"${system.task.instance.id}"},{"prop":"task.execute.path","direct":"IN","type":"VARCHAR","value":"${system.task.execute.path}"}],"rawScript":"chmod +x ${task.execute.path}/${datamax.home}/${datamax.version}/bin/*\n\n${task.execute.path}/${datamax.home}/${datamax.version}/bin/start.sh -m ${run.mode} -n ${task.name} -f ${task.script.file} -p ${task.execution.parallel} -s ${task.execution.slots} -j ${task.execution.yjm} -w ${task.execution.ytm} -r ${repository.home} -b ${task.execute.path} -c ${datamax.home}/${datamax.version}/lib/${datamax.core.jar}
-l ${flink.home}/${flink.version} -e ${profile.home}/${run.mode}"},"preTasks":[],"preTaskNodeList":[],"extras":null,"depList":[],"dependence":{},"conditionResult":{"successNode":[""],"failedNode":[""]},"taskInstancePriority":"MEDIUM","workerGroup":"default","timeout":{"enable":false,"strategy":null,"interval":0},"delayTime":0}],"globalParams":[{"prop":"repository.home","direct":"IN","type":"VARCHAR","value":"/data/online/datamax-repository"},{"prop":"run.mode","direct":"IN","type":"VARCHAR","value":"stream"},{"prop":"flink.home","direct":"IN","type":"VARCHAR","value":"./flink/"},{"prop":"datamax.home","direct":"IN","type":"VARCHAR","value":"./datamax/"},{"prop":"datamax.core.jar","direct":"IN","type":"VARCHAR","value":"platform-on-flink_1.13.0-2.0-SNAPSHOT.jar"},{"prop":"flink.version","direct":"IN","type":"VARCHAR","value":"1.13.0"},{"prop":"datamax.version","direct":"IN","type":"VARCHAR","value":"2.0.0"},{"prop":"profile.home","direct":"IN","type":"VARCHAR","value":"./profile/pro
d"},{"prop":"root.home","direct":"IN","type":"VARCHAR","value":"/data/online/datamax-repository"},{"prop":"current.date","direct":"IN","type":"VARCHAR","value":"${system.biz.curdate}"}],"timeout":0,"tenantId":4}}
[ERROR] 2021-07-22 11:51:07.697 org.apache.dolphinscheduler.api.exceptions.ApiExceptionHandler:[46] - 更新工作流定义错误
org.springframework.dao.DataIntegrityViolationException:
### Error updating database. Cause: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)
### The error may exist in org/apache/dolphinscheduler/dao/mapper/TaskDefinitionMapper.java (best guess)
### The error may involve org.apache.dolphinscheduler.dao.mapper.TaskDefinitionMapper.updateById-Inline
### The error occurred while setting parameters
### SQL: UPDATE t_ds_task_definition SET fail_retry_times=?, flag=?, code=?, task_priority=?, update_time=?, task_params=?, user_id=?, version=?, timeout=?, task_type=?, timeout_flag=?, create_time=?, project_code=?, fail_retry_interval=?, name=?, delay_time=?, worker_group=?, resource_ids=? WHERE id=?
### Cause: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)
; ERROR: value too long for type character varying(255); nested exception is org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)
at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81)
at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:74)
at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:440)
at com.sun.proxy.$Proxy101.update(Unknown Source)
at org.mybatis.spring.SqlSessionTemplate.update(SqlSessionTemplate.java:287)
at com.baomidou.mybatisplus.core.override.MybatisMapperMethod.execute(MybatisMapperMethod.java:63)
at com.baomidou.mybatisplus.core.override.MybatisMapperProxy.invoke(MybatisMapperProxy.java:61)
at com.sun.proxy.$Proxy122.updateById(Unknown Source)
at org.apache.dolphinscheduler.service.process.ProcessService.updateTaskDefinition(ProcessService.java:2156)
at org.apache.dolphinscheduler.service.process.ProcessService.handleTaskDefinition(ProcessService.java:2282)
at org.apache.dolphinscheduler.service.process.ProcessService.saveProcessDefinition(ProcessService.java:2212)
at org.apache.dolphinscheduler.service.process.ProcessService$$FastClassBySpringCGLIB$$ed138739.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:687)
at org.apache.dolphinscheduler.service.process.ProcessService$$EnhancerBySpringCGLIB$$d02bc54d.saveProcessDefinition(<generated>)
at org.apache.dolphinscheduler.api.service.impl.ProcessDefinitionServiceImpl.updateProcessDefinition(ProcessDefinitionServiceImpl.java:418)
at org.apache.dolphinscheduler.api.service.impl.ProcessDefinitionServiceImpl$$FastClassBySpringCGLIB$$e8e34ed9.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:687)
at org.apache.dolphinscheduler.api.service.impl.ProcessDefinitionServiceImpl$$EnhancerBySpringCGLIB$$4564e02d.updateProcessDefinition(<generated>)
at org.apache.dolphinscheduler.api.controller.ProcessDefinitionController.updateProcessDefinition(ProcessDefinitionController.java:242)
at org.apache.dolphinscheduler.api.controller.ProcessDefinitionController$$FastClassBySpringCGLIB$$dc9bf5db.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:752)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:88)
at org.apache.dolphinscheduler.api.aspect.AccessLogAspect.doAround(AccessLogAspect.java:86)
at sun.reflect.GeneratedMethodAccessor159.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:644)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:633)
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:93)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
at org.apache.dolphinscheduler.api.controller.ProcessDefinitionController$$EnhancerBySpringCGLIB$$22f5b91d.updateProcessDefinition(<generated>)
at sun.reflect.GeneratedMethodAccessor622.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:892)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:797)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763)
at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1633)
at com.github.xiaoymin.swaggerbootstrapui.filter.SecurityBasicAuthFilter.doFilter(SecurityBasicAuthFilter.java:84)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1609)
at com.github.xiaoymin.swaggerbootstrapui.filter.ProductionSecurityFilter.doFilter(ProductionSecurityFilter.java:53)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1609)
at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:97)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1609)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1609)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1609)
at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:94)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1609)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1609)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:561)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1612)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1582)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:766)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:143)
at org.postgresql.jdbc.PgPreparedStatement.execute(PgPreparedStatement.java:132)
at com.alibaba.druid.pool.DruidPooledPreparedStatement.execute(DruidPooledPreparedStatement.java:497)
at org.apache.ibatis.executor.statement.PreparedStatementHandler.update(PreparedStatementHandler.java:47)
at org.apache.ibatis.executor.statement.RoutingStatementHandler.update(RoutingStatementHandler.java:74)
at sun.reflect.GeneratedMethodAccessor431.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:63)
at com.sun.proxy.$Proxy177.update(Unknown Source)
at com.baomidou.mybatisplus.core.executor.MybatisSimpleExecutor.doUpdate(MybatisSimpleExecutor.java:54)
at org.apache.ibatis.executor.BaseExecutor.update(BaseExecutor.java:117)
at org.apache.ibatis.session.defaults.DefaultSqlSession.update(DefaultSqlSession.java:197)
at sun.reflect.GeneratedMethodAccessor505.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:426)
... 111 common frames omitted
```
**Requirement or improvement**
- I suggest to add some validation to check fields in the form.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org