[GitHub] [incubator-dolphinscheduler] zhaoyw1 commented on issue #1945: [QUESTION]About the commit record for issues#79

[GitBox <gi...@apache.org>]

zhaoyw1 commented on issue #1945:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/1945#issuecomment-727827487


   Yes,  have solved it by what I mentioned in this E-mail.
   Besides, we have launched lots of hudi tasks(spark) which are scheduled frequently for incremental synchronization. But every time they are scheduled, they will download the same jar-resources and then "not enough disk space" alert are triggered frequently. So I realize the resource reference and synchronization, this repetitive and unnecessary step has been avoided. Later I will share the solution in E-mail and hope it useful for community if no related pr is merged.
   
   这个问题已经解决了，用的就是我在邮件里面提到的方式。
   此外，我们近期上了大批量的调度非常频繁的hudi增量同步任务，由于每次任务都会从hdfs上拉取资源到本地，磁盘空间不足的告警会被频繁触发。所以我实现了资源引用与同步来解决这个问题，目前生产上已经不存在这个问题了。最近我会把我的实现方式通过邮件的方式发出来，如果社区还没有类似的pr提交的话，希望有所帮助。
   
   
   ------------------------------------------------------------------
   发件人：dailidong <no...@github.com>
   发送时间：2020年11月16日(星期一) 16:21
   收件人：apache/incubator-dolphinscheduler <in...@noreply.github.com>
   抄　送：赵玉威 <zh...@t3go.cn>; Author <au...@noreply.github.com>
   主　题：Re: [apache/incubator-dolphinscheduler] [QUESTION]About the commit record for issues#79 (#1945)
   
   Hi, how are you? Last email you provided the API DOC for dolphin scheduler, and it helps us a lot. Thx sincerely! But when I use ProcessInstanceController#updateProcessInstance, the param "scheduler time" cause some trouble. Here is the command how I tested: curl -X POST "http://incubator-t3-infra04:12345/dolphinscheduler/projects/test_api/instance/update?processInstanceJson=&processInstanceId=3&scheduleTime=&syncDefine=&locations=&connects=&flag=NO" -H "token:62ec76d988f746303226fe4216fd6247" (the token belong to admin and doesnt expried) Here is the response:{"code":0,"msg":"success","data":null} While the fact is the corresponding process instances flag doesn`t update to 0(from 1,get verfied through t_ds_process_definition), and check the log of dolphinscheduler-api-server.log, find that: [INFO] 2020-04-17 10:15:08.797 org.apache.dolphinscheduler.api.controller.ProcessInstanceController:[178] - updateProcessInstance process instance, login user:admin, project name:test_api, pro
 cess instance json:,process instance id:3, schedule time:, sync define:false, flag:NO, locations:, connects: [ERROR] 2020-04-17 10:15:08.800 org.apache.dolphinscheduler.common.utils.DateUtils:[123] - error while parse date: java.time.format.DateTimeParseException: Text '' could not be parsed at index 0 at java.time.format.DateTimeFormatter.parseResolved0(DateTimeFormatter.java:1949) at java.time.format.DateTimeFormatter.parse(DateTimeFormatter.java:1851) at java.time.LocalDateTime.parse(LocalDateTime.java:492) at org.apache.dolphinscheduler.common.utils.DateUtils.parse(DateUtils.java:120) at org.apache.dolphinscheduler.common.utils.DateUtils.stringToDate(DateUtils.java:136) at org.apache.dolphinscheduler.common.utils.DateUtils.getScheduleDate(DateUtils.java:220) at org.apache.dolphinscheduler.api.service.ProcessInstanceService.updateProcessInstance(ProcessInstanceService.java:375) at org.apache.dolphinscheduler.api.service.ProcessInstanceService$$FastClassBySpringCGLIB$$da33359.invo
 ke() in ProcessInstanceService.updateProcessInstance, the releated code is: So I am wondering whether "!schedulerTime.isEmpty()" could solve this error for ".scheduleTime=&." situation. Yours sincerely.
   …
    ------------------------------------------------------------------ 发件人：dailidong notifications@github.com 发送时间：2020年3月21日(星期六) 16:42 收件人：apache/incubator-dolphinscheduler incubator-dolphinscheduler@noreply.github.com 抄　送：赵玉威 zhaoyw@t3go.cn; Author author@noreply.github.com 主　题：Re: [apache/incubator-dolphinscheduler] [QUESTION]About the commit record for issues#79 (#1945) refer API DOC: http://106.75.43.194:8888/dolphinscheduler/doc.html?language=zh_CN&lang=zh this is 1.2.0 API doc, maybe there is little difference with 1.1.0 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.
   did you solve this problem?
   —
   You are receiving this because you authored the thread.
   Reply to this email directly, view it on GitHub, or unsubscribe. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org