You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2020/06/24 01:37:26 UTC
[GitHub] [incubator-dolphinscheduler] which520 opened a new issue #3048: Spark任务提交的时候，后台日志显示节点参数无效
which520 opened a new issue #3048:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3048
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] yangyichao-mango commented on issue #3048: Spark任务提交的时候，后台日志显示节点参数无效
Posted by GitBox <gi...@apache.org>.
yangyichao-mango commented on issue #3048:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3048#issuecomment-648533345
> [INFO] 2020-06-23 12:18:28.859 org.apache.dolphinscheduler.api.controller.ProcessDefinitionController:[159] - login user which, update process define, project name: test2, process define name: which_test, process_definition_json: {"globalParams":[],"tasks":[{"type":"SHELL","id":"tasks-56548","name":"wewe","params":{"resourceList":[],"localParams":[],"rawScript":"czczxcx"},"description":"","runFlag":"NORMAL","dependence":{},"maxRetryTimes":"0","retryInterval":"1","timeout":{"strategy":"","interval":null,"enable":false},"taskInstancePriority":"MEDIUM","workerGroupId":-1,"preTasks":[]},{"type":"PYTHON","id":"tasks-65022","name":"test","params":{"resourceList":[{"res":"python_test122.py"}],"localParams":[],"rawScript":"dsds"},"description":"","runFlag":"NORMAL","dependence":{},"maxRetryTimes":"0","retryInterval":"1","timeout":{"strategy":"","interval":null,"enable":false},"taskInstancePriority":"MEDIUM","workerGroupId":-1,"preTasks":[]},{"type":"FLINK","id":"tasks-31676","name":"dsds","params":{"mainClass":"","mainJar":{"res":"python_test122.py"},"deployMode":"cluster","resourceList":[],"localParams":[],"slot":1,"taskManager":"2","jobManagerMemory":"1G","taskManagerMemory":"2G","executorCores":2,"mainArgs":"","others":"","programType":"PYTHON"},"description":"","runFlag":"NORMAL","dependence":{},"maxRetryTimes":"0","retryInterval":"1","timeout":{"strategy":"","interval":null,"enable":false},"taskInstancePriority":"MEDIUM","workerGroupId":-1,"preTasks":[]},{"type":"SPARK","id":"tasks-74978","name":"dcsc","params":{"mainClass":"","mainJar":{"res":"python_test122.py"},"deployMode":"cluster","resourceList":[],"localParams":[],"driverCores":1,"driverMemory":"512M","numExecutors":2,"executorMemory":"2G","executorCores":2,"mainArgs":"","others":"","programType":"PYTHON"},"description":"","runFlag":"NORMAL","dependence":{},"maxRetryTimes":"0","retryInterval":"1","timeout":{"strategy":"","interval":null,"enable":false},"taskInstancePriority":"MEDIUM","workerGroupId":-1,"preTasks":[]}],"tenantId":-1,"timeout":"30"}, desc: dsfsdf, locations:{"tasks-56548":{"name":"wewe","targetarr":"","x":238,"y":104},"tasks-65022":{"name":"test","targetarr":"","x":457,"y":324},"tasks-31676":{"name":"dsds","targetarr":"","x":626,"y":219},"tasks-74978":{"name":"dcsc","targetarr":"","x":460,"y":161}}, connects:[]
> [ERROR] 2020-06-23 12:18:28.864 org.apache.dolphinscheduler.api.service.ProcessDefinitionService:[799] - task node dcsc parameter invalid
Which version?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] xingchun-chen closed issue #3048: Spark任务提交的时候，后台日志显示节点参数无效
Posted by GitBox <gi...@apache.org>.
xingchun-chen closed issue #3048:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3048
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] xingchun-chen commented on issue #3048: Spark任务提交的时候，后台日志显示节点参数无效
Posted by GitBox <gi...@apache.org>.
xingchun-chen commented on issue #3048:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3048#issuecomment-648543035
Your spark node does not fill in the mainClass and mainArgs parameters, and there is no "sparkVersion" parameter
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] which520 commented on issue #3048: Spark任务提交的时候，后台日志显示节点参数无效
Posted by GitBox <gi...@apache.org>.
which520 commented on issue #3048:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3048#issuecomment-648536708
public boolean checkParameters() {
return mainJar != null && programType != null && sparkVersion != null;
}
我怀疑是我的sparkVersion没有获取到，但是我不知道这个sparkVersion是怎么获取的
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] which520 commented on issue #3048: Spark任务提交的时候，后台日志显示节点参数无效
Posted by GitBox <gi...@apache.org>.
which520 commented on issue #3048:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3048#issuecomment-648587034
I lost an option for the Spark version of my UI,After the UI is reinstalled, it returns to normal
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] which520 commented on issue #3048: Spark任务提交的时候，后台日志显示节点参数无效
Posted by GitBox <gi...@apache.org>.
which520 commented on issue #3048:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3048#issuecomment-648530948
[INFO] 2020-06-23 12:18:28.859 org.apache.dolphinscheduler.api.controller.ProcessDefinitionController:[159] - login user which, update process define, project name: test2, process define name: which_test, process_definition_json: {"globalParams":[],"tasks":[{"type":"SHELL","id":"tasks-56548","name":"wewe","params":{"resourceList":[],"localParams":[],"rawScript":"czczxcx"},"description":"","runFlag":"NORMAL","dependence":{},"maxRetryTimes":"0","retryInterval":"1","timeout":{"strategy":"","interval":null,"enable":false},"taskInstancePriority":"MEDIUM","workerGroupId":-1,"preTasks":[]},{"type":"PYTHON","id":"tasks-65022","name":"test","params":{"resourceList":[{"res":"python_test122.py"}],"localParams":[],"rawScript":"dsds"},"description":"","runFlag":"NORMAL","dependence":{},"maxRetryTimes":"0","retryInterval":"1","timeout":{"strategy":"","interval":null,"enable":false},"taskInstancePriority":"MEDIUM","workerGroupId":-1,"preTasks":[]},{"type":"FLINK","id":"tasks-31676","name":"dsds","params":{"mainClass":"","mainJar":{"res":"python_test122.py"},"deployMode":"cluster","resourceList":[],"localParams":[],"slot":1,"taskManager":"2","jobManagerMemory":"1G","taskManagerMemory":"2G","executorCores":2,"mainArgs":"","others":"","programType":"PYTHON"},"description":"","runFlag":"NORMAL","dependence":{},"maxRetryTimes":"0","retryInterval":"1","timeout":{"strategy":"","interval":null,"enable":false},"taskInstancePriority":"MEDIUM","workerGroupId":-1,"preTasks":[]},{"type":"SPARK","id":"tasks-74978","name":"dcsc","params":{"mainClass":"","mainJar":{"res":"python_test122.py"},"deployMode":"cluster","resourceList":[],"localParams":[],"driverCores":1,"driverMemory":"512M","numExecutors":2,"executorMemory":"2G","executorCores":2,"mainArgs":"","others":"","programType":"PYTHON"},"description":"","runFlag":"NORMAL","dependence":{},"maxRetryTimes":"0","retryInterval":"1","timeout":{"strategy":"","interval":null,"enable":false},"taskInstancePriority":"MEDIUM","workerGroupId":-1,"preTasks":[]}],"tenantId":-1,"timeout":"30"}, desc: dsfsdf, locations:{"tasks-56548":{"name":"wewe","targetarr":"","x":238,"y":104},"tasks-65022":{"name":"test","targetarr":"","x":457,"y":324},"tasks-31676":{"name":"dsds","targetarr":"","x":626,"y":219},"tasks-74978":{"name":"dcsc","targetarr":"","x":460,"y":161}}, connects:[]
[ERROR] 2020-06-23 12:18:28.864 org.apache.dolphinscheduler.api.service.ProcessDefinitionService:[799] - task node dcsc parameter invalid
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] xingchun-chen commented on issue #3048: Spark任务提交的时候，后台日志显示节点参数无效
Posted by GitBox <gi...@apache.org>.
xingchun-chen commented on issue #3048:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3048#issuecomment-648543204
please update the title in english
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler] which520 commented on issue #3048: Spark任务提交的时候，后台日志显示节点参数无效
Posted by GitBox <gi...@apache.org>.
which520 commented on issue #3048:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/3048#issuecomment-648535525
1.2.1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org