You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by "TheWindIsRising (via GitHub)" <gi...@apache.org> on 2023/09/12 08:08:32 UTC
[GitHub] [dolphinscheduler] TheWindIsRising opened a new issue, #14903: dolphinscheduler 调度spark任务在yarn上完成后任务实例日志看不到show的部分
TheWindIsRising opened a new issue, #14903:
URL: https://github.com/apache/dolphinscheduler/issues/14903
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar feature requirement.
### Description
使用helm部署了一套dolphinscheduler3.1.2
后续写了一个spark测试模型通过dolphinscheduler调度到了yarn上，模型中用到了df.show函数打印数据，方便查看数据的准确性与测试数据，但是当我执行完任务 在任务日志实例中，并没有看到show的部分，是否可以增加一个全局变量参数设置一个日志的权限或者等级或者页面上增加一个选项
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] After dolphinscheduler schedules the spark task on yarn, the show part cannot be seen in the task instance log. [dolphinscheduler]
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS commented on issue #14903:
URL: https://github.com/apache/dolphinscheduler/issues/14903#issuecomment-1760679651
It's not a ds issue. `spark-submit` would not print any log of `df.show()`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] After dolphinscheduler schedules the spark task on yarn, the show part cannot be seen in the task instance log. [dolphinscheduler]
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS closed issue #14903: After dolphinscheduler schedules the spark task on yarn, the show part cannot be seen in the task instance log.
URL: https://github.com/apache/dolphinscheduler/issues/14903
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #14903: After dolphinscheduler schedules the spark task on yarn, the show part cannot be seen in the task instance log.
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #14903:
URL: https://github.com/apache/dolphinscheduler/issues/14903#issuecomment-1715215354
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar feature requirement.
### Description
Deployed a set of dolphinscheduler3.1.2 using helm
Subsequently, I wrote a spark test model and dispatched it to yarn through dolphinscheduler. The df.show function was used in the model to print data to facilitate checking the accuracy of the data and test data. However, when I finished executing the task, I did not see it in the task log instance. In the show part, can I add a global variable parameter to set the permission or level of a log or add an option on the page?
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org