You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by "TheWindIsRising (via GitHub)" <gi...@apache.org> on 2023/02/27 02:10:38 UTC

[GitHub] [dolphinscheduler] TheWindIsRising opened a new issue, #13633: 关于我把资源中心修改为minIO存储，执行spark任务，选项参数找不到资源的问题

TheWindIsRising opened a new issue, #13633:
URL: https://github.com/apache/dolphinscheduler/issues/13633

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar feature requirement.
   
   
   ### Description
   
   以上是我的资源配置
   当我执行任务时
   ![image](https://user-images.githubusercontent.com/124865830/221455646-eae9e87c-2e42-4b32-9a08-b7333717840a.png)
   租户为root的时候
   ![企业微信截图_167729453927](https://user-images.githubusercontent.com/124865830/221455692-f66b13b6-4a97-4569-afeb-d740dfb179cb.png)
   当我把租户换成hdfs时候
   我换了个hdfs租户它直接去hdfs上找了， 如果是root租户的话它会先从本地拉取然后上传到hdfs临时目录，然后root租户的时候它拉着拉着给我文件名称拉没了
   ![企业微信截图_16772946992914](https://user-images.githubusercontent.com/124865830/221455765-0afe23e9-7f23-4ea3-bc30-5f87e8415284.png)
   所以我在想这个问题是不是可以配置，或者哪个地方有冲突，权限不够什么
   
   ### Are you willing to submit a PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #13633: Regarding the problem that I changed the resource center to minIO storage and executed the spark task, the resource could not be found in the option parameter

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #13633:
URL: https://github.com/apache/dolphinscheduler/issues/13633#issuecomment-1445567373

   Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
   * In order for us to understand your request as soon as possible, please provide detailed information, version or pictures.
   * If you haven't received a reply for a long time, you can [join our slack](https://s.apache.org/dolphinscheduler-slack) and send your question to channel `#troubleshooting`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] TheWindIsRising commented on issue #13633: Regarding the problem that I changed the resource center to minIO storage and executed the spark task, the resource could not be found in the option parameter

Posted by "TheWindIsRising (via GitHub)" <gi...@apache.org>.
TheWindIsRising commented on issue #13633:
URL: https://github.com/apache/dolphinscheduler/issues/13633#issuecomment-1445567907

   如果简单在yarn上调度spark-examples_2.11-2.2.0.jarAPI的时候没用任何问题出现


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #13633: Regarding the problem that I changed the resource center to minIO storage and executed the spark task, the resource could not be found in the option parameter

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #13633:
URL: https://github.com/apache/dolphinscheduler/issues/13633#issuecomment-1445567253

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar feature requirement.
   
   
   ### Description
   
   The above is my resource configuration
   when i execute the task
   ![image](https://user-images.githubusercontent.com/124865830/221455646-eae9e87c-2e42-4b32-9a08-b7333717840a.png)
   When the tenant is root
   ![Enterprise WeChat screenshot_167729453927](https://user-images.githubusercontent.com/124865830/221455692-f66b13b6-4a97-4569-afeb-d740dfb179cb.png)
   When I change the tenant to hdfs
   I changed the hdfs tenant and it went directly to hdfs to find it. If it is a root tenant, it will first pull it from the local and then upload it to the hdfs temporary directory. Then, when it is a root tenant, it pulls and pulls and gives me the file name.
   ![Enterprise WeChat screenshot_16772946992914](https://user-images.githubusercontent.com/124865830/221455765-0afe23e9-7f23-4ea3-bc30-5f87e8415284.png)
   So I am wondering if this problem can be configured, or where there is a conflict, what is the lack of permissions?
   
   ### Are you willing to submit a PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] TheWindIsRising commented on issue #13633: Regarding the problem that I changed the resource center to minIO storage and executed the spark task, the resource could not be found in the option parameter

Posted by "TheWindIsRising (via GitHub)" <gi...@apache.org>.
TheWindIsRising commented on issue #13633:
URL: https://github.com/apache/dolphinscheduler/issues/13633#issuecomment-1445570152

   ![企业微信截图_16773120761080](https://user-images.githubusercontent.com/124865830/221456197-647ed701-fb53-4ef0-8440-0159c198eca8.png)
   spark.yarn.jars这个配置也不支持http，dolphinscheduler是容器部署 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] TheWindIsRising commented on issue #13633: Regarding the problem that I changed the resource center to minIO storage and executed the spark task, the resource could not be found in the option parameter

Posted by "TheWindIsRising (via GitHub)" <gi...@apache.org>.
TheWindIsRising commented on issue #13633:
URL: https://github.com/apache/dolphinscheduler/issues/13633#issuecomment-1445569620

   请问选项参数应该怎么指定关于--jars之类的资源路径


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Regarding the problem that I changed the resource center to minIO storage and executed the spark task, the resource could not be found in the option parameter [dolphinscheduler]

Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS closed issue #13633: Regarding the problem that I changed the resource center to minIO storage and executed the spark task, the resource could not be found in the option parameter
URL: https://github.com/apache/dolphinscheduler/issues/13633


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org