You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2021/11/02 14:47:06 UTC
[GitHub] [dolphinscheduler] jealone opened a new issue #6670: [Bug] [minio] create tenant error
jealone opened a new issue #6670:
URL: https://github.com/apache/dolphinscheduler/issues/6670
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
### What happened
I want use the s3/minio as the resources storage, but when I deploy the cluster, I have failed to create tenant. And I get the error logs as follow
```
ERROR] 2021-11-02 14:12:36.262 org.apache.dolphinscheduler.api.exceptions.ApiExceptionHandler:[45] - 创建租户错误
org.apache.hadoop.fs.FileAlreadyExistsException: Path is a file: /beeto_bi/resources
at org.apache.hadoop.fs.s3a.S3AFileSystem.mkdirs(S3AFileSystem.java:853)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1881)
at org.apache.dolphinscheduler.common.utils.HadoopUtils.mkdir(HadoopUtils.java:276)
at org.apache.dolphinscheduler.api.service.BaseService.createTenantDirIfNotExists(BaseService.java:129)
at org.apache.dolphinscheduler.api.service.TenantService.createTenant(TenantService.java:120)
at org.apache.dolphinscheduler.api.service.TenantService$$FastClassBySpringCGLIB$$13bc5043.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:752)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:295)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
at org.apache.dolphinscheduler.api.service.TenantService$$EnhancerBySpringCGLIB$$46064c30.createTenant(<generated>)
at org.apache.dolphinscheduler.api.controller.TenantController.createTenant(TenantController.java:85)
.....
```
And then I scan the source code，I find that tenant Controller will mkdir with udf and resources，but S3AFilesystem in hadoop-aws-2.7.3 will check if the path is directory. Unfortunately, the check rule is that if the path is end with '/'.
any instructions details to support minio?
### What you expected to happen
use minio as the resources storage
### How to reproduce
memention above
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #6670: [Bug] [minio] create tenant error
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #6670:
URL: https://github.com/apache/dolphinscheduler/issues/6670#issuecomment-985103775
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #6670: [Bug] [minio] create tenant error
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #6670:
URL: https://github.com/apache/dolphinscheduler/issues/6670#issuecomment-957738310
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #6670: [Bug] [minio] create tenant error
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #6670:
URL: https://github.com/apache/dolphinscheduler/issues/6670#issuecomment-957738965
Hi:
* Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
* In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
* If you haven't received a reply for a long time, you can subscribe to the developer's email，Mail subscription steps reference https://dolphinscheduler.apache.org/en-us/community/development/subscribe.html ,Then write the issue URL in the email content and send question to dev@dolphinscheduler.apache.org.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #6670: [Bug] [minio] create tenant error
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #6670:
URL: https://github.com/apache/dolphinscheduler/issues/6670#issuecomment-957738310
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
### What happened
I want use the s3/minio as the resources storage, but when I deploy the cluster, I have failed to create tenant. And I get the error logs as follow
```
ERROR] 2021-11-02 14:12:36.262 org.apache.dolphinscheduler.api.exceptions.ApiExceptionHandler:[45] - 创建租户错误
org.apache.hadoop.fs.FileAlreadyExistsException: Path is a file: /beeto_bi/resources
at org.apache.hadoop.fs.s3a.S3AFileSystem.mkdirs(S3AFileSystem.java:853)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1881)
at org.apache.dolphinscheduler.common.utils.HadoopUtils.mkdir(HadoopUtils.java:276)
at org.apache.dolphinscheduler.api.service.BaseService.createTenantDirIfNotExists(BaseService.java:129)
at org.apache.dolphinscheduler.api.service.TenantService.createTenant(TenantService.java:120)
at org.apache.dolphinscheduler.api.service.TenantService$$FastClassBySpringCGLIB$$13bc5043.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:752)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:295)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
at org.apache.dolphinscheduler.api.service.TenantService$$EnhancerBySpringCGLIB$$46064c30.createTenant(<generated>)
at org.apache.dolphinscheduler.api.controller.TenantController.createTenant(TenantController.java:85)
.....
```
And then I scan the source code，I find that tenant Controller will mkdir with udf and resources，but S3AFilesystem in hadoop-aws-2.7.3 will check if the path is directory. Unfortunately, the check rule is that if the path is end with '/'.
any instructions details to support minio?
### What you expected to happen
use minio as the resources storage
### How to reproduce
memention above
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #6670: [Bug] [minio] create tenant error
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #6670:
URL: https://github.com/apache/dolphinscheduler/issues/6670#issuecomment-957738310
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org