You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@uniffle.apache.org by GitBox <gi...@apache.org> on 2022/11/12 07:58:15 UTC
[GitHub] [incubator-uniffle] zuston opened a new issue, #317: [FEATURE] Introduce the general remote fs access layer
zuston opened a new issue, #317:
URL: https://github.com/apache/incubator-uniffle/issues/317
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/incubator-uniffle/issues?q=is%3Aissue) and found no similar issues.
### Describe the feature
Currently, we use the Hadoop filesystem interface to be interactive with S3/OBS/HDFS/COS, maybe this will make uniffle
bound to Hadoop ecosystem.
In my opinion, when deploying uniffle into the K8s and directly use the s3 as the backend storage, there is not necessary to use the Hadoop api to visit S3.
So we should introduce the general fs access layer to unify all fs visit, including HDFS/S3, like flink does.
### Motivation
_No response_
### Describe the solution
_No response_
### Additional context
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] jerqi commented on issue #317: [FEATURE] Introduce the general remote fs access layer
Posted by GitBox <gi...@apache.org>.
jerqi commented on issue #317:
URL: https://github.com/apache/incubator-uniffle/issues/317#issuecomment-1312420130
I have some concern about some low performance operations about object storage such as `Append` operation and `List` operation. Maybe we need extra optimization.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] zuston commented on issue #317: [FEATURE] Introduce the general remote fs access layer
Posted by "zuston (via GitHub)" <gi...@apache.org>.
zuston commented on issue #317:
URL: https://github.com/apache/incubator-uniffle/issues/317#issuecomment-1413081438
This issue may be related with the support of object store. Let's track this in https://github.com/apache/incubator-uniffle/issues/391
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] zuston closed issue #317: [FEATURE] Introduce the general remote fs access layer
Posted by "zuston (via GitHub)" <gi...@apache.org>.
zuston closed issue #317: [FEATURE] Introduce the general remote fs access layer
URL: https://github.com/apache/incubator-uniffle/issues/317
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org