You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by "Xuanwo (via GitHub)" <gi...@apache.org> on 2024/02/02 11:56:03 UTC
[I] GSoC Ideas: ofs, OpenDAL File System that allow users to mount storage services locally [opendal]
Xuanwo opened a new issue, #4130:
URL: https://github.com/apache/opendal/issues/4130
# Background
**OpenDAL** is a data access layer that allows users to easily and efficiently retrieve data from various storage services in a unified way. `ofs` can expose OpenDAL power in fuse way that allow users to mount storage services locally.
# Objectives
Implement `ofs`, allowing users to mount storage services locally for read and write.
## Features
In Scope:
- Continuous reading
- Continuous writing
- Random reading
- List dir
- Stat file
Out Scope:
- Random Write
- Xattrs
- Permissions
## Tasks
- Implement features that in scope
- Implement tests suite
# Recommended Skills
- Familiar with Rust
- Familiar with basic ideas of file system and fuse
- Familiar with OpenDAL Rust Core
# Mentor
Mailing List: dev@opendal.apache.org
Mentor: Xuanwo, Apache OpenDAL PMC, xuanwo@apache.org
*Please leave comments if you want to be a mentor*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] GSoC Ideas: ofs, OpenDAL File System via FUSE [opendal]
Posted by "simonsan (via GitHub)" <gi...@apache.org>.
simonsan commented on issue #4130:
URL: https://github.com/apache/opendal/issues/4130#issuecomment-1929611024
> > Will `ofs` (opendal-fs?) be a library or a CLI tool?
>
> I expect to have both lib and cli, but that depends on the code owner's decision.
Ok, I think it is a good way to [separate these concerns](https://doc.rust-lang.org/book/ch12-03-improving-error-handling-and-modularity.html?highlight=split#splitting-code-into-a-library-crate) for modularity and would be also helpful to link against it, when other projects, want to pull in that code. Not affiliated with it in any way, but I find it good, how [abscissa](https://github.com/iqlusioninc/abscissa) applies that when initializing a new project, it's a good way to quickly start. So just leaving it here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] GSoC Ideas: ofs, OpenDAL File System via FUSE [opendal]
Posted by "bokket (via GitHub)" <gi...@apache.org>.
bokket commented on issue #4130:
URL: https://github.com/apache/opendal/issues/4130#issuecomment-1969076909
we discussed in [discord](https://discord.com/channels/1081052318650339399/1081052319342407715/1212329570590654494).but not clear.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] GSoC Ideas: ofs, OpenDAL File System via FUSE [opendal]
Posted by "xiaguan (via GitHub)" <gi...@apache.org>.
xiaguan commented on issue #4130:
URL: https://github.com/apache/opendal/issues/4130#issuecomment-1968882306
Based on the existing `ofs` code, it seems. This idea would require implementing a reader and writer to handle prefetch and concurrent asynchronous write-backs, while also managing a certain amount of memory usage. There are also some things to consider on the metadata level.
It looks like my previous `page_cache` repo might be of some help. I'm considering applying for this idea.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] GSoC Ideas: ofs, OpenDAL File System via FUSE [opendal]
Posted by "Zheaoli (via GitHub)" <gi...@apache.org>.
Zheaoli commented on issue #4130:
URL: https://github.com/apache/opendal/issues/4130#issuecomment-1952800654
I'm familiar with the fuse before. I would like to be one of the mentor for this idea
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] GSoC Ideas: ofs, OpenDAL File System via FUSE [opendal]
Posted by "ho-229 (via GitHub)" <gi...@apache.org>.
ho-229 commented on issue #4130:
URL: https://github.com/apache/opendal/issues/4130#issuecomment-2011548094
I want to support Windows by implementing `CloudFilter` for ofs, I will write proposal later
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] GSoC Ideas: ofs, OpenDAL File System via FUSE [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #4130:
URL: https://github.com/apache/opendal/issues/4130#issuecomment-1929377725
> Will `ofs` (opendal-fs?) be a library or a CLI tool?
I expect to have both lib and cli, but that depends on the code owner's decision.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] GSoC Ideas: ofs, OpenDAL File System via FUSE [opendal]
Posted by "oowl (via GitHub)" <gi...@apache.org>.
oowl commented on issue #4130:
URL: https://github.com/apache/opendal/issues/4130#issuecomment-1925196567
Can I mentor this GSOC project? And we have done some work for it before, but I am not sure if this project for GSOC is reasonable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] GSoC Ideas: ofs, OpenDAL File System via FUSE [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo closed issue #4130: GSoC Ideas: ofs, OpenDAL File System via FUSE
URL: https://github.com/apache/opendal/issues/4130
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] GSoC Ideas: ofs, OpenDAL File System via FUSE [opendal]
Posted by "oowl (via GitHub)" <gi...@apache.org>.
oowl commented on issue #4130:
URL: https://github.com/apache/opendal/issues/4130#issuecomment-1925560420
> we do the linux [fuse](https://github.com/cberner/fuser) or [fuse3](https://github.com/Sherlock-Holo/fuse3) first, and then consider the [S3](https://github.com/awslabs/mountpoint-s3) support? @oowl
we do not need to implement all these fuse libraries, just choose one of them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] GSoC Ideas: ofs, OpenDAL File System via FUSE [opendal]
Posted by "xiaguan (via GitHub)" <gi...@apache.org>.
xiaguan commented on issue #4130:
URL: https://github.com/apache/opendal/issues/4130#issuecomment-1968868424
It seems to be a local mapping of cloud storage. It looks like switching out Datenlord's metadata engine, etcd, for a local SQLite should do the trick.
Rather than the data layer stuff, I'm more intersted about some decisions regarding metadata
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] GSoC Ideas: ofs, OpenDAL File System via FUSE [opendal]
Posted by "ho-229 (via GitHub)" <gi...@apache.org>.
ho-229 commented on issue #4130:
URL: https://github.com/apache/opendal/issues/4130#issuecomment-1959547916
I think we should update the goals of this idea, the goals listed above seems be implemented in https://github.com/apache/opendal/pull/4179
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] GSoC Ideas: ofs, OpenDAL File System via FUSE [opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on issue #4130:
URL: https://github.com/apache/opendal/issues/4130#issuecomment-1959634598
> I think we should update the goals of this idea, the goals listed above seems be implemented in https://github.com/apache/opendal/pull/4179
Agreed, I will update the goals.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] GSoC Ideas: ofs, OpenDAL File System via FUSE [opendal]
Posted by "simonsan (via GitHub)" <gi...@apache.org>.
simonsan commented on issue #4130:
URL: https://github.com/apache/opendal/issues/4130#issuecomment-1929329306
Will `ofs` (opendal-fs?) be a library or a CLI tool?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] GSoC Ideas: ofs, OpenDAL File System via FUSE [opendal]
Posted by "bokket (via GitHub)" <gi...@apache.org>.
bokket commented on issue #4130:
URL: https://github.com/apache/opendal/issues/4130#issuecomment-1925558200
I interested in.Whether we discuss it before, we do the linux [fuse](https://github.com/cberner/fuser) first, and then consider the [S3](https://github.com/awslabs/mountpoint-s3)? @oowl
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] GSoC Ideas: ofs, OpenDAL File System via FUSE [opendal]
Posted by "bokket (via GitHub)" <gi...@apache.org>.
bokket commented on issue #4130:
URL: https://github.com/apache/opendal/issues/4130#issuecomment-1925561924
> Welcome, but we need to wait for the nomination process of Gsoc.
The website shows that the organization is announced on February 21st.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org