You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by "Xuanwo (via GitHub)" <gi...@apache.org> on 2023/09/21 07:47:34 UTC
[GitHub] [incubator-opendal] Xuanwo opened a new issue, #3151: idea: Move `root` out of service config to operator level?
Xuanwo opened a new issue, #3151:
URL: https://github.com/apache/incubator-opendal/issues/3151
OpenDAL now has root configuration for all services, but many users desire dynamic root access to perform actions such as `cd`. For instance, they would like to read data from `/root` and then switch to `/root/subdir`. Without this feature, our users have do their own workaround:
From databend
```rust
#[derive(Clone, Debug)]
pub struct DataOperator {
operator: Operator,
params: StorageParams,
}
impl StorageParams {
pub fn map_root(mut self, f: impl Fn(&str) -> String) -> Self {
match &mut self {
StorageParams::Azblob(v) => v.root = f(&v.root),
StorageParams::Fs(v) => v.root = f(&v.root),
StorageParams::Ftp(v) => v.root = f(&v.root),
StorageParams::Hdfs(v) => v.root = f(&v.root),
StorageParams::Http(_) => {}
StorageParams::Ipfs(v) => v.root = f(&v.root),
StorageParams::Memory => {}
StorageParams::Moka(_) => {}
StorageParams::Obs(v) => v.root = f(&v.root),
StorageParams::Oss(v) => v.root = f(&v.root),
StorageParams::S3(v) => v.root = f(&v.root),
StorageParams::Gcs(v) => v.root = f(&v.root),
StorageParams::Redis(v) => v.root = f(&v.root),
StorageParams::Webhdfs(v) => v.root = f(&v.root),
StorageParams::Cos(v) => v.root = f(&v.root),
StorageParams::None => {}
};
self
}
}
```
From icelake
```rust
#[derive(Debug, Clone)]
pub struct OperatorArgs {
scheme: Scheme,
args: HashMap<String, String>,
}
pub struct StorageCatalogConfig {
warehouse: String,
base_catalog_config: BaseCatalogConfig,
op_args: OperatorArgs,
}
fn table_operator(&self, table_path: &str) -> Result<Operator> {
Operator::try_from(&self.config.op_args.sub_dir(table_path)?)
}
```
Besides, all our services need to do the exactly same thing for strip the path:
```rust
let de = oio::Entry::new(&build_rel_path(&self.core.root, &object.key), meta);
```
Maybe we can move `root` from services config to operator instead, so that users can
- `op.root()` to fetch current root
- `let op = op.with_root("abc/")` switch to a new root.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] idea: Move `root` out of service config to operator level? [incubator-opendal]
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo closed issue #3151: idea: Move `root` out of service config to operator level?
URL: https://github.com/apache/incubator-opendal/issues/3151
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] idea: Move `root` out of service config to operator level? [incubator-opendal]
Posted by "damooo (via GitHub)" <gi...@apache.org>.
damooo commented on issue #3151:
URL: https://github.com/apache/incubator-opendal/issues/3151#issuecomment-1742979451
It may be also useful to share underlying connection pool (for http based), by cloning the same hyper client, as recommended by hyper for newly created sub root service.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org