You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by "PsiACE (via GitHub)" <gi...@apache.org> on 2023/05/07 15:53:07 UTC
[GitHub] [incubator-opendal] PsiACE opened a new pull request, #2225: feat: migrate service dashmap
PsiACE opened a new pull request, #2225:
URL: https://github.com/apache/incubator-opendal/pull/2225
- [feat(typed_kv): add scan support for Typed KV Adapter](https://github.com/apache/incubator-opendal/commit/d45acbc8c72c497b3a8895e6b90198c4de75fba5)
- [feat(services/dashmap): migrate service dashmap](https://github.com/apache/incubator-opendal/commit/5e3e40ec969206c3832f02575bcb6a001323cb60)
Related: #2223
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] PsiACE commented on a diff in pull request #2225: feat: migrate service dashmap
Posted by "PsiACE (via GitHub)" <gi...@apache.org>.
PsiACE commented on code in PR #2225:
URL: https://github.com/apache/incubator-opendal/pull/2225#discussion_r1186878819
##########
core/src/services/dashmap/backend.rs:
##########
@@ -70,45 +70,36 @@ impl Builder for DashmapBuilder {
}
/// Backend is used to serve `Accessor` support in dashmap.
-pub type DashmapBackend = kv::Backend<Adapter>;
+pub type DashmapBackend = typed_kv::Backend<Adapter>;
#[derive(Debug, Clone)]
pub struct Adapter {
- inner: DashMap<String, Vec<u8>>,
+ inner: DashMap<String, typed_kv::Value>,
}
#[async_trait]
-impl kv::Adapter for Adapter {
- fn metadata(&self) -> kv::Metadata {
- kv::Metadata::new(
- Scheme::Dashmap,
- &format!("{:?}", &self.inner as *const _),
- Capability {
- read: true,
- write: true,
- scan: true,
- ..Default::default()
- },
- )
+impl typed_kv::Adapter for Adapter {
+ fn metadata(&self) -> (Scheme, String) {
Review Comment:
I note that https://github.com/apache/incubator-opendal/pull/2215 seems to be relevant.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] PsiACE commented on a diff in pull request #2225: feat: migrate service dashmap
Posted by "PsiACE (via GitHub)" <gi...@apache.org>.
PsiACE commented on code in PR #2225:
URL: https://github.com/apache/incubator-opendal/pull/2225#discussion_r1186879361
##########
core/src/services/dashmap/backend.rs:
##########
@@ -70,45 +70,36 @@ impl Builder for DashmapBuilder {
}
/// Backend is used to serve `Accessor` support in dashmap.
-pub type DashmapBackend = kv::Backend<Adapter>;
+pub type DashmapBackend = typed_kv::Backend<Adapter>;
#[derive(Debug, Clone)]
pub struct Adapter {
- inner: DashMap<String, Vec<u8>>,
+ inner: DashMap<String, typed_kv::Value>,
}
#[async_trait]
-impl kv::Adapter for Adapter {
- fn metadata(&self) -> kv::Metadata {
- kv::Metadata::new(
- Scheme::Dashmap,
- &format!("{:?}", &self.inner as *const _),
- Capability {
- read: true,
- write: true,
- scan: true,
- ..Default::default()
- },
- )
+impl typed_kv::Adapter for Adapter {
+ fn metadata(&self) -> (Scheme, String) {
Review Comment:
Got it and I'll add it in some time later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] Xuanwo commented on a diff in pull request #2225: feat: migrate service dashmap
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #2225:
URL: https://github.com/apache/incubator-opendal/pull/2225#discussion_r1186886060
##########
core/src/raw/adapters/typed_kv/api.rs:
##########
@@ -87,3 +117,47 @@ impl Value {
size_of::<Metadata>() + self.value.len()
}
}
+
+/// Info for this key value accessor.
+pub struct Info {
+ scheme: Scheme,
+ name: String,
+ capabilities: Capability,
+}
+
+impl Info {
+ /// Create a new KeyValueAccessorInfo.
+ pub fn new(scheme: Scheme, name: &str, capabilities: Capability) -> Self {
+ Self {
+ scheme,
+ name: name.to_string(),
+ capabilities,
+ }
+ }
+
+ /// Get the scheme.
+ pub fn scheme(&self) -> Scheme {
+ self.scheme
+ }
+
+ /// Get the name.
+ pub fn name(&self) -> &str {
+ &self.name
+ }
+
+ /// Get the capabilities.
+ pub fn capabilities(&self) -> Capability {
+ self.capabilities
+ }
+}
+
+impl From<Info> for AccessorInfo {
Review Comment:
I prefer to implment it in `Accessor::info()` directly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] Xuanwo commented on a diff in pull request #2225: feat: migrate service dashmap
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #2225:
URL: https://github.com/apache/incubator-opendal/pull/2225#discussion_r1186879133
##########
core/src/services/dashmap/backend.rs:
##########
@@ -70,45 +70,36 @@ impl Builder for DashmapBuilder {
}
/// Backend is used to serve `Accessor` support in dashmap.
-pub type DashmapBackend = kv::Backend<Adapter>;
+pub type DashmapBackend = typed_kv::Backend<Adapter>;
#[derive(Debug, Clone)]
pub struct Adapter {
- inner: DashMap<String, Vec<u8>>,
+ inner: DashMap<String, typed_kv::Value>,
}
#[async_trait]
-impl kv::Adapter for Adapter {
- fn metadata(&self) -> kv::Metadata {
- kv::Metadata::new(
- Scheme::Dashmap,
- &format!("{:?}", &self.inner as *const _),
- Capability {
- read: true,
- write: true,
- scan: true,
- ..Default::default()
- },
- )
+impl typed_kv::Adapter for Adapter {
+ fn metadata(&self) -> (Scheme, String) {
Review Comment:
> Perhaps the capability check will be reintroduced in a later PR.
No, they are different things. `typed_kv` must provide it's own Capability so that `typed_kv::Backend` can implement `Accessor::info()` correctly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] Xuanwo commented on a diff in pull request #2225: feat: migrate service dashmap
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #2225:
URL: https://github.com/apache/incubator-opendal/pull/2225#discussion_r1186884988
##########
core/src/raw/adapters/typed_kv/api.rs:
##########
@@ -110,3 +117,47 @@ impl Value {
size_of::<Metadata>() + self.value.len()
}
}
+
+/// Info for this key value accessor.
+pub struct Info {
+ scheme: Scheme,
+ name: String,
+ capabilities: Capability,
Review Comment:
> We should define a `typed_kv::Capability` to simplify.
Because all typed_kv will share the same backend, so we only need know if adapter supports `get`/`set`/`delete`/`scan`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] Xuanwo commented on a diff in pull request #2225: feat: migrate service dashmap
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #2225:
URL: https://github.com/apache/incubator-opendal/pull/2225#discussion_r1186876550
##########
core/src/services/dashmap/backend.rs:
##########
@@ -70,45 +70,36 @@ impl Builder for DashmapBuilder {
}
/// Backend is used to serve `Accessor` support in dashmap.
-pub type DashmapBackend = kv::Backend<Adapter>;
+pub type DashmapBackend = typed_kv::Backend<Adapter>;
#[derive(Debug, Clone)]
pub struct Adapter {
- inner: DashMap<String, Vec<u8>>,
+ inner: DashMap<String, typed_kv::Value>,
}
#[async_trait]
-impl kv::Adapter for Adapter {
- fn metadata(&self) -> kv::Metadata {
- kv::Metadata::new(
- Scheme::Dashmap,
- &format!("{:?}", &self.inner as *const _),
- Capability {
- read: true,
- write: true,
- scan: true,
- ..Default::default()
- },
- )
+impl typed_kv::Adapter for Adapter {
+ fn metadata(&self) -> (Scheme, String) {
Review Comment:
We need to return a `typed_kv::Metadata` here in which carries a `typed_kv::Capability {get: true, set: true, scan: true}`.
Maybe changed into `info() -> typed_kv::Info` to align with `Accessor`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] Xuanwo commented on a diff in pull request #2225: feat: migrate service dashmap
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #2225:
URL: https://github.com/apache/incubator-opendal/pull/2225#discussion_r1186879375
##########
core/src/services/moka/backend.rs:
##########
@@ -232,4 +232,17 @@ impl typed_kv::Adapter for Adapter {
Ok(())
}
+
+ async fn scan(&self, path: &str) -> Result<Vec<String>> {
Review Comment:
Moka should not implement `scan` since it's key could be expired at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] Xuanwo commented on a diff in pull request #2225: feat: migrate service dashmap
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #2225:
URL: https://github.com/apache/incubator-opendal/pull/2225#discussion_r1186884737
##########
core/src/raw/adapters/typed_kv/api.rs:
##########
@@ -110,3 +117,47 @@ impl Value {
size_of::<Metadata>() + self.value.len()
}
}
+
+/// Info for this key value accessor.
+pub struct Info {
+ scheme: Scheme,
+ name: String,
+ capabilities: Capability,
Review Comment:
We should define a `typed_kv::Capability` to simplify.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] Xuanwo merged pull request #2225: feat: migrate service dashmap
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo merged PR #2225:
URL: https://github.com/apache/incubator-opendal/pull/2225
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] Xuanwo commented on a diff in pull request #2225: feat: migrate service dashmap
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #2225:
URL: https://github.com/apache/incubator-opendal/pull/2225#discussion_r1186878996
##########
core/src/services/dashmap/backend.rs:
##########
@@ -70,45 +70,36 @@ impl Builder for DashmapBuilder {
}
/// Backend is used to serve `Accessor` support in dashmap.
-pub type DashmapBackend = kv::Backend<Adapter>;
+pub type DashmapBackend = typed_kv::Backend<Adapter>;
#[derive(Debug, Clone)]
pub struct Adapter {
- inner: DashMap<String, Vec<u8>>,
+ inner: DashMap<String, typed_kv::Value>,
}
#[async_trait]
-impl kv::Adapter for Adapter {
- fn metadata(&self) -> kv::Metadata {
- kv::Metadata::new(
- Scheme::Dashmap,
- &format!("{:?}", &self.inner as *const _),
- Capability {
- read: true,
- write: true,
- scan: true,
- ..Default::default()
- },
- )
+impl typed_kv::Adapter for Adapter {
+ fn metadata(&self) -> (Scheme, String) {
Review Comment:
> I note that #2215 seems to be relevant.
Not related. Please ignore it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] Xuanwo commented on a diff in pull request #2225: feat: migrate service dashmap
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #2225:
URL: https://github.com/apache/incubator-opendal/pull/2225#discussion_r1186884879
##########
core/src/raw/adapters/typed_kv/api.rs:
##########
@@ -110,3 +117,47 @@ impl Value {
size_of::<Metadata>() + self.value.len()
}
}
+
+/// Info for this key value accessor.
+pub struct Info {
+ scheme: Scheme,
+ name: String,
+ capabilities: Capability,
Review Comment:
And we can make all fields `pub` so that we don't need to add getter/setter functions for it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] Xuanwo commented on a diff in pull request #2225: feat: migrate service dashmap
Posted by "Xuanwo (via GitHub)" <gi...@apache.org>.
Xuanwo commented on code in PR #2225:
URL: https://github.com/apache/incubator-opendal/pull/2225#discussion_r1186876550
##########
core/src/services/dashmap/backend.rs:
##########
@@ -70,45 +70,36 @@ impl Builder for DashmapBuilder {
}
/// Backend is used to serve `Accessor` support in dashmap.
-pub type DashmapBackend = kv::Backend<Adapter>;
+pub type DashmapBackend = typed_kv::Backend<Adapter>;
#[derive(Debug, Clone)]
pub struct Adapter {
- inner: DashMap<String, Vec<u8>>,
+ inner: DashMap<String, typed_kv::Value>,
}
#[async_trait]
-impl kv::Adapter for Adapter {
- fn metadata(&self) -> kv::Metadata {
- kv::Metadata::new(
- Scheme::Dashmap,
- &format!("{:?}", &self.inner as *const _),
- Capability {
- read: true,
- write: true,
- scan: true,
- ..Default::default()
- },
- )
+impl typed_kv::Adapter for Adapter {
+ fn metadata(&self) -> (Scheme, String) {
Review Comment:
We need to return a `typed_kv::Metadata` here in which carries a `typed_kv::Capability {get: true, set: true, scan: true}`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-opendal] PsiACE commented on a diff in pull request #2225: feat: migrate service dashmap
Posted by "PsiACE (via GitHub)" <gi...@apache.org>.
PsiACE commented on code in PR #2225:
URL: https://github.com/apache/incubator-opendal/pull/2225#discussion_r1186878585
##########
core/src/services/dashmap/backend.rs:
##########
@@ -70,45 +70,36 @@ impl Builder for DashmapBuilder {
}
/// Backend is used to serve `Accessor` support in dashmap.
-pub type DashmapBackend = kv::Backend<Adapter>;
+pub type DashmapBackend = typed_kv::Backend<Adapter>;
#[derive(Debug, Clone)]
pub struct Adapter {
- inner: DashMap<String, Vec<u8>>,
+ inner: DashMap<String, typed_kv::Value>,
}
#[async_trait]
-impl kv::Adapter for Adapter {
- fn metadata(&self) -> kv::Metadata {
- kv::Metadata::new(
- Scheme::Dashmap,
- &format!("{:?}", &self.inner as *const _),
- Capability {
- read: true,
- write: true,
- scan: true,
- ..Default::default()
- },
- )
+impl typed_kv::Adapter for Adapter {
+ fn metadata(&self) -> (Scheme, String) {
Review Comment:
Perhaps the capability check will be reintroduced in a later PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org