You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by xu...@apache.org on 2023/04/24 15:00:14 UTC

[incubator-opendal] branch main updated: docs(refactor): Add more detailed description of operator, accessor, and builder (#2094)

This is an automated email from the ASF dual-hosted git repository.

xuanwo pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-opendal.git


The following commit(s) were added to refs/heads/main by this push:
     new f76b33cb docs(refactor): Add more detailed description of operator, accessor, and builder (#2094)
f76b33cb is described below

commit f76b33cb8275f891938a49ce918789d20efcd9f8
Author: Yansongsongsong <ya...@gmail.com>
AuthorDate: Mon Apr 24 23:00:10 2023 +0800

    docs(refactor): Add more detailed description of operator, accessor, and builder (#2094)
    
    * docs: fix some typos
    
    * docs: fix typo
    
    * docs: add more descriptions for builder API
    
    * docs: fix commet typo of memory builder
    
    * docs: fix typo of Accessor
    
    * docs: refactor operator accessor and builder related doc
    
    * fix: revert wrong fix
    
    * docs: fix incorrect description
    
    * docs: fix diagram representation
    
    * docs: fix unproper description of builder
    
    * docs: fix unproper decription of builder
    
    * docs: remove inappropriate analogy
---
 core/src/docs/concepts.rs           | 33 ++++++++++++++++++++++-----------
 core/src/docs/internals/accessor.rs |  2 +-
 core/src/raw/accessor.rs            | 13 +++++++++++--
 core/src/raw/adapters/kv/backend.rs |  8 +++++++-
 core/src/services/memory/backend.rs |  2 +-
 core/src/types/builder.rs           | 20 ++++++++++++++++----
 core/src/types/operator/operator.rs |  8 +++++++-
 7 files changed, 65 insertions(+), 21 deletions(-)

diff --git a/core/src/docs/concepts.rs b/core/src/docs/concepts.rs
index aa01e6e9..db4f06db 100644
--- a/core/src/docs/concepts.rs
+++ b/core/src/docs/concepts.rs
@@ -17,12 +17,15 @@
 
 //! The core concepts of OpenDAL's public API.
 //!
-//! OpenDAL provides a unified abstraction for all storage services.
+//! OpenDAL provides a unified abstraction that helps developers access all storage services.
 //!
 //! There are two core concepts in OpenDAL:
 //!
-//! - [`Builder`]: Build an instance of underlying services.
-//! - [`Operator`]: A bridge between underlying implementation detail and unified abstraction.
+//! - [`Builder`]: Builder accepts a series of parameters to set up an instance of underlying services.
+//! You can adjust the behaviour of underlying services with these parameters.
+//! - [`Operator`]: Developer can access underlying storage services with manipulating one Operator.
+//! The Operator is a delegate for underlying implementation detail, and provides one unified access interface,
+//! including `read`, `write`, `list` and so on.
 //!
 //! If you are interested in internal implementation details, please have a look at [`internals`][super::internals].
 //!
@@ -30,7 +33,9 @@
 //!
 //! Let's start with [`Builder`].
 //!
-//! A `Builder` is a trait that is implemented by the underlying services. We can use a `Builder` to configure and create a service. Builder is the only public API provided by services, and the detailed implementation is hidden.
+//! A `Builder` is a trait that is implemented by the underlying services. We can use a `Builder` to configure and create a service.
+//! Developer can only create one service via Builder, in other words, Builder is the only public API provided by services.
+//! And other detailed implementation will be hidden.
 //!
 //! ```text
 //! ┌───────────┐                 ┌───────────┐
@@ -41,6 +46,8 @@
 //! ```
 //!
 //! All [`Builder`] provided by OpenDAL is under [`services`][crate::services], we can refer to them like `opendal::services::S3`.
+//! By right the builder will be named like `OneServiceBuilder`, but usually we will export it to public with renaming it as one
+//! general name. For example, we will rename `S3Builder` to `S3` and developer will use `S3` finally.
 //!
 //! For example:
 //!
@@ -53,15 +60,19 @@
 //! ```
 //!
 //! # Operator
-//!
-//! The [`Operator`] is a bridge between the underlying implementation details and the unified abstraction. OpenDAL will erase all generic types and higher abstraction around it.
+//! The [`Operator`] is a delegate for Service, the underlying implementation detail that implements [`Accessor`][crate::raw::Accessor],
+//! and it also provides one unified access interface.
+//! It will hold one reference of Service with its all generic types erased by OpenDAL,
+//! which is the reason why we say the Operator is the delegate of one Service.
 //!
 //! ```text
-//! ┌───────────┐           ┌───────────┐              ┌─────────────────┐
-//! │           │  build()  │           │  type erase  │                 │
-//! │  Builder  ├──────────►│  Service  ├─────────────►│     Operator    │
-//! │           │           │           │              │                 │
-//! └───────────┘           └───────────┘              └─────────────────┘
+//!                   ┌────────────────────┐
+//!                   │      Operator      │
+//!                   │         │delegate  │
+//! ┌─────────┐ build │         ▼          │ rely on ┌─────────────────────┐
+//! │ Builder ├───────┼──►┌────────────┐   │◄────────┤ business logic code │
+//! └─────────┘       │   │  Service   │   │         └─────────────────────┘
+//!                   └───┴────────────┴───┘
 //! ```
 //!
 //! `Operator` can be built from `Builder`:
diff --git a/core/src/docs/internals/accessor.rs b/core/src/docs/internals/accessor.rs
index 463ac5c8..d149457e 100644
--- a/core/src/docs/internals/accessor.rs
+++ b/core/src/docs/internals/accessor.rs
@@ -148,7 +148,7 @@
 //!
 //! First of all, let's pick a good [`Scheme`] for our duck service. The
 //! scheme should be unique and easy to understand. Normally we should
-//! use it's formal name.
+//! use its formal name.
 //!
 //! For example, we will use `s3` for AWS S3 Compatible Storage Service
 //! instead of `aws` or `awss3`. This is because there are many storage
diff --git a/core/src/raw/accessor.rs b/core/src/raw/accessor.rs
index d551fda0..ee6e7618 100644
--- a/core/src/raw/accessor.rs
+++ b/core/src/raw/accessor.rs
@@ -26,6 +26,16 @@ use crate::*;
 
 /// Underlying trait of all backends for implementors.
 ///
+/// The actual data access of storage service happens in Accessor layer.
+/// Every storage supported by OpenDAL must implement [`Accessor`] but not all
+/// methods of [`Accessor`] will be implemented according to how the storage service is.
+///
+/// For example, user can not modify the content from one HTTP file server directly.
+/// So [`Http`][crate::services::Http] implements and provides only read related actions.
+///
+/// [`Accessor`] gives default implementation for all methods which will raise [`ErrorKind::Unsupported`] error.
+/// And what action this [`Accessor`] supports will be pointed out in [`AccessorInfo`].
+///
 /// # Note
 ///
 /// Visit [`internals`][crate::docs::internals] for more tutorials.
@@ -73,7 +83,6 @@ pub trait Accessor: Send + Sync + Debug + Unpin + 'static {
     ///
     /// - scheme: declare the scheme of backend.
     /// - capabilities: declare the capabilities of current backend.
-    /// - hints: declare the hints of current backend
     fn info(&self) -> AccessorInfo;
 
     /// Invoke the `create` operation on the specified path
@@ -487,7 +496,7 @@ impl<T: Accessor + ?Sized> Accessor for Arc<T> {
     }
 }
 
-/// FusedAccessor is the type erased accessor with `Box<dyn Read>`.
+/// FusedAccessor is the type erased accessor with `Arc<dyn Accessor>`.
 pub type FusedAccessor = Arc<
     dyn Accessor<
         Reader = oio::Reader,
diff --git a/core/src/raw/adapters/kv/backend.rs b/core/src/raw/adapters/kv/backend.rs
index 855b9475..55273102 100644
--- a/core/src/raw/adapters/kv/backend.rs
+++ b/core/src/raw/adapters/kv/backend.rs
@@ -25,7 +25,13 @@ use crate::ops::*;
 use crate::raw::*;
 use crate::*;
 
-/// Backend of kv service.
+/// Backend of kv service. If the storage service is one k-v-like service, it should implement this kv [`Backend`] by right.
+///
+/// `Backend` implements one general logic on how to read, write, scan the data from one kv store efficiently.
+/// And the [`Adapter`] held by `Backend` will handle how to communicate with one k-v-like service really and provides
+/// a series of basic operation for this service.
+///
+/// OpenDAL developer can implement one new k-v store backend easily with help of this Backend.
 #[derive(Debug, Clone)]
 pub struct Backend<S: Adapter> {
     kv: Arc<S>,
diff --git a/core/src/services/memory/backend.rs b/core/src/services/memory/backend.rs
index f34b2b0d..34c77350 100644
--- a/core/src/services/memory/backend.rs
+++ b/core/src/services/memory/backend.rs
@@ -43,7 +43,7 @@ pub struct MemoryBuilder {
 }
 
 impl MemoryBuilder {
-    /// Set the root for dashmap.
+    /// Set the root for BTreeMap.
     pub fn root(&mut self, path: &str) -> &mut Self {
         self.root = Some(path.into());
         self
diff --git a/core/src/types/builder.rs b/core/src/types/builder.rs
index 252ce9cc..237c103c 100644
--- a/core/src/types/builder.rs
+++ b/core/src/types/builder.rs
@@ -21,16 +21,28 @@ use std::env;
 use crate::raw::*;
 use crate::*;
 
-/// Builder is used to build a storage accessor used by [`Operator`].
+/// Builder is used to set up a real underlying service, i.e. storage accessor.
+///
+/// One builder is usually used by [`Operator`] during its initialization.
+/// It can be created by accepting several k-v pairs from one HashMap, one iterator and specific environment variables.
+///
+/// By default each builder of underlying service must support deriving from one HashMap.
+/// Besides that, according to the implementation, each builder will have its own special methods
+/// to control the behavior of initialization of the underlying service.
+/// It often provides semantic interface instead of using dynamic k-v strings directly.
+/// Because the latter way is obscure and hard to remember how many parameters it will have.
+///
+/// So it is recommended that developer should read related doc of builder carefully when you are working with one service.
+/// We also promise that every public API will provide detailed documentation.
 ///
 /// It's recommended to use [`Operator::new`] to avoid use `Builder` trait directly.
 pub trait Builder: Default {
-    /// Associated scheme for this builder.
+    /// Associated scheme for this builder. It indicates what underlying service is.
     const SCHEME: Scheme;
     /// The accessor that built by this builder.
     type Accessor: Accessor;
 
-    /// Construct a builder from given map.
+    /// Construct a builder from given map which contains several parameters needed by underlying service.
     fn from_map(map: HashMap<String, String>) -> Self;
 
     /// Construct a builder from given iterator.
@@ -41,7 +53,7 @@ pub trait Builder: Default {
         Self::from_map(iter.collect())
     }
 
-    /// Construct a builder from envs.
+    /// Construct a builder from envs. Please note that the env should begin with `opendal_{schema_name}_`.
     fn from_env() -> Self
     where
         Self: Sized,
diff --git a/core/src/types/operator/operator.rs b/core/src/types/operator/operator.rs
index 8187bdb1..8fdd9bf7 100644
--- a/core/src/types/operator/operator.rs
+++ b/core/src/types/operator/operator.rs
@@ -33,7 +33,10 @@ use crate::raw::*;
 use crate::*;
 
 /// Operator is the entry for all public async APIs.
+/// Developer should manipulate the data from storage service through Operator only by right.
 ///
+/// We will usually do some general checks and data transformations in this layer,
+/// like normalizing path from input, checking whether the path refers to one file or one directory, and so on.
 /// Read [`concepts`][docs::concepts] for know more about [`Operator`].
 ///
 /// # Examples
@@ -61,8 +64,10 @@ use crate::*;
 /// ```
 #[derive(Clone, Debug)]
 pub struct Operator {
+    // accessor is what Operator delegates for
     accessor: FusedAccessor,
 
+    // limit is usually the maximum size of data that operator will handle in one operation
     limit: usize,
 }
 
@@ -85,7 +90,8 @@ impl Operator {
         self.accessor
     }
 
-    /// Get current operator's limit
+    /// Get current operator's limit.
+    /// Limit is usually the maximum size of data that operator will handle in one operation.
     pub fn limit(&self) -> usize {
         self.limit
     }