You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by xu...@apache.org on 2026/08/28 05:53:33 UTC
(opendal) branch xuanwo/specifications created (now 65c19c7a26)
This is an automated email from the ASF dual-hosted git repository.
Xuanwo pushed a change to branch xuanwo/specifications
in repository https://gitbox.apache.org/repos/asf/opendal.git
at 65c19c7a26 Merge remote-tracking branch 'origin/main' into xuanwo/specifications
This branch includes the following new commits:
new 341b2c6a49 docs: establish OpenDAL specifications
new 65c19c7a26 Merge remote-tracking branch 'origin/main' into xuanwo/specifications
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
(opendal) 01/02: docs: establish OpenDAL specifications
Posted by xu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
Xuanwo pushed a commit to branch xuanwo/specifications
in repository https://gitbox.apache.org/repos/asf/opendal.git
commit 341b2c6a499350b5f7003562d36d73e991b2b90a
Author: Xuanwo <gi...@xuanwo.io>
AuthorDate: Fri Aug 28 13:50:33 2026 +0800
docs: establish OpenDAL specifications
---
AGENTS.md | 35 +
core/core/src/docs/concepts.rs | 164 ++---
core/core/src/docs/internals/accessor.rs | 355 ++--------
core/core/src/docs/internals/layer.rs | 102 ++-
core/core/src/docs/internals/mod.rs | 44 +-
core/core/src/docs/mod.rs | 16 +-
core/core/src/docs/performance/concurrent_write.md | 147 +++--
.../core/src/docs/performance/http_optimization.md | 224 ++++---
core/core/src/docs/performance/mod.rs | 14 +-
core/core/src/docs/rfcs/README.md | 17 +-
core/core/src/docs/specs/README.md | 32 +
core/core/src/docs/specs/conditional-operations.md | 152 +++++
core/core/src/docs/{performance => specs}/mod.rs | 18 +-
core/core/src/lib.rs | 1 +
core/core/src/types/capability.rs | 2 +
core/core/src/types/delete/input.rs | 48 +-
core/core/src/types/error.rs | 19 +-
core/core/src/types/operator/operator.rs | 25 +-
core/core/src/types/operator/operator_futures.rs | 288 ++++----
core/core/src/types/options.rs | 724 ++++++++++++++-------
website/docs/01-overview.md | 5 +-
website/docs/03-concepts.mdx | 70 +-
website/docs/10-core/01-overview.md | 15 +-
website/docs/10-core/04-tasks.md | 11 +-
website/docs/20-bindings/c/01-overview.md | 3 +-
website/docs/20-bindings/cpp/01-overview.md | 3 +-
website/docs/20-bindings/d/01-overview.md | 3 +-
website/docs/20-bindings/dart/01-overview.md | 3 +-
website/docs/20-bindings/dotnet/01-overview.md | 3 +-
website/docs/20-bindings/go/01-overview.md | 3 +-
website/docs/20-bindings/haskell/01-overview.md | 3 +-
website/docs/20-bindings/java/01-overview.md | 3 +-
website/docs/20-bindings/lua/01-overview.md | 3 +-
website/docs/20-bindings/nodejs/01-overview.md | 3 +-
website/docs/20-bindings/ocaml/01-overview.md | 3 +-
website/docs/20-bindings/php/01-overview.md | 3 +-
website/docs/20-bindings/python/01-overview.md | 3 +-
website/docs/20-bindings/ruby/01-overview.md | 3 +-
website/docs/20-bindings/swift/01-overview.md | 3 +-
website/docs/20-bindings/zig/01-overview.md | 3 +-
website/docs/sidebars.js | 5 +
website/docusaurus.config.js | 13 +
website/specifications.sidebars.js | 38 ++
website/src/components/concepts/MentalModel.jsx | 18 +-
website/src/components/concepts/samples.js | 5 +-
45 files changed, 1527 insertions(+), 1128 deletions(-)
diff --git a/AGENTS.md b/AGENTS.md
index 7ac8d79e7e..d310c03dde 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -59,6 +59,41 @@ The human contributor remains responsible for every submitted claim and change.
- Keep terminology consistent with the codebase, especially `service`, `layer`, `operator`, `storage` and `operation`.
- Use parallel structure in lists and punctuate complete sentences consistently.
+## Documentation Architecture
+
+OpenDAL splits documentation by audience. The website carries only content
+that holds for OpenDAL across languages; Rust-specific documentation lives in
+rustdoc.
+
+- `core/core/src/docs/specs/` contains the living portable contracts, the one
+ set of documents published to both destinations. Rustdoc publishes them
+ under `opendal::docs::specs`; the website reads the same Markdown at
+ `/docs/specifications/`.
+- `core/core/src/docs/concepts.rs`, `core/core/src/docs/internals/`, and
+ `core/core/src/docs/performance/` contain the Rust core guides: Rust types,
+ traits, implementation architecture, runtime resources, and transport
+ tuning. They are rustdoc-only; do not publish them on the website.
+- `website/docs/` contains cross-language concepts and non-normative guidance,
+ plus the per-language guides. Content there must hold for OpenDAL as a
+ whole. Do not present Rust implementation detail as cross-language behavior,
+ and do not claim every binding exposes a feature (layers, builders,
+ concurrency controls) unless the bindings demonstrate it.
+- `core/core/src/docs/rfcs/` contains immutable accepted design history. An
+ accepted RFC records the proposal that reached consensus; update a
+ specification, not the accepted RFC, when the current contract changes.
+- `core/core/src/docs/upgrade.md` contains upgrade procedures, and
+ `CHANGELOG.md` contains the release changelog. Rustdoc exposes them through
+ `opendal::docs`.
+- `website/docusaurus.config.js` and `website/specifications.sidebars.js`
+ define website publication and navigation for the specifications. Do not add
+ a website section that republishes rustdoc content, and do not copy
+ specifications into `website/docs/`.
+
+Public API documentation under `core/core/src/types/` must explain the
+observable behavior, capability requirements, and relevant errors at the API
+site. Link to a specification only after the local API contract is
+self-contained.
+
## Rust Workspace Commands
The Rust workspace for OpenDAL core lives under `core/`. There is no root `Cargo.toml`; run core cargo commands from `core/`.
diff --git a/core/core/src/docs/concepts.rs b/core/core/src/docs/concepts.rs
index 29383df770..423acabd3c 100644
--- a/core/core/src/docs/concepts.rs
+++ b/core/core/src/docs/concepts.rs
@@ -15,101 +15,104 @@
// specific language governing permissions and limitations
// under the License.
-//! The core concepts of OpenDAL's public API.
+//! The core concepts of OpenDAL's Rust API.
//!
-//! OpenDAL provides a unified abstraction for accessing storage services.
+//! OpenDAL gives applications one storage API across services. Rust
+//! applications use four concepts: a service describes the storage backend, a
+//! builder configures it, an operator exposes operations, and layers add
+//! behavior around those operations.
//!
-//! Public users mostly work with three concepts:
+//! For the language-independent model shared by every OpenDAL binding, see the
+//! [OpenDAL concepts guide](https://opendal.apache.org/docs/concepts/).
//!
-//! - [`Builder`]: configures and builds a storage service.
-//! - [`Operator`]: owns a service stack and exposes storage operations such as
-//! `read`, `write`, and `list`.
-//! - [`Layer`][crate::raw::Layer]: wraps an operator's service stack or runtime
-//! resources to add behavior such as retry, timeout, tracing, and metrics.
+//! # Service and builder
//!
-//! If you are interested in internal implementation details, please have a look at [`internals`][super::internals].
+//! A **service** is a storage backend such as S3, Google Cloud Storage, a
+//! local filesystem, or an in-memory store. Each service supports a different
+//! set of operations and optional features.
//!
-//! # Builder
-//!
-//! A [`Builder`] configures and creates the underlying storage service. Service
-//! crates expose builders through [`services`][crate::services], for example
-//! [`services::Memory`][crate::services::Memory].
+//! A [`Builder`] collects one service's configuration and constructs its
+//! implementation. The [`services`][crate::services] module exposes concrete
+//! builders, while [`Operator::new`] turns a builder into a ready-to-use
+//! operator.
//!
//! ```text
-//! ┌─────────┐ build() ┌─────────┐
-//! │ Builder ├────────────►│ Service │
-//! └─────────┘ └─────────┘
+//! configuration -> builder -> service -> operator
//! ```
//!
//! ```no_run
+//! # use opendal_core::Result;
+//! use opendal_core::Operator;
//! use opendal_core::services::Memory;
//!
+//! # fn test() -> Result<()> {
//! let builder = Memory::default();
+//! let op = Operator::new(builder)?;
+//! # let _ = op;
+//! # Ok(())
+//! # }
//! ```
//!
+//! OpenDAL does not make every service support every operation. Applications
+//! can inspect the operator's effective [`Capability`] before using an
+//! optional operation or option.
+//!
//! # Operator
//!
-//! An [`Operator`] is the public handle for a storage service stack. It stores:
+//! An [`Operator`] is the public handle for one configured service and root.
+//! It normalizes paths, validates options against effective capabilities, runs
+//! the configured layers, and dispatches each operation to the service.
//!
-//! - a base service created from the builder.
-//! - a base [`OperationContext`] with runtime resources such as HTTP transport
-//! and executor.
-//! - an ordered list of layers.
-//! - the composed service and context used by storage operations.
+//! Operators are cheap to clone, contain no caller-visible lifetime or service
+//! type parameter, and can be shared across threads. A clone refers to the
+//! same composed service stack. Methods that add a layer or replace runtime
+//! resources return a new operator; existing clones and in-flight operations
+//! keep their current stack.
//!
-//! ```text
-//! ┌─────────┐ Operator::new ┌────────────────────────────┐
-//! │ Builder ├──────────────────►│ Operator │
-//! └─────────┘ │ - base service │
-//! │ - base OperationContext │
-//! │ - layers │
-//! │ - composed service/context │
-//! └────────────────────────────┘
-//! ```
+//! # Operation
//!
-//! `Operator::new` returns a ready-to-use operator. There is no separate
-//! `finish` step.
+//! Operations are storage actions such as `read`, `write`, `stat`, `list`,
+//! `delete`, `copy`, and `rename`. Convenience methods use default options,
+//! while the corresponding `_with` methods expose operation-specific options.
//!
//! ```no_run
//! # use opendal_core::Result;
-//! use opendal_core::services::Memory;
//! use opendal_core::Operator;
+//! use opendal_core::services::Memory;
//!
-//! # fn test() -> Result<()> {
-//! let builder = Memory::default();
-//!
-//! let op = Operator::new(builder)?;
+//! # async fn test() -> Result<()> {
+//! let op = Operator::new(Memory::default())?;
+//! let bs = op.read("abc").await?;
+//! # let _ = bs;
//! # Ok(())
//! # }
//! ```
//!
-//! - `Operator` is cheap to clone because it stores shared handles internally.
-//! - `Operator` has no generic parameters or lifetimes, so it is easy to pass
-//! through application code.
-//! - `Operator` is `Send` and `Sync`, so it can be shared across threads.
-//! - Methods that change layers or runtime resources return a new operator;
-//! existing clones and in-flight operations keep using their current composed
-//! service and context.
+//! OpenDAL normalizes every path relative to the operator's root. `/`
+//! represents the root, a trailing `/` represents a directory, and any other
+//! normalized path represents a file. Operation documentation defines the
+//! observable behavior and errors. [Specifications][super::specs] define
+//! portable contracts that span multiple operations and services.
+//!
+//! # Layer and operation context
//!
-//! # Runtime resources and layers
+//! A [`Layer`][crate::raw::Layer] adds cross-cutting behavior such as retry,
+//! timeout, tracing, or metrics. Layers form an ordered stack around a
+//! service.
//!
-//! [`OperationContext`] carries runtime resources from the operator to services,
-//! such as [`HttpTransporter`][crate::HttpTransporter] and [`Executor`][crate::Executor].
-//! Operation arguments such as ranges, versions, and concurrency limits stay in
-//! the `Op*` argument structs used by each operation.
+//! An [`OperationContext`] carries runtime resources such as the HTTP
+//! transport and executor from the operator to the service.
+//! Operation-specific values such as ranges, versions, conditions, and
+//! concurrency remain in that operation's options.
//!
-//! Use [`Operator::with_context`] to replace the base context. Use
-//! [`Operator::layer`] to append a layer. In both cases, OpenDAL replays the full
-//! layer list from the base service and base context to produce a fresh composed
-//! service and context.
+//! Adding a layer with [`Operator::layer`] or replacing the base context with
+//! [`Operator::with_context`] rebuilds both the service stack and the composed
+//! context from the same ordered layer list:
//!
//! ```text
-//! base service ─┐
-//! ├─ layer replay ─► composed service ─┐
-//! base context ─┘ ├─► operation dispatch
-//! │ srv.read(&ctx, ...)
-//! ▼
-//! composed context
+//! base service -----+ +-> composed service --+
+//! +-> ordered layers --+ +-> operation
+//! base context -----+ +-> composed context --+
//! ```
//!
//! ```no_run
@@ -129,40 +132,13 @@
//! # }
//! ```
//!
-//! # Operations
-//!
-//! After creating an operator, use it to run operations on normalized paths.
-//!
-//! ```text
-//! ┌──────────────┐
-//! ┌────────►│ read("abc") │
-//! │ └──────────────┘
-//! ┌───────────┐ │
-//! │ Operator │ │ ┌──────────────┐
-//! │ ┌───────┐ ├────┼────────►│ write("def") │
-//! │ │Service│ │ │ └──────────────┘
-//! └─┴───────┴─┘ │
-//! │ ┌──────────────┐
-//! └────────►│ list("ghi/") │
-//! └──────────────┘
-//! ```
-//!
-//! We can read data with given path in this way:
-//!
-//! ```no_run
-//! # use opendal_core::Result;
-//! use opendal_core::services::Memory;
-//! use opendal_core::Operator;
-//!
-//! # async fn test() -> Result<()> {
-//! let builder = Memory::default();
-//!
-//! let op = Operator::new(builder)?;
-//! let bs: Vec<u8> = op.read("abc").await?;
-//! # Ok(())
-//! # }
-//! ```
+//! Most applications only need the public [`Operator`] API. Service and layer
+//! authors should continue with the [internals][super::internals] guide.
//!
//! [`Builder`]: crate::Builder
//! [`Operator`]: crate::Operator
+//! [`Operator::new`]: crate::Operator::new
+//! [`Operator::layer`]: crate::Operator::layer
+//! [`Operator::with_context`]: crate::Operator::with_context
+//! [`Capability`]: crate::Capability
//! [`OperationContext`]: crate::OperationContext
diff --git a/core/core/src/docs/internals/accessor.rs b/core/core/src/docs/internals/accessor.rs
index 633e7dbe78..fe40374222 100644
--- a/core/core/src/docs/internals/accessor.rs
+++ b/core/core/src/docs/internals/accessor.rs
@@ -15,318 +15,87 @@
// specific language governing permissions and limitations
// under the License.
-//! The internal implementation details of [`Service`].
+//! Implementing a service.
//!
-//! [`Service`] is the core trait of OpenDAL's raw API. We operate
-//! underlying storage services via APIs provided by [`Service`].
+//! Every OpenDAL backend implements the raw [`Service`] trait. A service owns
+//! protocol-specific request construction and response handling. The operator
+//! owns path normalization, option validation, layer composition, and
+//! dispatch.
//!
-//! # Introduction
+//! Service crates live under `core/services/<name>/`. Their public builder and
+//! configuration types construct a typed backend; the backend then implements
+//! [`Service`].
//!
-//! [`Service`] can be split in the following parts:
+//! # Service identity and capabilities
//!
-//! ```ignore
-//! // <----------Trait Bound-------------->
-//! pub trait Service: Send + Sync + Debug + Unpin + 'static {
-//! fn info(&self) -> ServiceInfo;
-//! fn capability(&self) -> Capability;
+//! [`Service::info`] returns immutable identity such as the scheme, root, and
+//! namespace name. Runtime resources do not belong in [`ServiceInfo`];
+//! services read them from [`OperationContext`].
//!
-//! type Reader: oio::Read;
-//! type Writer: oio::Write;
-//! type Lister: oio::List;
-//! type Deleter: oio::Delete;
-//! type Copier: oio::Copy;
+//! [`Service::capability`] reports the behavior implemented by that service
+//! stack. Set an operation or option capability only when the implementation
+//! satisfies the corresponding public contract. Operators reject options whose
+//! required capability is absent.
//!
-//! fn read(
-//! &self,
-//! ctx: &OperationContext,
-//! path: &str,
-//! args: OpRead,
-//! ) -> impl Future<Output = Result<Self::Reader>> + MaybeSend;
-//! }
-//! ```
-//!
-//! Let's go deep into [`Service`] line by line.
-//!
-//! ## Trait Bound
-//!
-//! First we will read the declaration of [`Service`] trait:
-//!
-//! ```ignore
-//! pub trait Service: Send + Sync + Debug + Unpin + 'static {}
-//! ```
-//!
-//! There are many trait boundings here. For now, [`Service`] requires the following bound:
-//!
-//! - [`Send`]: Allow user to send between threads without extra wrapper.
-//! - [`Sync`]: Allow user to sync between threads without extra lock.
-//! - [`Debug`][std::fmt::Debug]: Allow users to print underlying debug information of service.
-//! - [`Unpin`]: Make sure `Service` can be safely moved after being pinned, so users don't need to `Pin<Box<A>>`.
-//! - `'static`: Make sure `Service` is not a short-time reference, allow users to use `Service` in closures and futures without playing with lifetime.
-//!
-//! Implementer of `Service` should take care of the following things:
-//!
-//! - Implement `Debug` for backend, but don't leak credentials.
-//! - Make sure the backend is `Send` and `Sync`, wrap the internal struct with `Arc<Mutex<T>>` if necessary.
-//!
-//! ## Associated Type
-//!
-//! The middle block of [`Service`] is our associated types. We require
-//! implementers to specify concrete operation body types, avoiding extra
-//! dynamic dispatch inside the backend and typed service wrappers.
-//!
-//! [`Service`] has five associated types so far:
-//!
-//! - `Reader`: reader returned by `read` operation.
-//! - `Writer`: writer returned by `write` operation.
-//! - `Lister`: lister returned by `list` operation.
-//! - `Deleter`: deleter returned by `delete` operation.
-//! - `Copier`: copier returned by `copy` operation.
-//!
-//! Backend implementers should take care of the following things:
-//!
-//! - Return concrete operation body types instead of dynamic trait objects like `oio::Reader`.
-//! - Use `()` as the type if the operation is not supported.
-//! - Implement every operation method; unsupported operations should return
-//! [`ErrorKind::Unsupported`] explicitly.
-//!
-//! Runtime service wrappers that already receive a [`Servicer`] can forward the
-//! erased `oio::*` body types, but the original backend should not erase itself.
-//!
-//! OpenDAL erases these associated types at the [`ServiceDyn`] boundary.
-//! [`Servicer`] is `Arc<dyn ServiceDyn>` and is the handle used by
-//! [`Operator`] and runtime layer composition. This keeps backend and typed
-//! layer implementations concrete, while the composed operator stack can be
-//! stored and cloned through one object-safe service handle.
-//!
-//! ## API Style
-//!
-//! Every API of [`Service`] follows the same style:
-//!
-//! - All APIs have a unique [`Operation`] and [`Capability`]
-//! - All APIs are orthogonal and do not overlap with each other
-//! - Most APIs accept [`OperationContext`], `path` and `OpXxx`, and returns `RpXxx` with concrete operation bodies.
-//! - Most APIs have `async` and `blocking` variants, they share the same semantics but may have different underlying implementations.
-//!
-//! [`OperationContext`] carries operator resources composed by operator layers,
-//! such as HTTP client and executor. Backend implementations should use this
-//! context instead of storing mutable operator resources in [`ServiceInfo`].
-//!
-//! [`Service`] declares immutable identity facts via [`ServiceInfo`] and
-//! operation capability via [`Service::capability`]:
-//!
-//! ```ignore
-//! impl Service for MyBackend {
-//! type Reader = MyReader;
-//! type Writer = ();
-//! type Lister = ();
-//! type Deleter = ();
-//! type Copier = ();
-//!
-//! fn info(&self) -> ServiceInfo {
-//! ServiceInfo::new(MY_SCHEME, "/", "")
-//! }
-//!
-//! fn capability(&self) -> Capability {
-//! Capability {
-//! read: true,
-//! write: true,
-//! ..Default::default()
-//! }
-//! }
-//! }
-//! ```
-//!
-//! Now that you have mastered [`Service`], let's go and implement our own backend!
-//!
-//! # Tutorial
+//! # Operation body types
//!
-//! This tutorial implements a `duck` storage service that sends API
-//! requests to a super-powered duck. Gagaga!
+//! [`Service`] uses associated types for operation bodies:
//!
-//! ## Scheme
-//!
-//! First of all, let's pick a good scheme for our duck service. The
-//! scheme should be unique and easy to understand. Normally we should
-//! use its formal name.
-//!
-//! For example, we will use `s3` for AWS S3 Compatible Storage Service
-//! instead of `aws` or `awss3`. This is because there are many storage
-//! vendors that provide s3-like RESTful APIs, and our s3 service is
-//! implemented to support all of them, not just AWS S3.
-//!
-//! Obviously, we can use `duck` as scheme, let's add a new constant string:
-//!
-//! ```ignore
-//! pub const DUCK_SCHEME: &str = "duck";
+//! ```text
+//! type Reader: oio::Read;
+//! type Writer: oio::Write;
+//! type Lister: oio::List;
+//! type Deleter: oio::Delete;
+//! type Copier: oio::Copy;
//! ```
//!
-//! ## Builder
+//! A backend returns concrete body types so its implementation and typed
+//! wrappers do not pay for dynamic dispatch. Use `()` for an unsupported body
+//! type and return [`ErrorKind::Unsupported`] from the corresponding operation
+//! entry point.
//!
-//! Then we can implement a builder for the duck service. The [`Builder`]
-//! will provide APIs for users to configure, and they will create an
-//! instance of a particular service.
+//! OpenDAL erases these types once, at [`ServiceDyn`]. [`Servicer`] is
+//! `Arc<dyn ServiceDyn>` and is the handle used by operators and runtime layer
+//! composition. A wrapper that receives a [`Servicer`] may forward erased
+//! `oio::*` bodies, but a backend should keep its own bodies concrete.
//!
-//! Let's create a `backend` mod under `services/duck` directory, and adding the following code.
+//! # Operation methods
//!
-//! ```ignore
-//! use crate::raw::*;
-//! use crate::*;
+//! Each operation method receives normalized paths, an [`OperationContext`],
+//! and operation-specific arguments. The context supplies layer-composed
+//! runtime resources such as the HTTP transport and executor. Options such as
+//! ranges, versions, conditions, and concurrency remain in the operation
+//! arguments.
//!
-//! /// Duck Storage Service support. Gagaga!
-//! ///
-//! /// # Capabilities
-//! ///
-//! /// This service can be used to:
-//! ///
-//! /// - [x] read
-//! /// - [ ] write
-//! /// - [ ] list
-//! /// - [ ] presign
-//! ///
-//! /// # Configuration
-//! ///
-//! /// - `root`: Set the work dir for backend.
-//! ///
-//! /// ## Via Builder
-//! ///
-//! /// ```no_run
-//! /// use std::sync::Arc;
-//! ///
-//! /// use anyhow::Result;
-//! /// use opendal_core::services::Duck;
-//! /// use opendal_core::Operator;
-//! ///
-//! /// #[tokio::main]
-//! /// async fn main() -> Result<()> {
-//! /// // Create Duck backend builder.
-//! /// let mut builder = DuckBuilder::default();
-//! /// // Set the root for duck, all operations will happen under this root.
-//! /// //
-//! /// // NOTE: the root must be absolute path.
-//! /// builder.root("/path/to/dir");
-//! ///
-//! /// let op: Operator = Operator::new(builder)?;
-//! ///
-//! /// Ok(())
-//! /// }
-//! /// ```
-//! #[derive(Default, Debug, Serialize, Deserialize, Clone, PartialEq, Eq)]
-//! #[serde(default)]
-//! #[non_exhaustive]
-//! pub struct DuckConfig {
-//! pub root: Option<String>,
-//! }
+//! An implementation must:
//!
-//! #[derive(Default, Clone)]
-//! pub struct DuckBuilder {
-//! config: DuckConfig,
-//! }
-//! ```
-//!
-//! Now let's implement the required APIs for `DuckConfig`:
+//! - Map every advertised option to the native request without silently
+//! dropping it.
+//! - Preserve the operation's public success, error, and atomicity contract.
+//! - Return structured OpenDAL errors with the correct [`ErrorKind`] and
+//! useful context.
+//! - Keep credentials and other secrets out of `Debug` output and errors.
+//! - Forward cancellation and cleanup to protocol-specific readers, writers,
+//! deleters, and copiers.
//!
-//! ```ignore
-//! impl Configurator for DuckConfig {
-//! type Builder = DuckBuilder;
-//!
-//! fn into_builder(self) -> Self::Builder {
-//! DuckBuilder { config: self }
-//! }
-//! }
-//! ```
+//! # Adding or changing a service
//!
-//! Note that `DuckBuilder` is part of our public API, so it needs to be
-//! documented. And any changes you make will directly affect users, so
-//! please take it seriously. Otherwise, you will be hunted down by many
-//! angry ducks.
+//! Keep configuration, request construction, operation bodies, and error
+//! parsing at their existing service boundaries. Update the facade feature and
+//! service registration when the service must be available through the
+//! `opendal` crate.
//!
-//! Then, we can implement required APIs for `DuckBuilder`:
-//!
-//! ```ignore
-//! impl DuckBuilder {
-//! /// Set root of this backend.
-//! ///
-//! /// All operations will happen under this root.
-//! pub fn root(&mut self, root: &str) -> &mut Self {
-//! self.config.root = if root.is_empty() {
-//! None
-//! } else {
-//! Some(root.to_string())
-//! };
-//!
-//! self
-//! }
-//! }
-//!
-//! impl Builder for DuckBuilder {
-//! type Config = DuckConfig;
-//!
-//! fn build(self) -> Result<impl Service> {
-//! debug!("backend build started: {:?}", &self);
-//!
-//! let root = normalize_root(&self.config.root.clone().unwrap_or_default());
-//! debug!("backend use root {}", &root);
-//!
-//! Ok(DuckBackend { root })
-//! }
-//! }
-//! ```
-//!
-//! `DuckBuilder` is ready now, let's try to play with real ducks!
-//!
-//! ## Backend
-//!
-//! I'm sure you can see it already: `DuckBuilder` will build a
-//! `DuckBackend` that implements [`Service`]. The backend is what we used
-//! to communicate with the super-powered ducks!
-//!
-//! Let's keep adding more code under `backend.rs`:
-//!
-//! ```ignore
-//! /// Duck storage service backend
-//! #[derive(Clone, Debug)]
-//! pub struct DuckBackend {
-//! root: String,
-//! }
-//!
-//! impl Service for DuckBackend {
-//! type Reader = DuckReader;
-//! type Writer = ();
-//! type Lister = ();
-//! type Deleter = ();
-//! type Copier = ();
-//!
-//! fn info(&self) -> ServiceInfo {
-//! ServiceInfo::new(DUCK_SCHEME, &self.root, "")
-//! }
-//!
-//! fn capability(&self) -> Capability {
-//! Capability {
-//! read: true,
-//! ..Default::default()
-//! }
-//! }
-//!
-//! fn read(&self, ctx: &OperationContext, path: &str, args: OpRead) -> Result<Self::Reader> {
-//! gagaga!()
-//! }
-//! }
-//! ```
-//!
-//! Congratulations, we have implemented a [`Service`] that can talk to
-//! Super Power Ducks!
-//!
-//! What!? There are no Super Power Ducks? So sad, but never mind, we have
-//! really powerful storage services [here](https://github.com/apache/opendal/issues/5). Welcome to pick one to implement. I promise you won't
-//! have to `gagaga!()` this time.
+//! Before advertising new behavior, reproduce it against the actual service
+//! and run the matching capability-gated behavior tests. Protocol
+//! documentation, emulators, and fabricated responses can explain an
+//! implementation, but they do not establish real-service conformance.
//!
//! [`Service`]: crate::raw::Service
+//! [`Service::info`]: crate::raw::Service::info
+//! [`Service::capability`]: crate::raw::Service::capability
+//! [`ServiceInfo`]: crate::raw::ServiceInfo
//! [`ServiceDyn`]: crate::raw::ServiceDyn
//! [`Servicer`]: crate::raw::Servicer
-//! [`Operation`]: crate::raw::Operation
-//! [`Capability`]: crate::Capability
-//! [`ServiceInfo`]: crate::raw::ServiceInfo
-//! [`OperationContext`]: crate::raw::OperationContext
-//! [`Operator`]: crate::Operator
-//! [`Builder`]: crate::Builder
-//! [`Configurator`]: crate::Configurator
+//! [`OperationContext`]: crate::OperationContext
+//! [`ErrorKind`]: crate::ErrorKind
+//! [`ErrorKind::Unsupported`]: crate::ErrorKind::Unsupported
diff --git a/core/core/src/docs/internals/layer.rs b/core/core/src/docs/internals/layer.rs
index a45f6b07b1..6d5167e761 100644
--- a/core/core/src/docs/internals/layer.rs
+++ b/core/core/src/docs/internals/layer.rs
@@ -15,80 +15,64 @@
// specific language governing permissions and limitations
// under the License.
-//! The internal implementation details of [`Layer`].
+//! Implementing a layer.
//!
-//! OpenDAL has one layer composition surface:
+//! A [`Layer`] intercepts an operator's composed service, operation context,
+//! or both. Use a layer for behavior that applies across services, such as
+//! retry, timeout, tracing, metrics, or runtime resource replacement.
//!
-//! - [`Layer`] receives an already erased [`ServiceDyn`] stack and can wrap the
-//! service or its [`OperationContext`].
-//! - Layer wrappers still implement typed [`Service`], so their own reader,
-//! writer, lister, deleter, and copier bodies stay concrete.
-//! - The composition boundary is [`Servicer`]. Boxing operation
-//! bodies remains centralized in the blanket [`ServiceDyn`] implementation
-//! for typed [`Service`] values.
+//! # Two composition hooks
//!
-//! [`Layer`] itself is the runtime hook surface. Every hook returns `inner` by
-//! default, so a layer only implements the plane it needs:
+//! [`Layer`] exposes two hooks:
//!
-//! ```ignore
-//! pub trait Layer: Send + Sync + Debug + Unpin + 'static {
-//! fn apply_service(&self, srv: Servicer) -> Servicer;
-//! fn apply_context(&self, srv: Servicer, inner: OperationContext) -> OperationContext;
-//! }
+//! ```text
+//! fn apply_service(&self, service: Servicer) -> Servicer;
+//! fn apply_context(
+//! &self,
+//! service: Servicer,
+//! context: OperationContext,
+//! ) -> OperationContext;
//! ```
//!
-//! [`Operator`] replays the same ordered layer list over the base service and
-//! base [`OperationContext`]. It first composes the operation service stack.
-//! Then context hooks receive the final [`Servicer`] and compose runtime
-//! resources such as HTTP transport and executor. The composed context is passed
-//! to service operations.
+//! `apply_service` wraps storage operations. `apply_context` wraps or replaces
+//! runtime resources such as the HTTP transport and executor. Each hook
+//! returns its input unchanged by default, so a layer implements only the
+//! plane it owns.
//!
-//! An operation layer normally has two parts:
+//! The operator first applies every service hook in insertion order. It then
+//! applies every context hook in the same order, passing the final service
+//! stack to each context hook. Adding a layer or replacing the base context
+//! replays the complete layer list, producing a service stack and context from
+//! the same ordering.
//!
-//! - `XxxLayer` implements [`Layer`] and returns a typed service wrapper from
-//! `apply_service`.
-//! - `XxxService` stores the inner [`Servicer`] and implements [`Service`].
+//! # Operation layers
//!
-//! ```ignore
-//! pub struct XxxLayer;
+//! An operation layer normally contains:
//!
-//! impl Layer for XxxLayer {
-//! fn apply_service(&self, inner: Servicer) -> Servicer {
-//! Arc::new(XxxService { inner })
-//! }
-//! }
-//! ```
-//!
-//! Most operation layers only override the operations they need and forward the
-//! rest to `inner`. This works because [`Servicer`] implements [`Service`] by
-//! forwarding calls to [`ServiceDyn`], while the wrapper keeps concrete
-//! operation body types until it is returned as a [`Servicer`].
+//! - An `XxxLayer` that implements [`Layer::apply_service`].
+//! - An `XxxService` that stores the inner [`Servicer`] and implements
+//! [`Service`].
//!
-//! Resource-only layers can implement only `apply_context`. If they do not need
-//! the final service stack, they should name that parameter `_srv`.
+//! The wrapper overrides only the operations it owns and forwards the rest to
+//! the inner service. It must also return capabilities that describe the
+//! behavior of the wrapped stack. The wrapper keeps its own operation body
+//! types concrete until OpenDAL erases it back into a [`Servicer`].
//!
-//! ```ignore
-//! pub struct TransportLayer {
-//! transport: HttpTransporter,
-//! }
+//! # Resource layers
//!
-//! impl Layer for TransportLayer {
-//! fn apply_context(&self, _srv: Servicer, inner: OperationContext) -> OperationContext {
-//! inner.with_http_transport(self.transport.clone())
-//! }
-//! }
-//! ```
+//! A resource-only layer implements [`Layer::apply_context`]. It should
+//! preserve the previous resource when lower layers must remain effective. A
+//! layer that wraps an HTTP transport or executor must decide explicitly
+//! whether requests continue through the previous value or replace it
+//! entirely.
//!
-//! Layers that need consistent policy across planes can implement both hooks,
-//! for example operation and I/O timeout handling. Resource wrappers that
-//! replace HTTP transport or executor must forward to the previous value when
-//! they want lower layers to remain effective.
+//! Layers that coordinate policy across operation and I/O phases can implement
+//! both hooks. Shared mutable state requires interior mutability and must
+//! remain `Send` and `Sync`, because cloned operators can run operations
+//! concurrently.
//!
//! [`Layer`]: crate::raw::Layer
+//! [`Layer::apply_service`]: crate::raw::Layer::apply_service
+//! [`Layer::apply_context`]: crate::raw::Layer::apply_context
//! [`Service`]: crate::raw::Service
-//! [`ServiceDyn`]: crate::raw::ServiceDyn
//! [`Servicer`]: crate::raw::Servicer
-//! [`HttpTransporter`]: crate::HttpTransporter
-//! [`Executor`]: crate::Executor
-//! [`OperationContext`]: crate::raw::OperationContext
-//! [`Operator`]: crate::Operator
diff --git a/core/core/src/docs/internals/mod.rs b/core/core/src/docs/internals/mod.rs
index ee6b932b7f..f041195af1 100644
--- a/core/core/src/docs/internals/mod.rs
+++ b/core/core/src/docs/internals/mod.rs
@@ -17,37 +17,35 @@
//! The internal implementation details of OpenDAL.
//!
-//! OpenDAL provides unified abstraction via two-level API sets:
+//! This guide explains the extension boundaries used to implement OpenDAL
+//! storage services and layers. Application code should use the public
+//! [`Operator`] API; the raw APIs described here can change between minor
+//! releases.
//!
-//! - Public API like [`Operator`] provides user-level API.
-//! - Raw API like [`Service`], [`Layer`] provides developer-level API.
-//!
-//! OpenDAL tries its best to keep the public API stable. But raw APIs
-//! may change between minor releases from time to time. So most users
-//! should only use the public API. Developers use raw APIs to implement
-//! storage services or their own [`Layer`].
-//!
-//! In this section, we will talk about the following components:
-//!
-//! - [`Service`][accessor]: to connect underlying storage services.
-//! - [`Layer`][layer]: middleware/interceptor between storage services.
-//!
-//! The relation between [`Service`], [`Layer`], [`Servicer`], [`OperationContext`],
-//! and [`Operator`] looks like the following:
+//! OpenDAL keeps service implementations typed until it composes them into an
+//! operator:
//!
//! ```text
-//! ┌──────────────┐ ┌──────────────────────────────┐ ┌──────────────┐
-//! │ Service │ │ Servicer │ │ Operator │
-//! │ builders ├────►│ + Layer::apply_* replay ├────►│ srv + ctx │
-//! └──────────────┘ │ + OperationContext resources │ └──────────────┘
-//! └──────────────────────────────┘
+//! Builder -> Service -> ServiceDyn -> Servicer -> Layer stack -> Operator
+//! ^ |
+//! +------ type erasure ------+
//! ```
//!
-//! [`Builder`]: crate::Builder
+//! - A [`Service`] implements storage operations and reports the capabilities
+//! it provides.
+//! - [`ServiceDyn`] is the object-safe boundary that erases a typed service
+//! and its operation bodies.
+//! - A [`Servicer`] is the shared, type-erased service handle stored by
+//! operators and layers.
+//! - A [`Layer`] wraps the service stack, the operation context, or both.
+//!
+//! Continue with [implementing a service][accessor] or
+//! [implementing a layer][layer].
+//!
//! [`Operator`]: crate::Operator
//! [`Service`]: crate::raw::Service
+//! [`ServiceDyn`]: crate::raw::ServiceDyn
//! [`Servicer`]: crate::raw::Servicer
-//! [`OperationContext`]: crate::raw::OperationContext
//! [`Layer`]: crate::raw::Layer
pub mod accessor;
diff --git a/core/core/src/docs/mod.rs b/core/core/src/docs/mod.rs
index 68fe7d7760..ab925c6348 100644
--- a/core/core/src/docs/mod.rs
+++ b/core/core/src/docs/mod.rs
@@ -15,29 +15,33 @@
// specific language governing permissions and limitations
// under the License.
-//! This module holds documentation for OpenDAL.
+//! OpenDAL contracts, Rust core guides, and historical project documents.
//!
-//! It's highly recommended that you start by reading [`concepts`] first.
+//! Start with [`concepts`] for the Rust core API model. [`specs`] defines the
+//! current portable contracts implemented by OpenDAL. Service and layer authors
+//! can continue with [`internals`], while [`performance`] covers application
+//! tuning. Accepted RFCs in [`rfcs`] preserve the decisions and context that
+//! led to current designs.
#![allow(rustdoc::bare_urls)]
-pub mod comparisons;
-
pub mod concepts;
-
pub mod internals;
-
pub mod performance;
+pub mod specs;
/// Changes log for all OpenDAL released versions.
#[doc = include_str!("../../../CHANGELOG.md")]
#[cfg(not(doctest))]
+#[allow(rustdoc::broken_intra_doc_links, rustdoc::invalid_rust_codeblocks)]
pub mod changelog {}
#[cfg(not(doctest))]
+#[allow(rustdoc::broken_intra_doc_links, rustdoc::invalid_rust_codeblocks)]
pub mod rfcs;
/// Upgrade and migrate procedures while OpenDAL meets breaking changes.
#[doc = include_str!("upgrade.md")]
#[cfg(not(doctest))]
+#[allow(rustdoc::broken_intra_doc_links, rustdoc::invalid_rust_codeblocks)]
pub mod upgrade {}
diff --git a/core/core/src/docs/performance/concurrent_write.md b/core/core/src/docs/performance/concurrent_write.md
index ee5e34f26b..4f394e5895 100644
--- a/core/core/src/docs/performance/concurrent_write.md
+++ b/core/core/src/docs/performance/concurrent_write.md
@@ -1,101 +1,138 @@
-# Concurrent Write
+# Concurrent Writes
-OpenDAL writes data sequentially by default.
+OpenDAL writes parts sequentially by default. This keeps memory and request
+pressure low, but one in-flight request often cannot fill the available
+bandwidth on a high-latency path.
```rust
-# use opendal::Operator;
-# async fn test() {
-let w = op.writer("test.txt").await?;
-w.write(data1).await?;
-w.write(data2).await?;
-w.close().await?;
+# use opendal_core::{Buffer, Operator, Result};
+# async fn write_sequentially(
+# op: &Operator,
+# first: Buffer,
+# second: Buffer,
+# ) -> Result<()> {
+let mut writer = op.writer("large-object.bin").await?;
+writer.write(first).await?;
+writer.write(second).await?;
+writer.close().await?;
+# Ok(())
# }
```
-Most of the time, this can't maximize write performance due to limitations on a single connection. We can perform concurrent writes to improve performance.
+Set `concurrent` above `1` to let services with multipart or block-based write
+support upload independent parts in parallel:
```rust
-# use opendal::Operator;
-# async fn test() {
-let w = op
- .writer_with("test.txt")
+# use opendal_core::{Buffer, Operator, Result};
+# async fn write_concurrently(
+# op: &Operator,
+# first: Buffer,
+# second: Buffer,
+# ) -> Result<()> {
+let mut writer = op
+ .writer_with("large-object.bin")
.concurrent(8)
.await?;
-w.write(data1).await?;
-w.write(data2).await?;
-w.close().await?;
+writer.write(first).await?;
+writer.write(second).await?;
+writer.close().await?;
+# Ok(())
# }
```
-After setting `concurrent`, OpenDAL will attempt to write the specified file concurrently. The maximum level of concurrency is determined by the `concurrent` parameter. By default, it is set to 1, indicating a sequential write operation.
+For multipart and block writers, OpenDAL schedules part uploads through its
+[`Executor`][crate::Executor] and tracks their results in a task queue. A write
+call can return while earlier parts are still in flight when the queue has
+capacity. [`Writer::close`][crate::Writer::close] drains the queue and performs
+the final commit, so applications must always check the result from `close`.
-Under the hood, OpenDAL maintains a task queue to manage concurrent writes. It spawns asynchronous tasks in the background using the [`Executor`][crate::Executor] and tracks the status of each task. The task queue is flushed when the writer is closed, allowing data to be written concurrently without blocking the main thread.
-
-Take our example here, the write of `data1` will not block the write of `data2`. The two writes will be executed concurrently.
-
-The underlying implementation of concurrent writes may vary depending on the backend. For instance, the `s3` backend leverages the S3 Multipart Uploads API to handle concurrent writes, while the `azblob` backend utilizes the Block API for the same purpose.
+The storage service determines the native mechanism. S3 uses multipart upload,
+while Azure Blob uses block upload. Services without a concurrent write path
+continue to execute sequentially even when the option is present. Check
+[`Capability::write_can_multi`][crate::Capability::write_can_multi] before
+tuning multipart or block writes.
## Tuning
-There are two parameters that can be tuned to optimize concurrent writes:
+Concurrent writes have two controls:
-- `concurrent`: This parameter controls the maximum number of concurrent writes. The default value is 1.
-- `chunk`: This parameter specifies the size of each chunk of data to be written. The default value is vary for different storage services.
+- `concurrent` is the maximum number of part uploads OpenDAL attempts to keep
+ in flight. The default is `1`, and values below `1` are normalized to `1`.
+- `chunk` is the target size of each part. Its default and valid range depend on
+ the service.
### `concurrent`
-The most important thing to understand is that `concurrent` is not a strict limit. It represents the maximum number of concurrent writes that OpenDAL will attempt to perform. The actual number of concurrent writes may be lower, depending on the input data throughput.
-
-For example, if you set `concurrent` to 8, OpenDAL will attempt to perform up to 8 concurrent writes. However, if the input data throughput is low, it might only carry out 2 or 3 concurrent writes at a time, as there isn't enough data to keep all 8 writes active.
-
-The best value for `concurrent` depends on the specific use case and the underlying storage service. In general, a higher value can lead to better performance, but it highly depends on the storage service and the network conditions. For example, if the storage service is robust and bandwidth is sufficient, you may observe a linear increase in performance with higher `concurrent` values. However, if the storage service has request limits or the network is nearly saturated, increasing `con [...]
+`concurrent` is an upper bound, not a guaranteed level. Setting it to `8` does
+not create eight active requests unless the input arrives quickly enough and
+the object contains enough parts. Object size, request latency, and service
+support can all reduce the observed concurrency.
-It's recommended to start with a lower value like `2` or `4` and gradually increase it while monitoring performance and resource usage.
+Start with `2` or `4`, then try `8` while measuring the real workload. Higher
+concurrency can improve bandwidth utilization, but it also increases in-flight
+requests, buffered data, and pressure on service request limits. Once the
+network or service is saturated, increasing it further can reduce throughput by
+causing throttling and retries.
### `chunk`
-The `chunk` parameter specifies the size of each chunk of data to be written. A larger chunk size can improve performance, but it may also increase memory usage. The default value is vary for different storage services.
+OpenDAL buffers small writes until it can form a part of the configured size.
+This avoids issuing one storage request for every small input buffer. The final
+part may be smaller.
-For example, s3 is using `5MiB` as the default chunk size. It's also the minimum chunk size for s3. If you set a smaller chunk size, OpenDAL will automatically adjust it to `5MiB`.
+Larger chunks reduce request count and per-request overhead, but use more memory
+per in-flight part and require more data to be retried after a failed upload.
+Smaller chunks begin parallel work sooner, but increase request rate and may be
+invalid for the service. For example, S3 multipart upload has a 5 MiB minimum
+part size; OpenDAL raises smaller configured values to the service minimum.
-The best value for `chunk` depends on the specific use case and the underlying storage service. For most object storage services, a chunk size of `8MiB` or larger is recommended. However, if you're working with smaller files or have limited memory resources, you may want to use a smaller chunk size.
+Inspect [`Capability::write_multi_min_size`][crate::Capability::write_multi_min_size]
+and [`Capability::write_multi_max_size`][crate::Capability::write_multi_max_size]
+before overriding the service default. For large object-storage uploads, 8 MiB
+is a useful starting point, followed by measurements at the expected object
+sizes and concurrency.
-Please note that if you input small chunks of data, OpenDAL will attempt to merge them into a larger chunk before writing. This helps avoid the overhead of writing numerous small chunks, which can negatively affect performance.
+## One-shot and adapter APIs
-## Usage
-
-To upload a large in-memory chunk concurrently:
+`write_with` can split one in-memory buffer into concurrent parts:
```rust
-# use opendal::Operator;
-# async fn test() {
-let data = vec![0; 10 * 1024 * 1024]; // 10MiB
-let _ = op.write_with("test.txt", data).concurrent(4).await?;
+# use opendal_core::{Buffer, Operator, Result};
+# async fn upload(op: &Operator, data: Buffer) -> Result<()> {
+let _metadata = op
+ .write_with("large-object.bin", data)
+ .chunk(8 * 1024 * 1024)
+ .concurrent(4)
+ .await?;
+# Ok(())
# }
```
-`concurrent` and `chunk` also works in [`into_sink`][crate::Writer::into_sink], [`into_bytes_sink`][crate::Writer::into_bytes_sink] and [`into_futures_async_write`][crate::Writer::into_futures_async_write]:
+A configured writer keeps the same chunking and concurrency behavior after it
+is converted with [`Writer::into_sink`][crate::Writer::into_sink],
+[`Writer::into_bytes_sink`][crate::Writer::into_bytes_sink], or
+[`Writer::into_futures_async_write`][crate::Writer::into_futures_async_write].
```rust
use std::io;
-use bytes::Bytes;
use futures::SinkExt;
-use opendal::{Buffer, Operator};
-use opendal::Result;
-
-async fn test(op: Operator) -> io::Result<()> {
- let mut w = op
- .writer_with("hello.txt")
- .concurrent(8)
- .chunk(256)
+use opendal_core::{Buffer, Operator};
+
+async fn upload_with_sink(op: Operator, data: Buffer) -> io::Result<()> {
+ let mut sink = op
+ .writer_with("large-object.bin")
+ .chunk(8 * 1024 * 1024)
+ .concurrent(4)
.await?
.into_sink();
- let bs = "Hello, World!".as_bytes();
- w.send(Buffer::from(bs)).await?;
- w.close().await?;
+ sink.send(data).await?;
+ sink.close().await?;
Ok(())
}
```
+
+Measure end-to-end throughput and completion latency together with peak memory,
+request count, retries, throttling, and the result from the final close.
diff --git a/core/core/src/docs/performance/http_optimization.md b/core/core/src/docs/performance/http_optimization.md
index d3efd6301c..825554056e 100644
--- a/core/core/src/docs/performance/http_optimization.md
+++ b/core/core/src/docs/performance/http_optimization.md
@@ -1,138 +1,154 @@
# HTTP Optimization
-All OpenDAL HTTP-based storage services use the same
-[HttpTransporter][crate::HttpTransporter] abstraction. This design offers users
-a unified interface for configuring HTTP transports. The `opendal` facade
-installs [reqwest](https://crates.io/crates/reqwest) as the default HTTP transport
-when `opendal::install_default()` is called with the `http-transport-reqwest`
-feature enabled. The default `auto-register-services` feature calls
-`install_default()` before `main`. Applications that need to install their own
-process-wide transport should disable `auto-register-services`; transports
-configured directly on an operator are unaffected.
-
-Many of the services supported by OpenDAL are HTTP-based. This guide aims to provide optimization tips for using HTTP-based storage services. While these tips are also applicable to other HTTP clients, the configuration methods may vary.
-
-Please note that the following optimizations are based on experience and may not be suitable for all scenarios. The most effective way to determine the optimal configuration is to test it in your specific environment.
-
-## HTTP/1.1
-
-According to benchmarks from OpenDAL users, `HTTP/1.1` is generally faster than `HTTP/2` for large-scale download and upload operations.
+Most OpenDAL services use HTTP, so transport configuration can dominate
+throughput and tail latency. This guide uses reqwest because the `opendal`
+facade installs it as the default transport when an HTTP transport feature is
+enabled.
+
+The default `auto-register-services` feature calls `opendal::install_default()`
+before `main`. Applications that need a different process-wide default must
+disable `auto-register-services`, initialize the service registry when needed,
+and install their transport before another default is installed.
+
+A simpler option is to replace the transport for one operator. Add
+`opendal-http-transport-reqwest` and `reqwest` as direct dependencies, build one
+client, and reuse the resulting operator:
+
+```rust,ignore
+use opendal::{HttpTransporter, Operator};
+use opendal_http_transport_reqwest::ReqwestTransport;
+
+fn with_reqwest_client(op: Operator, client: reqwest::Client) -> Operator {
+ let transport = HttpTransporter::new(ReqwestTransport::new(client));
+ let ctx = op.base_context().with_http_transport(transport);
+ op.with_context(ctx)
+}
+```
-`reqwest` tends to maintain only a single TCP connection for `HTTP/2`, relying on its built-in multiplexing capabilities. There is a [known issue](https://github.com/hyperium/hyper/issues/3623) in the underlying `hyper-util` HTTP util library: an HTTP/2 connection will remain in the pool and continue to be reused unless it is poisoned or closed after an idle timeout, even when the connection's max concurrent streams limit has been reached, or the TCP connection is exhausted (i.e., close [...]
+Creating a client or operator per request discards DNS and connection-pool
+state and repeatedly pays connection setup costs.
-For HTTP/1.1, new connections will be created whenever there's no idle ones in the pool. While it solves the connection reuse issue, it also brings up possibility of excessive TCP connection. It's suggested to set a max concurrent request limit to avoid server overload and host bandwidth exhaustion.
+## Prefer HTTP/1.1 for large transfers
-When `HTTP/2` is disabled, `reqwest` falls back to `HTTP/1.1` and utilizes its default connection pool. This approach is better suited for large files, as it allows multiple TCP connections to be opened and used concurrently, significantly improving performance for large file downloads and uploads.
+For high-throughput uploads and downloads of large objects, benchmark HTTP/1.1
+first. OpenDAL users often see higher aggregate throughput with HTTP/1.1 because
+reqwest can open several pooled connections, while HTTP/2 normally multiplexes
+requests over one connection.
-If your workloads involve large files or require high throughput, and are not sensitive to latency, consider disabling `HTTP/2` in your configuration.
+One HTTP/2 connection can become the bottleneck when it reaches its stream
+limit or the bandwidth available to one TCP connection. The hyper client does
+not currently open another pooled HTTP/2 connection when the existing one
+reaches its maximum concurrent streams; see [hyper issue #3623]. This matters
+most for long-lived streams and highly concurrent large transfers.
-```rust
-let client = reqwest::ClientBuilder::new()
- // Disable http2 for better performance.
- .http1_only()
- .build()
- .expect("http client must be created");
+Force HTTP/1.1 with `ClientBuilder::http1_only`:
-// Replace the operator's base HTTP transport.
-let transport = HttpTransporter::new(ReqwestTransport::new(client));
-let op = op.with_context(OperationContext::new().with_http_transport(transport));
+```rust,ignore
+let client = reqwest::Client::builder()
+ .http1_only()
+ .build()?;
+let op = with_reqwest_client(op, client);
```
-## DNS Caching
-
-`reqwest` uses the DNS resolver provided by Rust's standard library by default, which is backed by the `getaddrinfo` system call under the hood. This system call does not cache results by default, meaning that each time you make a request to a new domain, a DNS lookup will be performed.
+HTTP/1.1 can open many TCP connections when no idle connection is available.
+Bound OpenDAL request concurrency so the client cannot exhaust host resources
+or overload the service. HTTP/2 can still be better for many small requests or
+when connection count matters, so measure both protocols against the real
+endpoint and object-size distribution.
-Under high-throughput workloads, this can cause a significant performance degradation, as each request incurs the overhead of a DNS lookup. It can also negatively affect the resolver, potentially overwhelming it with the volume of requests. In extreme cases, this may result in a DoS attack on the resolver, rendering it unresponsive.
+## Enable DNS caching
-To mitigate this issue, you can enable DNS caching in `reqwest` by using the `hickory-dns` feature. This feature provides a more efficient DNS resolver that caches results.
+Without the optional Hickory resolver, reqwest uses a `getaddrinfo`-based
+resolver and does not maintain a reqwest-level DNS cache. Repeated resolution
+can become visible at high request rates and can overload a constrained local
+resolver.
-```rust
-let client = reqwest::ClientBuilder::new()
- // Enable hickory dns for dns caching and async dns resolve.
- .hickory_dns(true)
- .build()
- .expect("http client must be created");
+Enable reqwest's `hickory-dns` feature in the application and select it on the
+client:
-// Replace the operator's base HTTP transport.
-let transport = HttpTransporter::new(ReqwestTransport::new(client));
-let op = op.with_context(OperationContext::new().with_http_transport(transport));
+```toml
+[dependencies]
+reqwest = { version = "0.13", features = ["hickory-dns"] }
```
-The default DNS cache settings from `hickory_dns` are generally sufficient for most workloads. However, if you have specific requirements—such as sharing the same DNS cache across multiple HTTP clients or configuring the DNS cache size—you can use the `Xuanwo/reqwest-hickory-resolver` crate to set up a custom DNS resolver.
-
-```rust
-/// Global shared hickory resolver.
-static GLOBAL_HICKORY_RESOLVER: LazyLock<Arc<HickoryResolver>> = LazyLock::new(|| {
- let mut opts = ResolverOpts::default();
- // Only query for the ipv4 address.
- opts.ip_strategy = LookupIpStrategy::Ipv4Only;
- // Use larger cache size for better performance.
- opts.cache_size = 1024;
- // Positive TTL is set to 5 minutes.
- opts.positive_min_ttl = Some(Duration::from_secs(300));
- // Negative TTL is set to 1 minute.
- opts.negative_min_ttl = Some(Duration::from_secs(60));
-
- Arc::new(
- HickoryResolver::default()
- // Always shuffle the DNS results for better performance.
- .with_shuffle(true)
- .with_options(opts),
- )
-});
-
-let client = reqwest::ClientBuilder::new()
- // Use our global hickory resolver instead.
- .dns_resolver(GLOBAL_HICKORY_RESOLVER.clone())
- .build()
- .expect("http client must be created");
-
-// Replace the operator's base HTTP transport.
-let transport = HttpTransporter::new(ReqwestTransport::new(client));
-let op = op.with_context(OperationContext::new().with_http_transport(transport));
+```rust,ignore
+let client = reqwest::Client::builder()
+ .hickory_dns(true)
+ .build()?;
+let op = with_reqwest_client(op, client);
```
-The `ResolverOpts` has many options that can be configured. For a complete list of options, please refer to the [hickory_resolver documentation](https://docs.rs/hickory-resolver/latest/hickory_resolver/config/struct.ResolverOpts.html).
+The built-in Hickory settings are a good starting point. Use
+`ClientBuilder::dns_resolver` when the application needs to share a custom
+resolver or control cache size, positive and negative TTLs, IP selection, or
+address shuffling. Longer TTLs reduce resolver traffic but retain stale records
+longer, so test failover behavior before increasing them.
-Here is a summary of the most commonly used options:
+## Set timeouts deliberately
-- `ip_strategy`: `hickory_resolver` default to use `Ipv4thenIpv6` strategy, which means it will first query for the IPv4 address and then the IPv6 address. This is generally a good strategy for most workloads. However, if you only need IPv4 addresses, you can set this option to `Ipv4Only` to avoid unnecessary DNS lookups.
-- `cache_size`: This option controls the size of the DNS cache. A larger cache size can improve performance, but it may also increase memory usage. The default value is `32`.
-- `positive_min_ttl` and `negative_min_ttl`: This option controls the minimum TTL for positive and negative DNS responses. A longer TTL can improve performance, but it may also increase the risk of stale DNS records. The default value is `None`. Some bad DNS servers may return a TTL of `0` even when the record is valid. In this case, you can set a longer TTL to avoid unnecessary DNS lookups.
+Reqwest 0.13 has no request, read, or connect timeout by default. At minimum,
+set a connect timeout so an unreachable endpoint cannot leave connection setup
+unbounded:
-In addition to the options mentioned above, `Xuanwo/reqwest-hickory-resolver` also offers a `shuffle` option. This setting determines whether the DNS results are shuffled before being returned. Shuffling can enhance performance by distributing the load more evenly across multiple IP addresses.
+```rust,ignore
+use std::time::Duration;
-## Timeout
+let client = reqwest::Client::builder()
+ .connect_timeout(Duration::from_secs(5))
+ .read_timeout(Duration::from_secs(30))
+ .build()?;
+let op = with_reqwest_client(op, client);
+```
+
+`read_timeout` limits inactivity between successful reads. A total request
+timeout created with `ClientBuilder::timeout` covers the complete transfer, so
+do not set it below the expected duration of a healthy large upload or
+download.
-`reqwest` didn't set a default timeout for HTTP requests. This means that if a request hangs or takes too long to complete, it can block the entire process, leading to performance degradation or even application crashes.
+The facade's optional `layers-timeout` feature also provides `TimeoutLayer` for
+OpenDAL operation and I/O timeouts:
-It's recommended to set a connect timeout for HTTP requests to prevent this issue.
+```rust,ignore
+use std::time::Duration;
-```rust
-let client = reqwest::ClientBuilder::new()
- // Set a connect timeout of 5 seconds.
- .connect_timeout(Duration::from_secs(5))
- .build()
- .expect("http client must be created");
+use opendal::layers::TimeoutLayer;
-// Replace the operator's base HTTP transport.
-let transport = HttpTransporter::new(ReqwestTransport::new(client));
-let op = op.with_context(OperationContext::new().with_http_transport(transport));
+let op = op.layer(
+ TimeoutLayer::new()
+ .with_timeout(Duration::from_secs(30))
+ .with_io_timeout(Duration::from_secs(60)),
+);
```
-It's also recommended to use opendal's [`TimeoutLayer`][crate::layers::TimeoutLayer] to prevent slow requests hangs forever. This layer will automatically cancel the request if it takes too long to complete.
+Transport timeouts bound HTTP phases. `TimeoutLayer` applies at OpenDAL's
+operation and I/O boundaries. Configure both when the application needs bounds
+at both levels.
-```rust
-let op = op.layer(TimeoutLayer::new());
-```
+## Size the connection pool
-## Connection Pool
+Reqwest reuses idle connections. In reqwest 0.13, the default idle timeout is
+90 seconds and the default maximum number of idle connections per host is
+unlimited. Tune both values for the traffic pattern; neither setting limits
+active connections:
-`reqwest` uses a connection pool to manage HTTP connections. This allows multiple requests to share the same connection, reducing the overhead of establishing new connections for each request.
+```rust,ignore
+use std::time::Duration;
+
+let client = reqwest::Client::builder()
+ .pool_idle_timeout(Duration::from_secs(60))
+ .pool_max_idle_per_host(32)
+ .build()?;
+let op = with_reqwest_client(op, client);
+```
-By default, the connection pool is unlimited, allowing `reqwest` to open as many connections as needed. The default keep-alive timeout is 90 seconds, meaning any connection idle for longer than that will be closed.
+- Keep enough idle connections for expected HTTP/1.1 concurrency so bursts do
+ not repeat DNS, TCP, and TLS setup.
+- Set a finite idle per-host limit for applications that contact many
+ endpoints. Bound active connections with OpenDAL request concurrency.
+- Shorten the idle timeout when traffic is sparse and connection reuse is rare.
-You can tune those settings via:
+Measure connection reuse, open sockets, handshake rate, request concurrency,
+throughput, and tail latency together. A pool setting that improves a steady
+single-endpoint workload can waste resources in a bursty multi-endpoint one.
-- [pool_idle_timeout](https://docs.rs/reqwest/0.13.2/reqwest/struct.ClientBuilder.html#method.pool_idle_timeout): Set an optional timeout for idle sockets being kept-alive.
-- [pool_max_idle_per_host](https://docs.rs/reqwest/0.13.2/reqwest/struct.ClientBuilder.html#method.pool_max_idle_per_host): Sets the maximum idle connection per host allowed in the pool.
+[hyper issue #3623]: https://github.com/hyperium/hyper/issues/3623
diff --git a/core/core/src/docs/performance/mod.rs b/core/core/src/docs/performance/mod.rs
index a817b84798..bf827fe009 100644
--- a/core/core/src/docs/performance/mod.rs
+++ b/core/core/src/docs/performance/mod.rs
@@ -17,13 +17,15 @@
//! OpenDAL Performance Guide
//!
-//! OpenDAL is designed in a zero-cost abstraction way, which means that
-//! users won't pay for the abstraction cost if they don't use it. But this
-//! can also means that users can't maximize the performance of OpenDAL
-//! if they don't know how to use it.
+//! OpenDAL keeps the storage abstraction lightweight, but applications still
+//! need to configure buffering, concurrency, and HTTP transport for their
+//! workload. This guide gives concrete Rust configuration examples and explains
+//! the trade-offs behind them.
//!
-//! This document will introduce some tips to improve the performance of
-//! OpenDAL.
+//! - [Concurrent writes][concurrent_write] explains how OpenDAL schedules write
+//! parts and how to tune `concurrent` and `chunk`.
+//! - [HTTP optimization][http_optimization] explains when to try HTTP/1.1 and
+//! how to configure DNS, timeouts, and connection pools.
#[doc = include_str!("concurrent_write.md")]
pub mod concurrent_write {}
diff --git a/core/core/src/docs/rfcs/README.md b/core/core/src/docs/rfcs/README.md
index a474cf89a2..ba8f3121b8 100644
--- a/core/core/src/docs/rfcs/README.md
+++ b/core/core/src/docs/rfcs/README.md
@@ -1,6 +1,12 @@
-# RFCs - OpenDAL Active RFC List
+# RFCs - OpenDAL Design History
-RFCs power OpenDAL's development.
+RFCs power OpenDAL's development and preserve the history behind substantial
+design decisions.
+
+An accepted RFC is an immutable historical document. It records the proposal
+that reached consensus at the time of acceptance, but it is not the current
+contract. [OpenDAL specifications][crate::docs::specs] define the living
+contracts implemented by current releases.
The "RFC" (request for comments) process is intended to provide a consistent and controlled path for changes to OpenDAL (such as new features) so that all stakeholders can be confident about the direction of the project.
@@ -56,7 +62,12 @@ Each accepted RFC is associated with an issue in the OpenDAL repository, which t
If you are interested in implementing an RFC but are unsure if someone else is already working on it,
feel free to inquire by leaving a comment on the associated issue.
+After implementation, update or add a specification when the RFC establishes
+portable user-visible behavior. Later contract changes update the specification
+and, when substantial, require a new RFC. They do not rewrite the accepted RFC.
+
## Some useful tips
- The author of an RFC may not be the same one as the implementer. Therefore, when submitting an RFC, it is advisable to include sufficient information.
-- If modifications are needed for an accepted RFC, please submit a new pull request or create a new RFC to propose changes.
+- Do not modify an accepted RFC. Correct the current contract in a specification
+ and create a new RFC when the change is substantial.
diff --git a/core/core/src/docs/specs/README.md b/core/core/src/docs/specs/README.md
new file mode 100644
index 0000000000..813f773b60
--- /dev/null
+++ b/core/core/src/docs/specs/README.md
@@ -0,0 +1,32 @@
+# OpenDAL Specifications
+
+Specifications define the portable behavior OpenDAL promises: the observable
+contracts that hold across every supporting service. They are living
+documents; implementations, API documentation, and behavior tests must remain
+consistent with them as OpenDAL evolves. Some promises apply only when the
+service advertises the corresponding capability, and an unsupported request
+fails with an explicit error. Behavior outside a specification's promises is
+outside the portable contract, and callers must not rely on it holding across
+services.
+
+Specifications state the contract. Behavior tests under `core/tests/behavior`
+provide executable conformance coverage for the specified contracts. Public
+API documentation explains the observable behavior, capability requirements,
+parameter meaning, and relevant errors at the API site, then links to the
+specification for the complete cross-operation contract.
+
+Accepted RFCs are immutable historical documents. They record the design that
+reached consensus, including its original motivation and trade-offs.
+Specifications describe the current contract, even when later implementation
+work or a newer RFC changes the accepted design.
+
+Editorial corrections and changes that codify already implemented behavior use
+the normal pull request process and include evidence for that behavior. A
+substantial contract change requires a new RFC. The resulting implementation,
+behavior tests, API documentation, and specification change land together.
+
+## Active specifications
+
+- [Conditional operations](https://opendal.apache.org/docs/specifications/conditional-operations/)
+ define portable predicate, error, and atomicity semantics, including the
+ behavior when the file does not exist.
diff --git a/core/core/src/docs/specs/conditional-operations.md b/core/core/src/docs/specs/conditional-operations.md
new file mode 100644
index 0000000000..89f449fd6b
--- /dev/null
+++ b/core/core/src/docs/specs/conditional-operations.md
@@ -0,0 +1,152 @@
+# Conditional operations
+
+Conditional operations let callers observe or mutate storage only when a file
+is in an expected state. A file is the object currently stored at an OpenDAL
+path.
+
+This specification defines the portable contract, and OpenDAL promises only
+the behavior it explicitly defines. A service may support a subset of the
+conditions through its advertised capabilities. Service-specific extensions do
+not change the meanings defined here.
+
+## Which file a condition checks
+
+Each condition checks one file:
+
+- `stat` and `read` check the file being observed.
+- `write` and `delete` check the file at the operation path.
+- `copy` and `rename` check the destination file.
+- `restore` checks the file currently stored at the restore path. The selected
+ version remains the source of the restore.
+
+A version selector chooses the stored version an operation acts on. A version
+condition compares the version of the file checked by the condition. They are
+different concepts.
+
+A copy or rename may also fail because its source does not exist, and a
+restore may also fail because its selected version does not exist. When such an
+error and a condition error occur together, this specification does not define
+which error the operation returns.
+
+## Capability contract
+
+OpenDAL validates conditional options before dispatching an operation. If the
+service does not advertise the required capability, the operation returns
+`Unsupported`. OpenDAL never silently ignores a condition.
+
+A service that advertises a conditional capability must implement the exact
+contract for that condition. Separate capability fields do not imply support
+for combinations of those conditions. If an implementation accepts a
+combination, the file must satisfy every condition. If it cannot preserve the
+complete conjunction, it returns `Unsupported` instead of dropping a
+condition.
+
+## Behavior when the file exists
+
+The ETag conditions in this table use concrete ETag values. A wildcard ETag
+has no portable meaning unless the documentation for a specific operation
+defines one.
+
+| Condition | Succeeds when |
+| --- | --- |
+| `if_match(etag)` | The file's ETag equals `etag`. |
+| `if_none_match(etag)` | The file's ETag differs from `etag`. |
+| `if_version_match(version)` | The file's version equals `version`. |
+| `if_version_not_match(version)` | The file's version differs from `version`. |
+| `if_not_exists()` | Never. |
+| `if_modified_since(time)` | The file was modified after `time`. |
+| `if_unmodified_since(time)` | The file was not modified after `time`. |
+| `if_not_changed(metadata)` | The file has the identity represented by `metadata`. |
+
+## Behavior when the file does not exist
+
+`stat` and `read` return `NotFound` regardless of the condition.
+
+For mutations:
+
+| Condition | Result |
+| --- | --- |
+| `if_match(etag)` | `ConditionNotMatch`. |
+| `if_version_match(version)` | `ConditionNotMatch`. |
+| `if_not_changed(metadata)` | `ConditionNotMatch`. |
+| `if_none_match(etag)` with a concrete ETag | The condition succeeds. |
+| `if_not_exists()` | The condition succeeds. |
+| `if_version_not_match(version)` | No portable behavior is defined. |
+
+After a condition succeeds, the operation's normal semantics apply. Deleting a
+file that does not exist remains a successful no-op. A write or copy may create
+the file. Rename and restore may still fail because their source or selected
+version does not exist.
+
+## `if_not_changed` lowering
+
+`if_not_changed(metadata)` is a positive identity condition derived from
+metadata previously returned by OpenDAL. OpenDAL lowers it as follows:
+
+1. Use a version match when the operation supports version conditions and the
+ metadata contains a version.
+2. Otherwise, use an ETag match when the operation supports ETag conditions and
+ the metadata contains an ETag.
+3. Return `Unsupported` when the operation supports neither primitive.
+4. Return `ConfigInvalid` when a primitive is supported but the metadata
+ contains no usable identity token.
+
+An identical explicit equality condition is redundant. A conflicting equality
+condition returns `ConditionNotMatch`. Other accepted conditions combine with
+the derived condition using AND semantics.
+
+## Error contract
+
+- A false explicit condition returns `ConditionNotMatch`.
+- `stat` and `read` return `NotFound` when the file does not exist.
+- A conflict reported by the service that is unrelated to an explicit OpenDAL
+ condition returns `Conflict`.
+- An unavailable condition or condition combination returns `Unsupported`.
+
+Protocol status codes do not determine the error kind on their own. For
+example, an HTTP `304 Not Modified` or `412 Precondition Failed` maps to
+`ConditionNotMatch` only when it is the service's response to a condition
+explicitly supplied by OpenDAL.
+
+## Atomicity and visibility
+
+A service advertises a mutation condition only when it can evaluate the
+condition atomically with the mutation's visible commit. An implementation must
+not emulate a conditional mutation by issuing `stat` followed by an
+unconditional mutation.
+
+For staged or multipart writes, the service evaluates the condition at the
+final commit. If the condition fails, the previously visible file remains
+unchanged.
+
+The point at which an operation surfaces a conditional error is unspecified.
+Depending on the operation and service, it may surface while constructing an
+I/O handle, during I/O, or when closing the handle. The final result and
+atomicity contract remain the same.
+
+A conditional batch delete evaluates every entry independently. A service may
+batch entries only when it preserves each entry's condition and error.
+Detailed partial-success reporting follows the delete operation's separate
+contract.
+
+## Metadata contract
+
+A service that advertises ETag or version match support must populate the
+corresponding `Metadata` field whenever the native response provides it. The
+identity token returned from a successful mutation or observation must be
+usable as input to a later matching condition for the same file.
+
+## Conformance
+
+Capability-gated behavior tests under `core/tests/behavior` validate portable
+conditions against real services. They cover only the behavior this
+specification defines and do not impose cross-service expectations beyond it.
+A service must pass the relevant portable behavior tests before advertising a
+conditional capability.
+
+## Historical decisions
+
+- [RFC 5485: Conditional Reader](https://github.com/apache/opendal/blob/main/core/core/src/docs/rfcs/5485_conditional_reader.md)
+- [RFC 7818: Rename If Not Exists](https://github.com/apache/opendal/blob/main/core/core/src/docs/rfcs/7818_rename_if_not_exists.md)
+- [RFC 8145: Conflict Error Kind](https://github.com/apache/opendal/blob/main/core/core/src/docs/rfcs/8145_conflict_error_kind.md)
+- [RFC 8147: Write If Not Changed](https://github.com/apache/opendal/blob/main/core/core/src/docs/rfcs/8147_write_if_not_changed.md)
diff --git a/core/core/src/docs/performance/mod.rs b/core/core/src/docs/specs/mod.rs
similarity index 60%
copy from core/core/src/docs/performance/mod.rs
copy to core/core/src/docs/specs/mod.rs
index a817b84798..2e4bf4ecb2 100644
--- a/core/core/src/docs/performance/mod.rs
+++ b/core/core/src/docs/specs/mod.rs
@@ -15,18 +15,8 @@
// specific language governing permissions and limitations
// under the License.
-//! OpenDAL Performance Guide
-//!
-//! OpenDAL is designed in a zero-cost abstraction way, which means that
-//! users won't pay for the abstraction cost if they don't use it. But this
-//! can also means that users can't maximize the performance of OpenDAL
-//! if they don't know how to use it.
-//!
-//! This document will introduce some tips to improve the performance of
-//! OpenDAL.
+#![doc = include_str!("README.md")]
-#[doc = include_str!("concurrent_write.md")]
-pub mod concurrent_write {}
-
-#[doc = include_str!("http_optimization.md")]
-pub mod http_optimization {}
+/// The portable contract for conditional operations.
+#[doc = include_str!("conditional-operations.md")]
+pub mod conditional_operations {}
diff --git a/core/core/src/lib.rs b/core/core/src/lib.rs
index 688f4d799e..ba797d6cc7 100644
--- a/core/core/src/lib.rs
+++ b/core/core/src/lib.rs
@@ -31,6 +31,7 @@ pub use types::*;
// Public modules, they will be accessed like `opendal::layers::Xxxx`
#[cfg(feature = "blocking")]
pub mod blocking;
+pub mod docs;
pub mod layers;
pub mod raw;
pub mod services;
diff --git a/core/core/src/types/capability.rs b/core/core/src/types/capability.rs
index 8797a36dae..2e3e1c9fe5 100644
--- a/core/core/src/types/capability.rs
+++ b/core/core/src/types/capability.rs
@@ -45,6 +45,8 @@ use serde::Serialize;
/// - Blocking operations are provided through the BlockingLayer
///
/// Developers should use `capability` to determine available operations.
+/// Conditional capability fields follow the
+/// [conditional operation specification][crate::docs::specs::conditional_operations].
///
/// # Field Naming Conventions
///
diff --git a/core/core/src/types/delete/input.rs b/core/core/src/types/delete/input.rs
index 1fed1fe2c2..53c60c53f7 100644
--- a/core/core/src/types/delete/input.rs
+++ b/core/core/src/types/delete/input.rs
@@ -20,40 +20,40 @@ use crate::{Entry, Metadata};
/// Per-entry input for delete operations.
///
-/// Conditional fields follow the current-target predicate contract documented
-/// by [`crate::options::DeleteOptions`]. A missing target fails positive match
-/// conditions with [`crate::ErrorKind::ConditionNotMatch`] and satisfies negative
-/// match conditions as a successful no-op.
+/// Each conditional field checks the file at this entry's path and follows
+/// the same contract as the matching [`crate::options::DeleteOptions`] field;
+/// refer there for the full behavior and error semantics.
#[non_exhaustive]
#[derive(Default, Debug)]
pub struct DeleteInput {
- /// The path of the path to delete.
+ /// The path of the file to delete.
pub path: String,
- /// The version of the path to delete.
+ /// The version of the file to delete.
+ ///
+ /// Refer to [`crate::options::DeleteOptions::version`] for the full contract.
pub version: Option<String>,
/// Whether to perform recursive deletion.
pub recursive: bool,
- /// Delete only when the current ETag matches this value.
+ /// Delete only when the file at this path has this exact ETag.
///
- /// A different or missing object returns [`crate::ErrorKind::ConditionNotMatch`].
+ /// Refer to [`crate::options::DeleteOptions::if_match`] for the full contract.
pub if_match: Option<String>,
- /// Delete only when the current ETag does not match this value.
+ /// Delete only when the file at this path does not have this ETag.
///
- /// An equal ETag returns [`crate::ErrorKind::ConditionNotMatch`]. A different
- /// or missing object satisfies the condition.
+ /// Refer to [`crate::options::DeleteOptions::if_none_match`] for the full contract.
pub if_none_match: Option<String>,
- /// Delete only when the current version matches this value.
+ /// Delete only when the file at this path has this exact version.
///
- /// A different or missing object returns [`crate::ErrorKind::ConditionNotMatch`].
+ /// Refer to [`crate::options::DeleteOptions::if_version_match`] for the full contract.
pub if_version_match: Option<String>,
- /// Delete only when the current version does not match this value.
+ /// Delete only when the file at this path does not have this version.
///
- /// An equal version returns [`crate::ErrorKind::ConditionNotMatch`]. A
- /// different or missing object satisfies the condition.
+ /// Refer to [`crate::options::DeleteOptions::if_version_not_match`] for the
+ /// full contract.
pub if_version_not_match: Option<String>,
- /// Delete only when the object still matches this metadata.
+ /// Delete only when the file still has the identity recorded in this metadata.
///
- /// A changed or missing object returns [`crate::ErrorKind::ConditionNotMatch`].
+ /// Refer to [`crate::options::DeleteOptions::if_not_changed`] for the full contract.
pub if_not_changed: Option<Metadata>,
}
@@ -66,7 +66,7 @@ impl DeleteInput {
}
}
- /// Select the object version to delete.
+ /// Select the file version to delete.
pub fn with_version(mut self, version: impl Into<String>) -> Self {
self.version = Some(version.into());
self
@@ -78,31 +78,31 @@ impl DeleteInput {
self
}
- /// Delete only when the current ETag matches `etag`.
+ /// Delete only when the file has this exact `etag`.
pub fn with_if_match(mut self, etag: impl Into<String>) -> Self {
self.if_match = Some(etag.into());
self
}
- /// Delete only when the current ETag does not match `etag`.
+ /// Delete only when the file does not have this `etag`.
pub fn with_if_none_match(mut self, etag: impl Into<String>) -> Self {
self.if_none_match = Some(etag.into());
self
}
- /// Delete only when the current version matches `version`.
+ /// Delete only when the file has this exact `version`.
pub fn with_if_version_match(mut self, version: impl Into<String>) -> Self {
self.if_version_match = Some(version.into());
self
}
- /// Delete only when the current version does not match `version`.
+ /// Delete only when the file does not have this `version`.
pub fn with_if_version_not_match(mut self, version: impl Into<String>) -> Self {
self.if_version_not_match = Some(version.into());
self
}
- /// Delete only when the object still matches `metadata`.
+ /// Delete only when the file still has the identity recorded in `metadata`.
pub fn with_if_not_changed(mut self, metadata: &Metadata) -> Self {
self.if_not_changed = Some(metadata.clone());
self
diff --git a/core/core/src/types/error.rs b/core/core/src/types/error.rs
index 3f59d80f7f..ad4538d5c2 100644
--- a/core/core/src/types/error.rs
+++ b/core/core/src/types/error.rs
@@ -56,6 +56,11 @@ pub enum ErrorKind {
Unsupported,
/// The config for this service is invalid.
+ ///
+ /// OpenDAL also returns this kind for an invalid option value or
+ /// combination, for example `if_not_changed` metadata that contains no
+ /// usable version or ETag, or a restore `if_not_exists` without a
+ /// restore version.
ConfigInvalid,
/// The given path is not found.
NotFound,
@@ -73,8 +78,14 @@ pub enum ErrorKind {
IsSameFile,
/// A condition supplied through the OpenDAL operation evaluated to false.
///
- /// The condition can originate from options such as `if_match`, `if_none_match`,
- /// or `if_not_exists`.
+ /// Options such as `if_match`, `if_not_exists`, and `if_not_changed`
+ /// produce this error. It is returned only for conditions the caller
+ /// supplied: a service-side conflict unrelated to such a condition
+ /// returns [`ErrorKind::Conflict`], and observing a missing file with
+ /// `stat` or `read` returns [`ErrorKind::NotFound`] regardless of
+ /// conditions.
+ /// See the [conditional operation specification][crate::docs::specs::conditional_operations]
+ /// for its portable error semantics.
ConditionNotMatch,
/// The range of the content is not satisfied.
///
@@ -82,6 +93,10 @@ pub enum ErrorKind {
RangeNotSatisfied,
/// The operation conflicts with the current or transitional state of the resource.
///
+ /// This covers conflicts the service reports on its own; a false
+ /// condition supplied through OpenDAL options returns
+ /// [`ErrorKind::ConditionNotMatch`] instead.
+ ///
/// This error kind does not indicate whether retrying the same operation is safe.
/// Inspect the error's retry status for that decision.
Conflict,
diff --git a/core/core/src/types/operator/operator.rs b/core/core/src/types/operator/operator.rs
index 7c03753a80..3697c874aa 100644
--- a/core/core/src/types/operator/operator.rs
+++ b/core/core/src/types/operator/operator.rs
@@ -818,7 +818,7 @@ impl Operator {
///
/// # Notes
///
- /// Read the [concurrent write guide](https://github.com/apache/opendal/blob/main/core/core/src/docs/performance/concurrent_write.md)
+ /// Read the [concurrent write guide][crate::docs::performance::concurrent_write]
/// for more details.
///
/// ## Extra Options
@@ -860,7 +860,7 @@ impl Operator {
///
/// # Notes
///
- /// Read the [concurrent write guide](https://github.com/apache/opendal/blob/main/core/core/src/docs/performance/concurrent_write.md)
+ /// Read the [concurrent write guide][crate::docs::performance::concurrent_write]
/// for more details.
///
/// ## Streaming Write
@@ -916,7 +916,7 @@ impl Operator {
///
/// # Notes
///
- /// Read the [concurrent write guide](https://github.com/apache/opendal/blob/main/core/core/src/docs/performance/concurrent_write.md)
+ /// Read the [concurrent write guide][crate::docs::performance::concurrent_write]
/// for more details.
///
/// ## Streaming Write
@@ -1044,7 +1044,7 @@ impl Operator {
/// [Capability](crate::types::Capability). However, you can override this by explicitly
/// setting the `chunk` parameter.
///
- /// Read the [concurrent write guide](https://github.com/apache/opendal/blob/main/core/core/src/docs/performance/concurrent_write.md)
+ /// Read the [concurrent write guide][crate::docs::performance::concurrent_write]
/// for more details.
///
/// # Examples
@@ -1099,7 +1099,7 @@ impl Operator {
/// [Capability](crate::types::Capability). However, you can override this by explicitly
/// setting the `chunk` parameter.
///
- /// Read the [concurrent write guide](https://github.com/apache/opendal/blob/main/core/core/src/docs/performance/concurrent_write.md)
+ /// Read the [concurrent write guide][crate::docs::performance::concurrent_write]
/// for more details.
///
/// # Examples
@@ -1732,8 +1732,7 @@ impl Operator {
///
/// # Notes
///
- /// - Deleting a file that does not exist won't return errors unless a condition requires a
- /// live target.
+ /// - Deleting a file that does not exist is a successful no-op.
///
/// # Examples
///
@@ -1754,12 +1753,13 @@ impl Operator {
///
/// # Notes
///
- /// - Deleting a file that does not exist won't return errors unless a condition requires a
- /// live target.
+ /// - Deleting a file that does not exist is a successful no-op, unless a
+ /// condition such as `if_match` fails first.
///
/// # Options
///
- /// Visit [`options::DeleteOptions`] for all available options.
+ /// Visit [`options::DeleteOptions`] for all available options and the
+ /// conditional delete contract.
///
/// # Examples
///
@@ -1790,8 +1790,9 @@ impl Operator {
///
/// # Notes
///
- /// - Deleting a file that does not exist won't return errors unless a condition requires a
- /// live target.
+ /// - Deleting a file that does not exist is a successful no-op, unless a
+ /// condition such as `if_match` fails first. Visit
+ /// [`options::DeleteOptions`] for the conditional delete contract.
///
/// # Examples
///
diff --git a/core/core/src/types/operator/operator_futures.rs b/core/core/src/types/operator/operator_futures.rs
index 080f57f6e1..dd4e62ad6d 100644
--- a/core/core/src/types/operator/operator_futures.rs
+++ b/core/core/src/types/operator/operator_futures.rs
@@ -85,53 +85,57 @@ where
pub type FutureStat<F> = OperatorFuture<options::StatOptions, Metadata, F>;
impl<F: Future<Output = Result<Metadata>>> FutureStat<F> {
- /// Set the If-Match for this operation.
+ /// Stat only when the file has this exact ETag.
///
- /// Refer to [`options::StatOptions::if_match`] for more details.
+ /// Refer to [`options::StatOptions::if_match`] for the full contract.
pub fn if_match(mut self, v: &str) -> Self {
self.args.if_match = Some(v.to_string());
self
}
- /// Set the If-None-Match for this operation.
+ /// Stat only when the file does not have this ETag.
///
- /// Refer to [`options::StatOptions::if_none_match`] for more details.
+ /// Refer to [`options::StatOptions::if_none_match`] for the full contract.
pub fn if_none_match(mut self, v: &str) -> Self {
self.args.if_none_match = Some(v.to_string());
self
}
- /// Refer to [`options::StatOptions::if_version_match`] for more details.
+ /// Stat only when the file has this exact version.
+ ///
+ /// Refer to [`options::StatOptions::if_version_match`] for the full contract.
pub fn if_version_match(mut self, v: &str) -> Self {
self.args.if_version_match = Some(v.to_string());
self
}
- /// Refer to [`options::StatOptions::if_version_not_match`] for more details.
+ /// Stat only when the file does not have this version.
+ ///
+ /// Refer to [`options::StatOptions::if_version_not_match`] for the full contract.
pub fn if_version_not_match(mut self, v: &str) -> Self {
self.args.if_version_not_match = Some(v.to_string());
self
}
- /// Set the If-Modified-Since for this operation.
+ /// Stat only when the file was modified after this timestamp.
///
- /// Refer to [`options::StatOptions::if_modified_since`] for more details.
+ /// Refer to [`options::StatOptions::if_modified_since`] for the full contract.
pub fn if_modified_since(mut self, v: Timestamp) -> Self {
self.args.if_modified_since = Some(v);
self
}
- /// Set the If-Unmodified-Since for this operation.
+ /// Stat only when the file was not modified after this timestamp.
///
- /// Refer to [`options::StatOptions::if_unmodified_since`] for more details.
+ /// Refer to [`options::StatOptions::if_unmodified_since`] for the full contract.
pub fn if_unmodified_since(mut self, v: Timestamp) -> Self {
self.args.if_unmodified_since = Some(v);
self
}
- /// Set the version for this operation.
+ /// Stat the given version of the file instead of the current one.
///
- /// Refer to [`options::StatOptions::version`] for more details.
+ /// Refer to [`options::StatOptions::version`] for the full contract.
pub fn version(mut self, v: &str) -> Self {
self.args.version = Some(v.to_string());
self
@@ -163,13 +167,17 @@ impl<F: Future<Output = Result<PresignedRequest>>> FuturePresignStat<F> {
self
}
- /// Refer to [`options::StatOptions::if_match`] for more details.
+ /// Make the presigned request stat only when the file has this exact ETag.
+ ///
+ /// Refer to [`options::StatOptions::if_match`] for the full contract.
pub fn if_match(mut self, v: &str) -> Self {
self.args.0.if_match = Some(v.to_string());
self
}
- /// Refer to [`options::StatOptions::if_none_match`] for more details.
+ /// Make the presigned request stat only when the file does not have this ETag.
+ ///
+ /// Refer to [`options::StatOptions::if_none_match`] for the full contract.
pub fn if_none_match(mut self, v: &str) -> Self {
self.args.0.if_none_match = Some(v.to_string());
self
@@ -209,13 +217,17 @@ impl<F: Future<Output = Result<PresignedRequest>>> FuturePresignRead<F> {
self
}
- /// Refer to [`options::ReadOptions::if_match`] for more details.
+ /// Make the presigned request read only when the file has this exact ETag.
+ ///
+ /// Refer to [`options::ReadOptions::if_match`] for the full contract.
pub fn if_match(mut self, v: &str) -> Self {
self.args.0.if_match = Some(v.to_string());
self
}
- /// Refer to [`options::ReadOptions::if_none_match`] for more details.
+ /// Make the presigned request read only when the file does not have this ETag.
+ ///
+ /// Refer to [`options::ReadOptions::if_none_match`] for the full contract.
pub fn if_none_match(mut self, v: &str) -> Self {
self.args.0.if_none_match = Some(v.to_string());
self
@@ -330,11 +342,9 @@ impl<F: Future<Output = Result<Buffer>>> FutureRead<F> {
self
}
- /// Set `version` for this `read` request.
+ /// Read the given version of the file instead of the current one.
///
- /// This feature can be used to retrieve the data of a specified version of the given path.
- ///
- /// If the version doesn't exist, an error with kind [`ErrorKind::NotFound`] will be returned.
+ /// Refer to [`options::ReadOptions::version`] for the full contract.
///
/// ```
/// # use opendal_core::Result;
@@ -350,12 +360,9 @@ impl<F: Future<Output = Result<Buffer>>> FutureRead<F> {
self
}
- /// Set `if_match` for this `read` request.
- ///
- /// This feature can be used to check if the file's `ETag` matches the given `ETag`.
+ /// Read only when the file has this exact ETag.
///
- /// If file exists and it's etag doesn't match, an error with kind [`ErrorKind::ConditionNotMatch`]
- /// will be returned.
+ /// Refer to [`options::ReadOptions::if_match`] for the full contract.
///
/// ```
/// # use opendal_core::Result;
@@ -370,12 +377,9 @@ impl<F: Future<Output = Result<Buffer>>> FutureRead<F> {
self
}
- /// Set `if_none_match` for this `read` request.
- ///
- /// This feature can be used to check if the file's `ETag` doesn't match the given `ETag`.
+ /// Read only when the file does not have this ETag.
///
- /// If file exists and it's etag match, an error with kind [`ErrorKind::ConditionNotMatch`]
- /// will be returned.
+ /// Refer to [`options::ReadOptions::if_none_match`] for the full contract.
///
/// ```
/// # use opendal_core::Result;
@@ -390,26 +394,25 @@ impl<F: Future<Output = Result<Buffer>>> FutureRead<F> {
self
}
- /// Set the current object version that this read must match.
+ /// Read only when the file has this exact version.
+ ///
+ /// Refer to [`options::ReadOptions::if_version_match`] for the full contract.
pub fn if_version_match(mut self, v: &str) -> Self {
self.args.if_version_match = Some(v.to_string());
self
}
- /// Set the current object version that this read must not match.
+ /// Read only when the file does not have this version.
+ ///
+ /// Refer to [`options::ReadOptions::if_version_not_match`] for the full contract.
pub fn if_version_not_match(mut self, v: &str) -> Self {
self.args.if_version_not_match = Some(v.to_string());
self
}
- /// ## `if_modified_since`
- ///
- /// Set `if_modified_since` for this `read` request.
+ /// Read only when the file was modified after this timestamp.
///
- /// This feature can be used to check if the file has been modified since the given timestamp.
- ///
- /// If file exists and it hasn't been modified since the specified time, an error with kind
- /// [`ErrorKind::ConditionNotMatch`] will be returned.
+ /// Refer to [`options::ReadOptions::if_modified_since`] for the full contract.
///
/// ```
/// # use opendal_core::Result;
@@ -425,12 +428,9 @@ impl<F: Future<Output = Result<Buffer>>> FutureRead<F> {
self
}
- /// Set `if_unmodified_since` for this `read` request.
- ///
- /// This feature can be used to check if the file hasn't been modified since the given timestamp.
+ /// Read only when the file was not modified after this timestamp.
///
- /// If file exists and it has been modified since the specified time, an error with kind
- /// [`ErrorKind::ConditionNotMatch`] will be returned.
+ /// Refer to [`options::ReadOptions::if_unmodified_since`] for the full contract.
///
/// ```
/// # use opendal_core::Result;
@@ -460,11 +460,9 @@ impl<F: Future<Output = Result<Buffer>>> FutureRead<F> {
pub type FutureReader<F> = OperatorFuture<options::ReaderOptions, Reader, F>;
impl<F: Future<Output = Result<Reader>>> FutureReader<F> {
- /// Set `version` for this `reader`.
- ///
- /// This feature can be used to retrieve the data of a specified version of the given path.
+ /// Read the given version of the file instead of the current one.
///
- /// If the version doesn't exist, an error with kind [`ErrorKind::NotFound`] will be returned.
+ /// Refer to [`options::ReaderOptions::version`] for the full contract.
///
/// ```
/// # use opendal_core::Result;
@@ -564,12 +562,9 @@ impl<F: Future<Output = Result<Reader>>> FutureReader<F> {
self
}
- /// Set `if-match` for this `read` request.
+ /// Read only when the file has this exact ETag.
///
- /// This feature can be used to check if the file's `ETag` matches the given `ETag`.
- ///
- /// If file exists and it's etag doesn't match, an error with kind [`ErrorKind::ConditionNotMatch`]
- /// will be returned.
+ /// Refer to [`options::ReaderOptions::if_match`] for the full contract.
///
/// ```
/// # use opendal_core::Result;
@@ -584,12 +579,9 @@ impl<F: Future<Output = Result<Reader>>> FutureReader<F> {
self
}
- /// Set `if-none-match` for this `read` request.
- ///
- /// This feature can be used to check if the file's `ETag` doesn't match the given `ETag`.
+ /// Read only when the file does not have this ETag.
///
- /// If file exists and it's etag match, an error with kind [`ErrorKind::ConditionNotMatch`]
- /// will be returned.
+ /// Refer to [`options::ReaderOptions::if_none_match`] for the full contract.
///
/// ```
/// # use opendal_core::Result;
@@ -604,24 +596,25 @@ impl<F: Future<Output = Result<Reader>>> FutureReader<F> {
self
}
- /// Set the current object version that this reader must match.
+ /// Read only when the file has this exact version.
+ ///
+ /// Refer to [`options::ReaderOptions::if_version_match`] for the full contract.
pub fn if_version_match(mut self, version: &str) -> Self {
self.args.if_version_match = Some(version.to_string());
self
}
- /// Set the current object version that this reader must not match.
+ /// Read only when the file does not have this version.
+ ///
+ /// Refer to [`options::ReaderOptions::if_version_not_match`] for the full contract.
pub fn if_version_not_match(mut self, version: &str) -> Self {
self.args.if_version_not_match = Some(version.to_string());
self
}
- /// Set `if-modified-since` for this `read` request.
- ///
- /// This feature can be used to check if the file has been modified since the given timestamp.
+ /// Read only when the file was modified after this timestamp.
///
- /// If file exists and it hasn't been modified since the specified time, an error with kind
- /// [`ErrorKind::ConditionNotMatch`] will be returned.
+ /// Refer to [`options::ReaderOptions::if_modified_since`] for the full contract.
///
/// ```
/// # use opendal_core::Result;
@@ -640,12 +633,9 @@ impl<F: Future<Output = Result<Reader>>> FutureReader<F> {
self
}
- /// Set `if-unmodified-since` for this `read` request.
- ///
- /// This feature can be used to check if the file hasn't been modified since the given timestamp.
+ /// Read only when the file was not modified after this timestamp.
///
- /// If file exists and it has been modified since the specified time, an error with kind
- /// [`ErrorKind::ConditionNotMatch`] will be returned.
+ /// Refer to [`options::ReaderOptions::if_unmodified_since`] for the full contract.
///
/// ```
/// # use opendal_core::Result;
@@ -856,9 +846,9 @@ impl<F: Future<Output = Result<Metadata>>> FutureWrite<F> {
self
}
- /// Sets If-Match header for this write request.
+ /// Write only when the file at the write path has this exact ETag.
///
- /// Refer to [`options::WriteOptions::if_match`] for more details.
+ /// Refer to [`options::WriteOptions::if_match`] for the full contract.
///
/// ### Example
///
@@ -882,9 +872,9 @@ impl<F: Future<Output = Result<Metadata>>> FutureWrite<F> {
self
}
- /// Sets If-None-Match header for this write request.
+ /// Write only when the file at the write path does not have this ETag.
///
- /// Refer to [`options::WriteOptions::if_none_match`] for more details.
+ /// Refer to [`options::WriteOptions::if_none_match`] for the full contract.
///
/// ### Example
///
@@ -908,27 +898,33 @@ impl<F: Future<Output = Result<Metadata>>> FutureWrite<F> {
self
}
- /// Set the current object version that this write must match.
+ /// Write only when the file at the write path has this exact version.
+ ///
+ /// Refer to [`options::WriteOptions::if_version_match`] for the full contract.
pub fn if_version_match(mut self, version: &str) -> Self {
self.args.0.if_version_match = Some(version.to_string());
self
}
- /// Set the current object version that this write must not match.
+ /// Write only when the file at the write path does not have this version.
+ ///
+ /// Refer to [`options::WriteOptions::if_version_not_match`] for the full contract.
pub fn if_version_not_match(mut self, version: &str) -> Self {
self.args.0.if_version_not_match = Some(version.to_string());
self
}
- /// Write only when the object still matches `metadata`.
+ /// Write only when the file still has the identity recorded in `metadata`.
+ ///
+ /// Refer to [`options::WriteOptions::if_not_changed`] for the full contract.
pub fn if_not_changed(mut self, metadata: &Metadata) -> Self {
self.args.0.if_not_changed = Some(metadata.clone());
self
}
- /// Sets the condition that write operation will succeed only if target does not exist.
+ /// Write only when no file exists at the write path.
///
- /// Refer to [`options::WriteOptions::if_not_exists`] for more details.
+ /// Refer to [`options::WriteOptions::if_not_exists`] for the full contract.
///
/// ### Example
///
@@ -1193,21 +1189,11 @@ impl<F: Future<Output = Result<Writer>>> FutureWriter<F> {
self
}
- /// Sets If-Match header for this write request.
+ /// Write only when the file at the write path has this exact ETag.
///
- /// Refer to [`options::WriteOptions::if_match`] for more details.
- ///
- /// ### Behavior
- ///
- /// - If supported, the write operation will only succeed if the target's ETag matches the specified value
- /// - The value should be a valid ETag string
- /// - Common values include:
- /// - A specific ETag value like `"686897696a7c876b7e"`
- /// - `*` - Matches any existing resource
- /// - If not supported, the value will be ignored
- ///
- /// This operation provides conditional write functionality based on ETag matching,
- /// helping prevent unintended overwrites in concurrent scenarios.
+ /// Refer to [`options::WriteOptions::if_match`] for the full contract.
+ /// Depending on the service, the conditional error may surface when
+ /// creating the writer, while writing, or when closing it.
///
/// ### Example
///
@@ -1234,9 +1220,9 @@ impl<F: Future<Output = Result<Writer>>> FutureWriter<F> {
self
}
- /// Sets If-None-Match header for this write request.
+ /// Write only when the file at the write path does not have this ETag.
///
- /// Refer to [`options::WriteOptions::if_none_match`] for more details.
+ /// Refer to [`options::WriteOptions::if_none_match`] for the full contract.
///
/// ### Example
///
@@ -1263,27 +1249,33 @@ impl<F: Future<Output = Result<Writer>>> FutureWriter<F> {
self
}
- /// Set the current object version that this writer must match.
+ /// Write only when the file at the write path has this exact version.
+ ///
+ /// Refer to [`options::WriteOptions::if_version_match`] for the full contract.
pub fn if_version_match(mut self, version: &str) -> Self {
self.args.if_version_match = Some(version.to_string());
self
}
- /// Set the current object version that this writer must not match.
+ /// Write only when the file at the write path does not have this version.
+ ///
+ /// Refer to [`options::WriteOptions::if_version_not_match`] for the full contract.
pub fn if_version_not_match(mut self, version: &str) -> Self {
self.args.if_version_not_match = Some(version.to_string());
self
}
- /// Write only when the object still matches `metadata`.
+ /// Write only when the file still has the identity recorded in `metadata`.
+ ///
+ /// Refer to [`options::WriteOptions::if_not_changed`] for the full contract.
pub fn if_not_changed(mut self, metadata: &Metadata) -> Self {
self.args.if_not_changed = Some(metadata.clone());
self
}
- /// Sets the condition that write operation will succeed only if target does not exist.
+ /// Write only when no file exists at the write path.
///
- /// Refer to [`options::WriteOptions::if_not_exists`] for more details.
+ /// Refer to [`options::WriteOptions::if_not_exists`] for the full contract.
///
/// ### Example
///
@@ -1345,43 +1337,57 @@ impl<F: Future<Output = Result<Writer>>> FutureWriter<F> {
pub type FutureDelete<F> = OperatorFuture<options::DeleteOptions, (), F>;
impl<F: Future<Output = Result<()>>> FutureDelete<F> {
- /// Change the version of this delete operation.
+ /// Delete the given version of the file instead of the current one.
+ ///
+ /// Refer to [`options::DeleteOptions::version`] for the full contract.
pub fn version(mut self, v: &str) -> Self {
self.args.version = Some(v.to_string());
self
}
/// Enable recursive deletion.
+ ///
+ /// Refer to [`options::DeleteOptions::recursive`] for the full contract.
pub fn recursive(mut self, recursive: bool) -> Self {
self.args.recursive = recursive;
self
}
- /// Set `if_match` for this delete operation.
+ /// Delete only when the file at the delete path has this exact ETag.
+ ///
+ /// Refer to [`options::DeleteOptions::if_match`] for the full contract.
pub fn if_match(mut self, etag: &str) -> Self {
self.args.if_match = Some(etag.to_string());
self
}
- /// Set `if_none_match` for this delete operation.
+ /// Delete only when the file at the delete path does not have this ETag.
+ ///
+ /// Refer to [`options::DeleteOptions::if_none_match`] for the full contract.
pub fn if_none_match(mut self, etag: &str) -> Self {
self.args.if_none_match = Some(etag.to_string());
self
}
- /// Set the current object version that this delete must match.
+ /// Delete only when the file at the delete path has this exact version.
+ ///
+ /// Refer to [`options::DeleteOptions::if_version_match`] for the full contract.
pub fn if_version_match(mut self, version: &str) -> Self {
self.args.if_version_match = Some(version.to_string());
self
}
- /// Set the current object version that this delete must not match.
+ /// Delete only when the file at the delete path does not have this version.
+ ///
+ /// Refer to [`options::DeleteOptions::if_version_not_match`] for the full contract.
pub fn if_version_not_match(mut self, version: &str) -> Self {
self.args.if_version_not_match = Some(version.to_string());
self
}
- /// Delete only when the object still matches `metadata`.
+ /// Delete only when the file still has the identity recorded in `metadata`.
+ ///
+ /// Refer to [`options::DeleteOptions::if_not_changed`] for the full contract.
pub fn if_not_changed(mut self, metadata: &Metadata) -> Self {
self.args.if_not_changed = Some(metadata.clone());
self
@@ -1521,9 +1527,9 @@ impl<F: Future<Output = Result<Lister>>> FutureLister<F> {
pub type FutureCopy<F> = OperatorFuture<(options::CopyOptions, String), Metadata, F>;
impl<F: Future<Output = Result<Metadata>>> FutureCopy<F> {
- /// Sets the condition that copy operation will succeed only if target does not exist.
+ /// Copy only when no file exists at the destination path.
///
- /// Refer to [`options::CopyOptions::if_not_exists`] for more details.
+ /// Refer to [`options::CopyOptions::if_not_exists`] for the full contract.
///
/// ### Example
///
@@ -1544,42 +1550,49 @@ impl<F: Future<Output = Result<Metadata>>> FutureCopy<F> {
self
}
- /// Sets the condition that copy operation will succeed only if the
- /// destination object currently has the given ETag.
+ /// Copy only when the destination file has this exact ETag.
///
- /// Refer to [`options::CopyOptions::if_match`] for more details.
+ /// Refer to [`options::CopyOptions::if_match`] for the full contract.
pub fn if_match(mut self, etag: &str) -> Self {
self.args.0.if_match = Some(etag.to_string());
self
}
- /// Set the destination ETag that this copy must not match.
+ /// Copy only when the destination file does not have this ETag.
+ ///
+ /// Refer to [`options::CopyOptions::if_none_match`] for the full contract.
pub fn if_none_match(mut self, etag: &str) -> Self {
self.args.0.if_none_match = Some(etag.to_string());
self
}
- /// Set the current destination version that this copy must match.
+ /// Copy only when the destination file has this exact version.
+ ///
+ /// Refer to [`options::CopyOptions::if_version_match`] for the full contract.
pub fn if_version_match(mut self, version: &str) -> Self {
self.args.0.if_version_match = Some(version.to_string());
self
}
- /// Set the current destination version that this copy must not match.
+ /// Copy only when the destination file does not have this version.
+ ///
+ /// Refer to [`options::CopyOptions::if_version_not_match`] for the full contract.
pub fn if_version_not_match(mut self, version: &str) -> Self {
self.args.0.if_version_not_match = Some(version.to_string());
self
}
- /// Copy only when the destination still matches `metadata`.
+ /// Copy only when the destination file still has the identity recorded in `metadata`.
+ ///
+ /// Refer to [`options::CopyOptions::if_not_changed`] for the full contract.
pub fn if_not_changed(mut self, metadata: &Metadata) -> Self {
self.args.0.if_not_changed = Some(metadata.clone());
self
}
- /// Sets source version for this copy operation.
+ /// Copy from this source file version instead of the current one.
///
- /// Refer to [`options::CopyOptions::source_version`] for more details.
+ /// Refer to [`options::CopyOptions::source_version`] for the full contract.
pub fn source_version(mut self, version: impl Into<String>) -> Self {
self.args.0.source_version = Some(version.into());
self
@@ -1616,50 +1629,57 @@ impl<F: Future<Output = Result<Metadata>>> FutureCopy<F> {
pub type FutureCopier<F> = OperatorFuture<(options::CopyOptions, String), Copier, F>;
impl<F: Future<Output = Result<Copier>>> FutureCopier<F> {
- /// Sets the condition that copy operation will succeed only if target does not exist.
+ /// Copy only when no file exists at the destination path.
///
- /// Refer to [`options::CopyOptions::if_not_exists`] for more details.
+ /// Refer to [`options::CopyOptions::if_not_exists`] for the full contract.
pub fn if_not_exists(mut self, v: bool) -> Self {
self.args.0.if_not_exists = v;
self
}
- /// Sets the condition that copy operation will succeed only if the
- /// destination object currently has the given ETag.
+ /// Copy only when the destination file has this exact ETag.
///
- /// Refer to [`options::CopyOptions::if_match`] for more details.
+ /// Refer to [`options::CopyOptions::if_match`] for the full contract.
pub fn if_match(mut self, etag: &str) -> Self {
self.args.0.if_match = Some(etag.to_string());
self
}
- /// Set the destination ETag that this copier must not match.
+ /// Copy only when the destination file does not have this ETag.
+ ///
+ /// Refer to [`options::CopyOptions::if_none_match`] for the full contract.
pub fn if_none_match(mut self, etag: &str) -> Self {
self.args.0.if_none_match = Some(etag.to_string());
self
}
- /// Set the current destination version that this copier must match.
+ /// Copy only when the destination file has this exact version.
+ ///
+ /// Refer to [`options::CopyOptions::if_version_match`] for the full contract.
pub fn if_version_match(mut self, version: &str) -> Self {
self.args.0.if_version_match = Some(version.to_string());
self
}
- /// Set the current destination version that this copier must not match.
+ /// Copy only when the destination file does not have this version.
+ ///
+ /// Refer to [`options::CopyOptions::if_version_not_match`] for the full contract.
pub fn if_version_not_match(mut self, version: &str) -> Self {
self.args.0.if_version_not_match = Some(version.to_string());
self
}
- /// Copy only when the destination still matches `metadata`.
+ /// Copy only when the destination file still has the identity recorded in `metadata`.
+ ///
+ /// Refer to [`options::CopyOptions::if_not_changed`] for the full contract.
pub fn if_not_changed(mut self, metadata: &Metadata) -> Self {
self.args.0.if_not_changed = Some(metadata.clone());
self
}
- /// Sets source version for this copier operation.
+ /// Copy from this source file version instead of the current one.
///
- /// Refer to [`options::CopyOptions::source_version`] for more details.
+ /// Refer to [`options::CopyOptions::source_version`] for the full contract.
pub fn source_version(mut self, version: impl Into<String>) -> Self {
self.args.0.source_version = Some(version.into());
self
@@ -1696,9 +1716,9 @@ impl<F: Future<Output = Result<Copier>>> FutureCopier<F> {
pub type FutureRename<F> = OperatorFuture<(options::RenameOptions, String), (), F>;
impl<F: Future<Output = Result<()>>> FutureRename<F> {
- /// Sets the condition that rename operation will succeed only if target does not exist.
+ /// Rename only when no file exists at the destination path.
///
- /// Refer to [`options::RenameOptions::if_not_exists`] for more details.
+ /// Refer to [`options::RenameOptions::if_not_exists`] for the full contract.
///
/// ### Example
///
@@ -1725,13 +1745,17 @@ impl<F: Future<Output = Result<()>>> FutureRename<F> {
pub type FutureRestore<F> = OperatorFuture<options::RestoreOptions, (), F>;
impl<F: Future<Output = Result<()>>> FutureRestore<F> {
- /// Restore the specified historical version as the current version.
+ /// Restore this historical version as the current version.
+ ///
+ /// Refer to [`options::RestoreOptions::version`] for the full contract.
pub fn version(mut self, version: impl Into<String>) -> Self {
self.args.version = Some(version.into());
self
}
- /// Restore the selected version only if the path does not currently exist.
+ /// Restore the selected version only when no file currently exists at the restore path.
+ ///
+ /// Refer to [`options::RestoreOptions::if_not_exists`] for the full contract.
pub fn if_not_exists(mut self, if_not_exists: bool) -> Self {
self.args.if_not_exists = if_not_exists;
self
diff --git a/core/core/src/types/options.rs b/core/core/src/types/options.rs
index 8646c76afc..a13a63a25e 100644
--- a/core/core/src/types/options.rs
+++ b/core/core/src/types/options.rs
@@ -24,17 +24,32 @@ use std::collections::HashMap;
/// Options for delete operations.
///
-/// Delete conditions are predicates over the current live target and are
-/// combined with logical AND. A missing target makes positive match conditions
-/// (`if_match`, `if_version_match`, and `if_not_changed`) false, so the delete
-/// returns [`crate::ErrorKind::ConditionNotMatch`]. It makes negative match
-/// conditions (`if_none_match` and `if_version_not_match`) true, so the delete
-/// succeeds as a no-op. An unconditional delete of a missing target also
-/// succeeds. Only concrete ETag values have portable delete semantics;
-/// wildcard values such as `"*"` are service-specific.
+/// Each condition checks the file currently stored at the delete path, and
+/// every condition must hold for the delete to proceed:
+///
+/// - A false condition fails the delete with
+/// [`crate::ErrorKind::ConditionNotMatch`].
+/// - A condition the service does not advertise through its capability fails
+/// the delete with [`crate::ErrorKind::Unsupported`]; OpenDAL never
+/// silently drops a condition. A service that cannot preserve a
+/// combination of conditions also returns `Unsupported`.
+/// - A service-side conflict unrelated to these conditions surfaces as
+/// [`crate::ErrorKind::Conflict`].
+///
+/// Once every condition holds, deleting a missing file remains a successful
+/// no-op.
+///
+/// See the [conditional operation specification][crate::docs::specs::conditional_operations]
+/// for the complete cross-operation contract.
#[derive(Debug, Clone, Default, Eq, PartialEq)]
pub struct DeleteOptions {
- /// The version of the file to delete.
+ /// Delete the given version of the file instead of the current one.
+ ///
+ /// This selects which stored version the delete removes; it is not a
+ /// condition on the file.
+ ///
+ /// Check [`crate::Capability::delete_with_version`] before using this
+ /// option.
pub version: Option<String>,
/// Whether to delete the target recursively.
///
@@ -42,45 +57,66 @@ pub struct DeleteOptions {
/// - If `true`, all entries under the path (or sharing the prefix for file-like paths)
/// will be removed.
pub recursive: bool,
- /// Delete only when the current object's ETag matches this value.
- ///
- /// ### Capability
- ///
- /// Check [`crate::Capability::delete_with_if_match`] before using this feature.
+ /// Delete only when the file at the delete path has this exact ETag.
///
- /// ### Behavior
+ /// The condition succeeds when the file exists and its ETag equals this
+ /// value. A file with a different ETag, or a missing file, fails the
+ /// delete with [`crate::ErrorKind::ConditionNotMatch`]. Only concrete
+ /// ETag values are portable; a wildcard such as `"*"` has no portable
+ /// meaning here.
///
- /// - An existing object with a different ETag returns
- /// [`crate::ErrorKind::ConditionNotMatch`].
- /// - A missing object returns [`crate::ErrorKind::ConditionNotMatch`].
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise [`crate::Capability::delete_with_if_match`].
pub if_match: Option<String>,
- /// Delete only when the current object's ETag does not match this value.
- ///
- /// An existing object with the same ETag returns
- /// [`crate::ErrorKind::ConditionNotMatch`]. A different or missing object
- /// satisfies the condition; deleting a missing object is a no-op.
- ///
- /// Check [`crate::Capability::delete_with_if_none_match`] before using this feature.
+ /// Delete only when the file at the delete path does not have this ETag.
+ ///
+ /// With a concrete ETag value, the condition succeeds when the file has
+ /// a different ETag or does not exist; deleting a missing file is then a
+ /// successful no-op. A file whose ETag equals this value fails the
+ /// delete with [`crate::ErrorKind::ConditionNotMatch`]. Only concrete
+ /// ETag values are portable; a wildcard such as `"*"` has no portable
+ /// meaning here.
+ ///
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::delete_with_if_none_match`].
pub if_none_match: Option<String>,
- /// Delete only when the current object's version matches this value.
+ /// Delete only when the file at the delete path has this exact version.
///
- /// A different or missing object returns [`crate::ErrorKind::ConditionNotMatch`].
+ /// The condition succeeds when the file exists and its version equals
+ /// this value. A file with a different version, or a missing file, fails
+ /// the delete with [`crate::ErrorKind::ConditionNotMatch`].
///
- /// Check [`crate::Capability::delete_with_if_version_match`] before using this feature.
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::delete_with_if_version_match`].
pub if_version_match: Option<String>,
- /// Delete only when the current object's version does not match this value.
+ /// Delete only when the file at the delete path does not have this
+ /// version.
///
- /// An existing object with the same version returns
- /// [`crate::ErrorKind::ConditionNotMatch`]. A different or missing object
- /// satisfies the condition; deleting a missing object is a no-op.
+ /// The condition succeeds when the file exists with a different version.
+ /// A file whose version equals this value fails the delete with
+ /// [`crate::ErrorKind::ConditionNotMatch`]. When no file exists at the
+ /// delete path, no portable behavior is defined.
///
- /// Check [`crate::Capability::delete_with_if_version_not_match`] before using this feature.
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::delete_with_if_version_not_match`].
pub if_version_not_match: Option<String>,
- /// Delete only when the object still matches the supplied metadata.
- ///
- /// OpenDAL prefers `version` when the service supports version preconditions and
- /// otherwise uses `ETag`. A changed or missing object returns
- /// [`crate::ErrorKind::ConditionNotMatch`].
+ /// Delete only when the file at the delete path still has the identity
+ /// recorded in this metadata.
+ ///
+ /// Pass metadata previously returned by OpenDAL for the same path.
+ /// OpenDAL derives a version match when the service supports version
+ /// conditions and the metadata contains a version, and otherwise derives
+ /// an ETag match. A changed or missing file fails the delete with
+ /// [`crate::ErrorKind::ConditionNotMatch`], as does combining this option
+ /// with a conflicting `if_match` or `if_version_match` value.
+ ///
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service supports neither version nor ETag conditions, and
+ /// [`crate::ErrorKind::ConfigInvalid`] when a condition is supported but
+ /// the metadata contains no usable version or ETag.
pub if_not_changed: Option<Metadata>,
}
@@ -108,9 +144,17 @@ pub struct ListOptions {
/// Options for read operations.
///
-/// Conditional reads return [`crate::ErrorKind::NotFound`] when no live target
-/// exists. An existing target that fails a condition returns
-/// [`crate::ErrorKind::ConditionNotMatch`].
+/// Each condition checks the file being read, and every condition must hold
+/// for the read to proceed. A missing file fails the read with
+/// [`crate::ErrorKind::NotFound`] no matter which conditions are set. An
+/// existing file that fails a condition returns
+/// [`crate::ErrorKind::ConditionNotMatch`], and a condition the service does
+/// not advertise through its capability returns
+/// [`crate::ErrorKind::Unsupported`]; OpenDAL never silently drops a
+/// condition.
+///
+/// See the [conditional operation specification][crate::docs::specs::conditional_operations]
+/// for the complete cross-operation contract.
#[derive(Debug, Clone, Default, Eq, PartialEq)]
pub struct ReadOptions {
/// Set `range` for this operation.
@@ -124,51 +168,82 @@ pub struct ReadOptions {
///
/// The type implements `From<RangeBounds<u64>>`, so users can use `(1024..).into()` instead.
pub range: BytesRange,
- /// Set `version` for this operation.
+ /// Read the given version of the file instead of the current one.
///
- /// This option can be used to retrieve the data of a specified version of the given path.
+ /// This selects which stored version the read returns; it is not a
+ /// condition on the file. A version that does not exist returns
+ /// [`crate::ErrorKind::NotFound`].
///
- /// If the version doesn't exist, an error with kind [`crate::ErrorKind::NotFound`] will be returned.
+ /// Check [`crate::Capability::read_with_version`] before using this
+ /// option.
pub version: Option<String>,
- /// Set `if_match` for this operation.
+ /// Read only when the file has this exact ETag.
///
- /// This option can be used to check if the file's `ETag` matches the given `ETag`.
+ /// The condition succeeds when the file exists and its ETag equals this
+ /// value. A file with a different ETag returns
+ /// [`crate::ErrorKind::ConditionNotMatch`]; a missing file returns
+ /// [`crate::ErrorKind::NotFound`]. Only concrete ETag values are
+ /// portable; a wildcard such as `"*"` has no portable meaning here.
///
- /// If file exists and it's etag doesn't match, an error with kind [`crate::ErrorKind::ConditionNotMatch`]
- /// will be returned.
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise [`crate::Capability::read_with_if_match`].
pub if_match: Option<String>,
- /// Set `if_none_match` for this operation.
+ /// Read only when the file does not have this ETag.
///
- /// This option can be used to check if the file's `ETag` doesn't match the given `ETag`.
+ /// With a concrete ETag value, the condition succeeds when the file
+ /// exists with a different ETag. A file whose ETag equals this value
+ /// returns [`crate::ErrorKind::ConditionNotMatch`]; a missing file
+ /// returns [`crate::ErrorKind::NotFound`]. Only concrete ETag values
+ /// are portable; a wildcard such as `"*"` has no portable meaning here.
///
- /// If file exists and it's etag match, an error with kind [`crate::ErrorKind::ConditionNotMatch`]
- /// will be returned.
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::read_with_if_none_match`].
pub if_none_match: Option<String>,
- /// Read only when the current object's version matches this value.
+ /// Read only when the file has this exact version.
///
- /// A missing live target returns [`crate::ErrorKind::NotFound`]. An existing
- /// target with a different version returns
- /// [`crate::ErrorKind::ConditionNotMatch`].
+ /// The condition succeeds when the file exists and its version equals
+ /// this value. A file with a different version returns
+ /// [`crate::ErrorKind::ConditionNotMatch`]; a missing file returns
+ /// [`crate::ErrorKind::NotFound`].
+ ///
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::read_with_if_version_match`].
pub if_version_match: Option<String>,
- /// Read only when the current object's version does not match this value.
+ /// Read only when the file does not have this version.
+ ///
+ /// The condition succeeds when the file exists with a different version.
+ /// A file whose version equals this value returns
+ /// [`crate::ErrorKind::ConditionNotMatch`]; a missing file returns
+ /// [`crate::ErrorKind::NotFound`].
///
- /// A missing live target returns [`crate::ErrorKind::NotFound`]. An existing
- /// target with the same version returns [`crate::ErrorKind::ConditionNotMatch`].
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::read_with_if_version_not_match`].
pub if_version_not_match: Option<String>,
- /// Set `if_modified_since` for this operation.
+ /// Read only when the file was modified after this timestamp.
///
- /// This option can be used to check if the file has been modified since the given timestamp.
+ /// The condition succeeds when the file exists and was modified after
+ /// this time. A file not modified since this time returns
+ /// [`crate::ErrorKind::ConditionNotMatch`]; a missing file returns
+ /// [`crate::ErrorKind::NotFound`].
///
- /// If file exists and it hasn't been modified since the specified time, an error with kind
- /// [`crate::ErrorKind::ConditionNotMatch`] will be returned.
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::read_with_if_modified_since`].
pub if_modified_since: Option<Timestamp>,
- /// Set `if_unmodified_since` for this operation.
+ /// Read only when the file was not modified after this timestamp.
///
- /// This feature can be used to check if the file hasn't been modified since the given timestamp.
+ /// The condition succeeds when the file exists and was not modified
+ /// after this time. A file modified after this time returns
+ /// [`crate::ErrorKind::ConditionNotMatch`]; a missing file returns
+ /// [`crate::ErrorKind::NotFound`].
///
- /// If file exists and it has been modified since the specified time, an error with kind
- /// [`crate::ErrorKind::ConditionNotMatch`] will be returned.
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::read_with_if_unmodified_since`].
pub if_unmodified_since: Option<Timestamp>,
/// Known content length of the object.
@@ -231,56 +306,96 @@ pub struct ReadOptions {
/// Options for reader operations.
///
-/// Conditional reads return [`crate::ErrorKind::NotFound`] when no live target
-/// exists. An existing target that fails a condition returns
-/// [`crate::ErrorKind::ConditionNotMatch`].
+/// Each condition checks the file being read, and every condition must hold
+/// for the reader to produce data. A missing file fails with
+/// [`crate::ErrorKind::NotFound`] no matter which conditions are set. An
+/// existing file that fails a condition returns
+/// [`crate::ErrorKind::ConditionNotMatch`], and a condition the service does
+/// not advertise through its capability returns
+/// [`crate::ErrorKind::Unsupported`]; OpenDAL never silently drops a
+/// condition. Depending on the service, a conditional error may surface when
+/// creating the reader or while reading from it.
+///
+/// See the [conditional operation specification][crate::docs::specs::conditional_operations]
+/// for the complete cross-operation contract.
#[derive(Debug, Clone, Default, Eq, PartialEq)]
pub struct ReaderOptions {
- /// Set `version` for this operation.
+ /// Read the given version of the file instead of the current one.
///
- /// This option can be used to retrieve the data of a specified version of the given path.
+ /// This selects which stored version the reader returns; it is not a
+ /// condition on the file. A version that does not exist returns
+ /// [`crate::ErrorKind::NotFound`].
///
- /// If the version doesn't exist, an error with kind [`crate::ErrorKind::NotFound`] will be returned.
+ /// Check [`crate::Capability::read_with_version`] before using this
+ /// option.
pub version: Option<String>,
- /// Set `if_match` for this operation.
+ /// Read only when the file has this exact ETag.
///
- /// This option can be used to check if the file's `ETag` matches the given `ETag`.
+ /// The condition succeeds when the file exists and its ETag equals this
+ /// value. A file with a different ETag returns
+ /// [`crate::ErrorKind::ConditionNotMatch`]; a missing file returns
+ /// [`crate::ErrorKind::NotFound`]. Only concrete ETag values are
+ /// portable; a wildcard such as `"*"` has no portable meaning here.
///
- /// If file exists and it's etag doesn't match, an error with kind [`crate::ErrorKind::ConditionNotMatch`]
- /// will be returned.
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise [`crate::Capability::read_with_if_match`].
pub if_match: Option<String>,
- /// Set `if_none_match` for this operation.
+ /// Read only when the file does not have this ETag.
///
- /// This option can be used to check if the file's `ETag` doesn't match the given `ETag`.
+ /// With a concrete ETag value, the condition succeeds when the file
+ /// exists with a different ETag. A file whose ETag equals this value
+ /// returns [`crate::ErrorKind::ConditionNotMatch`]; a missing file
+ /// returns [`crate::ErrorKind::NotFound`]. Only concrete ETag values
+ /// are portable; a wildcard such as `"*"` has no portable meaning here.
///
- /// If file exists and it's etag match, an error with kind [`crate::ErrorKind::ConditionNotMatch`]
- /// will be returned.
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::read_with_if_none_match`].
pub if_none_match: Option<String>,
- /// Read only when the current object's version matches this value.
+ /// Read only when the file has this exact version.
///
- /// A missing live target returns [`crate::ErrorKind::NotFound`]. An existing
- /// target with a different version returns
- /// [`crate::ErrorKind::ConditionNotMatch`].
+ /// The condition succeeds when the file exists and its version equals
+ /// this value. A file with a different version returns
+ /// [`crate::ErrorKind::ConditionNotMatch`]; a missing file returns
+ /// [`crate::ErrorKind::NotFound`].
+ ///
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::read_with_if_version_match`].
pub if_version_match: Option<String>,
- /// Read only when the current object's version does not match this value.
+ /// Read only when the file does not have this version.
+ ///
+ /// The condition succeeds when the file exists with a different version.
+ /// A file whose version equals this value returns
+ /// [`crate::ErrorKind::ConditionNotMatch`]; a missing file returns
+ /// [`crate::ErrorKind::NotFound`].
///
- /// A missing live target returns [`crate::ErrorKind::NotFound`]. An existing
- /// target with the same version returns [`crate::ErrorKind::ConditionNotMatch`].
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::read_with_if_version_not_match`].
pub if_version_not_match: Option<String>,
- /// Set `if_modified_since` for this operation.
+ /// Read only when the file was modified after this timestamp.
///
- /// This option can be used to check if the file has been modified since the given timestamp.
+ /// The condition succeeds when the file exists and was modified after
+ /// this time. A file not modified since this time returns
+ /// [`crate::ErrorKind::ConditionNotMatch`]; a missing file returns
+ /// [`crate::ErrorKind::NotFound`].
///
- /// If file exists and it hasn't been modified since the specified time, an error with kind
- /// [`crate::ErrorKind::ConditionNotMatch`] will be returned.
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::read_with_if_modified_since`].
pub if_modified_since: Option<Timestamp>,
- /// Set `if_unmodified_since` for this operation.
+ /// Read only when the file was not modified after this timestamp.
///
- /// This feature can be used to check if the file hasn't been modified since the given timestamp.
+ /// The condition succeeds when the file exists and was not modified
+ /// after this time. A file modified after this time returns
+ /// [`crate::ErrorKind::ConditionNotMatch`]; a missing file returns
+ /// [`crate::ErrorKind::NotFound`].
///
- /// If file exists and it has been modified since the specified time, an error with kind
- /// [`crate::ErrorKind::ConditionNotMatch`] will be returned.
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::read_with_if_unmodified_since`].
pub if_unmodified_since: Option<Timestamp>,
/// Known content length of the object.
@@ -344,56 +459,95 @@ pub struct ReaderOptions {
/// Options for stat operations.
///
-/// Conditional stats return [`crate::ErrorKind::NotFound`] when no live target
-/// exists. An existing target that fails a condition returns
-/// [`crate::ErrorKind::ConditionNotMatch`].
+/// Each condition checks the file being observed, and every condition must
+/// hold for the stat to return metadata. A missing file fails the stat with
+/// [`crate::ErrorKind::NotFound`] no matter which conditions are set. An
+/// existing file that fails a condition returns
+/// [`crate::ErrorKind::ConditionNotMatch`], and a condition the service does
+/// not advertise through its capability returns
+/// [`crate::ErrorKind::Unsupported`]; OpenDAL never silently drops a
+/// condition.
+///
+/// See the [conditional operation specification][crate::docs::specs::conditional_operations]
+/// for the complete cross-operation contract.
#[derive(Debug, Clone, Default, Eq, PartialEq)]
pub struct StatOptions {
- /// Set `version` for this operation.
+ /// Stat the given version of the file instead of the current one.
///
- /// This options can be used to retrieve the data of a specified version of the given path.
+ /// This selects which stored version the stat describes; it is not a
+ /// condition on the file. A version that does not exist returns
+ /// [`crate::ErrorKind::NotFound`].
///
- /// If the version doesn't exist, an error with kind [`crate::ErrorKind::NotFound`] will be returned.
+ /// Check [`crate::Capability::stat_with_version`] before using this
+ /// option.
pub version: Option<String>,
- /// Set `if_match` for this operation.
+ /// Stat only when the file has this exact ETag.
///
- /// This option can be used to check if the file's `ETag` matches the given `ETag`.
+ /// The condition succeeds when the file exists and its ETag equals this
+ /// value. A file with a different ETag returns
+ /// [`crate::ErrorKind::ConditionNotMatch`]; a missing file returns
+ /// [`crate::ErrorKind::NotFound`]. Only concrete ETag values are
+ /// portable; a wildcard such as `"*"` has no portable meaning here.
///
- /// If file exists and it's etag doesn't match, an error with kind [`crate::ErrorKind::ConditionNotMatch`]
- /// will be returned.
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise [`crate::Capability::stat_with_if_match`].
pub if_match: Option<String>,
- /// Set `if_none_match` for this operation.
+ /// Stat only when the file does not have this ETag.
///
- /// This option can be used to check if the file's `ETag` doesn't match the given `ETag`.
+ /// With a concrete ETag value, the condition succeeds when the file
+ /// exists with a different ETag. A file whose ETag equals this value
+ /// returns [`crate::ErrorKind::ConditionNotMatch`]; a missing file
+ /// returns [`crate::ErrorKind::NotFound`]. Only concrete ETag values
+ /// are portable; a wildcard such as `"*"` has no portable meaning here.
///
- /// If file exists and it's etag match, an error with kind [`crate::ErrorKind::ConditionNotMatch`]
- /// will be returned.
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::stat_with_if_none_match`].
pub if_none_match: Option<String>,
- /// Stat only when the current object's version matches this value.
+ /// Stat only when the file has this exact version.
///
- /// A missing live target returns [`crate::ErrorKind::NotFound`]. An existing
- /// target with a different version returns
- /// [`crate::ErrorKind::ConditionNotMatch`].
+ /// The condition succeeds when the file exists and its version equals
+ /// this value. A file with a different version returns
+ /// [`crate::ErrorKind::ConditionNotMatch`]; a missing file returns
+ /// [`crate::ErrorKind::NotFound`].
+ ///
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::stat_with_if_version_match`].
pub if_version_match: Option<String>,
- /// Stat only when the current object's version does not match this value.
+ /// Stat only when the file does not have this version.
+ ///
+ /// The condition succeeds when the file exists with a different version.
+ /// A file whose version equals this value returns
+ /// [`crate::ErrorKind::ConditionNotMatch`]; a missing file returns
+ /// [`crate::ErrorKind::NotFound`].
///
- /// A missing live target returns [`crate::ErrorKind::NotFound`]. An existing
- /// target with the same version returns [`crate::ErrorKind::ConditionNotMatch`].
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::stat_with_if_version_not_match`].
pub if_version_not_match: Option<String>,
- /// Set `if_modified_since` for this operation.
+ /// Stat only when the file was modified after this timestamp.
///
- /// This option can be used to check if the file has been modified since the given timestamp.
+ /// The condition succeeds when the file exists and was modified after
+ /// this time. A file not modified since this time returns
+ /// [`crate::ErrorKind::ConditionNotMatch`]; a missing file returns
+ /// [`crate::ErrorKind::NotFound`].
///
- /// If file exists and it hasn't been modified since the specified time, an error with kind
- /// [`crate::ErrorKind::ConditionNotMatch`] will be returned.
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::stat_with_if_modified_since`].
pub if_modified_since: Option<Timestamp>,
- /// Set `if_unmodified_since` for this operation.
+ /// Stat only when the file was not modified after this timestamp.
///
- /// This feature can be used to check if the file hasn't been modified since the given timestamp.
+ /// The condition succeeds when the file exists and was not modified
+ /// after this time. A file modified after this time returns
+ /// [`crate::ErrorKind::ConditionNotMatch`]; a missing file returns
+ /// [`crate::ErrorKind::NotFound`].
///
- /// If file exists and it has been modified since the specified time, an error with kind
- /// [`crate::ErrorKind::ConditionNotMatch`] will be returned.
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::stat_with_if_unmodified_since`].
pub if_unmodified_since: Option<Timestamp>,
/// Specify the content-type header that should be sent back by the operation.
@@ -411,6 +565,23 @@ pub struct StatOptions {
}
/// Options for write operations.
+///
+/// Each condition checks the file currently stored at the write path, and
+/// the service evaluates it atomically with the write's visible commit:
+///
+/// - A false condition fails the write with
+/// [`crate::ErrorKind::ConditionNotMatch`] and leaves the previously
+/// visible file unchanged. Depending on the service, the error may surface
+/// when the write starts, while writing, or when closing the writer.
+/// - A condition the service does not advertise through its capability fails
+/// the write with [`crate::ErrorKind::Unsupported`]; OpenDAL never
+/// silently drops a condition. A service that cannot preserve a
+/// combination of conditions also returns `Unsupported`.
+/// - A service-side conflict unrelated to these conditions surfaces as
+/// [`crate::ErrorKind::Conflict`].
+///
+/// See the [conditional operation specification][crate::docs::specs::conditional_operations]
+/// for the complete cross-operation contract.
#[derive(Debug, Clone, Default, Eq, PartialEq)]
pub struct WriteOptions {
/// Sets append mode for this operation.
@@ -529,72 +700,79 @@ pub struct WriteOptions {
/// This metadata can be retrieved later when reading the object.
pub user_metadata: Option<HashMap<String, String>>,
- /// Sets If-Match header for this write request.
- ///
- /// ### Capability
- ///
- /// Check [`crate::Capability::write_with_if_match`] before using this feature.
- ///
- /// ### Behavior
+ /// Write only when the file at the write path has this exact ETag.
///
- /// - If supported, the write operation will only succeed if the target's ETag matches the specified value
- /// - The value should be a valid ETag string
- /// - Common values include:
- /// - A specific ETag value like `"686897696a7c876b7e"`
- /// - `*` - Matches any existing resource
- /// - If not supported, the value will be ignored
+ /// The condition succeeds when the file exists and its ETag equals this
+ /// value. A file with a different ETag, or a missing file, fails the
+ /// write with [`crate::ErrorKind::ConditionNotMatch`]. Only concrete
+ /// ETag values are portable; a wildcard such as `"*"` has no portable
+ /// meaning here.
///
- /// This operation provides conditional write functionality based on ETag matching,
- /// helping prevent unintended overwrites in concurrent scenarios.
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise [`crate::Capability::write_with_if_match`].
pub if_match: Option<String>,
- /// Sets If-None-Match header for this write request.
- ///
- /// Note: Certain services, like `s3`, support `if_not_exists` but not `if_none_match`.
- /// Use `if_not_exists` if you only want to check whether a file exists.
- ///
- /// ### Capability
- ///
- /// Check [`crate::Capability::write_with_if_none_match`] before using this feature.
- ///
- /// ### Behavior
- ///
- /// - If supported, the write operation will only succeed if the target's ETag does not match the specified value
- /// - The value should be a valid ETag string
- /// - Common values include:
- /// - A specific ETag value like `"686897696a7c876b7e"`
- /// - `*` - Matches if the resource does not exist
- /// - If not supported, the value will be ignored
- ///
- /// This operation provides conditional write functionality based on ETag non-matching,
- /// useful for preventing overwriting existing resources or ensuring unique writes.
+ /// Write only when the file at the write path does not have this ETag.
+ ///
+ /// With a concrete ETag value, the condition succeeds when the file has
+ /// a different ETag or does not exist; the write then replaces or
+ /// creates the file. A file whose ETag equals this value fails the write
+ /// with [`crate::ErrorKind::ConditionNotMatch`]. Only concrete ETag
+ /// values are portable; a wildcard such as `"*"` has no portable meaning
+ /// here.
+ ///
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::write_with_if_none_match`]. Some services, such
+ /// as `s3`, support `if_not_exists` but not `if_none_match`; use
+ /// [`WriteOptions::if_not_exists`] when you only need to guard against
+ /// an existing file.
pub if_none_match: Option<String>,
- /// Write only when the current object's version matches this value.
+ /// Write only when the file at the write path has this exact version.
///
- /// Check [`crate::Capability::write_with_if_version_match`] before using this feature.
- pub if_version_match: Option<String>,
- /// Write only when the current object's version does not match this value.
+ /// The condition succeeds when the file exists and its version equals
+ /// this value. A file with a different version, or a missing file, fails
+ /// the write with [`crate::ErrorKind::ConditionNotMatch`].
///
- /// Check [`crate::Capability::write_with_if_version_not_match`] before using this feature.
- pub if_version_not_match: Option<String>,
- /// Sets the condition that write operation will succeed only if target does not exist.
- ///
- /// ### Capability
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::write_with_if_version_match`].
+ pub if_version_match: Option<String>,
+ /// Write only when the file at the write path does not have this
+ /// version.
///
- /// Check [`crate::Capability::write_with_if_not_exists`] before using this feature.
+ /// The condition succeeds when the file exists with a different version.
+ /// A file whose version equals this value fails the write with
+ /// [`crate::ErrorKind::ConditionNotMatch`]. When no file exists at the
+ /// write path, no portable behavior is defined.
///
- /// ### Behavior
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::write_with_if_version_not_match`].
+ pub if_version_not_match: Option<String>,
+ /// Write only when no file exists at the write path.
///
- /// - If supported, the write operation will only succeed if the target path does not exist
- /// - Will return error if target already exists
- /// - If not supported, the value will be ignored
+ /// The condition succeeds when the path is empty, so the write creates
+ /// the file. An existing file fails the write with
+ /// [`crate::ErrorKind::ConditionNotMatch`].
///
- /// This operation provides a way to ensure write operations only create new resources
- /// without overwriting existing ones, useful for implementing "create if not exists" logic.
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::write_with_if_not_exists`].
pub if_not_exists: bool,
- /// Write only when the object still matches the supplied metadata.
- ///
- /// OpenDAL prefers `version` when the service supports version preconditions and
- /// otherwise uses `ETag`.
+ /// Write only when the file at the write path still has the identity
+ /// recorded in this metadata.
+ ///
+ /// Pass metadata previously returned by OpenDAL for the same path.
+ /// OpenDAL derives a version match when the service supports version
+ /// conditions and the metadata contains a version, and otherwise derives
+ /// an ETag match. A changed or missing file fails the write with
+ /// [`crate::ErrorKind::ConditionNotMatch`], as does combining this option
+ /// with a conflicting `if_match` or `if_version_match` value.
+ ///
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service supports neither version nor ETag conditions, and
+ /// [`crate::ErrorKind::ConfigInvalid`] when a condition is supported but
+ /// the metadata contains no usable version or ETag.
pub if_not_changed: Option<Metadata>,
/// Sets concurrent write operations for this writer.
@@ -651,65 +829,107 @@ pub struct WriteOptions {
}
/// Options for copy operations.
+///
+/// Each condition checks the destination file, never the source, and every
+/// condition must hold for the copy to proceed:
+///
+/// - A false condition fails the copy with
+/// [`crate::ErrorKind::ConditionNotMatch`].
+/// - A condition the service does not advertise through its capability fails
+/// the copy with [`crate::ErrorKind::Unsupported`]; OpenDAL never silently
+/// drops a condition.
+/// - A missing source fails the copy with [`crate::ErrorKind::NotFound`];
+/// when a condition error applies at the same time, which error the copy
+/// returns is unspecified.
+/// - A service-side conflict unrelated to these conditions surfaces as
+/// [`crate::ErrorKind::Conflict`].
+///
+/// See the [conditional operation specification][crate::docs::specs::conditional_operations]
+/// for the complete cross-operation contract.
#[derive(Debug, Clone, Default, Eq, PartialEq)]
pub struct CopyOptions {
- /// Sets the condition that copy operation will succeed only if target does not exist.
+ /// Copy only when no file exists at the destination path.
///
- /// ### Capability
- ///
- /// Check [`crate::Capability::copy_with_if_not_exists`] before using this feature.
- ///
- /// ### Behavior
- ///
- /// - If supported, the copy operation will only succeed if the target path does not exist
- /// - Will return error if target already exists
- /// - If not supported, the value will be ignored
+ /// The condition succeeds when the destination path is empty, so the
+ /// copy creates the destination file. An existing destination file fails
+ /// the copy with [`crate::ErrorKind::ConditionNotMatch`].
///
- /// This operation provides a way to ensure copy operations only create new resources
- /// without overwriting existing ones, useful for implementing "copy if not exists" logic.
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::copy_with_if_not_exists`].
pub if_not_exists: bool,
- /// Sets the condition that copy operation will succeed only if the destination
- /// currently has the given ETag.
+ /// Copy only when the destination file has this exact ETag.
///
- /// ### Capability
- ///
- /// Check [`crate::Capability::copy_with_if_match`] before using this feature.
+ /// The condition succeeds when the destination file exists and its ETag
+ /// equals this value. A destination file with a different ETag, or a
+ /// missing destination file, fails the copy with
+ /// [`crate::ErrorKind::ConditionNotMatch`]. Only concrete ETag values
+ /// are portable; a wildcard such as `"*"` has no portable meaning here.
///
- /// ### Behavior
- ///
- /// - If supported, the copy operation will only succeed when the existing
- /// destination object's ETag matches the given value.
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise [`crate::Capability::copy_with_if_match`].
pub if_match: Option<String>,
- /// Copy only when the destination ETag does not match this value.
- ///
- /// Check [`crate::Capability::copy_with_if_none_match`] before using this feature.
+ /// Copy only when the destination file does not have this ETag.
+ ///
+ /// With a concrete ETag value, the condition succeeds when the
+ /// destination file has a different ETag or does not exist; the copy
+ /// then replaces or creates the destination file. A destination file
+ /// whose ETag equals this value fails the copy with
+ /// [`crate::ErrorKind::ConditionNotMatch`]. Only concrete ETag values
+ /// are portable; a wildcard such as `"*"` has no portable meaning here.
+ ///
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::copy_with_if_none_match`].
pub if_none_match: Option<String>,
- /// Copy only when the current destination version matches this value.
+ /// Copy only when the destination file has this exact version.
///
- /// Check [`crate::Capability::copy_with_if_version_match`] before using this feature.
+ /// The condition succeeds when the destination file exists and its
+ /// version equals this value. A destination file with a different
+ /// version, or a missing destination file, fails the copy with
+ /// [`crate::ErrorKind::ConditionNotMatch`].
+ ///
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::copy_with_if_version_match`].
pub if_version_match: Option<String>,
- /// Copy only when the current destination version does not match this value.
+ /// Copy only when the destination file does not have this version.
///
- /// Check [`crate::Capability::copy_with_if_version_not_match`] before using this feature.
- pub if_version_not_match: Option<String>,
- /// Copy only when the destination still matches the supplied metadata.
+ /// The condition succeeds when the destination file exists with a
+ /// different version. A destination file whose version equals this value
+ /// fails the copy with [`crate::ErrorKind::ConditionNotMatch`]. When no
+ /// destination file exists, no portable behavior is defined.
///
- /// OpenDAL prefers `version` when the service supports version preconditions and
- /// otherwise uses `ETag`.
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::copy_with_if_version_not_match`].
+ pub if_version_not_match: Option<String>,
+ /// Copy only when the destination file still has the identity recorded
+ /// in this metadata.
+ ///
+ /// Pass metadata previously returned by OpenDAL for the destination
+ /// path. OpenDAL derives a version match when the service supports
+ /// version conditions and the metadata contains a version, and otherwise
+ /// derives an ETag match. A changed or missing destination file fails
+ /// the copy with [`crate::ErrorKind::ConditionNotMatch`], as does
+ /// combining this option with a conflicting `if_match` or
+ /// `if_version_match` value.
+ ///
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service supports neither version nor ETag conditions, and
+ /// [`crate::ErrorKind::ConfigInvalid`] when a condition is supported but
+ /// the metadata contains no usable version or ETag.
pub if_not_changed: Option<Metadata>,
- /// Copy from a specific source object version.
- ///
- /// ### Capability
+ /// Copy from a specific source file version.
///
- /// Check [`crate::Capability::copy_with_source_version`] before using this feature.
+ /// This selects which stored version of the source the copy reads; it is
+ /// not a condition on the destination file. Destination behavior follows
+ /// normal copy semantics.
///
- /// ### Behavior
- ///
- /// - If supported, the copy operation will read from the specified source
- /// version instead of the current source object.
- /// - Destination behavior follows normal copy semantics.
+ /// Check [`crate::Capability::copy_with_source_version`] before using
+ /// this option.
pub source_version: Option<String>,
/// Known content length of the source object.
@@ -740,42 +960,58 @@ pub struct CopyOptions {
}
/// Options for rename operations.
+///
+/// The condition checks the destination file, never the source. A missing
+/// source fails the rename with [`crate::ErrorKind::NotFound`]; when a
+/// condition error applies at the same time, which error the rename returns
+/// is unspecified.
+///
+/// See the [conditional operation specification][crate::docs::specs::conditional_operations]
+/// for the complete cross-operation contract.
#[derive(Debug, Clone, Default, Eq, PartialEq)]
pub struct RenameOptions {
- /// Sets the condition that rename operation will succeed only if target does not exist.
- ///
- /// ### Capability
- ///
- /// Check [`crate::Capability::rename_with_if_not_exists`] before using this feature.
+ /// Rename only when no file exists at the destination path.
///
- /// ### Behavior
+ /// The condition succeeds when the destination path is empty. An
+ /// existing destination file fails the rename with
+ /// [`crate::ErrorKind::ConditionNotMatch`].
///
- /// - If the target does not exist, the rename operation succeeds.
- /// - If the target exists, the operation returns [`crate::ErrorKind::ConditionNotMatch`].
- /// - If the service does not support this condition, the operation returns
- /// [`crate::ErrorKind::Unsupported`].
+ /// The operation returns [`crate::ErrorKind::Unsupported`] when the
+ /// service does not advertise
+ /// [`crate::Capability::rename_with_if_not_exists`].
pub if_not_exists: bool,
}
/// Options for restore operations.
+///
+/// The condition checks the file currently stored at the restore path. The
+/// selected version remains the source of the restore, and the restore may
+/// still fail because that version does not exist.
+///
+/// See the [conditional operation specification][crate::docs::specs::conditional_operations]
+/// for the complete cross-operation contract.
#[derive(Debug, Clone, Default, Eq, PartialEq)]
pub struct RestoreOptions {
- /// Restore a specific historical version as the current version.
+ /// Restore this historical version as the current version.
///
- /// ### Capability
+ /// This selects which stored version the restore promotes; it is not a
+ /// condition on the file at the restore path.
///
- /// Check [`crate::Capability::restore_with_version`] before using this feature.
+ /// Check [`crate::Capability::restore_with_version`] before using this
+ /// option.
pub version: Option<String>,
- /// Restore the selected version only if the path does not currently exist.
- ///
- /// This option requires [`RestoreOptions::version`] to be set. It protects
- /// recovery workflows from overwriting an object recreated after the version
- /// to restore was selected.
- ///
- /// ### Capability
- ///
- /// Check [`crate::Capability::restore_with_if_not_exists`] before using this
- /// feature.
+ /// Restore the selected version only when no file currently exists at
+ /// the restore path.
+ ///
+ /// The condition succeeds when the restore path is empty. An existing
+ /// file fails the restore with [`crate::ErrorKind::ConditionNotMatch`],
+ /// which protects recovery workflows from overwriting a file recreated
+ /// after the version to restore was selected.
+ ///
+ /// This option requires [`RestoreOptions::version`]; setting it without
+ /// a version returns [`crate::ErrorKind::ConfigInvalid`]. The operation
+ /// returns [`crate::ErrorKind::Unsupported`] when the service does not
+ /// advertise [`crate::Capability::restore_with_if_not_exists`].
pub if_not_exists: bool,
}
diff --git a/website/docs/01-overview.md b/website/docs/01-overview.md
index 047c2a90a2..3376766ff3 100644
--- a/website/docs/01-overview.md
+++ b/website/docs/01-overview.md
@@ -9,5 +9,6 @@ OpenDAL represents **Open** **D**ata **A**ccess **L**ayer. Our vision is [**One
-New to OpenDAL? Start with [Concepts](./03-concepts.mdx) — the four ideas
-behind every OpenDAL API: service, operator, layer, and operation.
+New to OpenDAL? Start with [Concepts](./03-concepts.mdx) — the model behind
+every OpenDAL API: shared storage semantics and consistent path and operation
+concepts, expressed through each binding's native API.
diff --git a/website/docs/03-concepts.mdx b/website/docs/03-concepts.mdx
index f0a144999d..5988f5b70f 100644
--- a/website/docs/03-concepts.mdx
+++ b/website/docs/03-concepts.mdx
@@ -1,6 +1,6 @@
---
sidebar_label: Concepts
-description: The four ideas behind every OpenDAL API — service, operator, layer, and operation.
+description: The shared model behind every OpenDAL API — the storage semantics and path and operation concepts every binding expresses through its own native API.
---
import MentalModel from "@site/src/components/concepts/MentalModel";
@@ -9,10 +9,15 @@ import { conceptSamples } from "@site/src/components/concepts/samples";
# Concepts
-Every OpenDAL API, in every language, follows one model:
+Every OpenDAL API is built on the same core model:
-**Configure a service, build an operator, optionally wrap it with layers, then
-call operations.**
+**One set of storage semantics, expressed through each binding's native
+API.** Every binding builds on the same contracts for how operations and
+paths behave, and maps them onto its own native types and lifecycle. Which
+features a binding exposes and how it represents errors are the binding's
+own; its documentation is the reference for both. The picture below shows
+the model as the Rust core defines it — an operator built from one
+configured service, with optional layers around it:
<MentalModel />
@@ -23,19 +28,28 @@ filesystem, an in-memory store — [more than 50 in total](/services).
You never talk to a service directly. You describe it with plain configuration
— bucket, root, endpoint, credentials — and OpenDAL builds the client for you.
-Because a service is only configuration, switching backends is a configuration
-change, not a code change: `s3` in production, `fs` on your laptop, `memory`
-in tests.
+Because a service is only configuration, switching backends is mostly a
+configuration change: `s3` in production, `fs` on your laptop, `memory` in
+tests. Each service has its own configuration keys and capabilities, though,
+so switching means supplying the new service's configuration and checking that
+it supports the features you rely on.
## Operator
-The operator is the entry point for everything. It is built from one service's
-configuration, and every storage call in your application goes through it.
-
-Operators are lightweight handles: cheap to create, and safe to share across
-threads and tasks. There are no connections to pool or cursors to manage. One
-operator maps to one service and one root path — to work with two buckets,
-build two operators.
+In the Rust core, the operator is the handle every operation goes through: it
+is built from one service's configuration, and every storage call is a method
+on it. Many bindings expose it publicly under the same name — Python, Node.js,
+and Java all have an `Operator` — but a binding may also wrap or rename it to
+fit its language: the Dart binding, for example, exposes `Storage`, `File`,
+and `Directory` objects and keeps the operator internal.
+
+Wherever the operator sits, the same rules apply. One operator maps to one
+service and one root path — to work with two buckets, build two operators (or
+two of your binding's client objects). An operator is designed to be created
+once and reused for many calls. How you construct, share, and release a
+storage handle — configuration style, sharing model, explicit close — follows
+each language's own conventions; your binding's documentation describes its
+lifecycle.
## Layer
@@ -43,23 +57,37 @@ A layer adds behavior around an operator: automatic retry, logging, timeouts,
metrics, concurrency limits.
Layers nest like an onion: every operation passes through every layer on the
-way in and on the way out. Cross-cutting concerns are composed once at startup
-instead of being scattered through your code. Applying a layer returns a new
-operator and leaves the original untouched.
+way in and on the way out, so cross-cutting concerns are composed once at
+startup instead of being scattered through your code. Applying a layer returns
+a new operator and leaves the original untouched.
+
+Layers live in the Rust core, and bindings differ in what they expose. Some,
+such as Python, Node.js, and Java, let you apply layers yourself; some apply a
+fixed policy such as retry internally; others do not expose layers yet. Check
+your binding's documentation for what is available.
## Operation
Operations are the verbs: `read`, `write`, `stat`, `list`, `delete`, `copy`,
-`rename`, and friends. They are the same on every service, so code written
-against one backend runs against any other.
+`rename`, and friends. An operation has the same meaning on every service, so
+code written against one backend runs against any other that supports the same
+features.
+
+Not every service supports every operation or optional feature. Each operator
+carries a **capability** set describing what its service can do, and an
+unsupported call fails with an explicit error instead of silently misbehaving.
+Where a binding exposes capability inspection, check for optional behavior —
+such as conditional writes or multipart uploads — before relying on it.
Paths follow one rule everywhere: they are always relative to the operator's
root, and a trailing `/` means a directory. `logs/app/` is a directory;
`logs/app` is a file.
-## One model, every language
+## The model in code
-The same four steps, in the language you already ship:
+The same model, written in bindings that expose the operator and layers
+directly. In a binding with a different surface — such as Dart's `Storage` and
+`File` — the same steps happen behind its own types:
<CodeTabs samples={conceptSamples} title="service → operator → layer → operation" />
diff --git a/website/docs/10-core/01-overview.md b/website/docs/10-core/01-overview.md
index 82ff091b7f..119427fbe3 100644
--- a/website/docs/10-core/01-overview.md
+++ b/website/docs/10-core/01-overview.md
@@ -12,7 +12,8 @@ most complete and most mature surface of the project: every service, every
layer, and the full async API live here first.
New to the model behind the API? Read [Concepts](../03-concepts.mdx) first —
-service, operator, layer, operation are the same four ideas in every language.
+the shared storage semantics and path and operation concepts every binding
+expresses through its own native API.
## Status
@@ -47,9 +48,11 @@ opendal = { version = "0.57", features = ["services-s3"] }
## Useful Links
- **API reference**: [docs.rs/opendal][docs.rs] (release) · [dev build][dev-docs]
+- **Specifications**: [current portable contracts][Specifications]
+- **Rust core guides**: [concepts, internals, and performance][Rust core guides] (rustdoc)
- **Services & configuration**: [/services](/services)
-- **Upgrade guide**: [docs.rs][Upgrade Guide]
-- **Changelog**: [docs.rs][Changelog]
+- **Upgrade guide**: [dev documentation][Upgrade Guide]
+- **Changelog**: [dev documentation][Changelog]
- **Source & examples**: [`core/`][source] · [`core/examples/`][examples]
## Next steps
@@ -61,7 +64,9 @@ opendal = { version = "0.57", features = ["services-s3"] }
[docs.rs]: https://docs.rs/opendal/
[dev-docs]: https://opendal.apache.org/docs/rust/opendal/
-[Upgrade Guide]: https://docs.rs/opendal/latest/opendal/docs/upgrade/index.html
-[Changelog]: https://docs.rs/opendal/latest/opendal/docs/changelog/index.html
+[Specifications]: /docs/specifications/
+[Rust core guides]: https://opendal.apache.org/docs/rust/opendal/docs/
+[Upgrade Guide]: https://opendal.apache.org/docs/rust/opendal/docs/upgrade/
+[Changelog]: https://opendal.apache.org/docs/rust/opendal/docs/changelog/
[source]: https://github.com/apache/opendal/tree/main/core
[examples]: https://github.com/apache/opendal/tree/main/core/examples
diff --git a/website/docs/10-core/04-tasks.md b/website/docs/10-core/04-tasks.md
index 14c53e958b..6363408bf3 100644
--- a/website/docs/10-core/04-tasks.md
+++ b/website/docs/10-core/04-tasks.md
@@ -50,7 +50,8 @@ if let Some(etag) = meta.etag() {
If the ETag no longer matches between `stat` and `read_with`, the read returns
`ErrorKind::ConditionNotMatch`. `read_with` and `reader_with` also support
`if_none_match`, `if_modified_since`, and `if_unmodified_since`. The same
-conditions are available on `stat_with` for conditional metadata checks.
+conditions are available on `stat_with` for conditional metadata checks. See
+[Conditional operations] for the portable condition and error contract.
## Stream a large file
@@ -97,7 +98,9 @@ let _meta = op
```
A false condition returns `ErrorKind::ConditionNotMatch`. `if_none_match` is
-also available on services that advertise `write_with_if_none_match`.
+also available on services that advertise `write_with_if_none_match`. See
+[Conditional operations] for atomicity semantics and the behavior when the
+file does not exist.
## Stream a large upload
@@ -218,7 +221,8 @@ op.rename_with("old.txt", "new.txt")
Check `copy_with_if_not_exists` or `rename_with_if_not_exists` before using
these conditions. If the target exists, the operation returns
-`ErrorKind::ConditionNotMatch`.
+`ErrorKind::ConditionNotMatch`. Conditions guard the destination; see
+[Conditional operations] for the complete contract.
## Generate a presigned URL
@@ -236,3 +240,4 @@ let req = op.presign_read("path/to/file", Duration::from_secs(3600)).await?;
[`Buffer`]: https://docs.rs/opendal/latest/opendal/struct.Buffer.html
[`Writer`]: https://docs.rs/opendal/latest/opendal/struct.Writer.html
[`Lister`]: https://docs.rs/opendal/latest/opendal/struct.Lister.html
+[Conditional operations]: /docs/specifications/conditional-operations/
diff --git a/website/docs/20-bindings/c/01-overview.md b/website/docs/20-bindings/c/01-overview.md
index 7954fff870..aa16ad85d1 100644
--- a/website/docs/20-bindings/c/01-overview.md
+++ b/website/docs/20-bindings/c/01-overview.md
@@ -12,7 +12,8 @@ the local filesystem, and [50+ more services](/services) through one API backed
the Rust core.
New to the model behind the API? Read [Concepts](../../03-concepts.mdx) first —
-service, operator, layer, operation are the same four ideas in every language.
+the shared storage semantics and path and operation concepts every binding
+expresses through its own native API.
:::caution Experimental
This binding is **experimental and a work in progress**. The API surface is limited,
diff --git a/website/docs/20-bindings/cpp/01-overview.md b/website/docs/20-bindings/cpp/01-overview.md
index 6aaee5d63d..b574c00206 100644
--- a/website/docs/20-bindings/cpp/01-overview.md
+++ b/website/docs/20-bindings/cpp/01-overview.md
@@ -12,7 +12,8 @@ filesystem, and [50+ more services](/services) through one API, backed by the
Rust core.
New to the model behind the API? Read [Concepts](../../03-concepts.mdx) first —
-service, operator, layer, and operation are the same four ideas in every language.
+the shared storage semantics and path and operation concepts every binding
+expresses through its own native API.
:::warning Experimental
This binding is **experimental and a work in progress**. There is no published
diff --git a/website/docs/20-bindings/d/01-overview.md b/website/docs/20-bindings/d/01-overview.md
index dae1be3575..974a826fff 100644
--- a/website/docs/20-bindings/d/01-overview.md
+++ b/website/docs/20-bindings/d/01-overview.md
@@ -11,7 +11,8 @@ A D language binding for OpenDAL, wrapping the [C binding](../c) to give D progr
access to the same unified storage API as every other OpenDAL binding.
New to the model behind the API? Read [Concepts](../../03-concepts.mdx) first —
-service, operator, layer, and operation are the same four ideas in every language.
+the shared storage semantics and path and operation concepts every binding
+expresses through its own native API.
:::warning
This binding is **early-stage and experimental**. It is not yet released or
diff --git a/website/docs/20-bindings/dart/01-overview.md b/website/docs/20-bindings/dart/01-overview.md
index 7fd3e4c743..fa1e872178 100644
--- a/website/docs/20-bindings/dart/01-overview.md
+++ b/website/docs/20-bindings/dart/01-overview.md
@@ -15,7 +15,8 @@ The API follows the `dart:io` stdlib style — `Storage`, `File`, and `Directory
objects that should feel familiar to any Dart developer.
New to the model behind the API? Read [Concepts](../../03-concepts.mdx) first —
-service, operator, layer, and operation are the same four ideas in every language.
+the shared storage semantics and path and operation concepts every binding
+expresses through its own native API.
:::caution Experimental
This binding is a work in progress. The public API may change between releases.
diff --git a/website/docs/20-bindings/dotnet/01-overview.md b/website/docs/20-bindings/dotnet/01-overview.md
index a4f230a22a..bdf153a712 100644
--- a/website/docs/20-bindings/dotnet/01-overview.md
+++ b/website/docs/20-bindings/dotnet/01-overview.md
@@ -12,7 +12,8 @@ filesystem, and [50+ more services](/services) through one API, with the
performance of the Rust core underneath.
New to the model behind the API? Read [Concepts](../../03-concepts.mdx) first —
-service, operator, layer, operation are the same four ideas in every language.
+the shared storage semantics and path and operation concepts every binding
+expresses through its own native API.
:::note
This binding has its own version number, independent of the Rust core. Check
diff --git a/website/docs/20-bindings/go/01-overview.md b/website/docs/20-bindings/go/01-overview.md
index abe7417be3..4e5fbf5318 100644
--- a/website/docs/20-bindings/go/01-overview.md
+++ b/website/docs/20-bindings/go/01-overview.md
@@ -14,7 +14,8 @@ performance of the Rust core underneath. It is built on `opendal-c` via
required.
New to the model behind the API? Read [Concepts](../../03-concepts.mdx) first —
-service, operator, layer, operation are the same four ideas in every language.
+the shared storage semantics and path and operation concepts every binding
+expresses through its own native API.
:::note
This binding has its own version number, independent of the Rust core. Check
diff --git a/website/docs/20-bindings/haskell/01-overview.md b/website/docs/20-bindings/haskell/01-overview.md
index 6b5a04f9ec..5072cb4a4d 100644
--- a/website/docs/20-bindings/haskell/01-overview.md
+++ b/website/docs/20-bindings/haskell/01-overview.md
@@ -11,7 +11,8 @@ A Haskell binding for OpenDAL: access S3, GCS, Azure Blob, the local filesystem,
and [50+ more services](/services) through one API, with the Rust core underneath.
New to the model behind the API? Read [Concepts](../../03-concepts.mdx) first —
-service, operator, layer, operation are the same four ideas in every language.
+the shared storage semantics and path and operation concepts every binding
+expresses through its own native API.
:::note
This binding has its own version number, independent of the Rust core. When
diff --git a/website/docs/20-bindings/java/01-overview.md b/website/docs/20-bindings/java/01-overview.md
index 44b95cf11a..405c0382ad 100644
--- a/website/docs/20-bindings/java/01-overview.md
+++ b/website/docs/20-bindings/java/01-overview.md
@@ -12,7 +12,8 @@ filesystem, and [50+ more services](/services) through one API, with the
performance of the Rust core underneath.
New to the model behind the API? Read [Concepts](../../03-concepts.mdx) first —
-service, operator, layer, operation are the same four ideas in every language.
+the shared storage semantics and path and operation concepts every binding
+expresses through its own native API.
:::note
This binding has its own version number, independent of the Rust core. Check
diff --git a/website/docs/20-bindings/lua/01-overview.md b/website/docs/20-bindings/lua/01-overview.md
index c6e5cc9dc9..be39158edf 100644
--- a/website/docs/20-bindings/lua/01-overview.md
+++ b/website/docs/20-bindings/lua/01-overview.md
@@ -11,7 +11,8 @@ A Lua binding for OpenDAL: access the local filesystem, in-memory storage, S3,
and [50+ more services](/services) through one API, backed by the Rust core.
New to the model behind the API? Read [Concepts](../../03-concepts.mdx) first —
-service, operator, layer, operation are the same four ideas in every language.
+the shared storage semantics and path and operation concepts every binding
+expresses through its own native API.
:::warning Experimental / WIP
This binding is **unreleased and experimental**. APIs may change without notice.
diff --git a/website/docs/20-bindings/nodejs/01-overview.md b/website/docs/20-bindings/nodejs/01-overview.md
index 9412a5b3fd..ada400aaf6 100644
--- a/website/docs/20-bindings/nodejs/01-overview.md
+++ b/website/docs/20-bindings/nodejs/01-overview.md
@@ -12,7 +12,8 @@ filesystem, and [50+ more services](/services) through one API, with the
performance of the Rust core underneath.
New to the model behind the API? Read [Concepts](../../03-concepts.mdx) first —
-service, operator, layer, operation are the same four ideas in every language.
+the shared storage semantics and path and operation concepts every binding
+expresses through its own native API.
:::note
This binding has its own version number, independent of the Rust core. Check
diff --git a/website/docs/20-bindings/ocaml/01-overview.md b/website/docs/20-bindings/ocaml/01-overview.md
index 5e16a8e0b9..e154c13031 100644
--- a/website/docs/20-bindings/ocaml/01-overview.md
+++ b/website/docs/20-bindings/ocaml/01-overview.md
@@ -12,7 +12,8 @@ filesystem, and [50+ more services](/services) through one API, backed by the
Rust core.
New to the model behind the API? Read [Concepts](../../03-concepts.mdx) first —
-service, operator, layer, and operation are the same four ideas in every language.
+the shared storage semantics and path and operation concepts every binding
+expresses through its own native API.
:::warning Experimental / Work in Progress
This binding is **unreleased and experimental**. APIs may change without notice.
diff --git a/website/docs/20-bindings/php/01-overview.md b/website/docs/20-bindings/php/01-overview.md
index 0755017b90..f0552c0cca 100644
--- a/website/docs/20-bindings/php/01-overview.md
+++ b/website/docs/20-bindings/php/01-overview.md
@@ -12,7 +12,8 @@ filesystem, and [50+ more services](/services) through one synchronous API,
with the performance of the Rust core underneath.
New to the model behind the API? Read [Concepts](../../03-concepts.mdx) first —
-service, operator, layer, and operation are the same four ideas in every language.
+the shared storage semantics and path and operation concepts every binding
+expresses through its own native API.
## Status
diff --git a/website/docs/20-bindings/python/01-overview.md b/website/docs/20-bindings/python/01-overview.md
index 047d67e7b6..2694f1ff66 100644
--- a/website/docs/20-bindings/python/01-overview.md
+++ b/website/docs/20-bindings/python/01-overview.md
@@ -12,7 +12,8 @@ filesystem, and [50+ more services](/services) through one API, with the
performance of the Rust core underneath.
New to the model behind the API? Read [Concepts](../../03-concepts.mdx) first —
-service, operator, layer, operation are the same four ideas in every language.
+the shared storage semantics and path and operation concepts every binding
+expresses through its own native API.
:::note
This binding has its own version number, independent of the Rust core. Check
diff --git a/website/docs/20-bindings/ruby/01-overview.md b/website/docs/20-bindings/ruby/01-overview.md
index d276b3ae72..ed258777ec 100644
--- a/website/docs/20-bindings/ruby/01-overview.md
+++ b/website/docs/20-bindings/ruby/01-overview.md
@@ -12,7 +12,8 @@ filesystem, and [50+ more services](/services) through one API, with the
performance of the Rust core underneath.
New to the model behind the API? Read [Concepts](../../03-concepts.mdx) first —
-service, operator, layer, operation are the same four ideas in every language.
+the shared storage semantics and path and operation concepts every binding
+expresses through its own native API.
:::note
This binding has its own version number, independent of the Rust core. Check
diff --git a/website/docs/20-bindings/swift/01-overview.md b/website/docs/20-bindings/swift/01-overview.md
index bb8a1787ed..83c5eee607 100644
--- a/website/docs/20-bindings/swift/01-overview.md
+++ b/website/docs/20-bindings/swift/01-overview.md
@@ -11,7 +11,8 @@ A Swift binding for OpenDAL: access storage services through one API, backed
by the Rust core.
New to the model behind the API? Read [Concepts](../../03-concepts.mdx) first —
-service, operator, and operation are the same ideas in every language.
+the shared storage semantics and path and operation concepts every binding
+expresses through its own native API.
:::warning Experimental / WIP
This binding is unreleased and under active development. The API surface is
diff --git a/website/docs/20-bindings/zig/01-overview.md b/website/docs/20-bindings/zig/01-overview.md
index 49acedd31c..c166d3855c 100644
--- a/website/docs/20-bindings/zig/01-overview.md
+++ b/website/docs/20-bindings/zig/01-overview.md
@@ -12,7 +12,8 @@ the local filesystem, and [50+ more services](/services) through one API backed
Rust core.
New to the model behind the API? Read [Concepts](../../03-concepts.mdx) first —
-service, operator, layer, operation are the same four ideas in every language.
+the shared storage semantics and path and operation concepts every binding
+expresses through its own native API.
:::caution Early-stage / Experimental
This binding is **early-stage and a work in progress**. The API is incomplete,
diff --git a/website/docs/sidebars.js b/website/docs/sidebars.js
index 545294a2d9..8e1cecc1e5 100644
--- a/website/docs/sidebars.js
+++ b/website/docs/sidebars.js
@@ -45,6 +45,11 @@ const sidebars = {
'overview',
'vision',
'concepts',
+ {
+ type: 'link',
+ label: 'Specifications',
+ href: '/docs/specifications/',
+ },
// Divider between the concept docs and the per-language guides.
{
type: 'html',
diff --git a/website/docusaurus.config.js b/website/docusaurus.config.js
index 227ea683be..cede27cb43 100644
--- a/website/docusaurus.config.js
+++ b/website/docusaurus.config.js
@@ -112,6 +112,19 @@ const config = {
],
plugins: [
+ [
+ "@docusaurus/plugin-content-docs",
+ {
+ id: "specifications",
+ path: "../core/core/src/docs/specs",
+ routeBasePath: "docs/specifications",
+ sidebarPath: require.resolve("./specifications.sidebars.js"),
+ editUrl: ({ docPath }) =>
+ `${repoAddress}/edit/main/core/core/src/docs/specs/${docPath}`,
+ showLastUpdateAuthor: true,
+ showLastUpdateTime: true,
+ },
+ ],
[
"@docusaurus/plugin-content-docs",
{
diff --git a/website/specifications.sidebars.js b/website/specifications.sidebars.js
new file mode 100644
index 0000000000..fbc85f1b4e
--- /dev/null
+++ b/website/specifications.sidebars.js
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+// @ts-check
+
+/** @type {import('@docusaurus/plugin-content-docs').SidebarsConfig} */
+const sidebars = {
+ specs: [
+ {
+ type: "doc",
+ id: "README",
+ label: "Overview",
+ },
+ {
+ type: "doc",
+ id: "conditional-operations",
+ label: "Conditional operations",
+ },
+ ],
+};
+
+module.exports = sidebars;
diff --git a/website/src/components/concepts/MentalModel.jsx b/website/src/components/concepts/MentalModel.jsx
index 126e0f129d..0cf99e95e8 100644
--- a/website/src/components/concepts/MentalModel.jsx
+++ b/website/src/components/concepts/MentalModel.jsx
@@ -21,12 +21,14 @@ import React, { useState } from "react";
import CodeWindow from "../ui/CodeWindow";
import styles from "./styles.module.css";
-// The OpenDAL mental model in one picture: an application calls an operator,
-// the operator is wrapped by layers, and the operator talks to one configured
-// service. Selecting a concept highlights its place in the diagram; the
-// one-line hint lives in the window title bar. The diagram is decorative
-// (aria-hidden) — the page prose carries the same information for assistive
-// technology.
+// The OpenDAL mental model in one picture, as the Rust core defines it: an
+// application calls an operator, optional layers wrap the operator, and the
+// operator talks to one configured service. Bindings map this model onto
+// their own types — some rename or hide the operator, and not all expose
+// layers; the page prose carries those caveats. Selecting a concept
+// highlights its place in the diagram; the one-line hint lives in the window
+// title bar. The diagram is decorative (aria-hidden) — the page prose carries
+// the same information for assistive technology.
const CONCEPTS = [
{
id: "service",
@@ -44,13 +46,13 @@ const CONCEPTS = [
id: "layer",
index: "03",
name: "layer",
- hint: "wraps the operator, sees every call",
+ hint: "optional wrapper, sees every call",
},
{
id: "operation",
index: "04",
name: "operation",
- hint: "the same verbs on every backend",
+ hint: "the same verbs, bounded by capabilities",
},
];
diff --git a/website/src/components/concepts/samples.js b/website/src/components/concepts/samples.js
index f80e492f38..2d5eb1654d 100644
--- a/website/src/components/concepts/samples.js
+++ b/website/src/components/concepts/samples.js
@@ -18,8 +18,9 @@
*/
// The four-step mental model — service, operator, layer, operation — written
-// out in each binding's real API so the concepts page never drifts from the
-// libraries it describes.
+// out in bindings whose public API exposes the operator and layers directly,
+// so the concepts page never drifts from the libraries it describes. Bindings
+// with a different surface (for example Dart) are covered by the page prose.
export const conceptSamples = [
{
id: "rust",
(opendal) 02/02: Merge remote-tracking branch 'origin/main' into xuanwo/specifications
Posted by xu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
Xuanwo pushed a commit to branch xuanwo/specifications
in repository https://gitbox.apache.org/repos/asf/opendal.git
commit 65c19c7a263576c0bf47cc83fde5ba6e394b2299
Merge: 341b2c6a49 6e921175e5
Author: Xuanwo <gi...@xuanwo.io>
AuthorDate: Fri Aug 28 13:50:37 2026 +0800
Merge remote-tracking branch 'origin/main' into xuanwo/specifications
.../services/gcs_grpc/gcs_grpc/action.yml | 23 +-
.../OpenDAL/ServiceConfig/GcsGrpcServiceConfig.cs | 129 ++++
.../java/org/apache/opendal/ServiceConfig.java | 95 +++
bindings/python/Cargo.toml | 2 +
bindings/python/benchmark/README.md | 19 +
.../python/benchmark/blocking_gil_benchmark.py | 156 +++++
bindings/python/python/opendal/config.py | 31 +
bindings/python/python/opendal/services.pyi | 1 +
bindings/python/src/file.rs | 50 +-
bindings/python/src/operator.rs | 88 ++-
bindings/python/src/services.rs | 4 +
bindings/python/src/utils.rs | 59 --
bindings/python/tests/test_read.py | 39 ++
bindings/python/tests/test_sync_gil.py | 152 +++++
core/Cargo.lock | 23 +
core/Cargo.toml | 2 +
core/core/src/blocking/read/std_reader.rs | 25 +
core/core/src/types/read/futures_async_reader.rs | 116 ++++
core/services/aliyun-drive/src/core.rs | 195 ++++--
core/services/aliyun-drive/src/reader.rs | 5 +-
core/services/alluxio/src/core.rs | 193 +++---
core/services/alluxio/src/reader.rs | 7 +-
core/services/azdls/src/backend.rs | 19 +-
core/services/azdls/src/core.rs | 213 +++---
core/services/azdls/src/deleter.rs | 16 +-
core/services/azdls/src/lister.rs | 7 +-
core/services/azdls/src/reader.rs | 7 +-
core/services/azdls/src/writer.rs | 35 +-
core/services/azfile/src/backend.rs | 21 +-
core/services/azfile/src/core.rs | 168 ++---
core/services/azfile/src/deleter.rs | 9 +-
core/services/azfile/src/lister.rs | 7 +-
core/services/azfile/src/reader.rs | 7 +-
core/services/azfile/src/writer.rs | 22 +-
core/services/b2/src/backend.rs | 7 +-
core/services/b2/src/core.rs | 236 ++++---
core/services/b2/src/deleter.rs | 2 +-
core/services/b2/src/lister.rs | 7 +-
core/services/b2/src/reader.rs | 7 +-
core/services/b2/src/writer.rs | 27 +-
core/services/cloudflare-kv/src/backend.rs | 7 +-
core/services/cloudflare-kv/src/core.rs | 118 ++--
core/services/cloudflare-kv/src/deleter.rs | 15 +-
core/services/cloudflare-kv/src/lister.rs | 7 +-
core/services/cloudflare-kv/src/reader.rs | 12 +-
core/services/cloudflare-kv/src/writer.rs | 7 +-
core/services/cos/src/backend.rs | 10 +-
core/services/cos/src/core.rs | 160 ++---
core/services/cos/src/deleter.rs | 5 +-
core/services/cos/src/lister.rs | 10 +-
core/services/cos/src/reader.rs | 7 +-
core/services/cos/src/writer.rs | 35 +-
core/services/d1/src/core.rs | 136 ++--
core/services/dbfs/src/backend.rs | 17 +-
core/services/dbfs/src/core.rs | 115 ++--
core/services/dbfs/src/deleter.rs | 5 +-
core/services/dbfs/src/lister.rs | 4 +-
core/services/dbfs/src/writer.rs | 7 +-
core/services/dropbox/src/backend.rs | 9 +-
core/services/dropbox/src/core.rs | 126 ++--
core/services/dropbox/src/deleter.rs | 2 +-
core/services/dropbox/src/lister.rs | 9 +-
core/services/dropbox/src/reader.rs | 5 +-
core/services/dropbox/src/writer.rs | 7 +-
core/services/etcd/src/core.rs | 16 +-
core/services/fs/src/core.rs | 47 +-
core/services/gcs-grpc/Cargo.toml | 58 ++
core/services/gcs-grpc/README.md | 19 +
core/services/gcs-grpc/proto/README.md | 23 +
.../gcs-grpc/proto/google/storage/v2/storage.proto | 171 +++++
.../gcs-grpc/proto/upstream-descriptor.bin | Bin 0 -> 61045 bytes
core/services/gcs-grpc/src/backend.rs | 447 ++++++++++++
core/services/gcs-grpc/src/config.rs | 116 ++++
core/services/gcs-grpc/src/copier.rs | 92 +++
core/services/gcs-grpc/src/core.rs | 249 +++++++
core/services/gcs-grpc/src/deleter.rs | 62 ++
core/services/gcs-grpc/src/docs.md | 19 +
.../gcs-grpc/src/generated/google.storage.v2.rs | 541 +++++++++++++++
.../deleter.rs => gcs-grpc/src/generated/mod.rs} | 33 +-
.../{ipmfs/src/deleter.rs => gcs-grpc/src/lib.rs} | 50 +-
core/services/gcs-grpc/src/lister.rs | 123 ++++
core/services/gcs-grpc/src/reader.rs | 131 ++++
core/services/gcs-grpc/src/writer.rs | 748 +++++++++++++++++++++
core/services/gcs/src/core.rs | 15 +
core/services/gcs/src/deleter.rs | 60 +-
core/services/gdrive/src/backend.rs | 27 +-
core/services/gdrive/src/core.rs | 106 +--
core/services/gdrive/src/deleter.rs | 10 +-
core/services/gdrive/src/lister.rs | 26 +-
core/services/gdrive/src/reader.rs | 12 +-
core/services/gdrive/src/writer.rs | 9 +-
core/services/ghac/src/backend.rs | 5 +-
core/services/ghac/src/core.rs | 96 ++-
core/services/ghac/src/reader.rs | 7 +-
core/services/ghac/src/writer.rs | 5 +-
core/services/github/src/backend.rs | 12 +-
core/services/github/src/core.rs | 164 ++---
core/services/github/src/reader.rs | 7 +-
core/services/github/src/writer.rs | 7 +-
core/services/goosefs/src/core.rs | 410 ++++++-----
core/services/goosefs/src/reader.rs | 8 +-
core/services/goosefs/src/writer.rs | 19 +-
core/services/hdfs-native/src/core.rs | 76 +--
core/services/hf/src/core.rs | 256 +++----
core/services/hf/src/lister.rs | 20 +-
core/services/http/src/backend.rs | 8 +-
core/services/http/src/core.rs | 77 ++-
core/services/http/src/reader.rs | 7 +-
core/services/ipfs/src/backend.rs | 7 +-
core/services/ipfs/src/core.rs | 48 +-
core/services/ipfs/src/reader.rs | 7 +-
core/services/ipmfs/src/backend.rs | 12 +-
core/services/ipmfs/src/core.rs | 126 ++--
core/services/ipmfs/src/deleter.rs | 7 +-
core/services/ipmfs/src/lister.rs | 4 +-
core/services/ipmfs/src/reader.rs | 7 +-
core/services/ipmfs/src/writer.rs | 7 +-
core/services/koofr/src/backend.rs | 27 +-
core/services/koofr/src/core.rs | 108 +--
core/services/koofr/src/deleter.rs | 5 +-
core/services/koofr/src/lister.rs | 7 +-
core/services/koofr/src/reader.rs | 7 +-
core/services/koofr/src/writer.rs | 7 +-
core/services/lakefs/src/backend.rs | 12 +-
core/services/lakefs/src/core.rs | 126 ++--
core/services/lakefs/src/deleter.rs | 5 +-
core/services/lakefs/src/lister.rs | 4 +-
core/services/lakefs/src/reader.rs | 7 +-
core/services/lakefs/src/writer.rs | 12 +-
core/services/obs/src/backend.rs | 12 +-
core/services/obs/src/core.rs | 161 +++--
core/services/obs/src/deleter.rs | 5 +-
core/services/obs/src/lister.rs | 5 +-
core/services/obs/src/reader.rs | 7 +-
core/services/obs/src/writer.rs | 35 +-
core/services/onedrive/src/backend.rs | 9 +-
core/services/onedrive/src/core.rs | 178 +++--
core/services/onedrive/src/deleter.rs | 7 +-
core/services/onedrive/src/lister.rs | 7 +-
core/services/onedrive/src/reader.rs | 7 +-
core/services/onedrive/src/writer.rs | 19 +-
core/services/opfs/src/core.rs | 70 +-
core/services/oss/src/backend.rs | 10 +-
core/services/oss/src/core.rs | 157 +++--
core/services/oss/src/deleter.rs | 10 +-
core/services/oss/src/lister.rs | 10 +-
core/services/oss/src/reader.rs | 7 +-
core/services/oss/src/writer.rs | 35 +-
core/services/pcloud/src/backend.rs | 23 +-
core/services/pcloud/src/core.rs | 110 +--
core/services/pcloud/src/deleter.rs | 9 +-
core/services/pcloud/src/lister.rs | 5 +-
core/services/pcloud/src/reader.rs | 7 +-
core/services/pcloud/src/writer.rs | 7 +-
core/services/persy/src/core.rs | 75 ++-
core/services/seafile/src/core.rs | 168 +++--
core/services/seafile/src/reader.rs | 7 +-
core/services/seafile/src/writer.rs | 7 +-
core/services/sftp/src/core.rs | 72 +-
core/services/sled/src/core.rs | 34 +-
core/services/swift/src/backend.rs | 10 +-
core/services/swift/src/core.rs | 128 ++--
core/services/swift/src/deleter.rs | 10 +-
core/services/swift/src/lister.rs | 2 +-
core/services/swift/src/reader.rs | 7 +-
core/services/swift/src/writer.rs | 17 +-
core/services/tikv/src/core.rs | 43 +-
core/services/tos/src/backend.rs | 5 +-
core/services/tos/src/copier.rs | 30 +-
core/services/tos/src/core.rs | 217 +++---
core/services/tos/src/deleter.rs | 10 +-
core/services/tos/src/lister.rs | 10 +-
core/services/tos/src/reader.rs | 7 +-
core/services/tos/src/writer.rs | 25 +-
core/services/upyun/src/backend.rs | 20 +-
core/services/upyun/src/core.rs | 146 ++--
core/services/upyun/src/deleter.rs | 5 +-
core/services/upyun/src/lister.rs | 9 +-
core/services/upyun/src/reader.rs | 7 +-
core/services/upyun/src/writer.rs | 22 +-
core/services/vercel-artifacts/src/backend.rs | 7 +-
core/services/vercel-artifacts/src/core.rs | 59 +-
core/services/vercel-artifacts/src/reader.rs | 7 +-
core/services/vercel-artifacts/src/writer.rs | 7 +-
core/services/vercel-blob/src/backend.rs | 12 +-
core/services/vercel-blob/src/core.rs | 144 ++--
core/services/vercel-blob/src/reader.rs | 7 +-
core/services/vercel-blob/src/writer.rs | 22 +-
core/services/webdav/src/backend.rs | 10 +-
core/services/webdav/src/core.rs | 137 ++--
core/services/webdav/src/deleter.rs | 5 +-
core/services/webdav/src/lister.rs | 5 +-
core/services/webdav/src/reader.rs | 55 +-
core/services/webdav/src/writer.rs | 10 +-
core/services/webhdfs/src/backend.rs | 19 +-
core/services/webhdfs/src/core.rs | 193 +++---
core/services/webhdfs/src/deleter.rs | 7 +-
core/services/webhdfs/src/lister.rs | 16 +-
core/services/webhdfs/src/reader.rs | 7 +-
core/services/webhdfs/src/writer.rs | 49 +-
core/services/yandex-disk/src/backend.rs | 15 +-
core/services/yandex-disk/src/core.rs | 153 +++--
core/services/yandex-disk/src/deleter.rs | 5 +-
core/services/yandex-disk/src/lister.rs | 7 +-
core/services/yandex-disk/src/reader.rs | 7 +-
core/services/yandex-disk/src/writer.rs | 7 +-
core/src/lib.rs | 5 +
dev/Cargo.lock | 264 +++++++-
dev/Cargo.toml | 3 +
dev/src/generate/gcs_grpc.rs | 266 ++++++++
dev/src/generate/mod.rs | 6 +
justfile | 2 +-
website/data/services.json | 106 +++
213 files changed, 8527 insertions(+), 3114 deletions(-)