(opendal-yinyang) 01/01: Merge foundation 8ccf8ff into the writable layer

[xu...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

Xuanwo pushed a commit to branch xuanwo/linux-fuse-writable
in repository https://gitbox.apache.org/repos/asf/opendal-yinyang.git

commit 00487906de01260a7d73ccd27456b2aa1c5506b4
Merge: 88cf08c 8ccf8ff
Author: Xuanwo <gi...@xuanwo.io>
AuthorDate: Mon Oct 26 18:34:43 2026 +0800

    Merge foundation 8ccf8ff into the writable layer
    
    Keep shared staged-attribute lookup semantics while recording positive
    lookup dentries for explicit refresh. Retain writable callback and contract
    documentation alongside the notifier-backed SIGHUP lifecycle.
    
    Co-authored-by: Chojan Shang <ps...@apache.org>

 Cargo.lock           |   1 +
 Cargo.toml           |   1 +
 specs/linux-fuse.md  |  46 ++++++++++++--
 src/file_cli.rs      |   6 +-
 src/fuse/frontend.rs | 165 +++++++++++++++++++++++++++++++++++++++++++++++++--
 src/fuse/inode.rs    |   6 ++
 src/fuse/mod.rs      |  85 +++++++++++++++++---------
 7 files changed, 271 insertions(+), 39 deletions(-)

diff --cc specs/linux-fuse.md
index 0eb2698,ea8024b..31e29c0
--- a/specs/linux-fuse.md
+++ b/specs/linux-fuse.md
@@@ -91,55 -89,43 +91,90 @@@ version remains coherent until a later 
  that open's metadata and content never disagree. The frontend does not call
  `Mount::refresh` while references exist, because refresh rebases live handles;
  a writable layer that in-place refreshes must first define its open-fd and
 -kernel-cache invalidation policy.
 +kernel-cache invalidation policy. Before a new open or a path attribute, the
 +adapter retries reclaiming a clean unreferenced NodeId; a successful reclaim
 +makes the next observation use the latest remote version, while Busy retains
 +all still-pinned or dirty/frozen/error state.
 +
 +## Writable callbacks
 +
 +`write` stages bytes in the shared per-NodeId Mount handle. Each request uses
 +the file flags supplied by the kernel, so `fcntl(F_SETFL)` may toggle
 +`O_APPEND` and `O_SYNC`/`O_DSYNC` for later writes. An append request uses the
 +node's append path, which chooses its offset under the same per-node lock as
 +other writes; a regular write uses the kernel offset. The writable access mode
 +is fixed by the open request. `O_TRUNC` stages a zero length before the open
 +reply. A write with `O_SYNC`/`O_DSYNC` in its current flags attempts
 +publication before acknowledging the write. Unsupported write flags return an
 +explicit error instead of silently changing semantics.
 +
 +`flush` publishes a writable handle and reports the real remote outcome to the
 +closing descriptor. `release` never publishes and never discards staging.
 +Flush or fsync on a read-only reference is a successful no-op even when another
 +writable reference shares the node, so a reader close cannot publish somebody
 +else's dirty bytes. A failed publication remains attached to the shared handle
 +and volume ledger; later fsync/flush retries the original frozen request.
 +
 +`create` and `mkdir` use the Runtime helpers that return the NodeId produced by
 +the committing Planner, avoiding a post-commit path lookup. `unlink`, `rmdir`,
 +and path renames use the path-addressed Runtime methods, so lookup and commit
 +share one guarded Snapshot. Plain `rename` replaces an existing destination;
 +`RENAME_NOREPLACE` uses the no-replace form. Exchange and whiteout renames are
 +explicitly unsupported.
 +
 +`setattr` supports size and the core executable flag only. Size on an open
 +descriptor is staged for that descriptor's later flush/fsync; path truncation
 +publishes before returning success. A mode request is accepted only when it is
 +one of the synthesized modes above (`0444`/`0555` or `0644`/`0755` according to
 +read-only policy and the executable bit); other permission combinations return
 +`EOPNOTSUPP`. Linux may include `S_IFREG`/`S_IFDIR` type bits in the mode;
 +they are accepted when they match the node, while setuid/setgid/sticky and
 +other permission combinations are rejected. Kernel bookkeeping timestamps
 +attached to a chmod or truncate request are accepted and ignored, and the reply
 +always carries revision-derived timestamps. Standalone timestamp changes and
 +ownership changes remain explicitly unsupported. Before changing the
 +executable flag, the shared node lock
 +publishes dirty content, commits a Planner based on the handle's recorded
 +content revision, and advances only that clean handle to the receipt revision.
 +An external content change therefore produces Conflict rather than rebasing
 +old staged bytes onto new content. Ownership, timestamps and other mode bits
 +are not persisted.
  
+ ## Explicit refresh
+ 
+ `SIGHUP` triggers one explicit refresh; this is a manual invalidation entry
+ point, not an automatic change watch. There is no background polling, no
+ change-feed watcher, and no whole-tree enumeration. The refresh observes the
+ latest snapshot once and then:
+ 
+ - reclaims every clean, unreferenced node through the shared `Mount` contract,
+   so the next open materializes the current remote revision; nodes with live
+   references or retained state are left pinned (`Busy`);
+ - invalidates known inode attributes with `inval_inode(ino, -1, 0)` and
+   replied dentries with `inval_entry(parent, name)` through the fuser
+   `Notifier`. Only dentries this adapter actually replied to the kernel are
+   recorded: successful `LOOKUP` results. Negative lookups carry no timeout in
+   FUSE replies and are not recorded, and plain `READDIR` does not install
+   dentries without `READDIRPLUS`, so directory names are not recorded.
+ 
+ The adapter never calls `Mount::refresh`, because that replaces the shared
+ staged handle and would rebase open file descriptors. The refresh must also
+ hold no adapter lock while reclaiming nodes or sending notifications; targets
+ are collected under short locks, then the sends run on a blocking thread. At
+ most one refresh runs at a time and duplicate `SIGHUP`s while one is running
+ are coalesced by the signal stream. The SIGHUP loop is a tracked Tokio task:
+ the signal handler is registered synchronously before the mount becomes
+ visible, teardown sends it a stop signal, and the session join and the refresh
+ task share the same five-second grace period; a refresh already in progress
+ may finish inside that window. A `NotFound` notification result is expected
+ (the kernel already dropped the entry or inode) and is counted separately, so
+ only explicit connection-ended errors (`ENODEV`, `ENOTCONN`, `EPIPE`,
+ `ECONNRESET`, `ESHUTDOWN`) stop the batch before it would otherwise finish.
+ 
+ Stable `yy: fuse refresh:` log lines report the observed revision, the
+ reclaimed/active node counts, and the native notification counts at
+ completion, so an acceptance harness can confirm a refresh finished.
+ 
  ## Pinned directory cursors
  
  `opendir` pins one `Snapshot` revision and one directory identity. `readdir`
@@@ -202,17 -190,15 +237,16 @@@ there is no mid-session cancellation si
  - Every open materializes the complete file through the existing shared Mount
    staging contract; there is no partial-download or mmap coherence claim, and
    `FOPEN_DIRECT_IO` does not provide page-cache-backed mmap behavior.
 -- Writes, create, mkdir, unlink, rename, links, xattrs, ioctl, poll, locking
 -  and capacity accounting are not implemented; the kernel read-only mount and
 -  the callback errno policy reject them.
 +- Hard links, symlinks, mknod, xattrs, ioctl, poll, fallocate and capacity
 +  accounting are not implemented and return explicit `EOPNOTSUPP`/`ENOSYS`.
 +- The frontend does not negotiate `FUSE_POSIX_LOCKS` or `FLOCK_LOCKS`, and its
 +  getlk/setlk/flock callbacks are not implemented. If the kernel still provides
 +  POSIX or flock locks, they are local to this mount instance and carry no
 +  cross-mount or cross-client meaning; no distributed lock semantics are
 +  claimed. A writable mount must not claim that locking is rejected outright,
 +  because the kernel may return success for its own local lock manager.
- - The shared Mount node map is not held across first materialization or other
-   authority/staging I/O; the adapter adds no mount-wide lock across those
-   awaits.
- - Extension points for the next layer are the runtime namespace operations,
-   `Mount::refresh` guarded by reference counts, and the session `Notifier`
-   for kernel entry/inode invalidation once an in-place refresh policy exists.
+ - The module keeps existing core Mount behavior, including the node-map lock
+   that already spans first materialization; the adapter adds no mount-wide
+   lock across authority or staging I/O.
+ - The session `Notifier` is used only by the explicit `SIGHUP` refresh above;
 -  there is no automatic external-change watch, and its capability set does
 -  not claim recoverable staging or any write-side operation beyond the
 -  read-only contract. Remaining extension points for the next layer are the
 -  runtime namespace operations and `Mount::refresh` guarded by reference
 -  counts.
++  there is no automatic external-change watch.
diff --cc src/fuse/frontend.rs
index 5253bb5,239982b..a4e7142
--- a/src/fuse/frontend.rs
+++ b/src/fuse/frontend.rs
@@@ -46,11 -47,12 +46,12 @@@ use super::inode::{CachedInfo, InodeTab
  use crate::mount::{Mount, MountFile};
  use fuser::{
      AccessFlags, BsdFileFlags, FileAttr, FileHandle, FileType, Filesystem, FopenFlags, Generation,
-     INodeNo, InitFlags, KernelConfig, LockOwner, OpenAccMode, OpenFlags, RenameFlags, ReplyAttr,
-     ReplyCreate, ReplyData, ReplyDirectory, ReplyEmpty, ReplyEntry, ReplyOpen, ReplyStatfs,
-     ReplyWrite, ReplyXattr, Request, TimeOrNow, WriteFlags,
 -    INodeNo, InitFlags, KernelConfig, LockOwner, Notifier, OpenAccMode, OpenFlags, ReplyAttr,
 -    ReplyData, ReplyDirectory, ReplyEmpty, ReplyEntry, ReplyOpen, ReplyStatfs, ReplyWrite,
 -    ReplyXattr, Request, TimeOrNow, WriteFlags,
++    INodeNo, InitFlags, KernelConfig, LockOwner, Notifier, OpenAccMode, OpenFlags, RenameFlags,
++    ReplyAttr, ReplyCreate, ReplyData, ReplyDirectory, ReplyEmpty, ReplyEntry, ReplyOpen,
++    ReplyStatfs, ReplyWrite, ReplyXattr, Request, TimeOrNow, WriteFlags,
  };
- use std::collections::BTreeMap;
+ use std::collections::{BTreeMap, BTreeSet};
+ use std::ffi::OsStr;
  use std::future::Future;
  use std::sync::atomic::{AtomicU64, Ordering};
  use std::sync::{Arc, Mutex};
@@@ -67,9 -69,12 +68,13 @@@ pub(super) struct Inner 
      inodes: Mutex<InodeTable>,
      files: Mutex<BTreeMap<u64, Arc<OpenFile>>>,
      dirs: Mutex<BTreeMap<u64, Arc<AsyncMutex<DirCursor>>>>,
+     /// Dentry names this adapter actually replied to the kernel; only those
+     /// can be cached and invalidated by an explicit refresh.
+     dentries: Mutex<BTreeSet<(u64, String)>>,
+     notifier: Mutex<Option<Notifier>>,
      next_handle: AtomicU64,
      runtime: Handle,
 +    read_only: bool,
  }
  
  /// One kernel open maps to one reference on the node's shared staged version.
@@@ -141,9 -113,10 +146,11 @@@ impl Inner 
              inodes: Mutex::new(InodeTable::new(root)),
              files: Mutex::new(BTreeMap::new()),
              dirs: Mutex::new(BTreeMap::new()),
+             dentries: Mutex::new(BTreeSet::new()),
+             notifier: Mutex::new(None),
              next_handle: AtomicU64::new(0),
              runtime,
 +            read_only,
          }
      }
  
@@@ -290,8 -366,13 +421,13 @@@
              Ok(None) => return reply.error(fuser::Errno::ENOENT),
              Err(error) => return reply.error(core_errno(&error)),
          };
+         // A positive LOOKUP reply may install a kernel dentry; remember it
+         // for the explicit refresh. Negative results carry no timeout and are
+         // skipped, and a plain READDIR does not install positive dentries
+         // (that is READDIRPLUS), so scanned names are not recorded.
+         self.record_dentry(parent, &name);
 -        match self.attr_for(&snapshot, entry.node_id).await {
 -            Ok(attr) => {
 +        match self.shared_attr(entry.node_id).await {
 +            Ok(Some(attr)) => {
                  self.remember_lookup(attr.ino);
                  reply.entry(&ATTR_TTL, &attr, Generation(0));
              }
diff --cc src/fuse/mod.rs
index 2ba4675,f655759..fda4ab0
--- a/src/fuse/mod.rs
+++ b/src/fuse/mod.rs
@@@ -90,26 -85,34 +90,38 @@@ pub async fn run
          )
          .into());
      }
 -    let inner = Arc::new(frontend::Inner::new(
 +    let inner = Arc::new(frontend::Inner::with_read_only(
          mount,
          tokio::runtime::Handle::current(),
 +        read_only,
      ));
-     let filesystem = frontend::FuseFrontend { inner };
+     let filesystem = frontend::FuseFrontend {
+         inner: inner.clone(),
+     };
      let mut options = Config::default();
      options.mount_options = vec![
 -        MountOption::RO,
          MountOption::FSName(format!("yinyang-{}", config.name)),
          MountOption::Subtype("yinyang".into()),
      ];
 +    if read_only {
 +        options.mount_options.insert(0, MountOption::RO);
 +    }
      options.acl = SessionACL::Owner;
+     // Register SIGHUP before the mount becomes externally visible so a
+     // refresh signal cannot fall through to the default handler while the loop
+     // task waits for its first poll.
+     let hangup = match tokio::signal::unix::signal(tokio::signal::unix::SignalKind::hangup()) {
+         Ok(signal) => Some(signal),
+         Err(error) => {
+             eprintln!("yy: fuse refresh: cannot handle SIGHUP: {error}");
+             None
+         }
+     };
      let session = Session::new(filesystem, &mountpoint, &options)?;
      println!(
 -        "yy: mounted {} read-only at {} (fuse.yinyang)",
 +        "yy: mounted {} {} at {} (fuse.yinyang)",
          config.name,
 +        if read_only { "read-only" } else { "read-write" },
          mountpoint.display()
      );
      // `Session::spawn` moves the mount object into `BackgroundSession`, so a