You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by xu...@apache.org on 2026/10/26 10:00:51 UTC

(opendal-yinyang) branch xuanwo/linux-fuse-foundation updated: refactor: track the SIGHUP loop inside teardown

This is an automated email from the ASF dual-hosted git repository.

Xuanwo pushed a commit to branch xuanwo/linux-fuse-foundation
in repository https://gitbox.apache.org/repos/asf/opendal-yinyang.git


The following commit(s) were added to refs/heads/xuanwo/linux-fuse-foundation by this push:
     new 0f0f838  refactor: track the SIGHUP loop inside teardown
0f0f838 is described below

commit 0f0f838862e35d89525f75415e32e4703161788c
Author: Xuanwo <gi...@xuanwo.io>
AuthorDate: Mon Oct 26 18:00:45 2026 +0800

    refactor: track the SIGHUP loop inside teardown
    
    Register the SIGHUP handler synchronously before the mount becomes
    visible, then run one signal-driven loop as a tracked Tokio task instead
    of detached refresh flags. The stream coalesces duplicate requests while
    a refresh runs. Teardown sends the loop a stop signal and waits for the
    session join and the refresh task under the same five-second grace, so
    the CLI deadline really bounds both; an executing refresh may finish
    inside the window.
    
    Notification error handling matches the binding: fuser send_inval
    already accepts ENOENT, so only explicit connection-ended errnos
    (ENODEV/ENOTCONN/EPIPE/ECONNRESET/ESHUTDOWN) stop the batch and other
    failures are counted and skipped. Log lines count sent/acceptance of
    invalidations, not proof the kernel still cached the inode. Dentry
    recording covers positive LOOKUP replies only: plain READDIR does not
    install dentries without READDIRPLUS, and the comment and spec say so.
    
    Co-authored-by: Chojan Shang <ps...@apache.org>
---
 Cargo.toml           |   1 +
 specs/linux-fuse.md  |  18 +++++---
 src/fuse/frontend.rs | 115 ++++++++++++++-------------------------------------
 src/fuse/mod.rs      |  70 ++++++++++++++++++++-----------
 4 files changed, 88 insertions(+), 116 deletions(-)

diff --git a/Cargo.toml b/Cargo.toml
index 5f5e197..89537da 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -56,6 +56,7 @@ yinyang-core = { path = "crates/core" }
 
 [target.'cfg(target_os = "linux")'.dependencies]
 fuser = "0.17.0"
+libc = "0.2"
 nix = { version = "0.30.0", features = ["user"] }
 
 [workspace]
diff --git a/specs/linux-fuse.md b/specs/linux-fuse.md
index 8ca8f98..ea8024b 100644
--- a/specs/linux-fuse.md
+++ b/specs/linux-fuse.md
@@ -104,17 +104,23 @@ latest snapshot once and then:
 - invalidates known inode attributes with `inval_inode(ino, -1, 0)` and
   replied dentries with `inval_entry(parent, name)` through the fuser
   `Notifier`. Only dentries this adapter actually replied to the kernel are
-  recorded: successful `LOOKUP` results and `READDIR` entries. Negative
-  lookups carry no timeout in FUSE replies and are not recorded.
+  recorded: successful `LOOKUP` results. Negative lookups carry no timeout in
+  FUSE replies and are not recorded, and plain `READDIR` does not install
+  dentries without `READDIRPLUS`, so directory names are not recorded.
 
 The adapter never calls `Mount::refresh`, because that replaces the shared
 staged handle and would rebase open file descriptors. The refresh must also
 hold no adapter lock while reclaiming nodes or sending notifications; targets
 are collected under short locks, then the sends run on a blocking thread. At
-most one refresh run is in flight; duplicate `SIGHUP`s while one is running
-are merged into a single trailing run. The refresh task is concurrent with
-teardown and its lifetime sits inside the same bounded grace period: after
-unmount, notification sends fail and the refresh stops without retrying.
+most one refresh runs at a time and duplicate `SIGHUP`s while one is running
+are coalesced by the signal stream. The SIGHUP loop is a tracked Tokio task:
+the signal handler is registered synchronously before the mount becomes
+visible, teardown sends it a stop signal, and the session join and the refresh
+task share the same five-second grace period; a refresh already in progress
+may finish inside that window. A `NotFound` notification result is expected
+(the kernel already dropped the entry or inode) and is counted separately, so
+only explicit connection-ended errors (`ENODEV`, `ENOTCONN`, `EPIPE`,
+`ECONNRESET`, `ESHUTDOWN`) stop the batch before it would otherwise finish.
 
 Stable `yy: fuse refresh:` log lines report the observed revision, the
 reclaimed/active node counts, and the native notification counts at
diff --git a/src/fuse/frontend.rs b/src/fuse/frontend.rs
index 59c019e..ea81821 100644
--- a/src/fuse/frontend.rs
+++ b/src/fuse/frontend.rs
@@ -54,7 +54,7 @@ use fuser::{
 use std::collections::{BTreeMap, BTreeSet};
 use std::ffi::OsStr;
 use std::future::Future;
-use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
+use std::sync::atomic::{AtomicU64, Ordering};
 use std::sync::{Arc, Mutex};
 use std::time::SystemTime;
 use tokio::runtime::Handle;
@@ -73,8 +73,6 @@ pub(super) struct Inner {
     /// can be cached and invalidated by an explicit refresh.
     dentries: Mutex<BTreeSet<(u64, String)>>,
     notifier: Mutex<Option<Notifier>>,
-    refresh_inflight: AtomicBool,
-    refresh_pending: AtomicBool,
     next_handle: AtomicU64,
     runtime: Handle,
 }
@@ -117,8 +115,6 @@ impl Inner {
             dirs: Mutex::new(BTreeMap::new()),
             dentries: Mutex::new(BTreeSet::new()),
             notifier: Mutex::new(None),
-            refresh_inflight: AtomicBool::new(false),
-            refresh_pending: AtomicBool::new(false),
             next_handle: AtomicU64::new(0),
             runtime,
         }
@@ -142,43 +138,19 @@ impl Inner {
         guard(&self.dentries).insert((parent.0, name.to_owned()));
     }
 
-    /// Serve explicit refresh requests. This watches only SIGHUP; there is no
-    /// background polling or change-feed watch.
-    pub(super) async fn serve_hup(self: Arc<Self>) {
-        let mut signal = match tokio::signal::unix::signal(tokio::signal::unix::SignalKind::hangup())
-        {
-            Ok(signal) => signal,
-            Err(error) => {
-                eprintln!("yy: fuse refresh: cannot install SIGHUP handler: {error}");
-                return;
-            }
-        };
-        loop {
-            signal.recv().await;
-            self.request_refresh();
-        }
-    }
-
-    /// Start one refresh run; duplicate requests while one is in flight are
-    /// merged into a single trailing run instead of queued.
-    fn request_refresh(self: &Arc<Self>) {
-        if self.refresh_inflight.swap(true, Ordering::Acquire) {
-            self.refresh_pending.store(true, Ordering::Release);
-            return;
-        }
-        let inner = self.clone();
-        self.runtime
-            .spawn(async move { inner.refresh_runner().await });
-    }
-
-    async fn refresh_runner(self: Arc<Self>) {
+    /// Serve explicit refresh requests. The signal stream was registered
+    /// before the mount became visible; SIGHUPs arriving while a refresh runs
+    /// are coalesced by the stream, and `stop` ends the loop so the caller can
+    /// bound the task with the session teardown deadline.
+    pub(super) async fn serve_hup(
+        self: Arc<Self>,
+        mut signal: tokio::signal::unix::Signal,
+        mut stop: tokio::sync::oneshot::Receiver<()>,
+    ) {
         loop {
-            self.refresh_once().await;
-            // Another request arrived while this run was in flight: run once
-            // more so the newer known set is invalidated too.
-            if !self.refresh_pending.swap(false, Ordering::AcqRel) {
-                self.refresh_inflight.store(false, Ordering::Release);
-                return;
+            tokio::select! {
+                _ = signal.recv() => self.refresh_once().await,
+                _ = &mut stop => return,
             }
         }
     }
@@ -225,32 +197,38 @@ impl Inner {
             eprintln!("yy: fuse refresh: notifier not installed; skipping kernel invalidation");
             return;
         };
+        // fuser's send_inval already treats NotFound/ENOENT as an accepted
+        // notification, so Ok below means the kernel accepted the send, not
+        // that the inode or entry was still cached. Only explicit
+        // connection-ended errors stop the batch; other failures are counted
+        // and skipped so one vanished inode cannot abort the whole refresh.
         let attempts = inodes.len() + dentries.len();
         let (inode_ok, entry_ok, errors, ended) = tokio::task::spawn_blocking(move || {
             let mut inode_ok = 0usize;
             let mut entry_ok = 0usize;
             let mut errors = 0usize;
             let mut ended = false;
-            for inode in &inodes {
+            for inode in inodes.iter() {
                 match notifier.inval_inode(INodeNo(*inode), -1, 0) {
                     Ok(()) => inode_ok += 1,
-                    Err(_) => {
-                        // After unmount every send fails; stop instead of
-                        // spamming a closing channel.
+                    Err(error) if is_connection_ended(&error) => {
+                        // The session is gone; stop instead of spamming.
                         errors += 1;
                         ended = true;
                         return (inode_ok, entry_ok, errors, ended);
                     }
+                    Err(_) => errors += 1,
                 }
             }
-            for (parent, name) in &dentries {
+            for (parent, name) in dentries.iter() {
                 match notifier.inval_entry(INodeNo(*parent), OsStr::new(name)) {
                     Ok(()) => entry_ok += 1,
-                    Err(_) => {
+                    Err(error) if is_connection_ended(&error) => {
                         errors += 1;
                         ended = true;
                         return (inode_ok, entry_ok, errors, ended);
                     }
+                    Err(_) => errors += 1,
                 }
             }
             (inode_ok, entry_ok, errors, ended)
@@ -259,7 +237,7 @@ impl Inner {
         .unwrap_or((0, 0, attempts, false));
         if ended {
             eprintln!(
-                "yy: fuse refresh: channel ended; invalidated {inode_ok} inode attrs and {entry_ok} dentries before {errors} errors"
+                "yy: fuse refresh: channel ended; sent {inode_ok} inode and {entry_ok} entry invalidations before {errors} errors"
             );
         } else {
             eprintln!(
@@ -382,8 +360,10 @@ impl Inner {
             Ok(None) => return reply.error(fuser::Errno::ENOENT),
             Err(error) => return reply.error(core_errno(&error)),
         };
-        // Successful LOOKUP installs a kernel dentry; remember it for the
-        // explicit refresh. Negative results carry no timeout and are skipped.
+        // A positive LOOKUP reply may install a kernel dentry; remember it
+        // for the explicit refresh. Negative results carry no timeout and are
+        // skipped, and a plain READDIR does not install positive dentries
+        // (that is READDIRPLUS), so scanned names are not recorded.
         self.record_dentry(parent, &name);
         match self.attr_for(&snapshot, entry.node_id).await {
             Ok(attr) => {
@@ -650,8 +630,6 @@ impl Inner {
         cursor.complete = cursor.token.is_none();
         for entry in page.entries {
             let (ino, kind) = self.resolve_entry(&cursor.snapshot, entry.node_id).await?;
-            // readdir replies install kernel dentries; remember them too.
-            self.record_dentry(cursor.ino, &entry.name);
             cursor.entries.push(DirItem {
                 ino,
                 kind,
@@ -1093,9 +1071,7 @@ mod tests {
     use crate::mount::Mount;
     use crate::runtime::Runtime;
     use crate::support;
-    use fuser::INodeNo;
     use std::sync::Arc;
-    use std::sync::atomic::Ordering;
     use yinyang_core::Authority;
 
     /// An open read-only handle pins the staged version: while the file table
@@ -1232,35 +1208,4 @@ mod tests {
         drop(fresh);
         inner.mount.reclaim(id).await.unwrap();
     }
-
-    #[tokio::test]
-    async fn refresh_runner_completes_and_dentries_deduplicate() {
-        let temp = tempfile::tempdir().unwrap();
-        let backend = support::TestBackend::default();
-        Fs::create(backend.operator(), BackendProfile::Minio)
-            .await
-            .unwrap();
-        let authority: Arc<dyn Authority> = Arc::new(
-            Fs::open(backend.operator(), BackendProfile::Minio)
-                .await
-                .unwrap(),
-        );
-        let mount = Mount::open(authority, temp.path().join("ro"), true)
-            .await
-            .unwrap();
-        let inner = Arc::new(Inner::new(mount, tokio::runtime::Handle::current()));
-        inner.record_dentry(INodeNo(2), "a");
-        inner.record_dentry(INodeNo(2), "a");
-        inner.record_dentry(INodeNo(3), "b");
-        assert_eq!(guard(&inner.dentries).len(), 2);
-
-        inner.request_refresh();
-        for _ in 0..200 {
-            if !inner.refresh_inflight.load(Ordering::Relaxed) {
-                break;
-            }
-            tokio::time::sleep(std::time::Duration::from_millis(10)).await;
-        }
-        assert!(!inner.refresh_inflight.load(Ordering::Relaxed));
-    }
 }
diff --git a/src/fuse/mod.rs b/src/fuse/mod.rs
index 750cac8..b12eb44 100644
--- a/src/fuse/mod.rs
+++ b/src/fuse/mod.rs
@@ -99,6 +99,16 @@ pub async fn run(
         MountOption::Subtype("yinyang".into()),
     ];
     options.acl = SessionACL::Owner;
+    // Register SIGHUP before the mount becomes externally visible so a
+    // refresh signal cannot fall through to the default handler while the loop
+    // task waits for its first poll.
+    let hangup = match tokio::signal::unix::signal(tokio::signal::unix::SignalKind::hangup()) {
+        Ok(signal) => Some(signal),
+        Err(error) => {
+            eprintln!("yy: fuse refresh: cannot handle SIGHUP: {error}");
+            None
+        }
+    };
     let session = Session::new(filesystem, &mountpoint, &options)?;
     println!(
         "yy: mounted {} read-only at {} (fuse.yinyang)",
@@ -110,11 +120,15 @@ pub async fn run(
     // empty slot. One blocking owner holds the background session, polls its
     // public thread guard for a natural end (for example `fusermount3 -u`),
     // and receives the local teardown request. Exactly one `join` or
-    // `umount_and_join` runs.
+    // `umount_and_join` runs under the shared grace period.
     let background = session.spawn()?;
     inner.install_notifier(background.notifier());
-    let refresh = inner.clone();
-    tokio::runtime::Handle::current().spawn(async move { refresh.serve_hup().await });
+    let (stop_refresh, stop_refresh_rx) = tokio::sync::oneshot::channel::<()>();
+    let refresh_task = hangup.map(|signal| {
+        let refresh = inner.clone();
+        tokio::runtime::Handle::current()
+            .spawn(async move { refresh.serve_hup(signal, stop_refresh_rx).await })
+    });
     let (teardown, teardown_rx) = std::sync::mpsc::channel();
     let join = tokio::task::spawn_blocking(move || {
         loop {
@@ -131,36 +145,42 @@ pub async fn run(
         }
     });
     tokio::pin!(join);
-    let outcome = tokio::select! {
+    tokio::select! {
         result = &mut join => {
             // The session ended on its own, for example through an external
-            // `fusermount3 -u`. Returning now lets the process exit and release
-            // the staging lease instead of waiting for a signal forever.
+            // `fusermount3 -u`; fall through to bounded teardown.
             result.map_err(blocking_error)??;
-            Outcome::Unmounted
         }
         _ = shutdown_signal() => {
             let _ = teardown.send(());
-            match tokio::time::timeout(TEARDOWN_GRACE, &mut join).await {
-                Ok(result) => {
-                    result.map_err(blocking_error)??;
-                    Outcome::Unmounted
-                }
-                // A lazy unmount keeps the session alive while clients hold
-                // the mount. Report it to the CLI instead of returning into a
-                // runtime shutdown that would wait for the blocked session
-                // task; the CLI exits the process so the kernel aborts the
-                // connection.
-                Err(_) => Outcome::TeardownExpired {
-                    grace: TEARDOWN_GRACE,
-                },
-            }
         }
-    };
-    if matches!(outcome, Outcome::Unmounted) {
-        println!("yy: unmounted {}", mountpoint.display());
     }
-    Ok(outcome)
+    // One grace period covers the session teardown and the SIGHUP loop: stop
+    // the refresh task, then wait for both. A refresh in progress may finish
+    // inside this window; no cancellation is needed.
+    let _ = stop_refresh.send(());
+    match tokio::time::timeout(TEARDOWN_GRACE, async {
+        join.await.map_err(blocking_error)??;
+        if let Some(task) = refresh_task {
+            task.await.map_err(blocking_error)?;
+        }
+        Ok(())
+    })
+    .await
+    {
+        Ok(Ok(())) => {
+            println!("yy: unmounted {}", mountpoint.display());
+            Ok(Outcome::Unmounted)
+        }
+        Ok(Err(error)) => Err(error),
+        // A lazy unmount keeps the session alive while clients hold the
+        // mount. Report it to the CLI instead of returning into a runtime
+        // shutdown that would wait for the blocked session task; the CLI exits
+        // the process so the kernel aborts the connection.
+        Err(_) => Ok(Outcome::TeardownExpired {
+            grace: TEARDOWN_GRACE,
+        }),
+    }
 }
 
 fn blocking_error(error: tokio::task::JoinError) -> std::io::Error {