You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@opendal.apache.org by xu...@apache.org on 2026/10/26 10:00:51 UTC
(opendal-yinyang) branch xuanwo/linux-fuse-foundation updated: refactor: track the SIGHUP loop inside teardown
This is an automated email from the ASF dual-hosted git repository.
Xuanwo pushed a commit to branch xuanwo/linux-fuse-foundation
in repository https://gitbox.apache.org/repos/asf/opendal-yinyang.git
The following commit(s) were added to refs/heads/xuanwo/linux-fuse-foundation by this push:
new 0f0f838 refactor: track the SIGHUP loop inside teardown
0f0f838 is described below
commit 0f0f838862e35d89525f75415e32e4703161788c
Author: Xuanwo <gi...@xuanwo.io>
AuthorDate: Mon Oct 26 18:00:45 2026 +0800
refactor: track the SIGHUP loop inside teardown
Register the SIGHUP handler synchronously before the mount becomes
visible, then run one signal-driven loop as a tracked Tokio task instead
of detached refresh flags. The stream coalesces duplicate requests while
a refresh runs. Teardown sends the loop a stop signal and waits for the
session join and the refresh task under the same five-second grace, so
the CLI deadline really bounds both; an executing refresh may finish
inside the window.
Notification error handling matches the binding: fuser send_inval
already accepts ENOENT, so only explicit connection-ended errnos
(ENODEV/ENOTCONN/EPIPE/ECONNRESET/ESHUTDOWN) stop the batch and other
failures are counted and skipped. Log lines count sent/acceptance of
invalidations, not proof the kernel still cached the inode. Dentry
recording covers positive LOOKUP replies only: plain READDIR does not
install dentries without READDIRPLUS, and the comment and spec say so.
Co-authored-by: Chojan Shang <ps...@apache.org>
---
Cargo.toml | 1 +
specs/linux-fuse.md | 18 +++++---
src/fuse/frontend.rs | 115 ++++++++++++++-------------------------------------
src/fuse/mod.rs | 70 ++++++++++++++++++++-----------
4 files changed, 88 insertions(+), 116 deletions(-)
diff --git a/Cargo.toml b/Cargo.toml
index 5f5e197..89537da 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -56,6 +56,7 @@ yinyang-core = { path = "crates/core" }
[target.'cfg(target_os = "linux")'.dependencies]
fuser = "0.17.0"
+libc = "0.2"
nix = { version = "0.30.0", features = ["user"] }
[workspace]
diff --git a/specs/linux-fuse.md b/specs/linux-fuse.md
index 8ca8f98..ea8024b 100644
--- a/specs/linux-fuse.md
+++ b/specs/linux-fuse.md
@@ -104,17 +104,23 @@ latest snapshot once and then:
- invalidates known inode attributes with `inval_inode(ino, -1, 0)` and
replied dentries with `inval_entry(parent, name)` through the fuser
`Notifier`. Only dentries this adapter actually replied to the kernel are
- recorded: successful `LOOKUP` results and `READDIR` entries. Negative
- lookups carry no timeout in FUSE replies and are not recorded.
+ recorded: successful `LOOKUP` results. Negative lookups carry no timeout in
+ FUSE replies and are not recorded, and plain `READDIR` does not install
+ dentries without `READDIRPLUS`, so directory names are not recorded.
The adapter never calls `Mount::refresh`, because that replaces the shared
staged handle and would rebase open file descriptors. The refresh must also
hold no adapter lock while reclaiming nodes or sending notifications; targets
are collected under short locks, then the sends run on a blocking thread. At
-most one refresh run is in flight; duplicate `SIGHUP`s while one is running
-are merged into a single trailing run. The refresh task is concurrent with
-teardown and its lifetime sits inside the same bounded grace period: after
-unmount, notification sends fail and the refresh stops without retrying.
+most one refresh runs at a time and duplicate `SIGHUP`s while one is running
+are coalesced by the signal stream. The SIGHUP loop is a tracked Tokio task:
+the signal handler is registered synchronously before the mount becomes
+visible, teardown sends it a stop signal, and the session join and the refresh
+task share the same five-second grace period; a refresh already in progress
+may finish inside that window. A `NotFound` notification result is expected
+(the kernel already dropped the entry or inode) and is counted separately, so
+only explicit connection-ended errors (`ENODEV`, `ENOTCONN`, `EPIPE`,
+`ECONNRESET`, `ESHUTDOWN`) stop the batch before it would otherwise finish.
Stable `yy: fuse refresh:` log lines report the observed revision, the
reclaimed/active node counts, and the native notification counts at
diff --git a/src/fuse/frontend.rs b/src/fuse/frontend.rs
index 59c019e..ea81821 100644
--- a/src/fuse/frontend.rs
+++ b/src/fuse/frontend.rs
@@ -54,7 +54,7 @@ use fuser::{
use std::collections::{BTreeMap, BTreeSet};
use std::ffi::OsStr;
use std::future::Future;
-use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
+use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{Arc, Mutex};
use std::time::SystemTime;
use tokio::runtime::Handle;
@@ -73,8 +73,6 @@ pub(super) struct Inner {
/// can be cached and invalidated by an explicit refresh.
dentries: Mutex<BTreeSet<(u64, String)>>,
notifier: Mutex<Option<Notifier>>,
- refresh_inflight: AtomicBool,
- refresh_pending: AtomicBool,
next_handle: AtomicU64,
runtime: Handle,
}
@@ -117,8 +115,6 @@ impl Inner {
dirs: Mutex::new(BTreeMap::new()),
dentries: Mutex::new(BTreeSet::new()),
notifier: Mutex::new(None),
- refresh_inflight: AtomicBool::new(false),
- refresh_pending: AtomicBool::new(false),
next_handle: AtomicU64::new(0),
runtime,
}
@@ -142,43 +138,19 @@ impl Inner {
guard(&self.dentries).insert((parent.0, name.to_owned()));
}
- /// Serve explicit refresh requests. This watches only SIGHUP; there is no
- /// background polling or change-feed watch.
- pub(super) async fn serve_hup(self: Arc<Self>) {
- let mut signal = match tokio::signal::unix::signal(tokio::signal::unix::SignalKind::hangup())
- {
- Ok(signal) => signal,
- Err(error) => {
- eprintln!("yy: fuse refresh: cannot install SIGHUP handler: {error}");
- return;
- }
- };
- loop {
- signal.recv().await;
- self.request_refresh();
- }
- }
-
- /// Start one refresh run; duplicate requests while one is in flight are
- /// merged into a single trailing run instead of queued.
- fn request_refresh(self: &Arc<Self>) {
- if self.refresh_inflight.swap(true, Ordering::Acquire) {
- self.refresh_pending.store(true, Ordering::Release);
- return;
- }
- let inner = self.clone();
- self.runtime
- .spawn(async move { inner.refresh_runner().await });
- }
-
- async fn refresh_runner(self: Arc<Self>) {
+ /// Serve explicit refresh requests. The signal stream was registered
+ /// before the mount became visible; SIGHUPs arriving while a refresh runs
+ /// are coalesced by the stream, and `stop` ends the loop so the caller can
+ /// bound the task with the session teardown deadline.
+ pub(super) async fn serve_hup(
+ self: Arc<Self>,
+ mut signal: tokio::signal::unix::Signal,
+ mut stop: tokio::sync::oneshot::Receiver<()>,
+ ) {
loop {
- self.refresh_once().await;
- // Another request arrived while this run was in flight: run once
- // more so the newer known set is invalidated too.
- if !self.refresh_pending.swap(false, Ordering::AcqRel) {
- self.refresh_inflight.store(false, Ordering::Release);
- return;
+ tokio::select! {
+ _ = signal.recv() => self.refresh_once().await,
+ _ = &mut stop => return,
}
}
}
@@ -225,32 +197,38 @@ impl Inner {
eprintln!("yy: fuse refresh: notifier not installed; skipping kernel invalidation");
return;
};
+ // fuser's send_inval already treats NotFound/ENOENT as an accepted
+ // notification, so Ok below means the kernel accepted the send, not
+ // that the inode or entry was still cached. Only explicit
+ // connection-ended errors stop the batch; other failures are counted
+ // and skipped so one vanished inode cannot abort the whole refresh.
let attempts = inodes.len() + dentries.len();
let (inode_ok, entry_ok, errors, ended) = tokio::task::spawn_blocking(move || {
let mut inode_ok = 0usize;
let mut entry_ok = 0usize;
let mut errors = 0usize;
let mut ended = false;
- for inode in &inodes {
+ for inode in inodes.iter() {
match notifier.inval_inode(INodeNo(*inode), -1, 0) {
Ok(()) => inode_ok += 1,
- Err(_) => {
- // After unmount every send fails; stop instead of
- // spamming a closing channel.
+ Err(error) if is_connection_ended(&error) => {
+ // The session is gone; stop instead of spamming.
errors += 1;
ended = true;
return (inode_ok, entry_ok, errors, ended);
}
+ Err(_) => errors += 1,
}
}
- for (parent, name) in &dentries {
+ for (parent, name) in dentries.iter() {
match notifier.inval_entry(INodeNo(*parent), OsStr::new(name)) {
Ok(()) => entry_ok += 1,
- Err(_) => {
+ Err(error) if is_connection_ended(&error) => {
errors += 1;
ended = true;
return (inode_ok, entry_ok, errors, ended);
}
+ Err(_) => errors += 1,
}
}
(inode_ok, entry_ok, errors, ended)
@@ -259,7 +237,7 @@ impl Inner {
.unwrap_or((0, 0, attempts, false));
if ended {
eprintln!(
- "yy: fuse refresh: channel ended; invalidated {inode_ok} inode attrs and {entry_ok} dentries before {errors} errors"
+ "yy: fuse refresh: channel ended; sent {inode_ok} inode and {entry_ok} entry invalidations before {errors} errors"
);
} else {
eprintln!(
@@ -382,8 +360,10 @@ impl Inner {
Ok(None) => return reply.error(fuser::Errno::ENOENT),
Err(error) => return reply.error(core_errno(&error)),
};
- // Successful LOOKUP installs a kernel dentry; remember it for the
- // explicit refresh. Negative results carry no timeout and are skipped.
+ // A positive LOOKUP reply may install a kernel dentry; remember it
+ // for the explicit refresh. Negative results carry no timeout and are
+ // skipped, and a plain READDIR does not install positive dentries
+ // (that is READDIRPLUS), so scanned names are not recorded.
self.record_dentry(parent, &name);
match self.attr_for(&snapshot, entry.node_id).await {
Ok(attr) => {
@@ -650,8 +630,6 @@ impl Inner {
cursor.complete = cursor.token.is_none();
for entry in page.entries {
let (ino, kind) = self.resolve_entry(&cursor.snapshot, entry.node_id).await?;
- // readdir replies install kernel dentries; remember them too.
- self.record_dentry(cursor.ino, &entry.name);
cursor.entries.push(DirItem {
ino,
kind,
@@ -1093,9 +1071,7 @@ mod tests {
use crate::mount::Mount;
use crate::runtime::Runtime;
use crate::support;
- use fuser::INodeNo;
use std::sync::Arc;
- use std::sync::atomic::Ordering;
use yinyang_core::Authority;
/// An open read-only handle pins the staged version: while the file table
@@ -1232,35 +1208,4 @@ mod tests {
drop(fresh);
inner.mount.reclaim(id).await.unwrap();
}
-
- #[tokio::test]
- async fn refresh_runner_completes_and_dentries_deduplicate() {
- let temp = tempfile::tempdir().unwrap();
- let backend = support::TestBackend::default();
- Fs::create(backend.operator(), BackendProfile::Minio)
- .await
- .unwrap();
- let authority: Arc<dyn Authority> = Arc::new(
- Fs::open(backend.operator(), BackendProfile::Minio)
- .await
- .unwrap(),
- );
- let mount = Mount::open(authority, temp.path().join("ro"), true)
- .await
- .unwrap();
- let inner = Arc::new(Inner::new(mount, tokio::runtime::Handle::current()));
- inner.record_dentry(INodeNo(2), "a");
- inner.record_dentry(INodeNo(2), "a");
- inner.record_dentry(INodeNo(3), "b");
- assert_eq!(guard(&inner.dentries).len(), 2);
-
- inner.request_refresh();
- for _ in 0..200 {
- if !inner.refresh_inflight.load(Ordering::Relaxed) {
- break;
- }
- tokio::time::sleep(std::time::Duration::from_millis(10)).await;
- }
- assert!(!inner.refresh_inflight.load(Ordering::Relaxed));
- }
}
diff --git a/src/fuse/mod.rs b/src/fuse/mod.rs
index 750cac8..b12eb44 100644
--- a/src/fuse/mod.rs
+++ b/src/fuse/mod.rs
@@ -99,6 +99,16 @@ pub async fn run(
MountOption::Subtype("yinyang".into()),
];
options.acl = SessionACL::Owner;
+ // Register SIGHUP before the mount becomes externally visible so a
+ // refresh signal cannot fall through to the default handler while the loop
+ // task waits for its first poll.
+ let hangup = match tokio::signal::unix::signal(tokio::signal::unix::SignalKind::hangup()) {
+ Ok(signal) => Some(signal),
+ Err(error) => {
+ eprintln!("yy: fuse refresh: cannot handle SIGHUP: {error}");
+ None
+ }
+ };
let session = Session::new(filesystem, &mountpoint, &options)?;
println!(
"yy: mounted {} read-only at {} (fuse.yinyang)",
@@ -110,11 +120,15 @@ pub async fn run(
// empty slot. One blocking owner holds the background session, polls its
// public thread guard for a natural end (for example `fusermount3 -u`),
// and receives the local teardown request. Exactly one `join` or
- // `umount_and_join` runs.
+ // `umount_and_join` runs under the shared grace period.
let background = session.spawn()?;
inner.install_notifier(background.notifier());
- let refresh = inner.clone();
- tokio::runtime::Handle::current().spawn(async move { refresh.serve_hup().await });
+ let (stop_refresh, stop_refresh_rx) = tokio::sync::oneshot::channel::<()>();
+ let refresh_task = hangup.map(|signal| {
+ let refresh = inner.clone();
+ tokio::runtime::Handle::current()
+ .spawn(async move { refresh.serve_hup(signal, stop_refresh_rx).await })
+ });
let (teardown, teardown_rx) = std::sync::mpsc::channel();
let join = tokio::task::spawn_blocking(move || {
loop {
@@ -131,36 +145,42 @@ pub async fn run(
}
});
tokio::pin!(join);
- let outcome = tokio::select! {
+ tokio::select! {
result = &mut join => {
// The session ended on its own, for example through an external
- // `fusermount3 -u`. Returning now lets the process exit and release
- // the staging lease instead of waiting for a signal forever.
+ // `fusermount3 -u`; fall through to bounded teardown.
result.map_err(blocking_error)??;
- Outcome::Unmounted
}
_ = shutdown_signal() => {
let _ = teardown.send(());
- match tokio::time::timeout(TEARDOWN_GRACE, &mut join).await {
- Ok(result) => {
- result.map_err(blocking_error)??;
- Outcome::Unmounted
- }
- // A lazy unmount keeps the session alive while clients hold
- // the mount. Report it to the CLI instead of returning into a
- // runtime shutdown that would wait for the blocked session
- // task; the CLI exits the process so the kernel aborts the
- // connection.
- Err(_) => Outcome::TeardownExpired {
- grace: TEARDOWN_GRACE,
- },
- }
}
- };
- if matches!(outcome, Outcome::Unmounted) {
- println!("yy: unmounted {}", mountpoint.display());
}
- Ok(outcome)
+ // One grace period covers the session teardown and the SIGHUP loop: stop
+ // the refresh task, then wait for both. A refresh in progress may finish
+ // inside this window; no cancellation is needed.
+ let _ = stop_refresh.send(());
+ match tokio::time::timeout(TEARDOWN_GRACE, async {
+ join.await.map_err(blocking_error)??;
+ if let Some(task) = refresh_task {
+ task.await.map_err(blocking_error)?;
+ }
+ Ok(())
+ })
+ .await
+ {
+ Ok(Ok(())) => {
+ println!("yy: unmounted {}", mountpoint.display());
+ Ok(Outcome::Unmounted)
+ }
+ Ok(Err(error)) => Err(error),
+ // A lazy unmount keeps the session alive while clients hold the
+ // mount. Report it to the CLI instead of returning into a runtime
+ // shutdown that would wait for the blocked session task; the CLI exits
+ // the process so the kernel aborts the connection.
+ Err(_) => Ok(Outcome::TeardownExpired {
+ grace: TEARDOWN_GRACE,
+ }),
+ }
}
fn blocking_error(error: tokio::task::JoinError) -> std::io::Error {