[PR] perf(bindings/python): release GIL during blocking operator I/O [opendal]

["Xuanwo (via GitHub)" <gi...@apache.org>]

Xuanwo opened a new pull request, #8180:
URL: https://github.com/apache/opendal/pull/8180

   # Which issue does this PR close?
   
   Part of #8159.
   
   # Rationale for this change
   
   Blocking `Operator` methods currently keep the CPython GIL while OpenDAL waits for storage I/O. This prevents unrelated Python threads from progressing and serializes otherwise independent blocking requests. The module's `gil_used = false` declaration enables free-threaded Python support, but it does not release the GIL on regular CPython.
   
   # What changes are included in this PR?
   
   The blocking `Operator` now extracts Python paths, options, and payloads before entering `Python::detach`, runs the actual core I/O call with owned Rust values, and constructs Python return objects after reattaching. This covers `open`, `read`, `write`, `stat`, `copy`, `rename`, `remove_all`, `create_dir`, `delete`, `exists`, `list`, `scan`, and `check`.
   
   An event-coordinated local HTTP regression test verifies that another Python thread progresses while a blocking read is waiting and that two independent reads overlap. A standalone local benchmark measures single-thread latency and two-thread concurrency without service credentials.
   
   Mutable `File` operations and `BlockingLister.__next__` remain unchanged so their state-transfer boundaries can be handled separately.
   
   # Are there any user-facing changes?
   
   Yes. Blocking `Operator` storage calls no longer monopolize the GIL while waiting for I/O. Public method signatures, return values, and error behavior remain unchanged.
   
   ## Benchmark
   
   Release builds on CPython 3.14.5 for macOS arm64, using a local HTTP subprocess with a 50 ms response delay and 30 iterations. Commit `04f8e9b77` is compared with its `origin/main` baseline `e6ac14c93`.
   
   | Representative workload | Baseline median | This PR median | Effect |
   | --- | ---: | ---: | ---: |
   | One blocking read | 60.095 ms | 59.160 ms | No measurable regression |
   | Two independent reads from Python threads | 113.971 ms | 60.378 ms | 47.0% lower elapsed time |
   
   # AI Usage Statement
   
   AI materially assisted the implementation, regression-test and benchmark design, local validation, and drafting. The contributor directed the scope and reviewed the completed changes and evidence. Local runtime validation covered GIL-enabled CPython 3.14 and free-threaded CPython 3.14t; repository CI remains the authority for the full wheel and platform matrix.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] perf(bindings/python): release GIL during blocking operator I/O [opendal]

["Xuanwo (via GitHub)" <gi...@apache.org>]

Xuanwo merged PR #8180:
URL: https://github.com/apache/opendal/pull/8180


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@opendal.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org