Re: [I] [Feature] python agent should be shaded [skywalking]

["wu-sheng (via GitHub)" <gi...@apache.org>]

wu-sheng commented on issue #13437:
URL: https://github.com/apache/skywalking/issues/13437#issuecomment-4231223606

   Thanks for the suggestion. After investigation, here is the analysis for the Python agent:
   
   ## Current State
   
   The Python agent **does not pin/lock dependency versions**. In `pyproject.toml`:
   
   ```
   grpcio = '*'
   grpcio-tools = '*'
   kafka-python = { version = "*", optional = true }
   wrapt = '>=1.14'
   psutil = '*'
   ```
   
   This means pip resolves to **whatever version the user already has installed**, or the latest compatible version if not installed. The agent and the user's application share the same Python process and the same installed packages — there are no bundled/embedded copies.
   
   ## Why Shading Is Not Feasible in Python
   
   In Java, shading works by renaming package namespaces within bundled JARs (e.g., `com.google.protobuf` → `org.apache.skywalking.apm.dependencies.com.google.protobuf`). This is possible because JARs are self-contained and class loading supports namespace isolation.
   
   Python has no equivalent mechanism:
   - **C extensions** (grpcio, psutil) cannot be namespace-renamed — they're compiled shared libraries (`.so`/`.pyd`) with hardcoded symbols
   - **`sys.modules` is global** — once a module is imported, it's shared across the entire process. You cannot have two different versions of `grpcio` loaded simultaneously
   - **Vendoring** (copying source under `skywalking._vendor`) only works for pure-Python packages, not C extensions
   
   ## Why It's Not a Problem
   
   The APIs we consume from these dependencies are all stable, mature interfaces:
   - **grpcio**: `Channel`, `insecure_channel()`, stub patterns — stable since 1.x
   - **kafka-python**: `KafkaProducer.send()` — stable core API
   - **wrapt**: `ObjectProxy` — stable since 1.x  
   - **psutil**: `cpu_percent()`, `virtual_memory()` — stable for years
   - **requests**: `Session.request()` — stable since 2.x
   
   Since we use wide version ranges (`*`), there is no version conflict — the agent uses whatever version the user's application has installed. Breaking changes in these core APIs would affect the entire Python ecosystem, not just the agent.
   
   ## Conclusion
   
   The Python dependency model is fundamentally different from Java's. The Python agent already follows the Python-idiomatic approach: declare wide version ranges, share the user's installed packages, and consume only stable APIs. There is no practical need for shading, and true shading (like Java) is technically not feasible for C extensions in Python.
   
   If a specific version conflict is encountered in practice, please report it — we can adjust version ranges or add compatibility handling for that case.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org