You are viewing a plain text version of this content. The canonical link for it is here.
Posted to discuss-archive@tvm.apache.org by "junrushao (via GitHub)" <gi...@apache.org> on 2026/01/10 00:07:28 UTC
[PR] doc: Function and Module [tvm-ffi]
junrushao opened a new pull request, #394:
URL: https://github.com/apache/tvm-ffi/pull/394
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For additional commands, e-mail: discuss-archive-help@tvm.apache.org
Re: [PR] doc: Function, Exception and Module [tvm-ffi]
Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao commented on code in PR #394:
URL: https://github.com/apache/tvm-ffi/pull/394#discussion_r2679105288
##########
docs/concepts/func_module.rst:
##########
@@ -0,0 +1,564 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+Function, Exception and Module
+==============================
+
+TVM-FFI provides a unified and ABI-stable calling convention that enables
+cross-language function calls between C++, Python, Rust, and other languages.
+Functions are first-class :doc:`TVM-FFI objects <object_and_class>`.
+
+This tutorial covers everything you need to know about defining, registering,
+and calling TVM-FFI functions, their exception handling, and working with modules.
+
+Glossary
+--------
+
+TVM-FFI ABI. :cpp:type:`TVMFFISafeCallType`
+ A stable C calling convention where every function is represented by a single signature,
+ which enables type-erased, cross-language function calls.
+ This calling convention is used across all TVM-FFI function calls at the ABI boundary.
+ See :ref:`Stable C ABI <tvm_ffi_c_abi>` for a quick introduction.
+
+TVM-FFI Function. :py:class:`tvm_ffi.Function`, :cpp:class:`tvm::ffi::FunctionObj`, :cpp:class:`tvm::ffi::Function`
+ A reference-counted function object and its managed reference, which wraps any callable,
+ including language-agnostic functions and lambdas (C++, Python, Rust, etc.),
+ member functions, external C symbols, and other callable objects,
+ all sharing the same calling convention.
+
+TVM-FFI Module. :py:class:`tvm_ffi.Module`, :cpp:class:`tvm::ffi::ModuleObj`, :cpp:class:`tvm::ffi::Module`
+ A namespace for a collection of functions, loaded from a shared library via ``dlopen`` (Linux, macOS) or ``LoadLibraryW`` (Windows),
+ or statically linked to the current executable.
+
+Global Functions and Registry. :py:func:`tvm_ffi.get_global_func` and :py:func:`tvm_ffi.register_global_func`
+ A registry is a table that maps string names to :cpp:class:`~tvm::ffi::Function` objects
+ and their metadata (name, docs, signatures, etc.) for cross-language access.
+ Functions in the registry are called **global functions**.
+
+Common Usage
+------------
+
+TVM-FFI C Symbols
+~~~~~~~~~~~~~~~~~
+
+**Shared library**. Use :c:macro:`TVM_FFI_DLL_EXPORT_TYPED_FUNC` to export
+a function as a C symbol that follows the TVM-FFI ABI:
+
+.. code-block:: cpp
+
+ static int AddTwo(int x) { return x + 2; }
+
+ TVM_FFI_DLL_EXPORT_TYPED_FUNC(/*ExportName=*/add_two, /*Function=*/AddTwo)
+
+This creates a C symbol ``__tvm_ffi_<ExportName>`` in the shared library,
+which can then be loaded and called via :py:func:`tvm_ffi.load_module`:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ mod = tvm_ffi.load_module("path/to/library.so")
+ result = mod.add_two(40) # -> 42
+
+**System library**. For symbols bundled in the same executable, use :cpp:func:`TVMFFIEnvModRegisterSystemLibSymbol`
+to register each symbol during static initialization within a :c:macro:`TVM_FFI_STATIC_INIT_BLOCK`.
+See :py:func:`tvm_ffi.system_lib` for a complete workflow.
+
+
+Global Functions
+~~~~~~~~~~~~~~~~
+
+**Register a global function**. In C++, use :cpp:class:`tvm::ffi::reflection::GlobalDef` to
+register a function:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/tvm_ffi.h>
+
+ static int AddOne(int x) { return x + 1; }
+
+ TVM_FFI_STATIC_INIT_BLOCK() {
+ namespace refl = tvm::ffi::reflection;
+ refl::GlobalDef()
+ .def("my_ext.add_one", AddOne, "Add one to the input");
+ }
+
+The :c:macro:`TVM_FFI_STATIC_INIT_BLOCK` macro ensures that registration occurs
+during library initialization. The registered function is then accessible from
+Python by the name ``my_ext.add_one``.
+
+In Python, use the decorator :py:func:`tvm_ffi.register_global_func` to register a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ @tvm_ffi.register_global_func("my_ext.add_one")
+ def add_one(x: int) -> int:
+ return x + 1
+
+**Retrieve a global function**. After registration, functions are accessible by name.
+
+In Python, use :py:func:`tvm_ffi.get_global_func` to retrieve a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ # Get a function from the global registry
+ add_one = tvm_ffi.get_global_func("my_ext.add_one")
+ result = add_one(41) # -> 42
+
+In C++, use :cpp:func:`tvm::ffi::Function::GetGlobal` or :cpp:func:`tvm::ffi::Function::GetGlobalRequired`
+to retrieve a global function:
+
+.. code-block:: cpp
+
+ ffi::Function func = ffi::Function::GetGlobalRequired("my_ext.add_one");
+ int result = func(41) // -> 42
+
+
+Create Functions
+~~~~~~~~~~~~~~~~
+
+**From C++**. An :cpp:class:`tvm::ffi::Function` can be created via :cpp:func:`tvm::ffi::Function::FromTyped`
+or :cpp:class:`tvm::ffi::TypedFunction`'s constructor.
+
+.. code-block:: cpp
+
+ // Create type-erased function: add_type_erased
+ ffi::Function add_type_erased = ffi::Function::FromTyped([](int x, int y) {
+ return x + y;
+ });
+
+ // Create a typed function: add_typed
+ ffi::TypedFunction<int(int, int)> add_typed = [](int x, int y) {
+ return x + y;
+ };
+
+ // Convert a typed function to a type-erased function
+ ffi::Function generic = add_typed;
+
+**From Python**. Any Python :py:class:`Callable <collections.abc.Callable>` is automatically converted
+to a :py:class:`tvm_ffi.Function` at the ABI boundary. The example below demonstrates that in ``my_ext.bind``:
+
+- The input ``func`` is automatically converted to a :py:class:`tvm_ffi.Function`.
+- The returned lambda is also automatically converted to a :py:class:`tvm_ffi.Function`.
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ @tvm_ffi.register_global_func("my_ext.bind")
+ def bind(func, x):
+ assert isinstance(func, tvm_ffi.Function)
+ return lambda *args: func(x, *args) # converted to `tvm_ffi.Function`
+
+ def add_x_y(x, y):
+ return x + y
+
+ func_bind = tvm_ffi.get_global_func("my_ext.bind")
+ add_y = func_bind(add_x_y, 1) # bind x = 1
+ assert isinstance(add_y, tvm_ffi.Function)
+ print(add_y(2)) # -> 3
+
+
+:py:func:`tvm_ffi.convert` explicitly converts a Python callable to :py:class:`tvm_ffi.Function`:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ def add(x, y):
+ return x + y
+
+ func_add = tvm_ffi.convert(add)
+ print(func_add(1, 2))
+
+
+Exception ABI
+-------------
+
+This section describes the exception handling contract in the TVM-FFI Stable C ABI.
+Exceptions are first-class citizens in TVM-FFI, and this section specifies:
+
+- How to properly throw exceptions from a TVM-FFI ABI function
+- How to check for and propagate exceptions from a TVM-FFI ABI function
+
+TVM-FFI C ABI
+~~~~~~~~~~~~~
+
+All TVM-FFI functions ultimately conform to the :cpp:type:`TVMFFISafeCallType` signature,
+which provides a stable C ABI for cross-language calls. The C calling convention is defined as:
+
+.. code-block:: cpp
+
+ int tvm_ffi_c_abi(
+ void* handle, // Resource handle
+ const TVMFFIAny* args, // Input arguments (non-owning)
+ int32_t num_args, // Number of input arguments
+ TVMFFIAny* result // Output argument (owning, zero-initialized)
+ );
+
+**Input arguments**. The input arguments are passed as an array of :cpp:class:`tvm::ffi::AnyView` values,
+specified by ``args`` and ``num_args``.
+
+**Output argument**. The output argument ``result`` is an owning :cpp:type:`tvm::ffi::Any`
+that the caller must zero-initialize before the call.
+
+**Return value**. The ABI returns an **error code** that indicates:
+
+- ``0``: Success
+- ``-1``: Error occurred, retrievable with :cpp:func:`TVMFFIErrorMoveFromRaised`
+- ``-2``: Very rare frontend error
+
+.. hint::
+ See :doc:`Any <any>` for more details on the semantics of :cpp:type:`tvm::ffi::AnyView` and :cpp:type:`tvm::ffi::Any`.
+
+Retrieve Errors in C
+~~~~~~~~~~~~~~~~~~~~
+
+When a TVM-FFI function returns a non-zero code, it indicates that an error occurred
+and a :cpp:class:`tvm::ffi::ErrorObj` is stored in thread-local storage (TLS).
+This section shows how to retrieve the error object and print the error message and backtrace.
+
+.. note::
+
+ An :cpp:class:`~tvm::ffi::ErrorObj` is a :cpp:class:`~tvm::ffi::Object` with a :cpp:class:`TVMFFIErrorCell` payload
+ as defined below:
+
+ .. code-block:: cpp
+
+ typedef struct {
+ TVMFFIByteArray kind; // Error type (e.g., "ValueError")
+ TVMFFIByteArray message; // Error message
+ TVMFFIByteArray backtrace; // Stack trace (most-recent call first)
+ void (*update_backtrace)(...); // Hook to append/replace backtrace
+ } TVMFFIErrorCell;
+
+**Print an Error**. The example code below shows how to print an error message and backtrace.
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/c_api.h>
+
+ void PrintError(TVMFFIObject* err) {
+ TVMFFIErrorCell* cell = (TVMFFIErrorCell*)((char*)err + sizeof(TVMFFIObject));
+ fprintf(stderr, "%.*s: %.*s\n", (int)cell->kind.size, cell->kind.data, (int)cell->message.size, cell->message.data);
+ if (cell->backtrace.size) {
+ fprintf(stderr, "Backtrace:\n%.*s\n", (int)cell->backtrace.size, cell->backtrace.data);
+ }
+ }
+
+The payload of the error object is a :cpp:type:`TVMFFIErrorCell` structure
+containing the error kind, message, and backtrace. It can be accessed
+by skipping the :cpp:type:`TVMFFIObject` header using pointer arithmetic.
+
+**Retrieve the error object**. When the error code is ``-1``, the error object is stored in TLS
+and can be retrieved with :cpp:func:`TVMFFIErrorMoveFromRaised`.
+
+.. code-block:: cpp
+
+ void HandleReturnCode(int rc) {
+ TVMFFIObject* err = NULL;
+ if (rc == 0) {
+ // Success
+ } else if (rc == -1) {
+ // Move the raised error from TLS (clears TLS slot)
+ TVMFFIErrorMoveFromRaised(&err); // now `err` owns the error object
+ if (err != NULL) {
+ PrintError(err); // print the error
+ TVMFFIObjectDecRef(err); // Release the error object
+ }
+ } else if (rc == -2) {
+ // Frontend (e.g., Python) already has an exception set.
+ // Do not fetch from TLS; consult the frontend's error mechanism.
+ }
+ }
+
+This function transfers ownership of the error object to the caller and clears the TLS slot.
+You must call :cpp:func:`TVMFFIObjectDecRef` to release the object when done to avoid memory leaks.
+
+**Rare frontend errors**. Error code ``-2`` is reserved for rare frontend errors. It is returned only
+when the C API :cpp:func:`TVMFFIEnvCheckSignals` returns non-zero during execution, indicating that
+the Python side has a pending signal requiring attention. In this case, the caller should not fetch
+the error object from TLS but instead consult the frontend's error mechanism to handle the exception.
+
+Raise Errors in C
+~~~~~~~~~~~~~~~~~
+
+As part of TVM-FFI's calling convention, returning ``-1`` indicates that an error occurred
+and the error object is stored in the TLS slot. The error object can contain arbitrary
+user-defined information, such as error messages, backtraces, or Python frame-local variables.
+
+.. hint::
+ Compiler code generation may use similar patterns to raise errors in generated code.
+
+The example below sets the TLS error and returns ``-1`` using :cpp:func:`TVMFFIErrorSetRaisedFromCStr`:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/c_api.h>
+
+ int __tvm_ffi_my_kernel(void* handle, const TVMFFIAny* args,
+ int32_t num_args, TVMFFIAny* result) {
+ // Validate inputs
+ if (num_args < 2) {
+ TVMFFIErrorSetRaisedFromCStr("ValueError", "Expected at least 2 arguments");
+ return -1;
+ }
+ // ... kernel implementation ...
+ return 0;
+ }
+
+Alternatively, :cpp:func:`TVMFFIErrorSetRaisedFromCStrParts` accepts explicit string lengths,
+which is useful when the error kind and message are not null-terminated.
+
+**Propagating errors**. For chains of generated calls, simply propagate return codes—TLS carries
+the error details:
+
+.. code-block:: cpp
+
+ int outer_function(...) {
+ int err_code = 0;
+
+ err_code = inner_function(...);
+ if (err_code != 0) goto RAII; // Propagate error; TLS has the details
+
+ RAII:
Review Comment:
Sorry it's **my** style
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For additional commands, e-mail: discuss-archive-help@tvm.apache.org
Re: [PR] doc: Function and Module [tvm-ffi]
Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao commented on code in PR #394:
URL: https://github.com/apache/tvm-ffi/pull/394#discussion_r2678848944
##########
docs/concepts/func_module.rst:
##########
@@ -0,0 +1,584 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+Function and Module
+===================
+
+TVM-FFI provides a unified and ABI-stable calling convention that enables
+cross-language function calls between C++, Python, Rust and other languages.
+Functions are first-class :doc:`TVM-FFI objects <object_and_class>` that can be:
+
+- **Invoked with type-erased arguments** using a stable C ABI
+- **Registered globally** and accessed by name across languages
+- **Exported as C symbols** for efficient codegen and linking
+- **Wrapped in modules** for dynamic loading and namespace isolation
+
+This tutorial covers everything you need to know about defining, registering, and calling
+TVM-FFI functions, as well as working with modules.
+
+Glossary
+--------
+
+TVM-FFI ABI. :cpp:type:`TVMFFISafeCallType`
+ A stable C calling convention where every function is represented by a single signature,
+ which enables type-erased, cross-language function calls.
+ This calling convention is used across all TVM-FFI function calls at the ABI boundary.
+ See :ref:`Stable C ABI <tvm_ffi_c_abi>` for a quick introduction.
+
+TVM-FFI Function. :py:class:`tvm_ffi.Function`, :cpp:class:`tvm::ffi::FunctionObj`, :cpp:class:`tvm::ffi::Function`
+ A reference-counted function object and its managed reference, which wraps any callable,
+ including language-agnostic functions and lambdas (C++, Python, Rust, etc.),
+ member functions, external C symbols, and other callable objects,
+ all sharing the same calling convention.
+
+TVM-FFI Module. :py:class:`tvm_ffi.Module`, :cpp:class:`tvm::ffi::ModuleObj`, :cpp:class:`tvm::ffi::Module`
+ A namespace for a collection of functions, loaded from a shared library via ``dlopen`` (Linux, macOS) or ``LoadLibraryW`` (Windows),
+ or statically linked to the current executable.
+
+Global Functions and Registry. :py:func:`tvm_ffi.get_global_func` and :py:func:`tvm_ffi.register_global_func`
+ A registry is a table that maps string names to :cpp:class:`~tvm::ffi::Function` objects
+ and their metadata (name, docs, signatures, etc.) for cross-language access.
+ Functions in the registry are called **global functions**.
+
+Common Usage
+------------
+
+TVM-FFI C Symbols
+~~~~~~~~~~~~~~~~~
+
+**Shared library**. Use :c:macro:`TVM_FFI_DLL_EXPORT_TYPED_FUNC` to export
+a function as a C symbol that follows the TVM-FFI ABI:
+
+.. code-block:: cpp
+
+ static int AddTwo(int x) { return x + 2; }
+
+ TVM_FFI_DLL_EXPORT_TYPED_FUNC(/*ExportName=*/add_two, /*Function=*/AddTwo)
+
+This creates a C symbol ``__tvm_ffi_${ExportName}`` if the symbol is exported in a shared library,
+and later loaded and called via :py:func:`tvm_ffi.load_module`:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ mod = tvm_ffi.load_module("path/to/library.so")
+ result = mod.add_two(40) # -> 42
+
+**System library**. If the symbol is bundled in the same executable, :cpp:func:`TVMFFIEnvModRegisterSystemLibSymbol`
+is required to register the symbol during static initialization in :c:macro:`TVM_FFI_STATIC_INIT_BLOCK`.
+See examples in :py:func:`tvm_ffi.system_lib` for a complete workflow.
+
+
+Global Functions
+~~~~~~~~~~~~~~~~
+
+**Register a global function**. In C++, use :cpp:class:`tvm::ffi::reflection::GlobalDef` to
+register a function:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/tvm_ffi.h>
+
+ static int AddOne(int x) { return x + 1; }
+
+ TVM_FFI_STATIC_INIT_BLOCK() {
+ namespace refl = tvm::ffi::reflection;
+ refl::GlobalDef()
+ .def("my_ext.add_one", AddOne, "Add one to the input");
+ }
+
+:c:macro:`TVM_FFI_STATIC_INIT_BLOCK` macro ensures the registration logic happens
+during library initialization. The registered function is then accessible from
+Python under the name ``my_ext.add_one``.
+
+In Python, use the decorator :py:func:`tvm_ffi.register_global_func` to register a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ @tvm_ffi.register_global_func("my_ext.add_one")
+ def add_one(x: int) -> int:
+ return x + 1
+
+**Retrieve a global function**. After registration, functions are accessible by name.
+
+In Python, use :py:func:`tvm_ffi.get_global_func` to retrieve a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ # Get a function from the global registry
+ add_one = tvm_ffi.get_global_func("my_ext.add_one")
+ result = add_one(41) # -> 42
+
+In C++, use :cpp:func:`tvm::ffi::Function::GetGlobal` to retrieve a global function:
+
+.. code-block:: cpp
+
+ ffi::Function func = ffi::Function::GetGlobal("my_ext.add_one");
+ int result = func(41) // -> 42
+
+
+Create Functions
+~~~~~~~~~~~~~~~~
+
+**From C++**. An :cpp:class:`tvm::ffi::Function` can be created via :cpp:func:`tvm::ffi::Function::FromTyped`
+or :cpp:class:`tvm::ffi::TypedFunction`'s constructor.
+
+.. code-block:: cpp
+
+ // Create type-erased function: add_type_erased
+ ffi::Function add_type_erased = ffi::Function::FromTyped([](int x, int y) {
+ return x + y;
+ });
+
+ // Create a typed function: add_typed
+ ffi::TypedFunction<int(int, int)> add_typed = [](int x, int y) {
+ return x + y;
+ };
+
+ // Converts typed function to type-erased function
+ ffi::Function generic = add;
+
+**From Python**. An arbitrary Python :py:class:`Callable <collections.abc.Callable>` can be automatically converted
+to a :py:class:`tvm_ffi.Function` at ABI boundary. The example below shows that in function ``my_ext.bind``:
+
+- Its input ``func`` is automatically converted to :py:class:`tvm_ffi.Function`;
+- Its output, a lambda function is automatically converted to :py:class:`tvm_ffi.Function`.
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ @tvm_ffi.register_global_func("my_ext.bind")
+ def bind(func, x):
+ assert isinstance(func, tvm_ffi.Function)
+ return lambda *args: func(x, *args) # converted to `tvm_ffi.Function`
+
+ def add_x_y(x, y):
+ return x + y
+
+ func_bind = tvm_ffi.get_global_func("my_ext.bind")
+ add_y = func_bind(add_x_y, 1) # bind x = 1
+ assert isinstance(add_y, tvm_ffi.Function)
+ print(add_y(2)) # -> 3
+
+
+:py:func:`tvm_ffi.convert` explicitly converts a Python callable to :py:class:`tvm_ffi.Function`:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ def add(x, y):
+ return x + y
+
+ func_add = tvm_ffi.convert(add)
+ print(func_add(1, 2))
+
+
+C ABI Calling Convention
+------------------------
+
+.. important::
+
+ This section focuses on in-depth of TVM-FFI's stable C ABI definition,
+ describing it in pure C for precision and clarity.
+
+ Compiler code generation may follow similar patterns to call functions with TVM-FFI ABI,
+ or to generate functions that conform to TVM-FFI ABI.
+
+All TVM-FFI functions ultimately conform to the :cpp:type:`TVMFFISafeCallType` signature,
+which provides a stable C ABI for cross-language calls. The C calling convention is defined as:
+
+.. code-block:: cpp
+
+ int tvm_ffi_c_abi(
+ void* handle, // Resource handle
+ const TVMFFIAny* args, // Inputs (non-owning)
+ int num_args, // Number of arguments
+ TVMFFIAny* result // Output (owning)
+ );
+
+.. note::
+ As part of the contract, the return value ``result`` should be zero-initialized by the caller.
+
+The returning integer is **error code**, which indicates
+
+- ``0``: Success
+- ``-1``: Error occurred, retrieve via :cpp:func:`TVMFFIErrorMoveFromRaised`
+- ``-2``: Frontend error (e.g., Python exception)
+
+
+Input and Output Arguments
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The input and output arguments are passed as :cpp:type:`TVMFFIAny` pointers.
+
+- Input arguments are an array of ``num_args`` non-owning :cpp:type:`tvm::ffi::AnyView`;
+- Output argument is an owning :cpp:type:`tvm::ffi::Any`, which the caller has to guarantee to be zero-initialized.
+
+See :doc:`Any <any>` for more details on the semantics of :cpp:type:`tvm::ffi::AnyView` and :cpp:type:`tvm::ffi::Any`.
+
+Retrieve Errors in C
+~~~~~~~~~~~~~~~~~~~~
+
+When a TVM-FFI function returns a non-zero code, it indicates an error occurred,
+and the error object is stored in the TLS slot. This section shows how to retrieve
+the error object and print the error message and backtrace.
+
+.. hint::
+ Compiler code generation may use similar patterns to call any function with TVM-FFI ABI.
+
+
+**Print an Error**. The example code below shows how to print an error message and backtrace.
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/c_api.h>
+
+ static void PrintError(TVMFFIObject* err) {
+ TVMFFIErrorCell* cell = (TVMFFIErrorCell*)((char*)err + sizeof(TVMFFIObject));
+ fprintf(stderr, "%.*s: %.*s\n", (int)cell->kind.size, cell->kind.data, (int)cell->message.size, cell->message.data);
+ if (cell->backtrace.size) {
+ fprintf(stderr, "Backtrace:\n%.*s\n", (int)cell->backtrace.size, cell->backtrace.data);
+ }
+ }
+
+The payload of the error object is a :cpp:type:`TVMFFIErrorCell` structure,
+which contains the error kind, message, and backtrace, which can be accessed
+by skipping the :cpp:type:`TVMFFIObject` header with pointer arithmetics.
+
+.. note::
+
+ An :cpp:class:`~tvm::ffi::Error` object is a :cpp:class:`~tvm::ffi::Object` with a :cpp:class:`TVMFFIErrorCell` payload
+ as defined below:
+
+ .. code-block:: cpp
+
+ typedef struct {
+ TVMFFIByteArray kind; // Error type (e.g., "ValueError")
+ TVMFFIByteArray message; // Error message
+ TVMFFIByteArray backtrace; // Stack trace (most-recent call first)
+ void (*update_backtrace)(...); // Hook to append/replace backtrace
+ } TVMFFIErrorCell;
+
+
+**Retrieve error object**. When error code is ``-1``, the error object is stored in the TLS slot,
+and you can retrieve it with :cpp:func:`TVMFFIErrorMoveFromRaised`.
+
+.. code-block:: cpp
+
+ void HandleReturnCode(int rc) {
+ TVMFFIObject* err = NULL;
+ if (rc == 0) {
+ // Success
+ }
+ else if (rc == -1) {
+ // Move the raised error from TLS (clears TLS slot)
+ TVMFFIErrorMoveFromRaised(&err); // now `err` owns the error object
+ if (err != NULL) {
+ PrintError(err); // print the error
+ TVMFFIObjectDecRef(err); // Release the error object
+ }
+ }
+ else if (rc == -2) {
+ // Frontend (e.g., Python) already has an exception set.
+ // Do not fetch from TLS; consult the frontend's error mechanism.
+ }
+ }
+
+This method transfers ownership of the error object to the caller and clears the TLS slot,
+and therefore you must call :cpp:func:`TVMFFIObjectDecRef` to release the object when done
+to avoid memory leaks.
+
+Raise Errors in C
+~~~~~~~~~~~~~~~~~
+
+As part of TVM-FFI's calling convention, returning ``-1`` indicates an error occurred,
+and the error object is stored in the TLS slot. The error object may contain abitrary
+user-defined information, for example, error message, backtrace, or Python frame-local variables.
+
+.. hint::
+ Compiler code generation may use similar patterns to raise errors in generated code.
+
+The example below sets the TLS error and returns ``-1`` using :cpp:func:`TVMFFIErrorSetRaisedFromCStr`:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/c_api.h>
+
+ int __tvm_ffi_my_kernel(void* handle, const TVMFFIAny* args,
+ int32_t num_args, TVMFFIAny* result) {
+ // Validate inputs
+ if (num_args < 2) {
+ TVMFFIErrorSetRaisedFromCStr("ValueError", "Expected at least 2 arguments");
+ return -1;
+ }
+ // ... kernel implementation ...
+ return 0;
+ }
+
+Alternatively, :cpp:func:`TVMFFIErrorSetRaisedFromCStrParts` can also be used to concatenate
+multiple parts of the error message into a single string.
+
+**Propagating errors**. For chains of generated calls, simply propagate return codes - TLS carries
+the error details:
+
+.. code-block:: cpp
+
+ int outer_function(...) {
+ int err_code = 0;
+
+ err_code = inner_function(...);
+ if (err_code != 0) goto RAII; // Propagate error; TLS has the details
+
+ RAII:
+ // clean up owned resources
+ return err_code;
+ }
+
+Handling Non-C++ Errors
+~~~~~~~~~~~~~~~~~~~~~~~
+
+TVM-FFI preserves frontend errors (e.g., Python exceptions) through a dedicated ``-2`` return path.
+
+When a Python exception occurs during a TVM-FFI call:
+
+- The exception remains a native Python exception with its original traceback
+- The C++ side returns ``-2`` to indicate "frontend error already set"
+- Python retains the real exception object with actual frame locals
+
+This path preserves Python stack variables because the exception is never serialized into
+a TVM-FFI Error object.
+
+Function
+--------
+
+Layout and ABI
+~~~~~~~~~~~~~~
+
+:cpp:class:`tvm::ffi::FunctionObj` stores two call pointers in :cpp:class:`TVMFFIFunctionCell`:
+
+.. code-block:: cpp
+
+ typedef struct {
+ TVMFFISafeCallType safe_call;
+ void* cpp_call;
+ } TVMFFIFunctionCell;
+
+``safe_call`` is used for cross-ABI function calling, which intercepts and stores exceptions in TLS,
+while ``cpp_call`` is used inside the same DSO, and exceptions are thrown directly for performance.
+
+.. important::
+
+ Do not directly use ``safe_call`` or ``cpp_call`` to call functions. Instead, a C API :cpp:func:`TVMFFIFunctionCall`
+ is provided to handle error propagation and resource management.
+
+**Conversion with Any**. :py:class:`tvm_ffi.Function` is a TVM-FFI object, and thus conversion follows
+exactly the same rule as a TVM-FFI object does. See :ref:`Object Conversion with Any <object-conversion-with-any>` for more details.
+
+
+Throw and Catch Errors
+~~~~~~~~~~~~~~~~~~~~~~
+
+TVM-FFI gracefully handles exceptions across language boundaries without having to manually
+set error codes.
+
+**Python**. In Python, raise its language-native :py:class:`Exception <Exception>` and its
+derived classes, and TVM-FFI will catch it at ABI boundary, and convert it to a
+:cpp:class:`tvm::ffi::Error` object. If C++ code is calling into Python, the Python exception
+will be propagated back and re-raised as a :cpp:class:`tvm::ffi::Error` object to C++ side,
+and the C++ code can handle it appropriately.
+
+**C++**. In C++, use :cpp:class:`tvm::ffi::Error` or the :c:macro:`TVM_FFI_THROW` macro:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/error.h>
+
+ void ThrowError(int x) {
+ if (x < 0) {
+ TVM_FFI_THROW(ValueError) << "x must be non-negative, got " << x;
+ }
+ }
+
+The example above shows how to throw an error in C++. :c:macro:`TVM_FFI_THROW` captures
+the current file name, line number, stacktrace, and error message, and then constructs a
+:cpp:class:`tvm::ffi::Error` object. At ABI boundary, this :cpp:class:`tvm::ffi::Error`
+object is stored in the TLS slot, and the function returns ``-1`` as error code in the C ABI
+:cpp:type:`TVMFFISafeCallType` signature.
+
+.. hint::
+ A detailed implementation of such graceful handling behavior can be found
Review Comment:
It's needed for codeblock but not hint block
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For additional commands, e-mail: discuss-archive-help@tvm.apache.org
Re: [PR] doc: Function, Exception and Module [tvm-ffi]
Posted by "gemini-code-assist[bot] (via GitHub)" <gi...@apache.org>.
gemini-code-assist[bot] commented on code in PR #394:
URL: https://github.com/apache/tvm-ffi/pull/394#discussion_r2679104323
##########
docs/concepts/func_module.rst:
##########
@@ -0,0 +1,564 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+Function, Exception and Module
+==============================
+
+TVM-FFI provides a unified and ABI-stable calling convention that enables
+cross-language function calls between C++, Python, Rust, and other languages.
+Functions are first-class :doc:`TVM-FFI objects <object_and_class>`.
+
+This tutorial covers everything you need to know about defining, registering,
+and calling TVM-FFI functions, their exception handling, and working with modules.
+
+Glossary
+--------
+
+TVM-FFI ABI. :cpp:type:`TVMFFISafeCallType`
+ A stable C calling convention where every function is represented by a single signature,
+ which enables type-erased, cross-language function calls.
+ This calling convention is used across all TVM-FFI function calls at the ABI boundary.
+ See :ref:`Stable C ABI <tvm_ffi_c_abi>` for a quick introduction.
+
+TVM-FFI Function. :py:class:`tvm_ffi.Function`, :cpp:class:`tvm::ffi::FunctionObj`, :cpp:class:`tvm::ffi::Function`
+ A reference-counted function object and its managed reference, which wraps any callable,
+ including language-agnostic functions and lambdas (C++, Python, Rust, etc.),
+ member functions, external C symbols, and other callable objects,
+ all sharing the same calling convention.
+
+TVM-FFI Module. :py:class:`tvm_ffi.Module`, :cpp:class:`tvm::ffi::ModuleObj`, :cpp:class:`tvm::ffi::Module`
+ A namespace for a collection of functions, loaded from a shared library via ``dlopen`` (Linux, macOS) or ``LoadLibraryW`` (Windows),
+ or statically linked to the current executable.
+
+Global Functions and Registry. :py:func:`tvm_ffi.get_global_func` and :py:func:`tvm_ffi.register_global_func`
+ A registry is a table that maps string names to :cpp:class:`~tvm::ffi::Function` objects
+ and their metadata (name, docs, signatures, etc.) for cross-language access.
+ Functions in the registry are called **global functions**.
+
+Common Usage
+------------
+
+TVM-FFI C Symbols
+~~~~~~~~~~~~~~~~~
+
+**Shared library**. Use :c:macro:`TVM_FFI_DLL_EXPORT_TYPED_FUNC` to export
+a function as a C symbol that follows the TVM-FFI ABI:
+
+.. code-block:: cpp
+
+ static int AddTwo(int x) { return x + 2; }
+
+ TVM_FFI_DLL_EXPORT_TYPED_FUNC(/*ExportName=*/add_two, /*Function=*/AddTwo)
+
+This creates a C symbol ``__tvm_ffi_<ExportName>`` in the shared library,
+which can then be loaded and called via :py:func:`tvm_ffi.load_module`:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ mod = tvm_ffi.load_module("path/to/library.so")
+ result = mod.add_two(40) # -> 42
+
+**System library**. For symbols bundled in the same executable, use :cpp:func:`TVMFFIEnvModRegisterSystemLibSymbol`
+to register each symbol during static initialization within a :c:macro:`TVM_FFI_STATIC_INIT_BLOCK`.
+See :py:func:`tvm_ffi.system_lib` for a complete workflow.
+
+
+Global Functions
+~~~~~~~~~~~~~~~~
+
+**Register a global function**. In C++, use :cpp:class:`tvm::ffi::reflection::GlobalDef` to
+register a function:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/tvm_ffi.h>
+
+ static int AddOne(int x) { return x + 1; }
+
+ TVM_FFI_STATIC_INIT_BLOCK() {
+ namespace refl = tvm::ffi::reflection;
+ refl::GlobalDef()
+ .def("my_ext.add_one", AddOne, "Add one to the input");
+ }
+
+The :c:macro:`TVM_FFI_STATIC_INIT_BLOCK` macro ensures that registration occurs
+during library initialization. The registered function is then accessible from
+Python by the name ``my_ext.add_one``.
+
+In Python, use the decorator :py:func:`tvm_ffi.register_global_func` to register a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ @tvm_ffi.register_global_func("my_ext.add_one")
+ def add_one(x: int) -> int:
+ return x + 1
+
+**Retrieve a global function**. After registration, functions are accessible by name.
+
+In Python, use :py:func:`tvm_ffi.get_global_func` to retrieve a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ # Get a function from the global registry
+ add_one = tvm_ffi.get_global_func("my_ext.add_one")
+ result = add_one(41) # -> 42
+
+In C++, use :cpp:func:`tvm::ffi::Function::GetGlobal` or :cpp:func:`tvm::ffi::Function::GetGlobalRequired`
+to retrieve a global function:
+
+.. code-block:: cpp
+
+ ffi::Function func = ffi::Function::GetGlobalRequired("my_ext.add_one");
+ int result = func(41) // -> 42
Review Comment:

This line in the C++ code example is missing a semicolon at the end. Without it, the code is syntactically incorrect and would not compile.
```suggestion
int result = func(41); // -> 42
```
##########
docs/concepts/func_module.rst:
##########
@@ -0,0 +1,564 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+Function, Exception and Module
+==============================
+
+TVM-FFI provides a unified and ABI-stable calling convention that enables
+cross-language function calls between C++, Python, Rust, and other languages.
+Functions are first-class :doc:`TVM-FFI objects <object_and_class>`.
+
+This tutorial covers everything you need to know about defining, registering,
+and calling TVM-FFI functions, their exception handling, and working with modules.
+
+Glossary
+--------
+
+TVM-FFI ABI. :cpp:type:`TVMFFISafeCallType`
+ A stable C calling convention where every function is represented by a single signature,
+ which enables type-erased, cross-language function calls.
+ This calling convention is used across all TVM-FFI function calls at the ABI boundary.
+ See :ref:`Stable C ABI <tvm_ffi_c_abi>` for a quick introduction.
+
+TVM-FFI Function. :py:class:`tvm_ffi.Function`, :cpp:class:`tvm::ffi::FunctionObj`, :cpp:class:`tvm::ffi::Function`
+ A reference-counted function object and its managed reference, which wraps any callable,
+ including language-agnostic functions and lambdas (C++, Python, Rust, etc.),
+ member functions, external C symbols, and other callable objects,
+ all sharing the same calling convention.
+
+TVM-FFI Module. :py:class:`tvm_ffi.Module`, :cpp:class:`tvm::ffi::ModuleObj`, :cpp:class:`tvm::ffi::Module`
+ A namespace for a collection of functions, loaded from a shared library via ``dlopen`` (Linux, macOS) or ``LoadLibraryW`` (Windows),
+ or statically linked to the current executable.
+
+Global Functions and Registry. :py:func:`tvm_ffi.get_global_func` and :py:func:`tvm_ffi.register_global_func`
+ A registry is a table that maps string names to :cpp:class:`~tvm::ffi::Function` objects
+ and their metadata (name, docs, signatures, etc.) for cross-language access.
+ Functions in the registry are called **global functions**.
+
+Common Usage
+------------
+
+TVM-FFI C Symbols
+~~~~~~~~~~~~~~~~~
+
+**Shared library**. Use :c:macro:`TVM_FFI_DLL_EXPORT_TYPED_FUNC` to export
+a function as a C symbol that follows the TVM-FFI ABI:
+
+.. code-block:: cpp
+
+ static int AddTwo(int x) { return x + 2; }
+
+ TVM_FFI_DLL_EXPORT_TYPED_FUNC(/*ExportName=*/add_two, /*Function=*/AddTwo)
+
+This creates a C symbol ``__tvm_ffi_<ExportName>`` in the shared library,
+which can then be loaded and called via :py:func:`tvm_ffi.load_module`:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ mod = tvm_ffi.load_module("path/to/library.so")
+ result = mod.add_two(40) # -> 42
+
+**System library**. For symbols bundled in the same executable, use :cpp:func:`TVMFFIEnvModRegisterSystemLibSymbol`
+to register each symbol during static initialization within a :c:macro:`TVM_FFI_STATIC_INIT_BLOCK`.
+See :py:func:`tvm_ffi.system_lib` for a complete workflow.
+
+
+Global Functions
+~~~~~~~~~~~~~~~~
+
+**Register a global function**. In C++, use :cpp:class:`tvm::ffi::reflection::GlobalDef` to
+register a function:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/tvm_ffi.h>
+
+ static int AddOne(int x) { return x + 1; }
+
+ TVM_FFI_STATIC_INIT_BLOCK() {
+ namespace refl = tvm::ffi::reflection;
+ refl::GlobalDef()
+ .def("my_ext.add_one", AddOne, "Add one to the input");
+ }
+
+The :c:macro:`TVM_FFI_STATIC_INIT_BLOCK` macro ensures that registration occurs
+during library initialization. The registered function is then accessible from
+Python by the name ``my_ext.add_one``.
+
+In Python, use the decorator :py:func:`tvm_ffi.register_global_func` to register a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ @tvm_ffi.register_global_func("my_ext.add_one")
+ def add_one(x: int) -> int:
+ return x + 1
+
+**Retrieve a global function**. After registration, functions are accessible by name.
+
+In Python, use :py:func:`tvm_ffi.get_global_func` to retrieve a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ # Get a function from the global registry
+ add_one = tvm_ffi.get_global_func("my_ext.add_one")
+ result = add_one(41) # -> 42
+
+In C++, use :cpp:func:`tvm::ffi::Function::GetGlobal` or :cpp:func:`tvm::ffi::Function::GetGlobalRequired`
+to retrieve a global function:
+
+.. code-block:: cpp
+
+ ffi::Function func = ffi::Function::GetGlobalRequired("my_ext.add_one");
+ int result = func(41) // -> 42
+
+
+Create Functions
+~~~~~~~~~~~~~~~~
+
+**From C++**. An :cpp:class:`tvm::ffi::Function` can be created via :cpp:func:`tvm::ffi::Function::FromTyped`
+or :cpp:class:`tvm::ffi::TypedFunction`'s constructor.
+
+.. code-block:: cpp
+
+ // Create type-erased function: add_type_erased
+ ffi::Function add_type_erased = ffi::Function::FromTyped([](int x, int y) {
+ return x + y;
+ });
+
+ // Create a typed function: add_typed
+ ffi::TypedFunction<int(int, int)> add_typed = [](int x, int y) {
+ return x + y;
+ };
+
+ // Convert a typed function to a type-erased function
+ ffi::Function generic = add_typed;
+
+**From Python**. Any Python :py:class:`Callable <collections.abc.Callable>` is automatically converted
+to a :py:class:`tvm_ffi.Function` at the ABI boundary. The example below demonstrates that in ``my_ext.bind``:
+
+- The input ``func`` is automatically converted to a :py:class:`tvm_ffi.Function`.
+- The returned lambda is also automatically converted to a :py:class:`tvm_ffi.Function`.
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ @tvm_ffi.register_global_func("my_ext.bind")
+ def bind(func, x):
+ assert isinstance(func, tvm_ffi.Function)
+ return lambda *args: func(x, *args) # converted to `tvm_ffi.Function`
+
+ def add_x_y(x, y):
+ return x + y
+
+ func_bind = tvm_ffi.get_global_func("my_ext.bind")
+ add_y = func_bind(add_x_y, 1) # bind x = 1
+ assert isinstance(add_y, tvm_ffi.Function)
+ print(add_y(2)) # -> 3
+
+
+:py:func:`tvm_ffi.convert` explicitly converts a Python callable to :py:class:`tvm_ffi.Function`:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ def add(x, y):
+ return x + y
+
+ func_add = tvm_ffi.convert(add)
+ print(func_add(1, 2))
+
+
+Exception ABI
+-------------
+
+This section describes the exception handling contract in the TVM-FFI Stable C ABI.
+Exceptions are first-class citizens in TVM-FFI, and this section specifies:
+
+- How to properly throw exceptions from a TVM-FFI ABI function
+- How to check for and propagate exceptions from a TVM-FFI ABI function
+
+TVM-FFI C ABI
+~~~~~~~~~~~~~
+
+All TVM-FFI functions ultimately conform to the :cpp:type:`TVMFFISafeCallType` signature,
+which provides a stable C ABI for cross-language calls. The C calling convention is defined as:
+
+.. code-block:: cpp
+
+ int tvm_ffi_c_abi(
+ void* handle, // Resource handle
+ const TVMFFIAny* args, // Input arguments (non-owning)
+ int32_t num_args, // Number of input arguments
+ TVMFFIAny* result // Output argument (owning, zero-initialized)
+ );
+
+**Input arguments**. The input arguments are passed as an array of :cpp:class:`tvm::ffi::AnyView` values,
+specified by ``args`` and ``num_args``.
+
+**Output argument**. The output argument ``result`` is an owning :cpp:type:`tvm::ffi::Any`
+that the caller must zero-initialize before the call.
+
+**Return value**. The ABI returns an **error code** that indicates:
+
+- ``0``: Success
+- ``-1``: Error occurred, retrievable with :cpp:func:`TVMFFIErrorMoveFromRaised`
+- ``-2``: Very rare frontend error
+
+.. hint::
+ See :doc:`Any <any>` for more details on the semantics of :cpp:type:`tvm::ffi::AnyView` and :cpp:type:`tvm::ffi::Any`.
+
+Retrieve Errors in C
+~~~~~~~~~~~~~~~~~~~~
+
+When a TVM-FFI function returns a non-zero code, it indicates that an error occurred
+and a :cpp:class:`tvm::ffi::ErrorObj` is stored in thread-local storage (TLS).
+This section shows how to retrieve the error object and print the error message and backtrace.
+
+.. note::
+
+ An :cpp:class:`~tvm::ffi::ErrorObj` is a :cpp:class:`~tvm::ffi::Object` with a :cpp:class:`TVMFFIErrorCell` payload
+ as defined below:
+
+ .. code-block:: cpp
+
+ typedef struct {
+ TVMFFIByteArray kind; // Error type (e.g., "ValueError")
+ TVMFFIByteArray message; // Error message
+ TVMFFIByteArray backtrace; // Stack trace (most-recent call first)
+ void (*update_backtrace)(...); // Hook to append/replace backtrace
+ } TVMFFIErrorCell;
+
+**Print an Error**. The example code below shows how to print an error message and backtrace.
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/c_api.h>
+
+ void PrintError(TVMFFIObject* err) {
+ TVMFFIErrorCell* cell = (TVMFFIErrorCell*)((char*)err + sizeof(TVMFFIObject));
+ fprintf(stderr, "%.*s: %.*s\n", (int)cell->kind.size, cell->kind.data, (int)cell->message.size, cell->message.data);
+ if (cell->backtrace.size) {
+ fprintf(stderr, "Backtrace:\n%.*s\n", (int)cell->backtrace.size, cell->backtrace.data);
+ }
+ }
+
+The payload of the error object is a :cpp:type:`TVMFFIErrorCell` structure
+containing the error kind, message, and backtrace. It can be accessed
+by skipping the :cpp:type:`TVMFFIObject` header using pointer arithmetic.
+
+**Retrieve the error object**. When the error code is ``-1``, the error object is stored in TLS
+and can be retrieved with :cpp:func:`TVMFFIErrorMoveFromRaised`.
+
+.. code-block:: cpp
+
+ void HandleReturnCode(int rc) {
+ TVMFFIObject* err = NULL;
+ if (rc == 0) {
+ // Success
+ } else if (rc == -1) {
+ // Move the raised error from TLS (clears TLS slot)
+ TVMFFIErrorMoveFromRaised(&err); // now `err` owns the error object
+ if (err != NULL) {
+ PrintError(err); // print the error
+ TVMFFIObjectDecRef(err); // Release the error object
+ }
+ } else if (rc == -2) {
+ // Frontend (e.g., Python) already has an exception set.
+ // Do not fetch from TLS; consult the frontend's error mechanism.
+ }
+ }
+
+This function transfers ownership of the error object to the caller and clears the TLS slot.
+You must call :cpp:func:`TVMFFIObjectDecRef` to release the object when done to avoid memory leaks.
+
+**Rare frontend errors**. Error code ``-2`` is reserved for rare frontend errors. It is returned only
+when the C API :cpp:func:`TVMFFIEnvCheckSignals` returns non-zero during execution, indicating that
+the Python side has a pending signal requiring attention. In this case, the caller should not fetch
+the error object from TLS but instead consult the frontend's error mechanism to handle the exception.
+
+Raise Errors in C
+~~~~~~~~~~~~~~~~~
+
+As part of TVM-FFI's calling convention, returning ``-1`` indicates that an error occurred
+and the error object is stored in the TLS slot. The error object can contain arbitrary
+user-defined information, such as error messages, backtraces, or Python frame-local variables.
+
+.. hint::
+ Compiler code generation may use similar patterns to raise errors in generated code.
+
+The example below sets the TLS error and returns ``-1`` using :cpp:func:`TVMFFIErrorSetRaisedFromCStr`:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/c_api.h>
+
+ int __tvm_ffi_my_kernel(void* handle, const TVMFFIAny* args,
+ int32_t num_args, TVMFFIAny* result) {
+ // Validate inputs
+ if (num_args < 2) {
+ TVMFFIErrorSetRaisedFromCStr("ValueError", "Expected at least 2 arguments");
+ return -1;
+ }
+ // ... kernel implementation ...
+ return 0;
+ }
+
+Alternatively, :cpp:func:`TVMFFIErrorSetRaisedFromCStrParts` accepts explicit string lengths,
+which is useful when the error kind and message are not null-terminated.
+
+**Propagating errors**. For chains of generated calls, simply propagate return codes—TLS carries
+the error details:
+
+.. code-block:: cpp
+
+ int outer_function(...) {
+ int err_code = 0;
+
+ err_code = inner_function(...);
+ if (err_code != 0) goto RAII; // Propagate error; TLS has the details
+
+ RAII:
Review Comment:

The indentation of the `RAII:` label is inconsistent with the surrounding code. For better readability and to maintain a consistent coding style, it should be aligned with the function's block scope.
```suggestion
RAII:
```
##########
docs/concepts/func_module.rst:
##########
@@ -0,0 +1,564 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+Function, Exception and Module
+==============================
+
+TVM-FFI provides a unified and ABI-stable calling convention that enables
+cross-language function calls between C++, Python, Rust, and other languages.
+Functions are first-class :doc:`TVM-FFI objects <object_and_class>`.
+
+This tutorial covers everything you need to know about defining, registering,
+and calling TVM-FFI functions, their exception handling, and working with modules.
+
+Glossary
+--------
+
+TVM-FFI ABI. :cpp:type:`TVMFFISafeCallType`
+ A stable C calling convention where every function is represented by a single signature,
+ which enables type-erased, cross-language function calls.
+ This calling convention is used across all TVM-FFI function calls at the ABI boundary.
+ See :ref:`Stable C ABI <tvm_ffi_c_abi>` for a quick introduction.
+
+TVM-FFI Function. :py:class:`tvm_ffi.Function`, :cpp:class:`tvm::ffi::FunctionObj`, :cpp:class:`tvm::ffi::Function`
+ A reference-counted function object and its managed reference, which wraps any callable,
+ including language-agnostic functions and lambdas (C++, Python, Rust, etc.),
+ member functions, external C symbols, and other callable objects,
+ all sharing the same calling convention.
+
+TVM-FFI Module. :py:class:`tvm_ffi.Module`, :cpp:class:`tvm::ffi::ModuleObj`, :cpp:class:`tvm::ffi::Module`
+ A namespace for a collection of functions, loaded from a shared library via ``dlopen`` (Linux, macOS) or ``LoadLibraryW`` (Windows),
+ or statically linked to the current executable.
+
+Global Functions and Registry. :py:func:`tvm_ffi.get_global_func` and :py:func:`tvm_ffi.register_global_func`
+ A registry is a table that maps string names to :cpp:class:`~tvm::ffi::Function` objects
+ and their metadata (name, docs, signatures, etc.) for cross-language access.
+ Functions in the registry are called **global functions**.
+
+Common Usage
+------------
+
+TVM-FFI C Symbols
+~~~~~~~~~~~~~~~~~
+
+**Shared library**. Use :c:macro:`TVM_FFI_DLL_EXPORT_TYPED_FUNC` to export
+a function as a C symbol that follows the TVM-FFI ABI:
+
+.. code-block:: cpp
+
+ static int AddTwo(int x) { return x + 2; }
+
+ TVM_FFI_DLL_EXPORT_TYPED_FUNC(/*ExportName=*/add_two, /*Function=*/AddTwo)
+
+This creates a C symbol ``__tvm_ffi_<ExportName>`` in the shared library,
+which can then be loaded and called via :py:func:`tvm_ffi.load_module`:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ mod = tvm_ffi.load_module("path/to/library.so")
+ result = mod.add_two(40) # -> 42
+
+**System library**. For symbols bundled in the same executable, use :cpp:func:`TVMFFIEnvModRegisterSystemLibSymbol`
+to register each symbol during static initialization within a :c:macro:`TVM_FFI_STATIC_INIT_BLOCK`.
+See :py:func:`tvm_ffi.system_lib` for a complete workflow.
+
+
+Global Functions
+~~~~~~~~~~~~~~~~
+
+**Register a global function**. In C++, use :cpp:class:`tvm::ffi::reflection::GlobalDef` to
+register a function:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/tvm_ffi.h>
+
+ static int AddOne(int x) { return x + 1; }
+
+ TVM_FFI_STATIC_INIT_BLOCK() {
+ namespace refl = tvm::ffi::reflection;
+ refl::GlobalDef()
+ .def("my_ext.add_one", AddOne, "Add one to the input");
+ }
+
+The :c:macro:`TVM_FFI_STATIC_INIT_BLOCK` macro ensures that registration occurs
+during library initialization. The registered function is then accessible from
+Python by the name ``my_ext.add_one``.
+
+In Python, use the decorator :py:func:`tvm_ffi.register_global_func` to register a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ @tvm_ffi.register_global_func("my_ext.add_one")
+ def add_one(x: int) -> int:
+ return x + 1
+
+**Retrieve a global function**. After registration, functions are accessible by name.
+
+In Python, use :py:func:`tvm_ffi.get_global_func` to retrieve a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ # Get a function from the global registry
+ add_one = tvm_ffi.get_global_func("my_ext.add_one")
+ result = add_one(41) # -> 42
+
+In C++, use :cpp:func:`tvm::ffi::Function::GetGlobal` or :cpp:func:`tvm::ffi::Function::GetGlobalRequired`
+to retrieve a global function:
+
+.. code-block:: cpp
+
+ ffi::Function func = ffi::Function::GetGlobalRequired("my_ext.add_one");
+ int result = func(41) // -> 42
+
+
+Create Functions
+~~~~~~~~~~~~~~~~
+
+**From C++**. An :cpp:class:`tvm::ffi::Function` can be created via :cpp:func:`tvm::ffi::Function::FromTyped`
+or :cpp:class:`tvm::ffi::TypedFunction`'s constructor.
+
+.. code-block:: cpp
+
+ // Create type-erased function: add_type_erased
+ ffi::Function add_type_erased = ffi::Function::FromTyped([](int x, int y) {
+ return x + y;
+ });
+
+ // Create a typed function: add_typed
+ ffi::TypedFunction<int(int, int)> add_typed = [](int x, int y) {
+ return x + y;
+ };
+
+ // Convert a typed function to a type-erased function
+ ffi::Function generic = add_typed;
+
+**From Python**. Any Python :py:class:`Callable <collections.abc.Callable>` is automatically converted
+to a :py:class:`tvm_ffi.Function` at the ABI boundary. The example below demonstrates that in ``my_ext.bind``:
+
+- The input ``func`` is automatically converted to a :py:class:`tvm_ffi.Function`.
+- The returned lambda is also automatically converted to a :py:class:`tvm_ffi.Function`.
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ @tvm_ffi.register_global_func("my_ext.bind")
+ def bind(func, x):
+ assert isinstance(func, tvm_ffi.Function)
+ return lambda *args: func(x, *args) # converted to `tvm_ffi.Function`
+
+ def add_x_y(x, y):
+ return x + y
+
+ func_bind = tvm_ffi.get_global_func("my_ext.bind")
+ add_y = func_bind(add_x_y, 1) # bind x = 1
+ assert isinstance(add_y, tvm_ffi.Function)
+ print(add_y(2)) # -> 3
+
+
+:py:func:`tvm_ffi.convert` explicitly converts a Python callable to :py:class:`tvm_ffi.Function`:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ def add(x, y):
+ return x + y
+
+ func_add = tvm_ffi.convert(add)
+ print(func_add(1, 2))
+
+
+Exception ABI
+-------------
+
+This section describes the exception handling contract in the TVM-FFI Stable C ABI.
+Exceptions are first-class citizens in TVM-FFI, and this section specifies:
+
+- How to properly throw exceptions from a TVM-FFI ABI function
+- How to check for and propagate exceptions from a TVM-FFI ABI function
+
+TVM-FFI C ABI
+~~~~~~~~~~~~~
+
+All TVM-FFI functions ultimately conform to the :cpp:type:`TVMFFISafeCallType` signature,
+which provides a stable C ABI for cross-language calls. The C calling convention is defined as:
+
+.. code-block:: cpp
+
+ int tvm_ffi_c_abi(
+ void* handle, // Resource handle
+ const TVMFFIAny* args, // Input arguments (non-owning)
+ int32_t num_args, // Number of input arguments
+ TVMFFIAny* result // Output argument (owning, zero-initialized)
+ );
+
+**Input arguments**. The input arguments are passed as an array of :cpp:class:`tvm::ffi::AnyView` values,
+specified by ``args`` and ``num_args``.
+
+**Output argument**. The output argument ``result`` is an owning :cpp:type:`tvm::ffi::Any`
+that the caller must zero-initialize before the call.
+
+**Return value**. The ABI returns an **error code** that indicates:
+
+- ``0``: Success
+- ``-1``: Error occurred, retrievable with :cpp:func:`TVMFFIErrorMoveFromRaised`
+- ``-2``: Very rare frontend error
+
+.. hint::
+ See :doc:`Any <any>` for more details on the semantics of :cpp:type:`tvm::ffi::AnyView` and :cpp:type:`tvm::ffi::Any`.
+
+Retrieve Errors in C
+~~~~~~~~~~~~~~~~~~~~
+
+When a TVM-FFI function returns a non-zero code, it indicates that an error occurred
+and a :cpp:class:`tvm::ffi::ErrorObj` is stored in thread-local storage (TLS).
+This section shows how to retrieve the error object and print the error message and backtrace.
+
+.. note::
+
+ An :cpp:class:`~tvm::ffi::ErrorObj` is a :cpp:class:`~tvm::ffi::Object` with a :cpp:class:`TVMFFIErrorCell` payload
+ as defined below:
+
+ .. code-block:: cpp
+
+ typedef struct {
+ TVMFFIByteArray kind; // Error type (e.g., "ValueError")
+ TVMFFIByteArray message; // Error message
+ TVMFFIByteArray backtrace; // Stack trace (most-recent call first)
+ void (*update_backtrace)(...); // Hook to append/replace backtrace
+ } TVMFFIErrorCell;
+
+**Print an Error**. The example code below shows how to print an error message and backtrace.
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/c_api.h>
+
+ void PrintError(TVMFFIObject* err) {
+ TVMFFIErrorCell* cell = (TVMFFIErrorCell*)((char*)err + sizeof(TVMFFIObject));
+ fprintf(stderr, "%.*s: %.*s\n", (int)cell->kind.size, cell->kind.data, (int)cell->message.size, cell->message.data);
+ if (cell->backtrace.size) {
+ fprintf(stderr, "Backtrace:\n%.*s\n", (int)cell->backtrace.size, cell->backtrace.data);
+ }
+ }
+
+The payload of the error object is a :cpp:type:`TVMFFIErrorCell` structure
+containing the error kind, message, and backtrace. It can be accessed
+by skipping the :cpp:type:`TVMFFIObject` header using pointer arithmetic.
+
+**Retrieve the error object**. When the error code is ``-1``, the error object is stored in TLS
+and can be retrieved with :cpp:func:`TVMFFIErrorMoveFromRaised`.
+
+.. code-block:: cpp
+
+ void HandleReturnCode(int rc) {
+ TVMFFIObject* err = NULL;
+ if (rc == 0) {
+ // Success
+ } else if (rc == -1) {
+ // Move the raised error from TLS (clears TLS slot)
+ TVMFFIErrorMoveFromRaised(&err); // now `err` owns the error object
+ if (err != NULL) {
+ PrintError(err); // print the error
+ TVMFFIObjectDecRef(err); // Release the error object
+ }
+ } else if (rc == -2) {
+ // Frontend (e.g., Python) already has an exception set.
+ // Do not fetch from TLS; consult the frontend's error mechanism.
+ }
+ }
+
+This function transfers ownership of the error object to the caller and clears the TLS slot.
+You must call :cpp:func:`TVMFFIObjectDecRef` to release the object when done to avoid memory leaks.
+
+**Rare frontend errors**. Error code ``-2`` is reserved for rare frontend errors. It is returned only
+when the C API :cpp:func:`TVMFFIEnvCheckSignals` returns non-zero during execution, indicating that
+the Python side has a pending signal requiring attention. In this case, the caller should not fetch
+the error object from TLS but instead consult the frontend's error mechanism to handle the exception.
+
+Raise Errors in C
+~~~~~~~~~~~~~~~~~
+
+As part of TVM-FFI's calling convention, returning ``-1`` indicates that an error occurred
+and the error object is stored in the TLS slot. The error object can contain arbitrary
+user-defined information, such as error messages, backtraces, or Python frame-local variables.
+
+.. hint::
+ Compiler code generation may use similar patterns to raise errors in generated code.
+
+The example below sets the TLS error and returns ``-1`` using :cpp:func:`TVMFFIErrorSetRaisedFromCStr`:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/c_api.h>
+
+ int __tvm_ffi_my_kernel(void* handle, const TVMFFIAny* args,
+ int32_t num_args, TVMFFIAny* result) {
+ // Validate inputs
+ if (num_args < 2) {
+ TVMFFIErrorSetRaisedFromCStr("ValueError", "Expected at least 2 arguments");
+ return -1;
+ }
+ // ... kernel implementation ...
+ return 0;
+ }
+
+Alternatively, :cpp:func:`TVMFFIErrorSetRaisedFromCStrParts` accepts explicit string lengths,
+which is useful when the error kind and message are not null-terminated.
+
+**Propagating errors**. For chains of generated calls, simply propagate return codes—TLS carries
+the error details:
+
+.. code-block:: cpp
+
+ int outer_function(...) {
+ int err_code = 0;
+
+ err_code = inner_function(...);
+ if (err_code != 0) goto RAII; // Propagate error; TLS has the details
+
+ RAII:
+ // clean up owned resources
+ return err_code;
+ }
+
+Function
+--------
+
+Layout and ABI
+~~~~~~~~~~~~~~
+
+:cpp:class:`tvm::ffi::FunctionObj` stores two call pointers in :cpp:class:`TVMFFIFunctionCell`:
+
+.. code-block:: cpp
+
+ typedef struct {
+ TVMFFISafeCallType safe_call;
+ void* cpp_call;
+ } TVMFFIFunctionCell;
+
+``safe_call`` is used for cross-ABI function calls: it intercepts exceptions and stores them in TLS.
+``cpp_call`` is used within the same DSO, where exceptions are thrown directly for better performance.
+
+.. important::
+
+ :cpp:func:`TVMFFIFunctionCall` is the idiomatic way to call a :cpp:class:`tvm::ffi::FunctionObj` in C,
+ while ``safe_call`` or ``cpp_call`` remain low-level ABIs for fast access.
+
+**Conversion with Any**. Since :py:class:`tvm_ffi.Function` is a TVM-FFI object, it follows the same
+conversion rules as any other TVM-FFI object. See :ref:`Object Conversion with Any <object-conversion-with-any>` for details.
+
+
+Throw and Catch Errors
+~~~~~~~~~~~~~~~~~~~~~~
+
+TVM-FFI gracefully handles exceptions across language boundaries without requiring manual
+error code management.
+
+.. important::
+ Stack traces from all languages are properly preserved and concatenated in the TVM-FFI Stable C ABI.
+
+**Python**. In Python, raise native :py:class:`Exception <Exception>` instances or derived classes.
+TVM-FFI catches these at the ABI boundary and converts them to :cpp:class:`tvm::ffi::Error` objects.
+When C++ code calls into Python and a Python exception occurs, it propagates back to C++ as a
+:cpp:class:`tvm::ffi::Error`, which C++ code can handle appropriately.
+
+**C++**. In C++, use :cpp:class:`tvm::ffi::Error` or the :c:macro:`TVM_FFI_THROW` macro:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/error.h>
+
+ void ThrowError(int x) {
+ if (x < 0) {
+ TVM_FFI_THROW(ValueError) << "x must be non-negative, got " << x;
+ }
+ }
+
+The :c:macro:`TVM_FFI_THROW` macro captures the current file name, line number, stack trace,
+and error message, then constructs a :cpp:class:`tvm::ffi::Error` object. At the ABI boundary,
+this error is stored in TLS and the function returns ``-1`` per the :cpp:type:`TVMFFISafeCallType`
+calling convention.
+
+.. hint::
+ A detailed implementation of such graceful handling behavior can be found
+ in :c:macro:`TVM_FFI_SAFE_CALL_BEGIN` / :c:macro:`TVM_FFI_SAFE_CALL_END` macros.
+
+
+C Registry APIs
+~~~~~~~~~~~~~~~
+
+.. list-table::
+ :header-rows: 1
+ :widths: 40 60
+
+ * - C API
+ - Description
+ * - :cpp:func:`TVMFFIFunctionGetGlobal`
+ - Get a function by name; returns an owning handle.
+ * - :cpp:func:`TVMFFIFunctionSetGlobal`
+ - Register a function in the global registry.
+ * - :cpp:func:`TVMFFIFunctionCall`
+ - Call a function with the given arguments.
+
+Compiler developers commonly need to look up global functions in generated code. Use
+:cpp:func:`TVMFFIFunctionGetGlobal` to retrieve a function by name, then call it with :cpp:func:`TVMFFIFunctionCall`.
+The example below demonstrates how to look up and call a global function in C:
+
+.. code-block:: cpp
+
+ int LookupAndCall(const char* global_function_name, const TVMFFIAny* args, int num_args, TVMFFIAny* result) {
+ TVMFFIObject* func = NULL;
+ int err_code;
+ if ((err_code = TVMFFIFunctionGetGlobal(global_function_name, &func)) != 0)
+ goto RAII;
+ if ((err_code = TVMFFIFunctionCall(func, args, num_args, result)) != 0)
+ goto RAII;
+
+ RAII: // clean up owned resources
Review Comment:

The indentation of the `RAII:` label is inconsistent with the surrounding code. For better readability and to maintain a consistent coding style, it should be aligned with the function's block scope.
```suggestion
RAII: // clean up owned resources
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For additional commands, e-mail: discuss-archive-help@tvm.apache.org
Re: [PR] doc: Function, Exception and Module [tvm-ffi]
Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao commented on PR #394:
URL: https://github.com/apache/tvm-ffi/pull/394#issuecomment-3733708959
/gemini review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For additional commands, e-mail: discuss-archive-help@tvm.apache.org
Re: [PR] doc: Function and Module [tvm-ffi]
Posted by "gemini-code-assist[bot] (via GitHub)" <gi...@apache.org>.
gemini-code-assist[bot] commented on code in PR #394:
URL: https://github.com/apache/tvm-ffi/pull/394#discussion_r2677987105
##########
docs/concepts/func_module.rst:
##########
@@ -0,0 +1,427 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+Function and Module
+===================
+
+TVM-FFI provides a unified and ABI-stable calling convention that enables
+cross-language function calls between C++, Python, Rust and other languages.
+Functions are first-class :doc:`TVM-FFI objects <object_and_class>` that can be:
+
+- **Invoked with type-erased arguments** using a stable C ABI
+- **Registered globally** and accessed by name across languages
+- **Exported as C symbols** for efficient codegen and linking
+- **Wrapped in modules** for dynamic loading and namespace isolation
+
+This tutorial covers everything you need to know about defining, registering, and calling
+TVM-FFI functions, as well as working with modules.
+
+Glossary
+--------
+
+TVM-FFI ABI. :cpp:type:`TVMFFISafeCallType`
+ A stable C calling convention where every function is represented by a single signature:
+
+ .. code-block:: cpp
+
+ int func(void* handle, const TVMFFIAny* args, int num_args, TVMFFIAny* result);
+
+ Arguments are passed in as :cpp:class:`~tvm::ffi::AnyView`
+ and return value is passed in as :cpp:class:`~tvm::ffi::Any`,
+ enabling type-erased, cross-language function calls.
+ This calling convention is used across all TVM-FFI function calls at the ABI boundary.
+ See :ref:`Stable C ABI <tvm_ffi_c_abi>` for a quick introduction.
+
+TVM-FFI Function. :py:class:`tvm_ffi.Function`, :cpp:class:`tvm::ffi::FunctionObj`, :cpp:class:`tvm::ffi::Function`
+ A reference-counted function object and its managed reference, which wraps any callable,
+ including language-agnostic functions and lambdas (C++, Python, Rust, etc.),
+ member functions, external C symbols, and other callable objects,
+ all sharing the same calling convention.
+
+TVM-FFI Module. :py:class:`tvm_ffi.Module`, :cpp:class:`tvm::ffi::ModuleObj`, :cpp:class:`tvm::ffi::Module`
+ A namespace for a collection of functions, loaded from a shared library via ``dlopen`` (Linux, macOS) or ``LoadLibraryW`` (Windows),
+ or statically linked to the current executable.
+
+Global Functions and Registry. :py:func:`tvm_ffi.get_global_func` and :py:func:`tvm_ffi.register_global_func`
+ A registry is a table that maps string names to :cpp:class:`~tvm::ffi::Function` objects
+ and their metadata (name, docs, signatures, etc.) for cross-language access.
+ Functions in the registry are called **global functions**.
+
+Common Usage
+------------
+
+TVM-FFI C Symbols
+~~~~~~~~~~~~~~~~~
+
+**Shared library**. Use :c:macro:`TVM_FFI_DLL_EXPORT_TYPED_FUNC` to export
+a function as a C symbol that follows the TVM-FFI ABI:
+
+.. code-block:: cpp
+
+ static int AddTwo(int x) { return x + 2; }
+
+ TVM_FFI_DLL_EXPORT_TYPED_FUNC(/*ExportName=*/add_two, /*Function=*/AddTwo)
+
+This creates a C symbol ``__tvm_ffi_${ExportName}`` if the symbol is exported in a shared library,
+and later loaded and called via :py:func:`tvm_ffi.load_module`:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ mod = tvm_ffi.load_module("path/to/library.so")
+ result = mod.add_two(40) # -> 42
+
+**System library**. If the symbol is bundled in the same executable, :cpp:func:`TVMFFIEnvModRegisterSystemLibSymbol`
+is required to register the symbol during static initialization in :c:macro:`TVM_FFI_STATIC_INIT_BLOCK`.
+See examples in :py:func:`tvm_ffi.system_lib` for a complete workflow.
+
+
+Global Functions
+~~~~~~~~~~~~~~~~
+
+**Register a global function**. In C++, use :cpp:class:`tvm::ffi::reflection::GlobalDef` to
+register a function:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/tvm_ffi.h>
+
+ static int AddOne(int x) { return x + 1; }
+
+ TVM_FFI_STATIC_INIT_BLOCK() {
+ namespace refl = tvm::ffi::reflection;
+ refl::GlobalDef()
+ .def("my_ext.add_one", AddOne, "Add one to the input");
+ }
+
+:c:macro:`TVM_FFI_STATIC_INIT_BLOCK` macro ensures the registration logic happens
+during library initialization. The registered function is then accessible from
+Python under the name ``my_ext.add_one``.
+
+In Python, use the decorator :py:func:`tvm_ffi.register_global_func` to register a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ @tvm_ffi.register_global_func("my_ext.add_one")
+ def add_one(x: int) -> int:
+ return x + 1
+
+**Retrieve a global function**. After registration, functions are accessible by name.
+
+In Python, use :py:func:`tvm_ffi.get_global_func` to retrieve a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ # Get a function from the global registry
+ add_one = tvm_ffi.get_global_func("my_ext.add_one")
+ result = add_one(41) # -> 42
+
+In C++, use :cpp:func:`tvm::ffi::Function::GetGlobal` to retrieve a global function:
+
+.. code-block:: cpp
+
+ ffi::Function func = ffi::Function::GetGlobal("my_ext.add_one");
+ int result = func(41) // -> 42
+
+
+Create Functions
+~~~~~~~~~~~~~~~~
+
+**From C++**. An :cpp:class:`tvm::ffi::Function` can be created via :cpp:func:`tvm::ffi::Function::FromTyped`
+or :cpp:class:`tvm::ffi::TypedFunction`'s constructor.
+
+.. code-block:: cpp
+
+ // Create type-erased function: add_type_erased
+ ffi::Function add_type_erased = ffi::Function::FromTyped([](int x, int y) {
+ return x + y;
+ });
+
+ // Create a typed function: add_typed
+ ffi::TypedFunction<int(int, int)> add_typed = [](int x, int y) {
+ return x + y;
+ };
+
+ // Converts typed function to type-erased function
+ ffi::Function generic = add;
Review Comment:

The variable `add` is not defined in this code snippet. It seems you intended to use `add_typed`, which was defined in the preceding lines.
```suggestion
ffi::Function generic = add_typed;
```
##########
docs/concepts/func_module.rst:
##########
@@ -0,0 +1,427 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+Function and Module
+===================
+
+TVM-FFI provides a unified and ABI-stable calling convention that enables
+cross-language function calls between C++, Python, Rust and other languages.
+Functions are first-class :doc:`TVM-FFI objects <object_and_class>` that can be:
+
+- **Invoked with type-erased arguments** using a stable C ABI
+- **Registered globally** and accessed by name across languages
+- **Exported as C symbols** for efficient codegen and linking
+- **Wrapped in modules** for dynamic loading and namespace isolation
+
+This tutorial covers everything you need to know about defining, registering, and calling
+TVM-FFI functions, as well as working with modules.
+
+Glossary
+--------
+
+TVM-FFI ABI. :cpp:type:`TVMFFISafeCallType`
+ A stable C calling convention where every function is represented by a single signature:
+
+ .. code-block:: cpp
+
+ int func(void* handle, const TVMFFIAny* args, int num_args, TVMFFIAny* result);
+
+ Arguments are passed in as :cpp:class:`~tvm::ffi::AnyView`
+ and return value is passed in as :cpp:class:`~tvm::ffi::Any`,
+ enabling type-erased, cross-language function calls.
+ This calling convention is used across all TVM-FFI function calls at the ABI boundary.
+ See :ref:`Stable C ABI <tvm_ffi_c_abi>` for a quick introduction.
+
+TVM-FFI Function. :py:class:`tvm_ffi.Function`, :cpp:class:`tvm::ffi::FunctionObj`, :cpp:class:`tvm::ffi::Function`
+ A reference-counted function object and its managed reference, which wraps any callable,
+ including language-agnostic functions and lambdas (C++, Python, Rust, etc.),
+ member functions, external C symbols, and other callable objects,
+ all sharing the same calling convention.
+
+TVM-FFI Module. :py:class:`tvm_ffi.Module`, :cpp:class:`tvm::ffi::ModuleObj`, :cpp:class:`tvm::ffi::Module`
+ A namespace for a collection of functions, loaded from a shared library via ``dlopen`` (Linux, macOS) or ``LoadLibraryW`` (Windows),
+ or statically linked to the current executable.
+
+Global Functions and Registry. :py:func:`tvm_ffi.get_global_func` and :py:func:`tvm_ffi.register_global_func`
+ A registry is a table that maps string names to :cpp:class:`~tvm::ffi::Function` objects
+ and their metadata (name, docs, signatures, etc.) for cross-language access.
+ Functions in the registry are called **global functions**.
+
+Common Usage
+------------
+
+TVM-FFI C Symbols
+~~~~~~~~~~~~~~~~~
+
+**Shared library**. Use :c:macro:`TVM_FFI_DLL_EXPORT_TYPED_FUNC` to export
+a function as a C symbol that follows the TVM-FFI ABI:
+
+.. code-block:: cpp
+
+ static int AddTwo(int x) { return x + 2; }
+
+ TVM_FFI_DLL_EXPORT_TYPED_FUNC(/*ExportName=*/add_two, /*Function=*/AddTwo)
+
+This creates a C symbol ``__tvm_ffi_${ExportName}`` if the symbol is exported in a shared library,
Review Comment:

The placeholder `${ExportName}` might be confusing as it resembles shell variable expansion. Using a more conventional placeholder like `<ExportName>` would be clearer for readers.
```suggestion
This creates a C symbol ``__tvm_ffi_<ExportName>`` if the symbol is exported in a shared library,
```
##########
docs/concepts/func_module.rst:
##########
@@ -0,0 +1,427 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+Function and Module
+===================
+
+TVM-FFI provides a unified and ABI-stable calling convention that enables
+cross-language function calls between C++, Python, Rust and other languages.
+Functions are first-class :doc:`TVM-FFI objects <object_and_class>` that can be:
+
+- **Invoked with type-erased arguments** using a stable C ABI
+- **Registered globally** and accessed by name across languages
+- **Exported as C symbols** for efficient codegen and linking
+- **Wrapped in modules** for dynamic loading and namespace isolation
+
+This tutorial covers everything you need to know about defining, registering, and calling
+TVM-FFI functions, as well as working with modules.
+
+Glossary
+--------
+
+TVM-FFI ABI. :cpp:type:`TVMFFISafeCallType`
+ A stable C calling convention where every function is represented by a single signature:
+
+ .. code-block:: cpp
+
+ int func(void* handle, const TVMFFIAny* args, int num_args, TVMFFIAny* result);
+
+ Arguments are passed in as :cpp:class:`~tvm::ffi::AnyView`
+ and return value is passed in as :cpp:class:`~tvm::ffi::Any`,
+ enabling type-erased, cross-language function calls.
+ This calling convention is used across all TVM-FFI function calls at the ABI boundary.
+ See :ref:`Stable C ABI <tvm_ffi_c_abi>` for a quick introduction.
+
+TVM-FFI Function. :py:class:`tvm_ffi.Function`, :cpp:class:`tvm::ffi::FunctionObj`, :cpp:class:`tvm::ffi::Function`
+ A reference-counted function object and its managed reference, which wraps any callable,
+ including language-agnostic functions and lambdas (C++, Python, Rust, etc.),
+ member functions, external C symbols, and other callable objects,
+ all sharing the same calling convention.
+
+TVM-FFI Module. :py:class:`tvm_ffi.Module`, :cpp:class:`tvm::ffi::ModuleObj`, :cpp:class:`tvm::ffi::Module`
+ A namespace for a collection of functions, loaded from a shared library via ``dlopen`` (Linux, macOS) or ``LoadLibraryW`` (Windows),
+ or statically linked to the current executable.
+
+Global Functions and Registry. :py:func:`tvm_ffi.get_global_func` and :py:func:`tvm_ffi.register_global_func`
+ A registry is a table that maps string names to :cpp:class:`~tvm::ffi::Function` objects
+ and their metadata (name, docs, signatures, etc.) for cross-language access.
+ Functions in the registry are called **global functions**.
+
+Common Usage
+------------
+
+TVM-FFI C Symbols
+~~~~~~~~~~~~~~~~~
+
+**Shared library**. Use :c:macro:`TVM_FFI_DLL_EXPORT_TYPED_FUNC` to export
+a function as a C symbol that follows the TVM-FFI ABI:
+
+.. code-block:: cpp
+
+ static int AddTwo(int x) { return x + 2; }
+
+ TVM_FFI_DLL_EXPORT_TYPED_FUNC(/*ExportName=*/add_two, /*Function=*/AddTwo)
+
+This creates a C symbol ``__tvm_ffi_${ExportName}`` if the symbol is exported in a shared library,
+and later loaded and called via :py:func:`tvm_ffi.load_module`:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ mod = tvm_ffi.load_module("path/to/library.so")
+ result = mod.add_two(40) # -> 42
+
+**System library**. If the symbol is bundled in the same executable, :cpp:func:`TVMFFIEnvModRegisterSystemLibSymbol`
+is required to register the symbol during static initialization in :c:macro:`TVM_FFI_STATIC_INIT_BLOCK`.
+See examples in :py:func:`tvm_ffi.system_lib` for a complete workflow.
+
+
+Global Functions
+~~~~~~~~~~~~~~~~
+
+**Register a global function**. In C++, use :cpp:class:`tvm::ffi::reflection::GlobalDef` to
+register a function:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/tvm_ffi.h>
+
+ static int AddOne(int x) { return x + 1; }
+
+ TVM_FFI_STATIC_INIT_BLOCK() {
+ namespace refl = tvm::ffi::reflection;
+ refl::GlobalDef()
+ .def("my_ext.add_one", AddOne, "Add one to the input");
+ }
+
+:c:macro:`TVM_FFI_STATIC_INIT_BLOCK` macro ensures the registration logic happens
+during library initialization. The registered function is then accessible from
+Python under the name ``my_ext.add_one``.
+
+In Python, use the decorator :py:func:`tvm_ffi.register_global_func` to register a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ @tvm_ffi.register_global_func("my_ext.add_one")
+ def add_one(x: int) -> int:
+ return x + 1
+
+**Retrieve a global function**. After registration, functions are accessible by name.
+
+In Python, use :py:func:`tvm_ffi.get_global_func` to retrieve a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ # Get a function from the global registry
+ add_one = tvm_ffi.get_global_func("my_ext.add_one")
+ result = add_one(41) # -> 42
+
+In C++, use :cpp:func:`tvm::ffi::Function::GetGlobal` to retrieve a global function:
+
+.. code-block:: cpp
+
+ ffi::Function func = ffi::Function::GetGlobal("my_ext.add_one");
+ int result = func(41) // -> 42
Review Comment:

`tvm::ffi::Function::GetGlobal` returns a `std::optional<tvm::ffi::Function>`, not a `tvm::ffi::Function` directly. The code example as written would not compile. To fix this, you should either handle the optional (e.g., by dereferencing it) or use `GetGlobalRequired` for a more concise example that throws on failure.
```suggestion
ffi::Function func = ffi::Function::GetGlobalRequired("my_ext.add_one");
int result = func(41); // -> 42
```
##########
docs/concepts/func_module.rst:
##########
@@ -0,0 +1,427 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+Function and Module
+===================
+
+TVM-FFI provides a unified and ABI-stable calling convention that enables
+cross-language function calls between C++, Python, Rust and other languages.
+Functions are first-class :doc:`TVM-FFI objects <object_and_class>` that can be:
+
+- **Invoked with type-erased arguments** using a stable C ABI
+- **Registered globally** and accessed by name across languages
+- **Exported as C symbols** for efficient codegen and linking
+- **Wrapped in modules** for dynamic loading and namespace isolation
+
+This tutorial covers everything you need to know about defining, registering, and calling
+TVM-FFI functions, as well as working with modules.
+
+Glossary
+--------
+
+TVM-FFI ABI. :cpp:type:`TVMFFISafeCallType`
+ A stable C calling convention where every function is represented by a single signature:
+
+ .. code-block:: cpp
+
+ int func(void* handle, const TVMFFIAny* args, int num_args, TVMFFIAny* result);
Review Comment:

The type of `num_args` is specified as `int`, but the `TVMFFISafeCallType` typedef uses `int32_t` for this parameter. To maintain consistency with the actual ABI and other examples in this document (e.g., line 263), it should be `int32_t`.
```suggestion
int func(void* handle, const TVMFFIAny* args, int32_t num_args, TVMFFIAny* result);
```
##########
python/tvm_ffi/cython/function.pxi:
##########
@@ -889,6 +889,26 @@ cdef class Function(Object):
def __cinit__(self) -> None:
self.c_release_gil = _RELEASE_GIL_BY_DEFAULT
+ def __init__(self, func: Callable[..., Any]) -> None:
+ """Initialize a Function.
+
+ Parameters
+ ----------
+ func : Callable[..., Any]
+ Optional Python callable to wrap. If provided, creates an FFI
+ function that wraps this callable. If None, the Function is
+ expected to be initialized via static factory methods.
Review Comment:

The docstring for the `func` parameter is a bit confusing. It's described as "Optional", but the type hint `Callable[..., Any]` and the implementation `if not callable(func): ...` indicate it's a required argument. The part about `If None, ...` is also misleading, as passing `None` would raise a `TypeError`.
To improve clarity, I suggest rephrasing the docstring to reflect that `func` is a required callable.
```
The Python callable to wrap.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For additional commands, e-mail: discuss-archive-help@tvm.apache.org
Re: [PR] doc: Function and Module [tvm-ffi]
Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen commented on code in PR #394:
URL: https://github.com/apache/tvm-ffi/pull/394#discussion_r2678758130
##########
docs/concepts/func_module.rst:
##########
@@ -0,0 +1,584 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+Function and Module
+===================
+
+TVM-FFI provides a unified and ABI-stable calling convention that enables
+cross-language function calls between C++, Python, Rust and other languages.
+Functions are first-class :doc:`TVM-FFI objects <object_and_class>` that can be:
+
+- **Invoked with type-erased arguments** using a stable C ABI
+- **Registered globally** and accessed by name across languages
+- **Exported as C symbols** for efficient codegen and linking
+- **Wrapped in modules** for dynamic loading and namespace isolation
+
+This tutorial covers everything you need to know about defining, registering, and calling
+TVM-FFI functions, as well as working with modules.
+
+Glossary
+--------
+
+TVM-FFI ABI. :cpp:type:`TVMFFISafeCallType`
+ A stable C calling convention where every function is represented by a single signature,
+ which enables type-erased, cross-language function calls.
+ This calling convention is used across all TVM-FFI function calls at the ABI boundary.
+ See :ref:`Stable C ABI <tvm_ffi_c_abi>` for a quick introduction.
+
+TVM-FFI Function. :py:class:`tvm_ffi.Function`, :cpp:class:`tvm::ffi::FunctionObj`, :cpp:class:`tvm::ffi::Function`
+ A reference-counted function object and its managed reference, which wraps any callable,
+ including language-agnostic functions and lambdas (C++, Python, Rust, etc.),
+ member functions, external C symbols, and other callable objects,
+ all sharing the same calling convention.
+
+TVM-FFI Module. :py:class:`tvm_ffi.Module`, :cpp:class:`tvm::ffi::ModuleObj`, :cpp:class:`tvm::ffi::Module`
+ A namespace for a collection of functions, loaded from a shared library via ``dlopen`` (Linux, macOS) or ``LoadLibraryW`` (Windows),
+ or statically linked to the current executable.
+
+Global Functions and Registry. :py:func:`tvm_ffi.get_global_func` and :py:func:`tvm_ffi.register_global_func`
+ A registry is a table that maps string names to :cpp:class:`~tvm::ffi::Function` objects
+ and their metadata (name, docs, signatures, etc.) for cross-language access.
+ Functions in the registry are called **global functions**.
+
+Common Usage
+------------
+
+TVM-FFI C Symbols
+~~~~~~~~~~~~~~~~~
+
+**Shared library**. Use :c:macro:`TVM_FFI_DLL_EXPORT_TYPED_FUNC` to export
+a function as a C symbol that follows the TVM-FFI ABI:
+
+.. code-block:: cpp
+
+ static int AddTwo(int x) { return x + 2; }
+
+ TVM_FFI_DLL_EXPORT_TYPED_FUNC(/*ExportName=*/add_two, /*Function=*/AddTwo)
+
+This creates a C symbol ``__tvm_ffi_${ExportName}`` if the symbol is exported in a shared library,
+and later loaded and called via :py:func:`tvm_ffi.load_module`:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ mod = tvm_ffi.load_module("path/to/library.so")
+ result = mod.add_two(40) # -> 42
+
+**System library**. If the symbol is bundled in the same executable, :cpp:func:`TVMFFIEnvModRegisterSystemLibSymbol`
+is required to register the symbol during static initialization in :c:macro:`TVM_FFI_STATIC_INIT_BLOCK`.
+See examples in :py:func:`tvm_ffi.system_lib` for a complete workflow.
+
+
+Global Functions
+~~~~~~~~~~~~~~~~
+
+**Register a global function**. In C++, use :cpp:class:`tvm::ffi::reflection::GlobalDef` to
+register a function:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/tvm_ffi.h>
+
+ static int AddOne(int x) { return x + 1; }
+
+ TVM_FFI_STATIC_INIT_BLOCK() {
+ namespace refl = tvm::ffi::reflection;
+ refl::GlobalDef()
+ .def("my_ext.add_one", AddOne, "Add one to the input");
+ }
+
+:c:macro:`TVM_FFI_STATIC_INIT_BLOCK` macro ensures the registration logic happens
+during library initialization. The registered function is then accessible from
+Python under the name ``my_ext.add_one``.
+
+In Python, use the decorator :py:func:`tvm_ffi.register_global_func` to register a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ @tvm_ffi.register_global_func("my_ext.add_one")
+ def add_one(x: int) -> int:
+ return x + 1
+
+**Retrieve a global function**. After registration, functions are accessible by name.
+
+In Python, use :py:func:`tvm_ffi.get_global_func` to retrieve a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ # Get a function from the global registry
+ add_one = tvm_ffi.get_global_func("my_ext.add_one")
+ result = add_one(41) # -> 42
+
+In C++, use :cpp:func:`tvm::ffi::Function::GetGlobal` to retrieve a global function:
+
+.. code-block:: cpp
+
+ ffi::Function func = ffi::Function::GetGlobal("my_ext.add_one");
+ int result = func(41) // -> 42
+
+
+Create Functions
+~~~~~~~~~~~~~~~~
+
+**From C++**. An :cpp:class:`tvm::ffi::Function` can be created via :cpp:func:`tvm::ffi::Function::FromTyped`
+or :cpp:class:`tvm::ffi::TypedFunction`'s constructor.
+
+.. code-block:: cpp
+
+ // Create type-erased function: add_type_erased
+ ffi::Function add_type_erased = ffi::Function::FromTyped([](int x, int y) {
+ return x + y;
+ });
+
+ // Create a typed function: add_typed
+ ffi::TypedFunction<int(int, int)> add_typed = [](int x, int y) {
+ return x + y;
+ };
+
+ // Converts typed function to type-erased function
+ ffi::Function generic = add;
+
+**From Python**. An arbitrary Python :py:class:`Callable <collections.abc.Callable>` can be automatically converted
+to a :py:class:`tvm_ffi.Function` at ABI boundary. The example below shows that in function ``my_ext.bind``:
+
+- Its input ``func`` is automatically converted to :py:class:`tvm_ffi.Function`;
+- Its output, a lambda function is automatically converted to :py:class:`tvm_ffi.Function`.
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ @tvm_ffi.register_global_func("my_ext.bind")
+ def bind(func, x):
+ assert isinstance(func, tvm_ffi.Function)
+ return lambda *args: func(x, *args) # converted to `tvm_ffi.Function`
+
+ def add_x_y(x, y):
+ return x + y
+
+ func_bind = tvm_ffi.get_global_func("my_ext.bind")
+ add_y = func_bind(add_x_y, 1) # bind x = 1
+ assert isinstance(add_y, tvm_ffi.Function)
+ print(add_y(2)) # -> 3
+
+
+:py:func:`tvm_ffi.convert` explicitly converts a Python callable to :py:class:`tvm_ffi.Function`:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ def add(x, y):
+ return x + y
+
+ func_add = tvm_ffi.convert(add)
+ print(func_add(1, 2))
+
+
+C ABI Calling Convention
+------------------------
+
+.. important::
+
+ This section focuses on in-depth of TVM-FFI's stable C ABI definition,
+ describing it in pure C for precision and clarity.
+
+ Compiler code generation may follow similar patterns to call functions with TVM-FFI ABI,
+ or to generate functions that conform to TVM-FFI ABI.
+
+All TVM-FFI functions ultimately conform to the :cpp:type:`TVMFFISafeCallType` signature,
+which provides a stable C ABI for cross-language calls. The C calling convention is defined as:
+
+.. code-block:: cpp
+
+ int tvm_ffi_c_abi(
+ void* handle, // Resource handle
+ const TVMFFIAny* args, // Inputs (non-owning)
+ int num_args, // Number of arguments
+ TVMFFIAny* result // Output (owning)
+ );
+
+.. note::
+ As part of the contract, the return value ``result`` should be zero-initialized by the caller.
+
+The returning integer is **error code**, which indicates
+
+- ``0``: Success
+- ``-1``: Error occurred, retrieve via :cpp:func:`TVMFFIErrorMoveFromRaised`
+- ``-2``: Frontend error (e.g., Python exception)
+
+
+Input and Output Arguments
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The input and output arguments are passed as :cpp:type:`TVMFFIAny` pointers.
+
+- Input arguments are an array of ``num_args`` non-owning :cpp:type:`tvm::ffi::AnyView`;
+- Output argument is an owning :cpp:type:`tvm::ffi::Any`, which the caller has to guarantee to be zero-initialized.
+
+See :doc:`Any <any>` for more details on the semantics of :cpp:type:`tvm::ffi::AnyView` and :cpp:type:`tvm::ffi::Any`.
+
+Retrieve Errors in C
+~~~~~~~~~~~~~~~~~~~~
+
+When a TVM-FFI function returns a non-zero code, it indicates an error occurred,
+and the error object is stored in the TLS slot. This section shows how to retrieve
+the error object and print the error message and backtrace.
+
+.. hint::
+ Compiler code generation may use similar patterns to call any function with TVM-FFI ABI.
+
+
+**Print an Error**. The example code below shows how to print an error message and backtrace.
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/c_api.h>
+
+ static void PrintError(TVMFFIObject* err) {
+ TVMFFIErrorCell* cell = (TVMFFIErrorCell*)((char*)err + sizeof(TVMFFIObject));
+ fprintf(stderr, "%.*s: %.*s\n", (int)cell->kind.size, cell->kind.data, (int)cell->message.size, cell->message.data);
+ if (cell->backtrace.size) {
+ fprintf(stderr, "Backtrace:\n%.*s\n", (int)cell->backtrace.size, cell->backtrace.data);
+ }
+ }
+
+The payload of the error object is a :cpp:type:`TVMFFIErrorCell` structure,
+which contains the error kind, message, and backtrace, which can be accessed
+by skipping the :cpp:type:`TVMFFIObject` header with pointer arithmetics.
+
+.. note::
+
+ An :cpp:class:`~tvm::ffi::Error` object is a :cpp:class:`~tvm::ffi::Object` with a :cpp:class:`TVMFFIErrorCell` payload
+ as defined below:
+
+ .. code-block:: cpp
+
+ typedef struct {
+ TVMFFIByteArray kind; // Error type (e.g., "ValueError")
+ TVMFFIByteArray message; // Error message
+ TVMFFIByteArray backtrace; // Stack trace (most-recent call first)
+ void (*update_backtrace)(...); // Hook to append/replace backtrace
+ } TVMFFIErrorCell;
+
+
+**Retrieve error object**. When error code is ``-1``, the error object is stored in the TLS slot,
+and you can retrieve it with :cpp:func:`TVMFFIErrorMoveFromRaised`.
+
+.. code-block:: cpp
+
+ void HandleReturnCode(int rc) {
+ TVMFFIObject* err = NULL;
+ if (rc == 0) {
+ // Success
+ }
+ else if (rc == -1) {
+ // Move the raised error from TLS (clears TLS slot)
+ TVMFFIErrorMoveFromRaised(&err); // now `err` owns the error object
+ if (err != NULL) {
+ PrintError(err); // print the error
+ TVMFFIObjectDecRef(err); // Release the error object
+ }
+ }
+ else if (rc == -2) {
+ // Frontend (e.g., Python) already has an exception set.
+ // Do not fetch from TLS; consult the frontend's error mechanism.
+ }
+ }
+
+This method transfers ownership of the error object to the caller and clears the TLS slot,
+and therefore you must call :cpp:func:`TVMFFIObjectDecRef` to release the object when done
+to avoid memory leaks.
+
+Raise Errors in C
+~~~~~~~~~~~~~~~~~
+
+As part of TVM-FFI's calling convention, returning ``-1`` indicates an error occurred,
+and the error object is stored in the TLS slot. The error object may contain abitrary
+user-defined information, for example, error message, backtrace, or Python frame-local variables.
+
+.. hint::
+ Compiler code generation may use similar patterns to raise errors in generated code.
+
+The example below sets the TLS error and returns ``-1`` using :cpp:func:`TVMFFIErrorSetRaisedFromCStr`:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/c_api.h>
+
+ int __tvm_ffi_my_kernel(void* handle, const TVMFFIAny* args,
+ int32_t num_args, TVMFFIAny* result) {
+ // Validate inputs
+ if (num_args < 2) {
+ TVMFFIErrorSetRaisedFromCStr("ValueError", "Expected at least 2 arguments");
+ return -1;
+ }
+ // ... kernel implementation ...
+ return 0;
+ }
+
+Alternatively, :cpp:func:`TVMFFIErrorSetRaisedFromCStrParts` can also be used to concatenate
+multiple parts of the error message into a single string.
+
+**Propagating errors**. For chains of generated calls, simply propagate return codes - TLS carries
+the error details:
+
+.. code-block:: cpp
+
+ int outer_function(...) {
+ int err_code = 0;
+
+ err_code = inner_function(...);
+ if (err_code != 0) goto RAII; // Propagate error; TLS has the details
+
+ RAII:
+ // clean up owned resources
+ return err_code;
+ }
+
+Handling Non-C++ Errors
+~~~~~~~~~~~~~~~~~~~~~~~
+
+TVM-FFI preserves frontend errors (e.g., Python exceptions) through a dedicated ``-2`` return path.
Review Comment:
This is not accurate.
For common path, TVM-FFI catches translate python errors to ffi::Error with stacktrace. This allows python stack trace and c++ stack trace to be preserved across multiple cross function calls.
For rare cases where the python error is triggered by signal handlersg, such error cannot be caught by the c++ frontend, in such case, we raise an ErrorAlreadySet and translate to -2 in the case. see also https://github.com/apache/tvm-ffi/blob/main/include/tvm/ffi/error.h#L73
The code there is the only use-case of -2 return values atm
##########
docs/concepts/func_module.rst:
##########
@@ -0,0 +1,584 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+Function and Module
+===================
+
+TVM-FFI provides a unified and ABI-stable calling convention that enables
+cross-language function calls between C++, Python, Rust and other languages.
+Functions are first-class :doc:`TVM-FFI objects <object_and_class>` that can be:
+
+- **Invoked with type-erased arguments** using a stable C ABI
+- **Registered globally** and accessed by name across languages
+- **Exported as C symbols** for efficient codegen and linking
+- **Wrapped in modules** for dynamic loading and namespace isolation
+
+This tutorial covers everything you need to know about defining, registering, and calling
+TVM-FFI functions, as well as working with modules.
+
+Glossary
+--------
+
+TVM-FFI ABI. :cpp:type:`TVMFFISafeCallType`
+ A stable C calling convention where every function is represented by a single signature,
+ which enables type-erased, cross-language function calls.
+ This calling convention is used across all TVM-FFI function calls at the ABI boundary.
+ See :ref:`Stable C ABI <tvm_ffi_c_abi>` for a quick introduction.
+
+TVM-FFI Function. :py:class:`tvm_ffi.Function`, :cpp:class:`tvm::ffi::FunctionObj`, :cpp:class:`tvm::ffi::Function`
+ A reference-counted function object and its managed reference, which wraps any callable,
+ including language-agnostic functions and lambdas (C++, Python, Rust, etc.),
+ member functions, external C symbols, and other callable objects,
+ all sharing the same calling convention.
+
+TVM-FFI Module. :py:class:`tvm_ffi.Module`, :cpp:class:`tvm::ffi::ModuleObj`, :cpp:class:`tvm::ffi::Module`
+ A namespace for a collection of functions, loaded from a shared library via ``dlopen`` (Linux, macOS) or ``LoadLibraryW`` (Windows),
+ or statically linked to the current executable.
+
+Global Functions and Registry. :py:func:`tvm_ffi.get_global_func` and :py:func:`tvm_ffi.register_global_func`
+ A registry is a table that maps string names to :cpp:class:`~tvm::ffi::Function` objects
+ and their metadata (name, docs, signatures, etc.) for cross-language access.
+ Functions in the registry are called **global functions**.
+
+Common Usage
+------------
+
+TVM-FFI C Symbols
+~~~~~~~~~~~~~~~~~
+
+**Shared library**. Use :c:macro:`TVM_FFI_DLL_EXPORT_TYPED_FUNC` to export
+a function as a C symbol that follows the TVM-FFI ABI:
+
+.. code-block:: cpp
+
+ static int AddTwo(int x) { return x + 2; }
+
+ TVM_FFI_DLL_EXPORT_TYPED_FUNC(/*ExportName=*/add_two, /*Function=*/AddTwo)
+
+This creates a C symbol ``__tvm_ffi_${ExportName}`` if the symbol is exported in a shared library,
+and later loaded and called via :py:func:`tvm_ffi.load_module`:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ mod = tvm_ffi.load_module("path/to/library.so")
+ result = mod.add_two(40) # -> 42
+
+**System library**. If the symbol is bundled in the same executable, :cpp:func:`TVMFFIEnvModRegisterSystemLibSymbol`
+is required to register the symbol during static initialization in :c:macro:`TVM_FFI_STATIC_INIT_BLOCK`.
+See examples in :py:func:`tvm_ffi.system_lib` for a complete workflow.
+
+
+Global Functions
+~~~~~~~~~~~~~~~~
+
+**Register a global function**. In C++, use :cpp:class:`tvm::ffi::reflection::GlobalDef` to
+register a function:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/tvm_ffi.h>
+
+ static int AddOne(int x) { return x + 1; }
+
+ TVM_FFI_STATIC_INIT_BLOCK() {
+ namespace refl = tvm::ffi::reflection;
+ refl::GlobalDef()
+ .def("my_ext.add_one", AddOne, "Add one to the input");
+ }
+
+:c:macro:`TVM_FFI_STATIC_INIT_BLOCK` macro ensures the registration logic happens
+during library initialization. The registered function is then accessible from
+Python under the name ``my_ext.add_one``.
+
+In Python, use the decorator :py:func:`tvm_ffi.register_global_func` to register a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ @tvm_ffi.register_global_func("my_ext.add_one")
+ def add_one(x: int) -> int:
+ return x + 1
+
+**Retrieve a global function**. After registration, functions are accessible by name.
+
+In Python, use :py:func:`tvm_ffi.get_global_func` to retrieve a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ # Get a function from the global registry
+ add_one = tvm_ffi.get_global_func("my_ext.add_one")
+ result = add_one(41) # -> 42
+
+In C++, use :cpp:func:`tvm::ffi::Function::GetGlobal` to retrieve a global function:
+
+.. code-block:: cpp
+
+ ffi::Function func = ffi::Function::GetGlobal("my_ext.add_one");
+ int result = func(41) // -> 42
+
+
+Create Functions
+~~~~~~~~~~~~~~~~
+
+**From C++**. An :cpp:class:`tvm::ffi::Function` can be created via :cpp:func:`tvm::ffi::Function::FromTyped`
+or :cpp:class:`tvm::ffi::TypedFunction`'s constructor.
+
+.. code-block:: cpp
+
+ // Create type-erased function: add_type_erased
+ ffi::Function add_type_erased = ffi::Function::FromTyped([](int x, int y) {
+ return x + y;
+ });
+
+ // Create a typed function: add_typed
+ ffi::TypedFunction<int(int, int)> add_typed = [](int x, int y) {
+ return x + y;
+ };
+
+ // Converts typed function to type-erased function
+ ffi::Function generic = add;
+
+**From Python**. An arbitrary Python :py:class:`Callable <collections.abc.Callable>` can be automatically converted
+to a :py:class:`tvm_ffi.Function` at ABI boundary. The example below shows that in function ``my_ext.bind``:
+
+- Its input ``func`` is automatically converted to :py:class:`tvm_ffi.Function`;
+- Its output, a lambda function is automatically converted to :py:class:`tvm_ffi.Function`.
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ @tvm_ffi.register_global_func("my_ext.bind")
+ def bind(func, x):
+ assert isinstance(func, tvm_ffi.Function)
+ return lambda *args: func(x, *args) # converted to `tvm_ffi.Function`
+
+ def add_x_y(x, y):
+ return x + y
+
+ func_bind = tvm_ffi.get_global_func("my_ext.bind")
+ add_y = func_bind(add_x_y, 1) # bind x = 1
+ assert isinstance(add_y, tvm_ffi.Function)
+ print(add_y(2)) # -> 3
+
+
+:py:func:`tvm_ffi.convert` explicitly converts a Python callable to :py:class:`tvm_ffi.Function`:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ def add(x, y):
+ return x + y
+
+ func_add = tvm_ffi.convert(add)
+ print(func_add(1, 2))
+
+
+C ABI Calling Convention
+------------------------
+
+.. important::
+
+ This section focuses on in-depth of TVM-FFI's stable C ABI definition,
+ describing it in pure C for precision and clarity.
+
+ Compiler code generation may follow similar patterns to call functions with TVM-FFI ABI,
+ or to generate functions that conform to TVM-FFI ABI.
+
+All TVM-FFI functions ultimately conform to the :cpp:type:`TVMFFISafeCallType` signature,
+which provides a stable C ABI for cross-language calls. The C calling convention is defined as:
+
+.. code-block:: cpp
+
+ int tvm_ffi_c_abi(
+ void* handle, // Resource handle
+ const TVMFFIAny* args, // Inputs (non-owning)
+ int num_args, // Number of arguments
+ TVMFFIAny* result // Output (owning)
+ );
+
+.. note::
+ As part of the contract, the return value ``result`` should be zero-initialized by the caller.
+
+The returning integer is **error code**, which indicates
+
+- ``0``: Success
+- ``-1``: Error occurred, retrieve via :cpp:func:`TVMFFIErrorMoveFromRaised`
+- ``-2``: Frontend error (e.g., Python exception)
+
+
+Input and Output Arguments
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The input and output arguments are passed as :cpp:type:`TVMFFIAny` pointers.
+
+- Input arguments are an array of ``num_args`` non-owning :cpp:type:`tvm::ffi::AnyView`;
+- Output argument is an owning :cpp:type:`tvm::ffi::Any`, which the caller has to guarantee to be zero-initialized.
+
+See :doc:`Any <any>` for more details on the semantics of :cpp:type:`tvm::ffi::AnyView` and :cpp:type:`tvm::ffi::Any`.
+
+Retrieve Errors in C
+~~~~~~~~~~~~~~~~~~~~
+
+When a TVM-FFI function returns a non-zero code, it indicates an error occurred,
+and the error object is stored in the TLS slot. This section shows how to retrieve
+the error object and print the error message and backtrace.
+
+.. hint::
+ Compiler code generation may use similar patterns to call any function with TVM-FFI ABI.
+
+
+**Print an Error**. The example code below shows how to print an error message and backtrace.
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/c_api.h>
+
+ static void PrintError(TVMFFIObject* err) {
+ TVMFFIErrorCell* cell = (TVMFFIErrorCell*)((char*)err + sizeof(TVMFFIObject));
+ fprintf(stderr, "%.*s: %.*s\n", (int)cell->kind.size, cell->kind.data, (int)cell->message.size, cell->message.data);
+ if (cell->backtrace.size) {
+ fprintf(stderr, "Backtrace:\n%.*s\n", (int)cell->backtrace.size, cell->backtrace.data);
+ }
+ }
+
+The payload of the error object is a :cpp:type:`TVMFFIErrorCell` structure,
+which contains the error kind, message, and backtrace, which can be accessed
+by skipping the :cpp:type:`TVMFFIObject` header with pointer arithmetics.
+
+.. note::
+
+ An :cpp:class:`~tvm::ffi::Error` object is a :cpp:class:`~tvm::ffi::Object` with a :cpp:class:`TVMFFIErrorCell` payload
+ as defined below:
+
+ .. code-block:: cpp
+
+ typedef struct {
+ TVMFFIByteArray kind; // Error type (e.g., "ValueError")
+ TVMFFIByteArray message; // Error message
+ TVMFFIByteArray backtrace; // Stack trace (most-recent call first)
+ void (*update_backtrace)(...); // Hook to append/replace backtrace
+ } TVMFFIErrorCell;
+
+
+**Retrieve error object**. When error code is ``-1``, the error object is stored in the TLS slot,
+and you can retrieve it with :cpp:func:`TVMFFIErrorMoveFromRaised`.
+
+.. code-block:: cpp
+
+ void HandleReturnCode(int rc) {
+ TVMFFIObject* err = NULL;
+ if (rc == 0) {
+ // Success
+ }
+ else if (rc == -1) {
+ // Move the raised error from TLS (clears TLS slot)
+ TVMFFIErrorMoveFromRaised(&err); // now `err` owns the error object
+ if (err != NULL) {
+ PrintError(err); // print the error
+ TVMFFIObjectDecRef(err); // Release the error object
+ }
+ }
+ else if (rc == -2) {
+ // Frontend (e.g., Python) already has an exception set.
+ // Do not fetch from TLS; consult the frontend's error mechanism.
+ }
+ }
+
+This method transfers ownership of the error object to the caller and clears the TLS slot,
+and therefore you must call :cpp:func:`TVMFFIObjectDecRef` to release the object when done
+to avoid memory leaks.
+
+Raise Errors in C
+~~~~~~~~~~~~~~~~~
+
+As part of TVM-FFI's calling convention, returning ``-1`` indicates an error occurred,
+and the error object is stored in the TLS slot. The error object may contain abitrary
+user-defined information, for example, error message, backtrace, or Python frame-local variables.
+
+.. hint::
+ Compiler code generation may use similar patterns to raise errors in generated code.
+
+The example below sets the TLS error and returns ``-1`` using :cpp:func:`TVMFFIErrorSetRaisedFromCStr`:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/c_api.h>
+
+ int __tvm_ffi_my_kernel(void* handle, const TVMFFIAny* args,
+ int32_t num_args, TVMFFIAny* result) {
+ // Validate inputs
+ if (num_args < 2) {
+ TVMFFIErrorSetRaisedFromCStr("ValueError", "Expected at least 2 arguments");
+ return -1;
+ }
+ // ... kernel implementation ...
+ return 0;
+ }
+
+Alternatively, :cpp:func:`TVMFFIErrorSetRaisedFromCStrParts` can also be used to concatenate
+multiple parts of the error message into a single string.
+
+**Propagating errors**. For chains of generated calls, simply propagate return codes - TLS carries
+the error details:
+
+.. code-block:: cpp
+
+ int outer_function(...) {
+ int err_code = 0;
+
+ err_code = inner_function(...);
+ if (err_code != 0) goto RAII; // Propagate error; TLS has the details
+
+ RAII:
+ // clean up owned resources
+ return err_code;
+ }
+
+Handling Non-C++ Errors
+~~~~~~~~~~~~~~~~~~~~~~~
+
+TVM-FFI preserves frontend errors (e.g., Python exceptions) through a dedicated ``-2`` return path.
+
+When a Python exception occurs during a TVM-FFI call:
+
+- The exception remains a native Python exception with its original traceback
+- The C++ side returns ``-2`` to indicate "frontend error already set"
+- Python retains the real exception object with actual frame locals
+
+This path preserves Python stack variables because the exception is never serialized into
+a TVM-FFI Error object.
+
+Function
+--------
+
+Layout and ABI
+~~~~~~~~~~~~~~
+
+:cpp:class:`tvm::ffi::FunctionObj` stores two call pointers in :cpp:class:`TVMFFIFunctionCell`:
+
+.. code-block:: cpp
+
+ typedef struct {
+ TVMFFISafeCallType safe_call;
+ void* cpp_call;
+ } TVMFFIFunctionCell;
+
+``safe_call`` is used for cross-ABI function calling, which intercepts and stores exceptions in TLS,
+while ``cpp_call`` is used inside the same DSO, and exceptions are thrown directly for performance.
+
+.. important::
+
+ Do not directly use ``safe_call`` or ``cpp_call`` to call functions. Instead, a C API :cpp:func:`TVMFFIFunctionCall`
+ is provided to handle error propagation and resource management.
+
+**Conversion with Any**. :py:class:`tvm_ffi.Function` is a TVM-FFI object, and thus conversion follows
+exactly the same rule as a TVM-FFI object does. See :ref:`Object Conversion with Any <object-conversion-with-any>` for more details.
+
+
+Throw and Catch Errors
+~~~~~~~~~~~~~~~~~~~~~~
+
+TVM-FFI gracefully handles exceptions across language boundaries without having to manually
+set error codes.
+
+**Python**. In Python, raise its language-native :py:class:`Exception <Exception>` and its
+derived classes, and TVM-FFI will catch it at ABI boundary, and convert it to a
+:cpp:class:`tvm::ffi::Error` object. If C++ code is calling into Python, the Python exception
+will be propagated back and re-raised as a :cpp:class:`tvm::ffi::Error` object to C++ side,
+and the C++ code can handle it appropriately.
+
+**C++**. In C++, use :cpp:class:`tvm::ffi::Error` or the :c:macro:`TVM_FFI_THROW` macro:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/error.h>
+
+ void ThrowError(int x) {
+ if (x < 0) {
+ TVM_FFI_THROW(ValueError) << "x must be non-negative, got " << x;
+ }
+ }
+
+The example above shows how to throw an error in C++. :c:macro:`TVM_FFI_THROW` captures
+the current file name, line number, stacktrace, and error message, and then constructs a
+:cpp:class:`tvm::ffi::Error` object. At ABI boundary, this :cpp:class:`tvm::ffi::Error`
+object is stored in the TLS slot, and the function returns ``-1`` as error code in the C ABI
+:cpp:type:`TVMFFISafeCallType` signature.
+
+.. hint::
+ A detailed implementation of such graceful handling behavior can be found
Review Comment:
blank line needed in rst
##########
docs/concepts/func_module.rst:
##########
@@ -0,0 +1,584 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+Function and Module
+===================
+
+TVM-FFI provides a unified and ABI-stable calling convention that enables
+cross-language function calls between C++, Python, Rust and other languages.
+Functions are first-class :doc:`TVM-FFI objects <object_and_class>` that can be:
+
+- **Invoked with type-erased arguments** using a stable C ABI
+- **Registered globally** and accessed by name across languages
+- **Exported as C symbols** for efficient codegen and linking
+- **Wrapped in modules** for dynamic loading and namespace isolation
+
+This tutorial covers everything you need to know about defining, registering, and calling
+TVM-FFI functions, as well as working with modules.
+
+Glossary
+--------
+
+TVM-FFI ABI. :cpp:type:`TVMFFISafeCallType`
+ A stable C calling convention where every function is represented by a single signature,
+ which enables type-erased, cross-language function calls.
+ This calling convention is used across all TVM-FFI function calls at the ABI boundary.
+ See :ref:`Stable C ABI <tvm_ffi_c_abi>` for a quick introduction.
+
+TVM-FFI Function. :py:class:`tvm_ffi.Function`, :cpp:class:`tvm::ffi::FunctionObj`, :cpp:class:`tvm::ffi::Function`
+ A reference-counted function object and its managed reference, which wraps any callable,
+ including language-agnostic functions and lambdas (C++, Python, Rust, etc.),
+ member functions, external C symbols, and other callable objects,
+ all sharing the same calling convention.
+
+TVM-FFI Module. :py:class:`tvm_ffi.Module`, :cpp:class:`tvm::ffi::ModuleObj`, :cpp:class:`tvm::ffi::Module`
+ A namespace for a collection of functions, loaded from a shared library via ``dlopen`` (Linux, macOS) or ``LoadLibraryW`` (Windows),
+ or statically linked to the current executable.
+
+Global Functions and Registry. :py:func:`tvm_ffi.get_global_func` and :py:func:`tvm_ffi.register_global_func`
+ A registry is a table that maps string names to :cpp:class:`~tvm::ffi::Function` objects
+ and their metadata (name, docs, signatures, etc.) for cross-language access.
+ Functions in the registry are called **global functions**.
+
+Common Usage
+------------
+
+TVM-FFI C Symbols
+~~~~~~~~~~~~~~~~~
+
+**Shared library**. Use :c:macro:`TVM_FFI_DLL_EXPORT_TYPED_FUNC` to export
+a function as a C symbol that follows the TVM-FFI ABI:
+
+.. code-block:: cpp
+
+ static int AddTwo(int x) { return x + 2; }
+
+ TVM_FFI_DLL_EXPORT_TYPED_FUNC(/*ExportName=*/add_two, /*Function=*/AddTwo)
+
+This creates a C symbol ``__tvm_ffi_${ExportName}`` if the symbol is exported in a shared library,
+and later loaded and called via :py:func:`tvm_ffi.load_module`:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ mod = tvm_ffi.load_module("path/to/library.so")
+ result = mod.add_two(40) # -> 42
+
+**System library**. If the symbol is bundled in the same executable, :cpp:func:`TVMFFIEnvModRegisterSystemLibSymbol`
+is required to register the symbol during static initialization in :c:macro:`TVM_FFI_STATIC_INIT_BLOCK`.
+See examples in :py:func:`tvm_ffi.system_lib` for a complete workflow.
+
+
+Global Functions
+~~~~~~~~~~~~~~~~
+
+**Register a global function**. In C++, use :cpp:class:`tvm::ffi::reflection::GlobalDef` to
+register a function:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/tvm_ffi.h>
+
+ static int AddOne(int x) { return x + 1; }
+
+ TVM_FFI_STATIC_INIT_BLOCK() {
+ namespace refl = tvm::ffi::reflection;
+ refl::GlobalDef()
+ .def("my_ext.add_one", AddOne, "Add one to the input");
+ }
+
+:c:macro:`TVM_FFI_STATIC_INIT_BLOCK` macro ensures the registration logic happens
+during library initialization. The registered function is then accessible from
+Python under the name ``my_ext.add_one``.
+
+In Python, use the decorator :py:func:`tvm_ffi.register_global_func` to register a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ @tvm_ffi.register_global_func("my_ext.add_one")
+ def add_one(x: int) -> int:
+ return x + 1
+
+**Retrieve a global function**. After registration, functions are accessible by name.
+
+In Python, use :py:func:`tvm_ffi.get_global_func` to retrieve a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ # Get a function from the global registry
+ add_one = tvm_ffi.get_global_func("my_ext.add_one")
+ result = add_one(41) # -> 42
+
+In C++, use :cpp:func:`tvm::ffi::Function::GetGlobal` to retrieve a global function:
+
+.. code-block:: cpp
+
+ ffi::Function func = ffi::Function::GetGlobal("my_ext.add_one");
+ int result = func(41) // -> 42
+
+
+Create Functions
+~~~~~~~~~~~~~~~~
+
+**From C++**. An :cpp:class:`tvm::ffi::Function` can be created via :cpp:func:`tvm::ffi::Function::FromTyped`
+or :cpp:class:`tvm::ffi::TypedFunction`'s constructor.
+
+.. code-block:: cpp
+
+ // Create type-erased function: add_type_erased
+ ffi::Function add_type_erased = ffi::Function::FromTyped([](int x, int y) {
+ return x + y;
+ });
+
+ // Create a typed function: add_typed
+ ffi::TypedFunction<int(int, int)> add_typed = [](int x, int y) {
+ return x + y;
+ };
+
+ // Converts typed function to type-erased function
+ ffi::Function generic = add;
+
+**From Python**. An arbitrary Python :py:class:`Callable <collections.abc.Callable>` can be automatically converted
+to a :py:class:`tvm_ffi.Function` at ABI boundary. The example below shows that in function ``my_ext.bind``:
+
+- Its input ``func`` is automatically converted to :py:class:`tvm_ffi.Function`;
+- Its output, a lambda function is automatically converted to :py:class:`tvm_ffi.Function`.
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ @tvm_ffi.register_global_func("my_ext.bind")
+ def bind(func, x):
+ assert isinstance(func, tvm_ffi.Function)
+ return lambda *args: func(x, *args) # converted to `tvm_ffi.Function`
+
+ def add_x_y(x, y):
+ return x + y
+
+ func_bind = tvm_ffi.get_global_func("my_ext.bind")
+ add_y = func_bind(add_x_y, 1) # bind x = 1
+ assert isinstance(add_y, tvm_ffi.Function)
+ print(add_y(2)) # -> 3
+
+
+:py:func:`tvm_ffi.convert` explicitly converts a Python callable to :py:class:`tvm_ffi.Function`:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ def add(x, y):
+ return x + y
+
+ func_add = tvm_ffi.convert(add)
+ print(func_add(1, 2))
+
+
+C ABI Calling Convention
+------------------------
+
+.. important::
+
+ This section focuses on in-depth of TVM-FFI's stable C ABI definition,
+ describing it in pure C for precision and clarity.
+
+ Compiler code generation may follow similar patterns to call functions with TVM-FFI ABI,
+ or to generate functions that conform to TVM-FFI ABI.
+
+All TVM-FFI functions ultimately conform to the :cpp:type:`TVMFFISafeCallType` signature,
+which provides a stable C ABI for cross-language calls. The C calling convention is defined as:
+
+.. code-block:: cpp
+
+ int tvm_ffi_c_abi(
+ void* handle, // Resource handle
+ const TVMFFIAny* args, // Inputs (non-owning)
+ int num_args, // Number of arguments
+ TVMFFIAny* result // Output (owning)
+ );
+
+.. note::
+ As part of the contract, the return value ``result`` should be zero-initialized by the caller.
+
+The returning integer is **error code**, which indicates
+
+- ``0``: Success
+- ``-1``: Error occurred, retrieve via :cpp:func:`TVMFFIErrorMoveFromRaised`
+- ``-2``: Frontend error (e.g., Python exception)
+
+
+Input and Output Arguments
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The input and output arguments are passed as :cpp:type:`TVMFFIAny` pointers.
+
+- Input arguments are an array of ``num_args`` non-owning :cpp:type:`tvm::ffi::AnyView`;
+- Output argument is an owning :cpp:type:`tvm::ffi::Any`, which the caller has to guarantee to be zero-initialized.
+
+See :doc:`Any <any>` for more details on the semantics of :cpp:type:`tvm::ffi::AnyView` and :cpp:type:`tvm::ffi::Any`.
+
+Retrieve Errors in C
+~~~~~~~~~~~~~~~~~~~~
+
+When a TVM-FFI function returns a non-zero code, it indicates an error occurred,
+and the error object is stored in the TLS slot. This section shows how to retrieve
+the error object and print the error message and backtrace.
+
+.. hint::
+ Compiler code generation may use similar patterns to call any function with TVM-FFI ABI.
+
+
+**Print an Error**. The example code below shows how to print an error message and backtrace.
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/c_api.h>
+
+ static void PrintError(TVMFFIObject* err) {
+ TVMFFIErrorCell* cell = (TVMFFIErrorCell*)((char*)err + sizeof(TVMFFIObject));
+ fprintf(stderr, "%.*s: %.*s\n", (int)cell->kind.size, cell->kind.data, (int)cell->message.size, cell->message.data);
+ if (cell->backtrace.size) {
+ fprintf(stderr, "Backtrace:\n%.*s\n", (int)cell->backtrace.size, cell->backtrace.data);
+ }
+ }
+
+The payload of the error object is a :cpp:type:`TVMFFIErrorCell` structure,
+which contains the error kind, message, and backtrace, which can be accessed
+by skipping the :cpp:type:`TVMFFIObject` header with pointer arithmetics.
+
+.. note::
+
+ An :cpp:class:`~tvm::ffi::Error` object is a :cpp:class:`~tvm::ffi::Object` with a :cpp:class:`TVMFFIErrorCell` payload
+ as defined below:
+
+ .. code-block:: cpp
+
+ typedef struct {
+ TVMFFIByteArray kind; // Error type (e.g., "ValueError")
+ TVMFFIByteArray message; // Error message
+ TVMFFIByteArray backtrace; // Stack trace (most-recent call first)
+ void (*update_backtrace)(...); // Hook to append/replace backtrace
+ } TVMFFIErrorCell;
+
+
+**Retrieve error object**. When error code is ``-1``, the error object is stored in the TLS slot,
+and you can retrieve it with :cpp:func:`TVMFFIErrorMoveFromRaised`.
+
+.. code-block:: cpp
+
+ void HandleReturnCode(int rc) {
+ TVMFFIObject* err = NULL;
+ if (rc == 0) {
+ // Success
+ }
+ else if (rc == -1) {
+ // Move the raised error from TLS (clears TLS slot)
+ TVMFFIErrorMoveFromRaised(&err); // now `err` owns the error object
+ if (err != NULL) {
+ PrintError(err); // print the error
+ TVMFFIObjectDecRef(err); // Release the error object
+ }
+ }
+ else if (rc == -2) {
+ // Frontend (e.g., Python) already has an exception set.
+ // Do not fetch from TLS; consult the frontend's error mechanism.
+ }
+ }
+
+This method transfers ownership of the error object to the caller and clears the TLS slot,
+and therefore you must call :cpp:func:`TVMFFIObjectDecRef` to release the object when done
+to avoid memory leaks.
+
+Raise Errors in C
+~~~~~~~~~~~~~~~~~
+
+As part of TVM-FFI's calling convention, returning ``-1`` indicates an error occurred,
+and the error object is stored in the TLS slot. The error object may contain abitrary
+user-defined information, for example, error message, backtrace, or Python frame-local variables.
+
+.. hint::
+ Compiler code generation may use similar patterns to raise errors in generated code.
+
+The example below sets the TLS error and returns ``-1`` using :cpp:func:`TVMFFIErrorSetRaisedFromCStr`:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/c_api.h>
+
+ int __tvm_ffi_my_kernel(void* handle, const TVMFFIAny* args,
+ int32_t num_args, TVMFFIAny* result) {
+ // Validate inputs
+ if (num_args < 2) {
+ TVMFFIErrorSetRaisedFromCStr("ValueError", "Expected at least 2 arguments");
+ return -1;
+ }
+ // ... kernel implementation ...
+ return 0;
+ }
+
+Alternatively, :cpp:func:`TVMFFIErrorSetRaisedFromCStrParts` can also be used to concatenate
+multiple parts of the error message into a single string.
+
+**Propagating errors**. For chains of generated calls, simply propagate return codes - TLS carries
+the error details:
+
+.. code-block:: cpp
+
+ int outer_function(...) {
+ int err_code = 0;
+
+ err_code = inner_function(...);
+ if (err_code != 0) goto RAII; // Propagate error; TLS has the details
+
+ RAII:
+ // clean up owned resources
+ return err_code;
+ }
+
+Handling Non-C++ Errors
+~~~~~~~~~~~~~~~~~~~~~~~
+
+TVM-FFI preserves frontend errors (e.g., Python exceptions) through a dedicated ``-2`` return path.
+
+When a Python exception occurs during a TVM-FFI call:
+
+- The exception remains a native Python exception with its original traceback
+- The C++ side returns ``-2`` to indicate "frontend error already set"
+- Python retains the real exception object with actual frame locals
+
+This path preserves Python stack variables because the exception is never serialized into
+a TVM-FFI Error object.
+
+Function
+--------
+
+Layout and ABI
+~~~~~~~~~~~~~~
+
+:cpp:class:`tvm::ffi::FunctionObj` stores two call pointers in :cpp:class:`TVMFFIFunctionCell`:
+
+.. code-block:: cpp
+
+ typedef struct {
+ TVMFFISafeCallType safe_call;
+ void* cpp_call;
+ } TVMFFIFunctionCell;
+
+``safe_call`` is used for cross-ABI function calling, which intercepts and stores exceptions in TLS,
+while ``cpp_call`` is used inside the same DSO, and exceptions are thrown directly for performance.
+
+.. important::
+
+ Do not directly use ``safe_call`` or ``cpp_call`` to call functions. Instead, a C API :cpp:func:`TVMFFIFunctionCall`
+ is provided to handle error propagation and resource management.
+
+**Conversion with Any**. :py:class:`tvm_ffi.Function` is a TVM-FFI object, and thus conversion follows
+exactly the same rule as a TVM-FFI object does. See :ref:`Object Conversion with Any <object-conversion-with-any>` for more details.
+
+
+Throw and Catch Errors
+~~~~~~~~~~~~~~~~~~~~~~
+
+TVM-FFI gracefully handles exceptions across language boundaries without having to manually
+set error codes.
+
+**Python**. In Python, raise its language-native :py:class:`Exception <Exception>` and its
+derived classes, and TVM-FFI will catch it at ABI boundary, and convert it to a
+:cpp:class:`tvm::ffi::Error` object. If C++ code is calling into Python, the Python exception
+will be propagated back and re-raised as a :cpp:class:`tvm::ffi::Error` object to C++ side,
+and the C++ code can handle it appropriately.
+
+**C++**. In C++, use :cpp:class:`tvm::ffi::Error` or the :c:macro:`TVM_FFI_THROW` macro:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/error.h>
+
+ void ThrowError(int x) {
+ if (x < 0) {
+ TVM_FFI_THROW(ValueError) << "x must be non-negative, got " << x;
+ }
+ }
+
+The example above shows how to throw an error in C++. :c:macro:`TVM_FFI_THROW` captures
+the current file name, line number, stacktrace, and error message, and then constructs a
+:cpp:class:`tvm::ffi::Error` object. At ABI boundary, this :cpp:class:`tvm::ffi::Error`
+object is stored in the TLS slot, and the function returns ``-1`` as error code in the C ABI
+:cpp:type:`TVMFFISafeCallType` signature.
+
+.. hint::
+ A detailed implementation of such graceful handling behavior can be found
+ in :c:macro:`TVM_FFI_SAFE_CALL_BEGIN` / :c:macro:`TVM_FFI_SAFE_CALL_END` macros.
+
+
+C Registry APIs
+~~~~~~~~~~~~~~~
+
+.. list-table::
+ :header-rows: 1
+ :widths: 40 60
+
+ * - C API
+ - Description
+ * - :cpp:func:`TVMFFIFunctionGetGlobal`
+ - Get function by name. It returns an owning handle to the function.
+ * - :cpp:func:`TVMFFIFunctionSetGlobal`
+ - Register a function to the global registry.
+ * - :cpp:func:`TVMFFIFunctionCall`
+ - Call a function with variadic arguments.
+
+It is common need for a compiler developer to look up global functions in generated code. To do so,
+use :cpp:func:`TVMFFIFunctionGetGlobal` to get a function by name, and then call it with :cpp:func:`TVMFFIFunctionCall`.
+The example below shows how to lookup and call a global function in C code.
+
+.. code-block:: cpp
+
+ int LookupAndCall(const char* global_function_name, const TVMFFIAny* args, int num_args, TVMFFIAny* result) {
+ TVMFFIObject* func = NULL;
+ int err_code;
+ if ((err_code = TVMFFIFunctionGetGlobal(global_function_name, &func)) != 0)
+ goto RAII;
+ if ((err_code = TVMFFIFunctionCall(func, args, num_args, result)) != 0)
+ goto RAII;
+
+ RAII: // clean up owned resources
+ if (func != NULL) TVMFFIObjectDecRef(func);
+ return err_code;
+ }
+
+Modules
+-------
+
+A :py:class:`tvm_ffi.Module` is a namespace for a collection of functions that can be loaded
+from a shared library or bundled with the current executable. Modules provide namespace isolation
+and dynamic loading capabilities for TVM-FFI functions.
+
+Shared Library
+~~~~~~~~~~~~~~
+
+Shared library modules are loaded dynamically at runtime via ``dlopen`` (Linux, macOS) or
+``LoadLibraryW`` (Windows). This is the most common way to distribute and load compiled functions.
+
+**Export functions from C++**. Use :c:macro:`TVM_FFI_DLL_EXPORT_TYPED_FUNC` to export
+a function as a C symbol that follows the TVM-FFI ABI:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/tvm_ffi.h>
+
+ static int AddTwo(int x) { return x + 2; }
+
+ // Exports as symbol `__tvm_ffi_add_two`
+ TVM_FFI_DLL_EXPORT_TYPED_FUNC(add_two, AddTwo);
+
+**Load and call from Python**. Use :py:func:`tvm_ffi.load_module` to load the shared library:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ # Load the shared library
+ mod = tvm_ffi.load_module("path/to/library.so")
+
+ # Access functions by name
+ result = mod.add_two(40) # -> 42
+
+ # Alternative: explicit function retrieval
+ func = mod.get_function("add_two")
+ result = func(40) # -> 42
+
+**Build and load from source**. For rapid prototyping, :py:func:`tvm_ffi.cpp.load` compiles
+C++/CUDA source files and loads them as a module in one step:
+
+.. code-block:: python
+
+ import tvm_ffi.cpp
+
+ # Compile and load in one step
+ mod = tvm_ffi.cpp.load(
+ name="my_ops",
+ cpp_files="my_ops.cpp",
+ )
+ result = mod.add_two(40)
+
+Essentially, :py:func:`tvm_ffi.cpp.load` is a shortcut that combines JIT-building the library,
+and then loading the library into a :py:class:`tvm_ffi.Module`.
+
+System Library
+~~~~~~~~~~~~~~
+
+System library modules contain symbols that are statically linked to the current executable.
+This is useful for AOT (ahead-of-time) compilation where functions are bundled with the binary.
Review Comment:
This technique is useful when we would like to preserve module dynamic module loading behavior but do not have dlopen available in the system in a static bundle. Instead, we register the symbols in the system so they can be looked up via tvm_ffi.system_lib
##########
docs/concepts/func_module.rst:
##########
@@ -0,0 +1,584 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+Function and Module
+===================
+
+TVM-FFI provides a unified and ABI-stable calling convention that enables
+cross-language function calls between C++, Python, Rust and other languages.
+Functions are first-class :doc:`TVM-FFI objects <object_and_class>` that can be:
+
+- **Invoked with type-erased arguments** using a stable C ABI
+- **Registered globally** and accessed by name across languages
+- **Exported as C symbols** for efficient codegen and linking
+- **Wrapped in modules** for dynamic loading and namespace isolation
+
+This tutorial covers everything you need to know about defining, registering, and calling
+TVM-FFI functions, as well as working with modules.
+
+Glossary
+--------
+
+TVM-FFI ABI. :cpp:type:`TVMFFISafeCallType`
+ A stable C calling convention where every function is represented by a single signature,
+ which enables type-erased, cross-language function calls.
+ This calling convention is used across all TVM-FFI function calls at the ABI boundary.
+ See :ref:`Stable C ABI <tvm_ffi_c_abi>` for a quick introduction.
+
+TVM-FFI Function. :py:class:`tvm_ffi.Function`, :cpp:class:`tvm::ffi::FunctionObj`, :cpp:class:`tvm::ffi::Function`
+ A reference-counted function object and its managed reference, which wraps any callable,
+ including language-agnostic functions and lambdas (C++, Python, Rust, etc.),
+ member functions, external C symbols, and other callable objects,
+ all sharing the same calling convention.
+
+TVM-FFI Module. :py:class:`tvm_ffi.Module`, :cpp:class:`tvm::ffi::ModuleObj`, :cpp:class:`tvm::ffi::Module`
+ A namespace for a collection of functions, loaded from a shared library via ``dlopen`` (Linux, macOS) or ``LoadLibraryW`` (Windows),
+ or statically linked to the current executable.
+
+Global Functions and Registry. :py:func:`tvm_ffi.get_global_func` and :py:func:`tvm_ffi.register_global_func`
+ A registry is a table that maps string names to :cpp:class:`~tvm::ffi::Function` objects
+ and their metadata (name, docs, signatures, etc.) for cross-language access.
+ Functions in the registry are called **global functions**.
+
+Common Usage
+------------
+
+TVM-FFI C Symbols
+~~~~~~~~~~~~~~~~~
+
+**Shared library**. Use :c:macro:`TVM_FFI_DLL_EXPORT_TYPED_FUNC` to export
+a function as a C symbol that follows the TVM-FFI ABI:
+
+.. code-block:: cpp
+
+ static int AddTwo(int x) { return x + 2; }
+
+ TVM_FFI_DLL_EXPORT_TYPED_FUNC(/*ExportName=*/add_two, /*Function=*/AddTwo)
+
+This creates a C symbol ``__tvm_ffi_${ExportName}`` if the symbol is exported in a shared library,
+and later loaded and called via :py:func:`tvm_ffi.load_module`:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ mod = tvm_ffi.load_module("path/to/library.so")
+ result = mod.add_two(40) # -> 42
+
+**System library**. If the symbol is bundled in the same executable, :cpp:func:`TVMFFIEnvModRegisterSystemLibSymbol`
+is required to register the symbol during static initialization in :c:macro:`TVM_FFI_STATIC_INIT_BLOCK`.
+See examples in :py:func:`tvm_ffi.system_lib` for a complete workflow.
+
+
+Global Functions
+~~~~~~~~~~~~~~~~
+
+**Register a global function**. In C++, use :cpp:class:`tvm::ffi::reflection::GlobalDef` to
+register a function:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/tvm_ffi.h>
+
+ static int AddOne(int x) { return x + 1; }
+
+ TVM_FFI_STATIC_INIT_BLOCK() {
+ namespace refl = tvm::ffi::reflection;
+ refl::GlobalDef()
+ .def("my_ext.add_one", AddOne, "Add one to the input");
+ }
+
+:c:macro:`TVM_FFI_STATIC_INIT_BLOCK` macro ensures the registration logic happens
+during library initialization. The registered function is then accessible from
+Python under the name ``my_ext.add_one``.
+
+In Python, use the decorator :py:func:`tvm_ffi.register_global_func` to register a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ @tvm_ffi.register_global_func("my_ext.add_one")
+ def add_one(x: int) -> int:
+ return x + 1
+
+**Retrieve a global function**. After registration, functions are accessible by name.
+
+In Python, use :py:func:`tvm_ffi.get_global_func` to retrieve a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ # Get a function from the global registry
+ add_one = tvm_ffi.get_global_func("my_ext.add_one")
+ result = add_one(41) # -> 42
+
+In C++, use :cpp:func:`tvm::ffi::Function::GetGlobal` to retrieve a global function:
+
+.. code-block:: cpp
+
+ ffi::Function func = ffi::Function::GetGlobal("my_ext.add_one");
+ int result = func(41) // -> 42
+
+
+Create Functions
+~~~~~~~~~~~~~~~~
+
+**From C++**. An :cpp:class:`tvm::ffi::Function` can be created via :cpp:func:`tvm::ffi::Function::FromTyped`
+or :cpp:class:`tvm::ffi::TypedFunction`'s constructor.
+
+.. code-block:: cpp
+
+ // Create type-erased function: add_type_erased
+ ffi::Function add_type_erased = ffi::Function::FromTyped([](int x, int y) {
+ return x + y;
+ });
+
+ // Create a typed function: add_typed
+ ffi::TypedFunction<int(int, int)> add_typed = [](int x, int y) {
+ return x + y;
+ };
+
+ // Converts typed function to type-erased function
+ ffi::Function generic = add;
+
+**From Python**. An arbitrary Python :py:class:`Callable <collections.abc.Callable>` can be automatically converted
+to a :py:class:`tvm_ffi.Function` at ABI boundary. The example below shows that in function ``my_ext.bind``:
+
+- Its input ``func`` is automatically converted to :py:class:`tvm_ffi.Function`;
+- Its output, a lambda function is automatically converted to :py:class:`tvm_ffi.Function`.
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ @tvm_ffi.register_global_func("my_ext.bind")
+ def bind(func, x):
+ assert isinstance(func, tvm_ffi.Function)
+ return lambda *args: func(x, *args) # converted to `tvm_ffi.Function`
+
+ def add_x_y(x, y):
+ return x + y
+
+ func_bind = tvm_ffi.get_global_func("my_ext.bind")
+ add_y = func_bind(add_x_y, 1) # bind x = 1
+ assert isinstance(add_y, tvm_ffi.Function)
+ print(add_y(2)) # -> 3
+
+
+:py:func:`tvm_ffi.convert` explicitly converts a Python callable to :py:class:`tvm_ffi.Function`:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ def add(x, y):
+ return x + y
+
+ func_add = tvm_ffi.convert(add)
+ print(func_add(1, 2))
+
+
+C ABI Calling Convention
+------------------------
+
+.. important::
+
+ This section focuses on in-depth of TVM-FFI's stable C ABI definition,
+ describing it in pure C for precision and clarity.
+
+ Compiler code generation may follow similar patterns to call functions with TVM-FFI ABI,
+ or to generate functions that conform to TVM-FFI ABI.
+
+All TVM-FFI functions ultimately conform to the :cpp:type:`TVMFFISafeCallType` signature,
+which provides a stable C ABI for cross-language calls. The C calling convention is defined as:
+
+.. code-block:: cpp
+
+ int tvm_ffi_c_abi(
+ void* handle, // Resource handle
+ const TVMFFIAny* args, // Inputs (non-owning)
+ int num_args, // Number of arguments
+ TVMFFIAny* result // Output (owning)
+ );
+
+.. note::
+ As part of the contract, the return value ``result`` should be zero-initialized by the caller.
+
+The returning integer is **error code**, which indicates
+
+- ``0``: Success
+- ``-1``: Error occurred, retrieve via :cpp:func:`TVMFFIErrorMoveFromRaised`
+- ``-2``: Frontend error (e.g., Python exception)
+
+
+Input and Output Arguments
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The input and output arguments are passed as :cpp:type:`TVMFFIAny` pointers.
+
+- Input arguments are an array of ``num_args`` non-owning :cpp:type:`tvm::ffi::AnyView`;
+- Output argument is an owning :cpp:type:`tvm::ffi::Any`, which the caller has to guarantee to be zero-initialized.
+
+See :doc:`Any <any>` for more details on the semantics of :cpp:type:`tvm::ffi::AnyView` and :cpp:type:`tvm::ffi::Any`.
+
+Retrieve Errors in C
+~~~~~~~~~~~~~~~~~~~~
+
+When a TVM-FFI function returns a non-zero code, it indicates an error occurred,
+and the error object is stored in the TLS slot. This section shows how to retrieve
+the error object and print the error message and backtrace.
+
+.. hint::
+ Compiler code generation may use similar patterns to call any function with TVM-FFI ABI.
+
+
+**Print an Error**. The example code below shows how to print an error message and backtrace.
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/c_api.h>
+
+ static void PrintError(TVMFFIObject* err) {
+ TVMFFIErrorCell* cell = (TVMFFIErrorCell*)((char*)err + sizeof(TVMFFIObject));
+ fprintf(stderr, "%.*s: %.*s\n", (int)cell->kind.size, cell->kind.data, (int)cell->message.size, cell->message.data);
+ if (cell->backtrace.size) {
+ fprintf(stderr, "Backtrace:\n%.*s\n", (int)cell->backtrace.size, cell->backtrace.data);
+ }
+ }
+
+The payload of the error object is a :cpp:type:`TVMFFIErrorCell` structure,
+which contains the error kind, message, and backtrace, which can be accessed
+by skipping the :cpp:type:`TVMFFIObject` header with pointer arithmetics.
+
+.. note::
+
+ An :cpp:class:`~tvm::ffi::Error` object is a :cpp:class:`~tvm::ffi::Object` with a :cpp:class:`TVMFFIErrorCell` payload
+ as defined below:
+
+ .. code-block:: cpp
+
+ typedef struct {
+ TVMFFIByteArray kind; // Error type (e.g., "ValueError")
+ TVMFFIByteArray message; // Error message
+ TVMFFIByteArray backtrace; // Stack trace (most-recent call first)
+ void (*update_backtrace)(...); // Hook to append/replace backtrace
+ } TVMFFIErrorCell;
+
+
+**Retrieve error object**. When error code is ``-1``, the error object is stored in the TLS slot,
+and you can retrieve it with :cpp:func:`TVMFFIErrorMoveFromRaised`.
+
+.. code-block:: cpp
+
+ void HandleReturnCode(int rc) {
+ TVMFFIObject* err = NULL;
+ if (rc == 0) {
+ // Success
+ }
+ else if (rc == -1) {
+ // Move the raised error from TLS (clears TLS slot)
+ TVMFFIErrorMoveFromRaised(&err); // now `err` owns the error object
+ if (err != NULL) {
+ PrintError(err); // print the error
+ TVMFFIObjectDecRef(err); // Release the error object
+ }
+ }
+ else if (rc == -2) {
+ // Frontend (e.g., Python) already has an exception set.
+ // Do not fetch from TLS; consult the frontend's error mechanism.
+ }
+ }
+
+This method transfers ownership of the error object to the caller and clears the TLS slot,
+and therefore you must call :cpp:func:`TVMFFIObjectDecRef` to release the object when done
+to avoid memory leaks.
+
+Raise Errors in C
+~~~~~~~~~~~~~~~~~
+
+As part of TVM-FFI's calling convention, returning ``-1`` indicates an error occurred,
+and the error object is stored in the TLS slot. The error object may contain abitrary
+user-defined information, for example, error message, backtrace, or Python frame-local variables.
+
+.. hint::
+ Compiler code generation may use similar patterns to raise errors in generated code.
+
+The example below sets the TLS error and returns ``-1`` using :cpp:func:`TVMFFIErrorSetRaisedFromCStr`:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/c_api.h>
+
+ int __tvm_ffi_my_kernel(void* handle, const TVMFFIAny* args,
+ int32_t num_args, TVMFFIAny* result) {
+ // Validate inputs
+ if (num_args < 2) {
+ TVMFFIErrorSetRaisedFromCStr("ValueError", "Expected at least 2 arguments");
+ return -1;
+ }
+ // ... kernel implementation ...
+ return 0;
+ }
+
+Alternatively, :cpp:func:`TVMFFIErrorSetRaisedFromCStrParts` can also be used to concatenate
+multiple parts of the error message into a single string.
+
+**Propagating errors**. For chains of generated calls, simply propagate return codes - TLS carries
+the error details:
+
+.. code-block:: cpp
+
+ int outer_function(...) {
+ int err_code = 0;
+
+ err_code = inner_function(...);
+ if (err_code != 0) goto RAII; // Propagate error; TLS has the details
+
+ RAII:
+ // clean up owned resources
+ return err_code;
+ }
+
+Handling Non-C++ Errors
+~~~~~~~~~~~~~~~~~~~~~~~
+
+TVM-FFI preserves frontend errors (e.g., Python exceptions) through a dedicated ``-2`` return path.
+
+When a Python exception occurs during a TVM-FFI call:
+
+- The exception remains a native Python exception with its original traceback
+- The C++ side returns ``-2`` to indicate "frontend error already set"
+- Python retains the real exception object with actual frame locals
+
+This path preserves Python stack variables because the exception is never serialized into
+a TVM-FFI Error object.
+
+Function
+--------
+
+Layout and ABI
+~~~~~~~~~~~~~~
+
+:cpp:class:`tvm::ffi::FunctionObj` stores two call pointers in :cpp:class:`TVMFFIFunctionCell`:
+
+.. code-block:: cpp
+
+ typedef struct {
+ TVMFFISafeCallType safe_call;
+ void* cpp_call;
+ } TVMFFIFunctionCell;
+
+``safe_call`` is used for cross-ABI function calling, which intercepts and stores exceptions in TLS,
+while ``cpp_call`` is used inside the same DSO, and exceptions are thrown directly for performance.
+
+.. important::
+
+ Do not directly use ``safe_call`` or ``cpp_call`` to call functions. Instead, a C API :cpp:func:`TVMFFIFunctionCall`
Review Comment:
It is OK to use `safe_call` directly in low-level abi manipulations, `TVMFFIFunctionCall` was mainly meant as a way to expose the call in cases where function ptr manipulation is not accessible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For additional commands, e-mail: discuss-archive-help@tvm.apache.org
Re: [PR] doc: Function and Module [tvm-ffi]
Posted by "gemini-code-assist[bot] (via GitHub)" <gi...@apache.org>.
gemini-code-assist[bot] commented on PR #394:
URL: https://github.com/apache/tvm-ffi/pull/394#issuecomment-3731092830
## Summary of Changes
Hello @junrushao, I'm Gemini Code Assist[^1]! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request introduces extensive new documentation for TVM-FFI's function and module system, providing a detailed guide on how to define, register, and call functions across different languages using a stable C ABI. It also clarifies the role of modules in dynamic loading and namespace isolation. Complementing these documentation efforts, a functional enhancement allows Python callables to be directly wrapped into FFI functions, streamlining integration and aligning with the newly documented workflows.
### Highlights
* **New Documentation for Functions and Modules**: A comprehensive new documentation page, `docs/concepts/func_module.rst`, has been added. This page details the TVM-FFI function and module system, covering ABI-stable cross-language function calls, global function registration, module usage, and the stable C calling convention.
* **Enhanced Python Function Wrapping**: The Python `tvm_ffi.Function` class now supports direct initialization with a Python callable, simplifying the process of wrapping Python functions for use within the TVM-FFI system. This change is reflected in `python/tvm_ffi/cython/function.pxi` and its type stub.
* **Documentation Cross-Referencing**: Existing documentation files, including `docs/concepts/any.rst` and `docs/concepts/object_and_class.rst`, have been updated to include cross-references to the new function and module documentation, improving navigability and information discovery.
🧠 **New Feature in Public Preview:** You can now enable **Memory** to help **Gemini Code Assist** learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. **Click [here](https://codeassist.google/code-review/login) to enable Memory in your admin console.**
<details>
<summary><b>Using Gemini Code Assist</b></summary>
<br>
The full guide for Gemini Code Assist can be found on our [documentation page](https://developers.google.com/gemini-code-assist/docs/review-github-code), here are some quick tips.
<b>Invoking Gemini</b>
You can request assistance from Gemini at any point by creating a comment using either `/gemini <command>` or `@gemini-code-assist <command>`. Below is a summary of the supported commands on the current page.
Feature | Command | Description
--- | --- | ---
Code Review | `/gemini review` | Performs a code review for the current pull request in its current state.
Pull Request Summary | `/gemini summary` | Provides a summary of the current pull request in its current state.
Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help | `/gemini help` | Displays a list of available commands.
<b>Customization</b>
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a `.gemini/` folder in the base of the repository. Detailed instructions can be found [here](https://developers.google.com/gemini-code-assist/docs/customize-gemini-behavior-github).
<b>Limitations & Feedback</b>
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with :thumbsup: and :thumbsdown: on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up [here](https://google.qualtrics.com/jfe/form/SV_2cyuGuTWsEw84yG).
<b>You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the [Gemini Code Assist IDE Extension](https://cloud.google.com/products/gemini/code-assist).</b>
</details>
[^1]: Review the [Privacy Notices](https://policies.google.com/privacy), [Generative AI Prohibited Use Policy](https://policies.google.com/terms/generative-ai/use-policy), [Terms of Service](https://policies.google.com/terms), and learn how to configure Gemini Code Assist in GitHub [here](https://developers.google.com/gemini-code-assist/docs/customize-gemini-behavior-github). Gemini can make mistakes, so double check it and [use code with caution](https://support.google.com/legal/answer/13505487).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For additional commands, e-mail: discuss-archive-help@tvm.apache.org
Re: [PR] doc: Function, Exception and Module [tvm-ffi]
Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao merged PR #394:
URL: https://github.com/apache/tvm-ffi/pull/394
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For additional commands, e-mail: discuss-archive-help@tvm.apache.org
Re: [PR] doc: Function and Module [tvm-ffi]
Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao commented on code in PR #394:
URL: https://github.com/apache/tvm-ffi/pull/394#discussion_r2679095465
##########
docs/concepts/func_module.rst:
##########
@@ -0,0 +1,584 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+Function and Module
+===================
+
+TVM-FFI provides a unified and ABI-stable calling convention that enables
+cross-language function calls between C++, Python, Rust and other languages.
+Functions are first-class :doc:`TVM-FFI objects <object_and_class>` that can be:
+
+- **Invoked with type-erased arguments** using a stable C ABI
+- **Registered globally** and accessed by name across languages
+- **Exported as C symbols** for efficient codegen and linking
+- **Wrapped in modules** for dynamic loading and namespace isolation
+
+This tutorial covers everything you need to know about defining, registering, and calling
+TVM-FFI functions, as well as working with modules.
+
+Glossary
+--------
+
+TVM-FFI ABI. :cpp:type:`TVMFFISafeCallType`
+ A stable C calling convention where every function is represented by a single signature,
+ which enables type-erased, cross-language function calls.
+ This calling convention is used across all TVM-FFI function calls at the ABI boundary.
+ See :ref:`Stable C ABI <tvm_ffi_c_abi>` for a quick introduction.
+
+TVM-FFI Function. :py:class:`tvm_ffi.Function`, :cpp:class:`tvm::ffi::FunctionObj`, :cpp:class:`tvm::ffi::Function`
+ A reference-counted function object and its managed reference, which wraps any callable,
+ including language-agnostic functions and lambdas (C++, Python, Rust, etc.),
+ member functions, external C symbols, and other callable objects,
+ all sharing the same calling convention.
+
+TVM-FFI Module. :py:class:`tvm_ffi.Module`, :cpp:class:`tvm::ffi::ModuleObj`, :cpp:class:`tvm::ffi::Module`
+ A namespace for a collection of functions, loaded from a shared library via ``dlopen`` (Linux, macOS) or ``LoadLibraryW`` (Windows),
+ or statically linked to the current executable.
+
+Global Functions and Registry. :py:func:`tvm_ffi.get_global_func` and :py:func:`tvm_ffi.register_global_func`
+ A registry is a table that maps string names to :cpp:class:`~tvm::ffi::Function` objects
+ and their metadata (name, docs, signatures, etc.) for cross-language access.
+ Functions in the registry are called **global functions**.
+
+Common Usage
+------------
+
+TVM-FFI C Symbols
+~~~~~~~~~~~~~~~~~
+
+**Shared library**. Use :c:macro:`TVM_FFI_DLL_EXPORT_TYPED_FUNC` to export
+a function as a C symbol that follows the TVM-FFI ABI:
+
+.. code-block:: cpp
+
+ static int AddTwo(int x) { return x + 2; }
+
+ TVM_FFI_DLL_EXPORT_TYPED_FUNC(/*ExportName=*/add_two, /*Function=*/AddTwo)
+
+This creates a C symbol ``__tvm_ffi_${ExportName}`` if the symbol is exported in a shared library,
+and later loaded and called via :py:func:`tvm_ffi.load_module`:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ mod = tvm_ffi.load_module("path/to/library.so")
+ result = mod.add_two(40) # -> 42
+
+**System library**. If the symbol is bundled in the same executable, :cpp:func:`TVMFFIEnvModRegisterSystemLibSymbol`
+is required to register the symbol during static initialization in :c:macro:`TVM_FFI_STATIC_INIT_BLOCK`.
+See examples in :py:func:`tvm_ffi.system_lib` for a complete workflow.
+
+
+Global Functions
+~~~~~~~~~~~~~~~~
+
+**Register a global function**. In C++, use :cpp:class:`tvm::ffi::reflection::GlobalDef` to
+register a function:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/tvm_ffi.h>
+
+ static int AddOne(int x) { return x + 1; }
+
+ TVM_FFI_STATIC_INIT_BLOCK() {
+ namespace refl = tvm::ffi::reflection;
+ refl::GlobalDef()
+ .def("my_ext.add_one", AddOne, "Add one to the input");
+ }
+
+:c:macro:`TVM_FFI_STATIC_INIT_BLOCK` macro ensures the registration logic happens
+during library initialization. The registered function is then accessible from
+Python under the name ``my_ext.add_one``.
+
+In Python, use the decorator :py:func:`tvm_ffi.register_global_func` to register a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ @tvm_ffi.register_global_func("my_ext.add_one")
+ def add_one(x: int) -> int:
+ return x + 1
+
+**Retrieve a global function**. After registration, functions are accessible by name.
+
+In Python, use :py:func:`tvm_ffi.get_global_func` to retrieve a global function:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ # Get a function from the global registry
+ add_one = tvm_ffi.get_global_func("my_ext.add_one")
+ result = add_one(41) # -> 42
+
+In C++, use :cpp:func:`tvm::ffi::Function::GetGlobal` to retrieve a global function:
+
+.. code-block:: cpp
+
+ ffi::Function func = ffi::Function::GetGlobal("my_ext.add_one");
+ int result = func(41) // -> 42
+
+
+Create Functions
+~~~~~~~~~~~~~~~~
+
+**From C++**. An :cpp:class:`tvm::ffi::Function` can be created via :cpp:func:`tvm::ffi::Function::FromTyped`
+or :cpp:class:`tvm::ffi::TypedFunction`'s constructor.
+
+.. code-block:: cpp
+
+ // Create type-erased function: add_type_erased
+ ffi::Function add_type_erased = ffi::Function::FromTyped([](int x, int y) {
+ return x + y;
+ });
+
+ // Create a typed function: add_typed
+ ffi::TypedFunction<int(int, int)> add_typed = [](int x, int y) {
+ return x + y;
+ };
+
+ // Converts typed function to type-erased function
+ ffi::Function generic = add;
+
+**From Python**. An arbitrary Python :py:class:`Callable <collections.abc.Callable>` can be automatically converted
+to a :py:class:`tvm_ffi.Function` at ABI boundary. The example below shows that in function ``my_ext.bind``:
+
+- Its input ``func`` is automatically converted to :py:class:`tvm_ffi.Function`;
+- Its output, a lambda function is automatically converted to :py:class:`tvm_ffi.Function`.
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ @tvm_ffi.register_global_func("my_ext.bind")
+ def bind(func, x):
+ assert isinstance(func, tvm_ffi.Function)
+ return lambda *args: func(x, *args) # converted to `tvm_ffi.Function`
+
+ def add_x_y(x, y):
+ return x + y
+
+ func_bind = tvm_ffi.get_global_func("my_ext.bind")
+ add_y = func_bind(add_x_y, 1) # bind x = 1
+ assert isinstance(add_y, tvm_ffi.Function)
+ print(add_y(2)) # -> 3
+
+
+:py:func:`tvm_ffi.convert` explicitly converts a Python callable to :py:class:`tvm_ffi.Function`:
+
+.. code-block:: python
+
+ import tvm_ffi
+
+ def add(x, y):
+ return x + y
+
+ func_add = tvm_ffi.convert(add)
+ print(func_add(1, 2))
+
+
+C ABI Calling Convention
+------------------------
+
+.. important::
+
+ This section focuses on in-depth of TVM-FFI's stable C ABI definition,
+ describing it in pure C for precision and clarity.
+
+ Compiler code generation may follow similar patterns to call functions with TVM-FFI ABI,
+ or to generate functions that conform to TVM-FFI ABI.
+
+All TVM-FFI functions ultimately conform to the :cpp:type:`TVMFFISafeCallType` signature,
+which provides a stable C ABI for cross-language calls. The C calling convention is defined as:
+
+.. code-block:: cpp
+
+ int tvm_ffi_c_abi(
+ void* handle, // Resource handle
+ const TVMFFIAny* args, // Inputs (non-owning)
+ int num_args, // Number of arguments
+ TVMFFIAny* result // Output (owning)
+ );
+
+.. note::
+ As part of the contract, the return value ``result`` should be zero-initialized by the caller.
+
+The returning integer is **error code**, which indicates
+
+- ``0``: Success
+- ``-1``: Error occurred, retrieve via :cpp:func:`TVMFFIErrorMoveFromRaised`
+- ``-2``: Frontend error (e.g., Python exception)
+
+
+Input and Output Arguments
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The input and output arguments are passed as :cpp:type:`TVMFFIAny` pointers.
+
+- Input arguments are an array of ``num_args`` non-owning :cpp:type:`tvm::ffi::AnyView`;
+- Output argument is an owning :cpp:type:`tvm::ffi::Any`, which the caller has to guarantee to be zero-initialized.
+
+See :doc:`Any <any>` for more details on the semantics of :cpp:type:`tvm::ffi::AnyView` and :cpp:type:`tvm::ffi::Any`.
+
+Retrieve Errors in C
+~~~~~~~~~~~~~~~~~~~~
+
+When a TVM-FFI function returns a non-zero code, it indicates an error occurred,
+and the error object is stored in the TLS slot. This section shows how to retrieve
+the error object and print the error message and backtrace.
+
+.. hint::
+ Compiler code generation may use similar patterns to call any function with TVM-FFI ABI.
+
+
+**Print an Error**. The example code below shows how to print an error message and backtrace.
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/c_api.h>
+
+ static void PrintError(TVMFFIObject* err) {
+ TVMFFIErrorCell* cell = (TVMFFIErrorCell*)((char*)err + sizeof(TVMFFIObject));
+ fprintf(stderr, "%.*s: %.*s\n", (int)cell->kind.size, cell->kind.data, (int)cell->message.size, cell->message.data);
+ if (cell->backtrace.size) {
+ fprintf(stderr, "Backtrace:\n%.*s\n", (int)cell->backtrace.size, cell->backtrace.data);
+ }
+ }
+
+The payload of the error object is a :cpp:type:`TVMFFIErrorCell` structure,
+which contains the error kind, message, and backtrace, which can be accessed
+by skipping the :cpp:type:`TVMFFIObject` header with pointer arithmetics.
+
+.. note::
+
+ An :cpp:class:`~tvm::ffi::Error` object is a :cpp:class:`~tvm::ffi::Object` with a :cpp:class:`TVMFFIErrorCell` payload
+ as defined below:
+
+ .. code-block:: cpp
+
+ typedef struct {
+ TVMFFIByteArray kind; // Error type (e.g., "ValueError")
+ TVMFFIByteArray message; // Error message
+ TVMFFIByteArray backtrace; // Stack trace (most-recent call first)
+ void (*update_backtrace)(...); // Hook to append/replace backtrace
+ } TVMFFIErrorCell;
+
+
+**Retrieve error object**. When error code is ``-1``, the error object is stored in the TLS slot,
+and you can retrieve it with :cpp:func:`TVMFFIErrorMoveFromRaised`.
+
+.. code-block:: cpp
+
+ void HandleReturnCode(int rc) {
+ TVMFFIObject* err = NULL;
+ if (rc == 0) {
+ // Success
+ }
+ else if (rc == -1) {
+ // Move the raised error from TLS (clears TLS slot)
+ TVMFFIErrorMoveFromRaised(&err); // now `err` owns the error object
+ if (err != NULL) {
+ PrintError(err); // print the error
+ TVMFFIObjectDecRef(err); // Release the error object
+ }
+ }
+ else if (rc == -2) {
+ // Frontend (e.g., Python) already has an exception set.
+ // Do not fetch from TLS; consult the frontend's error mechanism.
+ }
+ }
+
+This method transfers ownership of the error object to the caller and clears the TLS slot,
+and therefore you must call :cpp:func:`TVMFFIObjectDecRef` to release the object when done
+to avoid memory leaks.
+
+Raise Errors in C
+~~~~~~~~~~~~~~~~~
+
+As part of TVM-FFI's calling convention, returning ``-1`` indicates an error occurred,
+and the error object is stored in the TLS slot. The error object may contain abitrary
+user-defined information, for example, error message, backtrace, or Python frame-local variables.
+
+.. hint::
+ Compiler code generation may use similar patterns to raise errors in generated code.
+
+The example below sets the TLS error and returns ``-1`` using :cpp:func:`TVMFFIErrorSetRaisedFromCStr`:
+
+.. code-block:: cpp
+
+ #include <tvm/ffi/c_api.h>
+
+ int __tvm_ffi_my_kernel(void* handle, const TVMFFIAny* args,
+ int32_t num_args, TVMFFIAny* result) {
+ // Validate inputs
+ if (num_args < 2) {
+ TVMFFIErrorSetRaisedFromCStr("ValueError", "Expected at least 2 arguments");
+ return -1;
+ }
+ // ... kernel implementation ...
+ return 0;
+ }
+
+Alternatively, :cpp:func:`TVMFFIErrorSetRaisedFromCStrParts` can also be used to concatenate
+multiple parts of the error message into a single string.
+
+**Propagating errors**. For chains of generated calls, simply propagate return codes - TLS carries
+the error details:
+
+.. code-block:: cpp
+
+ int outer_function(...) {
+ int err_code = 0;
+
+ err_code = inner_function(...);
+ if (err_code != 0) goto RAII; // Propagate error; TLS has the details
+
+ RAII:
+ // clean up owned resources
+ return err_code;
+ }
+
+Handling Non-C++ Errors
+~~~~~~~~~~~~~~~~~~~~~~~
+
+TVM-FFI preserves frontend errors (e.g., Python exceptions) through a dedicated ``-2`` return path.
Review Comment:
Updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For additional commands, e-mail: discuss-archive-help@tvm.apache.org
Re: [PR] doc: Function, Exception and Module [tvm-ffi]
Posted by "junrushao (via GitHub)" <gi...@apache.org>.
junrushao commented on PR #394:
URL: https://github.com/apache/tvm-ffi/pull/394#issuecomment-3733713279
@tqchen I made some updates accordingly. Could you please re-review?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For additional commands, e-mail: discuss-archive-help@tvm.apache.org