You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "tlopex (via GitHub)" <gi...@apache.org> on 2025/08/27 00:14:05 UTC
[PR] [Build] Migrate Python packaging to pyproject.toml with scikit-build-core [tvm]
tlopex opened a new pull request, #18239:
URL: https://github.com/apache/tvm/pull/18239
This pr migrates the TVM Python packaging system from the setup.py flow to the modern, PEP 517/518 compliant pyproject.toml standard, which allows us to produce a single, Python-version-agnostic wheel.
This change streamlines the process for both developers and users. For local development, you can now set up a fully-functional editable environment with a single command: `pip install -e .`. To create the distributable package for release, simply run `pip wheel -w dist .`, which will produce a universal wheel in the dist/ folder. This ensures that end-users can reliably install TVM with a standard pip install tvm, regardless of their specific Python 3 version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Build] Migrate Python packaging to pyproject.toml with scikit-build-core [tvm]
Posted by "tlopex (via GitHub)" <gi...@apache.org>.
tlopex commented on code in PR #18239:
URL: https://github.com/apache/tvm/pull/18239#discussion_r2305870497
##########
pyproject.toml:
##########
@@ -14,6 +14,190 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
+
+[build-system]
+requires = ["scikit-build-core>=0.10.0"]
+build-backend = "scikit_build_core.build"
+
+[project]
+name = "tvm"
+version = "0.22.0.dev0"
Review Comment:
Okay, added
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Build] Migrate Python packaging to pyproject.toml with scikit-build-core [tvm]
Posted by "tlopex (via GitHub)" <gi...@apache.org>.
tlopex commented on PR #18239:
URL: https://github.com/apache/tvm/pull/18239#issuecomment-3231615839
cc @MasterJH5574
I think this is ready for checking
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Build] Migrate Python packaging to pyproject.toml with scikit-build-core [tvm]
Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen commented on code in PR #18239:
URL: https://github.com/apache/tvm/pull/18239#discussion_r2303760557
##########
CMakeLists.txt:
##########
@@ -818,3 +820,123 @@ if(USE_ROCM AND USE_RCCL)
target_link_libraries(tvm PRIVATE rccl)
target_link_libraries(tvm_runtime PRIVATE rccl)
endif()
+
+# Python package installation configuration
+# This section ensures that all necessary files are installed for the Python wheel
+if(TVM_BUILD_PYTHON_MODULE)
+ message(STATUS "Configuring Python package installation")
+
+ # Install Python source files
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/python/tvm"
+ DESTINATION "tvm"
+ FILES_MATCHING
+ PATTERN "*.py"
+ PATTERN "*.pyi"
+ PATTERN "__pycache__" EXCLUDE
+ PATTERN "*.pyc" EXCLUDE
+ )
+
+ # Install compiled shared libraries
+ install(TARGETS tvm DESTINATION "tvm")
+ install(TARGETS tvm_runtime DESTINATION "tvm")
+
+ # Install third-party compiled dependencies
+ if(TARGET fpA_intB_gemm)
+ install(TARGETS fpA_intB_gemm DESTINATION "tvm")
+ endif()
+ if(TARGET flash_attn)
+ install(TARGETS flash_attn DESTINATION "tvm")
+ endif()
+
+ # Install minimal header files needed by Python extensions
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/include/tvm/runtime"
+ DESTINATION "tvm/include/tvm/runtime"
+ FILES_MATCHING
+ PATTERN "*.h"
+ )
+
+ # Install DLPack headers
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/ffi/3rdparty/dlpack/include"
+ DESTINATION "tvm/3rdparty/dlpack"
+ )
+
+ # Install libbacktrace only if enabled
+ if(USE_LIBBACKTRACE)
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/ffi/3rdparty/libbacktrace"
+ DESTINATION "tvm/3rdparty"
+ PATTERN ".git" EXCLUDE
+ PATTERN ".git*" EXCLUDE
+ PATTERN "*.tmp" EXCLUDE
+ )
+ endif()
+
+ # Install minimal CMake configuration
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/cmake/utils"
+ DESTINATION "tvm/cmake/utils"
+ FILES_MATCHING
+ PATTERN "*.cmake"
+ )
+
+ # Install minimal third-party files
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/3rdparty/dlpack/include"
+ DESTINATION "tvm/3rdparty/dlpack"
+ FILES_MATCHING
+ PATTERN "*.h"
+ )
+
+ # Install CUTLASS headers only if available
+ if(EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/3rdparty/cutlass/include")
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/3rdparty/cutlass/include"
+ DESTINATION "tvm/3rdparty/cutlass"
+ FILES_MATCHING
+ PATTERN "*.h"
+ PATTERN "*.hpp"
+ )
+ endif()
+
+ # Install other essential third-party headers
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/3rdparty/dmlc-core/include"
Review Comment:
they are shipped as part of ffi
##########
CMakeLists.txt:
##########
@@ -818,3 +820,123 @@ if(USE_ROCM AND USE_RCCL)
target_link_libraries(tvm PRIVATE rccl)
target_link_libraries(tvm_runtime PRIVATE rccl)
endif()
+
+# Python package installation configuration
+# This section ensures that all necessary files are installed for the Python wheel
+if(TVM_BUILD_PYTHON_MODULE)
+ message(STATUS "Configuring Python package installation")
+
+ # Install Python source files
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/python/tvm"
+ DESTINATION "tvm"
+ FILES_MATCHING
+ PATTERN "*.py"
+ PATTERN "*.pyi"
+ PATTERN "__pycache__" EXCLUDE
+ PATTERN "*.pyc" EXCLUDE
+ )
+
+ # Install compiled shared libraries
+ install(TARGETS tvm DESTINATION "tvm")
+ install(TARGETS tvm_runtime DESTINATION "tvm")
+
+ # Install third-party compiled dependencies
+ if(TARGET fpA_intB_gemm)
+ install(TARGETS fpA_intB_gemm DESTINATION "tvm")
+ endif()
+ if(TARGET flash_attn)
+ install(TARGETS flash_attn DESTINATION "tvm")
+ endif()
+
+ # Install minimal header files needed by Python extensions
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/include/tvm/runtime"
+ DESTINATION "tvm/include/tvm/runtime"
+ FILES_MATCHING
+ PATTERN "*.h"
+ )
+
+ # Install DLPack headers
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/ffi/3rdparty/dlpack/include"
+ DESTINATION "tvm/3rdparty/dlpack"
+ )
+
+ # Install libbacktrace only if enabled
+ if(USE_LIBBACKTRACE)
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/ffi/3rdparty/libbacktrace"
+ DESTINATION "tvm/3rdparty"
+ PATTERN ".git" EXCLUDE
+ PATTERN ".git*" EXCLUDE
+ PATTERN "*.tmp" EXCLUDE
+ )
+ endif()
+
+ # Install minimal CMake configuration
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/cmake/utils"
+ DESTINATION "tvm/cmake/utils"
+ FILES_MATCHING
+ PATTERN "*.cmake"
+ )
+
+ # Install minimal third-party files
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/3rdparty/dlpack/include"
+ DESTINATION "tvm/3rdparty/dlpack"
Review Comment:
we no longer need to install dlpack
##########
tests/lint/pylintrc:
##########
@@ -124,7 +124,8 @@ disable=
use-list-literal,
arguments-renamed,
super-init-not-called,
- c-extension-no-member
+ c-extension-no-member,
+ wrong-import-order
Review Comment:
disable import order in each file instead of globally
##########
pyproject.toml:
##########
@@ -14,6 +14,199 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
+
+[build-system]
+requires = ["scikit-build-core>=0.7.0"]
+build-backend = "scikit_build_core.build"
+
+[project]
+name = "tvm"
+version = "0.16.0.dev0"
+description = "Apache TVM: An End-to-End Deep Learning Compiler Stack"
+readme = "README.md"
+license = { text = "Apache-2.0" }
+requires-python = ">=3.8"
+authors = [
+ { name = "Apache TVM Community", email = "dev@tvm.apache.org" }
+]
+keywords = ["machine learning", "compiler", "deep learning", "inference"]
+classifiers = [
+ "Development Status :: 4 - Beta",
+ "Intended Audience :: Developers",
+ "Intended Audience :: Education",
+ "Intended Audience :: Science/Research",
+ "License :: OSI Approved :: Apache Software License",
+ "Programming Language :: Python :: 3",
+ "Programming Language :: Python :: 3.8",
+ "Programming Language :: Python :: 3.9",
+ "Programming Language :: Python :: 3.10",
+ "Programming Language :: Python :: 3.11",
+ "Programming Language :: Python :: 3.12",
+ "Topic :: Scientific/Engineering :: Artificial Intelligence",
+ "Topic :: Software Development :: Libraries :: Python Modules",
+]
+# Core dependencies - these are the minimum required for basic TVM functionality
+dependencies = [
+ "cloudpickle",
+ "ml_dtypes",
+ "numpy",
+ "packaging",
+ "psutil",
+ "scipy",
+ "tornado",
+ "typing_extensions",
+]
+
+# Optional dependencies for different features
+[project.optional-dependencies]
+# Model importers
+importer-coreml = ["coremltools"]
+importer-keras = ["tensorflow", "tensorflow-estimator"]
+importer-onnx = ["future", "onnx", "onnxoptimizer", "onnxruntime", "torch", "torchvision"]
+importer-pytorch = ["torch", "torchvision"]
+importer-tensorflow = ["tensorflow", "tensorflow-estimator"]
+importer-tflite = ["tflite"]
+importer-paddle = ["paddlepaddle"]
+
+# AutoTVM and autoscheduler
+autotvm = ["xgboost"]
+autoscheduler = ["xgboost"]
+
+# Development and testing
+dev = [
+ "black",
+ "isort",
+ "mypy",
+ "pylint",
+ "pytest",
+ "pytest-xdist",
+ "pytest-cov",
+ "pytest-mock",
+ "pytest-benchmark",
+ "pytest-timeout",
+ "pytest-rerunfailures",
+ "pytest-repeat",
+ "pytest-xdist",
+ "pytest-cov",
+ "pytest-mock",
+ "pytest-benchmark",
+ "pytest-timeout",
+ "pytest-rerunfailures",
+ "pytest-repeat",
+]
+
+# All optional dependencies (excluding dev)
+all = [
+ "coremltools",
+ "tensorflow",
+ "tensorflow-estimator",
+ "future",
+ "onnx",
+ "onnxoptimizer",
+ "onnxruntime",
+ "torch",
+ "torchvision",
+ "tflite",
+ "paddlepaddle",
+ "xgboost",
+]
+
+[project.urls]
+Homepage = "https://tvm.apache.org/"
+Documentation = "https://tvm.apache.org/docs/"
+Repository = "https://github.com/apache/tvm"
+"Bug Tracker" = "https://github.com/apache/tvm/issues"
+
+[tool.scikit-build]
+# Point to the root CMakeLists.txt
+cmake.source-dir = "."
+cmake.build-type = "Release"
+
+# Configure the wheel to be Python version-agnostic
+wheel.py-api = "py3"
+
+# Build configuration
+build-dir = "build"
+
+# CMake configuration - ensure proper installation paths
+cmake.args = [
+ "-DTVM_BUILD_PYTHON_MODULE=ON",
+ "-DTVM_FFI_BUILD_PYTHON_MODULE=ON",
+ "-DTVM_USE_CUTLASS=ON",
+ "-DTVM_USE_FLASH_ATTN=ON",
+ "-DTVM_USE_LLVM=ON",
+ "-DTVM_USE_CUDA=OFF", # Set based on your needs
Review Comment:
we only need to set -DTVM_BUILD_PYTHON_MODULE=ON, given we are shipping ffi separately, we should not set `-DTVM_FFI_BUILD_PYTHON_MODULE=ON`
##########
CMakeLists.txt:
##########
@@ -818,3 +820,123 @@ if(USE_ROCM AND USE_RCCL)
target_link_libraries(tvm PRIVATE rccl)
target_link_libraries(tvm_runtime PRIVATE rccl)
endif()
+
+# Python package installation configuration
+# This section ensures that all necessary files are installed for the Python wheel
+if(TVM_BUILD_PYTHON_MODULE)
+ message(STATUS "Configuring Python package installation")
+
+ # Install Python source files
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/python/tvm"
+ DESTINATION "tvm"
+ FILES_MATCHING
+ PATTERN "*.py"
+ PATTERN "*.pyi"
+ PATTERN "__pycache__" EXCLUDE
+ PATTERN "*.pyc" EXCLUDE
+ )
+
+ # Install compiled shared libraries
+ install(TARGETS tvm DESTINATION "tvm")
+ install(TARGETS tvm_runtime DESTINATION "tvm")
+
+ # Install third-party compiled dependencies
+ if(TARGET fpA_intB_gemm)
+ install(TARGETS fpA_intB_gemm DESTINATION "tvm")
+ endif()
+ if(TARGET flash_attn)
+ install(TARGETS flash_attn DESTINATION "tvm")
+ endif()
+
+ # Install minimal header files needed by Python extensions
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/include/tvm/runtime"
+ DESTINATION "tvm/include/tvm/runtime"
+ FILES_MATCHING
+ PATTERN "*.h"
+ )
+
+ # Install DLPack headers
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/ffi/3rdparty/dlpack/include"
+ DESTINATION "tvm/3rdparty/dlpack"
+ )
+
+ # Install libbacktrace only if enabled
+ if(USE_LIBBACKTRACE)
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/ffi/3rdparty/libbacktrace"
+ DESTINATION "tvm/3rdparty"
Review Comment:
they are shipped as part of ffi
##########
CMakeLists.txt:
##########
@@ -818,3 +820,123 @@ if(USE_ROCM AND USE_RCCL)
target_link_libraries(tvm PRIVATE rccl)
target_link_libraries(tvm_runtime PRIVATE rccl)
endif()
+
+# Python package installation configuration
+# This section ensures that all necessary files are installed for the Python wheel
+if(TVM_BUILD_PYTHON_MODULE)
+ message(STATUS "Configuring Python package installation")
+
+ # Install Python source files
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/python/tvm"
+ DESTINATION "tvm"
+ FILES_MATCHING
+ PATTERN "*.py"
+ PATTERN "*.pyi"
+ PATTERN "__pycache__" EXCLUDE
+ PATTERN "*.pyc" EXCLUDE
+ )
+
+ # Install compiled shared libraries
+ install(TARGETS tvm DESTINATION "tvm")
+ install(TARGETS tvm_runtime DESTINATION "tvm")
+
+ # Install third-party compiled dependencies
+ if(TARGET fpA_intB_gemm)
+ install(TARGETS fpA_intB_gemm DESTINATION "tvm")
+ endif()
+ if(TARGET flash_attn)
+ install(TARGETS flash_attn DESTINATION "tvm")
+ endif()
+
+ # Install minimal header files needed by Python extensions
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/include/tvm/runtime"
+ DESTINATION "tvm/include/tvm/runtime"
+ FILES_MATCHING
+ PATTERN "*.h"
+ )
+
+ # Install DLPack headers
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/ffi/3rdparty/dlpack/include"
+ DESTINATION "tvm/3rdparty/dlpack"
+ )
+
+ # Install libbacktrace only if enabled
+ if(USE_LIBBACKTRACE)
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/ffi/3rdparty/libbacktrace"
+ DESTINATION "tvm/3rdparty"
Review Comment:
we no longer need to install libbacktrace
##########
CMakeLists.txt:
##########
@@ -818,3 +820,123 @@ if(USE_ROCM AND USE_RCCL)
target_link_libraries(tvm PRIVATE rccl)
target_link_libraries(tvm_runtime PRIVATE rccl)
endif()
+
+# Python package installation configuration
+# This section ensures that all necessary files are installed for the Python wheel
+if(TVM_BUILD_PYTHON_MODULE)
+ message(STATUS "Configuring Python package installation")
+
+ # Install Python source files
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/python/tvm"
+ DESTINATION "tvm"
+ FILES_MATCHING
+ PATTERN "*.py"
+ PATTERN "*.pyi"
+ PATTERN "__pycache__" EXCLUDE
+ PATTERN "*.pyc" EXCLUDE
+ )
+
+ # Install compiled shared libraries
+ install(TARGETS tvm DESTINATION "tvm")
+ install(TARGETS tvm_runtime DESTINATION "tvm")
+
+ # Install third-party compiled dependencies
+ if(TARGET fpA_intB_gemm)
+ install(TARGETS fpA_intB_gemm DESTINATION "tvm")
+ endif()
+ if(TARGET flash_attn)
+ install(TARGETS flash_attn DESTINATION "tvm")
+ endif()
+
+ # Install minimal header files needed by Python extensions
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/include/tvm/runtime"
+ DESTINATION "tvm/include/tvm/runtime"
+ FILES_MATCHING
+ PATTERN "*.h"
+ )
+
+ # Install DLPack headers
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/ffi/3rdparty/dlpack/include"
+ DESTINATION "tvm/3rdparty/dlpack"
Review Comment:
we no longer need to ship dlpack
##########
CMakeLists.txt:
##########
@@ -818,3 +820,123 @@ if(USE_ROCM AND USE_RCCL)
target_link_libraries(tvm PRIVATE rccl)
target_link_libraries(tvm_runtime PRIVATE rccl)
endif()
+
+# Python package installation configuration
+# This section ensures that all necessary files are installed for the Python wheel
+if(TVM_BUILD_PYTHON_MODULE)
+ message(STATUS "Configuring Python package installation")
+
+ # Install Python source files
+ install(
+ DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/python/tvm"
+ DESTINATION "tvm"
Review Comment:
are we sure we want to install to the tvm subfolder? usually we need to install into the . folder in a python package, please cross confirm the wheel in pip install mode
Cross ref https://github.com/apache/tvm/blob/main/ffi/CMakeLists.txt for an example
##########
pyproject.toml:
##########
@@ -14,6 +14,199 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
+
+[build-system]
+requires = ["scikit-build-core>=0.7.0"]
+build-backend = "scikit_build_core.build"
+
+[project]
+name = "tvm"
+version = "0.16.0.dev0"
+description = "Apache TVM: An End-to-End Deep Learning Compiler Stack"
+readme = "README.md"
+license = { text = "Apache-2.0" }
+requires-python = ">=3.8"
+authors = [
+ { name = "Apache TVM Community", email = "dev@tvm.apache.org" }
+]
+keywords = ["machine learning", "compiler", "deep learning", "inference"]
+classifiers = [
+ "Development Status :: 4 - Beta",
+ "Intended Audience :: Developers",
+ "Intended Audience :: Education",
+ "Intended Audience :: Science/Research",
+ "License :: OSI Approved :: Apache Software License",
+ "Programming Language :: Python :: 3",
+ "Programming Language :: Python :: 3.8",
+ "Programming Language :: Python :: 3.9",
+ "Programming Language :: Python :: 3.10",
+ "Programming Language :: Python :: 3.11",
+ "Programming Language :: Python :: 3.12",
+ "Topic :: Scientific/Engineering :: Artificial Intelligence",
+ "Topic :: Software Development :: Libraries :: Python Modules",
+]
+# Core dependencies - these are the minimum required for basic TVM functionality
+dependencies = [
+ "cloudpickle",
+ "ml_dtypes",
+ "numpy",
+ "packaging",
+ "psutil",
+ "scipy",
+ "tornado",
+ "typing_extensions",
+]
+
+# Optional dependencies for different features
+[project.optional-dependencies]
+# Model importers
+importer-coreml = ["coremltools"]
+importer-keras = ["tensorflow", "tensorflow-estimator"]
+importer-onnx = ["future", "onnx", "onnxoptimizer", "onnxruntime", "torch", "torchvision"]
+importer-pytorch = ["torch", "torchvision"]
+importer-tensorflow = ["tensorflow", "tensorflow-estimator"]
+importer-tflite = ["tflite"]
+importer-paddle = ["paddlepaddle"]
+
+# AutoTVM and autoscheduler
+autotvm = ["xgboost"]
+autoscheduler = ["xgboost"]
+
+# Development and testing
+dev = [
+ "black",
+ "isort",
+ "mypy",
+ "pylint",
+ "pytest",
+ "pytest-xdist",
+ "pytest-cov",
+ "pytest-mock",
+ "pytest-benchmark",
+ "pytest-timeout",
+ "pytest-rerunfailures",
+ "pytest-repeat",
+ "pytest-xdist",
+ "pytest-cov",
+ "pytest-mock",
+ "pytest-benchmark",
+ "pytest-timeout",
+ "pytest-rerunfailures",
+ "pytest-repeat",
+]
+
+# All optional dependencies (excluding dev)
+all = [
+ "coremltools",
+ "tensorflow",
+ "tensorflow-estimator",
+ "future",
+ "onnx",
+ "onnxoptimizer",
+ "onnxruntime",
+ "torch",
+ "torchvision",
+ "tflite",
+ "paddlepaddle",
+ "xgboost",
+]
+
+[project.urls]
+Homepage = "https://tvm.apache.org/"
+Documentation = "https://tvm.apache.org/docs/"
+Repository = "https://github.com/apache/tvm"
+"Bug Tracker" = "https://github.com/apache/tvm/issues"
+
+[tool.scikit-build]
+# Point to the root CMakeLists.txt
+cmake.source-dir = "."
+cmake.build-type = "Release"
+
+# Configure the wheel to be Python version-agnostic
+wheel.py-api = "py3"
+
+# Build configuration
+build-dir = "build"
+
+# CMake configuration - ensure proper installation paths
+cmake.args = [
+ "-DTVM_BUILD_PYTHON_MODULE=ON",
+ "-DTVM_FFI_BUILD_PYTHON_MODULE=ON",
+ "-DTVM_USE_CUTLASS=ON",
+ "-DTVM_USE_FLASH_ATTN=ON",
+ "-DTVM_USE_LLVM=ON",
+ "-DTVM_USE_CUDA=OFF", # Set based on your needs
+ "-DTVM_USE_OPENCL=OFF",
+ "-DTVM_USE_VULKAN=OFF",
+ "-DTVM_USE_METAL=OFF",
+ "-DTVM_USE_OPENGL=OFF",
+ "-DTVM_USE_RPC=ON",
+ "-DTVM_USE_GRAPH_EXECUTOR=ON",
+ "-DTVM_USE_PROFILER=ON",
+ "-DTVM_USE_UTILS=ON",
+]
+
+# Ensure dependent libraries install to same location as libtvm.so
+cmake.define.CMAKE_INSTALL_PREFIX = "python/tvm"
+cmake.define.CMAKE_INSTALL_LIBDIR = "lib"
Review Comment:
we should not set cmake configs here, instead, we should set https://github.com/apache/tvm/blob/main/ffi/pyproject.toml#L77
##########
src/support/libinfo.cc:
##########
@@ -106,6 +106,10 @@
#define TVM_INFO_USE_HEXAGON_GTEST "NOT-FOUND"
#endif
+#ifndef TVM_INFO_TVM_BUILD_PYTHON_MODULE
+#define TVM_INFO_TVM_BUILD_PYTHON_MODULE "NOT-FOUND"
Review Comment:
this does not have to goto libinfo, as the flag is not as useful
##########
pyproject.toml:
##########
@@ -14,6 +14,199 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
+
+[build-system]
+requires = ["scikit-build-core>=0.7.0"]
+build-backend = "scikit_build_core.build"
+
+[project]
+name = "tvm"
+version = "0.16.0.dev0"
+description = "Apache TVM: An End-to-End Deep Learning Compiler Stack"
+readme = "README.md"
+license = { text = "Apache-2.0" }
+requires-python = ">=3.8"
+authors = [
+ { name = "Apache TVM Community", email = "dev@tvm.apache.org" }
+]
+keywords = ["machine learning", "compiler", "deep learning", "inference"]
+classifiers = [
+ "Development Status :: 4 - Beta",
+ "Intended Audience :: Developers",
+ "Intended Audience :: Education",
+ "Intended Audience :: Science/Research",
+ "License :: OSI Approved :: Apache Software License",
+ "Programming Language :: Python :: 3",
+ "Programming Language :: Python :: 3.8",
+ "Programming Language :: Python :: 3.9",
+ "Programming Language :: Python :: 3.10",
+ "Programming Language :: Python :: 3.11",
+ "Programming Language :: Python :: 3.12",
+ "Topic :: Scientific/Engineering :: Artificial Intelligence",
+ "Topic :: Software Development :: Libraries :: Python Modules",
+]
+# Core dependencies - these are the minimum required for basic TVM functionality
+dependencies = [
+ "cloudpickle",
+ "ml_dtypes",
+ "numpy",
+ "packaging",
+ "psutil",
+ "scipy",
+ "tornado",
+ "typing_extensions",
+]
+
+# Optional dependencies for different features
+[project.optional-dependencies]
+# Model importers
+importer-coreml = ["coremltools"]
+importer-keras = ["tensorflow", "tensorflow-estimator"]
+importer-onnx = ["future", "onnx", "onnxoptimizer", "onnxruntime", "torch", "torchvision"]
+importer-pytorch = ["torch", "torchvision"]
+importer-tensorflow = ["tensorflow", "tensorflow-estimator"]
+importer-tflite = ["tflite"]
+importer-paddle = ["paddlepaddle"]
+
+# AutoTVM and autoscheduler
+autotvm = ["xgboost"]
+autoscheduler = ["xgboost"]
+
+# Development and testing
+dev = [
+ "black",
+ "isort",
+ "mypy",
+ "pylint",
+ "pytest",
+ "pytest-xdist",
+ "pytest-cov",
+ "pytest-mock",
+ "pytest-benchmark",
+ "pytest-timeout",
+ "pytest-rerunfailures",
+ "pytest-repeat",
+ "pytest-xdist",
+ "pytest-cov",
+ "pytest-mock",
+ "pytest-benchmark",
+ "pytest-timeout",
+ "pytest-rerunfailures",
+ "pytest-repeat",
+]
+
+# All optional dependencies (excluding dev)
+all = [
+ "coremltools",
+ "tensorflow",
+ "tensorflow-estimator",
+ "future",
+ "onnx",
+ "onnxoptimizer",
+ "onnxruntime",
+ "torch",
+ "torchvision",
+ "tflite",
+ "paddlepaddle",
+ "xgboost",
+]
+
+[project.urls]
+Homepage = "https://tvm.apache.org/"
+Documentation = "https://tvm.apache.org/docs/"
+Repository = "https://github.com/apache/tvm"
+"Bug Tracker" = "https://github.com/apache/tvm/issues"
+
+[tool.scikit-build]
+# Point to the root CMakeLists.txt
+cmake.source-dir = "."
+cmake.build-type = "Release"
+
+# Configure the wheel to be Python version-agnostic
+wheel.py-api = "py3"
+
+# Build configuration
+build-dir = "build"
+
+# CMake configuration - ensure proper installation paths
+cmake.args = [
+ "-DTVM_BUILD_PYTHON_MODULE=ON",
+ "-DTVM_FFI_BUILD_PYTHON_MODULE=ON",
+ "-DTVM_USE_CUTLASS=ON",
+ "-DTVM_USE_FLASH_ATTN=ON",
+ "-DTVM_USE_LLVM=ON",
+ "-DTVM_USE_CUDA=OFF", # Set based on your needs
Review Comment:
do not set these configs by default, instead they can be picked up by config.cmake
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Build] Migrate Python packaging to pyproject.toml with scikit-build-core [tvm]
Posted by "MasterJH5574 (via GitHub)" <gi...@apache.org>.
MasterJH5574 commented on PR #18239:
URL: https://github.com/apache/tvm/pull/18239#issuecomment-3231759434
> I think this is ready for checking
Great, I'll find a time in the next few days. BTW there is a windows CI error that we need to fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Build] Migrate Python packaging to pyproject.toml with scikit-build-core [tvm]
Posted by "tlopex (via GitHub)" <gi...@apache.org>.
tlopex commented on code in PR #18239:
URL: https://github.com/apache/tvm/pull/18239#discussion_r2305004213
##########
pyproject.toml:
##########
@@ -14,6 +14,199 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
+
+[build-system]
+requires = ["scikit-build-core>=0.7.0"]
Review Comment:
I think so. I'll align it later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Build] Migrate Python packaging to pyproject.toml with scikit-build-core [tvm]
Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen commented on PR #18239:
URL: https://github.com/apache/tvm/pull/18239#issuecomment-3233063260
windows should be fixed already by https://github.com/apache/tvm/pull/18245.
We consider should followup and change the conda based windows ci to use the pip wheel based now that we have the scikit-build-core
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Build] Migrate Python packaging to pyproject.toml with scikit-build-core [tvm]
Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen commented on PR #18239:
URL: https://github.com/apache/tvm/pull/18239#issuecomment-3227998365
also cc @MasterJH5574 , would be great to test the wheel for specifically cuda related usecases like cutlass(maybe we need to also update mlc),
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Build] Migrate Python packaging to pyproject.toml with scikit-build-core [tvm]
Posted by "tlopex (via GitHub)" <gi...@apache.org>.
tlopex commented on PR #18239:
URL: https://github.com/apache/tvm/pull/18239#issuecomment-3231765074
> > I think this is ready for checking
>
> Great, I'll find a time in the next few days. BTW there is a windows CI error that we need to fix.
How could I fix it? Just by committing again?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Build] Migrate Python packaging to pyproject.toml with scikit-build-core [tvm]
Posted by "tlopex (via GitHub)" <gi...@apache.org>.
tlopex commented on PR #18239:
URL: https://github.com/apache/tvm/pull/18239#issuecomment-3226499397
cc @tqchen
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Build] Migrate Python packaging to pyproject.toml with scikit-build-core [tvm]
Posted by "MasterJH5574 (via GitHub)" <gi...@apache.org>.
MasterJH5574 commented on PR #18239:
URL: https://github.com/apache/tvm/pull/18239#issuecomment-3229329389
> would be great to test the wheel for specifically cuda related usecases like cutlass(maybe we need to also update mlc),
Sure. Happy to test when this PR gets ready.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Build] Migrate Python packaging to pyproject.toml with scikit-build-core [tvm]
Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen commented on code in PR #18239:
URL: https://github.com/apache/tvm/pull/18239#discussion_r2305458300
##########
pyproject.toml:
##########
@@ -14,6 +14,190 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
+
+[build-system]
+requires = ["scikit-build-core>=0.10.0"]
+build-backend = "scikit_build_core.build"
+
+[project]
+name = "tvm"
+version = "0.22.0.dev0"
Review Comment:
Add a comment that version.py needs to be called to update the version before we build the wheel, update
https://github.com/apache/tvm/blob/main/version.py to replace the version here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Build] Migrate Python packaging to pyproject.toml with scikit-build-core [tvm]
Posted by "MasterJH5574 (via GitHub)" <gi...@apache.org>.
MasterJH5574 merged PR #18239:
URL: https://github.com/apache/tvm/pull/18239
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Build] Migrate Python packaging to pyproject.toml with scikit-build-core [tvm]
Posted by "MasterJH5574 (via GitHub)" <gi...@apache.org>.
MasterJH5574 commented on code in PR #18239:
URL: https://github.com/apache/tvm/pull/18239#discussion_r2304992596
##########
pyproject.toml:
##########
@@ -14,6 +14,199 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
+
+[build-system]
+requires = ["scikit-build-core>=0.7.0"]
+build-backend = "scikit_build_core.build"
+
+[project]
+name = "tvm"
+version = "0.16.0.dev0"
Review Comment:
The latest version might be 0.22.0.dev0 according to https://github.com/apache/tvm/tags
##########
pyproject.toml:
##########
@@ -14,6 +14,199 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
+
+[build-system]
+requires = ["scikit-build-core>=0.7.0"]
+build-backend = "scikit_build_core.build"
+
+[project]
+name = "tvm"
+version = "0.16.0.dev0"
+description = "Apache TVM: An End-to-End Deep Learning Compiler Stack"
+readme = "README.md"
+license = { text = "Apache-2.0" }
+requires-python = ">=3.8"
+authors = [
+ { name = "Apache TVM Community", email = "dev@tvm.apache.org" }
+]
+keywords = ["machine learning", "compiler", "deep learning", "inference"]
+classifiers = [
+ "Development Status :: 4 - Beta",
+ "Intended Audience :: Developers",
+ "Intended Audience :: Education",
+ "Intended Audience :: Science/Research",
+ "License :: OSI Approved :: Apache Software License",
+ "Programming Language :: Python :: 3",
+ "Programming Language :: Python :: 3.8",
Review Comment:
FFI has a minimal Python version requirement of 3.9. Likely we need to align with that https://github.com/apache/tvm/blob/472b2fcd22c4147c35cbdf884d232f69bd86a23b/ffi/pyproject.toml#L34
##########
pyproject.toml:
##########
@@ -14,6 +14,199 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
+
+[build-system]
+requires = ["scikit-build-core>=0.7.0"]
Review Comment:
I saw ffi requires the minimal version of 0.10.0. Do we want to align the minimal version requirement?
https://github.com/apache/tvm/blob/472b2fcd22c4147c35cbdf884d232f69bd86a23b/ffi/pyproject.toml#L51-L53
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Build] Migrate Python packaging to pyproject.toml with scikit-build-core [tvm]
Posted by "tlopex (via GitHub)" <gi...@apache.org>.
tlopex commented on code in PR #18239:
URL: https://github.com/apache/tvm/pull/18239#discussion_r2305004213
##########
pyproject.toml:
##########
@@ -14,6 +14,199 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
+
+[build-system]
+requires = ["scikit-build-core>=0.7.0"]
Review Comment:
I think so. I'll align all of them later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Build] Migrate Python packaging to pyproject.toml with scikit-build-core [tvm]
Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen commented on PR #18239:
URL: https://github.com/apache/tvm/pull/18239#issuecomment-3229959701
@tlopex @MasterJH5574 please help cross check once ready
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org