You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/07/03 11:32:56 UTC
[GitHub] [tvm] Codle opened a new pull request, #12000: [Fix] fix python setup.py file bug
Codle opened a new pull request, #12000:
URL: https://github.com/apache/tvm/pull/12000
Signed-off-by: Zhengqiang Yin <co...@outlook.com>
The `data_files` filed in `setup_kwargs` seems only allow file path rather than dir path. I try to use `os.walk` to find all the files and re-set them to the `data_files` filed. This works fine for me and someone else.
Related issue is : https://github.com/apache/tvm/issues/11726
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] kparzysz-quic merged pull request #12000: [Fix] fix python setup.py file bug
Posted by GitBox <gi...@apache.org>.
kparzysz-quic merged PR #12000:
URL: https://github.com/apache/tvm/pull/12000
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Codle commented on a diff in pull request #12000: [Fix] fix python setup.py file bug
Posted by GitBox <gi...@apache.org>.
Codle commented on code in PR #12000:
URL: https://github.com/apache/tvm/pull/12000#discussion_r913208500
##########
python/setup.py:
##########
@@ -194,7 +194,15 @@ def is_pure(self):
curr_path = os.path.dirname(os.path.abspath(os.path.expanduser(__file__)))
for i, path in enumerate(LIB_LIST):
LIB_LIST[i] = os.path.relpath(path, curr_path)
- setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_LIST)]}
+ LIB_DATA_FILES = []
+ if os.path.isfile(path):
+ LIB_DATA_FILES.append(os.path.relpath(path, curr_path))
+ if os.path.isdir(path):
+ for root, dirs, files in os.walk(path):
+ for filename in files:
+ file_path = os.path.join(root, filename)
+ LIB_DATA_FILES.append(os.path.relpath(file_path, curr_path))
+ setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_DATA_FILES)]}
Review Comment:
I have made some changes for this, please help me do double-check. I found the `data_files` filed is valid for both `bdist_wheel` and other actions, so I think it was no need a conditional branch.
I have tested `python setup.py bdist_wheel` and `python setup.py install`, and they created the same library files (by unzipping .egg and .wheel file).
some references: https://stackoverflow.com/questions/24347450/how-do-you-add-additional-files-to-a-wheel
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] leandron commented on a diff in pull request #12000: [Fix] fix python setup.py file bug
Posted by GitBox <gi...@apache.org>.
leandron commented on code in PR #12000:
URL: https://github.com/apache/tvm/pull/12000#discussion_r913637173
##########
python/setup.py:
##########
@@ -194,7 +194,15 @@ def is_pure(self):
curr_path = os.path.dirname(os.path.abspath(os.path.expanduser(__file__)))
for i, path in enumerate(LIB_LIST):
LIB_LIST[i] = os.path.relpath(path, curr_path)
- setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_LIST)]}
+ LIB_DATA_FILES = []
+ if os.path.isfile(path):
+ LIB_DATA_FILES.append(os.path.relpath(path, curr_path))
+ if os.path.isdir(path):
+ for root, dirs, files in os.walk(path):
+ for filename in files:
+ file_path = os.path.join(root, filename)
+ LIB_DATA_FILES.append(os.path.relpath(file_path, curr_path))
+ setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_DATA_FILES)]}
Review Comment:
I think the issue with that approach is that internal subpaths are lost and the files are kept flat in the package. Can you send the output of `tree` on the directory you extract from the `.egg / .wheel` files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Codle commented on pull request #12000: [Fix] fix python setup.py file bug
Posted by GitBox <gi...@apache.org>.
Codle commented on PR #12000:
URL: https://github.com/apache/tvm/pull/12000#issuecomment-1173069062
I'm a little confused about choosing the corresponding reviewers for this PR. I'm sorry if this mention disturbed you, and please help me correct it. Thanks a lot.
@junrushao1994 @yongsun @ezyang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Codle commented on a diff in pull request #12000: [Fix] fix python setup.py file bug
Posted by GitBox <gi...@apache.org>.
Codle commented on code in PR #12000:
URL: https://github.com/apache/tvm/pull/12000#discussion_r913208500
##########
python/setup.py:
##########
@@ -194,7 +194,15 @@ def is_pure(self):
curr_path = os.path.dirname(os.path.abspath(os.path.expanduser(__file__)))
for i, path in enumerate(LIB_LIST):
LIB_LIST[i] = os.path.relpath(path, curr_path)
- setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_LIST)]}
+ LIB_DATA_FILES = []
+ if os.path.isfile(path):
+ LIB_DATA_FILES.append(os.path.relpath(path, curr_path))
+ if os.path.isdir(path):
+ for root, dirs, files in os.walk(path):
+ for filename in files:
+ file_path = os.path.join(root, filename)
+ LIB_DATA_FILES.append(os.path.relpath(file_path, curr_path))
+ setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_DATA_FILES)]}
Review Comment:
I have made some changes for this, please help me do double-check. I found the `data_files` filed is valid for both `bdist_wheel` and other actions, so I think it was no need a conditional branch.
I have tested `python setup.py bdist_wheel` and `python setup.py install`, and they have the same library files (by unzipping .egg and .wheel file).
some references: https://stackoverflow.com/questions/24347450/how-do-you-add-additional-files-to-a-wheel
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Codle commented on a diff in pull request #12000: [Fix] fix python setup.py file bug
Posted by GitBox <gi...@apache.org>.
Codle commented on code in PR #12000:
URL: https://github.com/apache/tvm/pull/12000#discussion_r913202840
##########
python/setup.py:
##########
@@ -194,7 +194,15 @@ def is_pure(self):
curr_path = os.path.dirname(os.path.abspath(os.path.expanduser(__file__)))
for i, path in enumerate(LIB_LIST):
LIB_LIST[i] = os.path.relpath(path, curr_path)
- setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_LIST)]}
+ LIB_DATA_FILES = []
+ if os.path.isfile(path):
Review Comment:
Sorry, it's my mistake and I have fixed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] leandron commented on a diff in pull request #12000: [Fix] fix python setup.py file bug
Posted by GitBox <gi...@apache.org>.
leandron commented on code in PR #12000:
URL: https://github.com/apache/tvm/pull/12000#discussion_r913150062
##########
python/setup.py:
##########
@@ -194,7 +194,15 @@ def is_pure(self):
curr_path = os.path.dirname(os.path.abspath(os.path.expanduser(__file__)))
for i, path in enumerate(LIB_LIST):
LIB_LIST[i] = os.path.relpath(path, curr_path)
- setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_LIST)]}
+ LIB_DATA_FILES = []
+ if os.path.isfile(path):
Review Comment:
The variable `path` in `if os.path.isfile(path):` is produced by the `enumerate(LIB_LIST)` above, right? So it seems the indentation of this block doesn't match with the variable of the iteration, and it is only catching the last value on the list? Can you double check on whether this is the expected behaviour?
##########
python/setup.py:
##########
@@ -194,7 +194,15 @@ def is_pure(self):
curr_path = os.path.dirname(os.path.abspath(os.path.expanduser(__file__)))
for i, path in enumerate(LIB_LIST):
LIB_LIST[i] = os.path.relpath(path, curr_path)
- setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_LIST)]}
+ LIB_DATA_FILES = []
+ if os.path.isfile(path):
+ LIB_DATA_FILES.append(os.path.relpath(path, curr_path))
+ if os.path.isdir(path):
+ for root, dirs, files in os.walk(path):
+ for filename in files:
+ file_path = os.path.join(root, filename)
+ LIB_DATA_FILES.append(os.path.relpath(file_path, curr_path))
+ setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_DATA_FILES)]}
Review Comment:
The logic here seems duplicated with `f"recursive-include tvm/{libname} *\n"` above. Is there a way to unify so that we can only add files in one standard way that works for local installation as well as packaging?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Codle commented on a diff in pull request #12000: [Fix] fix python setup.py file bug
Posted by GitBox <gi...@apache.org>.
Codle commented on code in PR #12000:
URL: https://github.com/apache/tvm/pull/12000#discussion_r914094183
##########
python/setup.py:
##########
@@ -194,7 +194,15 @@ def is_pure(self):
curr_path = os.path.dirname(os.path.abspath(os.path.expanduser(__file__)))
for i, path in enumerate(LIB_LIST):
LIB_LIST[i] = os.path.relpath(path, curr_path)
- setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_LIST)]}
+ LIB_DATA_FILES = []
+ if os.path.isfile(path):
+ LIB_DATA_FILES.append(os.path.relpath(path, curr_path))
+ if os.path.isdir(path):
+ for root, dirs, files in os.walk(path):
+ for filename in files:
+ file_path = os.path.join(root, filename)
+ LIB_DATA_FILES.append(os.path.relpath(file_path, curr_path))
+ setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_DATA_FILES)]}
Review Comment:
Yes, you are right, the files are kept flat in the package. I try to find out the best practice of packing non-package file in setup.py and this PR gives my idea: https://github.com/pypa/sampleproject/issues/30
While include_package_data=True, the setup program will run commands in MANIFEST.in, so it's less necessary to use data_files. This step is explained in [setuptools/docs/references/keywords.rst](https://github.com/pypa/setuptools/blob/main/docs/references/keywords.rst)
> include_package_data: If set to True, this tells setuptools to automatically include any data files it finds inside your package directories that are specified by your MANIFEST.in file. For more information, see the section on [:ref:`Including Data Files`](https://github.com/pypa/setuptools/blob/0bd847c4aea839b3a75f4e3f09a36526453bd267/docs/references/keywords.rst#id23).
In the first PR: https://github.com/apache/tvm/pull/380 for data_files field, I found the include_package_data was set to False in wheel packing. So the detail of include_package_data may be ignored by the commiter.
Here is the result of unzipping .egg / .whl file:
.whl
```
dist
├── tvm
│ ├── __init__.py
│ ├── _ffi
│ ├── arith
│ ├── auto_scheduler
│ ├── autotvm
│ ├── configs
│ ├── contrib
│ ├── driver
│ ├── error.py
│ ├── exec
│ ├── generic.py
│ ├── ir
│ ├── libtvm.so
│ ├── libtvm_runtime.so
│ ├── meta_schedule
│ ├── micro
│ ├── parser
│ ├── relay
│ ├── rpc
│ ├── runtime
│ ├── script
│ ├── support.py
│ ├── target
│ ├── te
│ ├── testing
│ ├── tir
│ ├── topi
│ └── utils
├── tvm-0.9.dev1753+g6236f2cb6-cp38-cp38-linux_x86_64.whl
└── tvm-0.9.dev1753+g6236f2cb6.dist-info
├── METADATA
├── RECORD
├── WHEEL
├── entry_points.txt
└── top_level.txt
```
.egg
```
dist
├── EGG-INFO
│ ├── PKG-INFO
│ ├── SOURCES.txt
│ ├── dependency_links.txt
│ ├── entry_points.txt
│ ├── native_libs.txt
│ ├── not-zip-safe
│ ├── requires.txt
│ └── top_level.txt
├── tvm
│ ├── __init__.py
│ ├── _ffi
│ ├── arith
│ ├── auto_scheduler
│ ├── autotvm
│ ├── configs
│ ├── contrib
│ ├── driver
│ ├── error.py
│ ├── exec
│ ├── generic.py
│ ├── ir
│ ├── libtvm.so
│ ├── libtvm_runtime.so
│ ├── meta_schedule
│ ├── micro
│ ├── parser
│ ├── relay
│ ├── rpc
│ ├── runtime
│ ├── script
│ ├── support.py
│ ├── target
│ ├── te
│ ├── testing
│ ├── tir
│ ├── topi
│ └── utils
└── tvm-0.9.dev1754+g2a71c380e-py3.8-linux-x86_64.egg
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Codle commented on a diff in pull request #12000: [Fix] fix python setup.py file bug
Posted by GitBox <gi...@apache.org>.
Codle commented on code in PR #12000:
URL: https://github.com/apache/tvm/pull/12000#discussion_r914094183
##########
python/setup.py:
##########
@@ -194,7 +194,15 @@ def is_pure(self):
curr_path = os.path.dirname(os.path.abspath(os.path.expanduser(__file__)))
for i, path in enumerate(LIB_LIST):
LIB_LIST[i] = os.path.relpath(path, curr_path)
- setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_LIST)]}
+ LIB_DATA_FILES = []
+ if os.path.isfile(path):
+ LIB_DATA_FILES.append(os.path.relpath(path, curr_path))
+ if os.path.isdir(path):
+ for root, dirs, files in os.walk(path):
+ for filename in files:
+ file_path = os.path.join(root, filename)
+ LIB_DATA_FILES.append(os.path.relpath(file_path, curr_path))
+ setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_DATA_FILES)]}
Review Comment:
Yes, you are right, the files are kept flat in the package. I try to find out the best practice of packing non-package file in setup.py and this PR gives my idea: https://github.com/pypa/sampleproject/issues/30
While include_package_data=True, the setup program will go to run command in MANIFEST.in. This step is explained in [setuptools/docs/references/keywords.rst](https://github.com/pypa/setuptools/blob/main/docs/references/keywords.rst)
> include_package_data: If set to True, this tells setuptools to automatically include any data files it finds inside your package directories that are specified by your MANIFEST.in file. For more information, see the section on [:ref:`Including Data Files`](https://github.com/pypa/setuptools/blob/0bd847c4aea839b3a75f4e3f09a36526453bd267/docs/references/keywords.rst#id23).
In the first PR: https://github.com/apache/tvm/pull/380 for data_files field, I found the include_package_data was set to False in wheel packing. So the detail of include_package_data may be ignored by the commiter.
Here is the result of unzipping .egg / .whl file:
.whl
```
dist
├── tvm
│ ├── __init__.py
│ ├── _ffi
│ ├── arith
│ ├── auto_scheduler
│ ├── autotvm
│ ├── configs
│ ├── contrib
│ ├── driver
│ ├── error.py
│ ├── exec
│ ├── generic.py
│ ├── ir
│ ├── libtvm.so
│ ├── libtvm_runtime.so
│ ├── meta_schedule
│ ├── micro
│ ├── parser
│ ├── relay
│ ├── rpc
│ ├── runtime
│ ├── script
│ ├── support.py
│ ├── target
│ ├── te
│ ├── testing
│ ├── tir
│ ├── topi
│ └── utils
├── tvm-0.9.dev1753+g6236f2cb6-cp38-cp38-linux_x86_64.whl
└── tvm-0.9.dev1753+g6236f2cb6.dist-info
├── METADATA
├── RECORD
├── WHEEL
├── entry_points.txt
└── top_level.txt
```
.egg
```
dist
├── EGG-INFO
│ ├── PKG-INFO
│ ├── SOURCES.txt
│ ├── dependency_links.txt
│ ├── entry_points.txt
│ ├── native_libs.txt
│ ├── not-zip-safe
│ ├── requires.txt
│ └── top_level.txt
├── tvm
│ ├── __init__.py
│ ├── _ffi
│ ├── arith
│ ├── auto_scheduler
│ ├── autotvm
│ ├── configs
│ ├── contrib
│ ├── driver
│ ├── error.py
│ ├── exec
│ ├── generic.py
│ ├── ir
│ ├── libtvm.so
│ ├── libtvm_runtime.so
│ ├── meta_schedule
│ ├── micro
│ ├── parser
│ ├── relay
│ ├── rpc
│ ├── runtime
│ ├── script
│ ├── support.py
│ ├── target
│ ├── te
│ ├── testing
│ ├── tir
│ ├── topi
│ └── utils
└── tvm-0.9.dev1754+g2a71c380e-py3.8-linux-x86_64.egg
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] leandron commented on a diff in pull request #12000: [Fix] fix python setup.py file bug
Posted by GitBox <gi...@apache.org>.
leandron commented on code in PR #12000:
URL: https://github.com/apache/tvm/pull/12000#discussion_r914646185
##########
python/setup.py:
##########
@@ -194,7 +194,15 @@ def is_pure(self):
curr_path = os.path.dirname(os.path.abspath(os.path.expanduser(__file__)))
for i, path in enumerate(LIB_LIST):
LIB_LIST[i] = os.path.relpath(path, curr_path)
- setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_LIST)]}
+ LIB_DATA_FILES = []
+ if os.path.isfile(path):
+ LIB_DATA_FILES.append(os.path.relpath(path, curr_path))
+ if os.path.isdir(path):
+ for root, dirs, files in os.walk(path):
+ for filename in files:
+ file_path = os.path.join(root, filename)
+ LIB_DATA_FILES.append(os.path.relpath(file_path, curr_path))
+ setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_DATA_FILES)]}
Review Comment:
Understood. Can you build TVM with `USE_MICRO ON`, so that more files are expected to be put in the package, namely `standalone_crt`, which has a deeper file structure?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Codle commented on a diff in pull request #12000: [Fix] fix python setup.py file bug
Posted by GitBox <gi...@apache.org>.
Codle commented on code in PR #12000:
URL: https://github.com/apache/tvm/pull/12000#discussion_r914094183
##########
python/setup.py:
##########
@@ -194,7 +194,15 @@ def is_pure(self):
curr_path = os.path.dirname(os.path.abspath(os.path.expanduser(__file__)))
for i, path in enumerate(LIB_LIST):
LIB_LIST[i] = os.path.relpath(path, curr_path)
- setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_LIST)]}
+ LIB_DATA_FILES = []
+ if os.path.isfile(path):
+ LIB_DATA_FILES.append(os.path.relpath(path, curr_path))
+ if os.path.isdir(path):
+ for root, dirs, files in os.walk(path):
+ for filename in files:
+ file_path = os.path.join(root, filename)
+ LIB_DATA_FILES.append(os.path.relpath(file_path, curr_path))
+ setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_DATA_FILES)]}
Review Comment:
Yes, you are right, the files are kept flat in the package. I try to find out the best practice of packing non-package file in setup.py and this PR gives my idea: https://github.com/pypa/sampleproject/issues/30
While include_package_data=True, the setup program will run commands in MANIFEST.in, so it's less necessary to use data_files. This step is explained in [setuptools/docs/references/keywords.rst](https://github.com/pypa/setuptools/blob/main/docs/references/keywords.rst)
> include_package_data: If set to True, this tells setuptools to automatically include any data files it finds inside your package directories that are specified by your MANIFEST.in file. For more information, see the section on [:ref:`Including Data Files`](https://github.com/pypa/setuptools/blob/0bd847c4aea839b3a75f4e3f09a36526453bd267/docs/references/keywords.rst#id23).
In the first PR: https://github.com/apache/tvm/pull/380 for data_files field, I found the include_package_data was set to False in wheel packing. So the automatically copy action of include_package_data may be ignored by the commiter.
Here is the result of unzipping .egg / .whl file, `configs` dir not be flatten to the json files.
.whl
```
dist
├── tvm
│ ├── __init__.py
│ ├── _ffi
│ ├── arith
│ ├── auto_scheduler
│ ├── autotvm
│ ├── configs
│ ├── contrib
│ ├── driver
│ ├── error.py
│ ├── exec
│ ├── generic.py
│ ├── ir
│ ├── libtvm.so
│ ├── libtvm_runtime.so
│ ├── meta_schedule
│ ├── micro
│ ├── parser
│ ├── relay
│ ├── rpc
│ ├── runtime
│ ├── script
│ ├── support.py
│ ├── target
│ ├── te
│ ├── testing
│ ├── tir
│ ├── topi
│ └── utils
├── tvm-0.9.dev1753+g6236f2cb6-cp38-cp38-linux_x86_64.whl
└── tvm-0.9.dev1753+g6236f2cb6.dist-info
├── METADATA
├── RECORD
├── WHEEL
├── entry_points.txt
└── top_level.txt
```
.egg
```
dist
├── EGG-INFO
│ ├── PKG-INFO
│ ├── SOURCES.txt
│ ├── dependency_links.txt
│ ├── entry_points.txt
│ ├── native_libs.txt
│ ├── not-zip-safe
│ ├── requires.txt
│ └── top_level.txt
├── tvm
│ ├── __init__.py
│ ├── _ffi
│ ├── arith
│ ├── auto_scheduler
│ ├── autotvm
│ ├── configs
│ ├── contrib
│ ├── driver
│ ├── error.py
│ ├── exec
│ ├── generic.py
│ ├── ir
│ ├── libtvm.so
│ ├── libtvm_runtime.so
│ ├── meta_schedule
│ ├── micro
│ ├── parser
│ ├── relay
│ ├── rpc
│ ├── runtime
│ ├── script
│ ├── support.py
│ ├── target
│ ├── te
│ ├── testing
│ ├── tir
│ ├── topi
│ └── utils
└── tvm-0.9.dev1754+g2a71c380e-py3.8-linux-x86_64.egg
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Codle commented on a diff in pull request #12000: [Fix] fix python setup.py file bug
Posted by GitBox <gi...@apache.org>.
Codle commented on code in PR #12000:
URL: https://github.com/apache/tvm/pull/12000#discussion_r915118964
##########
python/setup.py:
##########
@@ -194,7 +194,15 @@ def is_pure(self):
curr_path = os.path.dirname(os.path.abspath(os.path.expanduser(__file__)))
for i, path in enumerate(LIB_LIST):
LIB_LIST[i] = os.path.relpath(path, curr_path)
- setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_LIST)]}
+ LIB_DATA_FILES = []
+ if os.path.isfile(path):
+ LIB_DATA_FILES.append(os.path.relpath(path, curr_path))
+ if os.path.isdir(path):
+ for root, dirs, files in os.walk(path):
+ for filename in files:
+ file_path = os.path.join(root, filename)
+ LIB_DATA_FILES.append(os.path.relpath(file_path, curr_path))
+ setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_DATA_FILES)]}
Review Comment:
the unzipped .egg file:
```
.
├── EGG-INFO
│ ├── PKG-INFO
│ ├── SOURCES.txt
│ ├── dependency_links.txt
│ ├── entry_points.txt
│ ├── native_libs.txt
│ ├── not-zip-safe
│ ├── requires.txt
│ └── top_level.txt
├── tvm
│ ├── __init__.py
│ ├── _ffi
│ ├── arith
│ ├── auto_scheduler
│ ├── autotvm
│ ├── configs
│ ├── contrib
│ ├── driver
│ ├── error.py
│ ├── exec
│ ├── generic.py
│ ├── ir
│ ├── libtvm.so
│ ├── libtvm_runtime.so
│ ├── meta_schedule
│ ├── micro
│ ├── microtvm_template_projects
│ ├── parser
│ ├── relay
│ ├── rpc
│ ├── runtime
│ ├── script
│ ├── standalone_crt
│ ├── support.py
│ ├── target
│ ├── te
│ ├── testing
│ ├── tir
│ ├── topi
│ └── utils
└── tvm-0.9.dev1754+g2a71c380e-py3.8-linux-x86_64.egg
```
the `standalone_crt` folder:
```
(base) ➜ tree standalone_crt -L 2
standalone_crt
├── Makefile
├── host
│ └── Makefile
├── include
│ ├── checksum.h
│ ├── dlpack
│ ├── dmlc
│ └── tvm
├── src
│ ├── runtime
│ └── support
└── template
├── crt_config-template.h
└── host
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] leandron commented on a diff in pull request #12000: [Fix] fix python setup.py file bug
Posted by GitBox <gi...@apache.org>.
leandron commented on code in PR #12000:
URL: https://github.com/apache/tvm/pull/12000#discussion_r915199500
##########
python/setup.py:
##########
@@ -194,7 +194,15 @@ def is_pure(self):
curr_path = os.path.dirname(os.path.abspath(os.path.expanduser(__file__)))
for i, path in enumerate(LIB_LIST):
LIB_LIST[i] = os.path.relpath(path, curr_path)
- setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_LIST)]}
+ LIB_DATA_FILES = []
+ if os.path.isfile(path):
+ LIB_DATA_FILES.append(os.path.relpath(path, curr_path))
+ if os.path.isdir(path):
+ for root, dirs, files in os.walk(path):
+ for filename in files:
+ file_path = os.path.join(root, filename)
+ LIB_DATA_FILES.append(os.path.relpath(file_path, curr_path))
+ setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_DATA_FILES)]}
Review Comment:
That looks great! Thanks @Codle!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Codle commented on a diff in pull request #12000: [Fix] fix python setup.py file bug
Posted by GitBox <gi...@apache.org>.
Codle commented on code in PR #12000:
URL: https://github.com/apache/tvm/pull/12000#discussion_r913208500
##########
python/setup.py:
##########
@@ -194,7 +194,15 @@ def is_pure(self):
curr_path = os.path.dirname(os.path.abspath(os.path.expanduser(__file__)))
for i, path in enumerate(LIB_LIST):
LIB_LIST[i] = os.path.relpath(path, curr_path)
- setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_LIST)]}
+ LIB_DATA_FILES = []
+ if os.path.isfile(path):
+ LIB_DATA_FILES.append(os.path.relpath(path, curr_path))
+ if os.path.isdir(path):
+ for root, dirs, files in os.walk(path):
+ for filename in files:
+ file_path = os.path.join(root, filename)
+ LIB_DATA_FILES.append(os.path.relpath(file_path, curr_path))
+ setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_DATA_FILES)]}
Review Comment:
I have made some change for this, please help me do a double check. I found the `data_files` filed is valid for both `bdist_wheel` and other actions, so I think it was no need to create a conditional branch.
I have tested `python setup.py bdist_wheel` and `python setup.py install`, and they created the same library files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Codle commented on a diff in pull request #12000: [Fix] fix python setup.py file bug
Posted by GitBox <gi...@apache.org>.
Codle commented on code in PR #12000:
URL: https://github.com/apache/tvm/pull/12000#discussion_r914094183
##########
python/setup.py:
##########
@@ -194,7 +194,15 @@ def is_pure(self):
curr_path = os.path.dirname(os.path.abspath(os.path.expanduser(__file__)))
for i, path in enumerate(LIB_LIST):
LIB_LIST[i] = os.path.relpath(path, curr_path)
- setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_LIST)]}
+ LIB_DATA_FILES = []
+ if os.path.isfile(path):
+ LIB_DATA_FILES.append(os.path.relpath(path, curr_path))
+ if os.path.isdir(path):
+ for root, dirs, files in os.walk(path):
+ for filename in files:
+ file_path = os.path.join(root, filename)
+ LIB_DATA_FILES.append(os.path.relpath(file_path, curr_path))
+ setup_kwargs = {"include_package_data": True, "data_files": [("tvm", LIB_DATA_FILES)]}
Review Comment:
Yes, you are right, the files are kept flat in the package. I try to find out the best practice of packing non-package file in setup.py and this PR gives my idea: https://github.com/pypa/sampleproject/issues/30
While include_package_data=True, the setup program will run commands in MANIFEST.in, so it's less necessary to use data_files. This step is explained in [setuptools/docs/references/keywords.rst](https://github.com/pypa/setuptools/blob/main/docs/references/keywords.rst)
> include_package_data: If set to True, this tells setuptools to automatically include any data files it finds inside your package directories that are specified by your MANIFEST.in file. For more information, see the section on [:ref:`Including Data Files`](https://github.com/pypa/setuptools/blob/0bd847c4aea839b3a75f4e3f09a36526453bd267/docs/references/keywords.rst#id23).
In the first PR: https://github.com/apache/tvm/pull/380 for data_files field, I found the include_package_data was set to False in wheel packing. So the detail of include_package_data may be ignored by the commiter.
Here is the result of unzipping .egg / .whl file, `configs` dir not be flatten to the json files.
.whl
```
dist
├── tvm
│ ├── __init__.py
│ ├── _ffi
│ ├── arith
│ ├── auto_scheduler
│ ├── autotvm
│ ├── configs
│ ├── contrib
│ ├── driver
│ ├── error.py
│ ├── exec
│ ├── generic.py
│ ├── ir
│ ├── libtvm.so
│ ├── libtvm_runtime.so
│ ├── meta_schedule
│ ├── micro
│ ├── parser
│ ├── relay
│ ├── rpc
│ ├── runtime
│ ├── script
│ ├── support.py
│ ├── target
│ ├── te
│ ├── testing
│ ├── tir
│ ├── topi
│ └── utils
├── tvm-0.9.dev1753+g6236f2cb6-cp38-cp38-linux_x86_64.whl
└── tvm-0.9.dev1753+g6236f2cb6.dist-info
├── METADATA
├── RECORD
├── WHEEL
├── entry_points.txt
└── top_level.txt
```
.egg
```
dist
├── EGG-INFO
│ ├── PKG-INFO
│ ├── SOURCES.txt
│ ├── dependency_links.txt
│ ├── entry_points.txt
│ ├── native_libs.txt
│ ├── not-zip-safe
│ ├── requires.txt
│ └── top_level.txt
├── tvm
│ ├── __init__.py
│ ├── _ffi
│ ├── arith
│ ├── auto_scheduler
│ ├── autotvm
│ ├── configs
│ ├── contrib
│ ├── driver
│ ├── error.py
│ ├── exec
│ ├── generic.py
│ ├── ir
│ ├── libtvm.so
│ ├── libtvm_runtime.so
│ ├── meta_schedule
│ ├── micro
│ ├── parser
│ ├── relay
│ ├── rpc
│ ├── runtime
│ ├── script
│ ├── support.py
│ ├── target
│ ├── te
│ ├── testing
│ ├── tir
│ ├── topi
│ └── utils
└── tvm-0.9.dev1754+g2a71c380e-py3.8-linux-x86_64.egg
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org