You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@mxnet.apache.org by GitBox <gi...@apache.org> on 2020/09/28 02:40:20 UTC
[GitHub] [incubator-mxnet] xxxpsyduck opened a new issue #19239: mxnet 1.8 build python package issue
xxxpsyduck opened a new issue #19239:
URL: https://github.com/apache/incubator-mxnet/issues/19239
## Description
I followed exactly the build steps in `runtime_functions.sh` but got an error `OSError: libonnx_proto.so: cannot open shared object file: No such file or directory` when trying to import mxnet.
### Error Message
```
>>> import mxnet
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/indetail/.local/lib/python3.6/site-packages/mxnet/__init__.py", line 23, in <module>
from .context import Context, current_context, cpu, gpu, cpu_pinned
File "/home/indetail/.local/lib/python3.6/site-packages/mxnet/context.py", line 23, in <module>
from .base import classproperty, with_metaclass, _MXClassPropertyMetaClass
File "/home/indetail/.local/lib/python3.6/site-packages/mxnet/base.py", line 351, in <module>
_LIB = _load_lib()
File "/home/indetail/.local/lib/python3.6/site-packages/mxnet/base.py", line 342, in _load_lib
lib = ctypes.CDLL(lib_path[0], ctypes.RTLD_LOCAL)
File "/usr/lib/python3.6/ctypes/__init__.py", line 348, in __init__
self._handle = _dlopen(self._name, mode)
OSError: libonnx_proto.so: cannot open shared object file: No such file or directory
```
## To Reproduce
### Steps to reproduce
1. build mxnet from v1.8.x branch step by step following [https://github.com/apache/incubator-mxnet/blob/v1.8.x/ci/docker/runtime_functions.sh#L705](https://github.com/apache/incubator-mxnet/blob/v1.8.x/ci/docker/runtime_functions.sh#L705)
2. build python wheel step by step following [https://github.com/apache/incubator-mxnet/blob/v1.8.x/ci/docker/runtime_functions.sh#L113](https://github.com/apache/incubator-mxnet/blob/v1.8.x/ci/docker/runtime_functions.sh#L113)
3. Install mxnet by running the below command in mxnet/python/dist:
```
python3 -m pip install mxnet-1.8.0-py3-none-any.whl
```
4. Import mxnet and the error occurs
## What have you tried to solve it?
1. Searching for `libonnx_proto.so` from released 1.7 package but non-exist
2. All the .so needed for the build is already in `mxnet/lib` folder
## Environment
We recommend using our script for collecting the diagnositc information. Run the following command and paste the outputs below:
```
----------Python Info----------
Version : 3.6.9
Compiler : GCC 8.4.0
Build : ('default', 'Jul 17 2020 12:50:27')
Arch : ('64bit', 'ELF')
------------Pip Info-----------
Version : 20.2.3
Directory : /home/indetail/.local/lib/python3.6/site-packages/pip
----------MXNet Info-----------
Traceback (most recent call last):
File "<stdin>", line 95, in check_mxnet
File "/home/indetail/.local/lib/python3.6/site-packages/mxnet/__init__.py", line 23, in <module>
from .context import Context, current_context, cpu, gpu, cpu_pinned
File "/home/indetail/.local/lib/python3.6/site-packages/mxnet/context.py", line 23, in <module>
from .base import classproperty, with_metaclass, _MXClassPropertyMetaClass
File "/home/indetail/.local/lib/python3.6/site-packages/mxnet/base.py", line 351, in <module>
_LIB = _load_lib()
File "/home/indetail/.local/lib/python3.6/site-packages/mxnet/base.py", line 342, in _load_lib
lib = ctypes.CDLL(lib_path[0], ctypes.RTLD_LOCAL)
File "/usr/lib/python3.6/ctypes/__init__.py", line 348, in __init__
self._handle = _dlopen(self._name, mode)
OSError: libonnx_proto.so: cannot open shared object file: No such file or directory
----------System Info----------
Platform : Linux-5.4.0-48-generic-x86_64-with-Ubuntu-18.04-bionic
system : Linux
node : morhn1
release : 5.4.0-48-generic
version : #52~18.04.1-Ubuntu SMP Thu Sep 10 12:50:22 UTC 2020
----------Hardware Info----------
machine : x86_64
processor : x86_64
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 12
On-line CPU(s) list: 0-11
Thread(s) per core: 2
Core(s) per socket: 6
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 158
Model name: Intel(R) Core(TM) i7-8086K CPU @ 4.00GHz
Stepping: 10
CPU MHz: 4507.269
CPU max MHz: 5000.0000
CPU min MHz: 800.0000
BogoMIPS: 7999.96
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 12288K
NUMA node0 CPU(s): 0-11
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d
----------Network Test----------
Setting timeout: 10
Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0284 sec, LOAD: 1.2654 sec.
Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.3937 sec, LOAD: 0.4359 sec.
Error open Gluon Tutorial(cn): https://zh.gluon.ai, <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)>, DNS finished in 0.09618353843688965 sec.
Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.1073 sec, LOAD: 0.7933 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0671 sec, LOAD: 0.8329 sec.
Error open Conda: https://repo.continuum.io/pkgs/free/, HTTP Error 403: Forbidden, DNS finished in 0.031211137771606445 sec.
----------Environment----------
MXNET_LIBRARY_PATH="/home/indetail/work/build/libmxnet.so"
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] szha commented on issue #19239: mxnet-tensorrt 1.8 build python package issue
Posted by GitBox <gi...@apache.org>.
szha commented on issue #19239:
URL: https://github.com/apache/incubator-mxnet/issues/19239#issuecomment-699770316
cc @Kh4L
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] Kh4L commented on issue #19239: mxnet-tensorrt 1.8 build python package issue
Posted by GitBox <gi...@apache.org>.
Kh4L commented on issue #19239:
URL: https://github.com/apache/incubator-mxnet/issues/19239#issuecomment-707891636
@xxxpsyduck were you able to solve the issue?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] xxxpsyduck closed issue #19239: mxnet-tensorrt 1.8 build python package issue
Posted by GitBox <gi...@apache.org>.
xxxpsyduck closed issue #19239:
URL: https://github.com/apache/incubator-mxnet/issues/19239
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] szha commented on issue #19239: mxnet 1.8 build python package issue
Posted by GitBox <gi...@apache.org>.
szha commented on issue #19239:
URL: https://github.com/apache/incubator-mxnet/issues/19239#issuecomment-699738516
That missing file is from ONNX and I don't think we ever required linking to it. Since runtime_functions.sh have many different build settings, could you list the exact commands/function you used?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] xxxpsyduck commented on issue #19239: mxnet-tensorrt 1.8 build python package issue
Posted by GitBox <gi...@apache.org>.
xxxpsyduck commented on issue #19239:
URL: https://github.com/apache/incubator-mxnet/issues/19239#issuecomment-708157355
@Kh4L Not yet. I don't know if I did anything wrong. Steps I did:
```sh
git clone --recursive https://github.com/apache/incubator-mxnet.git mxnet
cd mxnet
git checkout v1.8.x
```
```sh
./ci/build.py -p ubuntu_gpu_tensorrt
docker run --gpus all -v `pwd`:/work/mxnet -it $ubuntu_gpu_tensorrt_image_id
```
```sh
./ci/docker/runtime_functions.sh build_ubuntu_gpu_tensorrt
./ci/docker/runtime_functions.sh build_wheel
cd python/dist
python3 -m pip install mxnet-1.8.0-py3-none-any.whl
```
every step success without errors. But when import:
```sh
python3
Python 3.6.9 (default, Jul 17 2020, 12:50:27)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import mxnet
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.6/dist-packages/mxnet/__init__.py", line 23, in <module>
from .context import Context, current_context, cpu, gpu, cpu_pinned
File "/usr/local/lib/python3.6/dist-packages/mxnet/context.py", line 23, in <module>
from .base import classproperty, with_metaclass, _MXClassPropertyMetaClass
File "/usr/local/lib/python3.6/dist-packages/mxnet/base.py", line 351, in <module>
_LIB = _load_lib()
File "/usr/local/lib/python3.6/dist-packages/mxnet/base.py", line 342, in _load_lib
lib = ctypes.CDLL(lib_path[0], ctypes.RTLD_LOCAL)
File "/usr/lib/python3.6/ctypes/__init__.py", line 348, in __init__
self._handle = _dlopen(self._name, mode)
OSError: libonnx_proto.so: cannot open shared object file: No such file or directory
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] leezu commented on issue #19239: mxnet-tensorrt 1.8 build python package issue
Posted by GitBox <gi...@apache.org>.
leezu commented on issue #19239:
URL: https://github.com/apache/incubator-mxnet/issues/19239#issuecomment-700156637
@xxxpsyduck you may just need to add a statement to copy the library in https://github.com/apache/incubator-mxnet/blob/8e9877b1cc33d271508075c336f8bd34a28e977b/tools/pip/setup.py#L149-L159
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] xxxpsyduck commented on issue #19239: mxnet 1.8 build python package issue
Posted by GitBox <gi...@apache.org>.
xxxpsyduck commented on issue #19239:
URL: https://github.com/apache/incubator-mxnet/issues/19239#issuecomment-699740475
@szha Yes. I listed them in "Step to reproduce"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] xxxpsyduck commented on issue #19239: mxnet-tensorrt 1.8 build python package issue
Posted by GitBox <gi...@apache.org>.
xxxpsyduck commented on issue #19239:
URL: https://github.com/apache/incubator-mxnet/issues/19239#issuecomment-708157911
@Kh4L Can you try these steps above with v1.8.x? Is there any changes should be made?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] xxxpsyduck edited a comment on issue #19239: mxnet-tensorrt 1.8 build python package issue
Posted by GitBox <gi...@apache.org>.
xxxpsyduck edited a comment on issue #19239:
URL: https://github.com/apache/incubator-mxnet/issues/19239#issuecomment-700406953
@leezu I added the following lines to `tools/pip/setup.py` and `python/setup.py`:
```
if platform.system() == 'Linux':
libdir, mxdir = os.path.dirname(LIB_PATH[0]), os.path.join(CURRENT_DIR, 'mxnet')
if os.path.exists(os.path.join(libdir, 'libonnx_proto.so')):
shutil.copy(os.path.join(libdir, 'libonnx_proto.so'), mxdir)
package_data['mxnet'].append('mxnet/libonnx_proto.so')
```
The error still occurs.
update: I checked and the file is missing from `libdir` so I moved the required file to the `libdir` folder. Built successfully but same error happens when I try to import
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] xxxpsyduck commented on issue #19239: mxnet-tensorrt 1.8 build python package issue
Posted by GitBox <gi...@apache.org>.
xxxpsyduck commented on issue #19239:
URL: https://github.com/apache/incubator-mxnet/issues/19239#issuecomment-700406953
@leezu I added the following lines to `tools/pip/setup.py` and `python/setup.py`:
```
if platform.system() == 'Linux':
libdir, mxdir = os.path.dirname(LIB_PATH[0]), os.path.join(CURRENT_DIR, 'mxnet')
if os.path.exists(os.path.join(libdir, 'libonnx_proto.so')):
shutil.copy(os.path.join(libdir, 'libonnx_proto.so'), mxdir)
package_data['mxnet'].append('mxnet/libonnx_proto.so')
```
The error still occurs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org