You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2020/04/14 16:05:48 UTC
[GitHub] [incubator-mxnet] RuRo opened a new pull request #18054: Bump ONNX
version to 1.5.0
RuRo opened a new pull request #18054: Bump ONNX version to 1.5.0
URL: https://github.com/apache/incubator-mxnet/pull/18054
## Description ##
This is a WIP pull request for increasing the supported `ONNX` version from `1.3.0` to `1.5.0`.
This change is motivated by the fact, that `ONNX v1.5.0` (opset 10) supports new operators, that should allow/simplify the implementation of some `mxnet -> ONNX` conversions, that are currently not implemented.
List of potentially useful operators, which were not available in `1.3.0`, but are available in `1.5.0`:
- `Compress`
- `ConstantOfShape`
- `EyeLike`
- `NonMaxSuppression`
- `NonZero`
- `OneHot`
- `Resize`
- `RoiAlign`
- `Scatter`
- `Where`
There are more operators, but these are the important ones, IMO.
This PR doesn't implement any of the above operators, but just bumps the `ONNX` version in CI and fixes the tests for the newer version.
## Changes ##
1) The `implement onnx translation helpers` commit just implements some helper functions, that I've used in all my PRs for ONNX translations
2) Tests for trigonometric functions `sin`/`cos`, `acos`/`asin`/`atan` now have `[^h]` appended, so that it doesn't accidentally match with **new** ONNX test for the Hyperbolic trigonometric functions, the conversion for which is currently not implemented. (`test_sinh` etc)
3) Test `test_matmul` changed to `test_matmul_`, so that it doesn't accidentally match with the **new** ONNX test for `matmulinteger`, the conversion for which is currently not implemented.
4) Test `test_cast` changed to `test_cast((?!STRING).)*$`, so that it doesn't accidentally match with the **new** ONNX test for casting to/from strings. `((?!STRING).)*$` matches any text until the end of string, as long as it doesn't contain the word `STRING`.
5) Updated the implementation for exporting `topk` and `slice_axis` operators, since they changed their API in the new opset version.
6) Tests for `test_slice` and `test_topk` were disabled, since the new versions of the operators moved some of the attributes to inputs (`k` for `TopK`, and `starts`, `ends`, `axes` and `steps` for `Slice`). Since mxnet (afaik) doesn't fully support dynamic shapes yet, importing the newer operators is currently not possible. Importing `TopK` and `Slice` operators, exported with the older onnx version and exporting to the newer opset still works, so this shouldn't be a breaking change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] mxnet-bot commented on issue #18054: Bump ONNX
version to 1.5.0
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on issue #18054: Bump ONNX version to 1.5.0
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-614254488
Jenkins CI successfully triggered : [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] mxnet-bot commented on issue #18054: Bump ONNX
version to 1.5.0
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on issue #18054: Bump ONNX version to 1.5.0
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-615297650
Jenkins CI successfully triggered : [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] QueensGambit commented on issue #18054: Bump ONNX
version to 1.5.0
Posted by GitBox <gi...@apache.org>.
QueensGambit commented on issue #18054: Bump ONNX version to 1.5.0
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-615876976
Thank you @RuRo for this PR.
I welcome the support of a newer ONNX version.
Should we instead aim for supporting the latest ONNX release **1.6.0** (Sep 28, 2019) instead?
Will this PR also allow dynamic shape export to ONNX as it is already available in Pytorch?
* https://pytorch.org/docs/stable/onnx.html
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] RuRo removed a comment on issue #18054: Bump ONNX
version to 1.5.0
Posted by GitBox <gi...@apache.org>.
RuRo removed a comment on issue #18054: Bump ONNX version to 1.5.0
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-615884655
@mxnet-bot run ci [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] QueensGambit commented on issue #18054: Bump ONNX
version to 1.5.0
Posted by GitBox <gi...@apache.org>.
QueensGambit commented on issue #18054: Bump ONNX version to 1.5.0
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-616215616
> I personally only want to export from MxNet to ONNX and don't care at all about importing from ONNX to MxNet, however I am not sure, if a PR which drops the support for importing the Pad operator from ONNX would ever get accepted. Even in this PR, I had to drop importing the newer opset versions of Slice and TopK and I am not sure, if this will get accepted without any complaints.
I'm also interested in MXNet to ONNX export to allow native TensorRT and possible onnxjs inference but I agree that this shouldn't break the ONNX to MXNet import.
> This PR doesn't add any new functionality and I think it should stay that way. This PR will just bump the ONNX version and any new operators and functionality, that can be implemented with the newer version will be in their separate PRs.
I asked for a ONNX 1.6.0 support to make use of the bug fixes from 1.6.0 and to allow export with a dynamic batch size.
However you're right that this PR shouldn't add to many things at once and that dynamic inputs could be added later.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] RuRo commented on issue #18054: Bump ONNX version
to 1.5.0
Posted by GitBox <gi...@apache.org>.
RuRo commented on issue #18054: Bump ONNX version to 1.5.0
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-615167758
@mxnet-bot run ci [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] mxnet-bot commented on issue #18054: Bump ONNX
version to 1.5.0
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on issue #18054: Bump ONNX version to 1.5.0
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-615884669
Jenkins CI successfully triggered : [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] RuRo removed a comment on issue #18054: Bump ONNX
version to 1.5.0
Posted by GitBox <gi...@apache.org>.
RuRo removed a comment on issue #18054: Bump ONNX version to 1.5.0
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-615167758
@mxnet-bot run ci [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] RuRo removed a comment on issue #18054: Bump ONNX
version to 1.5.0
Posted by GitBox <gi...@apache.org>.
RuRo removed a comment on issue #18054: Bump ONNX version to 1.5.0
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-614254442
@mxnet-bot run ci [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] RuRo commented on issue #18054: Bump ONNX version
to 1.5.0
Posted by GitBox <gi...@apache.org>.
RuRo commented on issue #18054: Bump ONNX version to 1.5.0
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-615423502
@mxnet-bot run ci [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] RuRo commented on issue #18054: Bump ONNX version
to 1.5.0
Posted by GitBox <gi...@apache.org>.
RuRo commented on issue #18054: Bump ONNX version to 1.5.0
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-614254442
@mxnet-bot run ci [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] mxnet-bot commented on issue #18054: Bump ONNX
version to 1.5.0
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on issue #18054: Bump ONNX version to 1.5.0
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-615167808
Jenkins CI successfully triggered : [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] RuRo commented on issue #18054: Bump ONNX version
to 1.5.0
Posted by GitBox <gi...@apache.org>.
RuRo commented on issue #18054: Bump ONNX version to 1.5.0
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-615884655
@mxnet-bot run ci [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] mxnet-bot commented on issue #18054: Bump ONNX
version to 1.5.0
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on issue #18054: Bump ONNX version to 1.5.0
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-613532305
Hey @RuRo , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:
- To trigger all jobs: @mxnet-bot run ci [all]
- To trigger specific jobs: @mxnet-bot run ci [job1, job2]
***
**CI supported jobs**: [website, miscellaneous, unix-cpu, unix-gpu, sanity, windows-cpu, windows-gpu, clang, edge, centos-cpu, centos-gpu]
***
_Note_:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] RuRo commented on pull request #18054: Fix tests for ONNX version 1.5.0 bump
Posted by GitBox <gi...@apache.org>.
RuRo commented on pull request #18054:
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-634959644
@mxnet-bot run ci [centos-cpu, miscellaneous, unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] szha commented on pull request #18054: Fix tests for ONNX version 1.5.0 bump
Posted by GitBox <gi...@apache.org>.
szha commented on pull request #18054:
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-634961954
Thanks @RuRo. I already bumped up the ONNX version in a previous PR. See the CI setup: https://github.com/apache/incubator-mxnet/blob/master/ci/docker/install/requirements#L26-L27
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] szha commented on pull request #18054: Fix tests for ONNX version 1.5.0 bump
Posted by GitBox <gi...@apache.org>.
szha commented on pull request #18054:
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-639778310
thanks for the fix
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] RuRo commented on issue #18054: Bump ONNX version
to 1.5.0
Posted by GitBox <gi...@apache.org>.
RuRo commented on issue #18054: Bump ONNX version to 1.5.0
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-615297569
@mxnet-bot run ci [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] mxnet-bot commented on issue #18054: Bump ONNX
version to 1.5.0
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on issue #18054: Bump ONNX version to 1.5.0
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-615423570
Jenkins CI successfully triggered : [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] RuRo removed a comment on issue #18054: Bump ONNX
version to 1.5.0
Posted by GitBox <gi...@apache.org>.
RuRo removed a comment on issue #18054: Bump ONNX version to 1.5.0
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-615297569
@mxnet-bot run ci [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] RuRo commented on issue #18054: Bump ONNX version
to 1.5.0
Posted by GitBox <gi...@apache.org>.
RuRo commented on issue #18054: Bump ONNX version to 1.5.0
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-615884610
@QueensGambit
> Should we instead aim for supporting the latest ONNX release 1.6.0 (Sep 28, 2019) instead?
I chose 1.5.0 as that was the smallest version, that supported the operators I needed. Are there any particular operators/features you need from 1.6.0? I've just tried upgrading to 1.6.0 and there are quite a lot of tests that are failing with 1.6.0, so it wouldn't be a seamless upgrade.
The main issue with upgrading ONNX is that in newer opset versions, ONNX started moving some of the static "attributes" to be dynamic inputs. (For example, the `Pad` operator in 1.6.0 accepts the pad sizes as an actual `int64` tensor instead of an attribute, that is just a static list of ints). It's pretty easy to **export** mxnet operators to such a format (just create a constant tensor), but it's AFAIK not currently possible to **import** such an operator, since `mx.sym.pad` expects the pad sizes to be known in advance.
I think, there may be some developments in the new `numpy` compatible API, that could allow for such operators to be imported, but I haven't looked into it yet.
I personally only want to export from MxNet to ONNX and don't care at all about importing from ONNX to MxNet, however I am not sure, if a PR which drops the support for importing the `Pad` operator from ONNX would ever get accepted. Even in this PR, I had to drop importing the newer opset versions of `Slice` and `TopK` and I am not sure, if this will get accepted without any complaints.
> Will this PR also allow dynamic shape export to ONNX as it is already available in Pytorch?
This PR doesn't add any new functionality and I think it should stay that way. This PR will just bump the ONNX version and any new operators and functionality, that can be implemented with the newer version will be in their separate PRs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] RuRo removed a comment on issue #18054: Bump ONNX
version to 1.5.0
Posted by GitBox <gi...@apache.org>.
RuRo removed a comment on issue #18054: Bump ONNX version to 1.5.0
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-615423502
@mxnet-bot run ci [unix-gpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-mxnet] RuRo commented on pull request #18054: Fix tests for ONNX version 1.5.0 bump
Posted by GitBox <gi...@apache.org>.
RuRo commented on pull request #18054:
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-638095415
@szha I am not too familiar with `pytest`/`unittest`. I've tried making a quick fix by adding an explicit `exit` call with an appropriate exit status, based on whether the tests are failing, but this doesn't work.
The problem is that the `tests/python/unittest/onnx/backend_test.py` file is scanned by pytest, when some other test are running. This means that the `exit` call is triggered during the test collection phase, which `pytest` treats as an error, no matter, what the exit status is (this behavior is probably correct).
<details>
```python
[2020-06-02T13:52:46.852Z] ==================================== ERRORS ====================================
[2020-06-02T13:52:46.852Z] _________ ERROR collecting tests/python/unittest/onnx/backend_test.py __________
[2020-06-02T13:52:46.852Z] tests/python/unittest/onnx/backend_test.py:96: in <module>
[2020-06-02T13:52:46.852Z] exit(any(res.failures for res in results))
[2020-06-02T13:52:46.852Z] /usr/lib/python3.6/_sitebuiltins.py:26: in __call__
[2020-06-02T13:52:46.852Z] raise SystemExit(code)
[2020-06-02T13:52:46.852Z] E SystemExit: False
```
</details>
There are also some failing/flaky ONNX tests on `centos-cpu` and `windows-cpu/gpu`, which are **not** failing on `unix-cpu`.
The `windows` tests are apparently (?) running in a dirty environment and some tests are failing, because the tests need to download some onnx model files and apparently the target directories already exist.
<details>
```python
[2020-06-02T14:22:01.951Z] ======================================================================
[2020-06-02T14:22:01.951Z] ERROR: test_bvlc_alexnet_cpu (backend_test.OnnxBackendRealModelTest)
[2020-06-02T14:22:01.951Z] ----------------------------------------------------------------------
[2020-06-02T14:22:01.951Z] Traceback (most recent call last):
[2020-06-02T14:22:01.951Z] File "C:\Python37\lib\site-packages\onnx\backend\test\runner\__init__.py", line 248, in device_test_func
[2020-06-02T14:22:01.951Z] return test_func(*args, device=device, **kwargs)
[2020-06-02T14:22:01.951Z] File "C:\Python37\lib\site-packages\onnx\backend\test\runner\__init__.py", line 268, in run
[2020-06-02T14:22:01.951Z] model_dir = self.prepare_model_data(model_test)
[2020-06-02T14:22:01.951Z] File "C:\Python37\lib\site-packages\onnx\backend\test\runner\__init__.py", line 218, in prepare_model_data
[2020-06-02T14:22:01.951Z] os.makedirs(model_dir)
[2020-06-02T14:22:01.951Z] File "C:\Python37\lib\os.py", line 221, in makedirs
[2020-06-02T14:22:01.951Z] mkdir(name, mode)
[2020-06-02T14:22:01.951Z] FileExistsError: [WinError 183] Cannot create a file when that file already exists: 'C:\\Windows\\system32\\config\\systemprofile\\.onnx\\models\\bvlc_alexnet'
[2020-06-02T14:22:01.951Z]
```
</details>
The `centos-cpu` tests are having some issues with precision and maybe partial model downloads. I am really not sure.
<details>
Partial downloads (maybe?).
```python
[2020-06-01T20:41:37.094Z] ======================================================================
[2020-06-01T20:41:37.094Z] ERROR: test_resnet50_cpu (backend_test.OnnxBackendRealModelTest)
[2020-06-01T20:41:37.094Z] ----------------------------------------------------------------------
[2020-06-01T20:41:37.094Z] Traceback (most recent call last):
[2020-06-01T20:41:37.094Z] File "/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-06-01T20:41:37.094Z] return test_func(*args, device=device, **kwargs)
[2020-06-01T20:41:37.094Z] File "/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/onnx/backend/test/runner/__init__.py", line 272, in run
[2020-06-01T20:41:37.094Z] model = onnx.load(model_pb_path)
[2020-06-01T20:41:37.094Z] File "/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/onnx/__init__.py", line 114, in load_model
[2020-06-01T20:41:37.094Z] model = load_model_from_string(s, format=format)
[2020-06-01T20:41:37.094Z] File "/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/onnx/__init__.py", line 151, in load_model_from_string
[2020-06-01T20:41:37.094Z] return _deserialize(s, ModelProto())
[2020-06-01T20:41:37.094Z] File "/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/onnx/__init__.py", line 94, in _deserialize
[2020-06-01T20:41:37.094Z] decoded = cast(Optional[int], proto.ParseFromString(s))
[2020-06-01T20:41:37.094Z] google.protobuf.message.DecodeError: Error parsing message
[2020-06-01T20:41:37.094Z]
[2020-06-01T20:41:37.094Z] ======================================================================
```
Precision problems. It's possible, the root cause of this error is the same as the `google.protobuf.message.DecodeError` issue, since they seem to occur together.
```python
[2020-06-01T20:41:37.094Z] ======================================================================
[2020-06-01T20:41:37.094Z] FAIL: test_vgg19_cpu (backend_test.OnnxBackendRealModelTest)
[2020-06-01T20:41:37.094Z] ----------------------------------------------------------------------
[2020-06-01T20:41:37.094Z] Traceback (most recent call last):
[2020-06-01T20:41:37.094Z] File "/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-06-01T20:41:37.094Z] return test_func(*args, device=device, **kwargs)
[2020-06-01T20:41:37.094Z] File "/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/onnx/backend/test/runner/__init__.py", line 290, in run
[2020-06-01T20:41:37.094Z] atol=model_test.atol)
[2020-06-01T20:41:37.094Z] File "/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/onnx/backend/test/runner/__init__.py", line 178, in assert_similar_outputs
[2020-06-01T20:41:37.094Z] atol=atol)
[2020-06-01T20:41:37.094Z] File "/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/numpy/testing/_private/utils.py", line 1533, in assert_allclose
[2020-06-01T20:41:37.094Z] verbose=verbose, header=header, equal_nan=equal_nan)
[2020-06-01T20:41:37.094Z] File "/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/numpy/testing/_private/utils.py", line 846, in assert_array_compare
[2020-06-01T20:41:37.094Z] raise AssertionError(msg)
[2020-06-01T20:41:37.094Z] AssertionError:
[2020-06-01T20:41:37.094Z] Not equal to tolerance rtol=0.001, atol=1e-07
[2020-06-01T20:41:37.094Z]
[2020-06-01T20:41:37.094Z] Mismatched elements: 997 / 1000 (99.7%)
[2020-06-01T20:41:37.094Z] Max absolute difference: 0.0330687
[2020-06-01T20:41:37.094Z] Max relative difference: 2.9896083
[2020-06-01T20:41:37.094Z] x: array([[2.939573e-04, 1.110041e-03, 4.796557e-04, 1.204622e-03,
[2020-06-01T20:41:37.094Z] 2.784455e-03, 4.492797e-03, 1.323598e-02, 1.580402e-04,
[2020-06-01T20:41:37.094Z] 1.594951e-04, 3.026387e-04, 3.349203e-04, 2.087024e-04,...
[2020-06-01T20:41:37.094Z] y: array([[7.276282e-04, 2.052591e-03, 7.338484e-04, 1.470099e-03,
[2020-06-01T20:41:37.094Z] 3.712713e-03, 3.797364e-03, 1.069483e-02, 3.067438e-04,
[2020-06-01T20:41:37.094Z] 3.405935e-04, 4.645915e-04, 3.827364e-04, 3.712076e-04,...
[2020-06-01T20:41:37.094Z]
[2020-06-01T20:41:37.094Z] ----------------------------------------------------------------------
```
</details>
The current tests (ONNX or otherwise) are quite a mess, IMO. I don't think it's feasible to fix everything in one PR (also, I don't really have too much time to spend on fixing the other issues). This PR was mainly focused on fixing the individual node tests broken by the ONNX version bump to 1.5.0 and I think, that the other issues should be resolved in a separate PR.
## TL;DR
I propose, that we merge this PR as it currently is and open separate issues for fixing the `unittest`/`pytest` integration and the `windows`/`centos` ONNX fails.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] RuRo commented on pull request #18054: Bump ONNX version to 1.5.0
Posted by GitBox <gi...@apache.org>.
RuRo commented on pull request #18054:
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-634623916
So I came back after my exams to find that the ONNX version was apparently bumped to 1.5.0 in #18025, but the tests weren't actually updated. I am not sure, why are the tests marked as passing even thought there is a ton of errors.
For example, [here](http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/master/1983/pipeline/376) ([raw](http://jenkins.mxnet-ci.amazon-ml.com/blue/rest/organizations/jenkins/pipelines/mxnet-validation/pipelines/unix-cpu/branches/master/runs/1983/nodes/376/log/?start=0)) you can see that there are a lot of ONNX tests failing, but the pipeline is still marked as green.
<details>
```
.s.s.sEsEs.s.sssssssssssssssss.s.s.s.s.s.s.s.s.s.s.s.s.s.sEsEs.s.sEsEs.sssss.sss.sssss.sssss.s.sssssssssss.s.s.s.s.s.sEs.s.s.s.s.s.s.s.s.s.s.sssssss.s.s.s.s.s.sss.sssssssssssssssssssssssssssssss.s.sEsEs.s.sss.s.s.s.s.s.s.s.s.sssssss.s.sssssssssssssssssss.s.s.s.s.sssss.s.s.s.sssssssssss.s.s.s.s.s.s.s.s.s.s.s.sssssssss.s.s.sssss.s.sssssssssssssssssssssss.s.s.sEs.s.s.sssssssssssssssssssssssssssssssss.s.s.s.s.s.sssssssss.s.s.sssss.s.sssssssssssssssssssssssssssssssssssssssssssssss.sss.s.sssssssssssss.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.sssssssssssssssssssssssssssssssssssssss.sss.s.sssss.s.sssssss.s.sEsEs.s.sEsssssEsEsEsssssss.sss.s.s.s.s.s.s.s.s.sssssss.s.s.sssssssssssss.s.s.s.s.s.s.s.s.s.sssssssssssssssssssssssssEs.s.s.s.s.s.s.s.sssssssssssssssssssss.s.sssss.sss.s.s.sssssssssssss.sssssssssssssssssssssssssss.s.s.s.s.sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss.s.s.s.s.s.s.s.s.s.s.s.sssssssssssssss.sssssssssss.s.s.s.sssssssssssssssssssssssssssssssss.sssssssss.s.sssssssssssssssssssssssss
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_acosh_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 63, in _convert_operator
[2020-05-27T04:58:58.762Z] raise NotImplementedError("Operator {} not implemented.".format(op_name))
[2020-05-27T04:58:58.762Z] NotImplementedError: Operator Acosh not implemented.
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_acosh_example_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 63, in _convert_operator
[2020-05-27T04:58:58.762Z] raise NotImplementedError("Operator {} not implemented.".format(op_name))
[2020-05-27T04:58:58.762Z] NotImplementedError: Operator Acosh not implemented.
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_asinh_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 63, in _convert_operator
[2020-05-27T04:58:58.762Z] raise NotImplementedError("Operator {} not implemented.".format(op_name))
[2020-05-27T04:58:58.762Z] NotImplementedError: Operator Asinh not implemented.
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_asinh_example_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 63, in _convert_operator
[2020-05-27T04:58:58.762Z] raise NotImplementedError("Operator {} not implemented.".format(op_name))
[2020-05-27T04:58:58.762Z] NotImplementedError: Operator Asinh not implemented.
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_atanh_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 63, in _convert_operator
[2020-05-27T04:58:58.762Z] raise NotImplementedError("Operator {} not implemented.".format(op_name))
[2020-05-27T04:58:58.762Z] NotImplementedError: Operator Atanh not implemented.
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_atanh_example_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 63, in _convert_operator
[2020-05-27T04:58:58.762Z] raise NotImplementedError("Operator {} not implemented.".format(op_name))
[2020-05-27T04:58:58.762Z] NotImplementedError: Operator Atanh not implemented.
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_cast_FLOAT_to_STRING_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 69, in _convert_operator
[2020-05-27T04:58:58.762Z] mxnet_sym = new_op(*inputs, **new_attrs)
[2020-05-27T04:58:58.762Z] File "<string>", line 65, in cast
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/_ctypes/symbol.py", line 132, in _symbol_creator
[2020-05-27T04:58:58.762Z] ctypes.byref(sym_handle)))
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/base.py", line 246, in check_call
[2020-05-27T04:58:58.762Z] raise get_last_ffi_error()
[2020-05-27T04:58:58.762Z] mxnet.base.MXNetError: MXNetError: Invalid Input: 'object', valid values are: {'bfloat16', 'bool', 'float16', 'float32', 'float64', 'int32', 'int64', 'int8', 'uint8'}, in operator Cast(name="", __profiler_scope__="<unk>:", dtype="object")
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_cosh_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 63, in _convert_operator
[2020-05-27T04:58:58.762Z] raise NotImplementedError("Operator {} not implemented.".format(op_name))
[2020-05-27T04:58:58.762Z] NotImplementedError: Operator Cosh not implemented.
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_cosh_example_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 63, in _convert_operator
[2020-05-27T04:58:58.762Z] raise NotImplementedError("Operator {} not implemented.".format(op_name))
[2020-05-27T04:58:58.762Z] NotImplementedError: Operator Cosh not implemented.
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_matmulinteger_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 63, in _convert_operator
[2020-05-27T04:58:58.762Z] raise NotImplementedError("Operator {} not implemented.".format(op_name))
[2020-05-27T04:58:58.762Z] NotImplementedError: Operator MatMulInteger not implemented.
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_sinh_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 63, in _convert_operator
[2020-05-27T04:58:58.762Z] raise NotImplementedError("Operator {} not implemented.".format(op_name))
[2020-05-27T04:58:58.762Z] NotImplementedError: Operator Sinh not implemented.
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_sinh_example_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 63, in _convert_operator
[2020-05-27T04:58:58.762Z] raise NotImplementedError("Operator {} not implemented.".format(op_name))
[2020-05-27T04:58:58.762Z] NotImplementedError: Operator Sinh not implemented.
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_slice_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 61, in _convert_operator
[2020-05-27T04:58:58.762Z] op_name, new_attrs, inputs = convert_map[op_name](attrs, inputs, self)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/_op_translations.py", line 511, in _slice
[2020-05-27T04:58:58.762Z] end = list(new_attrs.get('end'))
[2020-05-27T04:58:58.762Z] TypeError: 'NoneType' object is not iterable
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_slice_end_out_of_bounds_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 61, in _convert_operator
[2020-05-27T04:58:58.762Z] op_name, new_attrs, inputs = convert_map[op_name](attrs, inputs, self)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/_op_translations.py", line 511, in _slice
[2020-05-27T04:58:58.762Z] end = list(new_attrs.get('end'))
[2020-05-27T04:58:58.762Z] TypeError: 'NoneType' object is not iterable
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_slice_neg_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 61, in _convert_operator
[2020-05-27T04:58:58.762Z] op_name, new_attrs, inputs = convert_map[op_name](attrs, inputs, self)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/_op_translations.py", line 511, in _slice
[2020-05-27T04:58:58.762Z] end = list(new_attrs.get('end'))
[2020-05-27T04:58:58.762Z] TypeError: 'NoneType' object is not iterable
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_slice_neg_steps_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 61, in _convert_operator
[2020-05-27T04:58:58.762Z] op_name, new_attrs, inputs = convert_map[op_name](attrs, inputs, self)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/_op_translations.py", line 511, in _slice
[2020-05-27T04:58:58.762Z] end = list(new_attrs.get('end'))
[2020-05-27T04:58:58.762Z] TypeError: 'NoneType' object is not iterable
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_top_k_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 69, in _convert_operator
[2020-05-27T04:58:58.762Z] mxnet_sym = new_op(*inputs, **new_attrs)
[2020-05-27T04:58:58.762Z] File "<string>", line 106, in topk
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/_ctypes/symbol.py", line 132, in _symbol_creator
[2020-05-27T04:58:58.762Z] ctypes.byref(sym_handle)))
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/base.py", line 246, in check_call
[2020-05-27T04:58:58.762Z] raise get_last_ffi_error()
[2020-05-27T04:58:58.762Z] mxnet.base.MXNetError: MXNetError: Invalid Parameter format for axis expect int or None but value='<Symbol k>', in operator topk(name="", __profiler_scope__="<unk>:", axis="<Symbol k>", dtype="int64", ret_typ="both")
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Ran 1116 tests in 603.735s
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] FAILED (errors=17, skipped=853)
```
</details>
I am in the process of investigating, if my proposed fixes for the ONNX tests still work with the recent testing pipeline changes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] szha commented on pull request #18054: Fix tests for ONNX version 1.5.0 bump
Posted by GitBox <gi...@apache.org>.
szha commented on pull request #18054:
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-636378471
@RuRo thanks for catching it. I think the problem of CI for ONNX is in the way the backend_test.py is run. `python3 tests/python/unittest/onnx/backend_test.py`. We should change it to proper pytest instead
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] szha merged pull request #18054: Fix tests for ONNX version 1.5.0 bump
Posted by GitBox <gi...@apache.org>.
szha merged pull request #18054:
URL: https://github.com/apache/incubator-mxnet/pull/18054
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] RuRo commented on pull request #18054: Fix tests for ONNX version 1.5.0 bump
Posted by GitBox <gi...@apache.org>.
RuRo commented on pull request #18054:
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-634998080
@szha Did you check the CI logs, that I've attached in my previous comment?
<blockquote>
For example, [here](http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/master/1983/pipeline/376) ([raw](http://jenkins.mxnet-ci.amazon-ml.com/blue/rest/organizations/jenkins/pipelines/mxnet-validation/pipelines/unix-cpu/branches/master/runs/1983/nodes/376/log/?start=0)) you can see that there are a lot of ONNX tests failing, but the pipeline is still marked as green.
<details>
```
.s.s.sEsEs.s.sssssssssssssssss.s.s.s.s.s.s.s.s.s.s.s.s.s.sEsEs.s.sEsEs.sssss.sss.sssss.sssss.s.sssssssssss.s.s.s.s.s.sEs.s.s.s.s.s.s.s.s.s.s.sssssss.s.s.s.s.s.sss.sssssssssssssssssssssssssssssss.s.sEsEs.s.sss.s.s.s.s.s.s.s.s.sssssss.s.sssssssssssssssssss.s.s.s.s.sssss.s.s.s.sssssssssss.s.s.s.s.s.s.s.s.s.s.s.sssssssss.s.s.sssss.s.sssssssssssssssssssssss.s.s.sEs.s.s.sssssssssssssssssssssssssssssssss.s.s.s.s.s.sssssssss.s.s.sssss.s.sssssssssssssssssssssssssssssssssssssssssssssss.sss.s.sssssssssssss.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.s.sssssssssssssssssssssssssssssssssssssss.sss.s.sssss.s.sssssss.s.sEsEs.s.sEsssssEsEsEsssssss.sss.s.s.s.s.s.s.s.s.sssssss.s.s.sssssssssssss.s.s.s.s.s.s.s.s.s.sssssssssssssssssssssssssEs.s.s.s.s.s.s.s.sssssssssssssssssssss.s.sssss.sss.s.s.sssssssssssss.sssssssssssssssssssssssssss.s.s.s.s.sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss.s.s.s.s.s.s.s.s.s.s.s.sssssssssssssss.sssssssssss.s.s.s.sssssssssssssssssssssssssssssssss.sssssssss.s.sssssssssssssssssssssssss
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_acosh_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 63, in _convert_operator
[2020-05-27T04:58:58.762Z] raise NotImplementedError("Operator {} not implemented.".format(op_name))
[2020-05-27T04:58:58.762Z] NotImplementedError: Operator Acosh not implemented.
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_acosh_example_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 63, in _convert_operator
[2020-05-27T04:58:58.762Z] raise NotImplementedError("Operator {} not implemented.".format(op_name))
[2020-05-27T04:58:58.762Z] NotImplementedError: Operator Acosh not implemented.
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_asinh_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 63, in _convert_operator
[2020-05-27T04:58:58.762Z] raise NotImplementedError("Operator {} not implemented.".format(op_name))
[2020-05-27T04:58:58.762Z] NotImplementedError: Operator Asinh not implemented.
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_asinh_example_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 63, in _convert_operator
[2020-05-27T04:58:58.762Z] raise NotImplementedError("Operator {} not implemented.".format(op_name))
[2020-05-27T04:58:58.762Z] NotImplementedError: Operator Asinh not implemented.
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_atanh_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 63, in _convert_operator
[2020-05-27T04:58:58.762Z] raise NotImplementedError("Operator {} not implemented.".format(op_name))
[2020-05-27T04:58:58.762Z] NotImplementedError: Operator Atanh not implemented.
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_atanh_example_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 63, in _convert_operator
[2020-05-27T04:58:58.762Z] raise NotImplementedError("Operator {} not implemented.".format(op_name))
[2020-05-27T04:58:58.762Z] NotImplementedError: Operator Atanh not implemented.
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_cast_FLOAT_to_STRING_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 69, in _convert_operator
[2020-05-27T04:58:58.762Z] mxnet_sym = new_op(*inputs, **new_attrs)
[2020-05-27T04:58:58.762Z] File "<string>", line 65, in cast
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/_ctypes/symbol.py", line 132, in _symbol_creator
[2020-05-27T04:58:58.762Z] ctypes.byref(sym_handle)))
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/base.py", line 246, in check_call
[2020-05-27T04:58:58.762Z] raise get_last_ffi_error()
[2020-05-27T04:58:58.762Z] mxnet.base.MXNetError: MXNetError: Invalid Input: 'object', valid values are: {'bfloat16', 'bool', 'float16', 'float32', 'float64', 'int32', 'int64', 'int8', 'uint8'}, in operator Cast(name="", __profiler_scope__="<unk>:", dtype="object")
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_cosh_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 63, in _convert_operator
[2020-05-27T04:58:58.762Z] raise NotImplementedError("Operator {} not implemented.".format(op_name))
[2020-05-27T04:58:58.762Z] NotImplementedError: Operator Cosh not implemented.
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_cosh_example_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 63, in _convert_operator
[2020-05-27T04:58:58.762Z] raise NotImplementedError("Operator {} not implemented.".format(op_name))
[2020-05-27T04:58:58.762Z] NotImplementedError: Operator Cosh not implemented.
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_matmulinteger_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 63, in _convert_operator
[2020-05-27T04:58:58.762Z] raise NotImplementedError("Operator {} not implemented.".format(op_name))
[2020-05-27T04:58:58.762Z] NotImplementedError: Operator MatMulInteger not implemented.
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_sinh_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 63, in _convert_operator
[2020-05-27T04:58:58.762Z] raise NotImplementedError("Operator {} not implemented.".format(op_name))
[2020-05-27T04:58:58.762Z] NotImplementedError: Operator Sinh not implemented.
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_sinh_example_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 63, in _convert_operator
[2020-05-27T04:58:58.762Z] raise NotImplementedError("Operator {} not implemented.".format(op_name))
[2020-05-27T04:58:58.762Z] NotImplementedError: Operator Sinh not implemented.
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_slice_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 61, in _convert_operator
[2020-05-27T04:58:58.762Z] op_name, new_attrs, inputs = convert_map[op_name](attrs, inputs, self)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/_op_translations.py", line 511, in _slice
[2020-05-27T04:58:58.762Z] end = list(new_attrs.get('end'))
[2020-05-27T04:58:58.762Z] TypeError: 'NoneType' object is not iterable
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_slice_end_out_of_bounds_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 61, in _convert_operator
[2020-05-27T04:58:58.762Z] op_name, new_attrs, inputs = convert_map[op_name](attrs, inputs, self)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/_op_translations.py", line 511, in _slice
[2020-05-27T04:58:58.762Z] end = list(new_attrs.get('end'))
[2020-05-27T04:58:58.762Z] TypeError: 'NoneType' object is not iterable
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_slice_neg_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 61, in _convert_operator
[2020-05-27T04:58:58.762Z] op_name, new_attrs, inputs = convert_map[op_name](attrs, inputs, self)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/_op_translations.py", line 511, in _slice
[2020-05-27T04:58:58.762Z] end = list(new_attrs.get('end'))
[2020-05-27T04:58:58.762Z] TypeError: 'NoneType' object is not iterable
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_slice_neg_steps_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 61, in _convert_operator
[2020-05-27T04:58:58.762Z] op_name, new_attrs, inputs = convert_map[op_name](attrs, inputs, self)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/_op_translations.py", line 511, in _slice
[2020-05-27T04:58:58.762Z] end = list(new_attrs.get('end'))
[2020-05-27T04:58:58.762Z] TypeError: 'NoneType' object is not iterable
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ======================================================================
[2020-05-27T04:58:58.762Z] ERROR: test_top_k_cpu (__main__.OnnxBackendNodeModelTest)
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Traceback (most recent call last):
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 248, in device_test_func
[2020-05-27T04:58:58.762Z] return test_func(*args, device=device, **kwargs)
[2020-05-27T04:58:58.762Z] File "/usr/local/lib/python3.6/dist-packages/onnx/backend/test/runner/__init__.py", line 278, in run
[2020-05-27T04:58:58.762Z] prepared_model = self.backend.prepare(model, device)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/tests/python/unittest/onnx/backend.py", line 99, in prepare
[2020-05-27T04:58:58.762Z] sym, arg_params, aux_params = graph.from_onnx(model.graph)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 115, in from_onnx
[2020-05-27T04:58:58.762Z] mxnet_sym = self._convert_operator(node_name, op_name, onnx_attr, inputs)
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/contrib/onnx/onnx2mx/import_onnx.py", line 69, in _convert_operator
[2020-05-27T04:58:58.762Z] mxnet_sym = new_op(*inputs, **new_attrs)
[2020-05-27T04:58:58.762Z] File "<string>", line 106, in topk
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/_ctypes/symbol.py", line 132, in _symbol_creator
[2020-05-27T04:58:58.762Z] ctypes.byref(sym_handle)))
[2020-05-27T04:58:58.762Z] File "/work/mxnet/python/mxnet/base.py", line 246, in check_call
[2020-05-27T04:58:58.762Z] raise get_last_ffi_error()
[2020-05-27T04:58:58.762Z] mxnet.base.MXNetError: MXNetError: Invalid Parameter format for axis expect int or None but value='<Symbol k>', in operator topk(name="", __profiler_scope__="<unk>:", axis="<Symbol k>", dtype="int64", ret_typ="both")
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] ----------------------------------------------------------------------
[2020-05-27T04:58:58.762Z] Ran 1116 tests in 603.735s
[2020-05-27T04:58:58.762Z]
[2020-05-27T04:58:58.762Z] FAILED (errors=17, skipped=853)
```
</details>
</blockquote>
These are logs from a pretty recent commit from master. You can clearly see that the pipeline is green, but there are a lot of failed tests in `mxnet/tests/python/unittest/onnx`.
It seems to me, that 18025 just bumped the onnx version without updating the tests. Not sure, why the pipeline is marked as green in the Jenkins UI. Maybe there is a missing `set -ex` somewhere or something.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] mxnet-bot commented on pull request #18054: Fix tests for ONNX version 1.5.0 bump
Posted by GitBox <gi...@apache.org>.
mxnet-bot commented on pull request #18054:
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-634959690
Jenkins CI successfully triggered : [miscellaneous, unix-cpu, centos-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] RuRo commented on pull request #18054: Bump ONNX version to 1.5.0
Posted by GitBox <gi...@apache.org>.
RuRo commented on pull request #18054:
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-634957047
@szha, I am **not** proposing a bump to 1.6.0. The current version bump (1.3.0 to 1.5.0) seems to have broken the current ONNX tests (see my previous comment). For some reason, the ONNX tests fail, but the pipeline still succeeds.
I was going to bump the ONNX version in this PR, so I fixed most of the tests for ONNX v1.5.0.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] RuRo removed a comment on pull request #18054: Fix tests for ONNX version 1.5.0 bump
Posted by GitBox <gi...@apache.org>.
RuRo removed a comment on pull request #18054:
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-634959644
@mxnet-bot run ci [centos-cpu, miscellaneous, unix-cpu]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] RuRo commented on pull request #18054: Fix tests for ONNX version 1.5.0 bump
Posted by GitBox <gi...@apache.org>.
RuRo commented on pull request #18054:
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-639597045
@szha PTAL or @ some other reviewer plz?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] szha commented on pull request #18054: Bump ONNX version to 1.5.0
Posted by GitBox <gi...@apache.org>.
szha commented on pull request #18054:
URL: https://github.com/apache/incubator-mxnet/pull/18054#issuecomment-634894983
ONNX 1.6.0 switches the opset version to 2 so it would be a bigger change. https://github.com/onnx/onnx/blob/fcb553ec2376994853f147ca2ad9d4148bb9454a/docs/Versioning.md#released-versions
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org