You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2020/09/07 16:38:40 UTC
[GitHub] [incubator-tvm] leandron opened a new issue #6417: [FLAKY] relay/test_op_level1.py::test_binary_op failing intermittently
leandron opened a new issue #6417:
URL: https://github.com/apache/incubator-tvm/issues/6417
As part of #6302, it seems there is another flaky test, that is not related to the contents of the being submitted there: `tests/python/relay/test_op_level1.py::test_binary_op`. You can see this on Jenkins for the full stack trace: https://ci.tvm.ai/blue/organizations/jenkins/tvm/detail/PR-6302/14/pipeline/185
The error message I see is:
```
> np.testing.assert_allclose(op_res.asnumpy(), ref_res, rtol=0.01)
E AssertionError:
E Not equal to tolerance rtol=0.01, atol=0
E
E Mismatched elements: 1 / 250 (0.4%)
E Max absolute difference: 0.00023794
E Max relative difference: 0.01051613
E x: array([[[1.909631e-01, 2.819712e-02, 6.732421e-01, 6.087055e-01,
E 7.002974e-02],
E [3.610725e-03, 6.303363e-01, 2.848548e-03, 1.215679e-01,...
E y: array([[[1.9092e-01, 2.8198e-02, 6.7334e-01, 6.0889e-01, 7.0007e-02],
E [3.6106e-03, 6.3037e-01, 2.8477e-03, 1.2158e-01, 7.1533e-01],
E [2.4643e-02, 6.6943e-01, 7.9880e-03, 3.1323e-01, 7.0996e-01],...
```
I believe we will need to increase the tolerance, but I don't know by how much it would be adequate, so I'll just report it here.
cc @tqchen @merrymercy
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] leandron commented on issue #6417: [FLAKY] relay/test_op_level1.py::test_binary_op failing intermittently
Posted by GitBox <gi...@apache.org>.
leandron commented on issue #6417:
URL: https://github.com/apache/incubator-tvm/issues/6417#issuecomment-691627307
@tqchen Yes, I’ll have a look
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on issue #6417: [FLAKY] relay/test_op_level1.py::test_binary_op failing intermittently
Posted by GitBox <gi...@apache.org>.
tqchen commented on issue #6417:
URL: https://github.com/apache/incubator-tvm/issues/6417#issuecomment-688437913
related
https://github.com/apache/incubator-tvm/issues/6106,
https://github.com/apache/incubator-tvm/issues/4378,
https://github.com/apache/incubator-tvm/issues/4210
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on issue #6417: [FLAKY] relay/test_op_level1.py::test_binary_op failing intermittently
Posted by GitBox <gi...@apache.org>.
tqchen commented on issue #6417:
URL: https://github.com/apache/incubator-tvm/issues/6417#issuecomment-691542488
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on issue #6417: [FLAKY] relay/test_op_level1.py::test_binary_op failing intermittently
Posted by GitBox <gi...@apache.org>.
tqchen commented on issue #6417:
URL: https://github.com/apache/incubator-tvm/issues/6417#issuecomment-688437528
My guess the problem was due to floordivide/floormod, where there is a boundary case. So increasing tol may not help. We need to explicitly change the generator to avoid such boundary cases. There are some existing topi cases that already do this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] leandron commented on issue #6417: [FLAKY] relay/test_op_level1.py::test_binary_op failing intermittently
Posted by GitBox <gi...@apache.org>.
leandron commented on issue #6417:
URL: https://github.com/apache/incubator-tvm/issues/6417#issuecomment-717397927
Sorry I didn't manage to get into this yet.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on issue #6417: [FLAKY] relay/test_op_level1.py::test_binary_op failing intermittently
Posted by GitBox <gi...@apache.org>.
tqchen commented on issue #6417:
URL: https://github.com/apache/incubator-tvm/issues/6417#issuecomment-691696307
There are know fixes please take a look at the issues linked
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on issue #6417: [FLAKY] relay/test_op_level1.py::test_binary_op failing intermittently
Posted by GitBox <gi...@apache.org>.
tqchen commented on issue #6417:
URL: https://github.com/apache/incubator-tvm/issues/6417#issuecomment-709314955
ping @leandron :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tqchen commented on issue #6417: [FLAKY] relay/test_op_level1.py::test_binary_op failing intermittently
Posted by GitBox <gi...@apache.org>.
tqchen commented on issue #6417:
URL: https://github.com/apache/incubator-tvm/issues/6417#issuecomment-691542488
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] leandron commented on issue #6417: [FLAKY] relay/test_op_level1.py::test_binary_op failing intermittently
Posted by GitBox <gi...@apache.org>.
leandron commented on issue #6417:
URL: https://github.com/apache/incubator-tvm/issues/6417#issuecomment-693521798
Just to follow up here. I investigated how a fix would look like and it seems there is some manual filtering to be done on values around 0.5 for `floor_divide` and/or `floor_mod`, that might disagree when comparing relay vs. numpy.
I'll to submit a PR with this filter (soon), similar to #4382, plus improve diagnostic on that test case, so in case it fails, it will give us which function (out of 6 tested on that test) is the offending one.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org