You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2021/09/03 11:04:04 UTC
[GitHub] [tvm] vvchernov opened a new pull request #8923: [ONNX] [Test] fix GRU modification (linear before reset = False) and reduce tolerance for RNN tests
vvchernov opened a new pull request #8923:
URL: https://github.com/apache/tvm/pull/8923
During unification of GRU layer for frontends (#8781) the critical bag in GRU implementation on ONNX frontend side was observed. Inspite of unit test of GRU the bag was hidden by high tolerance (~1e-2) set for RNN tests. Moreover after bug fixing GRU unit test still requires high tolerance for correct pass. This fact was strange because local tests show very good coincidence with golden output from pytorch for any GRU modification (~1e-7).
1. I checked LSTM tests: tolerance 1e-6 is enough in most cases but sometimes it needs 1e-5 (it is tolerance for other operations tested for onnx frontend).
2. Bug in GRU was localized and fixed. it was related to implementation of specific GRU modification.
3. tolerance for RNN tests were reduced from 1e-2 to 1e-5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] vvchernov commented on pull request #8923: [ONNX] [Test] fix GRU modification (linear before reset = False) and reduce tolerance for RNN tests
Posted by GitBox <gi...@apache.org>.
vvchernov commented on pull request #8923:
URL: https://github.com/apache/tvm/pull/8923#issuecomment-912685320
It is a good question. Unfortunately there is only one test for operation (or its modification) to reduce testing time. Moreover numpy random is used with repeatable randomization. It means that it is not checked fully all cases for operation from which we could define tolerance thresholds. But it checks that operation works for one but orbitrary input for reasonable tolerance. For numerical calculations the tolerance of order of 1e-6 - 1e-5 seems reasonable. In my case it was high enough to hide problem in operation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] vvchernov commented on pull request #8923: [ONNX] [Test] fix GRU modification (linear before reset = False) and reduce tolerance for RNN tests
Posted by GitBox <gi...@apache.org>.
vvchernov commented on pull request #8923:
URL: https://github.com/apache/tvm/pull/8923#issuecomment-913820712
@AndrewZhaoLuo I have added seed for test reproduction and checked it 1000 times.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi merged pull request #8923: [ONNX] [Test] fix GRU modification and reduce tolerance for RNN tests
Posted by GitBox <gi...@apache.org>.
masahi merged pull request #8923:
URL: https://github.com/apache/tvm/pull/8923
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] AndrewZhaoLuo commented on pull request #8923: [ONNX] [Test] fix GRU modification (linear before reset = False) and reduce tolerance for RNN tests
Posted by GitBox <gi...@apache.org>.
AndrewZhaoLuo commented on pull request #8923:
URL: https://github.com/apache/tvm/pull/8923#issuecomment-912666438
Hey all, how did you check the required tolerance? Can you run it 1000 times to ensure these tolerances will pass if you have not already?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] AndrewZhaoLuo commented on pull request #8923: [ONNX] [Test] fix GRU modification (linear before reset = False) and reduce tolerance for RNN tests
Posted by GitBox <gi...@apache.org>.
AndrewZhaoLuo commented on pull request #8923:
URL: https://github.com/apache/tvm/pull/8923#issuecomment-912700316
I just want to make sure the tests are not super flaky.
You can do this by running `pytest tests/python/frontend/onnx/test_forward.py::test_gru --repeat 1000` after pip installing `pytest-repeat`.
Lower than 1 / 1000 failures will seem sufficient.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org