[GitHub] szha commented on issue #8689: Mark tests that should only be run nightly.

[GitBox <gi...@apache.org>]

szha commented on issue #8689: Mark tests that should only be run nightly.
URL: https://github.com/apache/incubator-mxnet/pull/8689#issuecomment-345336009
 
 
   I understand the motivation for moving tests to nightly. The concern I still have is if we disable certain tests on PR-level and only run them nightly, the problems in code may not surface in PR as they should. One might submit a patch for optimization or fix on existing op which actually contains bug, whose tests only run nightly. Reviewers might assume that the tests are run by the CI while in fact they are not. In general, removing tests from CI in PR is equivalent of removing the guarantee that the particular functionality still works after the PR.
   
   Admittedly some tests, including some of the tests I wrote, may be simple fuzzy tests with randomized inputs, that should have been designed as small fixtures to catch specific edge cases. That way, the each of the tests won't run for longer than seconds which is ideal. I agree we should do better at writing tests. Given the current situation in the test cases, it will be major endeavor moving from where we are to where the ideal world is. Given the constraint that we shouldn't be breaking mxnet in doing so, I think we should only replace existing long-running tests with carefully designed test cases instead of disabling them.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services