You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2021/10/13 18:59:51 UTC
[GitHub] [tvm] Lunderberg opened a new pull request #9275: [UnitTest][Flaky] In test_report_serialization, compare csv.
Lunderberg opened a new pull request #9275:
URL: https://github.com/apache/tvm/pull/9275
`str(report)` calls `ReportNode::AsTable()`, which includes aggregate values. Otherwise negligible differences in the computed value can be rounded differently after the round trip. This was first [noticed in CI](https://ci.tlcpack.ai/blue/organizations/jenkins/tvm/detail/PR-9194/7/pipeline/#step-246-log-1217) for an unrelated PR. Testing locally, this failure mode occurred 2 times out of 3000 trials.
Switching to `report.csv()` avoids this issue, as it does not include aggregates.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Lunderberg commented on pull request #9275: [UnitTest][Flaky] In test_report_serialization, compare csv.
Posted by GitBox <gi...@apache.org>.
Lunderberg commented on pull request #9275:
URL: https://github.com/apache/tvm/pull/9275#issuecomment-943374796
Thank you for catching that, and I've updated to expose the `ReportNode::AsTable`, explicitly disable the column summations, and use it instead of `.csv()` to have a consistent output.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tkonolige commented on pull request #9275: [UnitTest][Flaky] In test_report_serialization, compare csv.
Posted by GitBox <gi...@apache.org>.
tkonolige commented on pull request #9275:
URL: https://github.com/apache/tvm/pull/9275#issuecomment-944446206
Maybe just mark this test as `xfail`. I'll get a sorting PR in soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Lunderberg commented on pull request #9275: [UnitTest][Flaky] In test_report_serialization, compare csv.
Posted by GitBox <gi...@apache.org>.
Lunderberg commented on pull request #9275:
URL: https://github.com/apache/tvm/pull/9275#issuecomment-944307273
And it looks like `AsTable` has the same potential issue where the column names aren't sorted. I still like using `AsTable`, because it includes the `device_metrics` as part of the serialization/deserialization, so it looks like it should have the explicit column sorting added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tkonolige commented on pull request #9275: [UnitTest][Flaky] In test_report_serialization, compare csv.
Posted by GitBox <gi...@apache.org>.
tkonolige commented on pull request #9275:
URL: https://github.com/apache/tvm/pull/9275#issuecomment-942773268
There's no guarantee of ordering of columns in the csv, either we need to add ordering or make the test order them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi merged pull request #9275: [UnitTest][Flaky] In test_report_serialization, compare csv.
Posted by GitBox <gi...@apache.org>.
masahi merged pull request #9275:
URL: https://github.com/apache/tvm/pull/9275
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org