You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/02/07 18:28:45 UTC
[GitHub] [tvm] mbrookhart opened a new issue #10180: [CI Problem] x386 CI running out of RAM
mbrookhart opened a new issue #10180:
URL: https://github.com/apache/tvm/issues/10180
On a recent PR that added a few extra tests to Relay, we discovered that pytest was running over the 4GB RAM limit on the x386 CI job. We fixed this by reducing the memory use of the failing test ~10%, but we're getting to the point in our test size were running `pytest tests/python/relay` seems to be accumulating too much in RAM via the tests and pytest logs to actually run on x386. I imagine we'll hit this again in the future, should we perhaps write a bash script to run the test files 1 by 1 for the 32 bit job?
cc @driazati @areusch
Also wondering if @leandron might have some thoughts.
### Branch/PR Failing
https://github.com/apache/tvm/pull/10026
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on issue #10180: [CI Problem] x386 CI running out of RAM
Posted by GitBox <gi...@apache.org>.
areusch commented on issue #10180:
URL: https://github.com/apache/tvm/issues/10180#issuecomment-1031813291
we could also try to investigate why pytest-forked doesn't like GPUs. could you post any information you have about that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on issue #10180: [CI Problem] x386 CI running out of RAM
Posted by GitBox <gi...@apache.org>.
masahi commented on issue #10180:
URL: https://github.com/apache/tvm/issues/10180#issuecomment-1031959054
@mbrookhart Can you point to the failed log?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mbrookhart commented on issue #10180: [CI Problem] x386 CI running out of RAM
Posted by GitBox <gi...@apache.org>.
mbrookhart commented on issue #10180:
URL: https://github.com/apache/tvm/issues/10180#issuecomment-1033079857
@masahi https://ci.tlcpack.ai/blue/organizations/jenkins/tvm/detail/PR-10026/20/pipeline
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi edited a comment on issue #10180: [CI Problem] x386 CI running out of RAM
Posted by GitBox <gi...@apache.org>.
masahi edited a comment on issue #10180:
URL: https://github.com/apache/tvm/issues/10180#issuecomment-1031959054
@mbrookhart Can you point to the failed log from a job in https://github.com/apache/tvm/pull/10026?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] mbrookhart commented on issue #10180: [CI Problem] x386 CI running out of RAM
Posted by GitBox <gi...@apache.org>.
mbrookhart commented on issue #10180:
URL: https://github.com/apache/tvm/issues/10180#issuecomment-1031828030
I attempted to fix this using pytest --forked here: https://github.com/apache/tvm/pull/10174
But it failed a lot of tests on a lot of jobs related to GPU. I got the feeling that initializing the GPU interface on the main thread and then trying to access it from the forked thread broke an assumption somewhere in the stack, but I didn't dig very deeply on what the root cause might be.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] FranckQC commented on issue #10180: [CI Problem] x386 CI running out of RAM
Posted by GitBox <gi...@apache.org>.
FranckQC commented on issue #10180:
URL: https://github.com/apache/tvm/issues/10180#issuecomment-1032079462
Hi everyone.
It looks like I have the same issue on this i386 test on the CSE PR : https://github.com/apache/tvm/pull/9482
Let's see how the current build will end up (in theory the docs should be ok this time, and the Windows buikd too, it was just due to a URL change for the docs, and to a Github maintenance for the Windows build).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org