You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/09/21 15:33:18 UTC
[GitHub] [tvm] f2013519 opened a new issue, #12858: [Bug]
f2013519 opened a new issue, #12858:
URL: https://github.com/apache/tvm/issues/12858
Thanks for participating in the TVM community! We use https://discuss.tvm.ai for any general usage questions and discussions. The issue tracker is used for actionable items such as feature proposals discussion, roadmaps, and bug tracking. You are always welcomed to post on the forum first :smile_cat:
Issues that are inactive for a period of time may get closed. We adopt this policy so that we won't lose track of actionable issues that may fall at the bottom of the pile. Feel free to reopen a new one if you feel there is an additional problem that needs attention when an old one gets closed.
### Expected behavior
When we run tvmc run with repeats=0, I would expect that the inference runs for 1 time (no repeat). Also unrelated, I would expect that this is the default value, but that is debatable.
### Actual behavior
We receive an error
### Environment
OS: Linux 4.15.0-188-generic #199-Ubuntu SMP Wed Jun 15 20:42:56 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
tvm 0.9
### Steps to reproduce
Generate a model.tar from any model and run
tvmc run --inputs infer.npz --outputs predict.npz model.tar --print-time --repeat=0
/usr/local/lib/python3.7/dist-packages/numpy/core/fromnumeric.py:3373: RuntimeWarning: Mean of empty slice.
out=out, **kwargs)
/usr/local/lib/python3.7/dist-packages/numpy/core/_methods.py:170: RuntimeWarning: invalid value encountered in double_scalars
ret = ret.dtype.type(ret / rcount)
/usr/local/lib/python3.7/dist-packages/numpy/core/_methods.py:234: RuntimeWarning: Degrees of freedom <= 0 for slice
keepdims=keepdims)
/usr/local/lib/python3.7/dist-packages/numpy/core/_methods.py:195: RuntimeWarning: invalid value encountered in true_divide
arrmean, rcount, out=arrmean, casting='unsafe', subok=False)
/usr/local/lib/python3.7/dist-packages/numpy/core/_methods.py:226: RuntimeWarning: invalid value encountered in double_scalars
ret = ret.dtype.type(ret / rcount)
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/nfs/bnusers1/kbindumadhav/kbindumadhav/tvm3/tvm/python/tvm/driver/tvmc/__main__.py", line 24, in <module>
tvmc.main.main()
File "/nfs/bnusers1/kbindumadhav/kbindumadhav/tvm3/tvm/python/tvm/driver/tvmc/main.py", line 115, in main
sys.exit(_main(sys.argv[1:]))
File "/nfs/bnusers1/kbindumadhav/kbindumadhav/tvm3/tvm/python/tvm/driver/tvmc/main.py", line 103, in _main
return args.func(args)
File "/nfs/bnusers1/kbindumadhav/kbindumadhav/tvm3/tvm/python/tvm/driver/tvmc/runner.py", line 280, in drive_run
options=options,
File "/nfs/bnusers1/kbindumadhav/kbindumadhav/tvm3/tvm/python/tvm/driver/tvmc/runner.py", line 684, in run_module
times = module.benchmark(dev, number=number, repeat=repeat, end_to_end=end_to_end)
File "/nfs/bnusers1/kbindumadhav/kbindumadhav/tvm3/tvm/python/tvm/contrib/graph_executor.py", line 450, in benchmark
repeats_to_cooldown=repeats_to_cooldown,
File "/nfs/bnusers1/kbindumadhav/kbindumadhav/tvm3/tvm/python/tvm/runtime/module.py", line 354, in evaluator
return BenchmarkResult(results)
File "/nfs/bnusers1/kbindumadhav/kbindumadhav/tvm3/tvm/python/tvm/runtime/module.py", line 73, in __init__
self.min = np.min(self.results)
File "<__array_function__ internals>", line 6, in amin
File "/usr/local/lib/python3.7/dist-packages/numpy/core/fromnumeric.py", line 2831, in amin
keepdims=keepdims, initial=initial, where=where)
File "/usr/local/lib/python3.7/dist-packages/numpy/core/fromnumeric.py", line 87, in _wrapreduction
return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
ValueError: zero-size array to reduction operation minimum which has no identity
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhutton1 commented on issue #12858: [Bug] Issue with tvmc run with repeat=0
Posted by GitBox <gi...@apache.org>.
lhutton1 commented on issue #12858:
URL: https://github.com/apache/tvm/issues/12858#issuecomment-1254866832
Hi @f2013519, I submitted #12713 which makes sure the module is only run once when the user is _not_ worried about benchmarking so wanted to share some of my findings here. I also ran into a similar issue when benchmarking was being used (`--print-time=True`) and, as you correctly mention, the additional run is due to a warm-up use to prime caches and lazy optimizations. I found this a bit unintuitive at first so tried to document some of these quirks directly in the PR (#12713).
That said, it seems like a legitimate requirement to me to be able to benchmark just a single execution of the model, which currently is not possible. The additional run happens here: https://github.com/apache/tvm/blob/main/src/runtime/profiling.cc#L880 - we should make this additional run optional and expose the option to TVMC perhaps with a flag like `--disable-benchmark-warmup`.
cc @leandron @tkonolige @areusch would be interested to hear your thoughts
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] f2013519 commented on issue #12858: [Bug] Issue with tvmc run with repeat=0
Posted by GitBox <gi...@apache.org>.
f2013519 commented on issue #12858:
URL: https://github.com/apache/tvm/issues/12858#issuecomment-1254694090
Also with --number=0, we get the below warning:
/usr/local/lib/python3.7/dist-packages/numpy/core/_methods.py:202: RuntimeWarning: invalid value encountered in subtract x = asanyarray(arr - arrmean)
Use case is to run on a software emulator, so warm up is not required.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on issue #12858: [Bug] Issue with tvmc run with repeat=0
Posted by GitBox <gi...@apache.org>.
areusch commented on issue #12858:
URL: https://github.com/apache/tvm/issues/12858#issuecomment-1281018176
yeah could we maybe have like a `num_warmups=1` default parameter?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tkonolige commented on issue #12858: [Bug] Issue with tvmc run with repeat=0
Posted by GitBox <gi...@apache.org>.
tkonolige commented on issue #12858:
URL: https://github.com/apache/tvm/issues/12858#issuecomment-1255234586
I'm fine with adding a flag to disable the warmup iteration, but warmup iterations should be on by default. On CUDA, the first run on the model is very slow due to lazy compilation/loading.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org