You are viewing a plain text version of this content. The canonical link for it is here.

Posted to discuss-archive@tvm.apache.org by haozech via Apache TVM Discuss <no...@discuss.tvm.ai> on 2020/11/03 11:45:11 UTC

[Apache TVM Discuss] [Questions] How can I test the performance of a single operator?


Hi guys, I'm new to TVM and I was trying to test the performance of a single operator on NVGPU.
So far I found the doc about how to test model benchmark(https://github.com/apache/incubator-tvm/blob/main/apps/benchmark/README.md).
And the doc about [Tuning High Performance Convolution on NVIDIA GPUs] (https://tvm.apache.org/docs/tutorials/autotvm/tune_conv2d_cuda.html).
But neither of them satisfies my needs. What I want is to test the single op performance instead of testing a whole model nor self-defining an operator and tune it.
Can anybody tell me how to do this?





---
[Visit Topic](https://discuss.tvm.apache.org/t/how-can-i-test-the-performance-of-a-single-operator/8362/1) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/f46ecd1f754385224a66e11db83a135f78bc2bead632bd2bf832d6e09282ee93).

[Apache TVM Discuss] [Questions] How can I test the performance of a single operator?

Posted by haozech via Apache TVM Discuss <no...@discuss.tvm.ai>.


And also I got some error using method 1.
Here is my code:
```python
strides, padding, dilation = (1, 1), (1, 1), (1, 1)
data = te.placeholder((1, 512, 7, 7), name="data")
kernel = te.placeholder((512, 512, 3, 3), name="kernel")
cfg = autotvm.get_config()
task = autotvm.task.create(
    "conv2d_nchw.cuda", args=( cfg, data, kernel, strides, padding, dilation), target="cuda"
)
```
And the error info is:
```
Cannot find config for target=None, workload=None. A fallback configuration is used, which may bring great performance regression.
Traceback (most recent call last):
  File "tvm_op_test.py", line 105, in <module>
    "conv2d_nchw.cuda", args=( cfg, data, kernel, strides, padding, dilation), target="cuda"
  File "/WorkSpace/incubator-tvm/python/tvm/autotvm/task/task.py", line 445, in create
    args = serialize_args(args)
  File "/WorkSpace/incubator-tvm/python/tvm/autotvm/task/task.py", line 77, in serialize_args
    ret.append(_encode(t))
  File "/WorkSpace/incubator-tvm/python/tvm/autotvm/task/task.py", line 72, in _encode
    "primitive types or tvm.tir.Var only" % type(x)
RuntimeError: Do not support type "<class 'tvm.autotvm.task.space.FallbackConfigEntity'>" in argument. Consider to useprimitive types or tvm.tir.Var only
```





---
[Visit Topic](https://discuss.tvm.apache.org/t/how-can-i-test-the-performance-of-a-single-operator/8362/4) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/d3107b6134f5a1bf3832ec93490aff25a43317fc92abe327734ec64135556571).

[Apache TVM Discuss] [Questions] How can I test the performance of a single operator?

Posted by haozech via Apache TVM Discuss <no...@discuss.tvm.ai>.


Thank you for your reply! It's really helpful. Well, I found that in [Tuning High Performance Convolution on NVIDIA GPUs ](https://tvm.apache.org/docs/tutorials/autotvm/tune_conv2d_cuda.html), the step 2 will do tuning and find the best config. Is there any way to skip tuning and just test the default config's performance?





---
[Visit Topic](https://discuss.tvm.apache.org/t/how-can-i-test-the-performance-of-a-single-operator/8362/3) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/22e07e555edf69a7bf63304d9445258e5599c87b01583e742d563ce33148c10f).

[Apache TVM Discuss] [Questions] How can I test the performance of a single operator?

Posted by haozech via Apache TVM Discuss <no...@discuss.tvm.ai>.


Problem solved. Thank you!





---
[Visit Topic](https://discuss.tvm.apache.org/t/how-can-i-test-the-performance-of-a-single-operator/8362/7) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/66da986478b8fca3277e56fd3c3aa129e2b51ccb7e68d5c898bf12e2db03daed).

[Apache TVM Discuss] [Questions] How can I test the performance of a single operator?

Posted by Tristan Konolige via Apache TVM Discuss <no...@discuss.tvm.ai>.


To test without a config, remove the `autotvm.apply_history_best` with statement.

`params` is a dictionary mapping from the name of a weight to the actual values of the weight. In this case, you have no weights, so it is just the empty dictionary.





---
[Visit Topic](https://discuss.tvm.apache.org/t/how-can-i-test-the-performance-of-a-single-operator/8362/6) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/28fda6e1b2786e11f1123818cdcc36c3754e97af94852fe9a01f9798e7de2dce).

[Apache TVM Discuss] [Questions] How can I test the performance of a single operator?

Posted by haozech via Apache TVM Discuss <no...@discuss.tvm.ai>.


By the way, for 2, the function should return 4 values:`mod, params, input_shape, output_shape`. But I didn't see the params in the code? 
```python
x = relay.Var("x", tvm.relay.TensorType([40, 40]))
y = relay.Var("y", tvm.relay.TensorType([40, 40]))
mod = relay.Function(
    [x, y],
    relay.my_function(x, y)
)
```





---
[Visit Topic](https://discuss.tvm.apache.org/t/how-can-i-test-the-performance-of-a-single-operator/8362/5) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/b4cebc2b6950127a72440952bf7c58d3023aec06e8d1ab026d00a98db35a0e3a).

[Apache TVM Discuss] [Questions] How can I test the performance of a single operator?

Posted by Tristan Konolige via Apache TVM Discuss <no...@discuss.tvm.ai>.


Hello @haozech. There are two ways you can go about benchmarking a single operator. You can either 1. benchmark a specific implementation of the operator or 2. benchmark all implementations of the operator.

For 1, follow the [Tuning High Performance Convolution on NVIDIA GPUs](https://tvm.apache.org/docs/tutorials/autotvm/tune_conv2d_cuda.html) tutorial, but skip section 1. In section two, replace `"tutorial/conv2d_no_batching"` in `task = autotvm.task.create("tutorial/conv2d_no_batching", args=(N, H, W, CO, CI, KH, KW, strides, padding), target="cuda")` with the name of the implementation you want to benchmark. You can find the name of the implementation by greping the codebase for `@autotvm.register_topi_compute`. You'll also have to modify the inputs to they match what the function is expecting. Furthermore, you'll have to change `conv2d_no_batching` and `conv2d_nchw_python` in the last code block with the correct function names (these should be the name of the function annotated with `@autotvm.register_topi_compute`).

For 2, follow the [Auto-tuning a convolutional network for NVIDIA GPU](https://tvm.apache.org/docs/tutorials/autotvm/tune_relay_cuda.html) tutorial. Replace `get_network` with a function that returns a relay function with a single operator like so:
```
x = relay.Var("x", tvm.relay.TensorType([40, 40]))
y = relay.Var("y", tvm.relay.TensorType([40, 40]))
mod = relay.Function(
    [x, y],
    relay.my_function(x, y)
)
```





---
[Visit Topic](https://discuss.tvm.apache.org/t/how-can-i-test-the-performance-of-a-single-operator/8362/2) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/b74575d4c7f5d2b0894f8cf942f2ce25244e267cad638a71bc9b4cf1da48c22c).