You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2020/12/14 17:42:38 UTC
[GitHub] [tvm] ANSHUMAN87 opened a new pull request #7107: [Tutorial] Enabling sparse tutorial
ANSHUMAN87 opened a new pull request #7107:
URL: https://github.com/apache/tvm/pull/7107
cc @tkonolige , @jwfromm !
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ANSHUMAN87 commented on a change in pull request #7107: [Tutorial] Add output validation to sparse tutorial
Posted by GitBox <gi...@apache.org>.
ANSHUMAN87 commented on a change in pull request #7107:
URL: https://github.com/apache/tvm/pull/7107#discussion_r547988127
##########
File path: tests/scripts/task_ci_python_setup.sh
##########
@@ -31,3 +31,4 @@ set -o pipefail
echo "Addtiional setup in" ${CI_IMAGE_NAME}
python3 -m pip install --user tlcpack-sphinx-addon==0.1.3 synr==0.2.1
+python3 -m pip install --user tokenizers==0.9.4 transformers==4.0.1
Review comment:
Thanks @tkonolige ! I will look into it further.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ANSHUMAN87 commented on a change in pull request #7107: [Tutorial] Add output validation to sparse tutorial
Posted by GitBox <gi...@apache.org>.
ANSHUMAN87 commented on a change in pull request #7107:
URL: https://github.com/apache/tvm/pull/7107#discussion_r545081512
##########
File path: tutorials/frontend/deploy_sparse.py
##########
@@ -119,6 +117,9 @@
# determines how sparse the generated weights should be. The higher
# the sparsity, the faster the result.
sparsity = 0.85
+# Running benchmarking mode might overload CI,
+# so it is disabled by default.
+benchmark = False
Review comment:
@tkonolige : I think moving to end may not look good, as already there are many settings defined at one place in the top section.
And about the message i had changed to "If true, then all relay models(dense and sparse) will be benchmarked."
Please let me know, if this is not clear. TIA!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ANSHUMAN87 commented on a change in pull request #7107: [Tutorial] Add output validation to sparse tutorial
Posted by GitBox <gi...@apache.org>.
ANSHUMAN87 commented on a change in pull request #7107:
URL: https://github.com/apache/tvm/pull/7107#discussion_r545279781
##########
File path: tutorials/frontend/deploy_sparse.py
##########
@@ -119,6 +117,9 @@
# determines how sparse the generated weights should be. The higher
# the sparsity, the faster the result.
sparsity = 0.85
+# Running benchmarking mode might overload CI,
+# so it is disabled by default.
+benchmark = False
Review comment:
Okay agree with your point, thanks for clarification! Just to confirm we should move all the config or only benchmark config to bottom?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tkonolige commented on a change in pull request #7107: [Tutorial] Add output validation to sparse tutorial
Posted by GitBox <gi...@apache.org>.
tkonolige commented on a change in pull request #7107:
URL: https://github.com/apache/tvm/pull/7107#discussion_r543488900
##########
File path: tutorials/frontend/deploy_sparse.py
##########
@@ -210,29 +211,28 @@ def import_graphdef(
# the weights are sparse, we won't see any speedup because we are using
# regular dense matrix multiplications on these dense (but mostly zero)
# tensors instead of sparse aware kernels.
-def run_relay_graph(mod, params, shape_dict, target, ctx):
+def run_relay_graph(mod, params, shape_dict, target, ctx, input):
with relay.build_config(opt_level=3):
lib = relay.build(mod, target=target, params=params)
- input_shape = shape_dict["input_1"]
- dummy_data = np.random.uniform(size=input_shape, low=0, high=input_shape[1]).astype("int32")
m = graph_runtime.GraphModule(lib["default"](ctx))
- m.set_input(0, dummy_data)
+ m.set_input(0, input)
m.run()
tvm_output = m.get_output(0)
- ftimer = m.module.time_evaluator("run", ctx, repeat=5, number=5)
- prof_res = np.array(ftimer().results) * 1000
- print(
- "%-20s %-19s (%s)"
- % ("Runtime:", "%.2f ms" % np.mean(prof_res), "%.2f ms" % np.std(prof_res))
- )
- return tvm_output
+ if benchmark:
+ ftimer = m.module.time_evaluator("run", ctx, repeat=5, number=5)
+ prof_res = np.array(ftimer().results) * 1000
+ print(
+ "%-20s %-19s (%s)"
+ % ("Runtime:", "Avg-%.2f ms" % np.mean(prof_res), "Std-%.2f ms" % np.std(prof_res))
+ )
+ return tvm_output.asnumpy()
-def run_dense(mod, params, shape_dict, target, ctx):
- print("Dense Model Benchmark:")
- return run_relay_graph(mod, params, shape_dict, target, ctx)
+def run_dense(mod, params, shape_dict, target, ctx, input):
+ print("Dense Model Inference begins.")
Review comment:
How about
```suggestion
print("Starting inference for dense model.")
```
##########
File path: tutorials/frontend/deploy_sparse.py
##########
@@ -297,13 +297,13 @@ def deepcopy(param_dic):
return new_params
-def run_sparse(mod, params, shape_dict, target, ctx, bs_r, sparsity, gen_weights):
+def run_sparse(mod, params, shape_dict, target, ctx, bs_r, sparsity, gen_weights, input):
mod, params = ddo.simplify_fc_transpose.convert(mod["main"], params)
if gen_weights:
params = random_sparse_bert_params(mod, params, BS_R=bs_r, BS_C=1, density=1 - sparsity)
mod, params = ddo.bsr_dense.convert(mod, params, (bs_r, 1), sparsity_threshold=0.8)
- print("Block Sparse Model with {blocksize}x1 blocks:".format(blocksize=bs_r))
- return run_relay_graph(mod, params, shape_dict, target, ctx)
+ print("Block Sparse Model with {blocksize}x1 blocks Inference begins.".format(blocksize=bs_r))
Review comment:
How about
```suggestion
print("Starting inference for block sparse model with {blocksize}x1 blocks.".format(blocksize=bs_r))
```
##########
File path: tutorials/frontend/deploy_sparse.py
##########
@@ -312,15 +312,22 @@ def run_sparse(mod, params, shape_dict, target, ctx, bs_r, sparsity, gen_weights
# And that's it! Now we'll simply call all the needed function to benchmark
Review comment:
Could you update this comment please
##########
File path: tutorials/frontend/deploy_sparse.py
##########
@@ -119,6 +117,9 @@
# determines how sparse the generated weights should be. The higher
# the sparsity, the faster the result.
sparsity = 0.85
+# Running benchmarking mode might overload CI,
+# so it is disabled by default.
+benchmark = False
Review comment:
Could you say something like "Switch this to true to run the benchmark yourself"?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tkonolige commented on a change in pull request #7107: [Tutorial] Add output validation to sparse tutorial
Posted by GitBox <gi...@apache.org>.
tkonolige commented on a change in pull request #7107:
URL: https://github.com/apache/tvm/pull/7107#discussion_r545374179
##########
File path: tests/scripts/task_ci_python_setup.sh
##########
@@ -31,3 +31,4 @@ set -o pipefail
echo "Addtiional setup in" ${CI_IMAGE_NAME}
python3 -m pip install --user tlcpack-sphinx-addon==0.1.3 synr==0.2.1
+python3 -m pip install --user tokenizers==0.9.4 transformers==4.0.1
Review comment:
Is this necessary?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ANSHUMAN87 commented on pull request #7107: [Tutorial] Enabling sparse tutorial
Posted by GitBox <gi...@apache.org>.
ANSHUMAN87 commented on pull request #7107:
URL: https://github.com/apache/tvm/pull/7107#issuecomment-744606857
> I'm a little confused as to what is going on here. It seems like you've moved the input data generation from one function to another? Could you provide a little description of what you've changed in this PR?
The inputs were being formed randomly for dense & sparse inference separately which were resulting in different outputs. So i have changed to a single input for both the case. And the final output is being compared for validation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ANSHUMAN87 commented on a change in pull request #7107: [Tutorial] Add output validation to sparse tutorial
Posted by GitBox <gi...@apache.org>.
ANSHUMAN87 commented on a change in pull request #7107:
URL: https://github.com/apache/tvm/pull/7107#discussion_r546195312
##########
File path: tests/scripts/task_ci_python_setup.sh
##########
@@ -31,3 +31,4 @@ set -o pipefail
echo "Addtiional setup in" ${CI_IMAGE_NAME}
python3 -m pip install --user tlcpack-sphinx-addon==0.1.3 synr==0.2.1
+python3 -m pip install --user tokenizers==0.9.4 transformers==4.0.1
Review comment:
@tkonolige : I am getting a strange error in CI when it is trying to build for docs. I thought the necessary packages were missing, so trying with the CI script, but to in-vain. Would you please check once, i am not sure what i am missing here. TIA!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ANSHUMAN87 commented on a change in pull request #7107: [Tutorial] Add output validation to sparse tutorial
Posted by GitBox <gi...@apache.org>.
ANSHUMAN87 commented on a change in pull request #7107:
URL: https://github.com/apache/tvm/pull/7107#discussion_r547302078
##########
File path: tests/scripts/task_ci_python_setup.sh
##########
@@ -31,3 +31,4 @@ set -o pipefail
echo "Addtiional setup in" ${CI_IMAGE_NAME}
python3 -m pip install --user tlcpack-sphinx-addon==0.1.3 synr==0.2.1
+python3 -m pip install --user tokenizers==0.9.4 transformers==4.0.1
Review comment:
@tkonolige : I think the error is reproduced now. Would you please have a look once in the latest CI report.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] jroesch commented on pull request #7107: [Tutorial] Add output validation to sparse tutorial
Posted by GitBox <gi...@apache.org>.
jroesch commented on pull request #7107:
URL: https://github.com/apache/tvm/pull/7107#issuecomment-1016774626
This PR appears to be out of date, please feel free to reopen it if this is not the case.
As part of the new year we are attempting to triage the project's open pull requests to ensure that code which
is ready for review and/or merging receives adequate attention.
Thanks again for your contribution, and feel free to reach out to discuss these changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tkonolige commented on a change in pull request #7107: [Tutorial] Add output validation to sparse tutorial
Posted by GitBox <gi...@apache.org>.
tkonolige commented on a change in pull request #7107:
URL: https://github.com/apache/tvm/pull/7107#discussion_r547540336
##########
File path: tests/scripts/task_ci_python_setup.sh
##########
@@ -31,3 +31,4 @@ set -o pipefail
echo "Addtiional setup in" ${CI_IMAGE_NAME}
python3 -m pip install --user tlcpack-sphinx-addon==0.1.3 synr==0.2.1
+python3 -m pip install --user tokenizers==0.9.4 transformers==4.0.1
Review comment:
Ok, I think I have this figured out. This tutorial depends on transformers, but because the import was local to the download function (which wasn't run), we didn't hit it. That means that we need to install transformers in the ci_script (like you had done before). However, this model is pretty large (500MB), so I don't think we want to be downloading it for every run on CI. Do you have a small sparse model we could use instead? If not, I suggest we leave the finally running of the script commented out and let the user run it if they so choose.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ANSHUMAN87 commented on a change in pull request #7107: [Tutorial] Add output validation to sparse tutorial
Posted by GitBox <gi...@apache.org>.
ANSHUMAN87 commented on a change in pull request #7107:
URL: https://github.com/apache/tvm/pull/7107#discussion_r543532354
##########
File path: tutorials/frontend/deploy_sparse.py
##########
@@ -119,6 +117,9 @@
# determines how sparse the generated weights should be. The higher
# the sparsity, the faster the result.
sparsity = 0.85
+# Running benchmarking mode might overload CI,
+# so it is disabled by default.
+benchmark = False
Review comment:
Done!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] jroesch closed pull request #7107: [Tutorial] Add output validation to sparse tutorial
Posted by GitBox <gi...@apache.org>.
jroesch closed pull request #7107:
URL: https://github.com/apache/tvm/pull/7107
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] giuseros commented on pull request #7107: [Tutorial] Enabling sparse tutorial
Posted by GitBox <gi...@apache.org>.
giuseros commented on pull request #7107:
URL: https://github.com/apache/tvm/pull/7107#issuecomment-745173619
Hi @ANSHUMAN87 ,
maybe you could change the title of this PR in "Adding verification steps in sparse tutorial"? And maybe add a bit of description in the commit message?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tkonolige commented on a change in pull request #7107: [Tutorial] Add output validation to sparse tutorial
Posted by GitBox <gi...@apache.org>.
tkonolige commented on a change in pull request #7107:
URL: https://github.com/apache/tvm/pull/7107#discussion_r546840492
##########
File path: tests/scripts/task_ci_python_setup.sh
##########
@@ -31,3 +31,4 @@ set -o pipefail
echo "Addtiional setup in" ${CI_IMAGE_NAME}
python3 -m pip install --user tlcpack-sphinx-addon==0.1.3 synr==0.2.1
+python3 -m pip install --user tokenizers==0.9.4 transformers==4.0.1
Review comment:
I'm not sure which commit was broken. Can you remove these and then we can debug?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ANSHUMAN87 commented on a change in pull request #7107: [Tutorial] Add output validation to sparse tutorial
Posted by GitBox <gi...@apache.org>.
ANSHUMAN87 commented on a change in pull request #7107:
URL: https://github.com/apache/tvm/pull/7107#discussion_r546197218
##########
File path: tutorials/frontend/deploy_sparse.py
##########
@@ -119,6 +117,9 @@
# determines how sparse the generated weights should be. The higher
# the sparsity, the faster the result.
sparsity = 0.85
+# Running benchmarking mode might overload CI,
+# so it is disabled by default.
+benchmark = False
Review comment:
Done!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ANSHUMAN87 commented on a change in pull request #7107: [Tutorial] Enabling sparse tutorial
Posted by GitBox <gi...@apache.org>.
ANSHUMAN87 commented on a change in pull request #7107:
URL: https://github.com/apache/tvm/pull/7107#discussion_r542585227
##########
File path: tutorials/frontend/deploy_sparse.py
##########
@@ -314,13 +310,20 @@ def run_sparse(mod, params, shape_dict, target, ctx, bs_r, sparsity, gen_weights
# you'll need to uncomment the last line first.
def benchmark():
mod, params, shape_dict = import_graphdef(name, batch_size, seq_len)
- run_dense(mod, params, shape_dict, target, ctx)
+ input_shape = shape_dict["input_1"]
+ dummy_data = np.random.uniform(size=input_shape, low=0, high=input_shape[1]).astype("int32")
+ dense_output = run_dense(mod, params, shape_dict, target, ctx, dummy_data)
if measure_sparse:
gen_weights = "prune" not in name
- run_sparse(mod, params, shape_dict, target, ctx, bs_r, sparsity, gen_weights)
+ sparse_output = run_sparse(
+ mod, params, shape_dict, target, ctx, bs_r, sparsity, gen_weights, dummy_data
+ )
+ np.testing.assert_allclose(
+ dense_output, sparse_output, equal_nan=True, verbose=True, atol=1e-5, rtol=1e-5
+ )
-# benchmark()
+benchmark()
Review comment:
Got your point. Perhaps we disable the benchmarking part by default and compare only the output!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tkonolige commented on a change in pull request #7107: [Tutorial] Add output validation to sparse tutorial
Posted by GitBox <gi...@apache.org>.
tkonolige commented on a change in pull request #7107:
URL: https://github.com/apache/tvm/pull/7107#discussion_r545294924
##########
File path: tutorials/frontend/deploy_sparse.py
##########
@@ -119,6 +117,9 @@
# determines how sparse the generated weights should be. The higher
# the sparsity, the faster the result.
sparsity = 0.85
+# Running benchmarking mode might overload CI,
+# so it is disabled by default.
+benchmark = False
Review comment:
Just benchmark
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tkonolige commented on a change in pull request #7107: [Tutorial] Add output validation to sparse tutorial
Posted by GitBox <gi...@apache.org>.
tkonolige commented on a change in pull request #7107:
URL: https://github.com/apache/tvm/pull/7107#discussion_r544439232
##########
File path: tutorials/frontend/deploy_sparse.py
##########
@@ -119,6 +117,9 @@
# determines how sparse the generated weights should be. The higher
# the sparsity, the faster the result.
sparsity = 0.85
+# Running benchmarking mode might overload CI,
+# so it is disabled by default.
+benchmark = False
Review comment:
I think this should be a more explicit to the user. I can see people skipping over this. How about "Change this to to true to run the benchmark." Also, could you move it to the end of the script so it makes logical sense to someone reading it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ANSHUMAN87 commented on a change in pull request #7107: [Tutorial] Add output validation to sparse tutorial
Posted by GitBox <gi...@apache.org>.
ANSHUMAN87 commented on a change in pull request #7107:
URL: https://github.com/apache/tvm/pull/7107#discussion_r547091469
##########
File path: tests/scripts/task_ci_python_setup.sh
##########
@@ -31,3 +31,4 @@ set -o pipefail
echo "Addtiional setup in" ${CI_IMAGE_NAME}
python3 -m pip install --user tlcpack-sphinx-addon==0.1.3 synr==0.2.1
+python3 -m pip install --user tokenizers==0.9.4 transformers==4.0.1
Review comment:
Okay will revert this change and check the error.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ANSHUMAN87 commented on a change in pull request #7107: [Tutorial] Add output validation to sparse tutorial
Posted by GitBox <gi...@apache.org>.
ANSHUMAN87 commented on a change in pull request #7107:
URL: https://github.com/apache/tvm/pull/7107#discussion_r543532538
##########
File path: tutorials/frontend/deploy_sparse.py
##########
@@ -312,15 +312,22 @@ def run_sparse(mod, params, shape_dict, target, ctx, bs_r, sparsity, gen_weights
# And that's it! Now we'll simply call all the needed function to benchmark
Review comment:
Done!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] ANSHUMAN87 commented on pull request #7107: [Tutorial] Add output validation to sparse tutorial
Posted by GitBox <gi...@apache.org>.
ANSHUMAN87 commented on pull request #7107:
URL: https://github.com/apache/tvm/pull/7107#issuecomment-745375596
Thanks @tkonolige , @giuseros !
I believe your comments are addressed now. Please have a look. Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tkonolige commented on a change in pull request #7107: [Tutorial] Enabling sparse tutorial
Posted by GitBox <gi...@apache.org>.
tkonolige commented on a change in pull request #7107:
URL: https://github.com/apache/tvm/pull/7107#discussion_r542582066
##########
File path: tutorials/frontend/deploy_sparse.py
##########
@@ -314,13 +310,20 @@ def run_sparse(mod, params, shape_dict, target, ctx, bs_r, sparsity, gen_weights
# you'll need to uncomment the last line first.
def benchmark():
mod, params, shape_dict = import_graphdef(name, batch_size, seq_len)
- run_dense(mod, params, shape_dict, target, ctx)
+ input_shape = shape_dict["input_1"]
+ dummy_data = np.random.uniform(size=input_shape, low=0, high=input_shape[1]).astype("int32")
+ dense_output = run_dense(mod, params, shape_dict, target, ctx, dummy_data)
if measure_sparse:
gen_weights = "prune" not in name
- run_sparse(mod, params, shape_dict, target, ctx, bs_r, sparsity, gen_weights)
+ sparse_output = run_sparse(
+ mod, params, shape_dict, target, ctx, bs_r, sparsity, gen_weights, dummy_data
+ )
+ np.testing.assert_allclose(
+ dense_output, sparse_output, equal_nan=True, verbose=True, atol=1e-5, rtol=1e-5
+ )
-# benchmark()
+benchmark()
Review comment:
This should remain commented out so we don't spend our CI time on benchmarking.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tkonolige commented on a change in pull request #7107: [Tutorial] Add output validation to sparse tutorial
Posted by GitBox <gi...@apache.org>.
tkonolige commented on a change in pull request #7107:
URL: https://github.com/apache/tvm/pull/7107#discussion_r545271417
##########
File path: tutorials/frontend/deploy_sparse.py
##########
@@ -119,6 +117,9 @@
# determines how sparse the generated weights should be. The higher
# the sparsity, the faster the result.
sparsity = 0.85
+# Running benchmarking mode might overload CI,
+# so it is disabled by default.
+benchmark = False
Review comment:
All the other tutorials put it at the bottom, so we should be consistent. It also makes sense for the reader. The last step they want to do is run everything.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org