You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/06/06 05:49:57 UTC
[GitHub] [tvm] zxybazh opened a new pull request, #11587: [MetaSchedule] Add Testing Script with ONNX Support
zxybazh opened a new pull request, #11587:
URL: https://github.com/apache/tvm/pull/11587
This PR introduces 2 tuning script for meta schedule and auto scheduler tuning support with onnx files. Now we can easily introduce onnx models benchmarking with command line scripts. Sample tuning call looks similar to the following script
```
python3 -m tvm.meta_schedule.testing.tune_onnx_meta_schedule \
--model-name "$MODEL_NAME" \
--onnx-path "$ONNX_PATH" \
--input-shape "$INPUT_SHAPE" \
--target "$TARGET" \
--num-trials $NUM_TRIALS \
--rpc-host $RPC_HOST \
--rpc-port $RPC_PORT \
--rpc-key $RPC_KEY \
--rpc-workers $RPC_WORKERS \
--work-dir $WORK_DIR \
|& tee "$WORK_DIR/$MODEL_NAME.log"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zxybazh commented on a diff in pull request #11587: [MetaSchedule] Add Testing Script with ONNX Support
Posted by GitBox <gi...@apache.org>.
zxybazh commented on code in PR #11587:
URL: https://github.com/apache/tvm/pull/11587#discussion_r890807943
##########
python/tvm/meta_schedule/testing/tune_onnx_auto_scheduler.py:
##########
@@ -0,0 +1,238 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=missing-docstring
+import argparse
+import json
+import os
+
+import numpy as np # type: ignore
+import tvm
+from tvm.relay.frontend import from_onnx
+from tvm import auto_scheduler
+from tvm import meta_schedule as ms
+from tvm import relay
+from tvm.meta_schedule.testing.custom_builder_runner import run_module_via_rpc
+import onnx # type: ignore
+
+
+def _parse_args():
+ args = argparse.ArgumentParser()
+ args.add_argument(
+ "--model-name",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--onnx-path",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--input-shape",
+ type=str,
+ required=True,
+ # example: `[{"name": "input1", "dtype": "int64", "shape": [1, 1, 8]}]`
Review Comment:
Hi, do you mean add a file path for the config file or use dictionary string directly in CLI? In [this post](https://stackoverflow.com/questions/18608812/accepting-a-dictionary-as-an-argument-with-argparse-and-python) I found that latter one hard to implement maybe you are referring to add the config file path?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zxybazh commented on a diff in pull request #11587: [MetaSchedule] Add Testing Script with ONNX Support
Posted by GitBox <gi...@apache.org>.
zxybazh commented on code in PR #11587:
URL: https://github.com/apache/tvm/pull/11587#discussion_r890799018
##########
python/tvm/meta_schedule/testing/tune_onnx_auto_scheduler.py:
##########
@@ -0,0 +1,238 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=missing-docstring
+import argparse
+import json
+import os
+
+import numpy as np # type: ignore
+import tvm
+from tvm.relay.frontend import from_onnx
+from tvm import auto_scheduler
+from tvm import meta_schedule as ms
+from tvm import relay
+from tvm.meta_schedule.testing.custom_builder_runner import run_module_via_rpc
+import onnx # type: ignore
+
+
+def _parse_args():
+ args = argparse.ArgumentParser()
+ args.add_argument(
+ "--model-name",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--onnx-path",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--input-shape",
+ type=str,
+ required=True,
+ # example: `[{"name": "input1", "dtype": "int64", "shape": [1, 1, 8]}]`
+ )
+ args.add_argument(
+ "--target",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--num-trials",
+ type=int,
+ required=True,
+ )
+ args.add_argument(
+ "--rpc-host",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--rpc-port",
+ type=int,
+ required=True,
+ )
+ args.add_argument(
+ "--rpc-key",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--rpc-workers",
+ type=int,
+ required=True,
+ )
+ args.add_argument(
+ "--log-dir",
Review Comment:
Would be great if we have a consistent API, this is because our `tune_relay_auto_scheduler.py` uses `log-dir` instead of `work-dir`. Should be migrate all of them to `work-dir` instead?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao1994 commented on a diff in pull request #11587: [MetaSchedule] Add Testing Script with ONNX Support
Posted by GitBox <gi...@apache.org>.
junrushao1994 commented on code in PR #11587:
URL: https://github.com/apache/tvm/pull/11587#discussion_r890814882
##########
python/tvm/meta_schedule/testing/tune_onnx_auto_scheduler.py:
##########
@@ -0,0 +1,238 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=missing-docstring
+import argparse
+import json
+import os
+
+import numpy as np # type: ignore
+import tvm
+from tvm.relay.frontend import from_onnx
+from tvm import auto_scheduler
+from tvm import meta_schedule as ms
+from tvm import relay
+from tvm.meta_schedule.testing.custom_builder_runner import run_module_via_rpc
+import onnx # type: ignore
+
+
+def _parse_args():
+ args = argparse.ArgumentParser()
+ args.add_argument(
+ "--model-name",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--onnx-path",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--input-shape",
+ type=str,
+ required=True,
+ # example: `[{"name": "input1", "dtype": "int64", "shape": [1, 1, 8]}]`
Review Comment:
Add a help message. For example:
```python
parser.add_argument('integers', metavar='N', type=int, nargs='+',
help='an integer for the accumulator') # <==== this
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao1994 commented on a diff in pull request #11587: [MetaSchedule] Add Testing Script with ONNX Support
Posted by GitBox <gi...@apache.org>.
junrushao1994 commented on code in PR #11587:
URL: https://github.com/apache/tvm/pull/11587#discussion_r889896250
##########
python/tvm/meta_schedule/testing/tune_onnx_auto_scheduler.py:
##########
@@ -0,0 +1,238 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=missing-docstring
+import argparse
+import json
+import os
+
+import numpy as np # type: ignore
+import tvm
+from tvm.relay.frontend import from_onnx
+from tvm import auto_scheduler
+from tvm import meta_schedule as ms
+from tvm import relay
+from tvm.meta_schedule.testing.custom_builder_runner import run_module_via_rpc
+import onnx # type: ignore
+
+
+def _parse_args():
+ args = argparse.ArgumentParser()
+ args.add_argument(
+ "--model-name",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--onnx-path",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--input-shape",
+ type=str,
+ required=True,
+ # example: `[{"name": "input1", "dtype": "int64", "shape": [1, 1, 8]}]`
+ )
+ args.add_argument(
+ "--target",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--num-trials",
+ type=int,
+ required=True,
+ )
+ args.add_argument(
+ "--rpc-host",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--rpc-port",
+ type=int,
+ required=True,
+ )
+ args.add_argument(
+ "--rpc-key",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--rpc-workers",
+ type=int,
+ required=True,
+ )
+ args.add_argument(
+ "--log-dir",
Review Comment:
Shall we switch to `--work-dir` instead to be consistent with our `tune_relay` API?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao1994 merged pull request #11587: [MetaSchedule] Add Testing Script with ONNX Support
Posted by GitBox <gi...@apache.org>.
junrushao1994 merged PR #11587:
URL: https://github.com/apache/tvm/pull/11587
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao1994 commented on a diff in pull request #11587: [MetaSchedule] Add Testing Script with ONNX Support
Posted by GitBox <gi...@apache.org>.
junrushao1994 commented on code in PR #11587:
URL: https://github.com/apache/tvm/pull/11587#discussion_r889896788
##########
python/tvm/meta_schedule/testing/tune_onnx_auto_scheduler.py:
##########
@@ -0,0 +1,238 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=missing-docstring
+import argparse
+import json
+import os
+
+import numpy as np # type: ignore
+import tvm
+from tvm.relay.frontend import from_onnx
+from tvm import auto_scheduler
+from tvm import meta_schedule as ms
+from tvm import relay
+from tvm.meta_schedule.testing.custom_builder_runner import run_module_via_rpc
+import onnx # type: ignore
+
+
+def _parse_args():
+ args = argparse.ArgumentParser()
+ args.add_argument(
+ "--model-name",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--onnx-path",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--input-shape",
+ type=str,
+ required=True,
+ # example: `[{"name": "input1", "dtype": "int64", "shape": [1, 1, 8]}]`
Review Comment:
argparse actually allows to add documents in CLI, so you may use argparse API explicitly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zxybazh commented on a diff in pull request #11587: [MetaSchedule] Add Testing Script with ONNX Support
Posted by GitBox <gi...@apache.org>.
zxybazh commented on code in PR #11587:
URL: https://github.com/apache/tvm/pull/11587#discussion_r890799018
##########
python/tvm/meta_schedule/testing/tune_onnx_auto_scheduler.py:
##########
@@ -0,0 +1,238 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=missing-docstring
+import argparse
+import json
+import os
+
+import numpy as np # type: ignore
+import tvm
+from tvm.relay.frontend import from_onnx
+from tvm import auto_scheduler
+from tvm import meta_schedule as ms
+from tvm import relay
+from tvm.meta_schedule.testing.custom_builder_runner import run_module_via_rpc
+import onnx # type: ignore
+
+
+def _parse_args():
+ args = argparse.ArgumentParser()
+ args.add_argument(
+ "--model-name",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--onnx-path",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--input-shape",
+ type=str,
+ required=True,
+ # example: `[{"name": "input1", "dtype": "int64", "shape": [1, 1, 8]}]`
+ )
+ args.add_argument(
+ "--target",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--num-trials",
+ type=int,
+ required=True,
+ )
+ args.add_argument(
+ "--rpc-host",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--rpc-port",
+ type=int,
+ required=True,
+ )
+ args.add_argument(
+ "--rpc-key",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--rpc-workers",
+ type=int,
+ required=True,
+ )
+ args.add_argument(
+ "--log-dir",
Review Comment:
Would be great if we have a consistent API, this is because our `tune_relay_auto_scheduler.py` uses `log-dir` instead of `work-dir`. Should we migrate all of them to `work-dir` instead?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao1994 commented on a diff in pull request #11587: [MetaSchedule] Add Testing Script with ONNX Support
Posted by GitBox <gi...@apache.org>.
junrushao1994 commented on code in PR #11587:
URL: https://github.com/apache/tvm/pull/11587#discussion_r890814275
##########
python/tvm/meta_schedule/testing/tune_onnx_auto_scheduler.py:
##########
@@ -0,0 +1,238 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=missing-docstring
+import argparse
+import json
+import os
+
+import numpy as np # type: ignore
+import tvm
+from tvm.relay.frontend import from_onnx
+from tvm import auto_scheduler
+from tvm import meta_schedule as ms
+from tvm import relay
+from tvm.meta_schedule.testing.custom_builder_runner import run_module_via_rpc
+import onnx # type: ignore
+
+
+def _parse_args():
+ args = argparse.ArgumentParser()
+ args.add_argument(
+ "--model-name",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--onnx-path",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--input-shape",
+ type=str,
+ required=True,
+ # example: `[{"name": "input1", "dtype": "int64", "shape": [1, 1, 8]}]`
+ )
+ args.add_argument(
+ "--target",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--num-trials",
+ type=int,
+ required=True,
+ )
+ args.add_argument(
+ "--rpc-host",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--rpc-port",
+ type=int,
+ required=True,
+ )
+ args.add_argument(
+ "--rpc-key",
+ type=str,
+ required=True,
+ )
+ args.add_argument(
+ "--rpc-workers",
+ type=int,
+ required=True,
+ )
+ args.add_argument(
+ "--log-dir",
Review Comment:
Agreed. Let's migrate to work-dir for all existing scripts!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao1994 commented on pull request #11587: [MetaSchedule] Add Testing Script with ONNX Support
Posted by GitBox <gi...@apache.org>.
junrushao1994 commented on PR #11587:
URL: https://github.com/apache/tvm/pull/11587#issuecomment-1149005726
Thanks @zxybazh! This is super helpful to have for tuning ONNX from command line
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org