You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2021/05/18 05:08:05 UTC
[GitHub] [tvm] zackcquic opened a new pull request #7952: [IR][Pass][Instrument] Pass instrument framework
zackcquic opened a new pull request #7952:
URL: https://github.com/apache/tvm/pull/7952
This commit provides utilities to instrument passes:
1. Add a new namespace tvm.instrument
2. Introduce PassInstrument and PassInstrumentor to PassContext
Example
---------
passes_mem = #... Impl of memory instrument
passes_time = tvm.instrument.PassesTimeInstrument()
with tvm.transform.PassContext(
pass_instrumentor=PassInstrumentor([passes_mem, passes_time])):
tvm.relay.build(mod, 'llvm')
passes_mem.rendor()
passes_time.rendor()
3. Integrate existing PassContext::Trace() and timing profile
Thanks for contributing to TVM! Please refer to guideline https://tvm.apache.org/docs/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from [Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers) by @ them in the pull request thread.
Hi @altanh @tqchen:
I tried to integrate current passes profile mechanisms and make it more extendable, usage is as the commit's code example. Many parts are inspired by LLVM and MLIR.
How do you think?
This is my first attempt to TVM :), I have read through the guideline, but it there are stilling something wrong, please let me know.
Regards,
Zack
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r634818031
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-840237999
@tqchen Thanks a lot for review.
I will change it in these days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
areusch commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r635408096
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired by LLVM and MLIR.
+ * It inserts instrumentation points around passes.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrumentation points.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp()
+ *
+ * if (Instrument Before Pass())
+ * Pass1()
+ * Instrument After Pass()
+ *
+ * if (Instrument Before Pass())
+ * Pass2()
+ * Instrument After Pass()
+ *
+ * Instrument TearDown()
+ *
+ *
+ * The `Before Pass` instrumentation point can selectively disable passes by returning true (to
+ * enable) or false (to disable).
+ *
+ * If there are multiple pass instrumentations provided, `Before Pass` callbacks are applied in
+ * order. If one return false, then the pass will be skipped:
+ *
+ * for (auto pi : PassInstruments)
+ * if (pi->BeforePass())
+ * return False // Disable pass
+ *
+ * return True // All ok, enable pass
+ *
+ *
+ * \sa PassInstrument
+ * \sa PassContextNode::InstrumentBeforePass()
+ * \sa src/ir/transform.cc
+ */
+class PassInstrumentNode : public Object {
+ public:
+ virtual ~PassInstrumentNode() {}
+
+ /*! \brief Set up environment for instrumentation. */
+ virtual void SetUp() const = 0;
+
+ /*! \brief Clean up instrumentation environment. */
Review comment:
also specify here this runs once after all passes run. also, specify what will happen in these cases:
1. when SetUp has been called but a following SetUp (e.g. on another instrument) raises an exception
2. when an exception is raised in the course of running passes
it would also be great to add unit tests for these cases
##########
File path: python/tvm/ir/instrument.py
##########
@@ -0,0 +1,154 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=invalid-name,unused-argument
+"""Common pass instrumentation across IR variants."""
+import inspect
+import functools
+
+import tvm._ffi
+import tvm.runtime
+
+from . import _ffi_instrument_api
+
+
+@tvm._ffi.register_object("instrument.PassInstrument")
+class PassInstrument(tvm.runtime.Object):
+ """A pass instrument implementation.
+
+ Users don't need to interact with this class directly.
+ Instead, a `PassInstrument` instance should be created through `pass_instrument`.
+
+ See Also
+ --------
+ `pass_instrument`
+ """
+
+
+def _wrap_class_pass_instrument(pi_cls):
+ """Wrap a python class as pass instrument"""
+
+ class PyPassInstrument(PassInstrument):
+ """Internal wrapper class to create a class instance."""
+
+ def __init__(self, *args, **kwargs):
+ # initialize handle in cass pi_cls creation failed.fg
+ self.handle = None
+ inst = pi_cls(*args, **kwargs)
+
+ # check method declartion within class, if found, wrap it.
+ def create_method(method):
+ if hasattr(inst, method) and inspect.ismethod(getattr(inst, method)):
+
+ def func(*args):
+ return getattr(inst, method)(*args)
+
+ func.__name__ = "_" + method
+ return func
+ return None
+
+ # create runtime pass instrument object
+ # reister instance's run_before_pass, run_after_pass, set_up and tear_down method
+ # to it if present.
+ self.__init_handle_by_constructor__(
+ _ffi_instrument_api.NamedPassInstrument,
+ pi_cls.__name__,
+ create_method("run_before_pass"),
+ create_method("run_after_pass"),
+ create_method("set_up"),
+ create_method("tear_down"),
+ )
+
+ self._inst = inst
+
+ def __getattr__(self, name):
+ # fall back to instance attribute if there is not any
+ return self._inst.__getattribute__(name)
+
+ functools.update_wrapper(PyPassInstrument.__init__, pi_cls.__init__)
Review comment:
why do we need the metadata logic? can't we just pass a PackedFunc to the C++ FFI that binds to e.g. `self.run_before_pass`? Python captures `self` in that function pointer I believe.
##########
File path: python/tvm/ir/instrument.py
##########
@@ -0,0 +1,154 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=invalid-name,unused-argument
+"""Common pass instrumentation across IR variants."""
+import inspect
+import functools
+
+import tvm._ffi
+import tvm.runtime
+
+from . import _ffi_instrument_api
+
+
+@tvm._ffi.register_object("instrument.PassInstrument")
+class PassInstrument(tvm.runtime.Object):
+ """A pass instrument implementation.
+
+ Users don't need to interact with this class directly.
+ Instead, a `PassInstrument` instance should be created through `pass_instrument`.
+
+ See Also
+ --------
+ `pass_instrument`
+ """
+
+
+def _wrap_class_pass_instrument(pi_cls):
+ """Wrap a python class as pass instrument"""
+
+ class PyPassInstrument(PassInstrument):
+ """Internal wrapper class to create a class instance."""
+
+ def __init__(self, *args, **kwargs):
+ # initialize handle in cass pi_cls creation failed.fg
Review comment:
nit: remove fg
##########
File path: include/tvm/ir/transform.h
##########
@@ -95,8 +87,9 @@ class PassContextNode : public Object {
mutable Optional<DiagnosticContext> diag_ctx;
/*! \brief Pass specific configurations. */
Map<String, ObjectRef> config;
- /*! \brief Trace function to be invoked before and after each pass. */
- TraceFunc trace_func;
+
+ /*! \brief A list of pass instrument implementations. */
Review comment:
specify the order
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired by LLVM and MLIR.
+ * It inserts instrumentation points around passes.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrumentation points.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp()
+ *
+ * if (Instrument Before Pass())
Review comment:
i think it would be great to separate the before-pass setup from the decision of "should this pass run?" The reason is that this removes concerns about how the ordering of pass instrumentation may change the number of calls to a given instrument. For example, if you just want to log all of the passes executed, you need to make sure that it's the first pass instrument registered.
I'd suggest this:
```
should_run_pass = [pi.should_run(pass) for pi in pass_instruments]
if all(should_run_pass):
for pi in pass_instruments:
pi.before_pass(pass)
pass()
for pi in pass_instruments:
pi.after_pass(pass)
```
additionally, it would be great to document what happens if any of the non-pass logic raises an error here. the Python context manager framework should provide some good inspiration for this. it seems like it may be helpful to provide the after_pass with a "result" notion. also, can you add tests for these cases?
##########
File path: src/ir/instrument.cc
##########
@@ -0,0 +1,314 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file src/ir/instrument.cc
+ * \brief Infrastructure for instrumentation.
+ */
+#include <dmlc/thread_local.h>
+#include <tvm/ir/instrument.h>
+#include <tvm/ir/transform.h>
+#include <tvm/node/repr_printer.h>
+#include <tvm/runtime/registry.h>
+
+#include <stack>
+
+namespace tvm {
+namespace instrument {
+
+/*!
+ * \brief A named PassInstrument implementation
Review comment:
State the meaning of "named" here: "A PassInstrument implementation which also carries a human-readable name"
Why don't all PassInstruments have names? Seems impossible to refer to them in the output otherwise.
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired by LLVM and MLIR.
+ * It inserts instrumentation points around passes.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrumentation points.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp()
+ *
+ * if (Instrument Before Pass())
+ * Pass1()
+ * Instrument After Pass()
+ *
+ * if (Instrument Before Pass())
+ * Pass2()
+ * Instrument After Pass()
+ *
+ * Instrument TearDown()
+ *
+ *
+ * The `Before Pass` instrumentation point can selectively disable passes by returning true (to
+ * enable) or false (to disable).
+ *
+ * If there are multiple pass instrumentations provided, `Before Pass` callbacks are applied in
+ * order. If one return false, then the pass will be skipped:
+ *
+ * for (auto pi : PassInstruments)
+ * if (pi->BeforePass())
+ * return False // Disable pass
+ *
+ * return True // All ok, enable pass
+ *
+ *
+ * \sa PassInstrument
+ * \sa PassContextNode::InstrumentBeforePass()
+ * \sa src/ir/transform.cc
+ */
+class PassInstrumentNode : public Object {
+ public:
+ virtual ~PassInstrumentNode() {}
+
+ /*! \brief Set up environment for instrumentation. */
Review comment:
also specify here that this runs once before all passes run
##########
File path: python/tvm/ir/instrument.py
##########
@@ -0,0 +1,154 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=invalid-name,unused-argument
+"""Common pass instrumentation across IR variants."""
+import inspect
+import functools
+
+import tvm._ffi
+import tvm.runtime
+
+from . import _ffi_instrument_api
+
+
+@tvm._ffi.register_object("instrument.PassInstrument")
+class PassInstrument(tvm.runtime.Object):
+ """A pass instrument implementation.
+
+ Users don't need to interact with this class directly.
+ Instead, a `PassInstrument` instance should be created through `pass_instrument`.
+
+ See Also
+ --------
+ `pass_instrument`
+ """
+
+
+def _wrap_class_pass_instrument(pi_cls):
+ """Wrap a python class as pass instrument"""
+
+ class PyPassInstrument(PassInstrument):
+ """Internal wrapper class to create a class instance."""
+
+ def __init__(self, *args, **kwargs):
+ # initialize handle in cass pi_cls creation failed.fg
+ self.handle = None
+ inst = pi_cls(*args, **kwargs)
+
+ # check method declartion within class, if found, wrap it.
+ def create_method(method):
+ if hasattr(inst, method) and inspect.ismethod(getattr(inst, method)):
+
+ def func(*args):
+ return getattr(inst, method)(*args)
+
+ func.__name__ = "_" + method
+ return func
+ return None
+
+ # create runtime pass instrument object
+ # reister instance's run_before_pass, run_after_pass, set_up and tear_down method
+ # to it if present.
+ self.__init_handle_by_constructor__(
Review comment:
due to an issue with how pytest interacts with Node classes, it's preferable that this is the first thing in `__init__`. if anything between line 47 and here raises an exception, pytest will cause an infinite recursion rather than printing the actual error. the constraint is that `self.handle` needs to be initialized.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-850093392
Hi @tqchen,
Just a ping for reviewing :), thanks a lot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic removed a comment on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic removed a comment on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-845250654
Hi @areusch:
Could you elaborate more on "after_pass with a result notation"?
What kind of result and use scenario you expected?
That will be very helpful for me. Thanks a lot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic edited a comment on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic edited a comment on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-832372339
Hi @tqchen @jroesch
The CI failed, because of the following message:
```
docker: write /var/lib/docker/tmp/GetImageBlob707223609: no space left on device.
See 'docker run --help'.
script returned exit code 125
```
I think it may be not related to the change but the CI environment, right?
And another thing is the workflow:
```
1 workflow awaiting approval
First-time contributors need a maintainer to approve running workflows
```
I think it may require someone to approve, right?
Or I need to do something else?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-846538241
> > For the reason not to overload superclass, I don't know, but I have a guess, maybe need someone to confirm:
> > Currently, only FunctionPass, ModulePass, SequentialPass and PrimFuncPass extends Pass.
> > FunctionPass calls InferType at the end (see the comments for detail) while SequentialPass is just like a wrapper.
> > In these cases, InterType and SequentialPass maybe not want to be traced/instrumented.
>
> I disagree with this, mainly because some (perhaps not so) surprising results I got with my initial pass profiling implementation is that `InferType` is called **a lot** and often due to the `FunctionPass`/`SequentialPass` design, leading to quite a bit of compilation slowdown. This information would not have been obtained if we didn't include all passes that are run in the profiling.
>
> Now, I wonder if we should introduce a "universal" pass context which _every_ PassNode must respect and has access to (perhaps a thread local one)? I definitely want to make sure the timing instrument can time all passes that are run.
@altanh Thanks.
Just checked, YES, ```InferType``` is called **a lot** than I expected.
So I updated like you suggested, instrument in Pass::operator() instead of subclasses' operator().
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r637521086
##########
File path: python/tvm/ir/instrument.py
##########
@@ -0,0 +1,154 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=invalid-name,unused-argument
+"""Common pass instrumentation across IR variants."""
+import inspect
+import functools
+
+import tvm._ffi
+import tvm.runtime
+
+from . import _ffi_instrument_api
+
+
+@tvm._ffi.register_object("instrument.PassInstrument")
+class PassInstrument(tvm.runtime.Object):
+ """A pass instrument implementation.
+
+ Users don't need to interact with this class directly.
+ Instead, a `PassInstrument` instance should be created through `pass_instrument`.
+
+ See Also
+ --------
+ `pass_instrument`
+ """
+
+
+def _wrap_class_pass_instrument(pi_cls):
+ """Wrap a python class as pass instrument"""
+
+ class PyPassInstrument(PassInstrument):
+ """Internal wrapper class to create a class instance."""
+
+ def __init__(self, *args, **kwargs):
+ # initialize handle in cass pi_cls creation failed.fg
+ self.handle = None
+ inst = pi_cls(*args, **kwargs)
+
+ # check method declartion within class, if found, wrap it.
+ def create_method(method):
+ if hasattr(inst, method) and inspect.ismethod(getattr(inst, method)):
+
+ def func(*args):
+ return getattr(inst, method)(*args)
+
+ func.__name__ = "_" + method
+ return func
+ return None
+
+ # create runtime pass instrument object
+ # reister instance's run_before_pass, run_after_pass, set_up and tear_down method
+ # to it if present.
+ self.__init_handle_by_constructor__(
Review comment:
Yes, self.handle is initialized at line 49.
This wrapper function mimics ```_wrap_class_module_pass``` [transform.py:188](https://github.com/apache/tvm/blob/8934d6b5af1defc6ccd9e18fc26dac1a27f4546b/python/tvm/ir/transform.py#L188)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-848374711
@areusch @tqchen changes are done.
Thanks a lot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tkonolige commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
tkonolige commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-844369810
I agree that following precedent is useful for consistency, but it seems like it will cause a lot of problems here. What happens in this case:
The first instrument run before the pass initializes some state and returns true. The second returns false. Now there is no matching after pass call for the first instrument. When the next before pass gets called, the instrument doesn't know if it is now more deeply nested or if it had actually been canceled.
Also, given we already have a way to disable passes, wouldn't it make more sense to keep the existing interface?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tqchen commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r631011618
##########
File path: python/tvm/ir/instrument.py
##########
@@ -0,0 +1,128 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=invalid-name,unused-argument
+"""Common pass instrumentation across IR variants."""
+import tvm._ffi
+import tvm.runtime
+
+from . import _ffi_instrument_api
+
+
+@tvm._ffi.register_object("instrument.PassInstrument")
+class PassInstrument(tvm.runtime.Object):
Review comment:
Would be great to make it more python style(one decorator that registers all functions), see tricks here https://github.com/apache/tvm/blob/main/python/tvm/ir/transform.py#L253
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief A callback type for set up or clean up instrument environment.
+ */
+using InstrumentEnvFunc = runtime::TypedPackedFunc<void()>;
+
+/*!
+ * \brief A callback template for instrumenting before/after environment.
+ * \tparam RetTy the return type of callback.
+ */
+template <typename RetTy = void>
+using PassInstrumentFunc =
+ runtime::TypedPackedFunc<RetTy(const IRModule&, const transform::PassInfo&)>;
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
+ /*! \brief Name of this pass instrument object. */
+ String name;
+
+ /*! \brief Callback for instrumentation environment set up. */
+ InstrumentEnvFunc set_up_callback;
+ /*! \brief Callback for instrumentation environment clean up. */
+ InstrumentEnvFunc tear_down_callback;
+
+ /*! \brief Callback to run before a pass. */
+ PassInstrumentFunc</* RetTy */ bool> run_before_pass_callback;
Review comment:
would be good to directly inline the typed packedfunc as they are not too long
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief A callback type for set up or clean up instrument environment.
+ */
+using InstrumentEnvFunc = runtime::TypedPackedFunc<void()>;
+
+/*!
+ * \brief A callback template for instrumenting before/after environment.
+ * \tparam RetTy the return type of callback.
+ */
+template <typename RetTy = void>
+using PassInstrumentFunc =
+ runtime::TypedPackedFunc<RetTy(const IRModule&, const transform::PassInfo&)>;
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
+ /*! \brief Name of this pass instrument object. */
+ String name;
+
+ /*! \brief Callback for instrumentation environment set up. */
+ InstrumentEnvFunc set_up_callback;
+ /*! \brief Callback for instrumentation environment clean up. */
+ InstrumentEnvFunc tear_down_callback;
+
+ /*! \brief Callback to run before a pass. */
+ PassInstrumentFunc</* RetTy */ bool> run_before_pass_callback;
+ /*! \brief Callback to run after a pass. */
+ PassInstrumentFunc<> run_after_pass_callback;
+
+ /*!
+ * \brief Register a callback to run at set up point.
+ *
+ * \param callback The set up function.
+ */
+ void RegisterSetUpCallback(InstrumentEnvFunc callback) { set_up_callback = std::move(callback); }
+
+ /*
+ * \brief Register a callback to run at clean up point.
+ *
+ * \param callback The clean up function.
+ */
+ void RegisterTearDownCallback(InstrumentEnvFunc callback) {
+ tear_down_callback = std::move(callback);
+ }
+
+ /*!
+ * \brief Register a callback to run before pass run.
+ *
+ * \param callback The function to run before pass: return false to skip pass; return true to
+ * run pass.
+ */
+ void RegisterRunBeforePassCallback(PassInstrumentFunc<bool> callback) {
+ run_before_pass_callback = std::move(callback);
+ }
+
+ /*!
+ * \brief Register a callback to run after pass run.
+ *
+ * \param callback The function to run after pass.
+ */
+ void RegisterRunAfterPassCallback(PassInstrumentFunc<> callback) {
+ run_after_pass_callback = std::move(callback);
+ }
+
+ void VisitAttrs(AttrVisitor* v) { v->Visit("name", &name); }
+
+ /*! \brief Set up environment for instrumentation. */
+ void SetUp() const;
+
+ /*! \brief Clean up instrumentation environment. */
+ void TearDown() const;
+
+ /*!
+ * \brief Instrument before pass run, determine whether to run the pass or not.
+ * \param mod The module that an optimization pass runs on.
+ * \param info The pass information.
+ *
+ * \return true to run the pass; false to skip the pass.
+ */
+ bool RunBeforePass(const IRModule& mod, const transform::PassInfo& info) const;
+
+ /*!
+ * \brief Instrument after pass run.
+ *
+ * \param mod The module that an optimization pass runs on.
+ * \param info The pass information.
+ */
+ void RunAfterPass(const IRModule& mod, const transform::PassInfo& info) const;
+
+ static constexpr const char* _type_key = "instrument.PassInstrument";
+ TVM_DECLARE_FINAL_OBJECT_INFO(PassInstrumentNode, Object);
+};
+
+/*!
+ * \brief Managed reference class for PassInstrumentNode
+ * \sa PassInstrumentNode
+ */
+class PassInstrument : public ObjectRef {
+ public:
+ /*!
+ * \brief Constructor
+ * \param name Name for this instrumentation.
+ */
+ TVM_DLL PassInstrument(String name);
+
+ /*!
+ * \brief mutable accessor.
+ * \return mutable access pointer.
+ */
+ PassInstrumentNode* operator->() {
+ ICHECK(get() != nullptr);
+ return static_cast<PassInstrumentNode*>(get_mutable());
+ }
+
+ TVM_DEFINE_OBJECT_REF_METHODS(PassInstrument, ObjectRef, PassInstrumentNode);
+};
+
+/*!
+ * \brief PassInstrumentorNode collects a set of PassInstrument implementations, invokes the
+ * implementations' methods at different instrument points.
+ * \sa PassInstrumentor
+ */
+class PassInstrumentorNode : public Object {
+ public:
+ Array<PassInstrument> pass_instruments;
+
+ void VisitAttrs(AttrVisitor* v) { v->Visit("pass_instruments", &pass_instruments); }
+
+ /*! \brief Set up environment for instrument implementations. */
+ void SetUp() const;
+
+ /*! \brief Clean up environment for instrument implementations. */
+ void TearDown() const;
Review comment:
not sure setup and teardown is needed in our case. We can do setup in constructor and teardown in destructors, so likely we can omit these two calls for now.
If we want to keep these two functions, we need to discuss when to call these two functions. Perhaps setup should be called, in PassContext::EnterWithScope and TearDown should be called during PassContext::ExitWithScope
##########
File path: python/tvm/ir/transform.py
##########
@@ -65,12 +65,20 @@ class PassContext(tvm.runtime.Object):
disabled_pass : Optional[Union[List[str], Set[str], Tuple[str]]]
The list of passes that are disabled.
+ pass_instrumentor : Optional[tvm.instrument.PassInstrumentor]
+ The pass instrumentor that collects pass instrument implementations
+
config : Optional[Dict[str, Object]]
Additional configurations for specific passes.
"""
def __init__(
- self, opt_level=2, required_pass=None, disabled_pass=None, trace=None, config=None
+ self,
+ opt_level=2,
+ required_pass=None,
+ disabled_pass=None,
+ pass_instrumentor=None,
Review comment:
would be good to delibrate on API namings, per https://tvm.apache.org/docs/contribute/code_review.html#deliberate-on-api-and-data-structures.
For example given we are already in PassCOntext, perhaps we can remove the prefix `pass_`
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief A callback type for set up or clean up instrument environment.
+ */
+using InstrumentEnvFunc = runtime::TypedPackedFunc<void()>;
+
+/*!
+ * \brief A callback template for instrumenting before/after environment.
+ * \tparam RetTy the return type of callback.
+ */
+template <typename RetTy = void>
+using PassInstrumentFunc =
+ runtime::TypedPackedFunc<RetTy(const IRModule&, const transform::PassInfo&)>;
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
Review comment:
These callbakcs can sit in implementation of instrument.cc, since from they are only needed for fronend FFIs to provide instructment class. See https://github.com/apache/tvm/blob/main/include/tvm/ir/transform.h#L300 for a similar example. We can make PassInstrumentor a virtual class which allows sub-class overloading in normal C++ style
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief A callback type for set up or clean up instrument environment.
+ */
+using InstrumentEnvFunc = runtime::TypedPackedFunc<void()>;
+
+/*!
+ * \brief A callback template for instrumenting before/after environment.
+ * \tparam RetTy the return type of callback.
+ */
+template <typename RetTy = void>
+using PassInstrumentFunc =
+ runtime::TypedPackedFunc<RetTy(const IRModule&, const transform::PassInfo&)>;
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
+ /*! \brief Name of this pass instrument object. */
+ String name;
+
+ /*! \brief Callback for instrumentation environment set up. */
+ InstrumentEnvFunc set_up_callback;
+ /*! \brief Callback for instrumentation environment clean up. */
+ InstrumentEnvFunc tear_down_callback;
+
+ /*! \brief Callback to run before a pass. */
+ PassInstrumentFunc</* RetTy */ bool> run_before_pass_callback;
+ /*! \brief Callback to run after a pass. */
+ PassInstrumentFunc<> run_after_pass_callback;
+
+ /*!
+ * \brief Register a callback to run at set up point.
+ *
+ * \param callback The set up function.
+ */
+ void RegisterSetUpCallback(InstrumentEnvFunc callback) { set_up_callback = std::move(callback); }
+
+ /*
+ * \brief Register a callback to run at clean up point.
+ *
+ * \param callback The clean up function.
+ */
+ void RegisterTearDownCallback(InstrumentEnvFunc callback) {
+ tear_down_callback = std::move(callback);
+ }
+
+ /*!
+ * \brief Register a callback to run before pass run.
+ *
+ * \param callback The function to run before pass: return false to skip pass; return true to
+ * run pass.
+ */
+ void RegisterRunBeforePassCallback(PassInstrumentFunc<bool> callback) {
+ run_before_pass_callback = std::move(callback);
+ }
+
+ /*!
+ * \brief Register a callback to run after pass run.
+ *
+ * \param callback The function to run after pass.
+ */
+ void RegisterRunAfterPassCallback(PassInstrumentFunc<> callback) {
+ run_after_pass_callback = std::move(callback);
+ }
+
+ void VisitAttrs(AttrVisitor* v) { v->Visit("name", &name); }
+
+ /*! \brief Set up environment for instrumentation. */
+ void SetUp() const;
+
+ /*! \brief Clean up instrumentation environment. */
+ void TearDown() const;
+
+ /*!
+ * \brief Instrument before pass run, determine whether to run the pass or not.
+ * \param mod The module that an optimization pass runs on.
+ * \param info The pass information.
+ *
+ * \return true to run the pass; false to skip the pass.
+ */
+ bool RunBeforePass(const IRModule& mod, const transform::PassInfo& info) const;
+
+ /*!
+ * \brief Instrument after pass run.
+ *
+ * \param mod The module that an optimization pass runs on.
+ * \param info The pass information.
+ */
+ void RunAfterPass(const IRModule& mod, const transform::PassInfo& info) const;
+
+ static constexpr const char* _type_key = "instrument.PassInstrument";
+ TVM_DECLARE_FINAL_OBJECT_INFO(PassInstrumentNode, Object);
+};
+
+/*!
+ * \brief Managed reference class for PassInstrumentNode
+ * \sa PassInstrumentNode
+ */
+class PassInstrument : public ObjectRef {
+ public:
+ /*!
+ * \brief Constructor
+ * \param name Name for this instrumentation.
+ */
+ TVM_DLL PassInstrument(String name);
+
+ /*!
+ * \brief mutable accessor.
+ * \return mutable access pointer.
+ */
+ PassInstrumentNode* operator->() {
+ ICHECK(get() != nullptr);
+ return static_cast<PassInstrumentNode*>(get_mutable());
+ }
+
+ TVM_DEFINE_OBJECT_REF_METHODS(PassInstrument, ObjectRef, PassInstrumentNode);
+};
+
+/*!
+ * \brief PassInstrumentorNode collects a set of PassInstrument implementations, invokes the
+ * implementations' methods at different instrument points.
+ * \sa PassInstrumentor
+ */
+class PassInstrumentorNode : public Object {
Review comment:
Assuming we will only have one kind of compositional pattern(list, shall we directly put `Array<PassInstrumentor>` in the PassContext?)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r637521199
##########
File path: python/tvm/ir/instrument.py
##########
@@ -0,0 +1,154 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=invalid-name,unused-argument
+"""Common pass instrumentation across IR variants."""
+import inspect
+import functools
+
+import tvm._ffi
+import tvm.runtime
+
+from . import _ffi_instrument_api
+
+
+@tvm._ffi.register_object("instrument.PassInstrument")
+class PassInstrument(tvm.runtime.Object):
+ """A pass instrument implementation.
+
+ Users don't need to interact with this class directly.
+ Instead, a `PassInstrument` instance should be created through `pass_instrument`.
+
+ See Also
+ --------
+ `pass_instrument`
+ """
+
+
+def _wrap_class_pass_instrument(pi_cls):
+ """Wrap a python class as pass instrument"""
+
+ class PyPassInstrument(PassInstrument):
+ """Internal wrapper class to create a class instance."""
+
+ def __init__(self, *args, **kwargs):
+ # initialize handle in cass pi_cls creation failed.fg
+ self.handle = None
+ inst = pi_cls(*args, **kwargs)
+
+ # check method declartion within class, if found, wrap it.
+ def create_method(method):
+ if hasattr(inst, method) and inspect.ismethod(getattr(inst, method)):
+
+ def func(*args):
+ return getattr(inst, method)(*args)
+
+ func.__name__ = "_" + method
+ return func
+ return None
+
+ # create runtime pass instrument object
+ # reister instance's run_before_pass, run_after_pass, set_up and tear_down method
+ # to it if present.
+ self.__init_handle_by_constructor__(
+ _ffi_instrument_api.NamedPassInstrument,
+ pi_cls.__name__,
+ create_method("run_before_pass"),
+ create_method("run_after_pass"),
+ create_method("set_up"),
+ create_method("tear_down"),
+ )
+
+ self._inst = inst
+
+ def __getattr__(self, name):
+ # fall back to instance attribute if there is not any
+ return self._inst.__getattribute__(name)
+
+ functools.update_wrapper(PyPassInstrument.__init__, pi_cls.__init__)
Review comment:
Just keep consistent with ```_wrap_class_module_pass```[transform.py:206](https://github.com/apache/tvm/blob/8934d6b5af1defc6ccd9e18fc26dac1a27f4546b/python/tvm/ir/transform.py#L206)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] altanh commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
altanh commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-847533611
> > Thanks for the changes, this looks much better now! Mostly spelling nits here, but also could you please add back the test for the `PassTimingInstrument`? Looks like it got removed here (previously was in `tests/python/relay/test_pass_profiler.py`).
>
> I renamed `test_pass_profiler.py` to `test_pass_instrument.py`, and `PassTimingInstrument` are tested in this file.
Thanks, sorry I missed it (looks like github collapsed the file and I scrolled too fast...)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
areusch commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r638351376
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired by LLVM and MLIR.
+ * It inserts instrumentation points around passes.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrumentation points.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp()
+ *
+ * if (Instrument Before Pass())
Review comment:
hi @zackcquic, here i'm referencing the python `__exit__` function signature. you can see it used in context as "exit" [here](https://www.python.org/dev/peps/pep-0343/#specification-the-with-statement). i'm speaking fairly broadly, though--in Python, sys.exc_info() captures any pending exception as a tuple (type, value, traceback) whereas in c++ the most we get is information from catching e.g. `std::runtime_error`. By "result notation," I mainly mean some appropriate argument to `after_pass` to indicate why the `after_pass` is being invoked--it could be because:
1. a following `before_pass` errored
2. pass() was invoked and threw an error
3. pass() was invoked and finished cleanly
4. a preceding after_pass errored
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r638401833
##########
File path: python/tvm/ir/transform.py
##########
@@ -92,6 +104,9 @@ def __enter__(self):
def __exit__(self, ptype, value, trace):
_ffi_transform_api.ExitPassContext(self)
+ def override_instruments(self, instruments):
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r637521526
##########
File path: src/ir/instrument.cc
##########
@@ -0,0 +1,314 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file src/ir/instrument.cc
+ * \brief Infrastructure for instrumentation.
+ */
+#include <dmlc/thread_local.h>
+#include <tvm/ir/instrument.h>
+#include <tvm/ir/transform.h>
+#include <tvm/node/repr_printer.h>
+#include <tvm/runtime/registry.h>
+
+#include <stack>
+
+namespace tvm {
+namespace instrument {
+
+/*!
+ * \brief A named PassInstrument implementation
Review comment:
Changed:
- Made "name" a member in abstract class ```PassInstrumentNode```.
- Implement``BasePassInstrument``` for FFI here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-849471838
@areusch @tqchen change request are all done.
Thanks a lot for reviewing efforts. 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r631681526
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief A callback type for set up or clean up instrument environment.
+ */
+using InstrumentEnvFunc = runtime::TypedPackedFunc<void()>;
+
+/*!
+ * \brief A callback template for instrumenting before/after environment.
+ * \tparam RetTy the return type of callback.
+ */
+template <typename RetTy = void>
+using PassInstrumentFunc =
+ runtime::TypedPackedFunc<RetTy(const IRModule&, const transform::PassInfo&)>;
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
+ /*! \brief Name of this pass instrument object. */
+ String name;
+
+ /*! \brief Callback for instrumentation environment set up. */
+ InstrumentEnvFunc set_up_callback;
+ /*! \brief Callback for instrumentation environment clean up. */
+ InstrumentEnvFunc tear_down_callback;
+
+ /*! \brief Callback to run before a pass. */
+ PassInstrumentFunc</* RetTy */ bool> run_before_pass_callback;
+ /*! \brief Callback to run after a pass. */
+ PassInstrumentFunc<> run_after_pass_callback;
+
+ /*!
+ * \brief Register a callback to run at set up point.
+ *
+ * \param callback The set up function.
+ */
+ void RegisterSetUpCallback(InstrumentEnvFunc callback) { set_up_callback = std::move(callback); }
+
+ /*
+ * \brief Register a callback to run at clean up point.
+ *
+ * \param callback The clean up function.
+ */
+ void RegisterTearDownCallback(InstrumentEnvFunc callback) {
+ tear_down_callback = std::move(callback);
+ }
+
+ /*!
+ * \brief Register a callback to run before pass run.
+ *
+ * \param callback The function to run before pass: return false to skip pass; return true to
+ * run pass.
+ */
+ void RegisterRunBeforePassCallback(PassInstrumentFunc<bool> callback) {
+ run_before_pass_callback = std::move(callback);
+ }
+
+ /*!
+ * \brief Register a callback to run after pass run.
+ *
+ * \param callback The function to run after pass.
+ */
+ void RegisterRunAfterPassCallback(PassInstrumentFunc<> callback) {
+ run_after_pass_callback = std::move(callback);
+ }
+
+ void VisitAttrs(AttrVisitor* v) { v->Visit("name", &name); }
+
+ /*! \brief Set up environment for instrumentation. */
+ void SetUp() const;
+
+ /*! \brief Clean up instrumentation environment. */
+ void TearDown() const;
+
+ /*!
+ * \brief Instrument before pass run, determine whether to run the pass or not.
+ * \param mod The module that an optimization pass runs on.
+ * \param info The pass information.
+ *
+ * \return true to run the pass; false to skip the pass.
+ */
+ bool RunBeforePass(const IRModule& mod, const transform::PassInfo& info) const;
+
+ /*!
+ * \brief Instrument after pass run.
+ *
+ * \param mod The module that an optimization pass runs on.
+ * \param info The pass information.
+ */
+ void RunAfterPass(const IRModule& mod, const transform::PassInfo& info) const;
+
+ static constexpr const char* _type_key = "instrument.PassInstrument";
+ TVM_DECLARE_FINAL_OBJECT_INFO(PassInstrumentNode, Object);
+};
+
+/*!
+ * \brief Managed reference class for PassInstrumentNode
+ * \sa PassInstrumentNode
+ */
+class PassInstrument : public ObjectRef {
+ public:
+ /*!
+ * \brief Constructor
+ * \param name Name for this instrumentation.
+ */
+ TVM_DLL PassInstrument(String name);
+
+ /*!
+ * \brief mutable accessor.
+ * \return mutable access pointer.
+ */
+ PassInstrumentNode* operator->() {
+ ICHECK(get() != nullptr);
+ return static_cast<PassInstrumentNode*>(get_mutable());
+ }
+
+ TVM_DEFINE_OBJECT_REF_METHODS(PassInstrument, ObjectRef, PassInstrumentNode);
+};
+
+/*!
+ * \brief PassInstrumentorNode collects a set of PassInstrument implementations, invokes the
+ * implementations' methods at different instrument points.
+ * \sa PassInstrumentor
+ */
+class PassInstrumentorNode : public Object {
Review comment:
I updated to pass `Array<PassInstruments>`.
##########
File path: python/tvm/ir/transform.py
##########
@@ -65,12 +65,20 @@ class PassContext(tvm.runtime.Object):
disabled_pass : Optional[Union[List[str], Set[str], Tuple[str]]]
The list of passes that are disabled.
+ pass_instrumentor : Optional[tvm.instrument.PassInstrumentor]
+ The pass instrumentor that collects pass instrument implementations
+
config : Optional[Dict[str, Object]]
Additional configurations for specific passes.
"""
def __init__(
- self, opt_level=2, required_pass=None, disabled_pass=None, trace=None, config=None
+ self,
+ opt_level=2,
+ required_pass=None,
+ disabled_pass=None,
+ pass_instrumentor=None,
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tqchen commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
tqchen commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-844349244
given there is a precedence of API, (LLVM/MLIR) that fuses runBefore and shouldRun. I actually agree with @zackcquic that we should follow the previous API precedence so we get better consistency per https://tvm.apache.org/docs/contribute/code_review.html?highlight=review#deliberate-on-api-and-data-structures "Be consistent with existing well-known package’s APIs if the features overlap."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-844694203
@tqchen @tkonolige @areusch
Thanks for the discussions.
For ```ShouldRun``` and ```runBeforePass``` problem, I will check how LLVM and MLIR deal with multiple instrument instances in detail these days.
Also @areusch points to error handling, I really appreciate it. 👍
I will check these too, and then come up a solution.
If you have any other ideas/state of art impls, like **_after_pass with a "result" notion._**, please also let me know.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tkonolige commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
tkonolige commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r634825615
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired by LLVM and MLIR.
+ * It inserts instrumentation points around passes.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrumentation points.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass1())
+ * Pass1()
+ * Instrument After Pass1()
+ *
+ * if (Instrument Before Pass2())
+ * Pass2()
+ * Instrument After Pass2()
+ *
+ * Instrument TearDown
+ *
+ * The `Before Pass` instrumentation point can selectively disable passes by returning true (to
+ * enable) or false (to disable).
Review comment:
Can you document how multiple before passes are handled.
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired by LLVM and MLIR.
+ * It inserts instrumentation points around passes.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrumentation points.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass1())
Review comment:
`Instrument Before Pass1` and `Instrument Before Pass2` are the same right? So remove the numbers from them. (and from after pass).
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired by LLVM and MLIR.
+ * It inserts instrumentation points around passes.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrumentation points.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass1())
+ * Pass1()
+ * Instrument After Pass1()
+ *
+ * if (Instrument Before Pass2())
+ * Pass2()
+ * Instrument After Pass2()
+ *
+ * Instrument TearDown
Review comment:
```suggestion
* Instrument TearDown()
```
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired by LLVM and MLIR.
+ * It inserts instrumentation points around passes.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrumentation points.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
Review comment:
```suggestion
* Instrument SetUp()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-843732387
> What is the reasoning behind having instrumentation be able to turn on and off passes? It seems like this feature should be separate.
I don't have strong reason (examples).
This is inspired by LLVM. [llvm::PassInstrumentation::runBeforePass()](https://llvm.org/doxygen/classllvm_1_1PassInstrumentation.html#aead32b9af4b66a742d37585c6d6b4cbc)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r634819807
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
+ virtual ~PassInstrumentNode() {}
+
+ /*! \brief Set up environment for instrumentation. */
+ virtual void SetUp() const = 0;
+
+ /*! \brief Clean up instrumentation environment. */
+ virtual void TearDown() const = 0;
+
+ /*!
+ * \brief Instrument before pass run, determine whether to run the pass or not.
+ * \param mod The module that an optimization pass runs on.
+ * \param info The pass information.
+ *
+ * \return true to run the pass; false to skip the pass.
+ */
+ virtual bool RunBeforePass(const IRModule& mod, const transform::PassInfo& info) const = 0;
Review comment:
No. I think.
Like the ```InstrumentBeforePass``` logic before.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-850717593
> Thanks @zackcquic for putting all the hard work polishing the API, improving the code and bring the implementation.
>
> It would be great if we can followup to add design docs and examples :)
Sure, I will add it. Thank you very much.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic edited a comment on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic edited a comment on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-840237999
@tqchen Thanks a lot for review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tqchen commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
tqchen commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-830364732
Thanks @zackcquic . Looks like a good addition, do you mind also create an RFC discussion post? I agree it is important to have pass instrumentations, would be good to know how can it interact with Trace, since some of the callbacks might be similar and we need to unify the interface.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r631678904
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief A callback type for set up or clean up instrument environment.
+ */
+using InstrumentEnvFunc = runtime::TypedPackedFunc<void()>;
+
+/*!
+ * \brief A callback template for instrumenting before/after environment.
+ * \tparam RetTy the return type of callback.
+ */
+template <typename RetTy = void>
+using PassInstrumentFunc =
+ runtime::TypedPackedFunc<RetTy(const IRModule&, const transform::PassInfo&)>;
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
Review comment:
May I confirm it's "PassInstrumentor" or "PassInstrument"?
Currently they have different meaning:
PassInstrumentor collects a set of instrument implements (PassInstrument)
PassInstrument is an instrument implementation.
Or do you want make both PassInstrumentor and PassInstrument sub-class-able?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tqchen edited a comment on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
tqchen edited a comment on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-830364732
Thank you @zackcquic ! Looks like a good addition, do you mind also create an RFC discussion post?
I agree it is important to have pass instrumentations, would be good to know how can it interact with Trace, since some of the callbacks might be similar and we need to unify the interface. On the design side, I think the runtime profiling and pass instrumentation might be different enough that might worth two separate solutions(perhaps a bit of reuse of timer if needed) As the former have more complexity wrt to GPU timer etc, while the later allows more statistics to be collected
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r634902803
##########
File path: src/tir/ir/transform.cc
##########
@@ -87,9 +87,11 @@ PrimFuncPass::PrimFuncPass(
// Perform Module -> Module optimizations at the PrimFunc level.
IRModule PrimFuncPassNode::operator()(IRModule mod, const PassContext& pass_ctx) const {
- const PassInfo& pass_info = Info();
Review comment:
Fixed
##########
File path: src/ir/transform.cc
##########
@@ -464,12 +360,19 @@ IRModule ModulePassNode::operator()(IRModule mod, const PassContext& pass_ctx) c
<< "The diagnostic context was set at the top of this block this is a bug.";
const PassInfo& pass_info = Info();
+ ICHECK(mod.defined()) << "The input module must be set.";
+
+ if (!pass_ctx.InstrumentBeforePass(mod, pass_info)) {
+ DLOG(INFO) << "Skipping function pass : " << pass_info->name
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r634818188
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired from LLVM and MLIR.
Review comment:
Done
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
Review comment:
Done
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r634892551
##########
File path: src/ir/transform.cc
##########
@@ -162,170 +164,64 @@ void PassContext::RegisterConfigOption(const char* key, uint32_t value_type_inde
PassContext PassContext::Create() { return PassContext(make_object<PassContextNode>()); }
-void PassContext::Trace(const IRModule& module, const PassInfo& info, bool is_before) const {
+void PassContext::InstrumentSetUp() const {
auto pass_ctx_node = this->operator->();
- if (pass_ctx_node->trace_func != nullptr) {
- pass_ctx_node->trace_func(module, info, is_before);
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->SetUp();
+ }
}
}
-class ModulePass;
-
-/*! \brief PassProfile stores profiling information for a given pass and its sub-passes. */
-struct PassProfile {
- // TODO(@altanh): expose PassProfile through TVM Object API
- using Clock = std::chrono::steady_clock;
- using Duration = std::chrono::duration<double, std::micro>;
- using Time = std::chrono::time_point<Clock>;
-
- /*! \brief The name of the pass being profiled. */
- String name;
- /*! \brief The time when the pass was entered. */
- Time start;
- /*! \brief The time when the pass completed. */
- Time end;
- /*! \brief The total duration of the pass, i.e. end - start. */
- Duration duration;
- /*! \brief PassProfiles for all sub-passes invoked during the execution of the pass. */
- std::vector<PassProfile> children;
-
- explicit PassProfile(String name)
- : name(name), start(Clock::now()), end(Clock::now()), children() {}
-
- /*! \brief Gets the PassProfile of the currently executing pass. */
- static PassProfile* Current();
- /*! \brief Pushes a new PassProfile with the given pass name. */
- static void EnterPass(String name);
- /*! \brief Pops the current PassProfile. */
- static void ExitPass();
-};
-
-struct PassProfileThreadLocalEntry {
- /*! \brief The placeholder top-level PassProfile. */
- PassProfile root;
- /*! \brief The stack of PassProfiles for nested passes currently running. */
- std::stack<PassProfile*> profile_stack;
- /*! \brief Whether or not pass profiling is active. */
- bool active;
-
- PassProfileThreadLocalEntry() : root("root"), active(false) {}
-};
-
-/*! \brief Thread local store to hold the pass profiling data. */
-typedef dmlc::ThreadLocalStore<PassProfileThreadLocalEntry> PassProfileThreadLocalStore;
-
-void PassProfile::EnterPass(String name) {
- if (!PassProfileThreadLocalStore::Get()->active) return;
- PassProfile* cur = PassProfile::Current();
- cur->children.emplace_back(name);
- PassProfileThreadLocalStore::Get()->profile_stack.push(&cur->children.back());
+void PassContext::InstrumentTearDown() const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
Review comment:
I assume no.
Is there possible examples?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r631691711
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief A callback type for set up or clean up instrument environment.
+ */
+using InstrumentEnvFunc = runtime::TypedPackedFunc<void()>;
+
+/*!
+ * \brief A callback template for instrumenting before/after environment.
+ * \tparam RetTy the return type of callback.
+ */
+template <typename RetTy = void>
+using PassInstrumentFunc =
+ runtime::TypedPackedFunc<RetTy(const IRModule&, const transform::PassInfo&)>;
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
+ /*! \brief Name of this pass instrument object. */
+ String name;
+
+ /*! \brief Callback for instrumentation environment set up. */
+ InstrumentEnvFunc set_up_callback;
+ /*! \brief Callback for instrumentation environment clean up. */
+ InstrumentEnvFunc tear_down_callback;
+
+ /*! \brief Callback to run before a pass. */
+ PassInstrumentFunc</* RetTy */ bool> run_before_pass_callback;
+ /*! \brief Callback to run after a pass. */
+ PassInstrumentFunc<> run_after_pass_callback;
+
+ /*!
+ * \brief Register a callback to run at set up point.
+ *
+ * \param callback The set up function.
+ */
+ void RegisterSetUpCallback(InstrumentEnvFunc callback) { set_up_callback = std::move(callback); }
+
+ /*
+ * \brief Register a callback to run at clean up point.
+ *
+ * \param callback The clean up function.
+ */
+ void RegisterTearDownCallback(InstrumentEnvFunc callback) {
+ tear_down_callback = std::move(callback);
+ }
+
+ /*!
+ * \brief Register a callback to run before pass run.
+ *
+ * \param callback The function to run before pass: return false to skip pass; return true to
+ * run pass.
+ */
+ void RegisterRunBeforePassCallback(PassInstrumentFunc<bool> callback) {
+ run_before_pass_callback = std::move(callback);
+ }
+
+ /*!
+ * \brief Register a callback to run after pass run.
+ *
+ * \param callback The function to run after pass.
+ */
+ void RegisterRunAfterPassCallback(PassInstrumentFunc<> callback) {
+ run_after_pass_callback = std::move(callback);
+ }
+
+ void VisitAttrs(AttrVisitor* v) { v->Visit("name", &name); }
+
+ /*! \brief Set up environment for instrumentation. */
+ void SetUp() const;
+
+ /*! \brief Clean up instrumentation environment. */
+ void TearDown() const;
+
+ /*!
+ * \brief Instrument before pass run, determine whether to run the pass or not.
+ * \param mod The module that an optimization pass runs on.
+ * \param info The pass information.
+ *
+ * \return true to run the pass; false to skip the pass.
+ */
+ bool RunBeforePass(const IRModule& mod, const transform::PassInfo& info) const;
+
+ /*!
+ * \brief Instrument after pass run.
+ *
+ * \param mod The module that an optimization pass runs on.
+ * \param info The pass information.
+ */
+ void RunAfterPass(const IRModule& mod, const transform::PassInfo& info) const;
+
+ static constexpr const char* _type_key = "instrument.PassInstrument";
+ TVM_DECLARE_FINAL_OBJECT_INFO(PassInstrumentNode, Object);
+};
+
+/*!
+ * \brief Managed reference class for PassInstrumentNode
+ * \sa PassInstrumentNode
+ */
+class PassInstrument : public ObjectRef {
+ public:
+ /*!
+ * \brief Constructor
+ * \param name Name for this instrumentation.
+ */
+ TVM_DLL PassInstrument(String name);
+
+ /*!
+ * \brief mutable accessor.
+ * \return mutable access pointer.
+ */
+ PassInstrumentNode* operator->() {
+ ICHECK(get() != nullptr);
+ return static_cast<PassInstrumentNode*>(get_mutable());
+ }
+
+ TVM_DEFINE_OBJECT_REF_METHODS(PassInstrument, ObjectRef, PassInstrumentNode);
+};
+
+/*!
+ * \brief PassInstrumentorNode collects a set of PassInstrument implementations, invokes the
+ * implementations' methods at different instrument points.
+ * \sa PassInstrumentor
+ */
+class PassInstrumentorNode : public Object {
+ public:
+ Array<PassInstrument> pass_instruments;
+
+ void VisitAttrs(AttrVisitor* v) { v->Visit("pass_instruments", &pass_instruments); }
+
+ /*! \brief Set up environment for instrument implementations. */
+ void SetUp() const;
+
+ /*! \brief Clean up environment for instrument implementations. */
+ void TearDown() const;
Review comment:
This is inspired by passes time profiling before, they may have internal states want to setup/reset.
Integrated example is like [Passes time profiling](https://github.com/zackcquic/tvm/blob/0ad48d4f1e04f048bbae12daaec1cc100db458cd/src/ir/instrument.cc#L276)
If we only reset internal states at constructor and destructor:
```python
pi = PassInstrument
with PassContext(Instrumentor=[pi]):
xxxx
with PassContext(Instrumentor=[pi]):
xxxxx
```
The instrument result from second pass context will pollute the first one.
And yes, Setup and Teardown are called in EnterWithScope and ExitWithScope separately.
[transform.cc#L61](https://github.com/zackcquic/tvm/blob/0ad48d4f1e04f048bbae12daaec1cc100db458cd/src/ir/transform.cc#L61) and [transform.cc#L68](https://github.com/zackcquic/tvm/blob/0ad48d4f1e04f048bbae12daaec1cc100db458cd/src/ir/transform.cc#L68)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r637520285
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired by LLVM and MLIR.
+ * It inserts instrumentation points around passes.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrumentation points.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp()
+ *
+ * if (Instrument Before Pass())
+ * Pass1()
+ * Instrument After Pass()
+ *
+ * if (Instrument Before Pass())
+ * Pass2()
+ * Instrument After Pass()
+ *
+ * Instrument TearDown()
+ *
+ *
+ * The `Before Pass` instrumentation point can selectively disable passes by returning true (to
+ * enable) or false (to disable).
+ *
+ * If there are multiple pass instrumentations provided, `Before Pass` callbacks are applied in
+ * order. If one return false, then the pass will be skipped:
+ *
+ * for (auto pi : PassInstruments)
+ * if (pi->BeforePass())
+ * return False // Disable pass
+ *
+ * return True // All ok, enable pass
+ *
+ *
+ * \sa PassInstrument
+ * \sa PassContextNode::InstrumentBeforePass()
+ * \sa src/ir/transform.cc
+ */
+class PassInstrumentNode : public Object {
+ public:
+ virtual ~PassInstrumentNode() {}
+
+ /*! \brief Set up environment for instrumentation. */
Review comment:
Done
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired by LLVM and MLIR.
+ * It inserts instrumentation points around passes.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrumentation points.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp()
+ *
+ * if (Instrument Before Pass())
+ * Pass1()
+ * Instrument After Pass()
+ *
+ * if (Instrument Before Pass())
+ * Pass2()
+ * Instrument After Pass()
+ *
+ * Instrument TearDown()
+ *
+ *
+ * The `Before Pass` instrumentation point can selectively disable passes by returning true (to
+ * enable) or false (to disable).
+ *
+ * If there are multiple pass instrumentations provided, `Before Pass` callbacks are applied in
+ * order. If one return false, then the pass will be skipped:
+ *
+ * for (auto pi : PassInstruments)
+ * if (pi->BeforePass())
+ * return False // Disable pass
+ *
+ * return True // All ok, enable pass
+ *
+ *
+ * \sa PassInstrument
+ * \sa PassContextNode::InstrumentBeforePass()
+ * \sa src/ir/transform.cc
+ */
+class PassInstrumentNode : public Object {
+ public:
+ virtual ~PassInstrumentNode() {}
+
+ /*! \brief Set up environment for instrumentation. */
+ virtual void SetUp() const = 0;
+
+ /*! \brief Clean up instrumentation environment. */
Review comment:
Done
##########
File path: include/tvm/ir/transform.h
##########
@@ -95,8 +87,9 @@ class PassContextNode : public Object {
mutable Optional<DiagnosticContext> diag_ctx;
/*! \brief Pass specific configurations. */
Map<String, ObjectRef> config;
- /*! \brief Trace function to be invoked before and after each pass. */
- TraceFunc trace_func;
+
+ /*! \brief A list of pass instrument implementations. */
Review comment:
Done
##########
File path: python/tvm/ir/instrument.py
##########
@@ -0,0 +1,154 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=invalid-name,unused-argument
+"""Common pass instrumentation across IR variants."""
+import inspect
+import functools
+
+import tvm._ffi
+import tvm.runtime
+
+from . import _ffi_instrument_api
+
+
+@tvm._ffi.register_object("instrument.PassInstrument")
+class PassInstrument(tvm.runtime.Object):
+ """A pass instrument implementation.
+
+ Users don't need to interact with this class directly.
+ Instead, a `PassInstrument` instance should be created through `pass_instrument`.
+
+ See Also
+ --------
+ `pass_instrument`
+ """
+
+
+def _wrap_class_pass_instrument(pi_cls):
+ """Wrap a python class as pass instrument"""
+
+ class PyPassInstrument(PassInstrument):
+ """Internal wrapper class to create a class instance."""
+
+ def __init__(self, *args, **kwargs):
+ # initialize handle in cass pi_cls creation failed.fg
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r638393539
##########
File path: include/tvm/ir/transform.h
##########
@@ -189,12 +183,36 @@ class PassContext : public ObjectRef {
TVM_DLL static PassContext Current();
/*!
- * \brief Apply the tracing functions of the context to the module, with the info.
- * \param module The IRModule to trace.
+ * \brief Call instrument implementations' callbacks when entering PassContex.
+ * In order, if one raises exceptions, remaings will not be called.
+ */
+ TVM_DLL void InstrumentEnterPassContext();
+
+ /*!
+ * \brief Call instrument implementations' callback when exiting PassContext.
+ * In order, if one raises exceptions, remaings will not be called.
+ */
+ TVM_DLL void InstrumentExitPassContext();
+
+ /*!
+ * \brief Call intrument implementations' callbacks before a pass run.
+ * In order, if one raises exceptions, remaings will not be called.
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
areusch commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r639957850
##########
File path: src/ir/transform.cc
##########
@@ -162,170 +166,86 @@ void PassContext::RegisterConfigOption(const char* key, uint32_t value_type_inde
PassContext PassContext::Create() { return PassContext(make_object<PassContextNode>()); }
-void PassContext::Trace(const IRModule& module, const PassInfo& info, bool is_before) const {
+void PassContext::InstrumentEnterPassContext() {
auto pass_ctx_node = this->operator->();
- if (pass_ctx_node->trace_func != nullptr) {
- pass_ctx_node->trace_func(module, info, is_before);
+ if (pass_ctx_node->instruments.defined()) {
+ try {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->EnterPassContext();
+ }
+ } catch (const Error& e) {
+ LOG(INFO) << "Pass instrumentation entering pass context failed.";
+ LOG(INFO) << "Disable pass instrumentation.";
+ pass_ctx_node->instruments.clear();
Review comment:
here is where I'm suggesting you should call `pi->ExitPassContext()` for all `pi` whose `EnterPassContext` has successfully returned so far.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r634904870
##########
File path: python/tvm/ir/instrument.py
##########
@@ -0,0 +1,154 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=invalid-name,unused-argument
+"""Common pass instrumentation across IR variants."""
+import inspect
+import functools
+
+import tvm._ffi
+import tvm.runtime
+
+from . import _ffi_instrument_api
+
+
+@tvm._ffi.register_object("instrument.PassInstrument")
+class PassInstrument(tvm.runtime.Object):
+ """A pass instrument implementation.
+
+ Users don't need to interact with this class directly.
+ Instead, a `PassInstrument` instance should be created through `pass_instrument`.
+
+ See Also
+ --------
+ `pass_instrument`
+ """
+
+
+def _wrap_class_pass_instrument(pi_cls):
+ """Wrap a python class as pass instrument"""
+
+ class PyPassInstrument(PassInstrument):
+ """Internal wrapper class to create a class instance."""
+
+ def __init__(self, *args, **kwargs):
+ # initialize handle in cass pi_cls creation failed.fg
+ self.handle = None
+ inst = pi_cls(*args, **kwargs)
+
+ # check method declartion within class, if found, wrap it.
+ def create_method(method):
+ if hasattr(inst, method) and inspect.ismethod(getattr(inst, method)):
+
+ def func(*args):
+ return getattr(inst, method)(*args)
+
+ func.__name__ = "_" + method
+ return func
+ return None
+
+ # create runtime pass instrument object
+ # reister instance's run_before_pass, run_after_pass, set_up and tear_down method
+ # to it if present.
+ self.__init_handle_by_constructor__(
+ _ffi_instrument_api.NamedPassInstrument,
+ pi_cls.__name__,
+ create_method("run_before_pass"),
+ create_method("run_after_pass"),
+ create_method("set_up"),
+ create_method("tear_down"),
+ )
+
+ self._inst = inst
+
+ def __getattr__(self, name):
+ # fall back to instance attribute if there is not any
+ return self._inst.__getattribute__(name)
+
+ functools.update_wrapper(PyPassInstrument.__init__, pi_cls.__init__)
+ PyPassInstrument.__name__ = pi_cls.__name__
+ PyPassInstrument.__doc__ = pi_cls.__doc__
+ PyPassInstrument.__module__ = pi_cls.__module__
+ return PyPassInstrument
+
+
+def pass_instrument(pi_cls=None):
+ """Decorate a pass instrument.
+
+ Parameters
+ ----------
+ pi_class :
+
+ Examples
+ --------
+ The following code block decorates a pass instrument class.
+
+ .. code-block:: python
+ @tvm.instrument.pass_instrument
+ class SkipPass:
+ def __init__(self, skip_pass_name):
+ self.skip_pass_name = skip_pass_name
+
+ # Uncomment to customize
+ # def set_up(self):
+ # pass
+
+ # Uncomment to customize
+ # def tear_down(self):
+ # pass
+
+ # If pass name contains keyword, skip it by return False. (return True: not skip)
+ def run_before_pass(self, mod, pass_info):
+ if self.skip_pass_name in pass_info.name:
+ return False
+ return True
+
+ # Uncomment to customize
+ # def run_after_pass(self, mod, pass_info):
+ # pass
+
+ skip_annotate = SkipPass("AnnotateSpans")
+ with tvm.transform.PassContext(instruments=[skip_annotate]):
+ tvm.relay.build(mod, "llvm")
+ """
+
+ def create_pass_instrument(pi_cls):
+ if not inspect.isclass(pi_cls):
+ raise TypeError("pi_cls must be a class")
+
+ return _wrap_class_pass_instrument(pi_cls)
+
+ if pi_cls:
+ return create_pass_instrument(pi_cls)
+ return create_pass_instrument
+
+
+@tvm._ffi.register_object("instrument.PassInstrument")
+class PassesTimeInstrument(tvm.runtime.Object):
Review comment:
Done
##########
File path: include/tvm/ir/transform.h
##########
@@ -189,12 +182,32 @@ class PassContext : public ObjectRef {
TVM_DLL static PassContext Current();
/*!
- * \brief Apply the tracing functions of the context to the module, with the info.
- * \param module The IRModule to trace.
+ * \brief Set up for all the instrument implementations.
+ */
+ TVM_DLL void InstrumentSetUp() const;
+
+ /*!
+ * \brief Clean up for all the instrument implementations.
+ */
+ TVM_DLL void InstrumentTearDown() const;
+
+ /*!
+ * \brief Call intrument implementatations before a pass run.
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r631691711
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief A callback type for set up or clean up instrument environment.
+ */
+using InstrumentEnvFunc = runtime::TypedPackedFunc<void()>;
+
+/*!
+ * \brief A callback template for instrumenting before/after environment.
+ * \tparam RetTy the return type of callback.
+ */
+template <typename RetTy = void>
+using PassInstrumentFunc =
+ runtime::TypedPackedFunc<RetTy(const IRModule&, const transform::PassInfo&)>;
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
+ /*! \brief Name of this pass instrument object. */
+ String name;
+
+ /*! \brief Callback for instrumentation environment set up. */
+ InstrumentEnvFunc set_up_callback;
+ /*! \brief Callback for instrumentation environment clean up. */
+ InstrumentEnvFunc tear_down_callback;
+
+ /*! \brief Callback to run before a pass. */
+ PassInstrumentFunc</* RetTy */ bool> run_before_pass_callback;
+ /*! \brief Callback to run after a pass. */
+ PassInstrumentFunc<> run_after_pass_callback;
+
+ /*!
+ * \brief Register a callback to run at set up point.
+ *
+ * \param callback The set up function.
+ */
+ void RegisterSetUpCallback(InstrumentEnvFunc callback) { set_up_callback = std::move(callback); }
+
+ /*
+ * \brief Register a callback to run at clean up point.
+ *
+ * \param callback The clean up function.
+ */
+ void RegisterTearDownCallback(InstrumentEnvFunc callback) {
+ tear_down_callback = std::move(callback);
+ }
+
+ /*!
+ * \brief Register a callback to run before pass run.
+ *
+ * \param callback The function to run before pass: return false to skip pass; return true to
+ * run pass.
+ */
+ void RegisterRunBeforePassCallback(PassInstrumentFunc<bool> callback) {
+ run_before_pass_callback = std::move(callback);
+ }
+
+ /*!
+ * \brief Register a callback to run after pass run.
+ *
+ * \param callback The function to run after pass.
+ */
+ void RegisterRunAfterPassCallback(PassInstrumentFunc<> callback) {
+ run_after_pass_callback = std::move(callback);
+ }
+
+ void VisitAttrs(AttrVisitor* v) { v->Visit("name", &name); }
+
+ /*! \brief Set up environment for instrumentation. */
+ void SetUp() const;
+
+ /*! \brief Clean up instrumentation environment. */
+ void TearDown() const;
+
+ /*!
+ * \brief Instrument before pass run, determine whether to run the pass or not.
+ * \param mod The module that an optimization pass runs on.
+ * \param info The pass information.
+ *
+ * \return true to run the pass; false to skip the pass.
+ */
+ bool RunBeforePass(const IRModule& mod, const transform::PassInfo& info) const;
+
+ /*!
+ * \brief Instrument after pass run.
+ *
+ * \param mod The module that an optimization pass runs on.
+ * \param info The pass information.
+ */
+ void RunAfterPass(const IRModule& mod, const transform::PassInfo& info) const;
+
+ static constexpr const char* _type_key = "instrument.PassInstrument";
+ TVM_DECLARE_FINAL_OBJECT_INFO(PassInstrumentNode, Object);
+};
+
+/*!
+ * \brief Managed reference class for PassInstrumentNode
+ * \sa PassInstrumentNode
+ */
+class PassInstrument : public ObjectRef {
+ public:
+ /*!
+ * \brief Constructor
+ * \param name Name for this instrumentation.
+ */
+ TVM_DLL PassInstrument(String name);
+
+ /*!
+ * \brief mutable accessor.
+ * \return mutable access pointer.
+ */
+ PassInstrumentNode* operator->() {
+ ICHECK(get() != nullptr);
+ return static_cast<PassInstrumentNode*>(get_mutable());
+ }
+
+ TVM_DEFINE_OBJECT_REF_METHODS(PassInstrument, ObjectRef, PassInstrumentNode);
+};
+
+/*!
+ * \brief PassInstrumentorNode collects a set of PassInstrument implementations, invokes the
+ * implementations' methods at different instrument points.
+ * \sa PassInstrumentor
+ */
+class PassInstrumentorNode : public Object {
+ public:
+ Array<PassInstrument> pass_instruments;
+
+ void VisitAttrs(AttrVisitor* v) { v->Visit("pass_instruments", &pass_instruments); }
+
+ /*! \brief Set up environment for instrument implementations. */
+ void SetUp() const;
+
+ /*! \brief Clean up environment for instrument implementations. */
+ void TearDown() const;
Review comment:
This is inspired by passes time profiling before, they may have internal states want to setup/reset.
Integrated example is like [Passes time profiling](https://github.com/zackcquic/tvm/blob/0ad48d4f1e04f048bbae12daaec1cc100db458cd/src/ir/instrument.cc#L276)
If we only reset internal states at constructor and destructor:
```python
pi = PassInstrument
with PassContext(Instrumentor=[pi]):
xxxx
with PassContext(Instrumentor=[pi]):
xxxxx
```
The instrument result from second pass context will pollute the first one.
And yes, ```Setup``` and ```Teardown``` are called in EnterWithScope and ExitWithScope separately.
[transform.cc#L61](https://github.com/zackcquic/tvm/blob/0ad48d4f1e04f048bbae12daaec1cc100db458cd/src/ir/transform.cc#L61) and [transform.cc#L68](https://github.com/zackcquic/tvm/blob/0ad48d4f1e04f048bbae12daaec1cc100db458cd/src/ir/transform.cc#L68)
Do you think it is ok?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r639027984
##########
File path: tests/python/relay/test_pass_instrument.py
##########
@@ -205,6 +205,18 @@ def enter_pass_ctx(self):
assert cur_pass_ctx.instruments == None
+def test_enter_pass_ctx_exception_global(capsys):
+ @pass_instrument
+ class PIBroken:
+ def enter_pass_ctx(self):
+ raise RuntimeError("Just a dummy error")
+
+ cur_pass_ctx = tvm.transform.PassContext.current()
+ with pytest.raises(tvm.error.TVMError):
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r638400076
##########
File path: include/tvm/ir/transform.h
##########
@@ -189,12 +183,36 @@ class PassContext : public ObjectRef {
TVM_DLL static PassContext Current();
/*!
- * \brief Apply the tracing functions of the context to the module, with the info.
- * \param module The IRModule to trace.
+ * \brief Call instrument implementations' callbacks when entering PassContex.
+ * In order, if one raises exceptions, remaings will not be called.
+ */
+ TVM_DLL void InstrumentEnterPassContext();
+
+ /*!
+ * \brief Call instrument implementations' callback when exiting PassContext.
+ * In order, if one raises exceptions, remaings will not be called.
+ */
+ TVM_DLL void InstrumentExitPassContext();
+
+ /*!
+ * \brief Call intrument implementations' callbacks before a pass run.
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r640171977
##########
File path: src/ir/transform.cc
##########
@@ -162,170 +166,86 @@ void PassContext::RegisterConfigOption(const char* key, uint32_t value_type_inde
PassContext PassContext::Create() { return PassContext(make_object<PassContextNode>()); }
-void PassContext::Trace(const IRModule& module, const PassInfo& info, bool is_before) const {
+void PassContext::InstrumentEnterPassContext() {
auto pass_ctx_node = this->operator->();
- if (pass_ctx_node->trace_func != nullptr) {
- pass_ctx_node->trace_func(module, info, is_before);
+ if (pass_ctx_node->instruments.defined()) {
+ try {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->EnterPassContext();
+ }
+ } catch (const Error& e) {
+ LOG(INFO) << "Pass instrumentation entering pass context failed.";
+ LOG(INFO) << "Disable pass instrumentation.";
+ pass_ctx_node->instruments.clear();
Review comment:
Done, thanks a lot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r633069427
##########
File path: python/tvm/ir/instrument.py
##########
@@ -0,0 +1,128 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=invalid-name,unused-argument
+"""Common pass instrumentation across IR variants."""
+import tvm._ffi
+import tvm.runtime
+
+from . import _ffi_instrument_api
+
+
+@tvm._ffi.register_object("instrument.PassInstrument")
+class PassInstrument(tvm.runtime.Object):
Review comment:
Done
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief A callback type for set up or clean up instrument environment.
+ */
+using InstrumentEnvFunc = runtime::TypedPackedFunc<void()>;
+
+/*!
+ * \brief A callback template for instrumenting before/after environment.
+ * \tparam RetTy the return type of callback.
+ */
+template <typename RetTy = void>
+using PassInstrumentFunc =
+ runtime::TypedPackedFunc<RetTy(const IRModule&, const transform::PassInfo&)>;
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
+ /*! \brief Name of this pass instrument object. */
+ String name;
+
+ /*! \brief Callback for instrumentation environment set up. */
+ InstrumentEnvFunc set_up_callback;
+ /*! \brief Callback for instrumentation environment clean up. */
+ InstrumentEnvFunc tear_down_callback;
+
+ /*! \brief Callback to run before a pass. */
+ PassInstrumentFunc</* RetTy */ bool> run_before_pass_callback;
Review comment:
Done
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief A callback type for set up or clean up instrument environment.
+ */
+using InstrumentEnvFunc = runtime::TypedPackedFunc<void()>;
+
+/*!
+ * \brief A callback template for instrumenting before/after environment.
+ * \tparam RetTy the return type of callback.
+ */
+template <typename RetTy = void>
+using PassInstrumentFunc =
+ runtime::TypedPackedFunc<RetTy(const IRModule&, const transform::PassInfo&)>;
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
Review comment:
I made PassInstrumentNode able to sub-classing.
But I kept it in header files for that transform.cc needs the method declarations.
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief A callback type for set up or clean up instrument environment.
+ */
+using InstrumentEnvFunc = runtime::TypedPackedFunc<void()>;
+
+/*!
+ * \brief A callback template for instrumenting before/after environment.
+ * \tparam RetTy the return type of callback.
+ */
+template <typename RetTy = void>
+using PassInstrumentFunc =
+ runtime::TypedPackedFunc<RetTy(const IRModule&, const transform::PassInfo&)>;
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
+ /*! \brief Name of this pass instrument object. */
+ String name;
+
+ /*! \brief Callback for instrumentation environment set up. */
+ InstrumentEnvFunc set_up_callback;
+ /*! \brief Callback for instrumentation environment clean up. */
+ InstrumentEnvFunc tear_down_callback;
+
+ /*! \brief Callback to run before a pass. */
+ PassInstrumentFunc</* RetTy */ bool> run_before_pass_callback;
+ /*! \brief Callback to run after a pass. */
+ PassInstrumentFunc<> run_after_pass_callback;
+
+ /*!
+ * \brief Register a callback to run at set up point.
+ *
+ * \param callback The set up function.
+ */
+ void RegisterSetUpCallback(InstrumentEnvFunc callback) { set_up_callback = std::move(callback); }
+
+ /*
+ * \brief Register a callback to run at clean up point.
+ *
+ * \param callback The clean up function.
+ */
+ void RegisterTearDownCallback(InstrumentEnvFunc callback) {
+ tear_down_callback = std::move(callback);
+ }
+
+ /*!
+ * \brief Register a callback to run before pass run.
+ *
+ * \param callback The function to run before pass: return false to skip pass; return true to
+ * run pass.
+ */
+ void RegisterRunBeforePassCallback(PassInstrumentFunc<bool> callback) {
+ run_before_pass_callback = std::move(callback);
+ }
+
+ /*!
+ * \brief Register a callback to run after pass run.
+ *
+ * \param callback The function to run after pass.
+ */
+ void RegisterRunAfterPassCallback(PassInstrumentFunc<> callback) {
+ run_after_pass_callback = std::move(callback);
+ }
+
+ void VisitAttrs(AttrVisitor* v) { v->Visit("name", &name); }
+
+ /*! \brief Set up environment for instrumentation. */
+ void SetUp() const;
+
+ /*! \brief Clean up instrumentation environment. */
+ void TearDown() const;
+
+ /*!
+ * \brief Instrument before pass run, determine whether to run the pass or not.
+ * \param mod The module that an optimization pass runs on.
+ * \param info The pass information.
+ *
+ * \return true to run the pass; false to skip the pass.
+ */
+ bool RunBeforePass(const IRModule& mod, const transform::PassInfo& info) const;
+
+ /*!
+ * \brief Instrument after pass run.
+ *
+ * \param mod The module that an optimization pass runs on.
+ * \param info The pass information.
+ */
+ void RunAfterPass(const IRModule& mod, const transform::PassInfo& info) const;
+
+ static constexpr const char* _type_key = "instrument.PassInstrument";
+ TVM_DECLARE_FINAL_OBJECT_INFO(PassInstrumentNode, Object);
+};
+
+/*!
+ * \brief Managed reference class for PassInstrumentNode
+ * \sa PassInstrumentNode
+ */
+class PassInstrument : public ObjectRef {
+ public:
+ /*!
+ * \brief Constructor
+ * \param name Name for this instrumentation.
+ */
+ TVM_DLL PassInstrument(String name);
+
+ /*!
+ * \brief mutable accessor.
+ * \return mutable access pointer.
+ */
+ PassInstrumentNode* operator->() {
+ ICHECK(get() != nullptr);
+ return static_cast<PassInstrumentNode*>(get_mutable());
+ }
+
+ TVM_DEFINE_OBJECT_REF_METHODS(PassInstrument, ObjectRef, PassInstrumentNode);
+};
+
+/*!
+ * \brief PassInstrumentorNode collects a set of PassInstrument implementations, invokes the
+ * implementations' methods at different instrument points.
+ * \sa PassInstrumentor
+ */
+class PassInstrumentorNode : public Object {
Review comment:
I updated to pass `Array<PassInstruments>`.
##########
File path: python/tvm/ir/transform.py
##########
@@ -65,12 +65,20 @@ class PassContext(tvm.runtime.Object):
disabled_pass : Optional[Union[List[str], Set[str], Tuple[str]]]
The list of passes that are disabled.
+ pass_instrumentor : Optional[tvm.instrument.PassInstrumentor]
+ The pass instrumentor that collects pass instrument implementations
+
config : Optional[Dict[str, Object]]
Additional configurations for specific passes.
"""
def __init__(
- self, opt_level=2, required_pass=None, disabled_pass=None, trace=None, config=None
+ self,
+ opt_level=2,
+ required_pass=None,
+ disabled_pass=None,
+ pass_instrumentor=None,
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-830471526
@tqchen @tkonolige Here is the RFC:
https://discuss.tvm.apache.org/t/pass-instrument-framework-proposal/9874
And here is some reply for previous questions:
I would like to separate runtime profiling and pass instrumentation, too. This PR only focuses on pass profiling mechanisms like the passes time profiling, wants to make it easier to add more passes profiling implementations.
You might notice that this PR introduces a new namespace tvm.intrument. It intends to cover all instrument related (not limited to pass, but this PR only shows pass instrument), instead of mixing instrumentation/profiling code with transformation codes in tvm.transform. RuntimeProfiling could be add to this namespace, eg: tvm.instrument.Runtime.Profiling.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-845250654
Hi @areusch:
Could you elaborate more on "after_pass with a result notation"?
What kind of result and use scenario you expected?
That will be very helpful for me. Thanks a lot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] altanh commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
altanh commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-845458234
> For the reason not to overload superclass, I don't know, but I have a guess, maybe need someone to confirm:
Currently, only FunctionPass, ModulePass, SequentialPass and PrimFuncPass extends Pass.
FunctionPass calls InferType at the end (see the comments for detail) while SequentialPass is just like a wrapper.
>
> In these cases, InterType and SequentialPass maybe not want to be traced/instrumented.
I disagree with this, mainly because some (perhaps not so) surprising results I got with my initial pass profiling implementation is that `InferType` is called **a lot** and often due to the `FunctionPass`/`SequentialPass` design, leading to quite a bit of compilation slowdown. This information would not have been obtained if we didn't include all passes that are run in the profiling.
Now, I wonder if we should introduce a "universal" pass context which *every* PassNode must respect and has access to (perhaps a thread local one)? I definitely want to make sure the timing instrument can time all passes that are run.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-843832677
@altanh @tkonolige thanks for reviewing.
Changes are done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-843639724
@tkonolige Thanks a lot for review.
Changes are done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-832372339
Hi @tqchen @jroesch
The CI failed, because of the following message:
```
docker: write /var/lib/docker/tmp/GetImageBlob707223609: no space left on device.
See 'docker run --help'.
script returned exit code 125
```
I think it may be not related to the change but the CI environment, right?
And another thing is the workflow:
```
1 workflow awaiting approval
First-time contributors need a maintainer to approve running workflows
```
I think it may require someone to help approve, right?
Or I need to do something else?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r631681526
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief A callback type for set up or clean up instrument environment.
+ */
+using InstrumentEnvFunc = runtime::TypedPackedFunc<void()>;
+
+/*!
+ * \brief A callback template for instrumenting before/after environment.
+ * \tparam RetTy the return type of callback.
+ */
+template <typename RetTy = void>
+using PassInstrumentFunc =
+ runtime::TypedPackedFunc<RetTy(const IRModule&, const transform::PassInfo&)>;
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
+ /*! \brief Name of this pass instrument object. */
+ String name;
+
+ /*! \brief Callback for instrumentation environment set up. */
+ InstrumentEnvFunc set_up_callback;
+ /*! \brief Callback for instrumentation environment clean up. */
+ InstrumentEnvFunc tear_down_callback;
+
+ /*! \brief Callback to run before a pass. */
+ PassInstrumentFunc</* RetTy */ bool> run_before_pass_callback;
+ /*! \brief Callback to run after a pass. */
+ PassInstrumentFunc<> run_after_pass_callback;
+
+ /*!
+ * \brief Register a callback to run at set up point.
+ *
+ * \param callback The set up function.
+ */
+ void RegisterSetUpCallback(InstrumentEnvFunc callback) { set_up_callback = std::move(callback); }
+
+ /*
+ * \brief Register a callback to run at clean up point.
+ *
+ * \param callback The clean up function.
+ */
+ void RegisterTearDownCallback(InstrumentEnvFunc callback) {
+ tear_down_callback = std::move(callback);
+ }
+
+ /*!
+ * \brief Register a callback to run before pass run.
+ *
+ * \param callback The function to run before pass: return false to skip pass; return true to
+ * run pass.
+ */
+ void RegisterRunBeforePassCallback(PassInstrumentFunc<bool> callback) {
+ run_before_pass_callback = std::move(callback);
+ }
+
+ /*!
+ * \brief Register a callback to run after pass run.
+ *
+ * \param callback The function to run after pass.
+ */
+ void RegisterRunAfterPassCallback(PassInstrumentFunc<> callback) {
+ run_after_pass_callback = std::move(callback);
+ }
+
+ void VisitAttrs(AttrVisitor* v) { v->Visit("name", &name); }
+
+ /*! \brief Set up environment for instrumentation. */
+ void SetUp() const;
+
+ /*! \brief Clean up instrumentation environment. */
+ void TearDown() const;
+
+ /*!
+ * \brief Instrument before pass run, determine whether to run the pass or not.
+ * \param mod The module that an optimization pass runs on.
+ * \param info The pass information.
+ *
+ * \return true to run the pass; false to skip the pass.
+ */
+ bool RunBeforePass(const IRModule& mod, const transform::PassInfo& info) const;
+
+ /*!
+ * \brief Instrument after pass run.
+ *
+ * \param mod The module that an optimization pass runs on.
+ * \param info The pass information.
+ */
+ void RunAfterPass(const IRModule& mod, const transform::PassInfo& info) const;
+
+ static constexpr const char* _type_key = "instrument.PassInstrument";
+ TVM_DECLARE_FINAL_OBJECT_INFO(PassInstrumentNode, Object);
+};
+
+/*!
+ * \brief Managed reference class for PassInstrumentNode
+ * \sa PassInstrumentNode
+ */
+class PassInstrument : public ObjectRef {
+ public:
+ /*!
+ * \brief Constructor
+ * \param name Name for this instrumentation.
+ */
+ TVM_DLL PassInstrument(String name);
+
+ /*!
+ * \brief mutable accessor.
+ * \return mutable access pointer.
+ */
+ PassInstrumentNode* operator->() {
+ ICHECK(get() != nullptr);
+ return static_cast<PassInstrumentNode*>(get_mutable());
+ }
+
+ TVM_DEFINE_OBJECT_REF_METHODS(PassInstrument, ObjectRef, PassInstrumentNode);
+};
+
+/*!
+ * \brief PassInstrumentorNode collects a set of PassInstrument implementations, invokes the
+ * implementations' methods at different instrument points.
+ * \sa PassInstrumentor
+ */
+class PassInstrumentorNode : public Object {
Review comment:
If we put ```Array<PassInstrument>``` directly in the PassContext, then ```PassInstrumentor``` can be totally a internal data structure, and no need for sub-classing.
Either ```PassInstrumentor``` or ```Array<PassInstrument>``` is fine for me.
Which kind do you like?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r634899465
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired by LLVM and MLIR.
+ * It inserts instrumentation points around passes.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrumentation points.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass1())
+ * Pass1()
+ * Instrument After Pass1()
+ *
+ * if (Instrument Before Pass2())
+ * Pass2()
+ * Instrument After Pass2()
+ *
+ * Instrument TearDown
+ *
+ * The `Before Pass` instrumentation point can selectively disable passes by returning true (to
+ * enable) or false (to disable).
Review comment:
Done
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired by LLVM and MLIR.
+ * It inserts instrumentation points around passes.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrumentation points.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass1())
+ * Pass1()
+ * Instrument After Pass1()
+ *
+ * if (Instrument Before Pass2())
+ * Pass2()
+ * Instrument After Pass2()
+ *
+ * Instrument TearDown
Review comment:
Done
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired by LLVM and MLIR.
+ * It inserts instrumentation points around passes.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrumentation points.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
Review comment:
Done
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired by LLVM and MLIR.
+ * It inserts instrumentation points around passes.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrumentation points.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass1())
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r631678904
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief A callback type for set up or clean up instrument environment.
+ */
+using InstrumentEnvFunc = runtime::TypedPackedFunc<void()>;
+
+/*!
+ * \brief A callback template for instrumenting before/after environment.
+ * \tparam RetTy the return type of callback.
+ */
+template <typename RetTy = void>
+using PassInstrumentFunc =
+ runtime::TypedPackedFunc<RetTy(const IRModule&, const transform::PassInfo&)>;
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
Review comment:
May I confirm it's "PassInstrumentor" or "PassInstrument"?
Currently they have different meaning:
- PassInstrumentor collects a set of instrument implements (PassInstrument)
- PassInstrument is an instrument implementation.
Or do you want make both PassInstrumentor and PassInstrument sub-class-able?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r640196943
##########
File path: tests/python/relay/test_pass_instrument.py
##########
@@ -168,3 +169,329 @@ def run_after_pass(self, mod, info):
# Out of pass context scope, should be reset
assert passes_counter.run_before_count == 0
assert passes_counter.run_after_count == 0
+
+
+def test_enter_pass_ctx_exception(capsys):
+ @pass_instrument
+ class PI:
+ def __init__(self, id):
+ self.id = id
+
+ def enter_pass_ctx(self):
+ print(self.id + " enter ctx")
+
+ def exit_pass_ctx(self):
+ print(self.id + " exit ctx")
+
+ @pass_instrument
+ class PIBroken(PI):
+ def __init__(self, id):
+ super().__init__(id)
+
+ def enter_pass_ctx(self):
+ print(self.id + " enter ctx")
+ raise RuntimeError("Just a dummy error")
+
+ pass_ctx = tvm.transform.PassContext(instruments=[PI("%1"), PIBroken("%2"), PI("%3")])
+ with pytest.raises(tvm.error.TVMError):
+ with pass_ctx:
+ pass
+
+ assert "%1 enter ctx\n" "%2 enter ctx\n" == capsys.readouterr().out
Review comment:
Thanks for the detail explaining. I totally agree.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r634042815
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief A callback type for set up or clean up instrument environment.
+ */
+using InstrumentEnvFunc = runtime::TypedPackedFunc<void()>;
+
+/*!
+ * \brief A callback template for instrumenting before/after environment.
+ * \tparam RetTy the return type of callback.
+ */
+template <typename RetTy = void>
+using PassInstrumentFunc =
+ runtime::TypedPackedFunc<RetTy(const IRModule&, const transform::PassInfo&)>;
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
Review comment:
Moved to instrument.cc, and I use the name "NamedInstrumentPassNode" for that implementation.
(I think it's more consistent with naming like PassNode and ModuleNode.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] altanh commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
altanh commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-843324601
thanks for the PR and sorry for delay, I'll review this today!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-843832260
> I think this is a valuable change and should pave way for more powerful tools in debugging and profiling TVM compilation. Others have left good comments, I have a few additional questions:
>
> * what happens when PassContexts are nested? I'm not sure if this is allowed today but it feels to me like they could compose
> * the `disabled_pass` option feels redundant now for PassContext, perhaps we should keep it for backwards compatibility but it could be easily replaced using a `DisableByName` instrument (or something like that).
> * is there a reason why we couldn't overload the superclass (`PassNode`) `operator()` for doing the before and after instrumentation? the new changes have some duplicated lines for each type of PassNode.
>
> Related to the last point, we also have a problem with the new design regarding the previous Pass timing profiling: in particular, as passes can be nested, **every nested pass invocation must use the `operator()(IRModule, PassContext)` overload** to ensure the pass is recorded in the timing. I'm pretty sure this is not the case in a lot of existing passes, so we must fix this globally for the timing to be precise. But since passes can be called without a PassContext, it might be possible that there is no PassContext- maybe there's a global/thread local one? I'm not sure.
>
> Thanks for the work!
Hi @altanh, thanks a lot for reviewing along with these great questions. 👍
**Q. What happens when PassContexts are nested? I'm not sure if this is allowed today but it feels to me like they could compose.**
A. Yes, PassContext can be nested. Instrument will be active in the same level (in the view of PassContext).
``` python
def test_print():
x, y, z = [tvm.relay.var(c, shape=(3, 4), dtype="float32") for c in "xyz"]
e1 = op.add(x, y)
e2 = op.subtract(x, z)
e3 = op.multiply(e1, e1 / e2)
mod = tvm.IRModule.from_expr(e3 + e2)
@tvm.instrument.pass_instrument
class NamePrinter:
def run_before_pass(self, mod, pass_info):
print(pass_info.name)
return True
print_name = NamePrinter()
with tvm.transform.PassContext(instruments=[print_name]):
mod = tvm.relay.transform.AnnotateSpans()(mod)
with tvm.transform.PassContext():
mod = tvm.relay.transform.ToANormalForm()(mod)
mod = tvm.relay.transform.InferType()(mod)
test_print()
# output:
# AnnotateSpans
# InferType # Note: InferType is a nested pass, same pass ctx with AnnotateSpans. Still instrumented, since finally it goes through ModulePassNode::operator().
#
# Inner level passes are not printed, since no instruments provided.
```
**Q. The `disabled_pass` option feels redundant now for PassContext, perhaps we should keep it for backwards compatibility but it could be easily replaced using a `DisableByName` instrument (or something like that).**
A. Yes, I agree. I left it here. Maybe use another PRs to add/remove these in the future.
**Q. Is there a reason why we couldn't overload the superclass (`PassNode`) `operator()` for doing the before and after instrumentation? the new changes have some duplicated lines for each type of PassNode.**
A. Yes, I aware that.
Basically, this PR is just an enhanced "trace" (```with PassContext(treace=xxx)```), I just replace how the trace are implemented.
For the reason not to overload superclass, I don't know, but I have a guess, maybe need someone to confirm:
Currently, only ```FunctionPass, ModulePass, SequentialPass and PrimFuncPass``` extends ```Pass```.
```FunctionPass``` calls ```InferType``` at the end (see the comments for detail) while ```SequentialPass``` is just like a wrapper.
In these cases, ```InterType``` and ```SequentialPass``` maybe not want to be traced/instrumented.
**Q. Since passes can be called without a PassContext, it might be possible that there is no PassContext- maybe there's a global/thread local one? I'm not sure.**
A. I thought about that, but I prefer the old "trace" way (No pass context, no trace/instrument).
If I missed something, please let me know. e.g. There is a way to enable "trace" globally
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-841794965
@tqchen I just made modification based on comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
areusch commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r639378555
##########
File path: src/ir/transform.cc
##########
@@ -164,34 +167,57 @@ void PassContext::RegisterConfigOption(const char* key, uint32_t value_type_inde
PassContext PassContext::Create() { return PassContext(make_object<PassContextNode>()); }
-void PassContext::InstrumentSetUp() const {
+void PassContext::InstrumentEnterPassContext() {
auto pass_ctx_node = this->operator->();
if (pass_ctx_node->instruments.defined()) {
- for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
- pi->SetUp();
+ try {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->EnterPassContext();
+ }
+ } catch (const Error& e) {
+ LOG(INFO) << "Pass instrumentation entering pass context failed.";
+ LOG(INFO) << "Disable pass instrumentation.";
+ pass_ctx_node->instruments.clear();
+ throw e;
}
}
}
-void PassContext::InstrumentTearDown() const {
+void PassContext::InstrumentExitPassContext() {
auto pass_ctx_node = this->operator->();
if (pass_ctx_node->instruments.defined()) {
- for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
- pi->TearDown();
+ try {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->ExitPassContext();
+ }
+ } catch (const Error& e) {
+ LOG(INFO) << "Pass instrumentation exiting pass context failed.";
+ pass_ctx_node->instruments.clear();
+ throw e;
}
}
}
bool PassContext::InstrumentBeforePass(const IRModule& ir_module, const PassInfo& pass_info) const {
auto pass_ctx_node = this->operator->();
- if (pass_ctx_node->instruments.defined()) {
+ if (!pass_ctx_node->instruments.defined()) {
+ return true;
+ }
+
+ const bool pass_required = PassArrayContains(pass_ctx_node->required_pass, pass_info->name);
+ bool should_run = true;
+ if (!pass_required) {
+ const Array<instrument::PassInstrument>& instruments = pass_ctx_node->instruments;
+ should_run &= std::all_of(instruments.begin(), instruments.end(),
Review comment:
thanks! I think reading the LLVM API again, it actually does what I want, so no issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r638393612
##########
File path: include/tvm/ir/transform.h
##########
@@ -189,12 +183,36 @@ class PassContext : public ObjectRef {
TVM_DLL static PassContext Current();
/*!
- * \brief Apply the tracing functions of the context to the module, with the info.
- * \param module The IRModule to trace.
+ * \brief Call instrument implementations' callbacks when entering PassContex.
Review comment:
Done
##########
File path: include/tvm/ir/transform.h
##########
@@ -189,12 +183,36 @@ class PassContext : public ObjectRef {
TVM_DLL static PassContext Current();
/*!
- * \brief Apply the tracing functions of the context to the module, with the info.
- * \param module The IRModule to trace.
+ * \brief Call instrument implementations' callbacks when entering PassContex.
+ * In order, if one raises exceptions, remaings will not be called.
Review comment:
Done
##########
File path: include/tvm/ir/transform.h
##########
@@ -189,12 +183,36 @@ class PassContext : public ObjectRef {
TVM_DLL static PassContext Current();
/*!
- * \brief Apply the tracing functions of the context to the module, with the info.
- * \param module The IRModule to trace.
+ * \brief Call instrument implementations' callbacks when entering PassContex.
+ * In order, if one raises exceptions, remaings will not be called.
+ */
+ TVM_DLL void InstrumentEnterPassContext();
+
+ /*!
+ * \brief Call instrument implementations' callback when exiting PassContext.
+ * In order, if one raises exceptions, remaings will not be called.
Review comment:
Done
##########
File path: include/tvm/ir/transform.h
##########
@@ -189,12 +183,36 @@ class PassContext : public ObjectRef {
TVM_DLL static PassContext Current();
/*!
- * \brief Apply the tracing functions of the context to the module, with the info.
- * \param module The IRModule to trace.
+ * \brief Call instrument implementations' callbacks when entering PassContex.
+ * In order, if one raises exceptions, remaings will not be called.
+ */
+ TVM_DLL void InstrumentEnterPassContext();
+
+ /*!
+ * \brief Call instrument implementations' callback when exiting PassContext.
+ * In order, if one raises exceptions, remaings will not be called.
+ */
+ TVM_DLL void InstrumentExitPassContext();
+
+ /*!
+ * \brief Call intrument implementations' callbacks before a pass run.
+ * In order, if one raises exceptions, remaings will not be called.
+ *
+ * \param mod The module that an optimization pass runs on.
* \param info The pass information.
- * \param is_before Indicated whether the tracing is before or after a pass.
+ *
+ * \return false: the pass is skipped; true: the pass runs.
*/
- TVM_DLL void Trace(const IRModule& module, const PassInfo& info, bool is_before) const;
+ TVM_DLL bool InstrumentBeforePass(const IRModule& mod, const PassInfo& info) const;
+
+ /*!
+ * \brief Call instrument implementations callbacks after a pass run.
+ * In order, if one raises exceptions, remaings will not be called.
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r640171977
##########
File path: src/ir/transform.cc
##########
@@ -162,170 +166,86 @@ void PassContext::RegisterConfigOption(const char* key, uint32_t value_type_inde
PassContext PassContext::Create() { return PassContext(make_object<PassContextNode>()); }
-void PassContext::Trace(const IRModule& module, const PassInfo& info, bool is_before) const {
+void PassContext::InstrumentEnterPassContext() {
auto pass_ctx_node = this->operator->();
- if (pass_ctx_node->trace_func != nullptr) {
- pass_ctx_node->trace_func(module, info, is_before);
+ if (pass_ctx_node->instruments.defined()) {
+ try {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->EnterPassContext();
+ }
+ } catch (const Error& e) {
+ LOG(INFO) << "Pass instrumentation entering pass context failed.";
+ LOG(INFO) << "Disable pass instrumentation.";
+ pass_ctx_node->instruments.clear();
Review comment:
Done and test case updated, thanks a lot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r639407100
##########
File path: tests/python/relay/test_pass_instrument.py
##########
@@ -168,3 +169,329 @@ def run_after_pass(self, mod, info):
# Out of pass context scope, should be reset
assert passes_counter.run_before_count == 0
assert passes_counter.run_after_count == 0
+
+
+def test_enter_pass_ctx_exception(capsys):
+ @pass_instrument
+ class PI:
+ def __init__(self, id):
+ self.id = id
+
+ def enter_pass_ctx(self):
+ print(self.id + " enter ctx")
+
+ def exit_pass_ctx(self):
+ print(self.id + " exit ctx")
+
+ @pass_instrument
+ class PIBroken(PI):
+ def __init__(self, id):
+ super().__init__(id)
+
+ def enter_pass_ctx(self):
+ print(self.id + " enter ctx")
+ raise RuntimeError("Just a dummy error")
+
+ pass_ctx = tvm.transform.PassContext(instruments=[PI("%1"), PIBroken("%2"), PI("%3")])
+ with pytest.raises(tvm.error.TVMError):
+ with pass_ctx:
+ pass
+
+ assert "%1 enter ctx\n" "%2 enter ctx\n" == capsys.readouterr().out
Review comment:
I considered to catch all the exceptions in `InstrumentEnterPassContext` before:
if there is exception, disabling instruments within this context, and **NO** throw out to higher level.
But I finally throw it out, since I'd like to see the errors, and don't want to proceed with unstable states.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r634900983
##########
File path: src/ir/transform.cc
##########
@@ -162,170 +164,64 @@ void PassContext::RegisterConfigOption(const char* key, uint32_t value_type_inde
PassContext PassContext::Create() { return PassContext(make_object<PassContextNode>()); }
-void PassContext::Trace(const IRModule& module, const PassInfo& info, bool is_before) const {
+void PassContext::InstrumentSetUp() const {
auto pass_ctx_node = this->operator->();
- if (pass_ctx_node->trace_func != nullptr) {
- pass_ctx_node->trace_func(module, info, is_before);
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->SetUp();
+ }
}
}
-class ModulePass;
-
-/*! \brief PassProfile stores profiling information for a given pass and its sub-passes. */
-struct PassProfile {
- // TODO(@altanh): expose PassProfile through TVM Object API
- using Clock = std::chrono::steady_clock;
- using Duration = std::chrono::duration<double, std::micro>;
- using Time = std::chrono::time_point<Clock>;
-
- /*! \brief The name of the pass being profiled. */
- String name;
- /*! \brief The time when the pass was entered. */
- Time start;
- /*! \brief The time when the pass completed. */
- Time end;
- /*! \brief The total duration of the pass, i.e. end - start. */
- Duration duration;
- /*! \brief PassProfiles for all sub-passes invoked during the execution of the pass. */
- std::vector<PassProfile> children;
-
- explicit PassProfile(String name)
- : name(name), start(Clock::now()), end(Clock::now()), children() {}
-
- /*! \brief Gets the PassProfile of the currently executing pass. */
- static PassProfile* Current();
- /*! \brief Pushes a new PassProfile with the given pass name. */
- static void EnterPass(String name);
- /*! \brief Pops the current PassProfile. */
- static void ExitPass();
-};
-
-struct PassProfileThreadLocalEntry {
- /*! \brief The placeholder top-level PassProfile. */
- PassProfile root;
- /*! \brief The stack of PassProfiles for nested passes currently running. */
- std::stack<PassProfile*> profile_stack;
- /*! \brief Whether or not pass profiling is active. */
- bool active;
-
- PassProfileThreadLocalEntry() : root("root"), active(false) {}
-};
-
-/*! \brief Thread local store to hold the pass profiling data. */
-typedef dmlc::ThreadLocalStore<PassProfileThreadLocalEntry> PassProfileThreadLocalStore;
-
-void PassProfile::EnterPass(String name) {
- if (!PassProfileThreadLocalStore::Get()->active) return;
- PassProfile* cur = PassProfile::Current();
- cur->children.emplace_back(name);
- PassProfileThreadLocalStore::Get()->profile_stack.push(&cur->children.back());
+void PassContext::InstrumentTearDown() const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->TearDown();
+ }
+ }
}
-void PassProfile::ExitPass() {
- if (!PassProfileThreadLocalStore::Get()->active) return;
- PassProfile* cur = PassProfile::Current();
- ICHECK_NE(cur->name, "root") << "mismatched enter/exit for pass profiling";
- cur->end = std::move(PassProfile::Clock::now());
- cur->duration = std::chrono::duration_cast<PassProfile::Duration>(cur->end - cur->start);
- PassProfileThreadLocalStore::Get()->profile_stack.pop();
+bool PassContext::InstrumentBeforePass(const IRModule& ir_module, const PassInfo& pass_info) const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ if (!pi->RunBeforePass(ir_module, pass_info)) {
+ return false;
+ }
+ }
+ return true;
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r634820790
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
Review comment:
Done
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] altanh commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
altanh commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r634812967
##########
File path: src/ir/transform.cc
##########
@@ -162,170 +164,64 @@ void PassContext::RegisterConfigOption(const char* key, uint32_t value_type_inde
PassContext PassContext::Create() { return PassContext(make_object<PassContextNode>()); }
-void PassContext::Trace(const IRModule& module, const PassInfo& info, bool is_before) const {
+void PassContext::InstrumentSetUp() const {
auto pass_ctx_node = this->operator->();
- if (pass_ctx_node->trace_func != nullptr) {
- pass_ctx_node->trace_func(module, info, is_before);
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->SetUp();
+ }
}
}
-class ModulePass;
-
-/*! \brief PassProfile stores profiling information for a given pass and its sub-passes. */
-struct PassProfile {
- // TODO(@altanh): expose PassProfile through TVM Object API
- using Clock = std::chrono::steady_clock;
- using Duration = std::chrono::duration<double, std::micro>;
- using Time = std::chrono::time_point<Clock>;
-
- /*! \brief The name of the pass being profiled. */
- String name;
- /*! \brief The time when the pass was entered. */
- Time start;
- /*! \brief The time when the pass completed. */
- Time end;
- /*! \brief The total duration of the pass, i.e. end - start. */
- Duration duration;
- /*! \brief PassProfiles for all sub-passes invoked during the execution of the pass. */
- std::vector<PassProfile> children;
-
- explicit PassProfile(String name)
- : name(name), start(Clock::now()), end(Clock::now()), children() {}
-
- /*! \brief Gets the PassProfile of the currently executing pass. */
- static PassProfile* Current();
- /*! \brief Pushes a new PassProfile with the given pass name. */
- static void EnterPass(String name);
- /*! \brief Pops the current PassProfile. */
- static void ExitPass();
-};
-
-struct PassProfileThreadLocalEntry {
- /*! \brief The placeholder top-level PassProfile. */
- PassProfile root;
- /*! \brief The stack of PassProfiles for nested passes currently running. */
- std::stack<PassProfile*> profile_stack;
- /*! \brief Whether or not pass profiling is active. */
- bool active;
-
- PassProfileThreadLocalEntry() : root("root"), active(false) {}
-};
-
-/*! \brief Thread local store to hold the pass profiling data. */
-typedef dmlc::ThreadLocalStore<PassProfileThreadLocalEntry> PassProfileThreadLocalStore;
-
-void PassProfile::EnterPass(String name) {
- if (!PassProfileThreadLocalStore::Get()->active) return;
- PassProfile* cur = PassProfile::Current();
- cur->children.emplace_back(name);
- PassProfileThreadLocalStore::Get()->profile_stack.push(&cur->children.back());
+void PassContext::InstrumentTearDown() const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
Review comment:
is there ever a use case where different pass instruments could depend on each other?
##########
File path: include/tvm/ir/transform.h
##########
@@ -189,12 +182,32 @@ class PassContext : public ObjectRef {
TVM_DLL static PassContext Current();
/*!
- * \brief Apply the tracing functions of the context to the module, with the info.
- * \param module The IRModule to trace.
+ * \brief Set up for all the instrument implementations.
+ */
+ TVM_DLL void InstrumentSetUp() const;
+
+ /*!
+ * \brief Clean up for all the instrument implementations.
+ */
+ TVM_DLL void InstrumentTearDown() const;
+
+ /*!
+ * \brief Call intrument implementatations before a pass run.
Review comment:
```suggestion
* \brief Call instrument implementations before a pass run.
```
##########
File path: python/tvm/ir/transform.py
##########
@@ -80,9 +88,13 @@ def __init__(
if not isinstance(disabled, (list, tuple)):
raise TypeError("disabled_pass is expected to be the type of " + "list/tuple/set.")
+ instruments = list(instruments) if instruments else []
+ if not isinstance(instruments, (list, tuple)):
+ raise TypeError("disabled_pass is expected to be the type of " + "list/tuple/set.")
Review comment:
also correct error message from `disabled_pass` -> `instruments`
##########
File path: src/ir/transform.cc
##########
@@ -162,170 +164,64 @@ void PassContext::RegisterConfigOption(const char* key, uint32_t value_type_inde
PassContext PassContext::Create() { return PassContext(make_object<PassContextNode>()); }
-void PassContext::Trace(const IRModule& module, const PassInfo& info, bool is_before) const {
+void PassContext::InstrumentSetUp() const {
auto pass_ctx_node = this->operator->();
- if (pass_ctx_node->trace_func != nullptr) {
- pass_ctx_node->trace_func(module, info, is_before);
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->SetUp();
+ }
}
}
-class ModulePass;
-
-/*! \brief PassProfile stores profiling information for a given pass and its sub-passes. */
-struct PassProfile {
- // TODO(@altanh): expose PassProfile through TVM Object API
- using Clock = std::chrono::steady_clock;
- using Duration = std::chrono::duration<double, std::micro>;
- using Time = std::chrono::time_point<Clock>;
-
- /*! \brief The name of the pass being profiled. */
- String name;
- /*! \brief The time when the pass was entered. */
- Time start;
- /*! \brief The time when the pass completed. */
- Time end;
- /*! \brief The total duration of the pass, i.e. end - start. */
- Duration duration;
- /*! \brief PassProfiles for all sub-passes invoked during the execution of the pass. */
- std::vector<PassProfile> children;
-
- explicit PassProfile(String name)
- : name(name), start(Clock::now()), end(Clock::now()), children() {}
-
- /*! \brief Gets the PassProfile of the currently executing pass. */
- static PassProfile* Current();
- /*! \brief Pushes a new PassProfile with the given pass name. */
- static void EnterPass(String name);
- /*! \brief Pops the current PassProfile. */
- static void ExitPass();
-};
-
-struct PassProfileThreadLocalEntry {
- /*! \brief The placeholder top-level PassProfile. */
- PassProfile root;
- /*! \brief The stack of PassProfiles for nested passes currently running. */
- std::stack<PassProfile*> profile_stack;
- /*! \brief Whether or not pass profiling is active. */
- bool active;
-
- PassProfileThreadLocalEntry() : root("root"), active(false) {}
-};
-
-/*! \brief Thread local store to hold the pass profiling data. */
-typedef dmlc::ThreadLocalStore<PassProfileThreadLocalEntry> PassProfileThreadLocalStore;
-
-void PassProfile::EnterPass(String name) {
- if (!PassProfileThreadLocalStore::Get()->active) return;
- PassProfile* cur = PassProfile::Current();
- cur->children.emplace_back(name);
- PassProfileThreadLocalStore::Get()->profile_stack.push(&cur->children.back());
+void PassContext::InstrumentTearDown() const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->TearDown();
+ }
+ }
}
-void PassProfile::ExitPass() {
- if (!PassProfileThreadLocalStore::Get()->active) return;
- PassProfile* cur = PassProfile::Current();
- ICHECK_NE(cur->name, "root") << "mismatched enter/exit for pass profiling";
- cur->end = std::move(PassProfile::Clock::now());
- cur->duration = std::chrono::duration_cast<PassProfile::Duration>(cur->end - cur->start);
- PassProfileThreadLocalStore::Get()->profile_stack.pop();
+bool PassContext::InstrumentBeforePass(const IRModule& ir_module, const PassInfo& pass_info) const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ if (!pi->RunBeforePass(ir_module, pass_info)) {
+ return false;
+ }
+ }
+ return true;
Review comment:
redundant return
##########
File path: src/tir/ir/transform.cc
##########
@@ -87,9 +87,11 @@ PrimFuncPass::PrimFuncPass(
// Perform Module -> Module optimizations at the PrimFunc level.
IRModule PrimFuncPassNode::operator()(IRModule mod, const PassContext& pass_ctx) const {
- const PassInfo& pass_info = Info();
Review comment:
what's happening here?
##########
File path: python/tvm/ir/instrument.py
##########
@@ -0,0 +1,154 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=invalid-name,unused-argument
+"""Common pass instrumentation across IR variants."""
+import inspect
+import functools
+
+import tvm._ffi
+import tvm.runtime
+
+from . import _ffi_instrument_api
+
+
+@tvm._ffi.register_object("instrument.PassInstrument")
+class PassInstrument(tvm.runtime.Object):
+ """A pass instrument implementation.
+
+ Users don't need to interact with this class directly.
+ Instead, a `PassInstrument` instance should be created through `pass_instrument`.
+
+ See Also
+ --------
+ `pass_instrument`
+ """
+
+
+def _wrap_class_pass_instrument(pi_cls):
+ """Wrap a python class as pass instrument"""
+
+ class PyPassInstrument(PassInstrument):
+ """Internal wrapper class to create a class instance."""
+
+ def __init__(self, *args, **kwargs):
+ # initialize handle in cass pi_cls creation failed.fg
+ self.handle = None
+ inst = pi_cls(*args, **kwargs)
+
+ # check method declartion within class, if found, wrap it.
+ def create_method(method):
+ if hasattr(inst, method) and inspect.ismethod(getattr(inst, method)):
+
+ def func(*args):
+ return getattr(inst, method)(*args)
+
+ func.__name__ = "_" + method
+ return func
+ return None
+
+ # create runtime pass instrument object
+ # reister instance's run_before_pass, run_after_pass, set_up and tear_down method
+ # to it if present.
+ self.__init_handle_by_constructor__(
+ _ffi_instrument_api.NamedPassInstrument,
+ pi_cls.__name__,
+ create_method("run_before_pass"),
+ create_method("run_after_pass"),
+ create_method("set_up"),
+ create_method("tear_down"),
+ )
+
+ self._inst = inst
+
+ def __getattr__(self, name):
+ # fall back to instance attribute if there is not any
+ return self._inst.__getattribute__(name)
+
+ functools.update_wrapper(PyPassInstrument.__init__, pi_cls.__init__)
+ PyPassInstrument.__name__ = pi_cls.__name__
+ PyPassInstrument.__doc__ = pi_cls.__doc__
+ PyPassInstrument.__module__ = pi_cls.__module__
+ return PyPassInstrument
+
+
+def pass_instrument(pi_cls=None):
+ """Decorate a pass instrument.
+
+ Parameters
+ ----------
+ pi_class :
+
+ Examples
+ --------
+ The following code block decorates a pass instrument class.
+
+ .. code-block:: python
+ @tvm.instrument.pass_instrument
+ class SkipPass:
+ def __init__(self, skip_pass_name):
+ self.skip_pass_name = skip_pass_name
+
+ # Uncomment to customize
+ # def set_up(self):
+ # pass
+
+ # Uncomment to customize
+ # def tear_down(self):
+ # pass
+
+ # If pass name contains keyword, skip it by return False. (return True: not skip)
+ def run_before_pass(self, mod, pass_info):
+ if self.skip_pass_name in pass_info.name:
+ return False
+ return True
+
+ # Uncomment to customize
+ # def run_after_pass(self, mod, pass_info):
+ # pass
+
+ skip_annotate = SkipPass("AnnotateSpans")
+ with tvm.transform.PassContext(instruments=[skip_annotate]):
+ tvm.relay.build(mod, "llvm")
+ """
+
+ def create_pass_instrument(pi_cls):
+ if not inspect.isclass(pi_cls):
+ raise TypeError("pi_cls must be a class")
+
+ return _wrap_class_pass_instrument(pi_cls)
+
+ if pi_cls:
+ return create_pass_instrument(pi_cls)
+ return create_pass_instrument
+
+
+@tvm._ffi.register_object("instrument.PassInstrument")
+class PassesTimeInstrument(tvm.runtime.Object):
Review comment:
could we rename this to something like `PassTimingInstrument`?
##########
File path: src/ir/transform.cc
##########
@@ -162,170 +164,64 @@ void PassContext::RegisterConfigOption(const char* key, uint32_t value_type_inde
PassContext PassContext::Create() { return PassContext(make_object<PassContextNode>()); }
-void PassContext::Trace(const IRModule& module, const PassInfo& info, bool is_before) const {
+void PassContext::InstrumentSetUp() const {
auto pass_ctx_node = this->operator->();
- if (pass_ctx_node->trace_func != nullptr) {
- pass_ctx_node->trace_func(module, info, is_before);
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->SetUp();
+ }
}
}
-class ModulePass;
-
-/*! \brief PassProfile stores profiling information for a given pass and its sub-passes. */
-struct PassProfile {
- // TODO(@altanh): expose PassProfile through TVM Object API
- using Clock = std::chrono::steady_clock;
- using Duration = std::chrono::duration<double, std::micro>;
- using Time = std::chrono::time_point<Clock>;
-
- /*! \brief The name of the pass being profiled. */
- String name;
- /*! \brief The time when the pass was entered. */
- Time start;
- /*! \brief The time when the pass completed. */
- Time end;
- /*! \brief The total duration of the pass, i.e. end - start. */
- Duration duration;
- /*! \brief PassProfiles for all sub-passes invoked during the execution of the pass. */
- std::vector<PassProfile> children;
-
- explicit PassProfile(String name)
- : name(name), start(Clock::now()), end(Clock::now()), children() {}
-
- /*! \brief Gets the PassProfile of the currently executing pass. */
- static PassProfile* Current();
- /*! \brief Pushes a new PassProfile with the given pass name. */
- static void EnterPass(String name);
- /*! \brief Pops the current PassProfile. */
- static void ExitPass();
-};
-
-struct PassProfileThreadLocalEntry {
- /*! \brief The placeholder top-level PassProfile. */
- PassProfile root;
- /*! \brief The stack of PassProfiles for nested passes currently running. */
- std::stack<PassProfile*> profile_stack;
- /*! \brief Whether or not pass profiling is active. */
- bool active;
-
- PassProfileThreadLocalEntry() : root("root"), active(false) {}
-};
-
-/*! \brief Thread local store to hold the pass profiling data. */
-typedef dmlc::ThreadLocalStore<PassProfileThreadLocalEntry> PassProfileThreadLocalStore;
-
-void PassProfile::EnterPass(String name) {
- if (!PassProfileThreadLocalStore::Get()->active) return;
- PassProfile* cur = PassProfile::Current();
- cur->children.emplace_back(name);
- PassProfileThreadLocalStore::Get()->profile_stack.push(&cur->children.back());
+void PassContext::InstrumentTearDown() const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->TearDown();
+ }
+ }
}
-void PassProfile::ExitPass() {
- if (!PassProfileThreadLocalStore::Get()->active) return;
- PassProfile* cur = PassProfile::Current();
- ICHECK_NE(cur->name, "root") << "mismatched enter/exit for pass profiling";
- cur->end = std::move(PassProfile::Clock::now());
- cur->duration = std::chrono::duration_cast<PassProfile::Duration>(cur->end - cur->start);
- PassProfileThreadLocalStore::Get()->profile_stack.pop();
+bool PassContext::InstrumentBeforePass(const IRModule& ir_module, const PassInfo& pass_info) const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ if (!pi->RunBeforePass(ir_module, pass_info)) {
+ return false;
+ }
+ }
+ return true;
+ }
+ return true;
}
-PassProfile* PassProfile::Current() {
- PassProfileThreadLocalEntry* entry = PassProfileThreadLocalStore::Get();
- if (!entry->profile_stack.empty()) {
- return entry->profile_stack.top();
- } else {
- return &entry->root;
+void PassContext::InstrumentAfterPass(const IRModule& ir_module, const PassInfo& pass_info) const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->RunAfterPass(ir_module, pass_info);
+ }
}
}
IRModule Pass::operator()(IRModule mod) const {
const PassNode* node = operator->();
ICHECK(node != nullptr);
- PassProfile::EnterPass(node->Info()->name);
+ // PassProfile::EnterPass(node->Info()->name);
auto ret = node->operator()(std::move(mod));
- PassProfile::ExitPass();
+ // PassProfile::ExitPass();
return std::move(ret);
}
IRModule Pass::operator()(IRModule mod, const PassContext& pass_ctx) const {
const PassNode* node = operator->();
ICHECK(node != nullptr);
- PassProfile::EnterPass(node->Info()->name);
+ // PassProfile::EnterPass(node->Info()->name);
Review comment:
```suggestion
```
##########
File path: src/ir/transform.cc
##########
@@ -162,170 +164,64 @@ void PassContext::RegisterConfigOption(const char* key, uint32_t value_type_inde
PassContext PassContext::Create() { return PassContext(make_object<PassContextNode>()); }
-void PassContext::Trace(const IRModule& module, const PassInfo& info, bool is_before) const {
+void PassContext::InstrumentSetUp() const {
auto pass_ctx_node = this->operator->();
- if (pass_ctx_node->trace_func != nullptr) {
- pass_ctx_node->trace_func(module, info, is_before);
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->SetUp();
+ }
}
}
-class ModulePass;
-
-/*! \brief PassProfile stores profiling information for a given pass and its sub-passes. */
-struct PassProfile {
- // TODO(@altanh): expose PassProfile through TVM Object API
- using Clock = std::chrono::steady_clock;
- using Duration = std::chrono::duration<double, std::micro>;
- using Time = std::chrono::time_point<Clock>;
-
- /*! \brief The name of the pass being profiled. */
- String name;
- /*! \brief The time when the pass was entered. */
- Time start;
- /*! \brief The time when the pass completed. */
- Time end;
- /*! \brief The total duration of the pass, i.e. end - start. */
- Duration duration;
- /*! \brief PassProfiles for all sub-passes invoked during the execution of the pass. */
- std::vector<PassProfile> children;
-
- explicit PassProfile(String name)
- : name(name), start(Clock::now()), end(Clock::now()), children() {}
-
- /*! \brief Gets the PassProfile of the currently executing pass. */
- static PassProfile* Current();
- /*! \brief Pushes a new PassProfile with the given pass name. */
- static void EnterPass(String name);
- /*! \brief Pops the current PassProfile. */
- static void ExitPass();
-};
-
-struct PassProfileThreadLocalEntry {
- /*! \brief The placeholder top-level PassProfile. */
- PassProfile root;
- /*! \brief The stack of PassProfiles for nested passes currently running. */
- std::stack<PassProfile*> profile_stack;
- /*! \brief Whether or not pass profiling is active. */
- bool active;
-
- PassProfileThreadLocalEntry() : root("root"), active(false) {}
-};
-
-/*! \brief Thread local store to hold the pass profiling data. */
-typedef dmlc::ThreadLocalStore<PassProfileThreadLocalEntry> PassProfileThreadLocalStore;
-
-void PassProfile::EnterPass(String name) {
- if (!PassProfileThreadLocalStore::Get()->active) return;
- PassProfile* cur = PassProfile::Current();
- cur->children.emplace_back(name);
- PassProfileThreadLocalStore::Get()->profile_stack.push(&cur->children.back());
+void PassContext::InstrumentTearDown() const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->TearDown();
+ }
+ }
}
-void PassProfile::ExitPass() {
- if (!PassProfileThreadLocalStore::Get()->active) return;
- PassProfile* cur = PassProfile::Current();
- ICHECK_NE(cur->name, "root") << "mismatched enter/exit for pass profiling";
- cur->end = std::move(PassProfile::Clock::now());
- cur->duration = std::chrono::duration_cast<PassProfile::Duration>(cur->end - cur->start);
- PassProfileThreadLocalStore::Get()->profile_stack.pop();
+bool PassContext::InstrumentBeforePass(const IRModule& ir_module, const PassInfo& pass_info) const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ if (!pi->RunBeforePass(ir_module, pass_info)) {
+ return false;
+ }
+ }
+ return true;
+ }
+ return true;
}
-PassProfile* PassProfile::Current() {
- PassProfileThreadLocalEntry* entry = PassProfileThreadLocalStore::Get();
- if (!entry->profile_stack.empty()) {
- return entry->profile_stack.top();
- } else {
- return &entry->root;
+void PassContext::InstrumentAfterPass(const IRModule& ir_module, const PassInfo& pass_info) const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->RunAfterPass(ir_module, pass_info);
+ }
}
}
IRModule Pass::operator()(IRModule mod) const {
const PassNode* node = operator->();
ICHECK(node != nullptr);
- PassProfile::EnterPass(node->Info()->name);
+ // PassProfile::EnterPass(node->Info()->name);
auto ret = node->operator()(std::move(mod));
- PassProfile::ExitPass();
+ // PassProfile::ExitPass();
return std::move(ret);
}
IRModule Pass::operator()(IRModule mod, const PassContext& pass_ctx) const {
const PassNode* node = operator->();
ICHECK(node != nullptr);
- PassProfile::EnterPass(node->Info()->name);
+ // PassProfile::EnterPass(node->Info()->name);
auto ret = node->operator()(std::move(mod), pass_ctx);
- PassProfile::ExitPass();
+ // PassProfile::ExitPass();
Review comment:
```suggestion
```
##########
File path: src/ir/transform.cc
##########
@@ -464,12 +360,19 @@ IRModule ModulePassNode::operator()(IRModule mod, const PassContext& pass_ctx) c
<< "The diagnostic context was set at the top of this block this is a bug.";
const PassInfo& pass_info = Info();
+ ICHECK(mod.defined()) << "The input module must be set.";
+
+ if (!pass_ctx.InstrumentBeforePass(mod, pass_info)) {
+ DLOG(INFO) << "Skipping function pass : " << pass_info->name
Review comment:
```suggestion
DLOG(INFO) << "Skipping module pass : " << pass_info->name
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
areusch commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r639966929
##########
File path: tests/python/relay/test_pass_instrument.py
##########
@@ -168,3 +169,329 @@ def run_after_pass(self, mod, info):
# Out of pass context scope, should be reset
assert passes_counter.run_before_count == 0
assert passes_counter.run_after_count == 0
+
+
+def test_enter_pass_ctx_exception(capsys):
+ @pass_instrument
+ class PI:
+ def __init__(self, id):
+ self.id = id
+
+ def enter_pass_ctx(self):
+ print(self.id + " enter ctx")
+
+ def exit_pass_ctx(self):
+ print(self.id + " exit ctx")
+
+ @pass_instrument
+ class PIBroken(PI):
+ def __init__(self, id):
+ super().__init__(id)
+
+ def enter_pass_ctx(self):
+ print(self.id + " enter ctx")
+ raise RuntimeError("Just a dummy error")
+
+ pass_ctx = tvm.transform.PassContext(instruments=[PI("%1"), PIBroken("%2"), PI("%3")])
+ with pytest.raises(tvm.error.TVMError):
+ with pass_ctx:
+ pass
+
+ assert "%1 enter ctx\n" "%2 enter ctx\n" == capsys.readouterr().out
Review comment:
> I am not quite understanding the sentence - i don't think you can implement __enter__ with knowledge of any following __enter___.
The tricky thing is that you're making `PassContext::EnterWithScope` into a composite with statement--in other words, calling `EnterWithScope` calls multiple `PassInstrument::EnterPassContext`. Each `EnterPassContext` can't know whether or not another `EnterPassContext` is going to error, so doing things like opening files isn't 100% safe here--a following `EnterPassContext` which errors would leave the file open. Depending on the severity of the error, the user could retry compilation and run into a more complex, harder-to-reproduce error.
However, in every other error case (i.e. if errors occur in shouldRun, beforePass, pass, afterPass), `EnterPassContext` can count on `ExitPassContext` running. So I'd just suggest to add logic to call `ExitPassContext` while looping through and calling `EnterPassContext`. I added a comment where I think it should go.
i think if you do that, then that will button up this interface and it should handle errors in the same way as you'd expect given logic in `include/tvm/support/with.h`. You can test this by adding the `%1 exit ctx` here.
lmk if that seems unreasonable, i could be missing something. i do think that fixing the hole in the ExitPassContext is worth the potential complexity of the double-error, because at least the user is working with a simpler abstraction layer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r639032543
##########
File path: src/ir/transform.cc
##########
@@ -164,34 +167,57 @@ void PassContext::RegisterConfigOption(const char* key, uint32_t value_type_inde
PassContext PassContext::Create() { return PassContext(make_object<PassContextNode>()); }
-void PassContext::InstrumentSetUp() const {
+void PassContext::InstrumentEnterPassContext() {
auto pass_ctx_node = this->operator->();
if (pass_ctx_node->instruments.defined()) {
- for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
- pi->SetUp();
+ try {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->EnterPassContext();
+ }
+ } catch (const Error& e) {
+ LOG(INFO) << "Pass instrumentation entering pass context failed.";
+ LOG(INFO) << "Disable pass instrumentation.";
+ pass_ctx_node->instruments.clear();
+ throw e;
}
}
}
-void PassContext::InstrumentTearDown() const {
+void PassContext::InstrumentExitPassContext() {
auto pass_ctx_node = this->operator->();
if (pass_ctx_node->instruments.defined()) {
- for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
- pi->TearDown();
+ try {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->ExitPassContext();
+ }
+ } catch (const Error& e) {
+ LOG(INFO) << "Pass instrumentation exiting pass context failed.";
+ pass_ctx_node->instruments.clear();
+ throw e;
}
}
}
bool PassContext::InstrumentBeforePass(const IRModule& ir_module, const PassInfo& pass_info) const {
auto pass_ctx_node = this->operator->();
- if (pass_ctx_node->instruments.defined()) {
+ if (!pass_ctx_node->instruments.defined()) {
+ return true;
+ }
+
+ const bool pass_required = PassArrayContains(pass_ctx_node->required_pass, pass_info->name);
+ bool should_run = true;
+ if (!pass_required) {
+ const Array<instrument::PassInstrument>& instruments = pass_ctx_node->instruments;
+ should_run &= std::all_of(instruments.begin(), instruments.end(),
Review comment:
Removed all_of (much like llvm's code above now).
For the second question, personally, I prefer to be consistent with llvm's API and its logics like above.
If there are strong desire to do that in the future, I will change it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tkonolige commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
tkonolige commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-830331744
Thanks for the contribution @zackcquic. In order to avoid duplicating code it might be worth unifying this with the runtime profiling framework (available here https://github.com/apache/tvm/blob/main/include/tvm/runtime/profiling.h). I have a branch (which I haven't submitted yet) that allows users to extend which kinds of information are collected. You can view the branch here: https://github.com/tkonolige/incubator-tvm/tree/profiler_papi. With this branch, I think the profiler supports all the features you want.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-846553197
### Collector of pass instrumentation instance
**Following table shows the class and API that collects a set of pass instrumentations in different projects:**
Note:
- [IO] = Pass instrumentations are processed in order
- [RO] = Pass instrumentations are processed in reverse order
- Entry empty = No mapping API
- mlir::PassInstrumentor provides thread-safety.
| This PR (PassContext.instruments) | llvm::PassInstrumentation | mlir::PassInstrumentor |
| -------------------------------------- | ----------------------------------- | ------------------------------ |
| [IO] bool InstrumentBeforePass() | [IO] bool runBeforePass() | [IO] void runBeforePass() |
| [IO] bool InstrumentAfterPass() | [IO] void runAfterPass() | [RO] void runAfterPass() |
| | [IO] void runAfterPassInvalidated() | |
| | | [RO] void runAfterPassFailed() |
| | [IO] void runBeforeAnalysis() | [IO] void runBeforeAnalysis() |
| | [IO] void runAfterAnalysis() | [RO] void runAfterAnalysis() |
| | [IO] void runAnalysisInvalidated() | |
| | [IO] void runAnalysesCleared() | |
| [IO] void InstrumentEnterPassContext() | | [IO] void runBeforePipeline() |
| [IO] void InstrumentExitPassContext() | | [RO] void runAfterPipeline() |
### Pass instrumentation instance
**Following table shows the class and API to implement a pass instrumentation in different projects:**
Note:
- In LLVM, a pass instrumentation provides a set of callbacks and register them in ```llvm::PassInstrumentationCallbacks```,
which will be called in collectors at different instrumentation points.
- In MLIR and this PR, pass instrumentation subclassing base class and overriding methods.
| This PR (instrument::PassInstrumentNode) | llvm::PassInstrumentationCallbacks | mlir::PassInstrumentation |
| ----------------------------------------- | ---------------------------------- | --------------------------|
| virtual bool ShouldRun() | void registerShouldRunOptionalPassCallback(CallableT) <br> // CallableT = bool(StringRef, Any), called in llvm::PassInstrumentation | |
| | void registerBeforeSkippedPassCallback(CallableT) <br> // CallableT = void(StringRef, Any), called in llvm::PassInstrumentation | |
| | void registerBeforeNonSkippedPassCallback(CallableT) <br> // CallableT = void(StringRef, Any), called in llvm::PassInstrumentation | |
| virtual void RunBeforePass() | | virtual void runBeforePass() |
| virtual void RunAfterPass() | void registerAfterPassCallback(CallableT) <br> // CallableT = void(StringRef, Any, const PreservedAnalyses &), called in llvm::PassInstrumentation | virtual void runAfterPass()
| |void registerAfterPassInvalidatedCallback(CallableT) <br> // CallableT = void(StringRef, const PreservedAnalyses &), called in llvm::PassInstrumentation | |
| | void registerBeforeAnalysisCallback(CallableT) <br> // CallableT = void(StringRef, Any), called in llvm::PassInstrumentation | virtual void runBeforeAnalysis() |
| | void registerAfterAnalysisCallback(CallableT) <br> // CallableT = void(StringRef, Any), called in llvm::PassInstrumentation | virtual void runAfterAnalysis() |
| | void registerAnalysisInvalidatedCallback(CallableT) <br> // CallableT = void(StringRef, Any), called in llvm::PassInstrumentation | |
| | void registerAnalysesClearedCallback(CallableT) <br> // CallableT = void(StringRef), called in llvm::PassInstrumentation | |
| virtual void EnterPassContext() | | virtual void runBeforePipeline() |
| virtual void ExitPassContext() | | virtual void runAfterPipeline() |
### RunBeforePass API
Following is the code snip from ```bool llvm::PassInstrumentation::runBeforePass()```.
It splits the ```ShouldRun()``` and ```RunBefore()``` logic from a pass instrumentation, like @areusch recommended.
```c++
/// BeforePass instrumentation point - takes \p Pass instance to be executed
/// and constant reference to IR it operates on. \Returns true if pass is
/// allowed to be executed. These are only run on optional pass since required
/// passes must always be run. This allows these callbacks to print info when
/// they want to skip a pass.
template <typename IRUnitT, typename PassT>
bool runBeforePass(const PassT &Pass, const IRUnitT &IR) const {
if (!Callbacks)
return true;
bool ShouldRun = true;
if (!isRequired(Pass)) {
for (auto &C : Callbacks->ShouldRunOptionalPassCallbacks)
ShouldRun &= C(Pass.name(), llvm::Any(&IR));
}
if (ShouldRun) {
for (auto &C : Callbacks->BeforeNonSkippedPassCallbacks)
C(Pass.name(), llvm::Any(&IR));
} else {
for (auto &C : Callbacks->BeforeSkippedPassCallbacks)
C(Pass.name(), llvm::Any(&IR));
}
return ShouldRun;
}
```
In the updated commit, I use this way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-846553311
@tkonolige @areusch @altanh @tqchen Changes are done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r631669510
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief A callback type for set up or clean up instrument environment.
+ */
+using InstrumentEnvFunc = runtime::TypedPackedFunc<void()>;
+
+/*!
+ * \brief A callback template for instrumenting before/after environment.
+ * \tparam RetTy the return type of callback.
+ */
+template <typename RetTy = void>
+using PassInstrumentFunc =
+ runtime::TypedPackedFunc<RetTy(const IRModule&, const transform::PassInfo&)>;
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
Review comment:
Do you mean making PassInstrument a virtual class? I agree with it.
Currently, PassInstrumentor and PassInstrument have different meanings, they don't have inheritance relationship (not like PassNode and ModulePassNode).
PassInstrumentor will collect a set of instrument implementations (PassInstrument), usage is like: [test_pass_instrument.py#L140](https://github.com/zackcquic/tvm/blob/0ad48d4f1e04f048bbae12daaec1cc100db458cd/tests/python/relay/test_pass_instrument.py#L140)
Do you think it's ok?
It not, could you explain more on "These callbacks can sit in implementation of instrument.cc" and "We can make PassInstrumentor a virtual class"?
Thanks a lot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tqchen edited a comment on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
tqchen edited a comment on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-830364732
Thank you @zackcquic ! Looks like a good addition, do you mind also create an RFC discussion post? I agree it is important to have pass instrumentations, would be good to know how can it interact with Trace, since some of the callbacks might be similar and we need to unify the interface.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r640196943
##########
File path: tests/python/relay/test_pass_instrument.py
##########
@@ -168,3 +169,329 @@ def run_after_pass(self, mod, info):
# Out of pass context scope, should be reset
assert passes_counter.run_before_count == 0
assert passes_counter.run_after_count == 0
+
+
+def test_enter_pass_ctx_exception(capsys):
+ @pass_instrument
+ class PI:
+ def __init__(self, id):
+ self.id = id
+
+ def enter_pass_ctx(self):
+ print(self.id + " enter ctx")
+
+ def exit_pass_ctx(self):
+ print(self.id + " exit ctx")
+
+ @pass_instrument
+ class PIBroken(PI):
+ def __init__(self, id):
+ super().__init__(id)
+
+ def enter_pass_ctx(self):
+ print(self.id + " enter ctx")
+ raise RuntimeError("Just a dummy error")
+
+ pass_ctx = tvm.transform.PassContext(instruments=[PI("%1"), PIBroken("%2"), PI("%3")])
+ with pytest.raises(tvm.error.TVMError):
+ with pass_ctx:
+ pass
+
+ assert "%1 enter ctx\n" "%2 enter ctx\n" == capsys.readouterr().out
Review comment:
Thanks for the detail explaining. I agree.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-842842084
> cc @jroesch @Hzfengsy @junrushao1994 @vinx13 @areusch @mbrookhart @yzhliu would be great if you can help to review this PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic edited a comment on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic edited a comment on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-843832260
> I think this is a valuable change and should pave way for more powerful tools in debugging and profiling TVM compilation. Others have left good comments, I have a few additional questions:
>
> * what happens when PassContexts are nested? I'm not sure if this is allowed today but it feels to me like they could compose
> * the `disabled_pass` option feels redundant now for PassContext, perhaps we should keep it for backwards compatibility but it could be easily replaced using a `DisableByName` instrument (or something like that).
> * is there a reason why we couldn't overload the superclass (`PassNode`) `operator()` for doing the before and after instrumentation? the new changes have some duplicated lines for each type of PassNode.
>
> Related to the last point, we also have a problem with the new design regarding the previous Pass timing profiling: in particular, as passes can be nested, **every nested pass invocation must use the `operator()(IRModule, PassContext)` overload** to ensure the pass is recorded in the timing. I'm pretty sure this is not the case in a lot of existing passes, so we must fix this globally for the timing to be precise. But since passes can be called without a PassContext, it might be possible that there is no PassContext- maybe there's a global/thread local one? I'm not sure.
>
> Thanks for the work!
Hi @altanh, thanks a lot for reviewing along with these great questions. 👍
**Q. What happens when PassContexts are nested? I'm not sure if this is allowed today but it feels to me like they could compose.**
A. Yes, PassContext can be nested. Instrument will be active in the same level (in the view of PassContext).
``` python
def test_print():
x, y, z = [tvm.relay.var(c, shape=(3, 4), dtype="float32") for c in "xyz"]
e1 = op.add(x, y)
e2 = op.subtract(x, z)
e3 = op.multiply(e1, e1 / e2)
mod = tvm.IRModule.from_expr(e3 + e2)
@tvm.instrument.pass_instrument
class NamePrinter:
def run_before_pass(self, mod, pass_info):
print(pass_info.name)
return True
print_name = NamePrinter()
with tvm.transform.PassContext(instruments=[print_name]):
mod = tvm.relay.transform.AnnotateSpans()(mod)
with tvm.transform.PassContext():
mod = tvm.relay.transform.ToANormalForm()(mod)
mod = tvm.relay.transform.InferType()(mod)
test_print()
# output:
# AnnotateSpans
# InferType # Note: InferType is a nested pass, same pass ctx with AnnotateSpans. Still instrumented, since finally it goes through ModulePassNode::operator().
#
# Inner level passes are not printed, since no instruments provided.
```
If users want to render nested passes like you did in pass timing, currently I don't have a good idea and I am not sure will it be common usage. The rendering effort currently is left to user.
Do you have any suggestions and more use scenarios (beside time)?
**Q. The `disabled_pass` option feels redundant now for PassContext, perhaps we should keep it for backwards compatibility but it could be easily replaced using a `DisableByName` instrument (or something like that).**
A. Yes, I agree. I left it here. Maybe use another PRs to add/remove these in the future.
**Q. Is there a reason why we couldn't overload the superclass (`PassNode`) `operator()` for doing the before and after instrumentation? the new changes have some duplicated lines for each type of PassNode.**
A. Yes, I aware that.
Basically, this PR is just an enhanced "trace" (```with PassContext(treace=xxx)```), I just replace how the trace are implemented.
For the reason not to overload superclass, I don't know, but I have a guess, maybe need someone to confirm:
Currently, only ```FunctionPass, ModulePass, SequentialPass and PrimFuncPass``` extends ```Pass```.
```FunctionPass``` calls ```InferType``` at the end (see the comments for detail) while ```SequentialPass``` is just like a wrapper.
In these cases, ```InterType``` and ```SequentialPass``` maybe not want to be traced/instrumented.
**Q. Since passes can be called without a PassContext, it might be possible that there is no PassContext- maybe there's a global/thread local one? I'm not sure.**
A. I thought about that, but I prefer the old "trace" way (No pass context, no trace/instrument).
If I missed something, please let me know. e.g. There is a way to enable "trace" globally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic removed a comment on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic removed a comment on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-843832677
@altanh @tkonolige thanks for reviewing.
Changes are done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r638393420
##########
File path: include/tvm/ir/transform.h
##########
@@ -189,12 +183,36 @@ class PassContext : public ObjectRef {
TVM_DLL static PassContext Current();
/*!
- * \brief Apply the tracing functions of the context to the module, with the info.
- * \param module The IRModule to trace.
+ * \brief Call instrument implementations' callbacks when entering PassContex.
+ * In order, if one raises exceptions, remaings will not be called.
+ */
+ TVM_DLL void InstrumentEnterPassContext();
+
+ /*!
+ * \brief Call instrument implementations' callback when exiting PassContext.
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r634817444
##########
File path: src/ir/instrument.cc
##########
@@ -0,0 +1,310 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file src/ir/instrument.cc
+ * \brief Infrastructure for instrumentation.
+ */
+#include <dmlc/thread_local.h>
+#include <tvm/ir/instrument.h>
+#include <tvm/ir/transform.h>
+#include <tvm/node/repr_printer.h>
+#include <tvm/runtime/registry.h>
+
+#include <stack>
+
+namespace tvm {
+namespace instrument {
+
+/*!
+ * \brief A named PassInstrument implementation
+ * \sa NamedPassInstrument
+ */
+class NamedPassInstrumentNode : public PassInstrumentNode {
+ public:
+ /*! \brief Name of this pass instrument object. */
+ String name;
+
+ /*! \brief Callback for instrumentation environment set up. */
+ runtime::TypedPackedFunc<void()> set_up_callback;
+ /*! \brief Callback for instrumentation environment clean up. */
+ runtime::TypedPackedFunc<void()> tear_down_callback;
+
+ /*! \brief Callback to run before a pass. */
+ runtime::TypedPackedFunc<bool(const IRModule&, const transform::PassInfo&)>
+ run_before_pass_callback;
+ /*! \brief Callback to run after a pass. */
+ runtime::TypedPackedFunc<void(const IRModule&, const transform::PassInfo&)>
+ run_after_pass_callback;
+
+ void VisitAttrs(AttrVisitor* v) { v->Visit("name", &name); }
+
+ /*! \brief Set up environment for instrumentation. */
+ void SetUp() const final;
+
+ /*! \brief Clean up instrumentation environment. */
+ void TearDown() const final;
+
+ /*!
+ * \brief Instrument before pass run, determine whether to run the pass or not.
+ * \param mod The module that an optimization pass runs on.
+ * \param info The pass information.
+ *
+ * \return true to run the pass; false to skip the pass.
+ */
+ bool RunBeforePass(const IRModule& mod, const transform::PassInfo& info) const final;
+
+ /*!
+ * \brief Instrument after pass run.
+ *
+ * \param mod The module that an optimization pass runs on.
+ * \param info The pass information.
+ */
+ void RunAfterPass(const IRModule& mod, const transform::PassInfo& info) const final;
+
+ static constexpr const char* _type_key = "instrument.NamedPassInstrument";
+ TVM_DECLARE_FINAL_OBJECT_INFO(NamedPassInstrumentNode, PassInstrumentNode);
+};
+
+/*!
+ * \brief Managed reference class for NamedPassInstrumentNode
+ * \sa NamedPassInstrumentNode
+ */
+class NamedPassInstrument : public PassInstrument {
+ public:
+ /*!
+ * \brief Constructor
+ * \param name Name for this instrumentation.
+ */
+ TVM_DLL NamedPassInstrument(String name);
Review comment:
Done
##########
File path: python/tvm/ir/transform.py
##########
@@ -65,12 +65,20 @@ class PassContext(tvm.runtime.Object):
disabled_pass : Optional[Union[List[str], Set[str], Tuple[str]]]
The list of passes that are disabled.
+ instruments : Optional[Union[List[PassInstrument], Set[PassInstrument], Tuple[PassInstrument]]]
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r639034502
##########
File path: src/ir/transform.cc
##########
@@ -164,34 +167,57 @@ void PassContext::RegisterConfigOption(const char* key, uint32_t value_type_inde
PassContext PassContext::Create() { return PassContext(make_object<PassContextNode>()); }
-void PassContext::InstrumentSetUp() const {
+void PassContext::InstrumentEnterPassContext() {
auto pass_ctx_node = this->operator->();
if (pass_ctx_node->instruments.defined()) {
- for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
- pi->SetUp();
+ try {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->EnterPassContext();
+ }
+ } catch (const Error& e) {
+ LOG(INFO) << "Pass instrumentation entering pass context failed.";
+ LOG(INFO) << "Disable pass instrumentation.";
+ pass_ctx_node->instruments.clear();
+ throw e;
}
}
}
-void PassContext::InstrumentTearDown() const {
+void PassContext::InstrumentExitPassContext() {
auto pass_ctx_node = this->operator->();
if (pass_ctx_node->instruments.defined()) {
- for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
- pi->TearDown();
+ try {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->ExitPassContext();
+ }
+ } catch (const Error& e) {
+ LOG(INFO) << "Pass instrumentation exiting pass context failed.";
+ pass_ctx_node->instruments.clear();
+ throw e;
}
}
}
bool PassContext::InstrumentBeforePass(const IRModule& ir_module, const PassInfo& pass_info) const {
auto pass_ctx_node = this->operator->();
- if (pass_ctx_node->instruments.defined()) {
+ if (!pass_ctx_node->instruments.defined()) {
+ return true;
+ }
+
+ const bool pass_required = PassArrayContains(pass_ctx_node->required_pass, pass_info->name);
+ bool should_run = true;
+ if (!pass_required) {
+ const Array<instrument::PassInstrument>& instruments = pass_ctx_node->instruments;
+ should_run &= std::all_of(instruments.begin(), instruments.end(),
Review comment:
Removed all_of (much like llvm's code above now).
For the second question, personally, I prefer to be consistent with llvm's API and its logics like above.
If there are strong desire to do that in the future, I will change it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic edited a comment on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic edited a comment on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-840237999
@tqchen Thanks a lot for review.
I just make modifications based on the comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r639396559
##########
File path: tests/python/relay/test_pass_instrument.py
##########
@@ -168,3 +169,329 @@ def run_after_pass(self, mod, info):
# Out of pass context scope, should be reset
assert passes_counter.run_before_count == 0
assert passes_counter.run_after_count == 0
+
+
+def test_enter_pass_ctx_exception(capsys):
+ @pass_instrument
+ class PI:
+ def __init__(self, id):
+ self.id = id
+
+ def enter_pass_ctx(self):
+ print(self.id + " enter ctx")
+
+ def exit_pass_ctx(self):
+ print(self.id + " exit ctx")
+
+ @pass_instrument
+ class PIBroken(PI):
+ def __init__(self, id):
+ super().__init__(id)
+
+ def enter_pass_ctx(self):
+ print(self.id + " enter ctx")
+ raise RuntimeError("Just a dummy error")
+
+ pass_ctx = tvm.transform.PassContext(instruments=[PI("%1"), PIBroken("%2"), PI("%3")])
+ with pytest.raises(tvm.error.TVMError):
+ with pass_ctx:
+ pass
+
+ assert "%1 enter ctx\n" "%2 enter ctx\n" == capsys.readouterr().out
Review comment:
I am not quite understanding the sentence - i don't think you can implement `__enter__` with knowledge of any following `__enter___`.
I think ```__enter__``` and ```__exit__``` throw exceptions is a common problem (like following example) beyond the scope of this PR. Maybe we can use another RFC/PR to address this?
```python
class OpenFile:
def __init__(self, file_name):
self.file_name = file_name
def __enter__(self):
print("Try opening " + self.file_name)
self.file = open(self.file_name)
print("Open " + self.file_name + "sccussed")
return self.file
def __exit__(self, exec_type, exc_eval, traceback):
print("Try colsing " + self.file_name)
self.file.close();
print(self.file_name + " closed")
with OpenFile("zz"):
pass
# output
Try opening zz
Traceback (most recent call last):
File "test.py", line 18, in <module>
with OpenFile("zz"):
File "test.py", line 8, in __enter__
self.file = open(self.file_name)
FileNotFoundError: [Errno 2] No such file or directory: 'zz'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
areusch commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r639382306
##########
File path: tests/python/relay/test_pass_instrument.py
##########
@@ -168,3 +169,329 @@ def run_after_pass(self, mod, info):
# Out of pass context scope, should be reset
assert passes_counter.run_before_count == 0
assert passes_counter.run_after_count == 0
+
+
+def test_enter_pass_ctx_exception(capsys):
+ @pass_instrument
+ class PI:
+ def __init__(self, id):
+ self.id = id
+
+ def enter_pass_ctx(self):
+ print(self.id + " enter ctx")
+
+ def exit_pass_ctx(self):
+ print(self.id + " exit ctx")
+
+ @pass_instrument
+ class PIBroken(PI):
+ def __init__(self, id):
+ super().__init__(id)
+
+ def enter_pass_ctx(self):
+ print(self.id + " enter ctx")
+ raise RuntimeError("Just a dummy error")
+
+ pass_ctx = tvm.transform.PassContext(instruments=[PI("%1"), PIBroken("%2"), PI("%3")])
+ with pytest.raises(tvm.error.TVMError):
+ with pass_ctx:
+ pass
+
+ assert "%1 enter ctx\n" "%2 enter ctx\n" == capsys.readouterr().out
Review comment:
hrm...I guess this one has no analogue in LLVM. but I'm not sure I agree--I think if an `__enter__` returns succesfully, the contract is that it's expected the `__exit__` will get called. i don't think you can implement `__enter__` with knowledge of any following `__enter__`, so this case would potentially leave state initialized...
since catching `std::runtime_error` is how we handle errors at the FFI, it seems like we should be able to catch that here and invoke the `__exit__` in reverse order. on the other hand, this gets messy, since `__exit__` can also throw exceptions. @tqchen maybe you have some thoughts on doing such complex stuff inside `libtvm_runtime`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r634818156
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic closed pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic closed pull request #7952:
URL: https://github.com/apache/tvm/pull/7952
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tqchen commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
tqchen commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-844372490
I believe traditional compilers like LLVM also have separate disable pass options. The situation being described seems also applies to the traditional compiler setup, so while this is something useful to note in the docs, I believe follow the lead of well established API is a good choice than have a specific opinion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r634902889
##########
File path: src/ir/transform.cc
##########
@@ -162,170 +164,64 @@ void PassContext::RegisterConfigOption(const char* key, uint32_t value_type_inde
PassContext PassContext::Create() { return PassContext(make_object<PassContextNode>()); }
-void PassContext::Trace(const IRModule& module, const PassInfo& info, bool is_before) const {
+void PassContext::InstrumentSetUp() const {
auto pass_ctx_node = this->operator->();
- if (pass_ctx_node->trace_func != nullptr) {
- pass_ctx_node->trace_func(module, info, is_before);
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->SetUp();
+ }
}
}
-class ModulePass;
-
-/*! \brief PassProfile stores profiling information for a given pass and its sub-passes. */
-struct PassProfile {
- // TODO(@altanh): expose PassProfile through TVM Object API
- using Clock = std::chrono::steady_clock;
- using Duration = std::chrono::duration<double, std::micro>;
- using Time = std::chrono::time_point<Clock>;
-
- /*! \brief The name of the pass being profiled. */
- String name;
- /*! \brief The time when the pass was entered. */
- Time start;
- /*! \brief The time when the pass completed. */
- Time end;
- /*! \brief The total duration of the pass, i.e. end - start. */
- Duration duration;
- /*! \brief PassProfiles for all sub-passes invoked during the execution of the pass. */
- std::vector<PassProfile> children;
-
- explicit PassProfile(String name)
- : name(name), start(Clock::now()), end(Clock::now()), children() {}
-
- /*! \brief Gets the PassProfile of the currently executing pass. */
- static PassProfile* Current();
- /*! \brief Pushes a new PassProfile with the given pass name. */
- static void EnterPass(String name);
- /*! \brief Pops the current PassProfile. */
- static void ExitPass();
-};
-
-struct PassProfileThreadLocalEntry {
- /*! \brief The placeholder top-level PassProfile. */
- PassProfile root;
- /*! \brief The stack of PassProfiles for nested passes currently running. */
- std::stack<PassProfile*> profile_stack;
- /*! \brief Whether or not pass profiling is active. */
- bool active;
-
- PassProfileThreadLocalEntry() : root("root"), active(false) {}
-};
-
-/*! \brief Thread local store to hold the pass profiling data. */
-typedef dmlc::ThreadLocalStore<PassProfileThreadLocalEntry> PassProfileThreadLocalStore;
-
-void PassProfile::EnterPass(String name) {
- if (!PassProfileThreadLocalStore::Get()->active) return;
- PassProfile* cur = PassProfile::Current();
- cur->children.emplace_back(name);
- PassProfileThreadLocalStore::Get()->profile_stack.push(&cur->children.back());
+void PassContext::InstrumentTearDown() const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->TearDown();
+ }
+ }
}
-void PassProfile::ExitPass() {
- if (!PassProfileThreadLocalStore::Get()->active) return;
- PassProfile* cur = PassProfile::Current();
- ICHECK_NE(cur->name, "root") << "mismatched enter/exit for pass profiling";
- cur->end = std::move(PassProfile::Clock::now());
- cur->duration = std::chrono::duration_cast<PassProfile::Duration>(cur->end - cur->start);
- PassProfileThreadLocalStore::Get()->profile_stack.pop();
+bool PassContext::InstrumentBeforePass(const IRModule& ir_module, const PassInfo& pass_info) const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ if (!pi->RunBeforePass(ir_module, pass_info)) {
+ return false;
+ }
+ }
+ return true;
+ }
+ return true;
}
-PassProfile* PassProfile::Current() {
- PassProfileThreadLocalEntry* entry = PassProfileThreadLocalStore::Get();
- if (!entry->profile_stack.empty()) {
- return entry->profile_stack.top();
- } else {
- return &entry->root;
+void PassContext::InstrumentAfterPass(const IRModule& ir_module, const PassInfo& pass_info) const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->RunAfterPass(ir_module, pass_info);
+ }
}
}
IRModule Pass::operator()(IRModule mod) const {
const PassNode* node = operator->();
ICHECK(node != nullptr);
- PassProfile::EnterPass(node->Info()->name);
+ // PassProfile::EnterPass(node->Info()->name);
auto ret = node->operator()(std::move(mod));
- PassProfile::ExitPass();
+ // PassProfile::ExitPass();
return std::move(ret);
}
IRModule Pass::operator()(IRModule mod, const PassContext& pass_ctx) const {
const PassNode* node = operator->();
ICHECK(node != nullptr);
- PassProfile::EnterPass(node->Info()->name);
+ // PassProfile::EnterPass(node->Info()->name);
Review comment:
Removed
##########
File path: src/ir/transform.cc
##########
@@ -162,170 +164,64 @@ void PassContext::RegisterConfigOption(const char* key, uint32_t value_type_inde
PassContext PassContext::Create() { return PassContext(make_object<PassContextNode>()); }
-void PassContext::Trace(const IRModule& module, const PassInfo& info, bool is_before) const {
+void PassContext::InstrumentSetUp() const {
auto pass_ctx_node = this->operator->();
- if (pass_ctx_node->trace_func != nullptr) {
- pass_ctx_node->trace_func(module, info, is_before);
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->SetUp();
+ }
}
}
-class ModulePass;
-
-/*! \brief PassProfile stores profiling information for a given pass and its sub-passes. */
-struct PassProfile {
- // TODO(@altanh): expose PassProfile through TVM Object API
- using Clock = std::chrono::steady_clock;
- using Duration = std::chrono::duration<double, std::micro>;
- using Time = std::chrono::time_point<Clock>;
-
- /*! \brief The name of the pass being profiled. */
- String name;
- /*! \brief The time when the pass was entered. */
- Time start;
- /*! \brief The time when the pass completed. */
- Time end;
- /*! \brief The total duration of the pass, i.e. end - start. */
- Duration duration;
- /*! \brief PassProfiles for all sub-passes invoked during the execution of the pass. */
- std::vector<PassProfile> children;
-
- explicit PassProfile(String name)
- : name(name), start(Clock::now()), end(Clock::now()), children() {}
-
- /*! \brief Gets the PassProfile of the currently executing pass. */
- static PassProfile* Current();
- /*! \brief Pushes a new PassProfile with the given pass name. */
- static void EnterPass(String name);
- /*! \brief Pops the current PassProfile. */
- static void ExitPass();
-};
-
-struct PassProfileThreadLocalEntry {
- /*! \brief The placeholder top-level PassProfile. */
- PassProfile root;
- /*! \brief The stack of PassProfiles for nested passes currently running. */
- std::stack<PassProfile*> profile_stack;
- /*! \brief Whether or not pass profiling is active. */
- bool active;
-
- PassProfileThreadLocalEntry() : root("root"), active(false) {}
-};
-
-/*! \brief Thread local store to hold the pass profiling data. */
-typedef dmlc::ThreadLocalStore<PassProfileThreadLocalEntry> PassProfileThreadLocalStore;
-
-void PassProfile::EnterPass(String name) {
- if (!PassProfileThreadLocalStore::Get()->active) return;
- PassProfile* cur = PassProfile::Current();
- cur->children.emplace_back(name);
- PassProfileThreadLocalStore::Get()->profile_stack.push(&cur->children.back());
+void PassContext::InstrumentTearDown() const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->TearDown();
+ }
+ }
}
-void PassProfile::ExitPass() {
- if (!PassProfileThreadLocalStore::Get()->active) return;
- PassProfile* cur = PassProfile::Current();
- ICHECK_NE(cur->name, "root") << "mismatched enter/exit for pass profiling";
- cur->end = std::move(PassProfile::Clock::now());
- cur->duration = std::chrono::duration_cast<PassProfile::Duration>(cur->end - cur->start);
- PassProfileThreadLocalStore::Get()->profile_stack.pop();
+bool PassContext::InstrumentBeforePass(const IRModule& ir_module, const PassInfo& pass_info) const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ if (!pi->RunBeforePass(ir_module, pass_info)) {
+ return false;
+ }
+ }
+ return true;
+ }
+ return true;
}
-PassProfile* PassProfile::Current() {
- PassProfileThreadLocalEntry* entry = PassProfileThreadLocalStore::Get();
- if (!entry->profile_stack.empty()) {
- return entry->profile_stack.top();
- } else {
- return &entry->root;
+void PassContext::InstrumentAfterPass(const IRModule& ir_module, const PassInfo& pass_info) const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->RunAfterPass(ir_module, pass_info);
+ }
}
}
IRModule Pass::operator()(IRModule mod) const {
const PassNode* node = operator->();
ICHECK(node != nullptr);
- PassProfile::EnterPass(node->Info()->name);
+ // PassProfile::EnterPass(node->Info()->name);
auto ret = node->operator()(std::move(mod));
- PassProfile::ExitPass();
+ // PassProfile::ExitPass();
return std::move(ret);
}
IRModule Pass::operator()(IRModule mod, const PassContext& pass_ctx) const {
const PassNode* node = operator->();
ICHECK(node != nullptr);
- PassProfile::EnterPass(node->Info()->name);
+ // PassProfile::EnterPass(node->Info()->name);
auto ret = node->operator()(std::move(mod), pass_ctx);
- PassProfile::ExitPass();
+ // PassProfile::ExitPass();
Review comment:
Removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r639027780
##########
File path: tests/python/relay/test_pass_instrument.py
##########
@@ -14,77 +14,91 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
+""" Instrument test cases.
+"""
+import pytest
import tvm
import tvm.relay
from tvm.relay import op
from tvm.ir.instrument import PassTimingInstrument, pass_instrument
-def test_pass_timing_instrument():
+def get_test_model():
x, y, z = [tvm.relay.var(c, shape=(3, 4), dtype="float32") for c in "xyz"]
e1 = op.add(x, y)
e2 = op.subtract(x, z)
e3 = op.multiply(e1, e1 / e2)
- mod = tvm.IRModule.from_expr(e3 + e2)
+ return tvm.IRModule.from_expr(e3 + e2)
+
+def test_pass_timing_instrument():
pass_timing = PassTimingInstrument()
- with tvm.transform.PassContext(instruments=[pass_timing]):
- mod = tvm.relay.transform.AnnotateSpans()(mod)
- mod = tvm.relay.transform.ToANormalForm()(mod)
- mod = tvm.relay.transform.InferType()(mod)
- profiles = pass_timing.render()
- assert "AnnotateSpans" in profiles
- assert "ToANormalForm" in profiles
- assert "InferType" in profiles
+ # Override current PassContext's instruments
+ tvm.transform.PassContext.current().override_instruments([pass_timing])
+ mod = get_test_model()
+ mod = tvm.relay.transform.AnnotateSpans()(mod)
+ mod = tvm.relay.transform.ToANormalForm()(mod)
+ mod = tvm.relay.transform.InferType()(mod)
-def test_custom_instrument(capsys):
- x, y, z = [tvm.relay.var(c, shape=(3, 4), dtype="float32") for c in "xyz"]
- e1 = op.add(x, y)
- e2 = op.subtract(x, z)
- e3 = op.multiply(e1, e1 / e2)
- mod = tvm.IRModule.from_expr(e3 + e2)
+ profiles = pass_timing.render()
+ assert "AnnotateSpans" in profiles
+ assert "ToANormalForm" in profiles
+ assert "InferType" in profiles
+
+ # Reset current PassContext's instruments to None
+ tvm.transform.PassContext.current().override_instruments(None)
+ mod = get_test_model()
+ mod = tvm.relay.transform.AnnotateSpans()(mod)
+ mod = tvm.relay.transform.ToANormalForm()(mod)
+ mod = tvm.relay.transform.InferType()(mod)
+
+ profiles = pass_timing.render()
+ assert profiles == ""
+
+
+def test_custom_instrument(capsys):
@pass_instrument
class MyTest:
- def set_up(self):
- print("set up")
+ def enter_pass_ctx(self):
+ print("enter ctx")
Review comment:
Done
print and capsys removed with events
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-833095927
> I will try and comment over the weekend, thanks for the PR!
Hi @jroesch:
Just a gentle ping to receive feedback on this PR.
Thanks,
Zack
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r639396559
##########
File path: tests/python/relay/test_pass_instrument.py
##########
@@ -168,3 +169,329 @@ def run_after_pass(self, mod, info):
# Out of pass context scope, should be reset
assert passes_counter.run_before_count == 0
assert passes_counter.run_after_count == 0
+
+
+def test_enter_pass_ctx_exception(capsys):
+ @pass_instrument
+ class PI:
+ def __init__(self, id):
+ self.id = id
+
+ def enter_pass_ctx(self):
+ print(self.id + " enter ctx")
+
+ def exit_pass_ctx(self):
+ print(self.id + " exit ctx")
+
+ @pass_instrument
+ class PIBroken(PI):
+ def __init__(self, id):
+ super().__init__(id)
+
+ def enter_pass_ctx(self):
+ print(self.id + " enter ctx")
+ raise RuntimeError("Just a dummy error")
+
+ pass_ctx = tvm.transform.PassContext(instruments=[PI("%1"), PIBroken("%2"), PI("%3")])
+ with pytest.raises(tvm.error.TVMError):
+ with pass_ctx:
+ pass
+
+ assert "%1 enter ctx\n" "%2 enter ctx\n" == capsys.readouterr().out
Review comment:
I am not quite understanding the sentence - i don't think you can implement `__enter__` with knowledge of any following `__enter___`.
I think ```__enter__``` and ```__exit__``` throw exception is a common problem. Maybe use another RFC/PR to address this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r631669510
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief A callback type for set up or clean up instrument environment.
+ */
+using InstrumentEnvFunc = runtime::TypedPackedFunc<void()>;
+
+/*!
+ * \brief A callback template for instrumenting before/after environment.
+ * \tparam RetTy the return type of callback.
+ */
+template <typename RetTy = void>
+using PassInstrumentFunc =
+ runtime::TypedPackedFunc<RetTy(const IRModule&, const transform::PassInfo&)>;
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
Review comment:
Do you mean making PassInstrument a virtual class? I agree with it.
Currently, PassInstrumentor and PassInstrument have different meanings, they don't have inheritance relationship (not like PassNode and ModulePassNode).
PassInstrumentor will collect a set of instrument implementations (PassInstrument), usage is like: [test_pass_instrument.py#L140](https://github.com/zackcquic/tvm/blob/0ad48d4f1e04f048bbae12daaec1cc100db458cd/tests/python/relay/test_pass_instrument.py#L140)
Do you think it's ok?
It not, could you explain more on "These callbacks can sit in implementation of instrument.cc" and "We can make PassInstrumentor a virtual class"?
Thanks a lot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic edited a comment on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic edited a comment on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-847472910
> Thanks for the changes, this looks much better now! Mostly spelling nits here, but also could you please add back the test for the `PassTimingInstrument`? Looks like it got removed here (previously was in `tests/python/relay/test_pass_profiler.py`).
I renamed `test_pass_profiler.py` to `test_pass_instrument.py`, and `PassTimingInstrument` are tested in this file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tqchen commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
tqchen commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r633897767
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief A callback type for set up or clean up instrument environment.
+ */
+using InstrumentEnvFunc = runtime::TypedPackedFunc<void()>;
+
+/*!
+ * \brief A callback template for instrumenting before/after environment.
+ * \tparam RetTy the return type of callback.
+ */
+template <typename RetTy = void>
+using PassInstrumentFunc =
+ runtime::TypedPackedFunc<RetTy(const IRModule&, const transform::PassInfo&)>;
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
Review comment:
Sorry, I actually meant making PassIntrumentNode a pure virtual class containing the method, and only implement a CallbackInstrumentor(used to back python side) on the cc file that is used by the python side
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic removed a comment on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic removed a comment on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-846553311
@tkonolige @areusch @altanh @tqchen Changes are done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r631678904
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief A callback type for set up or clean up instrument environment.
+ */
+using InstrumentEnvFunc = runtime::TypedPackedFunc<void()>;
+
+/*!
+ * \brief A callback template for instrumenting before/after environment.
+ * \tparam RetTy the return type of callback.
+ */
+template <typename RetTy = void>
+using PassInstrumentFunc =
+ runtime::TypedPackedFunc<RetTy(const IRModule&, const transform::PassInfo&)>;
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
Review comment:
I made PassInstrumentNode able to sub-classing.
But I kept it in header files for that transform.cc needs the method declarations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r638396364
##########
File path: tests/python/relay/test_pass_manager.py
##########
@@ -559,15 +569,20 @@ def test_print_debug_callback():
]
)
- assert __TRACE_COUNTER__ == 0
mod = tvm.IRModule({"main": func})
- with tvm.transform.PassContext(opt_level=3, trace=_tracer):
+ pass_counter = PassCounter()
+ with tvm.transform.PassContext(opt_level=3, instruments=[pass_counter]):
+ # Should be reseted when entering pass context
+ assert pass_counter.get_counts() == 0
mod = seq(mod)
- # TODO(@jroesch): when we remove new fn pass behavior we need to remove
- # change this back to 3
- assert __TRACE_COUNTER__ == 5
+ # TODO(@jroesch): when we remove new fn pass behavior we need to remove
+ # change this back to match correct behavior
+ assert pass_counter.get_counts() == 6
Review comment:
As mentioned before, FunctionPassNode's InfterType() and SequentialPass() are not traced before.
In this PR, we override pass::operator() to instrument instead of instrumenting in subclasses::operator().
In this test case, sequential pass now is instrumented (counted).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r634817629
##########
File path: python/tvm/ir/transform.py
##########
@@ -80,9 +88,13 @@ def __init__(
if not isinstance(disabled, (list, tuple)):
raise TypeError("disabled_pass is expected to be the type of " + "list/tuple/set.")
+ instruments = list(instruments) if instruments else []
+ if not isinstance(instruments, (list, tuple)):
+ raise TypeError("disabled_pass is expected to be the type of " + "list/tuple/set.")
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic removed a comment on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic removed a comment on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-848374711
@areusch @tqchen changes are done.
Thanks a lot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tkonolige commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
tkonolige commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r634823956
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
+ virtual ~PassInstrumentNode() {}
+
+ /*! \brief Set up environment for instrumentation. */
+ virtual void SetUp() const = 0;
+
+ /*! \brief Clean up instrumentation environment. */
+ virtual void TearDown() const = 0;
+
+ /*!
+ * \brief Instrument before pass run, determine whether to run the pass or not.
+ * \param mod The module that an optimization pass runs on.
+ * \param info The pass information.
+ *
+ * \return true to run the pass; false to skip the pass.
+ */
+ virtual bool RunBeforePass(const IRModule& mod, const transform::PassInfo& info) const = 0;
Review comment:
Well, passes can be nested, so `RunBeforePass` needs to be able to handle nested calls.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r637521860
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired by LLVM and MLIR.
+ * It inserts instrumentation points around passes.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrumentation points.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp()
+ *
+ * if (Instrument Before Pass())
Review comment:
Just checked LLVM and its logic is pretty like you commented.
Logic updated and test cases added.
For _**providing the after_pass with a "result" notion**_, could you give more elaboration?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] altanh commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
altanh commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r638212865
##########
File path: include/tvm/ir/transform.h
##########
@@ -189,12 +183,36 @@ class PassContext : public ObjectRef {
TVM_DLL static PassContext Current();
/*!
- * \brief Apply the tracing functions of the context to the module, with the info.
- * \param module The IRModule to trace.
+ * \brief Call instrument implementations' callbacks when entering PassContex.
+ * In order, if one raises exceptions, remaings will not be called.
+ */
+ TVM_DLL void InstrumentEnterPassContext();
+
+ /*!
+ * \brief Call instrument implementations' callback when exiting PassContext.
Review comment:
```suggestion
* \brief Call instrument implementations' callbacks when exiting PassContext.
```
##########
File path: include/tvm/ir/transform.h
##########
@@ -189,12 +183,36 @@ class PassContext : public ObjectRef {
TVM_DLL static PassContext Current();
/*!
- * \brief Apply the tracing functions of the context to the module, with the info.
- * \param module The IRModule to trace.
+ * \brief Call instrument implementations' callbacks when entering PassContex.
+ * In order, if one raises exceptions, remaings will not be called.
+ */
+ TVM_DLL void InstrumentEnterPassContext();
+
+ /*!
+ * \brief Call instrument implementations' callback when exiting PassContext.
+ * In order, if one raises exceptions, remaings will not be called.
Review comment:
see above
##########
File path: include/tvm/ir/transform.h
##########
@@ -189,12 +183,36 @@ class PassContext : public ObjectRef {
TVM_DLL static PassContext Current();
/*!
- * \brief Apply the tracing functions of the context to the module, with the info.
- * \param module The IRModule to trace.
+ * \brief Call instrument implementations' callbacks when entering PassContex.
Review comment:
```suggestion
* \brief Call instrument implementations' callbacks when entering PassContext.
```
##########
File path: include/tvm/ir/transform.h
##########
@@ -189,12 +183,36 @@ class PassContext : public ObjectRef {
TVM_DLL static PassContext Current();
/*!
- * \brief Apply the tracing functions of the context to the module, with the info.
- * \param module The IRModule to trace.
+ * \brief Call instrument implementations' callbacks when entering PassContex.
+ * In order, if one raises exceptions, remaings will not be called.
Review comment:
```suggestion
* The callbacks are called In order, and if one raises an exception, the rest will not be called.
```
##########
File path: python/tvm/ir/transform.py
##########
@@ -92,6 +104,9 @@ def __enter__(self):
def __exit__(self, ptype, value, trace):
_ffi_transform_api.ExitPassContext(self)
+ def override_instruments(self, instruments):
Review comment:
could you add a docstring here to describe this method
##########
File path: include/tvm/ir/transform.h
##########
@@ -189,12 +183,36 @@ class PassContext : public ObjectRef {
TVM_DLL static PassContext Current();
/*!
- * \brief Apply the tracing functions of the context to the module, with the info.
- * \param module The IRModule to trace.
+ * \brief Call instrument implementations' callbacks when entering PassContex.
+ * In order, if one raises exceptions, remaings will not be called.
+ */
+ TVM_DLL void InstrumentEnterPassContext();
+
+ /*!
+ * \brief Call instrument implementations' callback when exiting PassContext.
+ * In order, if one raises exceptions, remaings will not be called.
+ */
+ TVM_DLL void InstrumentExitPassContext();
+
+ /*!
+ * \brief Call intrument implementations' callbacks before a pass run.
Review comment:
```suggestion
* \brief Call instrument implementations' callbacks before a pass run.
```
##########
File path: include/tvm/ir/transform.h
##########
@@ -189,12 +183,36 @@ class PassContext : public ObjectRef {
TVM_DLL static PassContext Current();
/*!
- * \brief Apply the tracing functions of the context to the module, with the info.
- * \param module The IRModule to trace.
+ * \brief Call instrument implementations' callbacks when entering PassContex.
+ * In order, if one raises exceptions, remaings will not be called.
+ */
+ TVM_DLL void InstrumentEnterPassContext();
+
+ /*!
+ * \brief Call instrument implementations' callback when exiting PassContext.
+ * In order, if one raises exceptions, remaings will not be called.
+ */
+ TVM_DLL void InstrumentExitPassContext();
+
+ /*!
+ * \brief Call intrument implementations' callbacks before a pass run.
+ * In order, if one raises exceptions, remaings will not be called.
Review comment:
see previous suggestion
##########
File path: include/tvm/ir/transform.h
##########
@@ -189,12 +183,36 @@ class PassContext : public ObjectRef {
TVM_DLL static PassContext Current();
/*!
- * \brief Apply the tracing functions of the context to the module, with the info.
- * \param module The IRModule to trace.
+ * \brief Call instrument implementations' callbacks when entering PassContex.
+ * In order, if one raises exceptions, remaings will not be called.
+ */
+ TVM_DLL void InstrumentEnterPassContext();
+
+ /*!
+ * \brief Call instrument implementations' callback when exiting PassContext.
+ * In order, if one raises exceptions, remaings will not be called.
+ */
+ TVM_DLL void InstrumentExitPassContext();
+
+ /*!
+ * \brief Call intrument implementations' callbacks before a pass run.
+ * In order, if one raises exceptions, remaings will not be called.
+ *
+ * \param mod The module that an optimization pass runs on.
* \param info The pass information.
- * \param is_before Indicated whether the tracing is before or after a pass.
+ *
+ * \return false: the pass is skipped; true: the pass runs.
*/
- TVM_DLL void Trace(const IRModule& module, const PassInfo& info, bool is_before) const;
+ TVM_DLL bool InstrumentBeforePass(const IRModule& mod, const PassInfo& info) const;
+
+ /*!
+ * \brief Call instrument implementations callbacks after a pass run.
+ * In order, if one raises exceptions, remaings will not be called.
Review comment:
see previous
##########
File path: tests/python/relay/test_pass_manager.py
##########
@@ -559,15 +569,20 @@ def test_print_debug_callback():
]
)
- assert __TRACE_COUNTER__ == 0
mod = tvm.IRModule({"main": func})
- with tvm.transform.PassContext(opt_level=3, trace=_tracer):
+ pass_counter = PassCounter()
+ with tvm.transform.PassContext(opt_level=3, instruments=[pass_counter]):
+ # Should be reseted when entering pass context
+ assert pass_counter.get_counts() == 0
mod = seq(mod)
- # TODO(@jroesch): when we remove new fn pass behavior we need to remove
- # change this back to 3
- assert __TRACE_COUNTER__ == 5
+ # TODO(@jroesch): when we remove new fn pass behavior we need to remove
+ # change this back to match correct behavior
+ assert pass_counter.get_counts() == 6
Review comment:
how come this number changed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r633069436
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief A callback type for set up or clean up instrument environment.
+ */
+using InstrumentEnvFunc = runtime::TypedPackedFunc<void()>;
+
+/*!
+ * \brief A callback template for instrumenting before/after environment.
+ * \tparam RetTy the return type of callback.
+ */
+template <typename RetTy = void>
+using PassInstrumentFunc =
+ runtime::TypedPackedFunc<RetTy(const IRModule&, const transform::PassInfo&)>;
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
+ /*! \brief Name of this pass instrument object. */
+ String name;
+
+ /*! \brief Callback for instrumentation environment set up. */
+ InstrumentEnvFunc set_up_callback;
+ /*! \brief Callback for instrumentation environment clean up. */
+ InstrumentEnvFunc tear_down_callback;
+
+ /*! \brief Callback to run before a pass. */
+ PassInstrumentFunc</* RetTy */ bool> run_before_pass_callback;
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r631669510
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief A callback type for set up or clean up instrument environment.
+ */
+using InstrumentEnvFunc = runtime::TypedPackedFunc<void()>;
+
+/*!
+ * \brief A callback template for instrumenting before/after environment.
+ * \tparam RetTy the return type of callback.
+ */
+template <typename RetTy = void>
+using PassInstrumentFunc =
+ runtime::TypedPackedFunc<RetTy(const IRModule&, const transform::PassInfo&)>;
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
Review comment:
Do you mean making PassInstrument a virtual class? I agree with it.
Currently, PassInstrumentor and PassInstrument have different meanings, they don't have inheritance relationship.
PassInstrumentor will collect a set of instrument implementations (PassInstrument), usage is like: [test_pass_instrument.py#L140](https://github.com/zackcquic/tvm/blob/0ad48d4f1e04f048bbae12daaec1cc100db458cd/tests/python/relay/test_pass_instrument.py#L140)
Do you think it's ok?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic edited a comment on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic edited a comment on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-844694203
@tqchen @tkonolige @areusch
Thanks for the discussions.
For ```ShouldRun``` and ```runBeforePass``` problem, I will check how LLVM and MLIR deal with multiple instrument instances in detail these days.
Also @areusch points to error handling, I really appreciate it. 👍
I will check these too, and then come up a solution.
If you guys have any other ideas/state of art impls, like **_after_pass with a "result" notion._**, please also let me know.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r634819309
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
Review comment:
It will apply instruments' run_before_pass callback in order, if one return false, then the pass will run.
The logic is :
```cpp
// In src/ir/transform.cc
bool PassContext::InstrumentBeforePass(const IRModule& ir_module, const PassInfo& pass_info) const {
auto pass_ctx_node = this->operator->();
if (pass_ctx_node->instruments.defined()) {
for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
if (!pi->RunBeforePass(ir_module, pass_info)) {
return false;
}
}
return true;
}
return true;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r633069427
##########
File path: python/tvm/ir/instrument.py
##########
@@ -0,0 +1,128 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=invalid-name,unused-argument
+"""Common pass instrumentation across IR variants."""
+import tvm._ffi
+import tvm.runtime
+
+from . import _ffi_instrument_api
+
+
+@tvm._ffi.register_object("instrument.PassInstrument")
+class PassInstrument(tvm.runtime.Object):
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r631678904
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief A callback type for set up or clean up instrument environment.
+ */
+using InstrumentEnvFunc = runtime::TypedPackedFunc<void()>;
+
+/*!
+ * \brief A callback template for instrumenting before/after environment.
+ * \tparam RetTy the return type of callback.
+ */
+template <typename RetTy = void>
+using PassInstrumentFunc =
+ runtime::TypedPackedFunc<RetTy(const IRModule&, const transform::PassInfo&)>;
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
Review comment:
May I confirm it is "PassInstrumentor" or "PassInstrument"?
Currently, they have different meanings (not inheritance like ModulePassNode and PassNode):
- ```PassInstrumentor``` collects a set of instrument implementations (```PassInstrument```)
- ```PassInstrument``` is an instrument implementation.
Or do you want make both PassInstrumentor and PassInstrument sub-class-able?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tkonolige commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
tkonolige commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r634627982
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
Review comment:
Can you differentiate the two blocks here so it doesn't look like the same pass is being run twice.
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
Review comment:
```suggestion
* The `Before Pass` instrumentation point can selectively disable passes by returning true (to enable) or false (to disable).
```
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired from LLVM and MLIR.
Review comment:
```suggestion
* This file introduces a pass instrument infrastructure, inspired by LLVM and MLIR.
```
##########
File path: python/tvm/ir/transform.py
##########
@@ -80,9 +88,13 @@ def __init__(
if not isinstance(disabled, (list, tuple)):
raise TypeError("disabled_pass is expected to be the type of " + "list/tuple/set.")
+ instruments = list(instruments) if instruments else []
+ if not isinstance(instruments, (list, tuple)):
+ raise TypeError("disabled_pass is expected to be the type of " + "list/tuple/set.")
Review comment:
Add which type it actually is to the error message.
##########
File path: src/ir/transform.cc
##########
@@ -162,170 +164,64 @@ void PassContext::RegisterConfigOption(const char* key, uint32_t value_type_inde
PassContext PassContext::Create() { return PassContext(make_object<PassContextNode>()); }
-void PassContext::Trace(const IRModule& module, const PassInfo& info, bool is_before) const {
+void PassContext::InstrumentSetUp() const {
auto pass_ctx_node = this->operator->();
- if (pass_ctx_node->trace_func != nullptr) {
- pass_ctx_node->trace_func(module, info, is_before);
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->SetUp();
+ }
}
}
-class ModulePass;
-
-/*! \brief PassProfile stores profiling information for a given pass and its sub-passes. */
-struct PassProfile {
- // TODO(@altanh): expose PassProfile through TVM Object API
- using Clock = std::chrono::steady_clock;
- using Duration = std::chrono::duration<double, std::micro>;
- using Time = std::chrono::time_point<Clock>;
-
- /*! \brief The name of the pass being profiled. */
- String name;
- /*! \brief The time when the pass was entered. */
- Time start;
- /*! \brief The time when the pass completed. */
- Time end;
- /*! \brief The total duration of the pass, i.e. end - start. */
- Duration duration;
- /*! \brief PassProfiles for all sub-passes invoked during the execution of the pass. */
- std::vector<PassProfile> children;
-
- explicit PassProfile(String name)
- : name(name), start(Clock::now()), end(Clock::now()), children() {}
-
- /*! \brief Gets the PassProfile of the currently executing pass. */
- static PassProfile* Current();
- /*! \brief Pushes a new PassProfile with the given pass name. */
- static void EnterPass(String name);
- /*! \brief Pops the current PassProfile. */
- static void ExitPass();
-};
-
-struct PassProfileThreadLocalEntry {
- /*! \brief The placeholder top-level PassProfile. */
- PassProfile root;
- /*! \brief The stack of PassProfiles for nested passes currently running. */
- std::stack<PassProfile*> profile_stack;
- /*! \brief Whether or not pass profiling is active. */
- bool active;
-
- PassProfileThreadLocalEntry() : root("root"), active(false) {}
-};
-
-/*! \brief Thread local store to hold the pass profiling data. */
-typedef dmlc::ThreadLocalStore<PassProfileThreadLocalEntry> PassProfileThreadLocalStore;
-
-void PassProfile::EnterPass(String name) {
- if (!PassProfileThreadLocalStore::Get()->active) return;
- PassProfile* cur = PassProfile::Current();
- cur->children.emplace_back(name);
- PassProfileThreadLocalStore::Get()->profile_stack.push(&cur->children.back());
+void PassContext::InstrumentTearDown() const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->TearDown();
+ }
+ }
}
-void PassProfile::ExitPass() {
- if (!PassProfileThreadLocalStore::Get()->active) return;
- PassProfile* cur = PassProfile::Current();
- ICHECK_NE(cur->name, "root") << "mismatched enter/exit for pass profiling";
- cur->end = std::move(PassProfile::Clock::now());
- cur->duration = std::chrono::duration_cast<PassProfile::Duration>(cur->end - cur->start);
- PassProfileThreadLocalStore::Get()->profile_stack.pop();
+bool PassContext::InstrumentBeforePass(const IRModule& ir_module, const PassInfo& pass_info) const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ if (!pi->RunBeforePass(ir_module, pass_info)) {
+ return false;
+ }
+ }
+ return true;
+ }
+ return true;
}
-PassProfile* PassProfile::Current() {
- PassProfileThreadLocalEntry* entry = PassProfileThreadLocalStore::Get();
- if (!entry->profile_stack.empty()) {
- return entry->profile_stack.top();
- } else {
- return &entry->root;
+void PassContext::InstrumentAfterPass(const IRModule& ir_module, const PassInfo& pass_info) const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->RunAfterPass(ir_module, pass_info);
+ }
}
}
IRModule Pass::operator()(IRModule mod) const {
const PassNode* node = operator->();
ICHECK(node != nullptr);
- PassProfile::EnterPass(node->Info()->name);
+ // PassProfile::EnterPass(node->Info()->name);
auto ret = node->operator()(std::move(mod));
- PassProfile::ExitPass();
+ // PassProfile::ExitPass();
Review comment:
uncomment or delete
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
Review comment:
```suggestion
* It inserts instrumentation points around passes.
```
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
+ virtual ~PassInstrumentNode() {}
+
+ /*! \brief Set up environment for instrumentation. */
+ virtual void SetUp() const = 0;
+
+ /*! \brief Clean up instrumentation environment. */
+ virtual void TearDown() const = 0;
+
+ /*!
+ * \brief Instrument before pass run, determine whether to run the pass or not.
+ * \param mod The module that an optimization pass runs on.
+ * \param info The pass information.
+ *
+ * \return true to run the pass; false to skip the pass.
+ */
+ virtual bool RunBeforePass(const IRModule& mod, const transform::PassInfo& info) const = 0;
Review comment:
Can calls to `RunBeforePass` be nested?
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
Review comment:
```suggestion
* if (Instrument Before Pass())
```
To make it clear that you are running the before pass
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
Review comment:
How does disabling passes work if there are multiple instruments?
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
Review comment:
I would move this description into the `PassInstrumentNode` class
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,114 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
Review comment:
```suggestion
* It provides API for users to register callbacks at different instrumentation points.
```
##########
File path: python/tvm/ir/transform.py
##########
@@ -65,12 +65,20 @@ class PassContext(tvm.runtime.Object):
disabled_pass : Optional[Union[List[str], Set[str], Tuple[str]]]
The list of passes that are disabled.
+ instruments : Optional[Union[List[PassInstrument], Set[PassInstrument], Tuple[PassInstrument]]]
Review comment:
```suggestion
instruments : Optional[Sequence[PassInstrument]]
```
##########
File path: src/ir/transform.cc
##########
@@ -162,170 +164,64 @@ void PassContext::RegisterConfigOption(const char* key, uint32_t value_type_inde
PassContext PassContext::Create() { return PassContext(make_object<PassContextNode>()); }
-void PassContext::Trace(const IRModule& module, const PassInfo& info, bool is_before) const {
+void PassContext::InstrumentSetUp() const {
auto pass_ctx_node = this->operator->();
- if (pass_ctx_node->trace_func != nullptr) {
- pass_ctx_node->trace_func(module, info, is_before);
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->SetUp();
+ }
}
}
-class ModulePass;
-
-/*! \brief PassProfile stores profiling information for a given pass and its sub-passes. */
-struct PassProfile {
- // TODO(@altanh): expose PassProfile through TVM Object API
- using Clock = std::chrono::steady_clock;
- using Duration = std::chrono::duration<double, std::micro>;
- using Time = std::chrono::time_point<Clock>;
-
- /*! \brief The name of the pass being profiled. */
- String name;
- /*! \brief The time when the pass was entered. */
- Time start;
- /*! \brief The time when the pass completed. */
- Time end;
- /*! \brief The total duration of the pass, i.e. end - start. */
- Duration duration;
- /*! \brief PassProfiles for all sub-passes invoked during the execution of the pass. */
- std::vector<PassProfile> children;
-
- explicit PassProfile(String name)
- : name(name), start(Clock::now()), end(Clock::now()), children() {}
-
- /*! \brief Gets the PassProfile of the currently executing pass. */
- static PassProfile* Current();
- /*! \brief Pushes a new PassProfile with the given pass name. */
- static void EnterPass(String name);
- /*! \brief Pops the current PassProfile. */
- static void ExitPass();
-};
-
-struct PassProfileThreadLocalEntry {
- /*! \brief The placeholder top-level PassProfile. */
- PassProfile root;
- /*! \brief The stack of PassProfiles for nested passes currently running. */
- std::stack<PassProfile*> profile_stack;
- /*! \brief Whether or not pass profiling is active. */
- bool active;
-
- PassProfileThreadLocalEntry() : root("root"), active(false) {}
-};
-
-/*! \brief Thread local store to hold the pass profiling data. */
-typedef dmlc::ThreadLocalStore<PassProfileThreadLocalEntry> PassProfileThreadLocalStore;
-
-void PassProfile::EnterPass(String name) {
- if (!PassProfileThreadLocalStore::Get()->active) return;
- PassProfile* cur = PassProfile::Current();
- cur->children.emplace_back(name);
- PassProfileThreadLocalStore::Get()->profile_stack.push(&cur->children.back());
+void PassContext::InstrumentTearDown() const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->TearDown();
+ }
+ }
}
-void PassProfile::ExitPass() {
- if (!PassProfileThreadLocalStore::Get()->active) return;
- PassProfile* cur = PassProfile::Current();
- ICHECK_NE(cur->name, "root") << "mismatched enter/exit for pass profiling";
- cur->end = std::move(PassProfile::Clock::now());
- cur->duration = std::chrono::duration_cast<PassProfile::Duration>(cur->end - cur->start);
- PassProfileThreadLocalStore::Get()->profile_stack.pop();
+bool PassContext::InstrumentBeforePass(const IRModule& ir_module, const PassInfo& pass_info) const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ if (!pi->RunBeforePass(ir_module, pass_info)) {
+ return false;
+ }
+ }
+ return true;
+ }
+ return true;
}
-PassProfile* PassProfile::Current() {
- PassProfileThreadLocalEntry* entry = PassProfileThreadLocalStore::Get();
- if (!entry->profile_stack.empty()) {
- return entry->profile_stack.top();
- } else {
- return &entry->root;
+void PassContext::InstrumentAfterPass(const IRModule& ir_module, const PassInfo& pass_info) const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->RunAfterPass(ir_module, pass_info);
+ }
}
}
IRModule Pass::operator()(IRModule mod) const {
const PassNode* node = operator->();
ICHECK(node != nullptr);
- PassProfile::EnterPass(node->Info()->name);
+ // PassProfile::EnterPass(node->Info()->name);
Review comment:
uncomment or delete
##########
File path: tests/python/relay/test_pass_manager.py
##########
@@ -536,10 +537,12 @@ def test_print_ir(capfd):
__TRACE_COUNTER__ = 0
-def _tracer(module, info, is_before):
- global __TRACE_COUNTER__
- if bool(is_before):
+@tvm.instrument.pass_instrument
+class MyInstrument:
+ def run_before_pass(self, module, info):
+ global __TRACE_COUNTER__
Review comment:
Can you switch this test to not use a global.
##########
File path: src/ir/instrument.cc
##########
@@ -0,0 +1,310 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file src/ir/instrument.cc
+ * \brief Infrastructure for instrumentation.
+ */
+#include <dmlc/thread_local.h>
+#include <tvm/ir/instrument.h>
+#include <tvm/ir/transform.h>
+#include <tvm/node/repr_printer.h>
+#include <tvm/runtime/registry.h>
+
+#include <stack>
+
+namespace tvm {
+namespace instrument {
+
+/*!
+ * \brief A named PassInstrument implementation
+ * \sa NamedPassInstrument
+ */
+class NamedPassInstrumentNode : public PassInstrumentNode {
+ public:
+ /*! \brief Name of this pass instrument object. */
+ String name;
+
+ /*! \brief Callback for instrumentation environment set up. */
+ runtime::TypedPackedFunc<void()> set_up_callback;
+ /*! \brief Callback for instrumentation environment clean up. */
+ runtime::TypedPackedFunc<void()> tear_down_callback;
+
+ /*! \brief Callback to run before a pass. */
+ runtime::TypedPackedFunc<bool(const IRModule&, const transform::PassInfo&)>
+ run_before_pass_callback;
+ /*! \brief Callback to run after a pass. */
+ runtime::TypedPackedFunc<void(const IRModule&, const transform::PassInfo&)>
+ run_after_pass_callback;
+
+ void VisitAttrs(AttrVisitor* v) { v->Visit("name", &name); }
+
+ /*! \brief Set up environment for instrumentation. */
+ void SetUp() const final;
+
+ /*! \brief Clean up instrumentation environment. */
+ void TearDown() const final;
+
+ /*!
+ * \brief Instrument before pass run, determine whether to run the pass or not.
+ * \param mod The module that an optimization pass runs on.
+ * \param info The pass information.
+ *
+ * \return true to run the pass; false to skip the pass.
+ */
+ bool RunBeforePass(const IRModule& mod, const transform::PassInfo& info) const final;
+
+ /*!
+ * \brief Instrument after pass run.
+ *
+ * \param mod The module that an optimization pass runs on.
+ * \param info The pass information.
+ */
+ void RunAfterPass(const IRModule& mod, const transform::PassInfo& info) const final;
+
+ static constexpr const char* _type_key = "instrument.NamedPassInstrument";
+ TVM_DECLARE_FINAL_OBJECT_INFO(NamedPassInstrumentNode, PassInstrumentNode);
+};
+
+/*!
+ * \brief Managed reference class for NamedPassInstrumentNode
+ * \sa NamedPassInstrumentNode
+ */
+class NamedPassInstrument : public PassInstrument {
+ public:
+ /*!
+ * \brief Constructor
+ * \param name Name for this instrumentation.
+ */
+ TVM_DLL NamedPassInstrument(String name);
Review comment:
Why not have the constructor take all the `TypedPackedFunc`s as arguments? Then you can avoid the mutable accessor
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tqchen commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
tqchen commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-850574345
Thanks @zackcquic for putting all the hard work polishing the API, improving the code and bring the implementation.
It would be great if we can followup to add design docs and examples :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r634817313
##########
File path: tests/python/relay/test_pass_manager.py
##########
@@ -536,10 +537,12 @@ def test_print_ir(capfd):
__TRACE_COUNTER__ = 0
-def _tracer(module, info, is_before):
- global __TRACE_COUNTER__
- if bool(is_before):
+@tvm.instrument.pass_instrument
+class MyInstrument:
+ def run_before_pass(self, module, info):
+ global __TRACE_COUNTER__
Review comment:
Done
##########
File path: src/ir/transform.cc
##########
@@ -162,170 +164,64 @@ void PassContext::RegisterConfigOption(const char* key, uint32_t value_type_inde
PassContext PassContext::Create() { return PassContext(make_object<PassContextNode>()); }
-void PassContext::Trace(const IRModule& module, const PassInfo& info, bool is_before) const {
+void PassContext::InstrumentSetUp() const {
auto pass_ctx_node = this->operator->();
- if (pass_ctx_node->trace_func != nullptr) {
- pass_ctx_node->trace_func(module, info, is_before);
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->SetUp();
+ }
}
}
-class ModulePass;
-
-/*! \brief PassProfile stores profiling information for a given pass and its sub-passes. */
-struct PassProfile {
- // TODO(@altanh): expose PassProfile through TVM Object API
- using Clock = std::chrono::steady_clock;
- using Duration = std::chrono::duration<double, std::micro>;
- using Time = std::chrono::time_point<Clock>;
-
- /*! \brief The name of the pass being profiled. */
- String name;
- /*! \brief The time when the pass was entered. */
- Time start;
- /*! \brief The time when the pass completed. */
- Time end;
- /*! \brief The total duration of the pass, i.e. end - start. */
- Duration duration;
- /*! \brief PassProfiles for all sub-passes invoked during the execution of the pass. */
- std::vector<PassProfile> children;
-
- explicit PassProfile(String name)
- : name(name), start(Clock::now()), end(Clock::now()), children() {}
-
- /*! \brief Gets the PassProfile of the currently executing pass. */
- static PassProfile* Current();
- /*! \brief Pushes a new PassProfile with the given pass name. */
- static void EnterPass(String name);
- /*! \brief Pops the current PassProfile. */
- static void ExitPass();
-};
-
-struct PassProfileThreadLocalEntry {
- /*! \brief The placeholder top-level PassProfile. */
- PassProfile root;
- /*! \brief The stack of PassProfiles for nested passes currently running. */
- std::stack<PassProfile*> profile_stack;
- /*! \brief Whether or not pass profiling is active. */
- bool active;
-
- PassProfileThreadLocalEntry() : root("root"), active(false) {}
-};
-
-/*! \brief Thread local store to hold the pass profiling data. */
-typedef dmlc::ThreadLocalStore<PassProfileThreadLocalEntry> PassProfileThreadLocalStore;
-
-void PassProfile::EnterPass(String name) {
- if (!PassProfileThreadLocalStore::Get()->active) return;
- PassProfile* cur = PassProfile::Current();
- cur->children.emplace_back(name);
- PassProfileThreadLocalStore::Get()->profile_stack.push(&cur->children.back());
+void PassContext::InstrumentTearDown() const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->TearDown();
+ }
+ }
}
-void PassProfile::ExitPass() {
- if (!PassProfileThreadLocalStore::Get()->active) return;
- PassProfile* cur = PassProfile::Current();
- ICHECK_NE(cur->name, "root") << "mismatched enter/exit for pass profiling";
- cur->end = std::move(PassProfile::Clock::now());
- cur->duration = std::chrono::duration_cast<PassProfile::Duration>(cur->end - cur->start);
- PassProfileThreadLocalStore::Get()->profile_stack.pop();
+bool PassContext::InstrumentBeforePass(const IRModule& ir_module, const PassInfo& pass_info) const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ if (!pi->RunBeforePass(ir_module, pass_info)) {
+ return false;
+ }
+ }
+ return true;
+ }
+ return true;
}
-PassProfile* PassProfile::Current() {
- PassProfileThreadLocalEntry* entry = PassProfileThreadLocalStore::Get();
- if (!entry->profile_stack.empty()) {
- return entry->profile_stack.top();
- } else {
- return &entry->root;
+void PassContext::InstrumentAfterPass(const IRModule& ir_module, const PassInfo& pass_info) const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->RunAfterPass(ir_module, pass_info);
+ }
}
}
IRModule Pass::operator()(IRModule mod) const {
const PassNode* node = operator->();
ICHECK(node != nullptr);
- PassProfile::EnterPass(node->Info()->name);
+ // PassProfile::EnterPass(node->Info()->name);
auto ret = node->operator()(std::move(mod));
- PassProfile::ExitPass();
+ // PassProfile::ExitPass();
Review comment:
Done
##########
File path: src/ir/transform.cc
##########
@@ -162,170 +164,64 @@ void PassContext::RegisterConfigOption(const char* key, uint32_t value_type_inde
PassContext PassContext::Create() { return PassContext(make_object<PassContextNode>()); }
-void PassContext::Trace(const IRModule& module, const PassInfo& info, bool is_before) const {
+void PassContext::InstrumentSetUp() const {
auto pass_ctx_node = this->operator->();
- if (pass_ctx_node->trace_func != nullptr) {
- pass_ctx_node->trace_func(module, info, is_before);
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->SetUp();
+ }
}
}
-class ModulePass;
-
-/*! \brief PassProfile stores profiling information for a given pass and its sub-passes. */
-struct PassProfile {
- // TODO(@altanh): expose PassProfile through TVM Object API
- using Clock = std::chrono::steady_clock;
- using Duration = std::chrono::duration<double, std::micro>;
- using Time = std::chrono::time_point<Clock>;
-
- /*! \brief The name of the pass being profiled. */
- String name;
- /*! \brief The time when the pass was entered. */
- Time start;
- /*! \brief The time when the pass completed. */
- Time end;
- /*! \brief The total duration of the pass, i.e. end - start. */
- Duration duration;
- /*! \brief PassProfiles for all sub-passes invoked during the execution of the pass. */
- std::vector<PassProfile> children;
-
- explicit PassProfile(String name)
- : name(name), start(Clock::now()), end(Clock::now()), children() {}
-
- /*! \brief Gets the PassProfile of the currently executing pass. */
- static PassProfile* Current();
- /*! \brief Pushes a new PassProfile with the given pass name. */
- static void EnterPass(String name);
- /*! \brief Pops the current PassProfile. */
- static void ExitPass();
-};
-
-struct PassProfileThreadLocalEntry {
- /*! \brief The placeholder top-level PassProfile. */
- PassProfile root;
- /*! \brief The stack of PassProfiles for nested passes currently running. */
- std::stack<PassProfile*> profile_stack;
- /*! \brief Whether or not pass profiling is active. */
- bool active;
-
- PassProfileThreadLocalEntry() : root("root"), active(false) {}
-};
-
-/*! \brief Thread local store to hold the pass profiling data. */
-typedef dmlc::ThreadLocalStore<PassProfileThreadLocalEntry> PassProfileThreadLocalStore;
-
-void PassProfile::EnterPass(String name) {
- if (!PassProfileThreadLocalStore::Get()->active) return;
- PassProfile* cur = PassProfile::Current();
- cur->children.emplace_back(name);
- PassProfileThreadLocalStore::Get()->profile_stack.push(&cur->children.back());
+void PassContext::InstrumentTearDown() const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->TearDown();
+ }
+ }
}
-void PassProfile::ExitPass() {
- if (!PassProfileThreadLocalStore::Get()->active) return;
- PassProfile* cur = PassProfile::Current();
- ICHECK_NE(cur->name, "root") << "mismatched enter/exit for pass profiling";
- cur->end = std::move(PassProfile::Clock::now());
- cur->duration = std::chrono::duration_cast<PassProfile::Duration>(cur->end - cur->start);
- PassProfileThreadLocalStore::Get()->profile_stack.pop();
+bool PassContext::InstrumentBeforePass(const IRModule& ir_module, const PassInfo& pass_info) const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ if (!pi->RunBeforePass(ir_module, pass_info)) {
+ return false;
+ }
+ }
+ return true;
+ }
+ return true;
}
-PassProfile* PassProfile::Current() {
- PassProfileThreadLocalEntry* entry = PassProfileThreadLocalStore::Get();
- if (!entry->profile_stack.empty()) {
- return entry->profile_stack.top();
- } else {
- return &entry->root;
+void PassContext::InstrumentAfterPass(const IRModule& ir_module, const PassInfo& pass_info) const {
+ auto pass_ctx_node = this->operator->();
+ if (pass_ctx_node->instruments.defined()) {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->RunAfterPass(ir_module, pass_info);
+ }
}
}
IRModule Pass::operator()(IRModule mod) const {
const PassNode* node = operator->();
ICHECK(node != nullptr);
- PassProfile::EnterPass(node->Info()->name);
+ // PassProfile::EnterPass(node->Info()->name);
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
areusch commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r638946971
##########
File path: src/ir/transform.cc
##########
@@ -164,34 +167,57 @@ void PassContext::RegisterConfigOption(const char* key, uint32_t value_type_inde
PassContext PassContext::Create() { return PassContext(make_object<PassContextNode>()); }
-void PassContext::InstrumentSetUp() const {
+void PassContext::InstrumentEnterPassContext() {
auto pass_ctx_node = this->operator->();
if (pass_ctx_node->instruments.defined()) {
- for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
- pi->SetUp();
+ try {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->EnterPassContext();
+ }
+ } catch (const Error& e) {
+ LOG(INFO) << "Pass instrumentation entering pass context failed.";
+ LOG(INFO) << "Disable pass instrumentation.";
+ pass_ctx_node->instruments.clear();
+ throw e;
}
}
}
-void PassContext::InstrumentTearDown() const {
+void PassContext::InstrumentExitPassContext() {
auto pass_ctx_node = this->operator->();
if (pass_ctx_node->instruments.defined()) {
- for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
- pi->TearDown();
+ try {
+ for (instrument::PassInstrument pi : pass_ctx_node->instruments) {
+ pi->ExitPassContext();
+ }
+ } catch (const Error& e) {
+ LOG(INFO) << "Pass instrumentation exiting pass context failed.";
+ pass_ctx_node->instruments.clear();
+ throw e;
}
}
}
bool PassContext::InstrumentBeforePass(const IRModule& ir_module, const PassInfo& pass_info) const {
auto pass_ctx_node = this->operator->();
- if (pass_ctx_node->instruments.defined()) {
+ if (!pass_ctx_node->instruments.defined()) {
+ return true;
+ }
+
+ const bool pass_required = PassArrayContains(pass_ctx_node->required_pass, pass_info->name);
+ bool should_run = true;
+ if (!pass_required) {
+ const Array<instrument::PassInstrument>& instruments = pass_ctx_node->instruments;
+ should_run &= std::all_of(instruments.begin(), instruments.end(),
Review comment:
this is a bit counter to the stl, but I actually prefer if code like this is just explicitly written out. then it's clear what happens if any of `pi->ShouldRun` returns early.
Here, it might be slightly better to explicitly invoke all ShouldRun(), even if one returns False. Then, passes that simply implement some type of logging or timing can still include entries for those passes that didn't run. i'm happy to be swayed either way on this, but it seems more useful to me that way.
##########
File path: tests/python/relay/test_pass_instrument.py
##########
@@ -168,3 +169,329 @@ def run_after_pass(self, mod, info):
# Out of pass context scope, should be reset
assert passes_counter.run_before_count == 0
assert passes_counter.run_after_count == 0
+
+
+def test_enter_pass_ctx_exception(capsys):
+ @pass_instrument
+ class PI:
+ def __init__(self, id):
+ self.id = id
+
+ def enter_pass_ctx(self):
+ print(self.id + " enter ctx")
+
+ def exit_pass_ctx(self):
+ print(self.id + " exit ctx")
+
+ @pass_instrument
+ class PIBroken(PI):
+ def __init__(self, id):
+ super().__init__(id)
+
+ def enter_pass_ctx(self):
+ print(self.id + " enter ctx")
+ raise RuntimeError("Just a dummy error")
+
+ pass_ctx = tvm.transform.PassContext(instruments=[PI("%1"), PIBroken("%2"), PI("%3")])
+ with pytest.raises(tvm.error.TVMError):
+ with pass_ctx:
+ pass
+
+ assert "%1 enter ctx\n" "%2 enter ctx\n" == capsys.readouterr().out
Review comment:
should this also contain "%1 exit ctx\n"?
##########
File path: tests/python/relay/test_pass_instrument.py
##########
@@ -14,77 +14,91 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
+""" Instrument test cases.
+"""
+import pytest
import tvm
import tvm.relay
from tvm.relay import op
from tvm.ir.instrument import PassTimingInstrument, pass_instrument
-def test_pass_timing_instrument():
+def get_test_model():
x, y, z = [tvm.relay.var(c, shape=(3, 4), dtype="float32") for c in "xyz"]
e1 = op.add(x, y)
e2 = op.subtract(x, z)
e3 = op.multiply(e1, e1 / e2)
- mod = tvm.IRModule.from_expr(e3 + e2)
+ return tvm.IRModule.from_expr(e3 + e2)
+
+def test_pass_timing_instrument():
pass_timing = PassTimingInstrument()
- with tvm.transform.PassContext(instruments=[pass_timing]):
- mod = tvm.relay.transform.AnnotateSpans()(mod)
- mod = tvm.relay.transform.ToANormalForm()(mod)
- mod = tvm.relay.transform.InferType()(mod)
- profiles = pass_timing.render()
- assert "AnnotateSpans" in profiles
- assert "ToANormalForm" in profiles
- assert "InferType" in profiles
+ # Override current PassContext's instruments
+ tvm.transform.PassContext.current().override_instruments([pass_timing])
+ mod = get_test_model()
+ mod = tvm.relay.transform.AnnotateSpans()(mod)
+ mod = tvm.relay.transform.ToANormalForm()(mod)
+ mod = tvm.relay.transform.InferType()(mod)
-def test_custom_instrument(capsys):
- x, y, z = [tvm.relay.var(c, shape=(3, 4), dtype="float32") for c in "xyz"]
- e1 = op.add(x, y)
- e2 = op.subtract(x, z)
- e3 = op.multiply(e1, e1 / e2)
- mod = tvm.IRModule.from_expr(e3 + e2)
+ profiles = pass_timing.render()
+ assert "AnnotateSpans" in profiles
+ assert "ToANormalForm" in profiles
+ assert "InferType" in profiles
+
+ # Reset current PassContext's instruments to None
+ tvm.transform.PassContext.current().override_instruments(None)
+ mod = get_test_model()
+ mod = tvm.relay.transform.AnnotateSpans()(mod)
+ mod = tvm.relay.transform.ToANormalForm()(mod)
+ mod = tvm.relay.transform.InferType()(mod)
+
+ profiles = pass_timing.render()
+ assert profiles == ""
+
+
+def test_custom_instrument(capsys):
@pass_instrument
class MyTest:
- def set_up(self):
- print("set up")
+ def enter_pass_ctx(self):
+ print("enter ctx")
Review comment:
rather than print, can you call a function like:
```
def __init__(self):
self._events = []
def _log_event(self, msg):
print(msg) # or maybe logging.info
self._events.append(msg)
```
then just assert on `self._events`? otherwise print() in the middle of e.g. FFI could cause this test to fail (not that it would be committed, but could make debugging harder).
##########
File path: tests/python/relay/test_pass_instrument.py
##########
@@ -205,6 +205,18 @@ def enter_pass_ctx(self):
assert cur_pass_ctx.instruments == None
+def test_enter_pass_ctx_exception_global(capsys):
+ @pass_instrument
+ class PIBroken:
+ def enter_pass_ctx(self):
+ raise RuntimeError("Just a dummy error")
+
+ cur_pass_ctx = tvm.transform.PassContext.current()
+ with pytest.raises(tvm.error.TVMError):
Review comment:
i think you can do one better and assert the specific TVMError, something like:
```
with pytest.raises(tvm.error.TVMError) as cm:
cur_pass_ctx.override_instruments([PIBroken()])
assert "Just a dummy error" in str(cm.exception)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tqchen commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
tqchen commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-842673796
cc @jroesch @Hzfengsy @junrushao1994 @vinx13 @areusch @mbrookhart @yzhliu would be great if you can help to review this PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] jroesch commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
jroesch commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-830475873
I will try and comment over the weekend, thanks for the PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tqchen merged pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
tqchen merged pull request #7952:
URL: https://github.com/apache/tvm/pull/7952
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tqchen commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
tqchen commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-850572867
Thanks @zackcquic @areusch @tkonolige @altanh
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-842844673
@tqchen requested changes updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r638986565
##########
File path: tests/python/relay/test_pass_instrument.py
##########
@@ -168,3 +169,329 @@ def run_after_pass(self, mod, info):
# Out of pass context scope, should be reset
assert passes_counter.run_before_count == 0
assert passes_counter.run_after_count == 0
+
+
+def test_enter_pass_ctx_exception(capsys):
+ @pass_instrument
+ class PI:
+ def __init__(self, id):
+ self.id = id
+
+ def enter_pass_ctx(self):
+ print(self.id + " enter ctx")
+
+ def exit_pass_ctx(self):
+ print(self.id + " exit ctx")
+
+ @pass_instrument
+ class PIBroken(PI):
+ def __init__(self, id):
+ super().__init__(id)
+
+ def enter_pass_ctx(self):
+ print(self.id + " enter ctx")
+ raise RuntimeError("Just a dummy error")
+
+ pass_ctx = tvm.transform.PassContext(instruments=[PI("%1"), PIBroken("%2"), PI("%3")])
+ with pytest.raises(tvm.error.TVMError):
+ with pass_ctx:
+ pass
+
+ assert "%1 enter ctx\n" "%2 enter ctx\n" == capsys.readouterr().out
Review comment:
I think no, since `__enter__` didn't catch the exceptions if `EnterPassContent()` throws ([transform.py:89](https://github.com/apache/tvm/blob/6f82e98aac18619078792661d151f33e37afa94d/python/tvm/ir/transform.py#L89)).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-847472910
> Thanks for the changes, this looks much better now! Mostly spelling nits here, but also could you please add back the test for the `PassTimingInstrument`? Looks like it got removed here (previously was in `tests/python/relay/test_pass_profiler.py`).
I renamed test_pass_profiler.py to test_pass_instrument.py, and PassTimingInstrument are tested in this file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic edited a comment on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic edited a comment on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-840237999
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r631681526
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief A callback type for set up or clean up instrument environment.
+ */
+using InstrumentEnvFunc = runtime::TypedPackedFunc<void()>;
+
+/*!
+ * \brief A callback template for instrumenting before/after environment.
+ * \tparam RetTy the return type of callback.
+ */
+template <typename RetTy = void>
+using PassInstrumentFunc =
+ runtime::TypedPackedFunc<RetTy(const IRModule&, const transform::PassInfo&)>;
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
+ /*! \brief Name of this pass instrument object. */
+ String name;
+
+ /*! \brief Callback for instrumentation environment set up. */
+ InstrumentEnvFunc set_up_callback;
+ /*! \brief Callback for instrumentation environment clean up. */
+ InstrumentEnvFunc tear_down_callback;
+
+ /*! \brief Callback to run before a pass. */
+ PassInstrumentFunc</* RetTy */ bool> run_before_pass_callback;
+ /*! \brief Callback to run after a pass. */
+ PassInstrumentFunc<> run_after_pass_callback;
+
+ /*!
+ * \brief Register a callback to run at set up point.
+ *
+ * \param callback The set up function.
+ */
+ void RegisterSetUpCallback(InstrumentEnvFunc callback) { set_up_callback = std::move(callback); }
+
+ /*
+ * \brief Register a callback to run at clean up point.
+ *
+ * \param callback The clean up function.
+ */
+ void RegisterTearDownCallback(InstrumentEnvFunc callback) {
+ tear_down_callback = std::move(callback);
+ }
+
+ /*!
+ * \brief Register a callback to run before pass run.
+ *
+ * \param callback The function to run before pass: return false to skip pass; return true to
+ * run pass.
+ */
+ void RegisterRunBeforePassCallback(PassInstrumentFunc<bool> callback) {
+ run_before_pass_callback = std::move(callback);
+ }
+
+ /*!
+ * \brief Register a callback to run after pass run.
+ *
+ * \param callback The function to run after pass.
+ */
+ void RegisterRunAfterPassCallback(PassInstrumentFunc<> callback) {
+ run_after_pass_callback = std::move(callback);
+ }
+
+ void VisitAttrs(AttrVisitor* v) { v->Visit("name", &name); }
+
+ /*! \brief Set up environment for instrumentation. */
+ void SetUp() const;
+
+ /*! \brief Clean up instrumentation environment. */
+ void TearDown() const;
+
+ /*!
+ * \brief Instrument before pass run, determine whether to run the pass or not.
+ * \param mod The module that an optimization pass runs on.
+ * \param info The pass information.
+ *
+ * \return true to run the pass; false to skip the pass.
+ */
+ bool RunBeforePass(const IRModule& mod, const transform::PassInfo& info) const;
+
+ /*!
+ * \brief Instrument after pass run.
+ *
+ * \param mod The module that an optimization pass runs on.
+ * \param info The pass information.
+ */
+ void RunAfterPass(const IRModule& mod, const transform::PassInfo& info) const;
+
+ static constexpr const char* _type_key = "instrument.PassInstrument";
+ TVM_DECLARE_FINAL_OBJECT_INFO(PassInstrumentNode, Object);
+};
+
+/*!
+ * \brief Managed reference class for PassInstrumentNode
+ * \sa PassInstrumentNode
+ */
+class PassInstrument : public ObjectRef {
+ public:
+ /*!
+ * \brief Constructor
+ * \param name Name for this instrumentation.
+ */
+ TVM_DLL PassInstrument(String name);
+
+ /*!
+ * \brief mutable accessor.
+ * \return mutable access pointer.
+ */
+ PassInstrumentNode* operator->() {
+ ICHECK(get() != nullptr);
+ return static_cast<PassInstrumentNode*>(get_mutable());
+ }
+
+ TVM_DEFINE_OBJECT_REF_METHODS(PassInstrument, ObjectRef, PassInstrumentNode);
+};
+
+/*!
+ * \brief PassInstrumentorNode collects a set of PassInstrument implementations, invokes the
+ * implementations' methods at different instrument points.
+ * \sa PassInstrumentor
+ */
+class PassInstrumentorNode : public Object {
Review comment:
If we put Array<PassInstrument> directly in the PassContext, then PassInstrumentor can be totally a internal data structure, and no need for subclassing.
Either PassInstrumentor or Array<PassInstrument> is fine for me.
Which kind do you like?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r631669510
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,244 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file implements a pass instrument infrastructure, inspired from LLVM and MLIR.
+ * It inserts instrumentation points between passes run.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * if (Instrument Before Pass)
+ * Pass Run
+ * Instrument After Pass
+ *
+ * Instrument TearDown
+ *
+ *
+ * Instrument point before pass can determine particular pass is disable or not depends on the
+ * callback registered.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief A callback type for set up or clean up instrument environment.
+ */
+using InstrumentEnvFunc = runtime::TypedPackedFunc<void()>;
+
+/*!
+ * \brief A callback template for instrumenting before/after environment.
+ * \tparam RetTy the return type of callback.
+ */
+template <typename RetTy = void>
+using PassInstrumentFunc =
+ runtime::TypedPackedFunc<RetTy(const IRModule&, const transform::PassInfo&)>;
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrument point.
+ * \sa PassInstrument
+ */
+class PassInstrumentNode : public Object {
+ public:
Review comment:
Do you mean making PassInstrument a virtual class? I agree with it.
Currently, PassInstrumentor and PassInstrument have different meanings, they don't have inheritance relationship (like PassNode and ModulePassNode).
PassInstrumentor will collect a set of instrument implementations (PassInstrument), usage is like: [test_pass_instrument.py#L140](https://github.com/zackcquic/tvm/blob/0ad48d4f1e04f048bbae12daaec1cc100db458cd/tests/python/relay/test_pass_instrument.py#L140)
Do you think it's ok?
It not, could you explain more on "These callbacks can sit in implementation of instrument.cc" and "We can make PassInstrumentor a virtual class"?
Thanks a lot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] tqchen commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
tqchen commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-839755661
Sorry for the long delay. @zackcquic I have added review comments. One thing that would be helpful to get put more thoughts on is the naming. in particular what is the argument name when passing a list of pass instrumentors.
cc @junrushao1994 @vinx13 @altanh @zhiics @yzhliu Please also help to take a look
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on a change in pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on a change in pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#discussion_r638428979
##########
File path: include/tvm/ir/instrument.h
##########
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/ir/instrument.h
+ *
+ * This file introduces a pass instrument infrastructure, inspired by LLVM and MLIR.
+ * It inserts instrumentation points around passes.
+ */
+#ifndef TVM_IR_INSTRUMENT_H_
+#define TVM_IR_INSTRUMENT_H_
+
+#include <tvm/node/reflection.h>
+#include <tvm/runtime/container.h>
+
+#include <utility>
+#include <vector>
+
+namespace tvm {
+
+class IRModule;
+
+// Forward class for PassInstrumentNode methods
+namespace transform {
+class PassInfo;
+} // namespace transform
+
+namespace instrument {
+
+/*!
+ * \brief PassInstrumentNode forms an instrument implementation.
+ * It provides API for users to register callbacks at different instrumentation points.
+ *
+ * Within a pass context (tvm::transfom::PassContext), the instrumentation call sequence will like:
+ *
+ * Instrument SetUp()
+ *
+ * if (Instrument Before Pass())
Review comment:
I add test cases and documentation for these cases' behavior in the newest commits. (include/tvm/ir/instrument.h, python/relay/tests_pass_instrument.py)
I assume if any error raise in `before_pass()`, `pass()` and `after_pass()`, it should be stopped, instead of keep processing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] zackcquic commented on pull request #7952: [IR][Pass][Instrument] Pass instrument framework
Posted by GitBox <gi...@apache.org>.
zackcquic commented on pull request #7952:
URL: https://github.com/apache/tvm/pull/7952#issuecomment-841794965
@tqchen I just made modification based on comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org