You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2020/12/08 05:47:06 UTC
[GitHub] [tvm] antinucleon opened a new pull request #7053: [auto_scheduler] buffer support, correctness check
antinucleon opened a new pull request #7053:
URL: https://github.com/apache/tvm/pull/7053
This PR enables correctness check for a generated schedule.
It is useful for:
- Metal / ROCM: For some invalid schedule, driver may skip it instead of return any errors (which will show as an impossble large FLOPS number)
- Sparse kernel search
Example 1: correctness check. This will generate random buffers for correctness check
```
if train_flag:
#measure_ctx = auto_scheduler.LocalRPCMeasureContext(min_repeat_ms=300)
measure_runner = auto_scheduler.RPCRunner("m1", "127.0.0.1", 9190, min_repeat_ms=300, timeout=30, repeat=3)
tune_option = auto_scheduler.TuningOptions(
num_measure_trials=1500,
check_correctness=True,
builder_n_parallel=1,
runner=measure_runner,
measure_callbacks=[auto_scheduler.RecordToFile(log_file)],
verbose=2,
)
sch, args = auto_scheduler.auto_schedule(task, tuning_options=tune_option)
```
Example 2: Sparse tuning, this will register given buffers for measure.
```
if train_flag:
#measure_ctx = auto_scheduler.LocalRPCMeasureContext(min_repeat_ms=300)
measure_runner = auto_scheduler.RPCRunner("m1", "127.0.0.1", 9190, min_repeat_ms=300, timeout=30, repeat=3)
tune_option = auto_scheduler.TuningOptions(
num_measure_trials=1500,
#runner=measure_ctx.runner,
check_correctness=False,
builder_n_parallel=1,
runner=measure_runner,
measure_callbacks=[auto_scheduler.RecordToFile(log_file)],
verbose=2,
)
for k, v in BUFFER.items():
tune_option.register_buffer(k, v)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] FrozenGene commented on a change in pull request #7053: [auto_scheduler] buffer support, correctness check
Posted by GitBox <gi...@apache.org>.
FrozenGene commented on a change in pull request #7053:
URL: https://github.com/apache/tvm/pull/7053#discussion_r539080819
##########
File path: python/tvm/auto_scheduler/auto_schedule.py
##########
@@ -117,14 +132,21 @@ def __init__(
num_measure_trials=0,
early_stopping=None,
num_measures_per_round=64,
+ working_dir=None,
+ check_correctness=False,
verbose=1,
builder="local",
+ builder_n_parallel=-1,
Review comment:
Ok.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] merrymercy commented on a change in pull request #7053: [auto_scheduler] buffer support, correctness check
Posted by GitBox <gi...@apache.org>.
merrymercy commented on a change in pull request #7053:
URL: https://github.com/apache/tvm/pull/7053#discussion_r538808502
##########
File path: python/tvm/auto_scheduler/measure.py
##########
@@ -889,18 +929,43 @@ def _timed_rpc_run(
if error_no == 0:
try:
- args = [ndarray.empty(get_const_tuple(x.shape), x.dtype, ctx) for x in build_res.args]
- try:
- random_fill = remote.get_function("tvm.contrib.random.random_fill")
- except AttributeError:
- raise AttributeError(
- "Please make sure USE_RANDOM is ON in the config.cmake " "on the remote devices"
- )
- for arg in args:
- random_fill(arg)
+ if os.path.exists(working_dir):
+ buffer_path = os.path.join(working_dir, "buffer.pkl")
+ if os.path.exists(buffer_path):
+ with open(buffer_path, "rb") as fi:
+ buffer = pickle.load(fi)
+ # force last args to be empty
+ args = []
+ for i in range(len(build_res.args) - 1):
+ args.append(ndarray.array(buffer[build_res.args[i].name], ctx=ctx))
+ args.append(ndarray.empty(get_const_tuple(build_res.args[-1].shape),\
+ build_res.args[-1].dtype, ctx))
+ is_buffer_exist = True
+ else:
+ args = [ndarray.empty(get_const_tuple(x.shape), x.dtype, ctx) for x in build_res.args]
+ try:
+ random_fill = remote.get_function("tvm.contrib.random.random_fill")
+ except AttributeError:
+ raise AttributeError(
+ "Please make sure USE_RANDOM is ON in the config.cmake " "on the remote devices"
+ )
+ for arg in args:
+ random_fill(arg)
ctx.sync()
costs = time_f(*args).results
+ if is_buffer_exist:
+ # check answer, only support single output for now
Review comment:
```suggestion
# check answer, only support single output for now
# todo(antinucleon): get the output information from `task.compute_dag` to support multiple output
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] antinucleon closed pull request #7053: [wip][auto_scheduler] buffer support, correctness check
Posted by GitBox <gi...@apache.org>.
antinucleon closed pull request #7053:
URL: https://github.com/apache/tvm/pull/7053
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] merrymercy commented on a change in pull request #7053: [auto_scheduler] buffer support, correctness check
Posted by GitBox <gi...@apache.org>.
merrymercy commented on a change in pull request #7053:
URL: https://github.com/apache/tvm/pull/7053#discussion_r539517512
##########
File path: python/tvm/auto_scheduler/auto_schedule.py
##########
@@ -117,14 +132,21 @@ def __init__(
num_measure_trials=0,
early_stopping=None,
num_measures_per_round=64,
+ working_dir=None,
+ check_correctness=False,
verbose=1,
builder="local",
+ builder_n_parallel=-1,
Review comment:
Yeah. I forget to delete it. Fixed by #7071
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] antinucleon commented on a change in pull request #7053: [auto_scheduler] buffer support, correctness check
Posted by GitBox <gi...@apache.org>.
antinucleon commented on a change in pull request #7053:
URL: https://github.com/apache/tvm/pull/7053#discussion_r539060438
##########
File path: python/tvm/auto_scheduler/auto_schedule.py
##########
@@ -117,14 +132,21 @@ def __init__(
num_measure_trials=0,
early_stopping=None,
num_measures_per_round=64,
+ working_dir=None,
+ check_correctness=False,
verbose=1,
builder="local",
+ builder_n_parallel=-1,
Review comment:
This file is no longer used according to https://github.com/apache/tvm/pull/7053#issuecomment-741010499 I think @merrymercy forgets to delete it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] antinucleon commented on pull request #7053: [wip][auto_scheduler] buffer support, correctness check
Posted by GitBox <gi...@apache.org>.
antinucleon commented on pull request #7053:
URL: https://github.com/apache/tvm/pull/7053#issuecomment-742134344
I find out this solution only work for a single task, but doesn't work with TaskScheduler. Will update it later.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] antinucleon commented on pull request #7053: [auto_scheduler] buffer support, correctness check
Posted by GitBox <gi...@apache.org>.
antinucleon commented on pull request #7053:
URL: https://github.com/apache/tvm/pull/7053#issuecomment-741516669
@FrozenGene @merrymercy Updated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] FrozenGene commented on a change in pull request #7053: [auto_scheduler] buffer support, correctness check
Posted by GitBox <gi...@apache.org>.
FrozenGene commented on a change in pull request #7053:
URL: https://github.com/apache/tvm/pull/7053#discussion_r538062449
##########
File path: python/tvm/auto_scheduler/auto_schedule.py
##########
@@ -136,9 +159,14 @@ def __init__(
if isinstance(runner, str):
if runner == "local":
- runner = LocalRunner()
+ runner = LocalRunner(working_dir=self.temp_working_dir)
else:
raise ValueError("Invalid runner: " + runner)
+
+ elif isinstance(runner, RPCRunner):
+ rpc_kwargs = runner.kwargs
+ rpc_kwargs["working_dir"] = self.temp_working_dir
Review comment:
How this argument be used later? I can not find any logic to handle this parameter...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] antinucleon commented on a change in pull request #7053: [auto_scheduler] buffer support, correctness check
Posted by GitBox <gi...@apache.org>.
antinucleon commented on a change in pull request #7053:
URL: https://github.com/apache/tvm/pull/7053#discussion_r538065121
##########
File path: python/tvm/auto_scheduler/auto_schedule.py
##########
@@ -136,9 +159,14 @@ def __init__(
if isinstance(runner, str):
if runner == "local":
- runner = LocalRunner()
+ runner = LocalRunner(working_dir=self.temp_working_dir)
else:
raise ValueError("Invalid runner: " + runner)
+
+ elif isinstance(runner, RPCRunner):
+ rpc_kwargs = runner.kwargs
+ rpc_kwargs["working_dir"] = self.temp_working_dir
Review comment:
https://github.com/apache/tvm/pull/7053/files#diff-5e828f80da7fdc456523468e7f69c1617d8a88867500f68998567fbd6f95a1d7R183
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] antinucleon commented on a change in pull request #7053: [auto_scheduler] buffer support, correctness check
Posted by GitBox <gi...@apache.org>.
antinucleon commented on a change in pull request #7053:
URL: https://github.com/apache/tvm/pull/7053#discussion_r539060438
##########
File path: python/tvm/auto_scheduler/auto_schedule.py
##########
@@ -117,14 +132,21 @@ def __init__(
num_measure_trials=0,
early_stopping=None,
num_measures_per_round=64,
+ working_dir=None,
+ check_correctness=False,
verbose=1,
builder="local",
+ builder_n_parallel=-1,
Review comment:
This file is no longer used according to https://github.com/apache/tvm/pull/7053#issuecomment-741010499
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] antinucleon commented on pull request #7053: [auto_scheduler] buffer support, correctness check
Posted by GitBox <gi...@apache.org>.
antinucleon commented on pull request #7053:
URL: https://github.com/apache/tvm/pull/7053#issuecomment-741538366
> I think you want to update submodule? So you have change showing you have changed the vta-hw submodule.
Updated
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] FrozenGene commented on a change in pull request #7053: [auto_scheduler] buffer support, correctness check
Posted by GitBox <gi...@apache.org>.
FrozenGene commented on a change in pull request #7053:
URL: https://github.com/apache/tvm/pull/7053#discussion_r538060242
##########
File path: python/tvm/auto_scheduler/auto_schedule.py
##########
@@ -210,6 +253,35 @@ def auto_schedule(task, search_policy=None, tuning_options=TuningOptions()):
if search_policy is None:
cost_model = XGBModel()
search_policy = SketchPolicy(task, cost_model)
+
+ if tuning_options.check_correctness == True:
+ empty_sch, args = task.compute_dag.apply_steps_from_state(
+ task.compute_dag.get_init_state(), layout_rewrite=True)
+ cpu_func = build_module.build(
+ empty_sch, args, target="llvm", target_host=task.target_host
+ )
+ buffer_path = os.path.join(tuning_options.working_dir, "buffer.pkl")
+ if os.path.exists(buffer_path) is True:
+ with open(buffer_path, "rb") as fi:
+ buffer = pickle.load(fi)
+ if len(buffer) == len(args):
+ # we skip check each arg shape here
+ pass
+ elif len(buffer) == len(args) - 1:
+ # assume only one output
+ np_args = np.zeros(size=get_const_tuple(args[-1].shape)).astype(args[-1].dtype)
+ cpu_args = [v for _, v in buffer.items()] + [ndarray.array(np_args, ctx=tvm.cpu())]
+ cpu_func(*cpu_args)
+ ### save cpu result
+ answer = [x.asnumpy() for x in cpu_args]
+ tuning_options.register_buffer(args[-1].name, answer[-1])
+ else:
+ np_args = [np.random.uniform(-0.1, 0.1, size=get_const_tuple(x.shape)).astype(x.dtype) for x in args]
Review comment:
We should use `random_fill` function as it help us handle different types. For example, for quantized uint8 dtype, [-0.1, 0.1] of `np.random.uniform(-0.1, 0.1, size=get_const_tuple(x.shape)).astype(x.dtype) for x in args` will be all zeros, which is not we want.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] merrymercy commented on a change in pull request #7053: [auto_scheduler] buffer support, correctness check
Posted by GitBox <gi...@apache.org>.
merrymercy commented on a change in pull request #7053:
URL: https://github.com/apache/tvm/pull/7053#discussion_r538808502
##########
File path: python/tvm/auto_scheduler/measure.py
##########
@@ -889,18 +929,43 @@ def _timed_rpc_run(
if error_no == 0:
try:
- args = [ndarray.empty(get_const_tuple(x.shape), x.dtype, ctx) for x in build_res.args]
- try:
- random_fill = remote.get_function("tvm.contrib.random.random_fill")
- except AttributeError:
- raise AttributeError(
- "Please make sure USE_RANDOM is ON in the config.cmake " "on the remote devices"
- )
- for arg in args:
- random_fill(arg)
+ if os.path.exists(working_dir):
+ buffer_path = os.path.join(working_dir, "buffer.pkl")
+ if os.path.exists(buffer_path):
+ with open(buffer_path, "rb") as fi:
+ buffer = pickle.load(fi)
+ # force last args to be empty
+ args = []
+ for i in range(len(build_res.args) - 1):
+ args.append(ndarray.array(buffer[build_res.args[i].name], ctx=ctx))
+ args.append(ndarray.empty(get_const_tuple(build_res.args[-1].shape),\
+ build_res.args[-1].dtype, ctx))
+ is_buffer_exist = True
+ else:
+ args = [ndarray.empty(get_const_tuple(x.shape), x.dtype, ctx) for x in build_res.args]
+ try:
+ random_fill = remote.get_function("tvm.contrib.random.random_fill")
+ except AttributeError:
+ raise AttributeError(
+ "Please make sure USE_RANDOM is ON in the config.cmake " "on the remote devices"
+ )
+ for arg in args:
+ random_fill(arg)
ctx.sync()
costs = time_f(*args).results
+ if is_buffer_exist:
+ # check answer, only support single output for now
Review comment:
```suggestion
# check answer, only support single output for now
# todo(antinucleon): get the output information from `task.compute_dag` to support multiple output
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] merrymercy edited a comment on pull request #7053: [auto_scheduler] buffer support, correctness check
Posted by GitBox <gi...@apache.org>.
merrymercy edited a comment on pull request #7053:
URL: https://github.com/apache/tvm/pull/7053#issuecomment-741010499
I recently refactored the task interface (https://github.com/apache/tvm/pull/7028).
The `auto_schedule.py` file is deleted. Please rebase.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] FrozenGene commented on a change in pull request #7053: [auto_scheduler] buffer support, correctness check
Posted by GitBox <gi...@apache.org>.
FrozenGene commented on a change in pull request #7053:
URL: https://github.com/apache/tvm/pull/7053#discussion_r539004649
##########
File path: python/tvm/auto_scheduler/measure.py
##########
@@ -276,13 +279,16 @@ class LocalBuilder(ProgramBuilder):
timeout : int = 15
The timeout limit (in second) for each build thread.
This is used in a wrapper of the multiprocessing.Process.join().
- n_parallel : int = multiprocessing.cpu_count()
+ n_parallel : int = None
Number of threads used to build in parallel.
+ When the value is -1, will be set to multiprocessing.cpu_count() instead
Review comment:
`-1` should be `None`
##########
File path: python/tvm/auto_scheduler/search_task.py
##########
@@ -118,14 +130,21 @@ def __init__(
num_measure_trials=0,
early_stopping=None,
num_measures_per_round=64,
+ working_dir=None,
+ check_correctness=False,
verbose=1,
builder="local",
+ builder_n_parallel=-1,
Review comment:
The `-1` is conflict with `None` default value before? Could we unify the default value (-1 or None) when users don't set.? Currently, I am a bit confused with two default values
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] antinucleon commented on a change in pull request #7053: [auto_scheduler] buffer support, correctness check
Posted by GitBox <gi...@apache.org>.
antinucleon commented on a change in pull request #7053:
URL: https://github.com/apache/tvm/pull/7053#discussion_r538844883
##########
File path: python/tvm/auto_scheduler/auto_schedule.py
##########
@@ -210,6 +253,35 @@ def auto_schedule(task, search_policy=None, tuning_options=TuningOptions()):
if search_policy is None:
cost_model = XGBModel()
search_policy = SketchPolicy(task, cost_model)
+
+ if tuning_options.check_correctness == True:
+ empty_sch, args = task.compute_dag.apply_steps_from_state(
+ task.compute_dag.get_init_state(), layout_rewrite=True)
+ cpu_func = build_module.build(
+ empty_sch, args, target="llvm", target_host=task.target_host
+ )
+ buffer_path = os.path.join(tuning_options.working_dir, "buffer.pkl")
+ if os.path.exists(buffer_path) is True:
+ with open(buffer_path, "rb") as fi:
+ buffer = pickle.load(fi)
+ if len(buffer) == len(args):
+ # we skip check each arg shape here
+ pass
+ elif len(buffer) == len(args) - 1:
+ # assume only one output
+ np_args = np.zeros(size=get_const_tuple(args[-1].shape)).astype(args[-1].dtype)
+ cpu_args = [v for _, v in buffer.items()] + [ndarray.array(np_args, ctx=tvm.cpu())]
+ cpu_func(*cpu_args)
+ ### save cpu result
+ answer = [x.asnumpy() for x in cpu_args]
+ tuning_options.register_buffer(args[-1].name, answer[-1])
+ else:
+ np_args = [np.random.uniform(-0.1, 0.1, size=get_const_tuple(x.shape)).astype(x.dtype) for x in args]
Review comment:
Fixed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] merrymercy commented on a change in pull request #7053: [auto_scheduler] buffer support, correctness check
Posted by GitBox <gi...@apache.org>.
merrymercy commented on a change in pull request #7053:
URL: https://github.com/apache/tvm/pull/7053#discussion_r538817599
##########
File path: python/tvm/auto_scheduler/measure.py
##########
@@ -740,6 +775,7 @@ def local_run(
min_repeat_ms=0,
cooldown_interval=0,
enable_cpu_cache_flush=False,
+ working_dir="",
Review comment:
The correctness check is not implemented for LocalRunner.
We should at least add an assertion here.
##########
File path: python/tvm/auto_scheduler/auto_schedule.py
##########
@@ -210,6 +253,35 @@ def auto_schedule(task, search_policy=None, tuning_options=TuningOptions()):
if search_policy is None:
cost_model = XGBModel()
search_policy = SketchPolicy(task, cost_model)
+
+ if tuning_options.check_correctness == True:
+ empty_sch, args = task.compute_dag.apply_steps_from_state(
+ task.compute_dag.get_init_state(), layout_rewrite=True)
+ cpu_func = build_module.build(
+ empty_sch, args, target="llvm", target_host=task.target_host
+ )
+ buffer_path = os.path.join(tuning_options.working_dir, "buffer.pkl")
+ if os.path.exists(buffer_path) is True:
+ with open(buffer_path, "rb") as fi:
+ buffer = pickle.load(fi)
+ if len(buffer) == len(args):
+ # we skip check each arg shape here
+ pass
+ elif len(buffer) == len(args) - 1:
+ # assume only one output
Review comment:
```suggestion
# assume only one output
# todo(antinucleon): get the output information from `task.compute_dag` to support multiple output
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] merrymercy commented on a change in pull request #7053: [auto_scheduler] buffer support, correctness check
Posted by GitBox <gi...@apache.org>.
merrymercy commented on a change in pull request #7053:
URL: https://github.com/apache/tvm/pull/7053#discussion_r538809651
##########
File path: python/tvm/auto_scheduler/auto_schedule.py
##########
@@ -210,6 +253,35 @@ def auto_schedule(task, search_policy=None, tuning_options=TuningOptions()):
if search_policy is None:
cost_model = XGBModel()
search_policy = SketchPolicy(task, cost_model)
+
+ if tuning_options.check_correctness == True:
+ empty_sch, args = task.compute_dag.apply_steps_from_state(
+ task.compute_dag.get_init_state(), layout_rewrite=True)
Review comment:
```suggestion
task.compute_dag.get_init_state())
```
##########
File path: python/tvm/auto_scheduler/auto_schedule.py
##########
@@ -210,6 +253,35 @@ def auto_schedule(task, search_policy=None, tuning_options=TuningOptions()):
if search_policy is None:
cost_model = XGBModel()
search_policy = SketchPolicy(task, cost_model)
+
+ if tuning_options.check_correctness == True:
+ empty_sch, args = task.compute_dag.apply_steps_from_state(
+ task.compute_dag.get_init_state(), layout_rewrite=True)
+ cpu_func = build_module.build(
+ empty_sch, args, target="llvm", target_host=task.target_host
Review comment:
```suggestion
empty_sch, args, target="llvm",
```
This function runs on the host machine. So we should not use `target_host`.
##########
File path: python/tvm/auto_scheduler/measure.py
##########
@@ -889,18 +929,43 @@ def _timed_rpc_run(
if error_no == 0:
try:
- args = [ndarray.empty(get_const_tuple(x.shape), x.dtype, ctx) for x in build_res.args]
- try:
- random_fill = remote.get_function("tvm.contrib.random.random_fill")
- except AttributeError:
- raise AttributeError(
- "Please make sure USE_RANDOM is ON in the config.cmake " "on the remote devices"
- )
- for arg in args:
- random_fill(arg)
+ if os.path.exists(working_dir):
+ buffer_path = os.path.join(working_dir, "buffer.pkl")
+ if os.path.exists(buffer_path):
+ with open(buffer_path, "rb") as fi:
+ buffer = pickle.load(fi)
+ # force last args to be empty
+ args = []
+ for i in range(len(build_res.args) - 1):
+ args.append(ndarray.array(buffer[build_res.args[i].name], ctx=ctx))
Review comment:
This will cause a copy over RPC for every measurement. A better way is to copy the `buffer.pkl` to the remote device once and do the initialization/check on the remote device.
We can leave a "todo" here and leave this optimization to future PRs.
##########
File path: python/tvm/auto_scheduler/measure.py
##########
@@ -275,13 +278,16 @@ class LocalBuilder(ProgramBuilder):
timeout : int = 15
The timeout limit (in second) for each build thread.
This is used in a wrapper of the multiprocessing.Process.join().
- n_parallel : int = multiprocessing.cpu_count()
+ n_parallel : int = -1
Review comment:
```suggestion
n_parallel : Optional[int] = None
```
Use None as default
##########
File path: python/tvm/auto_scheduler/auto_schedule.py
##########
@@ -210,6 +253,35 @@ def auto_schedule(task, search_policy=None, tuning_options=TuningOptions()):
if search_policy is None:
cost_model = XGBModel()
search_policy = SketchPolicy(task, cost_model)
+
+ if tuning_options.check_correctness == True:
+ empty_sch, args = task.compute_dag.apply_steps_from_state(
+ task.compute_dag.get_init_state(), layout_rewrite=True)
+ cpu_func = build_module.build(
+ empty_sch, args, target="llvm", target_host=task.target_host
+ )
+ buffer_path = os.path.join(tuning_options.working_dir, "buffer.pkl")
+ if os.path.exists(buffer_path) is True:
+ with open(buffer_path, "rb") as fi:
+ buffer = pickle.load(fi)
+ if len(buffer) == len(args):
+ # we skip check each arg shape here
+ pass
+ elif len(buffer) == len(args) - 1:
+ # assume only one output
+ np_args = np.zeros(size=get_const_tuple(args[-1].shape)).astype(args[-1].dtype)
+ cpu_args = [v for _, v in buffer.items()] + [ndarray.array(np_args, ctx=tvm.cpu())]
Review comment:
```suggestion
cpu_args = [v for _, v in buffer.items()] + [ndarray.empty(shape=get_const_tuple(args[-1].shape), dtype=args[-1].dtype, ctx=tvm.cpu())]
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] FrozenGene commented on a change in pull request #7053: [auto_scheduler] buffer support, correctness check
Posted by GitBox <gi...@apache.org>.
FrozenGene commented on a change in pull request #7053:
URL: https://github.com/apache/tvm/pull/7053#discussion_r539040463
##########
File path: python/tvm/auto_scheduler/auto_schedule.py
##########
@@ -117,14 +132,21 @@ def __init__(
num_measure_trials=0,
early_stopping=None,
num_measures_per_round=64,
+ working_dir=None,
+ check_correctness=False,
verbose=1,
builder="local",
+ builder_n_parallel=-1,
Review comment:
Seems we still omit here of `-1`, there is no place to have logic to handle `-1` now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] merrymercy commented on pull request #7053: [auto_scheduler] buffer support, correctness check
Posted by GitBox <gi...@apache.org>.
merrymercy commented on pull request #7053:
URL: https://github.com/apache/tvm/pull/7053#issuecomment-741010499
I recently refactored the task interface (https://github.com/apache/tvm/pull/7028).
The `auto_schedule.py` file is deleted.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org