[GitHub] [incubator-tvm] comaniac opened a new pull request #4520: [AutoTVM] Fix Empty Config caused Crashing

[GitBox <gi...@apache.org>]

comaniac opened a new pull request #4520: [AutoTVM] Fix Empty Config caused Crashing
URL: https://github.com/apache/incubator-tvm/pull/4520
 
 
   ### **Problem Description**
   Some TOPI schedules such as `depthwise_conv2d_nchw` on x86 has no tuning space at all. However since `depthwise_conv2d_nchw` on CUDA has a tuning space, we still list `topi_nn_depthwise_conv2d_nchw` as an AutoTVM task. It means `depthwise_conv2d_nchw` on x86 will be extracted by `extract_from_program` and "tuned" by AutoTVM. The problem is that this will result in the following record in the history log file:
   
   ```bash
   {"i": ["llvm", "topi_nn_depthwise_conv2d_nchw", [["TENSOR", [1, 192, 28, 28], "float32"], ["TENSOR", [192, 1, 3, 3], "float32"], [2, 2], [1, 1], [1, 1], "float32"], {}, ["depthwise_conv2d_nchw", [1, 192, 28, 28, "float32"], [192, 1, 3, 3, "float32"], [2, 2], [1, 1], [1, 1], "float32"], {"i": 0, "t": "direct", "c": null, "e": []}], "r": ...}
   
   ```
   
   Taking a closer look,  we will find an empty config:
   
   ```bash
   {"i": 0, "t": "direct", "c": null, "e": []}
   ```
   
   Although it is empty, it is still a valid record and will be loaded when `ApplyHistoryBest`. The problem is that x86 with `opt_level=3`, TVM will use AlterOp to replace NCHW to NCHWc. During this process, it tries to apply the empty config to the NCHWc schedlue template, which results in the following error snippet:
   
   ```
     File "/home/ubuntu/tvm/python/tvm/_ffi/_ctypes/function.py", line 72, in cfun
       rv = local_pyfunc(*pyargs)
     File "/home/ubuntu/tvm/python/tvm/relay/op/nn/_nn.py", line 232, in alter_op_layout_conv2d
       return topi.nn.conv2d_alter_layout(attrs, inputs, tinfos, op)
     File "</usr/local/lib/python3.7/dist-packages/decorator.py:decorator-gen-36>", line 2, in conv2d_alter_layout
     File "/home/ubuntu/tvm/python/tvm/target.py", line 382, in dispatch_func
       return dispatch_dict[k](*args, **kwargs)
     File "/home/ubuntu/tvm/topi/python/topi/x86/conv2d_alter_op.py", line 98, in _alter_conv2d_layout
       ic_bn, oc_bn = cfg["tile_ic"].size[-1], cfg["tile_oc"].size[-1]
     File "/home/ubuntu/tvm/python/tvm/autotvm/task/space.py", line 806, in __getitem__
       return self._entity_map[name]
   KeyError: 'tile_ic'
   ```
   
   The most obvious case is tuning MoibleNet V2 on x86 which has lots of depthwise conv2d, and this problem has been reported at several topics [[1]](https://discuss.tvm.ai/t/keyerror-tile-ic-when-tuning-mobilenet-for-mobile-gpu/4217)[[2]](https://discuss.tvm.ai/t/keyerror-tile-oh-when-tuning-resnet50/4003)[[3]](https://discuss.tvm.ai/t/x86-relay-auto-tune-mobilefacenet-error-using-relay/2508/16)[[4]](https://discuss.tvm.ai/t/topi-depthwise-conv2d-config-application-error/4223). [The workaround](https://discuss.tvm.ai/t/solved-keyerror-tile-ic-when-tuning-mobilenet/3757/4) suggests users to manually replace NCHW with NCHWc, but it is still confusing.
   
   ### **Solution**
   This PR proposes a quick fix to this problem by ignoring the records with an empty config. I choose this solution instead of the following alternatives for reasons. The root solution is actually making up a tuning space for depthwise conv2d on x86 tho.
   
   * **Do not log empty config in callback**: We may be still interested in the result of this op with this schedule even it has no config.
   
   * **Use fallback config when the history best config has no entities**: Similar to what I proposed but it needs to modify more places.
   
   cc @eqy @merrymercy @kevinthesun 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services