[Apache TVM Discuss] [Development] Duplication of the driver between C++ and Python

[Matt Barrett via Apache TVM Discuss <no...@discuss.tvm.ai>]

I've been looking into the TVM lower/build pipeline recently and have encountered an unusual duplication around the 'driver'. In particular, we have two files
`src/driver/driver_api.cc` and `python/tvm/driver/build_module.py` which both seem to independently define almost identical functionality. What has motivated this design rather than using the ffi to just expose the C++ functionality through Python? I'm interested in making some changes in this part of the code so it would be good to know whether I have to make identical changes in both files.

Additionally, I'm seeing a strange 'mix-and-match' between these two files when invoking `relay.build`. In particular, we seem to use the Python version of `lower` but the C++ version of `build`. I can find no obvious motivation for this behaviour in the code.

Any insights would be greatly appreciated!

@manupa-arm @tqchen





---
[Visit Topic](https://discuss.tvm.apache.org/t/duplication-of-the-driver-between-c-and-python/9702/1) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/ea93ac117c74b21f5f386c69562cb90b37bb4dd1ed89113c07f7d518a4ff0f9a).

[Apache TVM Discuss] [Development] Duplication of the driver between C++ and Python

[Manupa Karunaratne via Apache TVM Discuss <no...@discuss.tvm.ai>]

Yes, this part had been a pain point in figuring out which part of the compilation pipeline is being run. 

Regarding, the *lower*, I think C++ version is not run (maybe not anywhere in the tvm compilation -- correct me if I am wrong) at the minute because there is a check for the registered python ffi of build(...). 

Moreover, I found out that some relay passes (e.g. foldconstants) rely on the interpreter (which runs the JIT() of compile engine) to build and execute part of the artifacts in the process of relay optimization pipeline. What is interesting is that JIT() of compile engine uses the driver/build_module.py's build though subsequently relay.build(...) will invoke the C++ version of build in driver_api.cc later on. 

Do we have a reason for the existence of both ?
Moreover, is there an assumption in the design these two build(IRModule, TE) --> runtime.Module functionality to be kept implicitly identical ?

also cc : @csullivan @jroesch





---
[Visit Topic](https://discuss.tvm.apache.org/t/duplication-of-the-driver-between-c-and-python/9702/2) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.apache.org/email/unsubscribe/3ab347efef68691d3e1112c6649e67eac924283f12fa8ba06604109ba68c4ae4).