[TVM Discuss] [Development/RFC] [RFC] CoreML Runtime

[Morita Kazutaka via TVM Discuss <no...@discuss.tvm.ai>]

In this RFC, we would like to propose adding a runtime to load and execute CoreML models from TVM.

## Motivation

- Currently, using CoreML is a defacto standard approach to run inference on iOS.  This runtime is
  useful to obtain a baseline benchmark and compare it with TVM.

- Using CoreML is the only way to leverage Apple's Neural Engine on
  iOS devices.  With this runtime, we could extract a subgraph and offload it
  to CoreML to achieve better performance in future.

## How this runtime works

Here is a proposal of how to run CoreML inference via TVM RPC.  In this workflow, we assume that a RPC proxy is running somewhere and iOS [TVM RPC app](https://github.com/apache/incubator-tvm/tree/master/apps/ios_rpc) is set up on the iOS device beforehand.

1. Compile CoreML model

    Although there is [an API to compile a CoreML model](https://developer.apple.com/documentation/coreml/mlmodel/2921516-compilemodelaturl), we don't use it.
It is because the API works only on the device and it makes development hard.  We have to use an iOS device (or simulator) even for testing.
Instead, we compile a CoreML model with the xcode `coremlc` command.  We can completely test CoreML runtime only on macOS with this approach.

2. Send a compiled model to the device and launch RPC server

    We pack a compiled CoreML model (modelname.mlmodelc) into the iOS TVM RPC app bundle,
and then, launch the RPC server and connect to the RPC proxy.  These are automated with
`xcodebuild test`.

3. Create CoreML module and run inference

    We create a CoreML module with a packed func (tvm.coreml_runtime.create), and run CoreML inference on iOS via the RPC proxy like other RPC usecases.

## Implementation

Implementation is available at https://github.com/apache/incubator-tvm/pull/5283.

cc @tqchen @ziheng @FrozenGene





---
[Visit Topic](https://discuss.tvm.ai/t/rfc-coreml-runtime/6309/1) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/a3e6572ab317019775484945a3084b7ee976f96cd62218c5857fd101d10cf9fd).

[TVM Discuss] [Development/RFC] [RFC] CoreML Runtime

[tqchen via TVM Discuss <no...@discuss.tvm.ai>]

Indeed, having a coreml runtime support would be quite interesting for our iOS backend





---
[Visit Topic](https://discuss.tvm.ai/t/rfc-coreml-runtime/6309/5) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/44f9873558316d1932de27fd6a099cc4610f92b9a53689bbd39b40a43869540a).

[TVM Discuss] [Development/RFC] [RFC] CoreML Runtime

[Zhao Wu via TVM Discuss <no...@discuss.tvm.ai>]

[quote="kazum, post:3, topic:6309"]
$(xcode-select -p)/usr/bin/coremlc
[/quote]

XCode 10.1 works. (Post must be 20 characters)





---
[Visit Topic](https://discuss.tvm.ai/t/rfc-coreml-runtime/6309/4) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/9084e723691f589c2394e4d83c9655fee171c406197ef725e5248f9559dc6efc).

[TVM Discuss] [Development/RFC] [RFC] CoreML Runtime

[Morita Kazutaka via TVM Discuss <no...@discuss.tvm.ai>]

@FrozenGene  Sorry, my description is ambiguous.  You can compile a CoreML model with the following command:

    $(xcode-select -p)/usr/bin/coremlc compile [model.mlmodel] [outputFolder]

or

    xcrun coremlcompiler compile [model.mlmodel] [outputFolder]

I tried Xcode 11.4 and 10.3, and I could compile a model on both environment. I think Xcode 10.1 also works.





---
[Visit Topic](https://discuss.tvm.ai/t/rfc-coreml-runtime/6309/3) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/36492f22e73accee03eef3273f2ff3d20e4817c52a64f7240d68760997030fff).

[TVM Discuss] [Development/RFC] [RFC] CoreML Runtime

[Zhao Wu via TVM Discuss <no...@discuss.tvm.ai>]

I think leveraging Apple’s Neural Engine is one good motivation (we could add one example how to leverage this). As we have TFLite's runtime, I think add CoreML runtime is reasonable. 

[quote="kazum, post:1, topic:6309"]
Instead, we compile a CoreML model with the xcode `coremlc` command.
[/quote]

Does this have one version of XCore requires? My XCode is 10.1 but I can not find `coremlc` and only find `coremlcompiler` command.





---
[Visit Topic](https://discuss.tvm.ai/t/rfc-coreml-runtime/6309/2) to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/28bd90d1b55f259ca0c7df411a8927ba45324d30cf64b54c6d83036676f22331).