You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Andy Ye <ye...@google.com> on 2022/02/11 23:31:31 UTC
[RFC][Design] RunInference for Scikit-Learn and Pytorch
TLDR: We want to enable Beam users to make machine learning predictions in
their pipelines using two popular ML frameworks: scikit-learn and Pytorch.
Please review our doc[1] to help us refine our design!
Hey everyone,
Users of machine learning frameworks must currently implement their own
transforms for running ML inferences. The exception is the TensorFlow
RunInference
transform
<https://github.com/tensorflow/tfx-bsl/blob/master/tfx_bsl/beam/run_inference.py>.
However, this is hosted in its own repo
<https://github.com/tensorflow/tfx-bsl>, and has an API
<https://www.tensorflow.org/tfx/tfx_bsl/api_docs/python/tfx_bsl/public/beam/RunInference>
that
is exclusively geared towards the TensorFlow TFX library. Our goal is to
add new implementations of RunInference for the two other popular machine
learning frameworks: scikit-learn and Pytorch.
@Ryan Thompson <ry...@google.com> and I would appreciate a review of
this design doc[1]. Thanks!
- Andy and Ryan
[1]
https://docs.google.com/document/d/1bVMU7Uo9Nzuu6aXR702j74nhQK4j6J1lkRVVBRySI0g/edit#