You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@beam.apache.org by Jack McCluskey via dev <de...@beam.apache.org> on 2025/03/24 14:11:51 UTC

[Design Doc] Generic Remote Model Handlers for RunInference

Hey everyone,

I've put together a design doc for a generic remote model handler base class
<https://docs.google.com/document/d/17A_oHJ7s3ol4TGCUpKeYc6iozkcTwN_e4H_J3kRlgPM/edit?usp=sharing>[1],
trying to abstract away some of the rougher edges of having inference calls
to a remote service in the middle of a pipeline. It's largely based on the
existing Vertex AI model handler that I implemented in Beam roughly two
years ago, so some of the code will look familiar if you've looked at that
class before. There's also a prototype PR
<https://github.com/apache/beam/pull/34379>[2] available with some small
unit tests and documentation. Please take a look and let me know what you
think!

Thanks,

Jack McCluskey

[1]
https://docs.google.com/document/d/17A_oHJ7s3ol4TGCUpKeYc6iozkcTwN_e4H_J3kRlgPM/edit?usp=sharing

[2] https://github.com/apache/beam/pull/34379

-- 


Jack McCluskey
SWE - DataPLS PLAT/ Dataflow ML
RDU
jrmccluskey@google.com

Re: [Design Doc] Generic Remote Model Handlers for RunInference

Posted by Kenneth Knowles <ke...@apache.org>.

+1 makes sense.

My questions (on the doc) are around how much this is user facing vs a
backdrop for Beam devs to build more stuff. That determines how much I care
about the nitty gritty details of the UX of making a new implementation of
the API. Having at least a couple implementations in mind also would add
confidence.

On Mon, Mar 24, 2025 at 3:12 PM Danny McCormick via dev <de...@beam.apache.org>
wrote:

> Thanks - left a UX suggestion, but overall I think this looks good and
> will be quite useful.
>
> On Mon, Mar 24, 2025 at 10:12 AM Jack McCluskey via dev <
> dev@beam.apache.org> wrote:
>
>> Hey everyone,
>>
>> I've put together a design doc for a generic remote model handler base
>> class
>> <https://docs.google.com/document/d/17A_oHJ7s3ol4TGCUpKeYc6iozkcTwN_e4H_J3kRlgPM/edit?usp=sharing>[1],
>> trying to abstract away some of the rougher edges of having inference calls
>> to a remote service in the middle of a pipeline. It's largely based on the
>> existing Vertex AI model handler that I implemented in Beam roughly two
>> years ago, so some of the code will look familiar if you've looked at that
>> class before. There's also a prototype PR
>> <https://github.com/apache/beam/pull/34379>[2] available with some small
>> unit tests and documentation. Please take a look and let me know what you
>> think!
>>
>> Thanks,
>>
>> Jack McCluskey
>>
>> [1]
>> https://docs.google.com/document/d/17A_oHJ7s3ol4TGCUpKeYc6iozkcTwN_e4H_J3kRlgPM/edit?usp=sharing
>>
>> [2] https://github.com/apache/beam/pull/34379
>>
>> --
>>
>>
>> Jack McCluskey
>> SWE - DataPLS PLAT/ Dataflow ML
>> RDU
>> jrmccluskey@google.com
>>
>>
>>

Re: [Design Doc] Generic Remote Model Handlers for RunInference

Posted by Danny McCormick via dev <de...@beam.apache.org>.

Thanks - left a UX suggestion, but overall I think this looks good and will
be quite useful.

On Mon, Mar 24, 2025 at 10:12 AM Jack McCluskey via dev <de...@beam.apache.org>
wrote:

> Hey everyone,
>
> I've put together a design doc for a generic remote model handler base
> class
> <https://docs.google.com/document/d/17A_oHJ7s3ol4TGCUpKeYc6iozkcTwN_e4H_J3kRlgPM/edit?usp=sharing>[1],
> trying to abstract away some of the rougher edges of having inference calls
> to a remote service in the middle of a pipeline. It's largely based on the
> existing Vertex AI model handler that I implemented in Beam roughly two
> years ago, so some of the code will look familiar if you've looked at that
> class before. There's also a prototype PR
> <https://github.com/apache/beam/pull/34379>[2] available with some small
> unit tests and documentation. Please take a look and let me know what you
> think!
>
> Thanks,
>
> Jack McCluskey
>
> [1]
> https://docs.google.com/document/d/17A_oHJ7s3ol4TGCUpKeYc6iozkcTwN_e4H_J3kRlgPM/edit?usp=sharing
>
> [2] https://github.com/apache/beam/pull/34379
>
> --
>
>
> Jack McCluskey
> SWE - DataPLS PLAT/ Dataflow ML
> RDU
> jrmccluskey@google.com
>
>
>