You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@beam.apache.org by Chamikara Jayalath via user <us...@beam.apache.org> on 2023/08/10 22:48:32 UTC

[ANNOUNCE] Transform Service

Hi All,

We recently added a Docker Compose based service named Transform Service to
Beam.

Transform service includes a number of transforms released with Beam and
provides a single endpoint for accessing them via the Beam's multi-language
pipelines framework.

I've updated Beam Java/Python SDKs to automatically use this service to
expand cross-language transforms used by multi-lang pipelines
when possible. This means that Beam pipelines can use cross-language
transforms without installing other language runtimes if they have Docker
(and Docker Compose which comes with Docker) available locally at job
submission. Go SDK updates are in development.

Users also have the option to manually startup a Transform Service with
utilities provided with Beam SDKs if needed.

For mode details regarding the Transform Service please see the
documentation here
<https://beam.apache.org/documentation/programming-guide/#transform-service>
.

A list of transforms currently included with the Transform Service is
available here
<https://cwiki.apache.org/confluence/display/BEAM/Transform+Service>.

Please see here
<https://lists.apache.org/thread/vdflzk3cz9g9xrhxwhyjvf5xdfhwjdko> for a
previous discussion on this and please let me know if you have any
questions.

Thanks,
Cham

Re: [ANNOUNCE] Transform Service

Posted by Ahmet Altay via dev <de...@beam.apache.org>.

Congratulations! This is a great usability improvement, lowering the bar
for using multi language features.

On Thu, Aug 10, 2023 at 3:48 PM Chamikara Jayalath via dev <
dev@beam.apache.org> wrote:

> Hi All,
>
> We recently added a Docker Compose based service named Transform Service
> to Beam.
>
> Transform service includes a number of transforms released with Beam and
> provides a single endpoint for accessing them via the Beam's multi-language
> pipelines framework.
>
> I've updated Beam Java/Python SDKs to automatically use this service to
> expand cross-language transforms used by multi-lang pipelines
> when possible. This means that Beam pipelines can use cross-language
> transforms without installing other language runtimes if they have Docker
> (and Docker Compose which comes with Docker) available locally at job
> submission. Go SDK updates are in development.
>
> Users also have the option to manually startup a Transform Service with
> utilities provided with Beam SDKs if needed.
>
> For mode details regarding the Transform Service please see the
> documentation here
> <https://beam.apache.org/documentation/programming-guide/#transform-service>
> .
>
> A list of transforms currently included with the Transform Service is
> available here
> <https://cwiki.apache.org/confluence/display/BEAM/Transform+Service>.
>
> Please see here
> <https://lists.apache.org/thread/vdflzk3cz9g9xrhxwhyjvf5xdfhwjdko> for a
> previous discussion on this and please let me know if you have any
> questions.
>
> Thanks,
> Cham
>
>