Using the (experimental) Spanner connector

[Mark Striebeck <ma...@gmail.com>]

Hi,

I was wondering if anybody here uses one of the two experimental Spanner
connectors in beam (io.gcp.spanner or io.gcp.experimental.spannerio). I
wanted to try it out for our pipeline but ran into what seems to be
compatibility issues between the beam SDP and the Google Spanner SDK.

And before I am diving more into this, I would just like to know if people
are using these for their production pipelines or if these are truly
experimental.

Thanks
     Mark

Re: Using the (experimental) Spanner connector

[Chamikara Jayalath <ch...@google.com>]

On Thu, May 19, 2022 at 11:09 AM Mark Striebeck <ma...@gmail.com>
wrote:

> Hi,
>
> I was wondering if anybody here uses one of the two experimental Spanner
> connectors in beam (io.gcp.spanner or io.gcp.experimental.spannerio). I
> wanted to try it out for our pipeline but ran into what seems to be
> compatibility issues between the beam SDP and the Google Spanner SDK.
>
> And before I am diving more into this, I would just like to know if people
> are using these for their production pipelines or if these are truly
> experimental.
>

It's been around for some time but I don't think we have adequate
integration testing to measure its performance for production use-cases.

We also have spanner.py [1] which is a multi-language [2] version of the
Java SpannerIO connector.

Thanks,
Cham

[1]
https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/gcp/spanner.py
[2]
https://beam.apache.org/documentation/programming-guide/#multi-language-pipelines


> Thanks
>      Mark
>