You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Andrew Jorgensen (Jira)" <ji...@apache.org> on 2022/03/30 01:53:00 UTC
[jira] [Updated] (BEAM-14202) SpannerConfig does not respect SPANNER_EMULATOR_HOST
[ https://issues.apache.org/jira/browse/BEAM-14202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Jorgensen updated BEAM-14202:
------------------------------------
Description:
[com.google.cloud.spanner.SpannerOptions|https://github.com/googleapis/java-spanner/blob/main/google-cloud-spanner/src/main/java/com/google/cloud/spanner/SpannerOptions.java] has to determine if the configuration should use the emulator or not. Essentially it will look for SPANNER_EMULATOR_HOST in the environment and if a spanner host is not explicitly set it will set the host as the emulator, [see here|[https://github.com/googleapis/java-spanner/blob/main/google-cloud-spanner/src/main/java/com/google/cloud/spanner/SpannerOptions.java#L1143-L1153]
By Contrast in Beam, SpannerConfig by default has a host set ([see here|[https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerConfig.java#L83]). When Beam goes to convert SpannerConfig to SpannerOptions in SpannerAccessor it will always call setHost on SpannerOptions which will set the emulatorHost to null, [see here|https://github.com/apache/beam/blob/47ab260aaad6b1014d86dd6cf9274f1045690631/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerAccessor.java#L173-L176].
The consequence of this is the SpannerAccessor will always ignore the SPANNER_EMULATOR_HOST environment variable unless withEmulatorHost is called explicitly on SpannerConfig. Ultimately this pushes the responsibility onto the user to manually check for the environment variable and conditionally set it rather than relying on the functionality that already exists in the underlying SpannerOptions provided by the spanner sdk.
I think the solution would be for Beam to also check to see if SPANNER_EMULATOR_HOST is set and then fall back to the provided host or the DEFAULT_HOST in the event that we are not connecting to the emulator.
was:
[com.google.cloud.spanner.SpannerOptions|https://github.com/googleapis/java-spanner/blob/main/google-cloud-spanner/src/main/java/com/google/cloud/spanner/SpannerOptions.java] has to determine if the configuration should use the emulator or not. Essentially it will look for SPANNER_EMULATOR_HOST in the environment and if a spanner host is not explicitly set it will set the host as the emulator, [see here|[https://github.com/googleapis/java-spanner/blob/main/google-cloud-spanner/src/main/java/com/google/cloud/spanner/SpannerOptions.java#L1143-L1153]
By Contrast in Beam, SpannerConfig by default has a host set ([see here|[https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerConfig.java#L83]). When Beam goes to convert SpannerConfig to SpannerOptions in SpannerAccessor it will always call setHost on SpannerOptions which will set the emulatorHost to null, [see here|https://github.com/apache/beam/blob/47ab260aaad6b1014d86dd6cf9274f1045690631/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerAccessor.java#L173-L176].
The consequence of this is the SpannerAccessor will always ignore the SPANNER_EMULATOR_HOST environment variable unless withEmulatorHost is called explicitly on SpannerConfig. Ultimately this pushes the responsibility onto the user to manually check for the environment variable and conditionally set it rather than relying on the functionality that already exists in the underlying SpannerOptions provided by the spanner sdk.
I think the solution would be for Beam to also check to see if SPANNER_EMULATOR_HOST is set and then fall back to the provided host or the DEFAULT_HOST.
> SpannerConfig does not respect SPANNER_EMULATOR_HOST
> ----------------------------------------------------
>
> Key: BEAM-14202
> URL: https://issues.apache.org/jira/browse/BEAM-14202
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Affects Versions: 2.37.0
> Reporter: Andrew Jorgensen
> Priority: P2
> Fix For: Not applicable
>
>
> [com.google.cloud.spanner.SpannerOptions|https://github.com/googleapis/java-spanner/blob/main/google-cloud-spanner/src/main/java/com/google/cloud/spanner/SpannerOptions.java] has to determine if the configuration should use the emulator or not. Essentially it will look for SPANNER_EMULATOR_HOST in the environment and if a spanner host is not explicitly set it will set the host as the emulator, [see here|[https://github.com/googleapis/java-spanner/blob/main/google-cloud-spanner/src/main/java/com/google/cloud/spanner/SpannerOptions.java#L1143-L1153]
> By Contrast in Beam, SpannerConfig by default has a host set ([see here|[https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerConfig.java#L83]). When Beam goes to convert SpannerConfig to SpannerOptions in SpannerAccessor it will always call setHost on SpannerOptions which will set the emulatorHost to null, [see here|https://github.com/apache/beam/blob/47ab260aaad6b1014d86dd6cf9274f1045690631/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerAccessor.java#L173-L176].
> The consequence of this is the SpannerAccessor will always ignore the SPANNER_EMULATOR_HOST environment variable unless withEmulatorHost is called explicitly on SpannerConfig. Ultimately this pushes the responsibility onto the user to manually check for the environment variable and conditionally set it rather than relying on the functionality that already exists in the underlying SpannerOptions provided by the spanner sdk.
> I think the solution would be for Beam to also check to see if SPANNER_EMULATOR_HOST is set and then fall back to the provided host or the DEFAULT_HOST in the event that we are not connecting to the emulator.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)