[jira] [Commented] (BEAM-14194) [SpannerIO.readChangeStream] Throw error when autoscaling algorithm is not NONE

["Thiago Nunes (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-14194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17513814#comment-17513814 ] 

Thiago Nunes commented on BEAM-14194:
-------------------------------------

This would require a cherry pick for 2.38.0 in order to prevent wrong autoscaling behaviour if using the SpannerIO.readChangeStreams() functionality.

Currently, the implementation is under-reporting the backlog estimation, which would most likely produce scale down events even when there is enough work for the current number of workers.

By adding the check here, we prevent surprising production behavior when using the Connector.

> [SpannerIO.readChangeStream] Throw error when autoscaling algorithm is not NONE
> -------------------------------------------------------------------------------
>
>                 Key: BEAM-14194
>                 URL: https://issues.apache.org/jira/browse/BEAM-14194
>             Project: Beam
>          Issue Type: New Feature
>          Components: io-java-gcp, runner-dataflow
>            Reporter: Thiago Nunes
>            Assignee: Thiago Nunes
>            Priority: P2
>             Fix For: 2.38.0
>
>          Time Spent: 40m
>  Remaining Estimate: 0h
>
> SpannerIO.readChangeStreams does not currently support the autoscaling feature. In order to avoid customer confusion, we decided to error out if an algorithm different than NONE is specified.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)