[jira] [Commented] (BEAM-3045) Can't find autoscaling algorithm for streaming workflows

["Thomas Pilewicz (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-3045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16696600#comment-16696600 ] 

Thomas Pilewicz commented on BEAM-3045:
---------------------------------------

Does this mean that implementation of worker autoscaling for streaming jobs written in python is on Google's side ?

> Can't find autoscaling algorithm for streaming workflows
> --------------------------------------------------------
>
>                 Key: BEAM-3045
>                 URL: https://issues.apache.org/jira/browse/BEAM-3045
>             Project: Beam
>          Issue Type: Improvement
>          Components: runner-dataflow
>            Reporter: Zhuo Liu
>            Assignee: Thomas Groh
>            Priority: Minor
>             Fix For: Not applicable
>
>
> Hi I see there is an auto-scaling algorithm for google dataflow.
> https://cloud.google.com/dataflow/service/dataflow-service-desc
> Also enum AutoscalingAlgorithmType{NONE, BASIC, THROUGHPUT_BASED} could be found in org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions, however, I was not able to find the actual algorithm implementation for the THROUGHPUT_BASED scaling algorithm.
> Anyone could help?
> I was especially interested in the autoscaling algorithm for streaming workflows.
> Thanks a lot!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)