You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Lukasz Gajowy (Jira)" <ji...@apache.org> on 2019/09/17 10:44:00 UTC
[jira] [Updated] (BEAM-8256) Set fixed number of workers for
File-based IOITs
[ https://issues.apache.org/jira/browse/BEAM-8256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lukasz Gajowy updated BEAM-8256:
--------------------------------
Description:
Autoscaling is a feature of google cloud dataflow runner that adds/removes worker nodes dynamically as the job runs. It can behave in a different way creating different test (runtime) results in consequent runs. In integration tests (such as IOIT but others also apply) we don't need such nondeterminism and it's best to have a fixed number of workers for every test execution.
IOITs use autoscaling but they shouldn't. This issue was created to disable it and set a fixed number of workers.
Side note: autoscaling is already disabled in Nexmark and load tests of core operations.
> Set fixed number of workers for File-based IOITs
> ------------------------------------------------
>
> Key: BEAM-8256
> URL: https://issues.apache.org/jira/browse/BEAM-8256
> Project: Beam
> Issue Type: Bug
> Components: testing
> Reporter: Michal Walenia
> Assignee: Michal Walenia
> Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Autoscaling is a feature of google cloud dataflow runner that adds/removes worker nodes dynamically as the job runs. It can behave in a different way creating different test (runtime) results in consequent runs. In integration tests (such as IOIT but others also apply) we don't need such nondeterminism and it's best to have a fixed number of workers for every test execution.
> IOITs use autoscaling but they shouldn't. This issue was created to disable it and set a fixed number of workers.
> Side note: autoscaling is already disabled in Nexmark and load tests of core operations.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)