You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Kenneth Knowles (Jira)" <ji...@apache.org> on 2021/11/08 19:13:00 UTC

[jira] [Commented] (BEAM-13137) make ElasticsearchIO$BoundedElasticsearchSource#getEstimatedSizeBytes deterministic

    [ https://issues.apache.org/jira/browse/BEAM-13137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17440688#comment-17440688 ] 

Kenneth Knowles commented on BEAM-13137:
----------------------------------------

I suggest that we should be able to make the tests non-flaky by testing only current behavior.

> make ElasticsearchIO$BoundedElasticsearchSource#getEstimatedSizeBytes deterministic
> -----------------------------------------------------------------------------------
>
>                 Key: BEAM-13137
>                 URL: https://issues.apache.org/jira/browse/BEAM-13137
>             Project: Beam
>          Issue Type: Improvement
>          Components: io-java-elasticsearch
>            Reporter: Etienne Chauchot
>            Assignee: Evan Galpin
>            Priority: P2
>
> Index size estimation is statistical in ES and varies. But it is the base for splitting so it needs to be more deterministic because that causes flakiness in the UTests in _testSplit_ and _testSizes_ and maybe entails sub-optimal splitting in production in some cases.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)