[jira] [Commented] (BEAM-6920) Expand aliases in ElasticsearchIO.Read

["Kenneth Knowles (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17547714#comment-17547714 ] 

Kenneth Knowles commented on BEAM-6920:
---------------------------------------

This issue has been migrated to https://github.com/apache/beam/issues/19441

> Expand aliases in ElasticsearchIO.Read
> --------------------------------------
>
>                 Key: BEAM-6920
>                 URL: https://issues.apache.org/jira/browse/BEAM-6920
>             Project: Beam
>          Issue Type: Improvement
>          Components: io-java-elasticsearch
>    Affects Versions: 2.11.0
>            Reporter: Egbert
>            Priority: P3
>
> I am building a pipeline that needs to process all records in a set of indexes, each suffixed with a timestamp. I have an alias that matches all of these indexes at once. However, I cannot use the alias name in ElasticsearchIO as it will try to read stats from this specific index. Because it is an alias and not an actual index, the response contains no count for the alias name itself and therefore Beam (Dataflow?) will estimate the size as being 0. This makes the pipeline end without even executing the query on the alias, even though that would have returned loads of documents.
> This should be easy to fix as the results of /<aliasname>/_stats only contains indexes references by that alias, so instead of looking for a key <aliasname> in the `indices` key in the returned JSON, it should consider all returned indexes and add the estimated sizes together.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)