You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/03 23:03:28 UTC
[GitHub] [beam] kennknowles opened a new issue, #19250: Support SDF within ElasticsearchIO
kennknowles opened a new issue, #19250:
URL: https://github.com/apache/beam/issues/19250
Similar to HbaseIO.ReadAll, it would be very useful to support a SDF based transform which can compose queries based off of input elements.
Implementation would split the work similarly to how the current Source API implementation works (leveraging slices within Elasticsearch).
This work can facilitate a transition of ElasticsearchIO to SDF when more runners have implemented support. Until then, I propose leaving the read() implementation alone and add a readAll method and associated transform to be implemented as an SDF. Once enough runner support is ready, the existing Read transform would be updated to use the same SDF as the ReadAll transform with a null input element.
A prototype implementation currently exists (albeit developed in a bubble) at [https://github.com/0xdecaf/beam/tree/feature/elasticsearchio.readall](https://github.com/0xdecaf/beam/tree/feature/elasticsearchio.readall)
Imported from Jira [BEAM-6654](https://issues.apache.org/jira/browse/BEAM-6654). Original Jira may contain additional context.
Reported by: tmoulton.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kennknowles commented on issue #19250: Support SDF within ElasticsearchIO
Posted by GitBox <gi...@apache.org>.
kennknowles commented on issue #19250:
URL: https://github.com/apache/beam/issues/19250#issuecomment-1332516242
Do you feel like you would like to attempt to build this functionality? The linked branch has not technically been contributed to Beam so if you want to base your work on it, you should ask the author. Though, I suppose it is posted publicly on GitHub with Apache license, so...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] bernardo-bruning commented on issue #19250: Support SDF within ElasticsearchIO
Posted by GitBox <gi...@apache.org>.
bernardo-bruning commented on issue #19250:
URL: https://github.com/apache/beam/issues/19250#issuecomment-1332270579
Any updates here? We need this feature.
How can I help @kennknowles?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org