ElasticsearchIO delete document

[Arif Alili <a....@propellor.eu>]

Hi all,

I am writing to Elasticsearch using Beam (Google Dataflow) class
ElasticsearchIO. Creating indexes and writing document goes well, however,
I am struggling to find a way to delete Elasticsearch documents, looking at
this documentation
https://beam.apache.org/releases/javadoc/2.5.0/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIO.html,
I see there's only read() and write() methods.

Anyone has any suggestions how to delete Elasticsearch documents using
Beam's ElasticsearchIO?

Best,
Arif

Re: ElasticsearchIO delete document

[Arif Alili <a....@propellor.eu>]

Hi  Jithin,

Updating Beam to 2.25.0 and using "withIsDeleteFn" works as expected.
Thanks a lot for the detailed answer!

Best,
Arif

On Tue, Dec 1, 2020 at 11:30 AM JITHIN SUKUMAR <ji...@gmail.com>
wrote:

> Hi Arif Alili,
>
> Deleting documents using ElasticsearchIO [1] was included in Apache Beam
> since version 2.25.0 [2]. You can check out the javadoc [3] or some example
> implementations [4].
>
> References:
> [1]: https://issues.apache.org/jira/browse/BEAM-5757
> [2]: https://beam.apache.org/blog/beam-2.25.0/
> [3]:
> https://beam.apache.org/releases/javadoc/2.25.0/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIO.Write.html#withIsDeleteFn-org.apache.beam.sdk.io.elasticsearch.ElasticsearchIO.Write.BooleanFieldValueExtractFn-
> [4]:
> https://github.com/apache/beam/blob/release-2.25.0/sdks/java/io/elasticsearch-tests/elasticsearch-tests-common/src/test/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIOTestCommon.java#L667-L749
>
> Hope that helps!
> Regards,
> Jithin
>
> On Tue, Dec 1, 2020 at 6:18 PM Arif Alili <a....@propellor.eu> wrote:
>
>> Hi all,
>>
>> I am writing to Elasticsearch using Beam (Google Dataflow) class
>> ElasticsearchIO. Creating indexes and writing document goes well, however,
>> I am struggling to find a way to delete Elasticsearch documents, looking at
>> this documentation
>> https://beam.apache.org/releases/javadoc/2.5.0/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIO.html,
>> I see there's only read() and write() methods.
>>
>> Anyone has any suggestions how to delete Elasticsearch documents using
>> Beam's ElasticsearchIO?
>>
>> Best,
>> Arif
>>
>

-- 
*Arif Alili*

A:

T:
E:
I:


Pilotenstraat 43 bg
1059 CH Amsterdam
020 - 6 71 71 71 <+31+20+6+71+71+71>
a.alili@propellor.eu <[n...@propellor.eu>
www.propellor.eu


<https://www.facebook.com/PropellorEU/>
<https://www.linkedin.com/company/10870471>
<https://twitter.com/PropellorEU/>

Re: ElasticsearchIO delete document

[JITHIN SUKUMAR <ji...@gmail.com>]

Hi Arif Alili,

Deleting documents using ElasticsearchIO [1] was included in Apache Beam
since version 2.25.0 [2]. You can check out the javadoc [3] or some example
implementations [4].

References:
[1]: https://issues.apache.org/jira/browse/BEAM-5757
[2]: https://beam.apache.org/blog/beam-2.25.0/
[3]:
https://beam.apache.org/releases/javadoc/2.25.0/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIO.Write.html#withIsDeleteFn-org.apache.beam.sdk.io.elasticsearch.ElasticsearchIO.Write.BooleanFieldValueExtractFn-
[4]:
https://github.com/apache/beam/blob/release-2.25.0/sdks/java/io/elasticsearch-tests/elasticsearch-tests-common/src/test/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIOTestCommon.java#L667-L749

Hope that helps!
Regards,
Jithin

On Tue, Dec 1, 2020 at 6:18 PM Arif Alili <a....@propellor.eu> wrote:

> Hi all,
>
> I am writing to Elasticsearch using Beam (Google Dataflow) class
> ElasticsearchIO. Creating indexes and writing document goes well, however,
> I am struggling to find a way to delete Elasticsearch documents, looking at
> this documentation
> https://beam.apache.org/releases/javadoc/2.5.0/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIO.html,
> I see there's only read() and write() methods.
>
> Anyone has any suggestions how to delete Elasticsearch documents using
> Beam's ElasticsearchIO?
>
> Best,
> Arif
>