Elastic search processor in NiFi

[Ravisankar Mani <rr...@gmail.com>]

Hi All,


Currently i tried to access the elastic search data using the
"Fetchelasticserach" processor. In this case i can able to access the data
based specific document id.(single row retrieved) But we need to fetch the
entire data from ES for the specific index and type. Is it any way to do
this process?


Regards,
Ravi

Re: Elastic search processor in NiFi

[Matt Burgess <ma...@gmail.com>]

Ravi,

You can use InvokeHttp along with the Elasticsearch Search API [1] to
get all the documents for a specific index and type. If you are
including the _source in your query then I think you get the contents
back too.  Otherwise if you are getting just the IDs back, then you
can use SplitJSON to get each result as a separate flow file (with a
JSON Path of something like "$.hits.hits[*]._id"), followed by
ExtractText to set an attribute (like "es.doc.id") for each flow file.
Then you can use FetchElasticsearch (with the Document Identifier
property set to "${es.doc.id}") to get the contents of each document.

[1] https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html

Regards,
Matt

On Tue, May 10, 2016 at 8:43 AM, Ravisankar Mani <rr...@gmail.com> wrote:
> Hi All,
>
>
> Currently i tried to access the elastic search data using the
> "Fetchelasticserach" processor. In this case i can able to access the data
> based specific document id.(single row retrieved) But we need to fetch the
> entire data from ES for the specific index and type. Is it any way to do
> this process?
>
>
> Regards,
> Ravi