what is the PutElasticsearch Identifier Attribute for?

[jose antonio rodriguez diaz <jo...@gmail.com>]

Hello guys,

I trying to define a data flow to export data for MySQL to Elasticsearch. For this I’m using this processors

ExecuteSQL -> ConvertAvroToJSON -> PutElasticsearch 

But I’m getting errors with the last one. So what is the identifier Attribute of the PutElasticsearch processor for? I just want to insert the data into ES and this one just generate an Id for every record.

Any example, suggestion, or help would be welcome.

Thanks in advance.

Jose Antonio

Re: what is the PutElasticsearch Identifier Attribute for?

[Matt Burgess <ma...@gmail.com>]

The Identifier Attribute property should contain the name of a Flow File
attribute, which in turn contains the ID of the document to be put into
Elasticsearch. Unfortunately it is a required property so having ES
auto-generate it is not yet supported [1].

If you don't care what the ID is but need it to be unique, try using "uuid"
(notice no Expression Language surrounding it) into the Identifier
Attribute property. This will use the UUID of the Flow File as the document
ID in Elasticsearch. I'm not sure if ES lets you have String IDs without a
mapping, so if that doesn't work you can try an UpdateAttribute processor
before the PutES, add an attribute like "doc.id" with value ${nextInt()},
then use "doc.id" as the Identifier Attribute.

Regards,
Matt

[1] https://issues.apache.org/jira/browse/NIFI-1576

On Wed, Mar 23, 2016 at 9:59 AM, jose antonio rodriguez diaz <
josearodriguezd@gmail.com> wrote:

> Hello guys,
>
> I trying to define a data flow to export data for MySQL to Elasticsearch.
> For this I’m using this processors
>
> ExecuteSQL -> ConvertAvroToJSON -> PutElasticsearch
>
> But I’m getting errors with the last one. So what is the identifier
> Attribute of the PutElasticsearch processor for? I just want to insert the
> data into ES and this one just generate an Id for every record.
>
> Any example, suggestion, or help would be welcome.
>
> Thanks in advance.
>
> Jose Antonio
>
>
>