You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Aquilino Viveiros (Jira)" <ji...@apache.org> on 2020/06/16 21:47:00 UTC
[jira] [Updated] (NIFI-7449) ScrollElasticsearchHttp processor
fails against ES 7.6 with error 400 Field expansion matches too many
fields, limit: 1024
[ https://issues.apache.org/jira/browse/NIFI-7449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aquilino Viveiros updated NIFI-7449:
------------------------------------
Status: Patch Available (was: Open)
> ScrollElasticsearchHttp processor fails against ES 7.6 with error 400 Field expansion matches too many fields, limit: 1024
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-7449
> URL: https://issues.apache.org/jira/browse/NIFI-7449
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.9.2, 1.9.0
> Reporter: Aquilino Viveiros
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Using Elasticsearch 7.6 (probably true for any 7.x version) and using the processor to scroll over the data Nifi receives a 400 error from ES with the following content => *Field expansion matches too many fields, limit: 1024*
>
> Precondition: Index documents and mapping must have fields over the limit above.
>
> When initiating the scroll, the Nifi processor (1) uses a GET with a query parameter as follow:
> *[http://localhost:9200/index/_search?q=type:*]*
> Which causes the error above. Removing the q parameter eliminates the error
>
> Few ideas to fix it
> 1) Make query parameter optional (keep GET)
> 2) Refactor existing code to use POST instead of GET.
> 3) Introduce a different property for a Query to be used as a POST (does not trigger the error above)
>
> Tested on
> Elasticsearch 7.6
> Nifi 1.9.0 and 1.9.2
>
> 1) [https://github.com/apache/nifi/blob/9608fe33fa4a3f39ee6ddca54184ccc85ef1a163/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/ScrollElasticsearchHttp.java#L284]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)