You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Chris Sampson (Jira)" <ji...@apache.org> on 2023/04/24 21:47:00 UTC
[jira] [Commented] (NIFI-10948) QueryElasticsearchHTTP doesnt work on Opensearch (and possibly newer Elasticsearch versions)
[ https://issues.apache.org/jira/browse/NIFI-10948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715997#comment-17715997 ]
Chris Sampson commented on NIFI-10948:
--------------------------------------
Note that the QueryElasticsearchHttp processor is deprecated in recent versions of NiFi 1.x and removed from the upcoming 2.0.0 release.
Suggested alternatives are [JsonQueryElasticsearch|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-restapi-nar/1.21.0/org.apache.nifi.processors.elasticsearch.JsonQueryElasticsearch/index.html] or [PaginatedJsonQueryElasticsearch|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-restapi-nar/1.21.0/org.apache.nifi.processors.elasticsearch.PaginatedJsonQueryElasticsearch/index.html]. These use the [ElasticSearchClientService|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-elasticsearch-client-service-nar/1.21.0/org.apache.nifi.elasticsearch.ElasticSearchClientServiceImpl/index.html] for creating connections using the Elastic low-level rest api client library under the hood.
They haven't been properly tested against OpenSearch, but you might find better compatibility with these more recent versions of the Elasticsearch processors.
> QueryElasticsearchHTTP doesnt work on Opensearch (and possibly newer Elasticsearch versions)
> --------------------------------------------------------------------------------------------
>
> Key: NIFI-10948
> URL: https://issues.apache.org/jira/browse/NIFI-10948
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Affects Versions: 1.15.3
> Environment: Nifi 1.15.3 on centos machine
> Reporter: David Vassallo
> Priority: Minor
> Attachments: image-2022-12-05-16-34-20-513.png
>
>
> The QueryElasticsearchHTTP processor will return a "Null Pointer Exception" when running agasint Opensearch and probably newer versions of Elasticsearch. I tracked down the null pointer to this line:
>
> [https://github.com/apache/nifi/blob/0ebc6d31489e975dcbbe078fa572332ef8ffa9e2/nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/QueryElasticsearchHttp.java#L418]
>
> The processor is looking for the "_type" field which is no longer in the right place. In opensearch v 1.x and above (possibly Elasticsearch too, but I dont have the means to test this), the field has moved to be a child of the "@metadata" field:
> !image-2022-12-05-16-34-20-513.png!
>
> If you try to circumvent this issue by populating the "Type" property of the processor, you will get an HTTP 400 Bad Message response since the processor builds a URL with the "size" and "from" arguments as part of the URL which are not supported by Opensearch/Elasticsearch in more recent versions, so you are forced to leave this property empty but then run into the null problem described above
--
This message was sent by Atlassian Jira
(v8.20.10#820010)