You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Chris Sampson (Jira)" <ji...@apache.org> on 2024/09/05 20:25:00 UTC

[jira] [Created] (NIFI-13719) PutElasticsearch* processors should handle Long "took" field values in _bulk API responses

Chris Sampson created NIFI-13719:
------------------------------------

             Summary: PutElasticsearch* processors should handle Long "took" field values in _bulk API responses
                 Key: NIFI-13719
                 URL: https://issues.apache.org/jira/browse/NIFI-13719
             Project: Apache NiFi
          Issue Type: Bug
    Affects Versions: 2.0.0-M4, 1.27.0
            Reporter: Chris Sampson


The Elasticsearch {{_bulk}} API response contains a {{took}} field, which indicates the number of milliseconds taken by Elasticsearch to perform the requested operation.

This value is [expected to be an integer|https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html#bulk-api-response-body].

In Elasticsearch 8.15.0, there [is a bug|https://github.com/elastic/elasticsearch/issues/111854] that causes the {{took}} field to be a very large value (typically resulting in a Long rather than an Integer when the response JSON is parsed).

This results in NiFi failing the parse the {{IndexOperationResponse}} with a {{ClassCastException}}:

{code:java}
2024-09-05 10:42:48,126 ERROR [Timer-Driven Process Thread-8] o.a.n.p.e.PutElasticsearchRecord PutElasticsearchRecord[id=17e51d66-18c9-1257-06b1-700d2a77894e] Encountered a server-side problem with Elasticsearch. Routing to failure
org.apache.nifi.elasticsearch.ElasticsearchException: java.lang.ClassCastException: class java.lang.Long cannot be cast to class java.lang.Integer (java.lang.Long and java.lang.Integer are in module java.base of loader 'bootstrap')
   at org.apache.nifi.elasticsearch.ElasticSearchClientServiceImpl.bulk(ElasticSearchClientServiceImpl.java:680)
   at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
   at java.base/java.lang.reflect.Method.invoke(Method.java:580)
   at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254)
   at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:105)
   at jdk.proxy80/jdk.proxy80.$Proxy324.bulk(Unknown Source)
   at org.apache.nifi.processors.elasticsearch.PutElasticsearchRecord.indexDocuments(PutElasticsearchRecord.java:536)
   at org.apache.nifi.processors.elasticsearch.PutElasticsearchRecord.operate(PutElasticsearchRecord.java:516)
   at org.apache.nifi.processors.elasticsearch.PutElasticsearchRecord.onTrigger(PutElasticsearchRecord.java:431)
   at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
   at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1274)
   at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:244)
   at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:102)
   at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
   at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
   at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358)
   at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
   at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
   at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
   at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.lang.ClassCastException: class java.lang.Long cannot be cast to class java.lang.Integer (java.lang.Long and java.lang.Integer are in module java.base of loader 'bootstrap')
   at org.apache.nifi.elasticsearch.IndexOperationResponse.fromJsonResponse(IndexOperationResponse.java:49)
   at org.apache.nifi.elasticsearch.ElasticSearchClientServiceImpl.bulk(ElasticSearchClientServiceImpl.java:678)
   ... 19 common frames omitted
{code}

While the Elasticsearch {{took}} field **should** be an Integer, NiFi should allow for this to be a Long (although the Elasticsearch issue has been fixed as of [8.15.1|https://github.com/elastic/elasticsearch/pull/111863]).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)