You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "David Handermann (Jira)" <ji...@apache.org> on 2022/03/17 01:35:00 UTC
[jira] [Updated] (NIFI-6920) InvokeHttp convertAttributesFromHeaders() method creates attributes with empty string keys, leading to failure on validation
[ https://issues.apache.org/jira/browse/NIFI-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Handermann updated NIFI-6920:
-----------------------------------
Fix Version/s: 1.14.0
Assignee: David Handermann
Resolution: Fixed
Status: Resolved (was: Patch Available)
Updates to InvokeHTTP the following Pull Request for NIFI-8304 improved the behavior of attribute to header conversion:
https://github.com/apache/nifi/pull/4892
> InvokeHttp convertAttributesFromHeaders() method creates attributes with empty string keys, leading to failure on validation
> ----------------------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-6920
> URL: https://issues.apache.org/jira/browse/NIFI-6920
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.9.2, 1.12.0
> Reporter: Andrew Greenburg
> Assignee: David Handermann
> Priority: Major
> Fix For: 1.14.0
>
> Attachments: NIFI-6920.patch
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> When the InvokeHttp processor receives a response, it automatically adds all of the HTTP response headers to the response flowfile as attributes.
> See line 854 of [https://github.com/apache/nifi/blob/rel/nifi-1.9.2/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/InvokeHTTP.java] :
> {code:java}
> // write the response headers as attributes
> // this will overwrite any existing flowfile attributes
> responseFlowFile = session.putAllAttributes(responseFlowFile, convertAttributesFromHeaders(url, responseHttp));
> {code}
>
> In some cases, the Response contains headers where the key is an empty string, but the convertAttributesFromHeaders() method only has a safety check for keys that are null:
> {code:Java}
> private Map<String, String> convertAttributesFromHeaders(URL url, Response responseHttp){
> // create a new hashmap to store the values from the connection
> Map<String, String> map = new HashMap<>();
> responseHttp.headers().names().forEach( (key) -> {
> if (key == null) {
> return;
> }
> List<String> values = responseHttp.headers().values(key);
> // we ignore any headers with no actual values (rare)
> if (values == null || values.isEmpty()) {
> return;
> }
> // create a comma separated string from the values, this is stored in the map
> String value = csv(values);
> // put the csv into the map
> map.put(key, value);
> });
> if (responseHttp.request().isHttps()) {
> Principal principal = responseHttp.handshake().peerPrincipal();
> if (principal != null) {
> map.put(REMOTE_DN, principal.getName());
> }
> }
> return map;
> }
> {code}
> This causes the following error in Nifi, which routes the flowfile to failure:
> {code}
> 2019-12-02 19:18:36,416 ERROR [Timer-Driven Process Thread-2] o.a.nifi.processors.standard.InvokeHTTP InvokeHTTP[id=e127afdd-12b8-38cc-a2d3-af051d3965cb] Routing to Failure due to exception: java.lang.IllegalArgumentException: Invalid attribute key: <Empty String>: java.lang.IllegalArgumentException: Invalid attribute key: <Empty String>
> java.lang.IllegalArgumentException: Invalid attribute key: <Empty String>
> at org.apache.nifi.flowfile.FlowFile$KeyValidator.validateKey(FlowFile.java:120)
> at org.apache.nifi.controller.repository.StandardFlowFileRecord$Builder.addAttributes(StandardFlowFileRecord.java:238)
> at org.apache.nifi.controller.repository.StandardProcessSession.putAllAttributes(StandardProcessSession.java:1796)
> at org.apache.nifi.processors.standard.InvokeHTTP.onTrigger(InvokeHTTP.java:854)
> at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162)
> at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:209)
> at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> Here is an HTTP response that I can consistently reproduce this issue with:
> {code}
> HTTP/1.1 200 OK
> Server: openresty/1.15.8.1
> Date: Wed, 27 Nov 2019 15:07:20 GMT
> Content-Type: application/json;charset=UTF-8
> Content-Length: 190
> Connection: keep-alive
> Cache-Control: no-cache, no-store, max-age=0, must-revalidate
> Pragma: no-cache
> Expires: 0
> X-Content-Type-Options: nosniff
> X-Frame-Options: DENY
> X-XSS-Protection: 1 ; mode=block
> Referrer-Policy: no-referrer
> set-cookie: access_token=vmpxF4JrHGOeRvoSB; Max-Age=1209600; Expires=Wed, 11 Dec 2019 15:07:20 GMT; Path=/
> X-Frame-Options: SAMEORIGIN
> X-Content-Type-Options: nosniff
> X-XSS-Protection: 1; mode=block
> Content-Security-Policy: script-src https://10.0.0.196:* 'unsafe-inline' 'unsafe-eval' blob: ;
> img-src https://10.0.0.196:* data: ;
> frame-src https://10.0.0.196:* 'unsafe-inline' 'unsafe-eval' blob: ;
> style-src https://10.0.0.196:* 'unsafe-inline' ;
> font-src https://10.0.0.196:* data: ;
> default-src https://10.0.0.196:* ;
> object-src 'none'
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)