[jira] [Created] (NIFI-6728) InvokeHttp / PostHttp should yield on HTTP-503 (Service Unavailable)

["Andrew Doumaux (Jira)" <ji...@apache.org>]

Andrew Doumaux created NIFI-6728:
------------------------------------

             Summary: InvokeHttp / PostHttp should yield on HTTP-503 (Service Unavailable)
                 Key: NIFI-6728
                 URL: https://issues.apache.org/jira/browse/NIFI-6728
             Project: Apache NiFi
          Issue Type: Improvement
          Components: Core Framework
    Affects Versions: 1.7.1
            Reporter: Andrew Doumaux


h3. Problem:

InvokeHttp/PostHttp only penalize a flowfile when a the remote host returns an error code.  It has no way to yield the processor when the remote host indicates it's unavailable.  

 
h3. Desired Behavior:

A configurable property to have the processor yield on server error codes (5xx).  Specifically HTTP-503.

Maybe user exceptions as well (4xx), specifically HTTP-401 & HTTP-403 .

 
h3. Use Case:

I'm using NIFI to load balance data to multiple horizontally scalable ETL processes (Standalone java spring boot apps).  The ETL application returns a HTTP-503 when it is saturated with work and have no available threads to process a new piece of data.  I have a separate PostHttp processor for each ETL application instance/host.  It was done this way since I had the misconception that NIFI would yield that specific processor and thus pause sending data to that ETL application as it worked off it's current load.  

This behavior of penalizing the file and not yielding the processor is especially problematic when attempting to process an excessive backlog of data.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)