You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Pablo Lopez <pa...@integrado.com.au> on 2016/12/12 23:19:15 UTC
Pull big size HTTP responses.
Hi,
We have a requirement to send HTTPs requests to an Application Server which
will give use back a very big size as a response back (~200 MB) which
include HTTP attachments. Basically to download big files.
My question is, would NiFi try to put the whole content of the response in
memory before it is processed, or is there a possibility to write this
content to disk while it gets downloaded?
Thanks,
Pablo.
Re: Pull big size HTTP responses.
Posted by Pablo Lopez <pa...@integrado.com.au>.
Hi Jeremy,
Thanks for the info. I'll try this then.
Pablo.
On Tue, Dec 13, 2016 at 11:29 AM, Jeremy Farbota <jf...@payoff.com>
wrote:
> Pablo,
>
> IIRC, Joe Witt commented on this topic a while back. The data is never
> transferred the way you describe unless you configure it to do so very
> intentionally. The data is streamed into partitions.
>
> On Mon, Dec 12, 2016 at 3:19 PM, Pablo Lopez <pablo.lopez@integrado.com.au
> > wrote:
>
>> Hi,
>>
>> We have a requirement to send HTTPs requests to an Application Server
>> which will give use back a very big size as a response back (~200 MB) which
>> include HTTP attachments. Basically to download big files.
>>
>> My question is, would NiFi try to put the whole content of the response
>> in memory before it is processed, or is there a possibility to write this
>> content to disk while it gets downloaded?
>>
>> Thanks,
>> Pablo.
>>
>
>
>
> --
>
> [image: Payoff, Inc.] <http://www.payoff.com/>
>
> Jeremy Farbota
> Software Engineer, Data
> jfarbota@payoff.com <em...@payoff.com> • (217) 898-8110 <(949)+430-0630>
>
> I'm a Storyteller. Discover your Financial Personality!
> <https://www.payoff.com/quiz>
>
> [image: Facebook] <https://www.facebook.com/payoff> [image: Twitter]
> <https://www.twitter.com/payoff> [image: Linkedin]
> <https://www.linkedin.com/company/payoff-com>
>
--
Pablo Lopez.
Integration Architect
Integrado Pty Ltd
M: 044 84 52 479
pablo.lopez@integrado.com.au
Re: Pull big size HTTP responses.
Posted by Jeremy Farbota <jf...@payoff.com>.
Pablo,
IIRC, Joe Witt commented on this topic a while back. The data is never
transferred the way you describe unless you configure it to do so very
intentionally. The data is streamed into partitions.
On Mon, Dec 12, 2016 at 3:19 PM, Pablo Lopez <pa...@integrado.com.au>
wrote:
> Hi,
>
> We have a requirement to send HTTPs requests to an Application Server
> which will give use back a very big size as a response back (~200 MB) which
> include HTTP attachments. Basically to download big files.
>
> My question is, would NiFi try to put the whole content of the response in
> memory before it is processed, or is there a possibility to write this
> content to disk while it gets downloaded?
>
> Thanks,
> Pablo.
>
--
[image: Payoff, Inc.] <http://www.payoff.com/>
Jeremy Farbota
Software Engineer, Data
jfarbota@payoff.com <em...@payoff.com> • (217) 898-8110 <(949)+430-0630>
I'm a Storyteller. Discover your Financial Personality!
<https://www.payoff.com/quiz>
[image: Facebook] <https://www.facebook.com/payoff> [image: Twitter]
<https://www.twitter.com/payoff> [image: Linkedin]
<https://www.linkedin.com/company/payoff-com>