You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by "Whitmore, Mattie [USA]" <Wh...@bah.com> on 2015/09/04 17:24:22 UTC
Re: [External] Re: GitHTTP out of memory error
Thanks for all the speedy replies!
1. I will up the heap currently it is set at: java.arg.3=-Xmx512m
2. I have 15 GB of memory (m3.xlarge)
3. I do have a number of custom processors in my workflow
I’ll update ya’ll about how the increase in heap affects this issue. I am also working on serializing these HTTP calls into one processor, do ya’ll have any thoughts on that solution?
Mattie Whitmore
From: Corey Flowers
Reply-To: "users@nifi.apache.org<ma...@nifi.apache.org>"
Date: Friday, September 4, 2015 at 11:05 AM
To: "users@nifi.apache.org<ma...@nifi.apache.org>"
Subject: [External] Re: GitHTTP out of memory error
What is your heap space set to and how much memory do you have on the box?
On Fri, Sep 4, 2015 at 10:57 AM, Whitmore, Mattie [USA] <Wh...@bah.com>> wrote:
Hi,
I am 8 GetHTTP processors which run nightly, however 5 hours after these run some threads out of memory:
ERROR [Timer-Driven Process Thread-23] o.a.nifi.processors.standard.GetHTTP GetHTTP[id=021562be-b123-4b3f-b5ed-2b8c47af6d4c] GetHTTP[id=021562be-b123-4b3f-b5ed-2b8c47af6d4c] failed to process session due to java.lang.OutOfMemoryError: Java heap space: java.lang.OutOfMemoryError: Java heap space
2015-09-03 07:10:41,167 ERROR [Timer-Driven Process Thread-11] o.a.nifi.processors.standard.GetHTTP GetHTTP[id=0a5158d2-f082-4007-b646-3da8fb436fd0] GetHTTP[id=0a5158d2-f082-4007-b646-3da8fb436fd0] failed to process session due to java.lang.OutOfMemoryError: Java heap space: java.lang.OutOfMemoryError: Java heap space
2015-09-03 07:10:41,168 WARN [Timer-Driven Process Thread-23] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding GetHTTP[id=021562be-b123-4b3f-b5ed-2b8c47af6d4c] due to uncaught Exception: java.lang.OutOfMemoryError: Java heap space
2015-09-03 07:10:41,168 WARN [Timer-Driven Process Thread-11] o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding GetHTTP[id=0a5158d2-f082-4007-b646-3da8fb436fd0] due to uncaught Exception: java.lang.OutOfMemoryError: Java heap space
I am running on Amazon Linux and have tuned the server as described in the admin page.
Thanks!
Mattie Whitmore
Lead Technologist
Booz | Allen | Hamilton
________________________
--
Corey Flowers
Vice President, Onyx Point, Inc
(410) 541-6699
cflowers@onyxpoint.com<ma...@onyxpoint.com>
-- This account not approved for unencrypted proprietary information --
Re: [External] Re: GitHTTP out of memory error
Posted by Corey Flowers <cf...@onyxpoint.com>.
Really the setting of that depends on what you are trying to do in the flow
and what else runs on your systems. I have a cluster of 4 boxes that have
12GB of Ram and I set those to 4096. That cluster runs just fine and pushes
a few hundred gigs a day at about 30% system usage. Depending on what your
custom processors are doing with the heap, I would boost that up a bit. You
should also look at ganglia or something in that family to watch for memory
leaks in the custom code. NIFI has a really good ganglia reporting task to
show you what is happening at a system level within the heap.
On Fri, Sep 4, 2015 at 11:24 AM, Whitmore, Mattie [USA] <
Whitmore_Mattie@bah.com> wrote:
> Thanks for all the speedy replies!
>
>
> 1. I will up the heap currently it is set at: java.arg.3=-Xmx512m
> 2. I have 15 GB of memory (m3.xlarge)
> 3. I do have a number of custom processors in my workflow
>
> I’ll update ya’ll about how the increase in heap affects this issue. I
> am also working on serializing these HTTP calls into one processor, do
> ya’ll have any thoughts on that solution?
>
>
> *Mattie Whitmore*
>
>
> From: Corey Flowers
> Reply-To: "users@nifi.apache.org"
> Date: Friday, September 4, 2015 at 11:05 AM
> To: "users@nifi.apache.org"
> Subject: [External] Re: GitHTTP out of memory error
>
> What is your heap space set to and how much memory do you have on the box?
>
>
> On Fri, Sep 4, 2015 at 10:57 AM, Whitmore, Mattie [USA] <
> Whitmore_Mattie@bah.com> wrote:
>
>> Hi,
>>
>> I am 8 GetHTTP processors which run nightly, however 5 hours after these
>> run some threads out of memory:
>>
>> ERROR [Timer-Driven Process Thread-23]
>> o.a.nifi.processors.standard.GetHTTP
>> GetHTTP[id=021562be-b123-4b3f-b5ed-2b8c47af6d4c]
>> GetHTTP[id=021562be-b123-4b3f-b5ed-2b8c47af6d4c] failed to process session
>> due to java.lang.OutOfMemoryError: Java heap space:
>> java.lang.OutOfMemoryError: Java heap space
>>
>> 2015-09-03 07:10:41,167 ERROR [Timer-Driven Process Thread-11]
>> o.a.nifi.processors.standard.GetHTTP
>> GetHTTP[id=0a5158d2-f082-4007-b646-3da8fb436fd0]
>> GetHTTP[id=0a5158d2-f082-4007-b646-3da8fb436fd0] failed to process session
>> due to java.lang.OutOfMemoryError: Java heap space:
>> java.lang.OutOfMemoryError: Java heap space
>>
>> 2015-09-03 07:10:41,168 WARN [Timer-Driven Process Thread-23]
>> o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding
>> GetHTTP[id=021562be-b123-4b3f-b5ed-2b8c47af6d4c] due to uncaught Exception:
>> java.lang.OutOfMemoryError: Java heap space
>>
>> 2015-09-03 07:10:41,168 WARN [Timer-Driven Process Thread-11]
>> o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding
>> GetHTTP[id=0a5158d2-f082-4007-b646-3da8fb436fd0] due to uncaught Exception:
>> java.lang.OutOfMemoryError: Java heap space
>>
>> I am running on Amazon Linux and have tuned the server as described in
>> the admin page.
>>
>> Thanks!
>>
>> *Mattie Whitmore*
>> Lead Technologist
>>
>> Booz | Allen | Hamilton
>> *________________________*
>>
>>
>>
>
>
> --
> Corey Flowers
> Vice President, Onyx Point, Inc
> (410) 541-6699
> cflowers@onyxpoint.com
>
> -- This account not approved for unencrypted proprietary information --
>
--
Corey Flowers
Vice President, Onyx Point, Inc
(410) 541-6699
cflowers@onyxpoint.com
-- This account not approved for unencrypted proprietary information --
RE: [External] Re: GitHTTP out of memory error
Posted by Mark Payne <ma...@hotmail.com>.
Mattie,
Sounds good. If you increase the heap (I'd recommend 2 or 4 GB) and you are still running into OutOfMemoryError,
then it would certainly be hinting at a memory leak of some sort. Please let us know if increasing the heap
size helps!
Thanks
-Mark
________________________________
> From: Whitmore_Mattie@bah.com
> To: users@nifi.apache.org
> Subject: Re: [External] Re: GitHTTP out of memory error
> Date: Fri, 4 Sep 2015 15:24:22 +0000
>
> Thanks for all the speedy replies!
>
>
> 1. I will up the heap currently it is set at: java.arg.3=-Xmx512m
> 2. I have 15 GB of memory (m3.xlarge)
> 3. I do have a number of custom processors in my workflow
>
> I’ll update ya’ll about how the increase in heap affects this issue. I
> am also working on serializing these HTTP calls into one processor, do
> ya’ll have any thoughts on that solution?
>
>
> Mattie Whitmore
>
>
> From: Corey Flowers
> Reply-To: "users@nifi.apache.org<ma...@nifi.apache.org>"
> Date: Friday, September 4, 2015 at 11:05 AM
> To: "users@nifi.apache.org<ma...@nifi.apache.org>"
> Subject: [External] Re: GitHTTP out of memory error
>
> What is your heap space set to and how much memory do you have on the box?
>
>
> On Fri, Sep 4, 2015 at 10:57 AM, Whitmore, Mattie [USA]
> <Wh...@bah.com>> wrote:
> Hi,
>
> I am 8 GetHTTP processors which run nightly, however 5 hours after
> these run some threads out of memory:
>
>
> ERROR [Timer-Driven Process Thread-23]
> o.a.nifi.processors.standard.GetHTTP
> GetHTTP[id=021562be-b123-4b3f-b5ed-2b8c47af6d4c]
> GetHTTP[id=021562be-b123-4b3f-b5ed-2b8c47af6d4c] failed to process
> session due to java.lang.OutOfMemoryError: Java heap space:
> java.lang.OutOfMemoryError: Java heap space
>
> 2015-09-03 07:10:41,167 ERROR [Timer-Driven Process Thread-11]
> o.a.nifi.processors.standard.GetHTTP
> GetHTTP[id=0a5158d2-f082-4007-b646-3da8fb436fd0]
> GetHTTP[id=0a5158d2-f082-4007-b646-3da8fb436fd0] failed to process
> session due to java.lang.OutOfMemoryError: Java heap space:
> java.lang.OutOfMemoryError: Java heap space
>
> 2015-09-03 07:10:41,168 WARN [Timer-Driven Process Thread-23]
> o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding
> GetHTTP[id=021562be-b123-4b3f-b5ed-2b8c47af6d4c] due to uncaught
> Exception: java.lang.OutOfMemoryError: Java heap space
>
> 2015-09-03 07:10:41,168 WARN [Timer-Driven Process Thread-11]
> o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding
> GetHTTP[id=0a5158d2-f082-4007-b646-3da8fb436fd0] due to uncaught
> Exception: java.lang.OutOfMemoryError: Java heap space
>
> I am running on Amazon Linux and have tuned the server as described in
> the admin page.
>
> Thanks!
>
> Mattie Whitmore
> Lead Technologist
>
> Booz | Allen | Hamilton
> ________________________
>
>
>
>
>
> --
> Corey Flowers
> Vice President, Onyx Point, Inc
> (410) 541-6699
> cflowers@onyxpoint.com<ma...@onyxpoint.com>
>
> -- This account not approved for unencrypted proprietary information --