Completed 2nd long run of Fluo

[Keith Turner <ke...@deenlo.com>]

A 2nd multi-day test of Fluo was completed on EC2 a week or two ago.  I
just got around to finishing writing it up. The test was run using Accumulo
1.7.1.

http://fluo.io/blog/2016/05/17/webindex-long-run-2

Re: Completed 2nd long run of Fluo

[Keith Turner <ke...@deenlo.com>]

On Fri, Jun 3, 2016 at 6:34 PM, Josh Elser <jo...@gmail.com> wrote:

> Great stuff, Keith! Looking forward to picking it apart in more detail:
>
> bq. In the previous run Fluo worker processes were constantly being killed
> by YARN for exceeding memory limits
>
> I remember stumbling across some (surprising) recommendations which were
> to just completely disable the vmem/pmem monitoring to avoid this. Nice
> that you were actually able to work around it "properly".


I am not quite sure why the workers processes are going over the heap usage
by so much, because no native code should be in use.   One possibility is
thread stacks.   The -Xmx jvm option only controls heap usage.  If there
are lots of threads where the stack has been deep in the past, then those
threads may continue to use a good bit of memory.


>
>
> Keith Turner wrote:
>
>> A 2nd multi-day test of Fluo was completed on EC2 a week or two ago.  I
>> just got around to finishing writing it up. The test was run using
>> Accumulo 1.7.1.
>>
>> http://fluo.io/blog/2016/05/17/webindex-long-run-2
>>
>

Re: Completed 2nd long run of Fluo

[Josh Elser <jo...@gmail.com>]

Great stuff, Keith! Looking forward to picking it apart in more detail:

bq. In the previous run Fluo worker processes were constantly being 
killed by YARN for exceeding memory limits

I remember stumbling across some (surprising) recommendations which were 
to just completely disable the vmem/pmem monitoring to avoid this. Nice 
that you were actually able to work around it "properly".

Keith Turner wrote:
> A 2nd multi-day test of Fluo was completed on EC2 a week or two ago.  I
> just got around to finishing writing it up. The test was run using
> Accumulo 1.7.1.
>
> http://fluo.io/blog/2016/05/17/webindex-long-run-2