About Memory usage

[Davood Rafiei <ra...@gmail.com>]

Hi community,


I am using   WordCount example in DataStream API. I want to know the memory
usage of each task and operator and possible modify them.

As far as I know, the memory(off-heap or heap) is retrieved through
MemoryManager.
There is, releaseAll(Object owner) method in MemoryManager, that is called
from Task class when releasing the resources. However, I could not find how
the operator gets the required memory (heap or off-heap) from
MemoryManager.


Cheers
Davood

Re: About Memory usage

[Maximilian Michels <mx...@apache.org>]

Hi Davood,

The streaming part of Flink currently does not support managed memory;
all memory is allocated directly on the heap (apart from the network
buffers). There are plans to change that in the future.

The batch side uses manages memory, e.g. see ReduceCombineDriver.

Cheers,
Max


On Sat, Jun 11, 2016 at 1:11 PM, Davood Rafiei <ra...@gmail.com> wrote:
> Hi community,
>
>
> I am using   WordCount example in DataStream API. I want to know the memory
> usage of each task and operator and possible modify them.
>
> As far as I know, the memory(off-heap or heap) is retrieved through
> MemoryManager.
> There is, releaseAll(Object owner) method in MemoryManager, that is called
> from Task class when releasing the resources. However, I could not find how
> the operator gets the required memory (heap or off-heap) from
> MemoryManager.
>
>
> Cheers
> Davood