You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@nemo.apache.org by Haeyoon Cho <ch...@gmail.com> on 2019/08/07 06:51:24 UTC

GSoC 2019: Updates on the off-heap implementation

Hi all,

I am glad to tell you that the off-heap memory management is merged (PR link
<https://github.com/apache/incubator-nemo/pull/223>).
Nemo now reuses the off-heap memory as much as possible to minimize the
malloc() function call by the support of centralized manager,
MemoryPoolAssigner. Also, users can set the maximum amount of off-heap
memory to use. So that if the job tries to allocate more than this limit,
exception will be thrown.

As the last step of the GSoC 2019, I am planning to implement JVM and
off-heap region separation, as described in the JIRA issue
<https://issues.apache.org/jira/browse/NEMO-397>. This will let much stable
support for off-heap memory usages.

If you have any suggestions or questions regarding this project, please let
me know.
Thank you very much for your help and support.

Best regards,
Haeyoon Cho

Re: GSoC 2019: Updates on the off-heap implementation

Posted by Gyewon Lee <st...@gmail.com>.

Hello, Haeyoon.

I'm very excited to see the PR is merged :) Thanks a lot for your hard
working and I believe that this feature should be a great benefit for the
Nemoers. I will look forward to your future contributions, including
memory-region separation :).

Best,
Gyewon

2019년 8월 7일 (수) 오후 4:44, Byung-Gon Chun <bg...@gmail.com>님이 작성:

> Thanks for the update!
> Great progress!
>
> - Gon
>
> Sent from my iPhone
>
> 2019. 8. 7. 오후 2:51, Haeyoon Cho <ch...@gmail.com> 작성:
>
> > Hi all,
> >
> > I am glad to tell you that the off-heap memory management is merged (PR
> link
> > <https://github.com/apache/incubator-nemo/pull/223>).
> > Nemo now reuses the off-heap memory as much as possible to minimize the
> > malloc() function call by the support of centralized manager,
> > MemoryPoolAssigner. Also, users can set the maximum amount of off-heap
> > memory to use. So that if the job tries to allocate more than this limit,
> > exception will be thrown.
> >
> > As the last step of the GSoC 2019, I am planning to implement JVM and
> > off-heap region separation, as described in the JIRA issue
> > <https://issues.apache.org/jira/browse/NEMO-397>. This will let much
> stable
> > support for off-heap memory usages.
> >
> > If you have any suggestions or questions regarding this project, please
> let
> > me know.
> > Thank you very much for your help and support.
> >
> > Best regards,
> > Haeyoon Cho
>

Re: GSoC 2019: Updates on the off-heap implementation

Posted by Byung-Gon Chun <bg...@gmail.com>.

Thanks for the update!
Great progress!

- Gon

Sent from my iPhone

2019. 8. 7. 오후 2:51, Haeyoon Cho <ch...@gmail.com> 작성:

> Hi all,
> 
> I am glad to tell you that the off-heap memory management is merged (PR link
> <https://github.com/apache/incubator-nemo/pull/223>).
> Nemo now reuses the off-heap memory as much as possible to minimize the
> malloc() function call by the support of centralized manager,
> MemoryPoolAssigner. Also, users can set the maximum amount of off-heap
> memory to use. So that if the job tries to allocate more than this limit,
> exception will be thrown.
> 
> As the last step of the GSoC 2019, I am planning to implement JVM and
> off-heap region separation, as described in the JIRA issue
> <https://issues.apache.org/jira/browse/NEMO-397>. This will let much stable
> support for off-heap memory usages.
> 
> If you have any suggestions or questions regarding this project, please let
> me know.
> Thank you very much for your help and support.
> 
> Best regards,
> Haeyoon Cho