You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nemo.apache.org by Haeyoon Cho <ch...@gmail.com> on 2019/04/04 09:03:43 UTC
GSoC proposal: Empower Nemo I/O with new hardwares and off-heap memory
Hi,
This is Haeyoon Cho, applying for GSoC 2019 to work on Apache Nemo.
I have submitted my GSoC proposal, and it would be very nice if I can get
some comments regarding the proposal. This is the link.
<https://docs.google.com/document/d/13Gs4nqU54hDP_EHTlBGMhxuzZf4BMN95I-0ci5m4lTo/edit?usp=sharing>
To sum up the topic, the goal is to optimize Nemo regarding the I/O by
following approach:
1. Integrating Apache Crail into Apache Nemo (NEMO-351
<https://issues.apache.org/jira/browse/NEMO-351>)
2. Utilizing off-heap memory (NEMO-350
<https://issues.apache.org/jira/browse/NEMO-350> and NEMO-370
<https://issues.apache.org/jira/browse/NEMO-370>)
For the third topic(NEMO-370
<https://issues.apache.org/jira/browse/NEMO-370>), I still have to do some
experiments whether this will actually cause meaningful performance gain.
If you have any better ideas, please let me know.
Thank you very much.
Best regards,
Haeyoon
Re: GSoC proposal: Empower Nemo I/O with new hardwares and off-heap memory
Posted by Haeyoon Cho <ch...@gmail.com>.
Hi Gyewon,
Thank you for your kindness :)
I revised the proposal, reflecting your comments. If it’s okay, I am
planning to submit it this evening. Thank you again for your ideas.
Best,
Haeyoon.
2019년 4월 4일 (목) 오후 6:27, Gyewon Lee <st...@gmail.com>님이 작성:
> Hello, Haeyoon.
>
> Welcome to the community and I really appreciate for sharing the proposal.
>
> The initial proposal looks great to me. I left some minor comments on your
> proposals.
>
> Best,
> Gyewon
>
> 2019년 4월 4일 (목) 오후 6:10, Byung-Gon Chun <bg...@gmail.com>님이 작성:
>
> > Haeyoon,
> >
> > Welcome! Thanks for sharing the proposal!
> > Off-heap memory and disaggregated storage (e.g., Crail) support are
> > important tasks. Sounds good.
> >
> > Best,
> > Gon
> >
> > On Thu, Apr 4, 2019 at 6:04 PM Haeyoon Cho <ch...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > This is Haeyoon Cho, applying for GSoC 2019 to work on Apache Nemo.
> > > I have submitted my GSoC proposal, and it would be very nice if I can
> get
> > > some comments regarding the proposal. This is the link.
> > > <
> > >
> >
> https://docs.google.com/document/d/13Gs4nqU54hDP_EHTlBGMhxuzZf4BMN95I-0ci5m4lTo/edit?usp=sharing
> > > >
> > >
> > > To sum up the topic, the goal is to optimize Nemo regarding the I/O by
> > > following approach:
> > >
> > > 1. Integrating Apache Crail into Apache Nemo (NEMO-351
> > > <https://issues.apache.org/jira/browse/NEMO-351>)
> > > 2. Utilizing off-heap memory (NEMO-350
> > > <https://issues.apache.org/jira/browse/NEMO-350> and NEMO-370
> > > <https://issues.apache.org/jira/browse/NEMO-370>)
> > >
> > > For the third topic(NEMO-370
> > > <https://issues.apache.org/jira/browse/NEMO-370>), I still have to do
> > some
> > > experiments whether this will actually cause meaningful performance
> gain.
> > > If you have any better ideas, please let me know.
> > >
> > > Thank you very much.
> > >
> > > Best regards,
> > > Haeyoon
> > >
> >
> >
> > --
> > Byung-Gon Chun
> >
>
Re: GSoC proposal: Empower Nemo I/O with new hardwares and off-heap memory
Posted by Gyewon Lee <st...@gmail.com>.
Hello, Haeyoon.
Welcome to the community and I really appreciate for sharing the proposal.
The initial proposal looks great to me. I left some minor comments on your
proposals.
Best,
Gyewon
2019년 4월 4일 (목) 오후 6:10, Byung-Gon Chun <bg...@gmail.com>님이 작성:
> Haeyoon,
>
> Welcome! Thanks for sharing the proposal!
> Off-heap memory and disaggregated storage (e.g., Crail) support are
> important tasks. Sounds good.
>
> Best,
> Gon
>
> On Thu, Apr 4, 2019 at 6:04 PM Haeyoon Cho <ch...@gmail.com> wrote:
>
> > Hi,
> >
> > This is Haeyoon Cho, applying for GSoC 2019 to work on Apache Nemo.
> > I have submitted my GSoC proposal, and it would be very nice if I can get
> > some comments regarding the proposal. This is the link.
> > <
> >
> https://docs.google.com/document/d/13Gs4nqU54hDP_EHTlBGMhxuzZf4BMN95I-0ci5m4lTo/edit?usp=sharing
> > >
> >
> > To sum up the topic, the goal is to optimize Nemo regarding the I/O by
> > following approach:
> >
> > 1. Integrating Apache Crail into Apache Nemo (NEMO-351
> > <https://issues.apache.org/jira/browse/NEMO-351>)
> > 2. Utilizing off-heap memory (NEMO-350
> > <https://issues.apache.org/jira/browse/NEMO-350> and NEMO-370
> > <https://issues.apache.org/jira/browse/NEMO-370>)
> >
> > For the third topic(NEMO-370
> > <https://issues.apache.org/jira/browse/NEMO-370>), I still have to do
> some
> > experiments whether this will actually cause meaningful performance gain.
> > If you have any better ideas, please let me know.
> >
> > Thank you very much.
> >
> > Best regards,
> > Haeyoon
> >
>
>
> --
> Byung-Gon Chun
>
Re: GSoC proposal: Empower Nemo I/O with new hardwares and off-heap memory
Posted by Haeyoon Cho <ch...@gmail.com>.
Hi Gyewon,
thank you for the review again.
I will submit the final proposal soon.
Looking forward to co-op with the team!
Best,
Haeyoon
2019년 4월 8일 (월) 오후 2:15, Gyewon Lee <st...@gmail.com>님이 작성:
> Hello, Haeyoon.
>
> I have taken a look at your revision, and it looks good to me. Just don't
> forget to finalize your proposal before the deadline :)
>
> Best,
> Gyewon
>
> 2019년 4월 8일 (월) 오후 12:20, Haeyoon Cho <ch...@gmail.com>님이 작성:
>
> > Hello,
> >
> > Thank you for your warm welcome!
> >
> > I will try my best on implementing bug-free and efficient source codes.
> > Thank you :)
> >
> > Best,
> > Haeyoon
> >
> > 2019년 4월 4일 (목) 오후 6:10, Byung-Gon Chun <bg...@gmail.com>님이 작성:
> >
> > > Haeyoon,
> > >
> > > Welcome! Thanks for sharing the proposal!
> > > Off-heap memory and disaggregated storage (e.g., Crail) support are
> > > important tasks. Sounds good.
> > >
> > > Best,
> > > Gon
> > >
> > > On Thu, Apr 4, 2019 at 6:04 PM Haeyoon Cho <ch...@gmail.com>
> wrote:
> > >
> > > > Hi,
> > > >
> > > > This is Haeyoon Cho, applying for GSoC 2019 to work on Apache Nemo.
> > > > I have submitted my GSoC proposal, and it would be very nice if I can
> > get
> > > > some comments regarding the proposal. This is the link.
> > > > <
> > > >
> > >
> >
> https://docs.google.com/document/d/13Gs4nqU54hDP_EHTlBGMhxuzZf4BMN95I-0ci5m4lTo/edit?usp=sharing
> > > > >
> > > >
> > > > To sum up the topic, the goal is to optimize Nemo regarding the I/O
> by
> > > > following approach:
> > > >
> > > > 1. Integrating Apache Crail into Apache Nemo (NEMO-351
> > > > <https://issues.apache.org/jira/browse/NEMO-351>)
> > > > 2. Utilizing off-heap memory (NEMO-350
> > > > <https://issues.apache.org/jira/browse/NEMO-350> and NEMO-370
> > > > <https://issues.apache.org/jira/browse/NEMO-370>)
> > > >
> > > > For the third topic(NEMO-370
> > > > <https://issues.apache.org/jira/browse/NEMO-370>), I still have to
> do
> > > some
> > > > experiments whether this will actually cause meaningful performance
> > gain.
> > > > If you have any better ideas, please let me know.
> > > >
> > > > Thank you very much.
> > > >
> > > > Best regards,
> > > > Haeyoon
> > > >
> > >
> > >
> > > --
> > > Byung-Gon Chun
> > >
> >
>
Re: GSoC proposal: Empower Nemo I/O with new hardwares and off-heap memory
Posted by Gyewon Lee <st...@gmail.com>.
Hello, Haeyoon.
I have taken a look at your revision, and it looks good to me. Just don't
forget to finalize your proposal before the deadline :)
Best,
Gyewon
2019년 4월 8일 (월) 오후 12:20, Haeyoon Cho <ch...@gmail.com>님이 작성:
> Hello,
>
> Thank you for your warm welcome!
>
> I will try my best on implementing bug-free and efficient source codes.
> Thank you :)
>
> Best,
> Haeyoon
>
> 2019년 4월 4일 (목) 오후 6:10, Byung-Gon Chun <bg...@gmail.com>님이 작성:
>
> > Haeyoon,
> >
> > Welcome! Thanks for sharing the proposal!
> > Off-heap memory and disaggregated storage (e.g., Crail) support are
> > important tasks. Sounds good.
> >
> > Best,
> > Gon
> >
> > On Thu, Apr 4, 2019 at 6:04 PM Haeyoon Cho <ch...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > This is Haeyoon Cho, applying for GSoC 2019 to work on Apache Nemo.
> > > I have submitted my GSoC proposal, and it would be very nice if I can
> get
> > > some comments regarding the proposal. This is the link.
> > > <
> > >
> >
> https://docs.google.com/document/d/13Gs4nqU54hDP_EHTlBGMhxuzZf4BMN95I-0ci5m4lTo/edit?usp=sharing
> > > >
> > >
> > > To sum up the topic, the goal is to optimize Nemo regarding the I/O by
> > > following approach:
> > >
> > > 1. Integrating Apache Crail into Apache Nemo (NEMO-351
> > > <https://issues.apache.org/jira/browse/NEMO-351>)
> > > 2. Utilizing off-heap memory (NEMO-350
> > > <https://issues.apache.org/jira/browse/NEMO-350> and NEMO-370
> > > <https://issues.apache.org/jira/browse/NEMO-370>)
> > >
> > > For the third topic(NEMO-370
> > > <https://issues.apache.org/jira/browse/NEMO-370>), I still have to do
> > some
> > > experiments whether this will actually cause meaningful performance
> gain.
> > > If you have any better ideas, please let me know.
> > >
> > > Thank you very much.
> > >
> > > Best regards,
> > > Haeyoon
> > >
> >
> >
> > --
> > Byung-Gon Chun
> >
>
Re: GSoC proposal: Empower Nemo I/O with new hardwares and off-heap memory
Posted by Haeyoon Cho <ch...@gmail.com>.
Hello,
Thank you for your warm welcome!
I will try my best on implementing bug-free and efficient source codes.
Thank you :)
Best,
Haeyoon
2019년 4월 4일 (목) 오후 6:10, Byung-Gon Chun <bg...@gmail.com>님이 작성:
> Haeyoon,
>
> Welcome! Thanks for sharing the proposal!
> Off-heap memory and disaggregated storage (e.g., Crail) support are
> important tasks. Sounds good.
>
> Best,
> Gon
>
> On Thu, Apr 4, 2019 at 6:04 PM Haeyoon Cho <ch...@gmail.com> wrote:
>
> > Hi,
> >
> > This is Haeyoon Cho, applying for GSoC 2019 to work on Apache Nemo.
> > I have submitted my GSoC proposal, and it would be very nice if I can get
> > some comments regarding the proposal. This is the link.
> > <
> >
> https://docs.google.com/document/d/13Gs4nqU54hDP_EHTlBGMhxuzZf4BMN95I-0ci5m4lTo/edit?usp=sharing
> > >
> >
> > To sum up the topic, the goal is to optimize Nemo regarding the I/O by
> > following approach:
> >
> > 1. Integrating Apache Crail into Apache Nemo (NEMO-351
> > <https://issues.apache.org/jira/browse/NEMO-351>)
> > 2. Utilizing off-heap memory (NEMO-350
> > <https://issues.apache.org/jira/browse/NEMO-350> and NEMO-370
> > <https://issues.apache.org/jira/browse/NEMO-370>)
> >
> > For the third topic(NEMO-370
> > <https://issues.apache.org/jira/browse/NEMO-370>), I still have to do
> some
> > experiments whether this will actually cause meaningful performance gain.
> > If you have any better ideas, please let me know.
> >
> > Thank you very much.
> >
> > Best regards,
> > Haeyoon
> >
>
>
> --
> Byung-Gon Chun
>
Re: GSoC proposal: Empower Nemo I/O with new hardwares and off-heap memory
Posted by Byung-Gon Chun <bg...@gmail.com>.
Haeyoon,
Welcome! Thanks for sharing the proposal!
Off-heap memory and disaggregated storage (e.g., Crail) support are
important tasks. Sounds good.
Best,
Gon
On Thu, Apr 4, 2019 at 6:04 PM Haeyoon Cho <ch...@gmail.com> wrote:
> Hi,
>
> This is Haeyoon Cho, applying for GSoC 2019 to work on Apache Nemo.
> I have submitted my GSoC proposal, and it would be very nice if I can get
> some comments regarding the proposal. This is the link.
> <
> https://docs.google.com/document/d/13Gs4nqU54hDP_EHTlBGMhxuzZf4BMN95I-0ci5m4lTo/edit?usp=sharing
> >
>
> To sum up the topic, the goal is to optimize Nemo regarding the I/O by
> following approach:
>
> 1. Integrating Apache Crail into Apache Nemo (NEMO-351
> <https://issues.apache.org/jira/browse/NEMO-351>)
> 2. Utilizing off-heap memory (NEMO-350
> <https://issues.apache.org/jira/browse/NEMO-350> and NEMO-370
> <https://issues.apache.org/jira/browse/NEMO-370>)
>
> For the third topic(NEMO-370
> <https://issues.apache.org/jira/browse/NEMO-370>), I still have to do some
> experiments whether this will actually cause meaningful performance gain.
> If you have any better ideas, please let me know.
>
> Thank you very much.
>
> Best regards,
> Haeyoon
>
--
Byung-Gon Chun