You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Slim Baltagi <sb...@gmail.com> on 2015/07/31 11:57:35 UTC
Off-heap memory in Flink?
Hi
I remember seeing that using off-heap memory was on Flink’s roadmap as well
as a related pull request https://github.com/apache/flink/pull/290
Any update on such effort?
Thanks
Slim Baltagi
--
View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Off-heap-memory-in-Flink-tp7285.html
Sent from the Apache Flink Mailing List archive. mailing list archive at Nabble.com.
Re: Off-heap memory in Flink?
Posted by Stephan Ewen <se...@apache.org>.
Hey!
There is the wiki on Flink's memory management:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741525
Greetings,
Stephan
On Fri, Jul 31, 2015 at 4:06 PM, Radu Tudoran <ra...@huawei.com>
wrote:
> Hi,
>
> Is there some info, description about how this off-heap memory is managed
> and its goals?
> Thanks
>
> Dr. Radu Tudoran
> Research Engineer
> IT R&D Division
>
>
> HUAWEI TECHNOLOGIES Duesseldorf GmbH
> European Research Center
> Riesstrasse 25, 80992 München
>
> E-mail: radu.tudoran@huawei.com
> Mobile: +49 15209084330
> Telephone: +49 891588344173
>
> HUAWEI TECHNOLOGIES Duesseldorf GmbH
> Hansaallee 205, 40549 Düsseldorf, Germany, www.huawei.com
> Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
> Managing Director: Jingwen TAO, Wanzhou MENG, Lifang CHEN
> Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
> Geschäftsführer: Jingwen TAO, Wanzhou MENG, Lifang CHEN
>
> -----Original Message-----
> From: ewenstephan@gmail.com [mailto:ewenstephan@gmail.com] On Behalf Of
> Stephan Ewen
> Sent: Friday, July 31, 2015 4:02 PM
> To: dev@flink.apache.org
> Subject: Re: Off-heap memory in Flink?
>
> The Pull Request is basically ready. I would like to benchmark a bit
> before merging it.
>
> The on-heap Flink-managed memory classes are highly optimized to be JIT
> friendly. Just want to make sure that we don't loose that.
>
> I have worked a lot on streaming issues lately, so this is still in my
> backlog.
>
> On Fri, Jul 31, 2015 at 2:09 PM, Maximilian Michels <mx...@apache.org>
> wrote:
>
> > Hi Slim,
> >
> > Off-heap memory has been postponed because it's not a pressing but
> > rather a nice-to-have feature. I know that Stephan continued to work
> > on the off-heap memory. I think we can get it in sometime this year.
> >
> > Best,
> > Max
> >
> > On Fri, Jul 31, 2015 at 11:57 AM, Slim Baltagi <sb...@gmail.com>
> wrote:
> >
> > > Hi
> > >
> > > I remember seeing that using off-heap memory was on Flink’s roadmap
> > > as
> > well
> > > as a related pull request https://github.com/apache/flink/pull/290
> > >
> > > Any update on such effort?
> > >
> > > Thanks
> > >
> > > Slim Baltagi
> > >
> > >
> > >
> > > --
> > > View this message in context:
> > >
> > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Off-hea
> > p-memory-in-Flink-tp7285.html
> > > Sent from the Apache Flink Mailing List archive. mailing list
> > > archive at Nabble.com.
> > >
> >
>
RE: Off-heap memory in Flink?
Posted by Radu Tudoran <ra...@huawei.com>.
Hi,
Is there some info, description about how this off-heap memory is managed and its goals?
Thanks
Dr. Radu Tudoran
Research Engineer
IT R&D Division
HUAWEI TECHNOLOGIES Duesseldorf GmbH
European Research Center
Riesstrasse 25, 80992 München
E-mail: radu.tudoran@huawei.com
Mobile: +49 15209084330
Telephone: +49 891588344173
HUAWEI TECHNOLOGIES Duesseldorf GmbH
Hansaallee 205, 40549 Düsseldorf, Germany, www.huawei.com
Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
Managing Director: Jingwen TAO, Wanzhou MENG, Lifang CHEN
Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
Geschäftsführer: Jingwen TAO, Wanzhou MENG, Lifang CHEN
-----Original Message-----
From: ewenstephan@gmail.com [mailto:ewenstephan@gmail.com] On Behalf Of Stephan Ewen
Sent: Friday, July 31, 2015 4:02 PM
To: dev@flink.apache.org
Subject: Re: Off-heap memory in Flink?
The Pull Request is basically ready. I would like to benchmark a bit before merging it.
The on-heap Flink-managed memory classes are highly optimized to be JIT friendly. Just want to make sure that we don't loose that.
I have worked a lot on streaming issues lately, so this is still in my backlog.
On Fri, Jul 31, 2015 at 2:09 PM, Maximilian Michels <mx...@apache.org> wrote:
> Hi Slim,
>
> Off-heap memory has been postponed because it's not a pressing but
> rather a nice-to-have feature. I know that Stephan continued to work
> on the off-heap memory. I think we can get it in sometime this year.
>
> Best,
> Max
>
> On Fri, Jul 31, 2015 at 11:57 AM, Slim Baltagi <sb...@gmail.com> wrote:
>
> > Hi
> >
> > I remember seeing that using off-heap memory was on Flink’s roadmap
> > as
> well
> > as a related pull request https://github.com/apache/flink/pull/290
> >
> > Any update on such effort?
> >
> > Thanks
> >
> > Slim Baltagi
> >
> >
> >
> > --
> > View this message in context:
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Off-hea
> p-memory-in-Flink-tp7285.html
> > Sent from the Apache Flink Mailing List archive. mailing list
> > archive at Nabble.com.
> >
>
Re: Off-heap memory in Flink?
Posted by Stephan Ewen <se...@apache.org>.
The Pull Request is basically ready. I would like to benchmark a bit before
merging it.
The on-heap Flink-managed memory classes are highly optimized to be JIT
friendly. Just want to make sure that we don't loose that.
I have worked a lot on streaming issues lately, so this is still in my
backlog.
On Fri, Jul 31, 2015 at 2:09 PM, Maximilian Michels <mx...@apache.org> wrote:
> Hi Slim,
>
> Off-heap memory has been postponed because it's not a pressing but rather a
> nice-to-have feature. I know that Stephan continued to work on the off-heap
> memory. I think we can get it in sometime this year.
>
> Best,
> Max
>
> On Fri, Jul 31, 2015 at 11:57 AM, Slim Baltagi <sb...@gmail.com> wrote:
>
> > Hi
> >
> > I remember seeing that using off-heap memory was on Flink’s roadmap as
> well
> > as a related pull request https://github.com/apache/flink/pull/290
> >
> > Any update on such effort?
> >
> > Thanks
> >
> > Slim Baltagi
> >
> >
> >
> > --
> > View this message in context:
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Off-heap-memory-in-Flink-tp7285.html
> > Sent from the Apache Flink Mailing List archive. mailing list archive at
> > Nabble.com.
> >
>
Re: Off-heap memory in Flink?
Posted by Maximilian Michels <mx...@apache.org>.
Hi Slim,
Off-heap memory has been postponed because it's not a pressing but rather a
nice-to-have feature. I know that Stephan continued to work on the off-heap
memory. I think we can get it in sometime this year.
Best,
Max
On Fri, Jul 31, 2015 at 11:57 AM, Slim Baltagi <sb...@gmail.com> wrote:
> Hi
>
> I remember seeing that using off-heap memory was on Flink’s roadmap as well
> as a related pull request https://github.com/apache/flink/pull/290
>
> Any update on such effort?
>
> Thanks
>
> Slim Baltagi
>
>
>
> --
> View this message in context:
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Off-heap-memory-in-Flink-tp7285.html
> Sent from the Apache Flink Mailing List archive. mailing list archive at
> Nabble.com.
>