You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Aleksey Shipilev <al...@gmail.com> on 2008/03/27 18:10:54 UTC
[GSoC] Appliance for harmony-gc-4 "Unify the native memory management of Harmony DRLVM"
Hi all, Xiao Feng, Andrey (Yakushev),
I want to apply for GSoC with the idea of unification of native memory
management of Harmony DRLVM. I had found some issues recently on
SerialBench that stresses the memory allocation subsystem and will be
glad to refactor NMM on DRLVM. I had filed the draft of application on
GSoC site, can you please review it?
Thanks,
Aleksey.
Re: [GSoC] Appliance for harmony-gc-4 "Unify the native memory management of Harmony DRLVM"
Posted by Andrey Yakushev <an...@gmail.com>.
Aleksey,
I looked at your proposal and it seems to be the pretty complete list
of native memory management problems and good plan for starting of
them resolving.
I could only suggest more clear formulating the final goals of your
efforts. Some kind of success metric.
Would it be the reduction of used APIs, so it means more clear DRLVM
architecture? How many of them should be remained; and where is the
optimum? Does the freedom of API choice means better modularity?
Or it would be the resolving of known bugs. Which ones and how many?
Or performance improvement. What speedup is expected?
Or management improving. What could be improved precisely?
--
Thanks,
Andrey
On Mon, Mar 31, 2008 at 5:52 PM, Aleksey Shipilev
<al...@gmail.com> wrote:
> Alexei,
>
> I had addressed both of your issues in the updated proposal [1]:
> a. Added portlib pools as the candidate.
> b. Rephrased abstract a little, with emphasis on unifying.
> c. "Class-based service" added to improvements plan: service critical
> customers like exception handlers, etc.
> d. "Wrapping malloc/free" added to improvements plan.
>
> I want to thank you with the idea (d) - that's the way of moving
> Classlib native code to UMM.
>
> Xiao Feng, Andrey (Yakushev), can you please review?
>
>
> Thanks,
> Aleksey.
>
> [1] http://wiki.apache.org/general/AlekseyShipilev/GSoC2008/harmony-gc-4
>
>
> > > On Sun, Mar 30, 2008 at 10:01 PM, Alexei Fedotov
> > > <al...@gmail.com> wrote:
>
>
> > > > You forgot portlib pools. All this reads in a following way "We have
> > > > seven memory subsystems, and I want to implement the eighth which
> > > > would be the best." Ok, I put the same intention into STD_MALLOC three
> > > > years ago. We attached APR for the same reason.You suggest adding UMM.
> > > > Amount of systems continues to grow. Contrary, I would suggest having
> > > > less memory management subsystems on completion of your project.
> > > >
> > > > BTW, two important use cases are missed in your proposal. The native
> > > > memory subsystem should continue serving critical customers such as
> > > > lazy exception messages or finalizers even when it reported to others
> > > > that the memory is exhausted. To prevent user's JNI code from
> > > > exhausting memory we did not find a solution other than substitution
> > > > of function table in C runtime library with our functions. Redirecting
> > > > calls to our functions allows us plugging OS-level optimized memory
> > > > managers and in this case we can continue using conventional malloc
> > > > and free rather than inventing hymalloc.
>
Re: [GSoC] Appliance for harmony-gc-4 "Unify the native memory management of Harmony DRLVM"
Posted by Ligang Wang <wa...@gmail.com>.
Hello all, Aleksey,
I have seen your proposal on UMM for GSoC. It is a very good idea to refine
the native memory management in DRLVM. I am also interesed in this topic.
How is it going now?
Thanks,
Ligang
On 4/3/08, Xiao-Feng Li <xi...@gmail.com> wrote:
>
> On Thu, Apr 3, 2008 at 9:41 PM, Andrey Yakushev
> <an...@gmail.com> wrote:
> > You want to have big gap from the beginning. :)
> > I think its OK, but I suggest removing SPECjbb as not too
> > representative for native memory usage.
> >
>
> Agreed with Yakushev.
> I personally think that both footprint and performance should be
> better finally.
>
> As a start point, Shipilev's targets are ok anyway.
>
> Thanks,
> xiaofeng
>
> > On 4/2/08, Aleksey Shipilev <al...@gmail.com> wrote:
> >
> >
> > > On Wed, Apr 2, 2008 at 3:15 PM, Andrey Yakushev
> > > <an...@gmail.com> wrote:
> > > > OK, let it would be the first step of investigation. But at least
> add
> > > > note that performance and memory footprint wouldn't be worse then
> now
> > > > on defined set of tests.
> > > Right. But once again, without any prototype it's hard to guess the
> > > performance changes.
> > >
> > > Would these requirements fit?
> > > a. "The performance DRLVM with UMM enabled should be at least 80% of
> > > DRLVM with legacy memory management, as measured by execution on
> > > Dacapo, SPECjbb2005 and Eclipse startup".
> > > b. "The memory footprint of DRLVM with UMM enabled should be not
> > > larger than 120% of DRLVM with legacy memory management, as measured
> > > by execution on Dacapo, SPECjbb2005 and Eclipse startup".
> > >
> > > Though these requirements are more or less loose, they protect from
> > > UMM implementation that bloats up the performance or memory footprint
> > > many times to be considered successful.
> > >
> > > Thanks,
> > > Aleksey.
> > >
> >
> >
> > --
> > Thanks,
> > Andrey
> >
>
>
>
> --
> http://xiao-feng.blogspot.com
>
Re: [GSoC] Appliance for harmony-gc-4 "Unify the native memory management of Harmony DRLVM"
Posted by Xiao-Feng Li <xi...@gmail.com>.
On Thu, Apr 3, 2008 at 9:41 PM, Andrey Yakushev
<an...@gmail.com> wrote:
> You want to have big gap from the beginning. :)
> I think its OK, but I suggest removing SPECjbb as not too
> representative for native memory usage.
>
Agreed with Yakushev.
I personally think that both footprint and performance should be
better finally.
As a start point, Shipilev's targets are ok anyway.
Thanks,
xiaofeng
> On 4/2/08, Aleksey Shipilev <al...@gmail.com> wrote:
>
>
> > On Wed, Apr 2, 2008 at 3:15 PM, Andrey Yakushev
> > <an...@gmail.com> wrote:
> > > OK, let it would be the first step of investigation. But at least add
> > > note that performance and memory footprint wouldn't be worse then now
> > > on defined set of tests.
> > Right. But once again, without any prototype it's hard to guess the
> > performance changes.
> >
> > Would these requirements fit?
> > a. "The performance DRLVM with UMM enabled should be at least 80% of
> > DRLVM with legacy memory management, as measured by execution on
> > Dacapo, SPECjbb2005 and Eclipse startup".
> > b. "The memory footprint of DRLVM with UMM enabled should be not
> > larger than 120% of DRLVM with legacy memory management, as measured
> > by execution on Dacapo, SPECjbb2005 and Eclipse startup".
> >
> > Though these requirements are more or less loose, they protect from
> > UMM implementation that bloats up the performance or memory footprint
> > many times to be considered successful.
> >
> > Thanks,
> > Aleksey.
> >
>
>
> --
> Thanks,
> Andrey
>
--
http://xiao-feng.blogspot.com
Re: [GSoC] Appliance for harmony-gc-4 "Unify the native memory management of Harmony DRLVM"
Posted by Andrey Yakushev <an...@gmail.com>.
You want to have big gap from the beginning. :)
I think its OK, but I suggest removing SPECjbb as not too
representative for native memory usage.
On 4/2/08, Aleksey Shipilev <al...@gmail.com> wrote:
> On Wed, Apr 2, 2008 at 3:15 PM, Andrey Yakushev
> <an...@gmail.com> wrote:
> > OK, let it would be the first step of investigation. But at least add
> > note that performance and memory footprint wouldn't be worse then now
> > on defined set of tests.
> Right. But once again, without any prototype it's hard to guess the
> performance changes.
>
> Would these requirements fit?
> a. "The performance DRLVM with UMM enabled should be at least 80% of
> DRLVM with legacy memory management, as measured by execution on
> Dacapo, SPECjbb2005 and Eclipse startup".
> b. "The memory footprint of DRLVM with UMM enabled should be not
> larger than 120% of DRLVM with legacy memory management, as measured
> by execution on Dacapo, SPECjbb2005 and Eclipse startup".
>
> Though these requirements are more or less loose, they protect from
> UMM implementation that bloats up the performance or memory footprint
> many times to be considered successful.
>
> Thanks,
> Aleksey.
>
--
Thanks,
Andrey
Re: [GSoC] Appliance for harmony-gc-4 "Unify the native memory management of Harmony DRLVM"
Posted by Aleksey Shipilev <al...@gmail.com>.
On Wed, Apr 2, 2008 at 3:15 PM, Andrey Yakushev
<an...@gmail.com> wrote:
> OK, let it would be the first step of investigation. But at least add
> note that performance and memory footprint wouldn't be worse then now
> on defined set of tests.
Right. But once again, without any prototype it's hard to guess the
performance changes.
Would these requirements fit?
a. "The performance DRLVM with UMM enabled should be at least 80% of
DRLVM with legacy memory management, as measured by execution on
Dacapo, SPECjbb2005 and Eclipse startup".
b. "The memory footprint of DRLVM with UMM enabled should be not
larger than 120% of DRLVM with legacy memory management, as measured
by execution on Dacapo, SPECjbb2005 and Eclipse startup".
Though these requirements are more or less loose, they protect from
UMM implementation that bloats up the performance or memory footprint
many times to be considered successful.
Thanks,
Aleksey.
Re: [GSoC] Appliance for harmony-gc-4 "Unify the native memory management of Harmony DRLVM"
Posted by Andrey Yakushev <an...@gmail.com>.
On 4/2/08, Aleksey Shipilev <al...@gmail.com> wrote:
> Thanks, Andrey, Xiao Feng!
> Your comments are very valuable.
>
> 1. I had updated the wiki [1] with new version of proposal, emphasized
> once more on goals.
>
> 2. Metric of success. I think that (a) projected boosts could not be
> estimated right now, without working prototype, (b) bugs in memory
> management are also covered by multiple wrappers and I expect some of
> them will be exposed after moving to UMM, but still it's hard to
> reveal without working prototype. So, I choose the "UMM is ready and
> used in most of VM components" as good enough starting metric. What do
> you think, Andrey?
OK, let it would be the first step of investigation. But at least add
note that performance and memory footprint wouldn't be worse then now
on defined set of tests.
>
> 3. GC and mmap. Though I had little experience with GC heap adjustment
> mechanisms, why can't GC just use hyvmmap wrapper for mmap? If we will
> cover the same interface as mmap do, then there should be no problems,
> right? Moreover, it can eliminate cross-platform memory management
> wrappers inside the GC and delegate this to UMM. Xiao Feng, what do
> you think?
>
> Thanks,
> Aleksey.
>
> [1] http://wiki.apache.org/general/AlekseyShipilev/GSoC2008/harmony-gc-4
>
> On Tue, Apr 1, 2008 at 5:51 AM, Xiao-Feng Li <xi...@gmail.com> wrote:
> > Hi, Aleksey S. very good proposal. It deserves Google support. :-)
> >
> > Two comments:
> > 1. goal of the project. Your stated goals are related only to software
> > engineering issues. I think it should also be related to performance
> > issue and research issue. Specifically, a unified native MM can open
> > up new opportunities such as large page for the entire system, data
> > prefetch, JIT code caching management, Java directBuffer improvement,
> > etc.
> > 2. GC related. GC uses mmap for good reasons. It need to map certain
> > pages or unmap them sometimes. This support is important for GC heap
> > adjustment. It's common for GC to reserve certain space, then commit
> > or decommit part of it. So basically GC has two set of memory uses:
> > one is like malloc for its own runtime allocations, the other is like
> > mmap for Java heap management. I would suggest we ignore the latter
> > category at the beginning of the project. It's not an easy stuff until
> > we have good understandings about the unified NMM.
> >
> > Thanks,
> > xiaofeng
> >
> > On Mon, Mar 31, 2008 at 9:52 PM, Aleksey Shipilev
> >
> > <al...@gmail.com> wrote:
> >
> >
> > > Alexei,
> > >
> > > I had addressed both of your issues in the updated proposal [1]:
> > > a. Added portlib pools as the candidate.
> > > b. Rephrased abstract a little, with emphasis on unifying.
> > > c. "Class-based service" added to improvements plan: service critical
> > > customers like exception handlers, etc.
> > > d. "Wrapping malloc/free" added to improvements plan.
> > >
> > > I want to thank you with the idea (d) - that's the way of moving
> > > Classlib native code to UMM.
> > >
> > > Xiao Feng, Andrey (Yakushev), can you please review?
> > >
> > >
> > > Thanks,
> > > Aleksey.
> > >
> > > [1] http://wiki.apache.org/general/AlekseyShipilev/GSoC2008/harmony-gc-4
> > >
> > >
> > > > > On Sun, Mar 30, 2008 at 10:01 PM, Alexei Fedotov
> > > > > <al...@gmail.com> wrote:
> > >
> > >
> > > > > > You forgot portlib pools. All this reads in a following way "We have
> > > > > > seven memory subsystems, and I want to implement the eighth which
> > > > > > would be the best." Ok, I put the same intention into STD_MALLOC three
> > > > > > years ago. We attached APR for the same reason.You suggest adding UMM.
> > > > > > Amount of systems continues to grow. Contrary, I would suggest having
> > > > > > less memory management subsystems on completion of your project.
> > > > > >
> > > > > > BTW, two important use cases are missed in your proposal. The native
> > > > > > memory subsystem should continue serving critical customers such as
> > > > > > lazy exception messages or finalizers even when it reported to others
> > > > > > that the memory is exhausted. To prevent user's JNI code from
> > > > > > exhausting memory we did not find a solution other than substitution
> > > > > > of function table in C runtime library with our functions. Redirecting
> > > > > > calls to our functions allows us plugging OS-level optimized memory
> > > > > > managers and in this case we can continue using conventional malloc
> > > > > > and free rather than inventing hymalloc.
> > >
> >
> >
> >
> > --
> > http://xiao-feng.blogspot.com
> >
>
--
Thanks,
Andrey
Re: [GSoC] Appliance for harmony-gc-4 "Unify the native memory management of Harmony DRLVM"
Posted by Xiao-Feng Li <xi...@gmail.com>.
On Wed, Apr 2, 2008 at 6:26 PM, Aleksey Shipilev
<al...@gmail.com> wrote:
> Thanks, Andrey, Xiao Feng!
> Your comments are very valuable.
>
> 1. I had updated the wiki [1] with new version of proposal, emphasized
> once more on goals.
>
> 2. Metric of success. I think that (a) projected boosts could not be
> estimated right now, without working prototype, (b) bugs in memory
> management are also covered by multiple wrappers and I expect some of
> them will be exposed after moving to UMM, but still it's hard to
> reveal without working prototype. So, I choose the "UMM is ready and
> used in most of VM components" as good enough starting metric. What do
> you think, Andrey?
>
> 3. GC and mmap. Though I had little experience with GC heap adjustment
> mechanisms, why can't GC just use hyvmmap wrapper for mmap? If we will
> cover the same interface as mmap do, then there should be no problems,
> right? Moreover, it can eliminate cross-platform memory management
> wrappers inside the GC and delegate this to UMM. Xiao Feng, what do
> you think?
GC definitely should finally sit on a same layer as other parts. The
problem is, it's not that easy as you may expect. To define the
project within a controllable scope is good for you to accomplish.
Thanks,
xiaofeng
>
> Thanks,
> Aleksey.
>
> [1] http://wiki.apache.org/general/AlekseyShipilev/GSoC2008/harmony-gc-4
>
>
>
> On Tue, Apr 1, 2008 at 5:51 AM, Xiao-Feng Li <xi...@gmail.com> wrote:
> > Hi, Aleksey S. very good proposal. It deserves Google support. :-)
> >
> > Two comments:
> > 1. goal of the project. Your stated goals are related only to software
> > engineering issues. I think it should also be related to performance
> > issue and research issue. Specifically, a unified native MM can open
> > up new opportunities such as large page for the entire system, data
> > prefetch, JIT code caching management, Java directBuffer improvement,
> > etc.
> > 2. GC related. GC uses mmap for good reasons. It need to map certain
> > pages or unmap them sometimes. This support is important for GC heap
> > adjustment. It's common for GC to reserve certain space, then commit
> > or decommit part of it. So basically GC has two set of memory uses:
> > one is like malloc for its own runtime allocations, the other is like
> > mmap for Java heap management. I would suggest we ignore the latter
> > category at the beginning of the project. It's not an easy stuff until
> > we have good understandings about the unified NMM.
> >
> > Thanks,
> > xiaofeng
> >
> > On Mon, Mar 31, 2008 at 9:52 PM, Aleksey Shipilev
> >
> > <al...@gmail.com> wrote:
> >
> >
> > > Alexei,
> > >
> > > I had addressed both of your issues in the updated proposal [1]:
> > > a. Added portlib pools as the candidate.
> > > b. Rephrased abstract a little, with emphasis on unifying.
> > > c. "Class-based service" added to improvements plan: service critical
> > > customers like exception handlers, etc.
> > > d. "Wrapping malloc/free" added to improvements plan.
> > >
> > > I want to thank you with the idea (d) - that's the way of moving
> > > Classlib native code to UMM.
> > >
> > > Xiao Feng, Andrey (Yakushev), can you please review?
> > >
> > >
> > > Thanks,
> > > Aleksey.
> > >
> > > [1] http://wiki.apache.org/general/AlekseyShipilev/GSoC2008/harmony-gc-4
> > >
> > >
> > > > > On Sun, Mar 30, 2008 at 10:01 PM, Alexei Fedotov
> > > > > <al...@gmail.com> wrote:
> > >
> > >
> > > > > > You forgot portlib pools. All this reads in a following way "We have
> > > > > > seven memory subsystems, and I want to implement the eighth which
> > > > > > would be the best." Ok, I put the same intention into STD_MALLOC three
> > > > > > years ago. We attached APR for the same reason.You suggest adding UMM.
> > > > > > Amount of systems continues to grow. Contrary, I would suggest having
> > > > > > less memory management subsystems on completion of your project.
> > > > > >
> > > > > > BTW, two important use cases are missed in your proposal. The native
> > > > > > memory subsystem should continue serving critical customers such as
> > > > > > lazy exception messages or finalizers even when it reported to others
> > > > > > that the memory is exhausted. To prevent user's JNI code from
> > > > > > exhausting memory we did not find a solution other than substitution
> > > > > > of function table in C runtime library with our functions. Redirecting
> > > > > > calls to our functions allows us plugging OS-level optimized memory
> > > > > > managers and in this case we can continue using conventional malloc
> > > > > > and free rather than inventing hymalloc.
> > >
> >
> >
> >
> > --
> > http://xiao-feng.blogspot.com
> >
>
--
http://xiao-feng.blogspot.com
Re: [GSoC] Appliance for harmony-gc-4 "Unify the native memory management of Harmony DRLVM"
Posted by Aleksey Shipilev <al...@gmail.com>.
Thanks, Andrey, Xiao Feng!
Your comments are very valuable.
1. I had updated the wiki [1] with new version of proposal, emphasized
once more on goals.
2. Metric of success. I think that (a) projected boosts could not be
estimated right now, without working prototype, (b) bugs in memory
management are also covered by multiple wrappers and I expect some of
them will be exposed after moving to UMM, but still it's hard to
reveal without working prototype. So, I choose the "UMM is ready and
used in most of VM components" as good enough starting metric. What do
you think, Andrey?
3. GC and mmap. Though I had little experience with GC heap adjustment
mechanisms, why can't GC just use hyvmmap wrapper for mmap? If we will
cover the same interface as mmap do, then there should be no problems,
right? Moreover, it can eliminate cross-platform memory management
wrappers inside the GC and delegate this to UMM. Xiao Feng, what do
you think?
Thanks,
Aleksey.
[1] http://wiki.apache.org/general/AlekseyShipilev/GSoC2008/harmony-gc-4
On Tue, Apr 1, 2008 at 5:51 AM, Xiao-Feng Li <xi...@gmail.com> wrote:
> Hi, Aleksey S. very good proposal. It deserves Google support. :-)
>
> Two comments:
> 1. goal of the project. Your stated goals are related only to software
> engineering issues. I think it should also be related to performance
> issue and research issue. Specifically, a unified native MM can open
> up new opportunities such as large page for the entire system, data
> prefetch, JIT code caching management, Java directBuffer improvement,
> etc.
> 2. GC related. GC uses mmap for good reasons. It need to map certain
> pages or unmap them sometimes. This support is important for GC heap
> adjustment. It's common for GC to reserve certain space, then commit
> or decommit part of it. So basically GC has two set of memory uses:
> one is like malloc for its own runtime allocations, the other is like
> mmap for Java heap management. I would suggest we ignore the latter
> category at the beginning of the project. It's not an easy stuff until
> we have good understandings about the unified NMM.
>
> Thanks,
> xiaofeng
>
> On Mon, Mar 31, 2008 at 9:52 PM, Aleksey Shipilev
>
> <al...@gmail.com> wrote:
>
>
> > Alexei,
> >
> > I had addressed both of your issues in the updated proposal [1]:
> > a. Added portlib pools as the candidate.
> > b. Rephrased abstract a little, with emphasis on unifying.
> > c. "Class-based service" added to improvements plan: service critical
> > customers like exception handlers, etc.
> > d. "Wrapping malloc/free" added to improvements plan.
> >
> > I want to thank you with the idea (d) - that's the way of moving
> > Classlib native code to UMM.
> >
> > Xiao Feng, Andrey (Yakushev), can you please review?
> >
> >
> > Thanks,
> > Aleksey.
> >
> > [1] http://wiki.apache.org/general/AlekseyShipilev/GSoC2008/harmony-gc-4
> >
> >
> > > > On Sun, Mar 30, 2008 at 10:01 PM, Alexei Fedotov
> > > > <al...@gmail.com> wrote:
> >
> >
> > > > > You forgot portlib pools. All this reads in a following way "We have
> > > > > seven memory subsystems, and I want to implement the eighth which
> > > > > would be the best." Ok, I put the same intention into STD_MALLOC three
> > > > > years ago. We attached APR for the same reason.You suggest adding UMM.
> > > > > Amount of systems continues to grow. Contrary, I would suggest having
> > > > > less memory management subsystems on completion of your project.
> > > > >
> > > > > BTW, two important use cases are missed in your proposal. The native
> > > > > memory subsystem should continue serving critical customers such as
> > > > > lazy exception messages or finalizers even when it reported to others
> > > > > that the memory is exhausted. To prevent user's JNI code from
> > > > > exhausting memory we did not find a solution other than substitution
> > > > > of function table in C runtime library with our functions. Redirecting
> > > > > calls to our functions allows us plugging OS-level optimized memory
> > > > > managers and in this case we can continue using conventional malloc
> > > > > and free rather than inventing hymalloc.
> >
>
>
>
> --
> http://xiao-feng.blogspot.com
>
Re: [GSoC] Appliance for harmony-gc-4 "Unify the native memory management of Harmony DRLVM"
Posted by Xiao-Feng Li <xi...@gmail.com>.
Hi, Aleksey S. very good proposal. It deserves Google support. :-)
Two comments:
1. goal of the project. Your stated goals are related only to software
engineering issues. I think it should also be related to performance
issue and research issue. Specifically, a unified native MM can open
up new opportunities such as large page for the entire system, data
prefetch, JIT code caching management, Java directBuffer improvement,
etc.
2. GC related. GC uses mmap for good reasons. It need to map certain
pages or unmap them sometimes. This support is important for GC heap
adjustment. It's common for GC to reserve certain space, then commit
or decommit part of it. So basically GC has two set of memory uses:
one is like malloc for its own runtime allocations, the other is like
mmap for Java heap management. I would suggest we ignore the latter
category at the beginning of the project. It's not an easy stuff until
we have good understandings about the unified NMM.
Thanks,
xiaofeng
On Mon, Mar 31, 2008 at 9:52 PM, Aleksey Shipilev
<al...@gmail.com> wrote:
> Alexei,
>
> I had addressed both of your issues in the updated proposal [1]:
> a. Added portlib pools as the candidate.
> b. Rephrased abstract a little, with emphasis on unifying.
> c. "Class-based service" added to improvements plan: service critical
> customers like exception handlers, etc.
> d. "Wrapping malloc/free" added to improvements plan.
>
> I want to thank you with the idea (d) - that's the way of moving
> Classlib native code to UMM.
>
> Xiao Feng, Andrey (Yakushev), can you please review?
>
>
> Thanks,
> Aleksey.
>
> [1] http://wiki.apache.org/general/AlekseyShipilev/GSoC2008/harmony-gc-4
>
>
> > > On Sun, Mar 30, 2008 at 10:01 PM, Alexei Fedotov
> > > <al...@gmail.com> wrote:
>
>
> > > > You forgot portlib pools. All this reads in a following way "We have
> > > > seven memory subsystems, and I want to implement the eighth which
> > > > would be the best." Ok, I put the same intention into STD_MALLOC three
> > > > years ago. We attached APR for the same reason.You suggest adding UMM.
> > > > Amount of systems continues to grow. Contrary, I would suggest having
> > > > less memory management subsystems on completion of your project.
> > > >
> > > > BTW, two important use cases are missed in your proposal. The native
> > > > memory subsystem should continue serving critical customers such as
> > > > lazy exception messages or finalizers even when it reported to others
> > > > that the memory is exhausted. To prevent user's JNI code from
> > > > exhausting memory we did not find a solution other than substitution
> > > > of function table in C runtime library with our functions. Redirecting
> > > > calls to our functions allows us plugging OS-level optimized memory
> > > > managers and in this case we can continue using conventional malloc
> > > > and free rather than inventing hymalloc.
>
--
http://xiao-feng.blogspot.com
Re: [GSoC] Appliance for harmony-gc-4 "Unify the native memory management of Harmony DRLVM"
Posted by Aleksey Shipilev <al...@gmail.com>.
Alexei,
I had addressed both of your issues in the updated proposal [1]:
a. Added portlib pools as the candidate.
b. Rephrased abstract a little, with emphasis on unifying.
c. "Class-based service" added to improvements plan: service critical
customers like exception handlers, etc.
d. "Wrapping malloc/free" added to improvements plan.
I want to thank you with the idea (d) - that's the way of moving
Classlib native code to UMM.
Xiao Feng, Andrey (Yakushev), can you please review?
Thanks,
Aleksey.
[1] http://wiki.apache.org/general/AlekseyShipilev/GSoC2008/harmony-gc-4
> > On Sun, Mar 30, 2008 at 10:01 PM, Alexei Fedotov
> > <al...@gmail.com> wrote:
> > > You forgot portlib pools. All this reads in a following way "We have
> > > seven memory subsystems, and I want to implement the eighth which
> > > would be the best." Ok, I put the same intention into STD_MALLOC three
> > > years ago. We attached APR for the same reason.You suggest adding UMM.
> > > Amount of systems continues to grow. Contrary, I would suggest having
> > > less memory management subsystems on completion of your project.
> > >
> > > BTW, two important use cases are missed in your proposal. The native
> > > memory subsystem should continue serving critical customers such as
> > > lazy exception messages or finalizers even when it reported to others
> > > that the memory is exhausted. To prevent user's JNI code from
> > > exhausting memory we did not find a solution other than substitution
> > > of function table in C runtime library with our functions. Redirecting
> > > calls to our functions allows us plugging OS-level optimized memory
> > > managers and in this case we can continue using conventional malloc
> > > and free rather than inventing hymalloc.
Re: [GSoC] Appliance for harmony-gc-4 "Unify the native memory management of Harmony DRLVM"
Posted by Alexei Fedotov <al...@gmail.com>.
Right!
On Mon, Mar 31, 2008 at 1:09 AM, Aleksey Shipilev
<al...@gmail.com> wrote:
> Thanks, Alexei! I will try to address your concerns in next revision.
>
> As for the style, I would prefer to have proposal technically
> complete, to have the mutual expectations in place and keep in mind
> what's we are really doing. Of course, I'm risking on criticizing the
> exact ideas, but that's the point of this discussion, right? :)
>
> Thanks again,
> Aleksey.
>
>
>
> On Sun, Mar 30, 2008 at 10:01 PM, Alexei Fedotov
> <al...@gmail.com> wrote:
> > Hello Alexei,
> > Good job. I like your enthusiasm, and a technical style of your paper.
> > But exposing technical details you put your ideas in risk of being
> > criticized, your know. I hope that your effort would be useful
> > regardless of few comments I have.
> >
> > > Now DRLVM components use multiple wrappers for memory management: APR pools, STD_MALLOC, PoolManager, MemoryManager, CRT malloc, VirtualAlloc and others.
> > > The primary goal of this project is to implement Unified Memory Manager (UMM)
> >
> > You forgot portlib pools. All this reads in a following way "We have
> > seven memory subsystems, and I want to implement the eighth which
> > would be the best." Ok, I put the same intention into STD_MALLOC three
> > years ago. We attached APR for the same reason.You suggest adding UMM.
> > Amount of systems continues to grow. Contrary, I would suggest having
> > less memory management subsystems on completion of your project.
> >
> > BTW, two important use cases are missed in your proposal. The native
> > memory subsystem should continue serving critical customers such as
> > lazy exception messages or finalizers even when it reported to others
> > that the memory is exhausted. To prevent user's JNI code from
> > exhausting memory we did not find a solution other than substitution
> > of function table in C runtime library with our functions. Redirecting
> > calls to our functions allows us plugging OS-level optimized memory
> > managers and in this case we can continue using conventional malloc
> > and free rather than inventing hymalloc.
> >
> > Thanks!
> >
> >
> > On Sun, Mar 30, 2008 at 6:53 PM, Aleksey Shipilev
> >
> > <al...@gmail.com> wrote:
> > > Hi,
> > >
> > > I had updated the proposal on GSoC site and made the copy on Apache Wiki [1].
> > > I would appreciate anyone's comments and corrections, especially from mentors.
> > >
> > > Thanks,
> > > Aleksey.
> > >
> > > [1] http://wiki.apache.org/general/AlekseyShipilev/GSoC2008/harmony-gc-4
> > >
> > >
> > >
> > > On Thu, Mar 27, 2008 at 9:10 PM, Aleksey Shipilev
> > > <al...@gmail.com> wrote:
> > > > Hi all, Xiao Feng, Andrey (Yakushev),
> > > >
> > > > I want to apply for GSoC with the idea of unification of native memory
> > > > management of Harmony DRLVM. I had found some issues recently on
> > > > SerialBench that stresses the memory allocation subsystem and will be
> > > > glad to refactor NMM on DRLVM. I had filed the draft of application on
> > > > GSoC site, can you please review it?
> > > >
> > > > Thanks,
> > > > Aleksey.
> > > >
> > >
> >
> >
> >
> > --
> > With best regards,
> > Alexei
> >
>
--
With best regards,
Alexei
Re: [GSoC] Appliance for harmony-gc-4 "Unify the native memory management of Harmony DRLVM"
Posted by Aleksey Shipilev <al...@gmail.com>.
Thanks, Alexei! I will try to address your concerns in next revision.
As for the style, I would prefer to have proposal technically
complete, to have the mutual expectations in place and keep in mind
what's we are really doing. Of course, I'm risking on criticizing the
exact ideas, but that's the point of this discussion, right? :)
Thanks again,
Aleksey.
On Sun, Mar 30, 2008 at 10:01 PM, Alexei Fedotov
<al...@gmail.com> wrote:
> Hello Alexei,
> Good job. I like your enthusiasm, and a technical style of your paper.
> But exposing technical details you put your ideas in risk of being
> criticized, your know. I hope that your effort would be useful
> regardless of few comments I have.
>
> > Now DRLVM components use multiple wrappers for memory management: APR pools, STD_MALLOC, PoolManager, MemoryManager, CRT malloc, VirtualAlloc and others.
> > The primary goal of this project is to implement Unified Memory Manager (UMM)
>
> You forgot portlib pools. All this reads in a following way "We have
> seven memory subsystems, and I want to implement the eighth which
> would be the best." Ok, I put the same intention into STD_MALLOC three
> years ago. We attached APR for the same reason.You suggest adding UMM.
> Amount of systems continues to grow. Contrary, I would suggest having
> less memory management subsystems on completion of your project.
>
> BTW, two important use cases are missed in your proposal. The native
> memory subsystem should continue serving critical customers such as
> lazy exception messages or finalizers even when it reported to others
> that the memory is exhausted. To prevent user's JNI code from
> exhausting memory we did not find a solution other than substitution
> of function table in C runtime library with our functions. Redirecting
> calls to our functions allows us plugging OS-level optimized memory
> managers and in this case we can continue using conventional malloc
> and free rather than inventing hymalloc.
>
> Thanks!
>
>
> On Sun, Mar 30, 2008 at 6:53 PM, Aleksey Shipilev
>
> <al...@gmail.com> wrote:
> > Hi,
> >
> > I had updated the proposal on GSoC site and made the copy on Apache Wiki [1].
> > I would appreciate anyone's comments and corrections, especially from mentors.
> >
> > Thanks,
> > Aleksey.
> >
> > [1] http://wiki.apache.org/general/AlekseyShipilev/GSoC2008/harmony-gc-4
> >
> >
> >
> > On Thu, Mar 27, 2008 at 9:10 PM, Aleksey Shipilev
> > <al...@gmail.com> wrote:
> > > Hi all, Xiao Feng, Andrey (Yakushev),
> > >
> > > I want to apply for GSoC with the idea of unification of native memory
> > > management of Harmony DRLVM. I had found some issues recently on
> > > SerialBench that stresses the memory allocation subsystem and will be
> > > glad to refactor NMM on DRLVM. I had filed the draft of application on
> > > GSoC site, can you please review it?
> > >
> > > Thanks,
> > > Aleksey.
> > >
> >
>
>
>
> --
> With best regards,
> Alexei
>
Re: [GSoC] Appliance for harmony-gc-4 "Unify the native memory management of Harmony DRLVM"
Posted by Alexei Fedotov <al...@gmail.com>.
Hello Alexei,
Good job. I like your enthusiasm, and a technical style of your paper.
But exposing technical details you put your ideas in risk of being
criticized, your know. I hope that your effort would be useful
regardless of few comments I have.
> Now DRLVM components use multiple wrappers for memory management: APR pools, STD_MALLOC, PoolManager, MemoryManager, CRT malloc, VirtualAlloc and others.
> The primary goal of this project is to implement Unified Memory Manager (UMM)
You forgot portlib pools. All this reads in a following way "We have
seven memory subsystems, and I want to implement the eighth which
would be the best." Ok, I put the same intention into STD_MALLOC three
years ago. We attached APR for the same reason.You suggest adding UMM.
Amount of systems continues to grow. Contrary, I would suggest having
less memory management subsystems on completion of your project.
BTW, two important use cases are missed in your proposal. The native
memory subsystem should continue serving critical customers such as
lazy exception messages or finalizers even when it reported to others
that the memory is exhausted. To prevent user's JNI code from
exhausting memory we did not find a solution other than substitution
of function table in C runtime library with our functions. Redirecting
calls to our functions allows us plugging OS-level optimized memory
managers and in this case we can continue using conventional malloc
and free rather than inventing hymalloc.
Thanks!
On Sun, Mar 30, 2008 at 6:53 PM, Aleksey Shipilev
<al...@gmail.com> wrote:
> Hi,
>
> I had updated the proposal on GSoC site and made the copy on Apache Wiki [1].
> I would appreciate anyone's comments and corrections, especially from mentors.
>
> Thanks,
> Aleksey.
>
> [1] http://wiki.apache.org/general/AlekseyShipilev/GSoC2008/harmony-gc-4
>
>
>
> On Thu, Mar 27, 2008 at 9:10 PM, Aleksey Shipilev
> <al...@gmail.com> wrote:
> > Hi all, Xiao Feng, Andrey (Yakushev),
> >
> > I want to apply for GSoC with the idea of unification of native memory
> > management of Harmony DRLVM. I had found some issues recently on
> > SerialBench that stresses the memory allocation subsystem and will be
> > glad to refactor NMM on DRLVM. I had filed the draft of application on
> > GSoC site, can you please review it?
> >
> > Thanks,
> > Aleksey.
> >
>
--
With best regards,
Alexei
Re: [GSoC] Appliance for harmony-gc-4 "Unify the native memory management of Harmony DRLVM"
Posted by Aleksey Shipilev <al...@gmail.com>.
Hi,
I had updated the proposal on GSoC site and made the copy on Apache Wiki [1].
I would appreciate anyone's comments and corrections, especially from mentors.
Thanks,
Aleksey.
[1] http://wiki.apache.org/general/AlekseyShipilev/GSoC2008/harmony-gc-4
On Thu, Mar 27, 2008 at 9:10 PM, Aleksey Shipilev
<al...@gmail.com> wrote:
> Hi all, Xiao Feng, Andrey (Yakushev),
>
> I want to apply for GSoC with the idea of unification of native memory
> management of Harmony DRLVM. I had found some issues recently on
> SerialBench that stresses the memory allocation subsystem and will be
> glad to refactor NMM on DRLVM. I had filed the draft of application on
> GSoC site, can you please review it?
>
> Thanks,
> Aleksey.
>