You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Xiao-Feng Li <xi...@gmail.com> on 2009/08/05 03:55:58 UTC
[GSoC] time to close
Hi, Simon, it is time to conclude your project soon. It is probably
good for you to let the community know your current status.
Thanks,
xiaofeng
--
http://people.apache.org/~xli
Re: [GSoC] time to close
Posted by Xiao-Feng Li <xi...@gmail.com>.
Yes, it makes sense. Thanks.
-xiaofeng
On Fri, Aug 21, 2009 at 10:55 AM, Simon Zhou<si...@gmail.com> wrote:
> Hi, Xiaofeng,
>
> For now, just turn on USE_UNIQUE_MARK_SWEEP_GC in gc_common.h will be OK,
> because the gc.ignore_finref is false by default.
> On the contrary, to turn off the weak reference supporting, add
> -XX:gc.ignore_finref=true in the command line will work.
> For the test cases, I am sorry I have not special test cases for this
> project, I just using specjbb and dacapo, and exam the dump information to
> perform testing.
> I think these tests can cover most cases, except they do not strickly check
> the enqueue operations.
>
> I thought about how to write java code test to strickly check the enqueue
> operations in Concurrent GC, but I found it is a little hard. The reason is
> that, the application code can not know the start time and end time of
> Concurrent GC, while the enqueue operations only should be performed after
> GC.
>
> It is a little different from the case of STW GC, in smoke test, application
> code can just call System.gc() to force a STW GC, and then check if the
> enqueue operations are right. In such a case, if the referent become
> NULL and enqueue operations are not performed, it could be regarded as a GC
> fault (Or not?). But in current design of Concurrent GC, this may really
> happend because the enqueue operations happens in the end of GC, while 'set
> NULL' happens in the process of GC.
>
> So, IMHO, there are 2 options to solve this issue, one is change the current
> design, make the 'set NULL' and 'enqueue' in a atomic transaction (which is
> not in java spec [1]) which is transparent to the application code, but
> it may impact the performance.
> Another choice is that, write special elaborate test cases which waits some
> manully-setting period after finding the referent is NULL, then check the
> enqueue operations (at this time, Concurrent GC will hopfully be finished
> and enqueue the references). But how long of the manully-setting period
> will depend on the implementation.
>
> Do you think this make sense? Thanks!
>
> [1], http://java.sun.com/javase/6/docs/api/java/lang/ref/WeakReference.html
>
> Thanks
> Simon
>
> 2009/8/20 Xiao-Feng Li <xi...@gmail.com>
>>
>> Simon, the summary looks good.
>>
>> Still, would you like to write a brief doc on how to turn on your
>> work and test it? Have you any special test cases developed for the
>> project?
>>
>> Thanks,
>> xiaofeng
>>
>> On Mon, Aug 17, 2009 at 10:47 PM, Simon Zhou<si...@gmail.com>
>> wrote:
>> > Hi Xiaofeng,
>> >
>> > I would like to give a brief summary of Concurrent GC weak reference
>> > supporting project
>> >
>> >
>> >
>> > 1, Goal
>> >
>> > The main goal of this project is to make the Harmony Concurrent GC
>> > process
>> > weak/soft/phantom reference and finalizable objects properly. Weak
>> > reference
>> > is in java spec and a important feature supported by java languages,
>> > programmers can use them to give hint to JVM to manage their
>> > applications'
>> > memory.
>> >
>> >
>> >
>> > 2, Design and implementation
>> >
>> > There are 2 major difference between weak references processing in
>> > STW/Generational GC and that of Concurrent GC.
>> >
>> >
>> >
>> > First,
>> >
>> > we must intercept all the reference.get() invokes to remember the
>> > referent
>> > whose reference is 'leaked out'.
>> >
>> >
>> >
>> > Second,
>> >
>> > the weak/soft reference processing may change the referent's
>> > reachability,
>> > and does not change other part of the object referencing graph. So we
>> > can do
>> > this job concurrently after marking is finished.
>> >
>> > On the other hand, phantom reference processing should be after
>> > finalizable
>> > objects processing, while the later one will change the object
>> > referencing
>> > graph. So we should do this job in a STW phase.
>> >
>> >
>> >
>> > For the first issue, it is necessary before we can move forward to next
>> > steps., I use 2 different technicals to catch the get() method, one is
>> > JNI
>> > implementation, which is straightforward and easy to implement, but has
>> > big
>> > overhead because of the frequent JNI calls. this implementation can be
>> > treated like a stepping-stone. The other one is a VMMagic version, which
>> > is
>> > just like the write barrier implementation in current Concurrent GC. By
>> > modify some passes of JIT, the dynamic compiler will generate a native
>> > call
>> > to the barrier function when compiling java reference.get() method. this
>> > version of barrier has a low overhead. now, it is default.
>> >
>> > For the second issue, which is the main part of this project. I uploaded
>> > a
>> > doc to HARMONY-6258, it describe some details of implementation and I
>> > think
>> > the diagram can explain better.
>> >
>> >
>> >
>> > 3, Result
>> >
>> > All the functionalities and thoughts mentioned in the proposal has been
>> > implemented, and I have submitted the code patch to HARMONY-6258.
>> >
>> > Until now, I mainly used specJbb to evaluate the result, which uses
>> > weak/soft/phantom reference and finalizable object when creating each of
>> > its
>> > working houses. Besides I use dacapo to do some short-time-run test. I
>> > used
>> > to dump the object info of get() barrier and reference processing phase.
>> > They work well and correct, and the dump information is just as
>> > expected.
>> >
>> > As we know, the evaluation of weak reference processing should include
>> > the
>> > enqueue operations. but until now, I have not found a long-time-run
>> > (because
>> > it is a concurrent GC) benchmark which focus on automatically
>> > and strictly examining when the referent should be dead (it is somehow
>> > depend on different implementations) and if dead references are all
>> > execute
>> > the weak reference enqueue operations properly. I do have planned to
>> > write
>> > such a test suite and use it to evaluate my project, but it is
>> > an elaborate work and is not ready yet. Sorry for this, I know it is
>> > important, and I will finish it before any other improvements.
>> >
>> >
>> >
>> > 4, Future work and Further ideas
>> >
>> > After the implementation of this project, I am thinking of
>> > some improvement I can do in the future.
>> >
>> > A,Benchmark or test suite which focus on weak/soft reference processing
>> > (heap usages), checking the enqueue operations, finalizing and phantom
>> > reference.
>> >
>> >
>> >
>> > B,Now, weak reference processing is done by the last concurrent marking
>> > thread. When the weak references are only a very small part of the heap,
>> > this works fine, but when there are lots of weak references to be
>> > processed
>> > in the concurrent phase, the last marker's workload is heavy. To improve
>> > the
>> > load balance, we need a parallel, concurrent weak reference processing
>> > phase
>> > just after the concurrent marking.
>> >
>> >
>> >
>> > C,I am thinking of if it is possible to implement the resurrect phase of
>> > finalizable objects processing in a concurrent phase. This may need a
>> > more
>> > complicated write barrier which will only be 'turning on' in resurrect
>> > phase
>> > and a some tracing work for the dirty objects.
>> >
>> > Thanks for your great mentoring!
>> >
>> > Thanks
>> > Simon
>> > 2009/8/17 Xiao-Feng Li <xi...@gmail.com>
>> >>
>> >> Simon, today is for final "pencil down" and starting the GSoC project
>> >> evaluation.
>> >>
>> >> I've read your doc in Harmony-6258, and reviewed the patch. They look
>> >> good to me.
>> >>
>> >> One thing I want to know is, how you tested your work? what tests did
>> >> you use to demonstrate your work is correct?
>> >>
>> >> Would you please write a doc on the steps to build and test your code?
>> >> (and upload the test cases you developed?)
>> >>
>> >> Thanks,
>> >> xiaofeng
>> >>
>> >> On Mon, Aug 10, 2009 at 8:47 PM, Xiao-Feng Li<xi...@gmail.com>
>> >> wrote:
>> >> > Simon, thanks.
>> >> >
>> >> > I will review your patch and doc. If they are ok, I guess we can call
>> >> > it a successful project. But the end goal is to commit the code
>> >> > successfully. :)
>> >> >
>> >> > Thanks,
>> >> > xiaofeng
>> >> >
>> >> > On Mon, Aug 10, 2009 at 12:50 PM, Simon Zhou<si...@gmail.com>
>> >> > wrote:
>> >> >> Hi Xiaofeng,
>> >> >>
>> >> >> I have submitted a new patch for Concurrent Weak Reference
>> >> >> project.
>> >> >> Now, Concurrent Weak Reference for mostly concurrent algorithm is
>> >> >> ready.
>> >> >> Concurrent Weak Reference for STAB algorithm is almost ready, but
>> >> >> a
>> >> >> bug
>> >> >> only when using concurrent mark & concurrent sweep running after
>> >> >> dozens
>> >> >> of
>> >> >> GC iterations, I am checking this bug to find the root cause.
>> >> >> besides these, I am planning to write a simple doc for the
>> >> >> implementation
>> >> >> of concurrent weak reference and will submit with the patch.
>> >> >> Is there any other documentations I need provide for concluding
>> >> >> this
>> >> >> project? Thanks!
>> >> >>
>> >> >> Thanks
>> >> >> Simon
>> >> >> 2009/8/5 Xiao-Feng Li <xi...@gmail.com>
>> >> >>>
>> >> >>> Simon, thanks for the update.
>> >> >>>
>> >> >>> Thanks,
>> >> >>> xiaofeng
>> >> >>>
>> >> >>> On Wed, Aug 5, 2009 at 3:03 PM, Simon Zhou<si...@gmail.com>
>> >> >>> wrote:
>> >> >>> > Hi Xiaofeng,
>> >> >>> >
>> >> >>> > Now, I have completed the WeakReference Processing (get
>> >> >>> > barrier,
>> >> >>> > weakref,
>> >> >>> > softref, finalizable, phantomref) for mostly concurrent
>> >> >>> > algorithm.
>> >> >>> > For STAB algorithms' implementation is still in debugging (get
>> >> >>> > barrier,
>> >> >>> > weakref, softref is OK).
>> >> >>> > So I will send a patch included mostly concurrent algorithm
>> >> >>> > implementation
>> >> >>> > first.
>> >> >>> >
>> >> >>> > Thanks
>> >> >>> > Simon
>> >> >>> >
>> >> >>> > 2009/8/5 Xiao-Feng Li <xi...@gmail.com>
>> >> >>> >>
>> >> >>> >> Hi, Simon, it is time to conclude your project soon. It is
>> >> >>> >> probably
>> >> >>> >> good for you to let the community know your current status.
>> >> >>> >>
>> >> >>> >> Thanks,
>> >> >>> >> xiaofeng
>> >> >>> >>
>> >> >>> >> --
>> >> >>> >> http://people.apache.org/~xli
>> >> >>> >
>> >> >>> >
>> >> >>> >
>> >> >>> > --
>> >> >>> > From : Simon.Zhou@PPI, Fudan University
>> >> >>> >
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> --
>> >> >>> http://people.apache.org/~xli
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> From : Simon.Zhou@PPI, Fudan University
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > http://people.apache.org/~xli
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> http://people.apache.org/~xli
>> >
>> >
>> >
>> > --
>> > From : Simon.Zhou@PPI, Fudan University
>> >
>>
>>
>>
>> --
>> http://people.apache.org/~xli
>
>
>
> --
> From : Simon.Zhou@PPI, Fudan University
>
--
http://people.apache.org/~xli
Re: [GSoC] time to close
Posted by Simon Zhou <si...@gmail.com>.
Hi, Xiaofeng,
For now, just turn on USE_UNIQUE_MARK_SWEEP_GC in gc_common.h will be OK,
because the gc.ignore_finref is false by default.
On the contrary, to turn off the weak reference supporting, add
-XX:gc.ignore_finref=true in the command line will work.
For the test cases, I am sorry I have not special test cases for this
project, I just using specjbb and dacapo, and exam the dump information to
perform testing.
I think these tests can cover most cases, except they do not strickly check
the enqueue operations.
I thought about how to write java code test to strickly check the enqueue
operations in Concurrent GC, but I found it is a little hard. The reason is
that, the application code can not know the start time and end time of
Concurrent GC, while the enqueue operations only should be performed after
GC.
It is a little different from the case of STW GC, in smoke test, application
code can just call System.gc() to force a STW GC, and then check if the
enqueue operations are right. In such a case, if the referent become
NULL and enqueue operations are not performed, it could be regarded as a GC
fault (Or not?). But in current design of Concurrent GC, this may really
happend because the enqueue operations happens in the end of GC, while 'set
NULL' happens in the process of GC.
So, IMHO, there are 2 options to solve this issue, one is change the current
design, make the 'set NULL' and 'enqueue' in a atomic transaction (which is
not in java spec [1]) which is transparent to the application code, but
it may impact the performance.
Another choice is that, write special elaborate test cases which waits some
manully-setting period after finding the referent is NULL, then check the
enqueue operations (at this time, Concurrent GC will hopfully be finished
and enqueue the references). But how long of the manully-setting period
will depend on the implementation.
Do you think this make sense? Thanks!
[1], http://java.sun.com/javase/6/docs/api/java/lang/ref/WeakReference.html<http://java.sun.com/javase/6/docs/api/index.html>
Thanks
Simon
2009/8/20 Xiao-Feng Li <xi...@gmail.com>
> Simon, the summary looks good.
>
> Still, would you like to write a brief doc on how to turn on your
> work and test it? Have you any special test cases developed for the
> project?
>
> Thanks,
> xiaofeng
>
> On Mon, Aug 17, 2009 at 10:47 PM, Simon Zhou<si...@gmail.com>
> wrote:
> > Hi Xiaofeng,
> >
> > I would like to give a brief summary of Concurrent GC weak reference
> > supporting project
> >
> >
> >
> > 1, Goal
> >
> > The main goal of this project is to make the Harmony Concurrent GC
> process
> > weak/soft/phantom reference and finalizable objects properly. Weak
> reference
> > is in java spec and a important feature supported by java languages,
> > programmers can use them to give hint to JVM to manage their
> applications'
> > memory.
> >
> >
> >
> > 2, Design and implementation
> >
> > There are 2 major difference between weak references processing in
> > STW/Generational GC and that of Concurrent GC.
> >
> >
> >
> > First,
> >
> > we must intercept all the reference.get() invokes to remember the
> referent
> > whose reference is 'leaked out'.
> >
> >
> >
> > Second,
> >
> > the weak/soft reference processing may change the referent's
> reachability,
> > and does not change other part of the object referencing graph. So we can
> do
> > this job concurrently after marking is finished.
> >
> > On the other hand, phantom reference processing should be after
> finalizable
> > objects processing, while the later one will change the object
> referencing
> > graph. So we should do this job in a STW phase.
> >
> >
> >
> > For the first issue, it is necessary before we can move forward to next
> > steps., I use 2 different technicals to catch the get() method, one is
> JNI
> > implementation, which is straightforward and easy to implement, but has
> big
> > overhead because of the frequent JNI calls. this implementation can be
> > treated like a stepping-stone. The other one is a VMMagic version, which
> is
> > just like the write barrier implementation in current Concurrent GC. By
> > modify some passes of JIT, the dynamic compiler will generate a native
> call
> > to the barrier function when compiling java reference.get() method. this
> > version of barrier has a low overhead. now, it is default.
> >
> > For the second issue, which is the main part of this project. I uploaded
> a
> > doc to HARMONY-6258, it describe some details of implementation and I
> think
> > the diagram can explain better.
> >
> >
> >
> > 3, Result
> >
> > All the functionalities and thoughts mentioned in the proposal has been
> > implemented, and I have submitted the code patch to HARMONY-6258.
> >
> > Until now, I mainly used specJbb to evaluate the result, which uses
> > weak/soft/phantom reference and finalizable object when creating each of
> its
> > working houses. Besides I use dacapo to do some short-time-run test. I
> used
> > to dump the object info of get() barrier and reference processing phase.
> > They work well and correct, and the dump information is just as expected.
> >
> > As we know, the evaluation of weak reference processing should include
> the
> > enqueue operations. but until now, I have not found a long-time-run
> (because
> > it is a concurrent GC) benchmark which focus on automatically
> > and strictly examining when the referent should be dead (it is somehow
> > depend on different implementations) and if dead references are all
> execute
> > the weak reference enqueue operations properly. I do have planned to
> write
> > such a test suite and use it to evaluate my project, but it is
> > an elaborate work and is not ready yet. Sorry for this, I know it is
> > important, and I will finish it before any other improvements.
> >
> >
> >
> > 4, Future work and Further ideas
> >
> > After the implementation of this project, I am thinking of
> > some improvement I can do in the future.
> >
> > A,Benchmark or test suite which focus on weak/soft reference processing
> > (heap usages), checking the enqueue operations, finalizing and phantom
> > reference.
> >
> >
> >
> > B,Now, weak reference processing is done by the last concurrent marking
> > thread. When the weak references are only a very small part of the heap,
> > this works fine, but when there are lots of weak references to be
> processed
> > in the concurrent phase, the last marker's workload is heavy. To improve
> the
> > load balance, we need a parallel, concurrent weak reference processing
> phase
> > just after the concurrent marking.
> >
> >
> >
> > C,I am thinking of if it is possible to implement the resurrect phase of
> > finalizable objects processing in a concurrent phase. This may need a
> more
> > complicated write barrier which will only be 'turning on' in resurrect
> phase
> > and a some tracing work for the dirty objects.
> >
> > Thanks for your great mentoring!
> >
> > Thanks
> > Simon
> > 2009/8/17 Xiao-Feng Li <xi...@gmail.com>
> >>
> >> Simon, today is for final "pencil down" and starting the GSoC project
> >> evaluation.
> >>
> >> I've read your doc in Harmony-6258, and reviewed the patch. They look
> >> good to me.
> >>
> >> One thing I want to know is, how you tested your work? what tests did
> >> you use to demonstrate your work is correct?
> >>
> >> Would you please write a doc on the steps to build and test your code?
> >> (and upload the test cases you developed?)
> >>
> >> Thanks,
> >> xiaofeng
> >>
> >> On Mon, Aug 10, 2009 at 8:47 PM, Xiao-Feng Li<xi...@gmail.com>
> >> wrote:
> >> > Simon, thanks.
> >> >
> >> > I will review your patch and doc. If they are ok, I guess we can call
> >> > it a successful project. But the end goal is to commit the code
> >> > successfully. :)
> >> >
> >> > Thanks,
> >> > xiaofeng
> >> >
> >> > On Mon, Aug 10, 2009 at 12:50 PM, Simon Zhou<si...@gmail.com>
> >> > wrote:
> >> >> Hi Xiaofeng,
> >> >>
> >> >> I have submitted a new patch for Concurrent Weak Reference project.
> >> >> Now, Concurrent Weak Reference for mostly concurrent algorithm is
> >> >> ready.
> >> >> Concurrent Weak Reference for STAB algorithm is almost ready, but a
> >> >> bug
> >> >> only when using concurrent mark & concurrent sweep running after
> dozens
> >> >> of
> >> >> GC iterations, I am checking this bug to find the root cause.
> >> >> besides these, I am planning to write a simple doc for the
> >> >> implementation
> >> >> of concurrent weak reference and will submit with the patch.
> >> >> Is there any other documentations I need provide for concluding
> this
> >> >> project? Thanks!
> >> >>
> >> >> Thanks
> >> >> Simon
> >> >> 2009/8/5 Xiao-Feng Li <xi...@gmail.com>
> >> >>>
> >> >>> Simon, thanks for the update.
> >> >>>
> >> >>> Thanks,
> >> >>> xiaofeng
> >> >>>
> >> >>> On Wed, Aug 5, 2009 at 3:03 PM, Simon Zhou<si...@gmail.com>
> >> >>> wrote:
> >> >>> > Hi Xiaofeng,
> >> >>> >
> >> >>> > Now, I have completed the WeakReference Processing (get barrier,
> >> >>> > weakref,
> >> >>> > softref, finalizable, phantomref) for mostly concurrent algorithm.
> >> >>> > For STAB algorithms' implementation is still in debugging (get
> >> >>> > barrier,
> >> >>> > weakref, softref is OK).
> >> >>> > So I will send a patch included mostly concurrent algorithm
> >> >>> > implementation
> >> >>> > first.
> >> >>> >
> >> >>> > Thanks
> >> >>> > Simon
> >> >>> >
> >> >>> > 2009/8/5 Xiao-Feng Li <xi...@gmail.com>
> >> >>> >>
> >> >>> >> Hi, Simon, it is time to conclude your project soon. It is
> probably
> >> >>> >> good for you to let the community know your current status.
> >> >>> >>
> >> >>> >> Thanks,
> >> >>> >> xiaofeng
> >> >>> >>
> >> >>> >> --
> >> >>> >> http://people.apache.org/~xli
> >> >>> >
> >> >>> >
> >> >>> >
> >> >>> > --
> >> >>> > From : Simon.Zhou@PPI, Fudan University
> >> >>> >
> >> >>>
> >> >>>
> >> >>>
> >> >>> --
> >> >>> http://people.apache.org/~xli
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> From : Simon.Zhou@PPI, Fudan University
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > http://people.apache.org/~xli
> >> >
> >>
> >>
> >>
> >> --
> >> http://people.apache.org/~xli
> >
> >
> >
> > --
> > From : Simon.Zhou@PPI, Fudan University
> >
>
>
>
> --
> http://people.apache.org/~xli
>
--
>From : Simon.Zhou@PPI, Fudan University
Re: [GSoC] time to close
Posted by Xiao-Feng Li <xi...@gmail.com>.
Simon, the summary looks good.
Still, would you like to write a brief doc on how to turn on your
work and test it? Have you any special test cases developed for the
project?
Thanks,
xiaofeng
On Mon, Aug 17, 2009 at 10:47 PM, Simon Zhou<si...@gmail.com> wrote:
> Hi Xiaofeng,
>
> I would like to give a brief summary of Concurrent GC weak reference
> supporting project
>
>
>
> 1, Goal
>
> The main goal of this project is to make the Harmony Concurrent GC process
> weak/soft/phantom reference and finalizable objects properly. Weak reference
> is in java spec and a important feature supported by java languages,
> programmers can use them to give hint to JVM to manage their applications'
> memory.
>
>
>
> 2, Design and implementation
>
> There are 2 major difference between weak references processing in
> STW/Generational GC and that of Concurrent GC.
>
>
>
> First,
>
> we must intercept all the reference.get() invokes to remember the referent
> whose reference is 'leaked out'.
>
>
>
> Second,
>
> the weak/soft reference processing may change the referent's reachability,
> and does not change other part of the object referencing graph. So we can do
> this job concurrently after marking is finished.
>
> On the other hand, phantom reference processing should be after finalizable
> objects processing, while the later one will change the object referencing
> graph. So we should do this job in a STW phase.
>
>
>
> For the first issue, it is necessary before we can move forward to next
> steps., I use 2 different technicals to catch the get() method, one is JNI
> implementation, which is straightforward and easy to implement, but has big
> overhead because of the frequent JNI calls. this implementation can be
> treated like a stepping-stone. The other one is a VMMagic version, which is
> just like the write barrier implementation in current Concurrent GC. By
> modify some passes of JIT, the dynamic compiler will generate a native call
> to the barrier function when compiling java reference.get() method. this
> version of barrier has a low overhead. now, it is default.
>
> For the second issue, which is the main part of this project. I uploaded a
> doc to HARMONY-6258, it describe some details of implementation and I think
> the diagram can explain better.
>
>
>
> 3, Result
>
> All the functionalities and thoughts mentioned in the proposal has been
> implemented, and I have submitted the code patch to HARMONY-6258.
>
> Until now, I mainly used specJbb to evaluate the result, which uses
> weak/soft/phantom reference and finalizable object when creating each of its
> working houses. Besides I use dacapo to do some short-time-run test. I used
> to dump the object info of get() barrier and reference processing phase.
> They work well and correct, and the dump information is just as expected.
>
> As we know, the evaluation of weak reference processing should include the
> enqueue operations. but until now, I have not found a long-time-run (because
> it is a concurrent GC) benchmark which focus on automatically
> and strictly examining when the referent should be dead (it is somehow
> depend on different implementations) and if dead references are all execute
> the weak reference enqueue operations properly. I do have planned to write
> such a test suite and use it to evaluate my project, but it is
> an elaborate work and is not ready yet. Sorry for this, I know it is
> important, and I will finish it before any other improvements.
>
>
>
> 4, Future work and Further ideas
>
> After the implementation of this project, I am thinking of
> some improvement I can do in the future.
>
> A,Benchmark or test suite which focus on weak/soft reference processing
> (heap usages), checking the enqueue operations, finalizing and phantom
> reference.
>
>
>
> B,Now, weak reference processing is done by the last concurrent marking
> thread. When the weak references are only a very small part of the heap,
> this works fine, but when there are lots of weak references to be processed
> in the concurrent phase, the last marker's workload is heavy. To improve the
> load balance, we need a parallel, concurrent weak reference processing phase
> just after the concurrent marking.
>
>
>
> C,I am thinking of if it is possible to implement the resurrect phase of
> finalizable objects processing in a concurrent phase. This may need a more
> complicated write barrier which will only be 'turning on' in resurrect phase
> and a some tracing work for the dirty objects.
>
> Thanks for your great mentoring!
>
> Thanks
> Simon
> 2009/8/17 Xiao-Feng Li <xi...@gmail.com>
>>
>> Simon, today is for final "pencil down" and starting the GSoC project
>> evaluation.
>>
>> I've read your doc in Harmony-6258, and reviewed the patch. They look
>> good to me.
>>
>> One thing I want to know is, how you tested your work? what tests did
>> you use to demonstrate your work is correct?
>>
>> Would you please write a doc on the steps to build and test your code?
>> (and upload the test cases you developed?)
>>
>> Thanks,
>> xiaofeng
>>
>> On Mon, Aug 10, 2009 at 8:47 PM, Xiao-Feng Li<xi...@gmail.com>
>> wrote:
>> > Simon, thanks.
>> >
>> > I will review your patch and doc. If they are ok, I guess we can call
>> > it a successful project. But the end goal is to commit the code
>> > successfully. :)
>> >
>> > Thanks,
>> > xiaofeng
>> >
>> > On Mon, Aug 10, 2009 at 12:50 PM, Simon Zhou<si...@gmail.com>
>> > wrote:
>> >> Hi Xiaofeng,
>> >>
>> >> I have submitted a new patch for Concurrent Weak Reference project.
>> >> Now, Concurrent Weak Reference for mostly concurrent algorithm is
>> >> ready.
>> >> Concurrent Weak Reference for STAB algorithm is almost ready, but a
>> >> bug
>> >> only when using concurrent mark & concurrent sweep running after dozens
>> >> of
>> >> GC iterations, I am checking this bug to find the root cause.
>> >> besides these, I am planning to write a simple doc for the
>> >> implementation
>> >> of concurrent weak reference and will submit with the patch.
>> >> Is there any other documentations I need provide for concluding this
>> >> project? Thanks!
>> >>
>> >> Thanks
>> >> Simon
>> >> 2009/8/5 Xiao-Feng Li <xi...@gmail.com>
>> >>>
>> >>> Simon, thanks for the update.
>> >>>
>> >>> Thanks,
>> >>> xiaofeng
>> >>>
>> >>> On Wed, Aug 5, 2009 at 3:03 PM, Simon Zhou<si...@gmail.com>
>> >>> wrote:
>> >>> > Hi Xiaofeng,
>> >>> >
>> >>> > Now, I have completed the WeakReference Processing (get barrier,
>> >>> > weakref,
>> >>> > softref, finalizable, phantomref) for mostly concurrent algorithm.
>> >>> > For STAB algorithms' implementation is still in debugging (get
>> >>> > barrier,
>> >>> > weakref, softref is OK).
>> >>> > So I will send a patch included mostly concurrent algorithm
>> >>> > implementation
>> >>> > first.
>> >>> >
>> >>> > Thanks
>> >>> > Simon
>> >>> >
>> >>> > 2009/8/5 Xiao-Feng Li <xi...@gmail.com>
>> >>> >>
>> >>> >> Hi, Simon, it is time to conclude your project soon. It is probably
>> >>> >> good for you to let the community know your current status.
>> >>> >>
>> >>> >> Thanks,
>> >>> >> xiaofeng
>> >>> >>
>> >>> >> --
>> >>> >> http://people.apache.org/~xli
>> >>> >
>> >>> >
>> >>> >
>> >>> > --
>> >>> > From : Simon.Zhou@PPI, Fudan University
>> >>> >
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> http://people.apache.org/~xli
>> >>
>> >>
>> >>
>> >> --
>> >> From : Simon.Zhou@PPI, Fudan University
>> >>
>> >
>> >
>> >
>> > --
>> > http://people.apache.org/~xli
>> >
>>
>>
>>
>> --
>> http://people.apache.org/~xli
>
>
>
> --
> From : Simon.Zhou@PPI, Fudan University
>
--
http://people.apache.org/~xli
Re: [GSoC] time to close
Posted by Simon Zhou <si...@gmail.com>.
Hi Xiaofeng,
I would like to give a brief summary of Concurrent GC weak reference
supporting project
1, Goal
The main goal of this project is to make the Harmony Concurrent GC process
weak/soft/phantom reference and finalizable objects properly. Weak reference
is in java spec and a important feature supported by java languages,
programmers can use them to give hint to JVM to manage their applications'
memory.
2, Design and implementation
There are 2 major difference between weak references processing in
STW/Generational GC and that of Concurrent GC.
First,
we must intercept all the reference.get() invokes to remember the referent
whose reference is 'leaked out'.
Second,
the weak/soft reference processing may change the referent's reachability,
and does not change other part of the object referencing graph. So we can do
this job concurrently after marking is finished.
On the other hand, phantom reference processing should be after finalizable
objects processing, while the later one will change the object referencing
graph. So we should do this job in a STW phase.
For the first issue, it is necessary before we can move forward to next
steps., I use 2 different technicals to catch the get() method, one is JNI
implementation, which is straightforward and easy to implement, but has big
overhead because of the frequent JNI calls. this implementation can be
treated like a stepping-stone. The other one is a VMMagic version, which is
just like the write barrier implementation in current Concurrent GC. By
modify some passes of JIT, the dynamic compiler will generate a native call
to the barrier function when compiling java reference.get() method. this
version of barrier has a low overhead. now, it is default.
For the second issue, which is the main part of this project. I uploaded a
doc to HARMONY-6258, it describe some details of implementation and I think
the diagram can explain better.
3, Result
All the functionalities and thoughts mentioned in the proposal has been
implemented, and I have submitted the code patch to HARMONY-6258.
Until now, I mainly used specJbb to evaluate the result, which uses
weak/soft/phantom reference and finalizable object when creating each of its
working houses. Besides I use dacapo to do some short-time-run test. I used
to dump the object info of get() barrier and reference processing phase.
They work well and correct, and the dump information is just as expected.
As we know, the evaluation of weak reference processing should include the
enqueue operations. but until now, I have not found a long-time-run (because
it is a concurrent GC) benchmark which focus on automatically
and strictly examining when the referent should be dead (it is somehow
depend on different implementations) and if dead references are all execute
the weak reference enqueue operations properly. I do have planned to write
such a test suite and use it to evaluate my project, but it is an
elaborate work
and is not ready yet. Sorry for this, I know it is important, and I will
finish it before any other improvements.
4, Future work and Further ideas
After the implementation of this project, I am thinking of
some improvement I can do in the future.
A,Benchmark or test suite which focus on weak/soft reference processing
(heap usages), checking the enqueue operations, finalizing and phantom
reference.
B,Now, weak reference processing is done by the last concurrent marking
thread. When the weak references are only a very small part of the heap,
this works fine, but when there are lots of weak references to be processed
in the concurrent phase, the last marker's workload is heavy. To improve the
load balance, we need a parallel, concurrent weak reference processing phase
just after the concurrent marking.
C,I am thinking of if it is possible to implement the resurrect phase of
finalizable objects processing in a concurrent phase. This may need a more
complicated write barrier which will only be 'turning on' in resurrect phase
and a some tracing work for the dirty objects.
Thanks for your great mentoring!
ThanksSimon
2009/8/17 Xiao-Feng Li <xi...@gmail.com>
> Simon, today is for final "pencil down" and starting the GSoC project
> evaluation.
>
> I've read your doc in Harmony-6258, and reviewed the patch. They look
> good to me.
>
> One thing I want to know is, how you tested your work? what tests did
> you use to demonstrate your work is correct?
>
> Would you please write a doc on the steps to build and test your code?
> (and upload the test cases you developed?)
>
> Thanks,
> xiaofeng
>
> On Mon, Aug 10, 2009 at 8:47 PM, Xiao-Feng Li<xi...@gmail.com>
> wrote:
> > Simon, thanks.
> >
> > I will review your patch and doc. If they are ok, I guess we can call
> > it a successful project. But the end goal is to commit the code
> > successfully. :)
> >
> > Thanks,
> > xiaofeng
> >
> > On Mon, Aug 10, 2009 at 12:50 PM, Simon Zhou<si...@gmail.com>
> wrote:
> >> Hi Xiaofeng,
> >>
> >> I have submitted a new patch for Concurrent Weak Reference project.
> >> Now, Concurrent Weak Reference for mostly concurrent algorithm is
> ready.
> >> Concurrent Weak Reference for STAB algorithm is almost ready, but a
> bug
> >> only when using concurrent mark & concurrent sweep running after dozens
> of
> >> GC iterations, I am checking this bug to find the root cause.
> >> besides these, I am planning to write a simple doc for the
> implementation
> >> of concurrent weak reference and will submit with the patch.
> >> Is there any other documentations I need provide for concluding this
> >> project? Thanks!
> >>
> >> Thanks
> >> Simon
> >> 2009/8/5 Xiao-Feng Li <xi...@gmail.com>
> >>>
> >>> Simon, thanks for the update.
> >>>
> >>> Thanks,
> >>> xiaofeng
> >>>
> >>> On Wed, Aug 5, 2009 at 3:03 PM, Simon Zhou<si...@gmail.com>
> wrote:
> >>> > Hi Xiaofeng,
> >>> >
> >>> > Now, I have completed the WeakReference Processing (get barrier,
> >>> > weakref,
> >>> > softref, finalizable, phantomref) for mostly concurrent algorithm.
> >>> > For STAB algorithms' implementation is still in debugging (get
> >>> > barrier,
> >>> > weakref, softref is OK).
> >>> > So I will send a patch included mostly concurrent algorithm
> >>> > implementation
> >>> > first.
> >>> >
> >>> > Thanks
> >>> > Simon
> >>> >
> >>> > 2009/8/5 Xiao-Feng Li <xi...@gmail.com>
> >>> >>
> >>> >> Hi, Simon, it is time to conclude your project soon. It is probably
> >>> >> good for you to let the community know your current status.
> >>> >>
> >>> >> Thanks,
> >>> >> xiaofeng
> >>> >>
> >>> >> --
> >>> >> http://people.apache.org/~xli
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > From : Simon.Zhou@PPI, Fudan University
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> http://people.apache.org/~xli
> >>
> >>
> >>
> >> --
> >> From : Simon.Zhou@PPI, Fudan University
> >>
> >
> >
> >
> > --
> > http://people.apache.org/~xli
> >
>
>
>
> --
> http://people.apache.org/~xli
>
--
>From : Simon.Zhou@PPI, Fudan University
Re: [GSoC] time to close
Posted by Xiao-Feng Li <xi...@gmail.com>.
Simon, today is for final "pencil down" and starting the GSoC project
evaluation.
I've read your doc in Harmony-6258, and reviewed the patch. They look
good to me.
One thing I want to know is, how you tested your work? what tests did
you use to demonstrate your work is correct?
Would you please write a doc on the steps to build and test your code?
(and upload the test cases you developed?)
Thanks,
xiaofeng
On Mon, Aug 10, 2009 at 8:47 PM, Xiao-Feng Li<xi...@gmail.com> wrote:
> Simon, thanks.
>
> I will review your patch and doc. If they are ok, I guess we can call
> it a successful project. But the end goal is to commit the code
> successfully. :)
>
> Thanks,
> xiaofeng
>
> On Mon, Aug 10, 2009 at 12:50 PM, Simon Zhou<si...@gmail.com> wrote:
>> Hi Xiaofeng,
>>
>> I have submitted a new patch for Concurrent Weak Reference project.
>> Now, Concurrent Weak Reference for mostly concurrent algorithm is ready.
>> Concurrent Weak Reference for STAB algorithm is almost ready, but a bug
>> only when using concurrent mark & concurrent sweep running after dozens of
>> GC iterations, I am checking this bug to find the root cause.
>> besides these, I am planning to write a simple doc for the implementation
>> of concurrent weak reference and will submit with the patch.
>> Is there any other documentations I need provide for concluding this
>> project? Thanks!
>>
>> Thanks
>> Simon
>> 2009/8/5 Xiao-Feng Li <xi...@gmail.com>
>>>
>>> Simon, thanks for the update.
>>>
>>> Thanks,
>>> xiaofeng
>>>
>>> On Wed, Aug 5, 2009 at 3:03 PM, Simon Zhou<si...@gmail.com> wrote:
>>> > Hi Xiaofeng,
>>> >
>>> > Now, I have completed the WeakReference Processing (get barrier,
>>> > weakref,
>>> > softref, finalizable, phantomref) for mostly concurrent algorithm.
>>> > For STAB algorithms' implementation is still in debugging (get
>>> > barrier,
>>> > weakref, softref is OK).
>>> > So I will send a patch included mostly concurrent algorithm
>>> > implementation
>>> > first.
>>> >
>>> > Thanks
>>> > Simon
>>> >
>>> > 2009/8/5 Xiao-Feng Li <xi...@gmail.com>
>>> >>
>>> >> Hi, Simon, it is time to conclude your project soon. It is probably
>>> >> good for you to let the community know your current status.
>>> >>
>>> >> Thanks,
>>> >> xiaofeng
>>> >>
>>> >> --
>>> >> http://people.apache.org/~xli
>>> >
>>> >
>>> >
>>> > --
>>> > From : Simon.Zhou@PPI, Fudan University
>>> >
>>>
>>>
>>>
>>> --
>>> http://people.apache.org/~xli
>>
>>
>>
>> --
>> From : Simon.Zhou@PPI, Fudan University
>>
>
>
>
> --
> http://people.apache.org/~xli
>
--
http://people.apache.org/~xli
Re: [GSoC] time to close
Posted by Xiao-Feng Li <xi...@gmail.com>.
Simon, thanks.
I will review your patch and doc. If they are ok, I guess we can call
it a successful project. But the end goal is to commit the code
successfully. :)
Thanks,
xiaofeng
On Mon, Aug 10, 2009 at 12:50 PM, Simon Zhou<si...@gmail.com> wrote:
> Hi Xiaofeng,
>
> I have submitted a new patch for Concurrent Weak Reference project.
> Now, Concurrent Weak Reference for mostly concurrent algorithm is ready.
> Concurrent Weak Reference for STAB algorithm is almost ready, but a bug
> only when using concurrent mark & concurrent sweep running after dozens of
> GC iterations, I am checking this bug to find the root cause.
> besides these, I am planning to write a simple doc for the implementation
> of concurrent weak reference and will submit with the patch.
> Is there any other documentations I need provide for concluding this
> project? Thanks!
>
> Thanks
> Simon
> 2009/8/5 Xiao-Feng Li <xi...@gmail.com>
>>
>> Simon, thanks for the update.
>>
>> Thanks,
>> xiaofeng
>>
>> On Wed, Aug 5, 2009 at 3:03 PM, Simon Zhou<si...@gmail.com> wrote:
>> > Hi Xiaofeng,
>> >
>> > Now, I have completed the WeakReference Processing (get barrier,
>> > weakref,
>> > softref, finalizable, phantomref) for mostly concurrent algorithm.
>> > For STAB algorithms' implementation is still in debugging (get
>> > barrier,
>> > weakref, softref is OK).
>> > So I will send a patch included mostly concurrent algorithm
>> > implementation
>> > first.
>> >
>> > Thanks
>> > Simon
>> >
>> > 2009/8/5 Xiao-Feng Li <xi...@gmail.com>
>> >>
>> >> Hi, Simon, it is time to conclude your project soon. It is probably
>> >> good for you to let the community know your current status.
>> >>
>> >> Thanks,
>> >> xiaofeng
>> >>
>> >> --
>> >> http://people.apache.org/~xli
>> >
>> >
>> >
>> > --
>> > From : Simon.Zhou@PPI, Fudan University
>> >
>>
>>
>>
>> --
>> http://people.apache.org/~xli
>
>
>
> --
> From : Simon.Zhou@PPI, Fudan University
>
--
http://people.apache.org/~xli
Re: [GSoC] time to close
Posted by Simon Zhou <si...@gmail.com>.
Hi Xiaofeng,
I have submitted a new patch for Concurrent Weak Reference project.
Now, Concurrent Weak Reference for mostly concurrent algorithm is ready.
Concurrent Weak Reference for STAB algorithm is almost ready, but a bug
only when using concurrent mark & concurrent sweep running after dozens of
GC iterations, I am checking this bug to find the root cause.
besides these, I am planning to write a simple doc for the implementation
of concurrent weak reference and will submit with the patch.
Is there any other documentations I need provide for concluding this
project? Thanks!
Thanks
Simon
2009/8/5 Xiao-Feng Li <xi...@gmail.com>
> Simon, thanks for the update.
>
> Thanks,
> xiaofeng
>
> On Wed, Aug 5, 2009 at 3:03 PM, Simon Zhou<si...@gmail.com> wrote:
> > Hi Xiaofeng,
> >
> > Now, I have completed the WeakReference Processing (get barrier,
> weakref,
> > softref, finalizable, phantomref) for mostly concurrent algorithm.
> > For STAB algorithms' implementation is still in debugging (get barrier,
> > weakref, softref is OK).
> > So I will send a patch included mostly concurrent algorithm
> implementation
> > first.
> >
> > Thanks
> > Simon
> >
> > 2009/8/5 Xiao-Feng Li <xi...@gmail.com>
> >>
> >> Hi, Simon, it is time to conclude your project soon. It is probably
> >> good for you to let the community know your current status.
> >>
> >> Thanks,
> >> xiaofeng
> >>
> >> --
> >> http://people.apache.org/~xli
> >
> >
> >
> > --
> > From : Simon.Zhou@PPI, Fudan University
> >
>
>
>
> --
> http://people.apache.org/~xli
>
--
>From : Simon.Zhou@PPI, Fudan University
Re: [GSoC] time to close
Posted by Xiao-Feng Li <xi...@gmail.com>.
Simon, thanks for the update.
Thanks,
xiaofeng
On Wed, Aug 5, 2009 at 3:03 PM, Simon Zhou<si...@gmail.com> wrote:
> Hi Xiaofeng,
>
> Now, I have completed the WeakReference Processing (get barrier, weakref,
> softref, finalizable, phantomref) for mostly concurrent algorithm.
> For STAB algorithms' implementation is still in debugging (get barrier,
> weakref, softref is OK).
> So I will send a patch included mostly concurrent algorithm implementation
> first.
>
> Thanks
> Simon
>
> 2009/8/5 Xiao-Feng Li <xi...@gmail.com>
>>
>> Hi, Simon, it is time to conclude your project soon. It is probably
>> good for you to let the community know your current status.
>>
>> Thanks,
>> xiaofeng
>>
>> --
>> http://people.apache.org/~xli
>
>
>
> --
> From : Simon.Zhou@PPI, Fudan University
>
--
http://people.apache.org/~xli
Re: [GSoC] time to close
Posted by Simon Zhou <si...@gmail.com>.
Hi Xiaofeng,
Now, I have completed the WeakReference Processing (get barrier, weakref,
softref, finalizable, phantomref) for mostly concurrent algorithm.
For STAB algorithms' implementation is still in debugging (get barrier,
weakref, softref is OK).
So I will send a patch included mostly concurrent algorithm implementation
first.
Thanks
Simon
2009/8/5 Xiao-Feng Li <xi...@gmail.com>
> Hi, Simon, it is time to conclude your project soon. It is probably
> good for you to let the community know your current status.
>
> Thanks,
> xiaofeng
>
> --
> http://people.apache.org/~xli
>
--
>From : Simon.Zhou@PPI, Fudan University