You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by "Paulex Yang (JIRA)" <ji...@apache.org> on 2006/01/23 12:33:09 UTC
[jira] Created: (HARMONY-38)
com.ibm.platform.struct.AbstractMemorySpy doesn't actually autoFree native
memory
com.ibm.platform.struct.AbstractMemorySpy doesn't actually autoFree native memory
---------------------------------------------------------------------------------
Key: HARMONY-38
URL: http://issues.apache.org/jira/browse/HARMONY-38
Project: Harmony
Type: Bug
Components: Classlib
Reporter: Paulex Yang
Priority: Critical
com.ibm.platform.struct.IMemorySpy is a memory watcher, and the AbstractMemorySpy is a default implementation working with nio direct buffers to free the native memories when direct buffer instances are garbage collected. But currently its auto free donsn't work.
Look inside the orphanedMemory(Object obj) method, the memory is freed only if the local variable wrapper can be found from the memoryInUse, a Map which records the native memories used by direct buffers, but actually it will be removed when autoFree(PlatfromAddress) is invoked. Checking the autoFree() reference, I found that all DirectByteBuffer will invoke autoFree() in constructor, which means no memory will be freed.
A simple fix:
modify the Ln 94 in autoFree() from:
wrapper = (AddressWrapper) memoryInUse.remove(address);
to:
wrapper = (AddressWrapper) memoryInUse.get(address);
and it should work.
The steps to reproduce the bug:
1. Add the following line in the the Ln109, which indicates the memory is auto freed by AbstractMemorySpy
System.out.println("MemorySpy autofree:"+wrapper.shadow);
2. Run the test below:
public class MemorySpyTest {
public static void main(String[] args) throws Exception {
for (int i = 0; i < 1000; i++) {
try{
ByteBuffer buffer = ByteBuffer.allocateDirect(1024);
buffer = null;
}catch(Exception e){
e.printStackTrace();
}
System.gc();
}
}
Before modification: application exits silently.
After modification, hundres of lines printed out, like this:
.....
MemorySpy autofree:PlatformAddress[9310024]
MemorySpy autofree:PlatformAddress[9311056]
MemorySpy autofree:PlatformAddress[9312088]
MemorySpy autofree:PlatformAddress[9313120]
MemorySpy autofree:PlatformAddress[9314152]
MemorySpy autofree:PlatformAddress[9315184]
MemorySpy autofree:PlatformAddress[9316216]
MemorySpy autofree:PlatformAddress[9317248]
.......
Another possible pitfall in AbstractMemorySpy is it uses WeakReference and ReferenceQueue to get notification when the buffers are ready to be garbage collected. But it is not so safe yet to free the native memory, because the weakly-reachable object has its chance to be resurrected to strongly-reachable, if only some finalize() method doing this is executed. A safer solution is to use PhantomReference instead, because phantom-reachable object has no chance to survive. The only concern is the PhantomReference should be explicitly cleared by application, but that's OK because the orphanedMemory() method is a perfect candidate to do this.
I will attach the patch soon.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HARMONY-38)
com.ibm.platform.struct.AbstractMemorySpy doesn't actually autoFree native
memory
Posted by "Paulex Yang (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-38?page=comments#action_12363944 ]
Paulex Yang commented on HARMONY-38:
------------------------------------
Sorry, there is a typo on my comment above, I meant the patches for AbstractMemorySpy and RuntimeMemorySpy, as what the attachments' names imply
> com.ibm.platform.struct.AbstractMemorySpy doesn't actually autoFree native memory
> ---------------------------------------------------------------------------------
>
> Key: HARMONY-38
> URL: http://issues.apache.org/jira/browse/HARMONY-38
> Project: Harmony
> Type: Bug
> Components: Classlib
> Reporter: Paulex Yang
> Assignee: Tim Ellison
> Priority: Critical
> Attachments: Harmony38-AbstractMemorySpy-patch.txt, Harmony38-RuntimeMemorySpy-patch.txt
>
> com.ibm.platform.struct.IMemorySpy is a memory watcher, and the AbstractMemorySpy is a default implementation working with nio direct buffers to free the native memories when direct buffer instances are garbage collected. But currently its auto free donsn't work.
> Look inside the orphanedMemory(Object obj) method, the memory is freed only if the local variable wrapper can be found from the memoryInUse, a Map which records the native memories used by direct buffers, but actually it will be removed when autoFree(PlatfromAddress) is invoked. Checking the autoFree() reference, I found that all DirectByteBuffer will invoke autoFree() in constructor, which means no memory will be freed.
> A simple fix:
> modify the Ln 94 in autoFree() from:
> wrapper = (AddressWrapper) memoryInUse.remove(address);
> to:
> wrapper = (AddressWrapper) memoryInUse.get(address);
> and it should work.
> The steps to reproduce the bug:
> 1. Add the following line in the the Ln109, which indicates the memory is auto freed by AbstractMemorySpy
> System.out.println("MemorySpy autofree:"+wrapper.shadow);
> 2. Run the test below:
> public class MemorySpyTest {
> public static void main(String[] args) throws Exception {
> for (int i = 0; i < 1000; i++) {
> try{
> ByteBuffer buffer = ByteBuffer.allocateDirect(1024);
> buffer = null;
> }catch(Exception e){
> e.printStackTrace();
> }
> System.gc();
> }
> }
> Before modification: application exits silently.
> After modification, hundres of lines printed out, like this:
> .....
> MemorySpy autofree:PlatformAddress[9310024]
> MemorySpy autofree:PlatformAddress[9311056]
> MemorySpy autofree:PlatformAddress[9312088]
> MemorySpy autofree:PlatformAddress[9313120]
> MemorySpy autofree:PlatformAddress[9314152]
> MemorySpy autofree:PlatformAddress[9315184]
> MemorySpy autofree:PlatformAddress[9316216]
> MemorySpy autofree:PlatformAddress[9317248]
> .......
> Another possible pitfall in AbstractMemorySpy is it uses WeakReference and ReferenceQueue to get notification when the buffers are ready to be garbage collected. But it is not so safe yet to free the native memory, because the weakly-reachable object has its chance to be resurrected to strongly-reachable, if only some finalize() method doing this is executed. A safer solution is to use PhantomReference instead, because phantom-reachable object has no chance to survive. The only concern is the PhantomReference should be explicitly cleared by application, but that's OK because the orphanedMemory() method is a perfect candidate to do this.
> I will attach the patch soon.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HARMONY-38)
com.ibm.platform.struct.AbstractMemorySpy doesn't actually autoFree native
memory
Posted by "Tim Ellison (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-38?page=comments#action_12363811 ]
Tim Ellison commented on HARMONY-38:
------------------------------------
Paulex,
I've fixed the autoFree() problem described above in NIO com.ibm.platform.struct.AbstractMemorySpy.java at repo revision 371895.
If you have a patch for the WeakReference/PhantomReference issue please send it along, otherwise I'm happy to take a look at it myself.
> com.ibm.platform.struct.AbstractMemorySpy doesn't actually autoFree native memory
> ---------------------------------------------------------------------------------
>
> Key: HARMONY-38
> URL: http://issues.apache.org/jira/browse/HARMONY-38
> Project: Harmony
> Type: Bug
> Components: Classlib
> Reporter: Paulex Yang
> Assignee: Tim Ellison
> Priority: Critical
>
> com.ibm.platform.struct.IMemorySpy is a memory watcher, and the AbstractMemorySpy is a default implementation working with nio direct buffers to free the native memories when direct buffer instances are garbage collected. But currently its auto free donsn't work.
> Look inside the orphanedMemory(Object obj) method, the memory is freed only if the local variable wrapper can be found from the memoryInUse, a Map which records the native memories used by direct buffers, but actually it will be removed when autoFree(PlatfromAddress) is invoked. Checking the autoFree() reference, I found that all DirectByteBuffer will invoke autoFree() in constructor, which means no memory will be freed.
> A simple fix:
> modify the Ln 94 in autoFree() from:
> wrapper = (AddressWrapper) memoryInUse.remove(address);
> to:
> wrapper = (AddressWrapper) memoryInUse.get(address);
> and it should work.
> The steps to reproduce the bug:
> 1. Add the following line in the the Ln109, which indicates the memory is auto freed by AbstractMemorySpy
> System.out.println("MemorySpy autofree:"+wrapper.shadow);
> 2. Run the test below:
> public class MemorySpyTest {
> public static void main(String[] args) throws Exception {
> for (int i = 0; i < 1000; i++) {
> try{
> ByteBuffer buffer = ByteBuffer.allocateDirect(1024);
> buffer = null;
> }catch(Exception e){
> e.printStackTrace();
> }
> System.gc();
> }
> }
> Before modification: application exits silently.
> After modification, hundres of lines printed out, like this:
> .....
> MemorySpy autofree:PlatformAddress[9310024]
> MemorySpy autofree:PlatformAddress[9311056]
> MemorySpy autofree:PlatformAddress[9312088]
> MemorySpy autofree:PlatformAddress[9313120]
> MemorySpy autofree:PlatformAddress[9314152]
> MemorySpy autofree:PlatformAddress[9315184]
> MemorySpy autofree:PlatformAddress[9316216]
> MemorySpy autofree:PlatformAddress[9317248]
> .......
> Another possible pitfall in AbstractMemorySpy is it uses WeakReference and ReferenceQueue to get notification when the buffers are ready to be garbage collected. But it is not so safe yet to free the native memory, because the weakly-reachable object has its chance to be resurrected to strongly-reachable, if only some finalize() method doing this is executed. A safer solution is to use PhantomReference instead, because phantom-reachable object has no chance to survive. The only concern is the PhantomReference should be explicitly cleared by application, but that's OK because the orphanedMemory() method is a perfect candidate to do this.
> I will attach the patch soon.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Assigned: (HARMONY-38)
com.ibm.platform.struct.AbstractMemorySpy doesn't actually autoFree native
memory
Posted by "Tim Ellison (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-38?page=all ]
Tim Ellison reassigned HARMONY-38:
----------------------------------
Assign To: Tim Ellison
> com.ibm.platform.struct.AbstractMemorySpy doesn't actually autoFree native memory
> ---------------------------------------------------------------------------------
>
> Key: HARMONY-38
> URL: http://issues.apache.org/jira/browse/HARMONY-38
> Project: Harmony
> Type: Bug
> Components: Classlib
> Reporter: Paulex Yang
> Assignee: Tim Ellison
> Priority: Critical
>
> com.ibm.platform.struct.IMemorySpy is a memory watcher, and the AbstractMemorySpy is a default implementation working with nio direct buffers to free the native memories when direct buffer instances are garbage collected. But currently its auto free donsn't work.
> Look inside the orphanedMemory(Object obj) method, the memory is freed only if the local variable wrapper can be found from the memoryInUse, a Map which records the native memories used by direct buffers, but actually it will be removed when autoFree(PlatfromAddress) is invoked. Checking the autoFree() reference, I found that all DirectByteBuffer will invoke autoFree() in constructor, which means no memory will be freed.
> A simple fix:
> modify the Ln 94 in autoFree() from:
> wrapper = (AddressWrapper) memoryInUse.remove(address);
> to:
> wrapper = (AddressWrapper) memoryInUse.get(address);
> and it should work.
> The steps to reproduce the bug:
> 1. Add the following line in the the Ln109, which indicates the memory is auto freed by AbstractMemorySpy
> System.out.println("MemorySpy autofree:"+wrapper.shadow);
> 2. Run the test below:
> public class MemorySpyTest {
> public static void main(String[] args) throws Exception {
> for (int i = 0; i < 1000; i++) {
> try{
> ByteBuffer buffer = ByteBuffer.allocateDirect(1024);
> buffer = null;
> }catch(Exception e){
> e.printStackTrace();
> }
> System.gc();
> }
> }
> Before modification: application exits silently.
> After modification, hundres of lines printed out, like this:
> .....
> MemorySpy autofree:PlatformAddress[9310024]
> MemorySpy autofree:PlatformAddress[9311056]
> MemorySpy autofree:PlatformAddress[9312088]
> MemorySpy autofree:PlatformAddress[9313120]
> MemorySpy autofree:PlatformAddress[9314152]
> MemorySpy autofree:PlatformAddress[9315184]
> MemorySpy autofree:PlatformAddress[9316216]
> MemorySpy autofree:PlatformAddress[9317248]
> .......
> Another possible pitfall in AbstractMemorySpy is it uses WeakReference and ReferenceQueue to get notification when the buffers are ready to be garbage collected. But it is not so safe yet to free the native memory, because the weakly-reachable object has its chance to be resurrected to strongly-reachable, if only some finalize() method doing this is executed. A safer solution is to use PhantomReference instead, because phantom-reachable object has no chance to survive. The only concern is the PhantomReference should be explicitly cleared by application, but that's OK because the orphanedMemory() method is a perfect candidate to do this.
> I will attach the patch soon.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Resolved: (HARMONY-38)
com.ibm.platform.struct.AbstractMemorySpy doesn't actually autoFree native
memory
Posted by "Tim Ellison (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-38?page=all ]
Tim Ellison resolved HARMONY-38:
--------------------------------
Resolution: Fixed
Paulex,
Fixed in NIO module at repo revision 372204
- com.ibm.platform.struct.AbstractMemorySpy.java
- com.ibm.platform.struct.RuntimeMemorySpy.java
Please check that this fully resolves the problem.
> com.ibm.platform.struct.AbstractMemorySpy doesn't actually autoFree native memory
> ---------------------------------------------------------------------------------
>
> Key: HARMONY-38
> URL: http://issues.apache.org/jira/browse/HARMONY-38
> Project: Harmony
> Type: Bug
> Components: Classlib
> Reporter: Paulex Yang
> Assignee: Tim Ellison
> Priority: Critical
> Attachments: Harmony38-AbstractMemorySpy-patch.txt, Harmony38-RuntimeMemorySpy-patch.txt
>
> com.ibm.platform.struct.IMemorySpy is a memory watcher, and the AbstractMemorySpy is a default implementation working with nio direct buffers to free the native memories when direct buffer instances are garbage collected. But currently its auto free donsn't work.
> Look inside the orphanedMemory(Object obj) method, the memory is freed only if the local variable wrapper can be found from the memoryInUse, a Map which records the native memories used by direct buffers, but actually it will be removed when autoFree(PlatfromAddress) is invoked. Checking the autoFree() reference, I found that all DirectByteBuffer will invoke autoFree() in constructor, which means no memory will be freed.
> A simple fix:
> modify the Ln 94 in autoFree() from:
> wrapper = (AddressWrapper) memoryInUse.remove(address);
> to:
> wrapper = (AddressWrapper) memoryInUse.get(address);
> and it should work.
> The steps to reproduce the bug:
> 1. Add the following line in the the Ln109, which indicates the memory is auto freed by AbstractMemorySpy
> System.out.println("MemorySpy autofree:"+wrapper.shadow);
> 2. Run the test below:
> public class MemorySpyTest {
> public static void main(String[] args) throws Exception {
> for (int i = 0; i < 1000; i++) {
> try{
> ByteBuffer buffer = ByteBuffer.allocateDirect(1024);
> buffer = null;
> }catch(Exception e){
> e.printStackTrace();
> }
> System.gc();
> }
> }
> Before modification: application exits silently.
> After modification, hundres of lines printed out, like this:
> .....
> MemorySpy autofree:PlatformAddress[9310024]
> MemorySpy autofree:PlatformAddress[9311056]
> MemorySpy autofree:PlatformAddress[9312088]
> MemorySpy autofree:PlatformAddress[9313120]
> MemorySpy autofree:PlatformAddress[9314152]
> MemorySpy autofree:PlatformAddress[9315184]
> MemorySpy autofree:PlatformAddress[9316216]
> MemorySpy autofree:PlatformAddress[9317248]
> .......
> Another possible pitfall in AbstractMemorySpy is it uses WeakReference and ReferenceQueue to get notification when the buffers are ready to be garbage collected. But it is not so safe yet to free the native memory, because the weakly-reachable object has its chance to be resurrected to strongly-reachable, if only some finalize() method doing this is executed. A safer solution is to use PhantomReference instead, because phantom-reachable object has no chance to survive. The only concern is the PhantomReference should be explicitly cleared by application, but that's OK because the orphanedMemory() method is a perfect candidate to do this.
> I will attach the patch soon.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (HARMONY-38)
com.ibm.platform.struct.AbstractMemorySpy doesn't actually autoFree native
memory
Posted by "Paulex Yang (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HARMONY-38?page=all ]
Paulex Yang updated HARMONY-38:
-------------------------------
Attachment: Harmony38-AbstractMemorySpy-patch.txt
Harmony38-RuntimeMemorySpy-patch.txt
Thank you, Tim. Here goes the patches for PhantomReference issue, including ones for AbstractMemorySpy and PhantomMemorySpy.
> com.ibm.platform.struct.AbstractMemorySpy doesn't actually autoFree native memory
> ---------------------------------------------------------------------------------
>
> Key: HARMONY-38
> URL: http://issues.apache.org/jira/browse/HARMONY-38
> Project: Harmony
> Type: Bug
> Components: Classlib
> Reporter: Paulex Yang
> Assignee: Tim Ellison
> Priority: Critical
> Attachments: Harmony38-AbstractMemorySpy-patch.txt, Harmony38-RuntimeMemorySpy-patch.txt
>
> com.ibm.platform.struct.IMemorySpy is a memory watcher, and the AbstractMemorySpy is a default implementation working with nio direct buffers to free the native memories when direct buffer instances are garbage collected. But currently its auto free donsn't work.
> Look inside the orphanedMemory(Object obj) method, the memory is freed only if the local variable wrapper can be found from the memoryInUse, a Map which records the native memories used by direct buffers, but actually it will be removed when autoFree(PlatfromAddress) is invoked. Checking the autoFree() reference, I found that all DirectByteBuffer will invoke autoFree() in constructor, which means no memory will be freed.
> A simple fix:
> modify the Ln 94 in autoFree() from:
> wrapper = (AddressWrapper) memoryInUse.remove(address);
> to:
> wrapper = (AddressWrapper) memoryInUse.get(address);
> and it should work.
> The steps to reproduce the bug:
> 1. Add the following line in the the Ln109, which indicates the memory is auto freed by AbstractMemorySpy
> System.out.println("MemorySpy autofree:"+wrapper.shadow);
> 2. Run the test below:
> public class MemorySpyTest {
> public static void main(String[] args) throws Exception {
> for (int i = 0; i < 1000; i++) {
> try{
> ByteBuffer buffer = ByteBuffer.allocateDirect(1024);
> buffer = null;
> }catch(Exception e){
> e.printStackTrace();
> }
> System.gc();
> }
> }
> Before modification: application exits silently.
> After modification, hundres of lines printed out, like this:
> .....
> MemorySpy autofree:PlatformAddress[9310024]
> MemorySpy autofree:PlatformAddress[9311056]
> MemorySpy autofree:PlatformAddress[9312088]
> MemorySpy autofree:PlatformAddress[9313120]
> MemorySpy autofree:PlatformAddress[9314152]
> MemorySpy autofree:PlatformAddress[9315184]
> MemorySpy autofree:PlatformAddress[9316216]
> MemorySpy autofree:PlatformAddress[9317248]
> .......
> Another possible pitfall in AbstractMemorySpy is it uses WeakReference and ReferenceQueue to get notification when the buffers are ready to be garbage collected. But it is not so safe yet to free the native memory, because the weakly-reachable object has its chance to be resurrected to strongly-reachable, if only some finalize() method doing this is executed. A safer solution is to use PhantomReference instead, because phantom-reachable object has no chance to survive. The only concern is the PhantomReference should be explicitly cleared by application, but that's OK because the orphanedMemory() method is a perfect candidate to do this.
> I will attach the patch soon.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira