[VM]How to trigue GC while free native memory is low.

[Leo Li <li...@gmail.com>]

Hi, all:
    After applying patch3073, I added the support for direct byte buffer in
nio module as spec requires. But now here exists a problem: The native
memory allocated to the byte buffer can be released when gc releases the
byte buffer object, however, if the native heap is full while java heap
still has space, gc will not be triggered.
    It seems that RI will start gc before native memory heap is depleted and
thus prevents out-of-memory error.

   Then our work focuses on:
   1. When gc is required.
   2. Trigger a gc.
   The first one requires that we get support from operating system, since
the memory allocated in the native code, for example by malloc, is out of
the control of java VM.( I have ever thought of counting the used memory in
hymemory_allocate, but the plan fails since hymemory_free will not report
how much space is released.)
   The second one needs the help from VM. System.gc() is not so reliable as
spec says, so it is necessary to have a internal channel to notify VM to
start gc.

    One solution, I think, is to let a monitor thread in VM to check whether
OS physical memory is low. For example, the QueryMemoryResourceNotification
of win32 API is a candidate. Although the interrupt model is more effective:
win32 SDK provides a CreateMemoryResourceNotification to get a handler on
which the monitor thread can wait, maybe on other platforms, OS does not
supply such a convenience. So the monitor thread in the VM might have to
check the OS resource once for a while and if necessary the monitor thread
will call a GC.

   My suggestion is first to add some function to monitor memory in portlib,
since it is highly related to platforms and in portlib there has been
some useful tools.(Thanks Mark to point out that.) On the other hand, we can
negotiate a channel to trigger gc in VM. Actually I am not an expert on VM
since I am not sure whether there has been some monitor thread and the load
on performance if such a monitor thread is added to the VM...


-- 
Leo Li
China Software Development Lab, IBM

Re: [VM]How to trigue GC while free native memory is low.

[Ivan Volosyuk <iv...@gmail.com>]

On 2/5/07, Tim Ellison <t....@gmail.com> wrote:
> Xiao-Feng Li wrote:
> > On 2/5/07, Tim Ellison <t....@gmail.com> wrote:
> >> Xiao-Feng Li wrote:
> >> > It's a little big strange to me if API encourages this kind of
> >> > behavior that programmer grabs resources freely while relying on
> >> > certain unreliable behavior to release the resource.
> >>
> >> Had to smile at this, I'm sure a C programmer would say the same of the
> >> 'new' keyword -- then you GC-folk would be out of a job <g>
> >
> > Hmm, this is different, Tim. The keyword is "unreliable". GC can and
> > must provide reliable memory management. And automatic memory
> > management is mandatory for type-safe programming languages. It is not
> > optional to use `free' or not as C does.
>
> I was being somewhat flippant xiaofeng by comparing the world in which
> an explicit call to free() makes the resource available again, and the
> automatic memory management that relies on GC at some indeterminate (by
> the application) point in the future.
>
> I'd be much happier if we could get to a model of continuous
> profile-directed GC, rather than relying on allocation failures from
> heap management.  But that's getting off-topic, and subject for
> discussion over a beer!
>
> > The native resources we are discussing here in Java are different from
> > Java managed heap. The only binding here is the association between a
> > Java object and its represented native resource. If there was not this
> > association, GC might not need to be involved at all. The problem is,
> > the implication of this association is not explicitly documented or
> > specified.
> >
> > Since the direct ByteBuffer is with a piece of memory, it gives people
> > an impression that it's memory resource, must work as Java managed
> > heap. I am afraid this is inaccurate. Surely we can manage it in Java
> > heap to make its management reliable, then freeDirect is nonsense; but
> > I don't know if all the native resources can be managed reliably in
> > the similar way. And if we extend it to higher level resources, we
> > know explicit management is not a bad idea.
>
> There are so many external resources that could use this type of
> management requirement that solving it only for a memory block probably
> isn't the right answer.  Letting the malloc failure return to the class
> libraries and allowing the class library code an attempt to recover
> seems like the right answer.
>
> The problem seems to be that the only tools we have in the class library
> are System#gc() and System#runFinalization().  A full GC is likely too
> heavyweight (we don't actually want the Java heap reclaiming, and
> certainly don't need to incur compaction etc.), and draining the
> finalizer queue doesn't ensure that the unreachable objects have been
> enqueued.

I can say here that in DRLVM's GC v4.1 System.gc() is quite
lightweight and does exactly what you have said - detects unreachable
objects and push them into finalizer queue. :)

>
> We need the intermediate goal of 'enqueue finalizable objects', or in
> the more general case, 'enqueue PhantomReferences'.  We can probably
> even agree on creating a specific Java type for a
> QuicklyDetectAsCollectablePhantomReference <g> if that helps, though I'm
> not sure it will since it is the referent that is being considered.
>
> > To manage it at higher level, probably we can accept the OOM
> > blissfully when allocDirect fails? This is not only for ByteBuffer
> > handling, but a pattern applicable to other native resources.
> >
> > How do you think?
>
> The best we can do today is to call System.gc(), but it seems to be much
> more than we actually want to do.
>
> Regards,
> Tim

-- 
Ivan
Intel Enterprise Solutions Software Division

Re: [VM]How to trigue GC while free native memory is low.

[Tim Ellison <t....@gmail.com>]

Xiao-Feng Li wrote:
> On 2/5/07, Tim Ellison <t....@gmail.com> wrote:
>> Xiao-Feng Li wrote:
>> > It's a little big strange to me if API encourages this kind of
>> > behavior that programmer grabs resources freely while relying on
>> > certain unreliable behavior to release the resource.
>>
>> Had to smile at this, I'm sure a C programmer would say the same of the
>> 'new' keyword -- then you GC-folk would be out of a job <g>
> 
> Hmm, this is different, Tim. The keyword is "unreliable". GC can and
> must provide reliable memory management. And automatic memory
> management is mandatory for type-safe programming languages. It is not
> optional to use `free' or not as C does.

I was being somewhat flippant xiaofeng by comparing the world in which
an explicit call to free() makes the resource available again, and the
automatic memory management that relies on GC at some indeterminate (by
the application) point in the future.

I'd be much happier if we could get to a model of continuous
profile-directed GC, rather than relying on allocation failures from
heap management.  But that's getting off-topic, and subject for
discussion over a beer!

> The native resources we are discussing here in Java are different from
> Java managed heap. The only binding here is the association between a
> Java object and its represented native resource. If there was not this
> association, GC might not need to be involved at all. The problem is,
> the implication of this association is not explicitly documented or
> specified.
> 
> Since the direct ByteBuffer is with a piece of memory, it gives people
> an impression that it's memory resource, must work as Java managed
> heap. I am afraid this is inaccurate. Surely we can manage it in Java
> heap to make its management reliable, then freeDirect is nonsense; but
> I don't know if all the native resources can be managed reliably in
> the similar way. And if we extend it to higher level resources, we
> know explicit management is not a bad idea.

There are so many external resources that could use this type of
management requirement that solving it only for a memory block probably
isn't the right answer.  Letting the malloc failure return to the class
libraries and allowing the class library code an attempt to recover
seems like the right answer.

The problem seems to be that the only tools we have in the class library
are System#gc() and System#runFinalization().  A full GC is likely too
heavyweight (we don't actually want the Java heap reclaiming, and
certainly don't need to incur compaction etc.), and draining the
finalizer queue doesn't ensure that the unreachable objects have been
enqueued.

We need the intermediate goal of 'enqueue finalizable objects', or in
the more general case, 'enqueue PhantomReferences'.  We can probably
even agree on creating a specific Java type for a
QuicklyDetectAsCollectablePhantomReference <g> if that helps, though I'm
not sure it will since it is the referent that is being considered.

> To manage it at higher level, probably we can accept the OOM
> blissfully when allocDirect fails? This is not only for ByteBuffer
> handling, but a pattern applicable to other native resources.
> 
> How do you think?

The best we can do today is to call System.gc(), but it seems to be much
more than we actually want to do.

Regards,
Tim

Re: [VM]How to trigue GC while free native memory is low.

[Xiao-Feng Li <xi...@gmail.com>]

On 2/5/07, Tim Ellison <t....@gmail.com> wrote:
> Xiao-Feng Li wrote:
> > This might be crazy, but is it possible or reasonable to have a method
> > ByteBuffer.freeDirect()?
>
> No, we cannot extend the API.
>
> > It's a little big strange to me if API encourages this kind of
> > behavior that programmer grabs resources freely while relying on
> > certain unreliable behavior to release the resource.
>
> Had to smile at this, I'm sure a C programmer would say the same of the
> 'new' keyword -- then you GC-folk would be out of a job <g>

Hmm, this is different, Tim. The keyword is "unreliable". GC can and
must provide reliable memory management. And automatic memory
management is mandatory for type-safe programming languages. It is not
optional to use `free' or not as C does.

The native resources we are discussing here in Java are different from
Java managed heap. The only binding here is the association between a
Java object and its represented native resource. If there was not this
association, GC might not need to be involved at all. The problem is,
the implication of this association is not explicitly documented or
specified.

Since the direct ByteBuffer is with a piece of memory, it gives people
an impression that it's memory resource, must work as Java managed
heap. I am afraid this is inaccurate. Surely we can manage it in Java
heap to make its management reliable, then freeDirect is nonsense; but
I don't know if all the native resources can be managed reliably in
the similar way. And if we extend it to higher level resources, we
know explicit management is not a bad idea.

To manage it at higher level, probably we can accept the OOM
blissfully when allocDirect fails? This is not only for ByteBuffer
handling, but a pattern applicable to other native resources.

How do you think?

Thanks,
xiaofeng

> Regards,
> Tim
>

Re: [VM]How to trigue GC while free native memory is low.

[Tim Ellison <t....@gmail.com>]

Xiao-Feng Li wrote:
> This might be crazy, but is it possible or reasonable to have a method
> ByteBuffer.freeDirect()?

No, we cannot extend the API.

> It's a little big strange to me if API encourages this kind of
> behavior that programmer grabs resources freely while relying on
> certain unreliable behavior to release the resource.

Had to smile at this, I'm sure a C programmer would say the same of the
'new' keyword -- then you GC-folk would be out of a job <g>

Regards,
Tim

Re: [VM]How to trigue GC while free native memory is low.

[Xiao-Feng Li <xi...@gmail.com>]

On 2/2/07, Leo Li <li...@gmail.com> wrote:
> Hi, Xiao-Feng:
>      Excuse me if I am confusing you.
>      The direct byte buffer holds a block of native memory space used in nio
> operation. The native byte buffer will be freed through the byte buffer is
> gc collected and added to ReferenceQueue monitored by MemorySpy series
> classes.
>      But the gc will not be triggered if the java heap is still empty. Here
> is an example: I allocate a large block of native memory for a byte buffer
> while the byte buffer itself is quite small. Thus quickly native heap is
> depleted while VM still does not think gc is needed since java heap has a
> lot of free memory.
>
>
> import java.nio.*;
> public class Test {
>     public static void main(String[] args) throws Exception {
>         for(int i = 0;i<1000;i++)
>         {
>             ByteBuffer byteBuffer = ByteBuffer.allocateDirect(10240000);
>             //System.gc();
>         }
>    }
> }
> RI runs well but Harmony will soon throw OutOfMemory exception.
> But if remove the comment before System.gc(), Harmony will become ok.
> So my point is that we need a mechanism to notify VM to start gc if we have
> no more native memory because sometimes problem will be solved if gc is
> fired. Of cause we cannot avoid a  user always malloc space but never free
> them.:)

This might be crazy, but is it possible or reasonable to have a method
ByteBuffer.freeDirect()?

It's a little big strange to me if API encourages this kind of
behavior that programmer grabs resources freely while relying on
certain unreliable behavior to release the resource. The situation is
even worse than finalizer, well we know using finalizer is discouraged
for resource management.

Thanks,
xiaofeng

Re: [VM]How to trigue GC while free native memory is low.

[Ivan Volosyuk <iv...@gmail.com>]

Proactive freeing such native resources may lead to instability. We
don't have enough control of all references to that native resources.

On 2/3/07, Rana Dasgupta <rd...@gmail.com> wrote:
> I don't know much about DirectByteBuffer implementation, but it would be
> nice to keep the memory management of managed and unmanaged heaps seperate
> from each other since they are independent resources. Housekeeping the
> active java proxy reference and freeing up native memory proactively/lazily
> when there are none would be great, but both would need some kind of
> refcounting on these instances independent of gc. Some jit support would be
> needed to update the refcount when these proxies( instances of this class
> ) are getting copied around, like a write barrier call.
>
> Depending on hooks to monitor the unmanaged heap has gaps. There could be
> other running processes on the machine with unpredictable resource requests.
> It may be easier to empirically call System.GC() periodically based on
> an estimation of unmanaged heap allocation and tune it to be realistic.
>
> Thanks,
> Rana
>
>
>
> On 2/2/07, Tim Ellison <t....@gmail.com> wrote:
> >
> > You don't want to invoke a GC though, as you said there is no allocation
> > failure in the Java heap.
> >
> > If you are eager you just want to know when there are no more references
> > to the 'byteBuffer' instance, so you can free the native resources it
> > references.  And if you are lazy you would wait for a resource
> > allocation failure when creating a new direct byte buffer then be
> > prepared to ask if there are any collectable 'byteBuffer' instances.
> >
> > IIRC the NIO buffer code tries to behave eagerly, and uses a
> > PhantomReference to the object holding onto the resource.  I'm assuming
> > that you see the out of malloc failure because the phantom is only
> > enqueued by invoking System.gc(), right?
> >
> > Regards,
> > Tim
> >
> > Leo Li wrote:
> > > Hi, Xiao-Feng:
> > >     Excuse me if I am confusing you.
> > >     The direct byte buffer holds a block of native memory space used in
> > nio
> > > operation. The native byte buffer will be freed through the byte buffer
> > is
> > > gc collected and added to ReferenceQueue monitored by MemorySpy series
> > > classes.
> > >     But the gc will not be triggered if the java heap is still empty.
> > Here
> > > is an example: I allocate a large block of native memory for a byte
> > buffer
> > > while the byte buffer itself is quite small. Thus quickly native heap is
> > > depleted while VM still does not think gc is needed since java heap has
> > a
> > > lot of free memory.
> > >
> > >
> > > import java.nio.*;
> > > public class Test {
> > >    public static void main(String[] args) throws Exception {
> > >        for(int i = 0;i<1000;i++)
> > >        {
> > >            ByteBuffer byteBuffer = ByteBuffer.allocateDirect(10240000);
> > >            //System.gc();
> > >        }
> > >   }
> > > }
> > > RI runs well but Harmony will soon throw OutOfMemory exception.
> > > But if remove the comment before System.gc(), Harmony will become ok.
> > > So my point is that we need a mechanism to notify VM to start gc if we
> > have
> > > no more native memory because sometimes problem will be solved if gc is
> > > fired. Of cause we cannot avoid a  user always malloc space but never
> > free
> > > them.:)
> > >
> > > Good luck!
> > >
> > > On 2/2/07, Xiao-Feng Li <xi...@gmail.com> wrote:
> > >>
> > >> Leo, I can't fully understand your problem and proposal. I am working
> > >> in DRLVM GC component, I want to understand how GC can tackle the
> > >> issue you meet.
> > >>
> > >> You said the native memory is allocated (probably with malloc) for
> > >> byte buffer. I wonder how the process goes. I assume it's in native
> > >> code. Then you use malloc to allocate a block of memory, and put some
> > >> Java object (byte buffer) into it? Is this what you do? I wonder how
> > >> do you manipulate Java object placement in native code; and if the
> > >> native memory block is out of Java heap, do you expect GC to manage
> > >> it?
> > >>
> > >> You said when GC reclaims those byte buffer object, the native memory
> > >> block can be freed in native code. How do you know if an object is
> > >> reclaimed by GC or not?
> > >>
> > >> Thanks,
> > >> xiaofeng
> > >>
> > >> On 2/2/07, Leo Li < liyilei1979@gmail.com> wrote:
> > >> > Hi, all:
> > >> >     After applying patch3073, I added the support for direct byte
> > >> buffer
> > >> in
> > >> > nio module as spec requires. But now here exists a problem: The
> > native
> > >> > memory allocated to the byte buffer can be released when gc releases
> > >> the
> > >> > byte buffer object, however, if the native heap is full while java
> > heap
> > >> > still has space, gc will not be triggered.
> > >> >     It seems that RI will start gc before native memory heap is
> > >> depleted
> > >> and
> > >> > thus prevents out-of-memory error.
> > >> >
> > >> >    Then our work focuses on:
> > >> >    1. When gc is required.
> > >> >    2. Trigger a gc.
> > >> >    The first one requires that we get support from operating system,
> > >> since
> > >> > the memory allocated in the native code, for example by malloc, is
> > out
> > >> of
> > >> > the control of java VM.( I have ever thought of counting the used
> > >> memory
> > >> in
> > >> > hymemory_allocate, but the plan fails since hymemory_free will not
> > >> report
> > >> > how much space is released.)
> > >> >    The second one needs the help from VM. System.gc() is not so
> > >> reliable
> > >> as
> > >> > spec says, so it is necessary to have a internal channel to notify
> > >> VM to
> > >> > start gc.
> > >> >
> > >> >     One solution, I think, is to let a monitor thread in VM to check
> > >> whether
> > >> > OS physical memory is low. For example, the
> > >> QueryMemoryResourceNotification
> > >> > of win32 API is a candidate. Although the interrupt model is more
> > >> effective:
> > >> > win32 SDK provides a CreateMemoryResourceNotification to get a
> > handler
> > >> on
> > >> > which the monitor thread can wait, maybe on other platforms, OS does
> > >> not
> > >> > supply such a convenience. So the monitor thread in the VM might
> > >> have to
> > >>
> > >> > check the OS resource once for a while and if necessary the monitor
> > >> thread
> > >> > will call a GC.
> > >> >
> > >> >    My suggestion is first to add some function to monitor memory in
> > >> portlib,
> > >> > since it is highly related to platforms and in portlib there has been
> > >> > some useful tools.(Thanks Mark to point out that.) On the other
> > >> hand, we
> > >> can
> > >> > negotiate a channel to trigger gc in VM. Actually I am not an expert
> > on
> > >> VM
> > >> > since I am not sure whether there has been some monitor thread and
> > the
> > >> load
> > >> > on performance if such a monitor thread is added to the VM...
> > >> >
> > >> >
> > >> > --
> > >> > Leo Li
> > >> > China Software Development Lab, IBM
> > >> >
> > >> >
> > >>
> > >
> > >
> > >
> >
>
>


-- 
Ivan
Intel Enterprise Solutions Software Division

Re: [VM]How to trigue GC while free native memory is low.

[Rana Dasgupta <rd...@gmail.com>]

I don't know much about DirectByteBuffer implementation, but it would be
nice to keep the memory management of managed and unmanaged heaps seperate
from each other since they are independent resources. Housekeeping the
active java proxy reference and freeing up native memory proactively/lazily
when there are none would be great, but both would need some kind of
refcounting on these instances independent of gc. Some jit support would be
needed to update the refcount when these proxies( instances of this class
) are getting copied around, like a write barrier call.

Depending on hooks to monitor the unmanaged heap has gaps. There could be
other running processes on the machine with unpredictable resource requests.
It may be easier to empirically call System.GC() periodically based on
an estimation of unmanaged heap allocation and tune it to be realistic.

Thanks,
Rana



On 2/2/07, Tim Ellison <t....@gmail.com> wrote:
>
> You don't want to invoke a GC though, as you said there is no allocation
> failure in the Java heap.
>
> If you are eager you just want to know when there are no more references
> to the 'byteBuffer' instance, so you can free the native resources it
> references.  And if you are lazy you would wait for a resource
> allocation failure when creating a new direct byte buffer then be
> prepared to ask if there are any collectable 'byteBuffer' instances.
>
> IIRC the NIO buffer code tries to behave eagerly, and uses a
> PhantomReference to the object holding onto the resource.  I'm assuming
> that you see the out of malloc failure because the phantom is only
> enqueued by invoking System.gc(), right?
>
> Regards,
> Tim
>
> Leo Li wrote:
> > Hi, Xiao-Feng:
> >     Excuse me if I am confusing you.
> >     The direct byte buffer holds a block of native memory space used in
> nio
> > operation. The native byte buffer will be freed through the byte buffer
> is
> > gc collected and added to ReferenceQueue monitored by MemorySpy series
> > classes.
> >     But the gc will not be triggered if the java heap is still empty.
> Here
> > is an example: I allocate a large block of native memory for a byte
> buffer
> > while the byte buffer itself is quite small. Thus quickly native heap is
> > depleted while VM still does not think gc is needed since java heap has
> a
> > lot of free memory.
> >
> >
> > import java.nio.*;
> > public class Test {
> >    public static void main(String[] args) throws Exception {
> >        for(int i = 0;i<1000;i++)
> >        {
> >            ByteBuffer byteBuffer = ByteBuffer.allocateDirect(10240000);
> >            //System.gc();
> >        }
> >   }
> > }
> > RI runs well but Harmony will soon throw OutOfMemory exception.
> > But if remove the comment before System.gc(), Harmony will become ok.
> > So my point is that we need a mechanism to notify VM to start gc if we
> have
> > no more native memory because sometimes problem will be solved if gc is
> > fired. Of cause we cannot avoid a  user always malloc space but never
> free
> > them.:)
> >
> > Good luck!
> >
> > On 2/2/07, Xiao-Feng Li <xi...@gmail.com> wrote:
> >>
> >> Leo, I can't fully understand your problem and proposal. I am working
> >> in DRLVM GC component, I want to understand how GC can tackle the
> >> issue you meet.
> >>
> >> You said the native memory is allocated (probably with malloc) for
> >> byte buffer. I wonder how the process goes. I assume it's in native
> >> code. Then you use malloc to allocate a block of memory, and put some
> >> Java object (byte buffer) into it? Is this what you do? I wonder how
> >> do you manipulate Java object placement in native code; and if the
> >> native memory block is out of Java heap, do you expect GC to manage
> >> it?
> >>
> >> You said when GC reclaims those byte buffer object, the native memory
> >> block can be freed in native code. How do you know if an object is
> >> reclaimed by GC or not?
> >>
> >> Thanks,
> >> xiaofeng
> >>
> >> On 2/2/07, Leo Li < liyilei1979@gmail.com> wrote:
> >> > Hi, all:
> >> >     After applying patch3073, I added the support for direct byte
> >> buffer
> >> in
> >> > nio module as spec requires. But now here exists a problem: The
> native
> >> > memory allocated to the byte buffer can be released when gc releases
> >> the
> >> > byte buffer object, however, if the native heap is full while java
> heap
> >> > still has space, gc will not be triggered.
> >> >     It seems that RI will start gc before native memory heap is
> >> depleted
> >> and
> >> > thus prevents out-of-memory error.
> >> >
> >> >    Then our work focuses on:
> >> >    1. When gc is required.
> >> >    2. Trigger a gc.
> >> >    The first one requires that we get support from operating system,
> >> since
> >> > the memory allocated in the native code, for example by malloc, is
> out
> >> of
> >> > the control of java VM.( I have ever thought of counting the used
> >> memory
> >> in
> >> > hymemory_allocate, but the plan fails since hymemory_free will not
> >> report
> >> > how much space is released.)
> >> >    The second one needs the help from VM. System.gc() is not so
> >> reliable
> >> as
> >> > spec says, so it is necessary to have a internal channel to notify
> >> VM to
> >> > start gc.
> >> >
> >> >     One solution, I think, is to let a monitor thread in VM to check
> >> whether
> >> > OS physical memory is low. For example, the
> >> QueryMemoryResourceNotification
> >> > of win32 API is a candidate. Although the interrupt model is more
> >> effective:
> >> > win32 SDK provides a CreateMemoryResourceNotification to get a
> handler
> >> on
> >> > which the monitor thread can wait, maybe on other platforms, OS does
> >> not
> >> > supply such a convenience. So the monitor thread in the VM might
> >> have to
> >>
> >> > check the OS resource once for a while and if necessary the monitor
> >> thread
> >> > will call a GC.
> >> >
> >> >    My suggestion is first to add some function to monitor memory in
> >> portlib,
> >> > since it is highly related to platforms and in portlib there has been
> >> > some useful tools.(Thanks Mark to point out that.) On the other
> >> hand, we
> >> can
> >> > negotiate a channel to trigger gc in VM. Actually I am not an expert
> on
> >> VM
> >> > since I am not sure whether there has been some monitor thread and
> the
> >> load
> >> > on performance if such a monitor thread is added to the VM...
> >> >
> >> >
> >> > --
> >> > Leo Li
> >> > China Software Development Lab, IBM
> >> >
> >> >
> >>
> >
> >
> >
>

Re: [VM]How to trigue GC while free native memory is low.

[Tim Ellison <t....@gmail.com>]

You don't want to invoke a GC though, as you said there is no allocation
failure in the Java heap.

If you are eager you just want to know when there are no more references
to the 'byteBuffer' instance, so you can free the native resources it
references.  And if you are lazy you would wait for a resource
allocation failure when creating a new direct byte buffer then be
prepared to ask if there are any collectable 'byteBuffer' instances.

IIRC the NIO buffer code tries to behave eagerly, and uses a
PhantomReference to the object holding onto the resource.  I'm assuming
that you see the out of malloc failure because the phantom is only
enqueued by invoking System.gc(), right?

Regards,
Tim

Leo Li wrote:
> Hi, Xiao-Feng:
>     Excuse me if I am confusing you.
>     The direct byte buffer holds a block of native memory space used in nio
> operation. The native byte buffer will be freed through the byte buffer is
> gc collected and added to ReferenceQueue monitored by MemorySpy series
> classes.
>     But the gc will not be triggered if the java heap is still empty. Here
> is an example: I allocate a large block of native memory for a byte buffer
> while the byte buffer itself is quite small. Thus quickly native heap is
> depleted while VM still does not think gc is needed since java heap has a
> lot of free memory.
> 
> 
> import java.nio.*;
> public class Test {
>    public static void main(String[] args) throws Exception {
>        for(int i = 0;i<1000;i++)
>        {
>            ByteBuffer byteBuffer = ByteBuffer.allocateDirect(10240000);
>            //System.gc();
>        }
>   }
> }
> RI runs well but Harmony will soon throw OutOfMemory exception.
> But if remove the comment before System.gc(), Harmony will become ok.
> So my point is that we need a mechanism to notify VM to start gc if we have
> no more native memory because sometimes problem will be solved if gc is
> fired. Of cause we cannot avoid a  user always malloc space but never free
> them.:)
> 
> Good luck!
> 
> On 2/2/07, Xiao-Feng Li <xi...@gmail.com> wrote:
>>
>> Leo, I can't fully understand your problem and proposal. I am working
>> in DRLVM GC component, I want to understand how GC can tackle the
>> issue you meet.
>>
>> You said the native memory is allocated (probably with malloc) for
>> byte buffer. I wonder how the process goes. I assume it's in native
>> code. Then you use malloc to allocate a block of memory, and put some
>> Java object (byte buffer) into it? Is this what you do? I wonder how
>> do you manipulate Java object placement in native code; and if the
>> native memory block is out of Java heap, do you expect GC to manage
>> it?
>>
>> You said when GC reclaims those byte buffer object, the native memory
>> block can be freed in native code. How do you know if an object is
>> reclaimed by GC or not?
>>
>> Thanks,
>> xiaofeng
>>
>> On 2/2/07, Leo Li < liyilei1979@gmail.com> wrote:
>> > Hi, all:
>> >     After applying patch3073, I added the support for direct byte
>> buffer
>> in
>> > nio module as spec requires. But now here exists a problem: The native
>> > memory allocated to the byte buffer can be released when gc releases
>> the
>> > byte buffer object, however, if the native heap is full while java heap
>> > still has space, gc will not be triggered.
>> >     It seems that RI will start gc before native memory heap is
>> depleted
>> and
>> > thus prevents out-of-memory error.
>> >
>> >    Then our work focuses on:
>> >    1. When gc is required.
>> >    2. Trigger a gc.
>> >    The first one requires that we get support from operating system,
>> since
>> > the memory allocated in the native code, for example by malloc, is out
>> of
>> > the control of java VM.( I have ever thought of counting the used
>> memory
>> in
>> > hymemory_allocate, but the plan fails since hymemory_free will not
>> report
>> > how much space is released.)
>> >    The second one needs the help from VM. System.gc() is not so
>> reliable
>> as
>> > spec says, so it is necessary to have a internal channel to notify
>> VM to
>> > start gc.
>> >
>> >     One solution, I think, is to let a monitor thread in VM to check
>> whether
>> > OS physical memory is low. For example, the
>> QueryMemoryResourceNotification
>> > of win32 API is a candidate. Although the interrupt model is more
>> effective:
>> > win32 SDK provides a CreateMemoryResourceNotification to get a handler
>> on
>> > which the monitor thread can wait, maybe on other platforms, OS does
>> not
>> > supply such a convenience. So the monitor thread in the VM might
>> have to
>>
>> > check the OS resource once for a while and if necessary the monitor
>> thread
>> > will call a GC.
>> >
>> >    My suggestion is first to add some function to monitor memory in
>> portlib,
>> > since it is highly related to platforms and in portlib there has been
>> > some useful tools.(Thanks Mark to point out that.) On the other
>> hand, we
>> can
>> > negotiate a channel to trigger gc in VM. Actually I am not an expert on
>> VM
>> > since I am not sure whether there has been some monitor thread and the
>> load
>> > on performance if such a monitor thread is added to the VM...
>> >
>> >
>> > --
>> > Leo Li
>> > China Software Development Lab, IBM
>> >
>> >
>>
> 
> 
> 

Re: [VM]How to trigue GC while free native memory is low.

[LvJimmy，Jing <fi...@gmail.com>]

2007/2/3, Ivan Volosyuk <iv...@gmail.com>:
> Let me summarize:
>
> Sometimes we have small java heap objects which relates to much bigger
> native memory chunks. In some cases this memory may exceed limits and
> VM will crash. Calling GC from the other side will make this memory
> free. There are quite a few place in VM and classlib which uses such
> design pattern. In order to deal with this correctly some general
> solution required.
>
> We can do the following:
>
> 1. We can monitor virtual memory of running process via some hooks
> from operating system or use a special thread to do so.
> Actually I have already experimented with this for a while and it
> become clear to me that it cannot be use reliably. We can detect
> increase in virtual memory consumption when process virtual memory
> increases, if running GC some of memory will be freed but virtual
> memory consumption will not necessary decrease. Filling in the hidden
> from use freed memory is not visible and then will see next phase on
> virtual memory increase. After a few cycles we will end up with
> process crash.
>

We may not need to monitor memory, a simple thought
// try to malloc a block of memory
void * somepointer = malloc(memsize);
if (0 == somepointer){
     call system gc
}
// try again, if failed, throw out-of-memory error
...

> 2. We can use custom native memory allocator which override system one
> for all memory allocations or just for the ones originating from
> VM/classlib.
> We can have much more control on memory consumption then. Currently
> I'm investigating this possibility. It require careful design in order
> to make it work reliable, predictable and efficient. If somebody have
> fresh ideas or can give links for corresponding articles we could
> gather its here and develop a general solution.
>

It is something like java GC itself right? Then we have a native GC.
I'm thinking of it as well,
but if we have to control a memory pool, use original GC, use java
heap may be a better way?

> 3... ?
>
> --
> Ivan
> Intel Enterprise Solutions Software Division
>
> On 2/2/07, Gregory Shimansky <gs...@gmail.com> wrote:
> > Leo Li wrote:
> > > Hi, Xiao-Feng:
> > >     Excuse me if I am confusing you.
> > >     The direct byte buffer holds a block of native memory space used in nio
> > > operation. The native byte buffer will be freed through the byte buffer is
> > > gc collected and added to ReferenceQueue monitored by MemorySpy series
> > > classes.
> > >     But the gc will not be triggered if the java heap is still empty. Here
> > > is an example: I allocate a large block of native memory for a byte buffer
> > > while the byte buffer itself is quite small. Thus quickly native heap is
> > > depleted while VM still does not think gc is needed since java heap has a
> > > lot of free memory.
> > >
> > >
> > > import java.nio.*;
> > > public class Test {
> > >    public static void main(String[] args) throws Exception {
> > >        for(int i = 0;i<1000;i++)
> > >        {
> > >            ByteBuffer byteBuffer = ByteBuffer.allocateDirect(10240000);
> > >            //System.gc();
> > >        }
> > >   }
> > > }
> > > RI runs well but Harmony will soon throw OutOfMemory exception.
> > > But if remove the comment before System.gc(), Harmony will become ok.
> > > So my point is that we need a mechanism to notify VM to start gc if we have
> > > no more native memory because sometimes problem will be solved if gc is
> > > fired. Of cause we cannot avoid a  user always malloc space but never free
> > > them.:)
> >
> > ByteBuffer is not the only one class which has associated native
> > resources that are freed when Java object is collected. And it is not
> > the only class which may run out of such native resources before garbage
> > collection is needed for Java heap. You can see a similar problem
> > described in HARMONY-1831 where GZIPOutputStream allocates associated
> > native memory in ZLIB which ends up before GC collects unused objects.
> >
> > Other types of native resources could be handles and other OS specific
> > stuff. I am not sure, but it is probably easier for a particular type of
> > Java API class to realize that there are no more specific native
> > resources (like when ZLIB returns Z_MEM_ERROR in HARMONY-1831) left for
> > it and trigger GC unconditionally, then retry allocating resource again.
> >
> > > On 2/2/07, Xiao-Feng Li <xi...@gmail.com> wrote:
> > >>
> > >> Leo, I can't fully understand your problem and proposal. I am working
> > >> in DRLVM GC component, I want to understand how GC can tackle the
> > >> issue you meet.
> > >>
> > >> You said the native memory is allocated (probably with malloc) for
> > >> byte buffer. I wonder how the process goes. I assume it's in native
> > >> code. Then you use malloc to allocate a block of memory, and put some
> > >> Java object (byte buffer) into it? Is this what you do? I wonder how
> > >> do you manipulate Java object placement in native code; and if the
> > >> native memory block is out of Java heap, do you expect GC to manage
> > >> it?
> > >>
> > >> You said when GC reclaims those byte buffer object, the native memory
> > >> block can be freed in native code. How do you know if an object is
> > >> reclaimed by GC or not?
> > >>
> > >> Thanks,
> > >> xiaofeng
> > >>
> > >> On 2/2/07, Leo Li < liyilei1979@gmail.com> wrote:
> > >> > Hi, all:
> > >> >     After applying patch3073, I added the support for direct byte
> > >> buffer
> > >> in
> > >> > nio module as spec requires. But now here exists a problem: The native
> > >> > memory allocated to the byte buffer can be released when gc releases
> > >> the
> > >> > byte buffer object, however, if the native heap is full while java heap
> > >> > still has space, gc will not be triggered.
> > >> >     It seems that RI will start gc before native memory heap is
> > >> depleted
> > >> and
> > >> > thus prevents out-of-memory error.
> > >> >
> > >> >    Then our work focuses on:
> > >> >    1. When gc is required.
> > >> >    2. Trigger a gc.
> > >> >    The first one requires that we get support from operating system,
> > >> since
> > >> > the memory allocated in the native code, for example by malloc, is out
> > >> of
> > >> > the control of java VM.( I have ever thought of counting the used
> > >> memory
> > >> in
> > >> > hymemory_allocate, but the plan fails since hymemory_free will not
> > >> report
> > >> > how much space is released.)
> > >> >    The second one needs the help from VM. System.gc() is not so
> > >> reliable
> > >> as
> > >> > spec says, so it is necessary to have a internal channel to notify
> > >> VM to
> > >> > start gc.
> > >> >
> > >> >     One solution, I think, is to let a monitor thread in VM to check
> > >> whether
> > >> > OS physical memory is low. For example, the
> > >> QueryMemoryResourceNotification
> > >> > of win32 API is a candidate. Although the interrupt model is more
> > >> effective:
> > >> > win32 SDK provides a CreateMemoryResourceNotification to get a handler
> > >> on
> > >> > which the monitor thread can wait, maybe on other platforms, OS does
> > >> not
> > >> > supply such a convenience. So the monitor thread in the VM might
> > >> have to
> > >>
> > >> > check the OS resource once for a while and if necessary the monitor
> > >> thread
> > >> > will call a GC.
> > >> >
> > >> >    My suggestion is first to add some function to monitor memory in
> > >> portlib,
> > >> > since it is highly related to platforms and in portlib there has been
> > >> > some useful tools.(Thanks Mark to point out that.) On the other
> > >> hand, we
> > >> can
> > >> > negotiate a channel to trigger gc in VM. Actually I am not an expert on
> > >> VM
> > >> > since I am not sure whether there has been some monitor thread and the
> > >> load
> > >> > on performance if such a monitor thread is added to the VM...
> > >> >
> > >> >
> > >> > --
> > >> > Leo Li
> > >> > China Software Development Lab, IBM
> > >> >
> > >> >
> > >>
> > >
> > >
> > >
> >
> >
> > --
> > Gregory
> >
> >
>


-- 

Best Regards!

Jimmy, Jing Lv
China Software Development Lab, IBM

Re: [VM]How to trigue GC while free native memory is low.

[Ivan Volosyuk <iv...@gmail.com>]

Let me summarize:

Sometimes we have small java heap objects which relates to much bigger
native memory chunks. In some cases this memory may exceed limits and
VM will crash. Calling GC from the other side will make this memory
free. There are quite a few place in VM and classlib which uses such
design pattern. In order to deal with this correctly some general
solution required.

We can do the following:

1. We can monitor virtual memory of running process via some hooks
from operating system or use a special thread to do so.
Actually I have already experimented with this for a while and it
become clear to me that it cannot be use reliably. We can detect
increase in virtual memory consumption when process virtual memory
increases, if running GC some of memory will be freed but virtual
memory consumption will not necessary decrease. Filling in the hidden
from use freed memory is not visible and then will see next phase on
virtual memory increase. After a few cycles we will end up with
process crash.

2. We can use custom native memory allocator which override system one
for all memory allocations or just for the ones originating from
VM/classlib.
We can have much more control on memory consumption then. Currently
I'm investigating this possibility. It require careful design in order
to make it work reliable, predictable and efficient. If somebody have
fresh ideas or can give links for corresponding articles we could
gather its here and develop a general solution.

3... ?

-- 
Ivan
Intel Enterprise Solutions Software Division

On 2/2/07, Gregory Shimansky <gs...@gmail.com> wrote:
> Leo Li wrote:
> > Hi, Xiao-Feng:
> >     Excuse me if I am confusing you.
> >     The direct byte buffer holds a block of native memory space used in nio
> > operation. The native byte buffer will be freed through the byte buffer is
> > gc collected and added to ReferenceQueue monitored by MemorySpy series
> > classes.
> >     But the gc will not be triggered if the java heap is still empty. Here
> > is an example: I allocate a large block of native memory for a byte buffer
> > while the byte buffer itself is quite small. Thus quickly native heap is
> > depleted while VM still does not think gc is needed since java heap has a
> > lot of free memory.
> >
> >
> > import java.nio.*;
> > public class Test {
> >    public static void main(String[] args) throws Exception {
> >        for(int i = 0;i<1000;i++)
> >        {
> >            ByteBuffer byteBuffer = ByteBuffer.allocateDirect(10240000);
> >            //System.gc();
> >        }
> >   }
> > }
> > RI runs well but Harmony will soon throw OutOfMemory exception.
> > But if remove the comment before System.gc(), Harmony will become ok.
> > So my point is that we need a mechanism to notify VM to start gc if we have
> > no more native memory because sometimes problem will be solved if gc is
> > fired. Of cause we cannot avoid a  user always malloc space but never free
> > them.:)
>
> ByteBuffer is not the only one class which has associated native
> resources that are freed when Java object is collected. And it is not
> the only class which may run out of such native resources before garbage
> collection is needed for Java heap. You can see a similar problem
> described in HARMONY-1831 where GZIPOutputStream allocates associated
> native memory in ZLIB which ends up before GC collects unused objects.
>
> Other types of native resources could be handles and other OS specific
> stuff. I am not sure, but it is probably easier for a particular type of
> Java API class to realize that there are no more specific native
> resources (like when ZLIB returns Z_MEM_ERROR in HARMONY-1831) left for
> it and trigger GC unconditionally, then retry allocating resource again.
>
> > On 2/2/07, Xiao-Feng Li <xi...@gmail.com> wrote:
> >>
> >> Leo, I can't fully understand your problem and proposal. I am working
> >> in DRLVM GC component, I want to understand how GC can tackle the
> >> issue you meet.
> >>
> >> You said the native memory is allocated (probably with malloc) for
> >> byte buffer. I wonder how the process goes. I assume it's in native
> >> code. Then you use malloc to allocate a block of memory, and put some
> >> Java object (byte buffer) into it? Is this what you do? I wonder how
> >> do you manipulate Java object placement in native code; and if the
> >> native memory block is out of Java heap, do you expect GC to manage
> >> it?
> >>
> >> You said when GC reclaims those byte buffer object, the native memory
> >> block can be freed in native code. How do you know if an object is
> >> reclaimed by GC or not?
> >>
> >> Thanks,
> >> xiaofeng
> >>
> >> On 2/2/07, Leo Li < liyilei1979@gmail.com> wrote:
> >> > Hi, all:
> >> >     After applying patch3073, I added the support for direct byte
> >> buffer
> >> in
> >> > nio module as spec requires. But now here exists a problem: The native
> >> > memory allocated to the byte buffer can be released when gc releases
> >> the
> >> > byte buffer object, however, if the native heap is full while java heap
> >> > still has space, gc will not be triggered.
> >> >     It seems that RI will start gc before native memory heap is
> >> depleted
> >> and
> >> > thus prevents out-of-memory error.
> >> >
> >> >    Then our work focuses on:
> >> >    1. When gc is required.
> >> >    2. Trigger a gc.
> >> >    The first one requires that we get support from operating system,
> >> since
> >> > the memory allocated in the native code, for example by malloc, is out
> >> of
> >> > the control of java VM.( I have ever thought of counting the used
> >> memory
> >> in
> >> > hymemory_allocate, but the plan fails since hymemory_free will not
> >> report
> >> > how much space is released.)
> >> >    The second one needs the help from VM. System.gc() is not so
> >> reliable
> >> as
> >> > spec says, so it is necessary to have a internal channel to notify
> >> VM to
> >> > start gc.
> >> >
> >> >     One solution, I think, is to let a monitor thread in VM to check
> >> whether
> >> > OS physical memory is low. For example, the
> >> QueryMemoryResourceNotification
> >> > of win32 API is a candidate. Although the interrupt model is more
> >> effective:
> >> > win32 SDK provides a CreateMemoryResourceNotification to get a handler
> >> on
> >> > which the monitor thread can wait, maybe on other platforms, OS does
> >> not
> >> > supply such a convenience. So the monitor thread in the VM might
> >> have to
> >>
> >> > check the OS resource once for a while and if necessary the monitor
> >> thread
> >> > will call a GC.
> >> >
> >> >    My suggestion is first to add some function to monitor memory in
> >> portlib,
> >> > since it is highly related to platforms and in portlib there has been
> >> > some useful tools.(Thanks Mark to point out that.) On the other
> >> hand, we
> >> can
> >> > negotiate a channel to trigger gc in VM. Actually I am not an expert on
> >> VM
> >> > since I am not sure whether there has been some monitor thread and the
> >> load
> >> > on performance if such a monitor thread is added to the VM...
> >> >
> >> >
> >> > --
> >> > Leo Li
> >> > China Software Development Lab, IBM
> >> >
> >> >
> >>
> >
> >
> >
>
>
> --
> Gregory
>
>

Re: [VM]How to trigue GC while free native memory is low.

[Gregory Shimansky <gs...@gmail.com>]

Leo Li wrote:
> Hi, Xiao-Feng:
>     Excuse me if I am confusing you.
>     The direct byte buffer holds a block of native memory space used in nio
> operation. The native byte buffer will be freed through the byte buffer is
> gc collected and added to ReferenceQueue monitored by MemorySpy series
> classes.
>     But the gc will not be triggered if the java heap is still empty. Here
> is an example: I allocate a large block of native memory for a byte buffer
> while the byte buffer itself is quite small. Thus quickly native heap is
> depleted while VM still does not think gc is needed since java heap has a
> lot of free memory.
> 
> 
> import java.nio.*;
> public class Test {
>    public static void main(String[] args) throws Exception {
>        for(int i = 0;i<1000;i++)
>        {
>            ByteBuffer byteBuffer = ByteBuffer.allocateDirect(10240000);
>            //System.gc();
>        }
>   }
> }
> RI runs well but Harmony will soon throw OutOfMemory exception.
> But if remove the comment before System.gc(), Harmony will become ok.
> So my point is that we need a mechanism to notify VM to start gc if we have
> no more native memory because sometimes problem will be solved if gc is
> fired. Of cause we cannot avoid a  user always malloc space but never free
> them.:)

ByteBuffer is not the only one class which has associated native 
resources that are freed when Java object is collected. And it is not 
the only class which may run out of such native resources before garbage 
collection is needed for Java heap. You can see a similar problem 
described in HARMONY-1831 where GZIPOutputStream allocates associated 
native memory in ZLIB which ends up before GC collects unused objects.

Other types of native resources could be handles and other OS specific 
stuff. I am not sure, but it is probably easier for a particular type of 
Java API class to realize that there are no more specific native 
resources (like when ZLIB returns Z_MEM_ERROR in HARMONY-1831) left for 
it and trigger GC unconditionally, then retry allocating resource again.

> On 2/2/07, Xiao-Feng Li <xi...@gmail.com> wrote:
>>
>> Leo, I can't fully understand your problem and proposal. I am working
>> in DRLVM GC component, I want to understand how GC can tackle the
>> issue you meet.
>>
>> You said the native memory is allocated (probably with malloc) for
>> byte buffer. I wonder how the process goes. I assume it's in native
>> code. Then you use malloc to allocate a block of memory, and put some
>> Java object (byte buffer) into it? Is this what you do? I wonder how
>> do you manipulate Java object placement in native code; and if the
>> native memory block is out of Java heap, do you expect GC to manage
>> it?
>>
>> You said when GC reclaims those byte buffer object, the native memory
>> block can be freed in native code. How do you know if an object is
>> reclaimed by GC or not?
>>
>> Thanks,
>> xiaofeng
>>
>> On 2/2/07, Leo Li < liyilei1979@gmail.com> wrote:
>> > Hi, all:
>> >     After applying patch3073, I added the support for direct byte 
>> buffer
>> in
>> > nio module as spec requires. But now here exists a problem: The native
>> > memory allocated to the byte buffer can be released when gc releases 
>> the
>> > byte buffer object, however, if the native heap is full while java heap
>> > still has space, gc will not be triggered.
>> >     It seems that RI will start gc before native memory heap is 
>> depleted
>> and
>> > thus prevents out-of-memory error.
>> >
>> >    Then our work focuses on:
>> >    1. When gc is required.
>> >    2. Trigger a gc.
>> >    The first one requires that we get support from operating system,
>> since
>> > the memory allocated in the native code, for example by malloc, is out
>> of
>> > the control of java VM.( I have ever thought of counting the used 
>> memory
>> in
>> > hymemory_allocate, but the plan fails since hymemory_free will not
>> report
>> > how much space is released.)
>> >    The second one needs the help from VM. System.gc() is not so 
>> reliable
>> as
>> > spec says, so it is necessary to have a internal channel to notify 
>> VM to
>> > start gc.
>> >
>> >     One solution, I think, is to let a monitor thread in VM to check
>> whether
>> > OS physical memory is low. For example, the
>> QueryMemoryResourceNotification
>> > of win32 API is a candidate. Although the interrupt model is more
>> effective:
>> > win32 SDK provides a CreateMemoryResourceNotification to get a handler
>> on
>> > which the monitor thread can wait, maybe on other platforms, OS does 
>> not
>> > supply such a convenience. So the monitor thread in the VM might 
>> have to
>>
>> > check the OS resource once for a while and if necessary the monitor
>> thread
>> > will call a GC.
>> >
>> >    My suggestion is first to add some function to monitor memory in
>> portlib,
>> > since it is highly related to platforms and in portlib there has been
>> > some useful tools.(Thanks Mark to point out that.) On the other 
>> hand, we
>> can
>> > negotiate a channel to trigger gc in VM. Actually I am not an expert on
>> VM
>> > since I am not sure whether there has been some monitor thread and the
>> load
>> > on performance if such a monitor thread is added to the VM...
>> >
>> >
>> > --
>> > Leo Li
>> > China Software Development Lab, IBM
>> >
>> >
>>
> 
> 
> 


-- 
Gregory

Re: [VM]How to trigue GC while free native memory is low.

[Ivan Volosyuk <iv...@gmail.com>]

On 2/5/07, Ivan Volosyuk <iv...@gmail.com> wrote:
> On 2/5/07, Geir Magnusson Jr. <ge...@pobox.com> wrote:
> >
> > On Feb 4, 2007, at 8:24 PM, Ivan Volosyuk wrote:
> >
> > > On 2/5/07, Ivan Volosyuk <iv...@gmail.com> wrote:
> > >> On 2/4/07, Geir Magnusson Jr. <ge...@pobox.com> wrote:
> > >> >
> > >> > On Feb 4, 2007, at 8:41 AM, Gregory Shimansky wrote:
> > >> >
> > >> > > Leo Li wrote:
> > >> > >> On 2/4/07, Gregory Shimansky <gs...@gmail.com> wrote:
> > >> > >>>
> > >> > >>> I see no difference in the approach I've suggested already.
> > >> If we
> > >> > >>> have
> > >> > >>> to take care about all the native resource allocation inside of
> > >> > >>> classlib
> > >> > >>> APIs, then there is no difference which code calls GC, be it
> > >> > >>> gc_native_malloc, or the API native code like this:
> > >> > >>>
> > >> > >>> void *p = malloc(size);
> > >> > >>> if (NULL == p)
> > >> > >>> {
> > >> > >>>   gc_force_gc();
> > >> > >>>   p = malloc(size);
> > >> > >>>   if (NULL == p)
> > >> > >>>   {
> > >> > >>>       jni_env->ThrowNew("java/lang/OutOfMemoryError");
> > >> > >>>       return;
> > >> > >>>   }
> > >> > >>> }
> > >> > >>
> > >> > >>
> > >> > >> But I am worried whether it is too late to force gc when memory
> > >> > >> allocation
> > >> > >> fails. It will take some time to complete the release of
> > >> > >> resources, for
> > >> > >> example, in finalizer() and this period of time is not
> > >> > >> predictable. Thus
> > >> > >> the
> > >> > >> malloc will still fails...:(
> > >> > >> Futhermore, if we wait till memory allocation fails then to
> > >> force
> > >> > >> gc, it
> > >> > >> might be possible that more than one thread will encounter
> > >> such a
> > >> > >> problem
> > >> > >> and force gc. I am not sure whether it will lead to problem
> > >> in vm.
> > >> > >> So, I think it may be wiser to trigger a gc when free memory
> > >> ratio is
> > >> > >> low and before the failure in malloc actually occurs.
> > >> > >
> > >> > > Yes, I shouldn't have called the above code a "solution". I just
> > >> > > wanted
> > >> > > to point to Ivan that there is no difference with taking care
> > >> about
> > >> > > memory allocation in GC code or in classlib code.
> > >> > >
> > >> > > You've found another problem in such approach, just running GC
> > >> is not
> > >> > > enough in this case because finalization may not be done in
> > >> time to
> > >> > > free
> > >> > > needed native resources.
> > >> > >
> > >> > > Speaking in DRLVM terms, the above code could also contain
> > >> call to
> > >> > > vm_run_pending_finalizers, but it is still not a solution for
> > >> > > multithreaded case.
> > >> >
> > >> > I'm getting confused on where people think that this problem
> > >> needs to
> > >> > be managed.  Am I misunderstanding that there should be some
> > >> > awareness of this in the classlib or VM natives?  Seems like it can
> > >> > be localized.
> > >> >
> > >> > I think that if whoever calls malloc() finds that the system is out
> > >> > of memory, then it's too late.  I always thought that VMs manage
> > >> > their conceptual native "heap" (all native resources consumed from
> > >> > OS, including as Robin mentioned, filehandles) in a proactive
> > >> manner,
> > >> > like they do their java heap.  it's clear now that they don't.
> > >> >
> > >> > So when a caller to malloc() gets a null, it should simply throw an
> > >> > OOM, assuming that the runtime environment did all it could to
> > >> > prevent that.
> > >> >
> > >> > Can this be done through portlib, letting the VMs implementation of
> > >> > it manage to whatever degree of sophistication is warranted?
> > >>
> > >> Actually, the portlib is the lowest layer we rely on. If we can have
> > >> here a callback to VM's GC we can place the code in the portlib. I
> > >> would also distinguish memory allocations which associated with java
> > >> objects and that which doesn't. No need to account resources which
> > >> will not be freed after garbage collection.
> > >>
> > >> As was mentioned we have different types of critical system
> > >> resources.
> > >> It makes sense to make a list of such resources:
> > >>    malloc'ed memory
> > >>    mmap'ed memory
> > >>    shared memory (?)
> > >>    file descriptors (files, sockets)
> > >>    XWin / GDI resources (?)
> > >>    more?
> > >>
> > >> What would be the accounting strategy for the resources? We could
> > >> allocate them via some functions in this facility or we could only
> > >> account them here and call GC using some buildin criteria.
> > >
> > > More thoughts.
> > >
> > > It makes sense that allocation or accounting routine will block for
> > > some time until finalizers path completes as Robin suggested.
> >
> > That's not obvious to me.  Without thinking too hard about it, I'd
> > rather get an OOM than block.  For example, suppose you had some kind
> > of transaction processing system with some SLA.  When processing a
> > transaction, a OOM would at least give me a chance to fail/rollback
> > and let the transaction be tried again on another machine, whereas a
> > block seems I lose any decision-making control as an app.
>
> It makes sense. (After thinking hard) I have come to conclusion that
> we just need to account the allocation rate of system resources and
> after certain thresholds for each resource type trigger a GC. It will
> help us to keep the resources at some specific level depending on
> usage pattern and in case of abuse will lead to out of resource
> condition. Quite simple solution and I have already saw it done in
> DRLVM's thread manager implementation.

BTW, GC should be able to reset all of this resource counters to avoid
extra garbage collections in case of extensive usage of several
critical resources.

-- 
Ivan
Intel Enterprise Solutions Software Division

Re: [VM]How to trigue GC while free native memory is low.

[Ivan Volosyuk <iv...@gmail.com>]

On 2/5/07, Geir Magnusson Jr. <ge...@pobox.com> wrote:
>
> On Feb 4, 2007, at 8:24 PM, Ivan Volosyuk wrote:
>
> > On 2/5/07, Ivan Volosyuk <iv...@gmail.com> wrote:
> >> On 2/4/07, Geir Magnusson Jr. <ge...@pobox.com> wrote:
> >> >
> >> > On Feb 4, 2007, at 8:41 AM, Gregory Shimansky wrote:
> >> >
> >> > > Leo Li wrote:
> >> > >> On 2/4/07, Gregory Shimansky <gs...@gmail.com> wrote:
> >> > >>>
> >> > >>> I see no difference in the approach I've suggested already.
> >> If we
> >> > >>> have
> >> > >>> to take care about all the native resource allocation inside of
> >> > >>> classlib
> >> > >>> APIs, then there is no difference which code calls GC, be it
> >> > >>> gc_native_malloc, or the API native code like this:
> >> > >>>
> >> > >>> void *p = malloc(size);
> >> > >>> if (NULL == p)
> >> > >>> {
> >> > >>>   gc_force_gc();
> >> > >>>   p = malloc(size);
> >> > >>>   if (NULL == p)
> >> > >>>   {
> >> > >>>       jni_env->ThrowNew("java/lang/OutOfMemoryError");
> >> > >>>       return;
> >> > >>>   }
> >> > >>> }
> >> > >>
> >> > >>
> >> > >> But I am worried whether it is too late to force gc when memory
> >> > >> allocation
> >> > >> fails. It will take some time to complete the release of
> >> > >> resources, for
> >> > >> example, in finalizer() and this period of time is not
> >> > >> predictable. Thus
> >> > >> the
> >> > >> malloc will still fails...:(
> >> > >> Futhermore, if we wait till memory allocation fails then to
> >> force
> >> > >> gc, it
> >> > >> might be possible that more than one thread will encounter
> >> such a
> >> > >> problem
> >> > >> and force gc. I am not sure whether it will lead to problem
> >> in vm.
> >> > >> So, I think it may be wiser to trigger a gc when free memory
> >> ratio is
> >> > >> low and before the failure in malloc actually occurs.
> >> > >
> >> > > Yes, I shouldn't have called the above code a "solution". I just
> >> > > wanted
> >> > > to point to Ivan that there is no difference with taking care
> >> about
> >> > > memory allocation in GC code or in classlib code.
> >> > >
> >> > > You've found another problem in such approach, just running GC
> >> is not
> >> > > enough in this case because finalization may not be done in
> >> time to
> >> > > free
> >> > > needed native resources.
> >> > >
> >> > > Speaking in DRLVM terms, the above code could also contain
> >> call to
> >> > > vm_run_pending_finalizers, but it is still not a solution for
> >> > > multithreaded case.
> >> >
> >> > I'm getting confused on where people think that this problem
> >> needs to
> >> > be managed.  Am I misunderstanding that there should be some
> >> > awareness of this in the classlib or VM natives?  Seems like it can
> >> > be localized.
> >> >
> >> > I think that if whoever calls malloc() finds that the system is out
> >> > of memory, then it's too late.  I always thought that VMs manage
> >> > their conceptual native "heap" (all native resources consumed from
> >> > OS, including as Robin mentioned, filehandles) in a proactive
> >> manner,
> >> > like they do their java heap.  it's clear now that they don't.
> >> >
> >> > So when a caller to malloc() gets a null, it should simply throw an
> >> > OOM, assuming that the runtime environment did all it could to
> >> > prevent that.
> >> >
> >> > Can this be done through portlib, letting the VMs implementation of
> >> > it manage to whatever degree of sophistication is warranted?
> >>
> >> Actually, the portlib is the lowest layer we rely on. If we can have
> >> here a callback to VM's GC we can place the code in the portlib. I
> >> would also distinguish memory allocations which associated with java
> >> objects and that which doesn't. No need to account resources which
> >> will not be freed after garbage collection.
> >>
> >> As was mentioned we have different types of critical system
> >> resources.
> >> It makes sense to make a list of such resources:
> >>    malloc'ed memory
> >>    mmap'ed memory
> >>    shared memory (?)
> >>    file descriptors (files, sockets)
> >>    XWin / GDI resources (?)
> >>    more?
> >>
> >> What would be the accounting strategy for the resources? We could
> >> allocate them via some functions in this facility or we could only
> >> account them here and call GC using some buildin criteria.
> >
> > More thoughts.
> >
> > It makes sense that allocation or accounting routine will block for
> > some time until finalizers path completes as Robin suggested.
>
> That's not obvious to me.  Without thinking too hard about it, I'd
> rather get an OOM than block.  For example, suppose you had some kind
> of transaction processing system with some SLA.  When processing a
> transaction, a OOM would at least give me a chance to fail/rollback
> and let the transaction be tried again on another machine, whereas a
> block seems I lose any decision-making control as an app.

It makes sense. (After thinking hard) I have come to conclusion that
we just need to account the allocation rate of system resources and
after certain thresholds for each resource type trigger a GC. It will
help us to keep the resources at some specific level depending on
usage pattern and in case of abuse will lead to out of resource
condition. Quite simple solution and I have already saw it done in
DRLVM's thread manager implementation.

-- 
Ivan
Intel Enterprise Solutions Software Division

Re: [VM]How to trigue GC while free native memory is low.

[Tim Ellison <t....@gmail.com>]

Ivan Volosyuk wrote:
> FreeDirect is quite implementable. It is strange that it is not exist
> in API. May be it is done this way to be conceptually the same as
> ordinary java objects.

I believe the design for NIO buffers is quite nice for the API user --
and leaves the implementers to jump through the hoops, which is arguably
the right way around.

So as a user I can call:
  java.nio.ByteBuffer#allocate(int)   or
  java.nio.ByteBuffer#allocateDirect(int)

(there is a case that the user should not have to decide between direct
or not, but that's how it is)

Then in each case the user can discard the Java object to clean up the
memory.  It is the regular Java object lifecycle in each case, rather
than an alloc/free model, and no opportunity for invalid pointer
exceptions<g>.

In Harmony, ByteBuffer#allocateDirect(int) returns an instance of the
concrete package private type called java.nio.DirectByteBuffer, which
implements #free(), though of course apps cannot call that method
without first casting to the Harmony-specific type.

Regards,
Tim

Re: [VM]How to trigue GC while free native memory is low.

[Ivan Volosyuk <iv...@gmail.com>]

On 2/5/07, Xiao-Feng Li <xi...@gmail.com> wrote:
> On 2/5/07, Geir Magnusson Jr. <ge...@pobox.com> wrote:
> >
> > On Feb 4, 2007, at 8:24 PM, Ivan Volosyuk wrote:
> >
> > > On 2/5/07, Ivan Volosyuk <iv...@gmail.com> wrote:
> > >> On 2/4/07, Geir Magnusson Jr. <ge...@pobox.com> wrote:
> > >> >
> > >> > On Feb 4, 2007, at 8:41 AM, Gregory Shimansky wrote:
> > >> >
> > >> > > Leo Li wrote:
> > >> > >> On 2/4/07, Gregory Shimansky <gs...@gmail.com> wrote:
> > >> > >>>
> > >> > >>> I see no difference in the approach I've suggested already.
> > >> If we
> > >> > >>> have
> > >> > >>> to take care about all the native resource allocation inside of
> > >> > >>> classlib
> > >> > >>> APIs, then there is no difference which code calls GC, be it
> > >> > >>> gc_native_malloc, or the API native code like this:
> > >> > >>>
> > >> > >>> void *p = malloc(size);
> > >> > >>> if (NULL == p)
> > >> > >>> {
> > >> > >>>   gc_force_gc();
> > >> > >>>   p = malloc(size);
> > >> > >>>   if (NULL == p)
> > >> > >>>   {
> > >> > >>>       jni_env->ThrowNew("java/lang/OutOfMemoryError");
> > >> > >>>       return;
> > >> > >>>   }
> > >> > >>> }
> > >> > >>
> > >> > >>
> > >> > >> But I am worried whether it is too late to force gc when memory
> > >> > >> allocation
> > >> > >> fails. It will take some time to complete the release of
> > >> > >> resources, for
> > >> > >> example, in finalizer() and this period of time is not
> > >> > >> predictable. Thus
> > >> > >> the
> > >> > >> malloc will still fails...:(
> > >> > >> Futhermore, if we wait till memory allocation fails then to
> > >> force
> > >> > >> gc, it
> > >> > >> might be possible that more than one thread will encounter
> > >> such a
> > >> > >> problem
> > >> > >> and force gc. I am not sure whether it will lead to problem
> > >> in vm.
> > >> > >> So, I think it may be wiser to trigger a gc when free memory
> > >> ratio is
> > >> > >> low and before the failure in malloc actually occurs.
> > >> > >
> > >> > > Yes, I shouldn't have called the above code a "solution". I just
> > >> > > wanted
> > >> > > to point to Ivan that there is no difference with taking care
> > >> about
> > >> > > memory allocation in GC code or in classlib code.
> > >> > >
> > >> > > You've found another problem in such approach, just running GC
> > >> is not
> > >> > > enough in this case because finalization may not be done in
> > >> time to
> > >> > > free
> > >> > > needed native resources.
> > >> > >
> > >> > > Speaking in DRLVM terms, the above code could also contain
> > >> call to
> > >> > > vm_run_pending_finalizers, but it is still not a solution for
> > >> > > multithreaded case.
> > >> >
> > >> > I'm getting confused on where people think that this problem
> > >> needs to
> > >> > be managed.  Am I misunderstanding that there should be some
> > >> > awareness of this in the classlib or VM natives?  Seems like it can
> > >> > be localized.
> > >> >
> > >> > I think that if whoever calls malloc() finds that the system is out
> > >> > of memory, then it's too late.  I always thought that VMs manage
> > >> > their conceptual native "heap" (all native resources consumed from
> > >> > OS, including as Robin mentioned, filehandles) in a proactive
> > >> manner,
> > >> > like they do their java heap.  it's clear now that they don't.
> > >> >
> > >> > So when a caller to malloc() gets a null, it should simply throw an
> > >> > OOM, assuming that the runtime environment did all it could to
> > >> > prevent that.
> > >> >
> > >> > Can this be done through portlib, letting the VMs implementation of
> > >> > it manage to whatever degree of sophistication is warranted?
> > >>
> > >> Actually, the portlib is the lowest layer we rely on. If we can have
> > >> here a callback to VM's GC we can place the code in the portlib. I
> > >> would also distinguish memory allocations which associated with java
> > >> objects and that which doesn't. No need to account resources which
> > >> will not be freed after garbage collection.
> > >>
> > >> As was mentioned we have different types of critical system
> > >> resources.
> > >> It makes sense to make a list of such resources:
> > >>    malloc'ed memory
> > >>    mmap'ed memory
> > >>    shared memory (?)
> > >>    file descriptors (files, sockets)
> > >>    XWin / GDI resources (?)
> > >>    more?
> > >>
> > >> What would be the accounting strategy for the resources? We could
> > >> allocate them via some functions in this facility or we could only
> > >> account them here and call GC using some buildin criteria.
> > >
> > > More thoughts.
> > >
> > > It makes sense that allocation or accounting routine will block for
> > > some time until finalizers path completes as Robin suggested.
> >
> > That's not obvious to me.  Without thinking too hard about it, I'd
> > rather get an OOM than block.  For example, suppose you had some kind
> > of transaction processing system with some SLA.  When processing a
> > transaction, a OOM would at least give me a chance to fail/rollback
> > and let the transaction be tried again on another machine, whereas a
> > block seems I lose any decision-making control as an app.
> >
>
> I tend to think the same thing. That's why I wonder why the API has
> mallocDirect but no freeDirect. If memory management is a special
> exception (since VM has certain memory management facility like GC),
> I'd wish other native resources have their respective management APIs
> for application to use, or the application has to reply on VM's
> disposal.

FreeDirect is quite implementable. It is strange that it is not exist
in API. May be it is done this way to be conceptually the same as
ordinary java objects.

-- 
Ivan
Intel Enterprise Solutions Software Division

Re: [VM]How to trigue GC while free native memory is low.

[Xiao-Feng Li <xi...@gmail.com>]

On 2/5/07, Geir Magnusson Jr. <ge...@pobox.com> wrote:
>
> On Feb 4, 2007, at 8:24 PM, Ivan Volosyuk wrote:
>
> > On 2/5/07, Ivan Volosyuk <iv...@gmail.com> wrote:
> >> On 2/4/07, Geir Magnusson Jr. <ge...@pobox.com> wrote:
> >> >
> >> > On Feb 4, 2007, at 8:41 AM, Gregory Shimansky wrote:
> >> >
> >> > > Leo Li wrote:
> >> > >> On 2/4/07, Gregory Shimansky <gs...@gmail.com> wrote:
> >> > >>>
> >> > >>> I see no difference in the approach I've suggested already.
> >> If we
> >> > >>> have
> >> > >>> to take care about all the native resource allocation inside of
> >> > >>> classlib
> >> > >>> APIs, then there is no difference which code calls GC, be it
> >> > >>> gc_native_malloc, or the API native code like this:
> >> > >>>
> >> > >>> void *p = malloc(size);
> >> > >>> if (NULL == p)
> >> > >>> {
> >> > >>>   gc_force_gc();
> >> > >>>   p = malloc(size);
> >> > >>>   if (NULL == p)
> >> > >>>   {
> >> > >>>       jni_env->ThrowNew("java/lang/OutOfMemoryError");
> >> > >>>       return;
> >> > >>>   }
> >> > >>> }
> >> > >>
> >> > >>
> >> > >> But I am worried whether it is too late to force gc when memory
> >> > >> allocation
> >> > >> fails. It will take some time to complete the release of
> >> > >> resources, for
> >> > >> example, in finalizer() and this period of time is not
> >> > >> predictable. Thus
> >> > >> the
> >> > >> malloc will still fails...:(
> >> > >> Futhermore, if we wait till memory allocation fails then to
> >> force
> >> > >> gc, it
> >> > >> might be possible that more than one thread will encounter
> >> such a
> >> > >> problem
> >> > >> and force gc. I am not sure whether it will lead to problem
> >> in vm.
> >> > >> So, I think it may be wiser to trigger a gc when free memory
> >> ratio is
> >> > >> low and before the failure in malloc actually occurs.
> >> > >
> >> > > Yes, I shouldn't have called the above code a "solution". I just
> >> > > wanted
> >> > > to point to Ivan that there is no difference with taking care
> >> about
> >> > > memory allocation in GC code or in classlib code.
> >> > >
> >> > > You've found another problem in such approach, just running GC
> >> is not
> >> > > enough in this case because finalization may not be done in
> >> time to
> >> > > free
> >> > > needed native resources.
> >> > >
> >> > > Speaking in DRLVM terms, the above code could also contain
> >> call to
> >> > > vm_run_pending_finalizers, but it is still not a solution for
> >> > > multithreaded case.
> >> >
> >> > I'm getting confused on where people think that this problem
> >> needs to
> >> > be managed.  Am I misunderstanding that there should be some
> >> > awareness of this in the classlib or VM natives?  Seems like it can
> >> > be localized.
> >> >
> >> > I think that if whoever calls malloc() finds that the system is out
> >> > of memory, then it's too late.  I always thought that VMs manage
> >> > their conceptual native "heap" (all native resources consumed from
> >> > OS, including as Robin mentioned, filehandles) in a proactive
> >> manner,
> >> > like they do their java heap.  it's clear now that they don't.
> >> >
> >> > So when a caller to malloc() gets a null, it should simply throw an
> >> > OOM, assuming that the runtime environment did all it could to
> >> > prevent that.
> >> >
> >> > Can this be done through portlib, letting the VMs implementation of
> >> > it manage to whatever degree of sophistication is warranted?
> >>
> >> Actually, the portlib is the lowest layer we rely on. If we can have
> >> here a callback to VM's GC we can place the code in the portlib. I
> >> would also distinguish memory allocations which associated with java
> >> objects and that which doesn't. No need to account resources which
> >> will not be freed after garbage collection.
> >>
> >> As was mentioned we have different types of critical system
> >> resources.
> >> It makes sense to make a list of such resources:
> >>    malloc'ed memory
> >>    mmap'ed memory
> >>    shared memory (?)
> >>    file descriptors (files, sockets)
> >>    XWin / GDI resources (?)
> >>    more?
> >>
> >> What would be the accounting strategy for the resources? We could
> >> allocate them via some functions in this facility or we could only
> >> account them here and call GC using some buildin criteria.
> >
> > More thoughts.
> >
> > It makes sense that allocation or accounting routine will block for
> > some time until finalizers path completes as Robin suggested.
>
> That's not obvious to me.  Without thinking too hard about it, I'd
> rather get an OOM than block.  For example, suppose you had some kind
> of transaction processing system with some SLA.  When processing a
> transaction, a OOM would at least give me a chance to fail/rollback
> and let the transaction be tried again on another machine, whereas a
> block seems I lose any decision-making control as an app.
>

I tend to think the same thing. That's why I wonder why the API has
mallocDirect but no freeDirect. If memory management is a special
exception (since VM has certain memory management facility like GC),
I'd wish other native resources have their respective management APIs
for application to use, or the application has to reply on VM's
disposal.

Thanks,
xiaofeng

Re: [VM]How to trigue GC while free native memory is low.

["Geir Magnusson Jr." <ge...@pobox.com>]

On Feb 4, 2007, at 8:24 PM, Ivan Volosyuk wrote:

> On 2/5/07, Ivan Volosyuk <iv...@gmail.com> wrote:
>> On 2/4/07, Geir Magnusson Jr. <ge...@pobox.com> wrote:
>> >
>> > On Feb 4, 2007, at 8:41 AM, Gregory Shimansky wrote:
>> >
>> > > Leo Li wrote:
>> > >> On 2/4/07, Gregory Shimansky <gs...@gmail.com> wrote:
>> > >>>
>> > >>> I see no difference in the approach I've suggested already.  
>> If we
>> > >>> have
>> > >>> to take care about all the native resource allocation inside of
>> > >>> classlib
>> > >>> APIs, then there is no difference which code calls GC, be it
>> > >>> gc_native_malloc, or the API native code like this:
>> > >>>
>> > >>> void *p = malloc(size);
>> > >>> if (NULL == p)
>> > >>> {
>> > >>>   gc_force_gc();
>> > >>>   p = malloc(size);
>> > >>>   if (NULL == p)
>> > >>>   {
>> > >>>       jni_env->ThrowNew("java/lang/OutOfMemoryError");
>> > >>>       return;
>> > >>>   }
>> > >>> }
>> > >>
>> > >>
>> > >> But I am worried whether it is too late to force gc when memory
>> > >> allocation
>> > >> fails. It will take some time to complete the release of
>> > >> resources, for
>> > >> example, in finalizer() and this period of time is not
>> > >> predictable. Thus
>> > >> the
>> > >> malloc will still fails...:(
>> > >> Futhermore, if we wait till memory allocation fails then to  
>> force
>> > >> gc, it
>> > >> might be possible that more than one thread will encounter  
>> such a
>> > >> problem
>> > >> and force gc. I am not sure whether it will lead to problem  
>> in vm.
>> > >> So, I think it may be wiser to trigger a gc when free memory  
>> ratio is
>> > >> low and before the failure in malloc actually occurs.
>> > >
>> > > Yes, I shouldn't have called the above code a "solution". I just
>> > > wanted
>> > > to point to Ivan that there is no difference with taking care  
>> about
>> > > memory allocation in GC code or in classlib code.
>> > >
>> > > You've found another problem in such approach, just running GC  
>> is not
>> > > enough in this case because finalization may not be done in  
>> time to
>> > > free
>> > > needed native resources.
>> > >
>> > > Speaking in DRLVM terms, the above code could also contain  
>> call to
>> > > vm_run_pending_finalizers, but it is still not a solution for
>> > > multithreaded case.
>> >
>> > I'm getting confused on where people think that this problem  
>> needs to
>> > be managed.  Am I misunderstanding that there should be some
>> > awareness of this in the classlib or VM natives?  Seems like it can
>> > be localized.
>> >
>> > I think that if whoever calls malloc() finds that the system is out
>> > of memory, then it's too late.  I always thought that VMs manage
>> > their conceptual native "heap" (all native resources consumed from
>> > OS, including as Robin mentioned, filehandles) in a proactive  
>> manner,
>> > like they do their java heap.  it's clear now that they don't.
>> >
>> > So when a caller to malloc() gets a null, it should simply throw an
>> > OOM, assuming that the runtime environment did all it could to
>> > prevent that.
>> >
>> > Can this be done through portlib, letting the VMs implementation of
>> > it manage to whatever degree of sophistication is warranted?
>>
>> Actually, the portlib is the lowest layer we rely on. If we can have
>> here a callback to VM's GC we can place the code in the portlib. I
>> would also distinguish memory allocations which associated with java
>> objects and that which doesn't. No need to account resources which
>> will not be freed after garbage collection.
>>
>> As was mentioned we have different types of critical system  
>> resources.
>> It makes sense to make a list of such resources:
>>    malloc'ed memory
>>    mmap'ed memory
>>    shared memory (?)
>>    file descriptors (files, sockets)
>>    XWin / GDI resources (?)
>>    more?
>>
>> What would be the accounting strategy for the resources? We could
>> allocate them via some functions in this facility or we could only
>> account them here and call GC using some buildin criteria.
>
> More thoughts.
>
> It makes sense that allocation or accounting routine will block for
> some time until finalizers path completes as Robin suggested.

That's not obvious to me.  Without thinking too hard about it, I'd  
rather get an OOM than block.  For example, suppose you had some kind  
of transaction processing system with some SLA.  When processing a  
transaction, a OOM would at least give me a chance to fail/rollback  
and let the transaction be tried again on another machine, whereas a  
block seems I lose any decision-making control as an app.

geir


> To
> prevent deadlocks we should avoid blocking finalizer threads, though
> it can cause unavoidable massive-allocation-in-finalizer attack. Do we
> have any more possible deadlocks here?
>
> -- 
> Ivan
> Intel Enterprise Solutions Software Division

Re: [VM]How to trigue GC while free native memory is low.

[Ivan Volosyuk <iv...@gmail.com>]

On 2/5/07, Ivan Volosyuk <iv...@gmail.com> wrote:
> On 2/4/07, Geir Magnusson Jr. <ge...@pobox.com> wrote:
> >
> > On Feb 4, 2007, at 8:41 AM, Gregory Shimansky wrote:
> >
> > > Leo Li wrote:
> > >> On 2/4/07, Gregory Shimansky <gs...@gmail.com> wrote:
> > >>>
> > >>> I see no difference in the approach I've suggested already. If we
> > >>> have
> > >>> to take care about all the native resource allocation inside of
> > >>> classlib
> > >>> APIs, then there is no difference which code calls GC, be it
> > >>> gc_native_malloc, or the API native code like this:
> > >>>
> > >>> void *p = malloc(size);
> > >>> if (NULL == p)
> > >>> {
> > >>>   gc_force_gc();
> > >>>   p = malloc(size);
> > >>>   if (NULL == p)
> > >>>   {
> > >>>       jni_env->ThrowNew("java/lang/OutOfMemoryError");
> > >>>       return;
> > >>>   }
> > >>> }
> > >>
> > >>
> > >> But I am worried whether it is too late to force gc when memory
> > >> allocation
> > >> fails. It will take some time to complete the release of
> > >> resources, for
> > >> example, in finalizer() and this period of time is not
> > >> predictable. Thus
> > >> the
> > >> malloc will still fails...:(
> > >> Futhermore, if we wait till memory allocation fails then to force
> > >> gc, it
> > >> might be possible that more than one thread will encounter such a
> > >> problem
> > >> and force gc. I am not sure whether it will lead to problem in vm.
> > >> So, I think it may be wiser to trigger a gc when free memory ratio is
> > >> low and before the failure in malloc actually occurs.
> > >
> > > Yes, I shouldn't have called the above code a "solution". I just
> > > wanted
> > > to point to Ivan that there is no difference with taking care about
> > > memory allocation in GC code or in classlib code.
> > >
> > > You've found another problem in such approach, just running GC is not
> > > enough in this case because finalization may not be done in time to
> > > free
> > > needed native resources.
> > >
> > > Speaking in DRLVM terms, the above code could also contain call to
> > > vm_run_pending_finalizers, but it is still not a solution for
> > > multithreaded case.
> >
> > I'm getting confused on where people think that this problem needs to
> > be managed.  Am I misunderstanding that there should be some
> > awareness of this in the classlib or VM natives?  Seems like it can
> > be localized.
> >
> > I think that if whoever calls malloc() finds that the system is out
> > of memory, then it's too late.  I always thought that VMs manage
> > their conceptual native "heap" (all native resources consumed from
> > OS, including as Robin mentioned, filehandles) in a proactive manner,
> > like they do their java heap.  it's clear now that they don't.
> >
> > So when a caller to malloc() gets a null, it should simply throw an
> > OOM, assuming that the runtime environment did all it could to
> > prevent that.
> >
> > Can this be done through portlib, letting the VMs implementation of
> > it manage to whatever degree of sophistication is warranted?
>
> Actually, the portlib is the lowest layer we rely on. If we can have
> here a callback to VM's GC we can place the code in the portlib. I
> would also distinguish memory allocations which associated with java
> objects and that which doesn't. No need to account resources which
> will not be freed after garbage collection.
>
> As was mentioned we have different types of critical system resources.
> It makes sense to make a list of such resources:
>    malloc'ed memory
>    mmap'ed memory
>    shared memory (?)
>    file descriptors (files, sockets)
>    XWin / GDI resources (?)
>    more?
>
> What would be the accounting strategy for the resources? We could
> allocate them via some functions in this facility or we could only
> account them here and call GC using some buildin criteria.

More thoughts.

It makes sense that allocation or accounting routine will block for
some time until finalizers path completes as Robin suggested. To
prevent deadlocks we should avoid blocking finalizer threads, though
it can cause unavoidable massive-allocation-in-finalizer attack. Do we
have any more possible deadlocks here?

-- 
Ivan
Intel Enterprise Solutions Software Division

Re: [VM]How to trigue GC while free native memory is low.

[Ivan Volosyuk <iv...@gmail.com>]

On 2/5/07, Tim Ellison <t....@gmail.com> wrote:
> Geir Magnusson Jr. wrote:
> > On Feb 4, 2007, at 8:13 PM, Ivan Volosyuk wrote:
> >> On 2/4/07, Geir Magnusson Jr. <ge...@pobox.com> wrote:
> >>> I'm getting confused on where people think that this problem needs to
> >>> be managed.  Am I misunderstanding that there should be some
> >>> awareness of this in the classlib or VM natives?  Seems like it can
> >>> be localized.
> >>>
> >>> I think that if whoever calls malloc() finds that the system is out
> >>> of memory, then it's too late.  I always thought that VMs manage
> >>> their conceptual native "heap" (all native resources consumed from
> >>> OS, including as Robin mentioned, filehandles) in a proactive manner,
> >>> like they do their java heap.  it's clear now that they don't.
> >>>
> >>> So when a caller to malloc() gets a null, it should simply throw an
> >>> OOM, assuming that the runtime environment did all it could to
> >>> prevent that.
> >>>
> >>> Can this be done through portlib, letting the VMs implementation of
> >>> it manage to whatever degree of sophistication is warranted?
> >>
> >> Actually, the portlib is the lowest layer we rely on. If we can have
> >> here a callback to VM's GC we can place the code in the portlib. I
> >> would also distinguish memory allocations which associated with java
> >> objects and that which doesn't. No need to account resources which
> >> will not be freed after garbage collection.
> >
> > That's true.  I was just thinking about keeping a clear idea of the
> > native resource usage, but that's actually a separate concept from
> > coupling to java objects and their lifecycle.
>
> I agree, and it is generally a good idea for the VM to track it's
> resource usage through the portlib so that apps can embed the VM with
> conditions that it uses no more than a given set of limits.  The class
> library can do the same thing in the core libraries, but it soon gets
> out of hand when you start interacting with more third-party libraries
> that are not telling you about their resource usage, or have limits of
> their own (such as GUI libraries).

I see here the main issue with this approach. Yes, third-party
libraries may not tell us about their resource usage.

Thus it is quite interesting to look at existing commercial VMs how
its deal with this situation. If such resources are not tracked by the
VM, then it may be possible for the library to account its resources
and call GC itself. But what should be done if we have thin java layer
on top of complicated third-party native library which provides very
complicated system using a lot of native resources, for example KDE
java bindings.

>
> >> As was mentioned we have different types of critical system resources.
> >> It makes sense to make a list of such resources:
> >>   malloc'ed memory
> >>   mmap'ed memory
> >>   shared memory (?)
> >>   file descriptors (files, sockets)
> >>   XWin / GDI resources (?)
> >>   more?
> >>
> >> What would be the accounting strategy for the resources? We could
> >> allocate them via some functions in this facility or we could only
> >> account them here and call GC using some buildin criteria.
> >
> > It's clear to me that accounting and "free when associated java object
> > is free" are two different notions...
>
> Yes.
>
> Regards,
> Tim
>


-- 
Ivan
Intel Enterprise Solutions Software Division

Re: [VM]How to trigue GC while free native memory is low.

[Tim Ellison <t....@gmail.com>]

Geir Magnusson Jr. wrote:
> On Feb 4, 2007, at 8:13 PM, Ivan Volosyuk wrote:
>> On 2/4/07, Geir Magnusson Jr. <ge...@pobox.com> wrote:
>>> I'm getting confused on where people think that this problem needs to
>>> be managed.  Am I misunderstanding that there should be some
>>> awareness of this in the classlib or VM natives?  Seems like it can
>>> be localized.
>>>
>>> I think that if whoever calls malloc() finds that the system is out
>>> of memory, then it's too late.  I always thought that VMs manage
>>> their conceptual native "heap" (all native resources consumed from
>>> OS, including as Robin mentioned, filehandles) in a proactive manner,
>>> like they do their java heap.  it's clear now that they don't.
>>>
>>> So when a caller to malloc() gets a null, it should simply throw an
>>> OOM, assuming that the runtime environment did all it could to
>>> prevent that.
>>>
>>> Can this be done through portlib, letting the VMs implementation of
>>> it manage to whatever degree of sophistication is warranted?
>>
>> Actually, the portlib is the lowest layer we rely on. If we can have
>> here a callback to VM's GC we can place the code in the portlib. I
>> would also distinguish memory allocations which associated with java
>> objects and that which doesn't. No need to account resources which
>> will not be freed after garbage collection.
> 
> That's true.  I was just thinking about keeping a clear idea of the
> native resource usage, but that's actually a separate concept from
> coupling to java objects and their lifecycle.

I agree, and it is generally a good idea for the VM to track it's
resource usage through the portlib so that apps can embed the VM with
conditions that it uses no more than a given set of limits.  The class
library can do the same thing in the core libraries, but it soon gets
out of hand when you start interacting with more third-party libraries
that are not telling you about their resource usage, or have limits of
their own (such as GUI libraries).

>> As was mentioned we have different types of critical system resources.
>> It makes sense to make a list of such resources:
>>   malloc'ed memory
>>   mmap'ed memory
>>   shared memory (?)
>>   file descriptors (files, sockets)
>>   XWin / GDI resources (?)
>>   more?
>>
>> What would be the accounting strategy for the resources? We could
>> allocate them via some functions in this facility or we could only
>> account them here and call GC using some buildin criteria.
> 
> It's clear to me that accounting and "free when associated java object
> is free" are two different notions...

Yes.

Regards,
Tim

Re: [VM]How to trigue GC while free native memory is low.

["Geir Magnusson Jr." <ge...@pobox.com>]

On Feb 4, 2007, at 8:13 PM, Ivan Volosyuk wrote:

> On 2/4/07, Geir Magnusson Jr. <ge...@pobox.com> wrote:
>>
>> On Feb 4, 2007, at 8:41 AM, Gregory Shimansky wrote:
>>
>> > Leo Li wrote:
>> >> On 2/4/07, Gregory Shimansky <gs...@gmail.com> wrote:
>> >>>
>> >>> I see no difference in the approach I've suggested already. If we
>> >>> have
>> >>> to take care about all the native resource allocation inside of
>> >>> classlib
>> >>> APIs, then there is no difference which code calls GC, be it
>> >>> gc_native_malloc, or the API native code like this:
>> >>>
>> >>> void *p = malloc(size);
>> >>> if (NULL == p)
>> >>> {
>> >>>   gc_force_gc();
>> >>>   p = malloc(size);
>> >>>   if (NULL == p)
>> >>>   {
>> >>>       jni_env->ThrowNew("java/lang/OutOfMemoryError");
>> >>>       return;
>> >>>   }
>> >>> }
>> >>
>> >>
>> >> But I am worried whether it is too late to force gc when memory
>> >> allocation
>> >> fails. It will take some time to complete the release of
>> >> resources, for
>> >> example, in finalizer() and this period of time is not
>> >> predictable. Thus
>> >> the
>> >> malloc will still fails...:(
>> >> Futhermore, if we wait till memory allocation fails then to force
>> >> gc, it
>> >> might be possible that more than one thread will encounter such a
>> >> problem
>> >> and force gc. I am not sure whether it will lead to problem in vm.
>> >> So, I think it may be wiser to trigger a gc when free memory  
>> ratio is
>> >> low and before the failure in malloc actually occurs.
>> >
>> > Yes, I shouldn't have called the above code a "solution". I just
>> > wanted
>> > to point to Ivan that there is no difference with taking care about
>> > memory allocation in GC code or in classlib code.
>> >
>> > You've found another problem in such approach, just running GC  
>> is not
>> > enough in this case because finalization may not be done in time to
>> > free
>> > needed native resources.
>> >
>> > Speaking in DRLVM terms, the above code could also contain call to
>> > vm_run_pending_finalizers, but it is still not a solution for
>> > multithreaded case.
>>
>> I'm getting confused on where people think that this problem needs to
>> be managed.  Am I misunderstanding that there should be some
>> awareness of this in the classlib or VM natives?  Seems like it can
>> be localized.
>>
>> I think that if whoever calls malloc() finds that the system is out
>> of memory, then it's too late.  I always thought that VMs manage
>> their conceptual native "heap" (all native resources consumed from
>> OS, including as Robin mentioned, filehandles) in a proactive manner,
>> like they do their java heap.  it's clear now that they don't.
>>
>> So when a caller to malloc() gets a null, it should simply throw an
>> OOM, assuming that the runtime environment did all it could to
>> prevent that.
>>
>> Can this be done through portlib, letting the VMs implementation of
>> it manage to whatever degree of sophistication is warranted?
>
> Actually, the portlib is the lowest layer we rely on. If we can have
> here a callback to VM's GC we can place the code in the portlib. I
> would also distinguish memory allocations which associated with java
> objects and that which doesn't. No need to account resources which
> will not be freed after garbage collection.

That's true.  I was just thinking about keeping a clear idea of the  
native resource usage, but that's actually a separate concept from  
coupling to java objects and their lifecycle.

>
> As was mentioned we have different types of critical system resources.
> It makes sense to make a list of such resources:
>   malloc'ed memory
>   mmap'ed memory
>   shared memory (?)
>   file descriptors (files, sockets)
>   XWin / GDI resources (?)
>   more?
>
> What would be the accounting strategy for the resources? We could
> allocate them via some functions in this facility or we could only
> account them here and call GC using some buildin criteria.

It's clear to me that accounting and "free when associated java  
object is free" are two different notions...

geir

>
> -- 
> Ivan
> Intel Enterprise Solutions Software Division

Re: [VM]How to trigue GC while free native memory is low.

[Ivan Volosyuk <iv...@gmail.com>]

On 2/4/07, Geir Magnusson Jr. <ge...@pobox.com> wrote:
>
> On Feb 4, 2007, at 8:41 AM, Gregory Shimansky wrote:
>
> > Leo Li wrote:
> >> On 2/4/07, Gregory Shimansky <gs...@gmail.com> wrote:
> >>>
> >>> I see no difference in the approach I've suggested already. If we
> >>> have
> >>> to take care about all the native resource allocation inside of
> >>> classlib
> >>> APIs, then there is no difference which code calls GC, be it
> >>> gc_native_malloc, or the API native code like this:
> >>>
> >>> void *p = malloc(size);
> >>> if (NULL == p)
> >>> {
> >>>   gc_force_gc();
> >>>   p = malloc(size);
> >>>   if (NULL == p)
> >>>   {
> >>>       jni_env->ThrowNew("java/lang/OutOfMemoryError");
> >>>       return;
> >>>   }
> >>> }
> >>
> >>
> >> But I am worried whether it is too late to force gc when memory
> >> allocation
> >> fails. It will take some time to complete the release of
> >> resources, for
> >> example, in finalizer() and this period of time is not
> >> predictable. Thus
> >> the
> >> malloc will still fails...:(
> >> Futhermore, if we wait till memory allocation fails then to force
> >> gc, it
> >> might be possible that more than one thread will encounter such a
> >> problem
> >> and force gc. I am not sure whether it will lead to problem in vm.
> >> So, I think it may be wiser to trigger a gc when free memory ratio is
> >> low and before the failure in malloc actually occurs.
> >
> > Yes, I shouldn't have called the above code a "solution". I just
> > wanted
> > to point to Ivan that there is no difference with taking care about
> > memory allocation in GC code or in classlib code.
> >
> > You've found another problem in such approach, just running GC is not
> > enough in this case because finalization may not be done in time to
> > free
> > needed native resources.
> >
> > Speaking in DRLVM terms, the above code could also contain call to
> > vm_run_pending_finalizers, but it is still not a solution for
> > multithreaded case.
>
> I'm getting confused on where people think that this problem needs to
> be managed.  Am I misunderstanding that there should be some
> awareness of this in the classlib or VM natives?  Seems like it can
> be localized.
>
> I think that if whoever calls malloc() finds that the system is out
> of memory, then it's too late.  I always thought that VMs manage
> their conceptual native "heap" (all native resources consumed from
> OS, including as Robin mentioned, filehandles) in a proactive manner,
> like they do their java heap.  it's clear now that they don't.
>
> So when a caller to malloc() gets a null, it should simply throw an
> OOM, assuming that the runtime environment did all it could to
> prevent that.
>
> Can this be done through portlib, letting the VMs implementation of
> it manage to whatever degree of sophistication is warranted?

Actually, the portlib is the lowest layer we rely on. If we can have
here a callback to VM's GC we can place the code in the portlib. I
would also distinguish memory allocations which associated with java
objects and that which doesn't. No need to account resources which
will not be freed after garbage collection.

As was mentioned we have different types of critical system resources.
It makes sense to make a list of such resources:
   malloc'ed memory
   mmap'ed memory
   shared memory (?)
   file descriptors (files, sockets)
   XWin / GDI resources (?)
   more?

What would be the accounting strategy for the resources? We could
allocate them via some functions in this facility or we could only
account them here and call GC using some buildin criteria.

-- 
Ivan
Intel Enterprise Solutions Software Division

Re: [VM]How to trigue GC while free native memory is low.

["Geir Magnusson Jr." <ge...@pobox.com>]

On Feb 4, 2007, at 8:41 AM, Gregory Shimansky wrote:

> Leo Li wrote:
>> On 2/4/07, Gregory Shimansky <gs...@gmail.com> wrote:
>>>
>>> I see no difference in the approach I've suggested already. If we  
>>> have
>>> to take care about all the native resource allocation inside of  
>>> classlib
>>> APIs, then there is no difference which code calls GC, be it
>>> gc_native_malloc, or the API native code like this:
>>>
>>> void *p = malloc(size);
>>> if (NULL == p)
>>> {
>>>   gc_force_gc();
>>>   p = malloc(size);
>>>   if (NULL == p)
>>>   {
>>>       jni_env->ThrowNew("java/lang/OutOfMemoryError");
>>>       return;
>>>   }
>>> }
>>
>>
>> But I am worried whether it is too late to force gc when memory  
>> allocation
>> fails. It will take some time to complete the release of  
>> resources, for
>> example, in finalizer() and this period of time is not  
>> predictable. Thus
>> the
>> malloc will still fails...:(
>> Futhermore, if we wait till memory allocation fails then to force  
>> gc, it
>> might be possible that more than one thread will encounter such a  
>> problem
>> and force gc. I am not sure whether it will lead to problem in vm.
>> So, I think it may be wiser to trigger a gc when free memory ratio is
>> low and before the failure in malloc actually occurs.
>
> Yes, I shouldn't have called the above code a "solution". I just  
> wanted
> to point to Ivan that there is no difference with taking care about
> memory allocation in GC code or in classlib code.
>
> You've found another problem in such approach, just running GC is not
> enough in this case because finalization may not be done in time to  
> free
> needed native resources.
>
> Speaking in DRLVM terms, the above code could also contain call to
> vm_run_pending_finalizers, but it is still not a solution for
> multithreaded case.

I'm getting confused on where people think that this problem needs to  
be managed.  Am I misunderstanding that there should be some  
awareness of this in the classlib or VM natives?  Seems like it can  
be localized.

I think that if whoever calls malloc() finds that the system is out  
of memory, then it's too late.  I always thought that VMs manage  
their conceptual native "heap" (all native resources consumed from  
OS, including as Robin mentioned, filehandles) in a proactive manner,  
like they do their java heap.  it's clear now that they don't.

So when a caller to malloc() gets a null, it should simply throw an  
OOM, assuming that the runtime environment did all it could to  
prevent that.

Can this be done through portlib, letting the VMs implementation of  
it manage to whatever degree of sophistication is warranted?

geir

Re: [VM]How to trigue GC while free native memory is low.

[Ivan Volosyuk <iv...@gmail.com>]

On 2/4/07, Gregory Shimansky <gs...@gmail.com> wrote:
> Leo Li wrote:
> > On 2/4/07, Gregory Shimansky <gs...@gmail.com> wrote:
> >>
> >> Ivan Volosyuk wrote:
> >> > On 2/3/07, LvJimmy，Jing <fi...@gmail.com> wrote:
> >> >> 2007/2/2, Xiao-Feng Li <xi...@gmail.com>:
> >> >> > Thank you, Jimmy and Leo, for the clear explanations.
> >> >> >
> >> >> > I think temporarily you can put the System.gc() there to workaround
> >> >> > this issue. Yes, this is only a workaround since we cannot rely
> >> on it
> >> >> > without explicit guarantee in its semantics.
> >> >> >
> >> >> > To really solve this problem, we might need a careful design. Your
> >> >> > proposed approach looks like a possible solution. Have you
> >> thought to
> >> >> > solve the problem by using GC to manage the raw memory blocks?
> >> Without
> >> >> > deep understanding of the problem, I have some wild idea below.
> >> Please
> >> >> > see if it is reasonable or stupid. I will read the spec of
> >> ByteBuffer
> >> >> > to get more understanding.
> >> >> >
> >> >> > Since what the ByeBuffer needs are starting address and capacity, it
> >> >> > doesn't really care if this piece of memory is in Java heap or in
> >> >> > native runtime memory ( I assume.) We probably can provide some
> >> >> > special kind of virtual Java object that serves as raw memory block
> >> to
> >> >> > nio. In this works, we need not monitor the native runtime memory
> >> >> > usage.
> >> >> >
> >> >>
> >> >> Ah, this solution sounds interesting to me, yes, if we malloc memory
> >> >> in heap then we no longer need to monitor the native memory. But we
> >> >> may need a kit of methods offer this heap-memory service:
> >> >> 1) a function of malloc_in_heap()
> >> >> 2) a function of free_heap_block()
> >> >> 3) heap monitor should record this malloc/free action
> >> >> 4) ...
> >> >> However I do think this may be a good solution, and it still depends
> >> >> on how easy can it be implemented. Note not only direct ByteBuffer
> >> >> require this native memory usage.
> >> >
> >> > I like this idea. I think we can express it in more general way. We
> >> > need native memory which is somehow related to java heap objects. When
> >> > calling garbage collection there is high possibility that some of this
> >> > memory will be freed. So here is my variant of this proposal:
> >> > 1. a function gc_native_malloc() which allocates native memory and
> >> > triggers GC on a certain threshold.
> >> > 2. a fenction gc_native_free() which deallocates native memory and
> >> > updates counter for the used space.
> >> > This memory may be allocated in general native heap but it should be
> >> > accounted by GC. It is not even required that it should be the GC who
> >> > implements this functionality, just some code which will can call GC
> >> > and able to allocate native memory.
> >> >
> >> > One drawback of this approach that we should know all the places where
> >> > we have a relation between java heap object and native memory. Some
> >> > custom native library may disobey this rule.
> >>
> >> (Sorry for repeating this through gmane. Gmail today is acting weird and
> >> doesn't accept my emails through SSMPT)
> >>
> >> I see no difference in the approach I've suggested already. If we have
> >> to take care about all the native resource allocation inside of classlib
> >> APIs, then there is no difference which code calls GC, be it
> >> gc_native_malloc, or the API native code like this:
> >>
> >> void *p = malloc(size);
> >> if (NULL == p)
> >> {
> >>   gc_force_gc();
> >>   p = malloc(size);
> >>   if (NULL == p)
> >>   {
> >>       jni_env->ThrowNew("java/lang/OutOfMemoryError");
> >>       return;
> >>   }
> >> }
> >
> >
> > But I am worried whether it is too late to force gc when memory allocation
> > fails. It will take some time to complete the release of resources, for
> > example, in finalizer() and this period of time is not predictable. Thus
> > the
> > malloc will still fails...:(
> > Futhermore, if we wait till memory allocation fails then to force gc, it
> > might be possible that more than one thread will encounter such a problem
> > and force gc. I am not sure whether it will lead to problem in vm.
> > So, I think it may be wiser to trigger a gc when free memory ratio is
> > low and before the failure in malloc actually occurs.
>
> Yes, I shouldn't have called the above code a "solution". I just wanted
> to point to Ivan that there is no difference with taking care about
> memory allocation in GC code or in classlib code.

There only difference here is conceptual. If we request a resource
from GC via gc_malloc_native() or something like that, we may be sure
that GC will do the job of tracking that resources and will try to do
something when we go out of free space.

>
> You've found another problem in such approach, just running GC is not
> enough in this case because finalization may not be done in time to free
> needed native resources.
>
> Speaking in DRLVM terms, the above code could also contain call to
> vm_run_pending_finalizers, but it is still not a solution for
> multithreaded case.
>
> --
> Gregory
>
>


-- 
Ivan
Intel Enterprise Solutions Software Division

Re: [VM]How to trigue GC while free native memory is low.

[Gregory Shimansky <gs...@gmail.com>]

Leo Li wrote:
> On 2/4/07, Gregory Shimansky <gs...@gmail.com> wrote:
>>
>> Ivan Volosyuk wrote:
>> > On 2/3/07, LvJimmy，Jing <fi...@gmail.com> wrote:
>> >> 2007/2/2, Xiao-Feng Li <xi...@gmail.com>:
>> >> > Thank you, Jimmy and Leo, for the clear explanations.
>> >> >
>> >> > I think temporarily you can put the System.gc() there to workaround
>> >> > this issue. Yes, this is only a workaround since we cannot rely 
>> on it
>> >> > without explicit guarantee in its semantics.
>> >> >
>> >> > To really solve this problem, we might need a careful design. Your
>> >> > proposed approach looks like a possible solution. Have you 
>> thought to
>> >> > solve the problem by using GC to manage the raw memory blocks?
>> Without
>> >> > deep understanding of the problem, I have some wild idea below.
>> Please
>> >> > see if it is reasonable or stupid. I will read the spec of 
>> ByteBuffer
>> >> > to get more understanding.
>> >> >
>> >> > Since what the ByeBuffer needs are starting address and capacity, it
>> >> > doesn't really care if this piece of memory is in Java heap or in
>> >> > native runtime memory ( I assume.) We probably can provide some
>> >> > special kind of virtual Java object that serves as raw memory block
>> to
>> >> > nio. In this works, we need not monitor the native runtime memory
>> >> > usage.
>> >> >
>> >>
>> >> Ah, this solution sounds interesting to me, yes, if we malloc memory
>> >> in heap then we no longer need to monitor the native memory. But we
>> >> may need a kit of methods offer this heap-memory service:
>> >> 1) a function of malloc_in_heap()
>> >> 2) a function of free_heap_block()
>> >> 3) heap monitor should record this malloc/free action
>> >> 4) ...
>> >> However I do think this may be a good solution, and it still depends
>> >> on how easy can it be implemented. Note not only direct ByteBuffer
>> >> require this native memory usage.
>> >
>> > I like this idea. I think we can express it in more general way. We
>> > need native memory which is somehow related to java heap objects. When
>> > calling garbage collection there is high possibility that some of this
>> > memory will be freed. So here is my variant of this proposal:
>> > 1. a function gc_native_malloc() which allocates native memory and
>> > triggers GC on a certain threshold.
>> > 2. a fenction gc_native_free() which deallocates native memory and
>> > updates counter for the used space.
>> > This memory may be allocated in general native heap but it should be
>> > accounted by GC. It is not even required that it should be the GC who
>> > implements this functionality, just some code which will can call GC
>> > and able to allocate native memory.
>> >
>> > One drawback of this approach that we should know all the places where
>> > we have a relation between java heap object and native memory. Some
>> > custom native library may disobey this rule.
>>
>> (Sorry for repeating this through gmane. Gmail today is acting weird and
>> doesn't accept my emails through SSMPT)
>>
>> I see no difference in the approach I've suggested already. If we have
>> to take care about all the native resource allocation inside of classlib
>> APIs, then there is no difference which code calls GC, be it
>> gc_native_malloc, or the API native code like this:
>>
>> void *p = malloc(size);
>> if (NULL == p)
>> {
>>   gc_force_gc();
>>   p = malloc(size);
>>   if (NULL == p)
>>   {
>>       jni_env->ThrowNew("java/lang/OutOfMemoryError");
>>       return;
>>   }
>> }
> 
> 
> But I am worried whether it is too late to force gc when memory allocation
> fails. It will take some time to complete the release of resources, for
> example, in finalizer() and this period of time is not predictable. Thus 
> the
> malloc will still fails...:(
> Futhermore, if we wait till memory allocation fails then to force gc, it
> might be possible that more than one thread will encounter such a problem
> and force gc. I am not sure whether it will lead to problem in vm.
> So, I think it may be wiser to trigger a gc when free memory ratio is
> low and before the failure in malloc actually occurs.

Yes, I shouldn't have called the above code a "solution". I just wanted
to point to Ivan that there is no difference with taking care about
memory allocation in GC code or in classlib code.

You've found another problem in such approach, just running GC is not
enough in this case because finalization may not be done in time to free
needed native resources.

Speaking in DRLVM terms, the above code could also contain call to
vm_run_pending_finalizers, but it is still not a solution for
multithreaded case.

-- 
Gregory

Re: [VM]How to trigue GC while free native memory is low.

[Leo Li <li...@gmail.com>]

On 2/4/07, Gregory Shimansky <gs...@gmail.com> wrote:
>
> Ivan Volosyuk wrote:
> > On 2/3/07, LvJimmy，Jing <fi...@gmail.com> wrote:
> >> 2007/2/2, Xiao-Feng Li <xi...@gmail.com>:
> >> > Thank you, Jimmy and Leo, for the clear explanations.
> >> >
> >> > I think temporarily you can put the System.gc() there to workaround
> >> > this issue. Yes, this is only a workaround since we cannot rely on it
> >> > without explicit guarantee in its semantics.
> >> >
> >> > To really solve this problem, we might need a careful design. Your
> >> > proposed approach looks like a possible solution. Have you thought to
> >> > solve the problem by using GC to manage the raw memory blocks?
> Without
> >> > deep understanding of the problem, I have some wild idea below.
> Please
> >> > see if it is reasonable or stupid. I will read the spec of ByteBuffer
> >> > to get more understanding.
> >> >
> >> > Since what the ByeBuffer needs are starting address and capacity, it
> >> > doesn't really care if this piece of memory is in Java heap or in
> >> > native runtime memory ( I assume.) We probably can provide some
> >> > special kind of virtual Java object that serves as raw memory block
> to
> >> > nio. In this works, we need not monitor the native runtime memory
> >> > usage.
> >> >
> >>
> >> Ah, this solution sounds interesting to me, yes, if we malloc memory
> >> in heap then we no longer need to monitor the native memory. But we
> >> may need a kit of methods offer this heap-memory service:
> >> 1) a function of malloc_in_heap()
> >> 2) a function of free_heap_block()
> >> 3) heap monitor should record this malloc/free action
> >> 4) ...
> >> However I do think this may be a good solution, and it still depends
> >> on how easy can it be implemented. Note not only direct ByteBuffer
> >> require this native memory usage.
> >
> > I like this idea. I think we can express it in more general way. We
> > need native memory which is somehow related to java heap objects. When
> > calling garbage collection there is high possibility that some of this
> > memory will be freed. So here is my variant of this proposal:
> > 1. a function gc_native_malloc() which allocates native memory and
> > triggers GC on a certain threshold.
> > 2. a fenction gc_native_free() which deallocates native memory and
> > updates counter for the used space.
> > This memory may be allocated in general native heap but it should be
> > accounted by GC. It is not even required that it should be the GC who
> > implements this functionality, just some code which will can call GC
> > and able to allocate native memory.
> >
> > One drawback of this approach that we should know all the places where
> > we have a relation between java heap object and native memory. Some
> > custom native library may disobey this rule.
>
> (Sorry for repeating this through gmane. Gmail today is acting weird and
> doesn't accept my emails through SSMPT)
>
> I see no difference in the approach I've suggested already. If we have
> to take care about all the native resource allocation inside of classlib
> APIs, then there is no difference which code calls GC, be it
> gc_native_malloc, or the API native code like this:
>
> void *p = malloc(size);
> if (NULL == p)
> {
>   gc_force_gc();
>   p = malloc(size);
>   if (NULL == p)
>   {
>       jni_env->ThrowNew("java/lang/OutOfMemoryError");
>       return;
>   }
> }


But I am worried whether it is too late to force gc when memory allocation
fails. It will take some time to complete the release of resources, for
example, in finalizer() and this period of time is not predictable. Thus the
malloc will still fails...:(
Futhermore, if we wait till memory allocation fails then to force gc, it
might be possible that more than one thread will encounter such a problem
and force gc. I am not sure whether it will lead to problem in vm.
So, I think it may be wiser to trigger a gc when free memory ratio is
low and before the failure in malloc actually occurs.

>>
> >> > This approach need certain contract between Java classes and GC about
> >> > the special kind of Java object. Probably we can write a layer of
> Java
> >> > class wrapper for raw memory allocation, which hides the contract
> from
> >> > other common classes.
> >> >
> >>
> >> As we have a portlib, this can be implemented in such hymem_malloc or
> so.
> >>
> >> > At the same time, I am thinking of your proposed approach, and will
> >> > get back later.
> >> >
> >>
> >> Thanks, waiting for you the GC guru's appoach towards heap memory
> >> allocation :)
> >>
> >> > Thanks,
> >> > xiaofeng
> >> >
> >> > On 2/2/07, Leo Li <li...@gmail.com> wrote:
> >> > > Hi, Xiao-Feng:
> >> > >      Excuse me if I am confusing you.
> >> > >      The direct byte buffer holds a block of native memory space
> >> used in nio
> >> > > operation. The native byte buffer will be freed through the byte
> >> buffer is
> >> > > gc collected and added to ReferenceQueue monitored by MemorySpy
> >> series
> >> > > classes.
> >> > >      But the gc will not be triggered if the java heap is still
> >> empty. Here
> >> > > is an example: I allocate a large block of native memory for a
> >> byte buffer
> >> > > while the byte buffer itself is quite small. Thus quickly native
> >> heap is
> >> > > depleted while VM still does not think gc is needed since java
> >> heap has a
> >> > > lot of free memory.
> >> > >
> >> > >
> >> > > import java.nio.*;
> >> > > public class Test {
> >> > >     public static void main(String[] args) throws Exception {
> >> > >         for(int i = 0;i<1000;i++)
> >> > >         {
> >> > >             ByteBuffer byteBuffer =
> >> ByteBuffer.allocateDirect(10240000);
> >> > >             //System.gc();
> >> > >         }
> >> > >    }
> >> > > }
> >> > > RI runs well but Harmony will soon throw OutOfMemory exception.
> >> > > But if remove the comment before System.gc(), Harmony will become
> ok.
> >> > > So my point is that we need a mechanism to notify VM to start gc
> >> if we have
> >> > > no more native memory because sometimes problem will be solved if
> >> gc is
> >> > > fired. Of cause we cannot avoid a  user always malloc space but
> >> never free
> >> > > them.:)
> >> > >
> >> > > Good luck!
> >> > >
> >> > > On 2/2/07, Xiao-Feng Li <xi...@gmail.com> wrote:
> >> > > >
> >> > > > Leo, I can't fully understand your problem and proposal. I am
> >> working
> >> > > > in DRLVM GC component, I want to understand how GC can tackle the
> >> > > > issue you meet.
> >> > > >
> >> > > > You said the native memory is allocated (probably with malloc)
> for
> >> > > > byte buffer. I wonder how the process goes. I assume it's in
> native
> >> > > > code. Then you use malloc to allocate a block of memory, and put
> >> some
> >> > > > Java object (byte buffer) into it? Is this what you do? I wonder
> >> how
> >> > > > do you manipulate Java object placement in native code; and if
> the
> >> > > > native memory block is out of Java heap, do you expect GC to
> manage
> >> > > > it?
> >> > > >
> >> > > > You said when GC reclaims those byte buffer object, the native
> >> memory
> >> > > > block can be freed in native code. How do you know if an object
> is
> >> > > > reclaimed by GC or not?
> >> > > >
> >> > > > Thanks,
> >> > > > xiaofeng
> >> > > >
> >> > > > On 2/2/07, Leo Li < liyilei1979@gmail.com> wrote:
> >> > > > > Hi, all:
> >> > > > >     After applying patch3073, I added the support for direct
> >> byte buffer
> >> > > > in
> >> > > > > nio module as spec requires. But now here exists a problem:
> >> The native
> >> > > > > memory allocated to the byte buffer can be released when gc
> >> releases the
> >> > > > > byte buffer object, however, if the native heap is full while
> >> java heap
> >> > > > > still has space, gc will not be triggered.
> >> > > > >     It seems that RI will start gc before native memory heap
> >> is depleted
> >> > > > and
> >> > > > > thus prevents out-of-memory error.
> >> > > > >
> >> > > > >    Then our work focuses on:
> >> > > > >    1. When gc is required.
> >> > > > >    2. Trigger a gc.
> >> > > > >    The first one requires that we get support from operating
> >> system,
> >> > > > since
> >> > > > > the memory allocated in the native code, for example by
> >> malloc, is out
> >> > > > of
> >> > > > > the control of java VM.( I have ever thought of counting the
> >> used memory
> >> > > > in
> >> > > > > hymemory_allocate, but the plan fails since hymemory_free will
> >> not
> >> > > > report
> >> > > > > how much space is released.)
> >> > > > >    The second one needs the help from VM. System.gc() is not
> >> so reliable
> >> > > > as
> >> > > > > spec says, so it is necessary to have a internal channel to
> >> notify VM to
> >> > > > > start gc.
> >> > > > >
> >> > > > >     One solution, I think, is to let a monitor thread in VM to
> >> check
> >> > > > whether
> >> > > > > OS physical memory is low. For example, the
> >> > > > QueryMemoryResourceNotification
> >> > > > > of win32 API is a candidate. Although the interrupt model is
> more
> >> > > > effective:
> >> > > > > win32 SDK provides a CreateMemoryResourceNotification to get a
> >> handler
> >> > > > on
> >> > > > > which the monitor thread can wait, maybe on other platforms,
> >> OS does not
> >> > > > > supply such a convenience. So the monitor thread in the VM
> >> might have to
> >> > > >
> >> > > > > check the OS resource once for a while and if necessary the
> >> monitor
> >> > > > thread
> >> > > > > will call a GC.
> >> > > > >
> >> > > > >    My suggestion is first to add some function to monitor
> >> memory in
> >> > > > portlib,
> >> > > > > since it is highly related to platforms and in portlib there
> >> has been
> >> > > > > some useful tools.(Thanks Mark to point out that.) On the
> >> other hand, we
> >> > > > can
> >> > > > > negotiate a channel to trigger gc in VM. Actually I am not an
> >> expert on
> >> > > > VM
> >> > > > > since I am not sure whether there has been some monitor thread
> >> and the
> >> > > > load
> >> > > > > on performance if such a monitor thread is added to the VM...
> >> > > > >
> >> > > > >
> >> > > > > --
> >> > > > > Leo Li
> >> > > > > China Software Development Lab, IBM
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Leo Li
> >> > > China Software Development Lab, IBM
> >> > >
> >> > >
> >> >
> >>
> >>
> >> --
> >>
> >> Best Regards!
> >>
> >> Jimmy, Jing Lv
> >> China Software Development Lab, IBM
> >>
> >
> >
>
>
> --
> Gregory
>
>


-- 
Leo Li
China Software Development Lab, IBM

Re: [VM]How to trigue GC while free native memory is low.

[Gregory Shimansky <gs...@gmail.com>]

Ivan Volosyuk wrote:
> On 2/3/07, LvJimmy，Jing <fi...@gmail.com> wrote:
>> 2007/2/2, Xiao-Feng Li <xi...@gmail.com>:
>> > Thank you, Jimmy and Leo, for the clear explanations.
>> >
>> > I think temporarily you can put the System.gc() there to workaround
>> > this issue. Yes, this is only a workaround since we cannot rely on it
>> > without explicit guarantee in its semantics.
>> >
>> > To really solve this problem, we might need a careful design. Your
>> > proposed approach looks like a possible solution. Have you thought to
>> > solve the problem by using GC to manage the raw memory blocks? Without
>> > deep understanding of the problem, I have some wild idea below. Please
>> > see if it is reasonable or stupid. I will read the spec of ByteBuffer
>> > to get more understanding.
>> >
>> > Since what the ByeBuffer needs are starting address and capacity, it
>> > doesn't really care if this piece of memory is in Java heap or in
>> > native runtime memory ( I assume.) We probably can provide some
>> > special kind of virtual Java object that serves as raw memory block to
>> > nio. In this works, we need not monitor the native runtime memory
>> > usage.
>> >
>>
>> Ah, this solution sounds interesting to me, yes, if we malloc memory
>> in heap then we no longer need to monitor the native memory. But we
>> may need a kit of methods offer this heap-memory service:
>> 1) a function of malloc_in_heap()
>> 2) a function of free_heap_block()
>> 3) heap monitor should record this malloc/free action
>> 4) ...
>> However I do think this may be a good solution, and it still depends
>> on how easy can it be implemented. Note not only direct ByteBuffer
>> require this native memory usage.
> 
> I like this idea. I think we can express it in more general way. We
> need native memory which is somehow related to java heap objects. When
> calling garbage collection there is high possibility that some of this
> memory will be freed. So here is my variant of this proposal:
> 1. a function gc_native_malloc() which allocates native memory and
> triggers GC on a certain threshold.
> 2. a fenction gc_native_free() which deallocates native memory and
> updates counter for the used space.
> This memory may be allocated in general native heap but it should be
> accounted by GC. It is not even required that it should be the GC who
> implements this functionality, just some code which will can call GC
> and able to allocate native memory.
> 
> One drawback of this approach that we should know all the places where
> we have a relation between java heap object and native memory. Some
> custom native library may disobey this rule.

(Sorry for repeating this through gmane. Gmail today is acting weird and
doesn't accept my emails through SSMPT)

I see no difference in the approach I've suggested already. If we have
to take care about all the native resource allocation inside of classlib
APIs, then there is no difference which code calls GC, be it
gc_native_malloc, or the API native code like this:

void *p = malloc(size);
if (NULL == p)
{
   gc_force_gc();
   p = malloc(size);
   if (NULL == p)
   {
       jni_env->ThrowNew("java/lang/OutOfMemoryError");
       return;
   }
}

>>
>> > This approach need certain contract between Java classes and GC about
>> > the special kind of Java object. Probably we can write a layer of Java
>> > class wrapper for raw memory allocation, which hides the contract from
>> > other common classes.
>> >
>>
>> As we have a portlib, this can be implemented in such hymem_malloc or so.
>>
>> > At the same time, I am thinking of your proposed approach, and will
>> > get back later.
>> >
>>
>> Thanks, waiting for you the GC guru's appoach towards heap memory 
>> allocation :)
>>
>> > Thanks,
>> > xiaofeng
>> >
>> > On 2/2/07, Leo Li <li...@gmail.com> wrote:
>> > > Hi, Xiao-Feng:
>> > >      Excuse me if I am confusing you.
>> > >      The direct byte buffer holds a block of native memory space 
>> used in nio
>> > > operation. The native byte buffer will be freed through the byte 
>> buffer is
>> > > gc collected and added to ReferenceQueue monitored by MemorySpy 
>> series
>> > > classes.
>> > >      But the gc will not be triggered if the java heap is still 
>> empty. Here
>> > > is an example: I allocate a large block of native memory for a 
>> byte buffer
>> > > while the byte buffer itself is quite small. Thus quickly native 
>> heap is
>> > > depleted while VM still does not think gc is needed since java 
>> heap has a
>> > > lot of free memory.
>> > >
>> > >
>> > > import java.nio.*;
>> > > public class Test {
>> > >     public static void main(String[] args) throws Exception {
>> > >         for(int i = 0;i<1000;i++)
>> > >         {
>> > >             ByteBuffer byteBuffer = 
>> ByteBuffer.allocateDirect(10240000);
>> > >             //System.gc();
>> > >         }
>> > >    }
>> > > }
>> > > RI runs well but Harmony will soon throw OutOfMemory exception.
>> > > But if remove the comment before System.gc(), Harmony will become ok.
>> > > So my point is that we need a mechanism to notify VM to start gc 
>> if we have
>> > > no more native memory because sometimes problem will be solved if 
>> gc is
>> > > fired. Of cause we cannot avoid a  user always malloc space but 
>> never free
>> > > them.:)
>> > >
>> > > Good luck!
>> > >
>> > > On 2/2/07, Xiao-Feng Li <xi...@gmail.com> wrote:
>> > > >
>> > > > Leo, I can't fully understand your problem and proposal. I am 
>> working
>> > > > in DRLVM GC component, I want to understand how GC can tackle the
>> > > > issue you meet.
>> > > >
>> > > > You said the native memory is allocated (probably with malloc) for
>> > > > byte buffer. I wonder how the process goes. I assume it's in native
>> > > > code. Then you use malloc to allocate a block of memory, and put 
>> some
>> > > > Java object (byte buffer) into it? Is this what you do? I wonder 
>> how
>> > > > do you manipulate Java object placement in native code; and if the
>> > > > native memory block is out of Java heap, do you expect GC to manage
>> > > > it?
>> > > >
>> > > > You said when GC reclaims those byte buffer object, the native 
>> memory
>> > > > block can be freed in native code. How do you know if an object is
>> > > > reclaimed by GC or not?
>> > > >
>> > > > Thanks,
>> > > > xiaofeng
>> > > >
>> > > > On 2/2/07, Leo Li < liyilei1979@gmail.com> wrote:
>> > > > > Hi, all:
>> > > > >     After applying patch3073, I added the support for direct 
>> byte buffer
>> > > > in
>> > > > > nio module as spec requires. But now here exists a problem: 
>> The native
>> > > > > memory allocated to the byte buffer can be released when gc 
>> releases the
>> > > > > byte buffer object, however, if the native heap is full while 
>> java heap
>> > > > > still has space, gc will not be triggered.
>> > > > >     It seems that RI will start gc before native memory heap 
>> is depleted
>> > > > and
>> > > > > thus prevents out-of-memory error.
>> > > > >
>> > > > >    Then our work focuses on:
>> > > > >    1. When gc is required.
>> > > > >    2. Trigger a gc.
>> > > > >    The first one requires that we get support from operating 
>> system,
>> > > > since
>> > > > > the memory allocated in the native code, for example by 
>> malloc, is out
>> > > > of
>> > > > > the control of java VM.( I have ever thought of counting the 
>> used memory
>> > > > in
>> > > > > hymemory_allocate, but the plan fails since hymemory_free will 
>> not
>> > > > report
>> > > > > how much space is released.)
>> > > > >    The second one needs the help from VM. System.gc() is not 
>> so reliable
>> > > > as
>> > > > > spec says, so it is necessary to have a internal channel to 
>> notify VM to
>> > > > > start gc.
>> > > > >
>> > > > >     One solution, I think, is to let a monitor thread in VM to 
>> check
>> > > > whether
>> > > > > OS physical memory is low. For example, the
>> > > > QueryMemoryResourceNotification
>> > > > > of win32 API is a candidate. Although the interrupt model is more
>> > > > effective:
>> > > > > win32 SDK provides a CreateMemoryResourceNotification to get a 
>> handler
>> > > > on
>> > > > > which the monitor thread can wait, maybe on other platforms, 
>> OS does not
>> > > > > supply such a convenience. So the monitor thread in the VM 
>> might have to
>> > > >
>> > > > > check the OS resource once for a while and if necessary the 
>> monitor
>> > > > thread
>> > > > > will call a GC.
>> > > > >
>> > > > >    My suggestion is first to add some function to monitor 
>> memory in
>> > > > portlib,
>> > > > > since it is highly related to platforms and in portlib there 
>> has been
>> > > > > some useful tools.(Thanks Mark to point out that.) On the 
>> other hand, we
>> > > > can
>> > > > > negotiate a channel to trigger gc in VM. Actually I am not an 
>> expert on
>> > > > VM
>> > > > > since I am not sure whether there has been some monitor thread 
>> and the
>> > > > load
>> > > > > on performance if such a monitor thread is added to the VM...
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Leo Li
>> > > > > China Software Development Lab, IBM
>> > > > >
>> > > > >
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Leo Li
>> > > China Software Development Lab, IBM
>> > >
>> > >
>> >
>>
>>
>> -- 
>>
>> Best Regards!
>>
>> Jimmy, Jing Lv
>> China Software Development Lab, IBM
>>
> 
> 


-- 
Gregory

Re: [VM]How to trigue GC while free native memory is low.

[Ivan Volosyuk <iv...@gmail.com>]

IMHO, converting mmaped byte buffer into java heap object can be a
good idea. I'm just have difficulties in understanding how to merge in
one memory chunk an ordinary object header and mmaped region.

Actually, mapped buffers are just one of the examples of native memory
usage from java. Others like thread objects, zlib functionality and so
on, should also be addressed. Not all of this entities can be
converted to java objects.

On 2/3/07, Xiao-Feng Li <xi...@gmail.com> wrote:
> On 2/3/07, Ivan Volosyuk <iv...@gmail.com> wrote:
> > On 2/3/07, LvJimmy，Jing <fi...@gmail.com> wrote:
> > > 2007/2/2, Xiao-Feng Li <xi...@gmail.com>:
> > > > Thank you, Jimmy and Leo, for the clear explanations.
> > > >
> > > > I think temporarily you can put the System.gc() there to workaround
> > > > this issue. Yes, this is only a workaround since we cannot rely on it
> > > > without explicit guarantee in its semantics.
> > > >
> > > > To really solve this problem, we might need a careful design. Your
> > > > proposed approach looks like a possible solution. Have you thought to
> > > > solve the problem by using GC to manage the raw memory blocks? Without
> > > > deep understanding of the problem, I have some wild idea below. Please
> > > > see if it is reasonable or stupid. I will read the spec of ByteBuffer
> > > > to get more understanding.
> > > >
> > > > Since what the ByeBuffer needs are starting address and capacity, it
> > > > doesn't really care if this piece of memory is in Java heap or in
> > > > native runtime memory ( I assume.) We probably can provide some
> > > > special kind of virtual Java object that serves as raw memory block to
> > > > nio. In this works, we need not monitor the native runtime memory
> > > > usage.
> > > >
> > >
> > > Ah, this solution sounds interesting to me, yes, if we malloc memory
> > > in heap then we no longer need to monitor the native memory. But we
> > > may need a kit of methods offer this heap-memory service:
> > > 1) a function of malloc_in_heap()
> > > 2) a function of free_heap_block()
> > > 3) heap monitor should record this malloc/free action
> > > 4) ...
> > > However I do think this may be a good solution, and it still depends
> > > on how easy can it be implemented. Note not only direct ByteBuffer
> > > require this native memory usage.
> >
> > I like this idea. I think we can express it in more general way. We
> > need native memory which is somehow related to java heap objects. When
> > calling garbage collection there is high possibility that some of this
> > memory will be freed. So here is my variant of this proposal:
> > 1. a function gc_native_malloc() which allocates native memory and
> > triggers GC on a certain threshold.
> > 2. a fenction gc_native_free() which deallocates native memory and
> > updates counter for the used space.
> > This memory may be allocated in general native heap but it should be
> > accounted by GC. It is not even required that it should be the GC who
> > implements this functionality, just some code which will can call GC
> > and able to allocate native memory.
> >
> > One drawback of this approach that we should know all the places where
> > we have a relation between java heap object and native memory. Some
> > custom native library may disobey this rule.
>
> I've just studied ByteBuffer and MappedByteBuffer. If I understand
> correctly, the latter one is similar to POSIX mmap, and it's a
> subclass of ByteBuffer and uses a direct ByteBuffer. If it's
> implemented in the way of mmap, we probably have no serious issue to
> provide the required support in GC. The advantage to provide this
> support in GC is, GC can decide when and how to start a collection.
> And if the GC has some mechanism of reference counting, it can serve
> the purpose even better.
>
> The other point is, in the spec, the ByteBuffer can be created by
> wrapping a Byte array. GC can provide this buffer as a direct buffer
> if that's really wanted in certain scenario.
>
> Thanks,
> xiaofeng
>


-- 
Ivan
Intel Enterprise Solutions Software Division

Re: [VM]How to trigue GC while free native memory is low.

[Robin Garner <ro...@anu.edu.au>]

I think a real solution addresses all native resources, not just memory
on the 'C' heap.  DaCapo xalan for example (pre dacapo-2006-10-MR1)
would run out of file handles in a 512m heap.  File handles, certain
finite GUI resources etc are all prone to this kind of problem.

The usual design pattern seems to be that when Java code allocates a
native resource, it frees it in a finalizer (and better code attempts to
explicitly free it earlier), so the issue is not just GC, but also
(timely) execution of finalizers.

As various people have pointed out tracking the native resource
consumption is non-trivial, and it's not something I'm going to attempt
to address :)

Adding a facility where the GC asks the runtime whether it needs a GC
(for one of the above reasons) would IMO solve the problem from the
point of view of the GC.  Then the VM can track usage of finite
resources on its side of the fence, and trigger GC when they are nearing
exhaustion.

Care is probably required to avoid thrashing:  since it is execution of
finalizers/reference type processing that frees native resources, there
will be a delay between requesting a GC and native resources being
freed.  For this my first thought is to insert a marker in the finalizer
queue, and to ignore GC requests for native resources until all
finalizers queued since the last request have completed.

I'm glad DRLVM is addressing this issue - it's one that has interested
me for a while.

cheers

(PS apologies for the initial blank reply :/ )

-- 
Robin Garner
Dept. of Computer Science
Australian National University
http://cs.anu.edu.au/people/Robin.Garner/

Re: [VM]How to trigue GC while free native memory is low.

[Rana Dasgupta <rd...@gmail.com>]

On 2/3/07, Xiao-Feng Li <xi...@gmail.com> wrote:
>
> >I've just studied ByteBuffer and MappedByteBuffer. If I understand
> >correctly, the latter one is similar to POSIX mmap, and it's a
> >subclass of ByteBuffer and uses a direct ByteBuffer. If it's
> >implemented in the way of mmap, we probably have no serious issue to
> >provide the required support in GC. The advantage to provide this
> >support in GC is, GC can decide when and how to start a collection.
> >And if the GC has some mechanism of reference counting, it can serve
> >the purpose even better.


Yes. If the GC could treat these as a special class of objects with
different lifetime semantics and collection rules, that would be a nice
approach. It may be better than invoking GC periodically.


> Thanks,
> xiaofeng
>

Re: [VM]How to trigue GC while free native memory is low.

[Xiao-Feng Li <xi...@gmail.com>]

On 2/3/07, Ivan Volosyuk <iv...@gmail.com> wrote:
> On 2/3/07, LvJimmy，Jing <fi...@gmail.com> wrote:
> > 2007/2/2, Xiao-Feng Li <xi...@gmail.com>:
> > > Thank you, Jimmy and Leo, for the clear explanations.
> > >
> > > I think temporarily you can put the System.gc() there to workaround
> > > this issue. Yes, this is only a workaround since we cannot rely on it
> > > without explicit guarantee in its semantics.
> > >
> > > To really solve this problem, we might need a careful design. Your
> > > proposed approach looks like a possible solution. Have you thought to
> > > solve the problem by using GC to manage the raw memory blocks? Without
> > > deep understanding of the problem, I have some wild idea below. Please
> > > see if it is reasonable or stupid. I will read the spec of ByteBuffer
> > > to get more understanding.
> > >
> > > Since what the ByeBuffer needs are starting address and capacity, it
> > > doesn't really care if this piece of memory is in Java heap or in
> > > native runtime memory ( I assume.) We probably can provide some
> > > special kind of virtual Java object that serves as raw memory block to
> > > nio. In this works, we need not monitor the native runtime memory
> > > usage.
> > >
> >
> > Ah, this solution sounds interesting to me, yes, if we malloc memory
> > in heap then we no longer need to monitor the native memory. But we
> > may need a kit of methods offer this heap-memory service:
> > 1) a function of malloc_in_heap()
> > 2) a function of free_heap_block()
> > 3) heap monitor should record this malloc/free action
> > 4) ...
> > However I do think this may be a good solution, and it still depends
> > on how easy can it be implemented. Note not only direct ByteBuffer
> > require this native memory usage.
>
> I like this idea. I think we can express it in more general way. We
> need native memory which is somehow related to java heap objects. When
> calling garbage collection there is high possibility that some of this
> memory will be freed. So here is my variant of this proposal:
> 1. a function gc_native_malloc() which allocates native memory and
> triggers GC on a certain threshold.
> 2. a fenction gc_native_free() which deallocates native memory and
> updates counter for the used space.
> This memory may be allocated in general native heap but it should be
> accounted by GC. It is not even required that it should be the GC who
> implements this functionality, just some code which will can call GC
> and able to allocate native memory.
>
> One drawback of this approach that we should know all the places where
> we have a relation between java heap object and native memory. Some
> custom native library may disobey this rule.

I've just studied ByteBuffer and MappedByteBuffer. If I understand
correctly, the latter one is similar to POSIX mmap, and it's a
subclass of ByteBuffer and uses a direct ByteBuffer. If it's
implemented in the way of mmap, we probably have no serious issue to
provide the required support in GC. The advantage to provide this
support in GC is, GC can decide when and how to start a collection.
And if the GC has some mechanism of reference counting, it can serve
the purpose even better.

The other point is, in the spec, the ByteBuffer can be created by
wrapping a Byte array. GC can provide this buffer as a direct buffer
if that's really wanted in certain scenario.

Thanks,
xiaofeng

Re: [VM]How to trigue GC while free native memory is low.

[Ivan Volosyuk <iv...@gmail.com>]

On 2/3/07, LvJimmy，Jing <fi...@gmail.com> wrote:
> 2007/2/2, Xiao-Feng Li <xi...@gmail.com>:
> > Thank you, Jimmy and Leo, for the clear explanations.
> >
> > I think temporarily you can put the System.gc() there to workaround
> > this issue. Yes, this is only a workaround since we cannot rely on it
> > without explicit guarantee in its semantics.
> >
> > To really solve this problem, we might need a careful design. Your
> > proposed approach looks like a possible solution. Have you thought to
> > solve the problem by using GC to manage the raw memory blocks? Without
> > deep understanding of the problem, I have some wild idea below. Please
> > see if it is reasonable or stupid. I will read the spec of ByteBuffer
> > to get more understanding.
> >
> > Since what the ByeBuffer needs are starting address and capacity, it
> > doesn't really care if this piece of memory is in Java heap or in
> > native runtime memory ( I assume.) We probably can provide some
> > special kind of virtual Java object that serves as raw memory block to
> > nio. In this works, we need not monitor the native runtime memory
> > usage.
> >
>
> Ah, this solution sounds interesting to me, yes, if we malloc memory
> in heap then we no longer need to monitor the native memory. But we
> may need a kit of methods offer this heap-memory service:
> 1) a function of malloc_in_heap()
> 2) a function of free_heap_block()
> 3) heap monitor should record this malloc/free action
> 4) ...
> However I do think this may be a good solution, and it still depends
> on how easy can it be implemented. Note not only direct ByteBuffer
> require this native memory usage.

I like this idea. I think we can express it in more general way. We
need native memory which is somehow related to java heap objects. When
calling garbage collection there is high possibility that some of this
memory will be freed. So here is my variant of this proposal:
1. a function gc_native_malloc() which allocates native memory and
triggers GC on a certain threshold.
2. a fenction gc_native_free() which deallocates native memory and
updates counter for the used space.
This memory may be allocated in general native heap but it should be
accounted by GC. It is not even required that it should be the GC who
implements this functionality, just some code which will can call GC
and able to allocate native memory.

One drawback of this approach that we should know all the places where
we have a relation between java heap object and native memory. Some
custom native library may disobey this rule.

>
> > This approach need certain contract between Java classes and GC about
> > the special kind of Java object. Probably we can write a layer of Java
> > class wrapper for raw memory allocation, which hides the contract from
> > other common classes.
> >
>
> As we have a portlib, this can be implemented in such hymem_malloc or so.
>
> > At the same time, I am thinking of your proposed approach, and will
> > get back later.
> >
>
> Thanks, waiting for you the GC guru's appoach towards heap memory allocation :)
>
> > Thanks,
> > xiaofeng
> >
> > On 2/2/07, Leo Li <li...@gmail.com> wrote:
> > > Hi, Xiao-Feng:
> > >      Excuse me if I am confusing you.
> > >      The direct byte buffer holds a block of native memory space used in nio
> > > operation. The native byte buffer will be freed through the byte buffer is
> > > gc collected and added to ReferenceQueue monitored by MemorySpy series
> > > classes.
> > >      But the gc will not be triggered if the java heap is still empty. Here
> > > is an example: I allocate a large block of native memory for a byte buffer
> > > while the byte buffer itself is quite small. Thus quickly native heap is
> > > depleted while VM still does not think gc is needed since java heap has a
> > > lot of free memory.
> > >
> > >
> > > import java.nio.*;
> > > public class Test {
> > >     public static void main(String[] args) throws Exception {
> > >         for(int i = 0;i<1000;i++)
> > >         {
> > >             ByteBuffer byteBuffer = ByteBuffer.allocateDirect(10240000);
> > >             //System.gc();
> > >         }
> > >    }
> > > }
> > > RI runs well but Harmony will soon throw OutOfMemory exception.
> > > But if remove the comment before System.gc(), Harmony will become ok.
> > > So my point is that we need a mechanism to notify VM to start gc if we have
> > > no more native memory because sometimes problem will be solved if gc is
> > > fired. Of cause we cannot avoid a  user always malloc space but never free
> > > them.:)
> > >
> > > Good luck!
> > >
> > > On 2/2/07, Xiao-Feng Li <xi...@gmail.com> wrote:
> > > >
> > > > Leo, I can't fully understand your problem and proposal. I am working
> > > > in DRLVM GC component, I want to understand how GC can tackle the
> > > > issue you meet.
> > > >
> > > > You said the native memory is allocated (probably with malloc) for
> > > > byte buffer. I wonder how the process goes. I assume it's in native
> > > > code. Then you use malloc to allocate a block of memory, and put some
> > > > Java object (byte buffer) into it? Is this what you do? I wonder how
> > > > do you manipulate Java object placement in native code; and if the
> > > > native memory block is out of Java heap, do you expect GC to manage
> > > > it?
> > > >
> > > > You said when GC reclaims those byte buffer object, the native memory
> > > > block can be freed in native code. How do you know if an object is
> > > > reclaimed by GC or not?
> > > >
> > > > Thanks,
> > > > xiaofeng
> > > >
> > > > On 2/2/07, Leo Li < liyilei1979@gmail.com> wrote:
> > > > > Hi, all:
> > > > >     After applying patch3073, I added the support for direct byte buffer
> > > > in
> > > > > nio module as spec requires. But now here exists a problem: The native
> > > > > memory allocated to the byte buffer can be released when gc releases the
> > > > > byte buffer object, however, if the native heap is full while java heap
> > > > > still has space, gc will not be triggered.
> > > > >     It seems that RI will start gc before native memory heap is depleted
> > > > and
> > > > > thus prevents out-of-memory error.
> > > > >
> > > > >    Then our work focuses on:
> > > > >    1. When gc is required.
> > > > >    2. Trigger a gc.
> > > > >    The first one requires that we get support from operating system,
> > > > since
> > > > > the memory allocated in the native code, for example by malloc, is out
> > > > of
> > > > > the control of java VM.( I have ever thought of counting the used memory
> > > > in
> > > > > hymemory_allocate, but the plan fails since hymemory_free will not
> > > > report
> > > > > how much space is released.)
> > > > >    The second one needs the help from VM. System.gc() is not so reliable
> > > > as
> > > > > spec says, so it is necessary to have a internal channel to notify VM to
> > > > > start gc.
> > > > >
> > > > >     One solution, I think, is to let a monitor thread in VM to check
> > > > whether
> > > > > OS physical memory is low. For example, the
> > > > QueryMemoryResourceNotification
> > > > > of win32 API is a candidate. Although the interrupt model is more
> > > > effective:
> > > > > win32 SDK provides a CreateMemoryResourceNotification to get a handler
> > > > on
> > > > > which the monitor thread can wait, maybe on other platforms, OS does not
> > > > > supply such a convenience. So the monitor thread in the VM might have to
> > > >
> > > > > check the OS resource once for a while and if necessary the monitor
> > > > thread
> > > > > will call a GC.
> > > > >
> > > > >    My suggestion is first to add some function to monitor memory in
> > > > portlib,
> > > > > since it is highly related to platforms and in portlib there has been
> > > > > some useful tools.(Thanks Mark to point out that.) On the other hand, we
> > > > can
> > > > > negotiate a channel to trigger gc in VM. Actually I am not an expert on
> > > > VM
> > > > > since I am not sure whether there has been some monitor thread and the
> > > > load
> > > > > on performance if such a monitor thread is added to the VM...
> > > > >
> > > > >
> > > > > --
> > > > > Leo Li
> > > > > China Software Development Lab, IBM
> > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Leo Li
> > > China Software Development Lab, IBM
> > >
> > >
> >
>
>
> --
>
> Best Regards!
>
> Jimmy, Jing Lv
> China Software Development Lab, IBM
>


-- 
Ivan
Intel Enterprise Solutions Software Division

Re: [VM]How to trigue GC while free native memory is low.

[Ivan Volosyuk <iv...@gmail.com>]

On 2/3/07, Leo Li <li...@gmail.com> wrote:
> Thanks for all the advices.
> Can we conclude that:
>
> 1. The problem is related to a series of problems, as Ivan summarized. It is
> about the pattern to release native resources in java.
> 2. It is related to gc, if we do not intend to design another mechanism to
> track object reference. Only when the object cannot be referenced, have we
> the right to release the native resource of it. Actually the problem to
> release native resources can occur in any object besides direct byte buffer
> and zip related objects.
>
> But now we have some design problems:
> 1. When to trigger gc?
>     It might be too late when native memory allocation fails. It will cost a
> period of time for VM to complete gc and invoke phantom reference, not to
> mention other resource releases written in finalize method, as Gregory
> mentions.
> 2. The decision will be made on which level?
>     System wide, per process, or per VM?
>     I think it is the easiest to monitor memory resource on system wide
> level: Normally os will report its free physical memory and total physical
> memory, which we can rely on to deduce a tension in memory allocation.
>     Per process is a little difficult:how much memory it used is regarded
> to exist memory shortage?
>     Per VM is the most difficult: besides the problem from per process, we
> can hardly discriminate the usage of each vm in one process. The normal
> c-library just tells me the size we allocate in malloc by no message of the
> memory it releases in free.
>
> 3. Is it wise to allocate memory in a managed heap? There is some difference
> between Xiao Feng and Jimmy's advice:
>      Xiao Feng considers a way to allocate direct byte buffer on java heap.
> It will be great if it works: Then we will have no problem in releasing
> native resources.
>      Jimmy proposes we can give each VM a memory booking. It also works, I
> believe. But is it a little too onerous for us to implement a malloc/free
> function ourselves?
>
> Here are my own opinion:
> Although system wide memory monitor can be influnced by other process, I
> prefer it. Base on per process or per vm memory allocation, we can get just
> the information about virtual memory allocation. Hardly can we deduce that
> memory shortage occurs only from the virtual memory one process or vm uses :
> by page exchanging, os is able to provide more physical memory for one
> process or another.Native memory allocation failure is the direct result
> from shortage of system physical memory : os cannot commit enough physial
> memory for the required virtual pages.
>
> Since I am not an expert in the area, excuse me if I miss something.:)

Actually, we have three different situations when we get this memory
allocation failure.
1. Out of virtual memory: When we have enough physical memory and swap
we can experience problem when we exhausted all available virtual
memory space.
2. Out of physical memory + swap, allocation failure: We run out of
memory available to the system and it prevents further allocations.
3. Out of physical memory + swap, out of memory killer: Linux kernel
may allow us to allocate more memory then the system can give us. Some
of the processes in system will be killed when we will try to use such
memory. It will most probable will be the VM process, but who knows ;)

We should keep this in mind when coming to the final design of such
subsystem. Ideas? Comments?

> On 2/3/07, LvJimmy，Jing <fi...@gmail.com> wrote:
> >
> > 2007/2/2, Xiao-Feng Li <xi...@gmail.com>:
> > > Thank you, Jimmy and Leo, for the clear explanations.
> > >
> > > I think temporarily you can put the System.gc() there to workaround
> > > this issue. Yes, this is only a workaround since we cannot rely on it
> > > without explicit guarantee in its semantics.
> > >
> > > To really solve this problem, we might need a careful design. Your
> > > proposed approach looks like a possible solution. Have you thought to
> > > solve the problem by using GC to manage the raw memory blocks? Without
> > > deep understanding of the problem, I have some wild idea below. Please
> > > see if it is reasonable or stupid. I will read the spec of ByteBuffer
> > > to get more understanding.
> > >
> > > Since what the ByeBuffer needs are starting address and capacity, it
> > > doesn't really care if this piece of memory is in Java heap or in
> > > native runtime memory ( I assume.) We probably can provide some
> > > special kind of virtual Java object that serves as raw memory block to
> > > nio. In this works, we need not monitor the native runtime memory
> > > usage.
> > >
> >
> > Ah, this solution sounds interesting to me, yes, if we malloc memory
> > in heap then we no longer need to monitor the native memory. But we
> > may need a kit of methods offer this heap-memory service:
> > 1) a function of malloc_in_heap()
> > 2) a function of free_heap_block()
> > 3) heap monitor should record this malloc/free action
> > 4) ...
> > However I do think this may be a good solution, and it still depends
> > on how easy can it be implemented. Note not only direct ByteBuffer
> > require this native memory usage.
> >
> > > This approach need certain contract between Java classes and GC about
> > > the special kind of Java object. Probably we can write a layer of Java
> > > class wrapper for raw memory allocation, which hides the contract from
> > > other common classes.
> > >
> >
> > As we have a portlib, this can be implemented in such hymem_malloc or so.
> >
> > > At the same time, I am thinking of your proposed approach, and will
> > > get back later.
> > >
> >
> > Thanks, waiting for you the GC guru's appoach towards heap memory
> > allocation :)
> >
> > > Thanks,
> > > xiaofeng
> > >
> > > On 2/2/07, Leo Li <li...@gmail.com> wrote:
> > > > Hi, Xiao-Feng:
> > > >      Excuse me if I am confusing you.
> > > >      The direct byte buffer holds a block of native memory space used
> > in nio
> > > > operation. The native byte buffer will be freed through the byte
> > buffer is
> > > > gc collected and added to ReferenceQueue monitored by MemorySpy series
> > > > classes.
> > > >      But the gc will not be triggered if the java heap is still empty.
> > Here
> > > > is an example: I allocate a large block of native memory for a byte
> > buffer
> > > > while the byte buffer itself is quite small. Thus quickly native heap
> > is
> > > > depleted while VM still does not think gc is needed since java heap
> > has a
> > > > lot of free memory.
> > > >
> > > >
> > > > import java.nio.*;
> > > > public class Test {
> > > >     public static void main(String[] args) throws Exception {
> > > >         for(int i = 0;i<1000;i++)
> > > >         {
> > > >             ByteBuffer byteBuffer = ByteBuffer.allocateDirect
> > (10240000);
> > > >             //System.gc();
> > > >         }
> > > >    }
> > > > }
> > > > RI runs well but Harmony will soon throw OutOfMemory exception.
> > > > But if remove the comment before System.gc(), Harmony will become ok.
> > > > So my point is that we need a mechanism to notify VM to start gc if we
> > have
> > > > no more native memory because sometimes problem will be solved if gc
> > is
> > > > fired. Of cause we cannot avoid a  user always malloc space but never
> > free
> > > > them.:)
> > > >
> > > > Good luck!
> > > >
> > > > On 2/2/07, Xiao-Feng Li <xi...@gmail.com> wrote:
> > > > >
> > > > > Leo, I can't fully understand your problem and proposal. I am
> > working
> > > > > in DRLVM GC component, I want to understand how GC can tackle the
> > > > > issue you meet.
> > > > >
> > > > > You said the native memory is allocated (probably with malloc) for
> > > > > byte buffer. I wonder how the process goes. I assume it's in native
> > > > > code. Then you use malloc to allocate a block of memory, and put
> > some
> > > > > Java object (byte buffer) into it? Is this what you do? I wonder how
> > > > > do you manipulate Java object placement in native code; and if the
> > > > > native memory block is out of Java heap, do you expect GC to manage
> > > > > it?
> > > > >
> > > > > You said when GC reclaims those byte buffer object, the native
> > memory
> > > > > block can be freed in native code. How do you know if an object is
> > > > > reclaimed by GC or not?
> > > > >
> > > > > Thanks,
> > > > > xiaofeng
> > > > >
> > > > > On 2/2/07, Leo Li < liyilei1979@gmail.com> wrote:
> > > > > > Hi, all:
> > > > > >     After applying patch3073, I added the support for direct byte
> > buffer
> > > > > in
> > > > > > nio module as spec requires. But now here exists a problem: The
> > native
> > > > > > memory allocated to the byte buffer can be released when gc
> > releases the
> > > > > > byte buffer object, however, if the native heap is full while java
> > heap
> > > > > > still has space, gc will not be triggered.
> > > > > >     It seems that RI will start gc before native memory heap is
> > depleted
> > > > > and
> > > > > > thus prevents out-of-memory error.
> > > > > >
> > > > > >    Then our work focuses on:
> > > > > >    1. When gc is required.
> > > > > >    2. Trigger a gc.
> > > > > >    The first one requires that we get support from operating
> > system,
> > > > > since
> > > > > > the memory allocated in the native code, for example by malloc, is
> > out
> > > > > of
> > > > > > the control of java VM.( I have ever thought of counting the used
> > memory
> > > > > in
> > > > > > hymemory_allocate, but the plan fails since hymemory_free will not
> > > > > report
> > > > > > how much space is released.)
> > > > > >    The second one needs the help from VM. System.gc() is not so
> > reliable
> > > > > as
> > > > > > spec says, so it is necessary to have a internal channel to notify
> > VM to
> > > > > > start gc.
> > > > > >
> > > > > >     One solution, I think, is to let a monitor thread in VM to
> > check
> > > > > whether
> > > > > > OS physical memory is low. For example, the
> > > > > QueryMemoryResourceNotification
> > > > > > of win32 API is a candidate. Although the interrupt model is more
> > > > > effective:
> > > > > > win32 SDK provides a CreateMemoryResourceNotification to get a
> > handler
> > > > > on
> > > > > > which the monitor thread can wait, maybe on other platforms, OS
> > does not
> > > > > > supply such a convenience. So the monitor thread in the VM might
> > have to
> > > > >
> > > > > > check the OS resource once for a while and if necessary the
> > monitor
> > > > > thread
> > > > > > will call a GC.
> > > > > >
> > > > > >    My suggestion is first to add some function to monitor memory
> > in
> > > > > portlib,
> > > > > > since it is highly related to platforms and in portlib there has
> > been
> > > > > > some useful tools.(Thanks Mark to point out that.) On the other
> > hand, we
> > > > > can
> > > > > > negotiate a channel to trigger gc in VM. Actually I am not an
> > expert on
> > > > > VM
> > > > > > since I am not sure whether there has been some monitor thread and
> > the
> > > > > load
> > > > > > on performance if such a monitor thread is added to the VM...
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Leo Li
> > > > > > China Software Development Lab, IBM
> > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Leo Li
> > > > China Software Development Lab, IBM
> > > >
> > > >
> > >
> >
> >
> > --
> >
> > Best Regards!
> >
> > Jimmy, Jing Lv
> > China Software Development Lab, IBM
> >
>
>
>
> --
> Leo Li
> China Software Development Lab, IBM
>
>


-- 
Ivan
Intel Enterprise Solutions Software Division

Re: [VM]How to trigue GC while free native memory is low.

[Tim Ellison <t....@gmail.com>]

(I'm struggling to see your comment in here Robin ... ?)

Robin Garner wrote:
> Leo Li wrote:
>> Thanks for all the advices.
>> Can we conclude that:
>>
>> 1. The problem is related to a series of problems, as Ivan summarized. 
>> It is
>> about the pattern to release native resources in java.
>> 2. It is related to gc, if we do not intend to design another mechanism to
>> track object reference. Only when the object cannot be referenced, have we
>> the right to release the native resource of it. Actually the problem to
>> release native resources can occur in any object besides direct byte buffer
>> and zip related objects.
>>
>> But now we have some design problems:
>> 1. When to trigger gc?
>>    It might be too late when native memory allocation fails. It will cost a
>> period of time for VM to complete gc and invoke phantom reference, not to
>> mention other resource releases written in finalize method, as Gregory
>> mentions.
>> 2. The decision will be made on which level?
>>    System wide, per process, or per VM?
>>    I think it is the easiest to monitor memory resource on system wide
>> level: Normally os will report its free physical memory and total physical
>> memory, which we can rely on to deduce a tension in memory allocation.
>>    Per process is a little difficult:how much memory it used is regarded
>> to exist memory shortage?
>>    Per VM is the most difficult: besides the problem from per process, we
>> can hardly discriminate the usage of each vm in one process. The normal
>> c-library just tells me the size we allocate in malloc by no message of the
>> memory it releases in free.
>>
>> 3. Is it wise to allocate memory in a managed heap? There is some 
>> difference
>> between Xiao Feng and Jimmy's advice:
>>     Xiao Feng considers a way to allocate direct byte buffer on java heap.
>> It will be great if it works: Then we will have no problem in releasing
>> native resources.
>>     Jimmy proposes we can give each VM a memory booking. It also works, I
>> believe. But is it a little too onerous for us to implement a malloc/free
>> function ourselves?
>>
>> Here are my own opinion:
>> Although system wide memory monitor can be influnced by other process, I
>> prefer it. Base on per process or per vm memory allocation, we can get just
>> the information about virtual memory allocation. Hardly can we deduce that
>> memory shortage occurs only from the virtual memory one process or vm 
>> uses :
>> by page exchanging, os is able to provide more physical memory for one
>> process or another.Native memory allocation failure is the direct result
>> from shortage of system physical memory : os cannot commit enough physial
>> memory for the required virtual pages.
>>
>> Since I am not an expert in the area, excuse me if I miss something.:)
>> On 2/3/07, LvJimmy,Jing <fi...@gmail.com> wrote:
>>> 2007/2/2, Xiao-Feng Li <xi...@gmail.com>:
>>>> Thank you, Jimmy and Leo, for the clear explanations.
>>>>
>>>> I think temporarily you can put the System.gc() there to workaround
>>>> this issue. Yes, this is only a workaround since we cannot rely on it
>>>> without explicit guarantee in its semantics.
>>>>
>>>> To really solve this problem, we might need a careful design. Your
>>>> proposed approach looks like a possible solution. Have you thought to
>>>> solve the problem by using GC to manage the raw memory blocks? Without
>>>> deep understanding of the problem, I have some wild idea below. Please
>>>> see if it is reasonable or stupid. I will read the spec of ByteBuffer
>>>> to get more understanding.
>>>>
>>>> Since what the ByeBuffer needs are starting address and capacity, it
>>>> doesn't really care if this piece of memory is in Java heap or in
>>>> native runtime memory ( I assume.) We probably can provide some
>>>> special kind of virtual Java object that serves as raw memory block to
>>>> nio. In this works, we need not monitor the native runtime memory
>>>> usage.
>>>>
>>> Ah, this solution sounds interesting to me, yes, if we malloc memory
>>> in heap then we no longer need to monitor the native memory. But we
>>> may need a kit of methods offer this heap-memory service:
>>> 1) a function of malloc_in_heap()
>>> 2) a function of free_heap_block()
>>> 3) heap monitor should record this malloc/free action
>>> 4) ...
>>> However I do think this may be a good solution, and it still depends
>>> on how easy can it be implemented. Note not only direct ByteBuffer
>>> require this native memory usage.
>>>
>>>> This approach need certain contract between Java classes and GC about
>>>> the special kind of Java object. Probably we can write a layer of Java
>>>> class wrapper for raw memory allocation, which hides the contract from
>>>> other common classes.
>>>>
>>> As we have a portlib, this can be implemented in such hymem_malloc or so.
>>>
>>>> At the same time, I am thinking of your proposed approach, and will
>>>> get back later.
>>>>
>>> Thanks, waiting for you the GC guru's appoach towards heap memory
>>> allocation :)
>>>
>>>> Thanks,
>>>> xiaofeng
>>>>
>>>> On 2/2/07, Leo Li <li...@gmail.com> wrote:
>>>>> Hi, Xiao-Feng:
>>>>>      Excuse me if I am confusing you.
>>>>>      The direct byte buffer holds a block of native memory space used
>>> in nio
>>>>> operation. The native byte buffer will be freed through the byte
>>> buffer is
>>>>> gc collected and added to ReferenceQueue monitored by MemorySpy 
>>> series
>>>>> classes.
>>>>>      But the gc will not be triggered if the java heap is still 
>>> empty.
>>> Here
>>>>> is an example: I allocate a large block of native memory for a byte
>>> buffer
>>>>> while the byte buffer itself is quite small. Thus quickly native heap
>>> is
>>>>> depleted while VM still does not think gc is needed since java heap
>>> has a
>>>>> lot of free memory.
>>>>>
>>>>>
>>>>> import java.nio.*;
>>>>> public class Test {
>>>>>     public static void main(String[] args) throws Exception {
>>>>>         for(int i = 0;i<1000;i++)
>>>>>         {
>>>>>             ByteBuffer byteBuffer = ByteBuffer.allocateDirect
>>> (10240000);
>>>>>             //System.gc();
>>>>>         }
>>>>>    }
>>>>> }
>>>>> RI runs well but Harmony will soon throw OutOfMemory exception.
>>>>> But if remove the comment before System.gc(), Harmony will become ok.
>>>>> So my point is that we need a mechanism to notify VM to start gc 
>>> if we
>>> have
>>>>> no more native memory because sometimes problem will be solved if gc
>>> is
>>>>> fired. Of cause we cannot avoid a  user always malloc space but never
>>> free
>>>>> them.:)
>>>>>
>>>>> Good luck!
>>>>>
>>>>> On 2/2/07, Xiao-Feng Li <xi...@gmail.com> wrote:
>>>>>> Leo, I can't fully understand your problem and proposal. I am
>>> working
>>>>>> in DRLVM GC component, I want to understand how GC can tackle the
>>>>>> issue you meet.
>>>>>>
>>>>>> You said the native memory is allocated (probably with malloc) for
>>>>>> byte buffer. I wonder how the process goes. I assume it's in native
>>>>>> code. Then you use malloc to allocate a block of memory, and put
>>> some
>>>>>> Java object (byte buffer) into it? Is this what you do? I wonder 
>>> how
>>>>>> do you manipulate Java object placement in native code; and if the
>>>>>> native memory block is out of Java heap, do you expect GC to manage
>>>>>> it?
>>>>>>
>>>>>> You said when GC reclaims those byte buffer object, the native
>>> memory
>>>>>> block can be freed in native code. How do you know if an object is
>>>>>> reclaimed by GC or not?
>>>>>>
>>>>>> Thanks,
>>>>>> xiaofeng
>>>>>>
>>>>>> On 2/2/07, Leo Li < liyilei1979@gmail.com> wrote:
>>>>>>> Hi, all:
>>>>>>>     After applying patch3073, I added the support for direct byte
>>> buffer
>>>>>> in
>>>>>>> nio module as spec requires. But now here exists a problem: The
>>> native
>>>>>>> memory allocated to the byte buffer can be released when gc
>>> releases the
>>>>>>> byte buffer object, however, if the native heap is full while 
>>> java
>>> heap
>>>>>>> still has space, gc will not be triggered.
>>>>>>>     It seems that RI will start gc before native memory heap is
>>> depleted
>>>>>> and
>>>>>>> thus prevents out-of-memory error.
>>>>>>>
>>>>>>>    Then our work focuses on:
>>>>>>>    1. When gc is required.
>>>>>>>    2. Trigger a gc.
>>>>>>>    The first one requires that we get support from operating
>>> system,
>>>>>> since
>>>>>>> the memory allocated in the native code, for example by 
>>> malloc, is
>>> out
>>>>>> of
>>>>>>> the control of java VM.( I have ever thought of counting the used
>>> memory
>>>>>> in
>>>>>>> hymemory_allocate, but the plan fails since hymemory_free will 
>>> not
>>>>>> report
>>>>>>> how much space is released.)
>>>>>>>    The second one needs the help from VM. System.gc() is not so
>>> reliable
>>>>>> as
>>>>>>> spec says, so it is necessary to have a internal channel to 
>>> notify
>>> VM to
>>>>>>> start gc.
>>>>>>>
>>>>>>>     One solution, I think, is to let a monitor thread in VM to
>>> check
>>>>>> whether
>>>>>>> OS physical memory is low. For example, the
>>>>>> QueryMemoryResourceNotification
>>>>>>> of win32 API is a candidate. Although the interrupt model is more
>>>>>> effective:
>>>>>>> win32 SDK provides a CreateMemoryResourceNotification to get a
>>> handler
>>>>>> on
>>>>>>> which the monitor thread can wait, maybe on other platforms, OS
>>> does not
>>>>>>> supply such a convenience. So the monitor thread in the VM might
>>> have to
>>>>>>> check the OS resource once for a while and if necessary the
>>> monitor
>>>>>> thread
>>>>>>> will call a GC.
>>>>>>>
>>>>>>>    My suggestion is first to add some function to monitor memory
>>> in
>>>>>> portlib,
>>>>>>> since it is highly related to platforms and in portlib there has
>>> been
>>>>>>> some useful tools.(Thanks Mark to point out that.) On the other
>>> hand, we
>>>>>> can
>>>>>>> negotiate a channel to trigger gc in VM. Actually I am not an
>>> expert on
>>>>>> VM
>>>>>>> since I am not sure whether there has been some monitor thread 
>>> and
>>> the
>>>>>> load
>>>>>>> on performance if such a monitor thread is added to the VM...
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Leo Li
>>>>>>> China Software Development Lab, IBM
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Leo Li
>>>>> China Software Development Lab, IBM
>>>>>
>>>>>
>>>
>>> -- 
>>>
>>> Best Regards!
>>>
>>> Jimmy, Jing Lv
>>> China Software Development Lab, IBM
>>>
>>
>>
> 
> 

Re: [VM]How to trigue GC while free native memory is low.

[Robin Garner <ro...@anu.edu.au>]

Leo Li wrote:
> Thanks for all the advices.
> Can we conclude that:
> 
> 1. The problem is related to a series of problems, as Ivan summarized. 
> It is
> about the pattern to release native resources in java.
> 2. It is related to gc, if we do not intend to design another mechanism to
> track object reference. Only when the object cannot be referenced, have we
> the right to release the native resource of it. Actually the problem to
> release native resources can occur in any object besides direct byte buffer
> and zip related objects.
> 
> But now we have some design problems:
> 1. When to trigger gc?
>    It might be too late when native memory allocation fails. It will cost a
> period of time for VM to complete gc and invoke phantom reference, not to
> mention other resource releases written in finalize method, as Gregory
> mentions.
> 2. The decision will be made on which level?
>    System wide, per process, or per VM?
>    I think it is the easiest to monitor memory resource on system wide
> level: Normally os will report its free physical memory and total physical
> memory, which we can rely on to deduce a tension in memory allocation.
>    Per process is a little difficult:how much memory it used is regarded
> to exist memory shortage?
>    Per VM is the most difficult: besides the problem from per process, we
> can hardly discriminate the usage of each vm in one process. The normal
> c-library just tells me the size we allocate in malloc by no message of the
> memory it releases in free.
> 
> 3. Is it wise to allocate memory in a managed heap? There is some 
> difference
> between Xiao Feng and Jimmy's advice:
>     Xiao Feng considers a way to allocate direct byte buffer on java heap.
> It will be great if it works: Then we will have no problem in releasing
> native resources.
>     Jimmy proposes we can give each VM a memory booking. It also works, I
> believe. But is it a little too onerous for us to implement a malloc/free
> function ourselves?
> 
> Here are my own opinion:
> Although system wide memory monitor can be influnced by other process, I
> prefer it. Base on per process or per vm memory allocation, we can get just
> the information about virtual memory allocation. Hardly can we deduce that
> memory shortage occurs only from the virtual memory one process or vm 
> uses :
> by page exchanging, os is able to provide more physical memory for one
> process or another.Native memory allocation failure is the direct result
> from shortage of system physical memory : os cannot commit enough physial
> memory for the required virtual pages.
> 
> Since I am not an expert in the area, excuse me if I miss something.:)
> On 2/3/07, LvJimmy，Jing <fi...@gmail.com> wrote:
>>
>> 2007/2/2, Xiao-Feng Li <xi...@gmail.com>:
>> > Thank you, Jimmy and Leo, for the clear explanations.
>> >
>> > I think temporarily you can put the System.gc() there to workaround
>> > this issue. Yes, this is only a workaround since we cannot rely on it
>> > without explicit guarantee in its semantics.
>> >
>> > To really solve this problem, we might need a careful design. Your
>> > proposed approach looks like a possible solution. Have you thought to
>> > solve the problem by using GC to manage the raw memory blocks? Without
>> > deep understanding of the problem, I have some wild idea below. Please
>> > see if it is reasonable or stupid. I will read the spec of ByteBuffer
>> > to get more understanding.
>> >
>> > Since what the ByeBuffer needs are starting address and capacity, it
>> > doesn't really care if this piece of memory is in Java heap or in
>> > native runtime memory ( I assume.) We probably can provide some
>> > special kind of virtual Java object that serves as raw memory block to
>> > nio. In this works, we need not monitor the native runtime memory
>> > usage.
>> >
>>
>> Ah, this solution sounds interesting to me, yes, if we malloc memory
>> in heap then we no longer need to monitor the native memory. But we
>> may need a kit of methods offer this heap-memory service:
>> 1) a function of malloc_in_heap()
>> 2) a function of free_heap_block()
>> 3) heap monitor should record this malloc/free action
>> 4) ...
>> However I do think this may be a good solution, and it still depends
>> on how easy can it be implemented. Note not only direct ByteBuffer
>> require this native memory usage.
>>
>> > This approach need certain contract between Java classes and GC about
>> > the special kind of Java object. Probably we can write a layer of Java
>> > class wrapper for raw memory allocation, which hides the contract from
>> > other common classes.
>> >
>>
>> As we have a portlib, this can be implemented in such hymem_malloc or so.
>>
>> > At the same time, I am thinking of your proposed approach, and will
>> > get back later.
>> >
>>
>> Thanks, waiting for you the GC guru's appoach towards heap memory
>> allocation :)
>>
>> > Thanks,
>> > xiaofeng
>> >
>> > On 2/2/07, Leo Li <li...@gmail.com> wrote:
>> > > Hi, Xiao-Feng:
>> > >      Excuse me if I am confusing you.
>> > >      The direct byte buffer holds a block of native memory space used
>> in nio
>> > > operation. The native byte buffer will be freed through the byte
>> buffer is
>> > > gc collected and added to ReferenceQueue monitored by MemorySpy 
>> series
>> > > classes.
>> > >      But the gc will not be triggered if the java heap is still 
>> empty.
>> Here
>> > > is an example: I allocate a large block of native memory for a byte
>> buffer
>> > > while the byte buffer itself is quite small. Thus quickly native heap
>> is
>> > > depleted while VM still does not think gc is needed since java heap
>> has a
>> > > lot of free memory.
>> > >
>> > >
>> > > import java.nio.*;
>> > > public class Test {
>> > >     public static void main(String[] args) throws Exception {
>> > >         for(int i = 0;i<1000;i++)
>> > >         {
>> > >             ByteBuffer byteBuffer = ByteBuffer.allocateDirect
>> (10240000);
>> > >             //System.gc();
>> > >         }
>> > >    }
>> > > }
>> > > RI runs well but Harmony will soon throw OutOfMemory exception.
>> > > But if remove the comment before System.gc(), Harmony will become ok.
>> > > So my point is that we need a mechanism to notify VM to start gc 
>> if we
>> have
>> > > no more native memory because sometimes problem will be solved if gc
>> is
>> > > fired. Of cause we cannot avoid a  user always malloc space but never
>> free
>> > > them.:)
>> > >
>> > > Good luck!
>> > >
>> > > On 2/2/07, Xiao-Feng Li <xi...@gmail.com> wrote:
>> > > >
>> > > > Leo, I can't fully understand your problem and proposal. I am
>> working
>> > > > in DRLVM GC component, I want to understand how GC can tackle the
>> > > > issue you meet.
>> > > >
>> > > > You said the native memory is allocated (probably with malloc) for
>> > > > byte buffer. I wonder how the process goes. I assume it's in native
>> > > > code. Then you use malloc to allocate a block of memory, and put
>> some
>> > > > Java object (byte buffer) into it? Is this what you do? I wonder 
>> how
>> > > > do you manipulate Java object placement in native code; and if the
>> > > > native memory block is out of Java heap, do you expect GC to manage
>> > > > it?
>> > > >
>> > > > You said when GC reclaims those byte buffer object, the native
>> memory
>> > > > block can be freed in native code. How do you know if an object is
>> > > > reclaimed by GC or not?
>> > > >
>> > > > Thanks,
>> > > > xiaofeng
>> > > >
>> > > > On 2/2/07, Leo Li < liyilei1979@gmail.com> wrote:
>> > > > > Hi, all:
>> > > > >     After applying patch3073, I added the support for direct byte
>> buffer
>> > > > in
>> > > > > nio module as spec requires. But now here exists a problem: The
>> native
>> > > > > memory allocated to the byte buffer can be released when gc
>> releases the
>> > > > > byte buffer object, however, if the native heap is full while 
>> java
>> heap
>> > > > > still has space, gc will not be triggered.
>> > > > >     It seems that RI will start gc before native memory heap is
>> depleted
>> > > > and
>> > > > > thus prevents out-of-memory error.
>> > > > >
>> > > > >    Then our work focuses on:
>> > > > >    1. When gc is required.
>> > > > >    2. Trigger a gc.
>> > > > >    The first one requires that we get support from operating
>> system,
>> > > > since
>> > > > > the memory allocated in the native code, for example by 
>> malloc, is
>> out
>> > > > of
>> > > > > the control of java VM.( I have ever thought of counting the used
>> memory
>> > > > in
>> > > > > hymemory_allocate, but the plan fails since hymemory_free will 
>> not
>> > > > report
>> > > > > how much space is released.)
>> > > > >    The second one needs the help from VM. System.gc() is not so
>> reliable
>> > > > as
>> > > > > spec says, so it is necessary to have a internal channel to 
>> notify
>> VM to
>> > > > > start gc.
>> > > > >
>> > > > >     One solution, I think, is to let a monitor thread in VM to
>> check
>> > > > whether
>> > > > > OS physical memory is low. For example, the
>> > > > QueryMemoryResourceNotification
>> > > > > of win32 API is a candidate. Although the interrupt model is more
>> > > > effective:
>> > > > > win32 SDK provides a CreateMemoryResourceNotification to get a
>> handler
>> > > > on
>> > > > > which the monitor thread can wait, maybe on other platforms, OS
>> does not
>> > > > > supply such a convenience. So the monitor thread in the VM might
>> have to
>> > > >
>> > > > > check the OS resource once for a while and if necessary the
>> monitor
>> > > > thread
>> > > > > will call a GC.
>> > > > >
>> > > > >    My suggestion is first to add some function to monitor memory
>> in
>> > > > portlib,
>> > > > > since it is highly related to platforms and in portlib there has
>> been
>> > > > > some useful tools.(Thanks Mark to point out that.) On the other
>> hand, we
>> > > > can
>> > > > > negotiate a channel to trigger gc in VM. Actually I am not an
>> expert on
>> > > > VM
>> > > > > since I am not sure whether there has been some monitor thread 
>> and
>> the
>> > > > load
>> > > > > on performance if such a monitor thread is added to the VM...
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Leo Li
>> > > > > China Software Development Lab, IBM
>> > > > >
>> > > > >
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Leo Li
>> > > China Software Development Lab, IBM
>> > >
>> > >
>> >
>>
>>
>> -- 
>>
>> Best Regards!
>>
>> Jimmy, Jing Lv
>> China Software Development Lab, IBM
>>
> 
> 
> 


-- 
Robin Garner
Dept. of Computer Science
Australian National University
http://cs.anu.edu.au/people/Robin.Garner/

Re: [VM]How to trigue GC while free native memory is low.

[Leo Li <li...@gmail.com>]

Thanks for all the advices.
Can we conclude that:

1. The problem is related to a series of problems, as Ivan summarized. It is
about the pattern to release native resources in java.
2. It is related to gc, if we do not intend to design another mechanism to
track object reference. Only when the object cannot be referenced, have we
the right to release the native resource of it. Actually the problem to
release native resources can occur in any object besides direct byte buffer
and zip related objects.

But now we have some design problems:
1. When to trigger gc?
    It might be too late when native memory allocation fails. It will cost a
period of time for VM to complete gc and invoke phantom reference, not to
mention other resource releases written in finalize method, as Gregory
mentions.
2. The decision will be made on which level?
    System wide, per process, or per VM?
    I think it is the easiest to monitor memory resource on system wide
level: Normally os will report its free physical memory and total physical
memory, which we can rely on to deduce a tension in memory allocation.
    Per process is a little difficult:how much memory it used is regarded
to exist memory shortage?
    Per VM is the most difficult: besides the problem from per process, we
can hardly discriminate the usage of each vm in one process. The normal
c-library just tells me the size we allocate in malloc by no message of the
memory it releases in free.

3. Is it wise to allocate memory in a managed heap? There is some difference
between Xiao Feng and Jimmy's advice:
     Xiao Feng considers a way to allocate direct byte buffer on java heap.
It will be great if it works: Then we will have no problem in releasing
native resources.
     Jimmy proposes we can give each VM a memory booking. It also works, I
believe. But is it a little too onerous for us to implement a malloc/free
function ourselves?

Here are my own opinion:
Although system wide memory monitor can be influnced by other process, I
prefer it. Base on per process or per vm memory allocation, we can get just
the information about virtual memory allocation. Hardly can we deduce that
memory shortage occurs only from the virtual memory one process or vm uses :
by page exchanging, os is able to provide more physical memory for one
process or another.Native memory allocation failure is the direct result
from shortage of system physical memory : os cannot commit enough physial
memory for the required virtual pages.

Since I am not an expert in the area, excuse me if I miss something.:)
On 2/3/07, LvJimmy，Jing <fi...@gmail.com> wrote:
>
> 2007/2/2, Xiao-Feng Li <xi...@gmail.com>:
> > Thank you, Jimmy and Leo, for the clear explanations.
> >
> > I think temporarily you can put the System.gc() there to workaround
> > this issue. Yes, this is only a workaround since we cannot rely on it
> > without explicit guarantee in its semantics.
> >
> > To really solve this problem, we might need a careful design. Your
> > proposed approach looks like a possible solution. Have you thought to
> > solve the problem by using GC to manage the raw memory blocks? Without
> > deep understanding of the problem, I have some wild idea below. Please
> > see if it is reasonable or stupid. I will read the spec of ByteBuffer
> > to get more understanding.
> >
> > Since what the ByeBuffer needs are starting address and capacity, it
> > doesn't really care if this piece of memory is in Java heap or in
> > native runtime memory ( I assume.) We probably can provide some
> > special kind of virtual Java object that serves as raw memory block to
> > nio. In this works, we need not monitor the native runtime memory
> > usage.
> >
>
> Ah, this solution sounds interesting to me, yes, if we malloc memory
> in heap then we no longer need to monitor the native memory. But we
> may need a kit of methods offer this heap-memory service:
> 1) a function of malloc_in_heap()
> 2) a function of free_heap_block()
> 3) heap monitor should record this malloc/free action
> 4) ...
> However I do think this may be a good solution, and it still depends
> on how easy can it be implemented. Note not only direct ByteBuffer
> require this native memory usage.
>
> > This approach need certain contract between Java classes and GC about
> > the special kind of Java object. Probably we can write a layer of Java
> > class wrapper for raw memory allocation, which hides the contract from
> > other common classes.
> >
>
> As we have a portlib, this can be implemented in such hymem_malloc or so.
>
> > At the same time, I am thinking of your proposed approach, and will
> > get back later.
> >
>
> Thanks, waiting for you the GC guru's appoach towards heap memory
> allocation :)
>
> > Thanks,
> > xiaofeng
> >
> > On 2/2/07, Leo Li <li...@gmail.com> wrote:
> > > Hi, Xiao-Feng:
> > >      Excuse me if I am confusing you.
> > >      The direct byte buffer holds a block of native memory space used
> in nio
> > > operation. The native byte buffer will be freed through the byte
> buffer is
> > > gc collected and added to ReferenceQueue monitored by MemorySpy series
> > > classes.
> > >      But the gc will not be triggered if the java heap is still empty.
> Here
> > > is an example: I allocate a large block of native memory for a byte
> buffer
> > > while the byte buffer itself is quite small. Thus quickly native heap
> is
> > > depleted while VM still does not think gc is needed since java heap
> has a
> > > lot of free memory.
> > >
> > >
> > > import java.nio.*;
> > > public class Test {
> > >     public static void main(String[] args) throws Exception {
> > >         for(int i = 0;i<1000;i++)
> > >         {
> > >             ByteBuffer byteBuffer = ByteBuffer.allocateDirect
> (10240000);
> > >             //System.gc();
> > >         }
> > >    }
> > > }
> > > RI runs well but Harmony will soon throw OutOfMemory exception.
> > > But if remove the comment before System.gc(), Harmony will become ok.
> > > So my point is that we need a mechanism to notify VM to start gc if we
> have
> > > no more native memory because sometimes problem will be solved if gc
> is
> > > fired. Of cause we cannot avoid a  user always malloc space but never
> free
> > > them.:)
> > >
> > > Good luck!
> > >
> > > On 2/2/07, Xiao-Feng Li <xi...@gmail.com> wrote:
> > > >
> > > > Leo, I can't fully understand your problem and proposal. I am
> working
> > > > in DRLVM GC component, I want to understand how GC can tackle the
> > > > issue you meet.
> > > >
> > > > You said the native memory is allocated (probably with malloc) for
> > > > byte buffer. I wonder how the process goes. I assume it's in native
> > > > code. Then you use malloc to allocate a block of memory, and put
> some
> > > > Java object (byte buffer) into it? Is this what you do? I wonder how
> > > > do you manipulate Java object placement in native code; and if the
> > > > native memory block is out of Java heap, do you expect GC to manage
> > > > it?
> > > >
> > > > You said when GC reclaims those byte buffer object, the native
> memory
> > > > block can be freed in native code. How do you know if an object is
> > > > reclaimed by GC or not?
> > > >
> > > > Thanks,
> > > > xiaofeng
> > > >
> > > > On 2/2/07, Leo Li < liyilei1979@gmail.com> wrote:
> > > > > Hi, all:
> > > > >     After applying patch3073, I added the support for direct byte
> buffer
> > > > in
> > > > > nio module as spec requires. But now here exists a problem: The
> native
> > > > > memory allocated to the byte buffer can be released when gc
> releases the
> > > > > byte buffer object, however, if the native heap is full while java
> heap
> > > > > still has space, gc will not be triggered.
> > > > >     It seems that RI will start gc before native memory heap is
> depleted
> > > > and
> > > > > thus prevents out-of-memory error.
> > > > >
> > > > >    Then our work focuses on:
> > > > >    1. When gc is required.
> > > > >    2. Trigger a gc.
> > > > >    The first one requires that we get support from operating
> system,
> > > > since
> > > > > the memory allocated in the native code, for example by malloc, is
> out
> > > > of
> > > > > the control of java VM.( I have ever thought of counting the used
> memory
> > > > in
> > > > > hymemory_allocate, but the plan fails since hymemory_free will not
> > > > report
> > > > > how much space is released.)
> > > > >    The second one needs the help from VM. System.gc() is not so
> reliable
> > > > as
> > > > > spec says, so it is necessary to have a internal channel to notify
> VM to
> > > > > start gc.
> > > > >
> > > > >     One solution, I think, is to let a monitor thread in VM to
> check
> > > > whether
> > > > > OS physical memory is low. For example, the
> > > > QueryMemoryResourceNotification
> > > > > of win32 API is a candidate. Although the interrupt model is more
> > > > effective:
> > > > > win32 SDK provides a CreateMemoryResourceNotification to get a
> handler
> > > > on
> > > > > which the monitor thread can wait, maybe on other platforms, OS
> does not
> > > > > supply such a convenience. So the monitor thread in the VM might
> have to
> > > >
> > > > > check the OS resource once for a while and if necessary the
> monitor
> > > > thread
> > > > > will call a GC.
> > > > >
> > > > >    My suggestion is first to add some function to monitor memory
> in
> > > > portlib,
> > > > > since it is highly related to platforms and in portlib there has
> been
> > > > > some useful tools.(Thanks Mark to point out that.) On the other
> hand, we
> > > > can
> > > > > negotiate a channel to trigger gc in VM. Actually I am not an
> expert on
> > > > VM
> > > > > since I am not sure whether there has been some monitor thread and
> the
> > > > load
> > > > > on performance if such a monitor thread is added to the VM...
> > > > >
> > > > >
> > > > > --
> > > > > Leo Li
> > > > > China Software Development Lab, IBM
> > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Leo Li
> > > China Software Development Lab, IBM
> > >
> > >
> >
>
>
> --
>
> Best Regards!
>
> Jimmy, Jing Lv
> China Software Development Lab, IBM
>



-- 
Leo Li
China Software Development Lab, IBM

Re: [VM]How to trigue GC while free native memory is low.

[LvJimmy，Jing <fi...@gmail.com>]

2007/2/2, Xiao-Feng Li <xi...@gmail.com>:
> Thank you, Jimmy and Leo, for the clear explanations.
>
> I think temporarily you can put the System.gc() there to workaround
> this issue. Yes, this is only a workaround since we cannot rely on it
> without explicit guarantee in its semantics.
>
> To really solve this problem, we might need a careful design. Your
> proposed approach looks like a possible solution. Have you thought to
> solve the problem by using GC to manage the raw memory blocks? Without
> deep understanding of the problem, I have some wild idea below. Please
> see if it is reasonable or stupid. I will read the spec of ByteBuffer
> to get more understanding.
>
> Since what the ByeBuffer needs are starting address and capacity, it
> doesn't really care if this piece of memory is in Java heap or in
> native runtime memory ( I assume.) We probably can provide some
> special kind of virtual Java object that serves as raw memory block to
> nio. In this works, we need not monitor the native runtime memory
> usage.
>

Ah, this solution sounds interesting to me, yes, if we malloc memory
in heap then we no longer need to monitor the native memory. But we
may need a kit of methods offer this heap-memory service:
1) a function of malloc_in_heap()
2) a function of free_heap_block()
3) heap monitor should record this malloc/free action
4) ...
However I do think this may be a good solution, and it still depends
on how easy can it be implemented. Note not only direct ByteBuffer
require this native memory usage.

> This approach need certain contract between Java classes and GC about
> the special kind of Java object. Probably we can write a layer of Java
> class wrapper for raw memory allocation, which hides the contract from
> other common classes.
>

As we have a portlib, this can be implemented in such hymem_malloc or so.

> At the same time, I am thinking of your proposed approach, and will
> get back later.
>

Thanks, waiting for you the GC guru's appoach towards heap memory allocation :)

> Thanks,
> xiaofeng
>
> On 2/2/07, Leo Li <li...@gmail.com> wrote:
> > Hi, Xiao-Feng:
> >      Excuse me if I am confusing you.
> >      The direct byte buffer holds a block of native memory space used in nio
> > operation. The native byte buffer will be freed through the byte buffer is
> > gc collected and added to ReferenceQueue monitored by MemorySpy series
> > classes.
> >      But the gc will not be triggered if the java heap is still empty. Here
> > is an example: I allocate a large block of native memory for a byte buffer
> > while the byte buffer itself is quite small. Thus quickly native heap is
> > depleted while VM still does not think gc is needed since java heap has a
> > lot of free memory.
> >
> >
> > import java.nio.*;
> > public class Test {
> >     public static void main(String[] args) throws Exception {
> >         for(int i = 0;i<1000;i++)
> >         {
> >             ByteBuffer byteBuffer = ByteBuffer.allocateDirect(10240000);
> >             //System.gc();
> >         }
> >    }
> > }
> > RI runs well but Harmony will soon throw OutOfMemory exception.
> > But if remove the comment before System.gc(), Harmony will become ok.
> > So my point is that we need a mechanism to notify VM to start gc if we have
> > no more native memory because sometimes problem will be solved if gc is
> > fired. Of cause we cannot avoid a  user always malloc space but never free
> > them.:)
> >
> > Good luck!
> >
> > On 2/2/07, Xiao-Feng Li <xi...@gmail.com> wrote:
> > >
> > > Leo, I can't fully understand your problem and proposal. I am working
> > > in DRLVM GC component, I want to understand how GC can tackle the
> > > issue you meet.
> > >
> > > You said the native memory is allocated (probably with malloc) for
> > > byte buffer. I wonder how the process goes. I assume it's in native
> > > code. Then you use malloc to allocate a block of memory, and put some
> > > Java object (byte buffer) into it? Is this what you do? I wonder how
> > > do you manipulate Java object placement in native code; and if the
> > > native memory block is out of Java heap, do you expect GC to manage
> > > it?
> > >
> > > You said when GC reclaims those byte buffer object, the native memory
> > > block can be freed in native code. How do you know if an object is
> > > reclaimed by GC or not?
> > >
> > > Thanks,
> > > xiaofeng
> > >
> > > On 2/2/07, Leo Li < liyilei1979@gmail.com> wrote:
> > > > Hi, all:
> > > >     After applying patch3073, I added the support for direct byte buffer
> > > in
> > > > nio module as spec requires. But now here exists a problem: The native
> > > > memory allocated to the byte buffer can be released when gc releases the
> > > > byte buffer object, however, if the native heap is full while java heap
> > > > still has space, gc will not be triggered.
> > > >     It seems that RI will start gc before native memory heap is depleted
> > > and
> > > > thus prevents out-of-memory error.
> > > >
> > > >    Then our work focuses on:
> > > >    1. When gc is required.
> > > >    2. Trigger a gc.
> > > >    The first one requires that we get support from operating system,
> > > since
> > > > the memory allocated in the native code, for example by malloc, is out
> > > of
> > > > the control of java VM.( I have ever thought of counting the used memory
> > > in
> > > > hymemory_allocate, but the plan fails since hymemory_free will not
> > > report
> > > > how much space is released.)
> > > >    The second one needs the help from VM. System.gc() is not so reliable
> > > as
> > > > spec says, so it is necessary to have a internal channel to notify VM to
> > > > start gc.
> > > >
> > > >     One solution, I think, is to let a monitor thread in VM to check
> > > whether
> > > > OS physical memory is low. For example, the
> > > QueryMemoryResourceNotification
> > > > of win32 API is a candidate. Although the interrupt model is more
> > > effective:
> > > > win32 SDK provides a CreateMemoryResourceNotification to get a handler
> > > on
> > > > which the monitor thread can wait, maybe on other platforms, OS does not
> > > > supply such a convenience. So the monitor thread in the VM might have to
> > >
> > > > check the OS resource once for a while and if necessary the monitor
> > > thread
> > > > will call a GC.
> > > >
> > > >    My suggestion is first to add some function to monitor memory in
> > > portlib,
> > > > since it is highly related to platforms and in portlib there has been
> > > > some useful tools.(Thanks Mark to point out that.) On the other hand, we
> > > can
> > > > negotiate a channel to trigger gc in VM. Actually I am not an expert on
> > > VM
> > > > since I am not sure whether there has been some monitor thread and the
> > > load
> > > > on performance if such a monitor thread is added to the VM...
> > > >
> > > >
> > > > --
> > > > Leo Li
> > > > China Software Development Lab, IBM
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > Leo Li
> > China Software Development Lab, IBM
> >
> >
>


-- 

Best Regards!

Jimmy, Jing Lv
China Software Development Lab, IBM

Re: [VM]How to trigue GC while free native memory is low.

[Robin Garner <ro...@anu.edu.au>]

Leo Li wrote:
> So is it possible for our VM support an option by which our customer can
> customize the policy while the VM still supply a default policy, for
> example, assuming the native heap limit as that of the java heap?

Naturally.  I don't think that's a good example, though, as much of the 
discussion on this thread has illustrated the connection between native 
resources and java objects is somewhat hard to pin down.

Policy could be as simple as "request a GC whenever native memory grows 
by 1MB" or "reaches 75% utilization".  Or arbitrarily complex, eg 
negotiated with some host environment.  The point is to simplify the 
design by separating policy from mechanism.

> On 2/9/07, Rana Dasgupta <rd...@gmail.com> wrote:
>>
>> On 2/8/07, Robin Garner <ro...@anu.edu.au> wrote:
>> >
>> > Tim Ellison wrote:
>> > > Rana Dasgupta wrote:
>> > >> On 2/5/07, Tim Ellison <t....@gmail.com> wrote:
>> >
>> > >>> I don't think that we need to expand the scope of the problem to do
>> > >>> all types of resource accounting in the JVM or in an underlying
>> > >>> platform layer. I don't see the spec providing any basic 
>> guidance in
>> > >>> this area, it falls in the category of failsafe JVM's which is an
>> > >>> interesting area, but IMHO beyond scope at this point. My 
>> suggestion
>> > >>> is that we treat this as a specific nio api implementation problem
>> > >>> for which we can think of adding some VM/GC support if it helps.
>> > >>>
>> > >> We may have to disagree here then, I think we should indeed be
>> > >> considering a general solution for detecting early when these Java
>> > >> 'handle' objects are collectable -- then we can apply it to direct
>> > >> buffers or other handle types.
>> >
>> > >My point was that by adding a single GC -> VM interface function, all
>> > >types of native resources could be addressed in a simple extensible
>> way.
>> > >The subsequent policy and mechanism decisions then become an ongoing
>> > >design process for the VM and classlib native implementers, but once
>> > >this interface function is added, GC can support freeing of native
>> > >resources by whatever mechanisms the VM chooses to implement.
>>
>>
>> I don't understand very well how the JVM can define policies on its
>> platform/OS resource usage when it may be hosted by a server which wants
>> to
>> define such policies itself. In other words, say the JVM is running an
>> RDBMS
>> managed stored procedure. Should it be making assumptions about how much
>> native heap to use etc. when the hosting RDBMS may have negotiated such
>> policies itself and may be choosing to invoke one or more instances of 
>> the
>> JVM?
>>
> 
> 
> 


-- 
Robin Garner
Dept. of Computer Science
Australian National University
http://cs.anu.edu.au/people/Robin.Garner/

Re: [VM]How to trigue GC while free native memory is low.

[Leo Li <li...@gmail.com>]

So is it possible for our VM support an option by which our customer can
customize the policy while the VM still supply a default policy, for
example, assuming the native heap limit as that of the java heap?

On 2/9/07, Rana Dasgupta <rd...@gmail.com> wrote:
>
> On 2/8/07, Robin Garner <ro...@anu.edu.au> wrote:
> >
> > Tim Ellison wrote:
> > > Rana Dasgupta wrote:
> > >> On 2/5/07, Tim Ellison <t....@gmail.com> wrote:
> >
> > >>> I don't think that we need to expand the scope of the problem to do
> > >>> all types of resource accounting in the JVM or in an underlying
> > >>> platform layer. I don't see the spec providing any basic guidance in
> > >>> this area, it falls in the category of failsafe JVM's which is an
> > >>> interesting area, but IMHO beyond scope at this point. My suggestion
> > >>> is that we treat this as a specific nio api implementation problem
> > >>> for which we can think of adding some VM/GC support if it helps.
> > >>>
> > >> We may have to disagree here then, I think we should indeed be
> > >> considering a general solution for detecting early when these Java
> > >> 'handle' objects are collectable -- then we can apply it to direct
> > >> buffers or other handle types.
> >
> > >My point was that by adding a single GC -> VM interface function, all
> > >types of native resources could be addressed in a simple extensible
> way.
> > >The subsequent policy and mechanism decisions then become an ongoing
> > >design process for the VM and classlib native implementers, but once
> > >this interface function is added, GC can support freeing of native
> > >resources by whatever mechanisms the VM chooses to implement.
>
>
> I don't understand very well how the JVM can define policies on its
> platform/OS resource usage when it may be hosted by a server which wants
> to
> define such policies itself. In other words, say the JVM is running an
> RDBMS
> managed stored procedure. Should it be making assumptions about how much
> native heap to use etc. when the hosting RDBMS may have negotiated such
> policies itself and may be choosing to invoke one or more instances of the
> JVM?
>



-- 
Leo Li
China Software Development Lab, IBM

Re: [VM]How to trigue GC while free native memory is low.

[Robin Garner <ro...@anu.edu.au>]

Rana Dasgupta wrote:
> On 2/8/07, Robin Garner <ro...@anu.edu.au> wrote:
>>
>> >My point was that by adding a single GC -> VM interface function, all
>> >types of native resources could be addressed in a simple extensible way.
>> >The subsequent policy and mechanism decisions then become an ongoing
>> >design process for the VM and classlib native implementers, but once
>> >this interface function is added, GC can support freeing of native
>> >resources by whatever mechanisms the VM chooses to implement.
> 
> 
> I don't understand very well how the JVM can define policies on its
> platform/OS resource usage when it may be hosted by a server which wants to
> define such policies itself. In other words, say the JVM is running an 
> RDBMS
> managed stored procedure. Should it be making assumptions about how much
> native heap to use etc. when the hosting RDBMS may have negotiated such
> policies itself and may be choosing to invoke one or more instances of the
> JVM?

This is an interesting issue but I don't think it's relevant to the 
design of the interface.

In fact it's a perfect illustration of why the GC should defer policy 
decisions to an opaque VM API function, so that the VM is free to 
negotiate policy (or lack thereof) with its surrounding environment.

-- 
Robin Garner
Dept. of Computer Science
Australian National University
http://cs.anu.edu.au/people/Robin.Garner/

Re: [VM]How to trigue GC while free native memory is low.

[Rana Dasgupta <rd...@gmail.com>]

On 2/8/07, Robin Garner <ro...@anu.edu.au> wrote:
>
> Tim Ellison wrote:
> > Rana Dasgupta wrote:
> >> On 2/5/07, Tim Ellison <t....@gmail.com> wrote:
>
> >>> I don't think that we need to expand the scope of the problem to do
> >>> all types of resource accounting in the JVM or in an underlying
> >>> platform layer. I don't see the spec providing any basic guidance in
> >>> this area, it falls in the category of failsafe JVM's which is an
> >>> interesting area, but IMHO beyond scope at this point. My suggestion
> >>> is that we treat this as a specific nio api implementation problem
> >>> for which we can think of adding some VM/GC support if it helps.
> >>>
> >> We may have to disagree here then, I think we should indeed be
> >> considering a general solution for detecting early when these Java
> >> 'handle' objects are collectable -- then we can apply it to direct
> >> buffers or other handle types.
>
> >My point was that by adding a single GC -> VM interface function, all
> >types of native resources could be addressed in a simple extensible way.
> >The subsequent policy and mechanism decisions then become an ongoing
> >design process for the VM and classlib native implementers, but once
> >this interface function is added, GC can support freeing of native
> >resources by whatever mechanisms the VM chooses to implement.


I don't understand very well how the JVM can define policies on its
platform/OS resource usage when it may be hosted by a server which wants to
define such policies itself. In other words, say the JVM is running an RDBMS
managed stored procedure. Should it be making assumptions about how much
native heap to use etc. when the hosting RDBMS may have negotiated such
policies itself and may be choosing to invoke one or more instances of the
JVM?

Re: [VM]How to trigue GC while free native memory is low.

[Robin Garner <ro...@anu.edu.au>]

Tim Ellison wrote:
> Rana Dasgupta wrote:
>> On 2/5/07, Tim Ellison <t....@gmail.com> wrote:
>>>> I don't see the distinction.  There is still a 1:1 between the Java
>>>> object and the native resource.  If you run out of file handles then it
>>>> is potentially as bad as running out of native memory, the disparity in
>>>> size between the Java resource and the native resource is immaterial
>>>> isn't it?
>>
>>    There is  a distinction. One could directallocate only a few buffers and
>> exhaust the C heap ( if I understand the api correctly ) if the api
>> implementation is not eager about releasing back unused resources. One is
>> unlikely to hit a limit with a few outstanding fd's.
> 
> My point was that we could potentially run out of any OS resource being
> held onto by a Java object before running out of Java heap for those
> objects.  Maybe file handles was a poor example.

I think the file handle mention was my fault, but it came from a 
concrete example: Dacapo xalan in the 2006-10 release, which would fail 
for exactly this reason if the heap was large enough.

>> I don't think that we need to expand the scope of the problem to do 
>> all types of resource accounting in the JVM or in an underlying 
>> platform layer. I don't see the spec providing any basic guidance in 
>> this area, it falls in the category of failsafe JVM's which is an
>> interesting area, but IMHO beyond scope at this point. My suggestion
>> is that we treat this as a specific nio api implementation problem
>> for which we can think of adding some VM/GC support if it helps.
> 
> We may have to disagree here then, I think we should indeed be
> considering a general solution for detecting early when these Java
> 'handle' objects are collectable -- then we can apply it to direct
> buffers or other handle types.

My point was that by adding a single GC -> VM interface function, all 
types of native resources could be addressed in a simple extensible way. 
  The subsequent policy and mechanism decisions then become an ongoing 
design process for the VM and classlib native implementers, but once 
this interface function is added, GC can support freeing of native 
resources by whatever mechanisms the VM chooses to implement.

>> The extent of the support is based on how widely this fails in real
>> usage scenarios. After all the doc seems to say that the directbuffer
>> is best chosen for long lived large buffers only, those that will
>> need infrequent collection, implying weak GC dependance.
> 
> Yep, while we have a test case that demonstrates the difference between
> Harmony and other implementations, a failing application would be more
> compelling.
> 
>> One way to do this could be to treat these as a special class of Java
>> objects with all allocation on the Java heap, as Xiao Feng suggests, and
>> let GC decide on handling the resource exhaustion.
> 
> I suggest that would have it's own problems though.  The memory would
> have to be contiguous and pinned, and may be large.  We could end up
> with a fragmented heap quite quickly.

I strongly disagree with special-casing any of these resources.  To me, 
supporting pinned allocation for direct buffers is orthogonal to 
providing a way for the VM to request GC when certain native resources 
are at risk of depletion.

>> The second is for the api to ask the GC some specific questions about
>> unreachable object instances to eagerly decide on deallocation. That
>> requires a private interface.
> 
> right, which is an optimization of the suggestion of using the public
> System.gc() API

Surely a well-written Java application releases native resources either 
explicitly or through a reference type or finalizer ?  Yes, it's nice to 
tolerate badly written apps but is that the first-order concern here ?

>> The third is for the api implementation to do most of the
>> housekeeping ( hence my suggestion about max memory ) without relying
>> on platform hooks, and heuristically decide on when to invoke GC.
>> This will fail sometimes, but that may be OK. The solution can be
>> made more efficient ( than finalizers etc. ) by making the interface
>> private.
> 
> What heuristic would you suggest beyond the malloc failure?

% (native)heap usage, with some kind of 'damping' heuristic to prevent 
thrashing.

>> The last would be fore the api to add platform hooks to help with its
>> housekeeping. But I don't believe that this is possible for this
>> solution to be perfect( imagine several VM instances running on a
>> client machine ), and forces a dependency on a third platform
>> dependant piece, if it is to be kept independent of classlib and VM.
> 
> Which 'platform hooks' do you mean? (not sure if you meant they are in
> Harmony at the moment or are speaking hypothetically).
> 
> Regards,
> Tim


-- 
Robin Garner
Dept. of Computer Science
Australian National University
http://cs.anu.edu.au/people/Robin.Garner/

Re: [VM]How to trigue GC while free native memory is low.

[Tim Ellison <t....@gmail.com>]

Rana Dasgupta wrote:
> On 2/5/07, Tim Ellison <t....@gmail.com> wrote:
>> >I don't see the distinction.  There is still a 1:1 between the Java
>> >object and the native resource.  If you run out of file handles then it
>> >is potentially as bad as running out of native memory, the disparity in
>> >size between the Java resource and the native resource is immaterial
>> >isn't it?
> 
> 
>    There is  a distinction. One could directallocate only a few buffers and
> exhaust the C heap ( if I understand the api correctly ) if the api
> implementation is not eager about releasing back unused resources. One is
> unlikely to hit a limit with a few outstanding fd's.

My point was that we could potentially run out of any OS resource being
held onto by a Java object before running out of Java heap for those
objects.  Maybe file handles was a poor example.

> I don't think that we need to expand the scope of the problem to do 
> all types of resource accounting in the JVM or in an underlying 
> platform layer. I don't see the spec providing any basic guidance in 
> this area, it falls in the category of failsafe JVM's which is an
> interesting area, but IMHO beyond scope at this point. My suggestion
> is that we treat this as a specific nio api implementation problem
> for which we can think of adding some VM/GC support if it helps.

We may have to disagree here then, I think we should indeed be
considering a general solution for detecting early when these Java
'handle' objects are collectable -- then we can apply it to direct
buffers or other handle types.

> The extent of the support is based on how widely this fails in real
> usage scenarios. After all the doc seems to say that the directbuffer
> is best chosen for long lived large buffers only, those that will
> need infrequent collection, implying weak GC dependance.

Yep, while we have a test case that demonstrates the difference between
Harmony and other implementations, a failing application would be more
compelling.

> One way to do this could be to treat these as a special class of Java
> objects with all allocation on the Java heap, as Xiao Feng suggests, and
> let GC decide on handling the resource exhaustion.

I suggest that would have it's own problems though.  The memory would
have to be contiguous and pinned, and may be large.  We could end up
with a fragmented heap quite quickly.

> The second is for the api to ask the GC some specific questions about
> unreachable object instances to eagerly decide on deallocation. That
> requires a private interface.

right, which is an optimization of the suggestion of using the public
System.gc() API

> The third is for the api implementation to do most of the
> housekeeping ( hence my suggestion about max memory ) without relying
> on platform hooks, and heuristically decide on when to invoke GC.
> This will fail sometimes, but that may be OK. The solution can be
> made more efficient ( than finalizers etc. ) by making the interface
> private.

What heuristic would you suggest beyond the malloc failure?

> The last would be fore the api to add platform hooks to help with its
> housekeeping. But I don't believe that this is possible for this
> solution to be perfect( imagine several VM instances running on a
> client machine ), and forces a dependency on a third platform
> dependant piece, if it is to be kept independent of classlib and VM.

Which 'platform hooks' do you mean? (not sure if you meant they are in
Harmony at the moment or are speaking hypothetically).

Regards,
Tim

Re: [VM]How to trigue GC while free native memory is low.

[Ivan Volosyuk <iv...@gmail.com>]

On 2/5/07, Rana Dasgupta <rd...@gmail.com> wrote:
> On 2/5/07, Tim Ellison <t....@gmail.com> wrote:
> >
> > Rana Dasgupta wrote:
> > >> On 2/3/07, Xiao-Feng Li <xi...@gmail.com> wrote:
> > >>>
> > >>
> > >>> >Tim, the pinned memory is required by the API spec. We have to
> > support
> > >>> >it either in VM or API. (GC to support it doesn't necessarily mean to
> > >>> >have a pinned space in GC managed heap.) It's probably good to
> > >>> >distinct two different situations:
> > >>>
> > >>> >1. a native resource is associated with a Java object, and the
> > >>> >resource can only be released when the object is strongly
> > unreachable.
> > >>> >In this case, the programmer who uses this class knows that a GC
> > cycle
> > >>> >can help to release the unused resource. Then in his code, he
> > probably
> > >>> >can directly invoke some VM interface to force a collection when the
> > >>> >resource is run out. We need certain VM support that serve the
> > >>> >purpose. Or the programmer just relies on the VM to provide sort of
> > >>> >automatic resource management.
> > >>>
> > >>> >2. a native resource is associated with a Java object, but managed by
> > >>> >the application; or the native resource is not associated with a Java
> > >>> >object at all (e.g., completely managed in native libraries). In this
> > >>> >case, the whole resource management burden is taken by the
> > application
> > >>> >itself. There is no requirement to VM.
> > >>
> > >>
> > >> I agree. My contention is that management of most unmanaged resources
> > come
> > >> under this category. C# for example, recommends a Dispose() design
> > pattern
> > >> to release resources and the language ties it back to the finalizer. I
> > >> would suggest that we focus on solving the unmanaged memory release
> > problem
> > >> which to me looks more critical because unlike threads, fd's etc, where
> > >> mappings are usually 1x1 between the unmanaged resource and the
> > >> corresponding java object, in this case, the api seems to allow tying
> > very
> > >> large chunks of memory to the bytebuffer without an explicit release
> > api. I
> > >> don't think that the JVM( except for specialized ones ) has the
> > >> responsibility to automatically account for and manage all types of
> > >> resources, but it is obliged to do automatic memory management as Xiao
> > Feng
> > >> point out below.
> >
> > >I don't see the distinction.  There is still a 1:1 between the Java
> > >object and the native resource.  If you run out of file handles then it
> > >is potentially as bad as running out of native memory, the disparity in
> > >size between the Java resource and the native resource is immaterial
> > >isn't it?
>
>
>     There is  a distinction. One could directallocate only a few buffers and
> exhaust the C heap ( if I understand the api correctly ) if the api
> implementation is not eager about releasing back unused resources. One is
> unlikely to hit a limit with a few outstanding fd's. I don't think that we
> need to expand the scope of the problem to do all types of resource
> accounting in the JVM or in an underlying platform layer. I don't see the
> spec providing any basic guidance in this area, it falls in the category of
> failsafe JVM's which is an interesting area, but IMHO beyond scope at this
> point. My suggestion is that we treat this as a specific nio api
> implementation problem for which we can think of adding some VM/GC support
> if it helps. The extent of  the support is based on how widely this fails in
> real usage scenarios. After all the doc seems to say that the directbuffer
> is best chosen for long lived large buffers only, those that will need
> infrequent collection, implying weak GC dependance.
>     One way to do this could be to treat these as a special class of Java
> objects with all allocation on the Java heap, as Xiao Feng suggests, and let
> GC decide on handling the resource exhaustion. The second is for the api to
> ask the GC some specific questions about unreachable object instances to
> eagerly decide on deallocation. That requires a private interface. The third
> is for the api implementation to do most of the housekeeping ( hence my
> suggestion about max memory ) without relying on platform hooks, and
> heuristically decide on when to invoke GC. This will fail sometimes, but
> that may be OK. The solution can be made more efficient ( than finalizers
> etc. ) by making the interface private. The last would be fore the api to
> add platform hooks to help with its housekeeping. But I don't believe that
> this is possible for this solution to be perfect( imagine several VM
> instances running on a client machine ), and forces a dependency on a third
> platform dependant piece, if it is to be kept independent of classlib and
> VM.
>
>
>
> > >>> The case for direct ByteBuffer belongs to 1. I am afraid certain VM
> > >>> >support is necessary. The question is what and how.
> > >>
> > >>
> > >> I am not clear on what's the best approach. It seems that the most
> > general
> > >> automatic solution would need to be based on reference counting, but
> > >> that is not a small task.
> >
> > >Reference counting of the Java object I presume?  How would that work?\
>
>
> I don't know of a good automatic way to do ref counting of the java object
> without generating additional code by the jit when references are being
> copied, etc. It is therefore not trivial to get right and has performance
> impacts. It is a generalized way for the GC to answer questions about
> instance lifetimes discussed above. Anyway, we should probably treat this as
> a GC implementation issue.

Reference counting  is not only a JIT and GC changes. This also
implies changes of entire VM which will possibly hit the performance.
I see this as a quite complicated task.

>
> >> I don't think that external monitoring threads etc. are
> > >> very neat. There is no way to get them perfectly right in spite of the
> > non
> > >> trivial development needed to build them.  Xiao Feng suggested a
> > >> feeDirect()
> > >> method which could be a good way, but one of the api people would need
> > to
> > >> confirm if this addition is feasible.
> >
> > >We already have a Harmony-specific way to free the memory, which we can
> > >use in implementing the class libraries themselves, but that does not
> > >help our users who will be writing portable code.
>
>
> This interface is not user visible. But I understand that it has some impact
> on coupling the api implementation with VM's that don't support the private
> interfaces. It makes those implementations somewhat less efficient.
>
> >>>  Is it feasible to add an option to the VM specifying a maxDirectMemory,
> > >>> and for the api implemententaion to keep an account of how much has
> > been
> > >>> allocated and freed ? And on being requested for a new allocation, if
> > it
> > >>> thinks necessary, invoke on a gc private interface to force cleanup?
> >
> > >That is one of the suggestions so far.  In the absence of a private
> > >interface it would have to be System.gc() to detect that the Java object
> > >is collectable, and run finalization/put the phantom ref on the queue --
> > >if we had a private interface we could ask the VM if a specific object
> > >is reachable (not sure how easy it is to answer that question in
> > general).
>
>
>
> Regards,
> > Tim
> >
> >
>
>


-- 
Ivan
Intel Enterprise Solutions Software Division

Re: [VM]How to trigue GC while free native memory is low.

[Rana Dasgupta <rd...@gmail.com>]

On 2/5/07, Tim Ellison <t....@gmail.com> wrote:
>
> Rana Dasgupta wrote:
> >> On 2/3/07, Xiao-Feng Li <xi...@gmail.com> wrote:
> >>>
> >>
> >>> >Tim, the pinned memory is required by the API spec. We have to
> support
> >>> >it either in VM or API. (GC to support it doesn't necessarily mean to
> >>> >have a pinned space in GC managed heap.) It's probably good to
> >>> >distinct two different situations:
> >>>
> >>> >1. a native resource is associated with a Java object, and the
> >>> >resource can only be released when the object is strongly
> unreachable.
> >>> >In this case, the programmer who uses this class knows that a GC
> cycle
> >>> >can help to release the unused resource. Then in his code, he
> probably
> >>> >can directly invoke some VM interface to force a collection when the
> >>> >resource is run out. We need certain VM support that serve the
> >>> >purpose. Or the programmer just relies on the VM to provide sort of
> >>> >automatic resource management.
> >>>
> >>> >2. a native resource is associated with a Java object, but managed by
> >>> >the application; or the native resource is not associated with a Java
> >>> >object at all (e.g., completely managed in native libraries). In this
> >>> >case, the whole resource management burden is taken by the
> application
> >>> >itself. There is no requirement to VM.
> >>
> >>
> >> I agree. My contention is that management of most unmanaged resources
> come
> >> under this category. C# for example, recommends a Dispose() design
> pattern
> >> to release resources and the language ties it back to the finalizer. I
> >> would suggest that we focus on solving the unmanaged memory release
> problem
> >> which to me looks more critical because unlike threads, fd's etc, where
> >> mappings are usually 1x1 between the unmanaged resource and the
> >> corresponding java object, in this case, the api seems to allow tying
> very
> >> large chunks of memory to the bytebuffer without an explicit release
> api. I
> >> don't think that the JVM( except for specialized ones ) has the
> >> responsibility to automatically account for and manage all types of
> >> resources, but it is obliged to do automatic memory management as Xiao
> Feng
> >> point out below.
>
> >I don't see the distinction.  There is still a 1:1 between the Java
> >object and the native resource.  If you run out of file handles then it
> >is potentially as bad as running out of native memory, the disparity in
> >size between the Java resource and the native resource is immaterial
> >isn't it?


    There is  a distinction. One could directallocate only a few buffers and
exhaust the C heap ( if I understand the api correctly ) if the api
implementation is not eager about releasing back unused resources. One is
unlikely to hit a limit with a few outstanding fd's. I don't think that we
need to expand the scope of the problem to do all types of resource
accounting in the JVM or in an underlying platform layer. I don't see the
spec providing any basic guidance in this area, it falls in the category of
failsafe JVM's which is an interesting area, but IMHO beyond scope at this
point. My suggestion is that we treat this as a specific nio api
implementation problem for which we can think of adding some VM/GC support
if it helps. The extent of  the support is based on how widely this fails in
real usage scenarios. After all the doc seems to say that the directbuffer
is best chosen for long lived large buffers only, those that will need
infrequent collection, implying weak GC dependance.
    One way to do this could be to treat these as a special class of Java
objects with all allocation on the Java heap, as Xiao Feng suggests, and let
GC decide on handling the resource exhaustion. The second is for the api to
ask the GC some specific questions about unreachable object instances to
eagerly decide on deallocation. That requires a private interface. The third
is for the api implementation to do most of the housekeeping ( hence my
suggestion about max memory ) without relying on platform hooks, and
heuristically decide on when to invoke GC. This will fail sometimes, but
that may be OK. The solution can be made more efficient ( than finalizers
etc. ) by making the interface private. The last would be fore the api to
add platform hooks to help with its housekeeping. But I don't believe that
this is possible for this solution to be perfect( imagine several VM
instances running on a client machine ), and forces a dependency on a third
platform dependant piece, if it is to be kept independent of classlib and
VM.



> >>> The case for direct ByteBuffer belongs to 1. I am afraid certain VM
> >>> >support is necessary. The question is what and how.
> >>
> >>
> >> I am not clear on what's the best approach. It seems that the most
> general
> >> automatic solution would need to be based on reference counting, but
> >> that is not a small task.
>
> >Reference counting of the Java object I presume?  How would that work?\


I don't know of a good automatic way to do ref counting of the java object
without generating additional code by the jit when references are being
copied, etc. It is therefore not trivial to get right and has performance
impacts. It is a generalized way for the GC to answer questions about
instance lifetimes discussed above. Anyway, we should probably treat this as
a GC implementation issue.

>> I don't think that external monitoring threads etc. are
> >> very neat. There is no way to get them perfectly right in spite of the
> non
> >> trivial development needed to build them.  Xiao Feng suggested a
> >> feeDirect()
> >> method which could be a good way, but one of the api people would need
> to
> >> confirm if this addition is feasible.
>
> >We already have a Harmony-specific way to free the memory, which we can
> >use in implementing the class libraries themselves, but that does not
> >help our users who will be writing portable code.


This interface is not user visible. But I understand that it has some impact
on coupling the api implementation with VM's that don't support the private
interfaces. It makes those implementations somewhat less efficient.

>>>  Is it feasible to add an option to the VM specifying a maxDirectMemory,
> >>> and for the api implemententaion to keep an account of how much has
> been
> >>> allocated and freed ? And on being requested for a new allocation, if
> it
> >>> thinks necessary, invoke on a gc private interface to force cleanup?
>
> >That is one of the suggestions so far.  In the absence of a private
> >interface it would have to be System.gc() to detect that the Java object
> >is collectable, and run finalization/put the phantom ref on the queue --
> >if we had a private interface we could ask the VM if a specific object
> >is reachable (not sure how easy it is to answer that question in
> general).



Regards,
> Tim
>
>

Re: [VM]How to trigue GC while free native memory is low.

[Tim Ellison <t....@gmail.com>]

Rana Dasgupta wrote:
> On 2/3/07, Xiao-Feng Li <xi...@gmail.com> wrote:
>>
>>
>> >Tim, the pinned memory is required by the API spec. We have to support
>> >it either in VM or API. (GC to support it doesn't necessarily mean to
>> >have a pinned space in GC managed heap.) It's probably good to
>> >distinct two different situations:
>>
>> >1. a native resource is associated with a Java object, and the
>> >resource can only be released when the object is strongly unreachable.
>> >In this case, the programmer who uses this class knows that a GC cycle
>> >can help to release the unused resource. Then in his code, he probably
>> >can directly invoke some VM interface to force a collection when the
>> >resource is run out. We need certain VM support that serve the
>> >purpose. Or the programmer just relies on the VM to provide sort of
>> >automatic resource management.
>>
>> >2. a native resource is associated with a Java object, but managed by
>> >the application; or the native resource is not associated with a Java
>> >object at all (e.g., completely managed in native libraries). In this
>> >case, the whole resource management burden is taken by the application
>> >itself. There is no requirement to VM.
> 
> 
> I agree. My contention is that management of most unmanaged resources come
> under this category. C# for example, recommends a Dispose() design pattern
> to release resources and the language ties it back to the finalizer. I
> would suggest that we focus on solving the unmanaged memory release problem
> which to me looks more critical because unlike threads, fd's etc, where
> mappings are usually 1x1 between the unmanaged resource and the
> corresponding java object, in this case, the api seems to allow tying very
> large chunks of memory to the bytebuffer without an explicit release api. I
> don't think that the JVM( except for specialized ones ) has the
> responsibility to automatically account for and manage all types of
> resources, but it is obliged to do automatic memory management as Xiao Feng
> point out below.

I don't see the distinction.  There is still a 1:1 between the Java
object and the native resource.  If you run out of file handles then it
is potentially as bad as running out of native memory, the disparity in
size between the Java resource and the native resource is immaterial
isn't it?

>> The case for direct ByteBuffer belongs to 1. I am afraid certain VM
>> >support is necessary. The question is what and how.
> 
> 
> I am not clear on what's the best approach. It seems that the most general
> automatic solution would need to be based on reference counting, but
> that is not a small task.

Reference counting of the Java object I presume?  How would that work?

> I don't think that external monitoring threads etc. are
> very neat. There is no way to get them perfectly right in spite of the non
> trivial development needed to build them.  Xiao Feng suggested a
> feeDirect()
> method which could be a good way, but one of the api people would need to
> confirm if this addition is feasible.

We already have a Harmony-specific way to free the memory, which we can
use in implementing the class libraries themselves, but that does not
help our users who will be writing portable code.

>  Is it feasible to add an option to the VM specifying a maxDirectMemory,
> and for the api implemententaion to keep an account of how much has been
> allocated and freed ? And on being requested for a new allocation, if it
> thinks necessary, invoke on a gc private interface to force cleanup?

That is one of the suggestions so far.  In the absence of a private
interface it would have to be System.gc() to detect that the Java object
is collectable, and run finalization/put the phantom ref on the queue --
if we had a private interface we could ask the VM if a specific object
is reachable (not sure how easy it is to answer that question in general).

Regards,
Tim

Re: [VM]How to trigue GC while free native memory is low.

[Rana Dasgupta <rd...@gmail.com>]

On 2/3/07, Xiao-Feng Li <xi...@gmail.com> wrote:
>
>
> >Tim, the pinned memory is required by the API spec. We have to support
> >it either in VM or API. (GC to support it doesn't necessarily mean to
> >have a pinned space in GC managed heap.) It's probably good to
> >distinct two different situations:
>
> >1. a native resource is associated with a Java object, and the
> >resource can only be released when the object is strongly unreachable.
> >In this case, the programmer who uses this class knows that a GC cycle
> >can help to release the unused resource. Then in his code, he probably
> >can directly invoke some VM interface to force a collection when the
> >resource is run out. We need certain VM support that serve the
> >purpose. Or the programmer just relies on the VM to provide sort of
> >automatic resource management.
>
> >2. a native resource is associated with a Java object, but managed by
> >the application; or the native resource is not associated with a Java
> >object at all (e.g., completely managed in native libraries). In this
> >case, the whole resource management burden is taken by the application
> >itself. There is no requirement to VM.


I agree. My contention is that management of most unmanaged resources come
under this category. C# for example, recommends a Dispose() design pattern
to release resources and the language ties it back to the finalizer. I
would suggest that we focus on solving the unmanaged memory release problem
which to me looks more critical because unlike threads, fd's etc, where
mappings are usually 1x1 between the unmanaged resource and the
corresponding java object, in this case, the api seems to allow tying very
large chunks of memory to the bytebuffer without an explicit release api. I
don't think that the JVM( except for specialized ones ) has the
responsibility to automatically account for and manage all types of
resources, but it is obliged to do automatic memory management as Xiao Feng
point out below.

>The case for direct ByteBuffer belongs to 1. I am afraid certain VM
> >support is necessary. The question is what and how.


I am not clear on what's the best approach. It seems that the most general
automatic solution would need to be based on reference counting, but that is
not a small task. I don't think that external monitoring threads etc. are
very neat. There is no way to get them perfectly right in spite of the non
trivial development needed to build them.  Xiao Feng suggested a feeDirect()
method which could be a good way, but one of the api people would need to
confirm if this addition is feasible.
  Is it feasible to add an option to the VM specifying a maxDirectMemory,
and for the api implemententaion to keep an account of how much has been
allocated and freed ? And on being requested for a new allocation, if it
thinks necessary, invoke on a gc private interface to force cleanup?

Thanks,
> xiaofeng
>
> > Regards,
> > Tim
> >
> >
>

Re: [VM]How to trigue GC while free native memory is low.

[Xiao-Feng Li <xi...@gmail.com>]

On 2/4/07, Tim Ellison <t....@gmail.com> wrote:
> Xiao-Feng Li wrote:
> > Since what the ByeBuffer needs are starting address and capacity, it
> > doesn't really care if this piece of memory is in Java heap or in
> > native runtime memory ( I assume.) We probably can provide some
> > special kind of virtual Java object that serves as raw memory block to
> > nio. In this works, we need not monitor the native runtime memory
> > usage.
> >
> > This approach need certain contract between Java classes and GC about
> > the special kind of Java object. Probably we can write a layer of Java
> > class wrapper for raw memory allocation, which hides the contract from
> > other common classes.
>
> There is a requirement that the 'raw memory block' is pinned though,
> because addresses in that block may be passed into OS calls (and the
> point of a direct byte buffer is that the memory is not copied). I don't
> believe that is going to be a reasonable requirement for all VM
> implementations.

Tim, the pinned memory is required by the API spec. We have to support
it either in VM or API. (GC to support it doesn't necessarily mean to
have a pinned space in GC managed heap.) It's probably good to
distinct two different situations:

1. a native resource is associated with a Java object, and the
resource can only be released when the object is strongly unreachable.
In this case, the programmer who uses this class knows that a GC cycle
can help to release the unused resource. Then in his code, he probably
can directly invoke some VM interface to force a collection when the
resource is run out. We need certain VM support that serve the
purpose. Or the programmer just relies on the VM to provide sort of
automatic resource management.

2. a native resource is associated with a Java object, but managed by
the application; or the native resource is not associated with a Java
object at all (e.g., completely managed in native libraries). In this
case, the whole resource management burden is taken by the application
itself. There is no requirement to VM.

The case for direct ByteBuffer belongs to 1. I am afraid certain VM
support is necessary. The question is what and how.

Thanks,
xiaofeng

> Regards,
> Tim
>
>

Re: [VM]How to trigue GC while free native memory is low.

[Tim Ellison <t....@gmail.com>]

Xiao-Feng Li wrote:
> Since what the ByeBuffer needs are starting address and capacity, it
> doesn't really care if this piece of memory is in Java heap or in
> native runtime memory ( I assume.) We probably can provide some
> special kind of virtual Java object that serves as raw memory block to
> nio. In this works, we need not monitor the native runtime memory
> usage.
> 
> This approach need certain contract between Java classes and GC about
> the special kind of Java object. Probably we can write a layer of Java
> class wrapper for raw memory allocation, which hides the contract from
> other common classes.

There is a requirement that the 'raw memory block' is pinned though,
because addresses in that block may be passed into OS calls (and the
point of a direct byte buffer is that the memory is not copied). I don't
believe that is going to be a reasonable requirement for all VM
implementations.

Regards,
Tim

Re: [VM]How to trigue GC while free native memory is low.

[Xiao-Feng Li <xi...@gmail.com>]

Thank you, Jimmy and Leo, for the clear explanations.

I think temporarily you can put the System.gc() there to workaround
this issue. Yes, this is only a workaround since we cannot rely on it
without explicit guarantee in its semantics.

To really solve this problem, we might need a careful design. Your
proposed approach looks like a possible solution. Have you thought to
solve the problem by using GC to manage the raw memory blocks? Without
deep understanding of the problem, I have some wild idea below. Please
see if it is reasonable or stupid. I will read the spec of ByteBuffer
to get more understanding.

Since what the ByeBuffer needs are starting address and capacity, it
doesn't really care if this piece of memory is in Java heap or in
native runtime memory ( I assume.) We probably can provide some
special kind of virtual Java object that serves as raw memory block to
nio. In this works, we need not monitor the native runtime memory
usage.

This approach need certain contract between Java classes and GC about
the special kind of Java object. Probably we can write a layer of Java
class wrapper for raw memory allocation, which hides the contract from
other common classes.

At the same time, I am thinking of your proposed approach, and will
get back later.

Thanks,
xiaofeng

On 2/2/07, Leo Li <li...@gmail.com> wrote:
> Hi, Xiao-Feng:
>      Excuse me if I am confusing you.
>      The direct byte buffer holds a block of native memory space used in nio
> operation. The native byte buffer will be freed through the byte buffer is
> gc collected and added to ReferenceQueue monitored by MemorySpy series
> classes.
>      But the gc will not be triggered if the java heap is still empty. Here
> is an example: I allocate a large block of native memory for a byte buffer
> while the byte buffer itself is quite small. Thus quickly native heap is
> depleted while VM still does not think gc is needed since java heap has a
> lot of free memory.
>
>
> import java.nio.*;
> public class Test {
>     public static void main(String[] args) throws Exception {
>         for(int i = 0;i<1000;i++)
>         {
>             ByteBuffer byteBuffer = ByteBuffer.allocateDirect(10240000);
>             //System.gc();
>         }
>    }
> }
> RI runs well but Harmony will soon throw OutOfMemory exception.
> But if remove the comment before System.gc(), Harmony will become ok.
> So my point is that we need a mechanism to notify VM to start gc if we have
> no more native memory because sometimes problem will be solved if gc is
> fired. Of cause we cannot avoid a  user always malloc space but never free
> them.:)
>
> Good luck!
>
> On 2/2/07, Xiao-Feng Li <xi...@gmail.com> wrote:
> >
> > Leo, I can't fully understand your problem and proposal. I am working
> > in DRLVM GC component, I want to understand how GC can tackle the
> > issue you meet.
> >
> > You said the native memory is allocated (probably with malloc) for
> > byte buffer. I wonder how the process goes. I assume it's in native
> > code. Then you use malloc to allocate a block of memory, and put some
> > Java object (byte buffer) into it? Is this what you do? I wonder how
> > do you manipulate Java object placement in native code; and if the
> > native memory block is out of Java heap, do you expect GC to manage
> > it?
> >
> > You said when GC reclaims those byte buffer object, the native memory
> > block can be freed in native code. How do you know if an object is
> > reclaimed by GC or not?
> >
> > Thanks,
> > xiaofeng
> >
> > On 2/2/07, Leo Li < liyilei1979@gmail.com> wrote:
> > > Hi, all:
> > >     After applying patch3073, I added the support for direct byte buffer
> > in
> > > nio module as spec requires. But now here exists a problem: The native
> > > memory allocated to the byte buffer can be released when gc releases the
> > > byte buffer object, however, if the native heap is full while java heap
> > > still has space, gc will not be triggered.
> > >     It seems that RI will start gc before native memory heap is depleted
> > and
> > > thus prevents out-of-memory error.
> > >
> > >    Then our work focuses on:
> > >    1. When gc is required.
> > >    2. Trigger a gc.
> > >    The first one requires that we get support from operating system,
> > since
> > > the memory allocated in the native code, for example by malloc, is out
> > of
> > > the control of java VM.( I have ever thought of counting the used memory
> > in
> > > hymemory_allocate, but the plan fails since hymemory_free will not
> > report
> > > how much space is released.)
> > >    The second one needs the help from VM. System.gc() is not so reliable
> > as
> > > spec says, so it is necessary to have a internal channel to notify VM to
> > > start gc.
> > >
> > >     One solution, I think, is to let a monitor thread in VM to check
> > whether
> > > OS physical memory is low. For example, the
> > QueryMemoryResourceNotification
> > > of win32 API is a candidate. Although the interrupt model is more
> > effective:
> > > win32 SDK provides a CreateMemoryResourceNotification to get a handler
> > on
> > > which the monitor thread can wait, maybe on other platforms, OS does not
> > > supply such a convenience. So the monitor thread in the VM might have to
> >
> > > check the OS resource once for a while and if necessary the monitor
> > thread
> > > will call a GC.
> > >
> > >    My suggestion is first to add some function to monitor memory in
> > portlib,
> > > since it is highly related to platforms and in portlib there has been
> > > some useful tools.(Thanks Mark to point out that.) On the other hand, we
> > can
> > > negotiate a channel to trigger gc in VM. Actually I am not an expert on
> > VM
> > > since I am not sure whether there has been some monitor thread and the
> > load
> > > on performance if such a monitor thread is added to the VM...
> > >
> > >
> > > --
> > > Leo Li
> > > China Software Development Lab, IBM
> > >
> > >
> >
>
>
>
> --
> Leo Li
> China Software Development Lab, IBM
>
>

Re: [VM]How to trigue GC while free native memory is low.

[Leo Li <li...@gmail.com>]

Hi, Xiao-Feng:
     Excuse me if I am confusing you.
     The direct byte buffer holds a block of native memory space used in nio
operation. The native byte buffer will be freed through the byte buffer is
gc collected and added to ReferenceQueue monitored by MemorySpy series
classes.
     But the gc will not be triggered if the java heap is still empty. Here
is an example: I allocate a large block of native memory for a byte buffer
while the byte buffer itself is quite small. Thus quickly native heap is
depleted while VM still does not think gc is needed since java heap has a
lot of free memory.


import java.nio.*;
public class Test {
    public static void main(String[] args) throws Exception {
        for(int i = 0;i<1000;i++)
        {
            ByteBuffer byteBuffer = ByteBuffer.allocateDirect(10240000);
            //System.gc();
        }
   }
}
RI runs well but Harmony will soon throw OutOfMemory exception.
But if remove the comment before System.gc(), Harmony will become ok.
So my point is that we need a mechanism to notify VM to start gc if we have
no more native memory because sometimes problem will be solved if gc is
fired. Of cause we cannot avoid a  user always malloc space but never free
them.:)

Good luck!

On 2/2/07, Xiao-Feng Li <xi...@gmail.com> wrote:
>
> Leo, I can't fully understand your problem and proposal. I am working
> in DRLVM GC component, I want to understand how GC can tackle the
> issue you meet.
>
> You said the native memory is allocated (probably with malloc) for
> byte buffer. I wonder how the process goes. I assume it's in native
> code. Then you use malloc to allocate a block of memory, and put some
> Java object (byte buffer) into it? Is this what you do? I wonder how
> do you manipulate Java object placement in native code; and if the
> native memory block is out of Java heap, do you expect GC to manage
> it?
>
> You said when GC reclaims those byte buffer object, the native memory
> block can be freed in native code. How do you know if an object is
> reclaimed by GC or not?
>
> Thanks,
> xiaofeng
>
> On 2/2/07, Leo Li < liyilei1979@gmail.com> wrote:
> > Hi, all:
> >     After applying patch3073, I added the support for direct byte buffer
> in
> > nio module as spec requires. But now here exists a problem: The native
> > memory allocated to the byte buffer can be released when gc releases the
> > byte buffer object, however, if the native heap is full while java heap
> > still has space, gc will not be triggered.
> >     It seems that RI will start gc before native memory heap is depleted
> and
> > thus prevents out-of-memory error.
> >
> >    Then our work focuses on:
> >    1. When gc is required.
> >    2. Trigger a gc.
> >    The first one requires that we get support from operating system,
> since
> > the memory allocated in the native code, for example by malloc, is out
> of
> > the control of java VM.( I have ever thought of counting the used memory
> in
> > hymemory_allocate, but the plan fails since hymemory_free will not
> report
> > how much space is released.)
> >    The second one needs the help from VM. System.gc() is not so reliable
> as
> > spec says, so it is necessary to have a internal channel to notify VM to
> > start gc.
> >
> >     One solution, I think, is to let a monitor thread in VM to check
> whether
> > OS physical memory is low. For example, the
> QueryMemoryResourceNotification
> > of win32 API is a candidate. Although the interrupt model is more
> effective:
> > win32 SDK provides a CreateMemoryResourceNotification to get a handler
> on
> > which the monitor thread can wait, maybe on other platforms, OS does not
> > supply such a convenience. So the monitor thread in the VM might have to
>
> > check the OS resource once for a while and if necessary the monitor
> thread
> > will call a GC.
> >
> >    My suggestion is first to add some function to monitor memory in
> portlib,
> > since it is highly related to platforms and in portlib there has been
> > some useful tools.(Thanks Mark to point out that.) On the other hand, we
> can
> > negotiate a channel to trigger gc in VM. Actually I am not an expert on
> VM
> > since I am not sure whether there has been some monitor thread and the
> load
> > on performance if such a monitor thread is added to the VM...
> >
> >
> > --
> > Leo Li
> > China Software Development Lab, IBM
> >
> >
>



-- 
Leo Li
China Software Development Lab, IBM

Re: [VM]How to trigue GC while free native memory is low.

[LvJimmy，Jing <fi...@gmail.com>]

2007/2/2, Xiao-Feng Li <xi...@gmail.com>:
> Leo, I can't fully understand your problem and proposal. I am working
> in DRLVM GC component, I want to understand how GC can tackle the
> issue you meet.
>
> You said the native memory is allocated (probably with malloc) for
> byte buffer. I wonder how the process goes. I assume it's in native
> code. Then you use malloc to allocate a block of memory, and put some
> Java object (byte buffer) into it? Is this what you do? I wonder how
> do you manipulate Java object placement in native code; and if the
> native memory block is out of Java heap, do you expect GC to manage
> it?
>
> You said when GC reclaims those byte buffer object, the native memory
> block can be freed in native code. How do you know if an object is
> reclaimed by GC or not?
>

Hi,
     What Leo is talking about is DirectByteBuffer, which is added in
Java 5. This class is very strange compared to early classes. In java5
,we can allocate "direct" java.nio.ByteBuffer referring to the block
of memory starting at the memory address address and extending
capacity bytes. This block of memory is not in java heap so that it
can not be control by GC. However, we have a ByteBuffer object in java
heap(referring to that block of memory), if we gc this ByteBuffer
object, the VM also free that block of memory(This prevents memory
leaking).
      But if we have a lot of un-used ByteBuffer(e.g, 100) in java
heap, they may refer to much native memory(e,g, each of them own a
memory block of 10m),  so we must have 1G native memory in use, but at
that time, java-heap may be not full(only 100+ objects in heap), the
GC will not invoke, but real native memory will be full, so next time
we try to new a DirectByteBuffer will cause a memory-out error, which
can be avoid if we ask GC to invoke and clear these 100 unused
ByteBuffers and VM will free 1G memory.
      So to conclude this question, we should invoke GC when we find
there's too much native memory in use(not how GC can handle this
problem), so
1) how to know there's too much?
2) how to invoke GC when we need to?
And my question is 3) where is suitable to put this part of codes?

Leo may offer some simple sample for this?

> Thanks,
> xiaofeng
>
> On 2/2/07, Leo Li <li...@gmail.com> wrote:
> > Hi, all:
> >     After applying patch3073, I added the support for direct byte buffer in
> > nio module as spec requires. But now here exists a problem: The native
> > memory allocated to the byte buffer can be released when gc releases the
> > byte buffer object, however, if the native heap is full while java heap
> > still has space, gc will not be triggered.
> >     It seems that RI will start gc before native memory heap is depleted and
> > thus prevents out-of-memory error.
> >
> >    Then our work focuses on:
> >    1. When gc is required.
> >    2. Trigger a gc.
> >    The first one requires that we get support from operating system, since
> > the memory allocated in the native code, for example by malloc, is out of
> > the control of java VM.( I have ever thought of counting the used memory in
> > hymemory_allocate, but the plan fails since hymemory_free will not report
> > how much space is released.)
> >    The second one needs the help from VM. System.gc() is not so reliable as
> > spec says, so it is necessary to have a internal channel to notify VM to
> > start gc.
> >
> >     One solution, I think, is to let a monitor thread in VM to check whether
> > OS physical memory is low. For example, the QueryMemoryResourceNotification
> > of win32 API is a candidate. Although the interrupt model is more effective:
> > win32 SDK provides a CreateMemoryResourceNotification to get a handler on
> > which the monitor thread can wait, maybe on other platforms, OS does not
> > supply such a convenience. So the monitor thread in the VM might have to
> > check the OS resource once for a while and if necessary the monitor thread
> > will call a GC.
> >
> >    My suggestion is first to add some function to monitor memory in portlib,
> > since it is highly related to platforms and in portlib there has been
> > some useful tools.(Thanks Mark to point out that.) On the other hand, we can
> > negotiate a channel to trigger gc in VM. Actually I am not an expert on VM
> > since I am not sure whether there has been some monitor thread and the load
> > on performance if such a monitor thread is added to the VM...
> >
> >
> > --
> > Leo Li
> > China Software Development Lab, IBM
> >
> >
>


-- 

Best Regards!

Jimmy, Jing Lv
China Software Development Lab, IBM

Re: [VM]How to trigue GC while free native memory is low.

[Xiao-Feng Li <xi...@gmail.com>]

Leo, I can't fully understand your problem and proposal. I am working
in DRLVM GC component, I want to understand how GC can tackle the
issue you meet.

You said the native memory is allocated (probably with malloc) for
byte buffer. I wonder how the process goes. I assume it's in native
code. Then you use malloc to allocate a block of memory, and put some
Java object (byte buffer) into it? Is this what you do? I wonder how
do you manipulate Java object placement in native code; and if the
native memory block is out of Java heap, do you expect GC to manage
it?

You said when GC reclaims those byte buffer object, the native memory
block can be freed in native code. How do you know if an object is
reclaimed by GC or not?

Thanks,
xiaofeng

On 2/2/07, Leo Li <li...@gmail.com> wrote:
> Hi, all:
>     After applying patch3073, I added the support for direct byte buffer in
> nio module as spec requires. But now here exists a problem: The native
> memory allocated to the byte buffer can be released when gc releases the
> byte buffer object, however, if the native heap is full while java heap
> still has space, gc will not be triggered.
>     It seems that RI will start gc before native memory heap is depleted and
> thus prevents out-of-memory error.
>
>    Then our work focuses on:
>    1. When gc is required.
>    2. Trigger a gc.
>    The first one requires that we get support from operating system, since
> the memory allocated in the native code, for example by malloc, is out of
> the control of java VM.( I have ever thought of counting the used memory in
> hymemory_allocate, but the plan fails since hymemory_free will not report
> how much space is released.)
>    The second one needs the help from VM. System.gc() is not so reliable as
> spec says, so it is necessary to have a internal channel to notify VM to
> start gc.
>
>     One solution, I think, is to let a monitor thread in VM to check whether
> OS physical memory is low. For example, the QueryMemoryResourceNotification
> of win32 API is a candidate. Although the interrupt model is more effective:
> win32 SDK provides a CreateMemoryResourceNotification to get a handler on
> which the monitor thread can wait, maybe on other platforms, OS does not
> supply such a convenience. So the monitor thread in the VM might have to
> check the OS resource once for a while and if necessary the monitor thread
> will call a GC.
>
>    My suggestion is first to add some function to monitor memory in portlib,
> since it is highly related to platforms and in portlib there has been
> some useful tools.(Thanks Mark to point out that.) On the other hand, we can
> negotiate a channel to trigger gc in VM. Actually I am not an expert on VM
> since I am not sure whether there has been some monitor thread and the load
> on performance if such a monitor thread is added to the VM...
>
>
> --
> Leo Li
> China Software Development Lab, IBM
>
>

Re: [VM]How to trigue GC while free native memory is low.

[LvJimmy，Jing <fi...@gmail.com>]

2007/2/2, Leo Li <li...@gmail.com>:
> Hi, all:
>     After applying patch3073, I added the support for direct byte buffer in
> nio module as spec requires. But now here exists a problem: The native
> memory allocated to the byte buffer can be released when gc releases the
> byte buffer object, however, if the native heap is full while java heap
> still has space, gc will not be triggered.
>     It seems that RI will start gc before native memory heap is depleted and
> thus prevents out-of-memory error.
>

Sounds interesting. And if RI can prevent out-memory error but Harmony
fails to do that, I think we need to improve Harmony code.

>    Then our work focuses on:
>    1. When gc is required.
>    2. Trigger a gc.
>    The first one requires that we get support from operating system, since
> the memory allocated in the native code, for example by malloc, is out of
> the control of java VM.( I have ever thought of counting the used memory in
> hymemory_allocate, but the plan fails since hymemory_free will not report
> how much space is released.)

I'm not sure, only a thought, if we can record all memory allocate
information in a table so that we can calculate when invoking free.
What's more, if we only focus on DirectByteBuffer, IMO, jni method
NewDirectByteBuffer/GetDirectBufferCapacity can easily take this task.
Notice these jni-methods are added in Java5 with DirectByteBuffer, I
think there may be some relation ship with DirectByteBuffer GC.
However, if we are looking forward to control all native memory, and
if we can get enough information about program memory use by system
call, I agree on this solution.
Some further questions remain, e.g, what can we do if memory is still
not big enough after GC?

>    The second one needs the help from VM. System.gc() is not so reliable as
> spec says, so it is necessary to have a internal channel to notify VM to
> start gc.
>
>     One solution, I think, is to let a monitor thread in VM to check whether
> OS physical memory is low. For example, the QueryMemoryResourceNotification
> of win32 API is a candidate. Although the interrupt model is more effective:
> win32 SDK provides a CreateMemoryResourceNotification to get a handler on
> which the monitor thread can wait, maybe on other platforms, OS does not
> supply such a convenience. So the monitor thread in the VM might have to
> check the OS resource once for a while and if necessary the monitor thread
> will call a GC.
>
>    My suggestion is first to add some function to monitor memory in portlib,
> since it is highly related to platforms and in portlib there has been
> some useful tools.(Thanks Mark to point out that.) On the other hand, we can
> negotiate a channel to trigger gc in VM. Actually I am not an expert on VM
> since I am not sure whether there has been some monitor thread and the load
> on performance if such a monitor thread is added to the VM...
>

I wonder where is more suitable to put this native-memory-GC code, VM
or portlib. Due to the similar mechanism of original GC, will it make
sense if we put it into VM? And I think we can avoid adding a new
monitor, e.g, just call for GC in jni method, NewDirectByteBuffer

Waiting for some VM and GC experts :)
>
> --
> Leo Li
> China Software Development Lab, IBM
>
>


-- 

Best Regards!

Jimmy, Jing Lv
China Software Development Lab, IBM