You are viewing a plain text version of this content. The canonical link for it is here.
Posted to kato-spec@incubator.apache.org by Steve Poole <sp...@googlemail.com> on 2009/07/25 17:17:07 UTC

New draft versin of spec

I've just posted to the wiki a docbook based version of the specification.
Still needs work but a lot better than it was.


http://cwiki.apache.org/confluence/download/attachments/2326547/spec-edr-25july2009.pdf

Cheers

Steve

Re: New draft versin of spec

Posted by Steve Poole <sp...@googlemail.com>.
On Mon, Jul 27, 2009 at 7:39 PM, Steve Poole <sp...@googlemail.com>wrote:

>
>
> On Mon, Jul 27, 2009 at 5:26 PM, Stuart Monteith <st...@stoo.me.uk>wrote:
>
>> Hi Konstantin,
>>   Thanks for you comments, I've added mine inline below.
>>
>> Regards,
>>   Stuart
>>
>> Bobrovsky, Konstantin S wrote:
>>
>>> Hi Steve, all,
>>>
>>> FYI: I added some comments to the Kato javadoc @
>>> http://cwiki.apache.org/confluence/display/KATO/JavaDoc+comments. I'm
>>> resending them to this list for convenience.
>>>
>>> Thanks,
>>> Konst
>>>  Intel Novosibirsk
>>> Closed Joint Stock Company Intel A/O
>>> Registered legal address: Krylatsky Hills Business Park, 17 Krylatskaya
>>> Str., Bldg 4, Moscow 121614, Russian Federation
>>>  Top-level
>>> =========
>>>
>>> (1) kato.java seems a specialization of kato.runtime, so
>>> kato.runtime.java could be used instead.
>>>
>>>
>>>
>> +1
>
> +1  (does that become +2 :-)
>
>
I've applied this change to the API. It will be visible in the next drop of
the spec doc

>
>
>>
>>  (2) I noticed that there are API elements in the form of List
>>> get[SomeEntityName]() and javadoc does not specify objects of which type
>>> will be returned in the resulting list. E.g. ImageThread.getStackFrames().
>>>
>>>
>>>
>> I'll fix these up when I come across them. I've been reviewing the Java
>> runtime.
>>
>>  (3) It would be nice to have an opportunity to create an Image* object
>>> from a pointer into an image (having ImagePointer instance which describes
>>> the start of the Image* entity)
>>>
>>>
>>>
>> Can you give an example? I can see how this might be useful for the java
>> API, but I don't see how it'd be useful for image.
>>
>>  org.apache.kato.image
>>> =====================
>>>
>>> ImageModule
>>> -----------
>>>
>>> There should also be a way to get the address the module was loaded at
>>> and, if possible, its size.
>>>
>>>
>>>
>> I think this is a JavaDoc issue. ImageModule.getSections() returns
>> ImageSections that contain the information you are asking for. It is not
>> clear from
>> the doc that this is the case.
>>
>>> ImageRegister
>>> -------------
>>>
>>> register names for different CPUs should probably follow the naming
>>> convention adopted in the official specification of this CPU. This way tools
>>> would work uniformly with different Kato implementations.
>>>
>>>
>>>
>> I think having us specify the register set for every possible processor
>> architecture is inappropriate, so I agree.
>>
>>> 'Long' or 'Integer' might be too vague for specifying register type. How
>>> about '64 bit' or '32 bit' plus an additional characteristic such as
>>> 'general purpose/floating point/application/flags/...' ?
>>>
>>>
>> The "Type" information is there to describe which subclass of Number will
>> be returned by ImageRegister.getValue(). But you're right, it doesn't
>> actually describe
>> what the register contains.
>>
>>> The register sets specified here should be complete, otherwise we could
>>> get implementation/tool incompatibilites for the registers missing from this
>>> javadoc. One of the alternative ways for specifying them could be just a
>>> link to the corresponding external specification.
>>>
>>>
>>>
>> I have to agree with linking to the external specification. There has to
>> be at least some agreement to the set of registers available.
>>
>>> org.apache.kato.java
>>> ====================
>>>
>>> JavaReference
>>> -------------
>>>
>>> (1)
>>> sometimes it is very helpful to know where the heap root actually resides
>>> - the corresponding CPU register and/or memory address (in a memory stack
>>> frame or register stack frame), and this info is usually available in Java
>>> runtimes.
>>>
>>>
>>>
>> So, this would be something like: ImagePointer JavaReference.getAddress()
>> and ImageRegister JavaReference.getRegister(), with a boolean for both?
>>
>>  (2)
>>> It seems the API here lacks a REFERENCE_DIRIVED reference type to
>>> describe derived references (pointers), which point to an interrior of a
>>> Java object - to an array element, for example. Derived references are
>>> reported to Garbage Collector in pairs with references to their base Java
>>> objects.
>>>
>>>
>>>
>>>
>> The JavaReference types are the same as what you would get from JVMTI, so
>> that's why we have that limited list.
>> If I were to implement this, I would have had  the reference just be like
>> any other reference to a field or element.
>> But if we were to expand it, I'd suggest something like the following:
>>
>>   REFERENCE_DERIVED_ARRAY_ELEMENT
>>   REFERENCE_DERIVED_OBJECT_FIELD
>>   REFERENCE_DERIVED_CLASS_FIELD
>>
>>
>>  JavaRuntime
>>> -----------
>>>
>>> (1) We should have somewhere in the API means to iterate trough generated
>>> code sequences other than managed methods - so-called 'stubs'. Each stub is
>>> similar to a method: it has a name, (possibly, multiple) code sections. This
>>> interface (JavaRuntime) seems an appropriate place for this API.
>>>
>>>  (2) Not all methods within a Java runtime are necessarily compiled. Many
>>> may remain interpreted, others might not even have a bytecode. So I would
>>> suggest to replace 'getCompiledMethods' with smth like 'getJavaMethods' or
>>> add 'getInterpretedMethods', 'getMethodsWithoutCode'.
>>>
>>>
>>>
>> The intention isn't for all of the methods to be compiled, the method will
>> just list those methods that are compiled. I would argue for us actually
>> removing the JavaRuntime.getCompiledMethods() if we don't have a reason for
>> including it.
>>
>> Apparently we may also have native methods that have their stubs compiled
>> too.
>>
>>> JavaStackFrame
>>> --------------
>>>
>>> (1)
>>> Usually, Java stack frames contain information about Java monitors held
>>> in this frame. This is necessary because Java spec requires "forced" release
>>> of all monitors held by a method in case it "completes abruptly". So
>>> something like
>>> List getHeldMonitors()
>>> would be really useful.
>>>
>>>
>>>
>> I agree, having the API explictly show where monitors are held would take
>> people closer towards diagnosing deadlocks, as opposed to having them read
>> through their code and work out where they were taken. It may not always be
>> obvious from reading code which monitor has actually been taken.
>>
>
> +1
>
>>
>>
>>  (2)
>>> API also needs to support 'split' stack frames on Itanium - every frame
>>> has a traditinal memory stack frame and additionally, a register stack frame
>>> where Register Stack Engine (RSE) spills current register frame during a
>>> call (and restores from upon return).
>>>
>>> Maybe this could be done via delegating responsibility for such support
>>> to an Itanium-specific implementaion of the spec?
>>>
>>>
>>>
>> The API isn't very good at exposing the native information. With the
>> currently JavaThread.getStackSections() could return two ImageSections if
>> you are on IA64.
>> Certainly that's what we do for ImageThread.getStackSections() on IBM's
>> 1.4.2 IA64 Linux implementation.
>>
>> One of the the unresolved problems I have with the API is how to relate
>> all of the different frame types. Just now Java frame could be on it's own
>> "Java stack",
>> on the native stack as interpreted or compiled code, could be a native
>> method with a stub calling a C/C++ function, and we don't help with this.
>> For example, if you have a stack like:
>>
>> C function frame
>> stub frame
>> native method frame
>> compiled method frame (on native stack)
>> interpreted method frame (on own stack)
>>
>> how do you derive that information from the Kato API? I don't think we
>> can.
>>
>>  (3)
>>> while getLocation() can provide information about what method this frame
>>> belongs to, this method may have several compiled or interpreted
>>> "incarnations". So, I would suggest to add the following:
>>>
>>> boolean isCompiledFrame()
>>> boolean isInterpretedFrame()
>>>
>>> /** @return an ImageSection containing the compiled code to which this
>>> stack frame corresponds */
>>> ImageSection getCode()
>>>
>>>
>>>
>> I agree with this, there can be several incarnations of a method inflight
>> that we aren't presenting. Contrary to your suggestion, should we make it
>> easy to
>> get all java method incarnations? I might suggest that we have:
>>
>> List<JavaMethodIncarnation> JavaMethod.getIncarnations()
>>
>> JavaMethodIncarnation JavaLocation.getMethodIncarnation()
>>
>> In JavaMethodIncarnation:
>>   JavaMethod getMethod()
>>   boolean isJITCompiled()
>>   boolean isNative()
>>   int compilationLevel()
>>   List<ImageSection> getSections();
>>
>>  (4)
>>> note that sometimes methods are compiled with the entry point being not
>>> the first method's bytecode (compilation for so-called on-stack-replacement
>>> or OSR). So each compiled version of a method should also be annotated with
>>> bytecode index where the entry point for this compilation is. In this sense,
>>> ImageSection is not enough to describe a compiled code.
>>>
>>>
>>>
>> Sure,
>>   ImagePointer JavaMethodIncarnation.getEntryPoint()
>> ?
>>
>> JavaMethodIncarnation is a placeholder for perhaps a more sensible name.
>>
>
> I can see the benefit and  I agree we could look for a different name.
>
>
>>
>>  -----Original Message-----
>>> From: Steve Poole [mailto:spoole167@googlemail.com] Sent: Sunday, July
>>> 26, 2009 12:17 AM
>>> To: kato-spec@incubator.apache.org
>>> Subject: New draft versin of spec
>>>
>>> I've just posted to the wiki a docbook based version of the
>>> specification.
>>> Still needs work but a lot better than it was.
>>>
>>>
>>>
>>> http://cwiki.apache.org/confluence/download/attachments/2326547/spec-edr-25july2009.pdf
>>>
>>> Cheers
>>>
>>> Steve
>>>
>>>
>>
>> --
>> Stuart Monteith
>> http://blog.stoo.me.uk/
>>
>>
>

Re: New draft versin of spec

Posted by Steve Poole <sp...@googlemail.com>.
On Mon, Jul 27, 2009 at 5:26 PM, Stuart Monteith <st...@stoo.me.uk> wrote:

> Hi Konstantin,
>   Thanks for you comments, I've added mine inline below.
>
> Regards,
>   Stuart
>
> Bobrovsky, Konstantin S wrote:
>
>> Hi Steve, all,
>>
>> FYI: I added some comments to the Kato javadoc @
>> http://cwiki.apache.org/confluence/display/KATO/JavaDoc+comments. I'm
>> resending them to this list for convenience.
>>
>> Thanks,
>> Konst
>>  Intel Novosibirsk
>> Closed Joint Stock Company Intel A/O
>> Registered legal address: Krylatsky Hills Business Park, 17 Krylatskaya
>> Str., Bldg 4, Moscow 121614, Russian Federation
>>  Top-level
>> =========
>>
>> (1) kato.java seems a specialization of kato.runtime, so kato.runtime.java
>> could be used instead.
>>
>>
>>
> +1

+1  (does that become +2 :-)


>
>
>  (2) I noticed that there are API elements in the form of List
>> get[SomeEntityName]() and javadoc does not specify objects of which type
>> will be returned in the resulting list. E.g. ImageThread.getStackFrames().
>>
>>
>>
> I'll fix these up when I come across them. I've been reviewing the Java
> runtime.
>
>  (3) It would be nice to have an opportunity to create an Image* object
>> from a pointer into an image (having ImagePointer instance which describes
>> the start of the Image* entity)
>>
>>
>>
> Can you give an example? I can see how this might be useful for the java
> API, but I don't see how it'd be useful for image.
>
>  org.apache.kato.image
>> =====================
>>
>> ImageModule
>> -----------
>>
>> There should also be a way to get the address the module was loaded at
>> and, if possible, its size.
>>
>>
>>
> I think this is a JavaDoc issue. ImageModule.getSections() returns
> ImageSections that contain the information you are asking for. It is not
> clear from
> the doc that this is the case.
>
>> ImageRegister
>> -------------
>>
>> register names for different CPUs should probably follow the naming
>> convention adopted in the official specification of this CPU. This way tools
>> would work uniformly with different Kato implementations.
>>
>>
>>
> I think having us specify the register set for every possible processor
> architecture is inappropriate, so I agree.
>
>> 'Long' or 'Integer' might be too vague for specifying register type. How
>> about '64 bit' or '32 bit' plus an additional characteristic such as
>> 'general purpose/floating point/application/flags/...' ?
>>
>>
> The "Type" information is there to describe which subclass of Number will
> be returned by ImageRegister.getValue(). But you're right, it doesn't
> actually describe
> what the register contains.
>
>> The register sets specified here should be complete, otherwise we could
>> get implementation/tool incompatibilites for the registers missing from this
>> javadoc. One of the alternative ways for specifying them could be just a
>> link to the corresponding external specification.
>>
>>
>>
> I have to agree with linking to the external specification. There has to be
> at least some agreement to the set of registers available.
>
>> org.apache.kato.java
>> ====================
>>
>> JavaReference
>> -------------
>>
>> (1)
>> sometimes it is very helpful to know where the heap root actually resides
>> - the corresponding CPU register and/or memory address (in a memory stack
>> frame or register stack frame), and this info is usually available in Java
>> runtimes.
>>
>>
>>
> So, this would be something like: ImagePointer JavaReference.getAddress()
> and ImageRegister JavaReference.getRegister(), with a boolean for both?
>
>  (2)
>> It seems the API here lacks a REFERENCE_DIRIVED reference type to describe
>> derived references (pointers), which point to an interrior of a Java object
>> - to an array element, for example. Derived references are reported to
>> Garbage Collector in pairs with references to their base Java objects.
>>
>>
>>
>>
> The JavaReference types are the same as what you would get from JVMTI, so
> that's why we have that limited list.
> If I were to implement this, I would have had  the reference just be like
> any other reference to a field or element.
> But if we were to expand it, I'd suggest something like the following:
>
>   REFERENCE_DERIVED_ARRAY_ELEMENT
>   REFERENCE_DERIVED_OBJECT_FIELD
>   REFERENCE_DERIVED_CLASS_FIELD
>
>
>  JavaRuntime
>> -----------
>>
>> (1) We should have somewhere in the API means to iterate trough generated
>> code sequences other than managed methods - so-called 'stubs'. Each stub is
>> similar to a method: it has a name, (possibly, multiple) code sections. This
>> interface (JavaRuntime) seems an appropriate place for this API.
>>
>>  (2) Not all methods within a Java runtime are necessarily compiled. Many
>> may remain interpreted, others might not even have a bytecode. So I would
>> suggest to replace 'getCompiledMethods' with smth like 'getJavaMethods' or
>> add 'getInterpretedMethods', 'getMethodsWithoutCode'.
>>
>>
>>
> The intention isn't for all of the methods to be compiled, the method will
> just list those methods that are compiled. I would argue for us actually
> removing the JavaRuntime.getCompiledMethods() if we don't have a reason for
> including it.
>
> Apparently we may also have native methods that have their stubs compiled
> too.
>
>> JavaStackFrame
>> --------------
>>
>> (1)
>> Usually, Java stack frames contain information about Java monitors held in
>> this frame. This is necessary because Java spec requires "forced" release of
>> all monitors held by a method in case it "completes abruptly". So something
>> like
>> List getHeldMonitors()
>> would be really useful.
>>
>>
>>
> I agree, having the API explictly show where monitors are held would take
> people closer towards diagnosing deadlocks, as opposed to having them read
> through their code and work out where they were taken. It may not always be
> obvious from reading code which monitor has actually been taken.
>

+1

>
>
>  (2)
>> API also needs to support 'split' stack frames on Itanium - every frame
>> has a traditinal memory stack frame and additionally, a register stack frame
>> where Register Stack Engine (RSE) spills current register frame during a
>> call (and restores from upon return).
>>
>> Maybe this could be done via delegating responsibility for such support to
>> an Itanium-specific implementaion of the spec?
>>
>>
>>
> The API isn't very good at exposing the native information. With the
> currently JavaThread.getStackSections() could return two ImageSections if
> you are on IA64.
> Certainly that's what we do for ImageThread.getStackSections() on IBM's
> 1.4.2 IA64 Linux implementation.
>
> One of the the unresolved problems I have with the API is how to relate all
> of the different frame types. Just now Java frame could be on it's own "Java
> stack",
> on the native stack as interpreted or compiled code, could be a native
> method with a stub calling a C/C++ function, and we don't help with this.
> For example, if you have a stack like:
>
> C function frame
> stub frame
> native method frame
> compiled method frame (on native stack)
> interpreted method frame (on own stack)
>
> how do you derive that information from the Kato API? I don't think we can.
>
>  (3)
>> while getLocation() can provide information about what method this frame
>> belongs to, this method may have several compiled or interpreted
>> "incarnations". So, I would suggest to add the following:
>>
>> boolean isCompiledFrame()
>> boolean isInterpretedFrame()
>>
>> /** @return an ImageSection containing the compiled code to which this
>> stack frame corresponds */
>> ImageSection getCode()
>>
>>
>>
> I agree with this, there can be several incarnations of a method inflight
> that we aren't presenting. Contrary to your suggestion, should we make it
> easy to
> get all java method incarnations? I might suggest that we have:
>
> List<JavaMethodIncarnation> JavaMethod.getIncarnations()
>
> JavaMethodIncarnation JavaLocation.getMethodIncarnation()
>
> In JavaMethodIncarnation:
>   JavaMethod getMethod()
>   boolean isJITCompiled()
>   boolean isNative()
>   int compilationLevel()
>   List<ImageSection> getSections();
>
>  (4)
>> note that sometimes methods are compiled with the entry point being not
>> the first method's bytecode (compilation for so-called on-stack-replacement
>> or OSR). So each compiled version of a method should also be annotated with
>> bytecode index where the entry point for this compilation is. In this sense,
>> ImageSection is not enough to describe a compiled code.
>>
>>
>>
> Sure,
>   ImagePointer JavaMethodIncarnation.getEntryPoint()
> ?
>
> JavaMethodIncarnation is a placeholder for perhaps a more sensible name.
>

I can see the benefit and  I agree we could look for a different name.


>
>  -----Original Message-----
>> From: Steve Poole [mailto:spoole167@googlemail.com] Sent: Sunday, July
>> 26, 2009 12:17 AM
>> To: kato-spec@incubator.apache.org
>> Subject: New draft versin of spec
>>
>> I've just posted to the wiki a docbook based version of the specification.
>> Still needs work but a lot better than it was.
>>
>>
>>
>> http://cwiki.apache.org/confluence/download/attachments/2326547/spec-edr-25july2009.pdf
>>
>> Cheers
>>
>> Steve
>>
>>
>
> --
> Stuart Monteith
> http://blog.stoo.me.uk/
>
>

RE: New draft versin of spec

Posted by "Bobrovsky, Konstantin S" <ko...@intel.com>.
Hi Stuart,

I've extracted the discussion points for convenience, and added few more comments.

>>> The JavaReference types are the same as what you would get from JVMTI,
>>> so that's why we have that limited list.
>>>
>>
>> [Bobrovsky, Konstantin S]
>> So it seems JVMTI does not fully cover the variety of references.
>>
>>
>[Monteith, S]
>Well, notionally a derived reference is a reference to the object the
>element/field is in as it is that it is keeping alive.

To avoid confusion: in my view, the reference to the object itself is the 'base reference' and the reference to the interior of the object (element or field) is the 'derived' reference.


>You could have a description of "Derived reference to element 42" from
>the JavaReference.
>You will want to know explicitly that it is a reference of a specific
>type on your specific JVM, and so the specificity is important.
>My question is, what other reference types are there?

I think these three you mentioned is enough. The only addition is that derived reference should have an API to get the base reference.
>>>    REFERENCE_DERIVED_ARRAY_ELEMENT
>>>    REFERENCE_DERIVED_OBJECT_FIELD
>>>    REFERENCE_DERIVED_CLASS_FIELD


>>>> (1) We should have somewhere in the API means to iterate trough
>generated
>>>>
>>> code sequences other than managed methods - so-called 'stubs'. Each stub
>is
>>> similar to a method: it has a name, (possibly, multiple) code sections.
>>> This interface (JavaRuntime) seems an appropriate place for this API.
>>>
>>
>> [Bobrovsky, Konstantin S]
>> (To develop this thought a bit...)
>> Maybe we could have
>>   interface CodeStub {
>>     String getName();
>>     ImageSection getCodeSections();
>>   }
>>
>> And
>> List JavaRuntime.getCodeStubs();
>>
>>
>[Monteith, S]
>I think I'd like to see how these are used. Presumably then they would
>be returned in the stacks.
>Presumably for our integrated stack frames we'd have a class hierarchy
>like:
>
>StackElement--+----ImageStackFrame
>              |
>              +----JavaStackFrame
>              |
>              +----CodeStub
>
>and:
>     List<StackElement> JavaThread.getStackElements()
>
>That would put the code stubs in some sort of context.

From my viewpoint a codestub is a piece of memory where executable code (plus some relocations, id, etc.) resides. A single code stub (for example, method resolution stub in Hotspot) is used in multiple threads. In contrast, a stack frame is associated with a particular invocation of a piece of code, plus it represents this invocation's 'local memory' rather than code. (Sorry for saying obvious things - this is just to explain my position). So, I think it is not quite correct to inherit CodeStub from the StackElement.

Few more examples of 'stubs' (in Hotspot terminology):

- efficient assembly sequences generated at run-time in VM for certain routine operations like CAS, other atomic operations, memory copying, etc.
- 'deoptimization blob' which is invoked from a compiled code
- stubs for calling interpreted methods from compiled ones and vise versa.

>[Monteith, S]
>I can see how there'd be difficulties, there is certainly a case for
>having ImageSections for Java and Image StackFrames, to show where they
>are and their extent.
>One thing that troubles me is how we'd handle the NaT bits - the
>cleanest option would be to leave an 8 byte whole where they've been
>spilled. Ideally
>the NaT bits could be reconstructed for the spilled registers, but that
>is a distinctly odd feature of IA64.

Yes, NaTs are 'hardcore' ia64 specifics :-). I think ia64 implementation could only provide access to the 64-bit NAT collections on the reg. stack for every 64 spilled GP regs, and client can use this info at on its own purpose.

>>> For example, if you have a stack like:
>>>
>>> C function frame
>>> stub frame
>>> native method frame
>>> compiled method frame (on native stack)
>>> interpreted method frame (on own stack)
>>>
>>> how do you derive that information from the Kato API? I don't think we
>can.
>>>
>>
>> [Bobrovsky, Konstantin S]
>> As I understand the question, the problem is in apparent complexity of
>writing a Kato implementation which would be able to iterate through
>various kinds of frames on the a thread's stack, and for each frame report
>its type, extent and other information. ?
>>
>> I think any Java runtime should be able to do this (Hotspot is), at least
>when collecting garbage. And if a Java runtime knows how to do this in a
>live VM, Java code (Kato implementation for this runtime) which does this
>for a dead VM should look similar.
>>
>[Monteith, S]
>See my comment above. It's not so much the implementation complexity I
>was worried about, but rather how we represent it in the API. We
>currently don't
>provide a way, that I can see, of sensibly communicating the layout of
>the stack.

Maybe I'm missing something, but providing such an API should not be too hard.

JavaThread.getStackFrames() returns a list of frames on this thread's stack, and I assume the list is ordered the same way the real call stack was. We can iterate through the list and for each JavaStackFrame as whether it is
isNative() or isCompiled(), etc.

?

Thanks,
Konst

Intel Novosibirsk
Closed Joint Stock Company Intel A/O
Registered legal address: Krylatsky Hills Business Park,
17 Krylatskaya Str., Bldg 4, Moscow 121614,
Russian Federation


>-----Original Message-----
>From: Stuart Monteith [mailto:stukato@stoo.me.uk]
>Sent: Tuesday, July 28, 2009 11:37 PM
>To: kato-spec@incubator.apache.org
>Subject: Re: New draft versin of spec
>
>Hi,
>    Seamonkey doesn't help much, so replies as before :-)
>
>
>Bobrovsky, Konstantin S wrote:
>> Hi Stuart,
>>
>> Some more of my comments are also inline below (marked with '[Bobrovsky,
>Konstantin S]' - that's how Outlook helps :-) )
>>
>> Thanks,
>> Konst
>>
>> Intel Novosibirsk
>> Closed Joint Stock Company Intel A/O
>> Registered legal address: Krylatsky Hills Business Park,
>> 17 Krylatskaya Str., Bldg 4, Moscow 121614,
>> Russian Federation
>>
>>
>>
>>> -----Original Message-----
>>> From: Stuart Monteith [mailto:stukato@stoo.me.uk]
>>> Sent: Monday, July 27, 2009 11:26 PM
>>> To: kato-spec@incubator.apache.org
>>> Subject: Re: New draft versin of spec
>>>
>>> Hi Konstantin,
>>>    Thanks for you comments, I've added mine inline below.
>>>
>>> Regards,
>>>    Stuart
>>>
>>> Bobrovsky, Konstantin S wrote:
>>>
>>>> Hi Steve, all,
>>>>
>>>> FYI: I added some comments to the Kato javadoc @
>>>>
>>> http://cwiki.apache.org/confluence/display/KATO/JavaDoc+comments. I'm
>>> resending them to this list for convenience.
>>>
>>>> Thanks,
>>>> Konst
>>>>
>>>> Intel Novosibirsk
>>>> Closed Joint Stock Company Intel A/O
>>>> Registered legal address: Krylatsky Hills Business Park,
>>>> 17 Krylatskaya Str., Bldg 4, Moscow 121614,
>>>> Russian Federation
>>>>
>>>> Top-level
>>>> =========
>>>>
>>>> (1) kato.java seems a specialization of kato.runtime, so
>>>>
>>> kato.runtime.java could be used instead.
>>>
>>>>
>>> +1
>>>
>>>
>>>> (2) I noticed that there are API elements in the form of List
>>>>
>>> get[SomeEntityName]() and javadoc does not specify objects of which type
>>> will be returned in the resulting list. E.g.
>ImageThread.getStackFrames().
>>>
>>>>
>>> I'll fix these up when I come across them. I've been reviewing the Java
>>> runtime.
>>>
>>>
>>>> (3) It would be nice to have an opportunity to create an Image* object
>>>>
>> >from a pointer into an image (having ImagePointer instance which
>describes
>>
>>> the start of the Image* entity)
>>>
>>>>
>>> Can you give an example? I can see how this might be useful for the java
>>> API, but I don't see how it'd be useful for image.
>>>
>>>
>>
>> [Bobrovsky, Konstantin S]
>> e.g.
>> ImageStackFrame ImageFactory.getImageStackFrameAt(ImagePointer addr)
>> ImageThread ImageFactory.getImageThreadAt(ImagePointer addr)
>>
>> And yes, this would be useful for the kato.java API too - I agree.
>>
>>
>[Monteith, S]
>
>I think I can understand how this applies to
>getImageStackFrameAt(ImagePointer) - a pointer that lies within
>the stack could be returned. It might be better though to implement
>ImageStackFrame.getSections() and search
>through them. This would also apply to IA64 split stacks.
>
>>>> org.apache.kato.image
>>>> =====================
>>>>
>>>> ImageModule
>>>> -----------
>>>>
>>>> There should also be a way to get the address the module was loaded at
>>>>
>>> and, if possible, its size.
>>>
>>>>
>>> I think this is a JavaDoc issue. ImageModule.getSections() returns
>>> ImageSections that contain the information you are asking for. It is not
>>> clear from
>>> the doc that this is the case.
>>>
>>>> ImageRegister
>>>> -------------
>>>>
>>>> register names for different CPUs should probably follow the naming
>>>>
>>> convention adopted in the official specification of this CPU. This way
>>> tools would work uniformly with different Kato implementations.
>>>
>>>>
>>> I think having us specify the register set for every possible processor
>>> architecture is inappropriate, so I agree.
>>>
>>>> 'Long' or 'Integer' might be too vague for specifying register type.
>How
>>>>
>>> about '64 bit' or '32 bit' plus an additional characteristic such as
>>> 'general purpose/floating point/application/flags/...' ?
>>>
>>> The "Type" information is there to describe which subclass of Number
>>> will be returned by ImageRegister.getValue(). But you're right, it
>>> doesn't actually describe
>>> what the register contains.
>>>
>>>> The register sets specified here should be complete, otherwise we could
>>>>
>>> get implementation/tool incompatibilites for the registers missing from
>>> this javadoc. One of the alternative ways for specifying them could be
>just
>>> a link to the corresponding external specification.
>>>
>>>>
>>> I have to agree with linking to the external specification. There has to
>>> be at least some agreement to the set of registers available.
>>>
>>>> org.apache.kato.java
>>>> ====================
>>>>
>>>> JavaReference
>>>> -------------
>>>>
>>>> (1)
>>>> sometimes it is very helpful to know where the heap root actually
>resides
>>>>
>>> - the corresponding CPU register and/or memory address (in a memory
>stack
>>> frame or register stack frame), and this info is usually available in
>Java
>>> runtimes.
>>>
>>>>
>>> So, this would be something like: ImagePointer
>>> JavaReference.getAddress() and ImageRegister
>>> JavaReference.getRegister(), with a boolean for both?
>>>
>>>
>>
>> [Bobrovsky, Konstantin S]
>> getAddress() and getRegister() look satisfactory. What the boolean is
>for?
>> Can getRegister() return just null if the reference resides in memory
>only?
>>
>>
>[Monteith, S]
>I think I was just being verbose. The boolean is just to indicate
>whether or not the reference is at an address and/or a register. With
>null's then
>it would be redundant, I agree.
>
>>>> (2)
>>>> It seems the API here lacks a REFERENCE_DIRIVED reference type to
>>>>
>>> describe derived references (pointers), which point to an interrior of a
>>> Java object - to an array element, for example. Derived references are
>>> reported to Garbage Collector in pairs with references to their base
>Java
>>> objects.
>>>
>>>>
>>>>
>>> The JavaReference types are the same as what you would get from JVMTI,
>>> so that's why we have that limited list.
>>>
>>
>> [Bobrovsky, Konstantin S]
>> So it seems JVMTI does not fully cover the variety of references.
>>
>>
>[Monteith, S]
>Well, notionally a derived reference is a reference to the object the
>element/field is in as it is that it is keeping alive.
>You could have a description of "Derived reference to element 42" from
>the JavaReference.
>You will want to know explicitly that it is a reference of a specific
>type on your specific JVM, and so the specificity is important.
>My question is, what other reference types are there?
>
>>> If I were to implement this, I would have had  the reference just be
>>> like any other reference to a field or element.
>>> But if we were to expand it, I'd suggest something like the following:
>>>
>>>    REFERENCE_DERIVED_ARRAY_ELEMENT
>>>    REFERENCE_DERIVED_OBJECT_FIELD
>>>    REFERENCE_DERIVED_CLASS_FIELD
>>>
>>>
>>
>> [Bobrovsky, Konstantin S] This looks good to me.
>>
>>
>>>> JavaRuntime
>>>> -----------
>>>>
>>>> (1) We should have somewhere in the API means to iterate trough
>generated
>>>>
>>> code sequences other than managed methods - so-called 'stubs'. Each stub
>is
>>> similar to a method: it has a name, (possibly, multiple) code sections.
>>> This interface (JavaRuntime) seems an appropriate place for this API.
>>>
>>
>> [Bobrovsky, Konstantin S]
>> (To develop this thought a bit...)
>> Maybe we could have
>>   interface CodeStub {
>>     String getName();
>>     ImageSection getCodeSections();
>>   }
>>
>> And
>> List JavaRuntime.getCodeStubs();
>>
>>
>[Monteith, S]
>I think I'd like to see how these are used. Presumably then they would
>be returned in the stacks.
>Presumably for our integrated stack frames we'd have a class hierarchy
>like:
>
>StackElement--+----ImageStackFrame
>              |
>              +----JavaStackFrame
>              |
>              +----CodeStub
>
>and:
>     List<StackElement> JavaThread.getStackElements()
>
>That would put the code stubs in some sort of context.
>>>> (2) Not all methods within a Java runtime are necessarily compiled.
>Many
>>>>
>>> may remain interpreted, others might not even have a bytecode. So I
>would
>>> suggest to replace 'getCompiledMethods' with smth like 'getJavaMethods'
>or
>>> add 'getInterpretedMethods', 'getMethodsWithoutCode'.
>>>
>>>>
>>> The intention isn't for all of the methods to be compiled, the method
>>> will just list those methods that are compiled. I would argue for us
>>> actually removing the JavaRuntime.getCompiledMethods() if we don't have
>>> a reason for including it.
>>>
>>
>> [Bobrovsky, Konstantin S]
>> I thought of this method as an iterator through all java methods known to
>the system - its javadoc implies this: 'This is short cut method. The same
>result can be found by iterating over all methods in all class loaders in
>all classes'. This iteration will also yield methods which are not
>compiled.
>>
>> So we can have all
>>   getCompiledMethods()
>>   getInterpretedMethods()
>>   getMethodsWithoutCode()
>> and
>>   getJavaMethods()
>>
>> or just
>>   List getJavaMethods(MethodFilter f)
>>
>> where f defines the acceptance criteria.
>>
>>
>[Monteith, S]
>Yes, the implication in the JavaDoc was that getCompiledMethods() would
>return every JavaMethod that returns something for
>JavaMethod.getCompiledSections() other than an empty list.
>I'd settle on either removing the convenience method or looking at
>patterns we could apply to the whole API.
>>> Apparently we may also have native methods that have their stubs
>>> compiled too.
>>>
>>>> JavaStackFrame
>>>> --------------
>>>>
>>>> (1)
>>>> Usually, Java stack frames contain information about Java monitors held
>>>>
>>> in this frame. This is necessary because Java spec requires "forced"
>>> release of all monitors held by a method in case it "completes
>abruptly".
>>> So something like
>>>
>>>> List getHeldMonitors()
>>>> would be really useful.
>>>>
>>>>
>>>>
>>> I agree, having the API explictly show where monitors are held would
>>> take people closer towards diagnosing deadlocks, as opposed to having
>>> them read through their code and work out where they were taken. It may
>>> not always be obvious from reading code which monitor has actually been
>>> taken.
>>>
>>>
>>>> (2)
>>>> API also needs to support 'split' stack frames on Itanium - every frame
>>>>
>>> has a traditinal memory stack frame and additionally, a register stack
>>> frame where Register Stack Engine (RSE) spills current register frame
>>> during a call (and restores from upon return).
>>>
>>>> Maybe this could be done via delegating responsibility for such support
>>>>
>>> to an Itanium-specific implementaion of the spec?
>>>
>>>>
>>> The API isn't very good at exposing the native information. With the
>>> currently JavaThread.getStackSections() could return two ImageSections
>>> if you are on IA64.
>>> Certainly that's what we do for ImageThread.getStackSections() on IBM's
>>> 1.4.2 IA64 Linux implementation.
>>>
>>
>> [Bobrovsky, Konstantin S]
>> Yes, this is a good catch. Stack is split on ia64 at the thread level
>too.
>> Though, I meant slightly different thing: on IA64, we'd like to have
>information about where register stack for a particular stack frame (rather
>than a thread) starts in memory.
>>
>>
>[Monteith, S]
>I can see how there'd be difficulties, there is certainly a case for
>having ImageSections for Java and Image StackFrames, to show where they
>are and their extent.
>One thing that troubles me is how we'd handle the NaT bits - the
>cleanest option would be to leave an 8 byte whole where they've been
>spilled. Ideally
>the NaT bits could be reconstructed for the spilled registers, but that
>is a distinctly odd feature of IA64.
>
>>> One of the the unresolved problems I have with the API is how to relate
>>> all of the different frame types. Just now Java frame could be on it's
>>> own "Java stack",
>>> on the native stack as interpreted or compiled code, could be a native
>>> method with a stub calling a C/C++ function, and we don't help with
>this.
>>> For example, if you have a stack like:
>>>
>>> C function frame
>>> stub frame
>>> native method frame
>>> compiled method frame (on native stack)
>>> interpreted method frame (on own stack)
>>>
>>> how do you derive that information from the Kato API? I don't think we
>can.
>>>
>>
>> [Bobrovsky, Konstantin S]
>> As I understand the question, the problem is in apparent complexity of
>writing a Kato implementation which would be able to iterate through
>various kinds of frames on the a thread's stack, and for each frame report
>its type, extent and other information. ?
>>
>> I think any Java runtime should be able to do this (Hotspot is), at least
>when collecting garbage. And if a Java runtime knows how to do this in a
>live VM, Java code (Kato implementation for this runtime) which does this
>for a dead VM should look similar.
>>
>[Monteith, S]
>See my comment above. It's not so much the implementation complexity I
>was worried about, but rather how we represent it in the API. We
>currently don't
>provide a way, that I can see, of sensibly communicating the layout of
>the stack.
>>
>>>> (3)
>>>> while getLocation() can provide information about what method this
>frame
>>>>
>>> belongs to, this method may have several compiled or interpreted
>>> "incarnations". So, I would suggest to add the following:
>>>
>>>> boolean isCompiledFrame()
>>>> boolean isInterpretedFrame()
>>>>
>>>> /** @return an ImageSection containing the compiled code to which this
>>>>
>>> stack frame corresponds */
>>>
>>>> ImageSection getCode()
>>>>
>>>>
>>>>
>>> I agree with this, there can be several incarnations of a method
>>> inflight that we aren't presenting. Contrary to your suggestion, should
>>> we make it easy to
>>> get all java method incarnations? I might suggest that we have:
>>>
>>> List<JavaMethodIncarnation> JavaMethod.getIncarnations()
>>>
>>> JavaMethodIncarnation JavaLocation.getMethodIncarnation()
>>>
>>> In JavaMethodIncarnation:
>>>    JavaMethod getMethod()
>>>    boolean isJITCompiled()
>>>    boolean isNative()
>>>    int compilationLevel()
>>>    List<ImageSection> getSections();
>>>
>>>
>>
>> [Bobrovsky, Konstantin S]
>> I think this is an interesting idea overall. But not solve the <stack
>frame> -> <method incarnation> mapping problem there should additionally be
>JavaMethodIncarnation JavaStackFrame.getMethodIncarnation()
>>
>>
>[Monteith, S]
>Yes, it's important to match them up.
>> Here is another angle to look from.
>> compilationLevel() and getSections() don't apply to an incarnation of a
>native method (which will always have one incarnation), or to an
>interpreted incarnation of a usual method. What will only be used for such
>incarnations is getMethod(), isJITCompiled() and isNative(). So the API
>looks a little 'irregular' to me. It seems the 'incarnation' semantics is
>more close to 'compiled code incarnation' of a particular method. So I
>would develop the suggested API into a little more specific one:
>>
>> JavaMethod JavaStackFrame.getMethod()
>> boolean JavaStackFrame.isCompiled()
>> boolean JavaStackFrame.isInterpreted()
>> boolean JavaStackFrame.isNative()
>>
>> // returns a non-null value only if isCompiled() is true
>> CompiledMethodIncarnation JavaStackFrame.getCompiledMethodIncarnation()
>>
>> interface CompiledMethodIncarnation {
>>     JavaMethod getMethod();
>>     int compilationLevel()
>>     List<ImageSection> getSections();
>>     int getBytecodeIndexForEntryPoint(); // repeats your suggestion below
>>     ...
>> }
>>
>> And all the details for native and interpreted stack frames (which can
>have only one 'incarnation') will be queried from the returned JavaMethod -
>such as bytecode, function address, etc.
>>
>>
>[Monteith, S]
>That looks good. It leaves a simple route for more basic uses (i.e.
>getting the name of the method and it's class), and it is sufficiently
>general to cover the various JVM implementations.
>>>> (4)
>>>> note that sometimes methods are compiled with the entry point being not
>>>>
>>> the first method's bytecode (compilation for so-called on-stack-
>replacement
>>> or OSR). So each compiled version of a method should also be annotated
>with
>>> bytecode index where the entry point for this compilation is. In this
>>> sense, ImageSection is not enough to describe a compiled code.
>>>
>>>>
>>> Sure,
>>>    ImagePointer JavaMethodIncarnation.getEntryPoint()
>>> ?
>>>
>>>
>>
>> [Bobrovsky, Konstantin S]
>> Yes, this approach looks good to me, with a slight change I described
>above.
>>
>>
>>> JavaMethodIncarnation is a placeholder for perhaps a more sensible name.
>>>
>>>> -----Original Message-----
>>>> From: Steve Poole [mailto:spoole167@googlemail.com]
>>>> Sent: Sunday, July 26, 2009 12:17 AM
>>>> To: kato-spec@incubator.apache.org
>>>> Subject: New draft versin of spec
>>>>
>>>> I've just posted to the wiki a docbook based version of the
>>>>
>>> specification.
>>>
>>>> Still needs work but a lot better than it was.
>>>>
>>>>
>>>> http://cwiki.apache.org/confluence/download/attachments/2326547/spec-
>edr-
>>>>
>>> 25july2009.pdf
>>>
>>>> Cheers
>>>>
>>>> Steve
>>>>
>>>>
>>> --
>>> Stuart Monteith
>>> http://blog.stoo.me.uk/
>>>
>>
>>
>
>Thanks,
>    Stuart
>
>--
>Stuart Monteith
>http://blog.stoo.me.uk/

Re: New draft versin of spec

Posted by Stuart Monteith <st...@stoo.me.uk>.
Hi,
    Seamonkey doesn't help much, so replies as before :-)


Bobrovsky, Konstantin S wrote:
> Hi Stuart,
>
> Some more of my comments are also inline below (marked with '[Bobrovsky, Konstantin S]' - that's how Outlook helps :-) )
>
> Thanks,
> Konst
>  
> Intel Novosibirsk
> Closed Joint Stock Company Intel A/O
> Registered legal address: Krylatsky Hills Business Park, 
> 17 Krylatskaya Str., Bldg 4, Moscow 121614, 
> Russian Federation
>  
>
>   
>> -----Original Message-----
>> From: Stuart Monteith [mailto:stukato@stoo.me.uk]
>> Sent: Monday, July 27, 2009 11:26 PM
>> To: kato-spec@incubator.apache.org
>> Subject: Re: New draft versin of spec
>>
>> Hi Konstantin,
>>    Thanks for you comments, I've added mine inline below.
>>
>> Regards,
>>    Stuart
>>
>> Bobrovsky, Konstantin S wrote:
>>     
>>> Hi Steve, all,
>>>
>>> FYI: I added some comments to the Kato javadoc @
>>>       
>> http://cwiki.apache.org/confluence/display/KATO/JavaDoc+comments. I'm
>> resending them to this list for convenience.
>>     
>>> Thanks,
>>> Konst
>>>
>>> Intel Novosibirsk
>>> Closed Joint Stock Company Intel A/O
>>> Registered legal address: Krylatsky Hills Business Park,
>>> 17 Krylatskaya Str., Bldg 4, Moscow 121614,
>>> Russian Federation
>>>
>>> Top-level
>>> =========
>>>
>>> (1) kato.java seems a specialization of kato.runtime, so
>>>       
>> kato.runtime.java could be used instead.
>>     
>>>       
>> +1
>>
>>     
>>> (2) I noticed that there are API elements in the form of List
>>>       
>> get[SomeEntityName]() and javadoc does not specify objects of which type
>> will be returned in the resulting list. E.g. ImageThread.getStackFrames().
>>     
>>>       
>> I'll fix these up when I come across them. I've been reviewing the Java
>> runtime.
>>
>>     
>>> (3) It would be nice to have an opportunity to create an Image* object
>>>       
> >from a pointer into an image (having ImagePointer instance which describes
>   
>> the start of the Image* entity)
>>     
>>>       
>> Can you give an example? I can see how this might be useful for the java
>> API, but I don't see how it'd be useful for image.
>>
>>     
>
> [Bobrovsky, Konstantin S] 
> e.g.
> ImageStackFrame ImageFactory.getImageStackFrameAt(ImagePointer addr)
> ImageThread ImageFactory.getImageThreadAt(ImagePointer addr)
>
> And yes, this would be useful for the kato.java API too - I agree.
>
>   
[Monteith, S]

I think I can understand how this applies to 
getImageStackFrameAt(ImagePointer) - a pointer that lies within
the stack could be returned. It might be better though to implement 
ImageStackFrame.getSections() and search
through them. This would also apply to IA64 split stacks.

>>> org.apache.kato.image
>>> =====================
>>>
>>> ImageModule
>>> -----------
>>>
>>> There should also be a way to get the address the module was loaded at
>>>       
>> and, if possible, its size.
>>     
>>>       
>> I think this is a JavaDoc issue. ImageModule.getSections() returns
>> ImageSections that contain the information you are asking for. It is not
>> clear from
>> the doc that this is the case.
>>     
>>> ImageRegister
>>> -------------
>>>
>>> register names for different CPUs should probably follow the naming
>>>       
>> convention adopted in the official specification of this CPU. This way
>> tools would work uniformly with different Kato implementations.
>>     
>>>       
>> I think having us specify the register set for every possible processor
>> architecture is inappropriate, so I agree.
>>     
>>> 'Long' or 'Integer' might be too vague for specifying register type. How
>>>       
>> about '64 bit' or '32 bit' plus an additional characteristic such as
>> 'general purpose/floating point/application/flags/...' ?
>>     
>> The "Type" information is there to describe which subclass of Number
>> will be returned by ImageRegister.getValue(). But you're right, it
>> doesn't actually describe
>> what the register contains.
>>     
>>> The register sets specified here should be complete, otherwise we could
>>>       
>> get implementation/tool incompatibilites for the registers missing from
>> this javadoc. One of the alternative ways for specifying them could be just
>> a link to the corresponding external specification.
>>     
>>>       
>> I have to agree with linking to the external specification. There has to
>> be at least some agreement to the set of registers available.
>>     
>>> org.apache.kato.java
>>> ====================
>>>
>>> JavaReference
>>> -------------
>>>
>>> (1)
>>> sometimes it is very helpful to know where the heap root actually resides
>>>       
>> - the corresponding CPU register and/or memory address (in a memory stack
>> frame or register stack frame), and this info is usually available in Java
>> runtimes.
>>     
>>>       
>> So, this would be something like: ImagePointer
>> JavaReference.getAddress() and ImageRegister
>> JavaReference.getRegister(), with a boolean for both?
>>
>>     
>
> [Bobrovsky, Konstantin S] 
> getAddress() and getRegister() look satisfactory. What the boolean is for?
> Can getRegister() return just null if the reference resides in memory only?
>
>   
[Monteith, S]
I think I was just being verbose. The boolean is just to indicate 
whether or not the reference is at an address and/or a register. With 
null's then
it would be redundant, I agree.

>>> (2)
>>> It seems the API here lacks a REFERENCE_DIRIVED reference type to
>>>       
>> describe derived references (pointers), which point to an interrior of a
>> Java object - to an array element, for example. Derived references are
>> reported to Garbage Collector in pairs with references to their base Java
>> objects.
>>     
>>>
>>>       
>> The JavaReference types are the same as what you would get from JVMTI,
>> so that's why we have that limited list.
>>     
>
> [Bobrovsky, Konstantin S] 
> So it seems JVMTI does not fully cover the variety of references.
>
>   
[Monteith, S]
Well, notionally a derived reference is a reference to the object the 
element/field is in as it is that it is keeping alive.
You could have a description of "Derived reference to element 42" from 
the JavaReference.
You will want to know explicitly that it is a reference of a specific 
type on your specific JVM, and so the specificity is important.
My question is, what other reference types are there?

>> If I were to implement this, I would have had  the reference just be
>> like any other reference to a field or element.
>> But if we were to expand it, I'd suggest something like the following:
>>
>>    REFERENCE_DERIVED_ARRAY_ELEMENT
>>    REFERENCE_DERIVED_OBJECT_FIELD
>>    REFERENCE_DERIVED_CLASS_FIELD
>>
>>     
>
> [Bobrovsky, Konstantin S] This looks good to me.
>
>   
>>> JavaRuntime
>>> -----------
>>>
>>> (1) We should have somewhere in the API means to iterate trough generated
>>>       
>> code sequences other than managed methods - so-called 'stubs'. Each stub is
>> similar to a method: it has a name, (possibly, multiple) code sections.
>> This interface (JavaRuntime) seems an appropriate place for this API.
>>     
>
> [Bobrovsky, Konstantin S] 
> (To develop this thought a bit...)
> Maybe we could have
>   interface CodeStub {
>     String getName();
>     ImageSection getCodeSections();
>   }
>
> And
> List JavaRuntime.getCodeStubs();
>
>   
[Monteith, S]
I think I'd like to see how these are used. Presumably then they would 
be returned in the stacks.
Presumably for our integrated stack frames we'd have a class hierarchy like:

StackElement--+----ImageStackFrame
              |
              +----JavaStackFrame
              |
              +----CodeStub

and:
     List<StackElement> JavaThread.getStackElements()

That would put the code stubs in some sort of context.
>>> (2) Not all methods within a Java runtime are necessarily compiled. Many
>>>       
>> may remain interpreted, others might not even have a bytecode. So I would
>> suggest to replace 'getCompiledMethods' with smth like 'getJavaMethods' or
>> add 'getInterpretedMethods', 'getMethodsWithoutCode'.
>>     
>>>       
>> The intention isn't for all of the methods to be compiled, the method
>> will just list those methods that are compiled. I would argue for us
>> actually removing the JavaRuntime.getCompiledMethods() if we don't have
>> a reason for including it.
>>     
>
> [Bobrovsky, Konstantin S] 
> I thought of this method as an iterator through all java methods known to the system - its javadoc implies this: 'This is short cut method. The same result can be found by iterating over all methods in all class loaders in all classes'. This iteration will also yield methods which are not compiled.
>
> So we can have all
>   getCompiledMethods()
>   getInterpretedMethods()
>   getMethodsWithoutCode()
> and
>   getJavaMethods()
>
> or just
>   List getJavaMethods(MethodFilter f)
>
> where f defines the acceptance criteria.
>
>   
[Monteith, S]
Yes, the implication in the JavaDoc was that getCompiledMethods() would 
return every JavaMethod that returns something for 
JavaMethod.getCompiledSections() other than an empty list.
I'd settle on either removing the convenience method or looking at 
patterns we could apply to the whole API.
>> Apparently we may also have native methods that have their stubs
>> compiled too.
>>     
>>> JavaStackFrame
>>> --------------
>>>
>>> (1)
>>> Usually, Java stack frames contain information about Java monitors held
>>>       
>> in this frame. This is necessary because Java spec requires "forced"
>> release of all monitors held by a method in case it "completes abruptly".
>> So something like
>>     
>>> List getHeldMonitors()
>>> would be really useful.
>>>
>>>
>>>       
>> I agree, having the API explictly show where monitors are held would
>> take people closer towards diagnosing deadlocks, as opposed to having
>> them read through their code and work out where they were taken. It may
>> not always be obvious from reading code which monitor has actually been
>> taken.
>>
>>     
>>> (2)
>>> API also needs to support 'split' stack frames on Itanium - every frame
>>>       
>> has a traditinal memory stack frame and additionally, a register stack
>> frame where Register Stack Engine (RSE) spills current register frame
>> during a call (and restores from upon return).
>>     
>>> Maybe this could be done via delegating responsibility for such support
>>>       
>> to an Itanium-specific implementaion of the spec?
>>     
>>>       
>> The API isn't very good at exposing the native information. With the
>> currently JavaThread.getStackSections() could return two ImageSections
>> if you are on IA64.
>> Certainly that's what we do for ImageThread.getStackSections() on IBM's
>> 1.4.2 IA64 Linux implementation.
>>     
>
> [Bobrovsky, Konstantin S] 
> Yes, this is a good catch. Stack is split on ia64 at the thread level too.
> Though, I meant slightly different thing: on IA64, we'd like to have information about where register stack for a particular stack frame (rather than a thread) starts in memory.
>
>   
[Monteith, S]
I can see how there'd be difficulties, there is certainly a case for 
having ImageSections for Java and Image StackFrames, to show where they 
are and their extent.
One thing that troubles me is how we'd handle the NaT bits - the 
cleanest option would be to leave an 8 byte whole where they've been 
spilled. Ideally
the NaT bits could be reconstructed for the spilled registers, but that 
is a distinctly odd feature of IA64.

>> One of the the unresolved problems I have with the API is how to relate
>> all of the different frame types. Just now Java frame could be on it's
>> own "Java stack",
>> on the native stack as interpreted or compiled code, could be a native
>> method with a stub calling a C/C++ function, and we don't help with this.
>> For example, if you have a stack like:
>>
>> C function frame
>> stub frame
>> native method frame
>> compiled method frame (on native stack)
>> interpreted method frame (on own stack)
>>
>> how do you derive that information from the Kato API? I don't think we can.
>>     
>
> [Bobrovsky, Konstantin S] 
> As I understand the question, the problem is in apparent complexity of writing a Kato implementation which would be able to iterate through various kinds of frames on the a thread's stack, and for each frame report its type, extent and other information. ?
>
> I think any Java runtime should be able to do this (Hotspot is), at least when collecting garbage. And if a Java runtime knows how to do this in a live VM, Java code (Kato implementation for this runtime) which does this for a dead VM should look similar.
>   
[Monteith, S]
See my comment above. It's not so much the implementation complexity I 
was worried about, but rather how we represent it in the API. We 
currently don't
provide a way, that I can see, of sensibly communicating the layout of 
the stack.
>   
>>> (3)
>>> while getLocation() can provide information about what method this frame
>>>       
>> belongs to, this method may have several compiled or interpreted
>> "incarnations". So, I would suggest to add the following:
>>     
>>> boolean isCompiledFrame()
>>> boolean isInterpretedFrame()
>>>
>>> /** @return an ImageSection containing the compiled code to which this
>>>       
>> stack frame corresponds */
>>     
>>> ImageSection getCode()
>>>
>>>
>>>       
>> I agree with this, there can be several incarnations of a method
>> inflight that we aren't presenting. Contrary to your suggestion, should
>> we make it easy to
>> get all java method incarnations? I might suggest that we have:
>>
>> List<JavaMethodIncarnation> JavaMethod.getIncarnations()
>>
>> JavaMethodIncarnation JavaLocation.getMethodIncarnation()
>>
>> In JavaMethodIncarnation:
>>    JavaMethod getMethod()
>>    boolean isJITCompiled()
>>    boolean isNative()
>>    int compilationLevel()
>>    List<ImageSection> getSections();
>>
>>     
>
> [Bobrovsky, Konstantin S]
> I think this is an interesting idea overall. But not solve the <stack frame> -> <method incarnation> mapping problem there should additionally be JavaMethodIncarnation JavaStackFrame.getMethodIncarnation()
>
>   
[Monteith, S]
Yes, it's important to match them up.
> Here is another angle to look from.
> compilationLevel() and getSections() don't apply to an incarnation of a native method (which will always have one incarnation), or to an interpreted incarnation of a usual method. What will only be used for such incarnations is getMethod(), isJITCompiled() and isNative(). So the API looks a little 'irregular' to me. It seems the 'incarnation' semantics is more close to 'compiled code incarnation' of a particular method. So I would develop the suggested API into a little more specific one:
>
> JavaMethod JavaStackFrame.getMethod()
> boolean JavaStackFrame.isCompiled()
> boolean JavaStackFrame.isInterpreted()  
> boolean JavaStackFrame.isNative()  
>
> // returns a non-null value only if isCompiled() is true
> CompiledMethodIncarnation JavaStackFrame.getCompiledMethodIncarnation()
>
> interface CompiledMethodIncarnation {
>     JavaMethod getMethod();
>     int compilationLevel()
>     List<ImageSection> getSections();
>     int getBytecodeIndexForEntryPoint(); // repeats your suggestion below
>     ...
> }
>
> And all the details for native and interpreted stack frames (which can have only one 'incarnation') will be queried from the returned JavaMethod - such as bytecode, function address, etc.
>
>   
[Monteith, S]
That looks good. It leaves a simple route for more basic uses (i.e. 
getting the name of the method and it's class), and it is sufficiently 
general to cover the various JVM implementations.
>>> (4)
>>> note that sometimes methods are compiled with the entry point being not
>>>       
>> the first method's bytecode (compilation for so-called on-stack-replacement
>> or OSR). So each compiled version of a method should also be annotated with
>> bytecode index where the entry point for this compilation is. In this
>> sense, ImageSection is not enough to describe a compiled code.
>>     
>>>       
>> Sure,
>>    ImagePointer JavaMethodIncarnation.getEntryPoint()
>> ?
>>
>>     
>
> [Bobrovsky, Konstantin S] 
> Yes, this approach looks good to me, with a slight change I described above.
>
>   
>> JavaMethodIncarnation is a placeholder for perhaps a more sensible name.
>>     
>>> -----Original Message-----
>>> From: Steve Poole [mailto:spoole167@googlemail.com]
>>> Sent: Sunday, July 26, 2009 12:17 AM
>>> To: kato-spec@incubator.apache.org
>>> Subject: New draft versin of spec
>>>
>>> I've just posted to the wiki a docbook based version of the
>>>       
>> specification.
>>     
>>> Still needs work but a lot better than it was.
>>>
>>>
>>> http://cwiki.apache.org/confluence/download/attachments/2326547/spec-edr-
>>>       
>> 25july2009.pdf
>>     
>>> Cheers
>>>
>>> Steve
>>>
>>>       
>> --
>> Stuart Monteith
>> http://blog.stoo.me.uk/
>>     
>
>   

Thanks,
    Stuart

-- 
Stuart Monteith
http://blog.stoo.me.uk/

RE: New draft versin of spec

Posted by "Bobrovsky, Konstantin S" <ko...@intel.com>.
Hi Stuart,

Some more of my comments are also inline below (marked with '[Bobrovsky, Konstantin S]' - that's how Outlook helps :-) )

Thanks,
Konst
 
Intel Novosibirsk
Closed Joint Stock Company Intel A/O
Registered legal address: Krylatsky Hills Business Park, 
17 Krylatskaya Str., Bldg 4, Moscow 121614, 
Russian Federation
 

>-----Original Message-----
>From: Stuart Monteith [mailto:stukato@stoo.me.uk]
>Sent: Monday, July 27, 2009 11:26 PM
>To: kato-spec@incubator.apache.org
>Subject: Re: New draft versin of spec
>
>Hi Konstantin,
>    Thanks for you comments, I've added mine inline below.
>
>Regards,
>    Stuart
>
>Bobrovsky, Konstantin S wrote:
>> Hi Steve, all,
>>
>> FYI: I added some comments to the Kato javadoc @
>http://cwiki.apache.org/confluence/display/KATO/JavaDoc+comments. I'm
>resending them to this list for convenience.
>>
>> Thanks,
>> Konst
>>
>> Intel Novosibirsk
>> Closed Joint Stock Company Intel A/O
>> Registered legal address: Krylatsky Hills Business Park,
>> 17 Krylatskaya Str., Bldg 4, Moscow 121614,
>> Russian Federation
>>
>> Top-level
>> =========
>>
>> (1) kato.java seems a specialization of kato.runtime, so
>kato.runtime.java could be used instead.
>>
>>
>+1
>
>> (2) I noticed that there are API elements in the form of List
>get[SomeEntityName]() and javadoc does not specify objects of which type
>will be returned in the resulting list. E.g. ImageThread.getStackFrames().
>>
>>
>I'll fix these up when I come across them. I've been reviewing the Java
>runtime.
>
>> (3) It would be nice to have an opportunity to create an Image* object
>from a pointer into an image (having ImagePointer instance which describes
>the start of the Image* entity)
>>
>>
>Can you give an example? I can see how this might be useful for the java
>API, but I don't see how it'd be useful for image.
>

[Bobrovsky, Konstantin S] 
e.g.
ImageStackFrame ImageFactory.getImageStackFrameAt(ImagePointer addr)
ImageThread ImageFactory.getImageThreadAt(ImagePointer addr)

And yes, this would be useful for the kato.java API too - I agree.

>> org.apache.kato.image
>> =====================
>>
>> ImageModule
>> -----------
>>
>> There should also be a way to get the address the module was loaded at
>and, if possible, its size.
>>
>>
>I think this is a JavaDoc issue. ImageModule.getSections() returns
>ImageSections that contain the information you are asking for. It is not
>clear from
>the doc that this is the case.
>> ImageRegister
>> -------------
>>
>> register names for different CPUs should probably follow the naming
>convention adopted in the official specification of this CPU. This way
>tools would work uniformly with different Kato implementations.
>>
>>
>I think having us specify the register set for every possible processor
>architecture is inappropriate, so I agree.
>> 'Long' or 'Integer' might be too vague for specifying register type. How
>about '64 bit' or '32 bit' plus an additional characteristic such as
>'general purpose/floating point/application/flags/...' ?
>>
>The "Type" information is there to describe which subclass of Number
>will be returned by ImageRegister.getValue(). But you're right, it
>doesn't actually describe
>what the register contains.
>> The register sets specified here should be complete, otherwise we could
>get implementation/tool incompatibilites for the registers missing from
>this javadoc. One of the alternative ways for specifying them could be just
>a link to the corresponding external specification.
>>
>>
>I have to agree with linking to the external specification. There has to
>be at least some agreement to the set of registers available.
>> org.apache.kato.java
>> ====================
>>
>> JavaReference
>> -------------
>>
>> (1)
>> sometimes it is very helpful to know where the heap root actually resides
>- the corresponding CPU register and/or memory address (in a memory stack
>frame or register stack frame), and this info is usually available in Java
>runtimes.
>>
>>
>So, this would be something like: ImagePointer
>JavaReference.getAddress() and ImageRegister
>JavaReference.getRegister(), with a boolean for both?
>

[Bobrovsky, Konstantin S] 
getAddress() and getRegister() look satisfactory. What the boolean is for?
Can getRegister() return just null if the reference resides in memory only?

>> (2)
>> It seems the API here lacks a REFERENCE_DIRIVED reference type to
>describe derived references (pointers), which point to an interrior of a
>Java object - to an array element, for example. Derived references are
>reported to Garbage Collector in pairs with references to their base Java
>objects.
>>
>>
>>
>The JavaReference types are the same as what you would get from JVMTI,
>so that's why we have that limited list.

[Bobrovsky, Konstantin S] 
So it seems JVMTI does not fully cover the variety of references.

>If I were to implement this, I would have had  the reference just be
>like any other reference to a field or element.
>But if we were to expand it, I'd suggest something like the following:
>
>    REFERENCE_DERIVED_ARRAY_ELEMENT
>    REFERENCE_DERIVED_OBJECT_FIELD
>    REFERENCE_DERIVED_CLASS_FIELD
>

[Bobrovsky, Konstantin S] This looks good to me.

>
>> JavaRuntime
>> -----------
>>
>> (1) We should have somewhere in the API means to iterate trough generated
>code sequences other than managed methods - so-called 'stubs'. Each stub is
>similar to a method: it has a name, (possibly, multiple) code sections.
>This interface (JavaRuntime) seems an appropriate place for this API.

[Bobrovsky, Konstantin S] 
(To develop this thought a bit...)
Maybe we could have
  interface CodeStub {
    String getName();
    ImageSection getCodeSections();
  }

And
List JavaRuntime.getCodeStubs();

>>
>>
>> (2) Not all methods within a Java runtime are necessarily compiled. Many
>may remain interpreted, others might not even have a bytecode. So I would
>suggest to replace 'getCompiledMethods' with smth like 'getJavaMethods' or
>add 'getInterpretedMethods', 'getMethodsWithoutCode'.
>>
>>
>The intention isn't for all of the methods to be compiled, the method
>will just list those methods that are compiled. I would argue for us
>actually removing the JavaRuntime.getCompiledMethods() if we don't have
>a reason for including it.

[Bobrovsky, Konstantin S] 
I thought of this method as an iterator through all java methods known to the system - its javadoc implies this: 'This is short cut method. The same result can be found by iterating over all methods in all class loaders in all classes'. This iteration will also yield methods which are not compiled.

So we can have all
  getCompiledMethods()
  getInterpretedMethods()
  getMethodsWithoutCode()
and
  getJavaMethods()

or just
  List getJavaMethods(MethodFilter f)

where f defines the acceptance criteria.

>
>Apparently we may also have native methods that have their stubs
>compiled too.
>> JavaStackFrame
>> --------------
>>
>> (1)
>> Usually, Java stack frames contain information about Java monitors held
>in this frame. This is necessary because Java spec requires "forced"
>release of all monitors held by a method in case it "completes abruptly".
>So something like
>> List getHeldMonitors()
>> would be really useful.
>>
>>
>I agree, having the API explictly show where monitors are held would
>take people closer towards diagnosing deadlocks, as opposed to having
>them read through their code and work out where they were taken. It may
>not always be obvious from reading code which monitor has actually been
>taken.
>
>> (2)
>> API also needs to support 'split' stack frames on Itanium - every frame
>has a traditinal memory stack frame and additionally, a register stack
>frame where Register Stack Engine (RSE) spills current register frame
>during a call (and restores from upon return).
>>
>> Maybe this could be done via delegating responsibility for such support
>to an Itanium-specific implementaion of the spec?
>>
>>
>The API isn't very good at exposing the native information. With the
>currently JavaThread.getStackSections() could return two ImageSections
>if you are on IA64.
>Certainly that's what we do for ImageThread.getStackSections() on IBM's
>1.4.2 IA64 Linux implementation.

[Bobrovsky, Konstantin S] 
Yes, this is a good catch. Stack is split on ia64 at the thread level too.
Though, I meant slightly different thing: on IA64, we'd like to have information about where register stack for a particular stack frame (rather than a thread) starts in memory.

>
>One of the the unresolved problems I have with the API is how to relate
>all of the different frame types. Just now Java frame could be on it's
>own "Java stack",
>on the native stack as interpreted or compiled code, could be a native
>method with a stub calling a C/C++ function, and we don't help with this.
>For example, if you have a stack like:
>
>C function frame
>stub frame
>native method frame
>compiled method frame (on native stack)
>interpreted method frame (on own stack)
>
>how do you derive that information from the Kato API? I don't think we can.

[Bobrovsky, Konstantin S] 
As I understand the question, the problem is in apparent complexity of writing a Kato implementation which would be able to iterate through various kinds of frames on the a thread's stack, and for each frame report its type, extent and other information. ?

I think any Java runtime should be able to do this (Hotspot is), at least when collecting garbage. And if a Java runtime knows how to do this in a live VM, Java code (Kato implementation for this runtime) which does this for a dead VM should look similar.

>
>> (3)
>> while getLocation() can provide information about what method this frame
>belongs to, this method may have several compiled or interpreted
>"incarnations". So, I would suggest to add the following:
>>
>> boolean isCompiledFrame()
>> boolean isInterpretedFrame()
>>
>> /** @return an ImageSection containing the compiled code to which this
>stack frame corresponds */
>> ImageSection getCode()
>>
>>
>I agree with this, there can be several incarnations of a method
>inflight that we aren't presenting. Contrary to your suggestion, should
>we make it easy to
>get all java method incarnations? I might suggest that we have:
>
>List<JavaMethodIncarnation> JavaMethod.getIncarnations()
>
>JavaMethodIncarnation JavaLocation.getMethodIncarnation()
>
>In JavaMethodIncarnation:
>    JavaMethod getMethod()
>    boolean isJITCompiled()
>    boolean isNative()
>    int compilationLevel()
>    List<ImageSection> getSections();
>

[Bobrovsky, Konstantin S]
I think this is an interesting idea overall. But not solve the <stack frame> -> <method incarnation> mapping problem there should additionally be JavaMethodIncarnation JavaStackFrame.getMethodIncarnation()

Here is another angle to look from.
compilationLevel() and getSections() don't apply to an incarnation of a native method (which will always have one incarnation), or to an interpreted incarnation of a usual method. What will only be used for such incarnations is getMethod(), isJITCompiled() and isNative(). So the API looks a little 'irregular' to me. It seems the 'incarnation' semantics is more close to 'compiled code incarnation' of a particular method. So I would develop the suggested API into a little more specific one:

JavaMethod JavaStackFrame.getMethod()
boolean JavaStackFrame.isCompiled()
boolean JavaStackFrame.isInterpreted()  
boolean JavaStackFrame.isNative()  

// returns a non-null value only if isCompiled() is true
CompiledMethodIncarnation JavaStackFrame.getCompiledMethodIncarnation()

interface CompiledMethodIncarnation {
    JavaMethod getMethod();
    int compilationLevel()
    List<ImageSection> getSections();
    int getBytecodeIndexForEntryPoint(); // repeats your suggestion below
    ...
}

And all the details for native and interpreted stack frames (which can have only one 'incarnation') will be queried from the returned JavaMethod - such as bytecode, function address, etc.

>> (4)
>> note that sometimes methods are compiled with the entry point being not
>the first method's bytecode (compilation for so-called on-stack-replacement
>or OSR). So each compiled version of a method should also be annotated with
>bytecode index where the entry point for this compilation is. In this
>sense, ImageSection is not enough to describe a compiled code.
>>
>>
>Sure,
>    ImagePointer JavaMethodIncarnation.getEntryPoint()
>?
>

[Bobrovsky, Konstantin S] 
Yes, this approach looks good to me, with a slight change I described above.

>JavaMethodIncarnation is a placeholder for perhaps a more sensible name.
>> -----Original Message-----
>> From: Steve Poole [mailto:spoole167@googlemail.com]
>> Sent: Sunday, July 26, 2009 12:17 AM
>> To: kato-spec@incubator.apache.org
>> Subject: New draft versin of spec
>>
>> I've just posted to the wiki a docbook based version of the
>specification.
>> Still needs work but a lot better than it was.
>>
>>
>> http://cwiki.apache.org/confluence/download/attachments/2326547/spec-edr-
>25july2009.pdf
>>
>> Cheers
>>
>> Steve
>>
>
>--
>Stuart Monteith
>http://blog.stoo.me.uk/

Re: New draft versin of spec

Posted by Stuart Monteith <st...@stoo.me.uk>.
Hi Konstantin,
    Thanks for you comments, I've added mine inline below.

Regards,
    Stuart

Bobrovsky, Konstantin S wrote:
> Hi Steve, all,
>
> FYI: I added some comments to the Kato javadoc @ http://cwiki.apache.org/confluence/display/KATO/JavaDoc+comments. I'm resending them to this list for convenience.
>
> Thanks,
> Konst
>  
> Intel Novosibirsk
> Closed Joint Stock Company Intel A/O
> Registered legal address: Krylatsky Hills Business Park, 
> 17 Krylatskaya Str., Bldg 4, Moscow 121614, 
> Russian Federation
>  
> Top-level
> =========
>
> (1) kato.java seems a specialization of kato.runtime, so kato.runtime.java could be used instead.
>
>   
+1

> (2) I noticed that there are API elements in the form of List get[SomeEntityName]() and javadoc does not specify objects of which type will be returned in the resulting list. E.g. ImageThread.getStackFrames().
>
>   
I'll fix these up when I come across them. I've been reviewing the Java 
runtime.

> (3) It would be nice to have an opportunity to create an Image* object from a pointer into an image (having ImagePointer instance which describes the start of the Image* entity)
>
>   
Can you give an example? I can see how this might be useful for the java 
API, but I don't see how it'd be useful for image.

> org.apache.kato.image
> =====================
>
> ImageModule
> -----------
>
> There should also be a way to get the address the module was loaded at and, if possible, its size.
>
>   
I think this is a JavaDoc issue. ImageModule.getSections() returns 
ImageSections that contain the information you are asking for. It is not 
clear from
the doc that this is the case.
> ImageRegister
> -------------
>
> register names for different CPUs should probably follow the naming convention adopted in the official specification of this CPU. This way tools would work uniformly with different Kato implementations.
>
>   
I think having us specify the register set for every possible processor 
architecture is inappropriate, so I agree.
> 'Long' or 'Integer' might be too vague for specifying register type. How about '64 bit' or '32 bit' plus an additional characteristic such as 'general purpose/floating point/application/flags/...' ?
>   
The "Type" information is there to describe which subclass of Number 
will be returned by ImageRegister.getValue(). But you're right, it 
doesn't actually describe
what the register contains.
> The register sets specified here should be complete, otherwise we could get implementation/tool incompatibilites for the registers missing from this javadoc. One of the alternative ways for specifying them could be just a link to the corresponding external specification.
>
>   
I have to agree with linking to the external specification. There has to 
be at least some agreement to the set of registers available.
> org.apache.kato.java
> ====================
>
> JavaReference
> -------------
>
> (1)
> sometimes it is very helpful to know where the heap root actually resides - the corresponding CPU register and/or memory address (in a memory stack frame or register stack frame), and this info is usually available in Java runtimes.
>
>   
So, this would be something like: ImagePointer 
JavaReference.getAddress() and ImageRegister 
JavaReference.getRegister(), with a boolean for both?

> (2)
> It seems the API here lacks a REFERENCE_DIRIVED reference type to describe derived references (pointers), which point to an interrior of a Java object - to an array element, for example. Derived references are reported to Garbage Collector in pairs with references to their base Java objects.
>
>
>   
The JavaReference types are the same as what you would get from JVMTI, 
so that's why we have that limited list.
If I were to implement this, I would have had  the reference just be 
like any other reference to a field or element.
But if we were to expand it, I'd suggest something like the following:

    REFERENCE_DERIVED_ARRAY_ELEMENT
    REFERENCE_DERIVED_OBJECT_FIELD
    REFERENCE_DERIVED_CLASS_FIELD


> JavaRuntime
> -----------
>
> (1) We should have somewhere in the API means to iterate trough generated code sequences other than managed methods - so-called 'stubs'. Each stub is similar to a method: it has a name, (possibly, multiple) code sections. This interface (JavaRuntime) seems an appropriate place for this API.
>
>   
> (2) Not all methods within a Java runtime are necessarily compiled. Many may remain interpreted, others might not even have a bytecode. So I would suggest to replace 'getCompiledMethods' with smth like 'getJavaMethods' or add 'getInterpretedMethods', 'getMethodsWithoutCode'.
>
>   
The intention isn't for all of the methods to be compiled, the method 
will just list those methods that are compiled. I would argue for us 
actually removing the JavaRuntime.getCompiledMethods() if we don't have 
a reason for including it.

Apparently we may also have native methods that have their stubs 
compiled too.
> JavaStackFrame
> --------------
>
> (1)
> Usually, Java stack frames contain information about Java monitors held in this frame. This is necessary because Java spec requires "forced" release of all monitors held by a method in case it "completes abruptly". So something like
> List getHeldMonitors()
> would be really useful.
>
>   
I agree, having the API explictly show where monitors are held would 
take people closer towards diagnosing deadlocks, as opposed to having 
them read through their code and work out where they were taken. It may 
not always be obvious from reading code which monitor has actually been 
taken.

> (2)
> API also needs to support 'split' stack frames on Itanium - every frame has a traditinal memory stack frame and additionally, a register stack frame where Register Stack Engine (RSE) spills current register frame during a call (and restores from upon return).
>
> Maybe this could be done via delegating responsibility for such support to an Itanium-specific implementaion of the spec?
>
>   
The API isn't very good at exposing the native information. With the 
currently JavaThread.getStackSections() could return two ImageSections 
if you are on IA64.
Certainly that's what we do for ImageThread.getStackSections() on IBM's 
1.4.2 IA64 Linux implementation.

One of the the unresolved problems I have with the API is how to relate 
all of the different frame types. Just now Java frame could be on it's 
own "Java stack",
on the native stack as interpreted or compiled code, could be a native 
method with a stub calling a C/C++ function, and we don't help with this.
For example, if you have a stack like:
 
C function frame
stub frame
native method frame
compiled method frame (on native stack)
interpreted method frame (on own stack)

how do you derive that information from the Kato API? I don't think we can.

> (3)
> while getLocation() can provide information about what method this frame belongs to, this method may have several compiled or interpreted "incarnations". So, I would suggest to add the following:
>
> boolean isCompiledFrame()
> boolean isInterpretedFrame()
>
> /** @return an ImageSection containing the compiled code to which this stack frame corresponds */
> ImageSection getCode()
>
>   
I agree with this, there can be several incarnations of a method 
inflight that we aren't presenting. Contrary to your suggestion, should 
we make it easy to
get all java method incarnations? I might suggest that we have:

List<JavaMethodIncarnation> JavaMethod.getIncarnations()

JavaMethodIncarnation JavaLocation.getMethodIncarnation()

In JavaMethodIncarnation:
    JavaMethod getMethod()
    boolean isJITCompiled()
    boolean isNative()
    int compilationLevel()
    List<ImageSection> getSections();

> (4)
> note that sometimes methods are compiled with the entry point being not the first method's bytecode (compilation for so-called on-stack-replacement or OSR). So each compiled version of a method should also be annotated with bytecode index where the entry point for this compilation is. In this sense, ImageSection is not enough to describe a compiled code.
>
>   
Sure,
    ImagePointer JavaMethodIncarnation.getEntryPoint()
?

JavaMethodIncarnation is a placeholder for perhaps a more sensible name.
> -----Original Message-----
> From: Steve Poole [mailto:spoole167@googlemail.com] 
> Sent: Sunday, July 26, 2009 12:17 AM
> To: kato-spec@incubator.apache.org
> Subject: New draft versin of spec
>
> I've just posted to the wiki a docbook based version of the specification.
> Still needs work but a lot better than it was.
>
>
> http://cwiki.apache.org/confluence/download/attachments/2326547/spec-edr-25july2009.pdf
>
> Cheers
>
> Steve
>   

-- 
Stuart Monteith
http://blog.stoo.me.uk/

RE: New draft versin of spec

Posted by "Bobrovsky, Konstantin S" <ko...@intel.com>.
Hi Steve, all,

FYI: I added some comments to the Kato javadoc @ http://cwiki.apache.org/confluence/display/KATO/JavaDoc+comments. I'm resending them to this list for convenience.

Thanks,
Konst
 
Intel Novosibirsk
Closed Joint Stock Company Intel A/O
Registered legal address: Krylatsky Hills Business Park, 
17 Krylatskaya Str., Bldg 4, Moscow 121614, 
Russian Federation
 
Top-level
=========

(1) kato.java seems a specialization of kato.runtime, so kato.runtime.java could be used instead.

(2) I noticed that there are API elements in the form of List get[SomeEntityName]() and javadoc does not specify objects of which type will be returned in the resulting list. E.g. ImageThread.getStackFrames().

(3) It would be nice to have an opportunity to create an Image* object from a pointer into an image (having ImagePointer instance which describes the start of the Image* entity)

org.apache.kato.image
=====================

ImageModule
-----------

There should also be a way to get the address the module was loaded at and, if possible, its size.


ImageRegister
-------------

register names for different CPUs should probably follow the naming convention adopted in the official specification of this CPU. This way tools would work uniformly with different Kato implementations.

'Long' or 'Integer' might be too vague for specifying register type. How about '64 bit' or '32 bit' plus an additional characteristic such as 'general purpose/floating point/application/flags/...' ?

The register sets specified here should be complete, otherwise we could get implementation/tool incompatibilites for the registers missing from this javadoc. One of the alternative ways for specifying them could be just a link to the corresponding external specification.


org.apache.kato.java
====================

JavaReference
-------------

(1)
sometimes it is very helpful to know where the heap root actually resides - the corresponding CPU register and/or memory address (in a memory stack frame or register stack frame), and this info is usually available in Java runtimes.

(2)
It seems the API here lacks a REFERENCE_DIRIVED reference type to describe derived references (pointers), which point to an interrior of a Java object - to an array element, for example. Derived references are reported to Garbage Collector in pairs with references to their base Java objects.


JavaRuntime
-----------

(1) We should have somewhere in the API means to iterate trough generated code sequences other than managed methods - so-called 'stubs'. Each stub is similar to a method: it has a name, (possibly, multiple) code sections. This interface (JavaRuntime) seems an appropriate place for this API.

(2) Not all methods within a Java runtime are necessarily compiled. Many may remain interpreted, others might not even have a bytecode. So I would suggest to replace 'getCompiledMethods' with smth like 'getJavaMethods' or add 'getInterpretedMethods', 'getMethodsWithoutCode'.


JavaStackFrame
--------------

(1)
Usually, Java stack frames contain information about Java monitors held in this frame. This is necessary because Java spec requires "forced" release of all monitors held by a method in case it "completes abruptly". So something like
List getHeldMonitors()
would be really useful.

(2)
API also needs to support 'split' stack frames on Itanium - every frame has a traditinal memory stack frame and additionally, a register stack frame where Register Stack Engine (RSE) spills current register frame during a call (and restores from upon return).

Maybe this could be done via delegating responsibility for such support to an Itanium-specific implementaion of the spec?

(3)
while getLocation() can provide information about what method this frame belongs to, this method may have several compiled or interpreted "incarnations". So, I would suggest to add the following:

boolean isCompiledFrame()
boolean isInterpretedFrame()

/** @return an ImageSection containing the compiled code to which this stack frame corresponds */
ImageSection getCode()

(4)
note that sometimes methods are compiled with the entry point being not the first method's bytecode (compilation for so-called on-stack-replacement or OSR). So each compiled version of a method should also be annotated with bytecode index where the entry point for this compilation is. In this sense, ImageSection is not enough to describe a compiled code.


-----Original Message-----
From: Steve Poole [mailto:spoole167@googlemail.com] 
Sent: Sunday, July 26, 2009 12:17 AM
To: kato-spec@incubator.apache.org
Subject: New draft versin of spec

I've just posted to the wiki a docbook based version of the specification.
Still needs work but a lot better than it was.


http://cwiki.apache.org/confluence/download/attachments/2326547/spec-edr-25july2009.pdf

Cheers

Steve