Re: Thoughts...

[Tim Ellison <t....@gmail.com>]

Vasily Levchenko wrote:
> Originally,  Java developers using JNI have to debug their application using
> two debugger for accessing java and native world(
> http://www-128.ibm.com/developerworks/java/library/j-jnidebug/index.html,
> http://developers.sun.com/learning/javaoneonline/2006/tools/TS-1011.pdf).
> This concept should be the foundation for "mixed" debugger. Native debugger
> (gdb, adb or other) to debug java world need information how to deal with
> java stack, other jdb functionality comes almost automatically.

I understand the issue well enough; it's been a wish for years when
running 'managed' and native code to have a debugger that can cross the
boundary seamlessly, so I'm very excited to see this is being addressed.

> Let me start
> from the final stage of the good "mixed  mode debugger". Ideal mixed mode
> debugger should open core or dump file of java application  crush  restoring
> internal state like it do for other application, it should attach and start
> application in debugging mode. If look on debuggers' history we can see
> similar task that was done before, e.g. thread handling in debugger. How
> debugger operates with thread? How it get TSD and TLS from the thread on
> remote application or core file? A long-long time ago Solaris or Sun OS
> introduced thread debugging interface libc_db
> http://docs.sun.com/app/docs/doc/816-5173/6mbb8adsh?l=en&a=view and then it
> migrated to Linux world (
> http://sources.redhat.com/cgi-bin/cvsweb.cgi/libc/nptl_db/?cvsroot=glibc)
> and BSD (http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/libpthread_dbg/) world
> (thread_db). This library inform debugger how to fetch thread information
> from core file or debugee process. Similar approach could be done with java
> world but before we need several baby steps.

Agreed, and this is going to be highly VM-specific, i.e. you likely want
to poke about in the VM data structures for crash diagnosis, so expect
it to be a build-specific tool.

> Short-term:
> 1. Using JVMTI possibilities provide information for debugger to operate
> with the java stack
> 2. Provide debugger operations throw JVMTI (break points, watch points and
> so on)
> 
> Result construction could used as a source for a long solution described as
> ideal mixed-mode-debugger.
> 
> BTW: here some links about ready to use short-term solutions:
> http://www.hp.com/products1/unix/java/pdfs/07_06Tools_NativeDebug.pdf
> http://docs.sun.com/source/819-3683/Java_debug.html

So I was just suggesting that, based on what I read on this thread, some
people wanted to discuss the topic before deciding on a vote to accept
this contribution.  I'm not suggesting the code would/would not be
accepted, but its usually better to discuss the viability before the
vote rather than afterwards.

Regards,
Tim

> On 10/12/07, Tim Ellison <t....@gmail.com> wrote:
>> It looks like there is some good discussion going on around this topic,
>> and possible alternatives being explored.  Rather than prematurely
>> forcing the issue with a vote, maybe we should stop the vote and
>> continue the discussion for a while until we reach a consensus, then
>> restart the vote.
>>
>> What do you think?
>>
>> Regards,
>> Tim
>>
>> Vasily Levchenko wrote:
>>> On 10/12/07, Alexei Fedotov <al...@gmail.com> wrote:
>>>> Vasily,
>>>>
>>>> You wrote,
>>>>> As I understand my project is single user of NCAI
>>>> I believe this is not the only use case for NCAI. Keeping internal
>>>> layer of debugging interfaces is important for VM development itself.
>>>
>>> Why are you so sure that it will help you in internal VM development?
>> you
>>> can used it for building some kind viewer of state in JNI wrapper, but
>> not
>>> more. Your "viewer" can't get debugging services provided by kernel.
>> You're
>>> breaking the model that people build for a long time ago and develop new
>> one
>>> instead.  You haven't (not personally you but harmony community I guess)
>>> even try  re-use  the existing solutions.
>>>
>>>
>>> Let me assure you that I'm far from convincing you to support NCAI in
>>>> your stepchild tool if you don't want to.
>>>>
>>>> Thanks.
>>>>
>>>> On 10/12/07, Vasily Levchenko < vasily.v.levchenko@gmail.com> wrote:
>>>>> On 10/12/07, Alexei Fedotov <al...@gmail.com> wrote:
>>>>>> Vasily,
>>>>>>
>>>>>> 1.
>>>>>> You use capitalized wordings like "Mixed Mode" debugging or
>>>>>> "Integrated Debugger was transferred to me". This sound like names of
>>>>>> internal projects you are participating. It would be great if you
>>>>>> start a new thread and disclose your activity to the community (and
>>>>>> your humble servant). Or if they are top secret projects, I have
>> never
>>>>>> seen these references on the dev list. :-) Thank you in advance.
>>>>>
>>>>> Yes it's public available semi-product that available on
>>>>> http://whatif.intel.com, and actually I'm the owner of Integrated
>>>> debugger
>>>>> for Java/JNI environments. As I understand my project is single user
>> of
>>>> NCAI
>>>>> and will available during Q4.
>>>>>
>>>>>
>>>>> 2.
>>>>>> Sorry for being unaware of projects like libthread_db.so. I've tried
>>>>>> to google it and the first search item didn't describe this project
>>>>>> well:
>>>>>>> 'Crashes in /lib/libthread_db.so.1' thread - MARC
>>>>>> Could you please support your words with references to these
>>>>>> alternative projects and their licenses?
>>>>> libthread_db.so is usual part of system libraries e.g. for GLIbc is
>> LGPL
>>>>> in BSD world  it's  BSDL and so on.
>>>>>
>>>>>
>>>>> 3.
>>>>>> In any case I believe we need to take the code now, and than compare
>>>>>> it with alternative when it is ready. Technologies quickly change and
>>>>>> replace each other, and I don't think we should reject a contribution
>>>>>> counting on its possible future obsolescence.
>>>>>
>>>>> I don't really undestand what harry for?
>>>>>
>>>>>
>>>>> Contrary, our mission is to make today's code a good base for the new
>>>>>> development.
>>>>> If you really want good code for development, read NetBSD source tree
>>>> and
>>>>> polish current code ;)
>>>>>
>>>>>
>>>>> Any code will become obsolete one day. I believe
>>>>>> interfaces are the easiest thing to be replaced. Other things in this
>>>>>> implementation such as stack frame enumeration will live much longer
>>>>>> because there are not too many sane maniacs which are able to
>>>>>> understand all our calling conventions.
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>>
>>>>>> On 10/12/07, Vasily Levchenko < vasily.v.levchenko@gmail.com> wrote:
>>>>>>> On 10/12/07, Alexei Fedotov <al...@gmail.com> wrote:
>>>>>>>> Hello Vasily,
>>>>>>>>
>>>>>>>> Now I see your attitude towards this donation. :-) When I initally
>>>>>>>> looked through interface specification, I was not very supportive
>>>> and
>>>>>>>> replied with the same argument about gdb.
>>>>>>>>
>>>>>>>> Unfortunately, gdb is not BSD-licensed or Apache compatible. So we
>>>>>>>> cannot link with it. It is important if we want to debug
>>>> application
>>>>>>>> in a JVMTI-like way from the same process.
>>>>>>> It's really doesn't matter you can find the sample of GDB agents
>>>> under
>>>>>> LGPL
>>>>>>> and BSD license, actually good example libthread_db.so that provide
>>>>>> access
>>>>>>> to libpthread.so internals fo GDB usually it's the same version as
>>>> the
>>>>>> rest
>>>>>>> of the system libraries.
>>>>>>>
>>>>>>>
>>>>>>> If you like solutions with external debuggers, you may come up with
>>>>>>>> one as well as Salikh did two years ago, and this proved to be
>>>> useful.
>>>>>>>> This just does not help in two areas I've initially outlined:
>>>>>>>> defensive programming and crash handling.
>>>>>>>
>>>>>>> It were actually one of the lack of good crash handling that we
>>>>>> discussed
>>>>>>> when the Integrated Debugger was transfered to me.
>>>>>>>
>>>>>>>
>>>>>>> Having a level of internal
>>>>>>>> reflection from inside VM is very important to express assertions
>>>>>>>> about correct VM state for a debug build.
>>>>>>>
>>>>>>> Yes, it's true but it should be done in similar way as thread_db
>>>>>> interface
>>>>>>> which could provide even core file analyzing, but with in-process
>>>>>> debugger
>>>>>>> it is impossible or very hard to implement.
>>>>>>>
>>>>>>>
>>>>>>> Finally I want to address your concern of a "pure viewing" nature of
>>>>>>>> NCAI. NCAI allows changes in runtime environment. For example, you
>>>> can
>>>>>>>> change values of variables, or any memory location.
>>>>>>>
>>>>>>> GDB or any other out-of-process debugger provides the same
>>>> functionality
>>>>>>> without any side-effects.
>>>>>>>
>>>>>>>
>>>>>>> Thanks.
>>>>>>>> On 10/12/07, Vasily Levchenko <va...@gmail.com>
>>>> wrote:
>>>>>>>>> On 10/12/07, Alexei Fedotov < alexei.fedotov@gmail.com> wrote:
>>>>>>>>>> +1
>>>>>>>>>> NCAI adds very useful architectural level for VM internal
>>>>>> debugging
>>>>>>>>>> and defensive programming. The best example is a crash
>>>> handler,
>>>>>> but
>>>>>>>>>> other utilities such us self-detection of deadlocks, is also
>>>> quite
>>>>>>>>>> interesting.
>>>>>>>>> NCAI could be interface to build some kind of viewer, but no
>>>> real
>>>>>>>> debugger.
>>>>>>>>> 1. It couldn't be very useful  just because it's single process
>>>>>>>> debugging
>>>>>>>>> model and if you set breakpoint incorrectly you could lock whole
>>>>>> JVM.
>>>>>>>>> 2. building debugger from the very beginning you try to invent
>>>> wheel
>>>>>>>> once
>>>>>>>>> again, you have to develop new mechanisms that already exists in
>>>>>> native
>>>>>>>>> debuggers.
>>>>>>>>> 3. building infrastructure for Mixed  Mode debugging using
>>>> native
>>>>>>>> debugger
>>>>>>>>> could be implemented in 1-2 month and it'll be really powerful
>>>> and
>>>>>>>> useful
>>>>>>>>> solution especially JNI developers.
>>>>>>>>>
>>>>>>>>> HP(gdb) and Sun (dbx) already introduced native debugger-centric
>>>>>>>> solutions
>>>>>>>>> for debugging in mixed environments. So NCAI seams to be
>>>> potential
>>>>>> dead
>>>>>>>> code
>>>>>>>>> and I won't be surprised if year or two later you'll be voting
>>>> for
>>>>>>>> removing
>>>>>>>>> this code.
>>>>>>>>>
>>>>>>>>> On 10/12/07, Vasily Levchenko <va...@gmail.com>
>>>> wrote:
>>>>>>>>>>> Hi Mikhail,
>>>>>>>>>>> Have you got any idea how to use it? Except "Integrated
>>>> Debugger
>>>>>> for
>>>>>>>>>>> Java/JNI environment" (former MMD).
>>>>>>>>>>>
>>>>>>>>>>> On 10/12/07, Mikhail Loenko < mloenko@gmail.com> wrote:
>>>>>>>>>>>> We now have all requisite paperwork in place for
>>>>>>>>>>>> http://issues.apache.org/jira/browse/HARMONY-4689, so
>>>> let's
>>>>>> vote
>>>>>>>> to
>>>>>>>>>>>> accept:
>>>>>>>>>>>>
>>>>>>>>>>>> [ ] +1 Accept this contribution
>>>>>>>>>>>> [ ] -1 Reject because...
>>>>>>>>>>>>
>>>>>>>>>>>> As usual, 3 days unless someone complains they need more
>>>> time.
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> --vvl
>>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> With best regards,
>>>>>>>>>> Alexei,
>>>>>>>>>> ESSD, Intel
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> --vvl
>>>>>>>>>
>>>>>>>> --
>>>>>>>> With best regards,
>>>>>>>> Alexei,
>>>>>>>> ESSD, Intel
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> --vvl
>>>>>>>
>>>>>> --
>>>>>> With best regards,
>>>>>> Alexei,
>>>>>> ESSD, Intel
>>>>>>
>>>>>
>>>>> --
>>>>> --vvl
>>>>>
>>>> --
>>>> With best regards,
>>>> Alexei,
>>>> ESSD, Intel
>>>>
>>>
>>>
> 
> 
>