Re: [jdktools][jdwp] Updated Java 6 JDWP contribution

[Gregory Shimansky <gs...@apache.org>]

This is very good news to me. I like to see this part of Harmony to continue 
its development since JDWP agent wasn't touched for quite some time since its 
contribution with just several bugs fixes by its original developers and me.

Could you give the details about what actually increased the performance in it 
and what kind of bugs were fixed?

On 29 April 2009 Oliver Deakin wrote:
> Hi all,
>
> A little while ago, IBM created a mirror of the Java 6 jdktools branch
> so that we could start bringing Harmony JDWP into the IBM Java releases.
> We began working in that mirror because it was convenient for our
> internal builds, with a plan to reflect all changes we made back into
> Harmony as we went. Unfortunately, as deadlines drew in and releases
> came and went those plans did not come into fruition - until now :)
>
> I'd like to announce the contribution of an enhanced Java 6 level JDWP
> agent and socket transport layer. It has been attached to HARMONY-6187
> [1] for everyone to try out - please take a look. It has the following
> enhancements:
>  - A large number of bug fixes.
>  - Ported to a wide variety of platforms - Linux x86/x86_64, Linux PPC
> 32/64, zLinux 31/64, Windows x86/x86_64, AIX PPC 32/64 and zOS 31/64.
>  - As part of the porting process, all non-portable C++ usage has been
> removed and rewritten, particularly use of C++ standard libraries.
>  - JSR 45 support implemented. The new JDWP agent allows debugging of
> non-Java stratum code running on the VM, for example JSPs.
>  - Significant performance improvements.
>
> All authors already have ACQs recorded. There may be some more work to
> be done to get the new JDWP functioning 100% with Harmony, but I'd
> rather do that work out in the open once it is committed. Please let me
> know if there are any objections/comments to this contribution! I'll
> give it a few days and then start a vote.
>
> Regards,
> Oliver
>
> [1] https://issues.apache.org/jira/browse/HARMONY-6187



-- 
Gregory

Re: [jdktools][jdwp] Updated Java 6 JDWP contribution

[Regis <xu...@gmail.com>]

Oliver Deakin wrote:
> Hi Gregory,
> 
> Regis made most of the performance improvements, so I will let him 
> answer on that front. From the point of view of bugs, here are some of 
> the rough areas we made fixes in:
> - Stepping correctness - there were some bugs related to debug stepping, 
> particularly once we had JSR-45 support in and started debugging 
> non-Java stratum code. As an example, there were some scenarios where a 
> user would have to click step-over twice to move over a single line of 
> Java code. In non-Java code, there were cases where the line map would 
> not be interpreted correctly and we would end up "running away" as if we 
> had resumed the code rather than stepping.
> - A number of crashes were uncovered, many from timing issues which were 
> exposed as the performance increased. These generally came from using 
> pointers that had already been freed, double freeing a pointer or 
> similar memory access/addressing issues.
> - Some bugs related to JDWP returning incorrect values to queries. For 
> example, getting a count of all instances of a class when debugging in 
> Eclipse would often return an incorrect value, as (iirc) the agent was 
> holding references to objects that should have been released and made 
> available for garbage collection. The number returned would then appear 
> higher than it should because JDWP was still keeping instances alive 
> that were no longer referenced by the application.
> - Issues around socket behaviour varying from platform to platform 
> created a number of bugs.
> 
> I hope this gives some idea of the kind of bugs we fixed - some related 
> to debug behaviour correctness, some related to porting to other 
> platforms and some were more general programming bugs (e.g. using freed 
> pointer).
> 
> Regards,
> Oliver
> 
> Gregory Shimansky wrote:
>> This is very good news to me. I like to see this part of Harmony to 
>> continue its development since JDWP agent wasn't touched for quite 
>> some time since its contribution with just several bugs fixes by its 
>> original developers and me.
>>
>> Could you give the details about what actually increased the 
>> performance in it and what kind of bugs were fixed?
>>
>> On 29 April 2009 Oliver Deakin wrote:
>>  
>>> Hi all,
>>>
>>> A little while ago, IBM created a mirror of the Java 6 jdktools branch
>>> so that we could start bringing Harmony JDWP into the IBM Java releases.
>>> We began working in that mirror because it was convenient for our
>>> internal builds, with a plan to reflect all changes we made back into
>>> Harmony as we went. Unfortunately, as deadlines drew in and releases
>>> came and went those plans did not come into fruition - until now :)
>>>
>>> I'd like to announce the contribution of an enhanced Java 6 level JDWP
>>> agent and socket transport layer. It has been attached to HARMONY-6187
>>> [1] for everyone to try out - please take a look. It has the following
>>> enhancements:
>>>  - A large number of bug fixes.
>>>  - Ported to a wide variety of platforms - Linux x86/x86_64, Linux PPC
>>> 32/64, zLinux 31/64, Windows x86/x86_64, AIX PPC 32/64 and zOS 31/64.
>>>  - As part of the porting process, all non-portable C++ usage has been
>>> removed and rewritten, particularly use of C++ standard libraries.
>>>  - JSR 45 support implemented. The new JDWP agent allows debugging of
>>> non-Java stratum code running on the VM, for example JSPs.
>>>  - Significant performance improvements.
>>>
>>> All authors already have ACQs recorded. There may be some more work to
>>> be done to get the new JDWP functioning 100% with Harmony, but I'd
>>> rather do that work out in the open once it is committed. Please let me
>>> know if there are any objections/comments to this contribution! I'll
>>> give it a few days and then start a vote.
>>>
>>> Regards,
>>> Oliver
>>>
>>> [1] https://issues.apache.org/jira/browse/HARMONY-6187
>>>     
>>
>>
>>
>>   
> 

There are several performance improvements. Some could be treated as bugs, for 
example, agent thread didn't manage JNI local reference (slow down jvmti call 
which need to allocate memeory on java stack frame), thread wasn't removed from 
thread list (slow down travelling list).

The most of improvements focus on memory manage and I/O:
- Use one worker thread to handle asynchronized command to avoid to start agent 
thread every time
- Reduce times to read/write socket
- Use pre-allocated buffer for reading/writing data from/to socket to avoid to 
allocate memory every time
- Optimize memory manage of PacketParser

-- 
Best Regards,
Regis.

Re: [jdktools][jdwp] Updated Java 6 JDWP contribution

[Oliver Deakin <ol...@googlemail.com>]

Hi Gregory,

Regis made most of the performance improvements, so I will let him 
answer on that front. From the point of view of bugs, here are some of 
the rough areas we made fixes in:
 - Stepping correctness - there were some bugs related to debug 
stepping, particularly once we had JSR-45 support in and started 
debugging non-Java stratum code. As an example, there were some 
scenarios where a user would have to click step-over twice to move over 
a single line of Java code. In non-Java code, there were cases where the 
line map would not be interpreted correctly and we would end up "running 
away" as if we had resumed the code rather than stepping.
 - A number of crashes were uncovered, many from timing issues which 
were exposed as the performance increased. These generally came from 
using pointers that had already been freed, double freeing a pointer or 
similar memory access/addressing issues.
 - Some bugs related to JDWP returning incorrect values to queries. For 
example, getting a count of all instances of a class when debugging in 
Eclipse would often return an incorrect value, as (iirc) the agent was 
holding references to objects that should have been released and made 
available for garbage collection. The number returned would then appear 
higher than it should because JDWP was still keeping instances alive 
that were no longer referenced by the application.
 - Issues around socket behaviour varying from platform to platform 
created a number of bugs.

I hope this gives some idea of the kind of bugs we fixed - some related 
to debug behaviour correctness, some related to porting to other 
platforms and some were more general programming bugs (e.g. using freed 
pointer).

Regards,
Oliver

Gregory Shimansky wrote:
> This is very good news to me. I like to see this part of Harmony to continue 
> its development since JDWP agent wasn't touched for quite some time since its 
> contribution with just several bugs fixes by its original developers and me.
>
> Could you give the details about what actually increased the performance in it 
> and what kind of bugs were fixed?
>
> On 29 April 2009 Oliver Deakin wrote:
>   
>> Hi all,
>>
>> A little while ago, IBM created a mirror of the Java 6 jdktools branch
>> so that we could start bringing Harmony JDWP into the IBM Java releases.
>> We began working in that mirror because it was convenient for our
>> internal builds, with a plan to reflect all changes we made back into
>> Harmony as we went. Unfortunately, as deadlines drew in and releases
>> came and went those plans did not come into fruition - until now :)
>>
>> I'd like to announce the contribution of an enhanced Java 6 level JDWP
>> agent and socket transport layer. It has been attached to HARMONY-6187
>> [1] for everyone to try out - please take a look. It has the following
>> enhancements:
>>  - A large number of bug fixes.
>>  - Ported to a wide variety of platforms - Linux x86/x86_64, Linux PPC
>> 32/64, zLinux 31/64, Windows x86/x86_64, AIX PPC 32/64 and zOS 31/64.
>>  - As part of the porting process, all non-portable C++ usage has been
>> removed and rewritten, particularly use of C++ standard libraries.
>>  - JSR 45 support implemented. The new JDWP agent allows debugging of
>> non-Java stratum code running on the VM, for example JSPs.
>>  - Significant performance improvements.
>>
>> All authors already have ACQs recorded. There may be some more work to
>> be done to get the new JDWP functioning 100% with Harmony, but I'd
>> rather do that work out in the open once it is committed. Please let me
>> know if there are any objections/comments to this contribution! I'll
>> give it a few days and then start a vote.
>>
>> Regards,
>> Oliver
>>
>> [1] https://issues.apache.org/jira/browse/HARMONY-6187
>>     
>
>
>
>   

-- 
Oliver Deakin
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU