[jdktools] svn commit r479920 - build and src tree

[Ilya Neverov <il...@gmail.com>]

Hi Geir,

I eventually managed to build 'jdktools' created in your commit [1] in
context of harmony-hdk-r468731 but the 'test' target failed - will try
to resolve tomorrow.

I'd like to understand future evolution of the 'jdktools' - could you
please comment the items below.

1) Do we need separate jdk/lib/tools.jar and jdk/jre/lib/jretools.jar?
I guess it's too early to think about separate Harmony-jre bundle so
all jdktools can yet go into single tools.jar.

2) Do we need a possibility to build/test jdktools without building
classlib (using harmony-hdk bundles) ? This feature can be useful for
people who want to work with some tool sources only.

3) Do we need selective build feature like '-Dbuild.module= ' for
classlib. Again it will be very useful for people working for example
on jdwp/ module.

I'm going to proceed with current state of the jdktools/ sources to
provide features 2) and 3) and it would be nice to know your plans in
this area.

Thank you.
Ilya

[1]
-------------------
r479920 | geirm | 2006-11-28 11:58:10 +0600 (Tue, 28 Nov 2006) | 4 lines

copied the high-level build files from classlib
and modified to just build launcher and tools jars
--------------------

Re: [jdktools] svn commit r479920 - build and src tree

["Geir Magnusson Jr." <ge...@pobox.com>]

Ilya Neverov wrote:
> On 11/28/06, Geir Magnusson Jr. <ge...@pobox.com> wrote:
>>
>> Ilya Neverov wrote:
>> > Hi Geir,
>> >
>> > I eventually managed to build 'jdktools' created in your commit [1] in
>> > context of harmony-hdk-r468731 but the 'test' target failed - will try
>> > to resolve tomorrow.
>>
>> I don't think they work - it's still a work in progress.  I was going to
>> do a little more today and update to the list.  I was working last night
>> based on your patch, but was really just hacking around.
>>
>> I didn't understand some of what you were trying to accomplish in your
>> patch, so I figured I'd try some myself to grok how things come
>> together.  For example, I couldn't see why things were being done in
>> "working_jdktools", since in jdktools/trunk, we should have everything
>> setup, and use the switch trick in the federated build.
> 
> It doesn't matter now -  it was an attempt to prepare changes which
> can be commited at once.

No problem.

> 
> In my variant I tried to avoid multiple antcalls and to use importing
> as much as possible. The idea was to gather all 'rules' into files in
> the jdktools/make directory and to have a little property definition
> in each <module>/build.xml. I think now that antcalls are not a big
> issue for building jdktools so we can return to assessing build system
> later, when jdktools build takes long time to complete. We can live
> some time with current build model.

Right - I have just committed further changes, to be modeled on the 
classlib build approach.  We can build jdwp agent and transport .so's 
now with make rather than cpptask.

I want to use jdktools to *converge* on a better build system, so if you 
have suggestions on how to improve, great.  But I'm not sure that 
inventing a new way makes sense right now.  Maybe we can evolve JDK 
tools and bring those fixes back to classlib.

This has been a refresher course in classlib build and overall, I really 
like it. Things are clear, clean and determinstic (mostly...).  There 
are probably places for improvement, so we can work forward on that.

[SNIP]

>>
>> >
>> > 2) Do we need a possibility to build/test jdktools without building
>> > classlib (using harmony-hdk bundles) ? This feature can be useful for
>> > people who want to work with some tool sources only.
>>
>> yes, it's that way already -  the current mode requires a -Dhy.hdk=
>> param to ant to get it to work as I didnt' think i was far enough to
>> integrate yet.
> 
> Now I see this in ant files. I was confused with 'hy.hdk' and 'hy.jdk'
> names - first one is for HDK used as build context but second name is
> for 'target' directory. Does it make sense to rename 'hy.jdk' to
> 'target.dir'?  I can prepare the patch.

I'm unwinding that.  I was sticking with the naming convention because 
I've copied the structure from classlib, which is where the hy.<junk> 
comes from, but am asserting my individualism with "harmony.jdktools" :)

Right now we need to reference :

  - HDK (classlib) to get to some headers and such
  - DRLVM, to get to usable jvmti headers
  - common resource, to grab ecj and other jars for jdk/lib

and by default, things are set correctly if you use it from the 
federated build.  First build classlib, then drlvm, and then you can 
jdktools to your hearts delight.

I used "hy.jdk", but I think you're right - there should be a better 
naming convention.

I'm not a fan of target, as I think "target" should be used where we use 
"build" now - the place where artifacts are generated.

I'd like to suggest "jdk.skeleton" because we do have a :

    deploy/
       jdk/
         ...

structure, and we just drop the launcher, agent and transport so's for 
jdwp into jre/bin, the tools jars into jdk/lib....

> 
>>
>> My thought was that this would be designed for use by default in the
>> federated build (so it would just reach over to the known location of
>> the working_classlib), but easily overridable w/ a property.
> 
> Default value for 'hy.hdk' should be "../working_classlib/deploy" to
> work in federated build w/o external definition. It would give also a
> good hint for those who executed build w/o defining "-Dhy.hdk=..." -
> ant error message will contain path to working_classlib.

Yes, that's done now.  It works cleanly from federated build.

> 
>>
>> >
>> > 3) Do we need selective build feature like '-Dbuild.module= ' for
>> > classlib. Again it will be very useful for people working for example
>> > on jdwp/ module.
>>
>> We could add that, or just go into the module and type "ant".  If we can
>> avoid the requirement to always run top level build w/ a -Dbuild.module
>> I'd like to.  Time will tell.
> 
> I agree - 'ant' in module dir looks enough. The only thing should be
> checked in such approach - creating and deleting 'deploy' and 'build'
> directories where build stores results and intermediate files. The
> same about testing.

right now, you can type "ant" and it builds everything, placing the 
generated artifacts of interest into the deploy/jdk tree.  "ant clean" 
goes through and [mostly] cleans everything up.

> 
>>
>> >
>> > I'm going to proceed with current state of the jdktools/ sources to
>> > provide features 2) and 3) and it would be nice to know your plans in
>> > this area.
>>
>> Sure!  And I'm interested in what you think of what's there.  It's
>> exactly modeled after what I thought you were doing, although I probably
>> did a bit of "rip and shred" to simplify, and probably missed important
>> details.  Please let me know what you think.
> 
> No big issues at the moment; I'm still looking into the 'test' part.

That's still broken.

> Do you think copying libraries from HDK to jre/bin should be done
> while building launcher? I think they are needed for testing only so
> this step can be moved to the 'test' target chain. Am I correct?

I don't yet know the correct answer.  I think that we should start 
working towards a federated build model where each components 
contribution to the "jdk skeleton" are filled in, and only those, and 
then the federated build gathers all together into the final JDK.

But in order for people to work, we still want the current situation 
where a full jre is assembled for unit testing, etc.

I think the simplest answer it to keep going as we are- every subproject 
  can make their deploy/ as rich as they need it, and the federated 
build does something less brain-dead than copy * - rather, things are 
mapped out.  I've started scribbleing something in wiki, and will bring 
that up as a separate thread.

geir

Re: [jdktools] svn commit r479920 - build and src tree

[Ilya Neverov <il...@gmail.com>]

On 11/28/06, Geir Magnusson Jr. <ge...@pobox.com> wrote:
>
> Ilya Neverov wrote:
> > Hi Geir,
> >
> > I eventually managed to build 'jdktools' created in your commit [1] in
> > context of harmony-hdk-r468731 but the 'test' target failed - will try
> > to resolve tomorrow.
>
> I don't think they work - it's still a work in progress.  I was going to
> do a little more today and update to the list.  I was working last night
> based on your patch, but was really just hacking around.
>
> I didn't understand some of what you were trying to accomplish in your
> patch, so I figured I'd try some myself to grok how things come
> together.  For example, I couldn't see why things were being done in
> "working_jdktools", since in jdktools/trunk, we should have everything
> setup, and use the switch trick in the federated build.

It doesn't matter now -  it was an attempt to prepare changes which
can be commited at once.

In my variant I tried to avoid multiple antcalls and to use importing
as much as possible. The idea was to gather all 'rules' into files in
the jdktools/make directory and to have a little property definition
in each <module>/build.xml. I think now that antcalls are not a big
issue for building jdktools so we can return to assessing build system
later, when jdktools build takes long time to complete. We can live
some time with current build model.

>
> >
> > I'd like to understand future evolution of the 'jdktools' - could you
> > please comment the items below.
> >
> > 1) Do we need separate jdk/lib/tools.jar and jdk/jre/lib/jretools.jar?
> > I guess it's too early to think about separate Harmony-jre bundle so
> > all jdktools can yet go into single tools.jar.
>
> Right, and copy to both places or something for now.

Consensus!

>
> >
> > 2) Do we need a possibility to build/test jdktools without building
> > classlib (using harmony-hdk bundles) ? This feature can be useful for
> > people who want to work with some tool sources only.
>
> yes, it's that way already -  the current mode requires a -Dhy.hdk=
> param to ant to get it to work as I didnt' think i was far enough to
> integrate yet.

Now I see this in ant files. I was confused with 'hy.hdk' and 'hy.jdk'
names - first one is for HDK used as build context but second name is
for 'target' directory. Does it make sense to rename 'hy.jdk' to
'target.dir'?  I can prepare the patch.

>
> My thought was that this would be designed for use by default in the
> federated build (so it would just reach over to the known location of
> the working_classlib), but easily overridable w/ a property.

Default value for 'hy.hdk' should be "../working_classlib/deploy" to
work in federated build w/o external definition. It would give also a
good hint for those who executed build w/o defining "-Dhy.hdk=..." -
ant error message will contain path to working_classlib.

>
> >
> > 3) Do we need selective build feature like '-Dbuild.module= ' for
> > classlib. Again it will be very useful for people working for example
> > on jdwp/ module.
>
> We could add that, or just go into the module and type "ant".  If we can
> avoid the requirement to always run top level build w/ a -Dbuild.module
> I'd like to.  Time will tell.

I agree - 'ant' in module dir looks enough. The only thing should be
checked in such approach - creating and deleting 'deploy' and 'build'
directories where build stores results and intermediate files. The
same about testing.

>
> >
> > I'm going to proceed with current state of the jdktools/ sources to
> > provide features 2) and 3) and it would be nice to know your plans in
> > this area.
>
> Sure!  And I'm interested in what you think of what's there.  It's
> exactly modeled after what I thought you were doing, although I probably
> did a bit of "rip and shred" to simplify, and probably missed important
> details.  Please let me know what you think.

No big issues at the moment; I'm still looking into the 'test' part.
Do you think copying libraries from HDK to jre/bin should be done
while building launcher? I think they are needed for testing only so
this step can be moved to the 'test' target chain. Am I correct?

>
> Today I was going to add JDWP and see how to integrate it...
>
> geir
>
> >
> > Thank you.
> > Ilya
> >
> > [1]
> > -------------------
> > r479920 | geirm | 2006-11-28 11:58:10 +0600 (Tue, 28 Nov 2006) | 4 lines
> >
> > copied the high-level build files from classlib
> > and modified to just build launcher and tools jars
> > --------------------
>

Re: [jdktools] svn commit r479920 - build and src tree

["Geir Magnusson Jr." <ge...@pobox.com>]

Ilya Neverov wrote:
> Hi Geir,
> 
> I eventually managed to build 'jdktools' created in your commit [1] in
> context of harmony-hdk-r468731 but the 'test' target failed - will try
> to resolve tomorrow.

I don't think they work - it's still a work in progress.  I was going to 
do a little more today and update to the list.  I was working last night 
based on your patch, but was really just hacking around.

I didn't understand some of what you were trying to accomplish in your 
patch, so I figured I'd try some myself to grok how things come 
together.  For example, I couldn't see why things were being done in 
"working_jdktools", since in jdktools/trunk, we should have everything 
setup, and use the switch trick in the federated build.


> 
> I'd like to understand future evolution of the 'jdktools' - could you
> please comment the items below.
> 
> 1) Do we need separate jdk/lib/tools.jar and jdk/jre/lib/jretools.jar?
> I guess it's too early to think about separate Harmony-jre bundle so
> all jdktools can yet go into single tools.jar.

Right, and copy to both places or something for now.

> 
> 2) Do we need a possibility to build/test jdktools without building
> classlib (using harmony-hdk bundles) ? This feature can be useful for
> people who want to work with some tool sources only.

yes, it's that way already -  the current mode requires a -Dhy.hdk= 
param to ant to get it to work as I didnt' think i was far enough to 
integrate yet.

My thought was that this would be designed for use by default in the 
federated build (so it would just reach over to the known location of 
the working_classlib), but easily overridable w/ a property.

> 
> 3) Do we need selective build feature like '-Dbuild.module= ' for
> classlib. Again it will be very useful for people working for example
> on jdwp/ module.

We could add that, or just go into the module and type "ant".  If we can 
avoid the requirement to always run top level build w/ a -Dbuild.module 
I'd like to.  Time will tell.

> 
> I'm going to proceed with current state of the jdktools/ sources to
> provide features 2) and 3) and it would be nice to know your plans in
> this area.

Sure!  And I'm interested in what you think of what's there.  It's 
exactly modeled after what I thought you were doing, although I probably 
did a bit of "rip and shred" to simplify, and probably missed important 
details.  Please let me know what you think.

Today I was going to add JDWP and see how to integrate it...

geir

> 
> Thank you.
> Ilya
> 
> [1]
> -------------------
> r479920 | geirm | 2006-11-28 11:58:10 +0600 (Tue, 28 Nov 2006) | 4 lines
> 
> copied the high-level build files from classlib
> and modified to just build launcher and tools jars
> --------------------