You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@buildr.apache.org by Adam Young <ay...@redhat.com> on 2010/04/19 19:23:02 UTC
Integrating with JPackage
We have been happily using builder for For the Candlepin project fora
few months now. As we get closer to "productization" I've been looking
into a way to build the project from sources, to include all of the
dependencies. Dependencies downloaded from the Maven repositories are
often based at some point on checked-in binaries. An alternative
approach, and the which is embraced by JPackage, is to:
A) Create Source level RPMS for all dependencies and
B) Create local maven repo out of those jars.
This is the approach I've been pursuing, and so far, so good. In order
to take advantage of this approach, I've had to modify my version of
builder as well as call builder with an additional module. The change to
buildr proper allows the local module to overload how the repository
path is built:
http://gist.github.com/368958
This creates a module level function named build_path with the existing
logic.
My overloaded version goes into a local script I am createivle calling
./localbuild.rb.
Here's a first draft.
http://gist.github.com/371298
Note that this version redefines build_path.
I used libxml to parse the maven repository mapping file, because
xmlsimple was flaking out on me. I'm sure I could have debugged it
further, but instead I went with what seems to be a faster and simpler
approach using streaming.
This page explains the Maven version of JPP, and the rules buildr needs
to follow to make use of the JPP approach.
http://fedoraproject.org/wiki/Java/JPPMavenReadme
Is there other interest in this approach?
Re: Integrating with JPackage
Posted by Adam Young <ay...@redhat.com>.
On 04/19/2010 04:08 PM, Antoine Toulme wrote:
> On Mon, Apr 19, 2010 at 12:35, Adam Young<ay...@redhat.com> wrote:
>
>
>> On 04/19/2010 01:31 PM, Antoine Toulme wrote:
>>
>>
>>> A couple of quick comments while reading your gist:
>>>
>>>
>>>
>> Thanks for the feed back. THis is "proof-of-concept" code, that will make
>> it possible to build, but certainly not ready for prime time.
>>
>>
>> You should avoid at all costs using global variables with $. In practice,
>>
>>> it
>>> is best to use a singleton approach.
>>>
>>>
>>>
>> There is a difference between a global variable and a a singleton. In this
>> case, the difference is academic, but In general is is poor design to use a
>> Class name as the way to locate an instance variable.
>>
>>
> Sure. Actually here you could maybe even be safe with a @field ? I would
> need specs to back that.
>
THat sounds about right. I'll ty it out.
>
>> In this case, it would more likely make sense for there to be some variable
>> at the module scope, but I have to admit that my ruby is fresh enough that I
>> don't know how to do this off the top of my head, or if Ruby even allows
>> such a thing. A better solution would be to make the repository itself an
>> object and the parsing to be part of the construction of the repo object.
>> Or something.
>>
>>
>> Use info, warn, error to log stuff.
>>
>>>
>>>
>> Yeah. Still developing this. puts is this poor man's debugger.
>>
>>
>> If you need to check that an object is nil, you can do object.nil?
>>
>>>
>>>
>> Thanks. That is a new rubiysm for me. I like the ability to avoid
>> accidental assignment.
>>
>> Last but actually the most important item, we need specs to integrate
>>
>>> those
>>> changes. For the first gist, a couple of specs demonstrating that you can
>>> monkey patch successfully the Repositories instance would be great. For
>>> the
>>> module itself, it'd be great for you to have specs to deal with the
>>> absence
>>> of the xml file, malformed file, etc.
>>>
>>>
>>>
>> specs? That word means a few different things to me, but I suspect it
>> means something specific here. Can you send me a pointer?
>>
> Check out rspec, and the spec folder under the buildr code.
> It helps you define what the code is intended for and avoid breaking it by
> mistake.
> It's almost a test unit - there is a specification aspect to the way you
> write them.
>
Sounds good. I'll check it out. I like the idea of something checking
my work for me.
>
>>
>> I think we would be interested into this extension to be part of Buidr
>>
>>> itself. I would recommend that people interested by this functionality do
>>> an
>>> extra require in their Buildfile: require "buildr/jpp" for example, much
>>> like you do an extra require to add ecj as a compiler today.
>>>
>>>
>>>
>> I'm almost tempted to say that it should be a switch to buildr itself as
>> opposed to a change to the buildfile. Chosing which repository scheme to
>> use is a distribution decsion, one that may not be mirrored by the people
>> writing the code that gets distributed. WHile in Candlepin we do have the
>> ability to dictate which version of buildr to use for our purposes, we have
>> the goal that one should not need JPackage to build, or even that the end
>> use is building on Fedora or even Linux.
>>
>>
> In your buildfile, you will always assume you are using jpp dependencies,
> correct ? Since you are going to name repositories anyway.
> So doing a require at the top of the Buildfile sounds like the right
> approach to me.
>
No, there will be no dep in the buildfile. THe buildfile will list the
Java dependencies, but, unless you build with the switch, it will
resolve the dependencies the way that buildr and maven do today: based
on the values set via
repositories.remote=
What the JPP approach would do would override that and set
repositories.remote=file:///usr/share/maven
For stricter builds.
But yes, you could force your porject to build this way if you really
wanted to. I think the switch approach would be required.
>
>
>> mvn-jpp is run a a wrapper that just call mvn with a couple of switches:
>>
>> -Dmaven2.offline.mode -Dmaven2.ignore.versions -Dmaven2.usejppjars
>>
>> I'm tempted to say that buildr should implement at least the last two as
>> switches.
>>
>>
> We can also do that.
>
>
>> In building the hash for jar to version, I'm currently adding two keys for
>> each package: One with the version info and one without. Ideally, we'd
>> allow multiple versions in JPP, and not have to use
>> -Dmaven2.ignore.versions.
>>
>> So perhaps we could allow a switch which is buildr --usejpp
>>
> OK. But really --usejpp would be the same as writing "require 'buildr/jpp'".
> I am certainly missing something, but I think we have a good agreement. And
> if we have 2 ways to do the same thing, I tend to like it.
>
>
>>
>>
>> Thanks for your interest in Buildr!
>>
>>> Antoine
>>>
>>>
>>> On Mon, Apr 19, 2010 at 10:23, Adam Young<ay...@redhat.com> wrote:
>>>
>>>
>>>
>>>
>>>> We have been happily using builder for For the Candlepin project fora few
>>>> months now. As we get closer to "productization" I've been looking into
>>>> a
>>>> way to build the project from sources, to include all of the
>>>> dependencies.
>>>> Dependencies downloaded from the Maven repositories are often based at
>>>> some
>>>> point on checked-in binaries. An alternative approach, and the which is
>>>> embraced by JPackage, is to:
>>>>
>>>> A) Create Source level RPMS for all dependencies and
>>>> B) Create local maven repo out of those jars.
>>>>
>>>> This is the approach I've been pursuing, and so far, so good. In order
>>>> to
>>>> take advantage of this approach, I've had to modify my version of builder
>>>> as
>>>> well as call builder with an additional module. The change to buildr
>>>> proper
>>>> allows the local module to overload how the repository path is built:
>>>>
>>>> http://gist.github.com/368958
>>>>
>>>> This creates a module level function named build_path with the existing
>>>> logic.
>>>>
>>>> My overloaded version goes into a local script I am createivle calling
>>>> ./localbuild.rb.
>>>>
>>>> Here's a first draft.
>>>>
>>>> http://gist.github.com/371298
>>>>
>>>>
>>>> Note that this version redefines build_path.
>>>>
>>>> I used libxml to parse the maven repository mapping file, because
>>>> xmlsimple
>>>> was flaking out on me. I'm sure I could have debugged it further, but
>>>> instead I went with what seems to be a faster and simpler approach using
>>>> streaming.
>>>>
>>>>
>>>> This page explains the Maven version of JPP, and the rules buildr needs
>>>> to
>>>> follow to make use of the JPP approach.
>>>>
>>>> http://fedoraproject.org/wiki/Java/JPPMavenReadme
>>>>
>>>> Is there other interest in this approach?
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
Re: Integrating with JPackage
Posted by Antoine Toulme <an...@lunar-ocean.com>.
On Mon, Apr 19, 2010 at 12:35, Adam Young <ay...@redhat.com> wrote:
> On 04/19/2010 01:31 PM, Antoine Toulme wrote:
>
>> A couple of quick comments while reading your gist:
>>
>>
>
> Thanks for the feed back. THis is "proof-of-concept" code, that will make
> it possible to build, but certainly not ready for prime time.
>
>
> You should avoid at all costs using global variables with $. In practice,
>> it
>> is best to use a singleton approach.
>>
>>
>
> There is a difference between a global variable and a a singleton. In this
> case, the difference is academic, but In general is is poor design to use a
> Class name as the way to locate an instance variable.
>
Sure. Actually here you could maybe even be safe with a @field ? I would
need specs to back that.
>
> In this case, it would more likely make sense for there to be some variable
> at the module scope, but I have to admit that my ruby is fresh enough that I
> don't know how to do this off the top of my head, or if Ruby even allows
> such a thing. A better solution would be to make the repository itself an
> object and the parsing to be part of the construction of the repo object.
> Or something.
>
>
> Use info, warn, error to log stuff.
>>
>>
> Yeah. Still developing this. puts is this poor man's debugger.
>
>
> If you need to check that an object is nil, you can do object.nil?
>>
>>
> Thanks. That is a new rubiysm for me. I like the ability to avoid
> accidental assignment.
>
> Last but actually the most important item, we need specs to integrate
>> those
>> changes. For the first gist, a couple of specs demonstrating that you can
>> monkey patch successfully the Repositories instance would be great. For
>> the
>> module itself, it'd be great for you to have specs to deal with the
>> absence
>> of the xml file, malformed file, etc.
>>
>>
> specs? That word means a few different things to me, but I suspect it
> means something specific here. Can you send me a pointer?
Check out rspec, and the spec folder under the buildr code.
It helps you define what the code is intended for and avoid breaking it by
mistake.
It's almost a test unit - there is a specification aspect to the way you
write them.
>
>
> I think we would be interested into this extension to be part of Buidr
>> itself. I would recommend that people interested by this functionality do
>> an
>> extra require in their Buildfile: require "buildr/jpp" for example, much
>> like you do an extra require to add ecj as a compiler today.
>>
>>
> I'm almost tempted to say that it should be a switch to buildr itself as
> opposed to a change to the buildfile. Chosing which repository scheme to
> use is a distribution decsion, one that may not be mirrored by the people
> writing the code that gets distributed. WHile in Candlepin we do have the
> ability to dictate which version of buildr to use for our purposes, we have
> the goal that one should not need JPackage to build, or even that the end
> use is building on Fedora or even Linux.
>
In your buildfile, you will always assume you are using jpp dependencies,
correct ? Since you are going to name repositories anyway.
So doing a require at the top of the Buildfile sounds like the right
approach to me.
>
> mvn-jpp is run a a wrapper that just call mvn with a couple of switches:
>
> -Dmaven2.offline.mode -Dmaven2.ignore.versions -Dmaven2.usejppjars
>
> I'm tempted to say that buildr should implement at least the last two as
> switches.
>
We can also do that.
>
> In building the hash for jar to version, I'm currently adding two keys for
> each package: One with the version info and one without. Ideally, we'd
> allow multiple versions in JPP, and not have to use
> -Dmaven2.ignore.versions.
>
> So perhaps we could allow a switch which is buildr --usejpp
OK. But really --usejpp would be the same as writing "require 'buildr/jpp'".
I am certainly missing something, but I think we have a good agreement. And
if we have 2 ways to do the same thing, I tend to like it.
>
>
>
> Thanks for your interest in Buildr!
>>
>> Antoine
>>
>>
>> On Mon, Apr 19, 2010 at 10:23, Adam Young<ay...@redhat.com> wrote:
>>
>>
>>
>>> We have been happily using builder for For the Candlepin project fora few
>>> months now. As we get closer to "productization" I've been looking into
>>> a
>>> way to build the project from sources, to include all of the
>>> dependencies.
>>> Dependencies downloaded from the Maven repositories are often based at
>>> some
>>> point on checked-in binaries. An alternative approach, and the which is
>>> embraced by JPackage, is to:
>>>
>>> A) Create Source level RPMS for all dependencies and
>>> B) Create local maven repo out of those jars.
>>>
>>> This is the approach I've been pursuing, and so far, so good. In order
>>> to
>>> take advantage of this approach, I've had to modify my version of builder
>>> as
>>> well as call builder with an additional module. The change to buildr
>>> proper
>>> allows the local module to overload how the repository path is built:
>>>
>>> http://gist.github.com/368958
>>>
>>> This creates a module level function named build_path with the existing
>>> logic.
>>>
>>> My overloaded version goes into a local script I am createivle calling
>>> ./localbuild.rb.
>>>
>>> Here's a first draft.
>>>
>>> http://gist.github.com/371298
>>>
>>>
>>> Note that this version redefines build_path.
>>>
>>> I used libxml to parse the maven repository mapping file, because
>>> xmlsimple
>>> was flaking out on me. I'm sure I could have debugged it further, but
>>> instead I went with what seems to be a faster and simpler approach using
>>> streaming.
>>>
>>>
>>> This page explains the Maven version of JPP, and the rules buildr needs
>>> to
>>> follow to make use of the JPP approach.
>>>
>>> http://fedoraproject.org/wiki/Java/JPPMavenReadme
>>>
>>> Is there other interest in this approach?
>>>
>>>
>>>
>>
>>
>
>
Re: Integrating with JPackage
Posted by Adam Young <ay...@redhat.com>.
On 04/19/2010 01:31 PM, Antoine Toulme wrote:
> A couple of quick comments while reading your gist:
>
Thanks for the feed back. THis is "proof-of-concept" code, that will
make it possible to build, but certainly not ready for prime time.
> You should avoid at all costs using global variables with $. In practice, it
> is best to use a singleton approach.
>
There is a difference between a global variable and a a singleton. In
this case, the difference is academic, but In general is is poor design
to use a Class name as the way to locate an instance variable.
In this case, it would more likely make sense for there to be some
variable at the module scope, but I have to admit that my ruby is fresh
enough that I don't know how to do this off the top of my head, or if
Ruby even allows such a thing. A better solution would be to make the
repository itself an object and the parsing to be part of the
construction of the repo object. Or something.
> Use info, warn, error to log stuff.
>
Yeah. Still developing this. puts is this poor man's debugger.
> If you need to check that an object is nil, you can do object.nil?
>
Thanks. That is a new rubiysm for me. I like the ability to avoid
accidental assignment.
> Last but actually the most important item, we need specs to integrate those
> changes. For the first gist, a couple of specs demonstrating that you can
> monkey patch successfully the Repositories instance would be great. For the
> module itself, it'd be great for you to have specs to deal with the absence
> of the xml file, malformed file, etc.
>
specs? That word means a few different things to me, but I suspect it
means something specific here. Can you send me a pointer?
> I think we would be interested into this extension to be part of Buidr
> itself. I would recommend that people interested by this functionality do an
> extra require in their Buildfile: require "buildr/jpp" for example, much
> like you do an extra require to add ecj as a compiler today.
>
I'm almost tempted to say that it should be a switch to buildr itself as
opposed to a change to the buildfile. Chosing which repository scheme
to use is a distribution decsion, one that may not be mirrored by the
people writing the code that gets distributed. WHile in Candlepin we do
have the ability to dictate which version of buildr to use for our
purposes, we have the goal that one should not need JPackage to build,
or even that the end use is building on Fedora or even Linux.
mvn-jpp is run a a wrapper that just call mvn with a couple of switches:
-Dmaven2.offline.mode -Dmaven2.ignore.versions -Dmaven2.usejppjars
I'm tempted to say that buildr should implement at least the last two as
switches.
In building the hash for jar to version, I'm currently adding two keys
for each package: One with the version info and one without. Ideally,
we'd allow multiple versions in JPP, and not have to use
-Dmaven2.ignore.versions.
So perhaps we could allow a switch which is buildr --usejpp
> Thanks for your interest in Buildr!
>
> Antoine
>
>
> On Mon, Apr 19, 2010 at 10:23, Adam Young<ay...@redhat.com> wrote:
>
>
>> We have been happily using builder for For the Candlepin project fora few
>> months now. As we get closer to "productization" I've been looking into a
>> way to build the project from sources, to include all of the dependencies.
>> Dependencies downloaded from the Maven repositories are often based at some
>> point on checked-in binaries. An alternative approach, and the which is
>> embraced by JPackage, is to:
>>
>> A) Create Source level RPMS for all dependencies and
>> B) Create local maven repo out of those jars.
>>
>> This is the approach I've been pursuing, and so far, so good. In order to
>> take advantage of this approach, I've had to modify my version of builder as
>> well as call builder with an additional module. The change to buildr proper
>> allows the local module to overload how the repository path is built:
>>
>> http://gist.github.com/368958
>>
>> This creates a module level function named build_path with the existing
>> logic.
>>
>> My overloaded version goes into a local script I am createivle calling
>> ./localbuild.rb.
>>
>> Here's a first draft.
>>
>> http://gist.github.com/371298
>>
>>
>> Note that this version redefines build_path.
>>
>> I used libxml to parse the maven repository mapping file, because xmlsimple
>> was flaking out on me. I'm sure I could have debugged it further, but
>> instead I went with what seems to be a faster and simpler approach using
>> streaming.
>>
>>
>> This page explains the Maven version of JPP, and the rules buildr needs to
>> follow to make use of the JPP approach.
>>
>> http://fedoraproject.org/wiki/Java/JPPMavenReadme
>>
>> Is there other interest in this approach?
>>
>>
>
Re: Integrating with JPackage
Posted by Antoine Toulme <an...@lunar-ocean.com>.
A couple of quick comments while reading your gist:
You should avoid at all costs using global variables with $. In practice, it
is best to use a singleton approach.
Use info, warn, error to log stuff.
If you need to check that an object is nil, you can do object.nil?
Last but actually the most important item, we need specs to integrate those
changes. For the first gist, a couple of specs demonstrating that you can
monkey patch successfully the Repositories instance would be great. For the
module itself, it'd be great for you to have specs to deal with the absence
of the xml file, malformed file, etc.
I think we would be interested into this extension to be part of Buidr
itself. I would recommend that people interested by this functionality do an
extra require in their Buildfile: require "buildr/jpp" for example, much
like you do an extra require to add ecj as a compiler today.
Thanks for your interest in Buildr!
Antoine
On Mon, Apr 19, 2010 at 10:23, Adam Young <ay...@redhat.com> wrote:
> We have been happily using builder for For the Candlepin project fora few
> months now. As we get closer to "productization" I've been looking into a
> way to build the project from sources, to include all of the dependencies.
> Dependencies downloaded from the Maven repositories are often based at some
> point on checked-in binaries. An alternative approach, and the which is
> embraced by JPackage, is to:
>
> A) Create Source level RPMS for all dependencies and
> B) Create local maven repo out of those jars.
>
> This is the approach I've been pursuing, and so far, so good. In order to
> take advantage of this approach, I've had to modify my version of builder as
> well as call builder with an additional module. The change to buildr proper
> allows the local module to overload how the repository path is built:
>
> http://gist.github.com/368958
>
> This creates a module level function named build_path with the existing
> logic.
>
> My overloaded version goes into a local script I am createivle calling
> ./localbuild.rb.
>
> Here's a first draft.
>
> http://gist.github.com/371298
>
>
> Note that this version redefines build_path.
>
> I used libxml to parse the maven repository mapping file, because xmlsimple
> was flaking out on me. I'm sure I could have debugged it further, but
> instead I went with what seems to be a faster and simpler approach using
> streaming.
>
>
> This page explains the Maven version of JPP, and the rules buildr needs to
> follow to make use of the JPP approach.
>
> http://fedoraproject.org/wiki/Java/JPPMavenReadme
>
> Is there other interest in this approach?
>