You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@maven.apache.org by "Henning P. Schmiedehausen" <hp...@intermeta.de> on 2003/01/07 16:04:54 UTC

A tale of two builds (or: Why I don't like the SNAPSHOT sematics)

Let me tell you a little story. This actually happened to me yesterday
and I needed the better part of my afternoon to clean it up and I
could've done more useful work in that time:

 _
| |
|_|nce upon a time (to be exact: on 2002-12-27), I decided to take
upon me the course of converting our currently developed application
to Maven. Until then I was using happily the 1.0b4 ant based building
process, So I already had a repository and project.xml files in place.

I pulled the CVS HEAD from the central jakarta repository, made a tar
ball, put that away and bootstrapped it as described. It went fine and
I ended up with a shiny maven program which I tested and it was
good. This was before Jasons' big changes checkin, so it might be that
some of my points are no longer valid. I doubt it, though.

As we run internal concurrent development and I want all of our
developers to use the same tools, I decided to do a site wide install
of maven. We use RedHat Linux based work stations, so common sense
dictated to build a RPM of the maven distribution. With some small
patches and a little tinkering, this went through (see
http://maven.intermeta.de/) and we now have maven on every workstation
(all three of them :-) ) and I have it on my laptop and my home
system.

Maven worked great for me and from 2002-12-27 to 2003-01-06 I happily
converted all of the modules of our current project (seventeen) from
the old ant-based building using maven-1.0b4 to the new, shiny central
installed /usr/bin/maven based approach. I had to jump some very minor
obstacles and was rewarded with a much speeded up build process and
very much cleared semantics (<preGoal> and <postGoal> really made the
conversion worthwhile).

In this time I had to rebuild our starting maven-1.0b8-1t.src.rpm
three times (2t, 3t, 4t). Every time, the building went without a
hitch. rpm -ba maven.spec, <wait>, done. Terrific, I thought. Maven
really takes the cake.

And then came Monday the 6th. I had to rebuild my maven RPM for the
fourth time, making it maven-1.0b8-5t. So I did as I did all the days
before. But now the building process suddently died in the phase-1
bootstrap. I was wondering why, because I didn't change anything
(actually I had just two small documentation fixes in the .spec
file. _NO_ changes to the actual maven code). I retried but it didn't
build. The same code that compiled fine the other day, didn't worked
at all. No chance.

I found the culprit pretty soon. I seemed that some jars on ibiblio
were upgraded to newer snapshots. So the SNAPSHOT version now pointed
to a different jar which caused the failures. As I had saved all the
jars needed to build my CVS snapshot in a big tar archive, I decided
to build offline.

This is somewhat obscure but semi-documented (you have to the value
"-o" for a property in the build-bootstrap.properties).  It failed
again. I was dispairing.  Maven was definitely being called with the
"-o" switch which is documented as "makes the build offline". I
debugged for quite a while and found a bug in it [1]. After that, a
patch to the project.properties actually made my rpm compile again.

So there was lots of frustration. It stemmed from a few things:

- Maven building relies on three "SNAPSHOT" jars in its POM. One of
  these changed (commons-jelly did) and removed some classes and/or
  methods. 

  The Result: My build _failed_ without me doing anything wrong or
              different!

  This wouldn't have happened if maven didn't decide to go for
  a walk on the Internet every time it finds "SNAPHSHOT" as a
  version. Because I have a tried and true set of classes and
  depending jars right here in front of me. These are about 25
  MBytes of code which I _KNOW_ that they compile. And these jars are 
  actually called "SNAPSHOT" on my local repository which is wrong
  (these are snapshots with snap.yyyymmdd.hhmmss version). 

[ Proposal: Abandon the soft links on ibiblio. Make the Web Server
  redirect to the appropriate "snap" jar and let maven register the
  30x return code to dynamically change the dependency. After all,
  that's what these return codes are for. Or (if you don't like this),
  make the SNAPSHOT not a soft link but a small file containing the
  name of the real yyyymmdd.hhmmss" jar and let maven rebuild the
  dependency by itself. ]

  But frankly: for me the SNAPSHOT stuff as it is does not work. If
  this version would've been "snap-yyyymmdd.hhmmss", then even
  checking in a new jar onto the repository wouldn't have broken my
  local build without me doing anything.

- Maven cannot be told to work completely offline. This might
  be nice for people with cable but try it if you're on GSM.

  [1] Actually, if you don't believe this, look at this code
  from org.apache.maven.cli.App

--- cut ---
    /**
     * Check the online status.
     */
    private void checkOnline()
    {
        if ( getContext().getOnline() != null )
        {
            return;
        }

        if ( getCli().hasOption( WORK_OFFLINE ) )
        {
            getContext().setOnline( Boolean.FALSE );
        }
        else
        {
            getContext().setOnline( Boolean.TRUE );
        }
    }
--- cut ---

Which is nonsense to me. It says "if you get a maven.mode.online value
from the properties, ignore the WORK_OFFLINE (-o) switch".  But you
always get a value from the properties. Because the _default_ is
"true". You never get "null". So the "-o" switch is (at least in this
CVS snapshot) meaningless. Because it is never asked. Look:

henning@forge 16:02 ~/cvs/domainverwaltung > rm -rf target
henning@forge 16:02 ~/cvs/domainverwaltung > maven -o webapp
 __  __
|  \/  |__ Jakarta _ ___
| |\/| / _` \ V / -_) ' \  ~ intelligent projects ~
|_|  |_\__,_|\_/\___|_||_|   v. 1.0-beta-8

/mnt/home.net/henning/cvs/domainverwaltung

Attempting to download commons-configuration-SNAPSHOT.jar.
Attempting to download commons-dbcp-SNAPSHOT.jar.
^C

q.e.d.


- Maven cannot be told to "do no SNAPSHOT checking if you already
  have a snapshot jar". This might be right (I let this for the 
  reader to decide) but shows the problems with the SNAPSHOT concept.
  A build that worked five minutes ago doesn't run at all now, because
  someone else did check a jar onto the repository. And if the uploaded
  jar is corrupted somehow or has a problem, _everything_ relying on
  that jar in SNAPSHOT version dies. Every single build!

My personal conclusion of this: SNAPSHOT is a flawed concept and it
should be abandoned. So why is it there? As far as I can see, there is
only one reason: Laziness of the people using maven (because they
don't have to rewrite their project.xml files every time a new
snapshot comes out and they want to use a feature). While Laziness is
a perfectly good excuse for something, not being able to reproduce
a process is not. 

Wen I do software development, being able to reproduce a certain build
version (when a customer reports a problem), being able to validate a
build with tests, being able to exactly pinpoint where something fails
is taking 70% of my time. In this respect, SNAPSHOT sucks.

Now consider a "maven-perfect" world where all the jakarta projects
are happily using maven to build and resolve their problems. Consider
many of them using "commons-lang in SNAPSHOT" version.

And then getting a jar checkin like the one in commons-lang-1.0-dev.
Sudden death of all the projects using commons-lang. No way around it,
because trying to rebuild gets the same errors all over again because
maven simply fetches the flawed SNAPSHOT jar again from ibiblio. 
Frantic checking, which of the snapshots on ibiblio with
snap.yyyymmdd.hhmmss tag did still build the project. Not just one but
literally hundreds with many many different developers.

People, don't you have _anything_ better to do with your time?

We already _HAD_ that. We already _GOT BURNED_ by exactly that
problem. And now we activly recreating this? This is like getting
blown up by a bomb we've been sitting on and after we pulled ourselves
together again, we sat down right on the next one.

For me and my company the solution is simple: I ruled "SNAPSHOT" out
and put it off limits for any of our developers to use. Any jars used
in internal builds _must_ be fully versioned. No discussion about
this. And I will actively lobby all of the Apache projects to do the
same.

I know that most of the other developers will simply say "maven is
right and you don't know cheese". Frankly I don't care much, because
each of you can be sure that this mail will sooner or later pop up
again in an "I told you so" reference. And we will get yet another
kludge like the "override" property. History repeating itself.

	Regards
		Henning


-- 
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen       -- Geschaeftsfuehrer
INTERMETA - Gesellschaft fuer Mehrwertdienste mbH     hps@intermeta.de

Am Schwabachgrund 22  Fon.: 09131 / 50654-0   info@intermeta.de
D-91054 Buckenhof     Fax.: 09131 / 50654-20

Re: A tale of two builds (or: Why I don't like the SNAPSHOT sematics)

Posted by Peter Donald <pe...@realityforge.org>.

On Sat, 11 Jan 2003 01:59, Jason van Zyl wrote:
> > Anyways for a bit more info scroll down to dependencies on
> > http://jakarta.apache.org/ant/myrmidon/librarys.html
> >
> > or lookup
> >
> > http://java.sun.com/j2se/1.3/docs/guide/extensions/versioning.html.
>
> Thanks will take a peek?

yea!

> Is there anything we can lift from myrmidom? I intend at some point in
> the future for Maven to run inside Plexus so I'm sure there's a lot of
> overlap and probably things you've already done that I hope you wouldn't
> mind if I just lifted :-)

Most of it is used in a few Avalon contains and is already extracted. You can 
grab it from 

jakarta-avalon-excalibur/extension

though I would probably recomend you only use the classes in 
o.a.excalibur.extension.*

That just represents the Extension (and Specification) objects and reads them 
from manifests.

o.a.excalibur.packagemanager.* Is meant to define a mechanism to traverse 
extensionmanager dependencies in a repository however it is not really 
designed with Mavens repository in mind and wont scale well enough (only used 
in Avalon containers which have <80 extension jars). So don't use that unless 
you rewrite it ... and if you rewrite it I would to see it decoupled enough 
that I could use it ;)

-- 
Cheers,

Peter Donald
---------------------------------------------------
"Wise men don't need advice. Fools don't take it." 
                        -Benjamin Franklin 
---------------------------------------------------

Re: A tale of two builds (or: Why I don't like the =?utf-8?q?SNAPSHOT

Posted by Peter Donald <pe...@realityforge.org>.

On Sat, 11 Jan 2003 03:58, Brian Ewins wrote:
> Peter Donald wrote:
> >On Wed, 8 Jan 2003 21:44, Brian Ewins wrote:
> >>I don't think indirect dependencies are quite so straightforward. [snip]
> >
> >I would highly recomend that you use the JDK "Optional Package" or
> > "Extension" mechanisms for this. Basically for each jar you define a set
> > of extensions that the jar provides and a set of extensions that the jar
> > needs.
>
> Well its the jar versioning mechanism spec you mean[1], 

No its not ;) It is the spec that was called "Extension" spec in JDK1.2 and 
got renamed to the "Optional Package" specification in JDK1.3. It is the same 
extension that servlet engines and EJB engines must support. 

See;
http://java.sun.com/j2se/1.3/docs/guide/extensions/versioning.html

> Unfortunately this mechanism is currently a fiction created to torment
> writers of build tools ;)

Its a bit like that. A worst thing is that some things that actually implement 
it (ie Tomcat) are broken with transitive dependencies in some arrangments ;(

>       <manifest>
>         <attribute name="Built-By" value="${user.name}" />
>         <section name="${pom.package}">
>           <attribute name="Specification-Title" value="${pom.id}" />
>           <attribute name="Specification-Version"
>                      value="${pom.currentVersion}" />
>           <attribute name="Specification-Vendor"
>                      value="${pom.organization.name}" />
>           <attribute name="Implementation-Title"
>                      value="${pom.package}" />
>           <attribute name="Implementation-Version"
>                      value="${pom.currentVersion}" />
>           <attribute name="Implementation-Vendor"
>                      value="${pom.organization.name}" />
>         </section>
>       </manifest>

The Extension spec has a slightly different format. Namely it replaces 
"Specification-Title" with "Extension-Name" and allows you to specify 
dependencies via something like

Extension-List: trax velocity
velocity-Extension-Name: org.apache.velocity
velocity-Specification-Version: 1.0
trax-Extension-Name: Java API for XML Parsing
trax-Specification-Version: 1.1
trax-Implementation-Title: org.apache.xalan.xslt
trax-Implementation-Version: 2.1.0
trax-Implementation-Vendor: Apache Software Foundation

Maven could add those dependencies by scanning and selecting dependencies from 
its POM. Though I am not sure how to determine which dependencies are runtime 
... does Maven have a way to do this?

> Note several POM entries are reused, instead of having separate entries
> in the POM for the different pieces, and use the values above as
> defaults. The only way to override the info at present is to write a
> manifest by hand. It seems to me like it should be possible - compulsory
> even ;) -  to have this info in the POM since maven is targeted at
> producing a single artifact per project.

+1

-- 
Cheers,

Peter Donald
*-----------------------------------------------------*
* "Faced with the choice between changing one's mind, *
* and proving that there is no need to do so - almost *
* everyone gets busy on the proof."                   *
*              - John Kenneth Galbraith               *
*-----------------------------------------------------*

Re: A tale of two builds (or: Why I don't like theSNAPSHOT sematics)

Posted by Jason van Zyl <ja...@zenplex.com>.

On Thu, 2003-01-09 at 16:28, Peter Donald wrote:
> On Wed, 8 Jan 2003 21:44, Brian Ewins wrote:
> > I don't think indirect dependencies are quite so straightforward. The
> > obvious example is JAXP; its not just versions of a dependency, but
> > alternatives to that dependency. [xerces-1.4.4], or [xercesImpl with
> > xml-apis], or [none required on JDK 1.4]. It is a PITA documenting all
> > the dependencies, but being explicit helps avoid confusion about where a
> > jar is coming from, and allows you to omit a dependency entirely.
> 
> I would highly recomend that you use the JDK "Optional Package" or "Extension" 
> mechanisms for this. Basically for each jar you define a set of extensions 
> that the jar provides and a set of extensions that the jar needs. 
> 
> Each extensions has a specification and an implementation and you can declare 
> dependencies on either the implementation or the specification. SO you could 
> declare a dependency on "jaxp" or "xerces2 jaxp implementation".
> 
> Anyways for a bit more info scroll down to dependencies on 
> http://jakarta.apache.org/ant/myrmidon/librarys.html
> 
> or lookup 
> 
> http://java.sun.com/j2se/1.3/docs/guide/extensions/versioning.html. 

Thanks will take a peek?

Is there anything we can lift from myrmidom? I intend at some point in
the future for Maven to run inside Plexus so I'm sure there's a lot of
overlap and probably things you've already done that I hope you wouldn't
mind if I just lifted :-)

-- 
jvz.

Jason van Zyl
jason@zenplex.com
http://tambora.zenplex.org

In short, man creates for himself a new religion of a rational
and technical order to justify his work and to be justified in it.
  
  -- Jacques Ellul, The Technological Society

Re: A tale of two builds (or: Why I don't like the SNAPSHOT sematics)

Posted by di...@multitask.com.au.

Gotta patch we can look at?
--
dIon Gillard, Multitask Consulting
Blog:      http://www.freeroller.net/page/dion/Weblog
Work:      http://www.multitask.com.au


Brian Ewins <Br...@i-documentsystems.com> wrote on 11/01/2003 
03:58:45 AM:

> 
> 
> Peter Donald wrote:
> 
> >On Wed, 8 Jan 2003 21:44, Brian Ewins wrote:
> > 
> >
> >>I don't think indirect dependencies are quite so straightforward. 
[snip]
> >>
> >I would highly recomend that you use the JDK "Optional Package" or 
> "Extension" 
> >mechanisms for this. Basically for each jar you define a set of 
extensions 
> >that the jar provides and a set of extensions that the jar needs. 
> >
> Well its the jar versioning mechanism spec you mean[1], but yes, that's 
> exactly what I was referring to when I wrote: " I tried to identify jar 
> versions and was disappointed with the  practically nonexistent 
> information in most manifests (including Sun's jars of javax.* stuff)". 
> I wrote a version of the axis fingerprint which extracted this info from 

> every jar, only to find that only a couple of jars had anything worth 
> reporting. IIRC this included things like the servlet APIs distributed 
> from tomcat, which being the reference impl should know better.
> 
> Unfortunately this mechanism is currently a fiction created to torment 
> writers of build tools ;)
> 
> Its a good discussion point actually - since maven provides some help in 

> generating manifests, over time the info available to maven will 
> improve. Support could be improved though. Here's what the ear plugin 
does:
> 
>       <manifest>
>         <attribute name="Built-By" value="${user.name}" />
>         <section name="${pom.package}">
>           <attribute name="Specification-Title" value="${pom.id}" />
>           <attribute name="Specification-Version"
>                      value="${pom.currentVersion}" />
>           <attribute name="Specification-Vendor"
>                      value="${pom.organization.name}" />
>           <attribute name="Implementation-Title"
>                      value="${pom.package}" />
>           <attribute name="Implementation-Version"
>                      value="${pom.currentVersion}" />
>           <attribute name="Implementation-Vendor"
>                      value="${pom.organization.name}" />
>         </section>
>       </manifest>
> 
> Note several POM entries are reused, instead of having separate entries 
> in the POM for the different pieces, and use the values above as 
> defaults. The only way to override the info at present is to write a 
> manifest by hand. It seems to me like it should be possible - compulsory 

> even ;) -  to have this info in the POM since maven is targeted at 
> producing a single artifact per project.
> 
> Cheers,
> Baz
> 
> [1] 
> http://java.sun.com/j2se/1.
> 3/docs/guide/versioning/spec/VersioningSpecification.html 
> . the optional/extension bit just depends on the versioning spec for 
> version info. Maven should probably not be in the game of writing info 
> from the extension spec into the manifest unless the user explicitly 
> requests it, because that's supposed to install jars on the fly into 
> lib/ext!
> 
> 
> 
> Privacy and Confidentiality Notice
> 
> ------------------------------------------------
> 
> The information contained in this E-Mail message is intended only 
> for the person or persons to whom it is addressed.  Such information
> is confidential and privileged and no mistake in transmission is 
> intended to waive or compromise such privilege.  If you have 
> received it in error, please destroy it and notify us on the 
> telephone number printed above.  If you do not receive complete and 
> legible copies, please telephone us immediately. Any opinions 
> expressed herein including attachments are those of the author only.
> i-documentsystems Ltd. does not accept responsibility for the 
> accuracy or completeness of the information provided or for any 
> changes to this Email, however made, after it was sent. (Please note
> that it is your responsibility to scan this message for viruses).
> 
> 
> --
> To unsubscribe, e-mail:   <mailto:turbine-maven-dev-
> unsubscribe@jakarta.apache.org>
> For additional commands, e-mail: <mailto:turbine-maven-dev-
> help@jakarta.apache.org>
> 

> ForwardSourceID:NT000A1B3E

Re: A tale of two builds (or: Why I don't like the =?iso-8859-1?q?SNAPSHOT sematics?=)

Posted by Brian Ewins <Br...@i-documentsystems.com>.

Peter Donald wrote:

>On Wed, 8 Jan 2003 21:44, Brian Ewins wrote:
>  
>
>>I don't think indirect dependencies are quite so straightforward. [snip]
>>
>I would highly recomend that you use the JDK "Optional Package" or "Extension" 
>mechanisms for this. Basically for each jar you define a set of extensions 
>that the jar provides and a set of extensions that the jar needs. 
>
Well its the jar versioning mechanism spec you mean[1], but yes, that's 
exactly what I was referring to when I wrote: " I tried to identify jar 
versions and was disappointed with the  practically nonexistent 
information in most manifests (including Sun's jars of javax.* stuff)". 
I wrote a version of the axis fingerprint which extracted this info from 
every jar, only to find that only a couple of jars had anything worth 
reporting. IIRC this included things like the servlet APIs distributed 
from tomcat, which being the reference impl should know better.

Unfortunately this mechanism is currently a fiction created to torment 
writers of build tools ;)

Its a good discussion point actually - since maven provides some help in 
generating manifests, over time the info available to maven will 
improve. Support could be improved though. Here's what the ear plugin does:

      <manifest>
        <attribute name="Built-By" value="${user.name}" />
        <section name="${pom.package}">
          <attribute name="Specification-Title" value="${pom.id}" />
          <attribute name="Specification-Version"
                     value="${pom.currentVersion}" />
          <attribute name="Specification-Vendor"
                     value="${pom.organization.name}" />
          <attribute name="Implementation-Title"
                     value="${pom.package}" />
          <attribute name="Implementation-Version"
                     value="${pom.currentVersion}" />
          <attribute name="Implementation-Vendor"
                     value="${pom.organization.name}" />
        </section>
      </manifest>

Note several POM entries are reused, instead of having separate entries 
in the POM for the different pieces, and use the values above as 
defaults. The only way to override the info at present is to write a 
manifest by hand. It seems to me like it should be possible - compulsory 
even ;) -  to have this info in the POM since maven is targeted at 
producing a single artifact per project.

Cheers,
Baz

[1] 
http://java.sun.com/j2se/1.3/docs/guide/versioning/spec/VersioningSpecification.html 
. the optional/extension bit just depends on the versioning spec for 
version info. Maven should probably not be in the game of writing info 
from the extension spec into the manifest unless the user explicitly 
requests it, because that's supposed to install jars on the fly into 
lib/ext!

Privacy and Confidentiality Notice

------------------------------------------------

The information contained in this E-Mail message is intended only for the person or persons to whom it is addressed.  Such information is confidential and privileged and no mistake in transmission is intended to waive or compromise such privilege.  If you have received it in error, please destroy it and notify us on the telephone number printed above.  If you do not receive complete and legible copies, please telephone us immediately. Any opinions expressed herein including attachments are those of the author only. i-documentsystems Ltd. does not accept responsibility for the accuracy or completeness of the information provided or for any changes to this Email, however made, after it was sent. (Please note that it is your responsibility to scan this message for viruses).

Re: A tale of two builds (or: Why I don't like the SNAPSHOT sematics)

Posted by Peter Donald <pe...@realityforge.org>.

On Wed, 8 Jan 2003 21:44, Brian Ewins wrote:
> I don't think indirect dependencies are quite so straightforward. The
> obvious example is JAXP; its not just versions of a dependency, but
> alternatives to that dependency. [xerces-1.4.4], or [xercesImpl with
> xml-apis], or [none required on JDK 1.4]. It is a PITA documenting all
> the dependencies, but being explicit helps avoid confusion about where a
> jar is coming from, and allows you to omit a dependency entirely.

I would highly recomend that you use the JDK "Optional Package" or "Extension" 
mechanisms for this. Basically for each jar you define a set of extensions 
that the jar provides and a set of extensions that the jar needs. 

Each extensions has a specification and an implementation and you can declare 
dependencies on either the implementation or the specification. SO you could 
declare a dependency on "jaxp" or "xerces2 jaxp implementation".

Anyways for a bit more info scroll down to dependencies on 
http://jakarta.apache.org/ant/myrmidon/librarys.html

or lookup 

http://java.sun.com/j2se/1.3/docs/guide/extensions/versioning.html. 
-- 
Cheers,

Peter Donald
*----------------------------------------------------*
|  "Luck is the residue of design" -- Branch Rickey  |
*----------------------------------------------------*

Re: A tale of two builds (or: Why I don't like the SNAPSHOT sematics)

Posted by Jason van Zyl <ja...@zenplex.com>.

On Tue, 2003-01-07 at 15:44, Jason van Zyl wrote:
> 
> Maven will eventually work so that all indirect dependencies will be
> gleaned, but if in the POM you state a version of one of those direct

correction: that is indirect dependency

> dependencies then the POM will be obeyed. You as a project developer
> will always have the choice, Maven will just try to help out.

-- 
jvz.

Jason van Zyl
jason@zenplex.com
http://tambora.zenplex.org

In short, man creates for himself a new religion of a rational
and technical order to justify his work and to be justified in it.
  
  -- Jacques Ellul, The Technological Society

Re: A tale of two builds (or: Why I don't like the SNAPSHOT sematics)

Posted by Brian Ewins <Br...@i-documentsystems.com>.

Jason van Zyl wrote:

>On Tue, 2003-01-07 at 14:25, Brian Ewins wrote:
>  
>
>>- there is the ugly "axis+saaj" wart to express dependencies on 
>>artifacts with 'the wrong name'
>>    
>>
>
>You can use <groupId> and <artifactId> if you wish, we were just trying
>to maintain backward compatibility with fugly +/: notation.
>  
>
Excellent.  I'll read up on this, but there's an obvious question - do 
you still need the + notation when calling plugin.getDependencyPath()?

>Naming conventions can certainly tell you something. Which is more
>meaningful to you:
>
>1) foo-1.0.jar
>2) foo.jar
>
I do agree that the first name is better, and maven should /create/ 
artifacts with names like this. However, I don't agree with renaming 
third party artifacts to get this effect, and it seems to me that in the 
past some of the renaming has been even more severe (merging jars, for 
example), making it harder to locate the central copy of classes - but 
this is all in the past if the groupId/artifactId stuff works well.

>We can certainly move toward
>gleaning information from the manifest but these too currently are all
>over the map. We did what we could with what we had.
>
I'm very much in agreement here. Before using maven, I wrote the 
'fingerprint.jsp' in Apache Axis, which tries to provide a view of what 
jars the user has in the classpath, in order that people can publish 
'working' setups or quickly describe 'broken' setups. I tried to 
identify jar versions and was disappointed with the  practically 
nonexistent information in most manifests (including Sun's jars of 
javax.* stuff). I ended up just printing file sizes as a guide.

>
>Maven will eventually work so that all indirect dependencies will be
>gleaned, but if in the POM you state a version of one of those direct
>dependencies then the POM will be obeyed. You as a project developer
>will always have the choice, Maven will just try to help out.
>
I don't think indirect dependencies are quite so straightforward. The 
obvious example is JAXP; its not just versions of a dependency, but 
alternatives to that dependency. [xerces-1.4.4], or [xercesImpl with 
xml-apis], or [none required on JDK 1.4]. It is a PITA documenting all 
the dependencies, but being explicit helps avoid confusion about where a 
jar is coming from, and allows you to omit a dependency entirely.

>I'm just trying to make things easier, but I'm not going to take away
>rope that you could potentially hang yourself with ;-)
>  
>
I think there are conflicting goals here - 'maven should make things 
easy' and 'maven should promote best practice'. If you're not explicit 
about everything your project depends on, is your build really 
repeatable? On the other hand, being explicit makes it harder to get a 
build up and running ("just give me whatever version of xalan and xerces 
work together!").

A suggestion: instead of letting maven automatically resolve everything, 
why not let the user ask maven to write its suggested dependencies into 
the project.xml? eg "maven dependency:guess". The newbie gets up and 
running quickly with something that is truly 'best practice', the user 
with special requirements doesn't need to learn special 'opt outs', and 
it doesnt matter so much if maven's guesses at what the dependencies 
really are are a bit off.

Anyway, thanks Jason, that was a really good reply to my bellyaching!

-Baz

Privacy and Confidentiality Notice

------------------------------------------------

The information contained in this E-Mail message is intended only for the person or persons to whom it is addressed.  Such information is confidential and privileged and no mistake in transmission is intended to waive or compromise such privilege.  If you have received it in error, please destroy it and notify us on the telephone number printed above.  If you do not receive complete and legible copies, please telephone us immediately. Any opinions expressed herein including attachments are those of the author only. i-documentsystems Ltd. does not accept responsibility for the accuracy or completeness of the information provided or for any changes to this Email, however made, after it was sent. (Please note that it is your responsibility to scan this message for viruses).

Re: A tale of two builds (or: Why I don't like the SNAPSHOT sematics)

Posted by Kurt Schrader <ks...@karmalab.org>.

On 7 Jan 2003, Jason van Zyl wrote:

> On Tue, 2003-01-07 at 14:25, Brian Ewins wrote:
>
> > Kurt made a decent proposal in November for using metadata files instead
> > of a file naming convention[2]. There was a counter proposal from
>
> I think Kurt's idea that he picked up from ApacheCon conversations is a
> good one. I just asked him to hold off until a large refactoring was
> done. The refactoring is finished and I'm cleaning up at this point and
> Kurt has some time again. So I hope this proposal gets more attention.

I'll try to get an e-mail out in the next couple of days detailing where I
am with this, now that the refactoring is done.  Don't worry, I'm not
going to let it die.  Next week I'll be in Canada for a bit with Bob and
Jason annoying them with questions in order to try to get something in
before the b8 release.

-Kurt

Re: A tale of two builds (or: Why I don't like the SNAPSHOT sematics)

Posted by Jason van Zyl <ja...@zenplex.com>.

On Tue, 2003-01-07 at 14:25, Brian Ewins wrote:
> Henning P. Schmiedehausen wrote:
> 
> >Hm. Maybe I wasn't clear enough in this paragraph. This would still
> >allow the usage of "SNAPSHOT" in the project.xml. But you would get
> >the snap.yyyymmdd.hhmmss jars downloaded (and the small file which is
> >actually the same as a soft link but not resolved by the OS but by
> >maven. Which would help OSes which don't have soft links).
> >
> (erm you've touched on the thing I like least about maven here... 
> apologies in advance for the rant)
> 
> IMHO: This is just a symptom of a broader problem. The use of naming 
> conventions for dependencies in maven is screwed. Soft links are just 
> being used to cover up the lack of metadata.
> 
> The evidence is everywhere:
> - jars need renamed to be used with maven. How do you relate these jars 
> to the 'real' originals produced by some other project (there have been 
> several cases of jars renamed to the wrong versions)

You do not _need_ to rename them, but I have in many cases to try and
get some level of consistency in the artifact naming. Not fully
successful, I admit, but you can leave the jar called what you like
using the <jar/> element or by using the <groupId> and <artifactId>
elements in the dependency.

> - there is the ugly "axis+saaj" wart to express dependencies on 
> artifacts with 'the wrong name'

You can use <groupId> and <artifactId> if you wish, we were just trying
to maintain backward compatibility with fugly +/: notation.

> - you can't explicitly say "I want the copy of the jar I have in the 
> local repo" (on a jar by jar basis, not '-o') for jars which havent made 
> it to ibiblio yet - once they appear on ibiblio you'll get a 
> possibly-incorrect download. (I'm saying you should be able to 
> differentiate between a remote repository cache and a local repository; 
> and you should be able to be explicit about which repository to use)

maven.jar.override = true
maven.jar.my-project-id = /path/to/your/jar

or

maven.jar.my-project-id = 1.0

And your local repository will be searched first. Check out the
touchstone build for an example of how to do this. Yes, these features
are only available in HEAD.

> - the repo can't answer basic questions like, /which/ jars contain the 
> class 'x.y.z.X' that I'm to be missing? (a problem made more acute by 
> jar renaming)
> - the repo can't tell you which artifacts an artifact thinks it depends 
> on[1]

These two things are end goals but require a great deal of input by
users and a collection of POMs. I have added a feature which allows
uploading POMs, deployers for ssh/http/ftp and soon there will be
registered access to the central repo so projects can push their
information to the repository. These last two are not trivial we need
the shared database of project information and that won't come until
there is a reliable collection of POMs.

> Fundamentally, naming conventions just don't have the bandwidth to carry 
> all the information needed to properly describe an artifact, and 
> renaming destroys some of the information we /do/ have.

Naming conventions can certainly tell you something. Which is more
meaningful to you:

1) foo-1.0.jar
2) foo.jar

With maven we also try to push as much information as we can into the
manifest but I'm still in favour of any method which provides as much
information as possible at all levels. This is not to say I want to try
and jam in 50 character jar names to describe everything but I think
that versions are helpful in JAR name. We can certainly move toward
gleaning information from the manifest but these too currently are all
over the map. We did what we could with what we had.

> Kurt made a decent proposal in November for using metadata files instead 
> of a file naming convention[2]. There was a counter proposal from 

I think Kurt's idea that he picked up from ApacheCon conversations is a
good one. I just asked him to hold off until a large refactoring was
done. The refactoring is finished and I'm cleaning up at this point and
Kurt has some time again. So I hope this proposal gets more attention.

> Alexei[3] -  which stated that backward compatibility with the naming 
> convention was a requirement. I'd be -infinity on that one, maven hasn't 
> gone 1.0 yet, there should be time yet for getting rid of the dead wood. [4]
> 
> Did anything ever come of these? Or Jason reading up on the various 
> other package managers? (this was mentioned on irc).

See above.

> -Baz
> 
> [1] I don't particularly want maven to automatically download deep 
> dependencies - it hides information that should be in the project.xml, 
> and requires too much inter-project agreement on metadata. For instance, 
> if I need support for JAXP, there's a whole bunch of parsers I could 
> use; but some are not going to be compatible with particular apps due to 
> their bugs - so you typically can't just say "these are both JAXP1.1, I 
> can use either". You'd also need to decide between branches of a 
> product: eg if I want 'the latest tomcat compatible with 4.0.3' should I 
> use 4.0.6 or 4.1.1? But getting maven to report to me dependencies that 
> are 'known to work' without forcing me to use a projects declared 
> dependency is the correct approach.

Maven will eventually work so that all indirect dependencies will be
gleaned, but if in the POM you state a version of one of those direct
dependencies then the POM will be obeyed. You as a project developer
will always have the choice, Maven will just try to help out.

In addition, for projects that require a large amount of integration
(which are the type I'm used to) I will generally use the Maven override
feature to make sure I'm using the same version of project for the whole
overall project. So again once Maven can glean all the indirect
dependencies that information will be available for use. But if you
choose to override the JARs or state a preference in your POM (say for a
specific JAXP implementation) it will be obeyed.

The goal is too provide as much information as possible with very little
input. The times where Maven has forced something has usually been an
implementation oversight on my part. The rule of thumb is that the POM
has the final say and you as the developer of a project control the POM.
I'm just trying to make things easier, but I'm not going to take away
rope that you could potentially hang yourself with ;-)

> [2] 
> http://nagoya.apache.org/eyebrowse/ReadMsg?listName=turbine-maven-dev@jakarta.apache.org&msgNo=5568
> [3] 
> http://nagoya.apache.org/eyebrowse/ReadMsg?listName=turbine-maven-dev@jakarta.apache.org&msgNo=5896
> [4] Personally I felt both proposals got mired too quickly in describing 
> their xml representation without first explaining the 
> repository/artifact model that they're trying to achieve.
> 
> 
> 
> Privacy and Confidentiality Notice
> 
> ------------------------------------------------
> 
> The information contained in this E-Mail message is intended only for the person or persons to whom it is addressed.  Such information is confidential and privileged and no mistake in transmission is intended to waive or compromise such privilege.  If you have received it in error, please destroy it and notify us on the telephone number printed above.  If you do not receive complete and legible copies, please telephone us immediately. Any opinions expressed herein including attachments are those of the author only. i-documentsystems Ltd. does not accept responsibility for the accuracy or completeness of the information provided or for any changes to this Email, however made, after it was sent. (Please note that it is your responsibility to scan this message for viruses).
> 
> 
> --
> To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
> For additional commands, e-mail: <ma...@jakarta.apache.org>
-- 
jvz.

Jason van Zyl
jason@zenplex.com
http://tambora.zenplex.org

In short, man creates for himself a new religion of a rational
and technical order to justify his work and to be justified in it.

  -- Jacques Ellul, The Technological Society

Re: A tale of two builds (or: Why I don't like the SNAPSHOT sematics)

Posted by Brian Ewins <Br...@i-documentsystems.com>.

Henning P. Schmiedehausen wrote:

>Hm. Maybe I wasn't clear enough in this paragraph. This would still
>allow the usage of "SNAPSHOT" in the project.xml. But you would get
>the snap.yyyymmdd.hhmmss jars downloaded (and the small file which is
>actually the same as a soft link but not resolved by the OS but by
>maven. Which would help OSes which don't have soft links).
>
(erm you've touched on the thing I like least about maven here...
apologies in advance for the rant)

IMHO: This is just a symptom of a broader problem. The use of naming
conventions for dependencies in maven is screwed. Soft links are just
being used to cover up the lack of metadata.

The evidence is everywhere:
- jars need renamed to be used with maven. How do you relate these jars
to the 'real' originals produced by some other project (there have been
several cases of jars renamed to the wrong versions)
- there is the ugly "axis+saaj" wart to express dependencies on
artifacts with 'the wrong name'
- you can't explicitly say "I want the copy of the jar I have in the
local repo" (on a jar by jar basis, not '-o') for jars which havent made
it to ibiblio yet - once they appear on ibiblio you'll get a
possibly-incorrect download. (I'm saying you should be able to
differentiate between a remote repository cache and a local repository;
and you should be able to be explicit about which repository to use)
- the repo can't answer basic questions like, /which/ jars contain the
class 'x.y.z.X' that I'm to be missing? (a problem made more acute by
jar renaming)
- the repo can't tell you which artifacts an artifact thinks it depends
on[1]

Fundamentally, naming conventions just don't have the bandwidth to carry
all the information needed to properly describe an artifact, and
renaming destroys some of the information we /do/ have.

Kurt made a decent proposal in November for using metadata files instead
of a file naming convention[2]. There was a counter proposal from
Alexei[3] - which stated that backward compatibility with the naming
convention was a requirement. I'd be -infinity on that one, maven hasn't
gone 1.0 yet, there should be time yet for getting rid of the dead wood. [4]

Did anything ever come of these? Or Jason reading up on the various
other package managers? (this was mentioned on irc).

-Baz

[1] I don't particularly want maven to automatically download deep
dependencies - it hides information that should be in the project.xml,
and requires too much inter-project agreement on metadata. For instance,
if I need support for JAXP, there's a whole bunch of parsers I could
use; but some are not going to be compatible with particular apps due to
their bugs - so you typically can't just say "these are both JAXP1.1, I
can use either". You'd also need to decide between branches of a
product: eg if I want 'the latest tomcat compatible with 4.0.3' should I
use 4.0.6 or 4.1.1? But getting maven to report to me dependencies that
are 'known to work' without forcing me to use a projects declared
dependency is the correct approach.
[2]
http://nagoya.apache.org/eyebrowse/ReadMsg?listName=turbine-maven-dev@jakarta.apache.org&msgNo=5568
[3]
http://nagoya.apache.org/eyebrowse/ReadMsg?listName=turbine-maven-dev@jakarta.apache.org&msgNo=5896
[4] Personally I felt both proposals got mired too quickly in describing
their xml representation without first explaining the
repository/artifact model that they're trying to achieve.

Privacy and Confidentiality Notice

------------------------------------------------

The information contained in this E-Mail message is intended only for the person or persons to whom it is addressed. Such information is confidential and privileged and no mistake in transmission is intended to waive or compromise such privilege. If you have received it in error, please destroy it and notify us on the telephone number printed above. If you do not receive complete and legible copies, please telephone us immediately. Any opinions expressed herein including attachments are those of the author only. i-documentsystems Ltd. does not accept responsibility for the accuracy or completeness of the information provided or for any changes to this Email, however made, after it was sent. (Please note that it is your responsibility to scan this message for viruses).