You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Ben Walding <be...@walding.com> on 2003/05/16 00:30:22 UTC
Artifact Downloading
There have recently been a lot of bugs found in the artifact
downloading, I introduced some of them (and I believe those are fixed now).
I think some of the confusion is coming from
A) What the correct procedure should be
B) The implementation is showing its age and needs to be cleaned a bit.
I don't think it is badly broken, just some of the contracts between
components are weak and need to be locked down via unit tests.
C) Lack of tests to prevent regressions
To ensure that nothing is missed (The wiki entry should attempt to
capture the scenarios so they are all tested)
http://nagoya.apache.org/wiki/apachewiki.cgi?Maven/ArtifactDownloading
If Ruper has achieved anything - I'll look at a conversion to that.
If Ruper has achieved nothing sensible -
I'll write a test harness using jetty (eg create "remote" repos using
jetty and test downloads across a variety of scenarios)
I'll aggregate comments into the main entry as appropriate.
I'm not looking to shake up downloading, just lock it down.
Cheers,
Ben
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Artifact Downloading
Posted by Ben Walding <de...@walding.com>.
Mark H. Wilkinson wrote:
>I can think of a couple of related things that probably should be
>covered in the spec. First up, what protocols are we expecting to be
>used to retrieve artifacts?
>
http and file
>There may be value in supporting local
>filesystem repositories for those who don't have a web server kicking
>around. Is there any point in supporting ftp? Should maven reject those
>which aren't explicitly supported and tested?
>
For the time being I'm only planning on testing http and possibly file.
I'll put in a warning for unknown url schemes so as to warn if you use
something that is untested.
>Second point: what's the actual method used for determining whether a
>newer snapshot dependency exists on the remote repository? You mention
>not using file modification time.
>
Only because I think it could be a bad thing to do. For the time being
it will suffice if the repository upload plugin sets the time correctly.
>Does it require the ability to perform
>the moral equivalent of 'ls http://blah/maven/group/jars/', which is
>somewhat tricky for protocols like http, less so for ftp and local
>files. Does it depend on certain admin practices on the server end (like
>having a link called artifact-SNAPSHOT.jar)?
>
In the first instance I'll probably just have it grab the SNAPSHOT and
check the file mod time.
I really don't want to put another file in the repository eg a
properties file to carry more metadata
>>I'll write a test harness using jetty (eg create "remote" repos using
>>jetty and test downloads across a variety of scenarios)
>>
>>
>
>This presupposes using http as the retrieval method - might need to
>think about other approaches for other download protocols.
>
As required. Writing a test harness for a file:// based system is
fairly simple. ftp:// less so.
>Need any help?
>
Should be ok for the first round, although anyone who wants to test it
will be most welcome.
Cheers,
Ben
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Artifact Downloading
Posted by Jason van Zyl <ja...@zenplex.com>.
On Tue, 2003-05-20 at 16:16, Colin Sampaleanu wrote:
> Jason van Zyl wrote:
>
> I see pros and cons to each approach, so I'm just trying to figure out
> why putting the version in the name is so absolutely the right thing to
> do...
Anything that lends clarity is the right thing to do. If you can look at
a JAR and tell what it is then that's a good thing. Ultimately any
Maven-based tool should know how to find the correct artifact in a
determinstic way. The version in the artifact name is purely a visual
helper for someone looking at the artifact. Having it in the path
doesn't really help you much when you copy it somewhere by mistake and
you need to figure out what it is. Or even worse if you copied
1.1/foo.jar into 1.2/foo.jar and didn't notice. Any Maven-based tool
working with artifacts shouldn't be thrown off by a little eye candy in
the artifact name.
> Regards,
> Colin
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
--
jvz.
Jason van Zyl
jason@zenplex.com
http://tambora.zenplex.org
In short, man creates for himself a new religion of a rational
and technical order to justify his work and to be justified in it.
-- Jacques Ellul, The Technological Society
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Artifact Downloading
Posted by Colin Sampaleanu <co...@exis.com>.
Jason van Zyl wrote:
>On Tue, 2003-05-20 at 15:35, Michal Maczka wrote:
>
>
>>I also want to put for public discussion following subject:
>>
>>Are we are going to have artifacts without version name included in file
>>name?
>>
>>
>-1
>
>> think that for some types of artifacts this will "a must" at least in
>>local repository.
>>For example for Tag Descriptors (tld files) is much easier to use files
>>without versions
>>then to change in web.xml in such cases when jstl change version form 1.01
>>to 1.02
>>
>>
>Providing any human readable information in the name of the artifact is
>IMO very important. If you have files where you don't want to have to
>update then you can always have some sort of step where you do the name
>translation i.e. if you want to copy foo-{1.0|1.1|1.2}.jar to just
>foo.jar then that's your problem.
>
>I am vehemently opposed to placing things in the repository without the
>version in the name of the artifact.
>
>Any sort of aliasing mechanisms can either be added in a standard way if
>that's something people desire or you can always rig something yourself.
>
>>f such artifacts should be supported by Maven- sorry to say -but the
>>solution which was disused is rather obsolete.
>>
>>
>In some extension but the artifacts cannot be stored in the repository
>without versioning information in the name.
>
>I can think of a simple extension where you might customize the
>copy-deps behavior which strips out the version information if you so
>desire. But the artifacts will always be placed in the repositories with
>version information.
>
To play the devil's advocate, why such a hard and fast rule? The version
number is metadata, along with other metadata which is currently
represented in the directory path. If (for example) putting the version
in the directory name makes it easier to alias to the file (a pretty
common situation, for example, we have local repos checked into CVS that
are used by maven but also used by stand-alone ant scripts for shared
libs), why is this inherently worst? The version is still very easy to
see, being in the pathname. You do have to redecorate the name on
copying it out of the repo if there is a need to have the version in the
name, but for some types of artifacts (not jars, and sometimes even for
jars) you'd normally not want the version in the name.
I see pros and cons to each approach, so I'm just trying to figure out
why putting the version in the name is so absolutely the right thing to
do...
Regards,
Colin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
RE: Artifact Downloading
Posted by Michal Maczka <mm...@interia.pl>.
> -----Original Message-----
> From: Jason van Zyl [mailto:jason@zenplex.com]
> Sent: Tuesday, May 20, 2003 5:32 PM
> To: Maven Developers List
> Subject: RE: Artifact Downloading
>
>
> On Tue, 2003-05-20 at 15:35, Michal Maczka wrote:
> > I also want to put for public discussion following subject:
> >
> > Are we are going to have artifacts without version name included in file
> > name?
>
> -1
>
> > I think that for some types of artifacts this will "a must" at least in
> > local repository.
> > For example for Tag Descriptors (tld files) is much easier to use files
> > without versions
> > then to change in web.xml in such cases when jstl change
> version form 1.01
> > to 1.02
>
> Providing any human readable information in the name of the artifact is
> IMO very important. If you have files where you don't want to have to
> update then you can always have some sort of step where you do the name
> translation i.e. if you want to copy foo-{1.0|1.1|1.2}.jar to just
> foo.jar then that's your problem.
>
I agree.
> I am vehemently opposed to placing things in the repository without the
> version in the name of the artifact.
>
> Any sort of aliasing mechanisms can either be added in a standard way if
> that's something people desire or you can always rig something yourself.
>
> > If such artifacts should be supported by Maven- sorry to say -but the
> > solution which was disused is rather obsolete.
>
> In some extension but the artifacts cannot be stored in the repository
> without versioning information in the name.
>
> I can think of a simple extension where you might customize the
> copy-deps behavior which strips out the version information if you so
> desire. But the artifacts will always be placed in the repositories with
> version information.
>
So we will lost version information will be lost any way...
That's why I said:"Realization of this idea will make many things much
simpler and many other
much harder..."
I feel that information about version in filename has at least two purposes:
o It is used for organizning repository
o It makes artifact directly recognizable by human when it used. So e.g.:
he/she knows: I am using: tigris-2.3.css
Unfortunatly second goal (which I percive, probably simillary to you, as
even more important),
if we want or not, will be in many cases for some types of artifacts simply
not reachable,
becouse it is to expenisive.
Just wanted to ask this questoin before some final desing choices
will be made so all the requirments are known before.
Michal
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
RE: Artifact Downloading
Posted by Jason van Zyl <ja...@zenplex.com>.
On Tue, 2003-05-20 at 15:35, Michal Maczka wrote:
> I also want to put for public discussion following subject:
>
> Are we are going to have artifacts without version name included in file
> name?
-1
> I think that for some types of artifacts this will "a must" at least in
> local repository.
> For example for Tag Descriptors (tld files) is much easier to use files
> without versions
> then to change in web.xml in such cases when jstl change version form 1.01
> to 1.02
Providing any human readable information in the name of the artifact is
IMO very important. If you have files where you don't want to have to
update then you can always have some sort of step where you do the name
translation i.e. if you want to copy foo-{1.0|1.1|1.2}.jar to just
foo.jar then that's your problem.
I am vehemently opposed to placing things in the repository without the
version in the name of the artifact.
Any sort of aliasing mechanisms can either be added in a standard way if
that's something people desire or you can always rig something yourself.
> If such artifacts should be supported by Maven- sorry to say -but the
> solution which was disused is rather obsolete.
In some extension but the artifacts cannot be stored in the repository
without versioning information in the name.
I can think of a simple extension where you might customize the
copy-deps behavior which strips out the version information if you so
desire. But the artifacts will always be placed in the repositories with
version information.
>
> In Maven-new I have marked how repository layouts for tlds files can look
> like:
>
>
> tld=${artifactId}/${groupId}/${tlds}/${artifactId}/${version}/${id}.tld
> or
> tld=${artifactId}/${groupId}/${tlds}/${artifactId}-${version}/${id}.tld
>
>
> so we can have for example:
>
> struts/tlds/html/1.1rc1/html.tld
> # or
> struts/tlds/html-1.1rc1/html.tld
>
>
> Realization of this idea will make many things much simpler and many other
> much harder...
> So decision is not so obvious
>
> Michal
>
>
>
> > -----Original Message-----
> > From: Rafal Krzewski [mailto:Rafal.Krzewski@caltha.pl]
> > Sent: Monday, May 19, 2003 10:51 AM
> > To: Maven Developers List
> > Subject: Re: Artifact Downloading
> >
> >
> > Brian Ewins wrote:
> > > Maven is a little confusing on this point. If you do:
> > > maven jar:deploy-snapshot
> > >
> > > and you are building 'foo' version '1.4', it creates a build like:
> > > foo-1.4.20030202.153859.jar
> > >
> > > and uploads it. Clearly, this is supposed to be the latest version of
> > > foo-1.4.jar, so either
> > > foo-1.4-SNAPSHOT.jar
> > > should be created also (by linking or copying). What the timestamp will
> > > be on the server is moot.
> > >
> > > However if you do a release build:
> > > foo-1.4.jar
> > >
> > > this is *also* the latest version of foo-1.4.jar, and by rights,
> > > foo-1.4-SNAPSHOT.jar should now be a copy of or a link to this jar,
> > > despite the fact that you didn't upload it as a snapshot!
> >
> > Good catch!
> >
> > > So what should really happen here? Uploading a second copy of the jar
> > > each time would work, but seems odd, and won't fix the timestamp issue.
> > > Also if the declared version of the artifact you are building is
> > > actually "1.4-SNAPSHOT", then surely only one version should be uploaded
> > > - you're explicitly declaring that this is not a release.
> > >
> > > The alternative is to upload some smaller metadata file with each build;
> > > this would identify the real current version without linking or copying,
> > > and should have a sequence no (not timestamp) in it as well. This would
> > > have to be uploaded with every single build to handle the issue that
> > > release builds are actually snapshots (in the sense of latest versions)
> > > too.
> >
> > I noticed that we need such a metadata file earlier, during the
> > build repetability discussion few months ago. My idea was that we need
> > a file that contains the information what was the 'newest SNAPSHOT' of
> > an artifact in a given point in the past. Such metadata file solves
> > also other concerns. More on my idea further down.
> >
> > > This is not what the download mechanism does right now, but I've
> > > seen snapshot info files up on ibiblio? Was someone trying this?
> >
> > Quite possible, discussions about this were going on for some time.
> >
> > > Essentially this would enable you to do something like an http redirect,
> > > but you'd probably want to save the download as 'foo-1.4-SNAPSHOT'
> > > rather than whatever the real file turned out to be, since otherwise a
> > > lot of plugins might break.
> >
> > Agreed, current practice requires -SNAPSHOT files in the local repo, we
> > should try to get rid of them in the remote (distribution) repositories
> > first.
> >
> > > To be concrete, what I'm proposing would work like this:
> > <snip>
> > > Make sense?
> >
> > Sure it does!, Now I'll tell a bit about my approach:
> >
> > Every time an artifact is deployed to the remote repo, the deployment
> > script appends a single line to the metadata file. In this line, there
> > is a timestamp, a flag indicating if this is a snapshot or release build
> > and a filename (or optioanlly a pathname relative to the artifact's
> > directory or absolute in the HTTP server's namespace, to give repository
> > maintainers more freedom).
> >
> > A word of explaination about snapshot/release flag. It would be used to
> > detect the sitiuation when someone requests artifact-1.4.jar, but only
> > SNAPSHOT versions are available. Thats the easy part. Another story
> > would be the situation when the metadata file contains more than one
> > versions flagged as release. Under normal circumstances this should not
> > happen - when you release, it's done. BUT when the artifact's author
> > decides to break the policy and redeploy a released artifact,
> > metainformation about this will be saved, AND repeptability of builds
> > can be maintained because the dependency resolution code will be able
> > to determine what was the 'newest released' version of artifact-1.4.jar
> > at a given point in the past.
> >
> > This kind of metadata file solves the directory listing concerns raised
> > by Mark Wilkinson. Maven would gather information about available
> > versions from the metadata file, abstracting from the trasport protocol
> > capabilities.
> >
> > The artifact sequence number concept of Brian's can be 1-1 mapped to the
> > number of entries in the metadata file, giving us the same download
> > optimization benefits (if I understood the idea correctly).
> >
> > R.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
> > ----------------------------------------------------------------------
> > Rozmowy na WARSZAWSKIE tematy >>> http://link.interia.pl/f1727
> >
> >
> >
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
--
jvz.
Jason van Zyl
jason@zenplex.com
http://tambora.zenplex.org
In short, man creates for himself a new religion of a rational
and technical order to justify his work and to be justified in it.
-- Jacques Ellul, The Technological Society
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Artifact Downloading
Posted by Jason van Zyl <ja...@zenplex.com>.
On Wed, 2003-05-21 at 11:14, Rafal Krzewski wrote:
> michal.maczka wrote:
>
> > I like this idea. It will be nice to see how "component oriented"
> > technologies
> > like JavaServerFaces will fit into it.
> >
> > I haven't seen any more complex JSF implementation.
> > For using MyServerFaces (open source project @sourceforge) it is required
> > to install some jars and tag descriptors.
> > MyServerFaces currently supports just basic set of widgets. Once more
> > complex widgets will be supported
> > by any JSF implementation
> > (e.g. trees) probably CSS/JS files might be required as well
> > (I might be wrong...as I don't know JSF spec and maybe this will be resolved
> > differently).
> >
> > It will be cool to support easy integration with such components.
> >
> > It they will be distributed as one archives...well my concerns about
> > artifacts versioning are vanished.
>
> Great to hear that! Convincing you took a good part of my day today,
> but I think it was worth it -- we came to a nice new idea!
I do something like this with the style.tigris.org project. It's a set
of css files, javascript files and images so you can make nice web
interfaces. Everything is just bundled up in a jar, I put it in my local
repo and state it as a dependency. During deployment I nab it and unpack
it into my webapp space before bundling it up.
> R.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
--
jvz.
Jason van Zyl
jason@zenplex.com
http://tambora.zenplex.org
In short, man creates for himself a new religion of a rational
and technical order to justify his work and to be justified in it.
-- Jacques Ellul, The Technological Society
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Artifact Downloading
Posted by Rafal Krzewski <Ra...@caltha.pl>.
michal.maczka wrote:
> I like this idea. It will be nice to see how "component oriented"
> technologies
> like JavaServerFaces will fit into it.
>
> I haven't seen any more complex JSF implementation.
> For using MyServerFaces (open source project @sourceforge) it is required
> to install some jars and tag descriptors.
> MyServerFaces currently supports just basic set of widgets. Once more
> complex widgets will be supported
> by any JSF implementation
> (e.g. trees) probably CSS/JS files might be required as well
> (I might be wrong...as I don't know JSF spec and maybe this will be resolved
> differently).
>
> It will be cool to support easy integration with such components.
>
> It they will be distributed as one archives...well my concerns about
> artifacts versioning are vanished.
Great to hear that! Convincing you took a good part of my day today,
but I think it was worth it -- we came to a nice new idea!
R.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
RE: Artifact Downloading
Posted by "michal.maczka" <mi...@cqs.ch>.
> -----Original Message-----
> From: Rafal Krzewski [mailto:Rafal.Krzewski@caltha.pl]
> Sent: Wednesday, May 21, 2003 4:36 PM
> To: Maven Developers List
> Subject: Re: Artifact Downloading
>
>
> michal.maczka wrote:
>
> > I am not thinking about single image file but about more
> complex use cases.
> > It will be nice to say: I use strutsmenu (group dependecy)
> > Strutsmenu delivers a set of java classes, css files, javascripts images
> > etc.
> > in your project (which delivers war artifacts) and all those artifacts
> > will be bundled in your war.
>
> Arent those web content files supposed to be bundled in a some sort of
> archive (zip or jar, probably)? I think it would be reasonable to have
> artifact type of 'web-content' that you could make your war depend on.
> War plugin would uncompress the contents of such archive into the root
> directory of your webapp. I think this is much much easier to work with
> than have each and every file provided as an artifact in the repo.
>
> R.
I like this idea. It will be nice to see how "component oriented"
technologies
like JavaServerFaces will fit into it.
I haven't seen any more complex JSF implementation.
For using MyServerFaces (open source project @sourceforge) it is required
to install some jars and tag descriptors.
MyServerFaces currently supports just basic set of widgets. Once more
complex widgets will be supported
by any JSF implementation
(e.g. trees) probably CSS/JS files might be required as well
(I might be wrong...as I don't know JSF spec and maybe this will be resolved
differently).
It will be cool to support easy integration with such components.
It they will be distributed as one archives...well my concerns about
artifacts versioning are vanished.
Michal
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Artifact Downloading
Posted by Rafal Krzewski <Ra...@caltha.pl>.
michal.maczka wrote:
> I am not thinking about single image file but about more complex use cases.
> It will be nice to say: I use strutsmenu (group dependecy)
> Strutsmenu delivers a set of java classes, css files, javascripts images
> etc.
> in your project (which delivers war artifacts) and all those artifacts
> will be bundled in your war.
Arent those web content files supposed to be bundled in a some sort of
archive (zip or jar, probably)? I think it would be reasonable to have
artifact type of 'web-content' that you could make your war depend on.
War plugin would uncompress the contents of such archive into the root
directory of your webapp. I think this is much much easier to work with
than have each and every file provided as an artifact in the repo.
R.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
RE: Artifact Downloading
Posted by "michal.maczka" <mi...@cqs.ch>.
[..]
> > CSS/ Javascripts/ Images(like company logo) and all other sorts of files
> > which you use in HTML pages.
> > Once you change a version of such file ... you don't want to
> edit your HTML
> > page.
>
> Are you sure you would like to have a single image file held as an
> artifact in the repository? I think that such files are usually bundled
> together as a part of project souce (thus end up in versioned 'sources'
> artifact), project's binary artifact (probably a war), or bundled
> project documentation (which is bound to become 'docs' artifact
> sometime).
>
I am not thinking about single image file but about more complex use cases.
It will be nice to say: I use strutsmenu (group dependecy)
Strutsmenu delivers a set of java classes, css files, javascripts images
etc.
in your project (which delivers war artifacts) and all those artifacts
will be bundled in your war.
[..]
Michal
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Artifact Downloading
Posted by Rafal Krzewski <Ra...@caltha.pl>.
michal.maczka wrote:
> There is no such artifact which must not have version number in repository.
> But there are/will be some with version number dropped at some stage.
>
> Example:
> maven project:get-project -Dname=commons-lang
>
> as a result you can get artifact "project.xml" not "project-1.01.xml" in
> your current folder.
> After you can use project.xml to check out project from CVS repository.
Ummm, I think you need to specify the version of the project you want do
download, becuse otherwise the plugin would have to discover available
versions and choose one (presumably the newest) for you. Once the file
is downloaded it will be renamed to project.xml, but this is done by the
project plugin and *not* the dependency handling code.
> CSS/ Javascripts/ Images(like company logo) and all other sorts of files
> which you use in HTML pages.
> Once you change a version of such file ... you don't want to edit your HTML
> page.
Are you sure you would like to have a single image file held as an
artifact in the repository? I think that such files are usually bundled
together as a part of project souce (thus end up in versioned 'sources'
artifact), project's binary artifact (probably a war), or bundled
project documentation (which is bound to become 'docs' artifact
sometime).
Each of the css, image, script files are versioned independently in CVS
(or other SCM), and project releases are place "tags" on these files,
by deploying them to the repo in an artifact carrying specific project
version number. Of course it's good idea to create a proper symbolic
tag in the SCM at the release point, but that's a different story.
>>If this is ever necessary, version information should be dropped as late
>>as it is possible.
>
> So I think that we have agreement between Jason, you & me here.
> Only problem is that this will required some magical/cryptic post processing
> and this can couse many troubles. So this is for sure not a choice without
> cons.
By 'as late as possible' probably means outside Maven core. I also don't
precieve much trouble with that post processing -- once you get hold on
an artifact object, you will be able to querry the solid filesystem
path of the artifact's location in the local repo, as well as the
artifacts name, version and extension. Do you need anything else to
copy the file to traget location with the version stripped from the
name?
> If we have choice left open if artifact file name has always "version" in it
> or not,
> we will have even less troubles as in such case we will be able to handle
> some
> pathological cases easily (if such case arise).
> Preservation of such flexibility has a cost - the code will have to be bent
> accordingly.
> I don't think that there are many places, which will be affected. SNAPSHOT
> resolving
> algorithm is one of them.
I have trouble understanding what you mean here...
> To summarize - my votes:
-1 to any "versionless" artifact handling by the Maven core.
R.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
RE: Artifact Downloading
Posted by "michal.maczka" <mi...@cqs.ch>.
> -----Original Message-----
> From: Rafal Krzewski [mailto:Rafal.Krzewski@caltha.pl]
> Sent: Wednesday, May 21, 2003 1:02 PM
> To: Maven Developers List
> Subject: Re: Artifact Downloading
>
>
> michal.maczka wrote:
>
> > I suppose that there will be other artifacts/dependices which should
> > become versionless in the end.
>
> I'm sorry but I completely miss what you are pursuing in the first
> place... Can you please give an example of artifact types that
> *must not* have their version number in the file name?
>
There is no such artifact which must not have version number in repository.
But there are/will be some with version number dropped at some stage.
Example:
maven project:get-project -Dname=commons-lang
as a result you can get artifact "project.xml" not "project-1.01.xml" in
your current folder.
After you can use project.xml to check out project from CVS repository.
Other examples:
CSS/ Javascripts/ Images(like company logo) and all other sorts of files
which you use in HTML pages.
Once you change a version of such file ... you don't want to edit your HTML
page.
> > In fact I see that ulimate goal
> > is/was not to have versionless artifacts even when Maven scope ends.
> > SO you (human) look at any file and know exactly what it is.
> > But I am almos sure that this goal cannot be achieved in case
> > of some types of artifacts. So this means that we are going
> > to fail with this goal anyway. My question was: When
> > are we going to fail: alredy in repository or later when we will
> > cut off version string from a filename.
>
> If this is ever necessary, version information should be dropped as late
> as it is possible.
So I think that we have agreement between Jason, you & me here.
Only problem is that this will required some magical/cryptic post processing
and this can couse many troubles. So this is for sure not a choice without
cons.
>Any other approach is IMO asking for trouble.
>
If we have choice left open if artifact file name has always "version" in it
or not,
we will have even less troubles as in such case we will be able to handle
some
pathological cases easily (if such case arise).
Preservation of such flexibility has a cost - the code will have to be bent
accordingly.
I don't think that there are many places, which will be affected. SNAPSHOT
resolving
algorithm is one of them.
To summarize - my votes:
a) do we support files without a version in their names in the
repository: -1
b) can our code internally support them: -1 (if it is hard to implement)
+1 (if we can have it almost for
free)
Michal
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Artifact Downloading
Posted by Rafal Krzewski <Ra...@caltha.pl>.
michal.maczka wrote:
> I suppose that there will be other artifacts/dependices which should
> become versionless in the end.
I'm sorry but I completly miss what you are pursuing in the first
place... Can you please give an example of artifact types that
*must not* have their version number in the file name?
> In fact I see that ulimate goal
> is/was not to have versionless artifacts even when Maven scope ends.
> SO you (human) look at any file and know exactly what it is.
> But I am almos sure that this goal cannot be achieved in case
> of some types of artifacts. So this means that we are going
> to fail with this goal anyway. My question was: When
> are we going to fail: alredy in repository or later when we will
> cut off version string from a filename.
If this is ever necessary, version information should be dropped as late
as it is possible. Any other approach is IMO asking for trouble.
> I see that for repository
> (as repository is "for computers" rather then "for humans") it does not make
> big deal if we have
>
> struts/tlds/html/1.1rc1/html.tld
> # or
> struts/tlds/html-1.1rc1/html.tld
>
> Required metadata is anyway there.
>
> But some "magical processing" will be applied later on.
Sure, the metadata is there but what win does this give us over the
current scheme?
> And for any Mavenized project we have POM which tells us any which version
> of artifacts
> are we using, so such information is quickly accesible.
As long as you know the exact revision of the POM your particular
instance of application was compiled from...
Imagine the following scenario: You get a service call from a customer.
You log on to the application and replicate the problem. You suspect
that the problem migh be caused by a bug in Struts taglib, but you
are not sure. You log on to the produciton server and look on a deployed
war. Now, it contains html.tld. You obviously don't know what version
of Struts was used to build that jar (sombebody else build it and
deployed it.). If the deployed war contains the information about
the particular revision of the project.xml that was used for building,
you can use your SCM client and retrieve that revision of POM and
check what version of struts it was.... If the deployed war does
not contain that information -- well, you're screwed...
I would much rather have the version information on all artifacts, Michal...
> I am +0, but I think it was nice to discuss this issue - as versionless
> artifacts
> could change "a rules of the game" for many things (e.g. how to resolve
> snapshot dependencies).
Snapshots mix nicely with versions, and if there is a need for dropping
version info (I sincerely hope there is none), the drop would occur
after snapshot is resolved, so I see no particular problem here.
R.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
RE: Artifact Downloading
Posted by "michal.maczka" <mi...@cqs.ch>.
> -----Original Message-----
> From: Rafal Krzewski [mailto:Rafal.Krzewski@caltha.pl]
> Sent: Wednesday, May 21, 2003 8:53 AM
> To: Maven Developers List
> Subject: Re: Artifact Downloading
>
>
> Michal Maczka wrote:
> > I also want to put for public discussion following subject:
> >
> > Are we are going to have artifacts without version name included in file
> > name?
> >
> > I think that for some types of artifacts this will "a must" at least in
> > local repository.
> > For example for Tag Descriptors (tld files) is much easier to use files
> > without versions
> > then to change in web.xml in such cases when jstl change
> version form 1.01
> > to 1.02
>
> In this particualr case, I think it would be much better if that part of
> web.xml was automagically generated by the war plugin. You make your
> war-producing project depend on struts-html 1.02, and hey presto - you
> have struts-html-1.02.tld included in your web.xml. This way you would
> be altering only your project's POM when you want to upgrade your
> dependencie's version.
>
I suppose that there will be other artifacts/dependices which should
become versionless in the end. In fact I see that ulimate goal
is/was not to have versionless artifacts even when Maven scope ends.
SO you (human) look at any file and know exactly what it is.
But I am almos sure that this goal cannot be achieved in case
of some types of artifacts. So this means that we are going
to fail with this goal anyway. My question was: When
are we going to fail: alredy in repository or later when we will
cut off version string from a filename.
I see that for repository
(as repository is "for computers" rather then "for humans") it does not make
big deal if we have
struts/tlds/html/1.1rc1/html.tld
# or
struts/tlds/html-1.1rc1/html.tld
Required metadata is anyway there.
But some "magical processing" will be applied later on.
And for any Mavenized project we have POM which tells us any which version
of artifacts
are we using, so such information is quickly accesible.
> BTW, I'm -1 to versionless artifact names too.
>
I am +0, but I think it was nice to discuss this issue - as versionless
artifacts
could change "a rules of the game" for many things (e.g. how to resolve
snapshot dependencies).
Coming back to snapshot dependencies... even if we are not going to support
versionless
dependencies in Maven World, it does not mean that it is not possible to
construct an algorithm which will work also
with such artifacts. It is clear that current sketch of algorithm does not
support
this feature. If somebody has an idea how without making it more complicated
(or even simplfy it ;) )
make it also open for such feature...this would be just greate.
Michal
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Artifact Downloading
Posted by Rafal Krzewski <Ra...@caltha.pl>.
Michal Maczka wrote:
> I also want to put for public discussion following subject:
>
> Are we are going to have artifacts without version name included in file
> name?
>
> I think that for some types of artifacts this will "a must" at least in
> local repository.
> For example for Tag Descriptors (tld files) is much easier to use files
> without versions
> then to change in web.xml in such cases when jstl change version form 1.01
> to 1.02
In this particualr case, I think it would be much better if that part of
web.xml was automagically generated by the war plugin. You make your
war-producing project depend on struts-html 1.02, and hey presto - you
have struts-html-1.02.tld included in your web.xml. This way you would
be altering only your project's POM when you want to upgrade your
dependencie's version.
BTW, I'm -1 to versionless artifact names too.
R.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
RE: Artifact Downloading
Posted by Michal Maczka <mm...@interia.pl>.
I also want to put for public discussion following subject:
Are we are going to have artifacts without version name included in file
name?
I think that for some types of artifacts this will "a must" at least in
local repository.
For example for Tag Descriptors (tld files) is much easier to use files
without versions
then to change in web.xml in such cases when jstl change version form 1.01
to 1.02
If such artifacts should be supported by Maven- sorry to say -but the
solution which was disused is rather obsolete.
In Maven-new I have marked how repository layouts for tlds files can look
like:
tld=${artifactId}/${groupId}/${tlds}/${artifactId}/${version}/${id}.tld
or
tld=${artifactId}/${groupId}/${tlds}/${artifactId}-${version}/${id}.tld
so we can have for example:
struts/tlds/html/1.1rc1/html.tld
# or
struts/tlds/html-1.1rc1/html.tld
Realization of this idea will make many things much simpler and many other
much harder...
So decision is not so obvious
Michal
> -----Original Message-----
> From: Rafal Krzewski [mailto:Rafal.Krzewski@caltha.pl]
> Sent: Monday, May 19, 2003 10:51 AM
> To: Maven Developers List
> Subject: Re: Artifact Downloading
>
>
> Brian Ewins wrote:
> > Maven is a little confusing on this point. If you do:
> > maven jar:deploy-snapshot
> >
> > and you are building 'foo' version '1.4', it creates a build like:
> > foo-1.4.20030202.153859.jar
> >
> > and uploads it. Clearly, this is supposed to be the latest version of
> > foo-1.4.jar, so either
> > foo-1.4-SNAPSHOT.jar
> > should be created also (by linking or copying). What the timestamp will
> > be on the server is moot.
> >
> > However if you do a release build:
> > foo-1.4.jar
> >
> > this is *also* the latest version of foo-1.4.jar, and by rights,
> > foo-1.4-SNAPSHOT.jar should now be a copy of or a link to this jar,
> > despite the fact that you didn't upload it as a snapshot!
>
> Good catch!
>
> > So what should really happen here? Uploading a second copy of the jar
> > each time would work, but seems odd, and won't fix the timestamp issue.
> > Also if the declared version of the artifact you are building is
> > actually "1.4-SNAPSHOT", then surely only one version should be uploaded
> > - you're explicitly declaring that this is not a release.
> >
> > The alternative is to upload some smaller metadata file with each build;
> > this would identify the real current version without linking or copying,
> > and should have a sequence no (not timestamp) in it as well. This would
> > have to be uploaded with every single build to handle the issue that
> > release builds are actually snapshots (in the sense of latest versions)
> > too.
>
> I noticed that we need such a metadata file earlier, during the
> build repetability discussion few months ago. My idea was that we need
> a file that contains the information what was the 'newest SNAPSHOT' of
> an artifact in a given point in the past. Such metadata file solves
> also other concerns. More on my idea further down.
>
> > This is not what the download mechanism does right now, but I've
> > seen snapshot info files up on ibiblio? Was someone trying this?
>
> Quite possible, discussions about this were going on for some time.
>
> > Essentially this would enable you to do something like an http redirect,
> > but you'd probably want to save the download as 'foo-1.4-SNAPSHOT'
> > rather than whatever the real file turned out to be, since otherwise a
> > lot of plugins might break.
>
> Agreed, current practice requires -SNAPSHOT files in the local repo, we
> should try to get rid of them in the remote (distribution) repositories
> first.
>
> > To be concrete, what I'm proposing would work like this:
> <snip>
> > Make sense?
>
> Sure it does!, Now I'll tell a bit about my approach:
>
> Every time an artifact is deployed to the remote repo, the deployment
> script appends a single line to the metadata file. In this line, there
> is a timestamp, a flag indicating if this is a snapshot or release build
> and a filename (or optioanlly a pathname relative to the artifact's
> directory or absolute in the HTTP server's namespace, to give repository
> maintainers more freedom).
>
> A word of explaination about snapshot/release flag. It would be used to
> detect the sitiuation when someone requests artifact-1.4.jar, but only
> SNAPSHOT versions are available. Thats the easy part. Another story
> would be the situation when the metadata file contains more than one
> versions flagged as release. Under normal circumstances this should not
> happen - when you release, it's done. BUT when the artifact's author
> decides to break the policy and redeploy a released artifact,
> metainformation about this will be saved, AND repeptability of builds
> can be maintained because the dependency resolution code will be able
> to determine what was the 'newest released' version of artifact-1.4.jar
> at a given point in the past.
>
> This kind of metadata file solves the directory listing concerns raised
> by Mark Wilkinson. Maven would gather information about available
> versions from the metadata file, abstracting from the trasport protocol
> capabilities.
>
> The artifact sequence number concept of Brian's can be 1-1 mapped to the
> number of entries in the metadata file, giving us the same download
> optimization benefits (if I understood the idea correctly).
>
> R.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
> ----------------------------------------------------------------------
> Rozmowy na WARSZAWSKIE tematy >>> http://link.interia.pl/f1727
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Artifact Downloading
Posted by Rafal Krzewski <Ra...@caltha.pl>.
Brian Ewins wrote:
> Maven is a little confusing on this point. If you do:
> maven jar:deploy-snapshot
>
> and you are building 'foo' version '1.4', it creates a build like:
> foo-1.4.20030202.153859.jar
>
> and uploads it. Clearly, this is supposed to be the latest version of
> foo-1.4.jar, so either
> foo-1.4-SNAPSHOT.jar
> should be created also (by linking or copying). What the timestamp will
> be on the server is moot.
>
> However if you do a release build:
> foo-1.4.jar
>
> this is *also* the latest version of foo-1.4.jar, and by rights,
> foo-1.4-SNAPSHOT.jar should now be a copy of or a link to this jar,
> despite the fact that you didn't upload it as a snapshot!
Good catch!
> So what should really happen here? Uploading a second copy of the jar
> each time would work, but seems odd, and won't fix the timestamp issue.
> Also if the declared version of the artifact you are building is
> actually "1.4-SNAPSHOT", then surely only one version should be uploaded
> - you're explicitly declaring that this is not a release.
>
> The alternative is to upload some smaller metadata file with each build;
> this would identify the real current version without linking or copying,
> and should have a sequence no (not timestamp) in it as well. This would
> have to be uploaded with every single build to handle the issue that
> release builds are actually snapshots (in the sense of latest versions)
> too.
I noticed that we need such a metadata file earlier, during the
build repetability discussion few months ago. My idea was that we need
a file that contains the information what was the 'newest SNAPSHOT' of
an artifact in a given point in the past. Such metadata file solves
also other concerns. More on my idea further down.
> This is not what the download mechanism does right now, but I've
> seen snapshot info files up on ibiblio? Was someone trying this?
Quite possible, discussions about this were going on for some time.
> Essentially this would enable you to do something like an http redirect,
> but you'd probably want to save the download as 'foo-1.4-SNAPSHOT'
> rather than whatever the real file turned out to be, since otherwise a
> lot of plugins might break.
Agreed, current practice requires -SNAPSHOT files in the local repo, we
should try to get rid of them in the remote (distribution) repositories
first.
> To be concrete, what I'm proposing would work like this:
<snip>
> Make sense?
Sure it does!, Now I'll tell a bit about my approach:
Every time an artifact is deployed to the remote repo, the deployment
script appends a single line to the metadata file. In this line, there
is a timestamp, a flag indicating if this is a snapshot or release build
and a filename (or optioanlly a pathname relative to the artifact's
directory or absolute in the HTTP server's namespace, to give repository
maintainers more freedom).
A word of explaination about snapshot/release flag. It would be used to
detect the sitiuation when someone requests artifact-1.4.jar, but only
SNAPSHOT versions are available. Thats the easy part. Another story
would be the situation when the metadata file contains more than one
versions flagged as release. Under normal circumstances this should not
happen - when you release, it's done. BUT when the artifact's author
decides to break the policy and redeploy a released artifact,
metainformation about this will be saved, AND repeptability of builds
can be maintained because the dependency resolution code will be able
to determine what was the 'newest released' version of artifact-1.4.jar
at a given point in the past.
This kind of metadata file solves the directory listing concerns raised
by Mark Wilkinson. Maven would gather information about available
versions from the metadata file, abstracting from the trasport protocol
capabilities.
The artifact sequence number concept of Brian's can be 1-1 mapped to the
number of entries in the metadata file, giving us the same download
optimization benefits (if I understood the idea correctly).
R.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Artifact Downloading
Posted by Brian Ewins <Br...@i-documentsystems.com>.
Mark H. Wilkinson wrote:
> On Thu, 2003-05-15 at 23:30, Ben Walding wrote:
> Second point: what's the actual method used for determining whether a
> newer snapshot dependency exists on the remote repository? You mention
> not using file modification time. Does it require the ability to perform
> the moral equivalent of 'ls http://blah/maven/group/jars/', which is
> somewhat tricky for protocols like http, less so for ftp and local
> files. Does it depend on certain admin practices on the server end (like
> having a link called artifact-SNAPSHOT.jar)?
Maven is a little confusing on this point. If you do:
maven jar:deploy-snapshot
and you are building 'foo' version '1.4', it creates a build like:
foo-1.4.20030202.153859.jar
and uploads it. Clearly, this is supposed to be the latest version of
foo-1.4.jar, so either
foo-1.4-SNAPSHOT.jar
should be created also (by linking or copying). What the timestamp will
be on the server is moot.
However if you do a release build:
foo-1.4.jar
this is *also* the latest version of foo-1.4.jar, and by rights,
foo-1.4-SNAPSHOT.jar should now be a copy of or a link to this jar,
despite the fact that you didn't upload it as a snapshot!
So what should really happen here? Uploading a second copy of the jar
each time would work, but seems odd, and won't fix the timestamp issue.
Also if the declared version of the artifact you are building is
actually "1.4-SNAPSHOT", then surely only one version should be uploaded
- you're explicitly declaring that this is not a release.
The alternative is to upload some smaller metadata file with each build;
this would identify the real current version without linking or copying,
and should have a sequence no (not timestamp) in it as well. This would
have to be uploaded with every single build to handle the issue that
release builds are actually snapshots (in the sense of latest versions)
too. This is not what the download mechanism does right now, but I've
seen snapshot info files up on ibiblio? Was someone trying this?
Essentially this would enable you to do something like an http redirect,
but you'd probably want to save the download as 'foo-1.4-SNAPSHOT'
rather than whatever the real file turned out to be, since otherwise a
lot of plugins might break.
To be concrete, what I'm proposing would work like this:
With project id=foo, version=1.4:
maven jar:deploy
downloads foo-1.4.jar.snapshot to check seq no. It is missing.
builds foo-1.4.jar
uploads foo-1.4.jar
uploads foo-1.4.jar.snapshot which contains the text (without the comments):
foo-1.4.jar # this is the latest file version
1 #this is the sequence no
Now do:
maven jar:deploy-snapshot
downloads foo-1.4.jar.snapshot to check seq no. It is 1.
builds foo-1.4.20030202.153859.jar
uploads foo-1.4.20030202.153859.jar
uploads foo-1.4.jar.snapshot which contains the text (without the comments):
foo-1.4.20030202.153859.jar # this is the latest file version
2 #this is the sequence no
Now do a build that depends on foo-1.4-SNAPSHOT.jar:
downloads foo-1.4.jar.snapshot. Checks filename.
downloads foo-1.4.20030202.153859.jar
saves it as foo-1.4-SNAPSHOT.jar locally also.
Now do a build that depends on foo-1.4-SNAPSHOT.jar:
downloads foo-1.4.jar.snapshot. Compares seq no to local seq no.
nothing is downloaded.
Now change to project id=foo, version=1.4-SNAPSHOT:
maven jar:deploy
downloads foo-1.4.jar.snapshot to check seq no. It is 2.
builds foo-1.4-SNAPSHOT.jar
uploads foo-1.4-SNAPSHOT.jar
uploads foo-1.4.jar.snapshot which contains the text (without the comments):
foo-1.4-SNAPSHOT.jar # this is the latest file version
3 #this is the sequence no
Now do a build that depends on foo-1.4-SNAPSHOT.jar:
downloads foo-1.4.jar.snapshot. Compares seq no to local seq no (was 2,
now 3).
downloads foo-1.4-SNAPSHOT.jar.
Make sense?
Baz
Privacy and Confidentiality Notice
------------------------------------------------
The information contained in this E-Mail message is intended only for the person or persons to whom it is addressed. Such information is confidential and privileged and no mistake in transmission is intended to waive or compromise such privilege. If you have received it in error, please destroy it and notify us on the telephone number printed above. If you do not receive complete and legible copies, please telephone us immediately. Any opinions expressed herein including attachments are those of the author only. i-documentsystems Ltd. does not accept responsibility for the accuracy or completeness of the information provided or for any changes to this Email, however made, after it was sent. (Please note that it is your responsibility to scan this message for viruses).
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Artifact Downloading
Posted by "Mark H. Wilkinson" <mh...@kremvax.net>.
On Thu, 2003-05-15 at 23:30, Ben Walding wrote:
> There have recently been a lot of bugs found in the artifact
> downloading, I introduced some of them (and I believe those are fixed now).
>
> I think some of the confusion is coming from
> A) What the correct procedure should be
> B) The implementation is showing its age and needs to be cleaned a bit.
> I don't think it is badly broken, just some of the contracts between
> components are weak and need to be locked down via unit tests.
> C) Lack of tests to prevent regressions
I'd agree with all of the above.
> To ensure that nothing is missed (The wiki entry should attempt to
> capture the scenarios so they are all tested)
> http://nagoya.apache.org/wiki/apachewiki.cgi?Maven/ArtifactDownloading
I can think of a couple of related things that probably should be
covered in the spec. First up, what protocols are we expecting to be
used to retrieve artifacts? At the moment the majority will be http, but
from reading the source it looks like any of the other URL schemes
handled by the JDK would at least cause something to happen (though they
may not actually work). There may be value in supporting local
filesystem repositories for those who don't have a web server kicking
around. Is there any point in supporting ftp? Should maven reject those
which aren't explicitly supported and tested?
Second point: what's the actual method used for determining whether a
newer snapshot dependency exists on the remote repository? You mention
not using file modification time. Does it require the ability to perform
the moral equivalent of 'ls http://blah/maven/group/jars/', which is
somewhat tricky for protocols like http, less so for ftp and local
files. Does it depend on certain admin practices on the server end (like
having a link called artifact-SNAPSHOT.jar)?
> I'll write a test harness using jetty (eg create "remote" repos using
> jetty and test downloads across a variety of scenarios)
This presupposes using http as the retrieval method - might need to
think about other approaches for other download protocols.
> I'm not looking to shake up downloading, just lock it down.
Need any help?
-Mark.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org