You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geronimo.apache.org by Jason Dillon <ja...@planet57.com> on 2006/10/27 07:57:50 UTC
Maven2 vs. Geronimo Maven2 repository problems
I think we have a major problem with out pseudo Maven2 repository
impl wrt how SNAPSHOT artifacts are handled.
The real m2 repo code when asked for a SNAPSHOT will consult a list
of remote repos, and if matched will download a time-stamped artifact
which will resolve to the SNAPSHOT version by metadata. Out pseudo
m2 repo code knows nothing about time-stamped artifacts and thus can
not resolve an artifact that is not named artifactId-n-SNAPHOT.ext,
even though m2 can.
This is a major problem when packaging cars, as the m2 build will
download and resolve SNAPSHOT artifacts, but G will not since it can
not find the time-stamped versions that resolve to the artifactId-n-
SNAPHOT.ext which was asked for.
I did some experiments to make a thin adapter for the car:package
goal to delegate to the real m2 repo API, so that it can find time-
stamped SNAPSHOT artifacts, but I still end up with exceptions while
packaging due to our repo API not resolving properly. I suspect that
our artifact resolver code needs to be changed to make this work, but
have yet to test/validate that it works.
It also looks like our repo API, or its usage is terribly
inefficient, as I see that artifacts are asked for over and over and
over... which causes a ton of redundant artifact resolutions... which
is why when building a car you see a ton of pom related failures over
and over and over for the same artifact.
All and all this causing major discontinuity between m2 and G repos
when SNAPSHOT artifacts are used... which is causing severe build
complications.
And in short... I have no idea how to fix this. Seems like we should
probably reuse the maven repo code... but our repo code is doing some
mysterious things that do no map well the the maven code... like
looking in parent configs and such.... and all that explicit
resolution mucky muck.
Erg... but then again the maven API is not really sock solid either.
I have found that if you create an artifact with artifactId-n-
SNAPHOT.ext and then resolve it, the resulting artifact's file is
groupId/artifactId/n-timestamp/artifactId-n-timestamp.ext instead of
what I had hoped groupId/artifactId/n-SNAPSHOT/artifactId-n-
timestamp.ext, but ArtifactRepository.pathOf(artifact) returns
groupId/artifactId/n-SNAPSHOT/artifactId-n-SNAPSHOT.ext and so with
the combonation of the two we can resolve the actual file.
I dunno... this is all pissing me off way to much....
--jason
Re: Maven2 vs. Geronimo Maven2 repository problems
Posted by Aaron Mulder <am...@alumni.princeton.edu>.
On 10/30/06, Jason Dillon <ja...@planet57.com> wrote:
> config.xml contains about 1% of the configuration of the server. 99%
> is in the compiled CAR file's config.ser.
But the beauty is, you can add more... Whatever you want... If you
wanted config.xml to contain all the attributes for all the GBeans in
every module, it could... and plugins can specify the content they
want to add to config.xml when they are installed.
Thanks,
Aaron
> On Oct 30, 2006, at 2:24 PM, Aaron Mulder wrote:
>
> > On 10/30/06, Jason Dillon <ja...@planet57.com> wrote:
> >> Geronimo is, IMO, fundamentally flawed in how it is configured.
> >> Because of this we can not even easily use Geronimo from a dist
> >> archive to run components for the gbuild.org hosts. We need to be
> >> able to easily change the configuration that it comes with, strip off
> >> a few bits, change a few ports, etc. This is a massive pain in the
> >> ass with Geronimo ASIS today... and this is primarily because of CAR
> >> files.
> >
> > I don't get it -- you can do that in config.xml. Which is an XML
> > file, not serialized, not in a JAR, etc. What's the problem?
> >
> > Thanks,
> > Aaron
>
>
Re: Maven2 vs. Geronimo Maven2 repository problems
Posted by Jason Dillon <ja...@planet57.com>.
config.xml contains about 1% of the configuration of the server. 99%
is in the compiled CAR file's config.ser.
--jason
On Oct 30, 2006, at 2:24 PM, Aaron Mulder wrote:
> On 10/30/06, Jason Dillon <ja...@planet57.com> wrote:
>> Geronimo is, IMO, fundamentally flawed in how it is configured.
>> Because of this we can not even easily use Geronimo from a dist
>> archive to run components for the gbuild.org hosts. We need to be
>> able to easily change the configuration that it comes with, strip off
>> a few bits, change a few ports, etc. This is a massive pain in the
>> ass with Geronimo ASIS today... and this is primarily because of CAR
>> files.
>
> I don't get it -- you can do that in config.xml. Which is an XML
> file, not serialized, not in a JAR, etc. What's the problem?
>
> Thanks,
> Aaron
Re: Maven2 vs. Geronimo Maven2 repository problems
Posted by Aaron Mulder <am...@alumni.princeton.edu>.
On 10/30/06, Jason Dillon <ja...@planet57.com> wrote:
> Geronimo is, IMO, fundamentally flawed in how it is configured.
> Because of this we can not even easily use Geronimo from a dist
> archive to run components for the gbuild.org hosts. We need to be
> able to easily change the configuration that it comes with, strip off
> a few bits, change a few ports, etc. This is a massive pain in the
> ass with Geronimo ASIS today... and this is primarily because of CAR
> files.
I don't get it -- you can do that in config.xml. Which is an XML
file, not serialized, not in a JAR, etc. What's the problem?
Thanks,
Aaron
Re: Maven2 vs. Geronimo Maven2 repository problems
Posted by Jason Dillon <ja...@planet57.com>.
On Oct 29, 2006, at 11:26 PM, David Jencks wrote:
> It's easy to tell you're not happy :-)
No... I'm not really happy. I'm frustrated with Maven and all its
little bugs and problems and I am really getting burt out on all of
he other build muck... which appears to have no end in sight problems
daily.
>> <sarcasm>
>> Ya, I really love having config files tucked away in little
>> objects zipped up in jar files. Ya, and then when I want to
>> change some of that config... I really love how I have to rebuild
>> the entire server... which is so easy... oh do I have the latest
>> dependencies in my local repo, oh... it will only take a few hours
>> to download, and pray my local repo does not have bunk metadata...
>> and oh wait are all those deps on the snapshot repo... didn't some
>> apache stuff crash again??!?
>> </sarcasm>
>
> This seems to be a problem with all but the simplest m2 builds. I
> don't think you can blame the non-working snapshot problems on
> geronimo rather than maven.
Sure, this is a problem with any m2 build... but the real issue that
I was being sarcastic about was that we hid the configuration, the
user editable configuration, of modules into CARs, which FORCE users
to deal with building components to change trivial settings... and
this has everything to do with Geronimo.
>> For example... CAR builds depend on artifacts which are not
>> directly listed in the Maven projects pom.xml... the deployers,
>> which the Kernel will do something with... and those deps need to
>> be fully transitively resolved in the Maven repo before that will
>> function correctly. None of those deps will show up in any Maven
>> trace, as Maven knows absolutely nothing about them.
>
> Well, they're pretty much like maven plugins, they're used in the
> build. Maybe there's some way to make them actually be maven plugins.
Um, CARs are also used at runtime are they not? Will G embed Maven
to process them? If so, someone besides myself had better start
learning Maven... because I am not gonna be very happy about
supporting this... since I think that it is a bad idea, bad concept.
>> And if you add the deps for the deployers to your module... then
>> the CAR gremlins start to do more muck with them, which often
>> causes a CAR build to fail, even if you set the scope to
>> provided... or was it test? I can't recall since we have
>> bastardized the maven scope mechanism to add custom semantics to
>> CAR dependencies.
>
> Would configuring each deployer as a maven plugin solve this?
If this was pure build time, then yes... but its not.
>> I have heard from a few peeps, like you, that you don't want to
>> get rid of CAR files... but I have yet to hear of any substantial
>> reasons why they should stay. I have also heard from many other
>> peeps about how they would like to see CAR files go away and be
>> replaced by simple XML... which is what I am a major supporter
>> of. I have on many occasions ranted about the issues I have with
>> CAR files, the added complexity, blah, blah, blah...
>
> Aaron did say he'd be happy with xml as the persistence format in a
> car file. Right now a car file can include both configuration and
> classes. If you want to eliminate this possibility we're going to
> have to have to give maven some classloaders that can load
> specified parts of a j2ee artifact such as an ear with wars and
> rars inside.
Right, be he is still talking about a build time thing to build that
xml-state and then zip it up into a jar, which I think is a huge
mistake... don't zip up your config files, don't follow the crappy
j2ee ear/war/ejb-jar code+config mess.
>> Where are the arguments for those few who are in favor of the CAR?
>>
>> Perhaps we should also take a poll from our users... and see if
>> they like having config compiled up into objects and zipped up and
>> tossed in the repo... or if they would like a set of plain XML
>> files? I wonder which they will choose...
>
> You aren't saying much definite about what you are proposing, so I
> can't discuss the merits or problems with what you want, since I
> don't have a clue what it is, except that it involves xml.
I am not proposing anything at this time... I never said I was. I am
still trying to understand why we have this super complex build to
support these CAR things which IMO suck ass.
> I'm not thrilled with xml configuration but I don't have any real
> problem with it. I still do like the idea of car files. I don't
> think that they need to be able to include classes, but that will
> require a lot of classloader magic to deal with actual j2ee
> artifacts in repos.
If you are not thrilled with XML configuration, then you probably are
not looking at the problem from a user's standpoint, or a sysadmin,
or really anyone who just wants to configure a G distro and now have
to deal with building anything.
Geronimo is, IMO, fundamentally flawed in how it is configured.
Because of this we can not even easily use Geronimo from a dist
archive to run components for the gbuild.org hosts. We need to be
able to easily change the configuration that it comes with, strip off
a few bits, change a few ports, etc. This is a massive pain in the
ass with Geronimo ASIS today... and this is primarily because of CAR
files.
I hate J2EE archives, but really CAR files suck much much more... as
I can't unzip a CAR, edit some XML, re-zip and then redeploy to get
my changes in. No... I have to go through this horrible build
process which is a MASSIVE_PITA for us developers... and is basically
not an option for anyone else. Just go try to explain to your
sysadmin that to change some configuration, that you have to go and
rebuild the server. That sounds like a great way to basically tell
people to go use some other server which is easier to configure.
> So, there are kind of 2 ideas behind geronimo modules, currently
> represented as car files. One is the idea of building a server
> adapted to a specific purpose by assembling prebuilt modules
> someone else created. The other is that, along the continuum from
> scripting languages to compiled languages, we want geronimo
> modules to be as compiled as possible to find as many problems as
> possible during "compiling" (deployment) and to make the resulting
> code run (starting the modules) as fast as possible, and to reduce
> the amount of infrastructure needed at runtime, and possibly to
> provide a security and authentication structure for module
> installation auditing.
I don't see any problems with the first idea of building a server
from modules for a specific purpose...
But, the second, is a major problem... this is the crux of the
configuration simplicity which I have been trying to express (over
and over).
I do not see any reason why one could not simply do some build-time
validation of xml config to ensure that it catches everything, but
still deploy the xml, keep the xml in the file system, so its easy to
grok and change. Compiling this into serialized objects is a very,
very, very bad idea.
Imagine, if Maven compiled its pom.xml into a pom.ser so that it was
easier for Maven to be able to load and not have to worry about
syntax errors... as someone might have changed the file... oh no.
How much would you like that? Maybe we can suggest that to them as a
simplification of their modularization and plugin architecture.
Well, if you did not guess it... I am kidding of course. Maven does
not even use compiled ser objects for bits which users are not
expected to edit, like mojo descriptors. Its all xml, nice, easy to
edit, usually easy to grok, xml. No special tools required, have vi
can edit, simple, simple, simple.
> I think that we are getting reasonably near the goal of being able
> to build lots of servers from prebuilt modules. Certainly the 4
> servers we distribute are built out of nearly the same set of
> modules, and with the plugin stuff it's getting easier to add and
> remove modules from an existing server. While there is more to do
> I think this is one of the successes of geronimo.
I am not against modules. I am against compiled config.
> If you accept this idea of a modular server, then the question is
> what should the modules look like. On the one hand you have jboss,
> which the last time I looked was heavily on the scripting end of
> the spectrum, starting from unprocessed xml descriptors and
> immediately constructing the runtime objects that will be the
> running server. On the other hand you have traditional j2ee
> servers that e.g. generate code for ejbs and compile it into a
> jar. The idea behind geronimo modules is to use deployers to turn
> whatever input is provided into persistable configuration
> information for services, and to save that configuration
> information in a standard format. So, for instance, in a plain
> gbean plan you can have specialized xml for login configuration,
> tss and css corba security configuration beans, and persistence
> units. They all get processed into GBeanDatas and get saved. J2EE
> artifacts typically have 2 sets of xml, the spec dds and a geronimo
> plan: these get combined into GBeanDatas and saved, again in a
> standard format. As a result, we can find a lot of problems while
> constructing a module before its installed anywhere, and we don't
> need any of the deployment infrastructure available when we need to
> actually start the module.
Honestly... I prefer JBoss-style configuration, where the xml is fed
into the deployment system and the application is assemblies/
configured on the spot from the xml input.
This is very sysadmin friendly.
But, no big surprise there as I was one of the people who helped
implement that system. I made many changes to that app server to
make it easier for sysadmin's to use and operate. And honestly... if
someone asked me which server I would recommend that would be easy
for a sysadmin to use... well, lets just say it would not be Geronimo.
--jason
Re: Maven2 vs. Geronimo Maven2 repository problems
Posted by David Jencks <da...@yahoo.com>.
It's easy to tell you're not happy :-)
On Oct 29, 2006, at 12:40 PM, Jason Dillon wrote:
> On Oct 29, 2006, at 5:11 AM, Aaron Mulder wrote:
>> Just for the record, I like having a "post-processed" module format.
>> I wouldn't mind if it had XML data instead of serialized objects, but
>> I am not really in favor of throwing it away and making a standard
>> ("pre-processed") J2EE JAR+DD+deployment plan into our official
>> module
>> format. So I'd rather find ways to make the CAR builds work right.
>> :)
>
> <sarcasm>
> Ya, I really love having config files tucked away in little objects
> zipped up in jar files. Ya, and then when I want to change some of
> that config... I really love how I have to rebuild the entire
> server... which is so easy... oh do I have the latest dependencies
> in my local repo, oh... it will only take a few hours to download,
> and pray my local repo does not have bunk metadata... and oh wait
> are all those deps on the snapshot repo... didn't some apache stuff
> crash again??!?
> </sarcasm>
This seems to be a problem with all but the simplest m2 builds. I
don't think you can blame the non-working snapshot problems on
geronimo rather than maven.
>
> CAR files are not simple from a users perspective, or from an
> application builders either. The only guy who things these are
> simple is the little gremlin that needs to load CARs into the
> server... which is great, we have pushed all of the complexity out
> of the server and into the build process, which I can certainly
> tell you had not simplified my life, as I have been chasing down
> CAR problems for weeks and weeks now.
>
> How do CAR files make anyones job easier (except for that little
> gremlin that is)? How do CAR files simplify the configuration of a
> server? How do CAR files simplify building a custom application on
> top of Geronimo? How do they make building plugins any easier?!
> From what I have gathered so far, from the mailing list, and other
> complaints about car plugin failures and other mvn related muck...
> is that CAR files only worsen the situation, making G more
> complicated to configure and comprehend.
>
> Months ago, when I came back to help on Geronimo... It took me
> weeks to discover where the configuration was for the ActiveMQ
> broker... and I just wanted to change one little attribute. And
> back then the m1 build would almost never make it 10 modules before
> it was puking on something, so I could not ever rebuild the server
> to change that one attribute.... and soon I gave up.
>
> The complexity that is comes along with CARs, whether it be a users
> frustration about finding and changing configuration, or a
> developers frustration about why a CAR module build is failing...
> and needing to track down David Jencks, who seems like one of the
> only folks able to comprehend what's going on, or maybe your
> frustration when trying to get a plugin CAR build to include the
> correct server deps...
>
> K.I.S.S and ditch the CARs.
>
>
>> But anyway, isn't there some way to get Maven to tell you why it's
>> chosen to download some particular module? It gives you that nice
>> little "dependency trace" when it fails to find something, but I
>> would
>> hope there's some debug mode where it shows that for everything...
>
> No Aaron, this is not Maven anymore... 90% of CAR'ing is done from
> the Kernel. Only a very thin Maven veneer is used to setup the
> Kernel components and feed it some data. After that it is the
> little gremlins at work that do the rest... and they do not know
> anything about Maven, or Mavens dependency tree, or really even how
> to barf up intelligent error messages, or provide more debug intel
> with -X
I've recommended for a while that we align geronimo and maven
dependencies, but this would cause complete disruption of the svn
tree and build order, and I haven't heard anyone say that they think
its a good idea before 1.2 is out. However, having them different
causes major confusion in understanding the build and cars.
>
> For example... CAR builds depend on artifacts which are not
> directly listed in the Maven projects pom.xml... the deployers,
> which the Kernel will do something with... and those deps need to
> be fully transitively resolved in the Maven repo before that will
> function correctly. None of those deps will show up in any Maven
> trace, as Maven knows absolutely nothing about them.
Well, they're pretty much like maven plugins, they're used in the
build. Maybe there's some way to make them actually be maven plugins.
>
> And if you add the deps for the deployers to your module... then
> the CAR gremlins start to do more muck with them, which often
> causes a CAR build to fail, even if you set the scope to
> provided... or was it test? I can't recall since we have
> bastardized the maven scope mechanism to add custom semantics to
> CAR dependencies.
Would configuring each deployer as a maven plugin solve this?
>
> * * *
>
> I have heard from a few peeps, like you, that you don't want to get
> rid of CAR files... but I have yet to hear of any substantial
> reasons why they should stay. I have also heard from many other
> peeps about how they would like to see CAR files go away and be
> replaced by simple XML... which is what I am a major supporter of.
> I have on many occasions ranted about the issues I have with CAR
> files, the added complexity, blah, blah, blah...
Aaron did say he'd be happy with xml as the persistence format in a
car file. Right now a car file can include both configuration and
classes. If you want to eliminate this possibility we're going to
have to have to give maven some classloaders that can load specified
parts of a j2ee artifact such as an ear with wars and rars inside.
>
> Where are the arguments for those few who are in favor of the CAR?
>
> Perhaps we should also take a poll from our users... and see if
> they like having config compiled up into objects and zipped up and
> tossed in the repo... or if they would like a set of plain XML
> files? I wonder which they will choose...
>
You aren't saying much definite about what you are proposing, so I
can't discuss the merits or problems with what you want, since I
don't have a clue what it is, except that it involves xml. I'm not
thrilled with xml configuration but I don't have any real problem
with it. I still do like the idea of car files. I don't think that
they need to be able to include classes, but that will require a lot
of classloader magic to deal with actual j2ee artifacts in repos.
So, there are kind of 2 ideas behind geronimo modules, currently
represented as car files. One is the idea of building a server
adapted to a specific purpose by assembling prebuilt modules someone
else created. The other is that, along the continuum from scripting
languages to compiled languages, we want geronimo modules to be as
compiled as possible to find as many problems as possible during
"compiling" (deployment) and to make the resulting code run (starting
the modules) as fast as possible, and to reduce the amount of
infrastructure needed at runtime, and possibly to provide a security
and authentication structure for module installation auditing.
I think that we are getting reasonably near the goal of being able to
build lots of servers from prebuilt modules. Certainly the 4 servers
we distribute are built out of nearly the same set of modules, and
with the plugin stuff it's getting easier to add and remove modules
from an existing server. While there is more to do I think this is
one of the successes of geronimo.
If you accept this idea of a modular server, then the question is
what should the modules look like. On the one hand you have jboss,
which the last time I looked was heavily on the scripting end of the
spectrum, starting from unprocessed xml descriptors and immediately
constructing the runtime objects that will be the running server. On
the other hand you have traditional j2ee servers that e.g. generate
code for ejbs and compile it into a jar. The idea behind geronimo
modules is to use deployers to turn whatever input is provided into
persistable configuration information for services, and to save that
configuration information in a standard format. So, for instance, in
a plain gbean plan you can have specialized xml for login
configuration, tss and css corba security configuration beans, and
persistence units. They all get processed into GBeanDatas and get
saved. J2EE artifacts typically have 2 sets of xml, the spec dds and
a geronimo plan: these get combined into GBeanDatas and saved, again
in a standard format. As a result, we can find a lot of problems
while constructing a module before its installed anywhere, and we
don't need any of the deployment infrastructure available when we
need to actually start the module.
I'm very attached to the ideas of a modular server and separation of
deploy-time and runtime activities. I'm not particularly attached to
the particular format of car files or the persisted configuration
data, or the current limitation that all modules have to be in the
same format.
I think that there are a lot of good ideas for the future here but
that most of them will take a really lot of work and discussion to
get right and working properly. I also think that these ideas won't
do anything to alleviate some of the problems you mentioned, such as
the disconnect between maven's view of dependencies and geronimo's
view of dependencies: if we could line them up then a lot of geronimo-
specific configuration would be unnecessary or completely generated
from poms. I think starting with things we can do now,
incrementally, without massive code disruption, will lead more
quickly to a server our users are happy with. I'm really reluctant
to start on another massive refactoring of everything in geronimo:
the last time we tried that (version independence) it took 6 months
during which we got just about no new features.
thanks
david jencks
> --jason
>
Re: Maven2 vs. Geronimo Maven2 repository problems
Posted by Jacek Laskowski <ja...@laskowski.net.pl>.
On 10/29/06, Jason Dillon <ja...@planet57.com> wrote:
> Perhaps we should also take a poll from our users... and see if they
> like having config compiled up into objects and zipped up and tossed
> in the repo... or if they would like a set of plain XML files? I
> wonder which they will choose...
It's sometimes said that excellent features are those that are hard to
comprehend and thus many can find them unimportant or cumbersome. It
takes time to get the grasp and get used to them. I understand CARs
very little, but I think there must've been something that led to
their creation. Why would people sit together for a long time and come
up with CARs? Therefore, I would rather avoid asking people about
their preferrence wrt CARs now as the vote could not show the whole
truth - the wisdom that comes with them. Let's ask the creators about
their initial goal behind them and only then call for a vote.
Just my 0.02PLN...
Jacek
--
Jacek Laskowski
http://www.jaceklaskowski.pl
Re: Maven2 vs. Geronimo Maven2 repository problems
Posted by Jason Dillon <ja...@planet57.com>.
On Oct 29, 2006, at 5:11 AM, Aaron Mulder wrote:
> Just for the record, I like having a "post-processed" module format.
> I wouldn't mind if it had XML data instead of serialized objects, but
> I am not really in favor of throwing it away and making a standard
> ("pre-processed") J2EE JAR+DD+deployment plan into our official module
> format. So I'd rather find ways to make the CAR builds work right.
> :)
<sarcasm>
Ya, I really love having config files tucked away in little objects
zipped up in jar files. Ya, and then when I want to change some of
that config... I really love how I have to rebuild the entire
server... which is so easy... oh do I have the latest dependencies in
my local repo, oh... it will only take a few hours to download, and
pray my local repo does not have bunk metadata... and oh wait are all
those deps on the snapshot repo... didn't some apache stuff crash
again??!?
</sarcasm>
CAR files are not simple from a users perspective, or from an
application builders either. The only guy who things these are
simple is the little gremlin that needs to load CARs into the
server... which is great, we have pushed all of the complexity out of
the server and into the build process, which I can certainly tell you
had not simplified my life, as I have been chasing down CAR problems
for weeks and weeks now.
How do CAR files make anyones job easier (except for that little
gremlin that is)? How do CAR files simplify the configuration of a
server? How do CAR files simplify building a custom application on
top of Geronimo? How do they make building plugins any easier?!
From what I have gathered so far, from the mailing list, and other
complaints about car plugin failures and other mvn related muck... is
that CAR files only worsen the situation, making G more complicated
to configure and comprehend.
Months ago, when I came back to help on Geronimo... It took me weeks
to discover where the configuration was for the ActiveMQ broker...
and I just wanted to change one little attribute. And back then the
m1 build would almost never make it 10 modules before it was puking
on something, so I could not ever rebuild the server to change that
one attribute.... and soon I gave up.
The complexity that is comes along with CARs, whether it be a users
frustration about finding and changing configuration, or a developers
frustration about why a CAR module build is failing... and needing to
track down David Jencks, who seems like one of the only folks able to
comprehend what's going on, or maybe your frustration when trying to
get a plugin CAR build to include the correct server deps...
K.I.S.S and ditch the CARs.
> But anyway, isn't there some way to get Maven to tell you why it's
> chosen to download some particular module? It gives you that nice
> little "dependency trace" when it fails to find something, but I would
> hope there's some debug mode where it shows that for everything...
No Aaron, this is not Maven anymore... 90% of CAR'ing is done from
the Kernel. Only a very thin Maven veneer is used to setup the
Kernel components and feed it some data. After that it is the little
gremlins at work that do the rest... and they do not know anything
about Maven, or Mavens dependency tree, or really even how to barf up
intelligent error messages, or provide more debug intel with -X
For example... CAR builds depend on artifacts which are not directly
listed in the Maven projects pom.xml... the deployers, which the
Kernel will do something with... and those deps need to be fully
transitively resolved in the Maven repo before that will function
correctly. None of those deps will show up in any Maven trace, as
Maven knows absolutely nothing about them.
And if you add the deps for the deployers to your module... then the
CAR gremlins start to do more muck with them, which often causes a
CAR build to fail, even if you set the scope to provided... or was it
test? I can't recall since we have bastardized the maven scope
mechanism to add custom semantics to CAR dependencies.
* * *
I have heard from a few peeps, like you, that you don't want to get
rid of CAR files... but I have yet to hear of any substantial reasons
why they should stay. I have also heard from many other peeps about
how they would like to see CAR files go away and be replaced by
simple XML... which is what I am a major supporter of. I have on
many occasions ranted about the issues I have with CAR files, the
added complexity, blah, blah, blah...
Where are the arguments for those few who are in favor of the CAR?
Perhaps we should also take a poll from our users... and see if they
like having config compiled up into objects and zipped up and tossed
in the repo... or if they would like a set of plain XML files? I
wonder which they will choose...
--jason
Re: Maven2 vs. Geronimo Maven2 repository problems
Posted by Aaron Mulder <am...@alumni.princeton.edu>.
Just for the record, I like having a "post-processed" module format.
I wouldn't mind if it had XML data instead of serialized objects, but
I am not really in favor of throwing it away and making a standard
("pre-processed") J2EE JAR+DD+deployment plan into our official module
format. So I'd rather find ways to make the CAR builds work right.
:)
But anyway, isn't there some way to get Maven to tell you why it's
chosen to download some particular module? It gives you that nice
little "dependency trace" when it fails to find something, but I would
hope there's some debug mode where it shows that for everything...
Thanks,
Aaron
On 10/28/06, Jason Dillon <ja...@planet57.com> wrote:
> Ugh... well, I think there may be more problems... as I just started
> to notice that builds of car modules downloads the deployed version
> of the same module while building...
>
> <snip>
> [INFO]
> ------------------------------------------------------------------------
> ----
> [INFO] Building Geronimo Configs :: UDDI Jetty
> [INFO] task-segment: [install]
> [INFO]
> ------------------------------------------------------------------------
> ----
> ...
> [INFO] [car:package]
> [INFO] Packaging module configuration: /home/gbuild/ws/server/configs/
> uddi-jetty/target/plan/plan.xml
> ...
> [INFO] snapshot org.apache.geronimo.configs:uddi-jetty:1.2-SNAPSHOT:
> checking for updates from apache-snapshots
> [INFO] snapshot org.apache.geronimo.configs:uddi-jetty:1.2-SNAPSHOT:
> checking for updates from codehaus-snapshots
> [INFO] snapshot org.apache.geronimo.configs:uddi-jetty:1.2-SNAPSHOT:
> checking for updates from apache.snapshots
> Downloading: http://people.apache.org/repo/m2-snapshot-repository/org/
> apache/geronimo/configs/uddi-jetty/1.2-SNAPSHOT/uddi-
> jetty-1.2-20061027.030123-3.car
> 4403K downloaded
> ...
> </snip>
>
> This is really maddening...
>
> The more I hack around CAR problems, the more that I find that CAR's
> are just adding more and more extra unneeded complication to
> Geronimo. In fact I believe that CAR are the reason why the G build
> is so complicated... and convoluted.
>
> At what point do we stop hacking around this and just toss CAR files
> out the window?
>
> Any simplicity we get from CARs when loading them in the server is
> IMO completely negated by the complexity of the build required to
> produce them.
>
> I'd really love to see CARs go away... and leave in there place plain
> xml files, which are easily editable and do not require and
> additional build processing to massage them.
>
> * * *
>
> But, until then... anyone know why the build of a CAR is trying to
> resolve itself as a dependency?
>
> --jason
>
>
>
> On Oct 28, 2006, at 2:17 PM, Jacek Laskowski wrote:
>
> > On 10/28/06, Jason Dillon <ja...@planet57.com> wrote:
> >
> >> But... in the meantime, I made a few changes which I think should
> >> help reduce strange problems with SNAPSHOT resolution.
> >
> > Thanks Jason for your "passion" for conducting such a painstaking
> > research and finally hunting it down. Much appreciated for such
> > followers like me.
> >
> > Jacek
> >
> > --
> > Jacek Laskowski
> > http://www.jaceklaskowski.pl
>
>
Re: Maven2 vs. Geronimo Maven2 repository problems
Posted by Jason Dillon <ja...@planet57.com>.
Ugh... well, I think there may be more problems... as I just started
to notice that builds of car modules downloads the deployed version
of the same module while building...
<snip>
[INFO]
------------------------------------------------------------------------
----
[INFO] Building Geronimo Configs :: UDDI Jetty
[INFO] task-segment: [install]
[INFO]
------------------------------------------------------------------------
----
...
[INFO] [car:package]
[INFO] Packaging module configuration: /home/gbuild/ws/server/configs/
uddi-jetty/target/plan/plan.xml
...
[INFO] snapshot org.apache.geronimo.configs:uddi-jetty:1.2-SNAPSHOT:
checking for updates from apache-snapshots
[INFO] snapshot org.apache.geronimo.configs:uddi-jetty:1.2-SNAPSHOT:
checking for updates from codehaus-snapshots
[INFO] snapshot org.apache.geronimo.configs:uddi-jetty:1.2-SNAPSHOT:
checking for updates from apache.snapshots
Downloading: http://people.apache.org/repo/m2-snapshot-repository/org/
apache/geronimo/configs/uddi-jetty/1.2-SNAPSHOT/uddi-
jetty-1.2-20061027.030123-3.car
4403K downloaded
...
</snip>
This is really maddening...
The more I hack around CAR problems, the more that I find that CAR's
are just adding more and more extra unneeded complication to
Geronimo. In fact I believe that CAR are the reason why the G build
is so complicated... and convoluted.
At what point do we stop hacking around this and just toss CAR files
out the window?
Any simplicity we get from CARs when loading them in the server is
IMO completely negated by the complexity of the build required to
produce them.
I'd really love to see CARs go away... and leave in there place plain
xml files, which are easily editable and do not require and
additional build processing to massage them.
* * *
But, until then... anyone know why the build of a CAR is trying to
resolve itself as a dependency?
--jason
On Oct 28, 2006, at 2:17 PM, Jacek Laskowski wrote:
> On 10/28/06, Jason Dillon <ja...@planet57.com> wrote:
>
>> But... in the meantime, I made a few changes which I think should
>> help reduce strange problems with SNAPSHOT resolution.
>
> Thanks Jason for your "passion" for conducting such a painstaking
> research and finally hunting it down. Much appreciated for such
> followers like me.
>
> Jacek
>
> --
> Jacek Laskowski
> http://www.jaceklaskowski.pl
Re: Maven2 vs. Geronimo Maven2 repository problems
Posted by Jacek Laskowski <ja...@laskowski.net.pl>.
On 10/28/06, Jason Dillon <ja...@planet57.com> wrote:
> But... in the meantime, I made a few changes which I think should
> help reduce strange problems with SNAPSHOT resolution.
Thanks Jason for your "passion" for conducting such a painstaking
research and finally hunting it down. Much appreciated for such
followers like me.
Jacek
--
Jacek Laskowski
http://www.jaceklaskowski.pl
Re: Maven2 vs. Geronimo Maven2 repository problems
Posted by Jason Dillon <ja...@planet57.com>.
It looks like the current state of the Maven artifact/repo API makes
it more difficult than desired to integrated directly into Geronimo.
Dain mentioned that Jason van Zyl might be cleaning up the API, so we
may be able to use the API directly once that has been done.
But... in the meantime, I made a few changes which I think should
help reduce strange problems with SNAPSHOT resolution.
First, I added a few ${version} property, which is always set to the
same value of ${pom.version}, but does not use that property
directly. ${pom.version} will change from the value which is
configured in poms to x-timestamo-build values when resolved from a
deployed SNAPSHOT release, and ends up causing massive problems when
resolving other artifacts.
I do not like using properties for versions like this... but short of
getting Maven changed this was the only way I could find that will
allow modules to depend on other modules w/o having to maintain a
massive dependencyManagement section in the top-level pom... which
would need ~140 dependency elements... which ends up at ~700 lines...
sucky. Use of this property also complicates the release process a
tad, as the ${version} property will need to be updated by hand
before running the release plugin... and we have yet to see how that
will work. I have asked the Maven dev list about this and so far no
one has responded back with any solution... other than not using $
{pom.version} or using non-unique versions when distribution...
neither of which I like very much.
I have also changed the car:package goal to delegate directly to the
Maven API for the sourceRepository, which will allow SNAPSHOT
versions to be resolved correctly to the versions which Maven has
downloaded. I also added explicit resolution of the deployer modules
used by car:package to ensure that the source repo always has all of
the required transitive dependencies for the deployers. This means
that external projects that package CAR files will function off of
downloaded dependencies w/o needing to explicitly add deployer deps
to the modules dependencies.
* * *
I have done some testing for G and CTS builds and so far it looks
like this is working... though I am not 100% sure that this fixes
everything.
I believe that it should be possible to build G online now with
configs deployed and get the right bits included into the assemblies
as one would expect.
For those of you who have taken to building offline to get around
problems, I'd like to ask that you try to build online and report
back if you run into any strange problems... like missing SNAPSHOT
deps, or code newly built not getting pulled into the assembly.
Please let me know if you run into anything.
Thanks,
--jason
On Oct 26, 2006, at 10:57 PM, Jason Dillon wrote:
> I think we have a major problem with out pseudo Maven2 repository
> impl wrt how SNAPSHOT artifacts are handled.
>
> The real m2 repo code when asked for a SNAPSHOT will consult a list
> of remote repos, and if matched will download a time-stamped
> artifact which will resolve to the SNAPSHOT version by metadata.
> Out pseudo m2 repo code knows nothing about time-stamped artifacts
> and thus can not resolve an artifact that is not named artifactId-n-
> SNAPHOT.ext, even though m2 can.
>
> This is a major problem when packaging cars, as the m2 build will
> download and resolve SNAPSHOT artifacts, but G will not since it
> can not find the time-stamped versions that resolve to the
> artifactId-n-SNAPHOT.ext which was asked for.
>
> I did some experiments to make a thin adapter for the car:package
> goal to delegate to the real m2 repo API, so that it can find time-
> stamped SNAPSHOT artifacts, but I still end up with exceptions
> while packaging due to our repo API not resolving properly. I
> suspect that our artifact resolver code needs to be changed to make
> this work, but have yet to test/validate that it works.
>
> It also looks like our repo API, or its usage is terribly
> inefficient, as I see that artifacts are asked for over and over
> and over... which causes a ton of redundant artifact resolutions...
> which is why when building a car you see a ton of pom related
> failures over and over and over for the same artifact.
>
> All and all this causing major discontinuity between m2 and G repos
> when SNAPSHOT artifacts are used... which is causing severe build
> complications.
>
> And in short... I have no idea how to fix this. Seems like we
> should probably reuse the maven repo code... but our repo code is
> doing some mysterious things that do no map well the the maven
> code... like looking in parent configs and such.... and all that
> explicit resolution mucky muck.
>
> Erg... but then again the maven API is not really sock solid
> either. I have found that if you create an artifact with
> artifactId-n-SNAPHOT.ext and then resolve it, the resulting
> artifact's file is groupId/artifactId/n-timestamp/artifactId-n-
> timestamp.ext instead of what I had hoped groupId/artifactId/n-
> SNAPSHOT/artifactId-n-timestamp.ext, but ArtifactRepository.pathOf
> (artifact) returns groupId/artifactId/n-SNAPSHOT/artifactId-n-
> SNAPSHOT.ext and so with the combonation of the two we can resolve
> the actual file.
>
> I dunno... this is all pissing me off way to much....
>
> --jason
Re: Maven2 vs. Geronimo Maven2 repository problems
Posted by Paul McMahan <pa...@gmail.com>.
I agree that these types of incompatibilities between the maven2 and
geronimo repos are troublesome and I am concerned that there will be
more rough spots ahead as both maven and geronimo continue to enhance
their repository logic in separate beakers. An additional dimension
of this discussion is that geronimo plugin repositories favor the
maven2 style layout and conventions (see GERONIMO-2521).
I am not a maven expert so it hurts my head to think about how
complicated it might be to replace the geronimo repo with a maven repo
or some extension of it. Stuff like the pain we went through because
of windows pathname limits comes to mind :-( But if its possible to
perform this brain surgery without killing the patient then I am in
definitely favor of the idea. If the effects can be localized enough
to avoid major disruptions to build, JEE5, and other ongoing
development then it seems like a worthwhile effort.
Best wishes,
Paul
On 10/27/06, Jason Dillon <ja...@planet57.com> wrote:
> I think we have a major problem with out pseudo Maven2 repository
> impl wrt how SNAPSHOT artifacts are handled.
>
> The real m2 repo code when asked for a SNAPSHOT will consult a list
> of remote repos, and if matched will download a time-stamped artifact
> which will resolve to the SNAPSHOT version by metadata. Out pseudo
> m2 repo code knows nothing about time-stamped artifacts and thus can
> not resolve an artifact that is not named artifactId-n-SNAPHOT.ext,
> even though m2 can.
>
> This is a major problem when packaging cars, as the m2 build will
> download and resolve SNAPSHOT artifacts, but G will not since it can
> not find the time-stamped versions that resolve to the artifactId-n-
> SNAPHOT.ext which was asked for.
>
> I did some experiments to make a thin adapter for the car:package
> goal to delegate to the real m2 repo API, so that it can find time-
> stamped SNAPSHOT artifacts, but I still end up with exceptions while
> packaging due to our repo API not resolving properly. I suspect that
> our artifact resolver code needs to be changed to make this work, but
> have yet to test/validate that it works.
>
> It also looks like our repo API, or its usage is terribly
> inefficient, as I see that artifacts are asked for over and over and
> over... which causes a ton of redundant artifact resolutions... which
> is why when building a car you see a ton of pom related failures over
> and over and over for the same artifact.
>
> All and all this causing major discontinuity between m2 and G repos
> when SNAPSHOT artifacts are used... which is causing severe build
> complications.
>
> And in short... I have no idea how to fix this. Seems like we should
> probably reuse the maven repo code... but our repo code is doing some
> mysterious things that do no map well the the maven code... like
> looking in parent configs and such.... and all that explicit
> resolution mucky muck.
>
> Erg... but then again the maven API is not really sock solid either.
> I have found that if you create an artifact with artifactId-n-
> SNAPHOT.ext and then resolve it, the resulting artifact's file is
> groupId/artifactId/n-timestamp/artifactId-n-timestamp.ext instead of
> what I had hoped groupId/artifactId/n-SNAPSHOT/artifactId-n-
> timestamp.ext, but ArtifactRepository.pathOf(artifact) returns
> groupId/artifactId/n-SNAPSHOT/artifactId-n-SNAPSHOT.ext and so with
> the combonation of the two we can resolve the actual file.
>
> I dunno... this is all pissing me off way to much....
>
> --jason
>
Re: Maven2 vs. Geronimo Maven2 repository problems
Posted by Jason Dillon <ja...@planet57.com>.
Do we need a wrapper? Or could we just toss our Artifact* bits and
use the Maven API directly?
--jason
On Oct 27, 2006, at 9:35 AM, Dain Sundstrom wrote:
> On Oct 26, 2006, at 10:57 PM, Jason Dillon wrote:
>
>> And in short... I have no idea how to fix this. Seems like we
>> should probably reuse the maven repo code... but our repo code is
>> doing some mysterious things that do no map well the the maven
>> code... like looking in parent configs and such.... and all that
>> explicit resolution mucky muck.
>
> The current code was supposed to be a temporary solution until
> someone wrote a wrapper around the real maven repository.
>
> -dain
Re: Maven2 vs. Geronimo Maven2 repository problems
Posted by Jason Dillon <ja...@planet57.com>.
Unfortunately the Maven2 API is not that intuitive to use/
configure... I'm also not quite sure what the full use of our
repository API is. I'm going to add some logging to try and get a
better idea what is really going on inside.
--jason
On Oct 27, 2006, at 12:05 PM, Jacek Laskowski wrote:
> On 10/27/06, Dain Sundstrom <da...@iq80.com> wrote:
>
>> The current code was supposed to be a temporary solution until
>> someone wrote a wrapper around the real maven repository.
>
> I've always appreciated working with people who can craft a solid
> solution that they used to call 'a temporary solution' ;-) Thanks Dain
> for raising the bar so high.
>
> Anyway, we've finally got Jason for it so it's just a matter of a few
> commit before it gets done ;-)
>
> Jacek
>
> --
> Jacek Laskowski
> http://www.jaceklaskowski.pl
Re: Maven2 vs. Geronimo Maven2 repository problems
Posted by Aaron Mulder <am...@alumni.princeton.edu>.
It woudl be great to switch the plugin installer to use the Maven code
to interact with the repository (search for matching versions, do the
downloads, etc.). I don't know much about the Maven code/libraries or
I would have done it by now.
Thanks,
Aaron
On 10/27/06, Jacek Laskowski <ja...@laskowski.net.pl> wrote:
> On 10/27/06, Dain Sundstrom <da...@iq80.com> wrote:
>
> > The current code was supposed to be a temporary solution until
> > someone wrote a wrapper around the real maven repository.
>
> I've always appreciated working with people who can craft a solid
> solution that they used to call 'a temporary solution' ;-) Thanks Dain
> for raising the bar so high.
>
> Anyway, we've finally got Jason for it so it's just a matter of a few
> commit before it gets done ;-)
>
> Jacek
>
> --
> Jacek Laskowski
> http://www.jaceklaskowski.pl
>
Re: Maven2 vs. Geronimo Maven2 repository problems
Posted by Jacek Laskowski <ja...@laskowski.net.pl>.
On 10/27/06, Dain Sundstrom <da...@iq80.com> wrote:
> The current code was supposed to be a temporary solution until
> someone wrote a wrapper around the real maven repository.
I've always appreciated working with people who can craft a solid
solution that they used to call 'a temporary solution' ;-) Thanks Dain
for raising the bar so high.
Anyway, we've finally got Jason for it so it's just a matter of a few
commit before it gets done ;-)
Jacek
--
Jacek Laskowski
http://www.jaceklaskowski.pl
Re: Maven2 vs. Geronimo Maven2 repository problems
Posted by Dain Sundstrom <da...@iq80.com>.
On Oct 26, 2006, at 10:57 PM, Jason Dillon wrote:
> And in short... I have no idea how to fix this. Seems like we
> should probably reuse the maven repo code... but our repo code is
> doing some mysterious things that do no map well the the maven
> code... like looking in parent configs and such.... and all that
> explicit resolution mucky muck.
The current code was supposed to be a temporary solution until
someone wrote a wrapper around the real maven repository.
-dain
Re: Maven2 vs. Geronimo Maven2 repository problems
Posted by anita kulshreshtha <a_...@yahoo.com>.
Jason Dillon <ja...@planet57.com> wrote: I think we have a major problem with out pseudo Maven2 repository
impl wrt how SNAPSHOT artifacts are handled.
The real m2 repo code when asked for a SNAPSHOT will consult a list
of remote repos, and if matched will download a time-stamped artifact
which will resolve to the SNAPSHOT version by metadata.
I think maven should be installing the artifacts in the local repo as SNAPSHOTs, not as a time-stamped artifact. IMO, If the builds have to explicitly deal with the time-stamped artifact, it defeats the purpose of SNAPSHOTs.
Thanks
Anita
---------------------------------
Get your own web address for just $1.99/1st yr. We'll help. Yahoo! Small Business.