You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by John Casey <ca...@gmail.com> on 2007/01/29 22:13:09 UTC
[PROPOSAL] maven-build-context (Shared context for Maven components and plugins)
Hi everyone,
I wanted to propose a new feature for Maven 2.1 (trunk). I think quite a few
people have run into something like this before, but I've come across a need
to detect some advanced build configuration using plugins and/or custom
components at the beginning of the build, and use that detected information
in other plugins later in the build process. I think it's important for
Maven to have a shared context for dynamic build-time information, to enable
the development of custom component implementations (like resolvers) that
have access to more than the API provides. The way I see it, this is akin to
a large EJB application that uses a shared context (JNDI) to access database
connection pools and things like that...which in turn can be used to
retrieve more information about the application and its state.
Now that we're putting increased focus on embeddability, it's important that
we don't enable the passing of things like System properties from the CLI to
a profile activator by bloating all the APIs in between. Also, it's becoming
important to allow custom components in the Maven system to have access to
build state in much the same way that mojos have it now.
What I'm proposing is to provide a formalized way to read/write data in the
core Plexus container, under whatever appropriate scoping, for components
(including plugins) participating in the build process to coordinate their
efforts. Does this open up the possibility of having hard-coded inter-plugin
dependencies and other related nastiness? Sure, though I'm hoping we can
limit exposure to such ugliness by using the right techniques for building a
shared datatype, storing/retrieving it, and (even better) wiring it into the
composition process for components in the container.
At any rate, I've added a new project to the maven trunk called
maven-build-context, which contains a beginning for this feature. I tagged
the sources before adding it, so it can be rolled out fairly easily. Also,
I've written up my thoughts on providing a shared context for Maven in the
following page:
http://docs.codehaus.org/display/MAVEN/Shared+Build+Context+for+Components+and+Plugins
I'll capture any salient detail from this conversation, and make sure it's
reflected on that page, but please feel free to comment on the page
directly.
Thanks,
John
Re: [PROPOSAL] maven-build-context (Shared context for Maven components and plugins)
Posted by Wendell Beckwith <wb...@gmail.com>.
> As for modding the build order, I'm still not sure that's possible (or
> advisable, in the larger context)...currently, I've only used the
> build-context concept in maven-core as mainly a read-only data structure,
> after it's setup. Are you talking about the need to inject new projects to
> an existing reactor, or...?
>
> -john
>
Yep, that's exactly what I mean. Maven's problem (not really a problem just
a current design constraint) is that it seems to think it knows everything
from the pom. In theory this is great, in practice not so much. Internally
the pom info is converted into internal datstructures, so if this is the
case then does the pom data have to come from an XML file. It should be
possible to extend maven to use other entities to create projects other than
just and XML file. In several specific cases, all the information necessary
to compile and package a OSGi bundle/eclipse plugin can derived from the
eclipse Manifest.mf and the build.properties file. Why is it necessary to
use a mojo to extract this information into a pom and then have maven build
against the pom in a 2nd pass? At the very least, a mojo should be able to
extract and create the pom and then update the current list of projects so
that it can be built in the current execution. Naturally updating the
project list will force another evaluation of the project build sequence.
Maven can have the notion of Advisors, similar to how eclipse has
implemented then for Windows, etc. The Reactor would pass the current Maven
projects to the advisor(s) and get back the final list in the correct
order. The advisor could change the order and/or construct new Maven
projects into the order for the Reactor to ultimately build. I don't want
to hijack your proposal thread, just wanted to state why I thought a shared
context could be useful for dynamically wiring pieces together. As the
Reactor builds modules, a new module type advisor/extension could
dynamically be added to the context to influence builds for projects of type
foo, for example.
Wb
Re: [PROPOSAL] maven-build-context (Shared context for Maven components and plugins)
Posted by John Casey <ca...@gmail.com>.
Actually, to be perfectly honest, I haven't been actively involved with
Kepler for some time now. This proposal actually comes from a year of work
I've done for a client, where the use of a build-context-like structure has
allowed me to maintain separation of concerns between several different
plugins, yet allowed them to cooperate in detecting whether/how a dependency
project was resolved or built prior to the main build running (among other
things). Now that I've started working on Maven's trunk once again, I have
found many places where this sort of shared context is useful (some of which
relates to the custom profiles and pre-scanning of build extensions that
I've committed lately).
As for modding the build order, I'm still not sure that's possible (or
advisable, in the larger context)...currently, I've only used the
build-context concept in maven-core as mainly a read-only data structure,
after it's setup. Are you talking about the need to inject new projects to
an existing reactor, or...?
-john
On 1/29/07, Wendell Beckwith <wb...@gmail.com> wrote:
>
> I've read the doc and the irc and while I can see the benefit to a shared
> context, I'm curious as to whether the current pressing need is coming
> more
> from maven or the kepler project? Doesn't matter either way just looking
> for the origin of the pain. This proposal looks to satisfy some of the
> OSGi
> builds issues I have encountered if maven's project build order was in the
> shared context and a plugin/componet could update /modify that order such
> that OSGi bundles could be built easier with maven.
>
> Wb
>
>
Re: [PROPOSAL] maven-build-context (Shared context for Maven components and plugins)
Posted by John Casey <ca...@gmail.com>.
I was actually hoping that we could limit the downside of this through some
good design discussion. I like your idea of using some sort of
expression/container-wiring to access the shared context, though I'm not
sure how a plugin could "export" something into the shared context via
container wiring...
After thinking about it, I think I definitely agree that such a
container-managed context would help us to map out context-based
interdependencies in advance and check for a consumer in the build that has
no corresponding producer ahead of it, and things like that. It would also
be a big help in documentation and automated error diagnosis.
I definitely agree that this is an important new feature to consider (or I
wouldn't have gone to the trouble of suggesting it, I suppose), but I'd like
to take some time to make sure we limit the downside risk as much as
possible.
-j
On 1/29/07, Jesse McConnell <je...@gmail.com> wrote:
>
> I have heard a lot of people ask for this kind of functionality on irc
> for the last year+, and I myself have desired something similar a
> couple of different times in the past.
>
> being able to have and share state between two plugins is incredibly
> useful, albeit a rather dangerous door to open up to general usage.
> most of my concerns were aired in the irc conversion I had with john
> on it a while back...but the actual benefit behind this is large and a
> great many people have asked or inquired about similar functionality.
>
> just opens up a can of worms is all :)
>
> jesse
>
> On 1/29/07, Wendell Beckwith <wb...@gmail.com> wrote:
> > I've read the doc and the irc and while I can see the benefit to a
> shared
> > context, I'm curious as to whether the current pressing need is coming
> more
> > from maven or the kepler project? Doesn't matter either way just
> looking
> > for the origin of the pain. This proposal looks to satisfy some of the
> OSGi
> > builds issues I have encountered if maven's project build order was in
> the
> > shared context and a plugin/componet could update /modify that order
> such
> > that OSGi bundles could be built easier with maven.
> >
> > Wb
> >
> >
>
>
> --
> jesse mcconnell
> jesse.mcconnell@gmail.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
Re: [PROPOSAL] maven-build-context (Shared context for Maven components and plugins)
Posted by Jesse McConnell <je...@gmail.com>.
I have heard a lot of people ask for this kind of functionality on irc
for the last year+, and I myself have desired something similar a
couple of different times in the past.
being able to have and share state between two plugins is incredibly
useful, albeit a rather dangerous door to open up to general usage.
most of my concerns were aired in the irc conversion I had with john
on it a while back...but the actual benefit behind this is large and a
great many people have asked or inquired about similar functionality.
just opens up a can of worms is all :)
jesse
On 1/29/07, Wendell Beckwith <wb...@gmail.com> wrote:
> I've read the doc and the irc and while I can see the benefit to a shared
> context, I'm curious as to whether the current pressing need is coming more
> from maven or the kepler project? Doesn't matter either way just looking
> for the origin of the pain. This proposal looks to satisfy some of the OSGi
> builds issues I have encountered if maven's project build order was in the
> shared context and a plugin/componet could update /modify that order such
> that OSGi bundles could be built easier with maven.
>
> Wb
>
>
--
jesse mcconnell
jesse.mcconnell@gmail.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [PROPOSAL] maven-build-context (Shared context for Maven components and plugins)
Posted by Jason van Zyl <ja...@maven.org>.
On 29 Jan 07, at 1:54 PM 29 Jan 07, Wendell Beckwith wrote:
> I've read the doc and the irc and while I can see the benefit to a
> shared
> context, I'm curious as to whether the current pressing need is
> coming more
> from maven or the kepler project?
The drive for this is generally useful, but the impetus is coming
from building C code actually.
Jason.
> Doesn't matter either way just looking
> for the origin of the pain. This proposal looks to satisfy some of
> the OSGi
> builds issues I have encountered if maven's project build order was
> in the
> shared context and a plugin/componet could update /modify that
> order such
> that OSGi bundles could be built easier with maven.
>
> Wb
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [PROPOSAL] maven-build-context (Shared context for Maven components and plugins)
Posted by Wendell Beckwith <wb...@gmail.com>.
I've read the doc and the irc and while I can see the benefit to a shared
context, I'm curious as to whether the current pressing need is coming more
from maven or the kepler project? Doesn't matter either way just looking
for the origin of the pain. This proposal looks to satisfy some of the OSGi
builds issues I have encountered if maven's project build order was in the
shared context and a plugin/componet could update /modify that order such
that OSGi bundles could be built easier with maven.
Wb
Re: [PROPOSAL] maven-build-context (Shared context for Maven components and plugins)
Posted by John Casey <ca...@gmail.com>.
For convenience, I'm inlining the document here. If anyone comments directly
in this thread, I'll try to keep the wiki page up to date. I'll also try to
copy over feedback on the wiki page here.
-j
=====================
1. Background A. Shared Context in Systems of Interchangeable Components
Most large, component-oriented systems provide a shared context for their
components, so alternative component implementations are not dependent on a
single, restrictive (or, even worse, all-anticipating) API to pass
information
across multiple subsystems. Such parameter-passing leads to bloated or
inadequate APIs, since the API designer can choose either to try to
accommodate every possible implementation for a component, or he can
restrict
the API to the minimum necessary to do the job with the original
implementation. Resources like database connection pools are a good example
of
items that live in this shared context. Components that require such
resources
simply look them up, or are wired to use them by a composing container,
instead of forcing all APIs in between the connection pool initializer and
consumer to pass the pool instance around.
B. Relevance to Maven
Maven is a large, component-oriented application built on top of a container
(Plexus) which is capable of this sort of composition. However, Maven's
component implementations are currently restricted to the APIs used to pass
information around the system. They don't have ready access to a shared
context for reading build-time information. Also, they cannot coordinate
build
activities with one another. This lack of shared context leads to an
unnecessarily restrictive set of component implementations and inefficient
build processes. Maven component implementations - including, but not
restricted to, plugins - must have the ability to retrieve as much
information
as possible about the current build environment.
2. Problem A. No Support for Querying Dynamic, Build-Time State
Currently, Maven provides a mechanism for injecting static information about
the current build into plugins. It also supports wiring plugins and other
components with component requirements, which allow them to retrieve further
information about the current build, though this information currently is
not
cached for successive querying in a threadsafe way. It does not provide a
mechanism for retrieving dynamic build-time state or progress for custom
components
and plugins to use.
B. Embeddable Property-Passing
The simple act of passing System properties in an embeddable way (where
"System" properties need to vary across multiple threads in a single Maven
instance) introduces major API changes, since these properties, which are
initialized at the beginning of the build, must be passed throughout the
system until they reach the appropriate components. In some cases, these
properties must propagate through five or six layers of Maven APIs before
they
can be used. Yet in many ways, the System properties represent a shared
context (though too widely shared, in the embedded scenario) by which
component-specific configurations can be communicated from the application
entry point to the appropriate component.
C. Plugins in Silos
Additionally, there is no way for plugins to mark progress in a build, or
signal that they have detected a state where the build has not failed but
also
should not proceed. Multiple mojos within a single plugin can pass
information
by way of a context map, but this context map restricts the consuming mojos
to
usage scenarios that also involve their counterpart mojos, since they cannot
function properly without their counterparts in place, regardless of whether
the functions of these counterparts could be fulfilled in other ways.
3. Proposed Solution A. The Build Context
As a solution to the problem of a build-time shared context, a new project
called maven-build-context should be created. This project would provide a
formal mechanism for storing and retrieving structured data from the Plexus
container's Context map. By using a set of tools to access the shared
context,
it can be scoped appropriately (per-project, per-thread, etc.) in such a way
as to avoid the problems encountered when using System properties in an
embedded environment. By using structured data, it's possible to avoid
datatype-conversion problems (coercing a String to an int, for example).
B. Plugin Cooperation
By using a shared datatype with a controlled vocabulary, it's possible for
plugins to cooperatively determine whether to execute, based on the actions
of
other plugins that were registered in the shared context. Plugins that go
further in detecting build configurations beyond that explicitly laid out in
the POM could inject this detected configuration for use by other plugins.
Such advanced state might include the source or method used to resolve each
project dependency, so that dependencies from certain sources can be handled
differently when the project is deployed (imagine building certain
dependencies from source, then deploying those built dependencies alongside
the main project artifact automatically).
C. Baking it into the Container
Ideally, these shared-context objects should be wired in as part of the
composition process for dependent components, and injected into mojos along
with things like ${localRepository} or ${project}. The act of querying the
application for this build-time information makes it much more difficult to
chart dependencies between components that produce and consume such
information. Therefore, it's important to specify this information as
declaratively as possible, in order to catch potential problems ahead of
time,
before the build is in progress. Rudimentary specification of a
<dependency/>
element in each component's POM that points to the shared datatype is a good
start, but still doesn't enable a more advanced validation that producer and
consumer of this shared datatype are present and aligned properly.
4. Potential Drawbacks A. Inter-Plugin Dependencies
Obviously, the biggest concern here is the ability to form inter-plugin
dependencies by specifying a shared datatype in one plugin, and developing
another plugin that depends on that datatype. However, such a dependency
doesn't preclude a new plugin from specifying a binary (not lifecycle)
dependency on the original plugin (the one with shared the datatype), then
acting as an alternative producer. An even better approach would be to
specify
the shared datatype in its own project, then provide dependencies on this
new
project for each of the three plugins. By doing this, a developer
effectively
creates an open protocol for an infinite array of plugins to communicate
with
one another through the shared context of the build.
5. Relevant Conversations
1. Jesse McConnell and John Casey (transcript from
IRC)<http://docs.codehaus.org/display/MAVEN/Jesse+McConnell+and+John+Casey+%28transcript+from+IRC%29>
[SNIP]