You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Vincent Massol <vm...@pivolis.com> on 2005/10/20 23:29:06 UTC
[M2][Proposal] J2EE builds best practices and conventions
Hi there,
I've been thinking about build best practices for J2EE projects in m2. I'd
like to run this by the community to see what are everyone's thought on it.
Build Features
==============
Basically here are the features that I'd like a full-fledge J2EE build to
perform:
* build J2EE modules (JAR, WAR, EJB, EAR, etc)
* support storing in the project the container's configuration files
- for weblogic that's the domain config
- for jboss that's the server config
- for tomcat that's server.xml, tomcat-users.xml, etc
* support building for different target environments
- support container-specific deployment descriptor for example
- support container config files that depend on the environment. For
example you may want to run on port 8080 in your dev env and on port 80 in
your production env. You may want to run on a single server in the dev env
but in cluster on the production env, etc.
- support different deployment models per environment. For example in one
environment you may want to have the web server and EJB server on the same
machine and in another environment you may want them located on 2 different
machines.
* support deploying config and J2EE modules to a container
* support automated functional tests execution
Directory structure
===================
In term of directory structure, I'm proposing the following:
<project>/
|
|_ modules/
|_ jars/
|_ <functional name for jar M>/
|_ ejbs/
|_ <functional name for ejb N>/
|_ wars/
|_ <functional name for war O>/
|_ ears/
|_ <functional name for ear P>/
|_ clients/
|_ <functional name for client Q>/
|_ rars/
|_ <functional name for rar R>/
|
|_ applications/
|_ <functional name for executable 1 - ex: database>/
|_ <functional name for executable 2 - ex: business>/
|_ <functional name for executable 3 - ex: wsclient>/
|_ system/
Note that for a simple project the structure can be simplified to:
<project>/
|
|_ modules/
|_ <functional name for jar M>/
|_ <functional name for ejb N>/
|_ <functional name for war O>/
|_ <functional name for ear P>/
|_ <functional name for client Q>/
|_ <functional name for rar R>/
|
|_ [...]
Some explanations:
* The applications/ directory contains subprojects (aka modules in m2 speak)
that contain executable applications, i.e. things that will make up the full
system. Applications are made of 2 things:
- modules
- configuration files including scripts (e.g. start/stop scripts, etc)
* The database/ application represents the database. This is where schema
and data could be located. This is also where database script files can be
stored for starting/stopping/analyzing logs/profiling/etc.
* The business/ application would be the business logic executing in the
J2EE container. It would package the J2EE modules (EARs and possibly others)
and the container configuration files.
* The wsclient/ application is just an example showing a client application
(ws stands for web services here). In this example this would be a sample
swing application.
* All the application projects above will generate a distribution (zipped
file, tar.gz, etc) per environment containing a directory structure with
scripts files in (say) bin/. Basically they should be self-running with only
pre-requisite software required to be installed on the target machines (OS,
JVM, container install, etc).
* The system/ project represents the full system. It is made of
applications. It'll also deliver a distribution (zipped file, tar.gz, etc)
which will aggregate all the distributions from the other applications. This
represents the full system for a target environment.
Other considerations
====================
* Environment will be managed using profiles.
* Environment specific resources can go in: src/main/resources-{env name}
* Common resources can go in src/main/resources and use filters (one filter
file per environment for example).
* Maven-runtime would be nice to generate scripts
* functional/system tests would be run in system/ (or application/)
* It would be nice to be able to automate the installation of the
distributions generated by the different application projects. This means
the ability to install locally but also remotely. Alternatively this would
mean building with m2 on all target machines to always install locally,
which is also possible using M2 in embedded mode for example and the repo as
a way to distribute the binary files - Actually this sounds cool :-). But
this still leaves the issue of triggering the executions of the embedded m2
remotely. This requires some new plugin with some xml file to configure IPs
and access to all the machines where to install the system.
* Cargo can probably be used somewhere (to run functional tests, to create
container configurationsn, to deploy to running containers, to start/stop
containers, etc). I still need to figure out the usage of cargo vs shell
scripts.
I'm pretty sure all this needs to be refined but I'm curious to get some
feedback from the community.
Thanks
-Vincent
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Cargo M2 plugin [was] Re: [M2][Proposal] J2EE builds ...
Posted by Nick Sieger <ni...@gmail.com>.
We are close to finishing it but we need some testers before we release it.
> In addition it's currently based on Cargo 0.7 snapshots.
>
> I'm making it available today as a snapshot version in a m2 repository on
> codehaus. In order to use it you'd simply need to add that repository to
> your pom.xml or settings.xml. Would you be interested in trying it out?
>
> What containers are you interested in? (just want to verify the ones
> you're
> interested in are working with the cargo 0.7 snapshot build that I plan to
> publish ;-)).
I'm hoping to start using M2 Cargo in the near future for OC4J. If OC4J
support is trailing the others I'd probably be willing to pitch in and get
it up to speed.
/Nick
Integration-test and, classpath etc
Posted by Pablo <pa...@tiger.com.pl>.
Hello there
I'm struggling with maven to make him work the way I want.
I've written a plugin which has two goals:
1) testCompile - compiles tests to be executed during integration-test
phase.
2) test - starts tomcat, run tests and stops tomcat
In all jars associated with tests there is a 'test scope' set.
Normal tests (junit tests) which are run in the 'test-compile phase' are
compiled without errors. These tests are based on StrutsTestCase which
are based on JUnit.
Somehow junit.jar is included in classpath during compilation and
execution of these tests.
It's fine.
And now, my plugin wants to compile integration tests and then run them.
The first goal 'testCompile' has the following attributes
* @goal testCompile
* @phase test-compile
* @requiresDependencyResolution test
However, all jars which have scope set to test are not present in the
classpath. Why?
There is no httpunit, htmlunit, junit etc.
Could some one tell me what I'm doing wrong and what I should do to make
it work?
Cheers
Pablo
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: problems with Integration-test and plexus compiler
Posted by Jesse McConnell <je...@gmail.com>.
I needed to do a bit of compiling after the process-classes phase for a
couple of custom plugins...so I pulled out the interesting parts of the
compiler mojo and used that...not sure if there is a better way, but this
certainly works..
jesse
private Compiler compiler = new JavacCompiler();
private void compile(String file) throws MojoExecutionException {
getLog().debug("outputDir " + outputDirectory);
getLog().debug("file " + file);
getLog().debug("build to " + buildDirectory);
try {
CompilerConfiguration compilerConfiguration = new CompilerConfiguration();
compilerConfiguration.setOutputLocation( buildDirectory + "/classes" );
ArrayList list = new ArrayList();
list.add(outputDirectory);
compilerConfiguration.setSourceLocations( list );
compilerConfiguration.setClasspathEntries(
project.getCompileClasspathElements() );
HashSet set = new HashSet();
set.add(new File(file));
compilerConfiguration.setSourceFiles( set );
compilerConfiguration.setDebug( true );
List messages = null;
messages = compiler.compile( compilerConfiguration );
}
catch ( Exception e )
{
e.printStackTrace();
// TODO: don't catch Exception
throw new MojoExecutionException( "Fatal error compiling", e );
}
}
On 11/15/05, Pablo <pa...@tiger.com.pl> wrote:
>
> Hello there
>
> I'm writting a plugin which is supposed to do a few things:
> 1) Compile test classes to be run in integration-test phase
> 2) Start tomcat
> 3) Run tests
> 4) Stop tomcat
>
> Steps 2, 3 and 4 are almost done. The main problem is step 1.
> I thought it would be too much to do by implementing compilation from
> scratch therefore I simply created a class extending TestCompilerMojo
> class.
> Unfortunatelly it doesn't work, throws exception because CompilerManager
> is not initialized. I tried to make it by myself by setting necessary
> field in execute() method (compilerManager, compilerId) and now it
> doesn't throw any exceptions.
>
> In plugin configuration I've placed 'compileSourceRoots' and
> ''outputDirectory' but it seems that these values are not set because I
> got a message:
> '[INFO] No sources to compile'
>
> Can someone point me to some tutorial or HOWTO describing all necessary
> steps to make it work?
>
> I don't want to place integration tests in the same directory as normal
> junit tests because these tests are supposed to be run after tomcat is
> started with web application running in the *integration-test* phase not
> the *test* phase. If they were placed in src/tests the *package* phase
> would return error since there would be no webapp running *(test* is
> before *package* phase).
>
> Thanks in advance.
> Pablo
>
>
--
jesse mcconnell
jesseDOTmcconnellATgmailDOTcom
problems with Integration-test and plexus compiler
Posted by Pablo <pa...@tiger.com.pl>.
Hello there
I'm writting a plugin which is supposed to do a few things:
1) Compile test classes to be run in integration-test phase
2) Start tomcat
3) Run tests
4) Stop tomcat
Steps 2, 3 and 4 are almost done. The main problem is step 1.
I thought it would be too much to do by implementing compilation from
scratch therefore I simply created a class extending TestCompilerMojo class.
Unfortunatelly it doesn't work, throws exception because CompilerManager
is not initialized. I tried to make it by myself by setting necessary
field in execute() method (compilerManager, compilerId) and now it
doesn't throw any exceptions.
In plugin configuration I've placed 'compileSourceRoots' and
''outputDirectory' but it seems that these values are not set because I
got a message:
'[INFO] No sources to compile'
Can someone point me to some tutorial or HOWTO describing all necessary
steps to make it work?
I don't want to place integration tests in the same directory as normal
junit tests because these tests are supposed to be run after tomcat is
started with web application running in the *integration-test* phase not
the *test* phase. If they were placed in src/tests the *package* phase
would return error since there would be no webapp running *(test* is
before *package* phase).
Thanks in advance.
Pablo
Re: Cargo M2 plugin [was] Re: [M2][Proposal] J2EE builds ...
Posted by Steve Loughran <st...@apache.org>.
Vincent Massol wrote:
> Hi David,
>
>
>>-----Original Message-----
>>From: david varnes [mailto:lists@varnes.net]
>>Sent: mardi 15 novembre 2005 12:13
>>To: Maven Developers List
>>Subject: Cargo M2 plugin [was] Re: [M2][Proposal] J2EE builds ...
>>
>>Hi Vincent,
>>
>>Is the Cargo M2 plugin released anywhere, or is it only available
>>in the beetlejuice ci 0.7-snapshot builds ?
>
>
> We are close to finishing it but we need some testers before we release it.
> In addition it's currently based on Cargo 0.7 snapshots.
>
> I'm making it available today as a snapshot version in a m2 repository on
> codehaus. In order to use it you'd simply need to add that repository to
> your pom.xml or settings.xml. Would you be interested in trying it out?
>
> What containers are you interested in? (just want to verify the ones you're
> interested in are working with the cargo 0.7 snapshot build that I plan to
> publish ;-)).
>
give me the URL and I'll test it too.
containers: tomcat5 and jboss4.
-Steve
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
RE: Cargo M2 plugin [was] Re: [M2][Proposal] J2EE builds ...
Posted by Vincent Massol <vm...@pivolis.com>.
Hi David,
> -----Original Message-----
> From: david varnes [mailto:lists@varnes.net]
> Sent: mardi 15 novembre 2005 12:13
> To: Maven Developers List
> Subject: Cargo M2 plugin [was] Re: [M2][Proposal] J2EE builds ...
>
> Hi Vincent,
>
> Is the Cargo M2 plugin released anywhere, or is it only available
> in the beetlejuice ci 0.7-snapshot builds ?
We are close to finishing it but we need some testers before we release it.
In addition it's currently based on Cargo 0.7 snapshots.
I'm making it available today as a snapshot version in a m2 repository on
codehaus. In order to use it you'd simply need to add that repository to
your pom.xml or settings.xml. Would you be interested in trying it out?
What containers are you interested in? (just want to verify the ones you're
interested in are working with the cargo 0.7 snapshot build that I plan to
publish ;-)).
Thanks
-Vincent
> Vincent Massol wrote:
>
> [snip]
>
> > OTOH, it is possible to use tools like the Cargo m2 plugin in the WAR or
> EAR
> > component projects to perform integration tests using a *default*
> > configuration.
>
> [snap]
>
> > Thanks
> > -Vincent
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Cargo M2 plugin [was] Re: [M2][Proposal] J2EE builds ...
Posted by david varnes <li...@varnes.net>.
Hi Vincent,
Is the Cargo M2 plugin released anywhere, or is it only available
in the beetlejuice ci 0.7-snapshot builds ?
thanks
davidv
Vincent Massol wrote:
[snip]
> OTOH, it is possible to use tools like the Cargo m2 plugin in the WAR or EAR
> component projects to perform integration tests using a *default*
> configuration.
[snap]
> Thanks
> -Vincent
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
RE: [M2][Proposal] J2EE builds best practices and conventions
Posted by Vincent Massol <vm...@pivolis.com>.
> -----Original Message-----
> From: Jesse McConnell [mailto:jesse.mcconnell@gmail.com]
> Sent: lundi 14 novembre 2005 15:09
> To: Maven Developers List
> Subject: Re: [M2][Proposal] J2EE builds best practices and conventions
[snip]
> ok, this is where you are losing me...what exactly are these extra
> directory
> structures gives us? You mention functional tests...is that was you are
> aggregating under the applications directory?
>From my first email on this thread:
"
[...]
* build J2EE modules (JAR, WAR, EJB, EAR, etc)
* support storing in the project the container's configuration files
- for weblogic that's the domain config
- for jboss that's the server config
- for tomcat that's server.xml, tomcat-users.xml, etc
* support building for different target environments
- support container-specific deployment descriptor for example
- support container config files that depend on the environment. For
example you may want to run on port 8080 in your dev env and on port 80 in
your production env. You may want to run on a single server in the dev env
but in cluster on the production env, etc.
- support different deployment models per environment. For example in one
environment you may want to have the web server and EJB server on the same
machine and in another environment you may want them located on 2 different
machines.
* support deploying config and J2EE modules to a container
* support automated functional tests execution
"
The first bullet point is what you have covered in your mail. I'm interested
in covering the other and for those I'm recommending the applications/
directory structure (and the system/ one).
Application = programs that can be executed (i.e. bundle of different
components + config + possibly start/stop scripts).
Here some details from that same initial email:
"
Some explanations:
* The applications/ directory contains subprojects (aka modules in m2 speak)
that contain executable applications, i.e. things that will make up the full
system. Applications are made of 2 things:
- modules
- configuration files including scripts (e.g. start/stop scripts, etc)
* The database/ application represents the database. This is where schema
and data could be located. This is also where database script files can be
stored for starting/stopping/analyzing logs/profiling/etc.
* The business/ application would be the business logic executing in the
J2EE container. It would package the J2EE modules (EARs and possibly others)
and the container configuration files.
* The wsclient/ application is just an example showing a client application
(ws stands for web services here). In this example this would be a sample
swing application.
* All the application projects above will generate a distribution (zipped
file, tar.gz, etc) per environment containing a directory structure with
scripts files in (say) bin/. Basically they should be self-running with only
pre-requisite software required to be installed on the target machines (OS,
JVM, container install, etc).
* The system/ project represents the full system. It is made of
applications. It'll also deliver a distribution (zipped file, tar.gz, etc)
which will aggregate all the distributions from the other applications. This
represents the full system for a target environment.
Other considerations
====================
[...]
* functional/system tests would be run in system/ (or application/)
* It would be nice to be able to automate the installation of the
distributions generated by the different application projects. This means
the ability to install locally but also remotely. Alternatively this would
mean building with m2 on all target machines to always install locally,
which is also possible using M2 in embedded mode for example and the repo as
a way to distribute the binary files - Actually this sounds cool :-). But
this still leaves the issue of triggering the executions of the embedded m2
remotely. This requires some new plugin with some xml file to configure IPs
and access to all the machines where to install the system.
* Cargo can probably be used somewhere (to run functional tests, to create
container configurationsn, to deploy to running containers, to start/stop
containers, etc). I still need to figure out the usage of cargo vs shell
scripts.
"
> This starts to get into the discussion we had a couple of months ago on
> irc
> in regards to the different testing methodologies to support under maven
> direction. The simplest is the surefire launched unit tests sitting in
> each
> subprojects src/test/java directory. Next up would be integration tests
> between different subprojects, different components in your above
> example...is that purpose of the applications directory?
Not quite. The applications directory has a goal: producing executable
applications (i.e. installers or zips containing not only the components but
also config files to configure the containers into which those components
will go + start/stop scripts to start/stop those containers, etc).
Obviously if you want to run functional tests you need a running
environment, i.e. you need to have your containers configured and your
components deployed into it. This is why running the functional tests in the
applications/ project is a good place.
Imagine you have an application that access a database and you want to run
integration tests say in the ear component. If the database is not up and
running with valid schemas and data your tests will fail. It is the role of
the applications/database project to ensure that the database is up with a
valid schema and proper data.
OTOH, it is possible to use tools like the Cargo m2 plugin in the WAR or EAR
component projects to perform integration tests using a *default*
configuration. But that will not be the *target* configuration. I'm
recommending to define the target configurations in applications projects
and then possibly use Cargo there, using this defined configuration instead
of generating a default one.
There's work to be done in this area of generating/defining configurations.
ATM Cargo is able to generate a default configuration that can be slightly
tuned (ports, logging levels, security, etc). We would need to work so that
users could override more of that. Smartfrog is also a good idea in this
domain as it has a language to define configurations.
> I think on that irc discussion this was about as far as we figure maven at
> this stage would try and formalize since post integration testing it get
> really localized to implementation...
>
> I would think that integration testing in a j2ee build, or any build
> really
> would just be another subproject that declared all of artifacts as
> dependencies. If you had too many different integration tests or wanted to
> break them out by functionality then adopt the same mechanism I mentioned
> earlier, name them function-integration, function2-integration,
> function3-integration...and if you get 5 or more then nest them in a
> directory called integration.
Yes, but it's not just about testing. It's about generating executable
distributions. If your project don't want to generate an executable
distribution then I agree you don't need the applications/ projects.
> what artifacts are you seeing will be produced in the applications
> directory? The war and ear artifacts? aren't they really just dependencies
> to other artifacts...
The war and ears are components produced by the components projects. The
applications project generate zips and the like.
> if applications is where you are envisioning storing configuration or
> different containers and whatnot then I am not sure why that means that
> the
> component directory needs to nest everything else..
The separation between components/ and applications/ is there simply to
distinguish them. Obviously if you wanted to have everything flat you could
also have that. For a project the size of the daytrader app though I would
recommend have a clean directory structure separating them.
> In your comment you
> mention starting and stopping the container and things..that almost sounds
> like a tools or utility functionality which I would just make a subproject
> for called deployment-tools or something...and in there either the pom
> that
> uses the relevant plugin or subprojects if there are multiple behaviours..
There are 2 things here:
- data
- tools that use this data to start/stop
The applications projects store the data. The tools are indeed generic
plugins (like the Cargo one or the db one that I'm planning to write). In
any case these tools will not be developed as part of the applications/
projects. They would just get used there.
<sidenote>We would also need a best practice to tell users where they should
put build tools (i.e. plugins) that they develop for their build.</sidenote>
> thanks for the response :) it looks like we are agreeing on a lot of the
> fundamentals which should let jason actually start writing about these
> things :P
I'm trying to go beyond the fundamentals but we don't have to make those
best practices. I'm just interested in writing about those in the J2EE
chapter I'm writing for the m2 book because I think they are real-world
things. There are probably different ways of doing this. I just know that
this strategy has worked quite well for me in the past on several projects.
As Steve was saying there are probably more robust solutions like using a
proper tool (Smartfrog for example) outside of your build for doing this.
I think the "Saint-Graal" in term of automated build if the ability to fully
automate the process from checking out the sources to deploying a working
solution on a pre-production platform. I used to call this "Agile Delivery"
in some of my past presentations. I've been able to get very close to this
once. One critical item is having automated end to end tests.
Am I making sense?
Thanks
-Vincent
> On 11/14/05, Vincent Massol <vm...@pivolis.com> wrote:
> >
> > Hi Jesse,
> >
> > Thanks for restarting this thread!
> >
> > See below
> >
> > > -----Original Message-----
> > > From: Jesse McConnell [mailto:jesse.mcconnell@gmail.com]
> > > Sent: lundi 7 novembre 2005 20:00
> > > To: Maven Developers List
> > > Subject: Re: [M2][Proposal] J2EE builds best practices and conventions
> > >
> > > ok, digging this old thing out the dust bin..
> > >
> > > I have abandoned my old layout (and the current structure provided by
> > the
> > > j2ee archetype) and shifting towards something that is more like the
> > > current
> > > state of the maven repo...and I am trying to decide the lvl of nesting
> > > that
> > > is useful for the components when factoring vincents threads above..
> > >
> > > first off, I concede the usefulness of having actual source in the war
> > > files...while my architecture doesn't really require/use that kinda
> > thing,
> > > it would be silly to design around the idea in any sort of best
> > practices
> > > way :) (that being that war's and the like don't need any compiled
> > source
> > > at
> > > all, just suck in the associated dependencies)
> >
> > Cool :-)
> >
> > > ok...I'll start with the basics of the maven layout since that is a
> > pretty
> > > simple no brainer place to start..and for this discussion to make
> sense
> > as
> > > a
> > > best practices discussion I'll invent a little project and if it makes
> > > sense
> > > to later we can apply the daytrader app to it (I don't know enough of
> > the
> > > daytrader to do it justice atm). I know brett alluded to an exercise
> > like
> > > this in the initial mail above..
> > >
> > > Lets say we have a mythical project called dropit...
> > >
> > > flat:
> > >
> > > dropit
> > > dropit-logging
> > > dropit-config
> > > dropit-interfaces
> > > dropit-frontend-servlet
> > > dropit-backdoor-servlet
> > > dropit-site
> > > dropit-war
> > > dropit-ear
> > >
> > > pretty simple project, three discrete source modules with a seperate
> > > interfaces module for other systems to program against..the site, a
> war
> > > and
> > > an ear..
> >
> > Could you explain what the dropit-*-servlet projects are? Shouldn't they
> > be
> > located in the dropit-war project?
> >
> > > starting simple I guess there are a couple of questions we ought to
> > answer
> > > if we want to arrive at a best practice module layout...
> > >
> > > 1. do we want to distingish wars, servlets, ears and other packaging
> > > concepts by the name of the directory or by nesting them? if so, how
> > many
> > > lvls deep?
> > >
> > > dropit (the primary source, sister element)
> > > wars
> > > wars/dropit-frontend
> > > wars/dropit-backdoor
> >
> > +1
> >
> > > or go for the gusto?
> > >
> > > ear/wars/dropit-frontend
> > > ear/wars/dropit-backend
> >
> > I don't think this can be considered a best practice. It may happen in
> > very
> > particular circumstances that it'll work but not generally speaking.
> > That's
> > because the same jar/war/etc can be packaged into different wars, ears,
> > etc.
> >
> > Also it sounds very weird to me to have a directory structure like:
> >
> > My project
> > |_ src/...
> > |_ pom.xml
> > |_ dropit-frontend/
> > |_ dropit-backend/
> >
> > Mixing src/ and subprojects at the same level doesn't resonate too well
> I
> > think.
> >
> > > 2. if we follow another common approach of nesting
> > >
> > > component/dropit
> >
> > What is this dropit project inside component?
> >
> > > component/dropit-logging
> > > component/dropit-config
> > > packaging/war/dropit-frontend
> > > packaging/war/dropit-backdoor
> > > packaging/dropit (type ear, generates dropit-1.0.ear)
> >
> > For me a jar, war, rar, even ear are all the same: they are modules (aka
> > components).
> >
> > > this kinda approach still kind of rubs me raw with there being actual
> > > source
> > > in the wars since that means you have source under packaging
> > > subprojects...
> >
> > Yep agreed. I don't like it either. I prefer the directory layout I had
> > proposed in my earlier posts.
> >
> > > now we can move on to my personal psychosis :)
> >
> > :-)
> >
> > > I think my personal problem with trying to figure out a nice clean
> > module
> > > layout is that with maven's multiproject abilities it makes it
> difficult
> > > to
> > > seperate my previous layout experiences stemming from one overriding
> > > build.xml file containing everything woven together, from the more
> clean
> > > seperation of discrete units that maven2 encourages...working through
> > the
> > > repository for unit artifacts as opposed to other directories in the
> > > project
> > > really frees things up a lot.
> > >
> > > to be honest, working out my current setup and reading through the
> > backlog
> > > of this message I find myself leaning towards a best practices j2ee
> > setup
> > > of
> > > very close to the maven-components one, just utilizing a naming
> > convention
> > > for j2ee components.
> > >
> > > dropit
> > > dropit-logging
> > > dropit-config
> > > dropit-interfaces-api
> > > dropit-frontend-servlet
> > > dropit-backdoor-servlet
> > > dropit-war
> > > dropit-ear
> >
> > Yes, I fine with this too except that I'd name it a bit differently:
> >
> > dropit/
> > |_ logging/
> > |_ config/
> > |_ interfaces-api/
> > |_ frontend-servlet/
> > |_ backend-servlet/
> > |_ war/
> > |_ ear/
> >
> > But again this is a very simple project that doesn't have everything
> > automated? What about functional tests? Where do they fit? What about
> > container configuration files? What about deployment to the container?
> > What
> > about database start/stop/loading of data? Etc...
> >
> > This is why I was suggesting an additional directory level in my
> previous
> > posts:
> >
> > dropit/
> > |_ components/
> > |_ [...]
> > |_ applications/
> > |_ [...]
> >
> > But I agree that if, as a user, you don't want to automate all those
> then
> > a
> > single directory level is enough.
> >
> > > For a simple project like this, there is no need to venture into lots
> of
> > > convoluted subdirectories that carry along a meaning with them (like
> the
> > > packaging thing and my hangup with there being source in there).
> >
> > Agreed.
> >
> > > I would even go so far as to say that if this where a best practices
> > > layout
> > > that a general rule of thumb could be that if you have 5 or more
> > *-servlet
> > > artifacts that you nest them into a servlets directory
> > >
> > > dropit
> > > dropit-logging
> > > dropit-config
> > > dropit-interfaces-api
> > > servlets/dropit-frontend-servlet
> > > servlets/dropit-backdoor-servlet
> > > servlets/dropit-fun-servlet
> > > servlets/dropit-silly-servlet
> > > servlets/dropit-humor-servlet
> > > dropit-war
> > > dropit-ear
> > >
> > > and the same thing would hold for the ejbs, wars, ears, whatever...
> >
> > +1
> >
> > > the
> > > only
> > > exception being artifacts that compile to be jars...then would all
> exist
> > > at
> > > the root of the project directory unless it really makes sense to nest
> > > components that have maybe 5 or more of a particular thing.
> >
> > I'd treat jars in the same manner as wars, ejbs, ears, etc.
> >
> > > which...when you look at it like this closely models the current
> > structure
> > > of the maven-components svn module and how maven-plugins was nested in
> > it
> > > with a whole lot of plugins in it.
> > >
> > > Perhaps we don't necessarily want to promote a j2ee best practices
> > layout
> > > beyond a general rule of thumb for maven2 projects having all discrete
> > > compiling units at the base directory of the project. That and a
> > suggested
> > > naming convention and nesting convention for j2ee artifacts -war, -
> ear,
> > > -ejb.
> >
> > Not sure about this naming conventions.
> >
> > > that accomplishs a lot:
> > >
> > > 1) it gets us out of the old habit of nesting things just to nest
> things
> > > 2) promotes the concept that a project is made up of a lot of
> discretely
> > > compiling dependencies
> > > 3) holds all artifacts as equals, nesting only based on shared
> > underlying
> > > functionality
> > > 4) all 'best practices' for maven2 archetype projects are the
> > same...they
> > > already have an established layout internal to the sub-project, src,
> > > target,etc..
> > > 5) best practices for different meta project organizations is reduced
> to
> > > naming conventions, j2ee using -war, -ejb, -ear...others can follow
> > their
> > > own setups...
> > > 6) following this kinda approach ties closely with the multi-project
> > > setup...embraces the flatness really...maybe making it easier to
> explain
> > > how
> > > inner project dependencies work without any sort of implied knowledge
> > > based
> > > on project directory layout...
> > >
> > > does this rambling make any sense to anyone else? :)
> >
> > Sure, it does! But it doesn't cover the full story.
> >
> > Thanks
> > -Vincent
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
>
>
> --
> jesse mcconnell
> jesseDOTmcconnellATgmailDOTcom
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [M2][Proposal] J2EE builds best practices and conventions
Posted by Jesse McConnell <je...@gmail.com>.
>
> Yes, I fine with this too except that I'd name it a bit differently:
>
> dropit/
> |_ logging/
> |_ config/
> |_ interfaces-api/
> |_ frontend-servlet/
> |_ backend-servlet/
> |_ war/
> |_ ear/
>
> But again this is a very simple project that doesn't have everything
> automated? What about functional tests? Where do they fit? What about
> container configuration files? What about deployment to the container?
> What
> about database start/stop/loading of data? Etc...
>
> This is why I was suggesting an additional directory level in my previous
> posts:
>
> dropit/
> |_ components/
> |_ [...]
> |_ applications/
> |_ [...]
>
> But I agree that if, as a user, you don't want to automate all those then
> a
> single directory level is enough.
>
ok, this is where you are losing me...what exactly are these extra directory
structures gives us? You mention functional tests...is that was you are
aggregating under the applications directory?
This starts to get into the discussion we had a couple of months ago on irc
in regards to the different testing methodologies to support under maven
direction. The simplest is the surefire launched unit tests sitting in each
subprojects src/test/java directory. Next up would be integration tests
between different subprojects, different components in your above
example...is that purpose of the applications directory?
I think on that irc discussion this was about as far as we figure maven at
this stage would try and formalize since post integration testing it get
really localized to implementation...
I would think that integration testing in a j2ee build, or any build really
would just be another subproject that declared all of artifacts as
dependencies. If you had too many different integration tests or wanted to
break them out by functionality then adopt the same mechanism I mentioned
earlier, name them function-integration, function2-integration,
function3-integration...and if you get 5 or more then nest them in a
directory called integration.
what artifacts are you seeing will be produced in the applications
directory? The war and ear artifacts? aren't they really just dependencies
to other artifacts...
if applications is where you are envisioning storing configuration or
different containers and whatnot then I am not sure why that means that the
component directory needs to nest everything else.. In your comment you
mention starting and stopping the container and things..that almost sounds
like a tools or utility functionality which I would just make a subproject
for called deployment-tools or something...and in there either the pom that
uses the relevant plugin or subprojects if there are multiple behaviours..
thanks for the response :) it looks like we are agreeing on a lot of the
fundamentals which should let jason actually start writing about these
things :P
jesse
On 11/14/05, Vincent Massol <vm...@pivolis.com> wrote:
>
> Hi Jesse,
>
> Thanks for restarting this thread!
>
> See below
>
> > -----Original Message-----
> > From: Jesse McConnell [mailto:jesse.mcconnell@gmail.com]
> > Sent: lundi 7 novembre 2005 20:00
> > To: Maven Developers List
> > Subject: Re: [M2][Proposal] J2EE builds best practices and conventions
> >
> > ok, digging this old thing out the dust bin..
> >
> > I have abandoned my old layout (and the current structure provided by
> the
> > j2ee archetype) and shifting towards something that is more like the
> > current
> > state of the maven repo...and I am trying to decide the lvl of nesting
> > that
> > is useful for the components when factoring vincents threads above..
> >
> > first off, I concede the usefulness of having actual source in the war
> > files...while my architecture doesn't really require/use that kinda
> thing,
> > it would be silly to design around the idea in any sort of best
> practices
> > way :) (that being that war's and the like don't need any compiled
> source
> > at
> > all, just suck in the associated dependencies)
>
> Cool :-)
>
> > ok...I'll start with the basics of the maven layout since that is a
> pretty
> > simple no brainer place to start..and for this discussion to make sense
> as
> > a
> > best practices discussion I'll invent a little project and if it makes
> > sense
> > to later we can apply the daytrader app to it (I don't know enough of
> the
> > daytrader to do it justice atm). I know brett alluded to an exercise
> like
> > this in the initial mail above..
> >
> > Lets say we have a mythical project called dropit...
> >
> > flat:
> >
> > dropit
> > dropit-logging
> > dropit-config
> > dropit-interfaces
> > dropit-frontend-servlet
> > dropit-backdoor-servlet
> > dropit-site
> > dropit-war
> > dropit-ear
> >
> > pretty simple project, three discrete source modules with a seperate
> > interfaces module for other systems to program against..the site, a war
> > and
> > an ear..
>
> Could you explain what the dropit-*-servlet projects are? Shouldn't they
> be
> located in the dropit-war project?
>
> > starting simple I guess there are a couple of questions we ought to
> answer
> > if we want to arrive at a best practice module layout...
> >
> > 1. do we want to distingish wars, servlets, ears and other packaging
> > concepts by the name of the directory or by nesting them? if so, how
> many
> > lvls deep?
> >
> > dropit (the primary source, sister element)
> > wars
> > wars/dropit-frontend
> > wars/dropit-backdoor
>
> +1
>
> > or go for the gusto?
> >
> > ear/wars/dropit-frontend
> > ear/wars/dropit-backend
>
> I don't think this can be considered a best practice. It may happen in
> very
> particular circumstances that it'll work but not generally speaking.
> That's
> because the same jar/war/etc can be packaged into different wars, ears,
> etc.
>
> Also it sounds very weird to me to have a directory structure like:
>
> My project
> |_ src/...
> |_ pom.xml
> |_ dropit-frontend/
> |_ dropit-backend/
>
> Mixing src/ and subprojects at the same level doesn't resonate too well I
> think.
>
> > 2. if we follow another common approach of nesting
> >
> > component/dropit
>
> What is this dropit project inside component?
>
> > component/dropit-logging
> > component/dropit-config
> > packaging/war/dropit-frontend
> > packaging/war/dropit-backdoor
> > packaging/dropit (type ear, generates dropit-1.0.ear)
>
> For me a jar, war, rar, even ear are all the same: they are modules (aka
> components).
>
> > this kinda approach still kind of rubs me raw with there being actual
> > source
> > in the wars since that means you have source under packaging
> > subprojects...
>
> Yep agreed. I don't like it either. I prefer the directory layout I had
> proposed in my earlier posts.
>
> > now we can move on to my personal psychosis :)
>
> :-)
>
> > I think my personal problem with trying to figure out a nice clean
> module
> > layout is that with maven's multiproject abilities it makes it difficult
> > to
> > seperate my previous layout experiences stemming from one overriding
> > build.xml file containing everything woven together, from the more clean
> > seperation of discrete units that maven2 encourages...working through
> the
> > repository for unit artifacts as opposed to other directories in the
> > project
> > really frees things up a lot.
> >
> > to be honest, working out my current setup and reading through the
> backlog
> > of this message I find myself leaning towards a best practices j2ee
> setup
> > of
> > very close to the maven-components one, just utilizing a naming
> convention
> > for j2ee components.
> >
> > dropit
> > dropit-logging
> > dropit-config
> > dropit-interfaces-api
> > dropit-frontend-servlet
> > dropit-backdoor-servlet
> > dropit-war
> > dropit-ear
>
> Yes, I fine with this too except that I'd name it a bit differently:
>
> dropit/
> |_ logging/
> |_ config/
> |_ interfaces-api/
> |_ frontend-servlet/
> |_ backend-servlet/
> |_ war/
> |_ ear/
>
> But again this is a very simple project that doesn't have everything
> automated? What about functional tests? Where do they fit? What about
> container configuration files? What about deployment to the container?
> What
> about database start/stop/loading of data? Etc...
>
> This is why I was suggesting an additional directory level in my previous
> posts:
>
> dropit/
> |_ components/
> |_ [...]
> |_ applications/
> |_ [...]
>
> But I agree that if, as a user, you don't want to automate all those then
> a
> single directory level is enough.
>
> > For a simple project like this, there is no need to venture into lots of
> > convoluted subdirectories that carry along a meaning with them (like the
> > packaging thing and my hangup with there being source in there).
>
> Agreed.
>
> > I would even go so far as to say that if this where a best practices
> > layout
> > that a general rule of thumb could be that if you have 5 or more
> *-servlet
> > artifacts that you nest them into a servlets directory
> >
> > dropit
> > dropit-logging
> > dropit-config
> > dropit-interfaces-api
> > servlets/dropit-frontend-servlet
> > servlets/dropit-backdoor-servlet
> > servlets/dropit-fun-servlet
> > servlets/dropit-silly-servlet
> > servlets/dropit-humor-servlet
> > dropit-war
> > dropit-ear
> >
> > and the same thing would hold for the ejbs, wars, ears, whatever...
>
> +1
>
> > the
> > only
> > exception being artifacts that compile to be jars...then would all exist
> > at
> > the root of the project directory unless it really makes sense to nest
> > components that have maybe 5 or more of a particular thing.
>
> I'd treat jars in the same manner as wars, ejbs, ears, etc.
>
> > which...when you look at it like this closely models the current
> structure
> > of the maven-components svn module and how maven-plugins was nested in
> it
> > with a whole lot of plugins in it.
> >
> > Perhaps we don't necessarily want to promote a j2ee best practices
> layout
> > beyond a general rule of thumb for maven2 projects having all discrete
> > compiling units at the base directory of the project. That and a
> suggested
> > naming convention and nesting convention for j2ee artifacts -war, -ear,
> > -ejb.
>
> Not sure about this naming conventions.
>
> > that accomplishs a lot:
> >
> > 1) it gets us out of the old habit of nesting things just to nest things
> > 2) promotes the concept that a project is made up of a lot of discretely
> > compiling dependencies
> > 3) holds all artifacts as equals, nesting only based on shared
> underlying
> > functionality
> > 4) all 'best practices' for maven2 archetype projects are the
> same...they
> > already have an established layout internal to the sub-project, src,
> > target,etc..
> > 5) best practices for different meta project organizations is reduced to
> > naming conventions, j2ee using -war, -ejb, -ear...others can follow
> their
> > own setups...
> > 6) following this kinda approach ties closely with the multi-project
> > setup...embraces the flatness really...maybe making it easier to explain
> > how
> > inner project dependencies work without any sort of implied knowledge
> > based
> > on project directory layout...
> >
> > does this rambling make any sense to anyone else? :)
>
> Sure, it does! But it doesn't cover the full story.
>
> Thanks
> -Vincent
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
--
jesse mcconnell
jesseDOTmcconnellATgmailDOTcom
RE: [M2][Proposal] J2EE builds best practices and conventions
Posted by Vincent Massol <vm...@pivolis.com>.
Hi Jesse,
Thanks for restarting this thread!
See below
> -----Original Message-----
> From: Jesse McConnell [mailto:jesse.mcconnell@gmail.com]
> Sent: lundi 7 novembre 2005 20:00
> To: Maven Developers List
> Subject: Re: [M2][Proposal] J2EE builds best practices and conventions
>
> ok, digging this old thing out the dust bin..
>
> I have abandoned my old layout (and the current structure provided by the
> j2ee archetype) and shifting towards something that is more like the
> current
> state of the maven repo...and I am trying to decide the lvl of nesting
> that
> is useful for the components when factoring vincents threads above..
>
> first off, I concede the usefulness of having actual source in the war
> files...while my architecture doesn't really require/use that kinda thing,
> it would be silly to design around the idea in any sort of best practices
> way :) (that being that war's and the like don't need any compiled source
> at
> all, just suck in the associated dependencies)
Cool :-)
> ok...I'll start with the basics of the maven layout since that is a pretty
> simple no brainer place to start..and for this discussion to make sense as
> a
> best practices discussion I'll invent a little project and if it makes
> sense
> to later we can apply the daytrader app to it (I don't know enough of the
> daytrader to do it justice atm). I know brett alluded to an exercise like
> this in the initial mail above..
>
> Lets say we have a mythical project called dropit...
>
> flat:
>
> dropit
> dropit-logging
> dropit-config
> dropit-interfaces
> dropit-frontend-servlet
> dropit-backdoor-servlet
> dropit-site
> dropit-war
> dropit-ear
>
> pretty simple project, three discrete source modules with a seperate
> interfaces module for other systems to program against..the site, a war
> and
> an ear..
Could you explain what the dropit-*-servlet projects are? Shouldn't they be
located in the dropit-war project?
> starting simple I guess there are a couple of questions we ought to answer
> if we want to arrive at a best practice module layout...
>
> 1. do we want to distingish wars, servlets, ears and other packaging
> concepts by the name of the directory or by nesting them? if so, how many
> lvls deep?
>
> dropit (the primary source, sister element)
> wars
> wars/dropit-frontend
> wars/dropit-backdoor
+1
> or go for the gusto?
>
> ear/wars/dropit-frontend
> ear/wars/dropit-backend
I don't think this can be considered a best practice. It may happen in very
particular circumstances that it'll work but not generally speaking. That's
because the same jar/war/etc can be packaged into different wars, ears, etc.
Also it sounds very weird to me to have a directory structure like:
My project
|_ src/...
|_ pom.xml
|_ dropit-frontend/
|_ dropit-backend/
Mixing src/ and subprojects at the same level doesn't resonate too well I
think.
> 2. if we follow another common approach of nesting
>
> component/dropit
What is this dropit project inside component?
> component/dropit-logging
> component/dropit-config
> packaging/war/dropit-frontend
> packaging/war/dropit-backdoor
> packaging/dropit (type ear, generates dropit-1.0.ear)
For me a jar, war, rar, even ear are all the same: they are modules (aka
components).
> this kinda approach still kind of rubs me raw with there being actual
> source
> in the wars since that means you have source under packaging
> subprojects...
Yep agreed. I don't like it either. I prefer the directory layout I had
proposed in my earlier posts.
> now we can move on to my personal psychosis :)
:-)
> I think my personal problem with trying to figure out a nice clean module
> layout is that with maven's multiproject abilities it makes it difficult
> to
> seperate my previous layout experiences stemming from one overriding
> build.xml file containing everything woven together, from the more clean
> seperation of discrete units that maven2 encourages...working through the
> repository for unit artifacts as opposed to other directories in the
> project
> really frees things up a lot.
>
> to be honest, working out my current setup and reading through the backlog
> of this message I find myself leaning towards a best practices j2ee setup
> of
> very close to the maven-components one, just utilizing a naming convention
> for j2ee components.
>
> dropit
> dropit-logging
> dropit-config
> dropit-interfaces-api
> dropit-frontend-servlet
> dropit-backdoor-servlet
> dropit-war
> dropit-ear
Yes, I fine with this too except that I'd name it a bit differently:
dropit/
|_ logging/
|_ config/
|_ interfaces-api/
|_ frontend-servlet/
|_ backend-servlet/
|_ war/
|_ ear/
But again this is a very simple project that doesn't have everything
automated? What about functional tests? Where do they fit? What about
container configuration files? What about deployment to the container? What
about database start/stop/loading of data? Etc...
This is why I was suggesting an additional directory level in my previous
posts:
dropit/
|_ components/
|_ [...]
|_ applications/
|_ [...]
But I agree that if, as a user, you don't want to automate all those then a
single directory level is enough.
> For a simple project like this, there is no need to venture into lots of
> convoluted subdirectories that carry along a meaning with them (like the
> packaging thing and my hangup with there being source in there).
Agreed.
> I would even go so far as to say that if this where a best practices
> layout
> that a general rule of thumb could be that if you have 5 or more *-servlet
> artifacts that you nest them into a servlets directory
>
> dropit
> dropit-logging
> dropit-config
> dropit-interfaces-api
> servlets/dropit-frontend-servlet
> servlets/dropit-backdoor-servlet
> servlets/dropit-fun-servlet
> servlets/dropit-silly-servlet
> servlets/dropit-humor-servlet
> dropit-war
> dropit-ear
>
> and the same thing would hold for the ejbs, wars, ears, whatever...
+1
> the
> only
> exception being artifacts that compile to be jars...then would all exist
> at
> the root of the project directory unless it really makes sense to nest
> components that have maybe 5 or more of a particular thing.
I'd treat jars in the same manner as wars, ejbs, ears, etc.
> which...when you look at it like this closely models the current structure
> of the maven-components svn module and how maven-plugins was nested in it
> with a whole lot of plugins in it.
>
> Perhaps we don't necessarily want to promote a j2ee best practices layout
> beyond a general rule of thumb for maven2 projects having all discrete
> compiling units at the base directory of the project. That and a suggested
> naming convention and nesting convention for j2ee artifacts -war, -ear,
> -ejb.
Not sure about this naming conventions.
> that accomplishs a lot:
>
> 1) it gets us out of the old habit of nesting things just to nest things
> 2) promotes the concept that a project is made up of a lot of discretely
> compiling dependencies
> 3) holds all artifacts as equals, nesting only based on shared underlying
> functionality
> 4) all 'best practices' for maven2 archetype projects are the same...they
> already have an established layout internal to the sub-project, src,
> target,etc..
> 5) best practices for different meta project organizations is reduced to
> naming conventions, j2ee using -war, -ejb, -ear...others can follow their
> own setups...
> 6) following this kinda approach ties closely with the multi-project
> setup...embraces the flatness really...maybe making it easier to explain
> how
> inner project dependencies work without any sort of implied knowledge
> based
> on project directory layout...
>
> does this rambling make any sense to anyone else? :)
Sure, it does! But it doesn't cover the full story.
Thanks
-Vincent
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [M2][Proposal] J2EE builds best practices and conventions
Posted by Jesse McConnell <je...@gmail.com>.
ok, digging this old thing out the dust bin..
I have abandoned my old layout (and the current structure provided by the
j2ee archetype) and shifting towards something that is more like the current
state of the maven repo...and I am trying to decide the lvl of nesting that
is useful for the components when factoring vincents threads above..
first off, I concede the usefulness of having actual source in the war
files...while my architecture doesn't really require/use that kinda thing,
it would be silly to design around the idea in any sort of best practices
way :) (that being that war's and the like don't need any compiled source at
all, just suck in the associated dependencies)
ok...I'll start with the basics of the maven layout since that is a pretty
simple no brainer place to start..and for this discussion to make sense as a
best practices discussion I'll invent a little project and if it makes sense
to later we can apply the daytrader app to it (I don't know enough of the
daytrader to do it justice atm). I know brett alluded to an exercise like
this in the initial mail above..
Lets say we have a mythical project called dropit...
flat:
dropit
dropit-logging
dropit-config
dropit-interfaces
dropit-frontend-servlet
dropit-backdoor-servlet
dropit-site
dropit-war
dropit-ear
pretty simple project, three discrete source modules with a seperate
interfaces module for other systems to program against..the site, a war and
an ear..
starting simple I guess there are a couple of questions we ought to answer
if we want to arrive at a best practice module layout...
1. do we want to distingish wars, servlets, ears and other packaging
concepts by the name of the directory or by nesting them? if so, how many
lvls deep?
dropit (the primary source, sister element)
wars
wars/dropit-frontend
wars/dropit-backdoor
or go for the gusto?
ear/wars/dropit-frontend
ear/wars/dropit-backend
2. if we follow another common approach of nesting
component/dropit
component/dropit-logging
component/dropit-config
packaging/war/dropit-frontend
packaging/war/dropit-backdoor
packaging/dropit (type ear, generates dropit-1.0.ear)
this kinda approach still kind of rubs me raw with there being actual source
in the wars since that means you have source under packaging subprojects...
now we can move on to my personal psychosis :)
I think my personal problem with trying to figure out a nice clean module
layout is that with maven's multiproject abilities it makes it difficult to
seperate my previous layout experiences stemming from one overriding
build.xml file containing everything woven together, from the more clean
seperation of discrete units that maven2 encourages...working through the
repository for unit artifacts as opposed to other directories in the project
really frees things up a lot.
to be honest, working out my current setup and reading through the backlog
of this message I find myself leaning towards a best practices j2ee setup of
very close to the maven-components one, just utilizing a naming convention
for j2ee components.
dropit
dropit-logging
dropit-config
dropit-interfaces-api
dropit-frontend-servlet
dropit-backdoor-servlet
dropit-war
dropit-ear
For a simple project like this, there is no need to venture into lots of
convoluted subdirectories that carry along a meaning with them (like the
packaging thing and my hangup with there being source in there).
I would even go so far as to say that if this where a best practices layout
that a general rule of thumb could be that if you have 5 or more *-servlet
artifacts that you nest them into a servlets directory
dropit
dropit-logging
dropit-config
dropit-interfaces-api
servlets/dropit-frontend-servlet
servlets/dropit-backdoor-servlet
servlets/dropit-fun-servlet
servlets/dropit-silly-servlet
servlets/dropit-humor-servlet
dropit-war
dropit-ear
and the same thing would hold for the ejbs, wars, ears, whatever...the only
exception being artifacts that compile to be jars...then would all exist at
the root of the project directory unless it really makes sense to nest
components that have maybe 5 or more of a particular thing.
which...when you look at it like this closely models the current structure
of the maven-components svn module and how maven-plugins was nested in it
with a whole lot of plugins in it.
Perhaps we don't necessarily want to promote a j2ee best practices layout
beyond a general rule of thumb for maven2 projects having all discrete
compiling units at the base directory of the project. That and a suggested
naming convention and nesting convention for j2ee artifacts -war, -ear,
-ejb.
that accomplishs a lot:
1) it gets us out of the old habit of nesting things just to nest things
2) promotes the concept that a project is made up of a lot of discretely
compiling dependencies
3) holds all artifacts as equals, nesting only based on shared underlying
functionality
4) all 'best practices' for maven2 archetype projects are the same...they
already have an established layout internal to the sub-project, src,
target,etc..
5) best practices for different meta project organizations is reduced to
naming conventions, j2ee using -war, -ejb, -ear...others can follow their
own setups...
6) following this kinda approach ties closely with the multi-project
setup...embraces the flatness really...maybe making it easier to explain how
inner project dependencies work without any sort of implied knowledge based
on project directory layout...
does this rambling make any sense to anyone else? :)
jesse
--
jesse mcconnell
jesseDOTmcconnellATgmailDOTcom
RE: [M2][Proposal] J2EE builds best practices and conventions
Posted by Vincent Massol <vm...@pivolis.com>.
> -----Original Message-----
> From: Steve Loughran [mailto:stevel@apache.org]
> Sent: mardi 25 octobre 2005 00:46
> To: Maven Developers List
> Subject: Re: [M2][Proposal] J2EE builds best practices and conventions
[snip]
> > Cool. I wonder if there could be synergies with Cargo... It seems there
> is
> > potentially some overlap.
> >
> > Cargo could even possibly be used internally by Smartfrog to
> > start/stop/deploy to containers as it already supports a big number of
> > containers... The opposite could be true so that Cargo could have an EL
> for
> > generating container configurations (ATM we have either a default config
> for
> > which users can configure important parameters such as ports, log
> levels,
> > security, etc) or a config provided by the user).
>
>
> If it works in a world of signed jars, then yes, we could use it.
I'm not sure whether this affects Cargo or not. The only jars manipulated by
Cargo are the J2EE modules (WAR, EAR, RAR, etc) that need to be
copied/deployed into the containers and those are not created by Cargo so
they're pretty transparent (one exception though: Cargo can do some
introspection to look for container-specific deployment descriptor in those
J2EE modules but that's optional).
> Another thing we like to add is liveness; that is every component
> supports a health check that can be triggered in a walk down the graph
> of deployed components. Some components do nothing but liveness, such as
> one that probes for a web page and fails if the response is out of
> range, the size too small or the content type wrong.
Cool. This is useful. Cargo is not a monitoring tool. It's just a wrapper
that provides a common Java API to configure, start, stop containers and
deploy J2EE archives into them. So I guess it could be used by a higher
level framework such as smartfrog for at least container start/stop. WRT the
configuration, we have something very basic that I'd like to improve but as
Cargo supports existing configuration too it should be easy to integrate
with any tool that generates the config itself.
In any case thanks for pointing out smartfrog. This gives me a new
perspective. I'll need to understand it better but it gives me a new
perspective for cargo to see how it fits into this ecosystem of J2EE
management in general.
I've started a wiki page on
http://cargo.codehaus.org/Comparisons+with+other+tools?refresh=1
I'd be interested to get your feedback on it.
[snip]
> Grab the latest release off the site -http://smartfrog.org/ . Start with
> the tutorial. I dont think we have any out the box demo apps, but maybe
> we should.
Ok. I'll try to find some time to play with it...
Thanks
-Vincent
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [M2][Proposal] J2EE builds best practices and conventions
Posted by Steve Loughran <st...@apache.org>.
>>>>
>>>>I actually work full time on complex system deployment, of which the
>>>>core runtime. SmartFrog, is LGPL-licensed and sourceforge hosted
>>>>(http://smartfrog.org/). If you want to know more, the latest slides are
>>>>
>>>>http://people.apache.org/~stevel/slides/oscon.pdf
>
>
> BTW, I have found this sentence in your OSCon PDF:
> "Systems are composed of applications that are composed of components".
>
> That's very cool as it matches almost exactly the directory structure I had
> in mind. I called components modules but we could them components :-)
>
Components and modules both have so many meanings; 'things you deploy'
is another one.
>
>>>Cool. I'll be reading this and checking out the smartfrog project. More
>>>specifically I need to find out if the open source part of smartfrog is
>>>enough to satisfy the needs for deploying a J2EE system or if it
>>
>>requires
>>
>>>people to buy additional things.
>>
>>to be honest, right now, there is no direct support in our code for
>>jboss or geronimo. I tried to persuade the geronimo people last week
>>that they need to think about it. But there is
>> -JMX control
>> -war deploy to jetty+tomcat
>> -a jboss component from Brazil to do
>>
>>Ear support is on my todo list, Jboss being the obvious target. The goal
>>is to decouple configuring the app from the server, so you can switch
>>from app server provider just by changing an option in a deployment
>>descriptor and redeploying -or better yet, creating a new descriptor
>>that extends a base one for use on a different cluster. That todo list
>>is budy with core work related stuff, which is primarily dull Ws-* and
>>grid related things.
>
>
> Cool. I wonder if there could be synergies with Cargo... It seems there is
> potentially some overlap.
>
> Cargo could even possibly be used internally by Smartfrog to
> start/stop/deploy to containers as it already supports a big number of
> containers... The opposite could be true so that Cargo could have an EL for
> generating container configurations (ATM we have either a default config for
> which users can configure important parameters such as ports, log levels,
> security, etc) or a config provided by the user).
If it works in a world of signed jars, then yes, we could use it.
Another thing we like to add is liveness; that is every component
supports a health check that can be triggered in a walk down the graph
of deployed components. Some components do nothing but liveness, such as
one that probes for a web page and fails if the response is out of
range, the size too small or the content type wrong.
>
>
>>NB, there is no business model of paid-for-premium things. We are part
>>of a research lab. Once you have automated deployment to the extent that
>>you need 10 boxes + the option of 500 more for peak loads, then you have
>>the option of talking to the bit of HP that sells CPU time on demand.
>>But that is merely an option.
>
>
> :-)
>
>
>>Incidentally, on the subject of testing, Vmware are now giving away free
>>vmware runtimes. This lets you test more deployments on a single host,
>>so even the developers can have multi-server deployments on their
>>laptops. slick.
>
>
> Sounds great. I went to the vmware website and couldn't see any offer. Would
> you have more detail?
>
http://www.vmware.com/products/player/
>>>That said, it seems to me that there are maybe different levels:
>>>
>>>- the poor man's solution which could be implemented using Maven. Again
>>
>>this
>>
>>>is what I've done at several places and I know several others who have
>>
>>done
>>
>>>the same successfully. I'm not saying this is the best, I'm just saying
>>
>>that
>>
>>>it can work in some circumstances.
>>
>>
>>Oh I agree, it works just like shell scripts and ant or custom RPMs. The
>>problem is that you cannot easily transfer all the configuraton work
>>into a production system, so the only thing that gets carried over from
>>dev to staging to live systems is experience in debugging configuration
>>problems.
>
>
> Sure. On some projects I had worked on we had retroactive loops where
> configuration and tuning from production teams get reinjected into our SCM
> so that next time we deliver a new version it's closer to the required
> production config. But honestly the production teams I have seen we really
> not at the level of doing something like what smartfrog does... Files were
> edited and modified by hand, no version control, no recording of what was
> done, etc...
Always the case, always the case.
>
>
>>Even small projects have repeated and different deployments. It just
>>doesnt hurt as much doing it by hand as it does on large scale 100+node
>>systems, where manual intervention is clearly impossible.
>
>
> I'd be curious to understand more about how to use smartfrom with a build
> system such as Ant or Maven. Where do you store smartfrog config files (in
> the SCM? In some external repos? In a Maven remote repo? :-)).
When running with security turned on (as you really ought to do
everywhere :). the runtime only deploys descriptors that are in signed
JAR files. Our components stick their templates in the JAR files that
implement the components themselves. Apps that deploy can include them
in their own JARs or in custom things. Once I add security to the Ant
tasks I'll have to create JAR files on the fly from the inline descriptors.
We also have late binding info; you can get at properties of the JVM
>
> Is there a sample application that I could check out from somewhere and run
> on my machine to check how it all works?
>
Grab the latest release off the site -http://smartfrog.org/ . Start with
the tutorial. I dont think we have any out the box demo apps, but maybe
we should.
-steve
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
RE: [M2][Proposal] J2EE builds best practices and conventions
Posted by Vincent Massol <vm...@pivolis.com>.
> -----Original Message-----
> From: Steve Loughran [mailto:stevel@apache.org]
> Sent: lundi 24 octobre 2005 14:24
> To: Maven Developers List
> Subject: Re: [M2][Proposal] J2EE builds best practices and conventions
[snip]
> >>>I think at least at this point configuration management is out of scope
> >>>for Maven. We could use archetypes or similar to generate appropriate
> >>>config files, but in general I've found the differences go beyond
> >>>templating and its best to have a general versioning and deployment
> >>>mechanism for your server setup.
> >>
> >>
> >>I have to agree.
> >>
> >>I actually work full time on complex system deployment, of which the
> >>core runtime. SmartFrog, is LGPL-licensed and sourceforge hosted
> >>(http://smartfrog.org/). If you want to know more, the latest slides are
> >>
> >>http://people.apache.org/~stevel/slides/oscon.pdf
BTW, I have found this sentence in your OSCon PDF:
"Systems are composed of applications that are composed of components".
That's very cool as it matches almost exactly the directory structure I had
in mind. I called components modules but we could them components :-)
> > Cool. I'll be reading this and checking out the smartfrog project. More
> > specifically I need to find out if the open source part of smartfrog is
> > enough to satisfy the needs for deploying a J2EE system or if it
> requires
> > people to buy additional things.
>
> to be honest, right now, there is no direct support in our code for
> jboss or geronimo. I tried to persuade the geronimo people last week
> that they need to think about it. But there is
> -JMX control
> -war deploy to jetty+tomcat
> -a jboss component from Brazil to do
>
> Ear support is on my todo list, Jboss being the obvious target. The goal
> is to decouple configuring the app from the server, so you can switch
> from app server provider just by changing an option in a deployment
> descriptor and redeploying -or better yet, creating a new descriptor
> that extends a base one for use on a different cluster. That todo list
> is budy with core work related stuff, which is primarily dull Ws-* and
> grid related things.
Cool. I wonder if there could be synergies with Cargo... It seems there is
potentially some overlap.
Cargo could even possibly be used internally by Smartfrog to
start/stop/deploy to containers as it already supports a big number of
containers... The opposite could be true so that Cargo could have an EL for
generating container configurations (ATM we have either a default config for
which users can configure important parameters such as ports, log levels,
security, etc) or a config provided by the user).
> NB, there is no business model of paid-for-premium things. We are part
> of a research lab. Once you have automated deployment to the extent that
> you need 10 boxes + the option of 500 more for peak loads, then you have
> the option of talking to the bit of HP that sells CPU time on demand.
> But that is merely an option.
:-)
> Incidentally, on the subject of testing, Vmware are now giving away free
> vmware runtimes. This lets you test more deployments on a single host,
> so even the developers can have multi-server deployments on their
> laptops. slick.
Sounds great. I went to the vmware website and couldn't see any offer. Would
you have more detail?
> >
> > That said, it seems to me that there are maybe different levels:
> >
> > - the poor man's solution which could be implemented using Maven. Again
> this
> > is what I've done at several places and I know several others who have
> done
> > the same successfully. I'm not saying this is the best, I'm just saying
> that
> > it can work in some circumstances.
>
>
> Oh I agree, it works just like shell scripts and ant or custom RPMs. The
> problem is that you cannot easily transfer all the configuraton work
> into a production system, so the only thing that gets carried over from
> dev to staging to live systems is experience in debugging configuration
> problems.
Sure. On some projects I had worked on we had retroactive loops where
configuration and tuning from production teams get reinjected into our SCM
so that next time we deliver a new version it's closer to the required
production config. But honestly the production teams I have seen we really
not at the level of doing something like what smartfrog does... Files were
edited and modified by hand, no version control, no recording of what was
done, etc...
> Even small projects have repeated and different deployments. It just
> doesnt hurt as much doing it by hand as it does on large scale 100+node
> systems, where manual intervention is clearly impossible.
I'd be curious to understand more about how to use smartfrom with a build
system such as Ant or Maven. Where do you store smartfrog config files (in
the SCM? In some external repos? In a Maven remote repo? :-)).
Is there a sample application that I could check out from somewhere and run
on my machine to check how it all works?
Thanks
-Vincent
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [M2][Proposal] J2EE builds best practices and conventions
Posted by Steve Loughran <st...@apache.org>.
Vincent Massol wrote:
>
>>-----Original Message-----
>>From: Steve Loughran [mailto:stevel@apache.org]
>>Sent: lundi 24 octobre 2005 12:52
>>To: Maven Developers List
>>Subject: Re: [M2][Proposal] J2EE builds best practices and conventions
>>
>>Brett Porter wrote:
>>
>>>Vincent Massol wrote:
>>>
>>>
>>>>* support storing in the project the container's configuration files
>>>> - for weblogic that's the domain config
>>>> - for jboss that's the server config
>>>> - for tomcat that's server.xml, tomcat-users.xml, etc
>>>
>>>
>>>I usually put this in a separate module (which I think agrees with
>>>this), or omit it altogether opting to set it up manually for the target
>>>environment or use defaults for testing (where the config for each
>>>webapp would go in src/deployment/tomcat-5/context.xml for example). I
>>>know this doesn't work for all of the servers though.
>>>
>>>
>>>>* support building for different target environments
>>>> - support container-specific deployment descriptor for example
>>>> - support container config files that depend on the environment. For
>>>>example you may want to run on port 8080 in your dev env and on port
>>>>80 in
>>>>your production env. You may want to run on a single server in the dev
>>>>env
>>>>but in cluster on the production env, etc.
>>>
>>>
>>>I think at least at this point configuration management is out of scope
>>>for Maven. We could use archetypes or similar to generate appropriate
>>>config files, but in general I've found the differences go beyond
>>>templating and its best to have a general versioning and deployment
>>>mechanism for your server setup.
>>
>>
>>I have to agree.
>>
>>I actually work full time on complex system deployment, of which the
>>core runtime. SmartFrog, is LGPL-licensed and sourceforge hosted
>>(http://smartfrog.org/). If you want to know more, the latest slides are
>>
>>http://people.apache.org/~stevel/slides/oscon.pdf
>
>
> Cool. I'll be reading this and checking out the smartfrog project. More
> specifically I need to find out if the open source part of smartfrog is
> enough to satisfy the needs for deploying a J2EE system or if it requires
> people to buy additional things.
to be honest, right now, there is no direct support in our code for
jboss or geronimo. I tried to persuade the geronimo people last week
that they need to think about it. But there is
-JMX control
-war deploy to jetty+tomcat
-a jboss component from Brazil to do
Ear support is on my todo list, Jboss being the obvious target. The goal
is to decouple configuring the app from the server, so you can switch
from app server provider just by changing an option in a deployment
descriptor and redeploying -or better yet, creating a new descriptor
that extends a base one for use on a different cluster. That todo list
is budy with core work related stuff, which is primarily dull Ws-* and
grid related things.
NB, there is no business model of paid-for-premium things. We are part
of a research lab. Once you have automated deployment to the extent that
you need 10 boxes + the option of 500 more for peak loads, then you have
the option of talking to the bit of HP that sells CPU time on demand.
But that is merely an option.
Incidentally, on the subject of testing, Vmware are now giving away free
vmware runtimes. This lets you test more deployments on a single host,
so even the developers can have multi-server deployments on their
laptops. slick.
>
> That said, it seems to me that there are maybe different levels:
>
> - the poor man's solution which could be implemented using Maven. Again this
> is what I've done at several places and I know several others who have done
> the same successfully. I'm not saying this is the best, I'm just saying that
> it can work in some circumstances.
Oh I agree, it works just like shell scripts and ant or custom RPMs. The
problem is that you cannot easily transfer all the configuraton work
into a production system, so the only thing that gets carried over from
dev to staging to live systems is experience in debugging configuration
problems.
Even small projects have repeated and different deployments. It just
doesnt hurt as much doing it by hand as it does on large scale 100+node
systems, where manual intervention is clearly impossible.
-Steve
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
RE: [M2][Proposal] J2EE builds best practices and conventions
Posted by Vincent Massol <vm...@pivolis.com>.
> -----Original Message-----
> From: Steve Loughran [mailto:stevel@apache.org]
> Sent: lundi 24 octobre 2005 12:52
> To: Maven Developers List
> Subject: Re: [M2][Proposal] J2EE builds best practices and conventions
>
> Brett Porter wrote:
> > Vincent Massol wrote:
> >
> >> * support storing in the project the container's configuration files
> >> - for weblogic that's the domain config
> >> - for jboss that's the server config
> >> - for tomcat that's server.xml, tomcat-users.xml, etc
> >
> >
> > I usually put this in a separate module (which I think agrees with
> > this), or omit it altogether opting to set it up manually for the target
> > environment or use defaults for testing (where the config for each
> > webapp would go in src/deployment/tomcat-5/context.xml for example). I
> > know this doesn't work for all of the servers though.
> >
> >>
> >> * support building for different target environments
> >> - support container-specific deployment descriptor for example
> >> - support container config files that depend on the environment. For
> >> example you may want to run on port 8080 in your dev env and on port
> >> 80 in
> >> your production env. You may want to run on a single server in the dev
> >> env
> >> but in cluster on the production env, etc.
> >
> >
> > I think at least at this point configuration management is out of scope
> > for Maven. We could use archetypes or similar to generate appropriate
> > config files, but in general I've found the differences go beyond
> > templating and its best to have a general versioning and deployment
> > mechanism for your server setup.
>
>
> I have to agree.
>
> I actually work full time on complex system deployment, of which the
> core runtime. SmartFrog, is LGPL-licensed and sourceforge hosted
> (http://smartfrog.org/). If you want to know more, the latest slides are
>
> http://people.apache.org/~stevel/slides/oscon.pdf
Cool. I'll be reading this and checking out the smartfrog project. More
specifically I need to find out if the open source part of smartfrog is
enough to satisfy the needs for deploying a J2EE system or if it requires
people to buy additional things.
That said, it seems to me that there are maybe different levels:
- the poor man's solution which could be implemented using Maven. Again this
is what I've done at several places and I know several others who have done
the same successfully. I'm not saying this is the best, I'm just saying that
it can work in some circumstances.
- using separate tools such as smartfrog. If someone knows of other tools
doing this please let me know.
> Key things about configuring production systems are that
> -you have many systems, all slightly different
> -ops dont always let engineering near them
> -every part of a system needs to be configured consistently, and their
> start/stop choreographed properly
>
> Maven, like ant is a build tool. Its used by developers; not
> operations. Its certainly something that can
> -take configuration hints from developers
> -populates deployment descriptors from this data. e.g. libraries &
> versions used at build times
>
> However, it cannot be the tool used by operations, and should not try to
> do this. its a different problem, arguably a more complex one.
>
> Generating stub web.xml, ear.xml config files is something that build
> tools (and xdoclet) try and do. But there the problem is really that a
> build time artifact (the WAR and EAR file) also contains late-binding,
> often cluster-specific, configuration data alongside static stuff like
> the names of servlets.
All true. What I think a Maven build can easily do is up to the packaging of
the solutions, including environment-specific configuration files.
I do agree that Maven is not geared towards full-fledged runtime management.
However in my past experience most companies using simple shell scripts to
do that and having Maven package those scripts is a simple solution.
Should people have more complex needs then they should definitely use a
proper system like smartfrog and it would be nice to see how Maven would
integrate with such a system.
> >> - support different deployment models per environment. For example
> >> in one
> >> environment you may want to have the web server and EJB server on the
> >> same
> >> machine and in another environment you may want them located on 2
> >> different
> >> machines.
> >
>
> That is exactly the kind of stuff we can do in smartfrog. If you want to
> do the maven/maven2 plugin, I will help you test it.
Very cool. Let me check the docs first so that I get a better understanding
of how it would fit. Thanks for the offer.
Thanks
-Vincent
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [M2][Proposal] J2EE builds best practices and conventions
Posted by Steve Loughran <st...@apache.org>.
Brett Porter wrote:
> Vincent Massol wrote:
>
>> * support storing in the project the container's configuration files
>> - for weblogic that's the domain config
>> - for jboss that's the server config
>> - for tomcat that's server.xml, tomcat-users.xml, etc
>
>
> I usually put this in a separate module (which I think agrees with
> this), or omit it altogether opting to set it up manually for the target
> environment or use defaults for testing (where the config for each
> webapp would go in src/deployment/tomcat-5/context.xml for example). I
> know this doesn't work for all of the servers though.
>
>>
>> * support building for different target environments
>> - support container-specific deployment descriptor for example
>> - support container config files that depend on the environment. For
>> example you may want to run on port 8080 in your dev env and on port
>> 80 in
>> your production env. You may want to run on a single server in the dev
>> env
>> but in cluster on the production env, etc.
>
>
> I think at least at this point configuration management is out of scope
> for Maven. We could use archetypes or similar to generate appropriate
> config files, but in general I've found the differences go beyond
> templating and its best to have a general versioning and deployment
> mechanism for your server setup.
I have to agree.
I actually work full time on complex system deployment, of which the
core runtime. SmartFrog, is LGPL-licensed and sourceforge hosted
(http://smartfrog.org/). If you want to know more, the latest slides are
http://people.apache.org/~stevel/slides/oscon.pdf
Key things about configuring production systems are that
-you have many systems, all slightly different
-ops dont always let engineering near them
-every part of a system needs to be configured consistently, and their
start/stop choreographed properly
Maven, like ant is a build tool. Its used by developers; not
operations. Its certainly something that can
-take configuration hints from developers
-populates deployment descriptors from this data. e.g. libraries &
versions used at build times
However, it cannot be the tool used by operations, and should not try to
do this. its a different problem, arguably a more complex one.
Generating stub web.xml, ear.xml config files is something that build
tools (and xdoclet) try and do. But there the problem is really that a
build time artifact (the WAR and EAR file) also contains late-binding,
often cluster-specific, configuration data alongside static stuff like
the names of servlets.
>
>> - support different deployment models per environment. For example
>> in one
>> environment you may want to have the web server and EJB server on the
>> same
>> machine and in another environment you may want them located on 2
>> different
>> machines.
>
That is exactly the kind of stuff we can do in smartfrog. If you want to
do the maven/maven2 plugin, I will help you test it.
-steve
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [M2][Proposal] J2EE builds best practices and conventions
Posted by Steve Loughran <st...@apache.org>.
Brett Porter wrote:
> Vincent Massol wrote:
>
>> I've used Maven in the past to do that and it worked very well. I'd
>> like to
>> understand better what alternatives you're suggesting?
>>
>> Note for versioning: we have that by storing the config in the SCM.
>>
>> I'm curious to know which tool you're using in the J2EE space for doing
>> "versioning and deployment". I don't know any.
>
>
> My point was similar to Steve's. Yes, we can provide tools to help, but
> I don't see us coming up with the grand unified theory of configuration
> management. We used to store our configs in SCM and have some simple
> server side tools (that a deployer could use, as developers certainly
> weren't getting up in the middle of the night for releases :)
yeah,. they wake you when you get it wrong anyway. Although we have a
gui for managing a running (remote) system, ops people like command line
tools. Indeed, they like command line tools in Perl ;)
>> Note: I've also tried some other solution. One of them was to have a
>> configuration module in charge of generating the configuration for a
>> given
>> server and a given environment. A centralized configuration. It didn't
>> work
>> too well. It was too hard to keep it centralized. We didn't use any
>> tool. We
>> built our solution based on templates/velocity. I've had better
>> success with
>> a distributed configuration approach where each module is in charge of
>> generating its configuration per environment. But I'm interested in other
>> people's feedback on this!
>
>
> Yes, I agree with this approach too. About the only time centralised
> configuration might make sense is if you are talking about application
> configuration via LDAP or a database, for example.
Here are some things I've done in the past
-war files that get their config from a per-hostname XML File in the
war. Lets you do one war, many servers. Ops cannot change things without
a rebuild, but it stops them doing changes that aint on SCM.
-LDAP. Another point of failure and config. Replication is good for scale.
-Database: no extra complexity if there is a db in it already.
-DNS. custom hostnames for boxes with different roles in the cluster.
If you look at what smartfrog does, it has the notion of
-components that configure things and move it through a lifecycle
-a language for describing the config that includes cross-referenced stuff
-runtimes that talk to each other using RMI over SSL.
-dynamic download of component implementation classes (with signed JARs)
-a component, Anubis, that does robust multicast coordination. you
need this manage things in clusters properly.
-the ability for things to add runtime data to the config graph., and
for that data to persist past the lifetimeof the component
-containers handling failures of their kids. lets you do workflow.
components that retry deploying with exponential backoff. and a bit of
jitter, etc. etc.
Runtime configuration management is a painful and complex beast. Believe
me. Build time generation of smartfrog descriptors might be something to
consider: creating the templates for other apps to handle. The problem
then becomes
-how to create the template
-how to deploy it
-maybe how to get runtime information back from the graph into maven
-how to undeploy everything afterwards.
-security
Steve
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
RE: [M2][Proposal] J2EE builds best practices and conventions
Posted by Vincent Massol <vm...@pivolis.com>.
> -----Original Message-----
> From: Brett Porter [mailto:brett@apache.org]
> Sent: lundi 24 octobre 2005 21:03
> To: Maven Developers List
> Subject: Re: [M2][Proposal] J2EE builds best practices and conventions
>
> Vincent Massol wrote:
> > I've used Maven in the past to do that and it worked very well. I'd like
> to
> > understand better what alternatives you're suggesting?
> >
> > Note for versioning: we have that by storing the config in the SCM.
> >
> > I'm curious to know which tool you're using in the J2EE space for doing
> > "versioning and deployment". I don't know any.
>
> My point was similar to Steve's. Yes, we can provide tools to help, but
> I don't see us coming up with the grand unified theory of configuration
> management. We used to store our configs in SCM and have some simple
> server side tools (that a deployer could use, as developers certainly
> weren't getting up in the middle of the night for releases :)
Sure...
My personal strategy has always been the following:
- automate as much as possible along the course of the project (improve
automation release after release) so that you can automated everything from
code to installation and deployment.
- take into the account the different environments (I usually have 2 or 3:
development, demo or integration and pre-production) in order to generate
distributions as usable as possible.
- for the development and demo/integration environments it's usually the
development team that manages it so no issue.
- for the pre-production environment, I just hand our distribution to the
production team and they do whatever they want with it. They use their own
tools to do deployment, they repackage everything, etc. We do try to work in
harmony so that we invite them into the development team ahead of putting
the application in production so that they tell us their needs and we
implement that in the build to get it as close as what they need.
Now I have seen the case where the production team is interested by the
tools we use for building the system and they have agreed to themselves use
Maven to build the system. What I haven't seen yet is the case where the
production team uses a tool like smartfrog. I'm curious to see what becomes
the contract between the development team and the production team. What do
they take from the development team in that case? Just simple and
non-configured EARs? Do smartfrog config files get committed to the SCM and
extend to the development and demo/integration environments so that only one
solution is used for configuration/deployment?
[snip]
> >>>* support automated functional tests execution
> >>
> >>Cactus, right? :)
> >
> >
> > Nope. Cargo :-)
>
> But cargo isn't doing the actual testing, its just starting and stopping
> the server and setting up the environment. What do you use for the
> testing aspect?
JUnit, TestNG, HttpUnit, Canoo, Abbot, whatever testing framework you want
to use. Cactus can be used too of course. Cargo's strengths is its ability
to perform integration tests that would be complex to do using a black box
strategy.
[snip]
> > The extra jars directories (ejbs/, ears/, etc) are not really required.
> They
> > may be required if you have a big applications with lots of ejbs, wars,
> etc.
> > I'm not recommending a fixed structure with this additional directory
> level.
> > I'm just saying it can exist.
>
> Well, anything can exist, I'm just not sure we'd recommend that layout
> (which is what I thought this was :)
It is... but there are parts for which we can't recommend a structure for as
it's too dependent on the system being built.
Here's an example of the directory structure that I would have used for a
past project:
<project>/
|_ components/
|_ businessServer/ (components for an EAR app)
|_ partners/ (ejb)
|_ clearing/ (ejb)
|_ positions/ (ejb)
|_ referenceData/ (ejb)
|_ [...]
|_ businessServer/ (ear)
|_ client/ (components for a swing app)
|_ partners/ (ejb)
|_ clearing/ (ejb)
|_ positions/ (ejb)
|_ referenceData/ (ejb)
|_ [...]
|_ client/ (jar containing the main() method)
|_ eai/
|_ migration/
|_ [...]
|_ applications/
|_ [...]
|_ system/
As you can see the top level structure is the same as the one proposed but
the nested components/ projects do not follow a specific pattern that we
could propose. BTW I personally prefer naming components with functional
names rather than ejb, jar, ear, etc. But it's hard to do so for a generic
archetype... :-)
> > I'm not sure what this is about?
> >
> > In any case a nested directory structure such as:
> >
> > ear/
> > |_ war/
> > |_ jar1/
> > |_ ejb/
> > |_ jar2/
> >
> > Wouldn't really work, as J2EE modules are created by composition and not
> by
> > inheritance.
> >
> > I don't know if you're referring to this or not.
>
> Yes, this is what I was referring to. I see this as composition with the
> ear as the end result. I understand that the jars might go directly in
> the EAR though and so be at the same level.
>
> I'm not certain of the cirucmstances under which either would be needed,
> and I'm happy to have the ear as a sibling to the others, I just
> remembered the aggregation alternative had been discussed.
Yep. There was a short discussion and we've also discussed it with Jason. It
can probably be done in some special cases but I wouldn't see it as a
general solution. I'll still mention it.
> > Note: I've also tried some other solution. One of them was to have a
> > configuration module in charge of generating the configuration for a
> given
> > server and a given environment. A centralized configuration. It didn't
> work
> > too well. It was too hard to keep it centralized. We didn't use any
> tool. We
> > built our solution based on templates/velocity. I've had better success
> with
> > a distributed configuration approach where each module is in charge of
> > generating its configuration per environment. But I'm interested in
> other
> > people's feedback on this!
>
> Yes, I agree with this approach too. About the only time centralised
> configuration might make sense is if you are talking about application
> configuration via LDAP or a database, for example.
>
> Overall sounds pretty good, I'd probably understand more from seeing it
> in action and fine tuning it by tinkering with it and oing through the
> process of doing deployments, etc.
Thanks for you feedback
-Vincent
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [M2][Proposal] J2EE builds best practices and conventions
Posted by Brett Porter <br...@apache.org>.
Vincent Massol wrote:
> I've used Maven in the past to do that and it worked very well. I'd like to
> understand better what alternatives you're suggesting?
>
> Note for versioning: we have that by storing the config in the SCM.
>
> I'm curious to know which tool you're using in the J2EE space for doing
> "versioning and deployment". I don't know any.
My point was similar to Steve's. Yes, we can provide tools to help, but
I don't see us coming up with the grand unified theory of configuration
management. We used to store our configs in SCM and have some simple
server side tools (that a deployer could use, as developers certainly
weren't getting up in the middle of the night for releases :)
>>>* support deploying config and J2EE modules to a container
>>
>>Cargo, right?
>
>
> Yes and no. Cargo does this for sure. However I'm still pondering whether
> it's interesting to bring Cargo to the runtime vs storing shell scripts.
> Cargo could be used to generate the config at build time. Now for run time
> we could also bring it by having a cargo-execution.jar for example (java
> -jar cargo-execution.jar start cargo.xml). Not sure yet if it's useful when
> compared with a shell script.
I was really looking at the deployment aspect here, rather than creating
shell scripts.
>
>
>>>* support automated functional tests execution
>>
>>Cactus, right? :)
>
>
> Nope. Cargo :-)
But cargo isn't doing the actual testing, its just starting and stopping
the server and setting up the environment. What do you use for the
testing aspect?
>
> It's just a convenient way of separating them. Putting them all at the same
> level would just be confusing.
ok.
>
> The extra jars directories (ejbs/, ears/, etc) are not really required. They
> may be required if you have a big applications with lots of ejbs, wars, etc.
> I'm not recommending a fixed structure with this additional directory level.
> I'm just saying it can exist.
Well, anything can exist, I'm just not sure we'd recommend that layout
(which is what I thought this was :)
> I'm not sure what this is about?
>
> In any case a nested directory structure such as:
>
> ear/
> |_ war/
> |_ jar1/
> |_ ejb/
> |_ jar2/
>
> Wouldn't really work, as J2EE modules are created by composition and not by
> inheritance.
>
> I don't know if you're referring to this or not.
Yes, this is what I was referring to. I see this as composition with the
ear as the end result. I understand that the jars might go directly in
the EAR though and so be at the same level.
I'm not certain of the cirucmstances under which either would be needed,
and I'm happy to have the ear as a sibling to the others, I just
remembered the aggregation alternative had been discussed.
> Note: I've also tried some other solution. One of them was to have a
> configuration module in charge of generating the configuration for a given
> server and a given environment. A centralized configuration. It didn't work
> too well. It was too hard to keep it centralized. We didn't use any tool. We
> built our solution based on templates/velocity. I've had better success with
> a distributed configuration approach where each module is in charge of
> generating its configuration per environment. But I'm interested in other
> people's feedback on this!
Yes, I agree with this approach too. About the only time centralised
configuration might make sense is if you are talking about application
configuration via LDAP or a database, for example.
Overall sounds pretty good, I'd probably understand more from seeing it
in action and fine tuning it by tinkering with it and oing through the
process of doing deployments, etc.
- Brett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
RE: [M2][Proposal] J2EE builds best practices and conventions
Posted by Vincent Massol <vm...@pivolis.com>.
Hi Brett,
> -----Original Message-----
> From: Brett Porter [mailto:brett@apache.org]
> Sent: lundi 24 octobre 2005 09:52
> To: Maven Developers List
> Subject: Re: [M2][Proposal] J2EE builds best practices and conventions
>
> Vincent Massol wrote:
> > * support storing in the project the container's configuration files
> > - for weblogic that's the domain config
> > - for jboss that's the server config
> > - for tomcat that's server.xml, tomcat-users.xml, etc
>
> I usually put this in a separate module (which I think agrees with
> this), or omit it altogether opting to set it up manually for the target
> environment or use defaults for testing (where the config for each
> webapp would go in src/deployment/tomcat-5/context.xml for example). I
> know this doesn't work for all of the servers though.
Yes we're in agreement.
BTW I haven't found a single J2EE servers yet where it's not possible to do
this. It's definitely not always possible for all types of servers though.
> > * support building for different target environments
> > - support container-specific deployment descriptor for example
> > - support container config files that depend on the environment. For
> > example you may want to run on port 8080 in your dev env and on port 80
> in
> > your production env. You may want to run on a single server in the dev
> env
> > but in cluster on the production env, etc.
>
> I think at least at this point configuration management is out of scope
> for Maven. We could use archetypes or similar to generate appropriate
> config files, but in general I've found the differences go beyond
> templating and its best to have a general versioning and deployment
> mechanism for your server setup.
I've used Maven in the past to do that and it worked very well. I'd like to
understand better what alternatives you're suggesting?
Note for versioning: we have that by storing the config in the SCM.
I'm curious to know which tool you're using in the J2EE space for doing
"versioning and deployment". I don't know any.
> > - support different deployment models per environment. For example in
> one
> > environment you may want to have the web server and EJB server on the
> same
> > machine and in another environment you may want them located on 2
> different
> > machines.
>
> This ties in to the above.
>
> >
> > * support deploying config and J2EE modules to a container
>
> Cargo, right?
Yes and no. Cargo does this for sure. However I'm still pondering whether
it's interesting to bring Cargo to the runtime vs storing shell scripts.
Cargo could be used to generate the config at build time. Now for run time
we could also bring it by having a cargo-execution.jar for example (java
-jar cargo-execution.jar start cargo.xml). Not sure yet if it's useful when
compared with a shell script.
> > * support automated functional tests execution
>
> Cactus, right? :)
Nope. Cargo :-)
> > Directory structure
> > ===================
> >
> > In term of directory structure, I'm proposing the following:
> >
> > <project>/
> > |
> > |_ modules/
> > |_ jars/
> > |_ <functional name for jar M>/
> > |_ ejbs/
> > |_ <functional name for ejb N>/
> > |_ wars/
> > |_ <functional name for war O>/
> > |_ ears/
> > |_ <functional name for ear P>/
> > |_ clients/
> > |_ <functional name for client Q>/
> > |_ rars/
> > |_ <functional name for rar R>/
>
> I don't understand why the extra modules and jars directories are
> needed. Is this really the separation you want, or is it separated by
> application structure and the packaging is part of that? That is the
> model we seem to have followed so far.
The extra modules/ dir is needed because at the top level you would have 3
dirs:
- modules/
- applications/
- system/
It's just a convenient way of separating them. Putting them all at the same
level would just be confusing.
The extra jars directories (ejbs/, ears/, etc) are not really required. They
may be required if you have a big applications with lots of ejbs, wars, etc.
I'm not recommending a fixed structure with this additional directory level.
I'm just saying it can exist.
> Also, are we still considering making the aggregating packagings the
> reactor root, using modules instead of dependencies?
I'm not sure what this is about?
In any case a nested directory structure such as:
ear/
|_ war/
|_ jar1/
|_ ejb/
|_ jar2/
Wouldn't really work, as J2EE modules are created by composition and not by
inheritance.
I don't know if you're referring to this or not.
> > * The applications/ directory contains subprojects (aka modules in m2
> speak)
> > that contain executable applications, i.e. things that will make up the
> full
> > system. Applications are made of 2 things:
> > - modules
> > - configuration files including scripts (e.g. start/stop scripts, etc)
>
> What really makes these any different from the other modules? In
> particular, clients?
Clients are inside applications/ yes...
The modules/ are not directly executables. They are libraries. To execute
them you need configuration, the ability to start/stop/deploy/etc.
> > * The database/ application represents the database. This is where
> schema
> > and data could be located. This is also where database script files can
> be
> > stored for starting/stopping/analyzing logs/profiling/etc.
>
> This seems fine.
>
> >
> > * The business/ application would be the business logic executing in the
> > J2EE container. It would package the J2EE modules (EARs and possibly
> others)
> > and the container configuration files.
>
> This name doesn't really represent this to me. It seems like server or
> system was better.
The name is yours to choose. I'm certainly not suggesting we standardize on
one. That's why I've written "<functional name>". I do suggest to use
functional name as much as possible but that's not always easy either.
A system represent the whole thing (EAI servers, J2EE servers, database
servers, web servers, etc).
A server is a generic name. There are several servers: EAI servers, J2EE
servers, database servers, etc.
> > * The wsclient/ application is just an example showing a client
> application
> > (ws stands for web services here). In this example this would be a
> sample
> > swing application.
>
> As above, I'm not sure why this is any different from other modules.
>
> > * The system/ project represents the full system. It is made of
> > applications. It'll also deliver a distribution (zipped file, tar.gz,
> etc)
> > which will aggregate all the distributions from the other applications.
> This
> > represents the full system for a target environment.
>
> Why is the whole system deployed as a single unit? Shouldn't it be
> possible to deploy individual units?
It is. This is just a convenient place to create a single distribution. I do
agree that individual units could/should be deployed separately in each
applications/ project.
[snip]
> > * functional/system tests would be run in system/ (or application/)
>
> is it necesary to get that far into the system to do this type of testing?
Actually you could start doing functional tests as early as in the modules
using Cargo. But this is only because Cargo has the ability to create a
container config from scratch for you and is able to start/stop containers
and deploy to them. However Cargo does not handle starting/stopping the
database for example, nor putting data in it, nor EAR servers (at least not
yet), etc.
Thus you could do some integration tests but they're not going to be really
fully functional. Also if you're doing it in ear/ for example you're not
going to test with your real target configuration.
You could also do the functional tests in applications/ but again you won't
have all the servers set up and started so you'll not be able to do real
functional tests. You could possibly do some integration tests though with
some parts being stubbed or mocked.
> > I'm pretty sure all this needs to be refined but I'm curious to get some
> > feedback from the community.
>
> My overall impression is that this is something that needs to be more
> flexible than anything else. The above sounds like a convenient way to
> deploy some environments, but I think it could end up being a bit too
> complex to cover all the necessary alternatives.
This would just be a canvas and a general solution for performing the full
development lifecycle of a J2EE application. As you mentioned above, if you
don't want to manage your configurations with Maven, you would simply
implement the modules/ part and not the applications/ ones.
I'm interested to know if there are situations that would not fit in the
described proposal.
> I think I'd like to see more about the initial requirements that led to
> this, and whether it can be built up of several smaller solutions?
My requirements come from the J2EE projects I've worked on (about 10 in
total I'd say). Among them were 3 big projects where I've implemented what's
described above. I've also worked on smaller ones where this would have
worked but the team hasn't reached that level of automation/maturity yet. In
others there wasn't the need expressed to go from code to production
quickly.
The proposal above is modular. I think there are 3 levels that are
independent:
- level 1: modules/ level
- level 2: applications/ level and storing container configuration in SCM
- level 3: automation of installations/deployments and control of runtime
scripts through Maven.
As a build engineer, you can start with level 1 and stop there. Or you can
decide to go to level 2. Again you can stop there or you can decide to go to
level 3.
Note: I've also tried some other solution. One of them was to have a
configuration module in charge of generating the configuration for a given
server and a given environment. A centralized configuration. It didn't work
too well. It was too hard to keep it centralized. We didn't use any tool. We
built our solution based on templates/velocity. I've had better success with
a distributed configuration approach where each module is in charge of
generating its configuration per environment. But I'm interested in other
people's feedback on this!
Thanks
-Vincent
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [M2][Proposal] J2EE builds best practices and conventions
Posted by Brett Porter <br...@apache.org>.
Vincent Massol wrote:
> * support storing in the project the container's configuration files
> - for weblogic that's the domain config
> - for jboss that's the server config
> - for tomcat that's server.xml, tomcat-users.xml, etc
I usually put this in a separate module (which I think agrees with
this), or omit it altogether opting to set it up manually for the target
environment or use defaults for testing (where the config for each
webapp would go in src/deployment/tomcat-5/context.xml for example). I
know this doesn't work for all of the servers though.
>
> * support building for different target environments
> - support container-specific deployment descriptor for example
> - support container config files that depend on the environment. For
> example you may want to run on port 8080 in your dev env and on port 80 in
> your production env. You may want to run on a single server in the dev env
> but in cluster on the production env, etc.
I think at least at this point configuration management is out of scope
for Maven. We could use archetypes or similar to generate appropriate
config files, but in general I've found the differences go beyond
templating and its best to have a general versioning and deployment
mechanism for your server setup.
> - support different deployment models per environment. For example in one
> environment you may want to have the web server and EJB server on the same
> machine and in another environment you may want them located on 2 different
> machines.
This ties in to the above.
>
> * support deploying config and J2EE modules to a container
Cargo, right?
>
> * support automated functional tests execution
Cactus, right? :)
> Directory structure
> ===================
>
> In term of directory structure, I'm proposing the following:
>
> <project>/
> |
> |_ modules/
> |_ jars/
> |_ <functional name for jar M>/
> |_ ejbs/
> |_ <functional name for ejb N>/
> |_ wars/
> |_ <functional name for war O>/
> |_ ears/
> |_ <functional name for ear P>/
> |_ clients/
> |_ <functional name for client Q>/
> |_ rars/
> |_ <functional name for rar R>/
I don't understand why the extra modules and jars directories are
needed. Is this really the separation you want, or is it separated by
application structure and the packaging is part of that? That is the
model we seem to have followed so far.
Also, are we still considering making the aggregating packagings the
reactor root, using modules instead of dependencies?
> * The applications/ directory contains subprojects (aka modules in m2 speak)
> that contain executable applications, i.e. things that will make up the full
> system. Applications are made of 2 things:
> - modules
> - configuration files including scripts (e.g. start/stop scripts, etc)
What really makes these any different from the other modules? In
particular, clients?
> * The database/ application represents the database. This is where schema
> and data could be located. This is also where database script files can be
> stored for starting/stopping/analyzing logs/profiling/etc.
This seems fine.
>
> * The business/ application would be the business logic executing in the
> J2EE container. It would package the J2EE modules (EARs and possibly others)
> and the container configuration files.
This name doesn't really represent this to me. It seems like server or
system was better.
>
> * The wsclient/ application is just an example showing a client application
> (ws stands for web services here). In this example this would be a sample
> swing application.
As above, I'm not sure why this is any different from other modules.
> * The system/ project represents the full system. It is made of
> applications. It'll also deliver a distribution (zipped file, tar.gz, etc)
> which will aggregate all the distributions from the other applications. This
> represents the full system for a target environment.
Why is the whole system deployed as a single unit? Shouldn't it be
possible to deploy individual units?
>
> Other considerations
> ====================
>
> * Environment will be managed using profiles.
> * Environment specific resources can go in: src/main/resources-{env name}
> * Common resources can go in src/main/resources and use filters (one filter
> file per environment for example).
> * Maven-runtime would be nice to generate scripts
sounds fine
> * functional/system tests would be run in system/ (or application/)
is it necesary to get that far into the system to do this type of testing?
> I'm pretty sure all this needs to be refined but I'm curious to get some
> feedback from the community.
My overall impression is that this is something that needs to be more
flexible than anything else. The above sounds like a convenient way to
deploy some environments, but I think it could end up being a bit too
complex to cover all the necessary alternatives.
I think I'd like to see more about the initial requirements that led to
this, and whether it can be built up of several smaller solutions?
Cheers,
Brett
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
RE: [M2][Proposal] J2EE builds best practices and conventions
Posted by Vincent Massol <vm...@pivolis.com>.
Hi David,
> -----Original Message-----
> From: David Le Strat [mailto:dlestrat@yahoo.com]
> Sent: samedi 22 octobre 2005 19:21
> To: Maven Developers List
> Subject: Re: [M2][Proposal] J2EE builds best practices and conventions
[snip]
> > * support storing in the project the container's
> > configuration files
> > - for weblogic that's the domain config
> > - for jboss that's the server config
> > - for tomcat that's server.xml, tomcat-users.xml,
> > etc
>
> For domain management, it could be interesting to
> provide a subsystem for domain creation management.
> For instance, with Weblogic 8.1 and above, you can use
> a Jython based scripting to programmatically manage
> the creation of domains. As domains often require
> configurations that are specific to the environment
> type this could be quite useful. Again, in the case
> of a Weblogic domain, you write a WLST (Jython
> script), that leverages a domain template (jar file)
> to generate the target domain. Often, the template
> needs to be adjusted for the type of domains that
> needs to be created. The startup scripts, config.xml,
> workshop config require settings that are slightly
> different for a production domain than for a
> development domain. It would be nice to provide a set
> of templates for different type of configurations
> (Cluster template, vs single domain template, etc...)
Could you please explain the need for a scripting language to generate
configuration files?
The way I've done it in the past was to have this kind of directory
structure:
applications/
|_ <project>/
|_ src/main/config
|_ development/
|_ <development-specific config files go here>
|_ integration/
|_ <integration-specific config files go here>
|_ production/
|_ <production-specific config files go here>
|_ [...]
|_ shared/
|_ <shared config files go here>
And then support filter tokens for shared config files.
Now, if some scripting language is required, I'd still see the same
directory structure but with the addition of a m2 plugin for the scripting
language that you would bind to the generate-resources phase so that it
generates some config files.
WDYT?
[snip]
> Regarding the DB, do you have any thoughts on how to
> manage DB scripts releases. For a development system,
> the database can often be recreated, therefore a list
> of baseline scripts is required as well as a set of
> data seeding scripts.
>
> For a production system, the approach is different,
> where a baseline is required when going live the first
> time and then the scripts are required to be
> incremental with in some instance point in time data
> migration scripts.
This is very true. However I don' think it changes the concept of having a
database/ project and the directory structure could again be similar to the
one above, split by environment-type. For the production environment the
distribution would simply package a set of scripts that would be run in a
well-defined order in order to move the database from one state to the new
state corresponding to the new release.
> This is quite a broad topic of discussion, but an
> important one as in my experience database script
> management is often a pain point. Applications usally
> do not contain checks providing them with the ability
> to tie the version of the database to the version of
> the code base.
Definitely!
> > Note that for a simple project the structure can be
> > simplified to:
> >
> > <project>/
> > |
> > |_ modules/
Oops... just noticed the typo. It should be applications/ and not modules/
here... sorry...
[snip]
> +1, I would add database version check as well.
What do you mean? Is this for all environments or only for the "live" ones?
[snip]
> Cargo looks interesting, I would also look like at a
> WLST like approach that provides a Jython wrapper
> around JMX.
There's a decision to take here for the runtime part. There are several
possibilities:
1/ Generate a distribution that includes all the scripts (Jython or other)
to perform all admin operations on the system (start, stop, deploy,
undeploy, etc).
2/ Assume that Maven is installed on the machines where the
deployment/installation will happen and use the applications/ projects to
run all admin operations (start, stop, deploy, undeploy, etc). This means
that Maven will be used not only for the development part but also at
runtime.
3/ Do the same as 1/ but hide Maven by embedding it in the distribution. All
admin operations are provided by m2 plugins as in 2 but there's no need to
have Maven installed on the target machines.
I feel 3/ would be the best.
> What are the next steps in your mind?
I'm going to write about all this. At the same time, I'm implementing it on
a J2EE project which I'll use as a sample for the writing. Send me an
offline email if you're interested in discussing more about it. I'll make
all of this public eventually once it's finished.
Thanks
-Vincent
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: [M2][Proposal] J2EE builds best practices and conventions
Posted by David Le Strat <dl...@yahoo.com>.
Vincent,
Thanks for sharing this proposal. Very well put. We
follow a similar structure in my correct project to
deal with code releases, DB scripts and scripts
promotion, domains management and other
configurations. I have some comments below:
--- Vincent Massol <vm...@pivolis.com> wrote:
>
> Build Features
> ==============
>
> Basically here are the features that I'd like a
> full-fledge J2EE build to
> perform:
>
> * build J2EE modules (JAR, WAR, EJB, EAR, etc)
>
> * support storing in the project the container's
> configuration files
> - for weblogic that's the domain config
> - for jboss that's the server config
> - for tomcat that's server.xml, tomcat-users.xml,
> etc
For domain management, it could be interesting to
provide a subsystem for domain creation management.
For instance, with Weblogic 8.1 and above, you can use
a Jython based scripting to programmatically manage
the creation of domains. As domains often require
configurations that are specific to the environment
type this could be quite useful. Again, in the case
of a Weblogic domain, you write a WLST (Jython
script), that leverages a domain template (jar file)
to generate the target domain. Often, the template
needs to be adjusted for the type of domains that
needs to be created. The startup scripts, config.xml,
workshop config require settings that are slightly
different for a production domain than for a
development domain. It would be nice to provide a set
of templates for different type of configurations
(Cluster template, vs single domain template, etc...)
>
> * support building for different target environments
> - support container-specific deployment descriptor
> for example
> - support container config files that depend on
> the environment. For
> example you may want to run on port 8080 in your dev
> env and on port 80 in
> your production env. You may want to run on a single
> server in the dev env
> but in cluster on the production env, etc.
> - support different deployment models per
> environment. For example in one
> environment you may want to have the web server and
> EJB server on the same
> machine and in another environment you may want them
> located on 2 different
> machines.
+1
>
> * support deploying config and J2EE modules to a
> container
>
> * support automated functional tests execution
>
> Directory structure
> ===================
>
> In term of directory structure, I'm proposing the
> following:
>
> <project>/
> |
> |_ modules/
> |_ jars/
> |_ <functional name for jar M>/
> |_ ejbs/
> |_ <functional name for ejb N>/
> |_ wars/
> |_ <functional name for war O>/
> |_ ears/
> |_ <functional name for ear P>/
> |_ clients/
> |_ <functional name for client Q>/
> |_ rars/
> |_ <functional name for rar R>/
> |
> |_ applications/
> |_ <functional name for executable 1 - ex:
> database>/
> |_ <functional name for executable 2 - ex:
> business>/
> |_ <functional name for executable 3 - ex:
> wsclient>/
> |_ system/
+1. As mentioned above, you may want to add a
directory structure for domain management.
Regarding the DB, do you have any thoughts on how to
manage DB scripts releases. For a development system,
the database can often be recreated, therefore a list
of baseline scripts is required as well as a set of
data seeding scripts.
For a production system, the approach is different,
where a baseline is required when going live the first
time and then the scripts are required to be
incremental with in some instance point in time data
migration scripts.
This is quite a broad topic of discussion, but an
important one as in my experience database script
management is often a pain point. Applications usally
do not contain checks providing them with the ability
to tie the version of the database to the version of
the code base.
>
> Note that for a simple project the structure can be
> simplified to:
>
> <project>/
> |
> |_ modules/
> |_ <functional name for jar M>/
> |_ <functional name for ejb N>/
> |_ <functional name for war O>/
> |_ <functional name for ear P>/
> |_ <functional name for client Q>/
> |_ <functional name for rar R>/
> |
> |_ [...]
>
> Some explanations:
>
> * The applications/ directory contains subprojects
> (aka modules in m2 speak)
> that contain executable applications, i.e. things
> that will make up the full
> system. Applications are made of 2 things:
> - modules
> - configuration files including scripts (e.g.
> start/stop scripts, etc)
Agreed see comment above on that one.
>
> * The database/ application represents the database.
> This is where schema
> and data could be located. This is also where
> database script files can be
> stored for starting/stopping/analyzing
> logs/profiling/etc.
>
> * The business/ application would be the business
> logic executing in the
> J2EE container. It would package the J2EE modules
> (EARs and possibly others)
> and the container configuration files.
>
> * The wsclient/ application is just an example
> showing a client application
> (ws stands for web services here). In this example
> this would be a sample
> swing application.
>
> * All the application projects above will generate a
> distribution (zipped
> file, tar.gz, etc) per environment containing a
> directory structure with
> scripts files in (say) bin/. Basically they should
> be self-running with only
> pre-requisite software required to be installed on
> the target machines (OS,
> JVM, container install, etc).
+1, I would add database version check as well.
>
> * The system/ project represents the full system. It
> is made of
> applications. It'll also deliver a distribution
> (zipped file, tar.gz, etc)
> which will aggregate all the distributions from the
> other applications. This
> represents the full system for a target environment.
> Other considerations
> ====================
>
> * Environment will be managed using profiles.
> * Environment specific resources can go in:
> src/main/resources-{env name}
> * Common resources can go in src/main/resources and
> use filters (one filter
> file per environment for example).
> * Maven-runtime would be nice to generate scripts
> * functional/system tests would be run in system/
> (or application/)
> * It would be nice to be able to automate the
> installation of the
> distributions generated by the different application
> projects. This means
> the ability to install locally but also remotely.
> Alternatively this would
> mean building with m2 on all target machines to
> always install locally,
> which is also possible using M2 in embedded mode for
> example and the repo as
> a way to distribute the binary files - Actually this
> sounds cool :-). But
> this still leaves the issue of triggering the
> executions of the embedded m2
> remotely. This requires some new plugin with some
> xml file to configure IPs
> and access to all the machines where to install the
> system.
> * Cargo can probably be used somewhere (to run
> functional tests, to create
> container configurationsn, to deploy to running
> containers, to start/stop
> containers, etc). I still need to figure out the
> usage of cargo vs shell
> scripts.
Cargo looks interesting, I would also look like at a
WLST like approach that provides a Jython wrapper
around JMX.
What are the next steps in your mind?
Regards,
David Le Strat
________________________
David Le Strat
Blogging @ http://dlsthoughts.blogspot.com
__________________________________
Yahoo! Mail - PC Magazine Editors' Choice 2005
http://mail.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org