You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Alex Huang <Al...@citrix.com> on 2013/07/27 00:09:34 UTC
[DISCUSS] maven dependencies...
Everyone,
After looking around in the maven documentation, I realized the way we are specifying dependencies is not quite right for a large project such as ours.
Currently, almost every project declare their own dependencies and version number of the dependent jar. For those of us who are conscious of the version number properties declared in the cloudstack pom file, we follow that example but in many places, the version numbers are actually hard coded, probably because the writer is not aware of this.
Maven actually has a way to do this. In the master pom file, we can declare in the <dependencyManagment> tags all of the third party dependencies we need and their version numbers. And then each individual module can declare their dependency without version number, which defaults to the version declared by CloudStack's master pom. If a version number is declared in the module's pom, it overrides the master's version number but there's a warning about this override.
Sounds good? If so I'll do a quick change to move it over.
--Alex
Re: [DISCUSS] maven dependencies...
Posted by Daan Hoogland <da...@gmail.com>.
ship it;)
On Sat, Jul 27, 2013 at 12:09 AM, Alex Huang <Al...@citrix.com> wrote:
> Everyone,
>
> After looking around in the maven documentation, I realized the way we are
> specifying dependencies is not quite right for a large project such as ours.
>
> Currently, almost every project declare their own dependencies and version
> number of the dependent jar. For those of us who are conscious of the
> version number properties declared in the cloudstack pom file, we follow
> that example but in many places, the version numbers are actually hard
> coded, probably because the writer is not aware of this.
>
> Maven actually has a way to do this. In the master pom file, we can
> declare in the <dependencyManagment> tags all of the third party
> dependencies we need and their version numbers. And then each individual
> module can declare their dependency without version number, which defaults
> to the version declared by CloudStack's master pom. If a version number is
> declared in the module's pom, it overrides the master's version number but
> there's a warning about this override.
>
> Sounds good? If so I'll do a quick change to move it over.
>
> --Alex
>
RE: [DISCUSS] maven dependencies...
Posted by Donal Lafferty <do...@citrix.com>.
FYI, org.apache.commons.io.FileUtils was missing due to an incorrect dependency spec.
For details, see http://dlafferty.blogspot.co.uk/2013/08/diagnosing-maven-dependency-problems.html
The problem has since been resolved.
DL
> -----Original Message-----
> From: Donal Lafferty [mailto:donal.lafferty@citrix.com]
> Sent: 30 July 2013 5:19 PM
> To: <de...@cloudstack.apache.org>
> Subject: RE: [DISCUSS] maven dependencies...
>
> BTW, how is the dependency on org.apache.commons.io.FileUtils
> expressed? Has this code changed recently?
>
> Using this morning's master, the management server can't find the FileUtils
> class. I.e.
>
> INFO [ConfigurationServerImpl] (Timer-2:null) Generated SSL keystore.
> Exception in thread "Timer-2" java.lang.NoClassDefFoundError:
> org/apache/commons/io/FileUtils
> at
> com.cloud.server.ConfigurationServerImpl.getBase64Keystore(Configuration
> ServerImpl.java:453)
> at
> com.cloud.server.ConfigurationServerImpl.updateSSLKeystore(Configuratio
> nServerImpl.java:520)
> at
> com.cloud.server.ConfigurationServerImpl.persistDefaultValues(Configurati
> onServerImpl.java:289)
> at
> com.cloud.server.ConfigurationServerImpl_EnhancerByCloudStack_4741108
> b.CGLIB$persistDefaultValues$14(<generated>)
> at
> com.cloud.server.ConfigurationServerImpl_EnhancerByCloudStack_4741108
> b_FastClassByCloudStack_d48cf40f.invoke(<generated>)
> at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
> at
> com.cloud.utils.component.ComponentInstantiationPostProcessor$Intercep
> torDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at
> com.cloud.server.ConfigurationServerImpl_EnhancerByCloudStack_4741108
> b.persistDefaultValues(<generated>)
> at
> com.cloud.server.ConfigurationServerImpl.configure(ConfigurationServerIm
> pl.java:157)
> at
> com.cloud.utils.component.ComponentContext.initComponentsLifeCycle(Co
> mponentContext.java:111)
> at
> com.cloud.servlet.CloudStartupServlet$1.run(CloudStartupServlet.java:50)
> at java.util.TimerThread.mainLoop(Timer.java:555)
> at java.util.TimerThread.run(Timer.java:505)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.commons.io.FileUtils
> at
> org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirs
> tStrategy.java:50)
> at
> org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.jav
> a:244)
> at
> org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.jav
> a:230)
> at
> org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoa
> der.java:401)
> at
> org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoa
> der.java:363)
> ... 13 more
>
> > -----Original Message-----
> > From: Chip Childers [mailto:chip.childers@sungard.com]
> > Sent: 30 July 2013 4:12 PM
> > To: <de...@cloudstack.apache.org>
> > Subject: Re: [DISCUSS] maven dependencies...
> >
> > On Tue, Jul 30, 2013 at 10:22 AM, Alex Huang <Al...@citrix.com>
> > wrote:
> >
> > > Chip and Donal,
> > >
> > > I should have replied to this earlier. I changed from a logging
> > > category of [cloud.vm.VirtualMachineManagerImpl] to
> > > [c.c.v.VirtualMachineManagerImpl]. It's shorter and really no one
> > > looks at the package any more with editors that can open any file
> > > from any package but every so often you need it and having the
> > > package abbreviation comes in handy. To do that I changed from
> > > %{c3} to %{c1.} but it required the use of EnhancedPatternLayout.
> > >
> > > What I failed to notice (because the startup log rolls by so fast)
> > > is that some of the appenders are not using EnhancedPatternLayout.
> > >
> > > I've already put in a fix after Donal's email to put the ones that
> > > are using PatternLayout to go back to the old %{c3}. They're
> > > specific to syslog and snmp so I didn't want to screw up anything in
> > > those implementations.
> > >
> > > I will revert Chip's commit that changed all the rest of them to
> > > %{c1} unless we rather not have the package abbreviations.
> > >
> > >
> > Good clarification. Revert away.
> >
> >
> > > --Alex
> > >
> > > > -----Original Message-----
> > > > From: Chip Childers [mailto:chip.childers@sungard.com]
> > > > Sent: Tuesday, July 30, 2013 6:05 AM
> > > > To: dev@cloudstack.apache.org
> > > > Subject: Re: [DISCUSS] maven dependencies...
> > > >
> > > > On Mon, Jul 29, 2013 at 08:40:44PM +0000, Donal Lafferty wrote:
> > > > > Could you also have a look at ./client/tomcatconf/log4j-cloud.xml.in?
> > > > >
> > > > > It looks like it was updated in the refactor.
> > > > >
> > > > > AFAIK, in '<param name="ConversionPattern" value="%-5p [%c{1.}]
> > > > (%t:%x) %m%n"/>', {1.} should be {1}.
> > > > >
> > > > > See "Conversion Character" chart under
> > > >
> > > http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/Pattern
> > > La
> > > you
> > > > t.html
> > > > >
> > > > > There's a lot of paper work for me to make the update :(
> > > > >
> > > > > DL
> > > >
> > > > Done!
> > > >
> > > > commit 4c729e7
> > >
> > >
RE: [DISCUSS] maven dependencies...
Posted by Donal Lafferty <do...@citrix.com>.
BTW, how is the dependency on org.apache.commons.io.FileUtils expressed? Has this code changed recently?
Using this morning's master, the management server can't find the FileUtils class. I.e.
INFO [ConfigurationServerImpl] (Timer-2:null) Generated SSL keystore.
Exception in thread "Timer-2" java.lang.NoClassDefFoundError: org/apache/commons/io/FileUtils
at com.cloud.server.ConfigurationServerImpl.getBase64Keystore(ConfigurationServerImpl.java:453)
at com.cloud.server.ConfigurationServerImpl.updateSSLKeystore(ConfigurationServerImpl.java:520)
at com.cloud.server.ConfigurationServerImpl.persistDefaultValues(ConfigurationServerImpl.java:289)
at com.cloud.server.ConfigurationServerImpl_EnhancerByCloudStack_4741108b.CGLIB$persistDefaultValues$14(<generated>)
at com.cloud.server.ConfigurationServerImpl_EnhancerByCloudStack_4741108b_FastClassByCloudStack_d48cf40f.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at com.cloud.server.ConfigurationServerImpl_EnhancerByCloudStack_4741108b.persistDefaultValues(<generated>)
at com.cloud.server.ConfigurationServerImpl.configure(ConfigurationServerImpl.java:157)
at com.cloud.utils.component.ComponentContext.initComponentsLifeCycle(ComponentContext.java:111)
at com.cloud.servlet.CloudStartupServlet$1.run(CloudStartupServlet.java:50)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.io.FileUtils
at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:244)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:230)
at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:401)
at org.mortbay.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:363)
... 13 more
> -----Original Message-----
> From: Chip Childers [mailto:chip.childers@sungard.com]
> Sent: 30 July 2013 4:12 PM
> To: <de...@cloudstack.apache.org>
> Subject: Re: [DISCUSS] maven dependencies...
>
> On Tue, Jul 30, 2013 at 10:22 AM, Alex Huang <Al...@citrix.com>
> wrote:
>
> > Chip and Donal,
> >
> > I should have replied to this earlier. I changed from a logging
> > category of [cloud.vm.VirtualMachineManagerImpl] to
> > [c.c.v.VirtualMachineManagerImpl]. It's shorter and really no one
> > looks at the package any more with editors that can open any file from
> > any package but every so often you need it and having the package
> > abbreviation comes in handy. To do that I changed from %{c3} to
> > %{c1.} but it required the use of EnhancedPatternLayout.
> >
> > What I failed to notice (because the startup log rolls by so fast) is
> > that some of the appenders are not using EnhancedPatternLayout.
> >
> > I've already put in a fix after Donal's email to put the ones that are
> > using PatternLayout to go back to the old %{c3}. They're specific to
> > syslog and snmp so I didn't want to screw up anything in those
> > implementations.
> >
> > I will revert Chip's commit that changed all the rest of them to %{c1}
> > unless we rather not have the package abbreviations.
> >
> >
> Good clarification. Revert away.
>
>
> > --Alex
> >
> > > -----Original Message-----
> > > From: Chip Childers [mailto:chip.childers@sungard.com]
> > > Sent: Tuesday, July 30, 2013 6:05 AM
> > > To: dev@cloudstack.apache.org
> > > Subject: Re: [DISCUSS] maven dependencies...
> > >
> > > On Mon, Jul 29, 2013 at 08:40:44PM +0000, Donal Lafferty wrote:
> > > > Could you also have a look at ./client/tomcatconf/log4j-cloud.xml.in?
> > > >
> > > > It looks like it was updated in the refactor.
> > > >
> > > > AFAIK, in '<param name="ConversionPattern" value="%-5p [%c{1.}]
> > > (%t:%x) %m%n"/>', {1.} should be {1}.
> > > >
> > > > See "Conversion Character" chart under
> > >
> > http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/PatternLa
> > you
> > > t.html
> > > >
> > > > There's a lot of paper work for me to make the update :(
> > > >
> > > > DL
> > >
> > > Done!
> > >
> > > commit 4c729e7
> >
> >
Re: [DISCUSS] maven dependencies...
Posted by Chip Childers <ch...@sungard.com>.
On Tue, Jul 30, 2013 at 10:22 AM, Alex Huang <Al...@citrix.com> wrote:
> Chip and Donal,
>
> I should have replied to this earlier. I changed from a logging category
> of [cloud.vm.VirtualMachineManagerImpl] to
> [c.c.v.VirtualMachineManagerImpl]. It's shorter and really no one looks at
> the package any more with editors that can open any file from any package
> but every so often you need it and having the package abbreviation comes in
> handy. To do that I changed from %{c3} to %{c1.} but it required the use
> of EnhancedPatternLayout.
>
> What I failed to notice (because the startup log rolls by so fast) is that
> some of the appenders are not using EnhancedPatternLayout.
>
> I've already put in a fix after Donal's email to put the ones that are
> using PatternLayout to go back to the old %{c3}. They're specific to
> syslog and snmp so I didn't want to screw up anything in those
> implementations.
>
> I will revert Chip's commit that changed all the rest of them to %{c1}
> unless we rather not have the package abbreviations.
>
>
Good clarification. Revert away.
> --Alex
>
> > -----Original Message-----
> > From: Chip Childers [mailto:chip.childers@sungard.com]
> > Sent: Tuesday, July 30, 2013 6:05 AM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [DISCUSS] maven dependencies...
> >
> > On Mon, Jul 29, 2013 at 08:40:44PM +0000, Donal Lafferty wrote:
> > > Could you also have a look at ./client/tomcatconf/log4j-cloud.xml.in?
> > >
> > > It looks like it was updated in the refactor.
> > >
> > > AFAIK, in '<param name="ConversionPattern" value="%-5p [%c{1.}]
> > (%t:%x) %m%n"/>', {1.} should be {1}.
> > >
> > > See "Conversion Character" chart under
> >
> http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/PatternLayou
> > t.html
> > >
> > > There's a lot of paper work for me to make the update :(
> > >
> > > DL
> >
> > Done!
> >
> > commit 4c729e7
>
>
RE: [DISCUSS] maven dependencies...
Posted by Alex Huang <Al...@citrix.com>.
Chip and Donal,
I should have replied to this earlier. I changed from a logging category of [cloud.vm.VirtualMachineManagerImpl] to [c.c.v.VirtualMachineManagerImpl]. It's shorter and really no one looks at the package any more with editors that can open any file from any package but every so often you need it and having the package abbreviation comes in handy. To do that I changed from %{c3} to %{c1.} but it required the use of EnhancedPatternLayout.
What I failed to notice (because the startup log rolls by so fast) is that some of the appenders are not using EnhancedPatternLayout.
I've already put in a fix after Donal's email to put the ones that are using PatternLayout to go back to the old %{c3}. They're specific to syslog and snmp so I didn't want to screw up anything in those implementations.
I will revert Chip's commit that changed all the rest of them to %{c1} unless we rather not have the package abbreviations.
--Alex
> -----Original Message-----
> From: Chip Childers [mailto:chip.childers@sungard.com]
> Sent: Tuesday, July 30, 2013 6:05 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] maven dependencies...
>
> On Mon, Jul 29, 2013 at 08:40:44PM +0000, Donal Lafferty wrote:
> > Could you also have a look at ./client/tomcatconf/log4j-cloud.xml.in?
> >
> > It looks like it was updated in the refactor.
> >
> > AFAIK, in '<param name="ConversionPattern" value="%-5p [%c{1.}]
> (%t:%x) %m%n"/>', {1.} should be {1}.
> >
> > See "Conversion Character" chart under
> http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/PatternLayou
> t.html
> >
> > There's a lot of paper work for me to make the update :(
> >
> > DL
>
> Done!
>
> commit 4c729e7
Re: [DISCUSS] maven dependencies...
Posted by Chip Childers <ch...@sungard.com>.
On Mon, Jul 29, 2013 at 08:40:44PM +0000, Donal Lafferty wrote:
> Could you also have a look at ./client/tomcatconf/log4j-cloud.xml.in?
>
> It looks like it was updated in the refactor.
>
> AFAIK, in '<param name="ConversionPattern" value="%-5p [%c{1.}] (%t:%x) %m%n"/>', {1.} should be {1}.
>
> See "Conversion Character" chart under http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/PatternLayout.html
>
> There's a lot of paper work for me to make the update :(
>
> DL
Done!
commit 4c729e7
RE: [DISCUSS] maven dependencies...
Posted by Donal Lafferty <do...@citrix.com>.
Could you also have a look at ./client/tomcatconf/log4j-cloud.xml.in?
It looks like it was updated in the refactor.
AFAIK, in '<param name="ConversionPattern" value="%-5p [%c{1.}] (%t:%x) %m%n"/>', {1.} should be {1}.
See "Conversion Character" chart under http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/PatternLayout.html
There's a lot of paper work for me to make the update :(
DL
> -----Original Message-----
> From: Alex Huang [mailto:Alex.Huang@citrix.com]
> Sent: 29 July 2013 6:56 PM
> To: dev@cloudstack.apache.org
> Subject: RE: [DISCUSS] maven dependencies...
>
> Thanks for pointing that out. I think the latest fixed this.
>
> --Alex
>
> > -----Original Message-----
> > From: Donal Lafferty [mailto:donal.lafferty@citrix.com]
> > Sent: Monday, July 29, 2013 10:45 AM
> > To: dev@cloudstack.apache.org
> > Subject: RE: [DISCUSS] maven dependencies...
> >
> > Is it correct for dependencies to appear twice in the base pom.xml?
> >
> > E.g. mysql-connector-java
> >
> > > -----Original Message-----
> > > From: Alex Huang [mailto:Alex.Huang@citrix.com]
> > > Sent: 26 July 2013 11:10 PM
> > > To: dev@cloudstack.apache.org
> > > Subject: [DISCUSS] maven dependencies...
> > >
> > > Everyone,
> > >
> > > After looking around in the maven documentation, I realized the way
> > > we are specifying dependencies is not quite right for a large
> > > project such as
> > ours.
> > >
> > > Currently, almost every project declare their own dependencies and
> > > version number of the dependent jar. For those of us who are
> > > conscious of the version number properties declared in the
> > > cloudstack pom file, we follow that example but in many places, the
> > > version numbers are actually hard coded, probably because the writer
> > > is not aware
> > of this.
> > >
> > > Maven actually has a way to do this. In the master pom file, we can
> > > declare in the <dependencyManagment> tags all of the third party
> > > dependencies we need and their version numbers. And then each
> > > individual module can declare their dependency without version
> > > number, which defaults to the version declared by CloudStack's
> > > master pom. If a version number is declared in the module's pom, it
> > > overrides the master's version number but there's a warning about this
> override.
> > >
> > > Sounds good? If so I'll do a quick change to move it over.
> > >
> > > --Alex
RE: [DISCUSS] maven dependencies...
Posted by Alex Huang <Al...@citrix.com>.
Thanks for pointing that out. I think the latest fixed this.
--Alex
> -----Original Message-----
> From: Donal Lafferty [mailto:donal.lafferty@citrix.com]
> Sent: Monday, July 29, 2013 10:45 AM
> To: dev@cloudstack.apache.org
> Subject: RE: [DISCUSS] maven dependencies...
>
> Is it correct for dependencies to appear twice in the base pom.xml?
>
> E.g. mysql-connector-java
>
> > -----Original Message-----
> > From: Alex Huang [mailto:Alex.Huang@citrix.com]
> > Sent: 26 July 2013 11:10 PM
> > To: dev@cloudstack.apache.org
> > Subject: [DISCUSS] maven dependencies...
> >
> > Everyone,
> >
> > After looking around in the maven documentation, I realized the way we
> > are specifying dependencies is not quite right for a large project such as
> ours.
> >
> > Currently, almost every project declare their own dependencies and
> > version number of the dependent jar. For those of us who are
> > conscious of the version number properties declared in the cloudstack
> > pom file, we follow that example but in many places, the version
> > numbers are actually hard coded, probably because the writer is not aware
> of this.
> >
> > Maven actually has a way to do this. In the master pom file, we can
> > declare in the <dependencyManagment> tags all of the third party
> > dependencies we need and their version numbers. And then each
> > individual module can declare their dependency without version number,
> > which defaults to the version declared by CloudStack's master pom. If
> > a version number is declared in the module's pom, it overrides the
> > master's version number but there's a warning about this override.
> >
> > Sounds good? If so I'll do a quick change to move it over.
> >
> > --Alex
RE: [DISCUSS] maven dependencies...
Posted by Donal Lafferty <do...@citrix.com>.
Is it correct for dependencies to appear twice in the base pom.xml?
E.g. mysql-connector-java
> -----Original Message-----
> From: Alex Huang [mailto:Alex.Huang@citrix.com]
> Sent: 26 July 2013 11:10 PM
> To: dev@cloudstack.apache.org
> Subject: [DISCUSS] maven dependencies...
>
> Everyone,
>
> After looking around in the maven documentation, I realized the way we are
> specifying dependencies is not quite right for a large project such as ours.
>
> Currently, almost every project declare their own dependencies and version
> number of the dependent jar. For those of us who are conscious of the
> version number properties declared in the cloudstack pom file, we follow
> that example but in many places, the version numbers are actually hard
> coded, probably because the writer is not aware of this.
>
> Maven actually has a way to do this. In the master pom file, we can declare
> in the <dependencyManagment> tags all of the third party dependencies we
> need and their version numbers. And then each individual module can
> declare their dependency without version number, which defaults to the
> version declared by CloudStack's master pom. If a version number is
> declared in the module's pom, it overrides the master's version number but
> there's a warning about this override.
>
> Sounds good? If so I'll do a quick change to move it over.
>
> --Alex