You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by Pierre-Arnaud Marcelot <pa...@marcelot.net> on 2010/06/10 19:02:51 UTC
[Studio] Improvement proposal for Studio's Maven build
Hi Dev,
I've been working on some improvements to Studio's Maven build these last days in this branch [1].
I tried to improve the way we generate distributions to reduce the amount of tasks (and time) still done by hand when releasing a new version of Studio.
I'm tired of wasting almost a full day when a new version needs to be released...
Basically, I tried to integrate in the build (with 'release' profile enabled) the generation of the Windows Installer Exe, the Mac OS X DMG, as well as the generation of a folder containing all our distributions (linux-x86 and linux-x86_64 tar.gz archives, Mac OS X DMG, Windows Installer Exe) and update site (as an archive to be deployed to the downloads area, and as a folder for the 'online' update site), all these being signed (ASC, MD5 and SHA1) automatically.
The complete build with 'release' profile enabled takes a little more time (around 8-9 minutes), but in the end, we get a folder, which is ready to be uploaded on people.apache.org for deployment.
A few facts on this new build:
- There is now a separated Maven module (packaged as a pom) for each "distribution" build (eg: Mac OS X), which generates a zip file stored into the local repo and accessible as a dependency to other modules.
- Same thing for:
- Apache Directory Studio features
- Apache Directory Studio plugins
- Apache Directory Studio Eclipse dependencies
- The 'updatesite' module now generates two zip files (one which can be used to install Studio in Eclipse, and another which is meant to be used on people.apache.org.
- The new build takes a few more seconds than the current one on daily basic tasks but not that much considering the added benefit.
(On my machine, a complete build takes 3:12 min with the current build and 3:32 with the new one. A 'fastbuild' takes 1:59 min with the current build and 1:59 with the new one).
BTW, in order to gain a few "precious" seconds, I'd also recommend that we exclude the "features" modules from the 'fastbuild' profile.
I'd like to know what you think about it, if you could test it, and if we can considering switching to this new build on trunk.
Thanks,
Pierre-Arnaud
[1] - https://svn.apache.org/repos/asf/directory/studio/branches/studio-improved-build/
Re: [Studio] Improvement proposal for Studio's Maven build
Posted by Felix Knecht <fe...@apache.org>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 06/10/10 22:31, Pierre-Arnaud Marcelot wrote:
> Ooops...
>
> Sorry Felix.
>
> A wrong dependency was set (I probably had this dependency in maven repository before, because when I dropped the repo, the error appeared).
>
> You should be able to pass this step, now.
Works fine now
Thanks
Felix
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.15 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAkwRUGMACgkQ2lZVCB08qHHtYACgpx1vS/NbLY8EaLtxNiPSANJs
J5EAn02sGwZN9JFL9Qmir1WwpCqKZ5sq
=b+ja
-----END PGP SIGNATURE-----
Re: [Studio] Improvement proposal for Studio's Maven build
Posted by Pierre-Arnaud Marcelot <pa...@marcelot.net>.
Ooops...
Sorry Felix.
A wrong dependency was set (I probably had this dependency in maven repository before, because when I dropped the repo, the error appeared).
You should be able to pass this step, now.
FYI, using '-Prelease' on a non Mac OS X machine fails at the moment, because the build of the module 'Mac OS X DMG' is dependent on a command line utility that only exists on Mac.
I need to find a way to make this pass on any platform. Probably by creating a "fake" package on un-supported systems.
It will the same thing with the 'Windows Installer Exe' module, if you don't have the 'makensis' command-line utility installed.
I will try to find and give you the link to a webpage which describes the way to compile and install this utility.
Regards,
Pierre-Arnaud
On 10 juin 2010, at 20:38, Felix Knecht wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 06/10/10 20:35, Felix Knecht wrote:
>>> I tried to improve the way we generate distributions to reduce the amount of tasks (and time) still done by hand when releasing a new version of Studio.
>>> I'm tired of wasting almost a full day when a new version needs to be released...
>>
>> Good job.
>>
>>
>>> I'd like to know what you think about it, if you could test it, and if we can considering switching to this new build on trunk.
>>
>> felix@asgard
>> ~/svn/apache/directory/studio/branches/studio-improved-build $ mvn clean
>> install -Prelease
>> [INFO] Scanning for projects...
>> [INFO] Reactor build order:
>>
>> ...
>>
>> [ERROR] BUILD ERROR
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] One or more required plugin parameters are invalid/missing for
>> 'jar:sign'
>>
>> [0] Inside the definition for plugin 'maven-jar-plugin' specify the
>> following:
>>
>> <configuration>
>> ...
>> <alias>VALUE</alias>
>> </configuration>
>>
>> -OR-
>>
>> on the command line, specify: '-Djarsign.alias=VALUE'
>>
>>
>
>
> mvn clean install (linux 64bit) -->
> [ERROR] BUILD ERROR
> [INFO]
> - ------------------------------------------------------------------------
> [INFO] Failed to resolve dependencies for one or more projects in the
> reactor. Reason: Missing:
> - ----------
> 1) org.apache.directory.studio:application-linux-x86:tar.gz:1.5.4-SNAPSHOT
>
> Try downloading the file manually from the project website.
>
> Then, install it using the command:
> mvn install:install-file -DgroupId=org.apache.directory.studio
> - -DartifactId=application-linux-x86 -Dversion=1.5.4-SNAPSHOT
> - -Dpackaging=tar.gz -Dfile=/path/to/file
>
> Alternatively, if you host your own repository you can deploy the file
> there:
> mvn deploy:deploy-file -DgroupId=org.apache.directory.studio
> - -DartifactId=application-linux-x86 -Dversion=1.5.4-SNAPSHOT
> - -Dpackaging=tar.gz -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
>
> Path to dependency:
> 1)
> org.apache.directory.studio:ApacheDirectoryStudio-linux-x86_64:pom:1.5.4-SNAPSHOT
> 2)
> org.apache.directory.studio:application-linux-x86:tar.gz:1.5.4-SNAPSHOT
>
> - ----------
> 1 required artifact is missing.
>
> for artifact:
>
> org.apache.directory.studio:ApacheDirectoryStudio-linux-x86_64:pom:1.5.4-SNAPSHOT
>
> from the specified remote repositories:
> apache.snapshots (http://repository.apache.org/snapshots),
> central (http://repo1.maven.org/maven2)
>
>> Any ideas?
>>
>> Regards
>> Felix
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.15 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkwRMSMACgkQ2lZVCB08qHET1ACfZivhZSQO3mx4LI+1PpKCSEml
> aQUAmgJuM+1eFXxcxPzRpVoHNiR04r5l
> =UUfP
> -----END PGP SIGNATURE-----
Re: [Studio] Improvement proposal for Studio's Maven build
Posted by Felix Knecht <fe...@apache.org>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 06/10/10 20:35, Felix Knecht wrote:
>> I tried to improve the way we generate distributions to reduce the amount of tasks (and time) still done by hand when releasing a new version of Studio.
>> I'm tired of wasting almost a full day when a new version needs to be released...
>
> Good job.
>
>
>> I'd like to know what you think about it, if you could test it, and if we can considering switching to this new build on trunk.
>
> felix@asgard
> ~/svn/apache/directory/studio/branches/studio-improved-build $ mvn clean
> install -Prelease
> [INFO] Scanning for projects...
> [INFO] Reactor build order:
>
> ...
>
> [ERROR] BUILD ERROR
> [INFO]
> ------------------------------------------------------------------------
> [INFO] One or more required plugin parameters are invalid/missing for
> 'jar:sign'
>
> [0] Inside the definition for plugin 'maven-jar-plugin' specify the
> following:
>
> <configuration>
> ...
> <alias>VALUE</alias>
> </configuration>
>
> -OR-
>
> on the command line, specify: '-Djarsign.alias=VALUE'
>
>
mvn clean install (linux 64bit) -->
[ERROR] BUILD ERROR
[INFO]
- ------------------------------------------------------------------------
[INFO] Failed to resolve dependencies for one or more projects in the
reactor. Reason: Missing:
- ----------
1) org.apache.directory.studio:application-linux-x86:tar.gz:1.5.4-SNAPSHOT
Try downloading the file manually from the project website.
Then, install it using the command:
mvn install:install-file -DgroupId=org.apache.directory.studio
- -DartifactId=application-linux-x86 -Dversion=1.5.4-SNAPSHOT
- -Dpackaging=tar.gz -Dfile=/path/to/file
Alternatively, if you host your own repository you can deploy the file
there:
mvn deploy:deploy-file -DgroupId=org.apache.directory.studio
- -DartifactId=application-linux-x86 -Dversion=1.5.4-SNAPSHOT
- -Dpackaging=tar.gz -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
Path to dependency:
1)
org.apache.directory.studio:ApacheDirectoryStudio-linux-x86_64:pom:1.5.4-SNAPSHOT
2)
org.apache.directory.studio:application-linux-x86:tar.gz:1.5.4-SNAPSHOT
- ----------
1 required artifact is missing.
for artifact:
org.apache.directory.studio:ApacheDirectoryStudio-linux-x86_64:pom:1.5.4-SNAPSHOT
from the specified remote repositories:
apache.snapshots (http://repository.apache.org/snapshots),
central (http://repo1.maven.org/maven2)
> Any ideas?
>
> Regards
> Felix
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.15 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAkwRMSMACgkQ2lZVCB08qHET1ACfZivhZSQO3mx4LI+1PpKCSEml
aQUAmgJuM+1eFXxcxPzRpVoHNiR04r5l
=UUfP
-----END PGP SIGNATURE-----
Re: [Studio] Improvement proposal for Studio's Maven build
Posted by Felix Knecht <fe...@apache.org>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
> I tried to improve the way we generate distributions to reduce the amount of tasks (and time) still done by hand when releasing a new version of Studio.
> I'm tired of wasting almost a full day when a new version needs to be released...
Good job.
> I'd like to know what you think about it, if you could test it, and if we can considering switching to this new build on trunk.
felix@asgard
~/svn/apache/directory/studio/branches/studio-improved-build $ mvn clean
install -Prelease
[INFO] Scanning for projects...
[INFO] Reactor build order:
...
[ERROR] BUILD ERROR
[INFO]
- ------------------------------------------------------------------------
[INFO] One or more required plugin parameters are invalid/missing for
'jar:sign'
[0] Inside the definition for plugin 'maven-jar-plugin' specify the
following:
<configuration>
...
<alias>VALUE</alias>
</configuration>
- -OR-
on the command line, specify: '-Djarsign.alias=VALUE'
Any ideas?
Regards
Felix
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.15 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAkwRMFYACgkQ2lZVCB08qHEihACghkxwe0kCEdUmHUIZ7Kh0gPtk
J3wAoKR0oI8oikVw2mOI853JDWkIW4BE
=iCdu
-----END PGP SIGNATURE-----
Re: [Studio] Improvement proposal for Studio's Maven build
Posted by Pierre-Arnaud Marcelot <pa...@marcelot.net>.
Done.
I removed the 'application-unpack' module and excluded the help plugins and features modules from the 'normal' build.
They are now available as a 'studio-full' profile.
On my machine, the difference is very interesting: 3 min 10 sec using the 'studio-full' profile against 1 min 51 sec with a normal build.
Regards,
Pierre-Arnaud
On 11 juin 2010, at 19:23, Pierre-Arnaud Marcelot wrote:
> Hi Stefan,
>
> On 11 juin 2010, at 12:26, Stefan Seelmann wrote:
>
>> Pierre-Arnaud Marcelot wrote:
>>> Hi Dev,
>>>
>>> I've been working on some improvements to Studio's Maven build these last days in this branch [1].
>>>
>>> I tried to improve the way we generate distributions to reduce the amount of tasks (and time) still done by hand when releasing a new version of Studio.
>>> I'm tired of wasting almost a full day when a new version needs to be released...
>>>
>>> Basically, I tried to integrate in the build (with 'release' profile enabled) the generation of the Windows Installer Exe, the Mac OS X DMG, as well as the generation of a folder containing all our distributions (linux-x86 and linux-x86_64 tar.gz archives, Mac OS X DMG, Windows Installer Exe) and update site (as an archive to be deployed to the downloads area, and as a folder for the 'online' update site), all these being signed (ASC, MD5 and SHA1) automatically.
>>>
>>> The complete build with 'release' profile enabled takes a little more time (around 8-9 minutes), but in the end, we get a folder, which is ready to be uploaded on people.apache.org for deployment.
>>>
>>> A few facts on this new build:
>>> - There is now a separated Maven module (packaged as a pom) for each "distribution" build (eg: Mac OS X), which generates a zip file stored into the local repo and accessible as a dependency to other modules.
>>> - Same thing for:
>>> - Apache Directory Studio features
>>> - Apache Directory Studio plugins
>>> - Apache Directory Studio Eclipse dependencies
>>
>> Do we still need to create the parent's target/distribution if
>> everything is build in application module?
>
> Nope, it is just copied to be more accessible, without a long command-line like :
> $ open application/application-macosx/target/ApacheDirectoryStudio-macosx/Apache Directory Studio.app
>
> But you're right, it is not absolutely necessary.
> Then I believe the 'application-unpack module' can be removed and we can gain a few (precious) seconds while building.
>
>
>>> - The 'updatesite' module now generates two zip files (one which can be used to install Studio in Eclipse, and another which is meant to be used on people.apache.org.
>>
>> Same here, do we still need the top-level updatesite project?
>
> Actually I was talking about the 'application-updatesite' module, the one at the top-level directory can be removed.
> BTW, I forgot to mention that the updatesite module is not built by default, but only with the 'updatesite' profile.
>
>
>>> - The new build takes a few more seconds than the current one on daily basic tasks but not that much considering the added benefit.
>>> (On my machine, a complete build takes 3:12 min with the current build and 3:32 with the new one. A 'fastbuild' takes 1:59 min with the current build and 1:59 with the new one).
>>> BTW, in order to gain a few "precious" seconds, I'd also recommend that we exclude the "features" modules from the 'fastbuild' profile.
>>
>> +1
>>
>> Is it possible to make the "fastbuild" the default?
>
> Sure, this is something I was thinking about too.
>
> I will modify the build on the branch in order to only build the 'code' plugins:
> - aciitemeditor
> - apacheds
> - apacheds-configuration
> - common-ui
> - connection-core
> - connection-ui
> - jars
> - ldapbrowser-common
> - ldapbrowser-core
> - ldapbrowser-ui
> - ldif-parser
> - ldifeditor
> - rcp
> - schemaeditor
> - valueeditors
>
> Am I missing something in the list ?
>
>
>> Today, I import all plugins and features into eclipse but close the help
>> plugin and feature projects afterwards.
>
> I do exactly the same as we modify these projects very rarely.
>
>
>> If by default build would only contain the the real plugin it would be
>> faster. As a side effect a studio:eclipse only creates eclipse files for
>> the plugins but not for the features and help modules, then the eclipse
>> import wizard only imports the real projects and we avoid that eclipse
>> shows build errors.
>>
>> Additionally we would need a profile "full" that builds the features and
>> help plugins.
>
> Sounds good to me.
> I'll make the modifications in the branch.
>
> As a summary, here's the list of all the profiles that we'll be able to use:
> - 'linux-x86' -> Forces the generation of the Linux x86 application (even if the machine is not running Linux x86)
> - 'linux-x86_64' -> Forces the generation of the Linux x86_64 application (even if the machine is not running Linux x86_64)
> - 'macosx' -> Forces the generation of the Mac OS X application (even if the machine is not running Mac OS X)
> - 'win32' -> Forces the generation of the Windows application (even if the machine is not running Windows)
> - 'update' -> Adds the update site to the build
> - 'full' -> Adds features and help plugins to the build
> - 'release' -> Adds the generation of the Mac OS X DMG, Windows Installer Exe and folder containing signed distributions and updatesite
>
>
>>
>>> I'd like to know what you think about it, if you could test it, and if we can considering switching to this new build on trunk.
>>
>> I think it looks very good. And I trust your opinion, you did all the
>> releases you know the pain.
>
> Héhé. Not that painful, but time consuming.
>
>
> Thanks,
> Pierre-Arnaud
Re: [Studio] Improvement proposal for Studio's Maven build
Posted by Pierre-Arnaud Marcelot <pa...@marcelot.net>.
Hi Stefan,
On 11 juin 2010, at 12:26, Stefan Seelmann wrote:
> Pierre-Arnaud Marcelot wrote:
>> Hi Dev,
>>
>> I've been working on some improvements to Studio's Maven build these last days in this branch [1].
>>
>> I tried to improve the way we generate distributions to reduce the amount of tasks (and time) still done by hand when releasing a new version of Studio.
>> I'm tired of wasting almost a full day when a new version needs to be released...
>>
>> Basically, I tried to integrate in the build (with 'release' profile enabled) the generation of the Windows Installer Exe, the Mac OS X DMG, as well as the generation of a folder containing all our distributions (linux-x86 and linux-x86_64 tar.gz archives, Mac OS X DMG, Windows Installer Exe) and update site (as an archive to be deployed to the downloads area, and as a folder for the 'online' update site), all these being signed (ASC, MD5 and SHA1) automatically.
>>
>> The complete build with 'release' profile enabled takes a little more time (around 8-9 minutes), but in the end, we get a folder, which is ready to be uploaded on people.apache.org for deployment.
>>
>> A few facts on this new build:
>> - There is now a separated Maven module (packaged as a pom) for each "distribution" build (eg: Mac OS X), which generates a zip file stored into the local repo and accessible as a dependency to other modules.
>> - Same thing for:
>> - Apache Directory Studio features
>> - Apache Directory Studio plugins
>> - Apache Directory Studio Eclipse dependencies
>
> Do we still need to create the parent's target/distribution if
> everything is build in application module?
Nope, it is just copied to be more accessible, without a long command-line like :
$ open application/application-macosx/target/ApacheDirectoryStudio-macosx/Apache Directory Studio.app
But you're right, it is not absolutely necessary.
Then I believe the 'application-unpack module' can be removed and we can gain a few (precious) seconds while building.
>> - The 'updatesite' module now generates two zip files (one which can be used to install Studio in Eclipse, and another which is meant to be used on people.apache.org.
>
> Same here, do we still need the top-level updatesite project?
Actually I was talking about the 'application-updatesite' module, the one at the top-level directory can be removed.
BTW, I forgot to mention that the updatesite module is not built by default, but only with the 'updatesite' profile.
>> - The new build takes a few more seconds than the current one on daily basic tasks but not that much considering the added benefit.
>> (On my machine, a complete build takes 3:12 min with the current build and 3:32 with the new one. A 'fastbuild' takes 1:59 min with the current build and 1:59 with the new one).
>> BTW, in order to gain a few "precious" seconds, I'd also recommend that we exclude the "features" modules from the 'fastbuild' profile.
>
> +1
>
> Is it possible to make the "fastbuild" the default?
Sure, this is something I was thinking about too.
I will modify the build on the branch in order to only build the 'code' plugins:
- aciitemeditor
- apacheds
- apacheds-configuration
- common-ui
- connection-core
- connection-ui
- jars
- ldapbrowser-common
- ldapbrowser-core
- ldapbrowser-ui
- ldif-parser
- ldifeditor
- rcp
- schemaeditor
- valueeditors
Am I missing something in the list ?
> Today, I import all plugins and features into eclipse but close the help
> plugin and feature projects afterwards.
I do exactly the same as we modify these projects very rarely.
> If by default build would only contain the the real plugin it would be
> faster. As a side effect a studio:eclipse only creates eclipse files for
> the plugins but not for the features and help modules, then the eclipse
> import wizard only imports the real projects and we avoid that eclipse
> shows build errors.
>
> Additionally we would need a profile "full" that builds the features and
> help plugins.
Sounds good to me.
I'll make the modifications in the branch.
As a summary, here's the list of all the profiles that we'll be able to use:
- 'linux-x86' -> Forces the generation of the Linux x86 application (even if the machine is not running Linux x86)
- 'linux-x86_64' -> Forces the generation of the Linux x86_64 application (even if the machine is not running Linux x86_64)
- 'macosx' -> Forces the generation of the Mac OS X application (even if the machine is not running Mac OS X)
- 'win32' -> Forces the generation of the Windows application (even if the machine is not running Windows)
- 'update' -> Adds the update site to the build
- 'full' -> Adds features and help plugins to the build
- 'release' -> Adds the generation of the Mac OS X DMG, Windows Installer Exe and folder containing signed distributions and updatesite
>
>> I'd like to know what you think about it, if you could test it, and if we can considering switching to this new build on trunk.
>
> I think it looks very good. And I trust your opinion, you did all the
> releases you know the pain.
Héhé. Not that painful, but time consuming.
Thanks,
Pierre-Arnaud
Re: [Studio] Improvement proposal for Studio's Maven build
Posted by Stefan Seelmann <se...@apache.org>.
Pierre-Arnaud Marcelot wrote:
> Hi Dev,
>
> I've been working on some improvements to Studio's Maven build these last days in this branch [1].
>
> I tried to improve the way we generate distributions to reduce the amount of tasks (and time) still done by hand when releasing a new version of Studio.
> I'm tired of wasting almost a full day when a new version needs to be released...
>
> Basically, I tried to integrate in the build (with 'release' profile enabled) the generation of the Windows Installer Exe, the Mac OS X DMG, as well as the generation of a folder containing all our distributions (linux-x86 and linux-x86_64 tar.gz archives, Mac OS X DMG, Windows Installer Exe) and update site (as an archive to be deployed to the downloads area, and as a folder for the 'online' update site), all these being signed (ASC, MD5 and SHA1) automatically.
>
> The complete build with 'release' profile enabled takes a little more time (around 8-9 minutes), but in the end, we get a folder, which is ready to be uploaded on people.apache.org for deployment.
>
> A few facts on this new build:
> - There is now a separated Maven module (packaged as a pom) for each "distribution" build (eg: Mac OS X), which generates a zip file stored into the local repo and accessible as a dependency to other modules.
> - Same thing for:
> - Apache Directory Studio features
> - Apache Directory Studio plugins
> - Apache Directory Studio Eclipse dependencies
Do we still need to create the parent's target/distribution if
everything is build in application module?
> - The 'updatesite' module now generates two zip files (one which can be used to install Studio in Eclipse, and another which is meant to be used on people.apache.org.
Same here, do we still need the top-level updatesite project?
> - The new build takes a few more seconds than the current one on daily basic tasks but not that much considering the added benefit.
> (On my machine, a complete build takes 3:12 min with the current build and 3:32 with the new one. A 'fastbuild' takes 1:59 min with the current build and 1:59 with the new one).
> BTW, in order to gain a few "precious" seconds, I'd also recommend that we exclude the "features" modules from the 'fastbuild' profile.
+1
Is it possible to make the "fastbuild" the default?
Today, I import all plugins and features into eclipse but close the help
plugin and feature projects afterwards.
If by default build would only contain the the real plugin it would be
faster. As a side effect a studio:eclipse only creates eclipse files for
the plugins but not for the features and help modules, then the eclipse
import wizard only imports the real projects and we avoid that eclipse
shows build errors.
Additionally we would need a profile "full" that builds the features and
help plugins.
> I'd like to know what you think about it, if you could test it, and if we can considering switching to this new build on trunk.
I think it looks very good. And I trust your opinion, you did all the
releases you know the pain.
Thanks,
Stefan