You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stanbol.apache.org by Fabian Christ <ch...@googlemail.com> on 2011/07/18 11:19:36 UTC
Release and Launchers
Hi,
for doing a release and excluding artifacts we will need a "release"
launcher. At the moment we have the "stable" launcher and the "full"
launcher. Integration tests are executed using the "full" launcher.
What do you want me to do:
1) Use the "full" launcher as the "release" launcher meaning that
everything that is in this launcher will be in the release? So I will
remove artifacts that are not in the release from the "full" launcher.
2) Create a new "release" launcher and adjust the integration tests to
use the new "release" launcher instead of the "full" launcher?
3) Use the "stable" launcher for the release as in 2) ?
Best,
- Fabian
--
Fabian
Re: Release and Launchers
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Tue, Jul 19, 2011 at 3:54 PM, Rupert Westenthaler
<ru...@gmail.com> wrote:
> ...Is it possible to have modules with LGPL dependencies in the trunk
> after the first release?
>
> Is it possible to have modules with LGPL dependencies in the trunk
> after graduation?
>
> Is there some point where we have to delete such modules or move them
> to an own branch?...
Having code that has required LGPL dependencies in trunk is ok during
incubation, while the cleanup period is going on.
We should just not *release* anything that requires an LGPL dependency
(see http://www.apache.org/legal/resolved.html for details), but
having that in trunk is ok.
-Bertrand
Re: Release and Launchers
Posted by Rupert Westenthaler <ru...@gmail.com>.
Some Questions:
Is it possible to have modules with LGPL dependencies in the trunk
after the first release?
Is it possible to have modules with LGPL dependencies in the trunk
after graduation?
Is there some point where we have to delete such modules or move them
to an own branch?
best
Rupert
On Tue, Jul 19, 2011 at 3:44 PM, Fabian Christ
<ch...@googlemail.com> wrote:
> Hi,
>
> since the license issues are solved we can release just from the
> trunk. The release will contain everything that is in the default
> build. I think, this is the easiest way to do it now.
>
> - Fabian
>
> 2011/7/19 Bertrand Delacretaz <bd...@apache.org>:
>> On Mon, Jul 18, 2011 at 11:19 AM, Fabian Christ
>> <ch...@googlemail.com> wrote:
>>> ...for doing a release and excluding artifacts we will need a "release"
>>> launcher. At the moment we have the "stable" launcher and the "full"
>>> launcher. Integration tests are executed using the "full" launcher....
>>
>>> ...3) Use the "stable" launcher for the release as in 2) ?...
>>
>> +1 on including only what we release in the stable launcher, that
>> makes sense and you don't need to create yet another launcher.
>>
>> -Bertrand
>>
>
>
>
> --
> Fabian
>
--
| Rupert Westenthaler rupert.westenthaler@gmail.com
| Bodenlehenstraße 11 ++43-699-11108907
| A-5500 Bischofshofen
Re: Release and Launchers
Posted by Fabian Christ <ch...@googlemail.com>.
2011/7/19 Olivier Grisel <ol...@ensta.org>:
> 2011/7/19 Fabian Christ <ch...@googlemail.com>:
>> Hi,
>>
>> since the license issues are solved we can release just from the
>> trunk. The release will contain everything that is in the default
>> build. I think, this is the easiest way to do it now.
>
> Great. Would it be possible build a 0.9.0-RC1-incubating release
> candidate set of artifacts that are not uploaded to the official maven
> repo but just used for testing purpose (e.g. to check that all legal
> files are included, and so on)?
I had a look at Apache Sling and their release process. When doing a
release all released artifacts are uploaded into a staging repository
for review and vote. In staging phase everyone can check the release
on legal issues etc.
> Also the stable launchers includes opennlp models that have no clear
> license status (statistical models derived from copyrighted documents)
> hence are not hostable / distributable on a the Apache maven repo.
> Right now the models are downloaded from
> http://opennlp.sourceforge.net/models-1.5/ and included in the
> defaultdata bundle that is then uploaded to the maven repo of Nuxeo.
> The OpenNLP project is working on building models out of pure open
> data corpus (wikipedia / wikinews / dbpedia) but this is still work in
> progress.
>
> I don't know what to do in the short term. Maybe the first release of
> Stanbol could be done without the launcher bundle? Maybe we could
> build a launcher that without the defaultdata bundle and tell in the
> readme to download it manually from the nuxeo maven repo [1] so as to
> but it in the right sling/ folder after the first start?
>
> https://maven.nuxeo.org/nexus/content/repositories/vendor-releases/org/apache/stanbol/org.apache.stanbol.defaultdata/
If I understand the Apache way right we are free in the way we want to
do it as long as we respect the licenses.
I would suggest to remove everything from the launchers that is not
conform with Apache license and make it optional in the build. IMO
there should be no difference between our standard build and doing a
release. That would make things really easy for the release manager.
--
Fabian
Re: Release and Launchers
Posted by Fabian Christ <ch...@googlemail.com>.
2011/7/20 Rupert Westenthaler <ru...@gmail.com>:
> On Wed, Jul 20, 2011 at 9:56 AM, Fabian Christ
> <ch...@googlemail.com> wrote:
>> 2011/7/19 Olivier Grisel <ol...@ensta.org>:
>>> Maybe we could work on extending the DataFileProvider to make the
>>> defaultdata provider only provide download URLs from the existing gray
>>> licensed opennlp 1.5 models from
>>> http://opennlp.sourceforge.net/models-1.5/ and let the
>>> DataFileProvider download them from there automatically the first time
>>> they are required. The issue then is that every integration tests job
>>> will re-download the same data from sourceforge over and over again...
>>> That will slowdown the builds / tests and waste bandwith for nothing +
>>> add a new way for the builds and test to fail (dependency on the
>>> network / sourceforge availability).
>>
>> I think having the OpenNLP models in our trunk and use them during
>> development in incubation is no problem. So we don't need to change
>> anything for build and integration tests right now.
>>
> The models where never in the trunk, but downloaded by using a shell
> script. I am currently upgrading this to use the maven-ant-plugin
> instead.
> This will allow to download data files automatically during the normal
> build process
>
>> I would propose to exclude the models when a release is made. In this
>> case the OpenNLP engine has to be prepared to recognize that the
>> required model is missing and download it from Sourceforge. If the
>> model is not missing as during development in our trunk everything is
>> fine.
>>
> In other words we will exclude such bundles from the release forcing users to
>
> * check out /data and build them locally or
> * download those bundles from a Maven repository.
>
> But if we want to also release the launchers, that we would need to
> include those bundles otherwise the launcher would not work out of the
> box - something very important for adoption.
>
> I would assume that the normal user would double-click the jar; open
> the Browser; paste some text to /engines - everything he will get with
> the missing models would be "Invalid query" :(
>
> We can not expect that he will go to the Felix Web Console; open the
> DataFileProvider tab; look at the list of missing files; download them
> from SourceForge and copy them to the /datafiles directory.
That's why I would like to separate releases of ready to use Stanbol
versions and the Stanbol framework. When releasing the framework we
can argue that this framework comes without data. If you want a ready
to use Stanbol for a specific scenario we will release it separately.
See my other threat about splitting the release.
--
Fabian
Re: Release and Launchers
Posted by Rupert Westenthaler <ru...@gmail.com>.
On Wed, Jul 20, 2011 at 9:56 AM, Fabian Christ
<ch...@googlemail.com> wrote:
> 2011/7/19 Olivier Grisel <ol...@ensta.org>:
>> Maybe we could work on extending the DataFileProvider to make the
>> defaultdata provider only provide download URLs from the existing gray
>> licensed opennlp 1.5 models from
>> http://opennlp.sourceforge.net/models-1.5/ and let the
>> DataFileProvider download them from there automatically the first time
>> they are required. The issue then is that every integration tests job
>> will re-download the same data from sourceforge over and over again...
>> That will slowdown the builds / tests and waste bandwith for nothing +
>> add a new way for the builds and test to fail (dependency on the
>> network / sourceforge availability).
>
> I think having the OpenNLP models in our trunk and use them during
> development in incubation is no problem. So we don't need to change
> anything for build and integration tests right now.
>
The models where never in the trunk, but downloaded by using a shell
script. I am currently upgrading this to use the maven-ant-plugin
instead.
This will allow to download data files automatically during the normal
build process
> I would propose to exclude the models when a release is made. In this
> case the OpenNLP engine has to be prepared to recognize that the
> required model is missing and download it from Sourceforge. If the
> model is not missing as during development in our trunk everything is
> fine.
>
In other words we will exclude such bundles from the release forcing users to
* check out /data and build them locally or
* download those bundles from a Maven repository.
But if we want to also release the launchers, that we would need to
include those bundles otherwise the launcher would not work out of the
box - something very important for adoption.
I would assume that the normal user would double-click the jar; open
the Browser; paste some text to /engines - everything he will get with
the missing models would be "Invalid query" :(
We can not expect that he will go to the Felix Web Console; open the
DataFileProvider tab; look at the list of missing files; download them
from SourceForge and copy them to the /datafiles directory.
best
Rupert
--
| Rupert Westenthaler rupert.westenthaler@gmail.com
| Bodenlehenstraße 11 ++43-699-11108907
| A-5500 Bischofshofen
Re: Release and Launchers
Posted by Fabian Christ <ch...@googlemail.com>.
2011/7/19 Olivier Grisel <ol...@ensta.org>:
> Maybe we could work on extending the DataFileProvider to make the
> defaultdata provider only provide download URLs from the existing gray
> licensed opennlp 1.5 models from
> http://opennlp.sourceforge.net/models-1.5/ and let the
> DataFileProvider download them from there automatically the first time
> they are required. The issue then is that every integration tests job
> will re-download the same data from sourceforge over and over again...
> That will slowdown the builds / tests and waste bandwith for nothing +
> add a new way for the builds and test to fail (dependency on the
> network / sourceforge availability).
I think having the OpenNLP models in our trunk and use them during
development in incubation is no problem. So we don't need to change
anything for build and integration tests right now.
I would propose to exclude the models when a release is made. In this
case the OpenNLP engine has to be prepared to recognize that the
required model is missing and download it from Sourceforge. If the
model is not missing as during development in our trunk everything is
fine.
--
Fabian
Re: Release and Launchers
Posted by Olivier Grisel <ol...@ensta.org>.
2011/7/19 Fabian Christ <ch...@googlemail.com>:
> 2011/7/19 Olivier Grisel <ol...@ensta.org>:
>> 2011/7/19 Fabian Christ <ch...@googlemail.com>:
>> Also the stable launchers includes opennlp models that have no clear
>> license status (statistical models derived from copyrighted documents)
>> hence are not hostable / distributable on a the Apache maven repo.
>> Right now the models are downloaded from
>> http://opennlp.sourceforge.net/models-1.5/ and included in the
>> defaultdata bundle that is then uploaded to the maven repo of Nuxeo.
>> The OpenNLP project is working on building models out of pure open
>> data corpus (wikipedia / wikinews / dbpedia) but this is still work in
>> progress.
>>
>> I don't know what to do in the short term. Maybe the first release of
>> Stanbol could be done without the launcher bundle? Maybe we could
>> build a launcher that without the defaultdata bundle and tell in the
>> readme to download it manually from the nuxeo maven repo [1] so as to
>> but it in the right sling/ folder after the first start?
>
> So everything in /data and /defaultdata is not clear in terms of
> license? At least we can not redistribute it under Apache, right?
The DBpedia stuff is fine (we probably just need to add a header /
NOTICE / README file to say that is doubled licensed under CC By SA
3.0 / GFDL), the opennlp is not (yet). Joern Kottmann and others the
OpenNLP developers are currently working on a annotated corpus
infrastructure to be able able to train models that are 100% free with
a clear license and thus directly embeddable in Apache distributed
jars. But this work is likely to take at least a couple of months
hence we cannot delay the Stanbol release.
Maybe we could work on extending the DataFileProvider to make the
defaultdata provider only provide download URLs from the existing gray
licensed opennlp 1.5 models from
http://opennlp.sourceforge.net/models-1.5/ and let the
DataFileProvider download them from there automatically the first time
they are required. The issue then is that every integration tests job
will re-download the same data from sourceforge over and over again...
That will slowdown the builds / tests and waste bandwith for nothing +
add a new way for the builds and test to fail (dependency on the
network / sourceforge availability).
--
Olivier
http://twitter.com/ogrisel - http://github.com/ogrisel
Re: Release and Launchers
Posted by Fabian Christ <ch...@googlemail.com>.
2011/7/19 Olivier Grisel <ol...@ensta.org>:
> 2011/7/19 Fabian Christ <ch...@googlemail.com>:
> Also the stable launchers includes opennlp models that have no clear
> license status (statistical models derived from copyrighted documents)
> hence are not hostable / distributable on a the Apache maven repo.
> Right now the models are downloaded from
> http://opennlp.sourceforge.net/models-1.5/ and included in the
> defaultdata bundle that is then uploaded to the maven repo of Nuxeo.
> The OpenNLP project is working on building models out of pure open
> data corpus (wikipedia / wikinews / dbpedia) but this is still work in
> progress.
>
> I don't know what to do in the short term. Maybe the first release of
> Stanbol could be done without the launcher bundle? Maybe we could
> build a launcher that without the defaultdata bundle and tell in the
> readme to download it manually from the nuxeo maven repo [1] so as to
> but it in the right sling/ folder after the first start?
So everything in /data and /defaultdata is not clear in terms of
license? At least we can not redistribute it under Apache, right?
--
Fabian
Re: Release and Launchers
Posted by Olivier Grisel <ol...@ensta.org>.
2011/7/19 Fabian Christ <ch...@googlemail.com>:
> Hi,
>
> since the license issues are solved we can release just from the
> trunk. The release will contain everything that is in the default
> build. I think, this is the easiest way to do it now.
Great. Would it be possible build a 0.9.0-RC1-incubating release
candidate set of artifacts that are not uploaded to the official maven
repo but just used for testing purpose (e.g. to check that all legal
files are included, and so on)?
Also the stable launchers includes opennlp models that have no clear
license status (statistical models derived from copyrighted documents)
hence are not hostable / distributable on a the Apache maven repo.
Right now the models are downloaded from
http://opennlp.sourceforge.net/models-1.5/ and included in the
defaultdata bundle that is then uploaded to the maven repo of Nuxeo.
The OpenNLP project is working on building models out of pure open
data corpus (wikipedia / wikinews / dbpedia) but this is still work in
progress.
I don't know what to do in the short term. Maybe the first release of
Stanbol could be done without the launcher bundle? Maybe we could
build a launcher that without the defaultdata bundle and tell in the
readme to download it manually from the nuxeo maven repo [1] so as to
but it in the right sling/ folder after the first start?
https://maven.nuxeo.org/nexus/content/repositories/vendor-releases/org/apache/stanbol/org.apache.stanbol.defaultdata/
--
Olivier
http://twitter.com/ogrisel - http://github.com/ogrisel
Re: Release and Launchers
Posted by Fabian Christ <ch...@googlemail.com>.
Hi,
since the license issues are solved we can release just from the
trunk. The release will contain everything that is in the default
build. I think, this is the easiest way to do it now.
- Fabian
2011/7/19 Bertrand Delacretaz <bd...@apache.org>:
> On Mon, Jul 18, 2011 at 11:19 AM, Fabian Christ
> <ch...@googlemail.com> wrote:
>> ...for doing a release and excluding artifacts we will need a "release"
>> launcher. At the moment we have the "stable" launcher and the "full"
>> launcher. Integration tests are executed using the "full" launcher....
>
>> ...3) Use the "stable" launcher for the release as in 2) ?...
>
> +1 on including only what we release in the stable launcher, that
> makes sense and you don't need to create yet another launcher.
>
> -Bertrand
>
--
Fabian
Re: Release and Launchers
Posted by Bertrand Delacretaz <bd...@apache.org>.
On Mon, Jul 18, 2011 at 11:19 AM, Fabian Christ
<ch...@googlemail.com> wrote:
> ...for doing a release and excluding artifacts we will need a "release"
> launcher. At the moment we have the "stable" launcher and the "full"
> launcher. Integration tests are executed using the "full" launcher....
> ...3) Use the "stable" launcher for the release as in 2) ?...
+1 on including only what we release in the stable launcher, that
makes sense and you don't need to create yet another launcher.
-Bertrand