You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@baremaps.apache.org by Bertil Chapuis <bc...@gmail.com> on 2022/11/18 17:33:29 UTC
Draft of the release process
Hello Everyone,
The following draft release has been generated with github actions:
https://github.com/apache/incubator-baremaps/releases/tag/untagged-4d07860016b500637021
We have two distributions (source and binary) created with the maven assembly plugin. The checksums and signatures are generated by the CI. For now, the PGP signatures are computed with the PGP key used to sign the maven artifacts. The release action is located here:
https://github.com/apache/incubator-baremaps/blob/cd0a1fdb7c7c29a6f41d1f299239cb61580b3036/.github/workflows/release.yml#L23
In order to publish a new release, the release manager would have to perform the following steps:
1) create a branch for the release
2) execute the mvn release:prepare command in this branch
3) check that the release action completes on github and produces a draft of the release
4) generate and edit the release notes on github
5) ask the mailing list to vote for the release
7) if the vote passes, merge the release branch into main and publish the release
What do you think about this process? Do not hesitate to share your thoughts.
Best,
Bertil
Re: Draft of the release process
Posted by Julian Hyde <jh...@gmail.com>.
I should have mentioned that much of this “good advice” - from me and other mentors - can be deferred. I would suggest logging cases and do only those necessary for the first release. Subsequent releases can be cleaner. As the mantra goes, “Release early, release often”.
Incubating releases can include a DISCLAIMER-WIP file [1] that describes all the stuff they know they haven’t fixed yet.
Julian
[1] https://incubator.apache.org/policy/incubation.html#disclaimers
> On Nov 20, 2022, at 5:50 PM, Calvin Kirs <ki...@apache.org> wrote:
>
> Hi,
>
> We are missing disclaimers[1].
>
> We should not include binary code in the source code, I think you
> forgot to exclude it(*.jar) when you packaged the source code.
> (btw,maven-wrapper is donated to ASF with a version upgraded, we
> better upgrade to the new version 3.1.x which was released under the
> ASF umbrella)
>
> We usually include a RELEASE-NOTE or CHANGE-LOG in the source package.
>
> Here is a complete release process[2][3][4]. maybe it will be useful.
>
> [1]https://incubator.apache.org/policy/incubation.html#disclaimers
> [2]https://incubator.apache.org/guides/releasemanagement.html#best-practice
> [3]https://www.apache.org/legal/release-policy.html#what
> [4]https://infra.apache.org/release-distribution
>
>> On Mon, Nov 21, 2022 at 6:24 AM Julian Hyde <jh...@apache.org> wrote:
>>
>> Thanks for creating a process. Now we have a process, we can iterate.
>> A few thoughts.
>>
>> 1. On terminology. The release itself consists of the source code,
>> literally the src.tar.gz file. People incorrectly use the term 'binary
>> release' but the binaries are just artifacts generated from the
>> release.
>>
>> Part of the reason that binaries are not part of the release is that
>> it's only practical to vote on source code. What you have done here,
>> generating and signing binaries from the source code, is about the
>> best we can do.
>>
>> An Apache project MAY distribute binary artifacts with a release.
>> Since Baremaps is a Java project, I would recommend deploying the jars
>> to Maven Central. There are Maven plugins to stage the jars in
>> Apache's repository so that people can inspect them during the vote.
>>
>> 2. On process. The release manager will send an email to vote on the
>> artifacts. The email contains the hashes of the objects (to prevent,
>> among other things, a man-in-the-middle attack where a server spoofs
>> github.com), and instructions to verify the artifacts.
>>
>> As a mentor, I generally recommend that the first release is source
>> only, and therefore you would not mention the binary artifacts in the
>> release vote.
>>
>> 3. On the contents of the release. I noticed a few errors in the files:
>> * the LICENSE file still contains its appendix; you should remove it;
>> * in NOTICE, '2022 and onwards' should be '2022';
>> * there will need to be a DISCLAIMER file noting the incubating status;
>> * maven-wrapper.jar should not be in the source distro (see
>> https://issues.apache.org/jira/browse/LEGAL-570);
>> * I strongly recommend that there is a README or README.txt file,
>> written for the person who has just downloaded and unzipped the tar
>> ball, describing what they have downloaded (including the version) and
>> how to build it (yes, it's unfortunate that this file clashes with the
>> README.md required by GitHub);
>> * I would add headers to every text file that supports it, e.g.
>> pom.xml, READM.md, baremaps.bat; there is an argument to skip for
>> files where size is important, e.g. .js files that won't be minified;
>> * Are there any generated files in the source code?
>> maplibre-gl-inspect.js and schema.db.
>>
>> Errors are expected at this point. I'm sure I will find many more when
>> there is a release candidate. :)
>>
>> 4. On licensing. The NOTICE file looks in fairly good shape. But I
>> think we will need to audit the various test data sets.
>>
>> 5. On the location. Generally RMs stage the files into Apache
>> subversion before a release vote. This ensures that the artifacts
>> cannot be accidentally updated or deleted. (It also makes it easier
>> for me, because I can just do 'svn update' on my directory that maps
>> to https://dist.apache.org/repos/dist/dev/incubator/baremaps.)
>>
>> Lastly, I recommend that java classes have javadoc, without exception.
>> (This advice pertains to the software development process, not
>> releases, so feel free to ignore it.) Javadoc is a great place to
>> document the purpose of a class; not just what it does, but why it is
>> necessary, how it works, how you should use it, and why we put it in
>> this place. If contributors see classes without javadoc they will
>> contribute classes without javadoc, so if you start badly, things will
>> only get worse over time.
>>
>> Julian
>>
>>> On Fri, Nov 18, 2022 at 9:33 AM Bertil Chapuis <bc...@gmail.com> wrote:
>>>
>>> Hello Everyone,
>>>
>>> The following draft release has been generated with github actions:
>>> https://github.com/apache/incubator-baremaps/releases/tag/untagged-4d07860016b500637021
>>>
>>> We have two distributions (source and binary) created with the maven assembly plugin. The checksums and signatures are generated by the CI. For now, the PGP signatures are computed with the PGP key used to sign the maven artifacts. The release action is located here:
>>> https://github.com/apache/incubator-baremaps/blob/cd0a1fdb7c7c29a6f41d1f299239cb61580b3036/.github/workflows/release.yml#L23
>>>
>>> In order to publish a new release, the release manager would have to perform the following steps:
>>> 1) create a branch for the release
>>> 2) execute the mvn release:prepare command in this branch
>>> 3) check that the release action completes on github and produces a draft of the release
>>> 4) generate and edit the release notes on github
>>> 5) ask the mailing list to vote for the release
>>> 7) if the vote passes, merge the release branch into main and publish the release
>>>
>>> What do you think about this process? Do not hesitate to share your thoughts.
>>>
>>> Best,
>>>
>>> Bertil
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@baremaps.apache.org
>> For additional commands, e-mail: dev-help@baremaps.apache.org
>>
>
>
> --
> Best wishes！
> CalvinKirs
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@baremaps.apache.org
> For additional commands, e-mail: dev-help@baremaps.apache.org
>
Re: Draft of the release process
Posted by Calvin Kirs <ki...@apache.org>.
Hi,
We are missing disclaimers[1].
We should not include binary code in the source code, I think you
forgot to exclude it(*.jar) when you packaged the source code.
(btw,maven-wrapper is donated to ASF with a version upgraded, we
better upgrade to the new version 3.1.x which was released under the
ASF umbrella)
We usually include a RELEASE-NOTE or CHANGE-LOG in the source package.
Here is a complete release process[2][3][4]. maybe it will be useful.
[1]https://incubator.apache.org/policy/incubation.html#disclaimers
[2]https://incubator.apache.org/guides/releasemanagement.html#best-practice
[3]https://www.apache.org/legal/release-policy.html#what
[4]https://infra.apache.org/release-distribution
On Mon, Nov 21, 2022 at 6:24 AM Julian Hyde <jh...@apache.org> wrote:
>
> Thanks for creating a process. Now we have a process, we can iterate.
> A few thoughts.
>
> 1. On terminology. The release itself consists of the source code,
> literally the src.tar.gz file. People incorrectly use the term 'binary
> release' but the binaries are just artifacts generated from the
> release.
>
> Part of the reason that binaries are not part of the release is that
> it's only practical to vote on source code. What you have done here,
> generating and signing binaries from the source code, is about the
> best we can do.
>
> An Apache project MAY distribute binary artifacts with a release.
> Since Baremaps is a Java project, I would recommend deploying the jars
> to Maven Central. There are Maven plugins to stage the jars in
> Apache's repository so that people can inspect them during the vote.
>
> 2. On process. The release manager will send an email to vote on the
> artifacts. The email contains the hashes of the objects (to prevent,
> among other things, a man-in-the-middle attack where a server spoofs
> github.com), and instructions to verify the artifacts.
>
> As a mentor, I generally recommend that the first release is source
> only, and therefore you would not mention the binary artifacts in the
> release vote.
>
> 3. On the contents of the release. I noticed a few errors in the files:
> * the LICENSE file still contains its appendix; you should remove it;
> * in NOTICE, '2022 and onwards' should be '2022';
> * there will need to be a DISCLAIMER file noting the incubating status;
> * maven-wrapper.jar should not be in the source distro (see
> https://issues.apache.org/jira/browse/LEGAL-570);
> * I strongly recommend that there is a README or README.txt file,
> written for the person who has just downloaded and unzipped the tar
> ball, describing what they have downloaded (including the version) and
> how to build it (yes, it's unfortunate that this file clashes with the
> README.md required by GitHub);
> * I would add headers to every text file that supports it, e.g.
> pom.xml, READM.md, baremaps.bat; there is an argument to skip for
> files where size is important, e.g. .js files that won't be minified;
> * Are there any generated files in the source code?
> maplibre-gl-inspect.js and schema.db.
>
> Errors are expected at this point. I'm sure I will find many more when
> there is a release candidate. :)
>
> 4. On licensing. The NOTICE file looks in fairly good shape. But I
> think we will need to audit the various test data sets.
>
> 5. On the location. Generally RMs stage the files into Apache
> subversion before a release vote. This ensures that the artifacts
> cannot be accidentally updated or deleted. (It also makes it easier
> for me, because I can just do 'svn update' on my directory that maps
> to https://dist.apache.org/repos/dist/dev/incubator/baremaps.)
>
> Lastly, I recommend that java classes have javadoc, without exception.
> (This advice pertains to the software development process, not
> releases, so feel free to ignore it.) Javadoc is a great place to
> document the purpose of a class; not just what it does, but why it is
> necessary, how it works, how you should use it, and why we put it in
> this place. If contributors see classes without javadoc they will
> contribute classes without javadoc, so if you start badly, things will
> only get worse over time.
>
> Julian
>
> On Fri, Nov 18, 2022 at 9:33 AM Bertil Chapuis <bc...@gmail.com> wrote:
> >
> > Hello Everyone,
> >
> > The following draft release has been generated with github actions:
> > https://github.com/apache/incubator-baremaps/releases/tag/untagged-4d07860016b500637021
> >
> > We have two distributions (source and binary) created with the maven assembly plugin. The checksums and signatures are generated by the CI. For now, the PGP signatures are computed with the PGP key used to sign the maven artifacts. The release action is located here:
> > https://github.com/apache/incubator-baremaps/blob/cd0a1fdb7c7c29a6f41d1f299239cb61580b3036/.github/workflows/release.yml#L23
> >
> > In order to publish a new release, the release manager would have to perform the following steps:
> > 1) create a branch for the release
> > 2) execute the mvn release:prepare command in this branch
> > 3) check that the release action completes on github and produces a draft of the release
> > 4) generate and edit the release notes on github
> > 5) ask the mailing list to vote for the release
> > 7) if the vote passes, merge the release branch into main and publish the release
> >
> > What do you think about this process? Do not hesitate to share your thoughts.
> >
> > Best,
> >
> > Bertil
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@baremaps.apache.org
> For additional commands, e-mail: dev-help@baremaps.apache.org
>
--
Best wishes！
CalvinKirs
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@baremaps.apache.org
For additional commands, e-mail: dev-help@baremaps.apache.org
Re: Draft of the release process
Posted by Bertrand Delacretaz <bd...@apache.org>.
Hi,
Julian Hyde <jh...@apache.org> wrote:
> ...Part of the reason that binaries are not part of the release is that
> it's only practical to vote on source code. What you have done here,
> generating and signing binaries from the source code, is about the
> best we can do...
I think Reproducible Builds also help make binaries safer, see links below.
-Bertrand
- https://reproducible-builds.org
- https://github.com/jvm-repo-rebuild/reproducible-central
- https://zlika.github.io/reproducible-build-maven-plugin/index.html
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@baremaps.apache.org
For additional commands, e-mail: dev-help@baremaps.apache.org
Re: Draft of the release process
Posted by Josh Fischer <jo...@joshfischer.io>.
Great job getting this started. Comments are inlined.
Websites referenced:
[1] https://creadur.apache.org/rat/
[2] https://incubator.apache.org/policy/incubation.html#disclaimers
On Sun, Nov 20, 2022 at 4:24 PM Julian Hyde <jh...@apache.org> wrote:
> Thanks for creating a process. Now we have a process, we can iterate.
> A few thoughts.
>
> 1. On terminology. The release itself consists of the source code,
> literally the src.tar.gz file. People incorrectly use the term 'binary
> release' but the binaries are just artifacts generated from the
> release.
>
> Part of the reason that binaries are not part of the release is that
> it's only practical to vote on source code. What you have done here,
> generating and signing binaries from the source code, is about the
> best we can do.
>
> An Apache project MAY distribute binary artifacts with a release.
> Since Baremaps is a Java project, I would recommend deploying the jars
> to Maven Central. There are Maven plugins to stage the jars in
> Apache's repository so that people can inspect them during the vote.
>
> 2. On process. The release manager will send an email to vote on the
> artifacts. The email contains the hashes of the objects (to prevent,
> among other things, a man-in-the-middle attack where a server spoofs
> github.com), and instructions to verify the artifacts.
>
> As a mentor, I generally recommend that the first release is source
> only, and therefore you would not mention the binary artifacts in the
> release vote.
>
> 3. On the contents of the release. I noticed a few errors in the files:
> * the LICENSE file still contains its appendix; you should remove it;
> * in NOTICE, '2022 and onwards' should be '2022';
> * there will need to be a DISCLAIMER file noting the incubating status;
> * maven-wrapper.jar should not be in the source distro (see
> https://issues.apache.org/jira/browse/LEGAL-570);
> * I strongly recommend that there is a README or README.txt file,
> written for the person who has just downloaded and unzipped the tar
> ball, describing what they have downloaded (including the version) and
> how to build it (yes, it's unfortunate that this file clashes with the
> README.md required by GitHub);
> * I would add headers to every text file that supports it, e.g.
> pom.xml, READM.md, baremaps.bat; there is an argument to skip for
> files where size is important, e.g. .js files that won't be minified;
> * Are there any generated files in the source code?
> maplibre-gl-inspect.js and schema.db.
>
> Errors are expected at this point. I'm sure I will find many more when
> there is a release candidate. :)
>
** I ran an Apache Rat [1] check in the repo. We need to add a rat
excludes to ignore files where appropriate and add header files, Like
Julian suggested.
The command I ran (from inside the root folder of the repo) was
$ java -jar ~/path-to-jar/apache-rat-0.15.jar .
Typically, going through the licensing is a tedious task and we can get
around doing this before the first release by adding a "Work in progress"
Disclaimer [2]
4. On licensing. The NOTICE file looks in fairly good shape. But I
> think we will need to audit the various test data sets.
>
> 5. On the location. Generally RMs stage the files into Apache
> subversion before a release vote. This ensures that the artifacts
> cannot be accidentally updated or deleted. (It also makes it easier
> for me, because I can just do 'svn update' on my directory that maps
> to https://dist.apache.org/repos/dist/dev/incubator/baremaps.)
>
> Lastly, I recommend that java classes have javadoc, without exception.
> (This advice pertains to the software development process, not
> releases, so feel free to ignore it.) Javadoc is a great place to
> document the purpose of a class; not just what it does, but why it is
> necessary, how it works, how you should use it, and why we put it in
> this place. If contributors see classes without javadoc they will
> contribute classes without javadoc, so if you start badly, things will
> only get worse over time.
>
> Julian
>
> On Fri, Nov 18, 2022 at 9:33 AM Bertil Chapuis <bc...@gmail.com> wrote:
> >
> > Hello Everyone,
> >
> > The following draft release has been generated with github actions:
> >
> https://github.com/apache/incubator-baremaps/releases/tag/untagged-4d07860016b500637021
> >
> > We have two distributions (source and binary) created with the maven
> assembly plugin. The checksums and signatures are generated by the CI. For
> now, the PGP signatures are computed with the PGP key used to sign the
> maven artifacts. The release action is located here:
> >
> https://github.com/apache/incubator-baremaps/blob/cd0a1fdb7c7c29a6f41d1f299239cb61580b3036/.github/workflows/release.yml#L23
> >
> > In order to publish a new release, the release manager would have to
> perform the following steps:
> > 1) create a branch for the release
> > 2) execute the mvn release:prepare command in this branch
> > 3) check that the release action completes on github and produces a
> draft of the release
> > 4) generate and edit the release notes on github
> > 5) ask the mailing list to vote for the release
> > 7) if the vote passes, merge the release branch into main and publish
> the release
> >
> > What do you think about this process? Do not hesitate to share your
> thoughts.
> >
> > Best,
> >
> > Bertil
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@baremaps.apache.org
> For additional commands, e-mail: dev-help@baremaps.apache.org
>
>
Re: Draft of the release process
Posted by Julian Hyde <jh...@apache.org>.
Thanks for creating a process. Now we have a process, we can iterate.
A few thoughts.
1. On terminology. The release itself consists of the source code,
literally the src.tar.gz file. People incorrectly use the term 'binary
release' but the binaries are just artifacts generated from the
release.
Part of the reason that binaries are not part of the release is that
it's only practical to vote on source code. What you have done here,
generating and signing binaries from the source code, is about the
best we can do.
An Apache project MAY distribute binary artifacts with a release.
Since Baremaps is a Java project, I would recommend deploying the jars
to Maven Central. There are Maven plugins to stage the jars in
Apache's repository so that people can inspect them during the vote.
2. On process. The release manager will send an email to vote on the
artifacts. The email contains the hashes of the objects (to prevent,
among other things, a man-in-the-middle attack where a server spoofs
github.com), and instructions to verify the artifacts.
As a mentor, I generally recommend that the first release is source
only, and therefore you would not mention the binary artifacts in the
release vote.
3. On the contents of the release. I noticed a few errors in the files:
* the LICENSE file still contains its appendix; you should remove it;
* in NOTICE, '2022 and onwards' should be '2022';
* there will need to be a DISCLAIMER file noting the incubating status;
* maven-wrapper.jar should not be in the source distro (see
https://issues.apache.org/jira/browse/LEGAL-570);
* I strongly recommend that there is a README or README.txt file,
written for the person who has just downloaded and unzipped the tar
ball, describing what they have downloaded (including the version) and
how to build it (yes, it's unfortunate that this file clashes with the
README.md required by GitHub);
* I would add headers to every text file that supports it, e.g.
pom.xml, READM.md, baremaps.bat; there is an argument to skip for
files where size is important, e.g. .js files that won't be minified;
* Are there any generated files in the source code?
maplibre-gl-inspect.js and schema.db.
Errors are expected at this point. I'm sure I will find many more when
there is a release candidate. :)
4. On licensing. The NOTICE file looks in fairly good shape. But I
think we will need to audit the various test data sets.
5. On the location. Generally RMs stage the files into Apache
subversion before a release vote. This ensures that the artifacts
cannot be accidentally updated or deleted. (It also makes it easier
for me, because I can just do 'svn update' on my directory that maps
to https://dist.apache.org/repos/dist/dev/incubator/baremaps.)
Lastly, I recommend that java classes have javadoc, without exception.
(This advice pertains to the software development process, not
releases, so feel free to ignore it.) Javadoc is a great place to
document the purpose of a class; not just what it does, but why it is
necessary, how it works, how you should use it, and why we put it in
this place. If contributors see classes without javadoc they will
contribute classes without javadoc, so if you start badly, things will
only get worse over time.
Julian
On Fri, Nov 18, 2022 at 9:33 AM Bertil Chapuis <bc...@gmail.com> wrote:
>
> Hello Everyone,
>
> The following draft release has been generated with github actions:
> https://github.com/apache/incubator-baremaps/releases/tag/untagged-4d07860016b500637021
>
> We have two distributions (source and binary) created with the maven assembly plugin. The checksums and signatures are generated by the CI. For now, the PGP signatures are computed with the PGP key used to sign the maven artifacts. The release action is located here:
> https://github.com/apache/incubator-baremaps/blob/cd0a1fdb7c7c29a6f41d1f299239cb61580b3036/.github/workflows/release.yml#L23
>
> In order to publish a new release, the release manager would have to perform the following steps:
> 1) create a branch for the release
> 2) execute the mvn release:prepare command in this branch
> 3) check that the release action completes on github and produces a draft of the release
> 4) generate and edit the release notes on github
> 5) ask the mailing list to vote for the release
> 7) if the vote passes, merge the release branch into main and publish the release
>
> What do you think about this process? Do not hesitate to share your thoughts.
>
> Best,
>
> Bertil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@baremaps.apache.org
For additional commands, e-mail: dev-help@baremaps.apache.org
Re: Draft of the release process
Posted by Bertil Chapuis <bc...@gmail.com>.
Hello Everyone,
Following the exchanges we had with Calvin on GitHub, I added a DISCLAIMER-WIP file to the pull request. As the latest release is pretty far from what we have on the main branch, I'd like to merge this PR and perform an intermediary release (v0.7.1). Is it possible to publish this kind of release?
Also, we still need to address several issues to comply with Apache guidelines and I was wondering how we could share some of this work. For instance, we need to add license headers to all the javascript files in the basemap and in the examples. We also need to figure out how minified javascript resources should be integrated in the project (as resources in the jar or as resources downloaded from a CDN). If you don't have much time, that's okay, but are there some small tasks such as these ones or those listed in #492 that you would be interested to take on?
Best,
Bertil
> On 5 Feb 2023, at 13:07, Bertil Chapuis <bc...@gmail.com> wrote:
>
> Hello Everyone,
>
> I hope you are doing well. I guess we all have been very busy lately ;-) So let me resume this thread.
>
> Most of the issues reported in this thread have been addressed [1] or added to the release checklist [2]. The main one being the inclusion of a DISCLAIMER-WIP file to the release until we are fully conform to Apache’s standards. From what I understand, this should allow us to make releases, even if we haven’t completely resolved the proj4j issue in Baremaps yet.
>
> Please, do not hesitate to provide feedbacks, report issues, or even to commit in the 492-release branch.
>
> Best,
>
> Bertil
>
> [1] https://github.com/apache/incubator-baremaps/pull/535
> [2] https://github.com/apache/incubator-baremaps/issues/492
> [3] https://github.com/apache/incubator-baremaps/releases/tag/v0.7.1-rc3
>
>> On 23 Nov 2022, at 00:19, Martin Desruisseaux <ma...@geomatys.com> wrote:
>>
>> Hello Bertil
>>
>> Le 22/11/2022 à 21:47, Bertil Chapuis a écrit :
>>
>>> The terms of the EPSG license are restrictive [1]: The data may not be distributed for profit by any third party. I’m quite amazed to discover that most geospatial solutions out there are probably violating this license.
>>>
>> Yes. In the case of PROJ version 6 and later, complying to the license would be just a matter of releasing PROJ under MIT + EPSG terms of use, not MIT alone. Because PROJ is used by GDAL, QGIS and PostGIS (among others), the issue apply to all those software as well and could be resolved in the same way. But my understanding is that adding EPSG Terms of Use would cause problems for packaging QGIS and PostGIS in some Linux distributions (it is a matter of license policy, like the policy we have at Apache), which is why PROJ maintainers are reluctant to address the issue.
>>
>> For PROJ version 4, the situation is more tricky because that old version modified the EPSG dataset, notably by changing axis order to (longitude, latitude). The EPSG license does not allow such modifications (because they change the numerical results of coordinate operations), or at least not under "EPSG" namespace. So PROJ4 and PROJ4J are more difficult to resolve. PROJ version 6 and later are better because since version 6, PROJ respects EPSG data integrity.
>>
>>
>>> [1]https://svn.osgeo.org/geotools/trunk/licenses/EPSG.html
>>
>> This is an old version of EPSG terms of use. Current version is https://epsg.org/terms-of-use.html
>>
>> Compared to the old version, the new version explains in more details what kinds of changes we are allowed to do. In brief we are allowed to do some changes, but not any changes; they provide a table listing what we can do. This restriction is based on a good intention: the changes shall not alter the result of coordinate operations, otherwise (in worst case) safety could be compromised. Other conditions worth to note are:
>>
>> * (…snip…) any use of the EPSG Facilities (…snip…), even if obtained
>> without clicking acceptance, will be acceptance of these Terms of Use.
>> * You are obliged to inform anyone to whom you provide the EPSG
>> Facilities of these Terms of Use.
>>
>> In other words, even if we obtained the EPSG data in another way than direct download from EPSG web site (e.g. if we obtained the EPSG data through PROJ4 CSV files), the EPSG terms of use still apply.
>>
>>
>>> Concretely, does this mean that we could include a couple of EPSGs (not the full database) without concern? In Baremaps, we mainly require EPSG:4326 and EPSG:3857. It not yet clear to me if we can import proj4j and exclude parts of the EPSG database.
>>>
>> In Apache SIS I took the risk of including a hard-coded definition of EPSG::4326 used as a fallback if the EPSG dataset is not installed, because that code is so widely used. I included a few other codes as well. I hope that they fall under the "Fair Use" exception, which is allowed by U.S. law [1]. However this is my uneducated interpretation and I may very well be wrong.
>>
>> [1]https://www.copyright.gov/help/faq/faq-fairuse.html
>>
>>
>>> In the context of a cli, I would prefer to find a solution where the user is not prompted to accept the terms of the EPSG license. For instance, could we have a command (e.g. ./baremaps non-free) that downloads these jars in the lib folder?
>>>
>> I don't know. We can ask for advice by opening a JIRA issue on https://issues.apache.org/jira/projects/LEGAL/
>>
>> For making the task easier for legal volunteers, we should not go in details of what is EPSG. Just said that it is a widely used dataset, that the license is classified Category X by Apache (with a link to the license on EPSG web site) and maybe said that a similar issue was discussed in https://issues.apache.org/jira/browse/LEGAL-183 and resolved by prompting users (when SIS is used as an application) or requiring developers to add themselves a Maven dependency (when SIS is used as a library).
>>
>> They may ask that EPSG data are not downloaded from an Apache server. Preference may be let users download themselves from EPSG web site, but I agree that it is so unconvenient (we have to open an account) that not many users will do that. In Apache SIS case, the data are downloaded from Maven Central.
>>
>>
>>> I think the Eclipse Foundation did an exception for the locationtech group. According to their FAQ [2] they asked the permission to redistribute some pervasive dependencies in geospatial projects: “We can re-distribute the binaries for certain components. We do have explicit permission to re-distribute Geotools, JTS, and GEOS.”.
>>>
>> In the case of Geotools, I guess Eclipse accepted the LGPL license. JTS was also used to have a non-compatible license, but they changed a few years ago to a more permissive one, so the exception may not apply anymore. I'm not sure they were aware of EPSG Terms of Use when they granted exception.
>>
>> Anyway, there is nothing wrong in distributing EPSG data in an application, provided that we include EPSG terms in the license folder. Whether a foundation (Apache, Eclipse, Linux distribution…) accepts EPSG license is a matter of foundation policy. Each organization, or each individual, can choose their own policy regarding which licenses to accept.
>>
>> Martin
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@baremaps.apache.org
For additional commands, e-mail: dev-help@baremaps.apache.org
Re: Draft of the release process
Posted by Bertil Chapuis <bc...@gmail.com>.
Hello Everyone,
I hope you are doing well. I guess we all have been very busy lately ;-) So let me resume this thread.
Most of the issues reported in this thread have been addressed [1] or added to the release checklist [2]. The main one being the inclusion of a DISCLAIMER-WIP file to the release until we are fully conform to Apache’s standards. From what I understand, this should allow us to make releases, even if we haven’t completely resolved the proj4j issue in Baremaps yet.
Please, do not hesitate to provide feedbacks, report issues, or even to commit in the 492-release branch.
Best,
Bertil
[1] https://github.com/apache/incubator-baremaps/pull/535 <https://github.com/apache/incubator-baremaps/issues/492>
[2] https://github.com/apache/incubator-baremaps/issues/492 <https://github.com/apache/incubator-baremaps/issues/492>
[3] https://github.com/apache/incubator-baremaps/releases/tag/v0.7.1-rc3 <https://github.com/apache/incubator-baremaps/releases/tag/v0.7.1-rc3>
> On 23 Nov 2022, at 00:19, Martin Desruisseaux <martin.desruisseaux@geomatys.com <ma...@geomatys.com>> wrote:
>
> Hello Bertil
>
> Le 22/11/2022 à 21:47, Bertil Chapuis a écrit :
>
>> The terms of the EPSG license are restrictive [1]: The data may not be distributed for profit by any third party. I’m quite amazed to discover that most geospatial solutions out there are probably violating this license.
>>
> Yes. In the case of PROJ version 6 and later, complying to the license would be just a matter of releasing PROJ under MIT + EPSG terms of use, not MIT alone. Because PROJ is used by GDAL, QGIS and PostGIS (among others), the issue apply to all those software as well and could be resolved in the same way. But my understanding is that adding EPSG Terms of Use would cause problems for packaging QGIS and PostGIS in some Linux distributions (it is a matter of license policy, like the policy we have at Apache), which is why PROJ maintainers are reluctant to address the issue.
>
> For PROJ version 4, the situation is more tricky because that old version modified the EPSG dataset, notably by changing axis order to (longitude, latitude). The EPSG license does not allow such modifications (because they change the numerical results of coordinate operations), or at least not under "EPSG" namespace. So PROJ4 and PROJ4J are more difficult to resolve. PROJ version 6 and later are better because since version 6, PROJ respects EPSG data integrity.
>
>
>> [1]https://svn.osgeo.org/geotools/trunk/licenses/EPSG.html <https://svn.osgeo.org/geotools/trunk/licenses/EPSG.html>
>
> This is an old version of EPSG terms of use. Current version is https://epsg.org/terms-of-use.html <https://epsg.org/terms-of-use.html>
>
> Compared to the old version, the new version explains in more details what kinds of changes we are allowed to do. In brief we are allowed to do some changes, but not any changes; they provide a table listing what we can do. This restriction is based on a good intention: the changes shall not alter the result of coordinate operations, otherwise (in worst case) safety could be compromised. Other conditions worth to note are:
>
> * (…snip…) any use of the EPSG Facilities (…snip…), even if obtained
> without clicking acceptance, will be acceptance of these Terms of Use.
> * You are obliged to inform anyone to whom you provide the EPSG
> Facilities of these Terms of Use.
>
> In other words, even if we obtained the EPSG data in another way than direct download from EPSG web site (e.g. if we obtained the EPSG data through PROJ4 CSV files), the EPSG terms of use still apply.
>
>
>> Concretely, does this mean that we could include a couple of EPSGs (not the full database) without concern? In Baremaps, we mainly require EPSG:4326 and EPSG:3857. It not yet clear to me if we can import proj4j and exclude parts of the EPSG database.
>>
> In Apache SIS I took the risk of including a hard-coded definition of EPSG::4326 used as a fallback if the EPSG dataset is not installed, because that code is so widely used. I included a few other codes as well. I hope that they fall under the "Fair Use" exception, which is allowed by U.S. law [1]. However this is my uneducated interpretation and I may very well be wrong.
>
> [1]https://www.copyright.gov/help/faq/faq-fairuse.html <https://www.copyright.gov/help/faq/faq-fairuse.html>
>
>
>> In the context of a cli, I would prefer to find a solution where the user is not prompted to accept the terms of the EPSG license. For instance, could we have a command (e.g. ./baremaps non-free) that downloads these jars in the lib folder?
>>
> I don't know. We can ask for advice by opening a JIRA issue on https://issues.apache.org/jira/projects/LEGAL/ <https://issues.apache.org/jira/projects/LEGAL/>
>
> For making the task easier for legal volunteers, we should not go in details of what is EPSG. Just said that it is a widely used dataset, that the license is classified Category X by Apache (with a link to the license on EPSG web site) and maybe said that a similar issue was discussed in https://issues.apache.org/jira/browse/LEGAL-183 <https://issues.apache.org/jira/browse/LEGAL-183> and resolved by prompting users (when SIS is used as an application) or requiring developers to add themselves a Maven dependency (when SIS is used as a library).
>
> They may ask that EPSG data are not downloaded from an Apache server. Preference may be let users download themselves from EPSG web site, but I agree that it is so unconvenient (we have to open an account) that not many users will do that. In Apache SIS case, the data are downloaded from Maven Central.
>
>
>> I think the Eclipse Foundation did an exception for the locationtech group. According to their FAQ [2] they asked the permission to redistribute some pervasive dependencies in geospatial projects: “We can re-distribute the binaries for certain components. We do have explicit permission to re-distribute Geotools, JTS, and GEOS.”.
>>
> In the case of Geotools, I guess Eclipse accepted the LGPL license. JTS was also used to have a non-compatible license, but they changed a few years ago to a more permissive one, so the exception may not apply anymore. I'm not sure they were aware of EPSG Terms of Use when they granted exception.
>
> Anyway, there is nothing wrong in distributing EPSG data in an application, provided that we include EPSG terms in the license folder. Whether a foundation (Apache, Eclipse, Linux distribution…) accepts EPSG license is a matter of foundation policy. Each organization, or each individual, can choose their own policy regarding which licenses to accept.
>
> Martin
>
Re: Draft of the release process
Posted by Martin Desruisseaux <ma...@geomatys.com>.
Hello Bertil
Le 22/11/2022 à 21:47, Bertil Chapuis a écrit :
> The terms of the EPSG license are restrictive [1]: The data may not be
> distributed for profit by any third party. I’m quite amazed to
> discover that most geospatial solutions out there are probably
> violating this license.
>
Yes. In the case of PROJ version 6 and later, complying to the license
would be just a matter of releasing PROJ under MIT + EPSG terms of use,
not MIT alone. Because PROJ is used by GDAL, QGIS and PostGIS (among
others), the issue apply to all those software as well and could be
resolved in the same way. But my understanding is that adding EPSG Terms
of Use would cause problems for packaging QGIS and PostGIS in some Linux
distributions (it is a matter of license policy, like the policy we have
at Apache), which is why PROJ maintainers are reluctant to address the
issue.
For PROJ version 4, the situation is more tricky because that old
version modified the EPSG dataset, notably by changing axis order to
(longitude, latitude). The EPSG license does not allow such
modifications (because they change the numerical results of coordinate
operations), or at least not under "EPSG" namespace. So PROJ4 and PROJ4J
are more difficult to resolve. PROJ version 6 and later are better
because since version 6, PROJ respects EPSG data integrity.
> [1]https://svn.osgeo.org/geotools/trunk/licenses/EPSG.html
This is an old version of EPSG terms of use. Current version is
https://epsg.org/terms-of-use.html
Compared to the old version, the new version explains in more details
what kinds of changes we are allowed to do. In brief we are allowed to
do some changes, but not any changes; they provide a table listing what
we can do. This restriction is based on a good intention: the changes
shall not alter the result of coordinate operations, otherwise (in worst
case) safety could be compromised. Other conditions worth to note are:
* (…snip…) any use of the EPSG Facilities (…snip…), even if obtained
without clicking acceptance, will be acceptance of these Terms of Use.
* You are obliged to inform anyone to whom you provide the EPSG
Facilities of these Terms of Use.
In other words, even if we obtained the EPSG data in another way than
direct download from EPSG web site (e.g. if we obtained the EPSG data
through PROJ4 CSV files), the EPSG terms of use still apply.
> Concretely, does this mean that we could include a couple of EPSGs
> (not the full database) without concern? In Baremaps, we mainly
> require EPSG:4326 and EPSG:3857. It not yet clear to me if we can
> import proj4j and exclude parts of the EPSG database.
>
In Apache SIS I took the risk of including a hard-coded definition of
EPSG::4326 used as a fallback if the EPSG dataset is not installed,
because that code is so widely used. I included a few other codes as
well. I hope that they fall under the "Fair Use" exception, which is
allowed by U.S. law [1]. However this is my uneducated interpretation
and I may very well be wrong.
[1]https://www.copyright.gov/help/faq/faq-fairuse.html
> In the context of a cli, I would prefer to find a solution where the
> user is not prompted to accept the terms of the EPSG license. For
> instance, could we have a command (e.g. ./baremaps non-free) that
> downloads these jars in the lib folder?
>
I don't know. We can ask for advice by opening a JIRA issue on
https://issues.apache.org/jira/projects/LEGAL/
For making the task easier for legal volunteers, we should not go in
details of what is EPSG. Just said that it is a widely used dataset,
that the license is classified Category X by Apache (with a link to the
license on EPSG web site) and maybe said that a similar issue was
discussed in https://issues.apache.org/jira/browse/LEGAL-183 and
resolved by prompting users (when SIS is used as an application) or
requiring developers to add themselves a Maven dependency (when SIS is
used as a library).
They may ask that EPSG data are not downloaded from an Apache server.
Preference may be let users download themselves from EPSG web site, but
I agree that it is so unconvenient (we have to open an account) that not
many users will do that. In Apache SIS case, the data are downloaded
from Maven Central.
> I think the Eclipse Foundation did an exception for the locationtech
> group. According to their FAQ [2] they asked the permission to
> redistribute some pervasive dependencies in geospatial projects: “We
> can re-distribute the binaries for certain components. We do have
> explicit permission to re-distribute Geotools, JTS, and GEOS.”.
>
In the case of Geotools, I guess Eclipse accepted the LGPL license. JTS
was also used to have a non-compatible license, but they changed a few
years ago to a more permissive one, so the exception may not apply
anymore. I'm not sure they were aware of EPSG Terms of Use when they
granted exception.
Anyway, there is nothing wrong in distributing EPSG data in an
application, provided that we include EPSG terms in the license folder.
Whether a foundation (Apache, Eclipse, Linux distribution…) accepts EPSG
license is a matter of foundation policy. Each organization, or each
individual, can choose their own policy regarding which licenses to accept.
Martin
Re: Draft of the release process
Posted by Bertil Chapuis <bc...@gmail.com>.
These clarifications are really interesting. I wasn’t aware of this issue and assumed that the license of proj4j (MIT) was compatible with Apache. The terms of the EPSG license are restrictive [1]: The data may not be distributed for profit by any third party. I’m quite amazed to discover that most geospatial solutions out there are probably violating this license.
[1] https://svn.osgeo.org/geotools/trunk/licenses/EPSG.html
>> Until now I thought it would be a good alternative lib besides EPSG approach.
>>
> When creating a Coordinate Reference System (CRS) from an EPSG code such as "EPSG:4326", we are using the EPSG geodetic dataset. I think that if a software contains e.g. 10 EPSG codes (compared to the ~6000 definitions in the full dataset), it could be considered Fair Use. But including the full dataset requires that we comply with EPSG Terms Of Use.
>
> However if CRS are created from user-supplied WKT strings (or PROJ4 parameter strings), then no EPSG code is involved. The WKT definition string may contain an AUTHORITY["EPSG", 4326] element, but it only means "see EPSG:4326 for more complete information".
Concretely, does this mean that we could include a couple of EPSGs (not the full database) without concern? In Baremaps, we mainly require EPSG:4326 and EPSG:3857. It not yet clear to me if we can import proj4j and exclude parts of the EPSG database.
>
>> So what would be your advise with proj4(j) and how to handle that kind of implementation in an Apache project?
>>
> EPSG geodetic dataset is not needed when the CRS is built from WKT, GML or some file formats like NetCDF and some (not all) GeoTIFF files. So the approach taken in Apache SIS applications (JavaFX and console application) is to start without EPSG data. Then the first time that an EPSG code is really needed, the application popups a dialog box (in JavaFX) or an interactive prompt (in console application) asking users if they want to download the EPSG geodetic dataset. If users click "yes", the EPSG Terms of Use is shown and the application asks users if they accept. If they confirm, then the EPSG geodetic dataset is downloaded and a database is created locally.
>
> Note that above procedure is needed only for software downloaded directly from Apache servers. Anyone can bundle an application outside Apache which contains the Apache software + EPSG data. The only condition for being allowed to do that is to include the EPSG Terms of Use (in addition of Apache license) in the application bundle.
In the context of a cli, I would prefer to find a solution where the user is not prompted to accept the terms of the EPSG license. For instance, could we have a command (e.g. ./baremaps non-free) that downloads these jars in the lib folder?
>> Officially it is running on MIT license.
>>
> Yes, this issue has been raised on their mailing list. PROJ maintainers have choose to ignore. Some of them pretended that EPSG Terms of Use do not apply to them anymore because they converted the PostgreSQL scripts to SQLite scripts. I'm not a lawyer, but I think that this argument has zero chance to hold in court. For now they rely on the fact that IOGP (the owner of EPSG database) does not want to upset the geodetic community.
I think the Eclipse Foundation did an exception for the locationtech group. According to their FAQ [2] they asked the permission to redistribute some pervasive dependencies in geospatial projects: “We can re-distribute the binaries for certain components. We do have explicit permission to re-distribute Geotools, JTS, and GEOS.”.
[2] https://wiki.eclipse.org/LocationTech/FAQ#Q:_I.27d_like_to_host_my_project_here.2C_using_another_license_not_mentioned
Re: Draft of the release process
Posted by Martin Desruisseaux <ma...@geomatys.com>.
Hello Florian
Le 22/11/2022 à 18:44, Florian Micklich a écrit :
> I didn't know that proj4j itself uses EPSG data.
>
The CSV files in [1] are derived from EPSG tables. The file structures
are different than EPSG database schema, but the data that they contain
are EPSG data. The same information is also provided in WKT format in
[2] (this file somewhat duplicates the CSV tables). Some peoples on PROJ
mailing list have argued that rewriting EPSG data in a different format
free them from EPSG licensing terms, but this is not true. Translating a
software in a different language is a derivative work and is still
subject to the licensing terms of the copyright owner. In the same way
that translating a book does not suppress the rights of the original
author on the translated book. In the case of EPSG data, IOGP allows us
to reformat the data in a different way, but under some conditions
specified in their Terms of Use. And doing so does not give us the right
to erase the EPSG terms of use from the derivative work.
The value of EPSG geodetic dataset is not in the database schema
(converting that to a different data structure is the easy part), but in
the data themselves. Creating a new "EPSG-like" dataset from scratch
would be a huge effort, well beyond our capability. We would have to
visit all national mapping agencies in the world, ask them to provide
documentation about the Coordinate Reference System they have defined
(including historical ones), consolidate all those information in some
tables, etc. So the CSV files [1] in PROJ4 contain an enormous amount of
EPSG work, even if the data structures are different.
[1]https://github.com/locationtech/proj4j/tree/master/src/main/resources/proj4
[2]https://raw.githubusercontent.com/locationtech/proj4j/master/src/main/resources/proj4/wkt/epsg.properties
> Until now I thought it would be a good alternative lib besides EPSG
> approach.
>
When creating a Coordinate Reference System (CRS) from an EPSG code such
as "EPSG:4326", we are using the EPSG geodetic dataset. I think that if
a software contains e.g. 10 EPSG codes (compared to the ~6000
definitions in the full dataset), it could be considered Fair Use. But
including the full dataset requires that we comply with EPSG Terms Of Use.
However if CRS are created from user-supplied WKT strings (or PROJ4
parameter strings), then no EPSG code is involved. The WKT definition
string may contain an AUTHORITY["EPSG", 4326] element, but it only means
"see EPSG:4326 for more complete information".
> So what would be your advise with proj4(j) and how to handle that kind
> of implementation in an Apache project?
>
EPSG geodetic dataset is not needed when the CRS is built from WKT, GML
or some file formats like NetCDF and some (not all) GeoTIFF files. So
the approach taken in Apache SIS applications (JavaFX and console
application) is to start without EPSG data. Then the first time that an
EPSG code is really needed, the application popups a dialog box (in
JavaFX) or an interactive prompt (in console application) asking users
if they want to download the EPSG geodetic dataset. If users click
"yes", the EPSG Terms of Use is shown and the application asks users if
they accept. If they confirm, then the EPSG geodetic dataset is
downloaded and a database is created locally.
Note that above procedure is needed only for software downloaded
directly from Apache servers. Anyone can bundle an application outside
Apache which contains the Apache software + EPSG data. The only
condition for being allowed to do that is to include the EPSG Terms of
Use (in addition of Apache license) in the application bundle.
> Officially it is running on MIT license.
>
Yes, this issue has been raised on their mailing list. PROJ maintainers
have choose to ignore. Some of them pretended that EPSG Terms of Use do
not apply to them anymore because they converted the PostgreSQL scripts
to SQLite scripts. I'm not a lawyer, but I think that this argument has
zero chance to hold in court. For now they rely on the fact that IOGP
(the owner of EPSG database) does not want to upset the geodetic community.
Martin
Re: Draft of the release process
Posted by Florian Micklich <mi...@apache.org>.
Hi all,
Hi Martin :)
I didn't know that proj4j itself uses EPSG data. Until now I thought it would be a good alternative lib besides EPSG approach.
So what would be your advise with proj4(j) and how to handle that kind of implementation in an Apache project?
Officially it is running on MIT license.
Greeting
Florian
Am Montag, dem 21.11.2022 um 10:39 +0100 schrieb Martin Desruisseaux:
> Hello Bertil<br>
>
> One thing to keep in mind (maybe not to be resolved in this release, but <br>
> before graduation). Proj4J (and its parent PROJ project) includes EPSG <br>
> data, so the use of Proj4J dependency implies accepting EPSG terms of <br>
> use [1]. PROJ claims to be under MIT license, but in my understanding <br>
> this is copyright infringement. PROJ license should be MIT + EPSG terms <br>
> of use.<br>
>
> EPSG terms of use is a Category X license for Apache. The use of EPSG <br>
> data has been discussed in Apache Legal [2][3]. The conclusion was that <br>
> an Apache project can include EPSG data if they are optional (i.e. the <br>
> application can still do useful work without them), the main release <br>
> does not contain them, and the data are available in a separated <br>
> download prominently labelled as under a more restrictive license than <br>
> Apache.<br>
>
> Martin<br>
>
> [1][https://epsg.org/terms-of-use.html](https://epsg.org/terms-of-use.html)<br>
> [2][https://issues.apache.org/jira/browse/LEGAL-183](https://issues.apache.org/jira/browse/LEGAL-183)<br>
> [3][https://issues.apache.org/jira/browse/LEGAL-347](https://issues.apache.org/jira/browse/LEGAL-347)<br>
Re: Draft of the release process
Posted by Martin Desruisseaux <ma...@geomatys.com>.
Hello Bertil
One thing to keep in mind (maybe not to be resolved in this release, but
before graduation). Proj4J (and its parent PROJ project) includes EPSG
data, so the use of Proj4J dependency implies accepting EPSG terms of
use [1]. PROJ claims to be under MIT license, but in my understanding
this is copyright infringement. PROJ license should be MIT + EPSG terms
of use.
EPSG terms of use is a Category X license for Apache. The use of EPSG
data has been discussed in Apache Legal [2][3]. The conclusion was that
an Apache project can include EPSG data if they are optional (i.e. the
application can still do useful work without them), the main release
does not contain them, and the data are available in a separated
download prominently labelled as under a more restrictive license than
Apache.
Martin
[1]https://epsg.org/terms-of-use.html
[2]https://issues.apache.org/jira/browse/LEGAL-183
[3]https://issues.apache.org/jira/browse/LEGAL-347