You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Vladimir Sitnikov <si...@gmail.com> on 2019/07/01 11:20:43 UTC
Preview repository for Calcite and Calcite Avatica releases
Hi,
I see Calcite/Avatica "release vote" mails use links like the following:
> You can read the release notes here:
> https://github.com/apache/calcite-avatica
/blob/branch-avatica-1.15/site/_docs/history.md
That works, however
1) GitHub formatting differs from
http://calcite.apache.org/avatica/docs/history.html
2) It might be a good idea to share reports for review as well.
For instance,
2.1) RAT report
2.2) OWASP report
2.3) JavaDoc preview (how do we review JavaDoc otherwise?)
2.4) LICENSE files (e.g. to review third-party license updates)
I suggest we create a Git repository for that purpose (single repository to
host pages of Calcite and Avatica).
The idea is we enable GitHub pages for that repository, so we could use it
as a site preview.
We would likely want to put robots.txt there to prevent that site from
appearing in Google searches for Calcite.
Note: I don't suggest to put "source" code there, rather I suggest we just
push generated site/reports under gh-pages branch.
As a plus, we could push reports like RAT, OWASP and things like that for
release review purposes.
The process would be to build release artifacts as usual, then push site
and reports to that preview repository.
I think it would simplify release review, and it would probably attract
people to validate releases (or provide suggestions re calcite.apache.org)
Any thoughts? Objections?
Vladimir
Re: Preview repository for Calcite and Calcite Avatica releases
Posted by Vladimir Sitnikov <si...@gmail.com>.
Michael>Sounds like a great idea to me. I assume the Javadoc there is for
the
current snapshot?
Javadocs are built during the release candidate preparation.
Here's Git history:
https://github.com/apache/jmeter-site-preview/commits/gh-pages
I have just created JMeter RC2, and you can see the diff between release
candidates:
https://github.com/apache/jmeter-site-preview/commit/478f5a6b97d0b7f936ca899525477aad0e7033b4
As I was preparing the mail, I thought the links should probably contain
"git tag name"
so we could evaluate/compare multiple versions of the site across release
candidates.
Vladimir
Re: Preview repository for Calcite and Calcite Avatica releases
Posted by Michael Mior <mm...@apache.org>.
Sounds like a great idea to me. I assume the Javadoc there is for the
current snapshot?
--
Michael Mior
mmior@apache.org
Le lun. 29 juil. 2019 à 18:06, Vladimir Sitnikov
<si...@gmail.com> a écrit :
>
> Here's "site preview" repository for Apache JMeter.
> I think it does simplify the review at virtually zero cost.
>
> Any thoughts?
>
> --- cut ---
>
> You can read the New and Noteworthy section with some screenshots to
> illustrate improvements and full list of changes at:
> https://apache.github.io/jmeter-site-preview/site/changes.html
>
> RAT report:
> https://apache.github.io/jmeter-site-preview/rat/rat-report.txt
>
> Site preview is here:
> https://apache.github.io/jmeter-site-preview/site/
>
> JavaDoc API preview is here:
> https://apache.github.io/jmeter-site-preview/site/api
>
> --- cut ---
>
> Vladimir
Re: Preview repository for Calcite and Calcite Avatica releases
Posted by Vladimir Sitnikov <si...@gmail.com>.
Here's "site preview" repository for Apache JMeter.
I think it does simplify the review at virtually zero cost.
Any thoughts?
--- cut ---
You can read the New and Noteworthy section with some screenshots to
illustrate improvements and full list of changes at:
https://apache.github.io/jmeter-site-preview/site/changes.html
RAT report:
https://apache.github.io/jmeter-site-preview/rat/rat-report.txt
Site preview is here:
https://apache.github.io/jmeter-site-preview/site/
JavaDoc API preview is here:
https://apache.github.io/jmeter-site-preview/site/api
--- cut ---
Vladimir
Re: Preview repository for Calcite and Calcite Avatica releases
Posted by Vladimir Sitnikov <si...@gmail.com>.
Julian>People reviewing the release can run those same reports
Julian>Those reports don’t necessarily need to be published.
Publishing the reports would simplify the review.
Of course people might ignore those reports, however the availability of
the reports does help with the review.
Vladimir
Re: Preview repository for Calcite and Calcite Avatica releases
Posted by Julian Hyde <jh...@apache.org>.
The web site process is not broken.
There are some minor licensing bugs. Let’s fix them. Let’s talk about how we can improve the process. The existence of bugs doesn’t mean the process is broken.
The release manager must review the licensing/security reports. People reviewing the release can run those same reports (e.g. people run RAT today on release artifacts). Those reports don’t necessarily need to be published.
Julian
> On Jul 2, 2019, at 12:09 PM, Vladimir Sitnikov <si...@gmail.com> wrote:
>
> Julian>And if it ain’t broke, don’t fix it.
>
> It is broken. Both Avatica and Calcite releases violate ASF licensing
> policy (at least as per https://www.apache.org/legal/resolved.html definition
> of that)
>
> Julian>This change would be adding another thing for people voting on
> releases to review
>
> Do you suggest people should NOT review licensing/security reports for ASF
> releases?
>
> Vladimir
Re: Preview repository for Calcite and Calcite Avatica releases
Posted by Vladimir Sitnikov <si...@gmail.com>.
Julian>And if it ain’t broke, don’t fix it.
It is broken. Both Avatica and Calcite releases violate ASF licensing
policy (at least as per https://www.apache.org/legal/resolved.html definition
of that)
Julian>This change would be adding another thing for people voting on
releases to review
Do you suggest people should NOT review licensing/security reports for ASF
releases?
Vladimir
Re: Preview repository for Calcite and Calcite Avatica releases
Posted by Julian Hyde <jh...@apache.org>.
When I said “is there a significant problem” I meant an existing problem. I don’t think there is an existing problem. And if it ain’t broke, don’t fix it.
This change would be adding another thing for people voting on releases to review. I don’t want to add that burden.
Julian
> On Jul 2, 2019, at 11:32 AM, Vladimir Sitnikov <si...@gmail.com> wrote:
>
> Michael>We regularly force push to the Calcite repo, so this is not a
> Michael>restriction we face.
>
> I suggest we just continue with that (fix small typos via amend/rebase)
>
> Michael>The two repos are already separate so I don't think this is a big
> Michael>departure
>
> How are you going to check(test) cross-references between Avatica and
> Calcite pages then?
>
>
> Juiian>At a time when our release managers are more burdened than ever.
>
> My experience with pgjdbc shows that the more automated the release is the
> simpler the role of a release manager is.
>
> Julian>Is there a significant problem where we publish a release and the
> site is screwed up?
>
> That is not a problem at all for Gradle-based approach.
> On top of that, Calcite/Avatica is not using GitHub pages yet, so there's
> nothing to rollback.
>
> Vladimir
Re: Preview repository for Calcite and Calcite Avatica releases
Posted by Vladimir Sitnikov <si...@gmail.com>.
Michael>We regularly force push to the Calcite repo, so this is not a
Michael>restriction we face.
I suggest we just continue with that (fix small typos via amend/rebase)
Michael>The two repos are already separate so I don't think this is a big
Michael>departure
How are you going to check(test) cross-references between Avatica and
Calcite pages then?
Juiian>At a time when our release managers are more burdened than ever.
My experience with pgjdbc shows that the more automated the release is the
simpler the role of a release manager is.
Julian>Is there a significant problem where we publish a release and the
site is screwed up?
That is not a problem at all for Gradle-based approach.
On top of that, Calcite/Avatica is not using GitHub pages yet, so there's
nothing to rollback.
Vladimir
Re: Preview repository for Calcite and Calcite Avatica releases
Posted by Julian Hyde <jh...@apache.org>.
Sigh. More process, more technology. At a time when our release managers are more burdened than ever.
Is there a significant problem where we publish a release and the site is screwed up?
If so, let’s roll back the change to go to GitHub pages. When we generated the site locally using Jekyll there was opportunity to generate the site and review before publishing.
If not, let’s stay as we are. If we notice problems after the release we can fix them within the hour.
Julian
> On Jul 2, 2019, at 10:50 AM, Michael Mior <mm...@apache.org> wrote:
>
> Comments inline.
>
> Le mar. 2 juil. 2019 à 16:17, Vladimir Sitnikov
> <sitnikov.vladimir@gmail.com <ma...@gmail.com>> a écrit :
>>
>> Micael>How so? Just delete the branch or rebase to delete the commits. I
>> fail
>> to see the complexity here.
>>
>> The complexity is not technical. It is INFRA who prohibits rebases.
>> That is why I suggest we don't put "build artifacts" into the main source
>> repositories.
>> In the worst case we just throw away "preview repo".
>>
>> Here is a recent relevant INFRA ticket:
>> https://issues.apache.org/jira/browse/INFRA-18499?focusedCommentId=16865680&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16865680
>>
>> TL;DR there was <<history overwrite in Apache source repositories requires "
>> vp-infra@apache.org or our infra admin escalation">>
>> They denied to remove third-party jar files from the repository (e.g.
>> lib/js_rhino1_6R5.jar), and I did not escalate that because, well, I don't
>> want spend time there.
>>
>
> We regularly force push to the Calcite repo, so this is not a
> restriction we face. That said, I don't have any major problems with
> creating a separate repo.
>
>> Michael>Why do we need both to be in the same repository if we're just
>> using it for testing?
>>
>> I just assume they both could share some resources (e.g. css? images?) and
>> have cross-links (e.g. Calcite site points to Avatica pages).
>> So I thought it would be just simpler to put everything into a single repo
>> so it looks closer to the production site.
>>
>> Do you suggest to have different "test" sites for Calcite and Avatica?
>>
>
> The two repos are already separate so I don't think this is a big
> departure. As above though, I'm not strictly opposed to a separate
> repo.
>
>> Michael>With GitHub pages, published content is stored on a
>> Michael>separate gh-pages branch.
>>
>> I'm used to `git clone https://github.com/organization/repo.git` <https://github.com/organization/repo.git%60>
>> It clones all the branches and tags.
>> Apparently gp-pages would be there as well.
>> Do you think everybody is cloning git repositories in a different way?
>>
>
> My mistake, this is a fair concern.
>
>> Vladimir
Re: Preview repository for Calcite and Calcite Avatica releases
Posted by Michael Mior <mm...@apache.org>.
Comments inline.
Le mar. 2 juil. 2019 à 16:17, Vladimir Sitnikov
<si...@gmail.com> a écrit :
>
> Micael>How so? Just delete the branch or rebase to delete the commits. I
> fail
> to see the complexity here.
>
> The complexity is not technical. It is INFRA who prohibits rebases.
> That is why I suggest we don't put "build artifacts" into the main source
> repositories.
> In the worst case we just throw away "preview repo".
>
> Here is a recent relevant INFRA ticket:
> https://issues.apache.org/jira/browse/INFRA-18499?focusedCommentId=16865680&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16865680
>
> TL;DR there was <<history overwrite in Apache source repositories requires "
> vp-infra@apache.org or our infra admin escalation">>
> They denied to remove third-party jar files from the repository (e.g.
> lib/js_rhino1_6R5.jar), and I did not escalate that because, well, I don't
> want spend time there.
>
We regularly force push to the Calcite repo, so this is not a
restriction we face. That said, I don't have any major problems with
creating a separate repo.
> Michael>Why do we need both to be in the same repository if we're just
> using it for testing?
>
> I just assume they both could share some resources (e.g. css? images?) and
> have cross-links (e.g. Calcite site points to Avatica pages).
> So I thought it would be just simpler to put everything into a single repo
> so it looks closer to the production site.
>
> Do you suggest to have different "test" sites for Calcite and Avatica?
>
The two repos are already separate so I don't think this is a big
departure. As above though, I'm not strictly opposed to a separate
repo.
> Michael>With GitHub pages, published content is stored on a
> Michael>separate gh-pages branch.
>
> I'm used to `git clone https://github.com/organization/repo.git`
> It clones all the branches and tags.
> Apparently gp-pages would be there as well.
> Do you think everybody is cloning git repositories in a different way?
>
My mistake, this is a fair concern.
> Vladimir
Re: Preview repository for Calcite and Calcite Avatica releases
Posted by Vladimir Sitnikov <si...@gmail.com>.
Micael>How so? Just delete the branch or rebase to delete the commits. I
fail
to see the complexity here.
The complexity is not technical. It is INFRA who prohibits rebases.
That is why I suggest we don't put "build artifacts" into the main source
repositories.
In the worst case we just throw away "preview repo".
Here is a recent relevant INFRA ticket:
https://issues.apache.org/jira/browse/INFRA-18499?focusedCommentId=16865680&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16865680
TL;DR there was <<history overwrite in Apache source repositories requires "
vp-infra@apache.org or our infra admin escalation">>
They denied to remove third-party jar files from the repository (e.g.
lib/js_rhino1_6R5.jar), and I did not escalate that because, well, I don't
want spend time there.
Michael>Why do we need both to be in the same repository if we're just
using it for testing?
I just assume they both could share some resources (e.g. css? images?) and
have cross-links (e.g. Calcite site points to Avatica pages).
So I thought it would be just simpler to put everything into a single repo
so it looks closer to the production site.
Do you suggest to have different "test" sites for Calcite and Avatica?
Michael>With GitHub pages, published content is stored on a
Michael>separate gh-pages branch.
I'm used to `git clone https://github.com/organization/repo.git`
It clones all the branches and tags.
Apparently gp-pages would be there as well.
Do you think everybody is cloning git repositories in a different way?
Vladimir
Re: Preview repository for Calcite and Calcite Avatica releases
Posted by Michael Mior <mm...@apache.org>.
>1) I guess we'd better have a single repository for both Avatica and Calcite
Why do we need both to be in the same repository if we're just using
it for testing?
>2) Pushing site/reports to Git repository (e.g. apache/calcite.git) would count towards repository size , thus it would impact every users of Avatica/Calcite sources
Only if people are fetching all branches, which I don't think is the
common case. With GitHub pages, published content is stored on a
separate gh-pages branch.
>removing old commits from regular source repositories is very complicated.
How so? Just delete the branch or rebase to delete the commits. I fail
to see the complexity here.
>I guess GitHub pages is the most simple and well-understood approach.
I was agreeing with your GitHub pages suggestion, just using the existing repo.
--
Michael Mior
mmior@apache.org
Le lun. 1 juil. 2019 à 14:35, Vladimir Sitnikov
<si...@gmail.com> a écrit :
>
> Michael>Why not just use GitHub pages
> Michael>on the existing repositories?
>
> 1) I guess we'd better have a single repository for both Avatica and Calcite
> 2) Pushing site/reports to Git repository (e.g. apache/calcite.git) would
> count towards repository size, thus it would impact every users of
> Avatica/Calcite sources
>
> Typically we don't want to store build artifacts side by side with the
> sources.
>
> If we use a dedicated repository, then we can throw it away if it becomes
> too big for some reason.
> On the other hand removing old commits from regular source repositories is
> very complicated.
>
> I guess GitHub pages is the most simple and well-understood approach. It
> works, and it has little damage scope.
>
> Vladimir
Re: Preview repository for Calcite and Calcite Avatica releases
Posted by Vladimir Sitnikov <si...@gmail.com>.
Michael>Why not just use GitHub pages
Michael>on the existing repositories?
1) I guess we'd better have a single repository for both Avatica and Calcite
2) Pushing site/reports to Git repository (e.g. apache/calcite.git) would
count towards repository size, thus it would impact every users of
Avatica/Calcite sources
Typically we don't want to store build artifacts side by side with the
sources.
If we use a dedicated repository, then we can throw it away if it becomes
too big for some reason.
On the other hand removing old commits from regular source repositories is
very complicated.
I guess GitHub pages is the most simple and well-understood approach. It
works, and it has little damage scope.
Vladimir
Re: Preview repository for Calcite and Calcite Avatica releases
Posted by Michael Mior <mm...@apache.org>.
I'm not opposed to the idea of publishing these things although I'm
not sure we need a separate repository. Why not just use GitHub pages
on the existing repositories?
--
Michael Mior
mmior@apache.org
Le lun. 1 juil. 2019 à 07:21, Vladimir Sitnikov
<si...@gmail.com> a écrit :
>
> Hi,
>
> I see Calcite/Avatica "release vote" mails use links like the following:
>
> > You can read the release notes here:
> > https://github.com/apache/calcite-avatica
> /blob/branch-avatica-1.15/site/_docs/history.md
>
> That works, however
> 1) GitHub formatting differs from
> http://calcite.apache.org/avatica/docs/history.html
> 2) It might be a good idea to share reports for review as well.
> For instance,
> 2.1) RAT report
> 2.2) OWASP report
> 2.3) JavaDoc preview (how do we review JavaDoc otherwise?)
> 2.4) LICENSE files (e.g. to review third-party license updates)
>
>
> I suggest we create a Git repository for that purpose (single repository to
> host pages of Calcite and Avatica).
> The idea is we enable GitHub pages for that repository, so we could use it
> as a site preview.
> We would likely want to put robots.txt there to prevent that site from
> appearing in Google searches for Calcite.
>
> Note: I don't suggest to put "source" code there, rather I suggest we just
> push generated site/reports under gh-pages branch.
> As a plus, we could push reports like RAT, OWASP and things like that for
> release review purposes.
>
> The process would be to build release artifacts as usual, then push site
> and reports to that preview repository.
>
> I think it would simplify release review, and it would probably attract
> people to validate releases (or provide suggestions re calcite.apache.org)
>
> Any thoughts? Objections?
>
> Vladimir