You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@netbeans.apache.org by Neil C Smith <ne...@apache.org> on 2020/05/30 13:32:05 UTC
[DISCUSS] update centre and plugin portal for dev (master) builds
Hi,
Just bringing the discussion part of
https://github.com/apache/netbeans-website/pull/473 on to here.
Matthias correctly pointed out in the above PR that we've not kept
redirects for the master branch update centre and plugin portal always
in line with the current release branch. Not just for 12.0 - probably
over each release for the last year. This is one reason a current
issue with the plugin portal may have gone unnoticed.
Over the last year or two we've moved to having all inbound links from
the IDE come in under https://netbeans.apache.org/nb. This has a
variety of benefits, one being we can easily manage redirecting things
to the right place in
https://github.com/apache/netbeans-website/blob/master/netbeans.apache.org/src/content/.htaccess
Useful, particularly during transition from Oracle to Apache
infrastructure, to change things without requiring IDE updates.
It has also given us predictable links for update centre and plugin
portal for each release and master, even with underlying changes.
After 12.0 is released we should probably make the following 3
changes, only the third part of which was really cause for discussion
on the PR.
# Stage 1
Change the dev redirects in the .htaccess to -
Redirect 302 /nb/updates/dev/ https://netbeans-vm.apache.org/uc/dev/
Redirect 302 /nb/plugins/dev/ https://plugins.netbeans.apache.org/data/dev/
This will require setting up /dev endpoints for the UC on the VM and
on the plugin portal. Initially as symlinks for 12.0?
# Stage 2
Once the above is working, we can remove a lot of the redirects and
move everything up a level - then we have no need to update the
.htaccess for each release -
Redirect 302 /nb/updates/ https://netbeans-vm.apache.org/uc/
Redirect 302 /nb/plugins/ https://plugins.netbeans.apache.org/data/
We could also just proxy those links from those places without
redirect? Might help with issue of being blocked when processing a
lot of updates?
# Stage 3
We can then make a decision on what update centre and plugin portal
catalogues make sense for dev builds.
IMO, and the bit that really caused discussion, we adjust the latest
master build on Jenkins to serve the NBMs and point at that (via the
VM). I think this is what happened before the Apache transition?
https://builds.apache.org/view/M-R/view/NetBeans/job/netbeans-TLP/job/netbeans/job/master/lastSuccessfulBuild/artifact/
Antonio raised concerns on the PR that infra might not appreciate that
- obviously we don't have to do it, but considering we've been
distributing beta builds direct from Jenkins, it's probably a very
minor increase in bandwidth. Would it actually be useful?
We could also consider a dev / next specific section on the plugin
portal to allow plugin authors to test with master? Or to try new
plugin portal features?
Sorry about the long email about what feel like minor changes, but
given the discussion on the PR, be good to know this is what we're
heading for, or not. Losing track of how much of this has happened by
release manager convention, and how much is an actual plan! :-)
Best wishes,
Neil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: [DISCUSS] update centre and plugin portal for dev (master) builds
Posted by Neil C Smith <ne...@apache.org>.
On Wed, 17 Jun 2020, 21:21 Matthias Bläsing, <mb...@doppel-helix.eu>
wrote:
> I would not use an extra "dev" version, as that will most probably just
> rot.
>
Yes, dev as synonym for next for plugins (not updates) always pointing to
the next release would be great. Having /dev for plugins handled at the
plugin portal end simplifies the redirects.
I assume, that we'd retain the proxy for the 8.2 update center unless
> oracle decommissions the old server?
>
I guess. Removing it for 12.0 was -1'd. It's separate from the standard
templated UC and plugin links anyway.
> > We could also just proxy those links from those places without
> > redirect? Might help with issue of being blocked when processing a
> > lot of updates?
>
> This I don't understand.
>
Sorry! If you try to use the Ant build support, say, for downloading the
platform (via nbms), you'll get blocked by ASF infrastructure before it
completes. This works fine if using the VM link or downloads.a.o directly.
Trying to install a whole cluster via IDE may also be problematic -
something to test.
This might be something we could use the ousol mirrors for. These are
> explicit mirrors, so bandwidth is either allocated to them or spare
> bandwith is used.
>
Maybe. The bandwidth usage will be minimal - probably more to mirror the
nightly build in the first place, given the catalog is almost the only
thing that will be downloaded. Is it worth the effort / complexity to
implement?
Best wishes,
Neil
Re: [DISCUSS] update centre and plugin portal for dev (master)
builds
Posted by Matthias Bläsing <mb...@doppel-helix.eu>.
Hi,
Am Samstag, den 30.05.2020, 14:32 +0100 schrieb Neil C Smith:
>
> # Stage 1
>
> Change the dev redirects in the .htaccess to -
>
> Redirect 302 /nb/updates/dev/ https://netbeans-vm.apache.org/uc/dev/
> Redirect 302 /nb/plugins/dev/
> https://plugins.netbeans.apache.org/data/dev/
>
> This will require setting up /dev endpoints for the UC on the VM and
> on the plugin portal. Initially as symlinks for 12.0?
I agree, that this might be something to be managed in the plugin
portal. I would associate dev with the highest version in the plugin
portal. So we could add 12.1 to plugin portal now and that would be
dev. From a plugin authors perspective, 12.1 and dev would be the
"next" netbeans release.
I would not use an extra "dev" version, as that will most probably just
rot.
> # Stage 2
>
> Once the above is working, we can remove a lot of the redirects and
> move everything up a level - then we have no need to update the
> .htaccess for each release -
>
> Redirect 302 /nb/updates/ https://netbeans-vm.apache.org/uc/
> Redirect 302 /nb/plugins/ https://plugins.netbeans.apache.org/data/
I assume, that we'd retain the proxy for the 8.2 update center unless
oracle decommissions the old server?
> We could also just proxy those links from those places without
> redirect? Might help with issue of being blocked when processing a
> lot of updates?
This I don't understand.
> # Stage 3
>
> We can then make a decision on what update centre and plugin portal
> catalogues make sense for dev builds.
>
> IMO, and the bit that really caused discussion, we adjust the latest
> master build on Jenkins to serve the NBMs and point at that (via the
> VM). I think this is what happened before the Apache transition?
>
> https://builds.apache.org/view/M-R/view/NetBeans/job/netbeans-TLP/job/netbeans/job/master/lastSuccessfulBuild/artifact/
>
> Antonio raised concerns on the PR that infra might not appreciate
> that
> - obviously we don't have to do it, but considering we've been
> distributing beta builds direct from Jenkins, it's probably a very
> minor increase in bandwidth. Would it actually be useful?
This might be something we could use the ousol mirrors for. These are
explicit mirrors, so bandwidth is either allocated to them or spare
bandwith is used.
> We could also consider a dev / next specific section on the plugin
> portal to allow plugin authors to test with master? Or to try new
> plugin portal features?
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: [DISCUSS] update centre and plugin portal for dev (master) builds
Posted by Neil C Smith <ne...@apache.org>.
Hi,
Thanks for thoughts.
On Sat, 30 May 2020 at 17:27, antonio <an...@vieiro.net> wrote:
> As far as I understand the PP2 will remain untouched. This is so because
> URLs are already fixed in code and because NetBeans < 12.0 uses plain
> http for plugins, whereas 12.0+ uses https.
It's only fixed in code for 11.0 and below. I'm fairly sure the JSON
file is accurate for the old builds too thanks to Eric -
https://github.com/apache/netbeans-jenkins-lib/blob/master/meta/netbeansrelease.json
I switched plugins to https and via NAO in 11.1.
As soon as we decide to stop supporting 11.0 we're in a better
position, although that links direct to old plugin portal anyway.
> What about this?
>
> https://NAO/nb/updates/dev -302-> https://PNAO/dev/updates
> https://NAO/nb/plugins/dev -302-> https://PNAO/dev/plugins
> https://NAO/nb/updates -302-> https://PNAO/12.0/updates
> https://NAO/nb/plugins -302-> https://PNAO/12.0/plugins
We already redirect updates via the NetBeans VM - there's probably no
point in hitting the plugin centre for it - the NBMs are actually
downloaded from Apache mirrors for all releases. But the catalog XML
file itself must be on the VM.
Also, the point would be to have updates/12.0, updates/dev, etc. on
the other end so we can blanket redirect nb/plugins/* and nb/updates/*
We only need two redirects then.
> Of course this will require hosting the binaries in PNAO (which may be
> somewhat slow, but faster than hosting them in the Jenkins builds).
Release update NBMs are already served by the Apache mirrors. The key
thing served from Jenkins for dev (as is already the case with all the
12.0 betas) would be the catalog XML for update checking. Unless a
spec version in that is updated, no NBM will be downloaded. It will
be a seldom used testing thing for developers, if it's useful at all?!
And we could just keep the dev UC pointing at the last release 12.0
as we have done. It's just better if we can handle this on the other
end with the others, rather than treat dev differently IMO.
Best wishes,
Neil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: [DISCUSS] update centre and plugin portal for dev (master) builds
Posted by antonio <an...@vieiro.net>.
Hi all,
I regret having to say this whole thing of UC and plugins is quite
complicated for me :-). During the years we have been improving things
(adding https, proxying for the old Oracle hosted update center in
netbeans-vm.apache.org/pluginportal2, etc.)
As far as I understand the PP2 will remain untouched. This is so because
URLs are already fixed in code and because NetBeans < 12.0 uses plain
http for plugins, whereas 12.0+ uses https.
For simplicity, let's abbreviate:
NAO - netbeans.apache.org
PNAO - plugins.netbeans.apache.org (hosted as virtual server in
netbeans-vm.apache.org [1])
What about this?
https://NAO/nb/updates/dev -302-> https://PNAO/dev/updates
https://NAO/nb/plugins/dev -302-> https://PNAO/dev/plugins
https://NAO/nb/updates -302-> https://PNAO/12.0/updates
https://NAO/nb/plugins -302-> https://PNAO/12.0/plugins
Of course this will require hosting the binaries in PNAO (which may be
somewhat slow, but faster than hosting them in the Jenkins builds).
We are already synchronizing javadocs, so there's no reason why we can't
synchronize binaries (note that Jenkins builds don't support rsync,
though, so we'll have to download whole zips, and this may be slow).
What say? Did I understand things correctly? Will this work?
Cheers,
Antonio
[1]
https://github.com/apache/infrastructure-puppet/blob/9bb6860a8f07c3a04e77177ba752012fb0c629d9/data/nodes/netbeans-vm.apache.org.yaml#L207
El 30/5/20 a las 15:32, Neil C Smith escribió:
> Hi,
>
> Just bringing the discussion part of
> https://github.com/apache/netbeans-website/pull/473 on to here.
>
> Matthias correctly pointed out in the above PR that we've not kept
> redirects for the master branch update centre and plugin portal always
> in line with the current release branch. Not just for 12.0 - probably
> over each release for the last year. This is one reason a current
> issue with the plugin portal may have gone unnoticed.
>
> Over the last year or two we've moved to having all inbound links from
> the IDE come in under https://netbeans.apache.org/nb. This has a
> variety of benefits, one being we can easily manage redirecting things
> to the right place in
> https://github.com/apache/netbeans-website/blob/master/netbeans.apache.org/src/content/.htaccess
> Useful, particularly during transition from Oracle to Apache
> infrastructure, to change things without requiring IDE updates.
>
> It has also given us predictable links for update centre and plugin
> portal for each release and master, even with underlying changes.
> After 12.0 is released we should probably make the following 3
> changes, only the third part of which was really cause for discussion
> on the PR.
>
> # Stage 1
>
> Change the dev redirects in the .htaccess to -
>
> Redirect 302 /nb/updates/dev/ https://netbeans-vm.apache.org/uc/dev/
> Redirect 302 /nb/plugins/dev/ https://plugins.netbeans.apache.org/data/dev/
>
> This will require setting up /dev endpoints for the UC on the VM and
> on the plugin portal. Initially as symlinks for 12.0?
>
> # Stage 2
>
> Once the above is working, we can remove a lot of the redirects and
> move everything up a level - then we have no need to update the
> .htaccess for each release -
>
> Redirect 302 /nb/updates/ https://netbeans-vm.apache.org/uc/
> Redirect 302 /nb/plugins/ https://plugins.netbeans.apache.org/data/
>
> We could also just proxy those links from those places without
> redirect? Might help with issue of being blocked when processing a
> lot of updates?
>
> # Stage 3
>
> We can then make a decision on what update centre and plugin portal
> catalogues make sense for dev builds.
>
> IMO, and the bit that really caused discussion, we adjust the latest
> master build on Jenkins to serve the NBMs and point at that (via the
> VM). I think this is what happened before the Apache transition?
>
> https://builds.apache.org/view/M-R/view/NetBeans/job/netbeans-TLP/job/netbeans/job/master/lastSuccessfulBuild/artifact/
>
> Antonio raised concerns on the PR that infra might not appreciate that
> - obviously we don't have to do it, but considering we've been
> distributing beta builds direct from Jenkins, it's probably a very
> minor increase in bandwidth. Would it actually be useful?
>
> We could also consider a dev / next specific section on the plugin
> portal to allow plugin authors to test with master? Or to try new
> plugin portal features?
>
> Sorry about the long email about what feel like minor changes, but
> given the discussion on the PR, be good to know this is what we're
> heading for, or not. Losing track of how much of this has happened by
> release manager convention, and how much is an actual plan! :-)
>
> Best wishes,
>
> Neil
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> For additional commands, e-mail: dev-help@netbeans.apache.org
>
> For further information about the NetBeans mailing lists, visit:
> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists