[DISCUSS] feedback on new experimental Cluster Management Service

[Owen Nichols <on...@pivotal.io>]

Starting with discussions over a year ago, the Geode community recognized the need for a new API for cluster management and cluster configuration management.  Currently this requires a mix of properties, xml files, and gfsh commands.  The envisioned “v2” API would expose identical REST and Java APIs and unify the configuration and management experience.

A vast amount of work remains to implement 100% of that vision, but the effort reached an important milestone this week.  As of PR 3827 <https://github.com/apache/geode/pull/3827>, the REST controllers for the handful of commands implemented so far <https://cwiki.apache.org/confluence/display/GEODE/Management+REST+API> are now enabled by default.  The hope is to start gathering feedback on the v2 API vision by making it easier for all users to try it.

Steps have been taken to make it clear that this API could change or be removed in a future release:
- all public classes are annotated @Experimental <https://geode.apache.org/releases/latest/javadoc/org/apache/geode/annotations/Experimental.html>
- all REST endpoints <https://cwiki.apache.org/confluence/display/GEODE/Management+REST+API> contain the word “experimental”
- all REST controllers can be disabled if needed by start locator --J=-Dgemfire.enable-management-rest-service=false

Recent discussion about the upcoming 1.10 release have some people wondering “Is there a better way to get feedback on something new in Geode?”

Here are a few opinions I’ve heard:
- it’s fine the way it is (clearly labeled and has a kill switch just in case)
- this kind of work should be happening on a long-lived feature branch, not on develop. 
- Geode could have a separate “Technology Preview” release track (somehow merging all feature branches).
- it’s open source; let's encourage all contributions and release often. 
- the community could vote that experimental features in Geode releases are ok if and only if they are behind a feature flag which is off by default
- the community could encourage more frequent discussion of new and experimental features so that concerns are raised and mitigated well in advance of the next planned release

This thread is open to discussion both on the new Cluster Management Service itself, as well as the general process by which feedback on new features should be gathered.

Re: [DISCUSS] feedback on new experimental Cluster Management Service

[Jens Deppe <jd...@pivotal.io>]

Thanks for starting this thread Owen! I would, however, request that we
keep the thread to one topic, namely feedback about the Cluster Management
Service.

Some background and motivation for this work:

- The intention of this work came about as a need to provide a 'proper' API
with functionality similar to what *gfsh* provides today - i.e. a way to
create Geode components and to persist the configuration. Currently the
former is possible through the Java API (executed on servers) and gfsh, but
the latter is only possible through gfsh.
- The Java API should be available in whatever context the user was
developing (Geode server, Geode client or plain Java app)

--Jens

On Fri, Jul 26, 2019 at 6:54 PM Owen Nichols <on...@pivotal.io> wrote:

> Starting with discussions over a year ago, the Geode community recognized
> the need for a new API for cluster management and cluster configuration
> management.  Currently this requires a mix of properties, xml files, and
> gfsh commands.  The envisioned “v2” API would expose identical REST and
> Java APIs and unify the configuration and management experience.
>
> A vast amount of work remains to implement 100% of that vision, but the
> effort reached an important milestone this week.  As of PR 3827 <
> https://github.com/apache/geode/pull/3827>, the REST controllers for the
> handful of commands implemented so far <
> https://cwiki.apache.org/confluence/display/GEODE/Management+REST+API>
> are now enabled by default.  The hope is to start gathering feedback on the
> v2 API vision by making it easier for all users to try it.
>
> Steps have been taken to make it clear that this API could change or be
> removed in a future release:
> - all public classes are annotated @Experimental <
> https://geode.apache.org/releases/latest/javadoc/org/apache/geode/annotations/Experimental.html
> >
> - all REST endpoints <
> https://cwiki.apache.org/confluence/display/GEODE/Management+REST+API>
> contain the word “experimental”
> - all REST controllers can be disabled if needed by start locator
> --J=-Dgemfire.enable-management-rest-service=false
>
> Recent discussion about the upcoming 1.10 release have some people
> wondering “Is there a better way to get feedback on something new in Geode?”
>
> Here are a few opinions I’ve heard:
> - it’s fine the way it is (clearly labeled and has a kill switch just in
> case)
> - this kind of work should be happening on a long-lived feature branch,
> not on develop.
> - Geode could have a separate “Technology Preview” release track (somehow
> merging all feature branches).
> - it’s open source; let's encourage all contributions and release often.
> - the community could vote that experimental features in Geode releases
> are ok if and only if they are behind a feature flag which is off by default
> - the community could encourage more frequent discussion of new and
> experimental features so that concerns are raised and mitigated well in
> advance of the next planned release
>
> This thread is open to discussion both on the new Cluster Management
> Service itself, as well as the general process by which feedback on new
> features should be gathered.