[jira] [Commented] (MESOS-9601) Persist `QuotaConfig`s in the registry.

["Meng Zhu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MESOS-9601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876515#comment-16876515 ] 

Meng Zhu commented on MESOS-9601:
---------------------------------

{noformat}
commit 3720e4cf5f7cb0d8e98afacea39528bd41c767b4
Author: Meng Zhu <mz...@mesosphere.io>
Date:   Fri Jun 28 14:16:00 2019 -0700

    Updated registry operation `UpdateQuota` to persist `QuotaConfig`.

    The new operations will mutate the `quota_configs` field in
    the registry to persist `QuotaConfigs` configured by the new
    `UPDATE_QUOTA` call as well as the legacy `SET_QUOTA` and
    `REMOVE_QUOTA` calls.

    The operation removes any entries in the legacy `quotas` field
    with the same role name. In addition, it also adds/removes the
    minimum capability `QUOTA_V2` accordingly: if `quota_configs`
    is empty the capability will be removed otherwise it will
    be added.

    This operation replaces the `REMOVE_QUOTA` operation.

    Also fixed/disabled affected tests.

    Review: https://reviews.apache.org/r/70951

commit c82847ad1b8d3760d34ee1e8869c2b7286ccfaa1
Author: Meng Zhu <mz...@mesosphere.io>
Date:   Fri Jun 28 14:15:02 2019 -0700

    Added helpers to add and remove master minimum capabilities.

    Also added a TODO about refactoring the helpers.

    Review: https://reviews.apache.org/r/70972

commit f37250f53e75e0442aed2f61bbedbc9b068821d5
Author: Meng Zhu <mz...@mesosphere.io>
Date:   Tue Jun 25 18:07:29 2019 -0700

    Added a registry field for `QuotaConfig`.

    A new field called `quota_configs` is added to persist the
    quota configurations of the cluster. This replaces the old
    `quotas` field which is deprecated and will be removed
    in Mesos 2.0.

    When users upgrade to Mesos 1.9, `quotas` will be preserved
    and recovered and `quota_configs` will be empty. As users
    configures new quotas, whether through the new `UPDATE_QUOTA`
    call or the deprecated `SET_QUTOA` call, the configured quotas
    will be persisted into the `quota_configs` field along with the
    `QUOTA_V2` minimum capability. The capability is removed only
    if `quota_configs` becomes empty again. If a role already has an
    entry in the old `quotas` field, it will be removed from `quotas`.
    In other words, once upgraded, `quotas` will still be preserved
    and honored, but it will never grow. Instead it will gradually
    shrink as the roles' quotas get updated or removed.

    Review: https://reviews.apache.org/r/70950

commit 0bc857d672189605f83acb7ef57bce89b141ba72
Author: Meng Zhu <mz...@mesosphere.io>
Date:   Tue Jun 25 15:19:44 2019 -0700

    Added master minimum capability `QUOTA_V2`.

    This adds a new enum for the revamped quota feature
    in the master. When quota is configured in Mesos 1.9
    or higher, the quota configurations will be persisted
    into the `quota_configs` field in the registry. And
    the `QUOTA_V2` minimum capability will be added to the
    registry as well. This will prevent any master downgrades
    until `quota_configs` becomes empty. This can be done by
    setting the quota of the roles listed in `quota_configs`
    back to the default (no guarantees and no limits).

    Note, since at the moment of adding this patch, the master
    is not yet capable of handling the new quota API. The
    `capability` is not added to the `MASTER_CAPABILITIES`.
    That should be done later together with the patches that
    enables master for handling the new quota calls.

    Review: https://reviews.apache.org/r/70949
{noformat}

> Persist `QuotaConfig`s in the registry.
> ---------------------------------------
>
>                 Key: MESOS-9601
>                 URL: https://issues.apache.org/jira/browse/MESOS-9601
>             Project: Mesos
>          Issue Type: Task
>            Reporter: Meng Zhu
>            Assignee: Meng Zhu
>            Priority: Major
>              Labels: resource-management
>
> We need to persist the new `QuotaConfig` in the registry.
> One thing to note is, the old masters only support quota guarantee which also servers as limits implicitly. Once new masters start to support both guarantees and limits, there is no safe downgrade path without altering the cluster behavior (if the new quota semantics are used). Thus, we need to ensure that alerts are given if such downgrades are attempted.
> To this end, if the quota is configured after this change, a new minimum capability `QUOTA_V2` will be persisted to the registry along with the new `QuotaConfig` message. Thanks to the minimum capability check, old masters (that do not possess the `QUOTA_V2` capability) will refuse to start in this case and we will print out suggestions to the operator.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)