You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudstack.apache.org by "MitchDrage (via GitHub)" <gi...@apache.org> on 2026/09/14 23:52:50 UTC
[I] Per-bucket object storage credentials with key rotation [cloudstack]
MitchDrage opened a new issue, #14167:
URL: https://github.com/apache/cloudstack/issues/14167
### The required feature described as a wish
## Problem
CloudStack provisions object storage credentials per **account**. The first bucket an account creates provisions one backend identity, its access and secret key are stored against the account, and every later bucket reuses that same pair. There is no API to rotate or regenerate it.
An S3 key often exists to be used outside the platform: by a backup tool, a CI pipeline, a third-party application. Those consumers sit outside the operator's span of control. Sharing one pair across every bucket an account owns means:
- one leaked key exposes every bucket that account owns, not just the one it was issued for;
- a bucket cannot be handed to an external consumer without handing over every other bucket with it;
- each bucket usually serves a different tool, with a different supplier and a different supply-chain risk, so each bucket is properly its own boundary;
- there is no safe way to rotate, because there is only one key and revoking it breaks every consumer at once.
Using a separate CloudStack account per bucket does give distinct keys, but it brings its own resource limits, users and networks, and confuses tenancy with a storage-scoping requirement.
## Proposed change
**A credential per bucket.** Each bucket gets its own backend identity instead of sharing the account's, so a leaked key exposes one bucket.
**Two key slots per credential.** Each credential holds up to two independent key pairs. Rotation is: create a new pair in the free slot while consumers keep using the other, move consumers across, then revoke the old one. The slots are independent rather than a rolling pipeline, which mirrors the two keys of an Azure storage account and the two access keys of an AWS IAM user. Rotating a credential's only active key is safe, because the new key is created before the old one is removed.
**Existing buckets are untouched.** A bucket with no credential of its own behaves exactly as today. The schema addition is additive and nothing is migrated.
## Implementation decisions
I considered two ways to give a bucket its own identity on Ceph RGW.
**Plain RGW users per bucket**
I found that the writer of an object owns the object, so giving an existing bucket a new identity splits ownership between the old account user and the new one. A public bucket policy then fails on the older objects, the old key keeps reaching them, and the admin REST API cannot repair it: it can relink a bucket to another owner, but only the `radosgw-admin bucket chown` command rewrites the objects inside, and CloudStack accesses the gateway over REST and S3 with no shell access.
**RGW accounts (the chosen option)**
Ceph Squid added accounts. An account sits above users, in the same way an AWS account sits above its IAM users. It holds users, each with their own keys and policies, and the account owns the buckets and objects, not the user that created them. This fixes the problem above: ownership no longer depends on which user wrote an object, so a bucket can be given its own identity and a bucket policy applies to everything in it.
Each CloudStack account maps to one RGW account on each object store. The account's existing RGW user becomes the root user of that RGW account. It is the same user, so its keys keep working and its buckets move into the account with it. Each bucket then gets an IAM user inside the account, with a policy limited to that bucket. That user's two access keys are the two key slots.
Objects written before the account was created are still recorded against the original user, and an IAM policy alone cannot read them. To cover this, the bucket policy also grants access to the bucket's IAM user. CloudStack generates that policy together with the bucket's public or private setting.
That introduces some minimum versions. As far as I can see, Cloudian is already using this model, however RGW accounts arrived in Ceph Squid (v19). Older gateways keep today's behaviour and report themselves as unable to support the feature, rather than getting a half-working version of it. Ceph before Squid is already end of life.
## Migration
Adopting an existing account into an RGW account is permanent at the backend, so it is never implicit. An administrator triggers it per account per store, existing buckets and their keys keep working throughout, and each bucket is then moved to its own credential individually. The final step rotates the account key, so that the key shared before the migration no longer opens anything.
## Scope
In scope: Ceph RGW, the per-bucket credential model, two-slot rotation and revocation, the migration path, and the API and UI for all of it.
Out of scope for the first pass: bulk migration of buckets, and implementations for MinIO, Cloudian and other providers. Those providers are unaffected and keep today's behaviour.
## Discussion
Raised on dev@ as "[DISCUSS] Per-bucket credentials and key rotation for object storage".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Per-bucket object storage credentials with key rotation [cloudstack]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #14167:
URL: https://github.com/apache/cloudstack/issues/14167#issuecomment-5681227561
## ๐ฏ Triage report
This is a well-researched design proposal for per-bucket object storage credentials (Ceph RGW), replacing the current per-account shared credential model. It proposes RGW accounts, a two-slot key rotation scheme, and an explicit per-account/per-store migration path. The author states a PR is forthcoming.
### ๐ Assessment
| Dimension | Value | Reasoning |
|---|---|---|
| **Type** | `type:new-feature` | Adds new capability (per-bucket credentials, rotation, migration) rather than fixing a defect. |
| **Component** | `component:secondary-storage`, `component:ceph` | Object storage/bucket credentials are part of secondary/object storage; the proposal is Ceph RGW specific. |
| **Severity** | n/a | Not a bug. |
| **Labels** | type:new-feature, component:secondary-storage, component:ceph | See above. |
| **Coding agent** | Not suitable | Requires architectural/design decisions (RGW accounts model, migration semantics, API/schema design) and community discussion (already raised on dev@) before implementation; not a self-contained, well-defined coding task yet. |
### ๐ Similar issues
No similar or duplicate open issues found via search.
<details><summary>๐ก Notes and suggestions</summary>
- The author has already opened a `[DISCUSS]` thread on dev@; maintainers should confirm the design lands there before a PR is merged, since it introduces a new minimum Ceph version dependency (Squid/v19+) and backend-permanent migration semantics.
- Key design decisions worth maintainer review: RGW-account-per-CloudStack-account mapping, the two-key-slot rotation model, and how bucket policies are generated/maintained for public vs private buckets.
- Consider whether the migration path (per account, per store, irreversible at the backend) needs additional safeguards or dry-run/preview support before implementation.
- Once a PR is submitted, request review from maintainers familiar with the object storage plugin framework (`org.apache.cloudstack.storage.object`).
</details>
> Generated by [Daily Issue Triage](https://github.com/apache/cloudstack/actions/runs/34975614423) ยท sonnet50 86.7K ยท [โท](https://github.com/search?q=repo%3Aapache%2Fcloudstack+%22gh-aw-workflow-call-id%3A+apache%2Fcloudstack%2Fdaily-issue-triage%22&type=issues)
>
<details>
<summary>Add this agentic workflows to your repo</summary>
To install this agentic workflow, run
```
gh aw add githubnext/agentics/workflows/daily-issue-triage.md@d7c1dc4b72b00607a67caaffdcc216cb64379cf9
```
</details>
<!-- gh-aw-agentic-workflow: Daily Issue Triage, engine: copilot, version: 1.0.52, model: claude-sonnet-5, id: 34975614423, workflow_id: daily-issue-triage, run: https://github.com/apache/cloudstack/actions/runs/34975614423 -->
<!-- gh-aw-workflow-call-id: apache/cloudstack/daily-issue-triage -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Per-bucket object storage credentials with key rotation [cloudstack]
Posted by "nvazquez (via GitHub)" <gi...@apache.org>.
nvazquez commented on issue #14167:
URL: https://github.com/apache/cloudstack/issues/14167#issuecomment-6512798390
PR in progress: https://github.com/apache/cloudstack/pull/14170
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Per-bucket object storage credentials with key rotation [cloudstack]
Posted by "MitchDrage (via GitHub)" <gi...@apache.org>.
MitchDrage commented on issue #14167:
URL: https://github.com/apache/cloudstack/issues/14167#issuecomment-5672497063
I've been working on this for a while and a PR is coming soon with more details about the decisions, API and UI changes, etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudstack.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org