You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@superset.apache.org by vi...@apache.org on 2026/10/23 04:48:31 UTC
(superset-kubernetes-operator) branch main updated: docs: fix failing examples and env var reference (#475)
This is an automated email from the ASF dual-hosted git repository.
villebro pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/superset-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new 5082162 docs: fix failing examples and env var reference (#475)
5082162 is described below
commit 5082162c5b7b081c405180c36b841bdd70ef487a
Author: Younsung Lee <cy...@kakao.com>
AuthorDate: Fri Oct 23 13:48:26 2026 +0900
docs: fix failing examples and env var reference (#475)
* docs: fix failing examples and env var reference
Several examples in the docs and samples fail when users copy them.
The getting started guide used the lean 6.1.0 image with a psycopg2
URI. The lean image has no database drivers, so the migrate task
fails with "No module named 'psycopg2'". The guide now uses the
6.1.0-dev image, which ships the driver, and the installation guide
and production sample explain that the lean image needs a driver.
The production sample set runAsNonRoot without runAsUser. The image
declares a named user, so the kubelet refuses to start the container.
The sample now sets a numeric UID and GID. The rotation example and
the dev mode example now set the required image field, and the
security context example no longer enables readOnlyRootFilesystem,
which makes the web server and Celery beat crash.
The reserved env var table listed SUPERSET_WEBSERVER_PORT, which is a
line in superset_config.py and not an env var. It also missed several
env vars the operator sets, including the Gunicorn ones. The table now
lists them and explains that operator env vars override user env vars
with the same name. The Celery example set
CeleryConfig.worker_prefetch_multiplier, which has no effect because
the operator always passes --prefetch-multiplier on the worker command
line. The example now uses celeryWorker.celery.prefetchMultiplier.
Signed-off-by: younsl <cy...@kakao.com>
* docs(config): show read-only root filesystem with an emptyDir at /tmp
The security context section told users to keep readOnlyRootFilesystem
unset or false, which made it sound like a read-only root filesystem
cannot work. The failures only come from writes to /tmp, so mounting an
emptyDir there is enough. The section now shows that setup instead.
On Kind, a Superset with readOnlyRootFilesystem set to true and an
emptyDir at /tmp ran migrate and init, and the web server, Celery
worker, and Celery beat all became ready. The stale mention of /app and
the redundant readOnlyRootFilesystem false line are removed.
Signed-off-by: younsl <cy...@kakao.com>
---------
Signed-off-by: younsl <cy...@kakao.com>
---
.../superset_v1alpha1_superset_dev_full.yaml | 4 +-
.../samples/superset_v1alpha1_superset_prod.yaml | 6 +++
docs/architecture/overview.md | 2 +-
docs/getting-started.md | 4 +-
docs/user-guide/configuration.md | 44 ++++++++++++++++++----
docs/user-guide/installation.md | 2 +
docs/user-guide/lifecycle.md | 2 +
7 files changed, 51 insertions(+), 13 deletions(-)
diff --git a/config/samples/superset_v1alpha1_superset_dev_full.yaml b/config/samples/superset_v1alpha1_superset_dev_full.yaml
index 581d324..97878c9 100644
--- a/config/samples/superset_v1alpha1_superset_dev_full.yaml
+++ b/config/samples/superset_v1alpha1_superset_dev_full.yaml
@@ -116,7 +116,7 @@ spec:
# readOnlyRootFilesystem: false # Superset needs /tmp and /app writes
# Shared Python config (applies to all Python components). WebDriver/screenshot
- # settings power thumbnails and alerts & reports; CeleryConfig.imports/prefetch
+ # settings power thumbnails and alerts & reports; CeleryConfig.imports/acks
# are shared Celery app behavior layered onto the operator-rendered CeleryConfig.
config: |
# --- Reports & thumbnails: WebDriver / screenshot settings ---
@@ -135,7 +135,6 @@ spec:
"superset.tasks.thumbnails",
"superset.tasks.cache",
)
- CeleryConfig.worker_prefetch_multiplier = 1
CeleryConfig.task_acks_late = False
webServer:
@@ -156,6 +155,7 @@ spec:
replicas: 2
celery:
preset: balanced
+ prefetchMultiplier: 1
# Singleton scheduler. Beat-only settings live here so schedule edits roll just
# the Beat Deployment. beat_schedule mutates the operator-rendered CeleryConfig.
diff --git a/config/samples/superset_v1alpha1_superset_prod.yaml b/config/samples/superset_v1alpha1_superset_prod.yaml
index a59c633..cfd2f37 100644
--- a/config/samples/superset_v1alpha1_superset_prod.yaml
+++ b/config/samples/superset_v1alpha1_superset_prod.yaml
@@ -16,6 +16,9 @@
# Production-mode sample. Secrets are referenced via *From fields.
# See docs/user-guide/installation.md for full setup instructions.
#
+# The lean image (e.g. 6.1.0) has no database drivers. Point spec.image at a
+# build that adds psycopg2 before running migrations against PostgreSQL.
+#
# Prerequisites:
# kubectl create secret generic superset-secret \
# --from-literal=secret-key="$(openssl rand -base64 42)"
@@ -70,6 +73,9 @@ spec:
podTemplate:
podSecurityContext:
runAsNonRoot: true
+ # Numeric UID required: the image declares a named user (`USER superset`).
+ runAsUser: 1000
+ runAsGroup: 1000
seccompProfile:
type: RuntimeDefault
container:
diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md
index 399b400..308aeab 100644
--- a/docs/architecture/overview.md
+++ b/docs/architecture/overview.md
@@ -213,7 +213,7 @@ ROW_LIMIT = 10000
CELERY_ANNOTATIONS = {"tasks.add": {"rate_limit": "10/s"}}
```
-Note: All operator-managed settings (`SECRET_KEY`, `SQLALCHEMY_DATABASE_URI`, web server port) are rendered into the config file from operator-internal `SUPERSET_OPERATOR__*` env vars. Both passthrough and structured metastore modes render `SQLALCHEMY_DATABASE_URI` from `SUPERSET_OPERATOR__DB_URI` (passthrough) or `SUPERSET_OPERATOR__DB_*` (structured).
+Note: `SECRET_KEY` and `SQLALCHEMY_DATABASE_URI` are rendered into the config file from operator-internal `SUPERSET_OPERATOR__*` env vars. The web server port is written as a fixed `SUPERSET_WEBSERVER_PORT = 8088` assignment. Both passthrough and structured metastore modes render `SQLALCHEMY_DATABASE_URI` from `SUPERSET_OPERATOR__DB_URI` (passthrough) or `SUPERSET_OPERATOR__DB_*` (structured).
| Config section | WebServer | CeleryWorker | CeleryBeat | CeleryFlower | McpServer |
|---|---|---|---|---|---|
diff --git a/docs/getting-started.md b/docs/getting-started.md
index eade607..8b07fa4 100644
--- a/docs/getting-started.md
+++ b/docs/getting-started.md
@@ -41,7 +41,7 @@ Replace `<version>` with a published chart version (e.g., `0.1.0`); see [Downloa
## 2. Deploy Superset
-Create a minimal Superset instance in dev mode (inline credentials for simplicity):
+Create a minimal Superset instance in dev mode (inline credentials for simplicity). The example uses the `-dev` image variant because it ships the `psycopg2` PostgreSQL driver. The lean image (e.g. `6.1.0`) has no database drivers, so migrations would fail:
```bash
kubectl apply -f - <<'EOF'
@@ -52,7 +52,7 @@ metadata:
spec:
environment: Development
image:
- tag: "6.1.0"
+ tag: "6.1.0-dev"
secretKey: thisIsNotSecure_changeInProduction!
metastore:
uri: postgresql+psycopg2://superset:superset@postgres:5432/superset
diff --git a/docs/user-guide/configuration.md b/docs/user-guide/configuration.md
index 3e1c20d..5cda2a6 100644
--- a/docs/user-guide/configuration.md
+++ b/docs/user-guide/configuration.md
@@ -34,6 +34,8 @@ The `environment` field controls validation strictness (enforced by [CEL](https:
```yaml
spec:
environment: Development
+ image:
+ tag: "6.1.0-dev"
secretKey: thisIsNotSecure_changeInProduction!
metastore:
uri: postgresql+psycopg2://superset:superset@postgres:5432/superset
@@ -319,13 +321,20 @@ The operator sets certain env vars automatically based on the CR spec. These are
|---|---|---|---|
| `SUPERSET_OPERATOR__INSTANCE_NAME` | Operator-internal | Operator (from parent CR `metadata.name`) | Parent CR name; available for use in raw `spec.config` (e.g. instance-scoped Celery queue names) |
| `SUPERSET_OPERATOR__SECRET_KEY` | Operator-internal | Operator (from `secretKey` or `secretKeyFrom`) | Superset session signing key |
+| `SUPERSET_OPERATOR__PREVIOUS_SECRET_KEY` | Operator-internal | Operator (from `previousSecretKey` or `previousSecretKeyFrom`) | Previous signing key used during secret key rotation |
| `SUPERSET_OPERATOR__DB_URI` | Operator-internal | Operator (from `metastore.uri` or `metastore.uriFrom`) | Full database connection URI |
| `SUPERSET_OPERATOR__DB_HOST`, `SUPERSET_OPERATOR__DB_PORT`, `SUPERSET_OPERATOR__DB_NAME` | Operator-internal | Operator (structured metastore) | Database connection fields |
| `SUPERSET_OPERATOR__DB_USER`, `SUPERSET_OPERATOR__DB_PASS` | Operator-internal | Operator (from metastore structured fields or `passwordFrom`) | Database credentials |
| `SUPERSET_OPERATOR__VALKEY_HOST`, `SUPERSET_OPERATOR__VALKEY_PORT` | Operator-internal | Operator (from `valkey`) | Valkey connection fields |
+| `SUPERSET_OPERATOR__VALKEY_USER` | Operator-internal | Operator (from `valkey.username` or `valkey.usernameFrom`) | Valkey ACL username |
| `SUPERSET_OPERATOR__VALKEY_PASS` | Operator-internal | Operator (from `valkey.password` or `valkey.passwordFrom`) | Valkey password |
+| `SUPERSET_OPERATOR__FLOWER_URL_PREFIX` | Operator-internal | Operator (from `celeryFlower.service.gatewayPath`, default `/flower`) | Flower URL prefix |
| `SUPERSET_OPERATOR__FORCE_RELOAD` | Operator-internal | Operator (from `spec.forceReload`) | Triggers rolling restart |
-| `SUPERSET_WEBSERVER_PORT` | Standard | Rendered in config | Web server port (8088) |
+| `SERVER_WORKER_AMOUNT`, `SERVER_THREADS_AMOUNT`, `SERVER_WORKER_CLASS`, `GUNICORN_TIMEOUT`, `GUNICORN_KEEPALIVE`, `WORKER_MAX_REQUESTS`, `WORKER_MAX_REQUESTS_JITTER`, `SERVER_LIMIT_REQUEST_LINE`, `SERVER_LIMIT_REQUEST_FIELD_SIZE`, `GUNICORN_LOGLEVEL` | Standard | Operator (from `webServer.gunicorn`, web server only, skipped when `preset: disabled`) | Gunicorn settings read by `run-server.sh` |
+
+Operator-set env vars are merged after `podTemplate.container.env`, so a user-defined env var with the same name is overridden and the operator logs `operator env var overrides user-specified value`. Use the matching CRD field instead, for example `webServer.gunicorn.workers` rather than `SERVER_WORKER_AMOUNT`.
+
+The web server port is not an env var. The operator writes `SUPERSET_WEBSERVER_PORT = 8088` directly into the web server's `superset_config.py`.
The operator does **not** set `PYTHONPATH` — it relies on the upstream Superset image's default (which already includes `/app/pythonpath`, where the operator mounts the rendered `superset_config.py`). Custom images must preserve this entry on `PYTHONPATH` for the rendered config to be picked up.
@@ -341,8 +350,8 @@ The operator does **not** set `PYTHONPATH` — it relies on the upstream Superse
| `SUPERSET_OPERATOR__DB_URI` | Set (plain text) | Set (`valueFrom`) | — |
| `SUPERSET_OPERATOR__DB_HOST` | — | — | Set |
| `SUPERSET_OPERATOR__DB_PORT` | — | — | Set |
-| `SUPERSET_OPERATOR__DB_NAME` | — | — | Set (if `database` provided) |
-| `SUPERSET_OPERATOR__DB_USER` | — | — | Set (if `username` provided) |
+| `SUPERSET_OPERATOR__DB_NAME` | — | — | Set (from `database` or `databaseFrom`) |
+| `SUPERSET_OPERATOR__DB_USER` | — | — | Set (from `username` or `usernameFrom`) |
| `SUPERSET_OPERATOR__DB_PASS` | — | — | Set (plain text or `valueFrom`) |
In both passthrough and structured modes, the operator renders `SQLALCHEMY_DATABASE_URI` in `superset_config.py` from the operator-internal env vars. Passthrough mode reads from `SUPERSET_OPERATOR__DB_URI`, while structured mode assembles an f-string URI from the `SUPERSET_OPERATOR__DB_*` env vars.
@@ -351,8 +360,9 @@ In both passthrough and structured modes, the operator renders `SQLALCHEMY_DATAB
| Env Var | Set when |
|---|---|
-| `SUPERSET_OPERATOR__VALKEY_HOST` | Always (from `valkey.host`) |
-| `SUPERSET_OPERATOR__VALKEY_PORT` | Always (from `valkey.port`, default 6379) |
+| `SUPERSET_OPERATOR__VALKEY_HOST` | Always (from `valkey.host` or `valkey.hostFrom`) |
+| `SUPERSET_OPERATOR__VALKEY_PORT` | Always (from `valkey.port` or `valkey.portFrom`, default 6379) |
+| `SUPERSET_OPERATOR__VALKEY_USER` | `valkey.username` or `valkey.usernameFrom` is set |
| `SUPERSET_OPERATOR__VALKEY_PASS` | `valkey.password` (dev, plain text) or `valkey.passwordFrom` (prod, `valueFrom`) |
## Custom Python Config
@@ -507,7 +517,9 @@ Because assigning `CELERY_CONFIG` replaces Superset's own Celery config class, p
spec:
valkey:
host: valkey.default.svc
- celeryWorker: {}
+ celeryWorker:
+ celery:
+ prefetchMultiplier: 1
celeryBeat:
config: |
from celery.schedules import crontab
@@ -533,7 +545,6 @@ spec:
"superset.tasks.cache",
"superset.tasks.slack",
)
- CeleryConfig.worker_prefetch_multiplier = 1
CeleryConfig.task_acks_late = False
CeleryConfig.task_annotations = {
"sql_lab.get_sql_results": {
@@ -542,6 +553,8 @@ spec:
}
```
+Set worker command-line options such as prefetch multiplier, concurrency, and pool through `spec.celeryWorker.celery` (see [Celery Worker Configuration](#celery-worker-configuration)), not through `CeleryConfig.worker_*` settings. The operator passes them as `celery worker` flags, and those flags take precedence over the matching `CeleryConfig` values.
+
#### Without Valkey
When `spec.valkey` is unset, the operator emits no `CeleryConfig` class. Provide both connectivity and app behavior manually via `spec.config`:
@@ -815,7 +828,6 @@ spec:
fsGroup: 1000
container:
securityContext:
- readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
```
@@ -830,6 +842,22 @@ spec:
runAsNonRoot: true
```
+If you enable `readOnlyRootFilesystem`, mount an `emptyDir` at `/tmp`. The web server and Celery beat write temporary files there, and fail to start without it:
+
+```yaml
+spec:
+ podTemplate:
+ volumes:
+ - name: tmp
+ emptyDir: {}
+ container:
+ securityContext:
+ readOnlyRootFilesystem: true
+ volumeMounts:
+ - name: tmp
+ mountPath: /tmp
+```
+
## Autoscaling (HPA)
```yaml
diff --git a/docs/user-guide/installation.md b/docs/user-guide/installation.md
index bcb4a03..0d0d9f1 100644
--- a/docs/user-guide/installation.md
+++ b/docs/user-guide/installation.md
@@ -176,6 +176,8 @@ EOF
The operator will create parent-owned lifecycle task Jobs to perform database migration and initialization, then create the web server Deployment, Service, and ConfigMap. Check task status with `kubectl describe superset my-superset`.
+The official lean Superset image (e.g. `6.1.0`) has no database drivers. Set `image` to a build that includes the driver for your connection URI, such as `psycopg2` for `postgresql+psycopg2`. See [Metastore](configuration.md#metastore) for driver details.
+
## 4. Watch it come up
```bash
diff --git a/docs/user-guide/lifecycle.md b/docs/user-guide/lifecycle.md
index 4059718..86aef82 100644
--- a/docs/user-guide/lifecycle.md
+++ b/docs/user-guide/lifecycle.md
@@ -396,6 +396,8 @@ kind: Superset
metadata:
name: my-superset
spec:
+ image:
+ tag: "6.1.0"
secretKeyFrom:
name: superset-secret-v2
key: secret-key