You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "mikebridge-gfm (via GitHub)" <gi...@apache.org> on 2026/04/16 16:10:47 UTC
[I] [SIP] Soft Delete for Charts, Dashboards, and Datasets [superset]
mikebridge-gfm opened a new issue, #39410:
URL: https://github.com/apache/superset/issues/39410
*Please make sure you are familiar with the SIP process documented [here](https://github.com/apache/superset/issues/5602). The SIP will be numbered by a committer upon acceptance.*
## [SIP] Soft Delete for Charts, Dashboards, and Datasets
### Motivation
When a user accidentally deletes a chart, dashboard, or dataset in Superset, the data is permanently removed from the database. There is no way to recover — the only option is to restore from a database backup, which requires admin intervention and may not be available.
This is a common source of support tickets and user frustration. Competing platforms (Tableau, Looker, Metabase) offer recycle bin / trash functionality that allows users to recover recently deleted objects. The lack of soft delete in Superset is a gap that affects user confidence, increases support burden, and creates risk for organizations managing critical dashboards and datasets.
### Proposed Change
Replace hard deletes with **soft deletes** for charts (`slices`), dashboards (`dashboards`), and datasets (`tables`). Soft delete sets a `deleted_at` timestamp on the row instead of removing it. Soft-deleted objects are automatically excluded from all standard API responses and UI views.
#### Architecture
**`SoftDeleteMixin`** — A SQLAlchemy mixin added to `Slice`, `Dashboard`, and `SqlaTable` that provides:
- A nullable, indexed `deleted_at` column
- `soft_delete()` and `restore()` instance methods
- An `is_deleted` hybrid property and `not_deleted()` class-level filter
**Global ORM filter** — A `do_orm_execute` event listener registered at app initialization that automatically appends `WHERE deleted_at IS NULL` to every ORM SELECT targeting a soft-delete-enabled model. This uses SQLAlchemy's recommended pattern for soft deletes, ensuring that no existing query path accidentally returns deleted rows without modifying every caller.
The filter can be bypassed by setting the `skip_visibility_filter` execution option on a query. This is used by restore commands and admin tooling (e.g., import/export).
**DAO routing** — `BaseDAO.delete()` checks whether the model includes `SoftDeleteMixin`. If so, it calls `soft_delete()` (sets `deleted_at`). Otherwise, it calls `hard_delete()` (the original `session.delete()` behaviour). This means all existing delete commands (single and bulk) automatically gain soft-delete behaviour without code changes.
**Restore commands** — New `RestoreChartCommand`, `RestoreDashboardCommand`, and `RestoreDatasetCommand` classes clear the `deleted_at` timestamp, making the object active again. These follow the existing command pattern with ownership and permission validation.
#### Cascade behaviour
Soft-deleting a dashboard or dataset does **not** cascade to dependent charts. Charts remain active and visible. This matches user expectations — removing a dashboard should not silently break charts that may be used elsewhere.
When a chart is soft-deleted, its rows in the `dashboard_slices` association table are hard-deleted to prevent stale references from appearing in dashboard layouts.
#### Import/export
The import pipeline uses `skip_visibility_filter` to query all rows (including soft-deleted ones) during UUID lookups and overwrite checks. This prevents duplicate key violations when re-importing an object that was previously soft-deleted. If a matching soft-deleted object is found during import, it is hard-deleted before the new import is inserted.
### New or Changed Public Interfaces
#### Changed endpoints (behaviour change, same request/response format)
| Endpoint | Change |
|----------|--------|
| `DELETE /api/v1/chart/<pk>` | Sets `deleted_at` instead of removing the row |
| `DELETE /api/v1/dashboard/<pk>` | Sets `deleted_at` instead of removing the row |
| `DELETE /api/v1/dataset/<pk>` | Sets `deleted_at` instead of removing the row |
| Bulk delete endpoints | Same soft-delete behaviour |
#### New endpoints
| Method | Path | Description |
|--------|------|-------------|
| `POST` | `/api/v1/chart/<pk>/restore` | Restore a soft-deleted chart |
| `POST` | `/api/v1/dashboard/<pk>/restore` | Restore a soft-deleted dashboard |
| `POST` | `/api/v1/dataset/<pk>/restore` | Restore a soft-deleted dataset |
Restore endpoints require the same permissions as the delete endpoint (editor access or admin).
#### New list endpoint parameter
| Endpoint | Parameter | Description |
|----------|-----------|-------------|
| `GET /api/v1/dataset/` | `include_deleted=true` | Include soft-deleted datasets in the response |
#### Model changes
- `Slice`, `Dashboard`, and `SqlaTable` gain `SoftDeleteMixin`, adding a `deleted_at` column, `is_deleted` property, and `soft_delete()`/`restore()` methods.
- No changes to existing column definitions or relationships.
### New dependencies
None. The implementation uses only SQLAlchemy's built-in `do_orm_execute` event system.
### Migration Plan and Compatibility
**Database migration:** A single Alembic migration adds a nullable `deleted_at` column and index to the `slices`, `dashboards`, and `tables` tables. The migration is backwards-compatible — the column defaults to `NULL` (not deleted), so all existing rows remain active.
**Rollback:** The downgrade drops the `deleted_at` column and index. Any rows that were soft-deleted (had `deleted_at` set) will be restored to active visibility, since the column is simply removed.
**API compatibility:** External API callers see no change in request/response format. The same endpoints return the same status codes. The only behavioural difference is that `DELETE` sets a timestamp rather than removing the row.
**Direct database access:** External tooling that queries the database directly (bypassing the API) may see "deleted" rows that are still present with `deleted_at IS NOT NULL`. These consumers should add `WHERE deleted_at IS NULL` to their queries if they want to exclude soft-deleted objects.
**Permanent deletion:** A future phase will add a scheduled task to permanently purge rows that have been soft-deleted for longer than a configurable retention period.
### Rejected Alternatives
1. **Trash/recycle bin as a separate table** — Moving deleted objects to a separate `_trash` table was considered. This keeps the primary tables clean but adds complexity: foreign keys break, the schema must be kept in sync, and restore requires re-inserting rows with the same PKs. The `deleted_at` column approach is simpler, preserves all relationships, and is a well-established pattern in the industry.
2. **Event-sourced deletion** — Recording delete events in an append-only log and reconstructing state. This is architecturally elegant but far more complex than needed for the use case, and would require significant changes to Superset's read path.
3. **Deletion only behind a feature flag** — Gating soft delete behind a feature flag was considered for gradual rollout. However, soft delete is purely additive (no data loss, same API contract), and the global ORM filter ensures correctness without opt-in. A feature flag would add configuration surface without meaningful risk reduction.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] [SIP] Soft Delete for Charts, Dashboards, and Datasets [superset]
Posted by "mikebridge (via GitHub)" <gi...@apache.org>.
mikebridge commented on issue #39410:
URL: https://github.com/apache/superset/issues/39410#issuecomment-4271143357
Using the wrong credentials; I will recreate this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] [SIP] Soft Delete for Charts, Dashboards, and Datasets [superset]
Posted by "mikebridge-gfm (via GitHub)" <gi...@apache.org>.
mikebridge-gfm closed issue #39410: [SIP] Soft Delete for Charts, Dashboards, and Datasets
URL: https://github.com/apache/superset/issues/39410
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org