[D] Feature Request: Bulk import of all tables/views in a database as Superset datasets, and allow filtering by database when creating charts [superset]

["yixinNB (via GitHub)" <gi...@apache.org>]

GitHub user yixinNB created a discussion: Feature Request: Bulk import of all tables/views in a database as Superset datasets, and allow filtering by database when creating charts

As a Superset 5.0.0 user managing multiple databases, I often face two major pain points:

1. **There’s no easy way to bulk-import datasets.**
   Superset currently requires manually creating each dataset (table or view) through the UI.
   When a database contains dozens or even hundreds of tables, this process becomes extremely time-consuming and error-prone.
   The existing “import/export datasource” YAML or ZIP mechanism is helpful for migration but doesn’t provide an automated way to *scan and register all tables/views in a database* as datasets.

2. **Chart creation lacks database-level filtering.**
   When creating a new chart, the dataset selector lists *all datasets across all databases* in one long dropdown.
   If multiple databases share similar table names or schemas, it becomes difficult to find the correct dataset.
   A database (and optionally schema) filter would make the dataset selection process much faster and less confusing.

---

**Proposed Solution**

1. **Bulk Import Functionality**

   * Add a feature in the UI or API to automatically scan a database (and optionally a schema) and register all tables/views as datasets in one operation.
   * Include configurable options such as:

     * Skip already imported datasets
     * Auto-generate default metrics for numeric columns (e.g., `SUM()`, `AVG()`)
     * Control whether to sync or overwrite existing datasets
   * Support both synchronous and asynchronous execution with progress tracking.

2. **Database Filter in Chart Creation Dialog**

   * In the “Create Chart” or “Choose Dataset” dialog, allow filtering by *database* (and schema).
   * This will significantly improve usability when working with many databases or large Superset installations.

---

**Benefits**

* Saves hours of manual dataset creation and maintenance
* Reduces human errors and improves metadata consistency
* Makes chart creation faster and more intuitive, especially in multi-database environments
* Enables teams to automate dataset management and keep Superset metadata in sync with evolving databases


GitHub link: https://github.com/apache/superset/discussions/35645

----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org

Re: [D] Feature Request: Bulk import of all tables/views in a database as Superset datasets, and allow filtering by database when creating charts [superset]

["Haseeb702 (via GitHub)" <gi...@apache.org>]

GitHub user Haseeb702 added a comment to the discussion: Feature Request: Bulk import of all tables/views in a database as Superset datasets, and allow filtering by database when creating charts

Hi, my team at the University of Toronto are able to pick this work up as a part of our open source group project.

Looking over the proposed scope and steps, I think adding a bulk registration workflow makes sense. Based on the current flow, after selecting a single table in the dataset we have the option to "create and explore". When adding the ability to import datasets in bulk we would let users select multiple tables from the dropdown without retrying it every time.

As an initial option, we would make this a synchronous workflow, and detect when multiple tables are selected via a checklist in the UI. A new query would be made, mimicking the single processing query currently available. Instead of creating multiple charts (as the workflow currently does), the V1 of this feature should just create the datasets, without routing users to the "create chart" page.

Our proposed idea includes:
- UI changes to allow multiselect via a checklist / select all
- New API endpoint that calls the current one repeatedly, with new error handling/validation
- New submit button that shows "create datasets" instead of "create and explore" when selecting multiple tables
- New screen that shows the result of creating multiple datasets (with messages informing users of the number of failures, successes, etc)
- New test files/validations ensuring correct behaviour

As we continue, we may be able to incorporate more of the functionality above as well. This includes progress tracking, auto skip-detection, generating default metrics and options to overwrite/sync existing DBs.

GitHub link: https://github.com/apache/superset/discussions/35645#discussioncomment-19186365

----
This is an automatically sent email for notifications@superset.apache.org.
To unsubscribe, please send an email to: notifications-unsubscribe@superset.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org