[GitHub] [incubator-superset] betodealmeida opened a new pull request #8059: [WIP] Persist SQL Lab state in the backend #8055

[GitBox <gi...@apache.org>]

betodealmeida opened a new pull request #8059: [WIP] Persist SQL Lab state in the backend #8055
URL: https://github.com/apache/incubator-superset/pull/8059
 
 
   ### CATEGORY
   
   Choose one
   
   - [ ] Bug Fix
   - [ ] Enhancement (new features, refinement)
   - [X] Refactor
   - [ ] Add tests
   - [ ] Build / Development Environment
   - [ ] Documentation
   
   ### SUMMARY
   <!--- Describe the change below, including rationale and design decisions -->
   
   This PR implements [SIP-23](https://github.com/apache/incubator-superset/issues/7748), moving the persistence of SQL Lab state from the browser's `localStorage` to the backend.
   
   Currently, the redux store in `localStorage` has the following structure:
   
   ```javascript
   sqlLab: {
     activeSouthPaneTab: "Results",
     alerts: [],
     databases: {...},
     offline: false,
     queries: {...},
     queriesLastUpdate: 1565918142000,
     queryEditors: [...],
     tabHistory: [...],
     tables: [...],
   }
   ```
   
   With this PR, when loading SQL Lab the following keys are loaded from the DB:
   
   - `databases`
   - `queries`
   - `queryEditors` (from the new tables `tab_state_view` and the `table_schema`)
   - `tables` (from the new table `table_schema`)
   
   As the user interacts with SQL Lab, changes to `queryEditors`, `tables` and `queries` are synced to the backend automatically. This includes the SQL being written, the table schemas being displayed (expanded or not), the database/schema selected, the active tab, etc.
   
   For queries, as soon as a new tab is open we create a row in the `query` table to store the query, even if it hasn't run. This and the syncing of `queryEditors` and `tables` is done through new custom views created in this PR, `/tabstateview/` and `/tableschemaview/`.
   
   For results, I changed the backend so that sync queries are also stored in the results backend if one is configured. This allows us to fetch the results when a tab is loaded, without having to store them in the database, since they can be large.
   
   #### Migration
   
   This PR requires a DB migration to set up the initial tables, `tab_state` and `table_schema`. When a user first visits SQL Lab after this migration, the state of the following keys will be a **combination** of the backend (nothing, initially) and their `localStorage`:
   
   - `queryEditors`
   - `tables`
   - `tabHistory`
   
   When `<TabbedSqlEditors>` mounts, it will identify objects in these keys that come from `localStorage`, and migrate them to the backend with a custom action `migrateLocalStorage`. Once one of these objects has been migrated successfully, it is removed from `localStorage`, so that the migration doesn't have to be atomic.
   
   Once everything has been migrated, the `redux` key in `localStorage` is removed.
   
   #### Concurrent access from multiple browser tabs
   
   It's a non-goal in this PR to handle users working on the same query editor from multiple tabs. The current behavior remains: the last update is persisted.
   
   ### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
   <!--- Skip this if not applicable -->
   
   There are no visible changes.
   
   ### TEST PLAN
   <!--- What steps should be taken to verify the changes -->
   
   ### ADDITIONAL INFORMATION
   <!--- Check any relevant boxes with "x" -->
   <!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->
   - [ ] Has associated issue:
   - [ ] Changes UI
   - [X] Requires DB Migration.
   - [X] Confirm DB Migration upgrade and downgrade tested.
   - [X] Introduces new feature or API
   - [ ] Removes existing feature or API
   
   ### REVIEWERS
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org