[GitHub] [incubator-superset] betodealmeida opened a new pull request #7416: Scheduling queries from SQL Lab

[GitBox <gi...@apache.org>]

betodealmeida opened a new pull request #7416: Scheduling queries from SQL Lab
URL: https://github.com/apache/incubator-superset/pull/7416
 
 
   ### CATEGORY
   
   Choose one
   
   - [ ] Bug Fix
   - [X] Enhancement (new features, refinement)
   - [ ] Refactor
   - [ ] Add tests
   - [ ] Build / Development Environment
   - [ ] Documentation
   
   ### SUMMARY
   <!--- Describe the change below, including rationale and design decisions -->
   
   This PR introduces a lightweight way of scheduling queries in SQL Lab. If the feature flag `SCHEDULED_QUERIES` is enabled with proper configuration, a button called "Schedule Query" will show up in SQL Lab. The button allows queries to be saved with extra metadata that allows an external scheduler to run it periodically by polling the `/savedqueryviewapi/api/read` endpoint.
   
   The sample configuration can be changed or expanded to support different metadata needed, depending on the scheduler. We tested it with [Apache Airflow](https://airflow.apache.org/) at Lyft successfully.
   
   ### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
   <!--- Skip this if not applicable -->
   
   <img width="1698" alt="Screen Shot 2019-04-30 at 1 31 14 PM" src="https://user-images.githubusercontent.com/1534870/56991363-55a3de80-6b4c-11e9-848b-71b6348ff378.png">
   <img width="1698" alt="Screen Shot 2019-04-30 at 1 31 46 PM" src="https://user-images.githubusercontent.com/1534870/56991381-59cffc00-6b4c-11e9-8f93-0da899648174.png">
   
   This will generate the following payload in `savedqueryviewapi/api/read`:
   
   ```json
   {
     ...
     "result": [
       ...
       {
         "description": null, 
         "extra_json": "{\"schedule_info\":{\"output_table\":\"beto_output_table\",\"start_date\":\"2019-04-30T16:00:00.000Z\",\"end_date\":\"2019-05-30T16:00:00.000Z\",\"schedule_interval\":\"@daily\",\"dependencies\":[\"hive://schema.table\"]}}", 
         "id": 4, 
         "label": "Untitled Query", 
         "schema": "null", 
         "sql": "SELECT *\nFROM\nWHERE", 
         "sqlalchemy_uri": "sqlite:////Users/bdealmeida/.superset/unittests.db", 
         "user_email": "admin@fab.org"
       }
     ],
   }
   ```
   
   ### TEST PLAN
   <!--- What steps should be taken to verify the changes -->
   
   Tested end-to-end at Lyft with a Hive query.
   
   ### ADDITIONAL INFORMATION
   <!--- Check any relevant boxes with "x" -->
   <!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->
   - [ ] Has associated issue:
   - [X] Changes UI
   - [ ] Requires DB Migration.
   - [ ] Confirm DB Migration upgrade and downgrade tested.
   - [X] Introduces new feature or API
   - [ ] Removes existing feature or API
   
   ### REVIEWERS
   
   @mistercrunch @datability-io @DiggidyDave @khtruong @argentfalcon

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org