[GitHub] [incubator-superset] vnourdin commented on issue #7416: feat: Scheduling queries from SQL Lab

[GitBox <gi...@apache.org>]

vnourdin commented on issue #7416: feat: Scheduling queries from SQL Lab
URL: https://github.com/apache/incubator-superset/pull/7416#issuecomment-490135639
 
 
   Hello!
   
   I'm working at M6 (french TV group) and we're working on a job that look a bit like scheduled queries, so I was wondering if we could exchange words on how things are planned in SuperSet and see if we could help each other.
   
   Our job is an "Aggregator" which basically run aggregation queries on daily basis. We structured it similarly you did with scheduled queries: a config file (currently hand made), an Airflow dag which create other dags based on config files, and a Scala/Spark job to run the query (and monitor number of lines to Grafana).
   We currently work with Hive as db, what is the target for SuperSet?
   
   You talked about Airflow integration, how did you managed that part at Lyft? Our need seems more specific but I think we can help.
   
   On the config part, there is some info missing for us (full/incremental query, partitions of input tables to put sensors on it, partitions of output table), is this part "adjustable"?
   
   Thanks for your time, I hope we'll can help and share our work!
   
   Have a nice day,
   Valentin

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org