[GitHub] [superset] sabiroid commented on issue #16209: [SIP-71] API to pass metadata over to Celery workers

[GitBox <gi...@apache.org>]

sabiroid commented on issue #16209:
URL: https://github.com/apache/superset/issues/16209#issuecomment-902117789


   Technically, all the necessary information (though maybe not in the most convenient form) can be found in `flask.g`, `flask.context`, and `flask.request`. At least, so far I haven't encountered any critical information truly missing there.
   
   I believe, we did not alter the Superset code too much (that's why I also want to eventually push this feature upstream so we have the minimal possible fork), we were mostly achieving our goals through passing over our own config file(s).
   
   In our case the authentication is done when the user logs in the Superset, so all synchronous requests are sorted out. But for asynchronous ones we need to pass the token over since the Celery worker wouldn't have any authorization otherwise.
   
   On top of that we also need Celery workers to have additional information about the query they are executing like user name and dashboard/chart ID, so we can set up proper resource management configuration in Presto (so, say, a single really bad dashboard with many unoptimized queries doesn't screw up the whole Presto cluster)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org