[GitHub] [incubator-superset] Pinimo edited a comment on issue #9597: Cache warm-ups never succeed

[GitBox <gi...@apache.org>]

Pinimo edited a comment on issue #9597:
URL: https://github.com/apache/incubator-superset/issues/9597#issuecomment-669896508


   @mukulsaini I have not found the time to address the issue, to my best knowledge it has not been solved yet. If you too find this issue is a real problem, I invite you to talk it over on [Superset's Slack :left_speech_bubble: ](https://join.slack.com/t/apache-superset/shared_invite/enQtNDMxMDY5NjM4MDU0LWJmOTcxYjlhZTRhYmEyYTMzOWYxOWEwMjcwZDZiNWRiNDY2NDUwNzcwMDFhNzE1ZmMxZTZlZWY0ZTQ2MzMyNTU)
   
   Here are a few educated guesses as how to solve the issue:
   
   1. We could work around the auth by... **signing in through a headless browser** in the Celery process. After thinking it over, it seems difficult to me: 
       * the browser software needs to be installed and launched on the Celery worker (looks like an overkill to me)
       * the passphrase and user name need to be handled as secrets
   
   2. Perhaps a better solution would involve **setting up an API server** with its own authentication procedure -- **or a new auth method on the same server**, to allow the Celery worker to perform cache requests:
       * Logging in would require a login token. The token would need to be handed over to the Celery worker on start-up, possibly through an environment variable or shell argument.
       * When the cache warm-up starts, the worker would use the token to authenticate through this special auth channel
       * He would not be granted any rights on the server except performing cache requests, i.e. `POST` requests that only return empty documents (not usable to extract data from the instance).
       * However, we should make sure this would not breach the server's security or reliability. For instance, the token should not be used to flood the server, DB engine and Redis instance (DDOS).
   
   3. Not sure at all about this last idea: we could code a new CLI route to return the chart data. The Celery worker would then execute the CLI (if I'm remembering right, all the configs and docker images are the same). However, that would possibly infringe memory limits for the Celery worker.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org