You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2021/02/13 09:11:10 UTC

[GitHub] [superset] leocape opened a new pull request #13104: update docs for new alerts and reporting feature

leocape opened a new pull request #13104:
URL: https://github.com/apache/superset/pull/13104


   ### SUMMARY
   Added docs for setting up and configuring the new alerts and reports in V.1.0
   
   ### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
   <!--- Skip this if not applicable -->
   
   ### TEST PLAN
   Using the templates outlined in the doc, run these steps to get the new alerts and reporting feature working. After these steps are completed, you should now be able to create and test alerts and reports
   
   1. Create a new directory and create the Dockerfile
   2. Build the extended image using the Dockerfile
   3. Create the `docker-compose.yaml` file in the same directory
   4. Create a new sub directory called `config`
   5. Create the `superset_config.py` file in the `config` sub directory
   6. Run the image using `docker-compose up` in the same directory as the `docker-compose.py` file
   
   ### ADDITIONAL INFORMATION
   <!--- Check any relevant boxes with "x" -->
   <!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->
   - [ ] Has associated issue:
   - [ ] Changes UI
   - [ ] Requires DB Migration.
   - [ ] Confirm DB Migration upgrade and downgrade tested.
   - [ ] Introduces new feature or API
   - [ ] Removes existing feature or API
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] leocape commented on pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
leocape commented on pull request #13104:
URL: https://github.com/apache/superset/pull/13104#issuecomment-779654430


   > @leocape this is an excellent community effort in documenting a highly requested and very confusing feature set. I'm wondering how much of the config changes can be included in the default superset files (`superset/config.py`, `Dockerfile`, `docker-compose.yml`), so only minimal changes are required to enable this feature for testing. Documentation can then focus more on how to deploy this in prod-like environments (ie, not using docker-compose). Thoughts @dpgaspar? Are there any blockers (licensing) to adding the browser + webdriver to the Dockerfile?
   
   Thanks @nytai with all the help from everyone on the Slack threads. Yes would be great to simplify this doc so that it focuses more/only on the feature related config. I could cut out the superfluous docker-compose and dockerfile if you can include the webdriver (and postgres?) into the default dockerfile?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] vnourdin commented on a change in pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
vnourdin commented on a change in pull request #13104:
URL: https://github.com/apache/superset/pull/13104#discussion_r576214695



##########
File path: docs/src/pages/docs/installation/email_reports.mdx
##########
@@ -6,8 +6,309 @@ index: 10
 version: 1
 ---
 
-## Scheduling and Emailing Reports
+## Alerts and Reports

Review comment:
       I'm not really comfy about putting the Alerts & Reports doc in a page named `email-reports`, as this doc concern alerts too, and not only emails but also Slack :shrug: 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] Kingflyinger edited a comment on pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
Kingflyinger edited a comment on pull request #13104:
URL: https://github.com/apache/superset/pull/13104#issuecomment-787055210


   `version: '3.6'
   services:
     redis-superset:
       image: redis:6.0.9-buster
       restart: on-failure`
   redis-superset should be replaced by redis,as well as superset_config.py.
   or change the depends_on .
   
   and after done all, what`s the account and password? I tried admin/admin, wrong


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] vnourdin commented on a change in pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
vnourdin commented on a change in pull request #13104:
URL: https://github.com/apache/superset/pull/13104#discussion_r576674097



##########
File path: docs/src/pages/docs/installation/email_reports.mdx
##########
@@ -6,8 +6,305 @@ index: 10
 version: 1
 ---
 
-## Scheduling and Emailing Reports
+## Alerts and Reports
+(version 1.0.1 and above)
+
+Users can configure automated alerts and reports to send charts and dashboards to an email recipient or Slack channel.
+
+- Alerts are sent when a specified condition is passed
+- Reports are sent on a specified schedule 
+
+### Turning on Alerts and reports
+Alerts and reports are not turned on by default. They are currently behind a feature flag, and require some additional services and configurations.
+
+#### Requirements:
+
+- `Dockerfile`
+	- webdriver to run a headless browser (for taking screenshots of the charts and dahboards)
+- `docker-compose.yaml`
+	- redis message broker
+	- replacing SQLlite DB with Postgres DB
+	- celery worker
+	- celery beat
+- `superset_config.py`
+	- feature flag turned to True
+	- all configs as outlined in the template below
+- At least one of these is needed to send alerts and reports:
+	- (optional) SMTP server for sending email
+	- (optional) Slack app integration for sending to Slack channels
+ 
+#### Summary of steps to turn on alerts and reporting:
+
+Using the templates below,
+1. Create a new directory and create the Dockerfile
+2. Build the extended image using the Dockerfile
+3. Create the `docker-compose.yaml` file in the same directory
+4. Create a new sub directory called `config`
+5. Create the `superset_config.py` file in the `config` sub directory
+6. Run the image using `docker-compose up` in the same directory as the `docker-compose.py` file
+ 
+(note: v 1.0.1 is current at time of writing, you can change the version number to the latest version if a newer version is available)
+### Dockerfile
+
+A webdriver (and headless browser) is needed to capture screenshots of the charts and dashboards which are then sent to the recipient. As the base image does not have a webdriver installed by default, we need to extend the base image and install the webdriver (this template uses the Chrome webdriver). We are also adding in connectors for Mysql and Postgres, as well as Redis and Flower (Flower and Mysql are optional depending on your requirements)
+
+You can extend the image by running this Docker build command from the directory that contains the Dockerfile:
+`docker build -t superset-1.0.1-extended -f Dockerfile`
+
+Config for `Dockerfile`:
+```docker
+FROM apache/superset:1.0.1
+USER root
+RUN apt update
+RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb && \
+    apt install -y --no-install-recommends ./google-chrome-stable_current_amd64.deb && \
+    wget https://chromedriver.storage.googleapis.com/88.0.4324.96/chromedriver_linux64.zip && \
+    unzip chromedriver_linux64.zip && \
+    chmod +x chromedriver && \
+    mv chromedriver /usr/bin && \
+    apt autoremove -yqq --purge && \
+    apt clean && \
+    rm -f google-chrome-stable_current_amd64.deb chromedriver_linux64.zip

Review comment:
       It works with `geckodriver` too, we can adapt the doc to propose both approaches.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] vnourdin commented on a change in pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
vnourdin commented on a change in pull request #13104:
URL: https://github.com/apache/superset/pull/13104#discussion_r576675251



##########
File path: docs/src/pages/docs/installation/email_reports.mdx
##########
@@ -6,8 +6,309 @@ index: 10
 version: 1
 ---
 
-## Scheduling and Emailing Reports
+## Alerts and Reports

Review comment:
       I agree, renaming and keep the old approach at the end seems the best!




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] nytai commented on pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
nytai commented on pull request #13104:
URL: https://github.com/apache/superset/pull/13104#issuecomment-779462357


   @leocape this is an excellent community effort in documenting a highly requested and very confusing feature set. I'm wondering how much of the config changes can be included in the default superset files (`superset/config.py`, `Dockerfile`, `docker-compose.yml`), so only minimal changes are required to enable this feature for testing. Documentation can then focus more on how to deploy this in prod-like environments (ie, not using docker-compose). Thoughts @dpgaspar? Are there any blockers (licensing) to adding the browser + webdriver to the Dockerfile?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] aliJabra edited a comment on pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
aliJabra edited a comment on pull request #13104:
URL: https://github.com/apache/superset/pull/13104#issuecomment-785185040


   @vnourdin - thanks for the documentation! I'm running superset locally installed via pip (1.0.1). Is docker also a preferred way to run the 'backend' for generating alerts and reports? 
   I've superset_config.py properly configured (i think), can see the alerts and reports, slack & email stmp tokens etc. set. I can create alerts and reports, but they never trigger.. 
   
   I have a celery worker: 
   `celery worker --app=superset.tasks.celery_app:app --pool=prefork -O fair -c 4 -E  --concurrency=500 &
   `
   and a beat process running in bg:
   `celery beat --app=superset.tasks.celery_app:app --pidfile /tmp/celerybeat.pid --schedule /tmp/celerybeat-schedule &
   `
   
   I've used the default celery settings as per config.py
   
   `class CeleryConfig:  # pylint: disable=too-few-public-methods
       BROKER_URL = "sqla+sqlite:///celerydb.sqlite"
       CELERY_IMPORTS = ("superset.sql_lab", "superset.tasks")
       CELERY_RESULT_BACKEND = "db+sqlite:///celery_results.sqlite"
       CELERYD_LOG_LEVEL = "DEBUG"
       CELERYD_PREFETCH_MULTIPLIER = 1
       CELERY_ACKS_LATE = True
       CELERY_ANNOTATIONS = {
           "sql_lab.get_sql_results": {"rate_limit": "100/s"},
           "email_reports.send": {
               "rate_limit": "1/s",
               "time_limit": 600,
               "soft_time_limit": 600,
               "ignore_result": True,
           },
       }
       CELERYBEAT_SCHEDULE = {
           "email_reports.schedule_hourly": {
               "task": "email_reports.schedule_hourly",
               "schedule": crontab(minute=1, hour="*"),
           },
           "reports.scheduler": {
               "task": "reports.scheduler",
               "schedule": crontab(minute="*", hour="*"),
           },
           "reports.prune_log": {
               "task": "reports.prune_log",
               "schedule": crontab(minute=0, hour=0),
           },
       }
   
   
   CELERY_CONFIG = CeleryConfig
   `
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] aliJabra removed a comment on pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
aliJabra removed a comment on pull request #13104:
URL: https://github.com/apache/superset/pull/13104#issuecomment-785185040


   @vnourdin - thanks for the documentation! I'm running superset locally installed via pip (1.0.1). Is docker also a preferred way to run the 'backend' for generating alerts and reports? 
   I've superset_config.py properly configured (i think), can see the alerts and reports, slack & email stmp tokens etc. set. I can create alerts and reports, but they never trigger.. 
   
   I have a celery worker: 
   `celery worker --app=superset.tasks.celery_app:app --pool=prefork -O fair -c 4 -E  --concurrency=500 &
   `
   and a beat process running in bg:
   `celery beat --app=superset.tasks.celery_app:app --pidfile /tmp/celerybeat.pid --schedule /tmp/celerybeat-schedule &
   `
   
   I've used the default celery settings as per config.py
   
   `
   ```
   class CeleryConfig:  # pylint: disable=too-few-public-methods
       BROKER_URL = "sqla+sqlite:///celerydb.sqlite"
       CELERY_IMPORTS = ("superset.sql_lab", "superset.tasks")
       CELERY_RESULT_BACKEND = "db+sqlite:///celery_results.sqlite"
       CELERYD_LOG_LEVEL = "DEBUG"
       CELERYD_PREFETCH_MULTIPLIER = 1
       CELERY_ACKS_LATE = True
       CELERY_ANNOTATIONS = {
           "sql_lab.get_sql_results": {"rate_limit": "100/s"},
           "email_reports.send": {
               "rate_limit": "1/s",
               "time_limit": 600,
               "soft_time_limit": 600,
               "ignore_result": True,
           },
       }
       CELERYBEAT_SCHEDULE = {
           "email_reports.schedule_hourly": {
               "task": "email_reports.schedule_hourly",
               "schedule": crontab(minute=1, hour="*"),
           },
           "reports.scheduler": {
               "task": "reports.scheduler",
               "schedule": crontab(minute="*", hour="*"),
           },
           "reports.prune_log": {
               "task": "reports.prune_log",
               "schedule": crontab(minute=0, hour=0),
           },
       }
   
   
   CELERY_CONFIG = CeleryConfig
   ```
   `
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] leocape commented on a change in pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
leocape commented on a change in pull request #13104:
URL: https://github.com/apache/superset/pull/13104#discussion_r576659636



##########
File path: docs/src/pages/docs/installation/email_reports.mdx
##########
@@ -6,8 +6,305 @@ index: 10
 version: 1
 ---
 
-## Scheduling and Emailing Reports
+## Alerts and Reports
+(version 1.0.1 and above)
+
+Users can configure automated alerts and reports to send charts and dashboards to an email recipient or Slack channel.
+
+- Alerts are sent when a specified condition is passed
+- Reports are sent on a specified schedule 
+
+### Turning on Alerts and reports
+Alerts and reports are not turned on by default. They are currently behind a feature flag, and require some additional services and configurations.
+
+#### Requirements:
+
+- `Dockerfile`
+	- webdriver to run a headless browser (for taking screenshots of the charts and dahboards)
+- `docker-compose.yaml`
+	- redis message broker
+	- replacing SQLlite DB with Postgres DB
+	- celery worker
+	- celery beat
+- `superset_config.py`
+	- feature flag turned to True
+	- all configs as outlined in the template below
+- At least one of these is needed to send alerts and reports:
+	- (optional) SMTP server for sending email
+	- (optional) Slack app integration for sending to Slack channels
+ 
+#### Summary of steps to turn on alerts and reporting:
+
+Using the templates below,
+1. Create a new directory and create the Dockerfile
+2. Build the extended image using the Dockerfile
+3. Create the `docker-compose.yaml` file in the same directory
+4. Create a new sub directory called `config`
+5. Create the `superset_config.py` file in the `config` sub directory
+6. Run the image using `docker-compose up` in the same directory as the `docker-compose.py` file
+ 
+(note: v 1.0.1 is current at time of writing, you can change the version number to the latest version if a newer version is available)
+### Dockerfile
+
+A webdriver (and headless browser) is needed to capture screenshots of the charts and dashboards which are then sent to the recipient. As the base image does not have a webdriver installed by default, we need to extend the base image and install the webdriver (this template uses the Chrome webdriver). We are also adding in connectors for Mysql and Postgres, as well as Redis and Flower (Flower and Mysql are optional depending on your requirements)
+
+You can extend the image by running this Docker build command from the directory that contains the Dockerfile:
+`docker build -t superset-1.0.1-extended -f Dockerfile`
+
+Config for `Dockerfile`:
+```docker
+FROM apache/superset:1.0.1
+USER root
+RUN apt update
+RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb && \
+    apt install -y --no-install-recommends ./google-chrome-stable_current_amd64.deb && \
+    wget https://chromedriver.storage.googleapis.com/88.0.4324.96/chromedriver_linux64.zip && \
+    unzip chromedriver_linux64.zip && \
+    chmod +x chromedriver && \
+    mv chromedriver /usr/bin && \
+    apt autoremove -yqq --purge && \
+    apt clean && \
+    rm -f google-chrome-stable_current_amd64.deb chromedriver_linux64.zip

Review comment:
       I couldn't get it working with geckodriver/firefox, no matter which version I tried, it wouldn't launch correctly when trying to take the screenshot... I suspect there is a missing config somewhere that is needed for it




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] Yann-OAF commented on pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
Yann-OAF commented on pull request #13104:
URL: https://github.com/apache/superset/pull/13104#issuecomment-779706489


   > > #13116
   > 
   > Thanks @Yann-OAF I could link to it from this doc where I've placed the link placeholder?
   
   Yes, sounds good. These are probably too much to integrate to this doc, and won't be of interest to everyone (only k8s users) so linking is probably enough! I'm thinking I might start a dedicated doc on running in Kubernetes


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] aliJabra edited a comment on pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
aliJabra edited a comment on pull request #13104:
URL: https://github.com/apache/superset/pull/13104#issuecomment-785185040


   @vnourdin - thanks for the documentation! I'm running superset locally installed via pip (1.0.1). Is docker also a preferred way to run the 'backend' for generating alerts and reports? 
   I've superset_config.py properly configured (i think), can see the alerts and reports, slack & email stmp tokens etc. set. I can create alerts and reports, but they never trigger.. 
   
   I have a celery worker: 
   `celery worker --app=superset.tasks.celery_app:app --pool=prefork -O fair -c 4 -E  --concurrency=500 &
   `
   and a beat process running in bg:
   `celery beat --app=superset.tasks.celery_app:app --pidfile /tmp/celerybeat.pid --schedule /tmp/celerybeat-schedule &
   `
   
   I've used the default celery settings as per config.py
   
   `
   ```
   class CeleryConfig:  # pylint: disable=too-few-public-methods
       BROKER_URL = "sqla+sqlite:///celerydb.sqlite"
       CELERY_IMPORTS = ("superset.sql_lab", "superset.tasks")
       CELERY_RESULT_BACKEND = "db+sqlite:///celery_results.sqlite"
       CELERYD_LOG_LEVEL = "DEBUG"
       CELERYD_PREFETCH_MULTIPLIER = 1
       CELERY_ACKS_LATE = True
       CELERY_ANNOTATIONS = {
           "sql_lab.get_sql_results": {"rate_limit": "100/s"},
           "email_reports.send": {
               "rate_limit": "1/s",
               "time_limit": 600,
               "soft_time_limit": 600,
               "ignore_result": True,
           },
       }
       CELERYBEAT_SCHEDULE = {
           "email_reports.schedule_hourly": {
               "task": "email_reports.schedule_hourly",
               "schedule": crontab(minute=1, hour="*"),
           },
           "reports.scheduler": {
               "task": "reports.scheduler",
               "schedule": crontab(minute="*", hour="*"),
           },
           "reports.prune_log": {
               "task": "reports.prune_log",
               "schedule": crontab(minute=0, hour=0),
           },
       }
   
   
   CELERY_CONFIG = CeleryConfig
   ```
   `
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] srinify merged pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
srinify merged pull request #13104:
URL: https://github.com/apache/superset/pull/13104


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] dpgaspar commented on a change in pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
dpgaspar commented on a change in pull request #13104:
URL: https://github.com/apache/superset/pull/13104#discussion_r576747287



##########
File path: docs/src/pages/docs/installation/email_reports.mdx
##########
@@ -6,8 +6,309 @@ index: 10
 version: 1
 ---
 
-## Scheduling and Emailing Reports
+## Alerts and Reports
+(version 1.0.1 and above)
+
+Users can configure automated alerts and reports to send charts and dashboards to an email recipient or Slack channel.
+
+- Alerts are sent when a specified condition is passed
+- Reports are sent on a specified schedule 
+
+### Turning on Alerts and reports
+Alerts and reports are not turned on by default. They are currently behind a feature flag, and require some additional services and configurations.
+
+#### Requirements:
+
+- `Dockerfile`
+	- webdriver to run a headless browser (for taking screenshots of the charts and dahboards)
+- `docker-compose.yaml`
+	- redis message broker
+	- replacing SQLlite DB with Postgres DB
+	- celery worker
+	- celery beat
+- `superset_config.py`
+	- feature flag turned to True
+	- all configs as outlined in the template below
+- At least one of these is needed to send alerts and reports:
+	- (optional) SMTP server for sending email
+	- (optional) Slack app integration for sending to Slack channels
+ 
+#### Summary of steps to turn on alerts and reporting:
+
+Using the templates below,
+1. Create a new directory and create the Dockerfile
+2. Build the extended image using the Dockerfile
+3. Create the `docker-compose.yaml` file in the same directory
+4. Create a new sub directory called `config`
+5. Create the `superset_config.py` file in the `config` sub directory
+6. Run the image using `docker-compose up` in the same directory as the `docker-compose.py` file
+7. In a new terminal window, upgrade the DB by running `docker exec -it superset-1.0.1-extended superset db upgrade`
+8. Then run `docker exec -it superset-1.0.1-extended superset init`
+9. Then setup your admin user if need be, `docker exec -it superset-1.0.1-extended superset fab create-admin`
+10. Finally, restart the running instance - `CTRL-C`, then `docker-compose up`
+ 
+(note: v 1.0.1 is current at time of writing, you can change the version number to the latest version if a newer version is available)
+### Dockerfile
+
+A webdriver (and headless browser) is needed to capture screenshots of the charts and dashboards which are then sent to the recipient. As the base image does not have a webdriver installed by default, we need to extend the base image and install the webdriver (this template uses the Chrome webdriver). We are also adding in connectors for Mysql and Postgres, as well as Redis and Flower (Flower and Mysql are optional depending on your requirements)
+
+You can extend the image by running this Docker build command from the directory that contains the Dockerfile:
+`docker build -t superset-1.0.1-extended -f Dockerfile .`
+
+Config for `Dockerfile`:
+```docker
+FROM apache/superset:1.0.1
+USER root
+RUN apt update
+RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb && \
+    apt install -y --no-install-recommends ./google-chrome-stable_current_amd64.deb && \
+    wget https://chromedriver.storage.googleapis.com/88.0.4324.96/chromedriver_linux64.zip && \

Review comment:
       These two versions will eventually differ




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] leocape commented on a change in pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
leocape commented on a change in pull request #13104:
URL: https://github.com/apache/superset/pull/13104#discussion_r576609031



##########
File path: docs/src/pages/docs/installation/email_reports.mdx
##########
@@ -6,8 +6,309 @@ index: 10
 version: 1
 ---
 
-## Scheduling and Emailing Reports
+## Alerts and Reports

Review comment:
       I'd definitely agree, was really just for organisation so that in the menu there isn't 'Alerts and reporting' and then below it 'Scheduling and Emailing reports' which would be confusing imo, but if @srinify can help we could put it into its own section 'Alerts and Reporting', and then have the legacy reporting stuff remain at the bottom of this file?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] Yann-OAF commented on a change in pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
Yann-OAF commented on a change in pull request #13104:
URL: https://github.com/apache/superset/pull/13104#discussion_r575839561



##########
File path: docs/src/pages/docs/installation/email_reports.mdx
##########
@@ -6,8 +6,305 @@ index: 10
 version: 1
 ---
 
-## Scheduling and Emailing Reports
+## Alerts and Reports
+(version 1.0.1 and above)
+
+Users can configure automated alerts and reports to send charts and dashboards to an email recipient or Slack channel.
+
+- Alerts are sent when a specified condition is passed
+- Reports are sent on a specified schedule 
+
+### Turning on Alerts and reports
+Alerts and reports are not turned on by default. They are currently behind a feature flag, and require some additional services and configurations.
+
+#### Requirements:
+
+- `Dockerfile`
+	- webdriver to run a headless browser (for taking screenshots of the charts and dahboards)
+- `docker-compose.yaml`
+	- redis message broker
+	- replacing SQLlite DB with Postgres DB
+	- celery worker
+	- celery beat
+- `superset_config.py`
+	- feature flag turned to True
+	- all configs as outlined in the template below
+- At least one of these is needed to send alerts and reports:
+	- (optional) SMTP server for sending email
+	- (optional) Slack app integration for sending to Slack channels
+ 
+#### Summary of steps to turn on alerts and reporting:
+
+Using the templates below,
+1. Create a new directory and create the Dockerfile
+2. Build the extended image using the Dockerfile
+3. Create the `docker-compose.yaml` file in the same directory
+4. Create a new sub directory called `config`
+5. Create the `superset_config.py` file in the `config` sub directory
+6. Run the image using `docker-compose up` in the same directory as the `docker-compose.py` file
+ 
+(note: v 1.0.1 is current at time of writing, you can change the version number to the latest version if a newer version is available)
+### Dockerfile
+
+A webdriver (and headless browser) is needed to capture screenshots of the charts and dashboards which are then sent to the recipient. As the base image does not have a webdriver installed by default, we need to extend the base image and install the webdriver (this template uses the Chrome webdriver). We are also adding in connectors for Mysql and Postgres, as well as Redis and Flower (Flower and Mysql are optional depending on your requirements)
+
+You can extend the image by running this Docker build command from the directory that contains the Dockerfile:
+`docker build -t superset-1.0.1-extended -f Dockerfile`
+
+Config for `Dockerfile`:
+```docker
+FROM apache/superset:1.0.1
+USER root
+RUN apt update
+RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb && \
+    apt install -y --no-install-recommends ./google-chrome-stable_current_amd64.deb && \
+    wget https://chromedriver.storage.googleapis.com/88.0.4324.96/chromedriver_linux64.zip && \
+    unzip chromedriver_linux64.zip && \
+    chmod +x chromedriver && \
+    mv chromedriver /usr/bin && \
+    apt autoremove -yqq --purge && \
+    apt clean && \
+    rm -f google-chrome-stable_current_amd64.deb chromedriver_linux64.zip

Review comment:
       I'm curious, would you have similar working instructions to install `geckodriver`? My attempts have been unsuccessful...




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] aliJabra commented on pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
aliJabra commented on pull request #13104:
URL: https://github.com/apache/superset/pull/13104#issuecomment-785185040


   @vnourdin - thanks for the documentation! I'm running superset locally installed via pip (1.0.1). Is docker also a preferred way to run the 'backend' for generating alerts and reports? 
   I've superset_config.py properly configured (i think), can see the alerts and reports, slack & email stmp tokens etc. set. I can create alerts and reports, but they never trigger.. 
   
   I have a celery worker: 
   `celery worker --app=superset.tasks.celery_app:app --pool=prefork -O fair -c 4 -E  --concurrency=500 &
   `
   and a beat process running in bg:
   `celery beat --app=superset.tasks.celery_app:app --pidfile /tmp/celerybeat.pid --schedule /tmp/celerybeat-schedule &
   `
   
   I've used the default celery settings as per config.py
   
   `class CeleryConfig:  # pylint: disable=too-few-public-methods
       BROKER_URL = "sqla+sqlite:///celerydb.sqlite"
       CELERY_IMPORTS = ("superset.sql_lab", "superset.tasks")
       CELERY_RESULT_BACKEND = "db+sqlite:///celery_results.sqlite"
       CELERYD_LOG_LEVEL = "DEBUG"
       CELERYD_PREFETCH_MULTIPLIER = 1
       CELERY_ACKS_LATE = True
       CELERY_ANNOTATIONS = {
           "sql_lab.get_sql_results": {"rate_limit": "100/s"},
           "email_reports.send": {
               "rate_limit": "1/s",
               "time_limit": 600,
               "soft_time_limit": 600,
               "ignore_result": True,
           },
       }
       CELERYBEAT_SCHEDULE = {
           "email_reports.schedule_hourly": {
               "task": "email_reports.schedule_hourly",
               "schedule": crontab(minute=1, hour="*"),
           },
           "reports.scheduler": {
               "task": "reports.scheduler",
               "schedule": crontab(minute="*", hour="*"),
           },
           "reports.prune_log": {
               "task": "reports.prune_log",
               "schedule": crontab(minute=0, hour=0),
           },
       }
   
   
   CELERY_CONFIG = CeleryConfig`
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] Kingflyinger commented on pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
Kingflyinger commented on pull request #13104:
URL: https://github.com/apache/superset/pull/13104#issuecomment-787055210


   `version: '3.6'
   services:
     redis-superset:
       image: redis:6.0.9-buster
       restart: on-failure`
   redis-superset should be replaced by redis,as well as superset_config.py.
   or change the depends_on 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] shemExelate commented on a change in pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
shemExelate commented on a change in pull request #13104:
URL: https://github.com/apache/superset/pull/13104#discussion_r577055458



##########
File path: docs/src/pages/docs/installation/email_reports.mdx
##########
@@ -6,8 +6,309 @@ index: 10
 version: 1
 ---
 
-## Scheduling and Emailing Reports
+## Alerts and Reports
+(version 1.0.1 and above)
+
+Users can configure automated alerts and reports to send charts and dashboards to an email recipient or Slack channel.
+
+- Alerts are sent when a specified condition is passed
+- Reports are sent on a specified schedule 
+
+### Turning on Alerts and reports
+Alerts and reports are not turned on by default. They are currently behind a feature flag, and require some additional services and configurations.
+
+#### Requirements:
+
+- `Dockerfile`
+	- webdriver to run a headless browser (for taking screenshots of the charts and dahboards)
+- `docker-compose.yaml`
+	- redis message broker
+	- replacing SQLlite DB with Postgres DB
+	- celery worker
+	- celery beat
+- `superset_config.py`
+	- feature flag turned to True
+	- all configs as outlined in the template below
+- At least one of these is needed to send alerts and reports:
+	- (optional) SMTP server for sending email
+	- (optional) Slack app integration for sending to Slack channels
+ 
+#### Summary of steps to turn on alerts and reporting:
+
+Using the templates below,
+1. Create a new directory and create the Dockerfile
+2. Build the extended image using the Dockerfile
+3. Create the `docker-compose.yaml` file in the same directory
+4. Create a new sub directory called `config`
+5. Create the `superset_config.py` file in the `config` sub directory
+6. Run the image using `docker-compose up` in the same directory as the `docker-compose.py` file
+7. In a new terminal window, upgrade the DB by running `docker exec -it superset-1.0.1-extended superset db upgrade`

Review comment:
       is there a better way to run 
   ```
   superset db upgrade
   superset init
   superset fab create-admin
   ```
   as part of startup without entering the container?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] nytai commented on pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
nytai commented on pull request #13104:
URL: https://github.com/apache/superset/pull/13104#issuecomment-780016484


   Ok, in that case we should keep the Dockerfile changes. I'm planning to add the beat container in https://github.com/apache/superset/pull/13143 so that should take care of some the docker-compose changes, though it will need to be updated to reference the new images. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] Yann-OAF commented on pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
Yann-OAF commented on pull request #13104:
URL: https://github.com/apache/superset/pull/13104#issuecomment-778802687


   FYI, I have just sent PR #13116 which would enable this on Kubernetes by (optionally) running Celery beat as a separate singleton pod.
   
   The rest of the configuration can be injected using the existing chart mechanism:
   * Install webdrivers by overriding the worker's command (although doing this at startup time isn't great... the best would be to have them preinstalled in the image...)
   * Pass all the necessary `superset_config.py` overrides from `values.yaml`
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] dpgaspar commented on a change in pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
dpgaspar commented on a change in pull request #13104:
URL: https://github.com/apache/superset/pull/13104#discussion_r576747287



##########
File path: docs/src/pages/docs/installation/email_reports.mdx
##########
@@ -6,8 +6,309 @@ index: 10
 version: 1
 ---
 
-## Scheduling and Emailing Reports
+## Alerts and Reports
+(version 1.0.1 and above)
+
+Users can configure automated alerts and reports to send charts and dashboards to an email recipient or Slack channel.
+
+- Alerts are sent when a specified condition is passed
+- Reports are sent on a specified schedule 
+
+### Turning on Alerts and reports
+Alerts and reports are not turned on by default. They are currently behind a feature flag, and require some additional services and configurations.
+
+#### Requirements:
+
+- `Dockerfile`
+	- webdriver to run a headless browser (for taking screenshots of the charts and dahboards)
+- `docker-compose.yaml`
+	- redis message broker
+	- replacing SQLlite DB with Postgres DB
+	- celery worker
+	- celery beat
+- `superset_config.py`
+	- feature flag turned to True
+	- all configs as outlined in the template below
+- At least one of these is needed to send alerts and reports:
+	- (optional) SMTP server for sending email
+	- (optional) Slack app integration for sending to Slack channels
+ 
+#### Summary of steps to turn on alerts and reporting:
+
+Using the templates below,
+1. Create a new directory and create the Dockerfile
+2. Build the extended image using the Dockerfile
+3. Create the `docker-compose.yaml` file in the same directory
+4. Create a new sub directory called `config`
+5. Create the `superset_config.py` file in the `config` sub directory
+6. Run the image using `docker-compose up` in the same directory as the `docker-compose.py` file
+7. In a new terminal window, upgrade the DB by running `docker exec -it superset-1.0.1-extended superset db upgrade`
+8. Then run `docker exec -it superset-1.0.1-extended superset init`
+9. Then setup your admin user if need be, `docker exec -it superset-1.0.1-extended superset fab create-admin`
+10. Finally, restart the running instance - `CTRL-C`, then `docker-compose up`
+ 
+(note: v 1.0.1 is current at time of writing, you can change the version number to the latest version if a newer version is available)
+### Dockerfile
+
+A webdriver (and headless browser) is needed to capture screenshots of the charts and dashboards which are then sent to the recipient. As the base image does not have a webdriver installed by default, we need to extend the base image and install the webdriver (this template uses the Chrome webdriver). We are also adding in connectors for Mysql and Postgres, as well as Redis and Flower (Flower and Mysql are optional depending on your requirements)
+
+You can extend the image by running this Docker build command from the directory that contains the Dockerfile:
+`docker build -t superset-1.0.1-extended -f Dockerfile .`
+
+Config for `Dockerfile`:
+```docker
+FROM apache/superset:1.0.1
+USER root
+RUN apt update
+RUN wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb && \
+    apt install -y --no-install-recommends ./google-chrome-stable_current_amd64.deb && \
+    wget https://chromedriver.storage.googleapis.com/88.0.4324.96/chromedriver_linux64.zip && \

Review comment:
       These two version will eventually differ




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] srinify commented on pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
srinify commented on pull request #13104:
URL: https://github.com/apache/superset/pull/13104#issuecomment-786710445


   Hey @vnourdin @leocape I'm merging this as-is for now! The clean up process is quite involved, I like some things about the older commit and the latest commit and there's a lot of context to resolve. For now, it'd be nice to just have the info out there.
   
   So I'm merging for now, and I'll do a second pass next week to clean up and organize better!


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] leocape commented on pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
leocape commented on pull request #13104:
URL: https://github.com/apache/superset/pull/13104#issuecomment-779655744


   > #13116
   
   Thanks @Yann-OAF I could link to it from this doc where I've placed the link placeholder?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] nytai commented on pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
nytai commented on pull request #13104:
URL: https://github.com/apache/superset/pull/13104#issuecomment-779510931


   I also opened https://github.com/apache/superset/pull/13143 which includes a new docker-compose file for non-dev use cases. I added a beat container which I realized was missing in the regular docker-compose workflow. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


[GitHub] [superset] nytai commented on a change in pull request #13104: chore: update docs for new alerts and reporting feature

Posted by GitBox <gi...@apache.org>.
nytai commented on a change in pull request #13104:
URL: https://github.com/apache/superset/pull/13104#discussion_r577070537



##########
File path: docs/src/pages/docs/installation/email_reports.mdx
##########
@@ -6,8 +6,309 @@ index: 10
 version: 1
 ---
 
-## Scheduling and Emailing Reports
+## Alerts and Reports
+(version 1.0.1 and above)
+
+Users can configure automated alerts and reports to send charts and dashboards to an email recipient or Slack channel.
+
+- Alerts are sent when a specified condition is passed
+- Reports are sent on a specified schedule 
+
+### Turning on Alerts and reports
+Alerts and reports are not turned on by default. They are currently behind a feature flag, and require some additional services and configurations.
+
+#### Requirements:
+
+- `Dockerfile`
+	- webdriver to run a headless browser (for taking screenshots of the charts and dahboards)
+- `docker-compose.yaml`
+	- redis message broker
+	- replacing SQLlite DB with Postgres DB
+	- celery worker
+	- celery beat
+- `superset_config.py`
+	- feature flag turned to True
+	- all configs as outlined in the template below
+- At least one of these is needed to send alerts and reports:
+	- (optional) SMTP server for sending email
+	- (optional) Slack app integration for sending to Slack channels
+ 
+#### Summary of steps to turn on alerts and reporting:
+
+Using the templates below,
+1. Create a new directory and create the Dockerfile
+2. Build the extended image using the Dockerfile
+3. Create the `docker-compose.yaml` file in the same directory
+4. Create a new sub directory called `config`
+5. Create the `superset_config.py` file in the `config` sub directory
+6. Run the image using `docker-compose up` in the same directory as the `docker-compose.py` file
+7. In a new terminal window, upgrade the DB by running `docker exec -it superset-1.0.1-extended superset db upgrade`

Review comment:
       i think the `superset-init` service in the docker-compose workflow should take care of this. 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org