You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2021/02/04 13:01:28 UTC
[GitHub] [superset] JMGGarcia opened a new issue #12942: Generating Reports
JMGGarcia opened a new issue #12942:
URL: https://github.com/apache/superset/issues/12942
(First of all, congratulations on the v1 release of Superset, it looks very nice and includes a lot of nice new features!)
On the old version of Superset, I had an issue generating reports on a server (#11552). I was trying out the new version of superset, using v1.0.1rc2 and the default docker configs to try and generate reports (first locally, and then I would try on a server). I define the report to run every 6 minutes, and use one of the default dashboards (covid), and set the report to be sent by email to me. On the config file, I put `ENABLE_SCHEDULED_EMAIL_REPORTS = True` and the SMTP configurations. I tried with the debug mode on and off.
The problem is that nothing seems to run, with errors or not. There are no logs, neither in the UI nor in the terminal.
### Expected results
Logs related to the generation of reports.
### Actual results
No logs are generated, nothing seems to happen.
#### How to reproduce the bug
1. Go to Alert & Report page
2. Define a report
3. Wait for logs to be generated
### Environment
(please complete the following information):
- superset version: `v1.0.1rc2`
- Using default docker-compose with configs defined to generate reports
### Checklist
Make sure to follow these steps before submitting your issue - thank you!
- [x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
- [x] I have reproduced the issue with at least the latest released version of superset.
- [x] I have checked the issue tracker for the same issue and I haven't found one similar.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] vnourdin commented on issue #12942: Generating Reports
Posted by GitBox <gi...@apache.org>.
vnourdin commented on issue #12942:
URL: https://github.com/apache/superset/issues/12942#issuecomment-778112851
@shemExelate the config for reports scheduling is missing from your `CeleryConfig`, you must add the content of [this PR](https://github.com/apache/superset/pull/12999/files).
This feature is not yet documented properly, we are working on it!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] vnourdin commented on issue #12942: Generating Reports
Posted by GitBox <gi...@apache.org>.
vnourdin commented on issue #12942:
URL: https://github.com/apache/superset/issues/12942#issuecomment-778209169
Are you creating your reports from the old menu or the new "Alerts & Reports" one?
The celery beat is only used with the new implementation I believe.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] zuzana-vej closed issue #12942: Generating Reports
Posted by GitBox <gi...@apache.org>.
zuzana-vej closed issue #12942:
URL: https://github.com/apache/superset/issues/12942
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] JMGGarcia commented on issue #12942: Generating Reports
Posted by GitBox <gi...@apache.org>.
JMGGarcia commented on issue #12942:
URL: https://github.com/apache/superset/issues/12942#issuecomment-778213090
I'm using the new one (although I tested and I'm able to run the old one).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] zuzana-vej commented on issue #12942: Generating Reports
Posted by GitBox <gi...@apache.org>.
zuzana-vej commented on issue #12942:
URL: https://github.com/apache/superset/issues/12942#issuecomment-824375704
Looks like a discussion/question opened 2+ months; If this question is still pending please reopen under the "Discussions"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] shemExelate commented on issue #12942: Generating Reports
Posted by GitBox <gi...@apache.org>.
shemExelate commented on issue #12942:
URL: https://github.com/apache/superset/issues/12942#issuecomment-774756667
I'm having the same issue, running latest from `/docker`
my config:
**docker-compose.yaml**
added:
```yaml
superset-beat:
image: *superset-image
container_name: superset_beat
command: ["/app/docker/docker-bootstrap.sh", "beat"]
env_file: docker/.env
restart: unless-stopped
depends_on: *superset-depends-on
user: "root"
volumes: *superset-volumes
```
**docker-bootstrap.sh**
modified to:
```sh
if [[ "${1}" == "worker" ]]; then
echo "Starting Celery worker..."
celery worker --app=superset.tasks.celery_app:app --pool=prefork -O fair -c 4 -l DEBUG
elif [[ "${1}" == "beat" ]]; then
echo "Starting Celery beat..."
celery beat --app=superset.tasks.celery_app:app -l DEBUG
elif [[ "${1}" == "app" ]]; then
echo "Starting web app..."
flask run -p 8088 --with-threads --reload --debugger --host=0.0.0.0
fi
```
**superset_config.py**
modified to:
```py
RESULTS_BACKEND = RedisCache(
host=REDIS_HOST, port=REDIS_PORT)
FEATURE_FLAGS = {
"ALERT_REPORTS": True,
}
class CeleryConfig(object):
BROKER_URL = f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_CELERY_DB}"
CELERY_IMPORTS = (
"superset.sql_lab",
"superset.tasks",
)
CELERY_RESULT_BACKEND = f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_RESULTS_DB}"
CELERY_ANNOTATIONS = {
"tasks.add": {
"rate_limit": "10/s",
},
"sql_lab.get_sql_results": {
"rate_limit": "100/s",
},
"email_reports.send": {
"rate_limit": "1/s",
"time_limit": 120,
"soft_time_limit": 150,
"ignore_result": True,
},
}
CELERYBEAT_SCHEDULE = {
"email_reports.schedule_hourly": {
"task": "email_reports.schedule_hourly",
"schedule": crontab(minute=1, hour="*"),
},
}
CELERY_TASK_PROTOCOL = 1
CACHE_CONFIG = {
"CACHE_TYPE": "redis",
"CACHE_DEFAULT_TIMEOUT": 60 * 60 * 24, # 1 day default (in secs)
"CACHE_KEY_PREFIX": "superset_results",
"CACHE_REDIS_URL": f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_RESULTS_DB}",
}
CELERY_CONFIG = CeleryConfig
SQLLAB_CTAS_NO_LIMIT = True
ENABLE_SCHEDULED_EMAIL_REPORTS = True
EMAIL_NOTIFICATIONS = True
SMTP_HOST = "smtp.gmail.com"
SMTP_STARTTLS = True
SMTP_SSL = True
SMTP_USER = "df.team.test@gmail.com"
SMTP_PORT = 465
SMTP_PASSWORD = os.environ.get("SMTP_PASSWORD")
SMTP_MAIL_FROM = "df.team.test@gmail.com"
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] JMGGarcia commented on issue #12942: Generating Reports
Posted by GitBox <gi...@apache.org>.
JMGGarcia commented on issue #12942:
URL: https://github.com/apache/superset/issues/12942#issuecomment-778169837
@vnourdin Thank you, but with the content of that PR, I'm still getting the same results (using basically the settings @shemExelate posted previously). The beat container only logs
```
superset_beat | [2021-02-12 11:45:15,424: DEBUG/MainProcess] beat: Ticking with max interval->5.00 minutes
superset_beat | [2021-02-12 11:45:15,425: DEBUG/MainProcess] beat: Waking up in 5.00 minutes.
superset_beat | [2021-02-12 11:50:15,525: DEBUG/MainProcess] beat: Synchronizing schedule...
superset_beat | [2021-02-12 11:50:15,531: DEBUG/MainProcess] beat: Waking up in 5.00 minutes.
superset_beat | [2021-02-12 11:55:15,631: DEBUG/MainProcess] beat: Synchronizing schedule...
superset_beat | [2021-02-12 11:55:15,633: DEBUG/MainProcess] beat: Waking up in 5.00 minutes.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] vnourdin commented on issue #12942: Generating Reports
Posted by GitBox <gi...@apache.org>.
vnourdin commented on issue #12942:
URL: https://github.com/apache/superset/issues/12942#issuecomment-778226133
Then I don't know what's wrong :thinking: I got those logs:
```
│ [2021-02-12 13:51:22,565: DEBUG/MainProcess] beat: Ticking with max interval->5.00 minutes
│ [2021-02-12 13:51:22,572: DEBUG/MainProcess] beat: Waking up in 37.42 seconds.
│ [2021-02-12 13:52:00,011: DEBUG/MainProcess] beat: Synchronizing schedule...
│ [2021-02-12 13:52:00,057: INFO/MainProcess] Scheduler: Sending due task reports.scheduler (reports.scheduler)
│ [2021-02-12 13:52:00,092: DEBUG/MainProcess] reports.scheduler sent. id->656ea944-efa3-4acb-a171-90394e58546d
│ [2021-02-12 13:52:00,093: DEBUG/MainProcess] beat: Waking up in 59.90 seconds.
```
The only main missing config I see is `WEBDRIVER_BASEURL` but you have no error...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org