You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@superset.apache.org by gi...@git.apache.org on 2017/10/10 16:02:40 UTC
[GitHub] alexhockley opened a new issue #3636: Unable to configure Async worker with RabbitMQ
alexhockley opened a new issue #3636: Unable to configure Async worker with RabbitMQ
URL: https://github.com/apache/incubator-superset/issues/3636
### Server information
Ubuntu 16.04.3 LTS on Amazon EC2
### Superset version
0.20.3
### Expected results
Async query finishes
### Actual results
Async query hangs.
gunicorn stdout output on query run:
`2017-10-10 15:58:24,179:INFO:root:[stats_logger] (incr) sql_json
2017-10-10 15:58:24,183:INFO:root:Parsing with sqlparse statement SELECT *
FROM main.ab_permission
2017-10-10 15:58:24,215:INFO:root:Triggering query_id: 13
2017-10-10 15:58:24,215:INFO:root:Running query on a Celery worker
/home/ubuntu/venv/local/lib/python2.7/site-packages/sqlalchemy/sql/sqltypes.py:596: SAWarning: Dialect sqlite+pysqlite does *not* support Decimal objects natively, and SQLAlchemy must convert from floating point - rounding errors and other issues may occur. Please consider storing Decimal numbers as strings or integers on this platform for lossless storage.
'storage.' % (dialect.name, dialect.driver))
2017-10-10 15:58:24,240:DEBUG:amqp:Start from server, version: 0.9, properties: {u'information': u'Licensed under the MPL. See http://www.rabbitmq.com/', u'product': u'RabbitMQ', u'copyright': u'Copyright (C) 2007-2015 Pivotal Software, Inc.', u'capabilities': {u'exchange_exchange_bindings': True, u'connection.blocked': True, u'authentication_failure_close': True, u'basic.nack': True, u'per_consumer_qos': True, u'consumer_priorities': True, u'consumer_cancel_notify': True, u'publisher_confirms': True}, u'cluster_name': u'rabbit@ip-172-31-48-129.ec2.internal', u'platform': u'Erlang/OTP', u'version': u'3.5.7'}, mechanisms: [u'AMQPLAIN', u'PLAIN'], locales: [u'en_US']
2017-10-10 15:58:24,240:DEBUG:amqp:Open OK!
2017-10-10 15:58:24,241:DEBUG:amqp:using channel_id: 1
2017-10-10 15:58:24,241:DEBUG:amqp:Channel open
99.225.182.133 - - [10/Oct/2017:15:58:24 +0000] "POST /superset/sql_json/ HTTP/1.1" 202 546 "http://<ip>:8080/superset/sqllab" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36"
2017-10-10 15:58:25,602:INFO:root:[stats_logger] (incr) queries
99.225.182.133 - - [10/Oct/2017:15:58:25 +0000] "GET /superset/queries/1507650390378.714 HTTP/1.1" 200 2201 "http://<ip>:8080/superset/sqllab" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36"
2017-10-10 15:58:27,595:INFO:root:[stats_logger] (incr) queries
99.225.182.133 - - [10/Oct/2017:15:58:27 +0000] "GET /superset/queries/1507651099210.269 HTTP/1.1" 200 550 "http://<ip>:8080/superset/sqllab" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36"
2017-10-10 15:58:29,592:INFO:root:[stats_logger] (incr) queries
99.225.182.133 - - [10/Oct/2017:15:58:29 +0000] "GET /superset/queries/1507651099210.269 HTTP/1.1" 200 550 "http://<ip>:8080/superset/sqllab" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36"`
tailing /var/log/rabbitmq/rabbit@ip-172-31-48-129.log has:
`=INFO REPORT==== 10-Oct-2017::15:58:24 ===
accepting AMQP connection <0.525.0> (127.0.0.1:37060 -> 127.0.0.1:5672)`
/tml/sqllab_cache exists and should have write permissions
### Steps to reproduce
- Boot fresh server on Amazon EC2
- Follow install instructions located on https://superset.incubator.apache.org/installation.html
- Configure gunicorn (doesnt work with regular runserver either though)
- Install RabbitMQ
- Run `gunicorn --error-logfile - --access-logfile - -w 8 -k gevent -b 0.0.0.0:8080 --timeout 120 --limit-request-line 0 --limit-request-field_size 0 --forwarded-allow-ips="*" superset:app` to start the web server
- Connect to <public ip>:8080, login, configure the sample database to use async workers instead of sync (it works fine with sync workers)
- Go to SQL Lab and run a simple SELECT * FROM main.ab_permission
Using the following superset custom config file, located in the same directory i run the gunicorn command from
`#---------------------------------------------------------
# Superset specific config
#---------------------------------------------------------
ROW_LIMIT = 5000
SUPERSET_WORKERS = 4
SUPERSET_WEBSERVER_PORT = 8088
#---------------------------------------------------------
#---------------------------------------------------------
# Flask App Builder configuration
#---------------------------------------------------------
# Your App secret key
SECRET_KEY = '\2\1thisismyscretkey\1\2\e\y\y\h'
# The SQLAlchemy connection string to your database backend
# This connection defines the path to the database that stores your
# superset metadata (slices, connections, tables, dashboards, ...).
# Note that the connection information to connect to the datasources
# you want to explore are managed directly in the web UI
#SQLALCHEMY_DATABASE_URI = 'sqlite:////path/to/superset.db'
# Flask-WTF flag for CSRF
WTF_CSRF_ENABLED = True
# Add endpoints that need to be exempt from CSRF protection
WTF_CSRF_EXEMPT_LIST = []
# Set this API key to enable Mapbox visualizations
MAPBOX_API_KEY = ''
from werkzeug.contrib.cache import FileSystemCache
RESULTS_BACKEND = FileSystemCache('/tmp/sqllab_cache', default_timeout=60*24*7)
`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services