You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "kvatsek (via GitHub)" <gi...@apache.org> on 2023/06/19 09:06:04 UTC
[GitHub] [superset] kvatsek opened a new issue, #24445: superset does not close db connections
kvatsek opened a new issue, #24445:
URL: https://github.com/apache/superset/issues/24445
Superset does not close db connections. Аs a result, connections hang for days and eventually exceed the allowed number
See error FATAL: too many connections for role "superset"
Is it possible to make a superset with close db connections?
version superset 2.1.0
superset running in kubernetes
db in psql cluster
python3.8
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] superset does not close db connections [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas closed issue #24445: superset does not close db connections
URL: https://github.com/apache/superset/issues/24445
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] superset does not close db connections [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #24445:
URL: https://github.com/apache/superset/issues/24445#issuecomment-1977334528
Not all DBs support query cancellation as such, but we've made improvements for those that do. It sounds like you might need to make some timeout adjustments at your DB layer? Closing this as stale since it's been silent for so long, and we're trying to steer toward a more actionable Issues backlog. If you are still encountering this in current versions (currently 3.x) please re-open this issue, open a new Issue with updated context, or raise a PR to address the problem. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] kvatsek commented on issue #24445: superset does not close db connections
Posted by "kvatsek (via GitHub)" <gi...@apache.org>.
kvatsek commented on issue #24445:
URL: https://github.com/apache/superset/issues/24445#issuecomment-1598219733
In web error: There was an error fetching your recent activity
In pods log:
10.42.5.224 - - [20/Jun/2023:06:42:55 +0000] "GET /api/v1/dashboard/?q=(filters:!((col:owners,opr:rel_m_m,value:%271%27)),order_column:changed_on_delta_humanized,order_direction:desc,page:0,page_size:5) HTTP/1.1" 200 569 "https://superset.prod.rancher.ex.com/superset/welcome/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36"
(psycopg2.OperationalError) connection to server at "cm-db-psql01.ex.com" (192.168.0.3), port 5555 failed: FATAL: too many connections for role "cm_superset_adm"
(Background on this error at: https://sqlalche.me/e/14/e3q8)
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 3280, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 310, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 868, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 476, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 146, in _do_get
self._dec_overflow()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
compat.raise_(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 143, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 256, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 371, in __init__
self.__connect()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 666, in __connect
pool.logger.debug("Error on connect(): %s", e)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
compat.raise_(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 661, in __connect
self.dbapi_connection = connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/create.py", line 590, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 597, in connect
return self.dbapi.connect(*cargs, **cparams)
File "/usr/local/lib/python3.8/site-packages/psycopg2/__init__.py", line 122, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: connection to server at "cm-db-psql01.ex.com" (192.168.0.3), port 5555 failed: FATAL: too many connections for role "cm_superset_adm"
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/superset/views/base.py", line 228, in wraps
return f(self, *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/security/decorators.py", line 136, in wraps
if permission_str in self.base_permissions and self.appbuilder.sm.has_access(
File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/security/manager.py", line 1559, in has_access
if current_user.is_authenticated:
File "/usr/local/lib/python3.8/site-packages/werkzeug/local.py", line 278, in __get__
obj = instance._get_current_object()
File "/usr/local/lib/python3.8/site-packages/werkzeug/local.py", line 407, in _get_current_object
return self.__local() # type: ignore
File "/usr/local/lib/python3.8/site-packages/flask_login/utils.py", line 25, in <lambda>
current_user = LocalProxy(lambda: _get_user())
File "/usr/local/lib/python3.8/site-packages/flask_login/utils.py", line 359, in _get_user
current_app.login_manager._load_user()
File "/usr/local/lib/python3.8/site-packages/flask_login/login_manager.py", line 355, in _load_user
user = self._user_callback(user_id)
File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/security/manager.py", line 2112, in load_user
return self.get_user_by_id(int(pk))
File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/security/sqla/manager.py", line 251, in get_user_by_id
return self.get_session.query(self.user_model).get(pk)
File "<string>", line 2, in get
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/deprecations.py", line 401, in warned
return fn(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 943, in get
return self._get_impl(ident, loading.load_on_pk_identity)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 947, in _get_impl
return self.session._get_impl(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 2896, in _get_impl
return db_load_fn(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/loading.py", line 530, in load_on_pk_identity
session.execute(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1695, in execute
conn = self._connection_for_bind(bind)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 1536, in _connection_for_bind
return self._transaction._connection_for_bind(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/orm/session.py", line 747, in _connection_for_bind
conn = bind.connect()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 3234, in connect
return self._connection_cls(self, close_with_result=close_with_result)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 96, in __init__
else engine.raw_connection()
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 3313, in raw_connection
return self._wrap_pool_connect(self.pool.connect, _connection)
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 3283, in _wrap_pool_connect
Connection._handle_dbapi_exception_noconnection(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2117, in _handle_dbapi_exception_noconnection
util.raise_(
File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] kvatsek commented on issue #24445: superset does not close db connections
Posted by "kvatsek (via GitHub)" <gi...@apache.org>.
kvatsek commented on issue #24445:
URL: https://github.com/apache/superset/issues/24445#issuecomment-1623054846
found out that hung sessions remain after SQL-lab
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org