You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2022/08/03 17:56:22 UTC
[GitHub] [superset] justineyster opened a new issue, #20960: Connection to Trino using SSL certificate no longer working
justineyster opened a new issue, #20960:
URL: https://github.com/apache/superset/issues/20960
My team is using Trino behind a company firewall and connecting to it using Superset. When we first set this up, we had some issues getting Superset to trust the SSL certificate used to access Trino through the firewall, and we fixed these issues by taking the following two steps:
1) Moved the certificate to `/etc/pki/ca-trust/source/anchors/cert.pem` and ran `update-ca-trust` during our Docker image build.
2) Added the following to the `secure_extra` section of our database connection settings in the Superset UI:
```
{ "connect_args": {
"verify": "/etc/pki/ca-trust/source/anchors/cert.pem"
}}
```
These steps fixed the issue for several months, but now we're once again seeing the following error:
```
Error: HTTPSConnectionPool(host='trino.host.name', port=443): Max retries exceeded with url: /v1/statement (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)')))
```
The certificate gets pulled every time we deploy the image, so it is definitely still valid. A former team member used the following command from inside a Superset worker pod to test the certificate:
```
curl https://trino.host.name/v1/info
```
This should throw an error message if there is a problem with the SSL certificate, however, the output is:
```
{"nodeVersion":{"version":"390"},"environment":"production","coordinator":true,"starting":false,"uptime":"22.65h"}
```
I am wondering if anything has changed on the Superset side that is interfering with our ability to make this connection. Note that we recently upgraded from Superset version 1.4.1 to 1.5.1, if that helps. We haven't upgraded to 2.0.0 yet due to breaking changes. Any hints or debugging tips would be helpful, as I am quite stuck with debugging this on my own.
#### How to reproduce the bug
1. Connect to Trino from Superset using an SSL certificate as described above.
### Expected results
Connection is successful.
### Actual results
SSL error.
#### Screenshots
N/A
### Environment
- browser type and version: Chrome
- superset version: `1.5.1`
- python version: `3.9.7`
- node.js version: `14.18.2`
- any feature flags active:
### Checklist
Make sure to follow these steps before submitting your issue - thank you!
- [x] I have checked the superset logs for python stacktraces and included it here as text if there are any.
- [ ] I have reproduced the issue with at least the latest released version of superset.
- [x] I have checked the issue tracker for the same issue and I haven't found one similar.
### Additional context
Python stack trace:
```
Traceback (most recent call last):
File "/opt/app-root/lib64/python3.9/site-packages/superset/connectors/sqla/models.py", line 1725, in query
df = self.database.get_df(sql, self.schema, mutator=assign_column_label)
File "/opt/app-root/lib64/python3.9/site-packages/superset/models/core.py", line 439, in get_df
with closing(engine.raw_connection()) as conn:
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 2369, in raw_connection
return self._wrap_pool_connect(
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 2336, in _wrap_pool_connect
return fn()
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/pool/base.py", line 304, in unique_connection
return _ConnectionFairy._checkout(self)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/pool/base.py", line 778, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/pool/base.py", line 495, in checkout
rec = pool._do_get()
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/pool/impl.py", line 241, in _do_get
return self._create_connection()
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/pool/base.py", line 309, in _create_connection
return _ConnectionRecord(self)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/pool/base.py", line 440, in __init__
self.__connect(first_connect_check=True)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/pool/base.py", line 664, in __connect
pool.dispatch.first_connect.for_modify(
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/event/attr.py", line 314, in exec_once_unless_exception
self._exec_once_impl(True, *args, **kw)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/event/attr.py", line 285, in _exec_once_impl
self(*args, **kw)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/event/attr.py", line 322, in __call__
fn(*args, **kw)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 1406, in go
return once_fn(*arg, **kw)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/engine/strategies.py", line 199, in first_connect
dialect.initialize(c)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/engine/default.py", line 311, in initialize
self.server_version_info = self._get_server_version_info(
File "/opt/app-root/lib64/python3.9/site-packages/trino/sqlalchemy/dialect.py", line 319, in _get_server_version_info
res = connection.execute(sql.text(query))
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
ret = self._execute_context(
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
self._handle_dbapi_exception(
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1514, in _handle_dbapi_exception
util.raise_(exc_info[1], with_traceback=exc_info[2])
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/opt/app-root/lib64/python3.9/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/opt/app-root/lib64/python3.9/site-packages/trino/sqlalchemy/dialect.py", line 333, in do_execute
cursor.execute(statement, parameters)
File "/opt/app-root/lib64/python3.9/site-packages/trino/dbapi.py", line 484, in execute
result = self._query.execute()
File "/opt/app-root/lib64/python3.9/site-packages/trino/client.py", line 770, in execute
response = self._request.post(self._sql, additional_http_headers)
File "/opt/app-root/lib64/python3.9/site-packages/trino/client.py", line 468, in post
http_response = self._post(
File "/opt/app-root/lib64/python3.9/site-packages/trino/client.py", line 854, in decorated
raise error
File "/opt/app-root/lib64/python3.9/site-packages/trino/client.py", line 841, in decorated
result = func(*args, **kwargs)
File "/opt/app-root/lib64/python3.9/site-packages/requests/sessions.py", line 635, in post
return self.request("POST", url, data=data, json=json, **kwargs)
File "/opt/app-root/lib64/python3.9/site-packages/requests/sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
File "/opt/app-root/lib64/python3.9/site-packages/requests/sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
File "/opt/app-root/lib64/python3.9/site-packages/requests/adapters.py", line 563, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='insights-trino.whitewater.ibm.com', port=443): Max retries exceeded with url: /v1/statement (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)')))
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] musicmuthu commented on issue #20960: Connection to Trino using SSL certificate no longer working
Posted by "musicmuthu (via GitHub)" <gi...@apache.org>.
musicmuthu commented on issue #20960:
URL: https://github.com/apache/superset/issues/20960#issuecomment-1400451960
#bug
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] musicmuthu commented on issue #20960: Connection to Trino using SSL certificate no longer working
Posted by GitBox <gi...@apache.org>.
musicmuthu commented on issue #20960:
URL: https://github.com/apache/superset/issues/20960#issuecomment-1398068915
@justineyster i have tried to connect the trino using certificate connection but it' still point to password authentication.
i tried from both versions superset 2.0.0 and 2.0.1
**trino://admin:admin@trino-test.dna.com:30999/dna?verify=/vault/secrets/admin.pem**
also the ca certificate also placed in ROOT Certificate path
If i try from trino.jar from my linux local machine i'm able to login using certificate but not from superset
my client certificate already there in superset and ca certificate loaded into the superset
my superset_config for certificate auth changes is below:
from typing import Any, Callable, Dict, List, Optional, Type
from trino.auth import Authentication
ALLOWED_EXTRA_AUTHENTICATIONS: Dict[str, Dict[str, Callable[..., Any]]] = {
"trino": {
"custom_auth": 'certificate'
},
}
SSL_CERT_FILE = "/vault/secrets/adfs.pem /usr/local/lib/python3.8/site-packages/certifi/cacert.pem"
SSL_CERT_PATH = "/etc/ssl/certs"
SSL_CERT_DIR = "/usr/local/lib/python3.8/site-packages/certifi"
can you please point out where i did the mistakes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] justineyster closed issue #20960: Connection to Trino using SSL certificate no longer working
Posted by GitBox <gi...@apache.org>.
justineyster closed issue #20960: Connection to Trino using SSL certificate no longer working
URL: https://github.com/apache/superset/issues/20960
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] justineyster commented on issue #20960: Connection to Trino using SSL certificate no longer working
Posted by GitBox <gi...@apache.org>.
justineyster commented on issue #20960:
URL: https://github.com/apache/superset/issues/20960#issuecomment-1253856036
We fixed this by specifying the cert in the database connection options (Advanced -> Security -> Root Certificate).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org