You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2022/04/11 09:56:32 UTC
[GitHub] [superset] PatBriPerso opened a new issue, #19643: Use keycloak as OIDC/IdP provider but cannot access to users list
PatBriPerso opened a new issue, #19643:
URL: https://github.com/apache/superset/issues/19643
I setup superset to authenticate the users with keycloak as IdP.
I'm able to connect to superset after validating a login/pwd on keycloak but I cannot access the users list page (/users/list) as an Admin.
#### How to reproduce the bug
Setup superset as described in "Additional context".
Connect to superset through keycloak as an Admin
Click on the menu Settings > List Users
### Expected results
See the users list
### Actual results
I'm back on the welcome page (/superset/welcome/) with a message saying "Access is Denied".
#### Screenshots
n/a
### Environment
(please complete the following information):
- browser type and version:
Brave Version 1.37.109 Chromium: 100.0.4896.60 (Build officiel) (64 bits)
- superset version: `superset version`
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Superset 1.4.2
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
- python version: `python --version`
Python 3.8.12
- node.js version: `node -v`
n/a
- any feature flags active:
ALERT_REPORTS
### Checklist
Make sure to follow these steps before submitting your issue - thank you!
- [X] I have checked the superset logs for python stacktraces and included it here as text if there are any.
No stacktrace
- [X] I have reproduced the issue with at least the latest released version of superset.
- [X] I have checked the issue tracker for the same issue and I haven't found one similar.
### Additional context
I have a keycloak setup with the url: https://auth.mydomain.com/
I create a realm named "demo" and a user on this realm.
I add a client named "superset" (client ID) on this realm with a Client Protocol "openid-connect" and a Root URL "https://superset.demo.mydomain.com/"
My superset is accessible with the url: https://superset.demo.mydomain.com/
I use the Docker version of superset deployed on a Docker Swarm cluster. I use Traefik to route the HTTP requests to the containers (superset and keycloak).
To setup superset with keycloak, I follow these posts:
- https://github.com/apache/superset/discussions/13915
- https://stackoverflow.com/questions/54010314/using-keycloakopenid-connect-with-apache-superset/54024394#54024394
But I modify some files so I describe below my whole configuration.
Content of `/app/docker/requirements-local.txt`:
```
clickhouse-driver>=0.2.0
clickhouse-sqlalchemy>=0.1.6,<0.2.0
mysql-connector-python
flask-oidc==1.3.0
```
The first 2 packages are used to connect to my clickhouse database. The third one is used to have the superset database on MySQL (instead of postgres).
Only the fourth one is related to keycloak to enable OIDC (OpenID Connect).
Content of `/app/pythonpath/superset_config.py`:
```python
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#
# This file is included in the final Docker image and SHOULD be overridden when
# deploying the image to prod. Settings configured here are intended for use in local
# development environments. Also note that superset_config_docker.py is imported
# as a final step as a means to override "defaults" configured here
#
import logging
import os
from datetime import timedelta
from typing import Optional
from cachelib.file import FileSystemCache
from celery.schedules import crontab
logger = logging.getLogger()
def get_env_variable(var_name: str, default: Optional[str] = None) -> str:
"""Get the environment variable or raise exception."""
try:
return os.environ[var_name]
except KeyError:
if default is not None:
return default
else:
error_msg = "The environment variable {} was missing, abort...".format(
var_name
)
raise EnvironmentError(error_msg)
DATABASE_DIALECT = get_env_variable("DATABASE_DIALECT")
DATABASE_USER = get_env_variable("DATABASE_USER")
DATABASE_PASSWORD = get_env_variable("DATABASE_PASSWORD")
DATABASE_HOST = get_env_variable("DATABASE_HOST")
DATABASE_PORT = get_env_variable("DATABASE_PORT")
DATABASE_DB = get_env_variable("DATABASE_DB")
# The SQLAlchemy connection string.
SQLALCHEMY_DATABASE_URI = "%s://%s:%s@%s:%s/%s" % (
DATABASE_DIALECT,
DATABASE_USER,
DATABASE_PASSWORD,
DATABASE_HOST,
DATABASE_PORT,
DATABASE_DB,
)
REDIS_HOST = get_env_variable("REDIS_HOST")
REDIS_PORT = get_env_variable("REDIS_PORT")
REDIS_CELERY_DB = get_env_variable("REDIS_CELERY_DB", "0")
REDIS_RESULTS_DB = get_env_variable("REDIS_RESULTS_DB", "1")
##### Change from original file
#RESULTS_BACKEND = FileSystemCache("/app/superset_home/sqllab")
# Put results on Redis not on a file that is not shared among container on Swarm stack
from cachelib.redis import RedisCache
RESULTS_BACKEND = RedisCache(host=REDIS_HOST, port=REDIS_PORT, key_prefix='superset_results')
##### End of change
class CeleryConfig(object):
BROKER_URL = f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_CELERY_DB}"
CELERY_IMPORTS = ("superset.sql_lab", "superset.tasks")
CELERY_RESULT_BACKEND = f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_RESULTS_DB}"
CELERYD_LOG_LEVEL = "DEBUG"
CELERYD_PREFETCH_MULTIPLIER = 1
CELERY_ACKS_LATE = False
CELERYBEAT_SCHEDULE = {
"reports.scheduler": {
"task": "reports.scheduler",
"schedule": crontab(minute="*", hour="*"),
},
"reports.prune_log": {
"task": "reports.prune_log",
"schedule": crontab(minute=10, hour=0),
},
}
CELERY_CONFIG = CeleryConfig
FEATURE_FLAGS = {"ALERT_REPORTS": True}
ALERT_REPORTS_NOTIFICATION_DRY_RUN = True
WEBDRIVER_BASEURL = "http://superset:8088/"
# The base URL for the email report hyperlinks.
WEBDRIVER_BASEURL_USER_FRIENDLY = WEBDRIVER_BASEURL
SQLLAB_CTAS_NO_LIMIT = True
#
# Optionally import superset_config_docker.py (which will have been included on
# the PYTHONPATH) in order to allow for local settings to be overridden
#
try:
import superset_config_docker
from superset_config_docker import * # noqa
logger.info(
f"Loaded your Docker configuration at " f"[{superset_config_docker.__file__}]"
)
except ImportError:
logger.info("Using default Docker config...")
```
Content of `/app/pythonpath/superset_config_docker.py`:
```python
import os
from typing import Optional
def get_env_variable(var_name: str, default: Optional[str] = None) -> str:
"""Get the environment variable or raise exception."""
try:
return os.environ[var_name]
except KeyError:
if default is not None:
return default
else:
error_msg = "The environment variable {} was missing, abort...".format(
var_name
)
raise EnvironmentError(error_msg)
# The allowed translation for you app
LANGUAGES = {
"en": {"flag": "us", "name": "English"},
#"es": {"flag": "es", "name": "Spanish"},
#"it": {"flag": "it", "name": "Italian"},
"fr": {"flag": "fr", "name": "French"},
#"zh": {"flag": "cn", "name": "Chinese"},
#"ja": {"flag": "jp", "name": "Japanese"},
#"de": {"flag": "de", "name": "German"},
#"pt": {"flag": "pt", "name": "Portuguese"},
#"pt_BR": {"flag": "br", "name": "Brazilian Portuguese"},
#"ru": {"flag": "ru", "name": "Russian"},
#"ko": {"flag": "kr", "name": "Korean"},
#"sk": {"flag": "sk", "name": "Slovak"},
#"sl": {"flag": "si", "name": "Slovenian"},
}
ENABLE_PROXY_FIX = True
#---------------------------KEYCLOACK ----------------------------
# See: https://github.com/apache/superset/discussions/13915
# See: https://stackoverflow.com/questions/54010314/using-keycloakopenid-connect-with-apache-superset/54024394#54024394
from keycloak_security_manager import OIDCSecurityManager
from flask_appbuilder.security.manager import AUTH_OID
OIDC_ENABLE = get_env_variable("OIDC_ENABLE", 'False')
if OIDC_ENABLE == 'True':
AUTH_TYPE = AUTH_OID
SECRET_KEY = get_env_variable("SECRET_KEY", 'ChangeThisKeyPlease')
OIDC_CLIENT_SECRETS = get_env_variable("OIDC_CLIENT_SECRETS", '/app/pythonpath/client_secret.json')
OIDC_ID_TOKEN_COOKIE_SECURE = False
OIDC_REQUIRE_VERIFIED_EMAIL = False
OIDC_OPENID_REALM = get_env_variable("OIDC_OPENID_REALM")
OIDC_INTROSPECTION_AUTH_METHOD = 'client_secret_post'
CUSTOM_SECURITY_MANAGER = OIDCSecurityManager
AUTH_USER_REGISTRATION = True
AUTH_USER_REGISTRATION_ROLE = get_env_variable("AUTH_USER_REGISTRATION_ROLE", 'Admin')
#--------------------------------------------------------------
```
Content of `/app/pythonpath/keycloak_security_manager.py`:
```python
from flask_appbuilder.security.manager import AUTH_OID
from superset.security import SupersetSecurityManager
from flask_oidc import OpenIDConnect
from flask_appbuilder.security.views import AuthOIDView
from flask_login import login_user
from urllib.parse import quote
from flask_appbuilder.views import ModelView, SimpleFormView, expose
import logging
from flask import redirect, request
class OIDCSecurityManager(SupersetSecurityManager):
def __init__(self, appbuilder):
super(OIDCSecurityManager, self).__init__(appbuilder)
if self.auth_type == AUTH_OID:
self.oid = OpenIDConnect(self.appbuilder.get_app)
self.authoidview = AuthOIDCView
class AuthOIDCView(AuthOIDView):
@expose('/login/', methods=['GET', 'POST'])
def login(self, flag=True):
sm = self.appbuilder.sm
oidc = sm.oid
@self.appbuilder.sm.oid.require_login
def handle_login():
user = sm.auth_user_oid(oidc.user_getfield('email'))
if user is None:
info = oidc.user_getinfo(['preferred_username', 'given_name', 'family_name', 'email'])
user = sm.add_user(info.get('preferred_username'), info.get('given_name'), info.get('family_name'),
info.get('email'), sm.find_role('Admin'))
login_user(user, remember=False)
return redirect(self.appbuilder.get_url_for_index)
return handle_login()
@expose('/logout/', methods=['GET', 'POST'])
def logout(self):
oidc = self.appbuilder.sm.oid
oidc.logout()
super(AuthOIDCView, self).logout()
redirect_url = request.url_root.strip('/') + self.appbuilder.get_url_for_login
return redirect(
oidc.client_secrets.get('issuer') + '/protocol/openid-connect/logout?redirect_uri=' + quote(redirect_url))
```
Content of `/app/pythonpath/client_secret.json`:
```json
{
"web": {
"issuer": "https://auth.mydomain.com/realms/demo",
"auth_uri": "https://auth.mydomain.com/realms/demo/protocol/openid-connect/auth",
"client_id": "superset",
"client_secret": "<Client Secret>",
"redirect_uris": [
"https://superset.demo.mydomain.com/*"
],
"userinfo_uri": "https://auth.mydomain.com/realms/demo/protocol/openid-connect/userinfo",
"token_uri": "https://auth.mydomain.com/realms/demo/protocol/openid-connect/token",
"token_introspection_uri": "https://auth.mydomain.com/realms/demo/protocol/openid-connect/token/introspect"
}
}
```
The environment variables I pass to my Docker Swarm service:
```
FLASK_ENV=production
SUPERSET_ENV=production
SUPERSET_LOAD_EXAMPLES=no
CYPRESS_CONFIG=false
SUPERSET_PORT=8088
OIDC_ENABLE=True
OIDC_OPENID_REALM=demo
```
NOTA: I remove my environment variables related to mysql and redis.
Thanks for your help. Tell me if some information is missing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Use keycloak as OIDC/IdP provider but cannot access to users list [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on issue #19643:
URL: https://github.com/apache/superset/issues/19643#issuecomment-1942655897
@PatBriPerso are you still facing this issue, or should we close it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] PatBriPerso commented on issue #19643: Use keycloak as OIDC/IdP provider but cannot access to users list
Posted by GitBox <gi...@apache.org>.
PatBriPerso commented on issue #19643:
URL: https://github.com/apache/superset/issues/19643#issuecomment-1096217047
Thanks @fedepad.
In fact, I change the role of my user with the superset UI and give him the Admin role.
Here is how I do that:
- I start my stack with `OIDC_ENABLE=False` so I can connect with the default Admin account (stored in the DB) and I change its password
- I update my stack with `OIDC_ENABLE=True` and connect with a keycloak account. A new user was created in superset DB
- I change back to `OIDC_ENABLE=False`, connect with the default Admin account and change the role of my keycloak user to Admin
- Finally, I change again to `OIDC_ENABLE=True`, connect with my keycloak account and have the access issue (to the users list page)
I have no specific roles for my keycloak user. I think he has the default keycloak roles but I do not know if those roles are sent to superset. I think roles on keycloak and roles on superset are not related (but I'm not sure).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] rahul149386 commented on issue #19643: Use keycloak as OIDC/IdP provider but cannot access to users list
Posted by GitBox <gi...@apache.org>.
rahul149386 commented on issue #19643:
URL: https://github.com/apache/superset/issues/19643#issuecomment-1284247060
@fedepad @PatBriPerso
I created same user in keycloak and superset eventhough im able to see None user inevent log and througing timeout error any help regarding this.
Seeing below error.
![Uploading 1666195343939712558927051194224.jpg…]()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Use keycloak as OIDC/IdP provider but cannot access to users list [superset]
Posted by "resulraveendran (via GitHub)" <gi...@apache.org>.
resulraveendran commented on issue #19643:
URL: https://github.com/apache/superset/issues/19643#issuecomment-2063229959
Hi,
I got the same issue when i integrate with Zitadel is there any solution for this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] fedepad commented on issue #19643: Use keycloak as OIDC/IdP provider but cannot access to users list
Posted by GitBox <gi...@apache.org>.
fedepad commented on issue #19643:
URL: https://github.com/apache/superset/issues/19643#issuecomment-1095565013
Questions which I think are helpful for debugging:
- What roles does your user on IDP has?
- Is `Admin` an actual role in the IDP or not? If not, do you have the mapping from IDP roles to FAB roles somewhere?
- Does the role of the admin user you are using to login is the one you expect in the superset/FAB backend db? Check which role this user has just by looking at the backend db (MySQL in your case, since you cannot check the users list from UI...)
These are just to start with, first sanity checks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] rahul149386 commented on issue #19643: Use keycloak as OIDC/IdP provider but cannot access to users list
Posted by GitBox <gi...@apache.org>.
rahul149386 commented on issue #19643:
URL: https://github.com/apache/superset/issues/19643#issuecomment-1284252669
![Uploading 16661956164843434709105318764320.jpg…]()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] debimishra89 commented on issue #19643: Use keycloak as OIDC/IdP provider but cannot access to users list
Posted by GitBox <gi...@apache.org>.
debimishra89 commented on issue #19643:
URL: https://github.com/apache/superset/issues/19643#issuecomment-1300731622
Can we do role mapping between keycloak and superset with AUTH_TYPE=AUTH_OID??
Or is it only available with AUTH_OAUTH
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] rahul149386 commented on issue #19643: Use keycloak as OIDC/IdP provider but cannot access to users list
Posted by GitBox <gi...@apache.org>.
rahul149386 commented on issue #19643:
URL: https://github.com/apache/superset/issues/19643#issuecomment-1284254201
> @fedepad @PatBriPerso I created same user in keycloak and superset eventhough im able to see None user inevent log and througing timeout error any help regarding this. Seeing below
![Uploading 20221019_213835.jpg…]()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org