You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/07/02 21:26:27 UTC
[GitHub] [airflow] mdcsaenz opened a new issue #16783: Airflow 2.1.0 Oauth for google Too Many Redirects b/c Google User does not have Role
mdcsaenz opened a new issue #16783:
URL: https://github.com/apache/airflow/issues/16783
The issue is similar to this, however I have an updated airflow version AND updated packages than the ones suggested here and I am still getting the same outcome. When using google auth in airflow and attempting to sign in, we get an ERR_TOO_MANY_REDIRECTS. Explanation, Symptom, and not a solution, but an obvious reason of why this occurs, just need a fix.
- **Apache Airflow version**:
Version: v2.1.0
Git Version: .release:2.1.0+304e174674ff6921cb7ed79c0158949b50eff8fe
- **Kubernetes version (if you are using kubernetes)** (use `kubectl version`):
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.7", GitCommit:"1dd5338295409edcfff11505e7bb246f0d325d15", GitTreeState:"clean", BuildDate:"2021-01-13T13:23:52Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"19+", GitVersion:"v1.19.10-gke.1600", GitCommit:"7b8e568a7fb4c9d199c2ba29a5f7d76f6b4341c2", GitTreeState:"clean", BuildDate:"2021-05-07T09:18:53Z", GoVersion:"go1.15.10b5", Compiler:"gc", Platform:"linux/amd64"}
- **Environment**: Staging
- **Cloud provider or hardware configuration**: GKE on
- **OS** (e.g. from /etc/os-release):
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
- **Kernel** (e.g. `uname -a`):
- **Install tools**:
- **Others**:
**What happened**:
When using google auth in airflow and attempting to sign in, we get an ERR_TOO_MANY_REDIRECTS.
ERR_TOO_MANY_REDIRECTS on WEBPAGE
**What you expected to happen**:
I log in as my user and it assigns a default Role over Viewer at the very least OR uses our mappings in web_server config python file.
<!-- What do you think went wrong? -->
We realized that we get stuck in the loop, b/c the user will be in the users table in airflow but without a Role (its literally empty).
**How to reproduce it**:
I add the Admin role, the page that has the redirects, then refreshes and lets me in to the Airflow UI. However, when I sign out and sigin in again, my users Role is then erased and it starts the redirect cycle again.
As you can see there is no Role after I attempt to log in.
```
id | username | email | first_name | last_name | roles
===+==============================+=========================+============+===========+======
1 | admin | admin@example.com | admin | admin | Admin
2 | google_############ | msaenz@company.com | Cat | Says |
```
I run the command: `airflow users add-role -r Admin -u google_#################`
Then the page takes me to the UI and the table now looks like this:
```
id | username | email | first_name | last_name | roles
===+==============================+=========================+============+===========+======
1 | admin | admin@example.com | admin | admin | Admin
2 | google_############ | msaenz@company.com | Cat | Says | Admin
```
As minimally and precisely as possible. Keep in mind we do not have access to your cluster or dags.
If you are using kubernetes, please attempt to recreate the issue using minikube or kind.
## Install minikube/kind
- Minikube https://minikube.sigs.k8s.io/docs/start/
- Kind https://kind.sigs.k8s.io/docs/user/quick-start/
How often does this problem occur? Once? Every time etc? This occurs all the time
Here is the webserver_config.py
```
import os
from flask_appbuilder.security.manager import AUTH_OAUTH
AUTH_TYPE = AUTH_OAUTH
AUTH_ROLE_ADMIN="Admin"
AUTH_USER_REGISTRATION = False
AUTH_USER_REGISTRATION_ROLE = "Admin"
OIDC_COOKIE_SECURE = False
CSRF_ENABLED = False
WTF_CSRF_ENABLED = True
AUTH_ROLES_MAPPING = {"Engineering": ["Ops"],"Admins": ["Admin"]}
AUTH_ROLES_SYNC_AT_LOGIN = True
OAUTH_PROVIDERS = [
{
'name': 'google', 'icon': 'fa-google',
'token_key': 'access_token',
'remote_app': {
'client_id': '#####################.apps.googleusercontent.com',
'client_secret': '######################',
'api_base_url': 'https://www.googleapis.com/oauth2/v2/',
'whitelist': ['@company.com'], # optional
'client_kwargs': {
'scope': 'email profile'
},
'request_token_url': None,
'access_token_url': 'https://accounts.google.com/o/oauth2/token',
'authorize_url': 'https://accounts.google.com/o/oauth2/auth'},
}
]
```
Here is the pip freeze:
```
adal==1.2.7
alembic==1.6.2
amqp==2.6.1
anyio==3.2.1
apache-airflow==2.1.0
apache-airflow-providers-amazon==1.4.0
apache-airflow-providers-celery==1.0.1
apache-airflow-providers-cncf-kubernetes==1.2.0
apache-airflow-providers-docker==1.2.0
apache-airflow-providers-elasticsearch==1.0.4
apache-airflow-providers-ftp==1.1.0
apache-airflow-providers-google==3.0.0
apache-airflow-providers-grpc==1.1.0
apache-airflow-providers-hashicorp==1.0.2
apache-airflow-providers-http==1.1.1
apache-airflow-providers-imap==1.0.1
apache-airflow-providers-microsoft-azure==2.0.0
apache-airflow-providers-mysql==1.1.0
apache-airflow-providers-postgres==1.0.2
apache-airflow-providers-redis==1.0.1
apache-airflow-providers-sendgrid==1.0.2
apache-airflow-providers-sftp==1.2.0
apache-airflow-providers-slack==3.0.0
apache-airflow-providers-sqlite==1.0.2
apache-airflow-providers-ssh==1.3.0
apispec==3.3.2
appdirs==1.4.4
argcomplete==1.12.3
async-generator==1.10
attrs==20.3.0
azure-batch==10.0.0
azure-common==1.1.27
azure-core==1.13.0
azure-cosmos==3.2.0
azure-datalake-store==0.0.52
azure-identity==1.5.0
azure-keyvault==4.1.0
azure-keyvault-certificates==4.2.1
azure-keyvault-keys==4.3.1
azure-keyvault-secrets==4.2.0
azure-kusto-data==0.0.45
azure-mgmt-containerinstance==1.5.0
azure-mgmt-core==1.2.2
azure-mgmt-datafactory==1.1.0
azure-mgmt-datalake-nspkg==3.0.1
azure-mgmt-datalake-store==0.5.0
azure-mgmt-nspkg==3.0.2
azure-mgmt-resource==16.1.0
azure-nspkg==3.0.2
azure-storage-blob==12.8.1
azure-storage-common==2.1.0
azure-storage-file==2.1.0
Babel==2.9.1
bcrypt==3.2.0
billiard==3.6.4.0
blinker==1.4
boto3==1.17.71
botocore==1.20.71
cached-property==1.5.2
cachetools==4.2.2
cattrs==1.0.0
celery==4.4.7
certifi==2020.12.5
cffi==1.14.5
chardet==3.0.4
click==7.1.2
clickclick==20.10.2
cloudpickle==1.4.1
colorama==0.4.4
colorlog==5.0.1
commonmark==0.9.1
contextvars==2.4
croniter==1.0.13
cryptography==3.4.7
dask==2021.3.0
dataclasses==0.7
defusedxml==0.7.1
dill==0.3.1.1
distlib==0.3.1
distributed==2.19.0
dnspython==1.16.0
docker==3.7.3
docker-pycreds==0.4.0
docutils==0.17.1
elasticsearch==7.5.1
elasticsearch-dbapi==0.1.0
elasticsearch-dsl==7.3.0
email-validator==1.1.2
eventlet==0.31.0
filelock==3.0.12
Flask==1.1.2
Flask-AppBuilder==3.3.0
Flask-Babel==1.0.0
Flask-Caching==1.10.1
Flask-JWT-Extended==3.25.1
Flask-Login==0.4.1
Flask-OpenID==1.2.5
Flask-SQLAlchemy==2.5.1
Flask-WTF==0.14.3
flower==0.9.7
gevent==21.1.2
google-ads==4.0.0
google-api-core==1.26.3
google-api-python-client==1.12.8
google-auth==1.30.0
google-auth-httplib2==0.1.0
google-auth-oauthlib==0.4.4
google-cloud-automl==2.3.0
google-cloud-bigquery==2.16.0
google-cloud-bigquery-datatransfer==3.1.1
google-cloud-bigquery-storage==2.4.0
google-cloud-bigtable==1.7.0
google-cloud-container==1.0.1
google-cloud-core==1.6.0
google-cloud-datacatalog==3.1.1
google-cloud-dataproc==2.3.1
google-cloud-dlp==1.0.0
google-cloud-kms==2.2.0
google-cloud-language==1.3.0
google-cloud-logging==2.3.1
google-cloud-memcache==0.3.0
google-cloud-monitoring==2.2.1
google-cloud-os-login==2.1.0
google-cloud-pubsub==2.4.2
google-cloud-redis==2.1.0
google-cloud-secret-manager==1.0.0
google-cloud-spanner==1.19.1
google-cloud-speech==1.3.2
google-cloud-storage==1.38.0
google-cloud-tasks==2.2.0
google-cloud-texttospeech==1.0.1
google-cloud-translate==1.7.0
google-cloud-videointelligence==1.16.1
google-cloud-vision==1.0.0
google-cloud-workflows==0.3.0
google-crc32c==1.1.2
google-resumable-media==1.2.0
googleapis-common-protos==1.53.0
graphviz==0.16
greenlet==1.1.0
grpc-google-iam-v1==0.12.3
grpcio==1.37.1
grpcio-gcp==0.2.2
gunicorn==20.1.0
h11==0.12.0
HeapDict==1.0.1
httpcore==0.13.6
httplib2==0.17.4
httpx==0.18.2
humanize==3.5.0
hvac==0.10.11
idna==2.10
immutables==0.15
importlib-metadata==1.7.0
importlib-resources==1.5.0
inflection==0.5.1
iso8601==0.1.14
isodate==0.6.0
itsdangerous==1.1.0
Jinja2==2.11.3
jmespath==0.10.0
json-merge-patch==0.2
jsonschema==3.2.0
kombu==4.6.11
kubernetes==11.0.0
lazy-object-proxy==1.4.3
ldap3==2.9
libcst==0.3.18
lockfile==0.12.2
Mako==1.1.4
Markdown==3.3.4
MarkupSafe==1.1.1
marshmallow==3.12.1
marshmallow-enum==1.5.1
marshmallow-oneofschema==2.1.0
marshmallow-sqlalchemy==0.23.1
msal==1.11.0
msal-extensions==0.3.0
msgpack==1.0.2
msrest==0.6.21
msrestazure==0.6.4
mypy-extensions==0.4.3
mysql-connector-python==8.0.22
mysqlclient==2.0.3
numpy==1.19.5
oauthlib==2.1.0
openapi-schema-validator==0.1.5
openapi-spec-validator==0.3.0
packaging==20.9
pandas==1.1.5
pandas-gbq==0.14.1
paramiko==2.7.2
pendulum==2.1.2
pep562==1.0
plyvel==1.3.0
portalocker==1.7.1
prison==0.1.3
prometheus-client==0.8.0
proto-plus==1.18.1
protobuf==3.16.0
psutil==5.8.0
psycopg2-binary==2.8.6
pyarrow==3.0.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.20
pydata-google-auth==1.2.0
Pygments==2.9.0
PyJWT==1.7.1
PyNaCl==1.4.0
pyOpenSSL==19.1.0
pyparsing==2.4.7
pyrsistent==0.17.3
pysftp==0.2.9
python-daemon==2.3.0
python-dateutil==2.8.1
python-editor==1.0.4
python-http-client==3.3.2
python-ldap==3.3.1
python-nvd3==0.15.0
python-slugify==4.0.1
python3-openid==3.2.0
pytz==2021.1
pytzdata==2020.1
PyYAML==5.4.1
redis==3.5.3
requests==2.25.1
requests-oauthlib==1.1.0
rfc3986==1.5.0
rich==9.2.0
rsa==4.7.2
s3transfer==0.4.2
sendgrid==6.7.0
setproctitle==1.2.2
six==1.16.0
slack-sdk==3.5.1
sniffio==1.2.0
sortedcontainers==2.3.0
SQLAlchemy==1.3.24
SQLAlchemy-JSONField==1.0.0
SQLAlchemy-Utils==0.37.2
sshtunnel==0.1.5
starkbank-ecdsa==1.1.0
statsd==3.3.0
swagger-ui-bundle==0.0.8
tabulate==0.8.9
tblib==1.7.0
tenacity==6.2.0
termcolor==1.1.0
text-unidecode==1.3
toolz==0.11.1
tornado==6.1
typing==3.7.4.3
typing-extensions==3.7.4.3
typing-inspect==0.6.0
unicodecsv==0.14.1
uritemplate==3.0.1
urllib3==1.25.11
vine==1.3.0
virtualenv==20.4.6
watchtower==0.7.3
websocket-client==0.59.0
Werkzeug==1.0.1
WTForms==2.3.3
zict==2.0.0
zipp==3.4.1
zope.event==4.5.0
zope.interface==5.4.0
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #16783: Airflow 2.1.0 Oauth for google Too Many Redirects b/c Google User does not have Role
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #16783:
URL: https://github.com/apache/airflow/issues/16783#issuecomment-875729318
@jedcunningham Cool
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on issue #16783: Airflow 2.1.0 Oauth for google Too Many Redirects b/c Google User does not have Role
Posted by GitBox <gi...@apache.org>.
jedcunningham commented on issue #16783:
URL: https://github.com/apache/airflow/issues/16783#issuecomment-875104038
> But it definitely erased the Role
Yep, that's the behavior with `AUTH_ROLES_SYNC_AT_LOGIN` and no matching roles in `AUTH_ROLE_MAPPING`. You could also disable `AUTH_ROLES_SYNC_AT_LOGIN` and manage the roles yourself.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on issue #16783: Airflow 2.1.0 Oauth for google Too Many Redirects b/c Google User does not have Role
Posted by GitBox <gi...@apache.org>.
jedcunningham commented on issue #16783:
URL: https://github.com/apache/airflow/issues/16783#issuecomment-873291945
`AUTH_USER_REGISTRATION_ROLE` only applies if `AUTH_USER_REGISTRATION` is true.
Looks like the FAB google oauth provider doesn't support AUTH_ROLE_MAPPING by default:
https://github.com/dpgaspar/Flask-AppBuilder/blob/95947e84e04a999a474dfe8620fb0f36d71f0467/flask_appbuilder/security/manager.py#L585-L590
You'd need a custom security manager to return `role_keys`, for example like is done in the OP of #14829.
(I'm sure FAB would love to have support for this with the google oauth provider, so consider contributing it back if you get it working).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #16783: Airflow 2.1.0 Oauth for google Too Many Redirects b/c Google User does not have Role
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #16783:
URL: https://github.com/apache/airflow/issues/16783#issuecomment-875528146
Is this an Airflow bug, or a FAB bug?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mdcsaenz commented on issue #16783: Airflow 2.1.0 Oauth for google Too Many Redirects b/c Google User does not have Role
Posted by GitBox <gi...@apache.org>.
mdcsaenz commented on issue #16783:
URL: https://github.com/apache/airflow/issues/16783#issuecomment-874996824
> `AUTH_USER_REGISTRATION_ROLE` only applies if `AUTH_USER_REGISTRATION` is true.
>
> Looks like the FAB google oauth provider doesn't support `AUTH_ROLE_MAPPING` by default:
> https://github.com/dpgaspar/Flask-AppBuilder/blob/95947e84e04a999a474dfe8620fb0f36d71f0467/flask_appbuilder/security/manager.py#L585-L590
>
> You'd need a custom security manager to return `role_keys`, for example like is done in the OP of #14829.
> (I'm sure FAB would love to have support for this with the google oauth provider, so consider contributing it back if you get it working).
Ah okay, I will see if that works and then close the ticket if that is the case. But it definitely erased the Role even with AUTH_USER_REGISTRATION_ROLE and AUTH_USER_REGISTRATION are both false as well. I made AUTH_USER_ROLE False as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr closed issue #16783: Airflow 2.1.0 Oauth for google Too Many Redirects b/c Google User does not have Role
Posted by GitBox <gi...@apache.org>.
uranusjr closed issue #16783:
URL: https://github.com/apache/airflow/issues/16783
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr closed issue #16783: Airflow 2.1.0 Oauth for google Too Many Redirects b/c Google User does not have Role
Posted by GitBox <gi...@apache.org>.
uranusjr closed issue #16783:
URL: https://github.com/apache/airflow/issues/16783
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #16783: Airflow 2.1.0 Oauth for google Too Many Redirects b/c Google User does not have Role
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #16783:
URL: https://github.com/apache/airflow/issues/16783#issuecomment-875725650
This is likely a "wont Fix" from airflow's side I'm afraid then.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on issue #16783: Airflow 2.1.0 Oauth for google Too Many Redirects b/c Google User does not have Role
Posted by GitBox <gi...@apache.org>.
jedcunningham commented on issue #16783:
URL: https://github.com/apache/airflow/issues/16783#issuecomment-875727800
I haven't poked around yet, but we may be able to handle the 'no role' scenario more gracefully on our side. Let me take a stab at that before we close this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham edited a comment on issue #16783: Airflow 2.1.0 Oauth for google Too Many Redirects b/c Google User does not have Role
Posted by GitBox <gi...@apache.org>.
jedcunningham edited a comment on issue #16783:
URL: https://github.com/apache/airflow/issues/16783#issuecomment-873291945
`AUTH_USER_REGISTRATION_ROLE` only applies if `AUTH_USER_REGISTRATION` is true.
Looks like the FAB google oauth provider doesn't support `AUTH_ROLE_MAPPING` by default:
https://github.com/dpgaspar/Flask-AppBuilder/blob/95947e84e04a999a474dfe8620fb0f36d71f0467/flask_appbuilder/security/manager.py#L585-L590
You'd need a custom security manager to return `role_keys`, for example like is done in the OP of #14829.
(I'm sure FAB would love to have support for this with the google oauth provider, so consider contributing it back if you get it working).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on issue #16783: Airflow 2.1.0 Oauth for google Too Many Redirects b/c Google User does not have Role
Posted by GitBox <gi...@apache.org>.
jedcunningham commented on issue #16783:
URL: https://github.com/apache/airflow/issues/16783#issuecomment-875636662
FAB.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org