You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/04/07 15:43:46 UTC
[GitHub] [airflow] Abhishekchechani opened a new issue #8179: Airflow LDAP
authentication with RBAC features
Abhishekchechani opened a new issue #8179: Airflow LDAP authentication with RBAC features
URL: https://github.com/apache/airflow/issues/8179
I am trying to enable Airflow LDAP authentication with RBAC features and did the following changes:
1. Removed LDAP section from airflow.cfg
2. Modified airflow.cfg: added rbac = true and removed authentication = True under the [webserver] section
3. Create a webserver_config.py file in the AIRFLOW_HOME directory
The webserver_config.py file contains:
import os
from airflow import configuration as conf
from flask_appbuilder.security.manager import AUTH_LDAP
basedir = os.path.abspath(os.path.dirname(__file__))
SQLALCHEMY_DATABASE_URI = conf.get(‘core’, ‘SQL_ALCHEMY_CONN’)
CSRF_ENABLED = True
AUTH_TYPE = AUTH_LDAP
AUTH_ROLE_ADMIN = ‘Admin’
AUTH_USER_REGISTRATION = True
AUTH_USER_REGISTRATION_ROLE = “Admin”
AUTH_LDAP_SERVER = ‘ldaps://ldap.xxx.yyy.net:636‘
AUTH_LDAP_SEARCH = “ou=Users,o=corp”
AUTH_LDAP_BIND_USER = ‘cn=ldap-proxy,ou=Users,o=corp’
AUTH_LDAP_BIND_PASSWORD = ‘YOUR_PASSWORD’
AUTH_LDAP_UID_FIELD = ‘uid’
AUTH_LDAP_USE_TLS = False
AUTH_LDAP_ALLOW_SELF_SIGNED = False
AUTH_LDAP_TLS_CACERTFILE = ‘/etc/ssl/certs/ldap.crt’
After the above changes, we are able to login to Airflow with LDAP credentials. But the problem is that all the users have the Admin role after self registration, because we have given this value in AUTH_USER_REGISTRATION_ROLE = “Admin”.
How can we dynamically assign the AUTH_USER_REGISTRATION_ROLE based on the users LDAP role? We have different users like tester, developer and operation user but with the above webserver config file all users are automatically assigned the Admin role via Flask_appbuilder.security under manager.py file.
Is there any way to create the customize manager file and while login refer this customize file instead of Flask_appbuilder.security.manager.py file.
Because I can not change directly in flask_appbuilder.security manager.py file and add the our customize role and assign in AUTH_USER_REGISTRATION_ROLE based on the users LDAP role
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] xpyct600 commented on issue #8179: Airflow LDAP
authentication with RBAC features
Posted by GitBox <gi...@apache.org>.
xpyct600 commented on issue #8179: Airflow LDAP authentication with RBAC features
URL: https://github.com/apache/airflow/issues/8179#issuecomment-612755217
Hi @Abhishekchechani ,
Seems it can't find ldap group.
Place "print(group_cn)" to try block to see group list. In my case:
```
CN=airflow_op_test,OU=airflow,OU=groups,OU=qiwi,DC=hq,DC=qiwi,DC=com
CN=airflow_admin,OU=airflow,OU=groups,OU=qiwi,DC=hq,DC=qiwi,DC=com
```
If you don't see that, try to change search filter in ldap_sync.yaml. In my case:
`group_search_filter: '(cn=airflow_*)'`
Also may be you need to change other parameters (I'm using Active Directory):
```
group_object_class: 'Group'
group_name_attr: 'cn'
group_member_attr: 'member'
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] ashb closed issue #8179: Airflow LDAP authentication with
RBAC features
Posted by GitBox <gi...@apache.org>.
ashb closed issue #8179: Airflow LDAP authentication with RBAC features
URL: https://github.com/apache/airflow/issues/8179
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] ashb commented on issue #8179: Airflow LDAP
authentication with RBAC features
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #8179: Airflow LDAP authentication with RBAC features
URL: https://github.com/apache/airflow/issues/8179#issuecomment-611735117
This is a question/feature request for the framework we use, Flask-AppBuilder and has been asked for upstream as https://github.com/dpgaspar/Flask-AppBuilder/issues/956
Closing this one for now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] xpyct600 commented on issue #8179: Airflow LDAP
authentication with RBAC features
Posted by GitBox <gi...@apache.org>.
xpyct600 commented on issue #8179: Airflow LDAP authentication with RBAC features
URL: https://github.com/apache/airflow/issues/8179#issuecomment-612689869
Check /var/log/airflow/ldap_sync/log please. Is there anything?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] xpyct600 edited a comment on issue #8179: Airflow LDAP
authentication with RBAC features
Posted by GitBox <gi...@apache.org>.
xpyct600 edited a comment on issue #8179: Airflow LDAP authentication with RBAC features
URL: https://github.com/apache/airflow/issues/8179#issuecomment-612261854
Hi, @Abhishekchechani
Yep, you can use script https://github.com/xpyct600/fab_ldap_sync for this.
Add all necessary LDAP groups and airflow roles to ldap_sync.yaml:
```
group_role_map:
'<ldap_group>': '<airflow_role>'
```
Disable AUTH_USER_REGISTRATION in webserver_config.py:
`AUTH_USER_REGISTRATION = False`
Put ldap_sync.py and ldap_sync.yaml to $AIRFLOW_HOME directory, configure run script via cron eg:
`0 * * * * flock -n /var/lock/airflow_ldap_sync.lock -c 'export AIRFLOW_HOME=/var/local/python/airflow && /var/local/python/airflow/ldap_sync.py' > /dev/null 2>&1`
When the script ldap_sync.py starts, it will add all users from groups listed in file ldap_sync.yaml to airflow. Then it will map them necessary roles.
If you delete user from LDAP or modify user's groups, just run script again to update airflow users.
Write me if you have questions akim.sysadm@gmail.com
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] xpyct600 edited a comment on issue #8179: Airflow LDAP
authentication with RBAC features
Posted by GitBox <gi...@apache.org>.
xpyct600 edited a comment on issue #8179: Airflow LDAP authentication with RBAC features
URL: https://github.com/apache/airflow/issues/8179#issuecomment-612609018
Airflow doesn't read these files.
Airflow reads sql_alchemy_database.
ldap_sync.py uses the airflow's embedded functions to manage users (adding users to the database
)
ldap_sync.py creates users in airflow's database.
So when you log in you are already in Airflow's database
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] xpyct600 commented on issue #8179: Airflow LDAP
authentication with RBAC features
Posted by GitBox <gi...@apache.org>.
xpyct600 commented on issue #8179: Airflow LDAP authentication with RBAC features
URL: https://github.com/apache/airflow/issues/8179#issuecomment-612261854
Hi, @Abhishekchechani
Yep, you can use script https://github.com/xpyct600/fab_ldap_sync for this.
Add all necessary LDAP groups and airflow roles to ldap_sync.yaml:
```
group_role_map:
'<ldap_group>': '<airflow_role>'
```
Disable AUTH_USER_REGISTRATION in webserver_config.py:
`AUTH_USER_REGISTRATION = False`
When the script ldap_sync.py starts, it will add all users from groups listen in file ldap_sync.yaml to airflow. Then it will map them necessary roles.
If you delete user from LDAP or modify user's groups, just run script again to update airflow users.
Write me if you have questions akim.sysadm@gmail.com
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] Abhishekchechani edited a comment on issue #8179: Airflow
LDAP authentication with RBAC features
Posted by GitBox <gi...@apache.org>.
Abhishekchechani edited a comment on issue #8179: Airflow LDAP authentication with RBAC features
URL: https://github.com/apache/airflow/issues/8179#issuecomment-612733118
HI @xpyct600 ,
There is some indentation problem is there , because in ldap-sync.py code you have define user_list under else block and it use outside the block due to reason its showing error user_list not define.
60 user_list = [sam_account_name.get(appbuilder.sm.auth_ld
61
62 # Adding new users:
63 for username in user_list:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] Abhishekchechani edited a comment on issue #8179: Airflow
LDAP authentication with RBAC features
Posted by GitBox <gi...@apache.org>.
Abhishekchechani edited a comment on issue #8179: Airflow LDAP authentication with RBAC features
URL: https://github.com/apache/airflow/issues/8179#issuecomment-612733118
HI @xpyct600 ,
There is some indentation problem is there , because in ldap-sync.py code you have define user_list under else block and it use outside the block due to reason its showing error user_list not define.
60 user_list = [sam_account_name.get(appbuilder.sm.auth_ld
61
62 # Adding new users:
63 for username in user_list:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] Abhishekchechani commented on issue #8179: Airflow LDAP
authentication with RBAC features
Posted by GitBox <gi...@apache.org>.
Abhishekchechani commented on issue #8179: Airflow LDAP authentication with RBAC features
URL: https://github.com/apache/airflow/issues/8179#issuecomment-612612272
Hi @xpyct600 ,
Getting below error while run ldap_sync.py file.
File "ldap_sync.py", line 62, in <module>
for username in user_list:
NameError: name 'user_list' is not defined
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] xpyct600 edited a comment on issue #8179: Airflow LDAP
authentication with RBAC features
Posted by GitBox <gi...@apache.org>.
xpyct600 edited a comment on issue #8179: Airflow LDAP authentication with RBAC features
URL: https://github.com/apache/airflow/issues/8179#issuecomment-612689869
Check /var/log/airflow/ldap_sync.log please. Is there anything?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] Abhishekchechani commented on issue #8179: Airflow LDAP
authentication with RBAC features
Posted by GitBox <gi...@apache.org>.
Abhishekchechani commented on issue #8179: Airflow LDAP authentication with RBAC features
URL: https://github.com/apache/airflow/issues/8179#issuecomment-612602591
Thanks @xpyct600 for the response on the above Airflow LDAP feature.
But i want to know some points regarding Airflow LDAP features.
Once i create the ldap_sync.py and ldap_sync.yaml how the Airflow will read these file instead of flask_appbuilder.security manager.py file while login in Airflow WEBUI.
Is there any other property need to be set?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] jumiller-cotiviti commented on issue #8179: Airflow LDAP
authentication with RBAC features
Posted by GitBox <gi...@apache.org>.
jumiller-cotiviti commented on issue #8179: Airflow LDAP authentication with RBAC features
URL: https://github.com/apache/airflow/issues/8179#issuecomment-612194146
Since this might leave a lot of people who were already using ldap with groups in the lurch here is a script a guy wrote that will sync ldap groups to roles
I made my dirty implementation of this feature if somebody need.
https://github.com/xpyct600/fab_ldap_sync
You may need to change something for your LDAP.
Sorry for my codestyle, I'm not a programmer.
Just run this via cron daemon. Don't forget to export AIRFLOW_HOME environment variable before run ldap_sync.py
👀 1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] boring-cyborg[bot] commented on issue #8179: Airflow LDAP
authentication with RBAC features
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #8179: Airflow LDAP authentication with RBAC features
URL: https://github.com/apache/airflow/issues/8179#issuecomment-610463321
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] xpyct600 edited a comment on issue #8179: Airflow LDAP
authentication with RBAC features
Posted by GitBox <gi...@apache.org>.
xpyct600 edited a comment on issue #8179: Airflow LDAP authentication with RBAC features
URL: https://github.com/apache/airflow/issues/8179#issuecomment-612261854
Hi, @Abhishekchechani
Yep, you can use script https://github.com/xpyct600/fab_ldap_sync for this.
Add all necessary LDAP groups and airflow roles to ldap_sync.yaml:
```
group_role_map:
'<ldap_group>': '<airflow_role>'
```
Disable AUTH_USER_REGISTRATION in webserver_config.py:
`AUTH_USER_REGISTRATION = False`
When the script ldap_sync.py starts, it will add all users from groups listed in file ldap_sync.yaml to airflow. Then it will map them necessary roles.
If you delete user from LDAP or modify user's groups, just run script again to update airflow users.
Write me if you have questions akim.sysadm@gmail.com
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] xpyct600 commented on issue #8179: Airflow LDAP
authentication with RBAC features
Posted by GitBox <gi...@apache.org>.
xpyct600 commented on issue #8179: Airflow LDAP authentication with RBAC features
URL: https://github.com/apache/airflow/issues/8179#issuecomment-612609018
Airflow doesn't read these files.
Airflow reads sql_alchemy_database.
ldap_sync.py uses the airflow'w embedded functions to manage users (adding users to the database
)
ldap_sync.py creates users in airflow's database.
So when you log in you are already in Airflow's database
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] Abhishekchechani commented on issue #8179: Airflow LDAP
authentication with RBAC features
Posted by GitBox <gi...@apache.org>.
Abhishekchechani commented on issue #8179: Airflow LDAP authentication with RBAC features
URL: https://github.com/apache/airflow/issues/8179#issuecomment-612733118
HI @xpyct600 ,
There is some indentation problem is there , because in ldap-sync.py code you have define user_list under else block and it use outside the block due to reason its showing error user_list not define.
60 user_list = [sam_account_name.get(appbuilder.sm.auth_ld
61
62 # Adding new users:
63 for username in user_list:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services