You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/11/10 15:56:35 UTC
[GitHub] [airflow] dimitar-petrunov-sagedata opened a new issue #19520: Passwords not masked
dimitar-petrunov-sagedata opened a new issue #19520:
URL: https://github.com/apache/airflow/issues/19520
### Apache Airflow version
2.2.1 (latest released)
### Operating System
Docker image apache/airflow:2.2.1-python3.7
### Versions of Apache Airflow Providers
apache-airflow-providers-amazon==2.3.0
apache-airflow-providers-celery==2.1.0
apache-airflow-providers-cncf-kubernetes==2.0.3
apache-airflow-providers-docker==2.2.0
apache-airflow-providers-elasticsearch==2.0.3
apache-airflow-providers-ftp==2.0.1
apache-airflow-providers-google==6.0.0
apache-airflow-providers-grpc==2.0.1
apache-airflow-providers-hashicorp==2.1.1
apache-airflow-providers-http==2.0.1
apache-airflow-providers-imap==2.0.1
apache-airflow-providers-microsoft-azure==3.2.0
apache-airflow-providers-mysql==2.1.1
apache-airflow-providers-odbc==2.0.1
apache-airflow-providers-postgres==2.3.0
apache-airflow-providers-redis==2.0.1
apache-airflow-providers-sendgrid==2.0.1
apache-airflow-providers-sftp==2.1.1
apache-airflow-providers-slack==4.1.0
apache-airflow-providers-snowflake==2.1.1
apache-airflow-providers-sqlite==2.0.1
apache-airflow-providers-ssh==2.2.0
### Deployment
Docker-Compose
### Deployment details
not relevant
### What happened
Connection passwords aren't masked in the rendered UI view for tasks in "upstream_failed" state.
### What you expected to happen
Consider task1 >> task2 dependency. Both are BashOperator tasks having a string in their command line that is a secret and should be masked. Task1 fails and its rendered UI view correctly masks the secret string as '***'. Taks2's rednered view displays the secret unmasked.
### How to reproduce
from datetime import datetime
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from airflow.utils.log.secrets_masker import mask_secret
MASK_ME = "mask_me"
mask_secret(MASK_ME)
with DAG(
'broken_masks',
start_date=datetime(2020, 12, 23),
schedule_interval=None,
catchup=False,
) as dag:
t1 = BashOperator(
task_id="rendered_view_shows_masked_secret",
bash_command=f"echo '{MASK_ME} correctly masked' && exit 1",
)
# Check Rendered view doesn't mask secret as upstream fails
t2 = BashOperator(
task_id="rendered_view_shows_unmasked_secret",
bash_command=f"echo '{MASK_ME} not masked'",
)
t1 >> t2
### Anything else
n/a
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #19520: Passwords not masked
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #19520:
URL: https://github.com/apache/airflow/issues/19520#issuecomment-965503523
Masking is only implemented in logs. You should not pass passwords via rendered templates, also because the rendered templates are stored in the database. Masking in the UI will not prevent it.
This behaviour is not going to be changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #19520: Passwords not masked
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #19520:
URL: https://github.com/apache/airflow/issues/19520#issuecomment-965791432
There is no way to pass secret values between tasks. You can mask secrets in-the tasks but not when you pass them between tasks. The 'mask_secret" function works when you execute inside the task not in the DAG definition,
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dimitar-petrunov-sagedata edited a comment on issue #19520: Passwords not masked
Posted by GitBox <gi...@apache.org>.
dimitar-petrunov-sagedata edited a comment on issue #19520:
URL: https://github.com/apache/airflow/issues/19520#issuecomment-965619672
@potiuk Appologies if this isn't the right place to ask but I don't seem to find information on the subject. Environment variables are also visible in plain text in the rendered templates. Does it mean env vars passed via bash operator aren't suitable for holding secrets(these secrets supposedly originating from airflow hooks)?
update: I have checked the rendered_task_instance_fields table in the database, strangely fields are masked there as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #19520: Passwords not masked
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #19520:
URL: https://github.com/apache/airflow/issues/19520#issuecomment-965503523
Masking is only implemented in logs. You should not pass passwords via rendered templates, also because the rendered templates are stored in the database. Masking in the UI will not prevent it.
This behavioure is not going to be changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #19520: Passwords not masked
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #19520:
URL: https://github.com/apache/airflow/issues/19520#issuecomment-965463640
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dimitar-petrunov-sagedata edited a comment on issue #19520: Passwords not masked
Posted by GitBox <gi...@apache.org>.
dimitar-petrunov-sagedata edited a comment on issue #19520:
URL: https://github.com/apache/airflow/issues/19520#issuecomment-965619672
@potiuk Appologies if this isn't the right place to ask but I don't seem to find information on the subject. Environment variables are also visible in plain text in the rendered templates. Does it mean env vars passed via bash operator aren't suitable for holding secrets(these secrets supposedly originating from airflow hooks)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #19520: Passwords not masked
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #19520:
URL: https://github.com/apache/airflow/issues/19520
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dimitar-petrunov-sagedata edited a comment on issue #19520: Passwords not masked
Posted by GitBox <gi...@apache.org>.
dimitar-petrunov-sagedata edited a comment on issue #19520:
URL: https://github.com/apache/airflow/issues/19520#issuecomment-965619672
@potiuk Appologies if this isn't the right place to ask but I don't seem to find information on the subject. Environment variables are also visible in plain text in the rendered templates. Does it mean env vars passed via bash operator aren't suitable for secrets(these secrets supposedly coming from airflow hooks)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dimitar-petrunov-sagedata commented on issue #19520: Passwords not masked
Posted by GitBox <gi...@apache.org>.
dimitar-petrunov-sagedata commented on issue #19520:
URL: https://github.com/apache/airflow/issues/19520#issuecomment-965619672
@potiuk Appologies if this isn't the right place to ask but I don't seem to find information on the subject. Environment variables are also visible in plain text in the rendered templates. Does it mean env vars passed to bash operator aren't suitable for secrets?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org