You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/08/27 14:49:24 UTC
[GitHub] [airflow] YanHaoChen opened a new issue #17871: string 'airflow' be Rendered to string '***' in bash_command
YanHaoChen opened a new issue #17871:
URL: https://github.com/apache/airflow/issues/17871
<!--
Welcome to Apache Airflow!
Please complete the next sections or the issue will be closed.
-->
**Apache Airflow version**: 2.1.0
<!-- AIRFLOW VERSION IS MANDATORY -->
**OS**: macOS 11.3.1
<!-- MANDATORY! You can get it via `cat /etc/os-release` for example -->
**Apache Airflow Provider versions**:
<!-- You can use `pip freeze | grep apache-airflow-providers` (you can leave only relevant ones)-->
**Deployment**: Docker-compose version 1.29.1
<!-- e.g. Virtualenv / VM / Docker-compose / K8S / Helm Chart / Managed Airflow Service -->
<!-- Please include your deployment tools and versions: docker-compose, k8s, helm, etc -->
**What happened**:
The string 'airflow' in bash_command was Rendered to '***'.
<h2>In Task Instance Details</2>
<h5>Attribute: bash_command</h5>
1 | pip3 install -r /opt/airflow/dags/tw-financial-report-analysis/dag/af_requirements.txt
-- | --
<img width="1182" alt="screen 2021-08-27-10 36 15" src="https://user-images.githubusercontent.com/10976112/131144456-b4a49712-2311-4674-81f0-186f62510ecd.png">
<h2>In Rendered Template</2>
<h5>bash_command</h5>
1 | pip3 install -r /opt/***/dags/tw-financial-report-analysis/dag/af_requirements.txt
-- | --
<img width="718" alt="screen 2021-08-27 下午10 36 31" src="https://user-images.githubusercontent.com/10976112/131144562-2cbf4ed4-c8e5-43d3-b6d1-e098745d43cf.png">
**What you expected to happen**:
The rendered string shouldn't be changed.
**How to reproduce it**:
Run the bashOperartor with Attribute, bash_command, includes string 'airflow'.
<!--
As minimally and precisely as possible. Keep in mind we do not have access to your cluster or dags.
If this is a UI bug, please provide a screenshot of the bug or a link to a youtube video of the bug in action.
You can include images/screen-casts etc. by drag-dropping the image here.
-->
**Anything else we need to know**:
The log of the error task.
```
--------------------------------------------------------------------------------
[2021-08-27 14:09:16,193] {taskinstance.py:1068} INFO - Starting attempt 4 of 4
[2021-08-27 14:09:16,196] {taskinstance.py:1069} INFO -
--------------------------------------------------------------------------------
[2021-08-27 14:09:16,206] {taskinstance.py:1087} INFO - Executing <Task(BashOperator): check_requirements> on 2021-08-26T00:00:00+00:00
[2021-08-27 14:09:16,212] {standard_task_runner.py:52} INFO - Started process 41 to run task
[2021-08-27 14:09:16,217] {standard_task_runner.py:76} INFO - Running: ['***', 'tasks', 'run', 'setup_parse_financial_report', 'check_requirements', '2021-08-26T00:00:00+00:00', '--job-id', '7', '--pool', 'default_pool', '--raw', '--subdir', 'DAGS_FOLDER/tw-financial-report-analysis/dags/parse_financial_report/setup_project.py', '--cfg-path', '/tmp/tmp8e3r9kye', '--error-file', '/tmp/tmpvcm3zeju']
[2021-08-27 14:09:16,218] {standard_task_runner.py:77} INFO - Job 7: Subtask check_requirements
[2021-08-27 14:09:16,280] {logging_mixin.py:104} INFO - Running <TaskInstance: setup_parse_financial_report.check_requirements 2021-08-26T00:00:00+00:00 [running]> on host f5719825fcd2
[2021-08-27 14:09:16,341] {taskinstance.py:1280} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_OWNER=sean
AIRFLOW_CTX_DAG_ID=setup_parse_financial_report
AIRFLOW_CTX_TASK_ID=check_requirements
AIRFLOW_CTX_EXECUTION_DATE=2021-08-26T00:00:00+00:00
AIRFLOW_CTX_DAG_RUN_ID=scheduled__2021-08-26T00:00:00+00:00
[2021-08-27 14:09:16,343] {subprocess.py:52} INFO - Tmp dir root location:
/tmp
[2021-08-27 14:09:16,345] {subprocess.py:63} INFO - Running command: ['bash', '-c', 'pip3 install -r /opt/***/dags/tw-financial-report-analysis/dag/af_requirements.txt']
[2021-08-27 14:09:16,356] {subprocess.py:75} INFO - Output:
[2021-08-27 14:09:16,916] {subprocess.py:79} INFO - WARNING: The directory '/home/***/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
[2021-08-27 14:09:16,919] {subprocess.py:79} INFO - Defaulting to user installation because normal site-packages is not writeable
[2021-08-27 14:09:16,980] {subprocess.py:79} INFO - ERROR: Could not open requirements file: [Errno 2] No such file or directory: '/opt/***/dags/tw-financial-report-analysis/dag/af_requirements.txt'
[2021-08-27 14:09:17,297] {subprocess.py:79} INFO - WARNING: You are using pip version 21.1.1; however, version 21.2.4 is available.
[2021-08-27 14:09:17,299] {subprocess.py:79} INFO - You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
[2021-08-27 14:09:17,357] {subprocess.py:83} INFO - Command exited with return code 1
[2021-08-27 14:09:17,370] {taskinstance.py:1481} ERROR - Task failed with exception
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1137, in _run_raw_task
self._prepare_and_execute_task_with_callbacks(context, task)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1311, in _prepare_and_execute_task_with_callbacks
result = self._execute_task(context, task_copy)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1341, in _execute_task
result = task_copy.execute(context=context)
File "/home/airflow/.local/lib/python3.8/site-packages/airflow/operators/bash.py", line 180, in execute
raise AirflowException('Bash command failed. The command returned a non-zero exit code.')
airflow.exceptions.AirflowException: Bash command failed. The command returned a non-zero exit code.
[2021-08-27 14:09:17,373] {taskinstance.py:1524} INFO - Marking task as FAILED. dag_id=setup_parse_financial_report, task_id=check_requirements, execution_date=20210826T000000, start_date=20210827T140916, end_date=20210827T140917
[2021-08-27 14:09:17,401] {local_task_job.py:151} INFO - Task exited with return code 1
```
<!--
How often does this problem occur? Once? Every time etc?
Any relevant logs to include? Put them here inside fenced
``` ``` blocks or inside a foldable details tag if it's long:
<details><summary>x.log</summary> lots of stuff </details>
-->
**Are you willing to submit a PR?**
No
<!---
This is absolutely not required, but we are happy to guide you in contribution process
especially if you already have a good understanding of how to implement the fix.
Airflow is a community-managed project and we love to bring new contributors in.
Find us in #airflow-how-to-pr on Slack!
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #17871: string 'airflow' be Rendered to string '***' in bash_command
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #17871:
URL: https://github.com/apache/airflow/issues/17871#issuecomment-907259348
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #17871: string 'airflow' be Rendered to string '***' in bash_command
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #17871:
URL: https://github.com/apache/airflow/issues/17871#issuecomment-1020227594
Editing file in your container is bad idea. You need ot follow https://airflow.apache.org/docs/apache-airflow/stable/howto/set-config.html to permanently set it (ideally in the image/compose/helm chart) - but not in the running container (this makes no sense whatsoever as your webserver is not restarted and restarting it will remove any changes you've made in your container.
This is basic things about containes - you should learn how they work really, if you want to use them - there are plenty of resources available (videos etc). You will easily find them.
For sure you have airflow as sensitive password/credential somewhere. Just change it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on issue #17871: string 'airflow' be Rendered to string '***' in bash_command
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #17871:
URL: https://github.com/apache/airflow/issues/17871#issuecomment-907314982
It caused by masking: http://apache-airflow-docs.s3-website.eu-central-1.amazonaws.com/docs/apache-airflow/latest/security/secrets/mask-sensitive-values.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kse-systematic commented on issue #17871: string 'airflow' be Rendered to string '***' in bash_command
Posted by GitBox <gi...@apache.org>.
kse-systematic commented on issue #17871:
URL: https://github.com/apache/airflow/issues/17871#issuecomment-1020189225
Hi,
I'm experiencing this very same issue. My username and password are not airflow. And I have edited my airflow.cfg inside my webserver container to change `hide_sensitive_var_conn_fields` to False. Does anyone have the same issue? Or know how to solve this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #17871: string 'airflow' be Rendered to string '***' in bash_command
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #17871:
URL: https://github.com/apache/airflow/issues/17871#issuecomment-1020227594
Editing file in your container is bad idea. You need to follow https://airflow.apache.org/docs/apache-airflow/stable/howto/set-config.html to permanently set it (ideally in the image/compose/helm chart) - but not in the running container (this makes no sense whatsoever as your webserver is not restarted and restarting it will remove any changes you've made in your container.
This is basic things about containes - you should learn how they work really, if you want to use them - there are plenty of resources available (videos etc). You will easily find them.
For sure you have airflow as sensitive password/credential somewhere. Just change it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #17871: string 'airflow' be Rendered to string '***' in bash_command
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #17871:
URL: https://github.com/apache/airflow/issues/17871
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #17871: string 'airflow' be Rendered to string '***' in bash_command
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #17871:
URL: https://github.com/apache/airflow/issues/17871#issuecomment-907398990
Yeah. if you are using `airflow' as pasword, it will be masked.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] YanHaoChen commented on issue #17871: string 'airflow' be Rendered to string '***' in bash_command
Posted by GitBox <gi...@apache.org>.
YanHaoChen commented on issue #17871:
URL: https://github.com/apache/airflow/issues/17871#issuecomment-907577004
Hi @mik-laj and @potiuk,
After changing the password, the mask is disappeared.
Thank you for your answers and time!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org