You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "noderat (via GitHub)" <gi...@apache.org> on 2024/11/13 23:33:55 UTC
[I] apache-airflow-providers-microsoft-psrp securestring template filter [airflow]
noderat opened a new issue, #43992:
URL: https://github.com/apache/airflow/issues/43992
### Apache Airflow Provider(s)
microsoft-psrp
### Versions of Apache Airflow Providers
apache-airflow-providers-microsoft-psrp==2.8.0
### Apache Airflow version
2.10.2
### Operating System
Debian GNU/Linux 12 (bookworm)
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
Using official helm chart and the following slightly extended Dockerfile:
```
FROM apache/airflow:2.10.2
RUN pip install --no-cache-dir "apache-airflow[microsoft-psrp]==${AIRFLOW_VERSION}" \
--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-3.9.txt"
```
### What happened
`jinja2.exceptions.TemplateAssertionError: No filter named 'securestring'`
It appears the securestring template filter is not working as intended.
Summarized Error:
```
ERROR - Exception rendering Jinja template for task 'Write-Output', field 'arguments'. Template: ["{{ 'foo' | securestring }}"]
( ... lengthy stack trace redacted ... )
jinja2.exceptions.TemplateAssertionError: No filter named 'securestring'.
```
<details>
<summary>
Full execution log including the redacted stack trace:
</summary>
```
[2024-11-13, 16:11:36 MST] {taskinstance.py:2865} INFO - Starting attempt 1 of 1
[2024-11-13, 16:11:36 MST] {taskinstance.py:2888} INFO - Executing <Task(PsrpOperator): Write-Output> on 2024-11-13 23:11:35.867453+00:00
[2024-11-13, 16:11:36 MST] {warnings.py:112} WARNING - /home/airflow/.local/lib/python3.12/site-packages/airflow/task/task_runner/standard_task_runner.py:70: DeprecationWarning: This process (pid=14633) is multi-threaded, use of fork() may lead to deadlocks in the child.
pid = os.fork()
[2024-11-13, 16:11:36 MST] {standard_task_runner.py:72} INFO - Started process 14644 to run task
[2024-11-13, 16:11:36 MST] {standard_task_runner.py:104} INFO - Running: ['airflow', 'tasks', 'run', 'psrpoperator_securestring_error_reproduction', 'Write-Output', 'manual__2024-11-13T23:11:35.867453+00:00', '--job-id', '802', '--raw', '--subdir', 'DAGS_FOLDER/psrp_operator_securestring_error_repoduction.py', '--cfg-path', '/tmp/tmpwrdenr76']
[2024-11-13, 16:11:36 MST] {standard_task_runner.py:105} INFO - Job 802: Subtask Write-Output
[2024-11-13, 16:11:37 MST] {task_command.py:467} INFO - Running <TaskInstance: psrpoperator_securestring_error_reproduction.Write-Output manual__2024-11-13T23:11:35.867453+00:00 [running]> on host experimental-temp-airflow-worker-0.experimental-temp-airflow-worker.experimental-temp-airflow.svc.cluster.local
[2024-11-13, 16:11:37 MST] {abstractoperator.py:778} ERROR - Exception rendering Jinja template for task 'Write-Output', field 'arguments'. Template: ["{{ 'foo' | securestring }}"]
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/abstractoperator.py", line 770, in _do_render_template_fields
rendered_content = self.render_template(
^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/template/templater.py", line 183, in render_template
return [self.render_template(element, context, jinja_env, oids) for element in value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/template/templater.py", line 170, in render_template
template = jinja_env.from_string(value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/jinja2/environment.py", line 1108, in from_string
return cls.from_code(self, self.compile(source), gs, None)
^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/jinja2/environment.py", line 768, in compile
self.handle_exception(source=source_hint)
File "/home/airflow/.local/lib/python3.12/site-packages/jinja2/environment.py", line 939, in handle_exception
raise rewrite_traceback_stack(source=source)
File "<unknown>", line 1, in template
jinja2.exceptions.TemplateAssertionError: No filter named 'securestring'.
[2024-11-13, 16:11:37 MST] {taskinstance.py:3310} ERROR - Task failed with exception
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/taskinstance.py", line 273, in _run_raw_task
TaskInstance._execute_task_with_callbacks(
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/taskinstance.py", line 3114, in _execute_task_with_callbacks
task_orig = self.render_templates(context=context, jinja_env=jinja_env)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/taskinstance.py", line 3533, in render_templates
original_task.render_template_fields(context, jinja_env)
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/baseoperator.py", line 1419, in render_template_fields
self._do_render_template_fields(self, self.template_fields, context, jinja_env, set())
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/abstractoperator.py", line 770, in _do_render_template_fields
rendered_content = self.render_template(
^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/template/templater.py", line 183, in render_template
return [self.render_template(element, context, jinja_env, oids) for element in value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/airflow/template/templater.py", line 170, in render_template
template = jinja_env.from_string(value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/jinja2/environment.py", line 1108, in from_string
return cls.from_code(self, self.compile(source), gs, None)
^^^^^^^^^^^^^^^^^^^^
File "/home/airflow/.local/lib/python3.12/site-packages/jinja2/environment.py", line 768, in compile
self.handle_exception(source=source_hint)
File "/home/airflow/.local/lib/python3.12/site-packages/jinja2/environment.py", line 939, in handle_exception
raise rewrite_traceback_stack(source=source)
File "<unknown>", line 1, in template
jinja2.exceptions.TemplateAssertionError: No filter named 'securestring'.
[2024-11-13, 16:11:37 MST] {taskinstance.py:1225} INFO - Marking task as FAILED. dag_id=psrpoperator_securestring_error_reproduction, task_id=Write-Output, run_id=manual__2024-11-13T23:11:35.867453+00:00, execution_date=20241113T231135, start_date=20241113T231136, end_date=20241113T231137
```
</summary>
### What you think should happen instead
The minimal example provided should return through XCom the ToString representation of a powershell SecureString which would be `"System.Security.SecureString"`
### How to reproduce
The following DAG can be used to reproduce the issue and does not require a remote connection as the task fails before attempting any connection:
```python
import datetime
from airflow import DAG
from airflow.providers.microsoft.psrp.operators.psrp import PsrpOperator
with DAG(
dag_id="psrpoperator_securestring_error_reproduction",
start_date=None,
schedule_interval=None,
render_template_as_native_obj=True,
):
PsrpOperator(
psrp_conn_id = "SomeConnection",
cmdlet = "Write-Output",
arguments = [ "{{ 'foo' | securestring }}" ],
)
```
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] microsoft-psrp securestring jinja2.exceptions.TemplateAssertionError [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #43992: microsoft-psrp securestring jinja2.exceptions.TemplateAssertionError
URL: https://github.com/apache/airflow/issues/43992
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] apache-airflow-providers-microsoft-psrp securestring template filter [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #43992:
URL: https://github.com/apache/airflow/issues/43992#issuecomment-2475037681
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org