You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Manny Cato (Jira)" <ji...@apache.org> on 2019/10/30 22:19:00 UTC
[jira] [Commented] (AIRFLOW-5774) PapermillOperator fails to render
parameters
[ https://issues.apache.org/jira/browse/AIRFLOW-5774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963475#comment-16963475 ]
Manny Cato commented on AIRFLOW-5774:
-------------------------------------
I see this has been closed due to being a duplicate, but there's no other task attached?
Can someone provide more information on how this is beign tracked? It's making the PapermillOperator unusable
> PapermillOperator fails to render parameters
> --------------------------------------------
>
> Key: AIRFLOW-5774
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5774
> Project: Apache Airflow
> Issue Type: Bug
> Components: utils
> Affects Versions: 1.10.4
> Reporter: Mariusz Górski
> Priority: Minor
> Labels: bug
>
> *Problem*
> When running sample PapermillOperator described in docs, it fails with
> {code:java}
> //[2019-10-27 10:48:53,134] {{taskinstance.py:1047}} ERROR - Can't compile non template nodes
> Traceback (most recent call last):
> File "/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 922, in _run_raw_task
> result = task_copy.execute(context=context)
> File "/usr/local/lib/python3.7/site-packages/airflow/operators/papermill_operator.py", line 71, in execute
> logging.info(inlet.as_dict())
> File "/usr/local/lib/python3.7/site-packages/airflow/lineage/datasets.py", line 85, in as_dict
> attributes[key] = env.from_string(value).render(**self.context)
> File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 880, in from_string
> return cls.from_code(self, self.compile(source), globals, None)
> File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 581, in compile
> defer_init=defer_init)
> File "/usr/local/lib/python3.7/site-packages/jinja2/environment.py", line 543, in _generate
> optimized=self.optimized)
> File "/usr/local/lib/python3.7/site-packages/jinja2/compiler.py", line 78, in generate
> raise TypeError('Can\'t compile non template nodes')
> TypeError: Can't compile non template nodes
> {code}
> *Reason*
> It's because: _class NoteBook(DataSet)_ expects _parameters_ to be _strings_, while papermill _execute_notebook_ function requires _parameters_ to be _dicts_. When we pass dict to _NoteBook.parameters_, then jinja templating fails when dict is passed to _env.from_string(value).render(**context)_ call.
> *Solution:*
> Handling of nested structs in DataSet class
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)