[GitHub] [airflow] yuqian90 commented on a change in pull request #11335: Make Dag Serialization a hard requirement

[GitBox <gi...@apache.org>]

yuqian90 commented on a change in pull request #11335:
URL: https://github.com/apache/airflow/pull/11335#discussion_r561822542



##########
File path: airflow/models/taskinstance.py
##########
@@ -1576,28 +1574,22 @@ def get(
         }
 
     def get_rendered_template_fields(self):
-        """
-        Fetch rendered template fields from DB if Serialization is enabled.
-        Else just render the templates
-        """
+        """Fetch rendered template fields from DB"""
         from airflow.models.renderedtifields import RenderedTaskInstanceFields
-        if STORE_SERIALIZED_DAGS:
-            rendered_task_instance_fields = RenderedTaskInstanceFields.get_templated_fields(self)
-            if rendered_task_instance_fields:
-                for field_name, rendered_value in rendered_task_instance_fields.items():
-                    setattr(self.task, field_name, rendered_value)
-            else:
-                try:
-                    self.render_templates()
-                except (TemplateAssertionError, UndefinedError) as e:
-                    raise AirflowException(
-                        "Webserver does not have access to User-defined Macros or Filters "
-                        "when Dag Serialization is enabled. Hence for the task that have not yet "
-                        "started running, please use 'airflow tasks render' for debugging the "
-                        "rendering of template_fields."
-                    ) from e
+        rendered_task_instance_fields = RenderedTaskInstanceFields.get_templated_fields(self)
+        if rendered_task_instance_fields:
+            for field_name, rendered_value in rendered_task_instance_fields.items():
+                setattr(self.task, field_name, rendered_value)
         else:
-            self.render_templates()
+            try:
+                self.render_templates()
+            except (TemplateAssertionError, UndefinedError) as e:
+                raise AirflowException(
+                    "Webserver does not have access to User-defined Macros or Filters "

Review comment:
       Hi @kaxil , if using Airflow 2.0 or using Airflow 1.10.* with `store_serialized_dags = True` turned on, we hit this error here when user clicks on "Rendered Template" on tasks that are using `user_defined_macros` in jinja template fields.
   
   We also hit a similar problem if user clears `ExternalTaskMarker` that uses `user_defined_macros` in the jinja template fields (because the `dag.clear()` function calls `ti.render_templates()` to figure out the actual values of the template fields. 
   
   How do you recommend addressing these issues going forward? Should we allow some webserver functions to get access to these `user_defined_macros`? Or should we serialize the rendered template values so that webserver can access them?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org