[GitHub] [airflow] ashmeet13 edited a comment on issue #11144: Create UndefinedJinjaVariablesRule to ease upgrade to 2.0

[GitBox <gi...@apache.org>]

ashmeet13 edited a comment on issue #11144:
URL: https://github.com/apache/airflow/issues/11144#issuecomment-701439517


   Some progress that I have made. Thanks @ashb  for guiding towards `TaskInstance` for getting the complete context.
   Also towards `BaseOperator.render_template`
   
   With this I was able to improve the logic to extract undefined variables in a template to this - 
   
   ```
   def check_rendered(rendered):
       if isinstance(rendered, six.string_types):
           return set(re.findall(r"{{(.*?)}}", rendered))
   
       elif isinstance(rendered, (tuple,list,set)):
           parsed_templates = set()
           for element in rendered:
               parsed_templates.union(check_rendered(element))
           return parsed_templates
   
       elif isinstance(rendered, dict):
           parsed_templates = set()
           for key, value in rendered.items():
               parsed_templates.union(check_rendered(value))
           return parsed_templates 
   
   
   
   def check(dag):
       dag.template_undefined = jinja2.DebugUndefined
       for task in dag.tasks:
           jinja_env = task.get_template_env()
           task_instance = TaskInstance(task=task, execution_date=timezone.utcnow())
           template_context = task_instance.get_template_context()
   
           for attr_name in task.template_fields:
               content = getattr(task, attr_name)
               if content:
                   rendered = task.render_template(content, template_context)
                   undefined = check_rendered(rendered)
                   for element in undefined:
                       print(element) 
   ```
   
   Take into the consideration the following DAG -
   ```
   dag = DAG(
       'tutorial',
       default_args=default_args,
       description='A simple tutorial DAG',
       schedule_interval=timedelta(days=1),
   )
   
   templated_command_string = """
   {% for i in range(5) %}
       echo "{{ params.defined }}"
       echo "{{ execution_date.month }}"
       echo "{{ params.undefined }}"
       echo "{{ foo }}"
   {% endfor %}
   """
   
   t1 = BashOperator(
       task_id='templated_string',
       depends_on_past=False,
       bash_command=templated_command_string,
       params={'defined': 'test_val'},
       dag=dag,
   )
   ```
   
   The above check would print/return this -  
   ```
   foo
   no such element: dict object['undefined'] 
   ```
   
   
   One blocker still that I have is how exactly will I be able to get the dags when the user runs `airflow upgrade-check`
   Thanks again for the help!
   
   
   Edited: Adding the [Stack Overflow](https://stackoverflow.com/questions/46619830/how-to-get-all-undefined-variables-from-a-jinja2-template) source I used


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org