[GitHub] [airflow] nathadfield edited a comment on pull request #17252: Enable specifying dictionary paths in `template_fields_renderers`

[GitBox <gi...@apache.org>]

nathadfield edited a comment on pull request #17252:
URL: https://github.com/apache/airflow/pull/17252#issuecomment-887436723


   @mehmax I can modify `views.py` again to look for a template field  equal to `op_kwargs` if content is a dict.  We can also look at the keys in `op_kwargs` to see if they match any existing renderers (e.g. `sql`) otherwise we default to `py` (or something else?).
   
   ```
   if isinstance(content, dict):
       if template_field == 'op_kwargs':
           for key, value in content.items():
               renderer = task.template_fields_renderers.get(key, key)
               if renderer in renderers:
                   html_dict[key] = renderers[renderer](value)
               else:
                   html_dict[key] = renderers['py'](value)
       else:
           for dict_keys in get_key_paths(content):
               template_path = '.'.join((template_field, dict_keys))
               renderer = task.template_fields_renderers.get(template_path, template_path)
               if renderer in renderers:
                   content_value = get_value_from_path(dict_keys, content)
                   html_dict[template_path] = renderers[renderer](content_value)
   ```
   
   Then if specify this in the DAG:
   
   ```
   @task(default_args=default_args, templates_exts=['.sql'])
   def extract(src_odbc_conn_id: str, dest_odbc_conn_id: str, dest_table: str, sql: str, **kwargs):
       src_odbc_hook = OdbcHook(src_odbc_conn_id)
       dest_odbc_hook = OdbcHook(dest_odbc_conn_id)
   
       df = src_odbc_hook.get_pandas_df(sql)
       df.to_sql(dest_table, con=dest_odbc_hook.get_sqlalchemy_engine())
   
   sql = '''
   INSERT INTO `target_project.target_dataset.target_table`
   (col1, col2, col3, col4)
   SELECT col1, col2, col3, col4
   FROM `source_project.source_dataset.source_table
   WHERE dt = {{ ds }}`
   '''.strip()
   
   t1extract = extract(
       src_odbc_conn_id='SOURCE',
       dest_odbc_conn_id='DESTINATION',
       sql=sql,
       dest_table='material'
   )
   ```
   
   This is how it gets rendered.
   
   <img width="1668" alt="Screenshot 2021-07-27 at 12 31 06" src="https://user-images.githubusercontent.com/967119/127146694-eb6334b6-be41-4099-9f14-4b1a5093db7b.png">
   
   Thoughts?
   
   Anything from you @potiuk?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org