[jira] [Created] (AIRFLOW-4724) Make params dict to be templated for operators

["jack (JIRA)" <ji...@apache.org>]

jack created AIRFLOW-4724:
-----------------------------

             Summary: Make params dict to be templated for operators
                 Key: AIRFLOW-4724
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-4724
             Project: Apache Airflow
          Issue Type: Bug
          Components: core, operators
    Affects Versions: 1.10.3
            Reporter: jack
             Fix For: 1.10.4


Currently using params dict as:
{code:java}
EXEC_TIMESEPOCH =  "{{ execution_date.strftime('%s') }}"
gcs_export_uri_template_filename = 'product_dwh-' + EXEC_TIMESEPOCH + '.csv'
upload_file_ftp_op = BashOperator(
   task_id='upload_file_ftp_task',
   params={'filename':gcs_export_uri_template_filename},
   bash_command="python3.6 /home/ubuntu/airflow/scripts/ranker.py  '{{ params.filename }}'  " ,
  dag=dag)

{code}
Gives:
{code:java}
python3.6 /home/ubuntu/airflow/scripts/ranker.py  'product_dwh-{{ execution_date.strftime('%s') }}.csv'{code}
 

The BaseOperator says:
{code:java}
self.params = params or {} # Available in templates!{code}
[https://github.com/apache/airflow/blob/master/airflow/models/baseoperator.py#L343]

But as you can see above the code wasn't templated as expected.

 

I worked-around this by not using params dict as:
{code:java}
cmd = """python3.6 /home/ubuntu/airflow/scripts/ranker.py 'product_dwh-{{ execution_date.strftime('%s') }}.csv' """

upload_file_ftp_op = BashOperator(
    task_id='upload_file_ftp_task',
    bash_command = cmd,
    dag=dag){code}
This code works perfectly and while it's simpler and better the first code should have still work.

 

A discussion about this has been on slack:

[https://apache-airflow.slack.com/archives/CCQ7EGB1P/p1559134166151100]

 

Since Slack doesn't save history forever [~feluelle] , [~dlamblin], if you have something to comment please post it here so there will be a reference to whomever pick this one up.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)