[jira] [Created] (AIRFLOW-240) TemplateNotFound Error when using PostgresOperator and SQL script

["Greg Reda (JIRA)" <ji...@apache.org>]

Greg Reda created AIRFLOW-240:
---------------------------------

             Summary: TemplateNotFound Error when using PostgresOperator and SQL script
                 Key: AIRFLOW-240
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-240
             Project: Apache Airflow
          Issue Type: Bug
          Components: operators
         Environment: Ubuntu 14.04
            Reporter: Greg Reda


When using the PostgresOperator, I've been unable to get templating for instance. Here's an example to reproduce the error:

/my_dag/csv_to_redshift.py
----
copy_s3_to_redshift = PostgresOperator(
    task_id='load_table',
    sql=this_dag_path + '/copy_to_redshift.sql',
    params=dict(
        AWS_ACCESS_KEY_ID=Variable.get('AWS_ACCESS_KEY_ID'),
        AWS_SECRET_ACCESS_KEY=Variable.get('AWS_SECRET_ACCESS_KEY')
    ),
    postgres_conn_id='postgres_redshift',
    autocommit=False,
    dag=dag
)


/my_dag/copy_to_redshift.sql
----
COPY public.table_foobar FROM 's3://sprout_data_science_s3/test-data/import/foobar.csv'
CREDENTIALS 'aws_access_key_id={{ AWS_ACCESS_KEY_ID }};aws_secret_access_key={{ AWS_SECRET_ACCESS_KEY }}'
CSV
NULL as 'null'
IGNOREHEADER as 1;


Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/airflow/models.py", line 2038, in resolve_template_files
    setattr(self, attr, env.loader.get_source(env, content)[0])
  File "/usr/local/lib/python2.7/dist-packages/jinja2/loaders.py", line 187, in get_source
    raise TemplateNotFound(template)
TemplateNotFound: /home/gregreda/airflow/dags/example_csv_to_redshift/copy_to_redshift.sql

Any ideas?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)