[jira] [Commented] (AIRFLOW-240) TemplateNotFound Error when using PostgresOperator and SQL script

["Greg Reda (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15330664#comment-15330664 ] 

Greg Reda commented on AIRFLOW-240:
-----------------------------------

Threw this into a Stack Overflow question in hopes of making it more easily Google-able: http://stackoverflow.com/questions/37822380/templatenotfound-when-using-airflows-postgresoperator-with-jinja-templating-and

> TemplateNotFound Error when using PostgresOperator and SQL script
> -----------------------------------------------------------------
>
>                 Key: AIRFLOW-240
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-240
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: operators
>         Environment: Ubuntu 14.04
>            Reporter: Greg Reda
>
> When using the PostgresOperator, I've been unable to get templating to work. It's quite possible I'm doing something wrong, but I'm pretty lost as to what the issue might be. Here's an example to reproduce the TemplateNotFound error I've been getting:
> {code:title=/my_dag/csv_to_redshift.py}
> copy_s3_to_redshift = PostgresOperator(
>     task_id='load_table',
>     sql=this_dag_path + '/copy_to_redshift.sql',
>     params=dict(
>         AWS_ACCESS_KEY_ID=Variable.get('AWS_ACCESS_KEY_ID'),
>         AWS_SECRET_ACCESS_KEY=Variable.get('AWS_SECRET_ACCESS_KEY')
>     ),
>     postgres_conn_id='postgres_redshift',
>     autocommit=False,
>     dag=dag
> )
> {code}
> {code:title=/my_dag/copy_to_redshift.sql}
> COPY public.table_foobar FROM 's3://mybucket/test-data/import/foobar.csv'
> CREDENTIALS 'aws_access_key_id={{ AWS_ACCESS_KEY_ID }};aws_secret_access_key={{ AWS_SECRET_ACCESS_KEY }}'
> CSV
> NULL as 'null'
> IGNOREHEADER as 1;
> {code}
> {code}
> Traceback (most recent call last):
>   File "/usr/local/lib/python2.7/dist-packages/airflow/models.py", line 2038, in resolve_template_files
>     setattr(self, attr, env.loader.get_source(env, content)[0])
>   File "/usr/local/lib/python2.7/dist-packages/jinja2/loaders.py", line 187, in get_source
>     raise TemplateNotFound(template)
> TemplateNotFound: /home/gregreda/airflow/dags/my_dag/copy_to_redshift.sql
> {code}
> Any ideas towards a fix are much appreciated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)