You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Aizhamal Nurmamat kyzy (JIRA)" <ji...@apache.org> on 2019/04/01 23:25:00 UTC
[jira] [Updated] (AIRFLOW-3889) Handle bytes type in
render_template_from_field for PubSubPublishOperator
[ https://issues.apache.org/jira/browse/AIRFLOW-3889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aizhamal Nurmamat kyzy updated AIRFLOW-3889:
--------------------------------------------
Labels: cloud-composer (was: )
> Handle bytes type in render_template_from_field for PubSubPublishOperator
> -------------------------------------------------------------------------
>
> Key: AIRFLOW-3889
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3889
> Project: Apache Airflow
> Issue Type: Bug
> Components: models
> Affects Versions: 1.10.0
> Environment: composer-1.3.0-airflow-1.10.0
> Google Cloud Composer version: 1.3.0
> Airflow version: 1.10.0
> Python version: 3.6.6
> Reporter: Tuan Vu
> Priority: Minor
> Labels: cloud-composer
>
> The render_template_from_field method of the BaseOperator class raises an exception when it encounters content that is bytes type. Specifically, the PubSubPublishOperator requires the message data to be encoded as base64.
> Error:
>
> {code:java}
> airflow.exceptions.AirflowException: Type '<class 'bytes'>' used for parameter 'messages[data]' is not supported for templating
> {code}
>
> Steps to reproduce:
> * Airflow version: 1.10.0 - I use Google Cloud Composer with Airflow v1.10.0
> * Python version: 3.6.6
> * Run through the example_pubsub_flow: [https://github.com/apache/airflow/blob/53b89b98371c7bb993b242c341d3941e9ce09f9a/airflow/contrib/example_dags/example_pubsub_flow.py]
> * Encounter errors on task t3 in the DAG.
> Input:
> {code:java}
> messages = [
> {'data': b64encode(b'Hello World')},
> {'data': b64encode(b'Another message')},
> {'data': b64encode(b'A final message')}
> ]
> {code}
> Task:
> {code:java}
> t3 = PubSubPublishOperator(
> task_id='publish-messages', messages=messages)
> {code}
> Output error:
> {code:java}
> airflow.exceptions.AirflowException: Type '<class 'bytes'>' used for parameter 'messages[data]' is not supported for templating{code}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)