[jira] [Commented] (AIRFLOW-1932) Add Pull and Acknowledgment capabilities for GCP Pub/Sub

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16299121#comment-16299121 ] 

ASF subversion and git services commented on AIRFLOW-1932:
----------------------------------------------------------

Commit 6645218092096e4b10fc737a62bacc2670e1d6dc in incubator-airflow's branch refs/heads/master from [~prodonjs]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=6645218 ]

[AIRFLOW-1932] Add GCP Pub/Sub Pull and Ack

Adds the necessary hooks to support pulling and
acknowleding Pub/Sub
messages. This is implemented by adding a
PubSubPullSensor operator
that will attempt to retrieve messages from a
specified subscription
and will meet its criteria when a message or
messages is available.
The configuration allows those messages to be
acknowledged immediately.
In addition, the messages are passed to downstream
workers via the
return value of operator's execute method.

An end-to-end example is included showing topic
and subscription
creation, parallel tasks to publish and pull
messages, and a downstream
chain to echo the contents of each message before
cleaning up.

Closes #2885 from prodonjs/airflow-1932-pr


> Add Pull and Acknowledgment capabilities for GCP Pub/Sub
> --------------------------------------------------------
>
>                 Key: AIRFLOW-1932
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1932
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: contrib, gcp, hooks, operators
>    Affects Versions: Airflow 1.8
>            Reporter: Jason Prodonovich
>            Assignee: Jason Prodonovich
>
> To complete the suite of operations available for Google Cloud Pub/Sub, add the necessary hooks to pull messages and acknowledge messages from a subscription. Also, create a sensor operator that will block until the subscription returns at least one message. The operator should return the retrieved messages so they can be used by downstream tasks if desired.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)