You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "Sergio Kef (Jira)" <ji...@apache.org> on 2019/09/28 19:35:00 UTC

[jira] [Commented] (AIRFLOW-487) Split/Chunk operator

    [ https://issues.apache.org/jira/browse/AIRFLOW-487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16940145#comment-16940145 ] 

Sergio Kef commented on AIRFLOW-487:
------------------------------------

I don't understand how this would look like. Can one give an example?

> Split/Chunk operator
> --------------------
>
>                 Key: AIRFLOW-487
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-487
>             Project: Apache Airflow
>          Issue Type: New Feature
>            Reporter: mohsen
>            Priority: Major
>
> it would be nice to have a split (python) operator, which takes a job that returns a list and streams each item to the specified job. like this pattern: http://www.workflowpatterns.com/patterns/control/basic/wcp2.php
> also, a chunk operator that splits a list to smaller chunks is more general solution
> i don't know if i should post this feature request here, or if there is any try on this already. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)