You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@beam.apache.org by "Reuven Lax (Jira)" <ji...@apache.org> on 2019/11/06 19:07:00 UTC

[jira] [Assigned] (BEAM-6756) Support lazy iterables in schemas

     [ https://issues.apache.org/jira/browse/BEAM-6756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Reuven Lax reassigned BEAM-6756:
--------------------------------

    Assignee: Reuven Lax  (was: Shehzaad Nakhoda)

> Support lazy iterables in schemas
> ---------------------------------
>
>                 Key: BEAM-6756
>                 URL: https://issues.apache.org/jira/browse/BEAM-6756
>             Project: Beam
>          Issue Type: Sub-task
>          Components: sdk-java-core
>            Reporter: Reuven Lax
>            Assignee: Reuven Lax
>            Priority: Major
>          Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> The iterables returned by GroupByKey and CoGroupByKey are lazy; this allows a runner to page data into memory if the full iterable is too large. We currently don't support this in Schemas, so the Schema Group and CoGroup transforms materialize all data into memory. We should add support for this.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)