You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@mrql.apache.org by "Leonidas Fegaras (JIRA)" <ji...@apache.org> on 2017/01/28 14:03:24 UTC

[jira] [Closed] (MRQL-101) Fix memory materialization of lazy bags

     [ https://issues.apache.org/jira/browse/MRQL-101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Leonidas Fegaras closed MRQL-101.
---------------------------------
    Resolution: Fixed

> Fix memory materialization of lazy bags
> ---------------------------------------
>
>                 Key: MRQL-101
>                 URL: https://issues.apache.org/jira/browse/MRQL-101
>             Project: MRQL
>          Issue Type: Bug
>          Components: Run-Time/Flink, Run-Time/Spark
>    Affects Versions: 0.9.8
>            Reporter: Leonidas Fegaras
>            Assignee: Leonidas Fegaras
>
> This patch addresses various bugs in bag Materialization. Materializing lazy bags (stream-based bags implemented as iterators) is needed in a MRQL plan If the plan returns a lazy Bag and the term occurs in the code multiple times. In that case, we generate code to materialize this Bag in memory (as an array). The new materialization algorithm in this patch is simpler and more general, and addresses various bugs which result to the error "The collection stream has already been consumed".



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)