You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@mrql.apache.org by "Leonidas Fegaras (JIRA)" <ji...@apache.org> on 2017/01/26 19:34:24 UTC

[jira] [Created] (MRQL-101) Fix memory materialization of lazy bags

Leonidas Fegaras created MRQL-101:
-------------------------------------

             Summary: Fix memory materialization of lazy bags
                 Key: MRQL-101
                 URL: https://issues.apache.org/jira/browse/MRQL-101
             Project: MRQL
          Issue Type: Bug
          Components: Run-Time/Flink, Run-Time/Spark
    Affects Versions: 0.9.8
            Reporter: Leonidas Fegaras
            Assignee: Leonidas Fegaras


This patch addresses various bugs in bag Materialization. Materializing lazy bags (stream-based bags implemented as iterators) is needed in a MRQL plan If the plan returns a lazy Bag and the term occurs in the code multiple times. In that case, we generate code to materialize this Bag in memory (as an array). The new materialization algorithm in this patch is simpler and more general, and addresses various bugs which result to the error "The collection stream has already been consumed".



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)