You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@calcite.apache.org by "liyang (JIRA)" <ji...@apache.org> on 2015/08/22 00:59:45 UTC

[jira] [Created] (CALCITE-853) EnumerableAggregate should take advantage of input collation

liyang created CALCITE-853:
------------------------------

             Summary: EnumerableAggregate should take advantage of input collation
                 Key: CALCITE-853
                 URL: https://issues.apache.org/jira/browse/CALCITE-853
             Project: Calcite
          Issue Type: Improvement
            Reporter: liyang
            Assignee: Julian Hyde


Li Yang <li...@apache.org>
Aug 20 (2 days ago)
		
I encountered Out Of Mem exception when a huge result set is passed into EnumerableAggregate and get aggregated in memory. I'm thinking if the input is sorted by the group-by key, then the groupBy() don't have to hold all data in memory any more.

Julian Hyde <jh...@apache.org>
2:20 PM (16 hours ago)
		
Yes, that would be useful. Please log a jira.

Enumerable.groupBy doesn't know its input's collation so can't make that decision, but EnumerableAggregate does. I think that EnumerableAggregate should have a "trigger key", a subset of its group key, and if the trigger key changes it will emit and flush its hash table.

As well as for your use case, it will be useful for streaming queries.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)