[jira] [Commented] (ASTERIXDB-2577) Not keeping one frame for each spilled partition before starting the probe phase in hybrid hash join

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ASTERIXDB-2577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16902660#comment-16902660 ] 

ASF subversion and git services commented on ASTERIXDB-2577:
------------------------------------------------------------

Commit c677bf58926da9139f3c93e73c9eae1c6b491629 in asterixdb's branch refs/heads/master from Shiva
[ https://gitbox.apache.org/repos/asf?p=asterixdb.git;h=c677bf5 ]

[ASTERIXDB-2577][RT] One frame per spilled partition in optimized hybrid hash join

- user model changes: no
- storage format changes: no
- interface changes: no

Details:

Makes sure there is one frame for each spilled partition during the build close.
Checks the length of large record before flushing a record to the disk blindly
as being large.

Change-Id: I82d4da57e9a9835cab61dc5cc43c45c728e0c2b6
Reviewed-on: https://asterix-gerrit.ics.uci.edu/3485
Contrib: Jenkins <je...@fulliautomatix.ics.uci.edu>
Tested-by: Jenkins <je...@fulliautomatix.ics.uci.edu>
Integration-Tests: Jenkins <je...@fulliautomatix.ics.uci.edu>
Reviewed-by: Ali Alsuliman <al...@gmail.com>


> Not keeping one frame for each spilled partition before starting the probe phase in hybrid hash join
> ----------------------------------------------------------------------------------------------------
>
>                 Key: ASTERIXDB-2577
>                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-2577
>             Project: Apache AsterixDB
>          Issue Type: Bug
>          Components: RT - Runtime
>    Affects Versions: 0.9.4.1
>            Reporter: Shiva Jahangiri
>            Priority: Major
>
> In probe() method in optimized hybrid hash join, if insertion fails on the current spilled partition, we try to find the biggest spilled partition and flush it as a victim. If we could not find any spilled partition with size > 0, then we ASSUME that the record is large and flush it as a big object. By running customerOrderCIDHybridHashJoin_Case3() test in 
> TPCHCustomerOrderHashJoinTest, it can be seen that the record size is 206 bytes (so it is smaller than a frame), but neither the spilled partitions nor the buffer manager has any frame (This is the problem, there should be 1 frame for each spilled partition). In this case, we flush the record as a large object. This means that every single record that is supposed to get inserted to a spilled partition during the probe, will get flushed separately. 



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)