[jira] [Created] (ASTERIXDB-2656) No need for frame constraint in probe phase of hybrid hash join

["Shiva Jahangiri (Jira)" <ji...@apache.org>]

Shiva Jahangiri created ASTERIXDB-2656:
------------------------------------------

             Summary: No need for frame constraint in probe phase of hybrid hash join
                 Key: ASTERIXDB-2656
                 URL: https://issues.apache.org/jira/browse/ASTERIXDB-2656
             Project: Apache AsterixDB
          Issue Type: Bug
          Components: *DB - AsterixDB
            Reporter: Shiva Jahangiri


In an optimized hybrid hash join, there is a constraint that a spilled partition cannot have more than 1 frame. While it seems to be necessary during the build phase to help with having as much data for in-memory partitions, it seems an unnecessary constraint during the probe phase.

During the probe, we should let the spilled partitions to grow and use the leftover frames not used by in-memory partitions. This also solves a bug that hybrid hash join currently has.

During the probe phase, in case there is not enough memory for inserting a large record, asterixdb may either flush the record directly to the disk or spill the largest spilled partition to the disk, depending on which one is larger. In the case that the biggest spilled partition is the vicitim to flush and release the frames, the large record cannot use the freed frames as it belongs to a spilled partition(otherwise it would have been probed) and as such 1-frame limit is applied on it.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)