You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Timothy Farkas (JIRA)" <ji...@apache.org> on 2018/07/27 19:42:00 UTC
[jira] [Updated] (DRILL-6644) In Some Cases The HashJoin Memory
Calculator Over Reserves Memory
[ https://issues.apache.org/jira/browse/DRILL-6644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Timothy Farkas updated DRILL-6644:
----------------------------------
Description:
There are two cases where the HashJoin Memory calculator over reserves memory:
1. It reserves a maximum incoming probe batch size during the build phase. This is not really necessary because we will not fetch probe data until the probe phase. We only have to account for the data received during OK_NEW_SCHEMA.
2. When preparing for the probe phase it uses the worst case hashtable size even after we have constructed a hash table. In this case when we already have a hash table we should not use it's predicted worst case size, we should use its actual size in the memory calculator.
was:
There are two cases where the HashJoin Memory calculator over reserves memory:
1. It reserves a maximum incoming probe batch size during the build phase. This is not really necessary because we will not fetch probe data until the probe phase. We only have to account for the space data received during OK_NEW_SCHEMA occupied.
2. When preparing for the probe phase it uses the worst case hashtable size even after we have constructed a hash table. In this case when we already have a hash table we should not use it's predicted worst case size, we should use its actual size in the memory calculator.
> In Some Cases The HashJoin Memory Calculator Over Reserves Memory
> -----------------------------------------------------------------
>
> Key: DRILL-6644
> URL: https://issues.apache.org/jira/browse/DRILL-6644
> Project: Apache Drill
> Issue Type: Improvement
> Reporter: Timothy Farkas
> Assignee: Timothy Farkas
> Priority: Major
>
> There are two cases where the HashJoin Memory calculator over reserves memory:
> 1. It reserves a maximum incoming probe batch size during the build phase. This is not really necessary because we will not fetch probe data until the probe phase. We only have to account for the data received during OK_NEW_SCHEMA.
> 2. When preparing for the probe phase it uses the worst case hashtable size even after we have constructed a hash table. In this case when we already have a hash table we should not use it's predicted worst case size, we should use its actual size in the memory calculator.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)