[jira] [Created] (HIVE-5165) Evaluate the possibility of enabling feature similar to pinning tables in memory

["Srikanth Sundarrajan (JIRA)" <ji...@apache.org>]

Srikanth Sundarrajan created HIVE-5165:
------------------------------------------

             Summary: Evaluate the possibility of enabling feature similar to pinning tables in memory
                 Key: HIVE-5165
                 URL: https://issues.apache.org/jira/browse/HIVE-5165
             Project: Hive
          Issue Type: Improvement
          Components: Query Processor
            Reporter: Srikanth Sundarrajan


There are many queries involving joins to small tables (ex. dimension tables). These can be loaded in memory in the map phase to avoid using reduce for the join operation, but this requires that adequate memory be available to each map to load the table contents. Further if more small related tables need to be loaded to realize a join, memory foot print increases, forcing the system to perhaps consider reduce side join. This issue is to consider the possibility of using something similar to 

{code}
CREATE TABLE t (i INT) ENGINE = MEMORY;
{code}

in mysql, where all relatively small tables (< million rows or < 1G) can be loaded in memory in a service that the map phase can leverage (over the network). 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira