You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Jason Dere <jd...@hortonworks.com> on 2017/12/11 22:16:26 UTC
Review Request 64511: HIVE-18252 Limit the size of the object
inspector caches
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64511/
-----------------------------------------------------------
Review request for hive and Ashutosh Chauhan.
Bugs: HIVE-18252
https://issues.apache.org/jira/browse/HIVE-18252
Repository: hive-git
Description
-------
Patch to limit size of the object inspector cache for complex types, and to not cache complex object inspectors that contain constants.
Diffs
-----
common/src/java/org/apache/hadoop/hive/common/LRUCache.java PRE-CREATION
common/src/test/org/apache/hadoop/hive/common/TestLRUCache.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java cb63d59
serde/src/test/org/apache/hadoop/hive/serde2/objectinspector/TestObjectInspectorFactory.java PRE-CREATION
Diff: https://reviews.apache.org/r/64511/diff/1/
Testing
-------
Added Junit tests
Thanks,
Jason Dere
Re: Review Request 64511: HIVE-18252 Limit the size of the object
inspector caches
Posted by Jason Dere <jd...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64511/
-----------------------------------------------------------
(Updated April 12, 2018, 11:45 p.m.)
Review request for hive and Ashutosh Chauhan.
Changes
-------
New approach:
- Remove caching for the complex object inspectors (list/map/union/struct).
- This requires implementing equals()/hashcode() for the complex object inspectors, as well as for the constant object inspectors.
Bugs: HIVE-18252
https://issues.apache.org/jira/browse/HIVE-18252
Repository: hive-git
Description
-------
Patch to limit size of the object inspector cache for complex types, and to not cache complex object inspectors that contain constants.
Diffs (updated)
-----
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java c0f9d08e48
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/StandardConstantListObjectInspector.java c0cdae4d32
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/StandardConstantMapObjectInspector.java 55b9fc8c2f
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/StandardConstantStructObjectInspector.java cf0be75d48
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/StandardListObjectInspector.java da35c5c901
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/StandardMapObjectInspector.java de41b97bbd
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/StandardStructObjectInspector.java 3de9a2ee31
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/StandardUnionObjectInspector.java 002514f304
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaConstantBinaryObjectInspector.java 2a4c6d9577
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaConstantBooleanObjectInspector.java d52b278bcc
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaConstantByteObjectInspector.java e24ed7c480
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaConstantDateObjectInspector.java bd86c2225d
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaConstantDoubleObjectInspector.java dcfdc142f6
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaConstantFloatObjectInspector.java c1bd97689c
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaConstantHiveCharObjectInspector.java d04140fd9f
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaConstantHiveDecimalObjectInspector.java 92f80982dd
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaConstantHiveVarcharObjectInspector.java e470f06148
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaConstantIntObjectInspector.java c0ff664f9a
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaConstantLongObjectInspector.java 3fccd3c3aa
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaConstantShortObjectInspector.java 6460318118
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaConstantStringObjectInspector.java 7be737ba75
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaConstantTimestampObjectInspector.java 2453bc67cb
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantBinaryObjectInspector.java b5582a4fab
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantBooleanObjectInspector.java 6e8152bec3
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantByteObjectInspector.java 7c88d4461d
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantDateObjectInspector.java 290fcd3ca3
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantDoubleObjectInspector.java 09c2a514be
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantFloatObjectInspector.java f02bf3e995
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantHiveCharObjectInspector.java 0746e4afb2
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantHiveDecimalObjectInspector.java 4e1d970eca
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantHiveIntervalDayTimeObjectInspector.java 6ef81189fc
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantHiveIntervalYearMonthObjectInspector.java a2f97dad95
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantHiveVarcharObjectInspector.java 76cd40369d
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantIntObjectInspector.java 129b681795
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantLongObjectInspector.java 0452def8b4
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantShortObjectInspector.java 3343b1ffc4
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantStringObjectInspector.java ba3183bf82
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantTimestampLocalTZObjectInspector.java bf461c0255
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantTimestampObjectInspector.java dc8fedfdd8
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableVoidObjectInspector.java cdd87018f6
serde/src/test/org/apache/hadoop/hive/serde2/avro/TestAvroObjectInspectorGenerator.java 3736a1f8fc
Diff: https://reviews.apache.org/r/64511/diff/2/
Changes: https://reviews.apache.org/r/64511/diff/1-2/
Testing
-------
Added Junit tests
Thanks,
Jason Dere