You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Jason Dere <jd...@hortonworks.com> on 2015/03/03 20:34:16 UTC
Review Request 31696: HIVE-9792 Support interval type in
expressions/predicates
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31696/
-----------------------------------------------------------
Review request for hive.
Bugs: HIVE-9792
https://issues.apache.org/jira/browse/HIVE-9792
Repository: hive-git
Description
-------
This adds 2 separate interval types:
- year-month interval, which represents a date difference in years and months. Internally represented as an offset of months.
- day-time interval, which represents a date difference in days/hours/minutes/seconds/nanoseconds. 1 day = 24 hours = 1440 minutes = 86400 seconds. Internally represented as an offset of seconds (long) and nanoseconds (int).
I've tried using the Oracle's Datetime and Interval Arithmetic rules with regard to timestamp arithmetic, where the timestamp arithmetic is done in UTC time. This results in similar behavior to below depending on the default time zone in Hive:
SQL> select timestamp '1999-09-15 00:00:00 America/Los_Angeles' - timestamp '1999-12-15 00:00:00 America/Los_Angeles' from dual;
TIMESTAMP'1999-09-1500:00:00AMERICA/LOS_ANGELES'-TIMESTAMP'1999-12-1500:00:
---------------------------------------------------------------------------
-000000091 01:00:00.000000000
SQL> select timestamp '1999-09-15 00:00:00 UTC' - timestamp '1999-12-15 00:00:00 UTC' from dual;
TIMESTAMP'1999-09-1500:00:00UTC'-TIMESTAMP'1999-12-1500:00:00UTC'
---------------------------------------------------------------------------
-000000091 00:00:00.000000000
Diffs
-----
common/src/java/org/apache/hadoop/hive/common/type/HiveIntervalDayTime.java PRE-CREATION
common/src/java/org/apache/hadoop/hive/common/type/HiveIntervalYearMonth.java PRE-CREATION
common/src/java/org/apache/hadoop/hive/common/type/MutableHiveIntervalDayTime.java PRE-CREATION
common/src/java/org/apache/hadoop/hive/common/type/MutableHiveIntervalYearMonth.java PRE-CREATION
common/src/java/org/apache/hive/common/util/DateTimeUtils.java PRE-CREATION
common/src/test/org/apache/hadoop/hive/common/type/TestHiveIntervalDayTime.java PRE-CREATION
common/src/test/org/apache/hadoop/hive/common/type/TestHiveIntervalYearMonth.java PRE-CREATION
common/src/test/org/apache/hive/common/util/TestDateTimeUtils.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 20811ab
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinKey.java 6a3c300
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTBuilder.java 4b1f5c1
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java 4dbac57
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java 88c989f
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 90b84ac
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 72b852e
ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g cabf971
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 706390b
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseArithmetic.java 575c764
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseDTI.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseUnary.java c5bec44
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDTIMinus.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDTIPlus.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMinus.java 5419a19
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPNegative.java d653264
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPPlus.java accd347
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToIntervalDayTime.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToIntervalYearMonth.java PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFOPMinus.java 24618c9
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFOPPlus.java 4b2f7fb
ql/src/test/queries/clientnegative/interval_1.q PRE-CREATION
ql/src/test/queries/clientnegative/interval_2.q PRE-CREATION
ql/src/test/queries/clientnegative/interval_3.q PRE-CREATION
ql/src/test/queries/clientnegative/invalid_arithmetic_type.q ad37cff
ql/src/test/queries/clientpositive/interval_1.q PRE-CREATION
ql/src/test/queries/clientpositive/interval_2.q PRE-CREATION
ql/src/test/queries/clientpositive/interval_3.q PRE-CREATION
ql/src/test/queries/clientpositive/interval_arithmetic.q PRE-CREATION
ql/src/test/queries/clientpositive/interval_comparison.q PRE-CREATION
ql/src/test/results/clientnegative/interval_1.q.out PRE-CREATION
ql/src/test/results/clientnegative/interval_2.q.out PRE-CREATION
ql/src/test/results/clientnegative/interval_3.q.out PRE-CREATION
ql/src/test/results/clientnegative/invalid_arithmetic_type.q.out afd09ca
ql/src/test/results/clientpositive/interval_1.q.out PRE-CREATION
ql/src/test/results/clientpositive/interval_2.q.out PRE-CREATION
ql/src/test/results/clientpositive/interval_3.q.out PRE-CREATION
ql/src/test/results/clientpositive/interval_arithmetic.q.out PRE-CREATION
ql/src/test/results/clientpositive/interval_comparison.q.out PRE-CREATION
serde/if/serde.thrift 2db27b5
serde/src/gen/thrift/gen-cpp/serde_constants.h 418f666
serde/src/gen/thrift/gen-cpp/serde_constants.cpp cc71b65
serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java e70d0c4
serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php c1d8085
serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py 7d7608f
serde/src/gen/thrift/gen-rb/serde_constants.rb 33f12b0
serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 2b7fba6
serde/src/java/org/apache/hadoop/hive/serde2/io/HiveIntervalDayTimeWritable.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/io/HiveIntervalYearMonthWritable.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 864d9aa
serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyHiveIntervalDayTime.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyHiveIntervalYearMonth.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 879743f
serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyHiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyHiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 7423c00
serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 8bd5838
serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryHiveIntervalDayTime.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryHiveIntervalYearMonth.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 62cba01
serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 11e8cf4
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 2a47d97
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 5e67a07
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 05aed0a
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/HiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/HiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaHiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaHiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java cb996a8
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java f466297
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 5ccacf1
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableHiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableHiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantHiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantHiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableHiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableHiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java e5c9f18
serde/src/test/org/apache/hadoop/hive/serde2/io/TestHiveIntervalDayTimeWritable.java PRE-CREATION
serde/src/test/org/apache/hadoop/hive/serde2/io/TestHiveIntervalYearMonthWritable.java PRE-CREATION
Diff: https://reviews.apache.org/r/31696/diff/
Testing
-------
Thanks,
Jason Dere
Re: Review Request 31696: HIVE-9792 Support interval type in
expressions/predicates
Posted by Jason Dere <jd...@hortonworks.com>.
> On March 17, 2015, 5:32 p.m., Ashutosh Chauhan wrote:
> > common/src/java/org/apache/hadoop/hive/common/type/MutableHiveIntervalDayTime.java, line 21
> > <https://reviews.apache.org/r/31696/diff/3/?file=892328#file892328line21>
> >
> > Unless there is an advantage of having seprate Mutable classes, its better to just make parent class mutable, otherwise it may be confusing to dev about which one to use when.
ok, will change
> On March 17, 2015, 5:32 p.m., Ashutosh Chauhan wrote:
> > common/src/java/org/apache/hadoop/hive/common/type/MutableHiveIntervalYearMonth.java, line 21
> > <https://reviews.apache.org/r/31696/diff/3/?file=892329#file892329line21>
> >
> > same as prev comment.
will change
> On March 17, 2015, 5:32 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDTIMinus.java, line 48
> > <https://reviews.apache.org/r/31696/diff/3/?file=892347#file892347line48>
> >
> > It will be good to provide @Description annotation here, so that info can be displayed to user via `describe function`.
will fix
> On March 17, 2015, 5:32 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDTIMinus.java, line 66
> > <https://reviews.apache.org/r/31696/diff/3/?file=892347#file892347line66>
> >
> > Will be good to check if there are other ops allowed by standard. If there are, it will be good to leave a note here saying that although standard allows those, we havent added support for those yet.
I believe I have included the all the appropriate operations for addition/subtraction. The standard also includes multiplication/division by numeric values, which can be added in followup work.
> On March 17, 2015, 5:32 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDTIPlus.java, line 48
> > <https://reviews.apache.org/r/31696/diff/3/?file=892348#file892348line48>
> >
> > @Description annotation.
will fix
> On March 17, 2015, 5:32 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDTIPlus.java, line 64
> > <https://reviews.apache.org/r/31696/diff/3/?file=892348#file892348line64>
> >
> > Check with standard about allowed operands.
this should be fine
> On March 17, 2015, 5:32 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToIntervalDayTime.java, line 39
> > <https://reviews.apache.org/r/31696/diff/3/?file=892352#file892352line39>
> >
> > @Description
will fix
> On March 17, 2015, 5:32 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToIntervalYearMonth.java, line 39
> > <https://reviews.apache.org/r/31696/diff/3/?file=892353#file892353line39>
> >
> > @Description
will fix
- Jason
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31696/#review76685
-----------------------------------------------------------
On March 12, 2015, 7:50 p.m., Jason Dere wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31696/
> -----------------------------------------------------------
>
> (Updated March 12, 2015, 7:50 p.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Bugs: HIVE-9792
> https://issues.apache.org/jira/browse/HIVE-9792
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> This adds 2 separate interval types:
> - year-month interval, which represents a date difference in years and months. Internally represented as an offset of months.
> - day-time interval, which represents a date difference in days/hours/minutes/seconds/nanoseconds. 1 day = 24 hours = 1440 minutes = 86400 seconds. Internally represented as an offset of seconds (long) and nanoseconds (int).
>
> I've tried using the Oracle's Datetime and Interval Arithmetic rules with regard to timestamp arithmetic, where the timestamp arithmetic is done in UTC time. This results in similar behavior to below depending on the default time zone in Hive:
>
> SQL> select timestamp '1999-09-15 00:00:00 America/Los_Angeles' - timestamp '1999-12-15 00:00:00 America/Los_Angeles' from dual;
>
> TIMESTAMP'1999-09-1500:00:00AMERICA/LOS_ANGELES'-TIMESTAMP'1999-12-1500:00:
> ---------------------------------------------------------------------------
> -000000091 01:00:00.000000000
>
> SQL> select timestamp '1999-09-15 00:00:00 UTC' - timestamp '1999-12-15 00:00:00 UTC' from dual;
>
> TIMESTAMP'1999-09-1500:00:00UTC'-TIMESTAMP'1999-12-1500:00:00UTC'
> ---------------------------------------------------------------------------
> -000000091 00:00:00.000000000
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/common/type/HiveIntervalDayTime.java PRE-CREATION
> common/src/java/org/apache/hadoop/hive/common/type/HiveIntervalYearMonth.java PRE-CREATION
> common/src/java/org/apache/hadoop/hive/common/type/MutableHiveIntervalDayTime.java PRE-CREATION
> common/src/java/org/apache/hadoop/hive/common/type/MutableHiveIntervalYearMonth.java PRE-CREATION
> common/src/java/org/apache/hive/common/util/DateTimeMath.java PRE-CREATION
> common/src/java/org/apache/hive/common/util/DateUtils.java 92b7f62
> common/src/test/org/apache/hadoop/hive/common/type/TestHiveIntervalDayTime.java PRE-CREATION
> common/src/test/org/apache/hadoop/hive/common/type/TestHiveIntervalYearMonth.java PRE-CREATION
> common/src/test/org/apache/hive/common/util/TestDateTimeMath.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 20811ab
> ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinKey.java 6a3c300
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTBuilder.java 3a2c96c
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java 4dbac57
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java 88c989f
> ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g e7de6c8
> ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g d2d9989
> ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 0a05ceb
> ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java e8217fd
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseArithmetic.java 575c764
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseDTI.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseUnary.java c5bec44
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDTIMinus.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDTIPlus.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMinus.java 5419a19
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPNegative.java d653264
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPPlus.java accd347
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToIntervalDayTime.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToIntervalYearMonth.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFOPMinus.java 24618c9
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFOPPlus.java 4b2f7fb
> ql/src/test/queries/clientnegative/interval_1.q PRE-CREATION
> ql/src/test/queries/clientnegative/interval_2.q PRE-CREATION
> ql/src/test/queries/clientnegative/interval_3.q PRE-CREATION
> ql/src/test/queries/clientnegative/invalid_arithmetic_type.q ad37cff
> ql/src/test/queries/clientpositive/interval_1.q PRE-CREATION
> ql/src/test/queries/clientpositive/interval_2.q PRE-CREATION
> ql/src/test/queries/clientpositive/interval_3.q PRE-CREATION
> ql/src/test/queries/clientpositive/interval_arithmetic.q PRE-CREATION
> ql/src/test/queries/clientpositive/interval_comparison.q PRE-CREATION
> ql/src/test/results/clientnegative/interval_1.q.out PRE-CREATION
> ql/src/test/results/clientnegative/interval_2.q.out PRE-CREATION
> ql/src/test/results/clientnegative/interval_3.q.out PRE-CREATION
> ql/src/test/results/clientnegative/invalid_arithmetic_type.q.out afd09ca
> ql/src/test/results/clientpositive/interval_1.q.out PRE-CREATION
> ql/src/test/results/clientpositive/interval_2.q.out PRE-CREATION
> ql/src/test/results/clientpositive/interval_3.q.out PRE-CREATION
> ql/src/test/results/clientpositive/interval_arithmetic.q.out PRE-CREATION
> ql/src/test/results/clientpositive/interval_comparison.q.out PRE-CREATION
> serde/if/serde.thrift 2db27b5
> serde/src/gen/thrift/gen-cpp/serde_constants.h 418f666
> serde/src/gen/thrift/gen-cpp/serde_constants.cpp cc71b65
> serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java e70d0c4
> serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php c1d8085
> serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py 7d7608f
> serde/src/gen/thrift/gen-rb/serde_constants.rb 33f12b0
> serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 2b7fba6
> serde/src/java/org/apache/hadoop/hive/serde2/io/HiveIntervalDayTimeWritable.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/io/HiveIntervalYearMonthWritable.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 864d9aa
> serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyHiveIntervalDayTime.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyHiveIntervalYearMonth.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 879743f
> serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyHiveIntervalDayTimeObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyHiveIntervalYearMonthObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 7423c00
> serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 8bd5838
> serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryHiveIntervalDayTime.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryHiveIntervalYearMonth.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 62cba01
> serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 11e8cf4
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 2a47d97
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 5e67a07
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 05aed0a
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/HiveIntervalDayTimeObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/HiveIntervalYearMonthObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaHiveIntervalDayTimeObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaHiveIntervalYearMonthObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java cb996a8
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java f466297
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 5ccacf1
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableHiveIntervalDayTimeObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableHiveIntervalYearMonthObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantHiveIntervalDayTimeObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantHiveIntervalYearMonthObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableHiveIntervalDayTimeObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableHiveIntervalYearMonthObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java e5c9f18
> serde/src/test/org/apache/hadoop/hive/serde2/io/TestHiveIntervalDayTimeWritable.java PRE-CREATION
> serde/src/test/org/apache/hadoop/hive/serde2/io/TestHiveIntervalYearMonthWritable.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/31696/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jason Dere
>
>
Re: Review Request 31696: HIVE-9792 Support interval type in
expressions/predicates
Posted by Ashutosh Chauhan <ha...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31696/#review76685
-----------------------------------------------------------
Thanks for excellent tests! (Both unit as well as system)
common/src/java/org/apache/hadoop/hive/common/type/MutableHiveIntervalDayTime.java
<https://reviews.apache.org/r/31696/#comment124316>
Unless there is an advantage of having seprate Mutable classes, its better to just make parent class mutable, otherwise it may be confusing to dev about which one to use when.
common/src/java/org/apache/hadoop/hive/common/type/MutableHiveIntervalYearMonth.java
<https://reviews.apache.org/r/31696/#comment124317>
same as prev comment.
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDTIMinus.java
<https://reviews.apache.org/r/31696/#comment124362>
It will be good to provide @Description annotation here, so that info can be displayed to user via `describe function`.
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDTIMinus.java
<https://reviews.apache.org/r/31696/#comment124363>
Will be good to check if there are other ops allowed by standard. If there are, it will be good to leave a note here saying that although standard allows those, we havent added support for those yet.
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDTIPlus.java
<https://reviews.apache.org/r/31696/#comment124364>
@Description annotation.
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDTIPlus.java
<https://reviews.apache.org/r/31696/#comment124365>
Check with standard about allowed operands.
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToIntervalDayTime.java
<https://reviews.apache.org/r/31696/#comment124366>
@Description
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToIntervalYearMonth.java
<https://reviews.apache.org/r/31696/#comment124367>
@Description
ql/src/test/results/clientpositive/interval_arithmetic.q.out
<https://reviews.apache.org/r/31696/#comment124370>
Constant folding kicked in. Cool!
- Ashutosh Chauhan
On March 12, 2015, 7:50 p.m., Jason Dere wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31696/
> -----------------------------------------------------------
>
> (Updated March 12, 2015, 7:50 p.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Bugs: HIVE-9792
> https://issues.apache.org/jira/browse/HIVE-9792
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> This adds 2 separate interval types:
> - year-month interval, which represents a date difference in years and months. Internally represented as an offset of months.
> - day-time interval, which represents a date difference in days/hours/minutes/seconds/nanoseconds. 1 day = 24 hours = 1440 minutes = 86400 seconds. Internally represented as an offset of seconds (long) and nanoseconds (int).
>
> I've tried using the Oracle's Datetime and Interval Arithmetic rules with regard to timestamp arithmetic, where the timestamp arithmetic is done in UTC time. This results in similar behavior to below depending on the default time zone in Hive:
>
> SQL> select timestamp '1999-09-15 00:00:00 America/Los_Angeles' - timestamp '1999-12-15 00:00:00 America/Los_Angeles' from dual;
>
> TIMESTAMP'1999-09-1500:00:00AMERICA/LOS_ANGELES'-TIMESTAMP'1999-12-1500:00:
> ---------------------------------------------------------------------------
> -000000091 01:00:00.000000000
>
> SQL> select timestamp '1999-09-15 00:00:00 UTC' - timestamp '1999-12-15 00:00:00 UTC' from dual;
>
> TIMESTAMP'1999-09-1500:00:00UTC'-TIMESTAMP'1999-12-1500:00:00UTC'
> ---------------------------------------------------------------------------
> -000000091 00:00:00.000000000
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/common/type/HiveIntervalDayTime.java PRE-CREATION
> common/src/java/org/apache/hadoop/hive/common/type/HiveIntervalYearMonth.java PRE-CREATION
> common/src/java/org/apache/hadoop/hive/common/type/MutableHiveIntervalDayTime.java PRE-CREATION
> common/src/java/org/apache/hadoop/hive/common/type/MutableHiveIntervalYearMonth.java PRE-CREATION
> common/src/java/org/apache/hive/common/util/DateTimeMath.java PRE-CREATION
> common/src/java/org/apache/hive/common/util/DateUtils.java 92b7f62
> common/src/test/org/apache/hadoop/hive/common/type/TestHiveIntervalDayTime.java PRE-CREATION
> common/src/test/org/apache/hadoop/hive/common/type/TestHiveIntervalYearMonth.java PRE-CREATION
> common/src/test/org/apache/hive/common/util/TestDateTimeMath.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 20811ab
> ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinKey.java 6a3c300
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTBuilder.java 3a2c96c
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java 4dbac57
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java 88c989f
> ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g e7de6c8
> ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g d2d9989
> ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 0a05ceb
> ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java e8217fd
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseArithmetic.java 575c764
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseDTI.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseUnary.java c5bec44
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDTIMinus.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDTIPlus.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMinus.java 5419a19
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPNegative.java d653264
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPPlus.java accd347
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToIntervalDayTime.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToIntervalYearMonth.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFOPMinus.java 24618c9
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFOPPlus.java 4b2f7fb
> ql/src/test/queries/clientnegative/interval_1.q PRE-CREATION
> ql/src/test/queries/clientnegative/interval_2.q PRE-CREATION
> ql/src/test/queries/clientnegative/interval_3.q PRE-CREATION
> ql/src/test/queries/clientnegative/invalid_arithmetic_type.q ad37cff
> ql/src/test/queries/clientpositive/interval_1.q PRE-CREATION
> ql/src/test/queries/clientpositive/interval_2.q PRE-CREATION
> ql/src/test/queries/clientpositive/interval_3.q PRE-CREATION
> ql/src/test/queries/clientpositive/interval_arithmetic.q PRE-CREATION
> ql/src/test/queries/clientpositive/interval_comparison.q PRE-CREATION
> ql/src/test/results/clientnegative/interval_1.q.out PRE-CREATION
> ql/src/test/results/clientnegative/interval_2.q.out PRE-CREATION
> ql/src/test/results/clientnegative/interval_3.q.out PRE-CREATION
> ql/src/test/results/clientnegative/invalid_arithmetic_type.q.out afd09ca
> ql/src/test/results/clientpositive/interval_1.q.out PRE-CREATION
> ql/src/test/results/clientpositive/interval_2.q.out PRE-CREATION
> ql/src/test/results/clientpositive/interval_3.q.out PRE-CREATION
> ql/src/test/results/clientpositive/interval_arithmetic.q.out PRE-CREATION
> ql/src/test/results/clientpositive/interval_comparison.q.out PRE-CREATION
> serde/if/serde.thrift 2db27b5
> serde/src/gen/thrift/gen-cpp/serde_constants.h 418f666
> serde/src/gen/thrift/gen-cpp/serde_constants.cpp cc71b65
> serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java e70d0c4
> serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php c1d8085
> serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py 7d7608f
> serde/src/gen/thrift/gen-rb/serde_constants.rb 33f12b0
> serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 2b7fba6
> serde/src/java/org/apache/hadoop/hive/serde2/io/HiveIntervalDayTimeWritable.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/io/HiveIntervalYearMonthWritable.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 864d9aa
> serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyHiveIntervalDayTime.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyHiveIntervalYearMonth.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 879743f
> serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyHiveIntervalDayTimeObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyHiveIntervalYearMonthObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 7423c00
> serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 8bd5838
> serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryHiveIntervalDayTime.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryHiveIntervalYearMonth.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 62cba01
> serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 11e8cf4
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 2a47d97
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 5e67a07
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 05aed0a
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/HiveIntervalDayTimeObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/HiveIntervalYearMonthObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaHiveIntervalDayTimeObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaHiveIntervalYearMonthObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java cb996a8
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java f466297
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 5ccacf1
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableHiveIntervalDayTimeObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableHiveIntervalYearMonthObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantHiveIntervalDayTimeObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantHiveIntervalYearMonthObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableHiveIntervalDayTimeObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableHiveIntervalYearMonthObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java e5c9f18
> serde/src/test/org/apache/hadoop/hive/serde2/io/TestHiveIntervalDayTimeWritable.java PRE-CREATION
> serde/src/test/org/apache/hadoop/hive/serde2/io/TestHiveIntervalYearMonthWritable.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/31696/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jason Dere
>
>
Re: Review Request 31696: HIVE-9792 Support interval type in
expressions/predicates
Posted by Ashutosh Chauhan <ha...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31696/#review76772
-----------------------------------------------------------
Ship it!
Ship It!
- Ashutosh Chauhan
On March 17, 2015, 6:37 p.m., Jason Dere wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31696/
> -----------------------------------------------------------
>
> (Updated March 17, 2015, 6:37 p.m.)
>
>
> Review request for hive and Ashutosh Chauhan.
>
>
> Bugs: HIVE-9792
> https://issues.apache.org/jira/browse/HIVE-9792
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> This adds 2 separate interval types:
> - year-month interval, which represents a date difference in years and months. Internally represented as an offset of months.
> - day-time interval, which represents a date difference in days/hours/minutes/seconds/nanoseconds. 1 day = 24 hours = 1440 minutes = 86400 seconds. Internally represented as an offset of seconds (long) and nanoseconds (int).
>
> I've tried using the Oracle's Datetime and Interval Arithmetic rules with regard to timestamp arithmetic, where the timestamp arithmetic is done in UTC time. This results in similar behavior to below depending on the default time zone in Hive:
>
> SQL> select timestamp '1999-09-15 00:00:00 America/Los_Angeles' - timestamp '1999-12-15 00:00:00 America/Los_Angeles' from dual;
>
> TIMESTAMP'1999-09-1500:00:00AMERICA/LOS_ANGELES'-TIMESTAMP'1999-12-1500:00:
> ---------------------------------------------------------------------------
> -000000091 01:00:00.000000000
>
> SQL> select timestamp '1999-09-15 00:00:00 UTC' - timestamp '1999-12-15 00:00:00 UTC' from dual;
>
> TIMESTAMP'1999-09-1500:00:00UTC'-TIMESTAMP'1999-12-1500:00:00UTC'
> ---------------------------------------------------------------------------
> -000000091 00:00:00.000000000
>
>
> Diffs
> -----
>
> common/src/java/org/apache/hadoop/hive/common/type/HiveIntervalDayTime.java PRE-CREATION
> common/src/java/org/apache/hadoop/hive/common/type/HiveIntervalYearMonth.java PRE-CREATION
> common/src/java/org/apache/hive/common/util/DateTimeMath.java PRE-CREATION
> common/src/java/org/apache/hive/common/util/DateUtils.java 92b7f62
> common/src/test/org/apache/hadoop/hive/common/type/TestHiveIntervalDayTime.java PRE-CREATION
> common/src/test/org/apache/hadoop/hive/common/type/TestHiveIntervalYearMonth.java PRE-CREATION
> common/src/test/org/apache/hive/common/util/TestDateTimeMath.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java fdbfab9
> ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinKey.java 6a3c300
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTBuilder.java 3a2c96c
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java 4dbac57
> ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java 88c989f
> ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g e7de6c8
> ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g d2d9989
> ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 0a05ceb
> ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java e8217fd
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseArithmetic.java 575c764
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseDTI.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseUnary.java c5bec44
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDTIMinus.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDTIPlus.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMinus.java 5419a19
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPNegative.java d653264
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPPlus.java accd347
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToIntervalDayTime.java PRE-CREATION
> ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToIntervalYearMonth.java PRE-CREATION
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFOPMinus.java 24618c9
> ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFOPPlus.java 4b2f7fb
> ql/src/test/queries/clientnegative/interval_1.q PRE-CREATION
> ql/src/test/queries/clientnegative/interval_2.q PRE-CREATION
> ql/src/test/queries/clientnegative/interval_3.q PRE-CREATION
> ql/src/test/queries/clientnegative/invalid_arithmetic_type.q ad37cff
> ql/src/test/queries/clientpositive/interval_1.q PRE-CREATION
> ql/src/test/queries/clientpositive/interval_2.q PRE-CREATION
> ql/src/test/queries/clientpositive/interval_3.q PRE-CREATION
> ql/src/test/queries/clientpositive/interval_arithmetic.q PRE-CREATION
> ql/src/test/queries/clientpositive/interval_comparison.q PRE-CREATION
> ql/src/test/results/clientnegative/interval_1.q.out PRE-CREATION
> ql/src/test/results/clientnegative/interval_2.q.out PRE-CREATION
> ql/src/test/results/clientnegative/interval_3.q.out PRE-CREATION
> ql/src/test/results/clientnegative/invalid_arithmetic_type.q.out afd09ca
> ql/src/test/results/clientpositive/interval_1.q.out PRE-CREATION
> ql/src/test/results/clientpositive/interval_2.q.out PRE-CREATION
> ql/src/test/results/clientpositive/interval_3.q.out PRE-CREATION
> ql/src/test/results/clientpositive/interval_arithmetic.q.out PRE-CREATION
> ql/src/test/results/clientpositive/interval_comparison.q.out PRE-CREATION
> serde/if/serde.thrift 2db27b5
> serde/src/gen/thrift/gen-cpp/serde_constants.h 418f666
> serde/src/gen/thrift/gen-cpp/serde_constants.cpp cc71b65
> serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java e70d0c4
> serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php c1d8085
> serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py 7d7608f
> serde/src/gen/thrift/gen-rb/serde_constants.rb 33f12b0
> serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 2b7fba6
> serde/src/java/org/apache/hadoop/hive/serde2/io/HiveIntervalDayTimeWritable.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/io/HiveIntervalYearMonthWritable.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 864d9aa
> serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyHiveIntervalDayTime.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyHiveIntervalYearMonth.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 879743f
> serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyHiveIntervalDayTimeObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyHiveIntervalYearMonthObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 7423c00
> serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 8bd5838
> serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryHiveIntervalDayTime.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryHiveIntervalYearMonth.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 62cba01
> serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 11e8cf4
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 2a47d97
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 5e67a07
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 05aed0a
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/HiveIntervalDayTimeObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/HiveIntervalYearMonthObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaHiveIntervalDayTimeObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaHiveIntervalYearMonthObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java cb996a8
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java f466297
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 5ccacf1
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableHiveIntervalDayTimeObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableHiveIntervalYearMonthObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantHiveIntervalDayTimeObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantHiveIntervalYearMonthObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableHiveIntervalDayTimeObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableHiveIntervalYearMonthObjectInspector.java PRE-CREATION
> serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java e5c9f18
> serde/src/test/org/apache/hadoop/hive/serde2/io/TestHiveIntervalDayTimeWritable.java PRE-CREATION
> serde/src/test/org/apache/hadoop/hive/serde2/io/TestHiveIntervalYearMonthWritable.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/31696/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Jason Dere
>
>
Re: Review Request 31696: HIVE-9792 Support interval type in
expressions/predicates
Posted by Jason Dere <jd...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31696/
-----------------------------------------------------------
(Updated March 17, 2015, 6:37 p.m.)
Review request for hive and Ashutosh Chauhan.
Changes
-------
Incorporating RB feedback from Ashutosh
Bugs: HIVE-9792
https://issues.apache.org/jira/browse/HIVE-9792
Repository: hive-git
Description
-------
This adds 2 separate interval types:
- year-month interval, which represents a date difference in years and months. Internally represented as an offset of months.
- day-time interval, which represents a date difference in days/hours/minutes/seconds/nanoseconds. 1 day = 24 hours = 1440 minutes = 86400 seconds. Internally represented as an offset of seconds (long) and nanoseconds (int).
I've tried using the Oracle's Datetime and Interval Arithmetic rules with regard to timestamp arithmetic, where the timestamp arithmetic is done in UTC time. This results in similar behavior to below depending on the default time zone in Hive:
SQL> select timestamp '1999-09-15 00:00:00 America/Los_Angeles' - timestamp '1999-12-15 00:00:00 America/Los_Angeles' from dual;
TIMESTAMP'1999-09-1500:00:00AMERICA/LOS_ANGELES'-TIMESTAMP'1999-12-1500:00:
---------------------------------------------------------------------------
-000000091 01:00:00.000000000
SQL> select timestamp '1999-09-15 00:00:00 UTC' - timestamp '1999-12-15 00:00:00 UTC' from dual;
TIMESTAMP'1999-09-1500:00:00UTC'-TIMESTAMP'1999-12-1500:00:00UTC'
---------------------------------------------------------------------------
-000000091 00:00:00.000000000
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/common/type/HiveIntervalDayTime.java PRE-CREATION
common/src/java/org/apache/hadoop/hive/common/type/HiveIntervalYearMonth.java PRE-CREATION
common/src/java/org/apache/hive/common/util/DateTimeMath.java PRE-CREATION
common/src/java/org/apache/hive/common/util/DateUtils.java 92b7f62
common/src/test/org/apache/hadoop/hive/common/type/TestHiveIntervalDayTime.java PRE-CREATION
common/src/test/org/apache/hadoop/hive/common/type/TestHiveIntervalYearMonth.java PRE-CREATION
common/src/test/org/apache/hive/common/util/TestDateTimeMath.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java fdbfab9
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinKey.java 6a3c300
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTBuilder.java 3a2c96c
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java 4dbac57
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java 88c989f
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g e7de6c8
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g d2d9989
ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 0a05ceb
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java e8217fd
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseArithmetic.java 575c764
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseDTI.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseUnary.java c5bec44
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDTIMinus.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDTIPlus.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMinus.java 5419a19
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPNegative.java d653264
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPPlus.java accd347
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToIntervalDayTime.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToIntervalYearMonth.java PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFOPMinus.java 24618c9
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFOPPlus.java 4b2f7fb
ql/src/test/queries/clientnegative/interval_1.q PRE-CREATION
ql/src/test/queries/clientnegative/interval_2.q PRE-CREATION
ql/src/test/queries/clientnegative/interval_3.q PRE-CREATION
ql/src/test/queries/clientnegative/invalid_arithmetic_type.q ad37cff
ql/src/test/queries/clientpositive/interval_1.q PRE-CREATION
ql/src/test/queries/clientpositive/interval_2.q PRE-CREATION
ql/src/test/queries/clientpositive/interval_3.q PRE-CREATION
ql/src/test/queries/clientpositive/interval_arithmetic.q PRE-CREATION
ql/src/test/queries/clientpositive/interval_comparison.q PRE-CREATION
ql/src/test/results/clientnegative/interval_1.q.out PRE-CREATION
ql/src/test/results/clientnegative/interval_2.q.out PRE-CREATION
ql/src/test/results/clientnegative/interval_3.q.out PRE-CREATION
ql/src/test/results/clientnegative/invalid_arithmetic_type.q.out afd09ca
ql/src/test/results/clientpositive/interval_1.q.out PRE-CREATION
ql/src/test/results/clientpositive/interval_2.q.out PRE-CREATION
ql/src/test/results/clientpositive/interval_3.q.out PRE-CREATION
ql/src/test/results/clientpositive/interval_arithmetic.q.out PRE-CREATION
ql/src/test/results/clientpositive/interval_comparison.q.out PRE-CREATION
serde/if/serde.thrift 2db27b5
serde/src/gen/thrift/gen-cpp/serde_constants.h 418f666
serde/src/gen/thrift/gen-cpp/serde_constants.cpp cc71b65
serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java e70d0c4
serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php c1d8085
serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py 7d7608f
serde/src/gen/thrift/gen-rb/serde_constants.rb 33f12b0
serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 2b7fba6
serde/src/java/org/apache/hadoop/hive/serde2/io/HiveIntervalDayTimeWritable.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/io/HiveIntervalYearMonthWritable.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 864d9aa
serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyHiveIntervalDayTime.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyHiveIntervalYearMonth.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 879743f
serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyHiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyHiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 7423c00
serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 8bd5838
serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryHiveIntervalDayTime.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryHiveIntervalYearMonth.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 62cba01
serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 11e8cf4
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 2a47d97
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 5e67a07
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 05aed0a
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/HiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/HiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaHiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaHiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java cb996a8
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java f466297
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 5ccacf1
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableHiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableHiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantHiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantHiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableHiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableHiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java e5c9f18
serde/src/test/org/apache/hadoop/hive/serde2/io/TestHiveIntervalDayTimeWritable.java PRE-CREATION
serde/src/test/org/apache/hadoop/hive/serde2/io/TestHiveIntervalYearMonthWritable.java PRE-CREATION
Diff: https://reviews.apache.org/r/31696/diff/
Testing
-------
Thanks,
Jason Dere
Re: Review Request 31696: HIVE-9792 Support interval type in
expressions/predicates
Posted by Jason Dere <jd...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31696/
-----------------------------------------------------------
(Updated March 12, 2015, 7:50 p.m.)
Review request for hive and Ashutosh Chauhan.
Changes
-------
- Rebasing patch with trunk, due to the parser changes in HIVE-6617. This adds year/month/day/hour/minute/second to the list of nonReserved words, and interval to the list of reserved words.
- Switches DateTimeMath to use Calendar rather than Joda, was getting a NPE while trying to do use DateTimeMath for vectorized intervals (work separate from this Jira). I suspect something in MutableDateTime was not getting serialized properly during plan serialization.
Bugs: HIVE-9792
https://issues.apache.org/jira/browse/HIVE-9792
Repository: hive-git
Description
-------
This adds 2 separate interval types:
- year-month interval, which represents a date difference in years and months. Internally represented as an offset of months.
- day-time interval, which represents a date difference in days/hours/minutes/seconds/nanoseconds. 1 day = 24 hours = 1440 minutes = 86400 seconds. Internally represented as an offset of seconds (long) and nanoseconds (int).
I've tried using the Oracle's Datetime and Interval Arithmetic rules with regard to timestamp arithmetic, where the timestamp arithmetic is done in UTC time. This results in similar behavior to below depending on the default time zone in Hive:
SQL> select timestamp '1999-09-15 00:00:00 America/Los_Angeles' - timestamp '1999-12-15 00:00:00 America/Los_Angeles' from dual;
TIMESTAMP'1999-09-1500:00:00AMERICA/LOS_ANGELES'-TIMESTAMP'1999-12-1500:00:
---------------------------------------------------------------------------
-000000091 01:00:00.000000000
SQL> select timestamp '1999-09-15 00:00:00 UTC' - timestamp '1999-12-15 00:00:00 UTC' from dual;
TIMESTAMP'1999-09-1500:00:00UTC'-TIMESTAMP'1999-12-1500:00:00UTC'
---------------------------------------------------------------------------
-000000091 00:00:00.000000000
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/common/type/HiveIntervalDayTime.java PRE-CREATION
common/src/java/org/apache/hadoop/hive/common/type/HiveIntervalYearMonth.java PRE-CREATION
common/src/java/org/apache/hadoop/hive/common/type/MutableHiveIntervalDayTime.java PRE-CREATION
common/src/java/org/apache/hadoop/hive/common/type/MutableHiveIntervalYearMonth.java PRE-CREATION
common/src/java/org/apache/hive/common/util/DateTimeMath.java PRE-CREATION
common/src/java/org/apache/hive/common/util/DateUtils.java 92b7f62
common/src/test/org/apache/hadoop/hive/common/type/TestHiveIntervalDayTime.java PRE-CREATION
common/src/test/org/apache/hadoop/hive/common/type/TestHiveIntervalYearMonth.java PRE-CREATION
common/src/test/org/apache/hive/common/util/TestDateTimeMath.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 20811ab
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinKey.java 6a3c300
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTBuilder.java 3a2c96c
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java 4dbac57
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java 88c989f
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g e7de6c8
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g d2d9989
ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 0a05ceb
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java e8217fd
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseArithmetic.java 575c764
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseDTI.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseUnary.java c5bec44
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDTIMinus.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDTIPlus.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMinus.java 5419a19
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPNegative.java d653264
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPPlus.java accd347
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToIntervalDayTime.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToIntervalYearMonth.java PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFOPMinus.java 24618c9
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFOPPlus.java 4b2f7fb
ql/src/test/queries/clientnegative/interval_1.q PRE-CREATION
ql/src/test/queries/clientnegative/interval_2.q PRE-CREATION
ql/src/test/queries/clientnegative/interval_3.q PRE-CREATION
ql/src/test/queries/clientnegative/invalid_arithmetic_type.q ad37cff
ql/src/test/queries/clientpositive/interval_1.q PRE-CREATION
ql/src/test/queries/clientpositive/interval_2.q PRE-CREATION
ql/src/test/queries/clientpositive/interval_3.q PRE-CREATION
ql/src/test/queries/clientpositive/interval_arithmetic.q PRE-CREATION
ql/src/test/queries/clientpositive/interval_comparison.q PRE-CREATION
ql/src/test/results/clientnegative/interval_1.q.out PRE-CREATION
ql/src/test/results/clientnegative/interval_2.q.out PRE-CREATION
ql/src/test/results/clientnegative/interval_3.q.out PRE-CREATION
ql/src/test/results/clientnegative/invalid_arithmetic_type.q.out afd09ca
ql/src/test/results/clientpositive/interval_1.q.out PRE-CREATION
ql/src/test/results/clientpositive/interval_2.q.out PRE-CREATION
ql/src/test/results/clientpositive/interval_3.q.out PRE-CREATION
ql/src/test/results/clientpositive/interval_arithmetic.q.out PRE-CREATION
ql/src/test/results/clientpositive/interval_comparison.q.out PRE-CREATION
serde/if/serde.thrift 2db27b5
serde/src/gen/thrift/gen-cpp/serde_constants.h 418f666
serde/src/gen/thrift/gen-cpp/serde_constants.cpp cc71b65
serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java e70d0c4
serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php c1d8085
serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py 7d7608f
serde/src/gen/thrift/gen-rb/serde_constants.rb 33f12b0
serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 2b7fba6
serde/src/java/org/apache/hadoop/hive/serde2/io/HiveIntervalDayTimeWritable.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/io/HiveIntervalYearMonthWritable.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 864d9aa
serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyHiveIntervalDayTime.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyHiveIntervalYearMonth.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 879743f
serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyHiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyHiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 7423c00
serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 8bd5838
serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryHiveIntervalDayTime.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryHiveIntervalYearMonth.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 62cba01
serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 11e8cf4
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 2a47d97
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 5e67a07
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 05aed0a
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/HiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/HiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaHiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaHiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java cb996a8
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java f466297
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 5ccacf1
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableHiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableHiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantHiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantHiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableHiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableHiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java e5c9f18
serde/src/test/org/apache/hadoop/hive/serde2/io/TestHiveIntervalDayTimeWritable.java PRE-CREATION
serde/src/test/org/apache/hadoop/hive/serde2/io/TestHiveIntervalYearMonthWritable.java PRE-CREATION
Diff: https://reviews.apache.org/r/31696/diff/
Testing
-------
Thanks,
Jason Dere
Re: Review Request 31696: HIVE-9792 Support interval type in
expressions/predicates
Posted by Jason Dere <jd...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31696/
-----------------------------------------------------------
(Updated March 10, 2015, 6:43 p.m.)
Review request for hive and Ashutosh Chauhan.
Changes
-------
patch v4 - a bit of refactoring of the date/timestamp/interval math to allow more object reuse when adding months. This is to allow the vectorized code to use these routines as well.
Bugs: HIVE-9792
https://issues.apache.org/jira/browse/HIVE-9792
Repository: hive-git
Description
-------
This adds 2 separate interval types:
- year-month interval, which represents a date difference in years and months. Internally represented as an offset of months.
- day-time interval, which represents a date difference in days/hours/minutes/seconds/nanoseconds. 1 day = 24 hours = 1440 minutes = 86400 seconds. Internally represented as an offset of seconds (long) and nanoseconds (int).
I've tried using the Oracle's Datetime and Interval Arithmetic rules with regard to timestamp arithmetic, where the timestamp arithmetic is done in UTC time. This results in similar behavior to below depending on the default time zone in Hive:
SQL> select timestamp '1999-09-15 00:00:00 America/Los_Angeles' - timestamp '1999-12-15 00:00:00 America/Los_Angeles' from dual;
TIMESTAMP'1999-09-1500:00:00AMERICA/LOS_ANGELES'-TIMESTAMP'1999-12-1500:00:
---------------------------------------------------------------------------
-000000091 01:00:00.000000000
SQL> select timestamp '1999-09-15 00:00:00 UTC' - timestamp '1999-12-15 00:00:00 UTC' from dual;
TIMESTAMP'1999-09-1500:00:00UTC'-TIMESTAMP'1999-12-1500:00:00UTC'
---------------------------------------------------------------------------
-000000091 00:00:00.000000000
Diffs (updated)
-----
common/src/java/org/apache/hadoop/hive/common/type/HiveIntervalDayTime.java PRE-CREATION
common/src/java/org/apache/hadoop/hive/common/type/HiveIntervalYearMonth.java PRE-CREATION
common/src/java/org/apache/hadoop/hive/common/type/MutableHiveIntervalDayTime.java PRE-CREATION
common/src/java/org/apache/hadoop/hive/common/type/MutableHiveIntervalYearMonth.java PRE-CREATION
common/src/java/org/apache/hive/common/util/DateTimeMath.java PRE-CREATION
common/src/java/org/apache/hive/common/util/DateUtils.java 92b7f62
common/src/test/org/apache/hadoop/hive/common/type/TestHiveIntervalDayTime.java PRE-CREATION
common/src/test/org/apache/hadoop/hive/common/type/TestHiveIntervalYearMonth.java PRE-CREATION
common/src/test/org/apache/hive/common/util/TestDateTimeMath.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 20811ab
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinKey.java 6a3c300
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/ASTBuilder.java 3a2c96c
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java 4dbac57
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java 88c989f
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 90b84ac
ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 2105e19
ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g cabf971
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java e8217fd
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseArithmetic.java 575c764
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseDTI.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBaseUnary.java c5bec44
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDTIMinus.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPDTIPlus.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPMinus.java 5419a19
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPNegative.java d653264
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFOPPlus.java accd347
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToIntervalDayTime.java PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToIntervalYearMonth.java PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFOPMinus.java 24618c9
ql/src/test/org/apache/hadoop/hive/ql/udf/generic/TestGenericUDFOPPlus.java 4b2f7fb
ql/src/test/queries/clientnegative/interval_1.q PRE-CREATION
ql/src/test/queries/clientnegative/interval_2.q PRE-CREATION
ql/src/test/queries/clientnegative/interval_3.q PRE-CREATION
ql/src/test/queries/clientnegative/invalid_arithmetic_type.q ad37cff
ql/src/test/queries/clientpositive/interval_1.q PRE-CREATION
ql/src/test/queries/clientpositive/interval_2.q PRE-CREATION
ql/src/test/queries/clientpositive/interval_3.q PRE-CREATION
ql/src/test/queries/clientpositive/interval_arithmetic.q PRE-CREATION
ql/src/test/queries/clientpositive/interval_comparison.q PRE-CREATION
ql/src/test/results/clientnegative/interval_1.q.out PRE-CREATION
ql/src/test/results/clientnegative/interval_2.q.out PRE-CREATION
ql/src/test/results/clientnegative/interval_3.q.out PRE-CREATION
ql/src/test/results/clientnegative/invalid_arithmetic_type.q.out afd09ca
ql/src/test/results/clientpositive/interval_1.q.out PRE-CREATION
ql/src/test/results/clientpositive/interval_2.q.out PRE-CREATION
ql/src/test/results/clientpositive/interval_3.q.out PRE-CREATION
ql/src/test/results/clientpositive/interval_arithmetic.q.out PRE-CREATION
ql/src/test/results/clientpositive/interval_comparison.q.out PRE-CREATION
serde/if/serde.thrift 2db27b5
serde/src/gen/thrift/gen-cpp/serde_constants.h 418f666
serde/src/gen/thrift/gen-cpp/serde_constants.cpp cc71b65
serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java e70d0c4
serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php c1d8085
serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py 7d7608f
serde/src/gen/thrift/gen-rb/serde_constants.rb 33f12b0
serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java 2b7fba6
serde/src/java/org/apache/hadoop/hive/serde2/io/HiveIntervalDayTimeWritable.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/io/HiveIntervalYearMonthWritable.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyFactory.java 864d9aa
serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyHiveIntervalDayTime.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyHiveIntervalYearMonth.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 879743f
serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyHiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyHiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java 7423c00
serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryFactory.java 8bd5838
serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryHiveIntervalDayTime.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryHiveIntervalYearMonth.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java 62cba01
serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java 11e8cf4
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorConverters.java 2a47d97
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java 5e67a07
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/PrimitiveObjectInspector.java 05aed0a
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/HiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/HiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaHiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaHiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorConverter.java cb996a8
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java f466297
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java 5ccacf1
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableHiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/SettableHiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantHiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableConstantHiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableHiveIntervalDayTimeObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableHiveIntervalYearMonthObjectInspector.java PRE-CREATION
serde/src/java/org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.java e5c9f18
serde/src/test/org/apache/hadoop/hive/serde2/io/TestHiveIntervalDayTimeWritable.java PRE-CREATION
serde/src/test/org/apache/hadoop/hive/serde2/io/TestHiveIntervalYearMonthWritable.java PRE-CREATION
Diff: https://reviews.apache.org/r/31696/diff/
Testing
-------
Thanks,
Jason Dere