You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Xuefu Zhang <xz...@cloudera.com> on 2013/10/28 05:49:40 UTC
Review Request 14985: HIVE-5354: Decimal precision/scale support in ORC file
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14985/
-----------------------------------------------------------
Review request for hive and Brock Noland.
Bugs: HIVE-5354
https://issues.apache.org/jira/browse/HIVE-5354
Repository: hive-git
Description
-------
Support decimal precision/scale for Orc file, as part of HIVE-3976.
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcStruct.java c993b37
ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java 71484a3
ql/src/java/org/apache/hadoop/hive/ql/io/orc/WriterImpl.java 7519fc1
ql/src/protobuf/org/apache/hadoop/hive/ql/io/orc/orc_proto.proto 53b93a0
Diff: https://reviews.apache.org/r/14985/diff/
Testing
-------
Thanks,
Xuefu Zhang
Re: Review Request 14985: HIVE-5354: Decimal precision/scale support in ORC
file
Posted by Jason Dere <jd...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14985/#review27598
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcStruct.java
<https://reviews.apache.org/r/14985/#comment53625>
This will cause existing decimal data to not be readable, so probably not best to throw exception here. Or were you planning to change this in HIVE-5564?
Are there any existing orc/decimal tests? Otherwise might be good to test that the precision/scale is preserved correctly for ORC.
- Jason Dere
On Oct. 28, 2013, 4:49 a.m., Xuefu Zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14985/
> -----------------------------------------------------------
>
> (Updated Oct. 28, 2013, 4:49 a.m.)
>
>
> Review request for hive and Brock Noland.
>
>
> Bugs: HIVE-5354
> https://issues.apache.org/jira/browse/HIVE-5354
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Support decimal precision/scale for Orc file, as part of HIVE-3976.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcStruct.java c993b37
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java 71484a3
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/WriterImpl.java 7519fc1
> ql/src/protobuf/org/apache/hadoop/hive/ql/io/orc/orc_proto.proto 53b93a0
>
> Diff: https://reviews.apache.org/r/14985/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Xuefu Zhang
>
>
Re: Review Request 14985: HIVE-5354: Decimal precision/scale support in ORC
file
Posted by Xuefu Zhang <xz...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14985/
-----------------------------------------------------------
(Updated Oct. 28, 2013, 7:51 p.m.)
Review request for hive and Brock Noland.
Bugs: HIVE-5354
https://issues.apache.org/jira/browse/HIVE-5354
Repository: hive-git
Description
-------
Support decimal precision/scale for Orc file, as part of HIVE-3976.
Diffs (updated)
-----
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcStruct.java c993b37
ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java 71484a3
ql/src/java/org/apache/hadoop/hive/ql/io/orc/WriterImpl.java 7519fc1
ql/src/protobuf/org/apache/hadoop/hive/ql/io/orc/orc_proto.proto 53b93a0
Diff: https://reviews.apache.org/r/14985/diff/
Testing
-------
Thanks,
Xuefu Zhang
Re: Review Request 14985: HIVE-5354: Decimal precision/scale support in ORC
file
Posted by Jason Dere <jd...@hortonworks.com>.
> On Oct. 28, 2013, 7:38 p.m., Xuefu Zhang wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcStruct.java, line 548
> > <https://reviews.apache.org/r/14985/diff/1/?file=372080#file372080line548>
> >
> > This is a valid concern. However, there is no way to "preserve" previous precision/scale, as previously precision is always 38, and scale is variable while here we need to re-fit it to a precision and a fixed scale. I think a default one, (38, 18), may satisfy (most of cases). What do you think?
Sounds like a sensible fallback.
- Jason
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14985/#review27632
-----------------------------------------------------------
On Oct. 28, 2013, 7:51 p.m., Xuefu Zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14985/
> -----------------------------------------------------------
>
> (Updated Oct. 28, 2013, 7:51 p.m.)
>
>
> Review request for hive and Brock Noland.
>
>
> Bugs: HIVE-5354
> https://issues.apache.org/jira/browse/HIVE-5354
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Support decimal precision/scale for Orc file, as part of HIVE-3976.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcStruct.java c993b37
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java 71484a3
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/WriterImpl.java 7519fc1
> ql/src/protobuf/org/apache/hadoop/hive/ql/io/orc/orc_proto.proto 53b93a0
>
> Diff: https://reviews.apache.org/r/14985/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Xuefu Zhang
>
>
Re: Review Request 14985: HIVE-5354: Decimal precision/scale support in ORC
file
Posted by Xuefu Zhang <xz...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14985/#review27632
-----------------------------------------------------------
ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcStruct.java
<https://reviews.apache.org/r/14985/#comment53683>
This is a valid concern. However, there is no way to "preserve" previous precision/scale, as previously precision is always 38, and scale is variable while here we need to re-fit it to a precision and a fixed scale. I think a default one, (38, 18), may satisfy (most of cases). What do you think?
- Xuefu Zhang
On Oct. 28, 2013, 4:49 a.m., Xuefu Zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14985/
> -----------------------------------------------------------
>
> (Updated Oct. 28, 2013, 4:49 a.m.)
>
>
> Review request for hive and Brock Noland.
>
>
> Bugs: HIVE-5354
> https://issues.apache.org/jira/browse/HIVE-5354
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Support decimal precision/scale for Orc file, as part of HIVE-3976.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcStruct.java c993b37
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java 71484a3
> ql/src/java/org/apache/hadoop/hive/ql/io/orc/WriterImpl.java 7519fc1
> ql/src/protobuf/org/apache/hadoop/hive/ql/io/orc/orc_proto.proto 53b93a0
>
> Diff: https://reviews.apache.org/r/14985/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Xuefu Zhang
>
>