You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "zengxl (JIRA)" <ji...@apache.org> on 2018/12/25 06:30:00 UTC
[jira] [Updated] (SPARK-26437) Decimal data becomes bigint to
query, unable to query
[ https://issues.apache.org/jira/browse/SPARK-26437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zengxl updated SPARK-26437:
---------------------------
Description:
this is my sql:
create table tmp.tmp_test_6387_1224_spark stored as ORCFile as select 0.00 as a
select a from tmp.tmp_test_6387_1224_spark
CREATE TABLE `tmp.tmp_test_6387_1224_spark`(
{color:#f79232} `a` decimal(2,2)){color}
ROW FORMAT SERDE
'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
STORED AS INPUTFORMAT
'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
When I query this table（use hive or sparksql，the exception is same）, I throw the following exception information
*Caused by: java.io.EOFException: Reading BigInteger past EOF from compressed stream Stream for column 1 kind DATA position: 0 length: 0 range: 0 offset: 0 limit: 0*
*at org.apache.hadoop.hive.ql.io.orc.SerializationUtils.readBigInteger(SerializationUtils.java:176)*
*at org.apache.hadoop.hive.ql.io.orc.TreeReaderFactory$DecimalTreeReader.next(TreeReaderFactory.java:1264)*
*at org.apache.hadoop.hive.ql.io.orc.TreeReaderFactory$StructTreeReader.next(TreeReaderFactory.java:2004)*
*at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.next(RecordReaderImpl.java:1039)*
was:
this is my sql:
create table tmp.tmp_test_6387_1224_spark stored as ORCFile as select 0.00 as a
select a from tmp.tmp_test_6387_1224_spark
When I query this table（use hive or sparksql，the exception is same）, I throw the following exception information
*Caused by: java.io.EOFException: Reading BigInteger past EOF from compressed stream Stream for column 1 kind DATA position: 0 length: 0 range: 0 offset: 0 limit: 0*
*at org.apache.hadoop.hive.ql.io.orc.SerializationUtils.readBigInteger(SerializationUtils.java:176)*
*at org.apache.hadoop.hive.ql.io.orc.TreeReaderFactory$DecimalTreeReader.next(TreeReaderFactory.java:1264)*
*at org.apache.hadoop.hive.ql.io.orc.TreeReaderFactory$StructTreeReader.next(TreeReaderFactory.java:2004)*
*at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.next(RecordReaderImpl.java:1039)*
> Decimal data becomes bigint to query, unable to query
> -----------------------------------------------------
>
> Key: SPARK-26437
> URL: https://issues.apache.org/jira/browse/SPARK-26437
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.3.1
> Reporter: zengxl
> Priority: Major
>
> this is my sql:
> create table tmp.tmp_test_6387_1224_spark stored as ORCFile as select 0.00 as a
> select a from tmp.tmp_test_6387_1224_spark
> CREATE TABLE `tmp.tmp_test_6387_1224_spark`(
> {color:#f79232} `a` decimal(2,2)){color}
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> When I query this table（use hive or sparksql，the exception is same）, I throw the following exception information
> *Caused by: java.io.EOFException: Reading BigInteger past EOF from compressed stream Stream for column 1 kind DATA position: 0 length: 0 range: 0 offset: 0 limit: 0*
> *at org.apache.hadoop.hive.ql.io.orc.SerializationUtils.readBigInteger(SerializationUtils.java:176)*
> *at org.apache.hadoop.hive.ql.io.orc.TreeReaderFactory$DecimalTreeReader.next(TreeReaderFactory.java:1264)*
> *at org.apache.hadoop.hive.ql.io.orc.TreeReaderFactory$StructTreeReader.next(TreeReaderFactory.java:2004)*
> *at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.next(RecordReaderImpl.java:1039)*
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org