You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Zhaoyang Qin (Jira)" <ji...@apache.org> on 2019/10/29 09:43:00 UTC
[jira] [Comment Edited] (SPARK-29634) spark-sql can't query hive
table values with schema Char by equality.
[ https://issues.apache.org/jira/browse/SPARK-29634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16961828#comment-16961828 ]
Zhaoyang Qin edited comment on SPARK-29634 at 10/29/19 9:42 AM:
----------------------------------------------------------------
About Char datatype,Hive Docs sees:
[https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types#LanguageManualTypes-CharcharChar]
But spark-sql CLI read this table by
*select * from foo where bar = 'something'* ，
There will be no results.You must add Spaces like this:
*select * from foo where bar = 'something '.*
This causes normal SQL fail to execute correctly.
And many TPCDS statements get no results，eg: sql7,sql71,sql 72 etc.
was (Author: kelvin.fe):
About Char datatype,Hive Docs sees:
[https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types#LanguageManualTypes-CharcharChar]
But spark-sql CLI read this table by *select * from foo where bar = 'something'* ，There will be no results.You must add Spaces like this:
*select * from foo where bar = 'something '.*
This causes normal SQL fail to execute correctly.
And many TPCDS statements to get no results.eg: sql7,sql71,sql 72 etc.
> spark-sql can't query hive table values with schema Char by equality.
> ---------------------------------------------------------------------
>
> Key: SPARK-29634
> URL: https://issues.apache.org/jira/browse/SPARK-29634
> Project: Spark
> Issue Type: Bug
> Components: Spark Shell, SQL
> Affects Versions: 2.2.1, 2.3.1
> Environment: Spark 2.3.1
> Hive 3.0.0
> TPCDS Data & Tables
> Reporter: Zhaoyang Qin
> Priority: Major
> Labels: spark-sql, spark-sql-perf
>
> spark-sql can't query hive table values that with schema Char by equality.
> When i use spark-sql CLI to execute a query with hive tables,The expected results can not be obtained. The query result is empty. Some equality conditions did not work as expected.I checked and found that the table fields had one thing in common: they were created as char，sometimes as varchar.Then I execute the following statement and return an empty result: select * from foo where bar = 'something'.In fact, the data does exist. Using hive sql returns the correct results.
> Simulation steps:（use spark-sql）
> {code:java}
> //spark-sql
> {code}
> ./spark-sql
> > create table test1 (name char(10), age int);
> > insert into test1 values('erya',15);
> > select * from test1 where name = 'erya'; // no results
> > select * from test1 where name = 'erya '; // add 6 spaces
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org