You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sujith (JIRA)" <ji...@apache.org> on 2019/02/03 11:43:00 UTC
[jira] [Commented] (SPARK-26821) filters not working with char
datatype when querying against hive table
[ https://issues.apache.org/jira/browse/SPARK-26821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16759381#comment-16759381 ]
Sujith commented on SPARK-26821:
--------------------------------
As per the initial analysis, this phenomenon is happening because the actual char data type length is 5 where as we are trying to insert a data with length 2, since its a char data type the system will pad the remaining part of the array block with 'space'. now when we try to apply a filter, the system will try to compare the predicate value with the actual table data which contains the space char like 'ds' == 'ds ' which leads to wrong result.
I am trying to analyze more on this issue please let me know for any suggestions or guidance. thanks
> filters not working with char datatype when querying against hive table
> -----------------------------------------------------------------------
>
> Key: SPARK-26821
> URL: https://issues.apache.org/jira/browse/SPARK-26821
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.4.0
> Reporter: Sujith
> Priority: Major
>
> creates a table with a char type field, While inserting data to char data type column, if the data string length is less than the specified datatype length, spark2x will not process filter query properly leading to incorrect result .
> 0: jdbc:hive2://10.19.89.222:22550/default> create table jj(id int, name char(5));
> +----------+-+
> |Result|
> +----------+-+
> +----------+-+
> No rows selected (0.894 seconds)
> 0: jdbc:hive2://10.19.89.222:22550/default> insert into table jj values(232,'ds');
> +----------+-+
> |Result|
> +----------+-+
> +----------+-+
> No rows selected (1.815 seconds)
> 0: jdbc:hive2://10.19.89.222:22550/default> select * from jj where name='ds';
> +------+------++--
> |id|name|
> +------+------++--
> +------+------++--
>
> The above query will not give any result.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org