You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Ted Yu (Jira)" <ji...@apache.org> on 2020/12/26 04:01:00 UTC
[jira] [Updated] (SPARK-33915) Allow json expression to be pushable
column
[ https://issues.apache.org/jira/browse/SPARK-33915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated SPARK-33915:
---------------------------
Description:
Currently PushableColumnBase provides no support for json / jsonb expression.
Example of json expression:
{code}
get_json_object(phone, '$.code') = '1200'
{code}
If non-string literal is part of the expression, the presence of cast() would complicate the situation.
Implication is that implementation of SupportsPushDownFilters doesn't have a chance to perform pushdown even if third party DB engine supports json expression pushdown.
This issue is for discussion and implementation of Spark core changes which would allow json expression to be recognized as pushable column.
was:
Currently PushableColumnBase provides no support for json / jsonb expression.
Example of json expression:
get_json_object(phone, '$.code') = '1200'
If non-string literal is part of the expression, the presence of cast() would complicate the situation.
Implication is that implementation of SupportsPushDownFilters doesn't have a chance to perform pushdown even if third party DB engine supports json expression pushdown.
This issue is for discussion and implementation of Spark core changes which would allow json expression to be recognized as pushable column.
> Allow json expression to be pushable column
> -------------------------------------------
>
> Key: SPARK-33915
> URL: https://issues.apache.org/jira/browse/SPARK-33915
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 3.0.1
> Reporter: Ted Yu
> Priority: Major
>
> Currently PushableColumnBase provides no support for json / jsonb expression.
> Example of json expression:
> {code}
> get_json_object(phone, '$.code') = '1200'
> {code}
> If non-string literal is part of the expression, the presence of cast() would complicate the situation.
> Implication is that implementation of SupportsPushDownFilters doesn't have a chance to perform pushdown even if third party DB engine supports json expression pushdown.
> This issue is for discussion and implementation of Spark core changes which would allow json expression to be recognized as pushable column.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org