You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Liang-Chi Hsieh (JIRA)" <ji...@apache.org> on 2016/03/16 06:04:33 UTC
[jira] [Updated] (SPARK-13903) Modify output nullability with
constraints for Join and Filter operators
[ https://issues.apache.org/jira/browse/SPARK-13903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Liang-Chi Hsieh updated SPARK-13903:
------------------------------------
Description:
With constraints and optimization, we can make sure some outputs of a Join (or Filter) operator are not nulls. We should modify output nullability accordingly. We can use this information in later execution to avoid null checking.
Another reason to modify plan output is that we will use the output to determine schema. We should keep correct nullability in the schema.
was:With constraints and optimization, we can make sure some outputs of a Join operator are not nulls. We should modify output nullability accordingly. We can use this information in later execution to avoid null checking.
> Modify output nullability with constraints for Join and Filter operators
> ------------------------------------------------------------------------
>
> Key: SPARK-13903
> URL: https://issues.apache.org/jira/browse/SPARK-13903
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Liang-Chi Hsieh
>
> With constraints and optimization, we can make sure some outputs of a Join (or Filter) operator are not nulls. We should modify output nullability accordingly. We can use this information in later execution to avoid null checking.
> Another reason to modify plan output is that we will use the output to determine schema. We should keep correct nullability in the schema.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org