You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Timothy Miller (Jira)" <ji...@apache.org> on 2023/03/15 18:04:00 UTC

[jira] [Updated] (SPARK-42776) BroadcastHashJoinExec.requiredChildDistribution called before columnar replacement rules

     [ https://issues.apache.org/jira/browse/SPARK-42776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Timothy Miller updated SPARK-42776:
-----------------------------------
    Description:     (was: I am trying to replace BroadcastHashJoinExec with a columnar equivalent. However, I noticed that BroadcastHashJoinExec.requiredChildDistribution gets called BEFORE the columnar replacement rules. As a result, the object that gets broadcast is the plain old hashmap created from row data. By the time the columnar replacement rules are applied, it's too late to get Spark to broadcast any other kind of object.)

> BroadcastHashJoinExec.requiredChildDistribution called before columnar replacement rules
> ----------------------------------------------------------------------------------------
>
>                 Key: SPARK-42776
>                 URL: https://issues.apache.org/jira/browse/SPARK-42776
>             Project: Spark
>          Issue Type: Bug
>          Components: Optimizer
>    Affects Versions: 3.3.1
>         Environment: I'm prototyping on a Mac, but that's not really relevant.
>            Reporter: Timothy Miller
>            Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org