[jira] [Created] (SPARK-26075) Cannot broadcast the table that is larger than 8GB : Spark 2.3

["Neeraj Bhadani (JIRA)" <ji...@apache.org>]

Neeraj Bhadani created SPARK-26075:
--------------------------------------

             Summary: Cannot broadcast the table that is larger than 8GB : Spark 2.3
                 Key: SPARK-26075
                 URL: https://issues.apache.org/jira/browse/SPARK-26075
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 2.3.1
            Reporter: Neeraj Bhadani


 I am trying to use the broadcast join but getting below error in Spark 2.3. However, the same code is working fine in Spark 2.2
 
Upon checking the size of the dataframes its merely 50 MB and I have set the threshold to 200 MB as well. As I mentioned above same code is working fine in Spark 2.2

 
{{Error: "Cannot broadcast the table that is larger than 8GB". }}
However, Disabling the broadcasting is working fine.
{{'spark.sql.autoBroadcastJoinThreshold': '-1'}}
 
{{Regards,}}
{{Neeraj}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org