You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Viacheslav Tradunsky (Jira)" <ji...@apache.org> on 2020/05/13 07:56:00 UTC

[jira] [Created] (SPARK-31698) NPE on big dataset plans

Viacheslav Tradunsky created SPARK-31698:
--------------------------------------------

             Summary: NPE on big dataset plans
                 Key: SPARK-31698
                 URL: https://issues.apache.org/jira/browse/SPARK-31698
             Project: Spark
          Issue Type: Bug
          Components: Spark Core
    Affects Versions: 2.4.4
         Environment: AWS EMR
            Reporter: Viacheslav Tradunsky


We have big dataset containing 275 SQL operations more than 275 joins.

On the terminal operation to write data, it fails with NullPointerException.

 

I understand that such big number of operations might not be what spark is designed for, but NullPointerException is not an ideal way to fail in this case.

 

For more details, please see the stacktrace.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org