You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "shikui ye (Jira)" <ji...@apache.org> on 2021/01/15 08:40:00 UTC

[jira] [Created] (SPARK-34126) SQL running error, spark does not exit, resulting in data quality problems

shikui ye created SPARK-34126:
---------------------------------

             Summary: SQL running error, spark does not exit, resulting in data quality problems
                 Key: SPARK-34126
                 URL: https://issues.apache.org/jira/browse/SPARK-34126
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 3.0.1
         Environment: spark3.0.1 on yarn 
            Reporter: shikui ye


Spark SQL executes a SQL file containing multiple SQL segments. Because one of the SQL segments fails to run, but spark driver or spark context does not exit, an error will occur. The table written by the SQL segment is empty or old data. Depending on this problematic table, the subsequent SQL will have data quality problems even if it runs successfully.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org