[jira] [Created] (SPARK-39412) IllegalStateException from connector does not work well with error class framework

["Jungtaek Lim (Jira)" <ji...@apache.org>]

Jungtaek Lim created SPARK-39412:
------------------------------------

             Summary: IllegalStateException from connector does not work well with error class framework
                 Key: SPARK-39412
                 URL: https://issues.apache.org/jira/browse/SPARK-39412
             Project: Spark
          Issue Type: Bug
          Components: Structured Streaming
    Affects Versions: 3.3.0
            Reporter: Jungtaek Lim


With SPARK-39346, Spark SQL binds several exceptions to the internal error, and produces different guidance on dealing with the exception. This assumes these exceptions are only used for noticing internal bugs.

This applies to "connectors" as well, and introduces side-effect on the error log. For Kafka data source, it is a breaking and unacceptable change, because there is an important case Kafka data source determines a case of "dataloss", and throws IllegalStateException with instruction message on workaround.

Two problems arise:

1) This does not mean Spark has a bug and end users have to report, hence the guidance message on internal error is misleading.

2) Most importantly, instruction message is shown after a long stack trace. With the modification of existing test suite, I see the message being appeared in "line 90" of the error log.

We should roll the right error message back, at least for Kafka's case.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org