[GitHub] [spark] RussellSpitzer commented on pull request #30167: [SPARK-33240][SQL][3.0] Fail fast when fails to instantiate configured v2 session catalog

[GitBox <gi...@apache.org>]

RussellSpitzer commented on pull request #30167:
URL: https://github.com/apache/spark/pull/30167#issuecomment-717887777


   I would consider the old behavior a "swallowed exception." There is almost no chance that a user intended the original catalog to be used, has no way of catching the error and retrying and cannot get any meaningful information from the failure if it occurs. 
   
   Like I mentioned on the Mailing list I think failing fast is the right thing to do, we could imagine this as a correctness issue. To give a silly example, imagine the replacement catalog redefined "INSERT" to not just write data to a file, but also simultaneously send some updates to a service which records data in a separate system. If the user has done a catalog replacement, but this behavior does not actually occur on insert, then their system will be inconsistent. If they were previously running this job before without error, they actually did have the error but it was ignorable so they just didn't know about it.
   
   At the very minimum we should dump the whole trace so that end users can at least see why something broke but I do think it's dangerous to go on with business as usual after we fail to replace the catalog.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org