You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "parthchandra (via GitHub)" <gi...@apache.org> on 2026/03/18 21:56:33 UTC

Re: [PR] fix: enable native_datafusion Spark SQL tests for #3320, #3401, #3719 [datafusion-comet]

parthchandra commented on code in PR #3718:
URL: https://github.com/apache/datafusion-comet/pull/3718#discussion_r2956462818


##########
native/core/src/errors.rs:
##########
@@ -474,6 +476,54 @@ fn throw_spark_error_as_json(
     )
 }
 
+/// Try to convert a DataFusion "Unable to get field named" error into a SparkError.
+/// DataFusion produces this error when reading Parquet files with duplicate field names
+/// in case-insensitive mode. For example, if a Parquet file has columns "B" and "b",
+/// DataFusion may deduplicate them and report: Unable to get field named "b". Valid
+/// fields: ["A", "B"]. When the requested field has a case-insensitive match among the
+/// valid fields, we convert this to Spark's _LEGACY_ERROR_TEMP_2093 error.
+fn try_convert_duplicate_field_error(error_msg: &str) -> Option<SparkError> {

Review Comment:
   late comment: 
   You're right, this is overkill. We can, if the need arises (as in this case), not convert the `_LEGACY_ ` errors? Or even more broadly, not do this for the errors that originate in `QueryExecutionErrors`. 
   The error framework is important for the errors in `org.apache.spark.sql.errors.ExecutionErrors` because those are SQL errors and correspond to some pre-defined error codes in ANSI.  But for `QueryExecutionErrors` we do not have to be strict. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org