You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "itholic (via GitHub)" <gi...@apache.org> on 2024/04/16 05:18:57 UTC
Re: [PR] [SPARK-47858][SPARK-47852][PYTHON][SQL] Refactoring the structure for DataFrame error context [spark]
itholic commented on code in PR #46063:
URL: https://github.com/apache/spark/pull/46063#discussion_r1565663205
##########
sql/api/src/main/scala/org/apache/spark/sql/catalyst/trees/origin.scala:
##########
@@ -76,6 +76,12 @@ object CurrentOrigin {
value.get.copy(line = Some(line), startPosition = Some(start)))
}
+ def setPySparkErrorContext(pysparkFragment: String, pysparkCallSite: String): Unit = {
+ val tupleInfo = (pysparkFragment, pysparkCallSite)
+ value.set(
+ value.get.copy(pysparkErrorContext = Some(tupleInfo)))
+ }
+
Review Comment:
cc @HyukjinKwon @cloud-fan
Try reusing the existing `CurrentOrigin` here so we don't need to add separate ThreadLocal and also don't need to add helper functions for each existing methods.
Some test cases are still need to be addressed, but I believe the overall structure is ready to review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org