You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "sandip-db (via GitHub)" <gi...@apache.org> on 2023/11/29 05:47:38 UTC
[PR] [SPARK-46152][SQL] XML: Add DecimalType support in XML schema inference [spark]
sandip-db opened a new pull request, #44069:
URL: https://github.com/apache/spark/pull/44069
### What changes were proposed in this pull request?
Add DecimalType support in XML schema inference
### Why are the changes needed?
Add DecimalType support in XML schema inference
### Does this PR introduce _any_ user-facing change?
Yes, XML schema inference will infer DecimalType correctly.
### How was this patch tested?
Added a new unit test
### Was this patch authored or co-authored using generative AI tooling?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46152][SQL] XML: Add DecimalType support in XML schema inference [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon closed pull request #44069: [SPARK-46152][SQL] XML: Add DecimalType support in XML schema inference
URL: https://github.com/apache/spark/pull/44069
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46152][SQL] XML: Add DecimalType support in XML schema inference [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #44069:
URL: https://github.com/apache/spark/pull/44069#issuecomment-1832998873
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46152][SQL] XML: Add DecimalType support in XML schema inference [spark]
Posted by "sandip-db (via GitHub)" <gi...@apache.org>.
sandip-db commented on code in PR #44069:
URL: https://github.com/apache/spark/pull/44069#discussion_r1410081476
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/xml/XmlInferSchema.scala:
##########
@@ -39,6 +40,8 @@ class XmlInferSchema(options: XmlOptions, caseSensitive: Boolean)
extends Serializable
with Logging {
+ private val decimalParser = ExprUtils.getDecimalParser(options.locale)
Review Comment:
This parser converts a decimal string to DecimalType. The one in XSDToSchema converts XSD type to DecimalType.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46152][SQL] XML: Add DecimalType support in XML schema inference [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #44069:
URL: https://github.com/apache/spark/pull/44069#discussion_r1410078368
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/xml/XmlInferSchema.scala:
##########
@@ -39,6 +40,8 @@ class XmlInferSchema(options: XmlOptions, caseSensitive: Boolean)
extends Serializable
with Logging {
+ private val decimalParser = ExprUtils.getDecimalParser(options.locale)
Review Comment:
If not, let's make a followup to match.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-46152][SQL] XML: Add DecimalType support in XML schema inference [spark]
Posted by "shujingyang-db (via GitHub)" <gi...@apache.org>.
shujingyang-db commented on code in PR #44069:
URL: https://github.com/apache/spark/pull/44069#discussion_r1409666292
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/xml/XmlInferSchema.scala:
##########
@@ -39,6 +40,8 @@ class XmlInferSchema(options: XmlOptions, caseSensitive: Boolean)
extends Serializable
with Logging {
+ private val decimalParser = ExprUtils.getDecimalParser(options.locale)
Review Comment:
a qq - Does this parser function the same as our [XSD](https://src.dev.databricks.com/databricks/runtime/-/blob/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/xml/XSDToSchema.scala?L89) conversion?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org