You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "stevomitric (via GitHub)" <gi...@apache.org> on 2026/08/12 12:40:05 UTC
[PR] [WIP][SPARK-58740][SQL][TESTS] Add to_xml test coverage for nanosecond-precision timestamp types [spark]
stevomitric opened a new pull request, #57963:
URL: https://github.com/apache/spark/pull/57963
### What changes were proposed in this pull request?
This PR adds test coverage for the `to_xml` write path with the nanosecond-precision timestamp types (`TIMESTAMP_NTZ(p)` / `TIMESTAMP_LTZ(p)`, `p in [7, 9]`). The write path exists in `StaxXmlGenerator` but had no dedicated tests, while `from_xml` (read) and both JSON directions were already covered.
Two tests are added to `XmlFunctionsSuite`:
- `to_xml with nanos timestamp types` — asserts the exact XML output for both NTZ and LTZ nanos types at every valid precision.
- `roundtrip in to_xml and from_xml - nanos timestamps` — verifies the write -> read round-trip is loss-free at every valid precision.
### Why are the changes needed?
The `to_xml` nanosecond write path was implemented but never exercised by a test, leaving a coverage gap for the XML datasource nanosecond support (SPARK-57458).
### Does this PR introduce _any_ user-facing change?
No. Test-only change.
### How was this patch tested?
`build/sbt "sql/testOnly org.apache.spark.sql.XmlFunctionsSuite"`.
### Was this patch authored or co-authored using generative AI tooling?
Co-authored-by: Claude Code 4.8
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58740][SQL][TESTS] Add to_xml test coverage for nanosecond-precision timestamp types [spark]
Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on code in PR #57963:
URL: https://github.com/apache/spark/pull/57963#discussion_r3768852284
##########
sql/core/src/test/scala/org/apache/spark/sql/XmlFunctionsSuite.scala:
##########
@@ -361,6 +361,69 @@ class XmlFunctionsSuite extends SharedSparkSession {
checkAnswer(dfTwo, readBackTwo)
}
+ test("SPARK-57458: to_xml with nanos timestamp types") {
+ withSQLConf(
+ SQLConf.TIMESTAMP_NANOS_TYPES_ENABLED.key -> "true",
+ SQLConf.SESSION_LOCAL_TIMEZONE.key -> "UTC") {
+ foreachNanosPrecision { p =>
+ // The pattern must carry `p` fractional digits to emit the full declared precision; the
+ // stored value has exactly `p` significant digits, so the rendered fraction is the first
Review Comment:
**Nit:**
`p` is fractional-second precision; `significant digits` is general numeric terminology and can be read as applying to the whole timestamp.
```suggestion
// stored value is truncated to precision `p`, so the rendered fraction is the first
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58740][SQL][TESTS] Add to_xml test coverage for nanosecond-precision timestamp types [spark]
Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b commented on PR #57963:
URL: https://github.com/apache/spark/pull/57963#issuecomment-5268474282
Thank you @stevomitric!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58740][SQL][TESTS] Add to_xml test coverage for nanosecond-precision timestamp types [spark]
Posted by "stevomitric (via GitHub)" <gi...@apache.org>.
stevomitric commented on PR #57963:
URL: https://github.com/apache/spark/pull/57963#issuecomment-5267662724
cc @uros-b, @cloud-fan PTAL when you get a chance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58740][SQL][TESTS] Add to_xml test coverage for nanosecond-precision timestamp types [spark]
Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b commented on PR #57963:
URL: https://github.com/apache/spark/pull/57963#issuecomment-5276491523
**Merge Summary:**
- merged into master https://github.com/apache/spark/commit/94cbfbaf818aa071eb989620aee4cf20a24133ad
- merged into branch-4.x https://github.com/apache/spark/commit/abab4ce86459be303e2a7d050b72604397727c33
- merged into branch-4.3 https://github.com/apache/spark/commit/f31412b77870aa7059732aee7dae5b978854240f
*Posted by `merge_spark_pr.py`*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-58740][SQL][TESTS] Add to_xml test coverage for nanosecond-precision timestamp types [spark]
Posted by "uros-b (via GitHub)" <gi...@apache.org>.
uros-b closed pull request #57963: [SPARK-58740][SQL][TESTS] Add to_xml test coverage for nanosecond-precision timestamp types
URL: https://github.com/apache/spark/pull/57963
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org