You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "jubins (via GitHub)" <gi...@apache.org> on 2026/06/28 15:47:32 UTC
[PR] [SPARK-57740][PYTHON] Fix unix_micros truncation disclaimer and add unix_nanos cross-references to unix_* See Also sections [spark]
jubins opened a new pull request, #56851:
URL: https://github.com/apache/spark/pull/56851
### What is the purpose of the change
Fixes SPARK-57740 — two related doc-correctness issues in the `unix_*` epoch-unit function family in `python/pyspark/sql/functions/builtin.py`:
1. **Missing truncation disclaimer in `unix_micros`.** `unix_millis` and `unix_seconds` both include `"Truncates higher levels of precision."` in their one-line description. `unix_micros` did not, despite silently dropping sub-microsecond digits when given a nanosecond-precision input (`TIMESTAMP_LTZ(p)` / `TIMESTAMP_NTZ(p)`, `p` in `[7, 9]`). This became a practical concern once `unix_nanos` (SPARK-57579) made nanosecond-precision timestamps usable in PySpark.
2. **Missing `unix_nanos` cross-references.** `unix_nanos` (SPARK-57579) correctly links to all three siblings in its `See Also` section. None of the three sibling functions were updated to link back to `unix_nanos`, breaking discoverability of the function family from the existing functions.
### Brief change log
- `python/pyspark/sql/functions/builtin.py`:
- `unix_micros`: added `"Truncates higher levels of precision."` to the one-line description
- `unix_micros`, `unix_millis`, `unix_seconds`: added `:meth:\`pyspark.sql.functions.unix_nanos\`` to each `See Also` section
### Verifying this change
These are documentation-only changes with no logic impact. Verified by inspection that:
- All four `unix_*` functions now consistently include a truncation disclaimer in their description
- All four `unix_*` functions now cross-reference each other in their `See Also` sections symmetrically
### Does this pull request potentially affect one of the following parts
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with `@Public`/`@Evolving`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: no
- The S3 file system connector: no
### Documentation
Does this pull request introduce a new feature? No — documentation fix only.
### Was generative AI tooling used to co-author this PR?
- [x] Yes — Claude Code was used as a pair-programming assistant. All code was written, understood, and verified by the author.
Generated-by: Claude Opus 4.8
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57740][PYTHON] Fix unix_micros truncation disclaimer and add unix_nanos cross-references to unix_* See Also sections [spark]
Posted by "MaxGekk (via GitHub)" <gi...@apache.org>.
MaxGekk commented on PR #56851:
URL: https://github.com/apache/spark/pull/56851#issuecomment-4826627776
@jubins please, make PR's title concise
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57740][PYTHON] Fix unix_micros truncation disclaimer and add unix_nanos cross-references [spark]
Posted by "MaxGekk (via GitHub)" <gi...@apache.org>.
MaxGekk closed pull request #56851: [SPARK-57740][PYTHON] Fix unix_micros truncation disclaimer and add unix_nanos cross-references
URL: https://github.com/apache/spark/pull/56851
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-57740][PYTHON] Fix unix_micros truncation disclaimer and add unix_nanos cross-references [spark]
Posted by "MaxGekk (via GitHub)" <gi...@apache.org>.
MaxGekk commented on PR #56851:
URL: https://github.com/apache/spark/pull/56851#issuecomment-4831284997
+1, LGTM. Merging to master/4.x.
Thank you, @jubins.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org