You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Jubin Soni (Jira)" <ji...@apache.org> on 2026/06/28 15:43:00 UTC
[jira] [Updated] (SPARK-57740) unix_micros, unix_millis, and unix_seconds See Also sections are missing a cross-reference to unix_nanos
[ https://issues.apache.org/jira/browse/SPARK-57740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jubin Soni updated SPARK-57740:
-------------------------------
Description:
*What is the issue?*
The four {{unix_*}} epoch-unit functions form a family. {{unix_nanos}} (added in SPARK-57579) correctly lists all three siblings in its {{See Also}} section. However, none of the three siblings were updated to link back to {{{}unix_nanos{}}}, breaking discoverability of the family from the existing functions.
Current {{See Also}} sections:
||Function||Links to||
|{{unix_micros}}|{{{}unix_date{}}}, {{{}unix_seconds{}}}, {{{}unix_millis{}}}, {{timestamp_micros}} — *missing {{unix_nanos}}*|
|{{unix_millis}}|{{{}unix_date{}}}, {{{}unix_seconds{}}}, {{{}unix_micros{}}}, {{timestamp_millis}} — *missing {{unix_nanos}}*|
|{{unix_seconds}}|{{{}unix_date{}}}, {{{}unix_millis{}}}, {{{}unix_micros{}}}, {{{}from_unixtime{}}}, {{timestamp_seconds}} — *missing {{unix_nanos}}*|
|{{unix_nanos}}|{{{}unix_date{}}}, {{{}unix_seconds{}}}, {{{}unix_millis{}}}, {{unix_micros}} — complete|
----
*How to reproduce*
Read the {{See Also}} section of {{{}pyspark.sql.functions.unix_micros{}}}, {{{}unix_millis{}}}, or {{unix_seconds}} in the PySpark docs or via {{{}help(){}}}. None of them mention {{{}unix_nanos{}}}.
----
*Actual behavior*
{{help(pyspark.sql.functions.unix_micros)
# See Also section lists: unix_date, unix_seconds, unix_millis, timestamp_micros
# unix_nanos is absent}}
----
*Expected behavior*
{{help(pyspark.sql.functions.unix_micros)
# See Also section lists: unix_date, unix_seconds, unix_millis, timestamp_micros, unix_nanos}}
Same for {{unix_millis}} and {{{}unix_seconds{}}}.
----
*Proposed fix*
Add {{{}:meth:\{}}}pyspark.sql.functions.unix_nanos`{{{}to the{}}}See Also{{{}sections of{}}}unix_micros{{{}, {}}}unix_millis{{{}, and {}}}unix_seconds{{{}in{}}}python/pyspark/sql/functions/builtin.py`. Three lines total.
was:
When {{unix_nanos}} was added to PySpark (SPARK-57579), its own {{See Also}} section correctly links to all three sibling functions ({{{}unix_micros{}}}, {{{}unix_millis{}}}, {{{}unix_seconds{}}}). However, none of the three sibling functions were updated to link back to {{{}unix_nanos{}}}.
A user reading the {{{}unix_micros{}}}, {{{}unix_millis{}}}, or {{unix_seconds}} docstring will not discover the nanosecond-precision variant.
*Fix:* Add {{{}:meth:\{}}}pyspark.sql.functions.unix_nanos`{{{}to the{}}}See Also{{{}sections of{}}}unix_micros{{{}, {}}}unix_millis{{{}, and {}}}unix_seconds{{{}in{}}}python/pyspark/sql/functions/builtin.py`.
> unix_micros, unix_millis, and unix_seconds See Also sections are missing a cross-reference to unix_nanos
> --------------------------------------------------------------------------------------------------------
>
> Key: SPARK-57740
> URL: https://issues.apache.org/jira/browse/SPARK-57740
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 4.3.0
> Reporter: Jubin Soni
> Priority: Minor
>
> *What is the issue?*
> The four {{unix_*}} epoch-unit functions form a family. {{unix_nanos}} (added in SPARK-57579) correctly lists all three siblings in its {{See Also}} section. However, none of the three siblings were updated to link back to {{{}unix_nanos{}}}, breaking discoverability of the family from the existing functions.
> Current {{See Also}} sections:
> ||Function||Links to||
> |{{unix_micros}}|{{{}unix_date{}}}, {{{}unix_seconds{}}}, {{{}unix_millis{}}}, {{timestamp_micros}} — *missing {{unix_nanos}}*|
> |{{unix_millis}}|{{{}unix_date{}}}, {{{}unix_seconds{}}}, {{{}unix_micros{}}}, {{timestamp_millis}} — *missing {{unix_nanos}}*|
> |{{unix_seconds}}|{{{}unix_date{}}}, {{{}unix_millis{}}}, {{{}unix_micros{}}}, {{{}from_unixtime{}}}, {{timestamp_seconds}} — *missing {{unix_nanos}}*|
> |{{unix_nanos}}|{{{}unix_date{}}}, {{{}unix_seconds{}}}, {{{}unix_millis{}}}, {{unix_micros}} — complete|
> ----
> *How to reproduce*
> Read the {{See Also}} section of {{{}pyspark.sql.functions.unix_micros{}}}, {{{}unix_millis{}}}, or {{unix_seconds}} in the PySpark docs or via {{{}help(){}}}. None of them mention {{{}unix_nanos{}}}.
> ----
> *Actual behavior*
> {{help(pyspark.sql.functions.unix_micros)
> # See Also section lists: unix_date, unix_seconds, unix_millis, timestamp_micros
> # unix_nanos is absent}}
> ----
> *Expected behavior*
> {{help(pyspark.sql.functions.unix_micros)
> # See Also section lists: unix_date, unix_seconds, unix_millis, timestamp_micros, unix_nanos}}
> Same for {{unix_millis}} and {{{}unix_seconds{}}}.
> ----
> *Proposed fix*
> Add {{{}:meth:\{}}}pyspark.sql.functions.unix_nanos`{{{}to the{}}}See Also{{{}sections of{}}}unix_micros{{{}, {}}}unix_millis{{{}, and {}}}unix_seconds{{{}in{}}}python/pyspark/sql/functions/builtin.py`. Three lines total.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org