You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/04/02 23:06:18 UTC

[GitHub] [spark] mbaldini opened a new pull request #32040: Add @ExpressionDescription to TimeWindow to generate missing documentation

mbaldini opened a new pull request #32040:
URL: https://github.com/apache/spark/pull/32040


   ### What changes were proposed in this pull request?
   Added missing `@ExpressionDescription` annotation to `TimeWindow` to generate a missing section of documentation for the sql `window` function.
   See: https://spark.apache.org/docs/latest/api/sql/index.html#window for current missing documentation.
   
   ### Why are the changes needed?
   The current documentation simply displays `N/A` for documentation for the sql `window` function.
   
   
   ### Does this PR introduce _any_ user-facing change?
   Yes. This adds documentation to the sql `window` function, which is currently missing.
   
   
   ### How was this patch tested?
   no tests added/modified. Manually verified by building docs and inspecting manually.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] mbaldini commented on pull request #32040: Add @ExpressionDescription to TimeWindow to generate missing documentation

Posted by GitBox <gi...@apache.org>.
mbaldini commented on pull request #32040:
URL: https://github.com/apache/spark/pull/32040#issuecomment-812761741


   Ah I see. Where would be the correct location? I'd be happy to update it.On Apr 2, 2021 4:56 PM, Takeshi Yamamuro ***@***.***> wrote:
   @maropu commented on this pull request.
   
   
   
   In sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/TimeWindow.scala:
   > +    Examples:
   +      > SELECT _FUNC_(timestamp('2020-04-25 15:49:11.914'), '5 minutes');
   +       2020-04-25 15:45:00, 2020-04-25 15:50:00
   +
   +      > SELECT _FUNC_(timestamp('2020-04-25 15:49:11.914'), '5 minutes', '2 minutes');
   +       2020-04-25 15:45:00, 2020-04-25 15:50:00
   +       2020-04-25 15:47:00, 2020-04-25 15:52:00
   +       2020-04-25 15:49:00, 2020-04-25 15:54:00
   +
   +      > SELECT _FUNC_(timestamp('2020-04-25 15:49:11.914'), '5 minutes', '2 minutes', '30 seconds');
   +       2020-04-25 15:44:30, 2020-04-25 15:49:30
   +       2020-04-25 15:46:30, 2020-04-25 15:51:30
   +       2020-04-25 15:48:30, 2020-04-25 15:53:30
   +  """,
   +  group = "datetime_funcs",
   +  since = "2.0.0")
   
   This is not a unevaluable SQL expr, so we don't put ExpressionDescription here.
   
   —You are receiving this because you authored the thread.Reply to this email directly, view it on GitHub, or unsubscribe.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] maropu commented on a change in pull request #32040: Add @ExpressionDescription to TimeWindow to generate missing documentation

Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #32040:
URL: https://github.com/apache/spark/pull/32040#discussion_r606513566



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/TimeWindow.scala
##########
@@ -27,6 +27,35 @@ import org.apache.spark.sql.catalyst.util.IntervalUtils
 import org.apache.spark.sql.types._
 import org.apache.spark.unsafe.types.UTF8String
 
+@ExpressionDescription(
+  usage = """
+    _FUNC_(time_column, window_duration) - Returns the TimeWindow which contains the given
+    timestamp, based on the window_duration provided.
+
+    _FUNC_(time_column, window_duration, slide_duration) - Returns the TimeWindow(s) which contain
+    the given timestamp, based on the window_duration and slide_duration provided.
+
+    _FUNC_(time_column, window_duration, slide_duration, start_time) - Returns the TimeWindow(s)
+    which contain the given timestamp, based on the window_duration, slide_duration, and start_time
+    provided.
+  """,
+  examples = """
+    Examples:
+      > SELECT _FUNC_(timestamp('2020-04-25 15:49:11.914'), '5 minutes');
+       2020-04-25 15:45:00, 2020-04-25 15:50:00
+
+      > SELECT _FUNC_(timestamp('2020-04-25 15:49:11.914'), '5 minutes', '2 minutes');
+       2020-04-25 15:45:00, 2020-04-25 15:50:00
+       2020-04-25 15:47:00, 2020-04-25 15:52:00
+       2020-04-25 15:49:00, 2020-04-25 15:54:00
+
+      > SELECT _FUNC_(timestamp('2020-04-25 15:49:11.914'), '5 minutes', '2 minutes', '30 seconds');
+       2020-04-25 15:44:30, 2020-04-25 15:49:30
+       2020-04-25 15:46:30, 2020-04-25 15:51:30
+       2020-04-25 15:48:30, 2020-04-25 15:53:30
+  """,
+  group = "datetime_funcs",
+  since = "2.0.0")

Review comment:
       This is a unevaluable non-SQL expr, so we don't put `ExpressionDescription` here.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] AmplabJenkins commented on pull request #32040: Add @ExpressionDescription to TimeWindow to generate missing documentation

Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32040:
URL: https://github.com/apache/spark/pull/32040#issuecomment-812748989


   Can one of the admins verify this patch?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] maropu commented on a change in pull request #32040: Add @ExpressionDescription to TimeWindow to generate missing documentation

Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #32040:
URL: https://github.com/apache/spark/pull/32040#discussion_r606513566



##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/TimeWindow.scala
##########
@@ -27,6 +27,35 @@ import org.apache.spark.sql.catalyst.util.IntervalUtils
 import org.apache.spark.sql.types._
 import org.apache.spark.unsafe.types.UTF8String
 
+@ExpressionDescription(
+  usage = """
+    _FUNC_(time_column, window_duration) - Returns the TimeWindow which contains the given
+    timestamp, based on the window_duration provided.
+
+    _FUNC_(time_column, window_duration, slide_duration) - Returns the TimeWindow(s) which contain
+    the given timestamp, based on the window_duration and slide_duration provided.
+
+    _FUNC_(time_column, window_duration, slide_duration, start_time) - Returns the TimeWindow(s)
+    which contain the given timestamp, based on the window_duration, slide_duration, and start_time
+    provided.
+  """,
+  examples = """
+    Examples:
+      > SELECT _FUNC_(timestamp('2020-04-25 15:49:11.914'), '5 minutes');
+       2020-04-25 15:45:00, 2020-04-25 15:50:00
+
+      > SELECT _FUNC_(timestamp('2020-04-25 15:49:11.914'), '5 minutes', '2 minutes');
+       2020-04-25 15:45:00, 2020-04-25 15:50:00
+       2020-04-25 15:47:00, 2020-04-25 15:52:00
+       2020-04-25 15:49:00, 2020-04-25 15:54:00
+
+      > SELECT _FUNC_(timestamp('2020-04-25 15:49:11.914'), '5 minutes', '2 minutes', '30 seconds');
+       2020-04-25 15:44:30, 2020-04-25 15:49:30
+       2020-04-25 15:46:30, 2020-04-25 15:51:30
+       2020-04-25 15:48:30, 2020-04-25 15:53:30
+  """,
+  group = "datetime_funcs",
+  since = "2.0.0")

Review comment:
       This is not a unevaluable SQL expr, so we don't put `ExpressionDescription` here.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] mbaldini closed pull request #32040: Add @ExpressionDescription to TimeWindow to generate missing documentation

Posted by GitBox <gi...@apache.org>.
mbaldini closed pull request #32040:
URL: https://github.com/apache/spark/pull/32040


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] [spark] maropu commented on pull request #32040: Add @ExpressionDescription to TimeWindow to generate missing documentation

Posted by GitBox <gi...@apache.org>.
maropu commented on pull request #32040:
URL: https://github.com/apache/spark/pull/32040#issuecomment-812766403


   Probably, we can add it somewhere in the `SQL, DataFrames, and Datasets` doc: https://spark.apache.org/docs/3.1.1/sql-getting-started.html just like the doc of structured streaming: https://spark.apache.org/docs/3.1.1/sql-getting-started.html#aggregate-functions


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org