You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (Jira)" <ji...@apache.org> on 2019/09/27 12:32:00 UTC
[jira] [Assigned] (SPARK-21914) Running examples as tests in SQL
builtin function documentation
[ https://issues.apache.org/jira/browse/SPARK-21914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon reassigned SPARK-21914:
------------------------------------
Assignee: Maxim Gekk
> Running examples as tests in SQL builtin function documentation
> ---------------------------------------------------------------
>
> Key: SPARK-21914
> URL: https://issues.apache.org/jira/browse/SPARK-21914
> Project: Spark
> Issue Type: Test
> Components: SQL
> Affects Versions: 2.3.0
> Reporter: Hyukjin Kwon
> Assignee: Maxim Gekk
> Priority: Major
>
> It looks we have added many examples in {{ExpressionDescription}} for builtin functions.
> Actually, if I have seen correctly, we have fixed many examples so far in some minor PRs and sometimes require to add the examples as tests sql and golden files.
> As we have formatted examples in {{ExpressionDescription.examples}} - https://github.com/apache/spark/blob/ba327ee54c32b11107793604895bd38559804858/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/ExpressionDescription.java#L44-L50, and we have `SQLQueryTestSuite`, I think we could run the examples as tests like Python's doctests.
> Rough way I am thinking:
> 1. Loads the example in {{ExpressionDescription}}.
> 2. identify queries by {{>}}.
> 3. identify the rest of them as the results.
> 4. run the examples by reusing {{SQLQueryTestSuite}} if possible.
> 5. compare the output by reusing {{SQLQueryTestSuite}} if possible.
> Advantages of doing this I could think for now:
> - Reduce the number of PRs to fix the examples
> - De-duplicate the test cases that should be added into sql and golden files.
> - Correct documentation with correct examples.
> - Reduce reviewing costs for documentation fix PRs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org