You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "zhengruifeng (via GitHub)" <gi...@apache.org> on 2023/09/20 08:09:23 UTC

[GitHub] [spark] zhengruifeng opened a new pull request, #43011: [SPARK-45232][DOC] Add missing function groups to SQL references

zhengruifeng opened a new pull request, #43011:
URL: https://github.com/apache/spark/pull/43011

   ### What changes were proposed in this pull request?
   Add missing function groups to SQL references:
   - xml_funcs
   - lambda_funcs
   - collection_funcs
   - url_funcs
   - hash_funcsx
   
   Note that this PR doesn't fix `table_funcs`:
   1, `gen-sql-functions-docs.py` doesn't work properly with `TableFunctionRegistry`, I took a cursory look but fail to fix it;
   2, table functions except `range` (e.g. `explode`) were already contained in `Generator Functions`, not sure we need to show them twice.
   
   
   ### Why are the changes needed?
   when I am referring to the SQL references, I find many functions are missing https://spark.apache.org/docs/latest/sql-ref-functions.html.
   
   
   
   
   
   ### Does this PR introduce _any_ user-facing change?
   yes
   
   
   ### How was this patch tested?
   manually check
   
   
   ### Was this patch authored or co-authored using generative AI tooling?
   no
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on pull request #43011: [SPARK-45232][SQL][DOCS] Add missing function groups to SQL references

Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #43011:
URL: https://github.com/apache/spark/pull/43011#issuecomment-1730749736

   Just to be clear, this is automatic documentation based on the current documentation. If the grouping is wrong, or to be fixed, we should fix `functions.scala` main documentation.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zhengruifeng commented on pull request #43011: [SPARK-45232][SQL][DOCS] Add missing function groups to SQL references

Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #43011:
URL: https://github.com/apache/spark/pull/43011#issuecomment-1732950643

   
   ![image](https://github.com/apache/spark/assets/7322292/d93fb9bb-6d00-4112-9104-dadfa3314fc4)
   
   
   ![image](https://github.com/apache/spark/assets/7322292/8503b5ab-794a-4189-8f8a-c82619a2a244)
   
   
   now we put `lambda functions` into `collection functions`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] srielau commented on pull request #43011: [SPARK-45232][DOCS] Add missing function groups to SQL references

Posted by "srielau (via GitHub)" <gi...@apache.org>.
srielau commented on PR #43011:
URL: https://github.com/apache/spark/pull/43011#issuecomment-1729845582

   What is the purpose of "lambda function"? All others are type-specific or "functionality"-specific.
   But lambda is "technology". What is the user journey that would drive one to browse lambda functions?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zhengruifeng commented on a diff in pull request #43011: [WIP][SPARK-45232][DOC] Add missing function groups to SQL references

Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on code in PR #43011:
URL: https://github.com/apache/spark/pull/43011#discussion_r1331200011


##########
sql/gen-sql-functions-docs.py:
##########
@@ -34,6 +34,8 @@
     "math_funcs", "conditional_funcs", "generator_funcs",
     "predicate_funcs", "string_funcs", "misc_funcs",
     "bitwise_funcs", "conversion_funcs", "csv_funcs",
+    "xml_funcs", "lambda_funcs", "collection_funcs",
+    "url_funcs", "hash_funcs",

Review Comment:
   manually check with
   ```
   ag --scala 'group = \"' sql
   ```
   
   all groups but `table_funcs` should be here



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zhengruifeng commented on a diff in pull request #43011: [WIP][SPARK-45232][DOC] Add missing function groups to SQL references

Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on code in PR #43011:
URL: https://github.com/apache/spark/pull/43011#discussion_r1331213799


##########
sql/gen-sql-functions-docs.py:
##########
@@ -34,6 +34,8 @@
     "math_funcs", "conditional_funcs", "generator_funcs",
     "predicate_funcs", "string_funcs", "misc_funcs",
     "bitwise_funcs", "conversion_funcs", "csv_funcs",
+    "xml_funcs", "lambda_funcs", "collection_funcs",
+    "url_funcs", "hash_funcs", "struct_funcs",

Review Comment:
   check against
   https://github.com/apache/spark/blob/37ab190dc5bfa59b4e06af9551c35ab179a05733/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/ExpressionInfo.java#L43-L48
   
   two difference:
   1, `table_funcs`: not support in `gen-sql-functions-docs.py`;
   2, `binary_funcs`: we don't have function using this group;



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zhengruifeng commented on pull request #43011: [SPARK-45232][SQL][DOCS] Add missing function groups to SQL references

Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #43011:
URL: https://github.com/apache/spark/pull/43011#issuecomment-1730672720

   > What is the purpose of "lambda function"? All others are type-specific or "functionality"-specific. But lambda is "technology".
   
   lambda functions were already exposed to end users (e.g. https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.functions.array_sort.html#pyspark.sql.functions.array_sort). 
   
   I think if we document other functions here, it is better to add lambda functions as well.
   
   > What is the user journey that would drive one to browse lambda functions?
   
   I think this could be an example: when a user try to sort array of structs by a specific order, he may refer to the document of `array_sort` 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zhengruifeng commented on a diff in pull request #43011: [WIP][SPARK-45232][DOC] Add missing function groups to SQL references

Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on code in PR #43011:
URL: https://github.com/apache/spark/pull/43011#discussion_r1331213799


##########
sql/gen-sql-functions-docs.py:
##########
@@ -34,6 +34,8 @@
     "math_funcs", "conditional_funcs", "generator_funcs",
     "predicate_funcs", "string_funcs", "misc_funcs",
     "bitwise_funcs", "conversion_funcs", "csv_funcs",
+    "xml_funcs", "lambda_funcs", "collection_funcs",
+    "url_funcs", "hash_funcs", "struct_funcs",

Review Comment:
   check against
   https://github.com/apache/spark/blob/37ab190dc5bfa59b4e06af9551c35ab179a05733/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/ExpressionInfo.java#L43-L48
   
   two difference:
   1, `table_funcs`: not support in `gen-sql-functions-docs.py`;
   2, `binary_funcs`: I can not find any function using this group;



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zhengruifeng commented on pull request #43011: [WIP][SPARK-45232][DOCS] Add missing function groups to SQL references

Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #43011:
URL: https://github.com/apache/spark/pull/43011#issuecomment-1728625579

   we can check the documents built in the GA of this PR,  https://github.com/zhengruifeng/spark/actions/runs/6249096629
   
   ![image](https://github.com/apache/spark/assets/7322292/ea1f1389-754b-479c-9f7c-1b9083180667)
   
   
   however, it expires after one day


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] HyukjinKwon commented on pull request #43011: [SPARK-45232][SQL][DOCS] Add missing function groups to SQL references

Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #43011:
URL: https://github.com/apache/spark/pull/43011#issuecomment-1730788674

   yeah, I mean individual `ExpressionDescription`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] srielau commented on pull request #43011: [SPARK-45232][SQL][DOCS] Add missing function groups to SQL references

Posted by "srielau (via GitHub)" <gi...@apache.org>.
srielau commented on PR #43011:
URL: https://github.com/apache/spark/pull/43011#issuecomment-1730804920

   > > What is the purpose of "lambda function"? All others are type-specific or "functionality"-specific. But lambda is "technology".
   > 
   > lambda functions were already exposed to end users (e.g. https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.functions.array_sort.html#pyspark.sql.functions.array_sort).
   > 
   > I think if we document other functions here, it is better to add lambda functions as well.
   > 
   > > What is the user journey that would drive one to browse lambda functions?
   > 
   > I think this could be an example: when a user try to sort array of structs by a specific order, he may refer to the document of `array_sort`
   
   If I try to find a function that sorts arrays I will try to find that function under collection functions.
   Just like substr() is not a math function, even though most of its arguments are integers. Substr operates on strings...
   array_sort operates on arrays.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zhengruifeng commented on pull request #43011: [SPARK-45232][SQL][DOCS] Add missing function groups to SQL references

Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #43011:
URL: https://github.com/apache/spark/pull/43011#issuecomment-1730830602

   > > > What is the purpose of "lambda function"? All others are type-specific or "functionality"-specific. But lambda is "technology".
   > > 
   > > 
   > > lambda functions were already exposed to end users (e.g. https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.functions.array_sort.html#pyspark.sql.functions.array_sort).
   > > I think if we document other functions here, it is better to add lambda functions as well.
   > > > What is the user journey that would drive one to browse lambda functions?
   > > 
   > > 
   > > I think this could be an example: when a user try to sort array of structs by a specific order, he may refer to the document of `array_sort`
   > 
   > If I try to find a function that sorts arrays I will try to find that function under collection functions. Just like substr() is not a math function, even though most of its arguments are integers. Substr operates on strings... array_sort operates on arrays.
   
   Got it.
   I think we can rename the group name in `.md` files, but need to be different from others. What about `Advanced Collection Functions`? @srielau 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zhengruifeng commented on pull request #43011: [SPARK-45232][SQL][DOCS] Add missing function groups to SQL references

Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #43011:
URL: https://github.com/apache/spark/pull/43011#issuecomment-1732777865

   > > > > > What is the purpose of "lambda function"? All others are type-specific or "functionality"-specific. But lambda is "technology".
   > > > > 
   > > > > 
   > > > > lambda functions were already exposed to end users (e.g. https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.functions.array_sort.html#pyspark.sql.functions.array_sort).
   > > > > I think if we document other functions here, it is better to add lambda functions as well.
   > > > > > What is the user journey that would drive one to browse lambda functions?
   > > > > 
   > > > > 
   > > > > I think this could be an example: when a user try to sort array of structs by a specific order, he may refer to the document of `array_sort`
   > > > 
   > > > 
   > > > If I try to find a function that sorts arrays I will try to find that function under collection functions. Just like substr() is not a math function, even though most of its arguments are integers. Substr operates on strings... array_sort operates on arrays.
   > > 
   > > 
   > > Got it. I think we can rename the group name in `.md` files, but need to be different from others. What about `Advanced Collection Functions`? @srielau
   > 
   > How about having our cake and eat it to? Can a function be in more than one group?
   
   probably we can. I will try to map `lambda` to `collection` just in the doc build.
   
   I think making a function in more than one group would be much complex.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] srielau commented on pull request #43011: [SPARK-45232][SQL][DOCS] Add missing function groups to SQL references

Posted by "srielau (via GitHub)" <gi...@apache.org>.
srielau commented on PR #43011:
URL: https://github.com/apache/spark/pull/43011#issuecomment-1732636254

   > > > > What is the purpose of "lambda function"? All others are type-specific or "functionality"-specific. But lambda is "technology".
   > > > 
   > > > 
   > > > lambda functions were already exposed to end users (e.g. https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.functions.array_sort.html#pyspark.sql.functions.array_sort).
   > > > I think if we document other functions here, it is better to add lambda functions as well.
   > > > > What is the user journey that would drive one to browse lambda functions?
   > > > 
   > > > 
   > > > I think this could be an example: when a user try to sort array of structs by a specific order, he may refer to the document of `array_sort`
   > > 
   > > 
   > > If I try to find a function that sorts arrays I will try to find that function under collection functions. Just like substr() is not a math function, even though most of its arguments are integers. Substr operates on strings... array_sort operates on arrays.
   > 
   > Got it. I think we can rename the group name in `.md` files, but need to be different from others. What about `Advanced Collection Functions`? @srielau
   
   How about having our cake and eat it to? Can a function be in more than one group?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zhengruifeng commented on a diff in pull request #43011: [SPARK-45232][DOC] Add missing function groups to SQL references

Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on code in PR #43011:
URL: https://github.com/apache/spark/pull/43011#discussion_r1331200011


##########
sql/gen-sql-functions-docs.py:
##########
@@ -34,6 +34,8 @@
     "math_funcs", "conditional_funcs", "generator_funcs",
     "predicate_funcs", "string_funcs", "misc_funcs",
     "bitwise_funcs", "conversion_funcs", "csv_funcs",
+    "xml_funcs", "lambda_funcs", "collection_funcs",
+    "url_funcs", "hash_funcs",

Review Comment:
   manually check with
   ```
   ag --scala 'group = \"' sql
   ```
   
   all groups but `table_funcs` should be here



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zhengruifeng closed pull request #43011: [SPARK-45232][SQL][DOCS] Add missing function groups to SQL references

Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng closed pull request #43011: [SPARK-45232][SQL][DOCS] Add missing function groups to SQL references
URL: https://github.com/apache/spark/pull/43011


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zhengruifeng commented on pull request #43011: [SPARK-45232][SQL][DOCS] Add missing function groups to SQL references

Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #43011:
URL: https://github.com/apache/spark/pull/43011#issuecomment-1734633746

   @srielau I have put `lambda functions` in `collection functions`, I think this PR is ready to merge?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zhengruifeng commented on pull request #43011: [WIP][SPARK-45232][DOCS] Add missing function groups to SQL references

Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #43011:
URL: https://github.com/apache/spark/pull/43011#issuecomment-1728624460

   
   ![image](https://github.com/apache/spark/assets/7322292/09ae6ec9-a2a6-4f00-b260-88886c680ff3)
   
   ![image](https://github.com/apache/spark/assets/7322292/15308ec1-aa0a-4495-9769-3e8707662b89)
   
   ![image](https://github.com/apache/spark/assets/7322292/fdd0e404-bffd-4663-bec1-82ead49052ce)
   
   ![image](https://github.com/apache/spark/assets/7322292/b8c11c51-979a-4f8e-9c29-75769b52dae1)
   
   ![image](https://github.com/apache/spark/assets/7322292/0f5426c4-6086-46b2-8d67-05f57e10f0fb)
   
   ![image](https://github.com/apache/spark/assets/7322292/066e7bea-646d-4bca-8d14-7e72e91aefd1)
   
   ![image](https://github.com/apache/spark/assets/7322292/e94734d1-3894-453f-b2e0-968e619ab513)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] allisonwang-db commented on a diff in pull request #43011: [WIP][SPARK-45232][DOC] Add missing function groups to SQL references

Posted by "allisonwang-db (via GitHub)" <gi...@apache.org>.
allisonwang-db commented on code in PR #43011:
URL: https://github.com/apache/spark/pull/43011#discussion_r1331925178


##########
sql/gen-sql-functions-docs.py:
##########
@@ -34,6 +34,8 @@
     "math_funcs", "conditional_funcs", "generator_funcs",
     "predicate_funcs", "string_funcs", "misc_funcs",
     "bitwise_funcs", "conversion_funcs", "csv_funcs",
+    "xml_funcs", "lambda_funcs", "collection_funcs",
+    "url_funcs", "hash_funcs", "struct_funcs",

Review Comment:
   QQ: For generator_funcs, do we have documentation for them when used in the FROM clause of a query? Functions like explode are typically considered table-valued generator functions.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] gatorsmile commented on pull request #43011: [WIP][SPARK-45232][DOCS] Add missing function groups to SQL references

Posted by "gatorsmile (via GitHub)" <gi...@apache.org>.
gatorsmile commented on PR #43011:
URL: https://github.com/apache/spark/pull/43011#issuecomment-1728585646

   cc @srielau 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zhengruifeng commented on pull request #43011: [WIP][SPARK-45232][DOCS] Add missing function groups to SQL references

Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #43011:
URL: https://github.com/apache/spark/pull/43011#issuecomment-1728581850

   @allisonwang-db I am not sure, I don't see document for FROM clause, you may check 3 places:
   
   - https://spark.apache.org/docs/latest/api/sql/index.html#explode
   - https://spark.apache.org/docs/latest/sql-ref-functions-builtin.html#generator-functions
   - https://spark.apache.org/docs/latest/sql-ref-syntax.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zhengruifeng commented on pull request #43011: [SPARK-45232][SQL][DOCS] Add missing function groups to SQL references

Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #43011:
URL: https://github.com/apache/spark/pull/43011#issuecomment-1734891541

   thanks @srielau @allisonwang-db @HyukjinKwon 
   
   merged to master


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

[GitHub] [spark] zhengruifeng commented on pull request #43011: [SPARK-45232][SQL][DOCS] Add missing function groups to SQL references

Posted by "zhengruifeng (via GitHub)" <gi...@apache.org>.
zhengruifeng commented on PR #43011:
URL: https://github.com/apache/spark/pull/43011#issuecomment-1730787450

   @HyukjinKwon this page is not built from `functions.scala`, but from the groups specified in expression definitions, like
   
   https://github.com/apache/spark/blob/6b747ab8cef44ea46102b2685b9254f7c4828ea8/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala#L1313
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org