You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/12/05 17:35:02 UTC
[GitHub] [iceberg] ajantha-bhat opened a new pull request, #6360: Docs: Update Zorder spark support versions.
ajantha-bhat opened a new pull request, #6360:
URL: https://github.com/apache/iceberg/pull/6360
Some users are using Zorder with spark-3.1 and facing a confusing error message. Hence, updating the document.
Also thought about updating the code to throw the unsupported exception. But maybe not required to modify the code as it is only the older versions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] RussellSpitzer commented on pull request #6360: Docs: Update Zorder spark support versions.
Posted by GitBox <gi...@apache.org>.
RussellSpitzer commented on PR #6360:
URL: https://github.com/apache/iceberg/pull/6360#issuecomment-1341111581
Thanks @ajantha-bhat , looking much clearer now!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #6360: Docs: Update Zorder spark support versions.
Posted by GitBox <gi...@apache.org>.
RussellSpitzer commented on code in PR #6360:
URL: https://github.com/apache/iceberg/pull/6360#discussion_r1040172058
##########
docs/spark-procedures.md:
##########
@@ -271,7 +271,7 @@ Iceberg can compact data files in parallel using Spark with the `rewriteDataFile
|---------------|-----------|------|-------------|
| `table` | ✔️ | string | Name of the table to update |
| `strategy` | | string | Name of the strategy - binpack or sort. Defaults to binpack strategy |
-| `sort_order` | | string | If Zorder, then comma separated column names within zorder() text. Example: zorder(c1,c2,c3). <br/>Else, Comma separated sort_order_column. Where sort_order_column is a space separated sort order info per column (ColumnName SortDirection NullOrder). <br/> SortDirection can be ASC or DESC. NullOrder can be NULLS FIRST or NULLS LAST |
+| `sort_order` | | string | If Zorder(supported in Spark 3.2 and above), then comma separated column names within zorder() text. Example: zorder(c1,c2,c3). <br/>Else, Comma separated sort_order_column. Where sort_order_column is a space separated sort order info per column (ColumnName SortDirection NullOrder). <br/> SortDirection can be ASC or DESC. NullOrder can be NULLS FIRST or NULLS LAST |
Review Comment:
One more comment here, instead of "if Zorder", maybe "For Zorder use a comma separated list of columns within zorder(). (Supported in Spark 3.2 and Above) Example: .... "
"Else, Comma separated sort orders in the format (ColumnName SortDirection NullOrder) where ....
Defaults to the table's sort order."
Just reading over the whole doc it is a little confusing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] RussellSpitzer merged pull request #6360: Docs: Update Zorder spark support versions.
Posted by GitBox <gi...@apache.org>.
RussellSpitzer merged PR #6360:
URL: https://github.com/apache/iceberg/pull/6360
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] RussellSpitzer commented on a diff in pull request #6360: Docs: Update Zorder spark support versions.
Posted by GitBox <gi...@apache.org>.
RussellSpitzer commented on code in PR #6360:
URL: https://github.com/apache/iceberg/pull/6360#discussion_r1039913771
##########
docs/spark-procedures.md:
##########
@@ -271,7 +271,7 @@ Iceberg can compact data files in parallel using Spark with the `rewriteDataFile
|---------------|-----------|------|-------------|
| `table` | ✔️ | string | Name of the table to update |
| `strategy` | | string | Name of the strategy - binpack or sort. Defaults to binpack strategy |
-| `sort_order` | | string | If Zorder, then comma separated column names within zorder() text. Example: zorder(c1,c2,c3). <br/>Else, Comma separated sort_order_column. Where sort_order_column is a space separated sort order info per column (ColumnName SortDirection NullOrder). <br/> SortDirection can be ASC or DESC. NullOrder can be NULLS FIRST or NULLS LAST |
+| `sort_order` | | string | If Zorder(supported from Spark-3.2 and above), then comma separated column names within zorder() text. Example: zorder(c1,c2,c3). <br/>Else, Comma separated sort_order_column. Where sort_order_column is a space separated sort order info per column (ColumnName SortDirection NullOrder). <br/> SortDirection can be ASC or DESC. NullOrder can be NULLS FIRST or NULLS LAST |
Review Comment:
I think either (Supported in Spark 3.2 and above) or (Supported from Spark 3.2) but the mix of "from" and "and above" is a little confusing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org