You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "manuzhang (via GitHub)" <gi...@apache.org> on 2024/03/07 08:20:40 UTC
[PR] Docs: Enhance create_changelog_view usage [iceberg]
manuzhang opened a new pull request, #9889:
URL: https://github.com/apache/iceberg/pull/9889
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Docs: Enhance create_changelog_view usage [iceberg]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #9889:
URL: https://github.com/apache/iceberg/pull/9889#discussion_r1521884083
##########
docs/docs/spark-procedures.md:
##########
@@ -823,6 +823,8 @@ second snapshot deleted 1 record.
|2 | Bob |INSERT |0 |5390529835796506035|
|1 | Alice |DELETE |1 |8764748981452218370|
+#### Net Changes
Review Comment:
I'm OK to bring it up to the same level as `Pre/Post Update Images`. By doing that, we will need to give a brief introduction before jumping into an example. Something like this:
```
The procedure can remove intermediate changes across multiple snapshots, and only outputs the net changes. Here is an example to create a changelog view that computes net changes.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Docs: Enhance create_changelog_view usage [iceberg]
Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on code in PR #9889:
URL: https://github.com/apache/iceberg/pull/9889#discussion_r1520931489
##########
docs/docs/spark-procedures.md:
##########
@@ -756,8 +756,8 @@ Creates a view that contains the changes from a given table.
| `table` | ✔️ | string | Name of the source table for the changelog |
| `changelog_view` | | string | Name of the view to create |
| `options` | | map<string, string> | A map of Spark read options to use |
-| `net_changes` | | boolean | Whether to output net changes (see below for more information). Defaults to false. |
-| `compute_updates` | | boolean | Whether to compute pre/post update images (see below for more information). Defaults to false. |
+| `net_changes` | | boolean | Whether to output net changes (see below for more information). Defaults to false. It must be false when `compute_updates` is true. |
Review Comment:
It's not disabled but throws error 'Not support net changes with update images' when both are true.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Docs: Enhance create_changelog_view usage [iceberg]
Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on code in PR #9889:
URL: https://github.com/apache/iceberg/pull/9889#discussion_r1520932990
##########
docs/docs/spark-procedures.md:
##########
@@ -823,6 +823,8 @@ second snapshot deleted 1 record.
|2 | Bob |INSERT |0 |5390529835796506035|
|1 | Alice |DELETE |1 |8764748981452218370|
+#### Net Changes
Review Comment:
I think `Net Changes` is a feature like `Pre/Post Update Images` so they should be on the same level.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Docs: Enhance create_changelog_view usage [iceberg]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #9889:
URL: https://github.com/apache/iceberg/pull/9889#discussion_r1520078687
##########
docs/docs/spark-procedures.md:
##########
@@ -756,8 +756,8 @@ Creates a view that contains the changes from a given table.
| `table` | ✔️ | string | Name of the source table for the changelog |
| `changelog_view` | | string | Name of the view to create |
| `options` | | map<string, string> | A map of Spark read options to use |
-| `net_changes` | | boolean | Whether to output net changes (see below for more information). Defaults to false. |
-| `compute_updates` | | boolean | Whether to compute pre/post update images (see below for more information). Defaults to false. |
+| `net_changes` | | boolean | Whether to output net changes (see below for more information). Defaults to false. It must be false when `compute_updates` is true. |
Review Comment:
`It must be false` -> `It is disabled`?
I feel `It must be false` means users still have an option to enable it, at least control it. The logic is that it is always disabled when `compute_updates` is true.
##########
docs/docs/spark-procedures.md:
##########
@@ -823,6 +823,8 @@ second snapshot deleted 1 record.
|2 | Bob |INSERT |0 |5390529835796506035|
|1 | Alice |DELETE |1 |8764748981452218370|
+#### Net Changes
Review Comment:
This is an item under the subtitle `example`, it should not be the same level as `example`. Can we change it to one level down, like `##### Net Changes`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Docs: Enhance create_changelog_view usage [iceberg]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain merged PR #9889:
URL: https://github.com/apache/iceberg/pull/9889
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Docs: Enhance create_changelog_view usage [iceberg]
Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on PR #9889:
URL: https://github.com/apache/iceberg/pull/9889#issuecomment-1996506484
@flyrain Thanks for review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Docs: Enhance create_changelog_view usage [iceberg]
Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on PR #9889:
URL: https://github.com/apache/iceberg/pull/9889#issuecomment-1987545272
@nastra @flyrain please help review this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Docs: Enhance create_changelog_view usage [iceberg]
Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #9889:
URL: https://github.com/apache/iceberg/pull/9889#discussion_r1521879235
##########
docs/docs/spark-procedures.md:
##########
@@ -756,8 +756,8 @@ Creates a view that contains the changes from a given table.
| `table` | ✔️ | string | Name of the source table for the changelog |
| `changelog_view` | | string | Name of the view to create |
| `options` | | map<string, string> | A map of Spark read options to use |
-| `net_changes` | | boolean | Whether to output net changes (see below for more information). Defaults to false. |
-| `compute_updates` | | boolean | Whether to compute pre/post update images (see below for more information). Defaults to false. |
+| `net_changes` | | boolean | Whether to output net changes (see below for more information). Defaults to false. It must be false when `compute_updates` is true. |
Review Comment:
Fair enough. Let's keep it as is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org