You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "tomtongue (via GitHub)" <gi...@apache.org> on 2023/09/14 11:33:20 UTC
[GitHub] [iceberg] tomtongue opened a new pull request, #8562: Doc: Fix CREATE BRANCH syntax and broken link in Writing to Branches section
tomtongue opened a new pull request, #8562:
URL: https://github.com/apache/iceberg/pull/8562
## Changes
* Fix `CREATE BRANCH` syntax in *Branching and Tagging* part and Spark DDL
* Fix broken link in *Writing to Branches* section in Spark Writes
## Details
There are several syntax descriptions that don't follow the current design in the doc.
The current doc shows the CREATE BRANCH in Branching and Tagging part like;
```sql
-- Create a branch "test-branch" which will be retained for 7 days along with the latest 2 snapshots
ALTER TABLE prod.db.table CREATE BRANCH test-branch RETAIN 7 DAYS WITH RETENTION 2 SNAPSHOTS
```
If you run the query, the query fails with the following error;
```scala
scala> spark.sql("ALTER TABLE glue_catalog.garbagedb.iceberg_branch CREATE BRANCH branch3 RETAIN 2 DAYS WITH RETENTION 3 SNAPSHOTS")
org.apache.spark.sql.catalyst.parser.extensions.IcebergParseException:
no viable alternative at input 'WITH RETENTION'
== SQL ==
ALTER TABLE glue_catalog.garbagedb.iceberg_branch CREATE BRANCH branch3 RETAIN 2 DAYS WITH RETENTION 3 SNAPSHOTS
at org.apache.spark.sql.catalyst.parser.extensions.IcebergParseException.withCommand(IcebergSparkSqlExtensionsParser.scala:393)
at org.apache.spark.sql.catalyst.parser.extensions.IcebergSparkSqlExtensionsParser.parse(IcebergSparkSqlExtensionsParser.scala:252)
at org.apache.spark.sql.catalyst.parser.extensions.IcebergSparkSqlExtensionsParser.parsePlan(IcebergSparkSqlExtensionsParser.scala:129)
at org.apache.spark.sql.SparkSession.$anonfun$sql$2(SparkSession.scala:633)
at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:192)
at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:632)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:827)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:630)
at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:671)
... 47 elided
```
Based on the [antlr source](https://github.com/apache/iceberg/blob/master/spark/v3.4/spark-extensions/src/main/antlr/org.apache.spark.sql.catalyst.parser.extensions/IcebergSqlExtensions.g4#L101) in the Iceberg, the following query looks correct (this query worked well in Spark 3.4 and the latest Iceberg built from source).
```scala
scala> spark.sql("ALTER TABLE glue_catalog.garbagedb.iceberg_branch CREATE BRANCH branch3 RETAIN 2 DAYS WITH SNAPSHOT RETENTION 3 DAYS").
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] nastra commented on a diff in pull request #8562: Docs: Fix CREATE BRANCH syntax and broken link in Writing to Branches section
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #8562:
URL: https://github.com/apache/iceberg/pull/8562#discussion_r1326883495
##########
docs/spark-writes.md:
##########
@@ -210,7 +210,7 @@ Branch writes can also be performed as part of a write-audit-publish (WAP) workf
Note WAP branch and branch identifier cannot both be specified.
Also, the branch must exist before performing the write.
The operation does **not** create the branch if it does not exist.
-For more information on branches please refer to [branches](../tables/branching)
+For more information on branches please refer to [branches](../branching)
Review Comment:
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] tomtongue commented on a diff in pull request #8562: Docs: Fix CREATE BRANCH syntax and broken link in Writing to Branches section
Posted by "tomtongue (via GitHub)" <gi...@apache.org>.
tomtongue commented on code in PR #8562:
URL: https://github.com/apache/iceberg/pull/8562#discussion_r1326970634
##########
docs/spark-writes.md:
##########
@@ -210,7 +210,7 @@ Branch writes can also be performed as part of a write-audit-publish (WAP) workf
Note WAP branch and branch identifier cannot both be specified.
Also, the branch must exist before performing the write.
The operation does **not** create the branch if it does not exist.
-For more information on branches please refer to [branches](../tables/branching)
+For more information on branches please refer to [branches](../branching)
Review Comment:
@manuzhang @nastra thanks for looking at this and pointing the page to me. Just a nit, the current doc returns 404 not found when choosing **branches** in https://iceberg.apache.org/docs/1.3.1/spark-writes/#writing-to-branches section. The link shows https://iceberg.apache.org/docs/tables/branching but the correct link seems to be https://iceberg.apache.org/docs/<version>/branching. That's why I updated the link.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] manuzhang commented on a diff in pull request #8562: Docs: Fix CREATE BRANCH syntax and broken link in Writing to Branches section
Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on code in PR #8562:
URL: https://github.com/apache/iceberg/pull/8562#discussion_r1326287181
##########
docs/spark-writes.md:
##########
@@ -210,7 +210,7 @@ Branch writes can also be performed as part of a write-audit-publish (WAP) workf
Note WAP branch and branch identifier cannot both be specified.
Also, the branch must exist before performing the write.
The operation does **not** create the branch if it does not exist.
-For more information on branches please refer to [branches](../tables/branching)
+For more information on branches please refer to [branches](../branching)
Review Comment:
Based on previous [review comment](https://github.com/apache/iceberg/pull/8165#pullrequestreview-1551505550), it should be `../tables/branching`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] tomtongue commented on pull request #8562: Docs: Fix CREATE BRANCH syntax and broken link in Writing to Branches section
Posted by "tomtongue (via GitHub)" <gi...@apache.org>.
tomtongue commented on PR #8562:
URL: https://github.com/apache/iceberg/pull/8562#issuecomment-1719774226
Thanks for quickly review this, @amogh-jahagirdar!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] nastra commented on a diff in pull request #8562: Docs: Fix CREATE BRANCH syntax and broken link in Writing to Branches section
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #8562:
URL: https://github.com/apache/iceberg/pull/8562#discussion_r1326885177
##########
docs/spark-writes.md:
##########
@@ -210,7 +210,7 @@ Branch writes can also be performed as part of a write-audit-publish (WAP) workf
Note WAP branch and branch identifier cannot both be specified.
Also, the branch must exist before performing the write.
The operation does **not** create the branch if it does not exist.
-For more information on branches please refer to [branches](../tables/branching)
+For more information on branches please refer to [branches](../branching)
Review Comment:
```suggestion
For more information on branches please refer to [branches](../tables/branching)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] manuzhang commented on a diff in pull request #8562: Docs: Fix CREATE BRANCH syntax and broken link in Writing to Branches section
Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on code in PR #8562:
URL: https://github.com/apache/iceberg/pull/8562#discussion_r1326287181
##########
docs/spark-writes.md:
##########
@@ -210,7 +210,7 @@ Branch writes can also be performed as part of a write-audit-publish (WAP) workf
Note WAP branch and branch identifier cannot both be specified.
Also, the branch must exist before performing the write.
The operation does **not** create the branch if it does not exist.
-For more information on branches please refer to [branches](../tables/branching)
+For more information on branches please refer to [branches](../branching)
Review Comment:
Based on previous [review comment](https://github.com/apache/iceberg/pull/8165#discussion_r1277194854), it should be `../tables/branching`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] tomtongue commented on pull request #8562: Docs: Fix CREATE BRANCH syntax and broken link in Writing to Branches section
Posted by "tomtongue (via GitHub)" <gi...@apache.org>.
tomtongue commented on PR #8562:
URL: https://github.com/apache/iceberg/pull/8562#issuecomment-1720793168
Thanks for reviewing @amogh-jahagirdar @manuzhang @nastra !!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] nastra merged pull request #8562: Docs: Fix CREATE BRANCH syntax and broken link in Writing to Branches section
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra merged PR #8562:
URL: https://github.com/apache/iceberg/pull/8562
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] tomtongue commented on a diff in pull request #8562: Docs: Fix CREATE BRANCH syntax and broken link in Writing to Branches section
Posted by "tomtongue (via GitHub)" <gi...@apache.org>.
tomtongue commented on code in PR #8562:
URL: https://github.com/apache/iceberg/pull/8562#discussion_r1326970634
##########
docs/spark-writes.md:
##########
@@ -210,7 +210,7 @@ Branch writes can also be performed as part of a write-audit-publish (WAP) workf
Note WAP branch and branch identifier cannot both be specified.
Also, the branch must exist before performing the write.
The operation does **not** create the branch if it does not exist.
-For more information on branches please refer to [branches](../tables/branching)
+For more information on branches please refer to [branches](../branching)
Review Comment:
@manuzhang @nastra thanks for looking at this and pointing the page to me. Just a nit, the current doc returns 404 not found when choosing **branches** in https://iceberg.apache.org/docs/1.3.1/spark-writes/#writing-to-branches section. The link shows https://iceberg.apache.org/docs/tables/branching but the correct link seems to be https://iceberg.apache.org/docs/<version>/branching. That's why I changed update the link.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org