You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/02/08 16:41:56 UTC
[GitHub] [iceberg-docs] RussellSpitzer opened a new pull request #48: [Docs] Update versions used in Spark Quickstart Commands
RussellSpitzer opened a new pull request #48:
URL: https://github.com/apache/iceberg-docs/pull/48
Previously we only specified the Spark 3.0 runtime jar which will not work correctly
with the latest Spark version 3.2. In this patch the identifiers are switched to
3.2 artifact locations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] RussellSpitzer merged pull request #48: [Docs] Update versions used in Spark Quickstart Commands
Posted by GitBox <gi...@apache.org>.
RussellSpitzer merged pull request #48:
URL: https://github.com/apache/iceberg-docs/pull/48
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] RussellSpitzer commented on a change in pull request #48: [Docs] Update versions used in Spark Quickstart Commands
Posted by GitBox <gi...@apache.org>.
RussellSpitzer commented on a change in pull request #48:
URL: https://github.com/apache/iceberg-docs/pull/48#discussion_r801850221
##########
File path: docs/content/docs/integrations/aws.md
##########
@@ -49,7 +49,7 @@ For example, to use AWS features with Spark 3 and AWS clients version 2.15.40, y
```sh
# add Iceberg dependency
ICEBERG_VERSION={{% icebergVersion %}}
-DEPENDENCIES="org.apache.iceberg:iceberg-spark3-runtime:$ICEBERG_VERSION"
+DEPENDENCIES="org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:$ICEBERG_VERSION"
Review comment:
@jackye1995 I wasn't sure if AWS is on 3.2 spark yet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] RussellSpitzer commented on pull request #48: [Docs] Update versions used in Spark Quickstart Commands
Posted by GitBox <gi...@apache.org>.
RussellSpitzer commented on pull request #48:
URL: https://github.com/apache/iceberg-docs/pull/48#issuecomment-1036438808
Thanks again everyone
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] jackye1995 commented on a change in pull request #48: [Docs] Update versions used in Spark Quickstart Commands
Posted by GitBox <gi...@apache.org>.
jackye1995 commented on a change in pull request #48:
URL: https://github.com/apache/iceberg-docs/pull/48#discussion_r802924924
##########
File path: docs/content/docs/integrations/aws.md
##########
@@ -49,7 +49,7 @@ For example, to use AWS features with Spark 3 and AWS clients version 2.15.40, y
```sh
# add Iceberg dependency
ICEBERG_VERSION={{% icebergVersion %}}
-DEPENDENCIES="org.apache.iceberg:iceberg-spark3-runtime:$ICEBERG_VERSION"
+DEPENDENCIES="org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:$ICEBERG_VERSION"
Review comment:
Actually not yet. I think we can just change L47 to say `to use AWS features with Spark 3.0`, given the fact that EMR will continue to have 3.0 support in old versions that users can choose.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] RussellSpitzer commented on a change in pull request #48: [Docs] Update versions used in Spark Quickstart Commands
Posted by GitBox <gi...@apache.org>.
RussellSpitzer commented on a change in pull request #48:
URL: https://github.com/apache/iceberg-docs/pull/48#discussion_r802964280
##########
File path: docs/content/docs/integrations/aws.md
##########
@@ -49,7 +49,7 @@ For example, to use AWS features with Spark 3 and AWS clients version 2.15.40, y
```sh
# add Iceberg dependency
ICEBERG_VERSION={{% icebergVersion %}}
-DEPENDENCIES="org.apache.iceberg:iceberg-spark3-runtime:$ICEBERG_VERSION"
+DEPENDENCIES="org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:$ICEBERG_VERSION"
Review comment:
ok I backed this out and just added "Spark 3.0" to the reference above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] RussellSpitzer commented on pull request #48: [Docs] Update versions used in Spark Quickstart Commands
Posted by GitBox <gi...@apache.org>.
RussellSpitzer commented on pull request #48:
URL: https://github.com/apache/iceberg-docs/pull/48#issuecomment-1032828620
@samredai + @jackye1995 , Just ran into a user attempting to use the quickstart guide but it doesn't quite work on Spark 3.2 because of the version differences. I updated all references to --packages spark3 -> spark-3.2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] RussellSpitzer edited a comment on pull request #48: [Docs] Update versions used in Spark Quickstart Commands
Posted by GitBox <gi...@apache.org>.
RussellSpitzer edited a comment on pull request #48:
URL: https://github.com/apache/iceberg-docs/pull/48#issuecomment-1032828620
@samredai + @jackye1995 , Just ran into a user attempting to use the quickstart guide but it doesn't work on Spark 3.2 because of the version differences. I updated all references to --packages spark3 -> spark-3.2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] nastra commented on a change in pull request #48: [Docs] Update versions used in Spark Quickstart Commands
Posted by GitBox <gi...@apache.org>.
nastra commented on a change in pull request #48:
URL: https://github.com/apache/iceberg-docs/pull/48#discussion_r803827653
##########
File path: docs/content/docs/integrations/nessie.md
##########
@@ -35,15 +35,15 @@ See [Project Nessie](https://projectnessie.org) for more information on Nessie.
The `iceberg-nessie` module is bundled with Spark and Flink runtimes for all versions from `0.11.0`. To get started
with Nessie and Iceberg simply add the Iceberg runtime to your process. Eg: `spark-sql --packages
-org.apache.iceberg:iceberg-spark3-runtime:{{% icebergVersion %}}`.
+org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:{{% icebergVersion %}}`.
Review comment:
as discussed on Slack this isn't currently supported by the Nessie Spark SQL Extensions as those are still running with Spark 3.1 and below. Once this is fixed on the Nessie side, I'll update the docs here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] RussellSpitzer commented on pull request #48: [Docs] Update versions used in Spark Quickstart Commands
Posted by GitBox <gi...@apache.org>.
RussellSpitzer commented on pull request #48:
URL: https://github.com/apache/iceberg-docs/pull/48#issuecomment-1034063519
Comment moved to OSS Apache Iceberg Repo -> https://github.com/apache/iceberg/pull/4078
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] RussellSpitzer commented on pull request #48: [Docs] Update versions used in Spark Quickstart Commands
Posted by GitBox <gi...@apache.org>.
RussellSpitzer commented on pull request #48:
URL: https://github.com/apache/iceberg-docs/pull/48#issuecomment-1032836084
> Thansk @RussellSpitzer. Does this effect the `0.12.1` runtime bundles? I'm wondering if we need to hotfix those docs as well.
Well anywhere 0.12.1 is specified Spark 3.2 won't work because we don't make a 0.12.1 Spark 3.2 artifact
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg-docs] samredai commented on pull request #48: [Docs] Update versions used in Spark Quickstart Commands
Posted by GitBox <gi...@apache.org>.
samredai commented on pull request #48:
URL: https://github.com/apache/iceberg-docs/pull/48#issuecomment-1032834345
Thansk @RussellSpitzer. Does this effect the `0.12.1` runtime bundles? I'm wondering if we need to hotfix those docs as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org