You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@polaris.apache.org by "MonkeyCanCode (via GitHub)" <gi...@apache.org> on 2024/09/17 05:01:04 UTC

[PR] Change scala from 2.13 to 2.12 [polaris]

MonkeyCanCode opened a new pull request, #305:
URL: https://github.com/apache/polaris/pull/305

   # Description
   
   As part of https://github.com/apache/polaris/pull/235, we had changed scala version used by spark from 2.13 to 2.12. This PR applied the missing change on jupyter notebook.
   
   ## Type of change
   
   Please delete options that are not relevant.
   
   - [ ] Bug fix (non-breaking change which fixes an issue)
   - [ ] Documentation update
   - [ ] New feature (non-breaking change which adds functionality)
   - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
   - [ ] This change requires a documentation update
   
   # How Has This Been Tested?
   
   Test locally by reran the jupyter notebook.
   
   # Checklist:
   
   Please delete options that are not relevant.
   
   - [x] I have performed a self-review of my code
   - [ ] I have commented my code, particularly in hard-to-understand areas
   - [ ] I have made corresponding changes to the documentation
   - [ ] My changes generate no new warnings
   - [ ] If adding new functionality, I have discussed my implementation with the community using the linked GitHub issue
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Change scala from 2.13 to 2.12 [polaris]

Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on code in PR #305:
URL: https://github.com/apache/polaris/pull/305#discussion_r1763563934


##########
notebooks/SparkPolaris.ipynb:
##########
@@ -258,7 +258,7 @@
     "\n",
     "spark = (SparkSession.builder\n",
     "  .config(\"spark.sql.catalog.spark_catalog\", \"org.apache.iceberg.spark.SparkSessionCatalog\")\n",
-    "  .config(\"spark.jars.packages\", \"org.apache.iceberg:iceberg-spark-runtime-3.5_2.13:1.5.0,org.apache.hadoop:hadoop-aws:3.4.0,software.amazon.awssdk:bundle:2.23.19,software.amazon.awssdk:url-connection-client:2.23.19\")\n",
+    "  .config(\"spark.jars.packages\", \"org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.5.0,org.apache.hadoop:hadoop-aws:3.4.0,software.amazon.awssdk:bundle:2.23.19,software.amazon.awssdk:url-connection-client:2.23.19\")\n",

Review Comment:
   Not a blocker: we may update the iceberg version to 1.5.2 as well. 
   
   Here is another place to upgrade as well:
   https://github.com/polaris-catalog/polaris/blob/68b248aa98a527fe2f15a8920d7524adf288d6ed/regtests/t_pyspark/src/iceberg_spark.py#L75-L75



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Change scala from 2.13 to 2.12 and iceberg runtime from 1.5.0 to 1.5.2 [polaris]

Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain merged PR #305:
URL: https://github.com/apache/polaris/pull/305


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Change scala from 2.13 to 2.12 [polaris]

Posted by "MonkeyCanCode (via GitHub)" <gi...@apache.org>.
MonkeyCanCode commented on code in PR #305:
URL: https://github.com/apache/polaris/pull/305#discussion_r1763570928


##########
notebooks/SparkPolaris.ipynb:
##########
@@ -258,7 +258,7 @@
     "\n",
     "spark = (SparkSession.builder\n",
     "  .config(\"spark.sql.catalog.spark_catalog\", \"org.apache.iceberg.spark.SparkSessionCatalog\")\n",
-    "  .config(\"spark.jars.packages\", \"org.apache.iceberg:iceberg-spark-runtime-3.5_2.13:1.5.0,org.apache.hadoop:hadoop-aws:3.4.0,software.amazon.awssdk:bundle:2.23.19,software.amazon.awssdk:url-connection-client:2.23.19\")\n",
+    "  .config(\"spark.jars.packages\", \"org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.5.0,org.apache.hadoop:hadoop-aws:3.4.0,software.amazon.awssdk:bundle:2.23.19,software.amazon.awssdk:url-connection-client:2.23.19\")\n",

Review Comment:
   Goot catch. Let me update that as well shortly. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Change scala from 2.13 to 2.12 and iceberg runtime from 1.5.0 to 1.5.2 [polaris]

Posted by "flyrain (via GitHub)" <gi...@apache.org>.
flyrain commented on PR #305:
URL: https://github.com/apache/polaris/pull/305#issuecomment-2356861841

   Thanks @MonkeyCanCode for the PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@polaris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org