You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by GitBox <gi...@apache.org> on 2022/06/30 01:42:17 UTC

[GitHub] [zeppelin] zjffdu opened a new pull request, #4388: [ZEPPELIN-5755] Support Spark 3.3

zjffdu opened a new pull request, #4388:
URL: https://github.com/apache/zeppelin/pull/4388

   ### What is this PR for?
   
   Trivial PR to support spark 3.3
   * Add profile spark-3.3
   * Add integration tests for spark-3.3
   * Due to api change, sqlContext in pyspark is set via different api in spark 3.3
   
   ### What type of PR is it?
   
   Feature
   
   ### Todos
   * [ ] - Task
   
   ### What is the Jira issue?
   * https://issues.apache.org/jira/startup.jsp?returnTo=%2Fbrowse%2FZEPPELIN-5755
   * 
   ### How should this be tested?
   * CI pass
   
   ### Screenshots (if appropriate)
   
   ### Questions:
   * Does the licenses files need to update? No
   * Is there breaking changes for older versions? No
   * Does this needs documentation? No
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [zeppelin] zjffdu merged pull request #4388: [ZEPPELIN-5755] Support Spark 3.3

Posted by GitBox <gi...@apache.org>.
zjffdu merged PR #4388:
URL: https://github.com/apache/zeppelin/pull/4388


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [zeppelin] shakraz commented on pull request #4388: [ZEPPELIN-5755] Support Spark 3.3

Posted by GitBox <gi...@apache.org>.
shakraz commented on PR #4388:
URL: https://github.com/apache/zeppelin/pull/4388#issuecomment-1189954989

   hey, zjffdu!
   quick question, not sure why but seems like the latest docker image (apache/zeppelin:0.10.1) doesn't support spark3.3, is it because it's simply not up to date?
   Thanks in advance


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [zeppelin] zjffdu commented on pull request #4388: [ZEPPELIN-5755] Support Spark 3.3

Posted by GitBox <gi...@apache.org>.
zjffdu commented on PR #4388:
URL: https://github.com/apache/zeppelin/pull/4388#issuecomment-1189963579

   @shakraz Spark 3.3 has not been released when Zeppelin 0.10.1 is released


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [zeppelin] zjffdu commented on pull request #4388: [ZEPPELIN-5755] Support Spark 3.3

Posted by GitBox <gi...@apache.org>.
zjffdu commented on PR #4388:
URL: https://github.com/apache/zeppelin/pull/4388#issuecomment-1178605080

   Will merge if no more comment


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [zeppelin] Reamer commented on a diff in pull request #4388: [ZEPPELIN-5755] Support Spark 3.3

Posted by GitBox <gi...@apache.org>.
Reamer commented on code in PR #4388:
URL: https://github.com/apache/zeppelin/pull/4388#discussion_r912839095


##########
.github/workflows/core.yml:
##########
@@ -395,6 +395,14 @@ jobs:
         run: |
           rm -rf spark/interpreter/metastore_db
           ./mvnw test -DskipRat -pl spark-submit,spark/interpreter -Pspark-3.2 -Pspark-scala-2.13 -Phadoop2 -Pintegration -B -DfailIfNoTests=false
+      - name: run spark-3.3 tests with scala-2.12 and python-${{ matrix.python }}
+        run: |
+          rm -rf spark/interpreter/metastore_db
+          ./mvnw test -DskipRat -pl spark-submit,spark/interpreter -Pspark-3.3 -Pspark-scala-2.12 -Phadoop2 -Pintegration -B -DfailIfNoTests=false

Review Comment:
   Can we test with Hadoop 3? Hadoop 3 has been available for a while now.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [zeppelin] zjffdu commented on a diff in pull request #4388: [ZEPPELIN-5755] Support Spark 3.3

Posted by GitBox <gi...@apache.org>.
zjffdu commented on code in PR #4388:
URL: https://github.com/apache/zeppelin/pull/4388#discussion_r916501175


##########
.github/workflows/core.yml:
##########
@@ -395,6 +395,14 @@ jobs:
         run: |
           rm -rf spark/interpreter/metastore_db
           ./mvnw test -DskipRat -pl spark-submit,spark/interpreter -Pspark-3.2 -Pspark-scala-2.13 -Phadoop2 -Pintegration -B -DfailIfNoTests=false
+      - name: run spark-3.3 tests with scala-2.12 and python-${{ matrix.python }}
+        run: |
+          rm -rf spark/interpreter/metastore_db
+          ./mvnw test -DskipRat -pl spark-submit,spark/interpreter -Pspark-3.3 -Pspark-scala-2.12 -Phadoop2 -Pintegration -B -DfailIfNoTests=false

Review Comment:
   Done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [zeppelin] cometta commented on pull request #4388: [ZEPPELIN-5755] Support Spark 3.3

Posted by "cometta (via GitHub)" <gi...@apache.org>.
cometta commented on PR #4388:
URL: https://github.com/apache/zeppelin/pull/4388#issuecomment-1430827058

   will the official release of zeppelin with 3.3 support available soon?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@zeppelin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org