You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@submarine.apache.org by GitBox <gi...@apache.org> on 2022/01/04 07:04:05 UTC
[GitHub] [submarine] RamakrishnaChilaka opened a new pull request #861: Feature/submarine security spark 3.2
RamakrishnaChilaka opened a new pull request #861:
URL: https://github.com/apache/submarine/pull/861
### What is this PR for?
This pr supports spark 3.2
### What type of PR is it?
[Bug Fix | Improvement | Feature | Documentation | Hot Fix | Refactoring]
Improvement, we are giving spark 3.2 support
### Todos
* [ ] - Task
### What is the Jira issue?
<!-- * Open an issue on Jira https://issues.apache.org/jira/browse/SUBMARINE/
* Put link here, and add [SUBMARINE-*Jira number*] in PR title, eg. `SUBMARINE-23. PR title`
-->
### How should this be tested?
<!--
* First time? Setup Travis CI as described on https://submarine.apache.org/contribution/contributions.html#continuous-integration
* Strongly recommended: add automated unit tests for any new or changed behavior
* Outline any manual steps to test the PR here.
-->
### Screenshots (if appropriate)
### Questions:
* Do the license files need updating? Yes/No
* Are there breaking changes for older versions? Yes/No
* Does this need new documentation? Yes/No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] anumee commented on a change in pull request #861: Feature/submarine security spark 3.2
Posted by GitBox <gi...@apache.org>.
anumee commented on a change in pull request #861:
URL: https://github.com/apache/submarine/pull/861#discussion_r785081213
##########
File path: submarine-security/spark-security/pom.xml
##########
@@ -363,7 +363,7 @@
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
- <version>3.2.2</version>
+ <version>4.5.4</version>
Review comment:
Please use latest v 4.5.6 instead. 4.5.4 has a dependency on log4j-core:jar:2.11.2 with CVE-2021-44228
##########
File path: submarine-security/spark-security/pom.xml
##########
@@ -572,6 +572,20 @@
</build>
<profiles>
+ <profile>
+ <id>spark-3.2</id>
+ <properties>
+ <spark.version>3.2.0</spark.version>
+ <scala.version>2.12.14</scala.version>
Review comment:
I agree with @akizminet. This should be 2.12.15 to match line #51.
##########
File path: submarine-security/spark-security/pom.xml
##########
@@ -572,6 +572,20 @@
</build>
<profiles>
+ <profile>
+ <id>spark-3.2</id>
+ <properties>
+ <spark.version>3.2.0</spark.version>
+ <scala.version>2.12.14</scala.version>
+ <scala.binary.version>2.12</scala.binary.version>
+ <!--<scalatest.version>3.2.0</scalatest.version>-->
+ <spark.compatible.version>3</spark.compatible.version>
+ <commons-lang3.version>3.10</commons-lang3.version>
+ <jackson-databind.version>2.10.5</jackson-databind.version>
+ <jackson-annotations.version>2.10.5</jackson-annotations.version>
Review comment:
Can we also add following here please?
`<ranger.version>2.1.0</ranger.version>`
`<ranger.major.version>2.1</ranger.major.version>`
`<httpcomponents.httpclient.version>4.5.13</httpcomponents.httpclient.version>`
`<httpcomponents.httpcore.version>4.4.15</httpcomponents.httpcore.version>`
`<httpcomponents.httpmime.version>4.5.13</httpcomponents.httpmime.version>`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] akizminet commented on a change in pull request #861: Feature/submarine security spark 3.2
Posted by GitBox <gi...@apache.org>.
akizminet commented on a change in pull request #861:
URL: https://github.com/apache/submarine/pull/861#discussion_r777904128
##########
File path: submarine-security/spark-security/pom.xml
##########
@@ -572,6 +572,20 @@
</build>
<profiles>
+ <profile>
+ <id>spark-3.2</id>
+ <properties>
+ <spark.version>3.2.0</spark.version>
+ <scala.version>2.12.14</scala.version>
Review comment:
why scala.version here is 2.12.14 instead of 2.12.15
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] xunliu closed pull request #861: Feature/submarine security spark 3.2
Posted by GitBox <gi...@apache.org>.
xunliu closed pull request #861:
URL: https://github.com/apache/submarine/pull/861
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] RamakrishnaChilaka commented on a change in pull request #861: Feature/submarine security spark 3.2
Posted by GitBox <gi...@apache.org>.
RamakrishnaChilaka commented on a change in pull request #861:
URL: https://github.com/apache/submarine/pull/861#discussion_r785660228
##########
File path: submarine-security/spark-security/pom.xml
##########
@@ -363,7 +363,7 @@
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
- <version>3.2.2</version>
+ <version>4.5.4</version>
Review comment:
Done
##########
File path: submarine-security/spark-security/pom.xml
##########
@@ -572,6 +572,20 @@
</build>
<profiles>
+ <profile>
+ <id>spark-3.2</id>
+ <properties>
+ <spark.version>3.2.0</spark.version>
+ <scala.version>2.12.14</scala.version>
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] RamakrishnaChilaka commented on a change in pull request #861: Feature/submarine security spark 3.2
Posted by GitBox <gi...@apache.org>.
RamakrishnaChilaka commented on a change in pull request #861:
URL: https://github.com/apache/submarine/pull/861#discussion_r777948979
##########
File path: submarine-security/spark-security/pom.xml
##########
@@ -572,6 +572,20 @@
</build>
<profiles>
+ <profile>
+ <id>spark-3.2</id>
+ <properties>
+ <spark.version>3.2.0</spark.version>
+ <scala.version>2.12.14</scala.version>
Review comment:
2.12.15 is the most latest release of scala.. No reason in particular.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] anumee commented on a change in pull request #861: Feature/submarine security spark 3.2
Posted by GitBox <gi...@apache.org>.
anumee commented on a change in pull request #861:
URL: https://github.com/apache/submarine/pull/861#discussion_r785187391
##########
File path: submarine-security/spark-security/pom.xml
##########
@@ -572,6 +572,20 @@
</build>
<profiles>
+ <profile>
+ <id>spark-3.2</id>
+ <properties>
+ <spark.version>3.2.0</spark.version>
+ <scala.version>2.12.14</scala.version>
Review comment:
I agree with @akizminet. This should be 2.12.15 to match line 51.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] xunliu commented on pull request #861: Feature/submarine security spark 3.2
Posted by GitBox <gi...@apache.org>.
xunliu commented on pull request #861:
URL: https://github.com/apache/submarine/pull/861#issuecomment-1047424313
hi @RamakrishnaChilaka , Security spark has been moved to the `apache/incubator-kyuubi` project, the `submarine` project will not contain the code for `security spark`. So this PR will not be merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] RamakrishnaChilaka commented on a change in pull request #861: Feature/submarine security spark 3.2
Posted by GitBox <gi...@apache.org>.
RamakrishnaChilaka commented on a change in pull request #861:
URL: https://github.com/apache/submarine/pull/861#discussion_r785660012
##########
File path: submarine-security/spark-security/pom.xml
##########
@@ -572,6 +572,20 @@
</build>
<profiles>
+ <profile>
+ <id>spark-3.2</id>
+ <properties>
+ <spark.version>3.2.0</spark.version>
+ <scala.version>2.12.14</scala.version>
+ <scala.binary.version>2.12</scala.binary.version>
+ <!--<scalatest.version>3.2.0</scalatest.version>-->
+ <spark.compatible.version>3</spark.compatible.version>
+ <commons-lang3.version>3.10</commons-lang3.version>
+ <jackson-databind.version>2.10.5</jackson-databind.version>
+ <jackson-annotations.version>2.10.5</jackson-annotations.version>
Review comment:
i agree @anumee , but ranger versions do have a seperate profile altogether.. as this spark 3.2 profile can be used with a different version of ranger as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org