You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "HyukjinKwon (via GitHub)" <gi...@apache.org> on 2024/02/19 03:34:36 UTC

[PR] [SPARK-47090][INFRA] Skip JDK 17/21 Maven compilation in branch-3.4 job [spark]

HyukjinKwon opened a new pull request, #45159:
URL: https://github.com/apache/spark/pull/45159

   ### What changes were proposed in this pull request?
   
   This PR proposes to skip JDK 17/21 Maven build for `branch-3.4`.
   
   ### Why are the changes needed?
   
   - Spark 3.4.0 is actually built on JDK 8 but supports 11 and 17. 
   - Official JDK support is 8/11/17 (https://spark.apache.org/docs/3.4.0/)
   
   To fix up the build https://github.com/apache/spark/actions/runs/7928294496/job/21664443573
   
   ```
   Error:  Failed to execute goal net.alchim31.maven:scala-maven-plugin:4.8.0:compile (scala-compile-first) on project spark-tags_2.12: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:4.8.0:compile failed.: CompileFailed -> [Help 1]
   Error:  
   Error:  To see the full stack trace of the errors, re-run Maven with the -e switch.
   Error:  Re-run Maven using the -X switch to enable full debug logging.
   Error:  
   Error:  For more information about the errors and possible solutions, please read the following articles:
   Error:  [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
   Error:  
   Error:  After correcting the problems, you can resume the build with the command
   Error:    mvn <args> -rf :spark-tags_2.12
   Error: Process completed with exit code 1.
   ```
   
   
   ### Does this PR introduce _any_ user-facing change?
   
   No, dev-only.
   
   ### How was this patch tested?
   
   Will be tested in this PR (as its negative condition to the scheduled build).
   
   ### Was this patch authored or co-authored using generative AI tooling?
   
   No.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-47090][INFRA] Skip JDK 17/21 Maven compilation in branch-3.4 job [spark]

Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon closed pull request #45159: [SPARK-47090][INFRA] Skip JDK 17/21 Maven compilation in branch-3.4 job
URL: https://github.com/apache/spark/pull/45159


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-47090][INFRA] Skip JDK 17/21 Maven compilation in branch-3.4 job [spark]

Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #45159:
URL: https://github.com/apache/spark/pull/45159#discussion_r1494022647


##########
.github/workflows/build_and_test.yml:
##########
@@ -790,7 +790,8 @@ jobs:
 
   maven-build:
     needs: precondition
-    if: fromJson(needs.precondition.outputs.required).maven-build == 'true'
+    # Skipped in branch-3.4, see SPARK-47090.
+    if: fromJson(needs.precondition.outputs.required).maven-build == 'true' && inputs.branch != 'branch-3.4'

Review Comment:
   cc @dongjoon-hyun do you have any preference on this?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-47090][INFRA] Skip JDK 17/21 Maven compilation in branch-3.4 job [spark]

Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #45159:
URL: https://github.com/apache/spark/pull/45159#discussion_r1493969416


##########
.github/workflows/build_and_test.yml:
##########
@@ -790,7 +790,8 @@ jobs:
 
   maven-build:
     needs: precondition
-    if: fromJson(needs.precondition.outputs.required).maven-build == 'true'
+    # Skipped in branch-3.4, see SPARK-47090.
+    if: fromJson(needs.precondition.outputs.required).maven-build == 'true' && inputs.branch != 'branch-3.4'

Review Comment:
   I can add more condition, e.g., only skip JDK 21 in branch-3.5 and branch-3.4 if that's preferred



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-47090][INFRA] Skip JDK 17/21 Maven compilation in branch-3.4 job [spark]

Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on code in PR #45159:
URL: https://github.com/apache/spark/pull/45159#discussion_r1494136868


##########
.github/workflows/build_and_test.yml:
##########
@@ -790,7 +790,8 @@ jobs:
 
   maven-build:
     needs: precondition
-    if: fromJson(needs.precondition.outputs.required).maven-build == 'true'
+    # Skipped in branch-3.4, see SPARK-47090.
+    if: fromJson(needs.precondition.outputs.required).maven-build == 'true' && inputs.branch != 'branch-3.4'

Review Comment:
   https://github.com/apache/spark/blob/c4e4497ff7e747eb71d087cdfb1b51673c53b83b/.github/workflows/build_branch34.yml#L43-L51
   
   For daily build, `maven-build` is not true,  and before https://github.com/apache/spark/pull/45148, the `apple-silicon` job was checking the value of `build`
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-47090][INFRA] Skip JDK 17/21 Maven compilation in branch-3.4 job [spark]

Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #45159:
URL: https://github.com/apache/spark/pull/45159#discussion_r1494252065


##########
.github/workflows/build_and_test.yml:
##########
@@ -790,7 +790,8 @@ jobs:
 
   maven-build:
     needs: precondition
-    if: fromJson(needs.precondition.outputs.required).maven-build == 'true'
+    # Skipped in branch-3.4, see SPARK-47090.
+    if: fromJson(needs.precondition.outputs.required).maven-build == 'true' && inputs.branch != 'branch-3.4'

Review Comment:
   Ohh okie that's good then :-). Let me close this



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-47090][INFRA] Skip JDK 17/21 Maven compilation in branch-3.4 job [spark]

Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on code in PR #45159:
URL: https://github.com/apache/spark/pull/45159#discussion_r1494133318


##########
.github/workflows/build_and_test.yml:
##########
@@ -790,7 +790,8 @@ jobs:
 
   maven-build:
     needs: precondition
-    if: fromJson(needs.precondition.outputs.required).maven-build == 'true'
+    # Skipped in branch-3.4, see SPARK-47090.
+    if: fromJson(needs.precondition.outputs.required).maven-build == 'true' && inputs.branch != 'branch-3.4'

Review Comment:
   hmm... I think after https://github.com/apache/spark/pull/45148/files, the daily job for branch-3.4 will no longer execute maven-build ...



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-47090][INFRA] Skip JDK 17/21 Maven compilation in branch-3.4 job [spark]

Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on code in PR #45159:
URL: https://github.com/apache/spark/pull/45159#discussion_r1494228629


##########
.github/workflows/build_and_test.yml:
##########
@@ -790,7 +790,8 @@ jobs:
 
   maven-build:
     needs: precondition
-    if: fromJson(needs.precondition.outputs.required).maven-build == 'true'
+    # Skipped in branch-3.4, see SPARK-47090.
+    if: fromJson(needs.precondition.outputs.required).maven-build == 'true' && inputs.branch != 'branch-3.4'

Review Comment:
   https://github.com/apache/spark/actions/runs/7957014021



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-47090][INFRA] Skip JDK 17/21 Maven compilation in branch-3.4 job [spark]

Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #45159:
URL: https://github.com/apache/spark/pull/45159#discussion_r1494123602


##########
.github/workflows/build_and_test.yml:
##########
@@ -790,7 +790,8 @@ jobs:
 
   maven-build:
     needs: precondition
-    if: fromJson(needs.precondition.outputs.required).maven-build == 'true'
+    # Skipped in branch-3.4, see SPARK-47090.
+    if: fromJson(needs.precondition.outputs.required).maven-build == 'true' && inputs.branch != 'branch-3.4'

Review Comment:
   Yeah we can do something more fine grained. I prefer to just take it out in branch-3.4 because compilation itself doesn't affect the Spark 3.4 releases.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-47090][INFRA] Skip JDK 17/21 Maven compilation in branch-3.4 job [spark]

Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on code in PR #45159:
URL: https://github.com/apache/spark/pull/45159#discussion_r1494118216


##########
.github/workflows/build_and_test.yml:
##########
@@ -790,7 +790,8 @@ jobs:
 
   maven-build:
     needs: precondition
-    if: fromJson(needs.precondition.outputs.required).maven-build == 'true'
+    # Skipped in branch-3.4, see SPARK-47090.
+    if: fromJson(needs.precondition.outputs.required).maven-build == 'true' && inputs.branch != 'branch-3.4'

Review Comment:
   Perhaps we can use a separate job to generate the content of `matrix` first? Although this might increase complexity



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-47090][INFRA] Skip JDK 17/21 Maven compilation in branch-3.4 job [spark]

Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on code in PR #45159:
URL: https://github.com/apache/spark/pull/45159#discussion_r1494118216


##########
.github/workflows/build_and_test.yml:
##########
@@ -790,7 +790,8 @@ jobs:
 
   maven-build:
     needs: precondition
-    if: fromJson(needs.precondition.outputs.required).maven-build == 'true'
+    # Skipped in branch-3.4, see SPARK-47090.
+    if: fromJson(needs.precondition.outputs.required).maven-build == 'true' && inputs.branch != 'branch-3.4'

Review Comment:
   Perhaps we can use a separate job to generate the content of `java-matrix` first? Although this might increase complexity



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-47090][INFRA] Skip JDK 17/21 Maven compilation in branch-3.4 job [spark]

Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on code in PR #45159:
URL: https://github.com/apache/spark/pull/45159#discussion_r1494136868


##########
.github/workflows/build_and_test.yml:
##########
@@ -790,7 +790,8 @@ jobs:
 
   maven-build:
     needs: precondition
-    if: fromJson(needs.precondition.outputs.required).maven-build == 'true'
+    # Skipped in branch-3.4, see SPARK-47090.
+    if: fromJson(needs.precondition.outputs.required).maven-build == 'true' && inputs.branch != 'branch-3.4'

Review Comment:
   https://github.com/apache/spark/blob/c4e4497ff7e747eb71d087cdfb1b51673c53b83b/.github/workflows/build_branch34.yml#L43-L51
   
   For daily build, `maven-build` is not true,  before https://github.com/apache/spark/pull/45148, the `apple-silicon` job was checking the value of `build`
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-47090][INFRA] Skip JDK 17/21 Maven compilation in branch-3.4 job [spark]

Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on code in PR #45159:
URL: https://github.com/apache/spark/pull/45159#discussion_r1494133318


##########
.github/workflows/build_and_test.yml:
##########
@@ -790,7 +790,8 @@ jobs:
 
   maven-build:
     needs: precondition
-    if: fromJson(needs.precondition.outputs.required).maven-build == 'true'
+    # Skipped in branch-3.4, see SPARK-47090.
+    if: fromJson(needs.precondition.outputs.required).maven-build == 'true' && inputs.branch != 'branch-3.4'

Review Comment:
   hmm... I think after https://github.com/apache/spark/pull/45148, the daily job for branch-3.4 will no longer execute maven-build ...



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org

Re: [PR] [SPARK-47090][INFRA] Skip JDK 17/21 Maven compilation in branch-3.4 job [spark]

Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on code in PR #45159:
URL: https://github.com/apache/spark/pull/45159#discussion_r1494228629


##########
.github/workflows/build_and_test.yml:
##########
@@ -790,7 +790,8 @@ jobs:
 
   maven-build:
     needs: precondition
-    if: fromJson(needs.precondition.outputs.required).maven-build == 'true'
+    # Skipped in branch-3.4, see SPARK-47090.
+    if: fromJson(needs.precondition.outputs.required).maven-build == 'true' && inputs.branch != 'branch-3.4'

Review Comment:
   https://github.com/apache/spark/actions/runs/7957014021
   
   ![image](https://github.com/apache/spark/assets/1475305/277ada51-ee06-435e-816d-5ee8fa7f3c11)
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org