You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/04/28 07:07:21 UTC
[GitHub] [arrow] raulcd opened a new pull request, #13021: [WIP] ARROW-16327: [Java][CI] Add Java 17 to CI matrix for java workflows
raulcd opened a new pull request, #13021:
URL: https://github.com/apache/arrow/pull/13021
This PR aims to add support for java 17 on CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on pull request #13021: [WIP] ARROW-16327: [Java][CI] Add Java 17 to CI matrix for java workflows
Posted by GitBox <gi...@apache.org>.
raulcd commented on PR #13021:
URL: https://github.com/apache/arrow/pull/13021#issuecomment-1111829516
@github-actions crossbow submit *java*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on pull request #13021: ARROW-16327: [Java][CI] Add Java 17 to CI matrix for java workflows
Posted by GitBox <gi...@apache.org>.
pitrou commented on PR #13021:
URL: https://github.com/apache/arrow/pull/13021#issuecomment-1118495781
@raulcd I think this is good to go, I'll let @kou comment if he wants to.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on a diff in pull request #13021: ARROW-16327: [Java][CI] Add Java 17 to CI matrix for java workflows
Posted by GitBox <gi...@apache.org>.
raulcd commented on code in PR #13021:
URL: https://github.com/apache/arrow/pull/13021#discussion_r862865731
##########
docker-compose.yml:
##########
@@ -1488,6 +1482,18 @@ services:
/arrow/ci/scripts/java_build.sh /arrow /build &&
/arrow/ci/scripts/java_test.sh /arrow /build"
+ oracle-java:
Review Comment:
the maven image for openjdk 17 is based on Oracle Linux instead of debian `maven:3.8.5-openjdk-17`:
```
$ docker run --rm maven:3.8.5-openjdk-17 cat /etc/system-release
Oracle Linux Server release 8.5
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pitrou commented on a diff in pull request #13021: ARROW-16327: [Java][CI] Add Java 17 to CI matrix for java workflows
Posted by GitBox <gi...@apache.org>.
pitrou commented on code in PR #13021:
URL: https://github.com/apache/arrow/pull/13021#discussion_r862836853
##########
docker-compose.yml:
##########
@@ -1488,6 +1482,18 @@ services:
/arrow/ci/scripts/java_build.sh /arrow /build &&
/arrow/ci/scripts/java_test.sh /arrow /build"
+ oracle-java:
Review Comment:
How is this different from `debian-java`? It seems to be based on the same base image.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on pull request #13021: ARROW-16327: [Java][CI] Add Java 17 to CI matrix for java workflows
Posted by GitBox <gi...@apache.org>.
kou commented on PR #13021:
URL: https://github.com/apache/arrow/pull/13021#issuecomment-1112624192
It seems that `maven:3.8.5-openjdk-17` is based on Oracle Linux 8 not Debian:
```console
$ docker run --rm maven:3.8.5-openjdk-17 cat /etc/system-release
Oracle Linux Server release 8.5
```
So existing `ci/docker/debian-9-java.dockerfile` filename isn't suitable for OpenJDK 17.
How about adding `ci/docker/oracle-8-java.dockerfile`, adding `oracle-java` to `docker-compose.yml` and removing `jdk_name` from `.env`?
BTW, it seems that nobody uses `ARROW_JAVA_SHADE_FLATBUFS` defined in `ci/docker/debian-9-java.dockerfile`. See also: 7bc2b0f3579b6fc2d1d6d9797f3a1e5e3726131f
We may be able to use `maven:*` images directly instead of defining our (thin) wrapper image. For example, we use existing images directly for `python-wheel-manylinux-test-imports`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou closed pull request #13021: ARROW-16327: [Java][CI] Add Java 17 to CI matrix for java workflows
Posted by GitBox <gi...@apache.org>.
kou closed pull request #13021: ARROW-16327: [Java][CI] Add Java 17 to CI matrix for java workflows
URL: https://github.com/apache/arrow/pull/13021
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #13021: ARROW-16327: [Java][CI] Add Java 17 to CI matrix for java workflows
Posted by GitBox <gi...@apache.org>.
ursabot commented on PR #13021:
URL: https://github.com/apache/arrow/pull/13021#issuecomment-1121169618
Benchmark runs are scheduled for baseline = 4ea776a01eed0505d57755c395863d34bce45ebb and contender = 1fc251f18d5b48f0c9fe8af8168237e7e6d05a45. 1fc251f18d5b48f0c9fe8af8168237e7e6d05a45 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/b27d690a5fa74024be08b0eb78f6bb59...1311f661896d46419311c3441c71f282/)
[Finished :arrow_down:0.47% :arrow_up:0.12%] [test-mac-arm](https://conbench.ursa.dev/compare/runs/f9748f4d95824a65b16f2b7cd3c29911...59d3cde4fd024cca9bffb41b9ab9204e/)
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/48527a6ca83b4a7ea2d06dc613ac9e6d...cdd42037eb5a4a40b6a68977baf40c7a/)
[Finished :arrow_down:0.63% :arrow_up:0.0%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/f481bd807d7143fcb299688848b4447b...fbf0cbe1bb9342de90967f4a4fc0c2ba/)
Buildkite builds:
[Finished] [`1fc251f1` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/708)
[Finished] [`1fc251f1` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/705)
[Finished] [`1fc251f1` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/694)
[Finished] [`1fc251f1` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/710)
[Finished] [`4ea776a0` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/707)
[Finished] [`4ea776a0` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/704)
[Finished] [`4ea776a0` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/693)
[Finished] [`4ea776a0` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/709)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on pull request #13021: ARROW-16327: [Java][CI] Add Java 17 to CI matrix for java workflows
Posted by GitBox <gi...@apache.org>.
raulcd commented on PR #13021:
URL: https://github.com/apache/arrow/pull/13021#issuecomment-1114707214
Thanks for the review @kou ! I have added a new `oracle-java` entry on the `docker-compose.yml` and I have updated the `debian-java` to use the `maven:*` images directly removing the wrapper image `ci/docker/debian-9-java.dockerfile`, as you said I can't find anywhere where the `ARROW_JAVA_SHADE_FLATBUFS` were being used. I have also removed the `JDK_NAME` and have added a JDK 8 CI test (as it seems to be the default defined on our .env).
As per the failure on JDK 17 it is expected until we merge https://github.com/apache/arrow/pull/12941, the idea is to rebase once that PR is merged and validate the JDK 17 image is passing at that point!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on pull request #13021: ARROW-16327: [Java][CI] Add Java 17 to CI matrix for java workflows
Posted by GitBox <gi...@apache.org>.
kou commented on PR #13021:
URL: https://github.com/apache/arrow/pull/13021#issuecomment-1116664931
@github-actions crossbow submit java-jars
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13021: ARROW-16327: [Java][CI] Add Java 17 to CI matrix for java workflows
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13021:
URL: https://github.com/apache/arrow/pull/13021#issuecomment-1111838407
https://issues.apache.org/jira/browse/ARROW-16327
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] raulcd commented on pull request #13021: ARROW-16327: [Java][CI] Add Java 17 to CI matrix for java workflows
Posted by GitBox <gi...@apache.org>.
raulcd commented on PR #13021:
URL: https://github.com/apache/arrow/pull/13021#issuecomment-1118372786
@kou @pitrou is there anything else you think we should do on this PR in order to get it merged?
About the `oracle-java` vs `debian-java` I am not sure how we could manage the inconsistency from the maven images repository better on our side.I did add the params to the usage on the `docker-compose` and the `openjdk` vs `jdk` naming convention for the image:
```
# Usage:
# docker-compose build oracle-java
# docker-compose run oracle-java
# Parameters:
# MAVEN: 3.8.5
# JDK: 17
image: ${ARCH}/maven:${MAVEN}-openjdk-${JDK}
```
vs
```
# Usage:
# docker-compose build debian-java
# docker-compose run debian-java
# Parameters:
# MAVEN: 3.5.4, 3.6.2
# JDK: 8, 11
image: ${ARCH}/maven:${MAVEN}-jdk-${JDK}
```
let me know if you have a better idea.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13021: ARROW-16327: [Java][CI] Add Java 17 to CI matrix for java workflows
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13021:
URL: https://github.com/apache/arrow/pull/13021#issuecomment-1116666034
Revision: c2238ba904fd78acb27e8b696992002618ceab98
Submitted crossbow builds: [ursacomputing/crossbow @ actions-2006](https://github.com/ursacomputing/crossbow/branches/all?query=actions-2006)
|Task|Status|
|----|------|
|java-jars|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-2006-github-java-jars)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13021: [WIP] ARROW-16327: [Java][CI] Add Java 17 to CI matrix for java workflows
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13021:
URL: https://github.com/apache/arrow/pull/13021#issuecomment-1111827329
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
Thanks for opening a pull request!
If this is not a [minor PR](https://github.com/apache/arrow/blob/master/CONTRIBUTING.md#Minor-Fixes). Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW
Opening JIRAs ahead of time contributes to the [Openness](http://theapacheway.com/open/#:~:text=Openness%20allows%20new%20users%20the,must%20happen%20in%20the%20open.) of the Apache Arrow project.
Then could you also rename pull request title in the following format?
ARROW-${JIRA_ID}: [${COMPONENT}] ${SUMMARY}
or
MINOR: [${COMPONENT}] ${SUMMARY}
See also:
* [Other pull requests](https://github.com/apache/arrow/pulls/)
* [Contribution Guidelines - How to contribute patches](https://arrow.apache.org/docs/developers/contributing.html#how-to-contribute-patches)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13021: [WIP] ARROW-16327: [Java][CI] Add Java 17 to CI matrix for java workflows
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13021:
URL: https://github.com/apache/arrow/pull/13021#issuecomment-1111830587
Revision: fe4354fcff315f59ef52a07b5107d1adcdb59877
Submitted crossbow builds: [ursacomputing/crossbow @ actions-1984](https://github.com/ursacomputing/crossbow/branches/all?query=actions-1984)
|Task|Status|
|----|------|
|java-jars|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1984-github-java-jars)|
|verify-rc-source-java-linux-almalinux-8-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1984-github-verify-rc-source-java-linux-almalinux-8-amd64)|
|verify-rc-source-java-linux-conda-latest-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1984-github-verify-rc-source-java-linux-conda-latest-amd64)|
|verify-rc-source-java-linux-ubuntu-18.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1984-github-verify-rc-source-java-linux-ubuntu-18.04-amd64)|
|verify-rc-source-java-linux-ubuntu-20.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1984-github-verify-rc-source-java-linux-ubuntu-20.04-amd64)|
|verify-rc-source-java-linux-ubuntu-22.04-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1984-github-verify-rc-source-java-linux-ubuntu-22.04-amd64)|
|verify-rc-source-java-macos-amd64|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1984-github-verify-rc-source-java-macos-amd64)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] kou commented on pull request #13021: ARROW-16327: [Java][CI] Add Java 17 to CI matrix for java workflows
Posted by GitBox <gi...@apache.org>.
kou commented on PR #13021:
URL: https://github.com/apache/arrow/pull/13021#issuecomment-1112626818
Could you also confirm CI failure? https://github.com/apache/arrow/runs/6218109762?check_suite_focus=true
```text
Error: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.2:compile (default-compile) on project arrow-format: Compilation failure -> [Help 1]
Error: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.2:compile (default-compile) on project arrow-plasma: Compilation failure -> [Help 1]
Error: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.2:compile (default-compile) on project arrow-memory-core: Compilation failure -> [Help 1]
```
The following log may be related:
```text
[INFO] -----------------< org.apache.arrow:arrow-memory-core >-----------------
[INFO] Building Arrow Memory - Core 8.0.0-SNAPSHOT [6/19]
[INFO] --------------------------------[ jar ]---------------------------------
Warning: *****************************************************************
Warning: * Your build is requesting parallel execution, but project *
Warning: * contains the following plugin(s) that have goals not marked *
Warning: * as @threadSafe to support parallel building. *
Warning: * While this /may/ work fine, please look for plugin updates *
Warning: * and/or request plugins be made thread-safe. *
Warning: * If reporting an issue, report it against the plugin in *
Warning: * question, not against maven-core *
Warning: *****************************************************************
Warning: The following plugins are not marked @threadSafe in Arrow Memory - Core:
Warning: org.apache.rat:apache-rat-plugin:0.13
Warning: Enable debug to see more precisely which goals are not marked @threadSafe.
Warning: *****************************************************************
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org