You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "mingmxu (via GitHub)" <gi...@apache.org> on 2023/06/20 01:40:30 UTC
[GitHub] [pinot] mingmxu opened a new issue, #10945: `mvn clean install` fail on Mac with M1 chip
mingmxu opened a new issue, #10945:
URL: https://github.com/apache/pinot/issues/10945
When running ` mvn clean install ` on latest code, some unit tests(from module `pinot-segment-spi`, `pinot-segment-local`, `pinot-core`, ...), fail with error message like below:
```
java.lang.UnsatisfiedLinkError: 'long xerial.larray.impl.LArrayNative.mmap(long, int, long, long)'
at xerial.larray.impl.LArrayNative.mmap(Native Method)
at xerial.larray.mmap.MMapBuffer.<init>(MMapBuffer.java:94)
at org.apache.pinot.segment.spi.memory.PinotNativeOrderLBuffer.mapFile(PinotNativeOrderLBuffer.java:49)
```
```
[ERROR] Failures:
[ERROR] PinotDefaultByteBufferTest>PinotDataBufferTest.testConstructors:294 » UnsatisfiedLink
[ERROR] PinotLArrayByteBufferTest>PinotDataBufferTest.testConstructors:294 » UnsatisfiedLink
[ERROR] PinotLArrayByteBufferTest>PinotDataBufferInstanceTestBase.testMapFileBE:140 » UnsatisfiedLink
[ERROR] PinotLArrayByteBufferTest>PinotDataBufferInstanceTestBase.testMapFileLE:153 » UnsatisfiedLink
[ERROR] PinotLArrayByteBufferTest>PinotDataBufferInstanceTestBase.testMultipleClose:78 » UnsatisfiedLink
[ERROR] PinotLArrayByteBufferTest>PinotDataBufferTest.testPinotNativeOrderBuffer:90->PinotDataBufferTest.testOrderBuffer:79 » UnsatisfiedLink
[ERROR] PinotLArrayByteBufferTest>PinotDataBufferTest.testPinotNativeOrderBufferReadWriteFile:136->PinotDataBufferTest.testPinotBufferReadWriteFile:120 » UnsatisfiedLink
[ERROR] PinotLArrayByteBufferTest>PinotDataBufferTest.testPinotNonNativeOrderBuffer:96->PinotDataBufferTest.testOrderBuffer:79 » UnsatisfiedLink
[ERROR] PinotLArrayByteBufferTest>PinotDataBufferTest.testPinotNonNativeOrderBufferReadWriteFile:142->PinotDataBufferTest.testPinotBufferReadWriteFile:120 » UnsatisfiedLink
[ERROR] PinotLArrayByteBufferTest>PinotDataBufferTest.testViewAndToDirectByteBuffer:205 » UnsatisfiedLink
[INFO]
[ERROR] Tests run: 128, Failures: 10, Errors: 0, Skipped: 0
```
I suspect it's related with [xerial/larray](https://github.com/xerial/larray) which doesn't mention MacM1/2 support. Should create a `mac_m1_m2` profile to exclude these tests?
```
larray-mmap (MMapBuffer and LArray.mmap) uses JNI and is available for the following major CPU architectures:
Windows (32/64-bit)
Linux (i368, amd64 (Intel 64-bit), arm, armhf)
Mac OSX (Intel 64bit)
```
cc @mayankshriv
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #10945: `mvn clean install` fail on Mac with M1 chip
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #10945:
URL: https://github.com/apache/pinot/issues/10945#issuecomment-1601937276
@gortiz I remember you encountered similar issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] gortiz commented on issue #10945: `mvn clean install` fail on Mac with M1 chip
Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz commented on issue #10945:
URL: https://github.com/apache/pinot/issues/10945#issuecomment-1602077781
Hi @mingmxu. You have to options:
1. Use Rosetta to run maven, as explained in [Pinot doc](https://docs.pinot.apache.org/basics/getting-started/running-pinot-locally#m1-and-m2-mac-support). Remember that Rosetta must be enabled in your shell. You can verify if it is the case by executing `mvn --version`, which will print the `arch` it is using. It should say `amd64`. In case Rosetta is disabled, it would say `arch: "aarch64"`.
2. Run maven with Java 17, which will not use LArray but the new buffer library that doesn't use JNI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] mingmxu commented on issue #10945: `mvn clean install` fail on Mac with M1 chip
Posted by "mingmxu (via GitHub)" <gi...@apache.org>.
mingmxu commented on issue #10945:
URL: https://github.com/apache/pinot/issues/10945#issuecomment-1603147778
Thanks @gortiz for the suggestions. I don't find a `amd64` JVM, would try the approach later.
Java17 helps to block the LArray JNI failure, there's another `clp-ffi` in `pinot-core` module, cc @kirkrodrigues who might have some context.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] gortiz commented on issue #10945: `mvn clean install` fail on Mac with M1 chip
Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz commented on issue #10945:
URL: https://github.com/apache/pinot/issues/10945#issuecomment-1603801121
I usually use sdkman to install different JDKs. In case you are using it, you need to enable Rosetta in sdkman as well in order to install the amd64 JDK. This article explains how to do use sdkman to install JDKs in different architectures: https://itnext.io/how-to-install-x86-and-arm-jdks-on-the-mac-m1-apple-silicon-using-sdkman-872a5adc050d.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] kirkrodrigues commented on issue #10945: `mvn clean install` fail on Mac with M1 chip
Posted by "kirkrodrigues (via GitHub)" <gi...@apache.org>.
kirkrodrigues commented on issue #10945:
URL: https://github.com/apache/pinot/issues/10945#issuecomment-1604558647
Hi @mingmxu, clp-ffi doesn't have aarch64 support yet. I will try and add it this weekend and submit a PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org