You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (Jira)" <ji...@apache.org> on 2024/07/30 05:45:00 UTC
[jira] [Resolved] (SPARK-49055) Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment
[ https://issues.apache.org/jira/browse/SPARK-49055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun resolved SPARK-49055.
-----------------------------------
Resolution: Not A Problem
After investigating with [~LuciferYang], we concluded that there is no functional change in Apache Hadoop code (for 10 years). Only the log level is changed by HADOOP-17982. So, I closed this issue as `Not A Problem`.
> Investigate OrcEncryptionSuite UnsatisfiedLinkError on AppleSilicon MacOS environment
> -------------------------------------------------------------------------------------
>
> Key: SPARK-49055
> URL: https://issues.apache.org/jira/browse/SPARK-49055
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.0.0
> Reporter: Yang Jie
> Priority: Major
> Attachments: image-2024-07-30-12-59-02-278.png
>
>
> {code}
> git reset --hard 49b4c3bc9c09325de941dfaf41e4fd3a4a4c345f // [SPARK-45393][BUILD] Upgrade Hadoop to 3.4.0
> build/sbt clean "sql/testOnly org.apache.spark.sql.execution.datasources.orc.OrcEncryptionSuite"
> ...
> [info] OrcEncryptionSuite:
> 12:42:55.441 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
> 12:42:57.950 WARN org.apache.hadoop.crypto.OpensslCipher: Failed to load OpenSSL Cipher.
> java.lang.UnsatisfiedLinkError: 'boolean org.apache.hadoop.util.NativeCodeLoader.buildSupportsOpenssl()'
> at org.apache.hadoop.util.NativeCodeLoader.buildSupportsOpenssl(Native Method)
> at org.apache.hadoop.crypto.OpensslCipher.<clinit>(OpensslCipher.java:86)
> at org.apache.hadoop.crypto.OpensslAesCtrCryptoCodec.<init>(OpensslAesCtrCryptoCodec.java:36)
> at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
> at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
> [info] - Write and read an encrypted file (2 seconds, 486 milliseconds)
> [info] - Write and read an encrypted table (402 milliseconds)
> [info] - SPARK-35325: Write and read encrypted nested columns (299 milliseconds)
> [info] - SPARK-35992: Write and read fully-encrypted columns with default masking (623 milliseconds)
> 12:42:59.856 WARN org.apache.spark.sql.execution.datasources.orc.OrcEncryptionSuite:
> ===== POSSIBLE THREAD LEAK IN SUITE o.a.s.sql.execution.datasources.orc.OrcEncryptionSuite, threads: rpc-boss-3-1 (daemon=true), Thread-17 (daemon=true), ForkJoinPool.commonPool-worker-2 (daemon=true), shuffle-boss-6-1 (daemon=true), ForkJoinPool.commonPool-worker-1 (daemon=true), Thread-18 (daemon=true), ForkJoinPool.commonPool-worker-3 (daemon=true) =====
> [info] Run completed in 5 seconds, 291 milliseconds.
> [info] Total number of tests run: 4
> [info] Suites: completed 1, aborted 0
> [info] Tests: succeeded 4, failed 0, canceled 0, ignored 0, pending 0
> [info] All tests passed.
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org