You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Kazuaki Ishizaki (JIRA)" <ji...@apache.org> on 2018/10/16 11:53:00 UTC
[jira] [Updated] (ARROW-3476) [Java] mvn test in memory fails on a
big-endian platform
[ https://issues.apache.org/jira/browse/ARROW-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kazuaki Ishizaki updated ARROW-3476:
------------------------------------
Description:
Apache Arrow is becoming commonplace to exchange data among important emerging analytics frameworks such as Pandas, numpy, and Spark.
[IBM Z|https://en.wikipedia.org/wiki/IBM_Z] is one of platforms to process critical transactions such as bank or credit card. Users of IBM Z want to extract insights from these transactions using these emerging analytics systems on IBM Z Linux. These analytics pipelines can be also fast and effective on IBM Z Linux by using Apache Arrow on memory.
Since IBM Z Linux uses big-endian data format, it is not possible to use Apache Arrow in this pipeline. If Apache Arrow could support big-endian, the use case would be expanded.
When I ran test case of Apache arrow on a big-endian platform (ppc64be), {{mvn test}} in memory causes a failure due to an assertion.
In {{TestEndianess.testLittleEndian}} test suite, the assertion occurs during an allocation of a {{RootAllocator}} class.
{code}
$ uname -a
Linux ppc64be.novalocal 4.5.7-300.fc24.ppc64 #1 SMP Fri Jun 10 20:29:32 UTC 2016 ppc64 ppc64 ppc64 GNU/Linux
$ arch
ppc64
$ cd java/memory
$ mvn test
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Arrow Memory 0.12.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
...
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.082 s - in org.apache.arrow.memory.TestAccountant
[INFO] Running org.apache.arrow.memory.TestLowCostIdentityHashMap
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.012 s - in org.apache.arrow.memory.TestLowCostIdentityHashMap
[INFO] Running org.apache.arrow.memory.TestBaseAllocator
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.746 s <<< FAILURE! - in org.apache.arrow.memory.TestEndianess
[ERROR] testLittleEndian(org.apache.arrow.memory.TestEndianess) Time elapsed: 0.313 s <<< ERROR!
java.lang.ExceptionInInitializerError
at org.apache.arrow.memory.TestEndianess.testLittleEndian(TestEndianess.java:31)
Caused by: java.lang.IllegalStateException: Arrow only runs on LittleEndian systems.
at org.apache.arrow.memory.TestEndianess.testLittleEndian(TestEndianess.java:31)
[ERROR] Tests run: 22, Failures: 0, Errors: 21, Skipped: 1, Time elapsed: 0.055 s <<< FAILURE! - in org.apache.arrow.memory.TestBaseAllocator
...
{code}
was:
On a big-endian platform, {{mvn test}} in memory causes a failure due to an assertion.
In {{TestEndianess.testLittleEndian}} test suite, the assertion occurs during an allocation of a {{RootAllocator}} class.
{code}
$ uname -a
Linux ppc64be.novalocal 4.5.7-300.fc24.ppc64 #1 SMP Fri Jun 10 20:29:32 UTC 2016 ppc64 ppc64 ppc64 GNU/Linux
$ arch
ppc64
$ cd java/memory
$ mvn test
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Arrow Memory 0.12.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
...
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.082 s - in org.apache.arrow.memory.TestAccountant
[INFO] Running org.apache.arrow.memory.TestLowCostIdentityHashMap
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.012 s - in org.apache.arrow.memory.TestLowCostIdentityHashMap
[INFO] Running org.apache.arrow.memory.TestBaseAllocator
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.746 s <<< FAILURE! - in org.apache.arrow.memory.TestEndianess
[ERROR] testLittleEndian(org.apache.arrow.memory.TestEndianess) Time elapsed: 0.313 s <<< ERROR!
java.lang.ExceptionInInitializerError
at org.apache.arrow.memory.TestEndianess.testLittleEndian(TestEndianess.java:31)
Caused by: java.lang.IllegalStateException: Arrow only runs on LittleEndian systems.
at org.apache.arrow.memory.TestEndianess.testLittleEndian(TestEndianess.java:31)
[ERROR] Tests run: 22, Failures: 0, Errors: 21, Skipped: 1, Time elapsed: 0.055 s <<< FAILURE! - in org.apache.arrow.memory.TestBaseAllocator
...
{code}
> [Java] mvn test in memory fails on a big-endian platform
> --------------------------------------------------------
>
> Key: ARROW-3476
> URL: https://issues.apache.org/jira/browse/ARROW-3476
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java
> Reporter: Kazuaki Ishizaki
> Priority: Major
>
> Apache Arrow is becoming commonplace to exchange data among important emerging analytics frameworks such as Pandas, numpy, and Spark.
> [IBM Z|https://en.wikipedia.org/wiki/IBM_Z] is one of platforms to process critical transactions such as bank or credit card. Users of IBM Z want to extract insights from these transactions using these emerging analytics systems on IBM Z Linux. These analytics pipelines can be also fast and effective on IBM Z Linux by using Apache Arrow on memory.
> Since IBM Z Linux uses big-endian data format, it is not possible to use Apache Arrow in this pipeline. If Apache Arrow could support big-endian, the use case would be expanded.
> When I ran test case of Apache arrow on a big-endian platform (ppc64be), {{mvn test}} in memory causes a failure due to an assertion.
> In {{TestEndianess.testLittleEndian}} test suite, the assertion occurs during an allocation of a {{RootAllocator}} class.
> {code}
> $ uname -a
> Linux ppc64be.novalocal 4.5.7-300.fc24.ppc64 #1 SMP Fri Jun 10 20:29:32 UTC 2016 ppc64 ppc64 ppc64 GNU/Linux
> $ arch
> ppc64
> $ cd java/memory
> $ mvn test
> [INFO] Scanning for projects...
> [INFO]
> [INFO] ------------------------------------------------------------------------
> [INFO] Building Arrow Memory 0.12.0-SNAPSHOT
> [INFO] ------------------------------------------------------------------------
> [INFO]
> ...
> [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.082 s - in org.apache.arrow.memory.TestAccountant
> [INFO] Running org.apache.arrow.memory.TestLowCostIdentityHashMap
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.012 s - in org.apache.arrow.memory.TestLowCostIdentityHashMap
> [INFO] Running org.apache.arrow.memory.TestBaseAllocator
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.746 s <<< FAILURE! - in org.apache.arrow.memory.TestEndianess
> [ERROR] testLittleEndian(org.apache.arrow.memory.TestEndianess) Time elapsed: 0.313 s <<< ERROR!
> java.lang.ExceptionInInitializerError
> at org.apache.arrow.memory.TestEndianess.testLittleEndian(TestEndianess.java:31)
> Caused by: java.lang.IllegalStateException: Arrow only runs on LittleEndian systems.
> at org.apache.arrow.memory.TestEndianess.testLittleEndian(TestEndianess.java:31)
> [ERROR] Tests run: 22, Failures: 0, Errors: 21, Skipped: 1, Time elapsed: 0.055 s <<< FAILURE! - in org.apache.arrow.memory.TestBaseAllocator
> ...
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)