[jira] [Commented] (ARROW-3476) [Java] mvn test in memory fails on a big-endian platform

["ketan kunde (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16813123#comment-16813123 ] 

ketan kunde commented on ARROW-3476:
------------------------------------

Hi [~kiszk], 

I wanted some more information about this test case related to 'Arrow'

1) Which feature of spark are this test targetting to?

2) Are this test specific to little endian environment and yet to be ported for Big endian?

 

Regards

Ketan 

> [Java] mvn test in memory fails on a big-endian platform
> --------------------------------------------------------
>
>                 Key: ARROW-3476
>                 URL: https://issues.apache.org/jira/browse/ARROW-3476
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Java
>            Reporter: Kazuaki Ishizaki
>            Priority: Major
>
> Apache Arrow is becoming commonplace to exchange data among important emerging analytics frameworks such as Pandas, Numpy, and Spark.
> [IBM Z|https://en.wikipedia.org/wiki/IBM_Z] is one of platforms to process critical transactions such as bank or credit card. Users of IBM Z want to extract insights from these transactions using the emerging analytics systems on IBM Z Linux. These analytics pipelines can be also fast and effective on IBM Z Linux by using Apache Arrow on memory.
> From the technical perspective, since IBM Z Linux uses big-endian data format, it is not possible to use Apache Arrow in this pipeline. If Apache Arrow could support big-endian, the use case would be expanded.
> When I ran test case of Apache arrow on a big-endian platform (ppc64be), {{mvn test}} in memory causes a failure due to an assertion.
> In {{TestEndianess.testLittleEndian}} test suite, the assertion occurs during an allocation of a {{RootAllocator}} class.
> {code}
> $ uname -a
> Linux ppc64be.novalocal 4.5.7-300.fc24.ppc64 #1 SMP Fri Jun 10 20:29:32 UTC 2016 ppc64 ppc64 ppc64 GNU/Linux
> $ arch  
> ppc64
> $ cd java/memory
> $ mvn test
> [INFO] Scanning for projects...
> [INFO]                                                                         
> [INFO] ------------------------------------------------------------------------
> [INFO] Building Arrow Memory 0.12.0-SNAPSHOT
> [INFO] ------------------------------------------------------------------------
> [INFO] 
> ...
> [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.082 s - in org.apache.arrow.memory.TestAccountant
> [INFO] Running org.apache.arrow.memory.TestLowCostIdentityHashMap
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.012 s - in org.apache.arrow.memory.TestLowCostIdentityHashMap
> [INFO] Running org.apache.arrow.memory.TestBaseAllocator
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.746 s <<< FAILURE! - in org.apache.arrow.memory.TestEndianess
> [ERROR] testLittleEndian(org.apache.arrow.memory.TestEndianess)  Time elapsed: 0.313 s  <<< ERROR!
> java.lang.ExceptionInInitializerError
> 	at org.apache.arrow.memory.TestEndianess.testLittleEndian(TestEndianess.java:31)
> Caused by: java.lang.IllegalStateException: Arrow only runs on LittleEndian systems.
> 	at org.apache.arrow.memory.TestEndianess.testLittleEndian(TestEndianess.java:31)
> [ERROR] Tests run: 22, Failures: 0, Errors: 21, Skipped: 1, Time elapsed: 0.055 s <<< FAILURE! - in org.apache.arrow.memory.TestBaseAllocator
> ...
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)