You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "David Li (Jira)" <ji...@apache.org> on 2022/08/08 19:46:00 UTC
[jira] [Resolved] (ARROW-17303) [Java] Read "arrow" (IPC and streaming) files using org.apache.arrow.dataset.jni.NativeDatasetFactory
[ https://issues.apache.org/jira/browse/ARROW-17303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Li resolved ARROW-17303.
------------------------------
Fix Version/s: 10.0.0
Resolution: Fixed
Issue resolved by pull request 13811
[https://github.com/apache/arrow/pull/13811]
> [Java] Read "arrow" (IPC and streaming) files using org.apache.arrow.dataset.jni.NativeDatasetFactory
> ------------------------------------------------------------------------------------------------------
>
> Key: ARROW-17303
> URL: https://issues.apache.org/jira/browse/ARROW-17303
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Java
> Affects Versions: 9.0.0
> Reporter: Igor Suhorukov
> Assignee: Igor Suhorukov
> Priority: Major
> Labels: pull-request-available
> Fix For: 10.0.0
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Fetch "arrow" (IPC and streaming) files using org.apache.arrow.dataset.jni.NativeDatasetFactory in Java API. This functionality required to implement Arrow file/Stream input format in my use case to process large amount of existing geospatial ARROW format data in Apache Spark data source. Optimized Analytics Package (OAP) for Spark also can leverage this feature of Dataset on JVM. They use FileSystemDatasetFactory in this [[Spark gazelle_plugin|https://github.com/oap-project/gazelle_plugin/blob/b28ec129211d4a4fb360b6b137847c36545e66f6/arrow-data-source/standard/src/main/scala/com/intel/oap/spark/sql/execution/datasources/v2/arrow/ArrowUtils.scala#L77]|https://github.com/oap-project/gazelle_plugin/blob/b28ec129211d4a4fb360b6b137847c36545e66f6/arrow-data-source/standard/src/main/scala/com/intel/oap/spark/sql/execution/datasources/v2/arrow/ArrowUtils.scala#L77] adapter
--
This message was sent by Atlassian Jira
(v8.20.10#820010)