You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/03/28 21:07:04 UTC
[GitHub] [arrow] davisusanibar opened a new pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
davisusanibar opened a new pull request #12739:
URL: https://github.com/apache/arrow/pull/12739
Getting Started Section - Landing Page for Java section
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] davisusanibar commented on a change in pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
davisusanibar commented on a change in pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#discussion_r836872197
##########
File path: docs/source/java/getstarted.rst
##########
@@ -0,0 +1,417 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _getstarted:
+
+Getting Started
+===============
+
+.. contents::
+
+This is the variety of arrow java documentation:
+
+* `Specification and protocols`_: This contains agnostic specification that is implemented in this case by arrow java modules.
+* `Supported environment`_ (like this): This contains answers for what-is-that arrow java module.
+* `Cookbook`_: This contains answers about how-to-use arrow java modules with practices examples.
+* `Development`_: This contains detailed information about what you need to consider to start with arrow java development.
+
+Arrow Java Modules
+------------------
+
+Arrow java modules is created using specification such as columnar format, off-heap
+memory, serialization and interprocess communication (IPC). Some of the java modules
+was created with their own native implementations and others through bindings.
+
+.. list-table:: Arrow Java Implementations and Bindings
+ :widths: 25 75
+ :header-rows: 1
+
+ * - Module
+ - Root Decision
+ * - Arrow Memory
+ - Native implementation.
+ * - Arrow Vector
+ - Native implementation.
+ * - Arrow Flight Grpc
+ - Native implementation.
+ * - Arrow Flight Sql
+ - Native implementation.
+ * - Arrow Algorithm
+ - Native implementation.
+ * - Arrow Compression
+ - Native implementation.
+ * - Arrow C Data Interface
+ - Native implementation.
+ * - Arrow Dataset
+ - Bindings
+ * - Arrow ORC
+ - Bindings
+ * - Arrow Gandiva
+ - Bindings
+
+Arrow java is divided in these modules to offer in-memory columnar data structures:
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 75
+ :header-rows: 1
+
+ * - Module
+ - Description
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ * - arrow-memory-unsafe
+ - Allocator and utils for allocating memory in Arrow based on sun.misc.Unsafe.
+ * - arrow-memory-netty
+ - Netty allocator and utils for allocating memory in Arrow.
+ * - arrow-vector
+ - An off-heap reference implementation for Arrow columnar data format.
+ * - arrow-tools
+ - Java applications for working with Arrow ValueVectors.
+ * - arrow-jdbc
+ - (`Experimental`) A library for converting JDBC data to Arrow data.
+ * - arrow-plasma
+ - (`Experimental`) Java client for the Plasma object store.
+ * - flight-core
+ - (`Experimental`) An RPC mechanism for transferring ValueVectors.
+ * - flight-grpc
+ - (`Experimental`) Contains utility class to expose Flight gRPC service and client
+ * - flight-sql
+ - (`Experimental`) Contains utility classes to expose Flight SQL semantics for clients and servers over Arrow Flight.
+ * - flight-integration-tests
+ - Integration tests for Flight RPC.
+ * - arrow-performance
+ - JMH Performance benchmarks for other Arrow libraries.
+ * - arrow-algorithm
+ - (`Experimental`) A collection of algorithms for working with ValueVectors.
+ * - arrow-avro
+ - (`Experimental`) A library for converting Avro data to Arrow data.
+ * - arrow-compression
+ - (`Experimental`) A library for working with the compression/decompression of Arrow data.
+ * - arrow-c-data
+ - Java implementation of C Data Interface
+ * - arrow-orc
+ - (`Experimental`) A JNI wrapper for the C++ ORC reader implementation.
+ * - arrow-gandiva
+ - Java wrappers around the native Gandiva SQL expression compiler.
+ * - arrow-dataset
+ - Java implementation of Arrow Dataset API/Framework
+
+For more detail about how to install this modules please review
+:doc:`Installing Java Modules <install>`.
+
+.. figure:: img/arrow_java_wks_deps.png
+ :scale: 75 %
+ :alt: Offer a workspace dependency view for arrow java modules
+
+ Workspace dependency view for arrow java modules.
Review comment:
Deleted
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] davisusanibar commented on a change in pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
davisusanibar commented on a change in pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#discussion_r838768695
##########
File path: docs/source/java/modules.rst
##########
@@ -0,0 +1,413 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_modules:
+
+=======
+Modules
+=======
+
+.. contents::
+
+This is the variety of arrow java documentation:
+
+* `Specification and protocols`_: This contains agnostic specification that is implemented in this case by arrow java modules.
+* `Supported environment`_ (like this): This contains answers for what-is-that arrow java module.
+* `Cookbook`_: This contains answers about how-to-use arrow java modules with practices examples.
+* `Development`_: This contains detailed information about what you need to consider to start with arrow java development.
+
+Arrow java modules is created using specification such as columnar format, off-heap
+memory, serialization and interprocess communication (IPC). Some of the java modules
+was created with their own native implementations and others through bindings.
+
+.. list-table:: Arrow Java Implementations and Bindings
+ :widths: 25 75
+ :header-rows: 1
+
+ * - Module
+ - Root Decision
+ * - Arrow Memory
+ - Native implementation.
+ * - Arrow Vector
+ - Native implementation.
+ * - Arrow Flight Grpc
+ - Native implementation.
+ * - Arrow Flight Sql
+ - Native implementation.
+ * - Arrow Algorithm
+ - Native implementation.
+ * - Arrow Compression
+ - Native implementation.
+ * - Arrow C Data Interface
+ - Native implementation.
+ * - Arrow Dataset
+ - Bindings
+ * - Arrow ORC
+ - Bindings
+ * - Arrow Gandiva
+ - Bindings
+
+Arrow java is divided in these modules to offer in-memory columnar data structures:
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 75
+ :header-rows: 1
+
+ * - Module
+ - Description
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ * - arrow-memory-unsafe
+ - Allocator and utils for allocating memory in Arrow based on sun.misc.Unsafe.
+ * - arrow-memory-netty
+ - Netty allocator and utils for allocating memory in Arrow.
+ * - arrow-vector
+ - An off-heap reference implementation for Arrow columnar data format.
+ * - arrow-tools
+ - Java applications for working with Arrow ValueVectors.
+ * - arrow-jdbc
+ - (`Experimental`) A library for converting JDBC data to Arrow data.
+ * - arrow-plasma
+ - (`Experimental`) Java client for the Plasma object store.
+ * - flight-core
+ - (`Experimental`) An RPC mechanism for transferring ValueVectors.
+ * - flight-grpc
+ - (`Experimental`) Contains utility class to expose Flight gRPC service and client
+ * - flight-sql
+ - (`Experimental`) Contains utility classes to expose Flight SQL semantics for clients and servers over Arrow Flight.
+ * - flight-integration-tests
+ - Integration tests for Flight RPC.
+ * - arrow-performance
+ - JMH Performance benchmarks for other Arrow libraries.
+ * - arrow-algorithm
+ - (`Experimental`) A collection of algorithms for working with ValueVectors.
+ * - arrow-avro
+ - (`Experimental`) A library for converting Avro data to Arrow data.
+ * - arrow-compression
+ - (`Experimental`) A library for working with the compression/decompression of Arrow data.
+ * - arrow-c-data
+ - Java implementation of C Data Interface
+ * - arrow-orc
+ - (`Experimental`) A JNI wrapper for the C++ ORC reader implementation.
+ * - arrow-gandiva
+ - Java wrappers around the native Gandiva SQL expression compiler.
+ * - arrow-dataset
+ - Java implementation of Arrow Dataset API/Framework
+
+For more detail about how to install this modules please review
+:doc:`Installing Java Modules <install>`.
+
+.. note::
+
+ Arrow java modules offer support to work data: (1) in-memory,
+ (2) at rest and (3) on-the-wire.
+
+Lets zoom-in arrow java modules to take advantage of this functionalities:
+
+Arrow Java In-Memory (The Physical Layer)
Review comment:
Changed
##########
File path: docs/source/java/modules.rst
##########
@@ -0,0 +1,413 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_modules:
+
+=======
+Modules
Review comment:
Changed
##########
File path: docs/source/java/modules.rst
##########
@@ -0,0 +1,413 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_modules:
+
+=======
+Modules
+=======
+
+.. contents::
+
+This is the variety of arrow java documentation:
+
+* `Specification and protocols`_: This contains agnostic specification that is implemented in this case by arrow java modules.
+* `Supported environment`_ (like this): This contains answers for what-is-that arrow java module.
+* `Cookbook`_: This contains answers about how-to-use arrow java modules with practices examples.
Review comment:
Changed
##########
File path: docs/source/java/modules.rst
##########
@@ -0,0 +1,413 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_modules:
+
+=======
+Modules
+=======
+
+.. contents::
+
+This is the variety of arrow java documentation:
+
+* `Specification and protocols`_: This contains agnostic specification that is implemented in this case by arrow java modules.
+* `Supported environment`_ (like this): This contains answers for what-is-that arrow java module.
+* `Cookbook`_: This contains answers about how-to-use arrow java modules with practices examples.
+* `Development`_: This contains detailed information about what you need to consider to start with arrow java development.
+
+Arrow java modules is created using specification such as columnar format, off-heap
Review comment:
Changed
##########
File path: docs/source/java/modules.rst
##########
@@ -0,0 +1,413 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_modules:
+
+=======
+Modules
+=======
+
+.. contents::
+
+This is the variety of arrow java documentation:
+
+* `Specification and protocols`_: This contains agnostic specification that is implemented in this case by arrow java modules.
+* `Supported environment`_ (like this): This contains answers for what-is-that arrow java module.
+* `Cookbook`_: This contains answers about how-to-use arrow java modules with practices examples.
+* `Development`_: This contains detailed information about what you need to consider to start with arrow java development.
+
+Arrow java modules is created using specification such as columnar format, off-heap
+memory, serialization and interprocess communication (IPC). Some of the java modules
+was created with their own native implementations and others through bindings.
+
+.. list-table:: Arrow Java Implementations and Bindings
+ :widths: 25 75
+ :header-rows: 1
+
+ * - Module
+ - Root Decision
+ * - Arrow Memory
+ - Native implementation.
+ * - Arrow Vector
+ - Native implementation.
+ * - Arrow Flight Grpc
+ - Native implementation.
+ * - Arrow Flight Sql
+ - Native implementation.
+ * - Arrow Algorithm
+ - Native implementation.
+ * - Arrow Compression
+ - Native implementation.
+ * - Arrow C Data Interface
+ - Native implementation.
+ * - Arrow Dataset
+ - Bindings
+ * - Arrow ORC
+ - Bindings
+ * - Arrow Gandiva
+ - Bindings
+
+Arrow java is divided in these modules to offer in-memory columnar data structures:
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 75
+ :header-rows: 1
+
+ * - Module
+ - Description
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ * - arrow-memory-unsafe
+ - Allocator and utils for allocating memory in Arrow based on sun.misc.Unsafe.
+ * - arrow-memory-netty
+ - Netty allocator and utils for allocating memory in Arrow.
+ * - arrow-vector
+ - An off-heap reference implementation for Arrow columnar data format.
+ * - arrow-tools
+ - Java applications for working with Arrow ValueVectors.
+ * - arrow-jdbc
+ - (`Experimental`) A library for converting JDBC data to Arrow data.
Review comment:
Deleted
##########
File path: docs/source/java/modules.rst
##########
@@ -0,0 +1,413 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_modules:
+
+=======
+Modules
+=======
+
+.. contents::
+
+This is the variety of arrow java documentation:
+
+* `Specification and protocols`_: This contains agnostic specification that is implemented in this case by arrow java modules.
+* `Supported environment`_ (like this): This contains answers for what-is-that arrow java module.
+* `Cookbook`_: This contains answers about how-to-use arrow java modules with practices examples.
+* `Development`_: This contains detailed information about what you need to consider to start with arrow java development.
+
+Arrow java modules is created using specification such as columnar format, off-heap
+memory, serialization and interprocess communication (IPC). Some of the java modules
+was created with their own native implementations and others through bindings.
+
+.. list-table:: Arrow Java Implementations and Bindings
+ :widths: 25 75
+ :header-rows: 1
+
+ * - Module
+ - Root Decision
+ * - Arrow Memory
+ - Native implementation.
+ * - Arrow Vector
+ - Native implementation.
+ * - Arrow Flight Grpc
+ - Native implementation.
+ * - Arrow Flight Sql
+ - Native implementation.
+ * - Arrow Algorithm
+ - Native implementation.
+ * - Arrow Compression
+ - Native implementation.
+ * - Arrow C Data Interface
+ - Native implementation.
+ * - Arrow Dataset
+ - Bindings
+ * - Arrow ORC
+ - Bindings
+ * - Arrow Gandiva
+ - Bindings
+
+Arrow java is divided in these modules to offer in-memory columnar data structures:
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 75
+ :header-rows: 1
+
+ * - Module
+ - Description
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ * - arrow-memory-unsafe
+ - Allocator and utils for allocating memory in Arrow based on sun.misc.Unsafe.
+ * - arrow-memory-netty
+ - Netty allocator and utils for allocating memory in Arrow.
+ * - arrow-vector
+ - An off-heap reference implementation for Arrow columnar data format.
+ * - arrow-tools
+ - Java applications for working with Arrow ValueVectors.
+ * - arrow-jdbc
+ - (`Experimental`) A library for converting JDBC data to Arrow data.
+ * - arrow-plasma
+ - (`Experimental`) Java client for the Plasma object store.
+ * - flight-core
+ - (`Experimental`) An RPC mechanism for transferring ValueVectors.
+ * - flight-grpc
+ - (`Experimental`) Contains utility class to expose Flight gRPC service and client
+ * - flight-sql
+ - (`Experimental`) Contains utility classes to expose Flight SQL semantics for clients and servers over Arrow Flight.
+ * - flight-integration-tests
+ - Integration tests for Flight RPC.
+ * - arrow-performance
+ - JMH Performance benchmarks for other Arrow libraries.
+ * - arrow-algorithm
+ - (`Experimental`) A collection of algorithms for working with ValueVectors.
+ * - arrow-avro
+ - (`Experimental`) A library for converting Avro data to Arrow data.
+ * - arrow-compression
+ - (`Experimental`) A library for working with the compression/decompression of Arrow data.
+ * - arrow-c-data
+ - Java implementation of C Data Interface
+ * - arrow-orc
+ - (`Experimental`) A JNI wrapper for the C++ ORC reader implementation.
+ * - arrow-gandiva
+ - Java wrappers around the native Gandiva SQL expression compiler.
+ * - arrow-dataset
+ - Java implementation of Arrow Dataset API/Framework
+
+For more detail about how to install this modules please review
+:doc:`Installing Java Modules <install>`.
+
+.. note::
+
+ Arrow java modules offer support to work data: (1) in-memory,
+ (2) at rest and (3) on-the-wire.
Review comment:
Updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] davisusanibar commented on a change in pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
davisusanibar commented on a change in pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#discussion_r839007685
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A VectorSchemaRoot
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+
+ System.out.println("VectorSchemaRoot: \n" + vectorSchemaRoot.contentToTSVString());
+ }
+ }
+
+Interprocess Communication (IPC)
+********************************
+
+Arrow data can be written to and read from disk, and both of these can be done in
+a streaming and/or random-access fashion depending on application requirements.
+
+**Create a IPC File or Random Access Format**
+
+Write File or Random Access Format: Write to a file a dataset with metadata
+that contains integer age and string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.ipc.ArrowFileWriter;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.io.File;
+ import java.io.FileOutputStream;
+ import java.io.IOException;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+ // Arrow Java At Rest
+ File file = new File("randon_access_to_file.arrow");
+ try (FileOutputStream fileOutputStream = new FileOutputStream(file);
+ ArrowFileWriter writer = new ArrowFileWriter(vectorSchemaRoot, null, fileOutputStream.getChannel())
+ ) {
+ writer.start();
+ writer.writeBatch();
+ writer.end();
+ System.out.println("Record batches written: " + writer.getRecordBlocks().size() + ". Number of rows written: " + vectorSchemaRoot.getRowCount());
Review comment:
Changed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] davisusanibar commented on pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
davisusanibar commented on pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#issuecomment-1081190228
> So just a quick comment: I don't think this really reflects the corresponding Python page that @amol- linked at all. The content here is mostly not related to how to actually use the Java library at all.
Ok let me think on that, one option could be to initially I just add a quick start guide of purely examples, and then, if the user like to know more about arrow java only continuing read that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] davisusanibar commented on a change in pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
davisusanibar commented on a change in pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#discussion_r840763139
##########
File path: docs/source/java/vector_schema_root.rst
##########
@@ -15,9 +15,6 @@
.. specific language governing permissions and limitations
.. under the License.
-.. default-domain:: java
-.. highlight:: java
-
Review comment:
Yes, this cause error like this: `/arrow/docs/source/java/vector_schema_root.rst:81: ERROR: Unknown interpreted text role "class".`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] davisusanibar commented on a change in pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
davisusanibar commented on a change in pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#discussion_r839006926
##########
File path: docs/source/java/examples/index.rst
##########
@@ -0,0 +1,24 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+Examples
+========
+
+.. toctree::
+ :maxdepth: 1
+
+ quickstartguide
Review comment:
Changed
##########
File path: docs/source/java/index.rst
##########
@@ -18,17 +18,9 @@
Java Implementation
===================
-This is the documentation of the Java API of Apache Arrow. For more details
-on the Arrow format and other language bindings see the :doc:`parent documentation <../index>`.
-
.. toctree::
:maxdepth: 2
- install
- memory
- vector
- vector_schema_root
- ipc
- algorithm
- dataset
+ getting_started
+ Examples <examples/index>
Review comment:
Deleted
##########
File path: docs/source/java/getting_started.rst
##########
@@ -0,0 +1,33 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+User Guide
+==========
+
+.. toctree::
Review comment:
Changed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#issuecomment-1081156702
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on a change in pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
lidavidm commented on a change in pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#discussion_r837791842
##########
File path: docs/source/java/modules.rst
##########
@@ -0,0 +1,413 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_modules:
+
+=======
+Modules
+=======
+
+.. contents::
+
+This is the variety of arrow java documentation:
+
+* `Specification and protocols`_: This contains agnostic specification that is implemented in this case by arrow java modules.
+* `Supported environment`_ (like this): This contains answers for what-is-that arrow java module.
+* `Cookbook`_: This contains answers about how-to-use arrow java modules with practices examples.
+* `Development`_: This contains detailed information about what you need to consider to start with arrow java development.
+
+Arrow java modules is created using specification such as columnar format, off-heap
+memory, serialization and interprocess communication (IPC). Some of the java modules
+was created with their own native implementations and others through bindings.
+
+.. list-table:: Arrow Java Implementations and Bindings
+ :widths: 25 75
+ :header-rows: 1
+
+ * - Module
+ - Root Decision
+ * - Arrow Memory
+ - Native implementation.
+ * - Arrow Vector
+ - Native implementation.
+ * - Arrow Flight Grpc
+ - Native implementation.
+ * - Arrow Flight Sql
+ - Native implementation.
+ * - Arrow Algorithm
+ - Native implementation.
+ * - Arrow Compression
+ - Native implementation.
+ * - Arrow C Data Interface
+ - Native implementation.
+ * - Arrow Dataset
+ - Bindings
+ * - Arrow ORC
+ - Bindings
+ * - Arrow Gandiva
+ - Bindings
+
+Arrow java is divided in these modules to offer in-memory columnar data structures:
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 75
+ :header-rows: 1
+
+ * - Module
+ - Description
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ * - arrow-memory-unsafe
+ - Allocator and utils for allocating memory in Arrow based on sun.misc.Unsafe.
+ * - arrow-memory-netty
+ - Netty allocator and utils for allocating memory in Arrow.
+ * - arrow-vector
+ - An off-heap reference implementation for Arrow columnar data format.
+ * - arrow-tools
+ - Java applications for working with Arrow ValueVectors.
+ * - arrow-jdbc
+ - (`Experimental`) A library for converting JDBC data to Arrow data.
Review comment:
We don't need backticks around 'experimental', and we can consolidate these two tables (if you decide to keep them as tables)
##########
File path: docs/source/java/modules.rst
##########
@@ -0,0 +1,413 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_modules:
+
+=======
+Modules
+=======
+
+.. contents::
+
+This is the variety of arrow java documentation:
+
+* `Specification and protocols`_: This contains agnostic specification that is implemented in this case by arrow java modules.
+* `Supported environment`_ (like this): This contains answers for what-is-that arrow java module.
+* `Cookbook`_: This contains answers about how-to-use arrow java modules with practices examples.
Review comment:
IMO, we don't need most of these links (they're part of the sidebar already). But we can link to the cookbook in the actual getting started page.
##########
File path: docs/source/java/modules.rst
##########
@@ -0,0 +1,413 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_modules:
+
+=======
+Modules
+=======
+
+.. contents::
+
+This is the variety of arrow java documentation:
+
+* `Specification and protocols`_: This contains agnostic specification that is implemented in this case by arrow java modules.
+* `Supported environment`_ (like this): This contains answers for what-is-that arrow java module.
+* `Cookbook`_: This contains answers about how-to-use arrow java modules with practices examples.
+* `Development`_: This contains detailed information about what you need to consider to start with arrow java development.
+
+Arrow java modules is created using specification such as columnar format, off-heap
Review comment:
nit, but please be sure to capitalize Java.
##########
File path: docs/source/java/modules.rst
##########
@@ -0,0 +1,413 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_modules:
+
+=======
+Modules
+=======
+
+.. contents::
+
+This is the variety of arrow java documentation:
+
+* `Specification and protocols`_: This contains agnostic specification that is implemented in this case by arrow java modules.
+* `Supported environment`_ (like this): This contains answers for what-is-that arrow java module.
+* `Cookbook`_: This contains answers about how-to-use arrow java modules with practices examples.
+* `Development`_: This contains detailed information about what you need to consider to start with arrow java development.
+
+Arrow java modules is created using specification such as columnar format, off-heap
+memory, serialization and interprocess communication (IPC). Some of the java modules
+was created with their own native implementations and others through bindings.
+
+.. list-table:: Arrow Java Implementations and Bindings
+ :widths: 25 75
+ :header-rows: 1
+
+ * - Module
+ - Root Decision
+ * - Arrow Memory
+ - Native implementation.
+ * - Arrow Vector
+ - Native implementation.
+ * - Arrow Flight Grpc
+ - Native implementation.
+ * - Arrow Flight Sql
+ - Native implementation.
+ * - Arrow Algorithm
+ - Native implementation.
+ * - Arrow Compression
+ - Native implementation.
+ * - Arrow C Data Interface
+ - Native implementation.
+ * - Arrow Dataset
+ - Bindings
+ * - Arrow ORC
+ - Bindings
+ * - Arrow Gandiva
+ - Bindings
+
+Arrow java is divided in these modules to offer in-memory columnar data structures:
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 75
+ :header-rows: 1
+
+ * - Module
+ - Description
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ * - arrow-memory-unsafe
+ - Allocator and utils for allocating memory in Arrow based on sun.misc.Unsafe.
+ * - arrow-memory-netty
+ - Netty allocator and utils for allocating memory in Arrow.
+ * - arrow-vector
+ - An off-heap reference implementation for Arrow columnar data format.
+ * - arrow-tools
+ - Java applications for working with Arrow ValueVectors.
+ * - arrow-jdbc
+ - (`Experimental`) A library for converting JDBC data to Arrow data.
+ * - arrow-plasma
+ - (`Experimental`) Java client for the Plasma object store.
+ * - flight-core
+ - (`Experimental`) An RPC mechanism for transferring ValueVectors.
+ * - flight-grpc
+ - (`Experimental`) Contains utility class to expose Flight gRPC service and client
+ * - flight-sql
+ - (`Experimental`) Contains utility classes to expose Flight SQL semantics for clients and servers over Arrow Flight.
+ * - flight-integration-tests
+ - Integration tests for Flight RPC.
+ * - arrow-performance
+ - JMH Performance benchmarks for other Arrow libraries.
+ * - arrow-algorithm
+ - (`Experimental`) A collection of algorithms for working with ValueVectors.
+ * - arrow-avro
+ - (`Experimental`) A library for converting Avro data to Arrow data.
+ * - arrow-compression
+ - (`Experimental`) A library for working with the compression/decompression of Arrow data.
+ * - arrow-c-data
+ - Java implementation of C Data Interface
+ * - arrow-orc
+ - (`Experimental`) A JNI wrapper for the C++ ORC reader implementation.
+ * - arrow-gandiva
+ - Java wrappers around the native Gandiva SQL expression compiler.
+ * - arrow-dataset
+ - Java implementation of Arrow Dataset API/Framework
+
+For more detail about how to install this modules please review
+:doc:`Installing Java Modules <install>`.
+
+.. note::
+
+ Arrow java modules offer support to work data: (1) in-memory,
+ (2) at rest and (3) on-the-wire.
Review comment:
I'm not sure this is useful as a note. This can be a brief sentence in either the modules overview or the getting started guide.
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,482 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_quickstartguide:
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow java manage data in Vector Schema Root (somewhat analogous to tables and record
+batches in the other Arrow implementations). Before to create a Vector Schema Root let's
+define another topics neededs for that purpose.
+
+Create A Value Vector
+*********************
+
+It's called `array` in the columnar format specification. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A Vector Schema Root
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+
+ System.out.println("Vector Schema Root: \n" + vectorSchemaRoot.contentToTSVString());
+ }
+ }
+
+Create a IPC File or Random Access Format
+*****************************************
+
+The Arrow Interprocess Communication (IPC) format defines two types of binary
+formats for serializing Arrow data: the streaming format and the file format
+(or random access format). Such files can be directly memory-mapped when read.
+
+**Write File or Random Access Format**: Write to a file a dataset with metadata
+that contains integer age and string names of data.
Review comment:
I think we can be briefer here and just say that Arrow data can be written to and read from disk, and both of these can be done in a streaming and/or random-access fashion depending on application requirements.
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,482 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_quickstartguide:
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow java manage data in Vector Schema Root (somewhat analogous to tables and record
+batches in the other Arrow implementations). Before to create a Vector Schema Root let's
+define another topics neededs for that purpose.
+
+Create A Value Vector
+*********************
+
+It's called `array` in the columnar format specification. Represent a one-dimensional
Review comment:
reST uses double backticks. But usually for this we should use quotes. `Also known as "arrays" in the columnar format.`
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,482 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_quickstartguide:
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow java manage data in Vector Schema Root (somewhat analogous to tables and record
+batches in the other Arrow implementations). Before to create a Vector Schema Root let's
Review comment:
We should probably just consistently use "VectorSchemaRoot"
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,482 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_quickstartguide:
Review comment:
note it's not too useful to have this anchor, 1) we don't use it and 2) for linking to a file, instead of a subheading, just use `:doc:`
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,482 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_quickstartguide:
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow java manage data in Vector Schema Root (somewhat analogous to tables and record
+batches in the other Arrow implementations). Before to create a Vector Schema Root let's
Review comment:
And we can introduce things more naturally, something like: Arrow provides several building blocks; ValueVectors are sequences of typed values; data types, fields, and schemas describe the types of values, and the types of columns in tabular data; VectorSchemaRoot represents tabular data. Arrow also provides readers and writers for loading data from and persisting data to storage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] davisusanibar commented on a change in pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
davisusanibar commented on a change in pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#discussion_r838769384
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,482 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_quickstartguide:
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow java manage data in Vector Schema Root (somewhat analogous to tables and record
+batches in the other Arrow implementations). Before to create a Vector Schema Root let's
Review comment:
Thanks, updated
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,482 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_quickstartguide:
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow java manage data in Vector Schema Root (somewhat analogous to tables and record
+batches in the other Arrow implementations). Before to create a Vector Schema Root let's
+define another topics neededs for that purpose.
+
+Create A Value Vector
+*********************
+
+It's called `array` in the columnar format specification. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A Vector Schema Root
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+
+ System.out.println("Vector Schema Root: \n" + vectorSchemaRoot.contentToTSVString());
+ }
+ }
+
+Create a IPC File or Random Access Format
+*****************************************
+
+The Arrow Interprocess Communication (IPC) format defines two types of binary
+formats for serializing Arrow data: the streaming format and the file format
+(or random access format). Such files can be directly memory-mapped when read.
+
+**Write File or Random Access Format**: Write to a file a dataset with metadata
+that contains integer age and string names of data.
Review comment:
Added
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,482 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_quickstartguide:
Review comment:
Deleted
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,482 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_quickstartguide:
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow java manage data in Vector Schema Root (somewhat analogous to tables and record
+batches in the other Arrow implementations). Before to create a Vector Schema Root let's
+define another topics neededs for that purpose.
+
+Create A Value Vector
+*********************
+
+It's called `array` in the columnar format specification. Represent a one-dimensional
Review comment:
Added
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] davisusanibar commented on pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
davisusanibar commented on pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#issuecomment-1081151222
@lidavidm please if you could help me wit this review, thank you in advance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] davisusanibar commented on pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
davisusanibar commented on pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#issuecomment-1081234137
> > So just a quick comment: I don't think this really reflects the corresponding Python page that @amol- linked at all. The content here is mostly not related to how to actually use the Java library at all.
>
> Ok let me think on that, one option could be to initially I just add a quick start guide of purely examples, and then, if the user like to know more about arrow java only continuing read that.
Just changed. Divided getting started guide in quick start guide (purely examples) and modules (more detail about what part is covered by java modules). Please let me know if this is ok.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on a change in pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
lidavidm commented on a change in pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#discussion_r840742654
##########
File path: docs/source/java/vector_schema_root.rst
##########
@@ -15,9 +15,6 @@
.. specific language governing permissions and limitations
.. under the License.
-.. default-domain:: java
-.. highlight:: java
-
Review comment:
Did you mean to remove this?
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,203 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+The Apache Arrow Java modules implement various specifications including the
+columnar format and IPC. Most modules are native Java implementations,
+but some modules are JNI bindings to the C++ library.
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 50 25
+ :header-rows: 1
+
+ * - Module
+ - Description
+ - Implementation
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ - Native
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ - Native
+ * - arrow-memory-unsafe
+ - Memory management implementation based on sun.misc.Unsafe.
+ - Native
+ * - arrow-memory-netty
+ - Memory management implementation based on Netty.
+ - Native
+ * - arrow-vector
+ - An off-heap reference implementation for Arrow columnar data format.
+ - Native
+ * - arrow-tools
+ - Java applications for working with Arrow ValueVectors.
+ - Native
+ * - arrow-jdbc
+ - (Experimental) A library for converting JDBC data to Arrow data.
+ - Native
+ * - arrow-plasma
+ - (Experimental) Java client for the Plasma object store.
+ - Native
+ * - flight-core
+ - (Experimental) An RPC mechanism for transferring ValueVectors.
+ - Native
+ * - flight-grpc
+ - (Experimental) Contains utility class to expose Flight gRPC service and client.
Review comment:
```suggestion
- (Experimental) Contains utility classes to expose Flight gRPC service and client.
```
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,203 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+The Apache Arrow Java modules implement various specifications including the
+columnar format and IPC. Most modules are native Java implementations,
+but some modules are JNI bindings to the C++ library.
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 50 25
+ :header-rows: 1
+
+ * - Module
+ - Description
+ - Implementation
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ - Native
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ - Native
+ * - arrow-memory-unsafe
+ - Memory management implementation based on sun.misc.Unsafe.
+ - Native
+ * - arrow-memory-netty
+ - Memory management implementation based on Netty.
+ - Native
+ * - arrow-vector
+ - An off-heap reference implementation for Arrow columnar data format.
+ - Native
+ * - arrow-tools
+ - Java applications for working with Arrow ValueVectors.
+ - Native
+ * - arrow-jdbc
+ - (Experimental) A library for converting JDBC data to Arrow data.
+ - Native
+ * - arrow-plasma
+ - (Experimental) Java client for the Plasma object store.
+ - Native
+ * - flight-core
+ - (Experimental) An RPC mechanism for transferring ValueVectors.
+ - Native
+ * - flight-grpc
+ - (Experimental) Contains utility class to expose Flight gRPC service and client.
+ - Native
+ * - flight-sql
+ - (Experimental) Contains utility classes to expose Flight SQL semantics for clients and servers over Arrow Flight.
Review comment:
```suggestion
- (Experimental) Implements SQL semantics for clients and servers over Arrow Flight.
```
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,203 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+The Apache Arrow Java modules implement various specifications including the
+columnar format and IPC. Most modules are native Java implementations,
+but some modules are JNI bindings to the C++ library.
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 50 25
+ :header-rows: 1
+
+ * - Module
+ - Description
+ - Implementation
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ - Native
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ - Native
+ * - arrow-memory-unsafe
+ - Memory management implementation based on sun.misc.Unsafe.
+ - Native
+ * - arrow-memory-netty
+ - Memory management implementation based on Netty.
+ - Native
+ * - arrow-vector
+ - An off-heap reference implementation for Arrow columnar data format.
+ - Native
+ * - arrow-tools
+ - Java applications for working with Arrow ValueVectors.
+ - Native
+ * - arrow-jdbc
+ - (Experimental) A library for converting JDBC data to Arrow data.
+ - Native
+ * - arrow-plasma
+ - (Experimental) Java client for the Plasma object store.
+ - Native
+ * - flight-core
+ - (Experimental) An RPC mechanism for transferring ValueVectors.
+ - Native
+ * - flight-grpc
+ - (Experimental) Contains utility class to expose Flight gRPC service and client.
+ - Native
+ * - flight-sql
+ - (Experimental) Contains utility classes to expose Flight SQL semantics for clients and servers over Arrow Flight.
+ - Native
+ * - flight-integration-tests
+ - Integration tests for Flight RPC.
+ - Native
+ * - arrow-performance
+ - JMH benchmarks for the Arrow libraries.
+ - Native
+ * - arrow-algorithm
+ - (Experimental) A collection of algorithms for working with ValueVectors.
+ - Native
+ * - arrow-avro
+ - (Experimental) A library for converting Avro data to Arrow data.
+ - Native
+ * - arrow-compression
+ - (Experimental) A library for working with compression/decompression of Arrow data.
+ - Native
+ * - arrow-c-data
+ - Java implementation of `C Data Interface`_
+ - JNI
+ * - arrow-orc
+ - (Experimental) A JNI wrapper for the C++ ORC reader implementation.
+ - JNI
+ * - arrow-gandiva
+ - Java wrappers around the native Gandiva SQL expression compiler.
+ - JNI
+ * - arrow-dataset
+ - Java bindings to the Arrow Datasets library.
+ - JNI
+
+Arrow Java modules support working with data (1) in-memory, (2) at rest, and (3) on-the-wire.
+
+For more detail about how to install this modules please review `Installing Java Modules`.
+
+Arrow Java In-Memory (The Physical Layer)
Review comment:
I think we can just keep the table and we don't need the rest of this file. Most of this content is already present in other documentation.
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,203 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+The Apache Arrow Java modules implement various specifications including the
+columnar format and IPC. Most modules are native Java implementations,
+but some modules are JNI bindings to the C++ library.
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 50 25
+ :header-rows: 1
+
+ * - Module
+ - Description
+ - Implementation
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ - Native
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ - Native
+ * - arrow-memory-unsafe
+ - Memory management implementation based on sun.misc.Unsafe.
+ - Native
+ * - arrow-memory-netty
+ - Memory management implementation based on Netty.
+ - Native
+ * - arrow-vector
+ - An off-heap reference implementation for Arrow columnar data format.
+ - Native
+ * - arrow-tools
+ - Java applications for working with Arrow ValueVectors.
+ - Native
+ * - arrow-jdbc
+ - (Experimental) A library for converting JDBC data to Arrow data.
+ - Native
+ * - arrow-plasma
+ - (Experimental) Java client for the Plasma object store.
+ - Native
+ * - flight-core
+ - (Experimental) An RPC mechanism for transferring ValueVectors.
+ - Native
+ * - flight-grpc
+ - (Experimental) Contains utility class to expose Flight gRPC service and client.
+ - Native
+ * - flight-sql
+ - (Experimental) Contains utility classes to expose Flight SQL semantics for clients and servers over Arrow Flight.
+ - Native
+ * - flight-integration-tests
+ - Integration tests for Flight RPC.
+ - Native
+ * - arrow-performance
+ - JMH benchmarks for the Arrow libraries.
+ - Native
+ * - arrow-algorithm
+ - (Experimental) A collection of algorithms for working with ValueVectors.
+ - Native
+ * - arrow-avro
+ - (Experimental) A library for converting Avro data to Arrow data.
+ - Native
+ * - arrow-compression
+ - (Experimental) A library for working with compression/decompression of Arrow data.
+ - Native
+ * - arrow-c-data
+ - Java implementation of `C Data Interface`_
+ - JNI
+ * - arrow-orc
+ - (Experimental) A JNI wrapper for the C++ ORC reader implementation.
+ - JNI
+ * - arrow-gandiva
+ - Java wrappers around the native Gandiva SQL expression compiler.
+ - JNI
+ * - arrow-dataset
+ - Java bindings to the Arrow Datasets library.
+ - JNI
+
+Arrow Java modules support working with data (1) in-memory, (2) at rest, and (3) on-the-wire.
+
+For more detail about how to install this modules please review `Installing Java Modules`.
Review comment:
I don't think we need this, don't we have install instructions as one of the sidebar items already?
##########
File path: docs/source/java/index.rst
##########
@@ -18,12 +18,11 @@
Java Implementation
===================
-This is the documentation of the Java API of Apache Arrow. For more details
-on the Arrow format and other language bindings see the :doc:`parent documentation <../index>`.
-
Review comment:
Did you mean to remove this?
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,311 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks. Data types describe the types of values;
+ValueVectors are sequences of typed values; fields describe the types of columns in
+tabular data; schemas describe a sequence of columns in tabular data, and
+VectorSchemaRoot represents tabular data. Arrow also provides readers and
+writers for loading data from and persisting data to storage.
+
+Create a ValueVector
+********************
+
+**ValueVectors** represent a sequence of values of the same type.
+They are also known as "arrays" in the columnar format.
+
+Example: create a vector of 32-bit integers representing ``[1, null, 2]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", allocator);
+ ){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [1, null, 2]
+
+
+Example: create a vector of UTF-8 encoded strings representing ``["one", "two", "three"]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", allocator);
+ ){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [one, two, three]
+
+Create a Field
+**************
+
+**Fields** are used to denote the particular columns of tabular data.
+They consist of a name, a data type, a flag indicating whether the column can have null values,
+and optional key-value metadata.
+
+Example: create a field named "document" of string type:
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata),
+ /*children*/ null);
+ System.out.println("Field created: " + document + ", Metadata: " + document.getMetadata());
+
+.. code-block:: shell
+
+ Field created: document: Utf8, Metadata: {A=Id card, B=Passport, C=Visa}
+
+Create a Schema
+***************
+
+**Schemas** hold a sequence of fields together with some optional metadata.
+
+Example: Create a schema describing datasets with two columns:
+an int32 column "A" and a UTF8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), /*children*/ null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), /*children*/ null);
+ Schema schema = new Schema(asList(a, b), metadata);
+ System.out.println("Schema created: " + schema);
+
+.. code-block:: shell
+
+ Schema created: Schema<A: Int(32, true), B: Utf8>(metadata: {K1=V1, K2=V2})
+
+Create a VectorSchemaRoot
+*************************
+
+A **VectorSchemaRoot** combines ValueVectors with a Schema to represent tabular data.
+
+Example: Create a dataset of names (strings) and ages (32-bit signed integers).
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Field age = new Field("age",
+ FieldType.nullable(new ArrowType.Int(32, true)),
+ /*children*/null
+ );
+ Field name = new Field("name",
+ FieldType.nullable(new ArrowType.Utf8()),
+ /*children*/null
+ );
+ Schema schema = new Schema(asList(age, name), /*metadata*/ null);
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator);
+ IntVector ageVector = (IntVector) root.getVector("age");
+ VarCharVector nameVector = (VarCharVector) root.getVector("name");
+ ){
+ root.setRowCount(3);
+ ageVector.allocateNew(3);
+ ageVector.set(0, 10);
+ ageVector.set(1, 20);
+ ageVector.set(2, 30);
+ nameVector.allocateNew(3);
+ nameVector.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ nameVector.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ nameVector.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+ System.out.println("VectorSchemaRoot created: \n" + root.contentToTSVString());
+ }
+
+.. code-block:: shell
+
+ VectorSchemaRoot created:
+ age name
+ 10 Dave
+ 20 Peter
+ 30 Mary
+
+
+Interprocess Communication (IPC)
+********************************
+
+Arrow data can be written to and read from disk, and both of these can be done in
+a streaming and/or random-access fashion depending on application requirements.
+
+**Write data to an arrow file**
+
+Example: Write the dataset from the previous example to an Arrow random-access file.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.ipc.ArrowFileWriter;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.io.File;
+ import java.io.FileOutputStream;
+ import java.io.IOException;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Field age = new Field("age",
+ FieldType.nullable(new ArrowType.Int(32, true)),
+ /*children*/ null);
+ Field name = new Field("name",
+ FieldType.nullable(new ArrowType.Utf8()),
+ /*children*/ null);
+ Schema schema = new Schema(asList(age, name));
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator);
+ IntVector ageVector = (IntVector) root.getVector("age");
+ VarCharVector nameVector = (VarCharVector) root.getVector("name");
+ ){
+ ageVector.allocateNew(3);
+ ageVector.set(0, 10);
+ ageVector.set(1, 20);
+ ageVector.set(2, 30);
+ nameVector.allocateNew(3);
+ nameVector.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ nameVector.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ nameVector.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+ root.setRowCount(3);
+ File file = new File("random_access_file.arrow");
+ try (
+ FileOutputStream fileOutputStream = new FileOutputStream(file);
+ ArrowFileWriter writer = new ArrowFileWriter(root, /*provider*/ null, fileOutputStream.getChannel());
+ ) {
+ writer.start();
+ writer.writeBatch();
+ writer.end();
+ System.out.println("Record batches written: " + writer.getRecordBlocks().size()
+ + ". Number of rows written: " + root.getRowCount());
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+ }
+
+.. code-block:: shell
+
+ Record batches written: 1. Number of rows written: 3
+
+**Read data from an arrow file**
+
+Example: Read the dataset from the previous example to an Arrow random-access file.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.ipc.ArrowFileReader;
+ import org.apache.arrow.vector.ipc.message.ArrowBlock;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import java.io.File;
+ import java.io.FileInputStream;
+ import java.io.FileOutputStream;
+ import java.io.IOException;
+
+ try(
+ BufferAllocator allocator = new RootAllocator(Long.MAX_VALUE);
+ FileInputStream fileInputStream = new FileInputStream(new File("random_access_file.arrow"));
+ ArrowFileReader reader = new ArrowFileReader(fileInputStream.getChannel(), allocator);
+ ){
+ System.out.println("Record batches in file: " + reader.getRecordBlocks().size());
+ for (ArrowBlock arrowBlock : reader.getRecordBlocks()) {
+ reader.loadRecordBatch(arrowBlock);
+ VectorSchemaRoot root = reader.getVectorSchemaRoot();
+ System.out.println("VectorSchemaRoot read: \n" + root.contentToTSVString());
+ }
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+
+.. code-block:: shell
+
+ VectorSchemaRoot read:
Review comment:
This output is missing the "Record batches in file" line
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on a change in pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
lidavidm commented on a change in pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#discussion_r839557302
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,325 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks. Data types describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data, and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create a ValueVector
+*********************
+
+**ValueVectors** represent a sequence of values of the same type.
+They are also known as "arrays" in the columnar format.
+
+Example: create a vector of 32-bit integers representing ``[1, null, 2]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", allocator);
+ ){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [1, null, 2]
+
+
+Example: create a vector of UTF-8 encoded strings representing ``["one", "two", "three"]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", allocator);
+ ){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [one, two, three]
+
+Create a Field
+**************
+
+**Fields** are used to denote the particular columns of tabular data.
+They consist of a name, a data type, a flag indicating whether the column can have null values,
+and optional key-value metadata.
+
+Example: create a field named "document" of string type:
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata),
+ /*children*/ null);
+ System.out.println("Field created: " + document + ", Metadata: " + document.getMetadata());
+
+.. code-block:: shell
+
+ Field created: document: Utf8, Metadata: {A=Id card, B=Passport, C=Visa}
+
+Create a Schema
+***************
+
+**Schema** holds a sequence of fields together with some optional metadata.
Review comment:
```suggestion
**Schemas** hold a sequence of fields together with some optional metadata.
```
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,325 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks. Data types describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
Review comment:
We don't need to put any of these terms in a code font
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,325 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks. Data types describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data, and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create a ValueVector
+*********************
+
+**ValueVectors** represent a sequence of values of the same type.
+They are also known as "arrays" in the columnar format.
+
+Example: create a vector of 32-bit integers representing ``[1, null, 2]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", allocator);
+ ){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [1, null, 2]
+
+
+Example: create a vector of UTF-8 encoded strings representing ``["one", "two", "three"]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", allocator);
+ ){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [one, two, three]
+
+Create a Field
+**************
+
+**Fields** are used to denote the particular columns of tabular data.
+They consist of a name, a data type, a flag indicating whether the column can have null values,
+and optional key-value metadata.
+
+Example: create a field named "document" of string type:
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata),
+ /*children*/ null);
+ System.out.println("Field created: " + document + ", Metadata: " + document.getMetadata());
+
+.. code-block:: shell
+
+ Field created: document: Utf8, Metadata: {A=Id card, B=Passport, C=Visa}
+
+Create a Schema
+***************
+
+**Schema** holds a sequence of fields together with some optional metadata.
+
+Example: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), /*children*/ null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), /*children*/ null);
+ Schema schema = new Schema(asList(a, b), metadata);
+ System.out.println("Schema created: " + schema);
+
+.. code-block:: shell
+
+ Schema created: Schema<A: Int(32, true), B: Utf8>(metadata: {K1=V1, K2=V2})
+
+Create a VectorSchemaRoot
+*************************
+
+**VectorSchemaRoot** combines ValueVectors with a Schema to represent tabular data.
+
+Example: Create a dataset with metadata that contains integer age and
+string names of data.
Review comment:
```suggestion
Example: Create a dataset of names (strings) and ages (32-bit signed integers).
```
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,325 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks. Data types describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data, and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create a ValueVector
+*********************
+
+**ValueVectors** represent a sequence of values of the same type.
+They are also known as "arrays" in the columnar format.
+
+Example: create a vector of 32-bit integers representing ``[1, null, 2]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", allocator);
+ ){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [1, null, 2]
+
+
+Example: create a vector of UTF-8 encoded strings representing ``["one", "two", "three"]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", allocator);
+ ){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [one, two, three]
+
+Create a Field
+**************
+
+**Fields** are used to denote the particular columns of tabular data.
+They consist of a name, a data type, a flag indicating whether the column can have null values,
+and optional key-value metadata.
+
+Example: create a field named "document" of string type:
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata),
+ /*children*/ null);
+ System.out.println("Field created: " + document + ", Metadata: " + document.getMetadata());
+
+.. code-block:: shell
+
+ Field created: document: Utf8, Metadata: {A=Id card, B=Passport, C=Visa}
+
+Create a Schema
+***************
+
+**Schema** holds a sequence of fields together with some optional metadata.
+
+Example: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), /*children*/ null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), /*children*/ null);
+ Schema schema = new Schema(asList(a, b), metadata);
+ System.out.println("Schema created: " + schema);
+
+.. code-block:: shell
+
+ Schema created: Schema<A: Int(32, true), B: Utf8>(metadata: {K1=V1, K2=V2})
+
+Create a VectorSchemaRoot
+*************************
+
+**VectorSchemaRoot** combines ValueVectors with a Schema to represent tabular data.
+
+Example: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Field a = new Field("age",
+ FieldType.nullable(new ArrowType.Int(32, true)),
+ /*children*/null
+ );
+ Field b = new Field("name",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, /*metadata*/ null),
Review comment:
Can we use `FieldType.nullable` here?
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,325 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks. Data types describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data, and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create a ValueVector
+*********************
+
+**ValueVectors** represent a sequence of values of the same type.
+They are also known as "arrays" in the columnar format.
+
+Example: create a vector of 32-bit integers representing ``[1, null, 2]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", allocator);
+ ){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [1, null, 2]
+
+
+Example: create a vector of UTF-8 encoded strings representing ``["one", "two", "three"]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", allocator);
+ ){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [one, two, three]
+
+Create a Field
+**************
+
+**Fields** are used to denote the particular columns of tabular data.
+They consist of a name, a data type, a flag indicating whether the column can have null values,
+and optional key-value metadata.
+
+Example: create a field named "document" of string type:
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata),
+ /*children*/ null);
+ System.out.println("Field created: " + document + ", Metadata: " + document.getMetadata());
+
+.. code-block:: shell
+
+ Field created: document: Utf8, Metadata: {A=Id card, B=Passport, C=Visa}
+
+Create a Schema
+***************
+
+**Schema** holds a sequence of fields together with some optional metadata.
+
+Example: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), /*children*/ null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), /*children*/ null);
+ Schema schema = new Schema(asList(a, b), metadata);
+ System.out.println("Schema created: " + schema);
+
+.. code-block:: shell
+
+ Schema created: Schema<A: Int(32, true), B: Utf8>(metadata: {K1=V1, K2=V2})
+
+Create a VectorSchemaRoot
+*************************
+
+**VectorSchemaRoot** combines ValueVectors with a Schema to represent tabular data.
+
+Example: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Field a = new Field("age",
+ FieldType.nullable(new ArrowType.Int(32, true)),
+ /*children*/null
+ );
+ Field b = new Field("name",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, /*metadata*/ null),
+ /*children*/null
+ );
+ Schema schema = new Schema(asList(a, b), /*metadata*/ null);
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator);
+ IntVector intVectorA = (IntVector) root.getVector("age");
+ VarCharVector varCharVectorB = (VarCharVector) root.getVector("name");
+ ){
+ root.setRowCount(3);
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+ System.out.println("VectorSchemaRoot created: \n" + root.contentToTSVString());
+ }
+
+.. code-block:: shell
+
+ VectorSchemaRoot created:
+ age name
+ 10 Dave
+ 20 Peter
+ 30 Mary
+
+
+Interprocess Communication (IPC)
+********************************
+
+Arrow data can be written to and read from disk, and both of these can be done in
+a streaming and/or random-access fashion depending on application requirements.
+
+**Write data to an arrow file**
+
+Example: Write the dataset from the previous example to an Arrow random-access file.
+
+.. code-block:: Java
+
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.ipc.ArrowFileWriter;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.io.File;
+ import java.io.FileOutputStream;
+ import java.io.IOException;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
Review comment:
IMO, we can remove the metadata here to keep this example more manageable.
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,325 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks. Data types describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data, and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create a ValueVector
+*********************
+
+**ValueVectors** represent a sequence of values of the same type.
+They are also known as "arrays" in the columnar format.
+
+Example: create a vector of 32-bit integers representing ``[1, null, 2]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", allocator);
+ ){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [1, null, 2]
+
+
+Example: create a vector of UTF-8 encoded strings representing ``["one", "two", "three"]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", allocator);
+ ){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [one, two, three]
+
+Create a Field
+**************
+
+**Fields** are used to denote the particular columns of tabular data.
+They consist of a name, a data type, a flag indicating whether the column can have null values,
+and optional key-value metadata.
+
+Example: create a field named "document" of string type:
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata),
+ /*children*/ null);
+ System.out.println("Field created: " + document + ", Metadata: " + document.getMetadata());
+
+.. code-block:: shell
+
+ Field created: document: Utf8, Metadata: {A=Id card, B=Passport, C=Visa}
+
+Create a Schema
+***************
+
+**Schema** holds a sequence of fields together with some optional metadata.
+
+Example: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), /*children*/ null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), /*children*/ null);
+ Schema schema = new Schema(asList(a, b), metadata);
+ System.out.println("Schema created: " + schema);
+
+.. code-block:: shell
+
+ Schema created: Schema<A: Int(32, true), B: Utf8>(metadata: {K1=V1, K2=V2})
+
+Create a VectorSchemaRoot
+*************************
+
+**VectorSchemaRoot** combines ValueVectors with a Schema to represent tabular data.
+
+Example: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Field a = new Field("age",
+ FieldType.nullable(new ArrowType.Int(32, true)),
+ /*children*/null
+ );
+ Field b = new Field("name",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, /*metadata*/ null),
+ /*children*/null
+ );
+ Schema schema = new Schema(asList(a, b), /*metadata*/ null);
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator);
+ IntVector intVectorA = (IntVector) root.getVector("age");
+ VarCharVector varCharVectorB = (VarCharVector) root.getVector("name");
+ ){
+ root.setRowCount(3);
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+ System.out.println("VectorSchemaRoot created: \n" + root.contentToTSVString());
+ }
+
+.. code-block:: shell
+
+ VectorSchemaRoot created:
+ age name
+ 10 Dave
+ 20 Peter
+ 30 Mary
+
+
+Interprocess Communication (IPC)
+********************************
+
+Arrow data can be written to and read from disk, and both of these can be done in
+a streaming and/or random-access fashion depending on application requirements.
+
+**Write data to an arrow file**
+
+Example: Write the dataset from the previous example to an Arrow random-access file.
+
+.. code-block:: Java
+
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.ipc.ArrowFileWriter;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.io.File;
+ import java.io.FileOutputStream;
+ import java.io.IOException;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age",
Review comment:
```suggestion
Field age = new Field("age",
```
##########
File path: docs/source/java/getting_started.rst
##########
@@ -0,0 +1,33 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+User Guide
+==========
+
+.. toctree::
Review comment:
This still doesn't seem right - keep the toctree in index.rst, and move the contents of "quickstartguide.rst" into this file.
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,325 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks. Data types describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data, and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create a ValueVector
+*********************
Review comment:
```suggestion
Create a ValueVector
********************
```
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,325 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks. Data types describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data, and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create a ValueVector
+*********************
+
+**ValueVectors** represent a sequence of values of the same type.
+They are also known as "arrays" in the columnar format.
+
+Example: create a vector of 32-bit integers representing ``[1, null, 2]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", allocator);
+ ){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [1, null, 2]
+
+
+Example: create a vector of UTF-8 encoded strings representing ``["one", "two", "three"]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", allocator);
+ ){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [one, two, three]
+
+Create a Field
+**************
+
+**Fields** are used to denote the particular columns of tabular data.
+They consist of a name, a data type, a flag indicating whether the column can have null values,
+and optional key-value metadata.
+
+Example: create a field named "document" of string type:
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata),
+ /*children*/ null);
+ System.out.println("Field created: " + document + ", Metadata: " + document.getMetadata());
+
+.. code-block:: shell
+
+ Field created: document: Utf8, Metadata: {A=Id card, B=Passport, C=Visa}
+
+Create a Schema
+***************
+
+**Schema** holds a sequence of fields together with some optional metadata.
+
+Example: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
Review comment:
```suggestion
an int32 column "A" and a UTF8-encoded string column "B"
```
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,325 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks. Data types describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data, and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create a ValueVector
+*********************
+
+**ValueVectors** represent a sequence of values of the same type.
+They are also known as "arrays" in the columnar format.
+
+Example: create a vector of 32-bit integers representing ``[1, null, 2]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", allocator);
+ ){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [1, null, 2]
+
+
+Example: create a vector of UTF-8 encoded strings representing ``["one", "two", "three"]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", allocator);
+ ){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [one, two, three]
+
+Create a Field
+**************
+
+**Fields** are used to denote the particular columns of tabular data.
+They consist of a name, a data type, a flag indicating whether the column can have null values,
+and optional key-value metadata.
+
+Example: create a field named "document" of string type:
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata),
+ /*children*/ null);
+ System.out.println("Field created: " + document + ", Metadata: " + document.getMetadata());
+
+.. code-block:: shell
+
+ Field created: document: Utf8, Metadata: {A=Id card, B=Passport, C=Visa}
+
+Create a Schema
+***************
+
+**Schema** holds a sequence of fields together with some optional metadata.
+
+Example: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), /*children*/ null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), /*children*/ null);
+ Schema schema = new Schema(asList(a, b), metadata);
+ System.out.println("Schema created: " + schema);
+
+.. code-block:: shell
+
+ Schema created: Schema<A: Int(32, true), B: Utf8>(metadata: {K1=V1, K2=V2})
+
+Create a VectorSchemaRoot
+*************************
+
+**VectorSchemaRoot** combines ValueVectors with a Schema to represent tabular data.
+
+Example: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Field a = new Field("age",
Review comment:
```suggestion
Field age = new Field("age",
```
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,325 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks. Data types describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data, and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create a ValueVector
+*********************
+
+**ValueVectors** represent a sequence of values of the same type.
+They are also known as "arrays" in the columnar format.
+
+Example: create a vector of 32-bit integers representing ``[1, null, 2]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", allocator);
+ ){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [1, null, 2]
+
+
+Example: create a vector of UTF-8 encoded strings representing ``["one", "two", "three"]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", allocator);
+ ){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [one, two, three]
+
+Create a Field
+**************
+
+**Fields** are used to denote the particular columns of tabular data.
+They consist of a name, a data type, a flag indicating whether the column can have null values,
+and optional key-value metadata.
+
+Example: create a field named "document" of string type:
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata),
+ /*children*/ null);
+ System.out.println("Field created: " + document + ", Metadata: " + document.getMetadata());
+
+.. code-block:: shell
+
+ Field created: document: Utf8, Metadata: {A=Id card, B=Passport, C=Visa}
+
+Create a Schema
+***************
+
+**Schema** holds a sequence of fields together with some optional metadata.
+
+Example: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), /*children*/ null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), /*children*/ null);
+ Schema schema = new Schema(asList(a, b), metadata);
+ System.out.println("Schema created: " + schema);
+
+.. code-block:: shell
+
+ Schema created: Schema<A: Int(32, true), B: Utf8>(metadata: {K1=V1, K2=V2})
+
+Create a VectorSchemaRoot
+*************************
+
+**VectorSchemaRoot** combines ValueVectors with a Schema to represent tabular data.
Review comment:
```suggestion
A **VectorSchemaRoot** combines ValueVectors with a Schema to represent tabular data.
```
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,325 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks. Data types describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data, and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create a ValueVector
+*********************
+
+**ValueVectors** represent a sequence of values of the same type.
+They are also known as "arrays" in the columnar format.
+
+Example: create a vector of 32-bit integers representing ``[1, null, 2]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", allocator);
+ ){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [1, null, 2]
+
+
+Example: create a vector of UTF-8 encoded strings representing ``["one", "two", "three"]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", allocator);
+ ){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [one, two, three]
+
+Create a Field
+**************
+
+**Fields** are used to denote the particular columns of tabular data.
+They consist of a name, a data type, a flag indicating whether the column can have null values,
+and optional key-value metadata.
+
+Example: create a field named "document" of string type:
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata),
+ /*children*/ null);
+ System.out.println("Field created: " + document + ", Metadata: " + document.getMetadata());
+
+.. code-block:: shell
+
+ Field created: document: Utf8, Metadata: {A=Id card, B=Passport, C=Visa}
+
+Create a Schema
+***************
+
+**Schema** holds a sequence of fields together with some optional metadata.
+
+Example: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), /*children*/ null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), /*children*/ null);
+ Schema schema = new Schema(asList(a, b), metadata);
+ System.out.println("Schema created: " + schema);
+
+.. code-block:: shell
+
+ Schema created: Schema<A: Int(32, true), B: Utf8>(metadata: {K1=V1, K2=V2})
+
+Create a VectorSchemaRoot
+*************************
+
+**VectorSchemaRoot** combines ValueVectors with a Schema to represent tabular data.
+
+Example: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Field a = new Field("age",
+ FieldType.nullable(new ArrowType.Int(32, true)),
+ /*children*/null
+ );
+ Field b = new Field("name",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, /*metadata*/ null),
+ /*children*/null
+ );
+ Schema schema = new Schema(asList(a, b), /*metadata*/ null);
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator);
+ IntVector intVectorA = (IntVector) root.getVector("age");
+ VarCharVector varCharVectorB = (VarCharVector) root.getVector("name");
Review comment:
```suggestion
IntVector ageVector = (IntVector) root.getVector("age");
VarCharVector nameVector = (VarCharVector) root.getVector("name");
```
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,325 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks. Data types describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data, and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create a ValueVector
+*********************
+
+**ValueVectors** represent a sequence of values of the same type.
+They are also known as "arrays" in the columnar format.
+
+Example: create a vector of 32-bit integers representing ``[1, null, 2]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", allocator);
+ ){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [1, null, 2]
+
+
+Example: create a vector of UTF-8 encoded strings representing ``["one", "two", "three"]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", allocator);
+ ){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [one, two, three]
+
+Create a Field
+**************
+
+**Fields** are used to denote the particular columns of tabular data.
+They consist of a name, a data type, a flag indicating whether the column can have null values,
+and optional key-value metadata.
+
+Example: create a field named "document" of string type:
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata),
+ /*children*/ null);
+ System.out.println("Field created: " + document + ", Metadata: " + document.getMetadata());
+
+.. code-block:: shell
+
+ Field created: document: Utf8, Metadata: {A=Id card, B=Passport, C=Visa}
+
+Create a Schema
+***************
+
+**Schema** holds a sequence of fields together with some optional metadata.
+
+Example: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), /*children*/ null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), /*children*/ null);
+ Schema schema = new Schema(asList(a, b), metadata);
+ System.out.println("Schema created: " + schema);
+
+.. code-block:: shell
+
+ Schema created: Schema<A: Int(32, true), B: Utf8>(metadata: {K1=V1, K2=V2})
+
+Create a VectorSchemaRoot
+*************************
+
+**VectorSchemaRoot** combines ValueVectors with a Schema to represent tabular data.
+
+Example: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Field a = new Field("age",
+ FieldType.nullable(new ArrowType.Int(32, true)),
+ /*children*/null
+ );
+ Field b = new Field("name",
Review comment:
```suggestion
Field name = new Field("name",
```
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,325 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks. Data types describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data, and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create a ValueVector
+*********************
+
+**ValueVectors** represent a sequence of values of the same type.
+They are also known as "arrays" in the columnar format.
+
+Example: create a vector of 32-bit integers representing ``[1, null, 2]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", allocator);
+ ){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [1, null, 2]
+
+
+Example: create a vector of UTF-8 encoded strings representing ``["one", "two", "three"]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", allocator);
+ ){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [one, two, three]
+
+Create a Field
+**************
+
+**Fields** are used to denote the particular columns of tabular data.
+They consist of a name, a data type, a flag indicating whether the column can have null values,
+and optional key-value metadata.
+
+Example: create a field named "document" of string type:
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata),
+ /*children*/ null);
+ System.out.println("Field created: " + document + ", Metadata: " + document.getMetadata());
+
+.. code-block:: shell
+
+ Field created: document: Utf8, Metadata: {A=Id card, B=Passport, C=Visa}
+
+Create a Schema
+***************
+
+**Schema** holds a sequence of fields together with some optional metadata.
+
+Example: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), /*children*/ null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), /*children*/ null);
+ Schema schema = new Schema(asList(a, b), metadata);
+ System.out.println("Schema created: " + schema);
+
+.. code-block:: shell
+
+ Schema created: Schema<A: Int(32, true), B: Utf8>(metadata: {K1=V1, K2=V2})
+
+Create a VectorSchemaRoot
+*************************
+
+**VectorSchemaRoot** combines ValueVectors with a Schema to represent tabular data.
+
+Example: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Field a = new Field("age",
+ FieldType.nullable(new ArrowType.Int(32, true)),
+ /*children*/null
Review comment:
Does Field require you to pass children?
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,325 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks. Data types describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data, and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create a ValueVector
+*********************
+
+**ValueVectors** represent a sequence of values of the same type.
+They are also known as "arrays" in the columnar format.
+
+Example: create a vector of 32-bit integers representing ``[1, null, 2]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", allocator);
+ ){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [1, null, 2]
+
+
+Example: create a vector of UTF-8 encoded strings representing ``["one", "two", "three"]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", allocator);
+ ){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [one, two, three]
+
+Create a Field
+**************
+
+**Fields** are used to denote the particular columns of tabular data.
+They consist of a name, a data type, a flag indicating whether the column can have null values,
+and optional key-value metadata.
+
+Example: create a field named "document" of string type:
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata),
+ /*children*/ null);
+ System.out.println("Field created: " + document + ", Metadata: " + document.getMetadata());
+
+.. code-block:: shell
+
+ Field created: document: Utf8, Metadata: {A=Id card, B=Passport, C=Visa}
+
+Create a Schema
+***************
+
+**Schema** holds a sequence of fields together with some optional metadata.
+
+Example: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), /*children*/ null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), /*children*/ null);
+ Schema schema = new Schema(asList(a, b), metadata);
+ System.out.println("Schema created: " + schema);
+
+.. code-block:: shell
+
+ Schema created: Schema<A: Int(32, true), B: Utf8>(metadata: {K1=V1, K2=V2})
+
+Create a VectorSchemaRoot
+*************************
+
+**VectorSchemaRoot** combines ValueVectors with a Schema to represent tabular data.
+
+Example: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Field a = new Field("age",
+ FieldType.nullable(new ArrowType.Int(32, true)),
+ /*children*/null
+ );
+ Field b = new Field("name",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, /*metadata*/ null),
+ /*children*/null
+ );
+ Schema schema = new Schema(asList(a, b), /*metadata*/ null);
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator);
+ IntVector intVectorA = (IntVector) root.getVector("age");
+ VarCharVector varCharVectorB = (VarCharVector) root.getVector("name");
+ ){
+ root.setRowCount(3);
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+ System.out.println("VectorSchemaRoot created: \n" + root.contentToTSVString());
+ }
+
+.. code-block:: shell
+
+ VectorSchemaRoot created:
+ age name
+ 10 Dave
+ 20 Peter
+ 30 Mary
+
+
+Interprocess Communication (IPC)
+********************************
+
+Arrow data can be written to and read from disk, and both of these can be done in
+a streaming and/or random-access fashion depending on application requirements.
+
+**Write data to an arrow file**
+
+Example: Write the dataset from the previous example to an Arrow random-access file.
+
+.. code-block:: Java
+
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.ipc.ArrowFileWriter;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.io.File;
+ import java.io.FileOutputStream;
+ import java.io.IOException;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age",
Review comment:
and similar below
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,325 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks. Data types describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data, and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create a ValueVector
+*********************
+
+**ValueVectors** represent a sequence of values of the same type.
+They are also known as "arrays" in the columnar format.
+
+Example: create a vector of 32-bit integers representing ``[1, null, 2]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", allocator);
+ ){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [1, null, 2]
+
+
+Example: create a vector of UTF-8 encoded strings representing ``["one", "two", "three"]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", allocator);
+ ){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [one, two, three]
+
+Create a Field
+**************
+
+**Fields** are used to denote the particular columns of tabular data.
+They consist of a name, a data type, a flag indicating whether the column can have null values,
+and optional key-value metadata.
+
+Example: create a field named "document" of string type:
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata),
+ /*children*/ null);
+ System.out.println("Field created: " + document + ", Metadata: " + document.getMetadata());
+
+.. code-block:: shell
+
+ Field created: document: Utf8, Metadata: {A=Id card, B=Passport, C=Visa}
+
+Create a Schema
+***************
+
+**Schema** holds a sequence of fields together with some optional metadata.
+
+Example: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), /*children*/ null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), /*children*/ null);
+ Schema schema = new Schema(asList(a, b), metadata);
+ System.out.println("Schema created: " + schema);
+
+.. code-block:: shell
+
+ Schema created: Schema<A: Int(32, true), B: Utf8>(metadata: {K1=V1, K2=V2})
+
+Create a VectorSchemaRoot
+*************************
+
+**VectorSchemaRoot** combines ValueVectors with a Schema to represent tabular data.
+
+Example: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Field a = new Field("age",
+ FieldType.nullable(new ArrowType.Int(32, true)),
+ /*children*/null
+ );
+ Field b = new Field("name",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, /*metadata*/ null),
+ /*children*/null
+ );
+ Schema schema = new Schema(asList(a, b), /*metadata*/ null);
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator);
+ IntVector intVectorA = (IntVector) root.getVector("age");
+ VarCharVector varCharVectorB = (VarCharVector) root.getVector("name");
+ ){
+ root.setRowCount(3);
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+ System.out.println("VectorSchemaRoot created: \n" + root.contentToTSVString());
+ }
+
+.. code-block:: shell
+
+ VectorSchemaRoot created:
+ age name
+ 10 Dave
+ 20 Peter
+ 30 Mary
+
+
+Interprocess Communication (IPC)
+********************************
+
+Arrow data can be written to and read from disk, and both of these can be done in
+a streaming and/or random-access fashion depending on application requirements.
+
+**Write data to an arrow file**
+
+Example: Write the dataset from the previous example to an Arrow random-access file.
+
+.. code-block:: Java
+
+
Review comment:
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] davisusanibar commented on a change in pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
davisusanibar commented on a change in pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#discussion_r839664147
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,325 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks. Data types describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data, and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create a ValueVector
+*********************
+
+**ValueVectors** represent a sequence of values of the same type.
+They are also known as "arrays" in the columnar format.
+
+Example: create a vector of 32-bit integers representing ``[1, null, 2]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", allocator);
+ ){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [1, null, 2]
+
+
+Example: create a vector of UTF-8 encoded strings representing ``["one", "two", "three"]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", allocator);
+ ){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [one, two, three]
+
+Create a Field
+**************
+
+**Fields** are used to denote the particular columns of tabular data.
+They consist of a name, a data type, a flag indicating whether the column can have null values,
+and optional key-value metadata.
+
+Example: create a field named "document" of string type:
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata),
+ /*children*/ null);
+ System.out.println("Field created: " + document + ", Metadata: " + document.getMetadata());
+
+.. code-block:: shell
+
+ Field created: document: Utf8, Metadata: {A=Id card, B=Passport, C=Visa}
+
+Create a Schema
+***************
+
+**Schema** holds a sequence of fields together with some optional metadata.
+
+Example: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), /*children*/ null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), /*children*/ null);
+ Schema schema = new Schema(asList(a, b), metadata);
+ System.out.println("Schema created: " + schema);
+
+.. code-block:: shell
+
+ Schema created: Schema<A: Int(32, true), B: Utf8>(metadata: {K1=V1, K2=V2})
+
+Create a VectorSchemaRoot
+*************************
+
+**VectorSchemaRoot** combines ValueVectors with a Schema to represent tabular data.
+
+Example: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Field a = new Field("age",
+ FieldType.nullable(new ArrowType.Int(32, true)),
+ /*children*/null
Review comment:
Yes, it is required
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,325 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks. Data types describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data, and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create a ValueVector
+*********************
+
+**ValueVectors** represent a sequence of values of the same type.
+They are also known as "arrays" in the columnar format.
+
+Example: create a vector of 32-bit integers representing ``[1, null, 2]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", allocator);
+ ){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [1, null, 2]
+
+
+Example: create a vector of UTF-8 encoded strings representing ``["one", "two", "three"]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", allocator);
+ ){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [one, two, three]
+
+Create a Field
+**************
+
+**Fields** are used to denote the particular columns of tabular data.
+They consist of a name, a data type, a flag indicating whether the column can have null values,
+and optional key-value metadata.
+
+Example: create a field named "document" of string type:
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata),
+ /*children*/ null);
+ System.out.println("Field created: " + document + ", Metadata: " + document.getMetadata());
+
+.. code-block:: shell
+
+ Field created: document: Utf8, Metadata: {A=Id card, B=Passport, C=Visa}
+
+Create a Schema
+***************
+
+**Schema** holds a sequence of fields together with some optional metadata.
+
+Example: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), /*children*/ null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), /*children*/ null);
+ Schema schema = new Schema(asList(a, b), metadata);
+ System.out.println("Schema created: " + schema);
+
+.. code-block:: shell
+
+ Schema created: Schema<A: Int(32, true), B: Utf8>(metadata: {K1=V1, K2=V2})
+
+Create a VectorSchemaRoot
+*************************
+
+**VectorSchemaRoot** combines ValueVectors with a Schema to represent tabular data.
+
+Example: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Field a = new Field("age",
+ FieldType.nullable(new ArrowType.Int(32, true)),
+ /*children*/null
+ );
+ Field b = new Field("name",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, /*metadata*/ null),
Review comment:
Changed
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,325 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks. Data types describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data, and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create a ValueVector
+*********************
+
+**ValueVectors** represent a sequence of values of the same type.
+They are also known as "arrays" in the columnar format.
+
+Example: create a vector of 32-bit integers representing ``[1, null, 2]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", allocator);
+ ){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [1, null, 2]
+
+
+Example: create a vector of UTF-8 encoded strings representing ``["one", "two", "three"]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", allocator);
+ ){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [one, two, three]
+
+Create a Field
+**************
+
+**Fields** are used to denote the particular columns of tabular data.
+They consist of a name, a data type, a flag indicating whether the column can have null values,
+and optional key-value metadata.
+
+Example: create a field named "document" of string type:
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata),
+ /*children*/ null);
+ System.out.println("Field created: " + document + ", Metadata: " + document.getMetadata());
+
+.. code-block:: shell
+
+ Field created: document: Utf8, Metadata: {A=Id card, B=Passport, C=Visa}
+
+Create a Schema
+***************
+
+**Schema** holds a sequence of fields together with some optional metadata.
+
+Example: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), /*children*/ null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), /*children*/ null);
+ Schema schema = new Schema(asList(a, b), metadata);
+ System.out.println("Schema created: " + schema);
+
+.. code-block:: shell
+
+ Schema created: Schema<A: Int(32, true), B: Utf8>(metadata: {K1=V1, K2=V2})
+
+Create a VectorSchemaRoot
+*************************
+
+**VectorSchemaRoot** combines ValueVectors with a Schema to represent tabular data.
+
+Example: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Field a = new Field("age",
+ FieldType.nullable(new ArrowType.Int(32, true)),
+ /*children*/null
+ );
+ Field b = new Field("name",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, /*metadata*/ null),
+ /*children*/null
+ );
+ Schema schema = new Schema(asList(a, b), /*metadata*/ null);
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator);
+ IntVector intVectorA = (IntVector) root.getVector("age");
+ VarCharVector varCharVectorB = (VarCharVector) root.getVector("name");
+ ){
+ root.setRowCount(3);
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+ System.out.println("VectorSchemaRoot created: \n" + root.contentToTSVString());
+ }
+
+.. code-block:: shell
+
+ VectorSchemaRoot created:
+ age name
+ 10 Dave
+ 20 Peter
+ 30 Mary
+
+
+Interprocess Communication (IPC)
+********************************
+
+Arrow data can be written to and read from disk, and both of these can be done in
+a streaming and/or random-access fashion depending on application requirements.
+
+**Write data to an arrow file**
+
+Example: Write the dataset from the previous example to an Arrow random-access file.
+
+.. code-block:: Java
+
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.ipc.ArrowFileWriter;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.io.File;
+ import java.io.FileOutputStream;
+ import java.io.IOException;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
Review comment:
Deleted
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on a change in pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
lidavidm commented on a change in pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#discussion_r837682632
##########
File path: docs/source/java/getstarted.rst
##########
@@ -0,0 +1,31 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _getstarted:
+
+===============
+Getting Started
Review comment:
I think it's fairly pointless to have a getting started page that just links to other pages. Let's move the main content back here.
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,482 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_quickstartguide:
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow java manage data in Vector Schema Root (somewhat analogous to tables and record
+batches in the other Arrow implementations). Before to create a Vector Schema Root let's
+define another topics neededs for that purpose.
+
+Create A Value Vector
+*********************
+
+It's called `array` in the columnar format specification. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A Vector Schema Root
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+
+ System.out.println("Vector Schema Root: \n" + vectorSchemaRoot.contentToTSVString());
+ }
+ }
+
+Create a IPC File or Random Access Format
+*****************************************
+
+The Arrow Interprocess Communication (IPC) format defines two types of binary
+formats for serializing Arrow data: the streaming format and the file format
+(or random access format). Such files can be directly memory-mapped when read.
+
+**Write File or Random Access Format**: Write to a file a dataset with metadata
+that contains integer age and string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.ipc.ArrowFileWriter;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.io.File;
+ import java.io.FileOutputStream;
+ import java.io.IOException;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+ // Arrow Java At Rest
+ File file = new File("randon_access_to_file.arrow");
+ try (FileOutputStream fileOutputStream = new FileOutputStream(file);
+ ArrowFileWriter writer = new ArrowFileWriter(vectorSchemaRoot, null, fileOutputStream.getChannel())
+ ) {
+ writer.start();
+ writer.writeBatch();
+ writer.end();
+ System.out.println("Record batches written: " + writer.getRecordBlocks().size() + ". Number of rows written: " + vectorSchemaRoot.getRowCount());
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+ }
+ }
+
+Create A Flight Server And Flight Client
Review comment:
IMO this section can be omitted, we can defer this for a dedicated page about Flight
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,482 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_quickstartguide:
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow java manage data in Vector Schema Root (somewhat analogous to tables and record
+batches in the other Arrow implementations). Before to create a Vector Schema Root let's
+define another topics neededs for that purpose.
+
+Create A Value Vector
+*********************
+
+It's called `array` in the columnar format specification. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A Vector Schema Root
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+
+ System.out.println("Vector Schema Root: \n" + vectorSchemaRoot.contentToTSVString());
+ }
+ }
+
+Create a IPC File or Random Access Format
Review comment:
Can we also have a section about reading a file?
##########
File path: docs/source/java/modules.rst
##########
@@ -0,0 +1,413 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_modules:
+
+=======
+Modules
+=======
+
+.. contents::
+
+This is the variety of arrow java documentation:
+
+* `Specification and protocols`_: This contains agnostic specification that is implemented in this case by arrow java modules.
+* `Supported environment`_ (like this): This contains answers for what-is-that arrow java module.
+* `Cookbook`_: This contains answers about how-to-use arrow java modules with practices examples.
+* `Development`_: This contains detailed information about what you need to consider to start with arrow java development.
+
+Arrow java modules is created using specification such as columnar format, off-heap
+memory, serialization and interprocess communication (IPC). Some of the java modules
+was created with their own native implementations and others through bindings.
+
+.. list-table:: Arrow Java Implementations and Bindings
+ :widths: 25 75
+ :header-rows: 1
+
+ * - Module
+ - Root Decision
+ * - Arrow Memory
+ - Native implementation.
+ * - Arrow Vector
+ - Native implementation.
+ * - Arrow Flight Grpc
+ - Native implementation.
+ * - Arrow Flight Sql
+ - Native implementation.
+ * - Arrow Algorithm
+ - Native implementation.
+ * - Arrow Compression
+ - Native implementation.
+ * - Arrow C Data Interface
+ - Native implementation.
+ * - Arrow Dataset
+ - Bindings
+ * - Arrow ORC
+ - Bindings
+ * - Arrow Gandiva
+ - Bindings
+
+Arrow java is divided in these modules to offer in-memory columnar data structures:
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 75
+ :header-rows: 1
+
+ * - Module
+ - Description
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ * - arrow-memory-unsafe
+ - Allocator and utils for allocating memory in Arrow based on sun.misc.Unsafe.
+ * - arrow-memory-netty
+ - Netty allocator and utils for allocating memory in Arrow.
+ * - arrow-vector
+ - An off-heap reference implementation for Arrow columnar data format.
+ * - arrow-tools
+ - Java applications for working with Arrow ValueVectors.
+ * - arrow-jdbc
+ - (`Experimental`) A library for converting JDBC data to Arrow data.
+ * - arrow-plasma
+ - (`Experimental`) Java client for the Plasma object store.
+ * - flight-core
+ - (`Experimental`) An RPC mechanism for transferring ValueVectors.
+ * - flight-grpc
+ - (`Experimental`) Contains utility class to expose Flight gRPC service and client
+ * - flight-sql
+ - (`Experimental`) Contains utility classes to expose Flight SQL semantics for clients and servers over Arrow Flight.
+ * - flight-integration-tests
+ - Integration tests for Flight RPC.
+ * - arrow-performance
+ - JMH Performance benchmarks for other Arrow libraries.
+ * - arrow-algorithm
+ - (`Experimental`) A collection of algorithms for working with ValueVectors.
+ * - arrow-avro
+ - (`Experimental`) A library for converting Avro data to Arrow data.
+ * - arrow-compression
+ - (`Experimental`) A library for working with the compression/decompression of Arrow data.
+ * - arrow-c-data
+ - Java implementation of C Data Interface
+ * - arrow-orc
+ - (`Experimental`) A JNI wrapper for the C++ ORC reader implementation.
+ * - arrow-gandiva
+ - Java wrappers around the native Gandiva SQL expression compiler.
+ * - arrow-dataset
+ - Java implementation of Arrow Dataset API/Framework
+
+For more detail about how to install this modules please review
+:doc:`Installing Java Modules <install>`.
+
+.. note::
+
+ Arrow java modules offer support to work data: (1) in-memory,
+ (2) at rest and (3) on-the-wire.
+
+Lets zoom-in arrow java modules to take advantage of this functionalities:
+
+Arrow Java In-Memory (The Physical Layer)
Review comment:
We should get rid of the rest of this file. Code examples can be developed more for the cookbook. Quick examples can go on the getstarted page. That page isn't meant to teach you everything, just give you a brief idea of what's possible.
##########
File path: docs/source/java/modules.rst
##########
@@ -0,0 +1,413 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_modules:
+
+=======
+Modules
Review comment:
Let's move this page to the top level. We can organize it like https://arrow.apache.org/docs/cpp/overview.html but for Java. How does that sound?
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,482 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_quickstartguide:
+
+=================
+Quick Start Guide
Review comment:
IMO this file should just be moved into getstarted.rst
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] davisusanibar commented on a change in pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
davisusanibar commented on a change in pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#discussion_r839663914
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,325 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks. Data types describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
Review comment:
Changed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on a change in pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
lidavidm commented on a change in pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#discussion_r836859872
##########
File path: docs/source/java/getstarted.rst
##########
@@ -0,0 +1,417 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _getstarted:
+
+Getting Started
+===============
Review comment:
```suggestion
===============
Getting Started
===============
```
##########
File path: docs/source/java/getstarted.rst
##########
@@ -0,0 +1,417 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _getstarted:
+
+Getting Started
+===============
+
+.. contents::
+
+This is the variety of arrow java documentation:
+
+* `Specification and protocols`_: This contains agnostic specification that is implemented in this case by arrow java modules.
+* `Supported environment`_ (like this): This contains answers for what-is-that arrow java module.
+* `Cookbook`_: This contains answers about how-to-use arrow java modules with practices examples.
+* `Development`_: This contains detailed information about what you need to consider to start with arrow java development.
+
+Arrow Java Modules
+------------------
+
+Arrow java modules is created using specification such as columnar format, off-heap
+memory, serialization and interprocess communication (IPC). Some of the java modules
+was created with their own native implementations and others through bindings.
+
+.. list-table:: Arrow Java Implementations and Bindings
+ :widths: 25 75
+ :header-rows: 1
+
+ * - Module
+ - Root Decision
+ * - Arrow Memory
+ - Native implementation.
+ * - Arrow Vector
+ - Native implementation.
+ * - Arrow Flight Grpc
+ - Native implementation.
+ * - Arrow Flight Sql
+ - Native implementation.
+ * - Arrow Algorithm
+ - Native implementation.
+ * - Arrow Compression
+ - Native implementation.
+ * - Arrow C Data Interface
+ - Native implementation.
+ * - Arrow Dataset
+ - Bindings
+ * - Arrow ORC
+ - Bindings
+ * - Arrow Gandiva
+ - Bindings
+
+Arrow java is divided in these modules to offer in-memory columnar data structures:
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 75
+ :header-rows: 1
+
+ * - Module
+ - Description
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ * - arrow-memory-unsafe
+ - Allocator and utils for allocating memory in Arrow based on sun.misc.Unsafe.
+ * - arrow-memory-netty
+ - Netty allocator and utils for allocating memory in Arrow.
+ * - arrow-vector
+ - An off-heap reference implementation for Arrow columnar data format.
+ * - arrow-tools
+ - Java applications for working with Arrow ValueVectors.
+ * - arrow-jdbc
+ - (`Experimental`) A library for converting JDBC data to Arrow data.
+ * - arrow-plasma
+ - (`Experimental`) Java client for the Plasma object store.
+ * - flight-core
+ - (`Experimental`) An RPC mechanism for transferring ValueVectors.
+ * - flight-grpc
+ - (`Experimental`) Contains utility class to expose Flight gRPC service and client
+ * - flight-sql
+ - (`Experimental`) Contains utility classes to expose Flight SQL semantics for clients and servers over Arrow Flight.
+ * - flight-integration-tests
+ - Integration tests for Flight RPC.
+ * - arrow-performance
+ - JMH Performance benchmarks for other Arrow libraries.
+ * - arrow-algorithm
+ - (`Experimental`) A collection of algorithms for working with ValueVectors.
+ * - arrow-avro
+ - (`Experimental`) A library for converting Avro data to Arrow data.
+ * - arrow-compression
+ - (`Experimental`) A library for working with the compression/decompression of Arrow data.
+ * - arrow-c-data
+ - Java implementation of C Data Interface
+ * - arrow-orc
+ - (`Experimental`) A JNI wrapper for the C++ ORC reader implementation.
+ * - arrow-gandiva
+ - Java wrappers around the native Gandiva SQL expression compiler.
+ * - arrow-dataset
+ - Java implementation of Arrow Dataset API/Framework
+
+For more detail about how to install this modules please review
+:doc:`Installing Java Modules <install>`.
+
+.. figure:: img/arrow_java_wks_deps.png
+ :scale: 75 %
+ :alt: Offer a workspace dependency view for arrow java modules
+
+ Workspace dependency view for arrow java modules.
Review comment:
I'm not sure this really is useful (it's rather messy)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] davisusanibar commented on a change in pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
davisusanibar commented on a change in pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#discussion_r839007325
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A VectorSchemaRoot
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
Review comment:
Deleted
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A VectorSchemaRoot
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
Review comment:
Changed
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A VectorSchemaRoot
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
Review comment:
Deleted
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A VectorSchemaRoot
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+
+ System.out.println("VectorSchemaRoot: \n" + vectorSchemaRoot.contentToTSVString());
+ }
+ }
+
+Interprocess Communication (IPC)
+********************************
+
+Arrow data can be written to and read from disk, and both of these can be done in
+a streaming and/or random-access fashion depending on application requirements.
+
+**Create a IPC File or Random Access Format**
Review comment:
Changed
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A VectorSchemaRoot
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+
+ System.out.println("VectorSchemaRoot: \n" + vectorSchemaRoot.contentToTSVString());
+ }
+ }
+
+Interprocess Communication (IPC)
+********************************
+
+Arrow data can be written to and read from disk, and both of these can be done in
+a streaming and/or random-access fashion depending on application requirements.
+
+**Create a IPC File or Random Access Format**
+
+Write File or Random Access Format: Write to a file a dataset with metadata
+that contains integer age and string names of data.
Review comment:
Changed
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A VectorSchemaRoot
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+
+ System.out.println("VectorSchemaRoot: \n" + vectorSchemaRoot.contentToTSVString());
+ }
+ }
+
+Interprocess Communication (IPC)
+********************************
+
+Arrow data can be written to and read from disk, and both of these can be done in
+a streaming and/or random-access fashion depending on application requirements.
+
+**Create a IPC File or Random Access Format**
+
+Write File or Random Access Format: Write to a file a dataset with metadata
+that contains integer age and string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.ipc.ArrowFileWriter;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.io.File;
+ import java.io.FileOutputStream;
+ import java.io.IOException;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+ // Arrow Java At Rest
Review comment:
Deleted
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] davisusanibar commented on a change in pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
davisusanibar commented on a change in pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#discussion_r839663699
##########
File path: docs/source/java/getting_started.rst
##########
@@ -0,0 +1,33 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+User Guide
+==========
+
+.. toctree::
Review comment:
Changed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] davisusanibar commented on a change in pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
davisusanibar commented on a change in pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#discussion_r838768182
##########
File path: docs/source/java/getstarted.rst
##########
@@ -0,0 +1,31 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _getstarted:
+
+===============
+Getting Started
Review comment:
Changed
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,482 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_quickstartguide:
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow java manage data in Vector Schema Root (somewhat analogous to tables and record
+batches in the other Arrow implementations). Before to create a Vector Schema Root let's
+define another topics neededs for that purpose.
+
+Create A Value Vector
+*********************
+
+It's called `array` in the columnar format specification. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A Vector Schema Root
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+
+ System.out.println("Vector Schema Root: \n" + vectorSchemaRoot.contentToTSVString());
+ }
+ }
+
+Create a IPC File or Random Access Format
+*****************************************
+
+The Arrow Interprocess Communication (IPC) format defines two types of binary
+formats for serializing Arrow data: the streaming format and the file format
+(or random access format). Such files can be directly memory-mapped when read.
+
+**Write File or Random Access Format**: Write to a file a dataset with metadata
+that contains integer age and string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.ipc.ArrowFileWriter;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.io.File;
+ import java.io.FileOutputStream;
+ import java.io.IOException;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+ // Arrow Java At Rest
+ File file = new File("randon_access_to_file.arrow");
+ try (FileOutputStream fileOutputStream = new FileOutputStream(file);
+ ArrowFileWriter writer = new ArrowFileWriter(vectorSchemaRoot, null, fileOutputStream.getChannel())
+ ) {
+ writer.start();
+ writer.writeBatch();
+ writer.end();
+ System.out.println("Record batches written: " + writer.getRecordBlocks().size() + ". Number of rows written: " + vectorSchemaRoot.getRowCount());
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+ }
+ }
+
+Create A Flight Server And Flight Client
Review comment:
Deleted
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,482 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_quickstartguide:
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow java manage data in Vector Schema Root (somewhat analogous to tables and record
+batches in the other Arrow implementations). Before to create a Vector Schema Root let's
+define another topics neededs for that purpose.
+
+Create A Value Vector
+*********************
+
+It's called `array` in the columnar format specification. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A Vector Schema Root
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+
+ System.out.println("Vector Schema Root: \n" + vectorSchemaRoot.contentToTSVString());
+ }
+ }
+
+Create a IPC File or Random Access Format
Review comment:
Added
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,482 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+.. _java_quickstartguide:
+
+=================
+Quick Start Guide
Review comment:
Changed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] davisusanibar commented on a change in pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
davisusanibar commented on a change in pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#discussion_r840778466
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,203 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+The Apache Arrow Java modules implement various specifications including the
+columnar format and IPC. Most modules are native Java implementations,
+but some modules are JNI bindings to the C++ library.
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 50 25
+ :header-rows: 1
+
+ * - Module
+ - Description
+ - Implementation
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ - Native
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ - Native
+ * - arrow-memory-unsafe
+ - Memory management implementation based on sun.misc.Unsafe.
+ - Native
+ * - arrow-memory-netty
+ - Memory management implementation based on Netty.
+ - Native
+ * - arrow-vector
+ - An off-heap reference implementation for Arrow columnar data format.
+ - Native
+ * - arrow-tools
+ - Java applications for working with Arrow ValueVectors.
+ - Native
+ * - arrow-jdbc
+ - (Experimental) A library for converting JDBC data to Arrow data.
+ - Native
+ * - arrow-plasma
+ - (Experimental) Java client for the Plasma object store.
+ - Native
+ * - flight-core
+ - (Experimental) An RPC mechanism for transferring ValueVectors.
+ - Native
+ * - flight-grpc
+ - (Experimental) Contains utility class to expose Flight gRPC service and client.
+ - Native
+ * - flight-sql
+ - (Experimental) Contains utility classes to expose Flight SQL semantics for clients and servers over Arrow Flight.
+ - Native
+ * - flight-integration-tests
+ - Integration tests for Flight RPC.
+ - Native
+ * - arrow-performance
+ - JMH benchmarks for the Arrow libraries.
+ - Native
+ * - arrow-algorithm
+ - (Experimental) A collection of algorithms for working with ValueVectors.
+ - Native
+ * - arrow-avro
+ - (Experimental) A library for converting Avro data to Arrow data.
+ - Native
+ * - arrow-compression
+ - (Experimental) A library for working with compression/decompression of Arrow data.
+ - Native
+ * - arrow-c-data
+ - Java implementation of `C Data Interface`_
+ - JNI
+ * - arrow-orc
+ - (Experimental) A JNI wrapper for the C++ ORC reader implementation.
+ - JNI
+ * - arrow-gandiva
+ - Java wrappers around the native Gandiva SQL expression compiler.
+ - JNI
+ * - arrow-dataset
+ - Java bindings to the Arrow Datasets library.
+ - JNI
+
+Arrow Java modules support working with data (1) in-memory, (2) at rest, and (3) on-the-wire.
+
+For more detail about how to install this modules please review `Installing Java Modules`.
+
+Arrow Java In-Memory (The Physical Layer)
Review comment:
Deleted
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,203 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+The Apache Arrow Java modules implement various specifications including the
+columnar format and IPC. Most modules are native Java implementations,
+but some modules are JNI bindings to the C++ library.
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 50 25
+ :header-rows: 1
+
+ * - Module
+ - Description
+ - Implementation
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ - Native
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ - Native
+ * - arrow-memory-unsafe
+ - Memory management implementation based on sun.misc.Unsafe.
+ - Native
+ * - arrow-memory-netty
+ - Memory management implementation based on Netty.
+ - Native
+ * - arrow-vector
+ - An off-heap reference implementation for Arrow columnar data format.
+ - Native
+ * - arrow-tools
+ - Java applications for working with Arrow ValueVectors.
+ - Native
+ * - arrow-jdbc
+ - (Experimental) A library for converting JDBC data to Arrow data.
+ - Native
+ * - arrow-plasma
+ - (Experimental) Java client for the Plasma object store.
+ - Native
+ * - flight-core
+ - (Experimental) An RPC mechanism for transferring ValueVectors.
+ - Native
+ * - flight-grpc
+ - (Experimental) Contains utility class to expose Flight gRPC service and client.
+ - Native
+ * - flight-sql
+ - (Experimental) Contains utility classes to expose Flight SQL semantics for clients and servers over Arrow Flight.
+ - Native
+ * - flight-integration-tests
+ - Integration tests for Flight RPC.
+ - Native
+ * - arrow-performance
+ - JMH benchmarks for the Arrow libraries.
+ - Native
+ * - arrow-algorithm
+ - (Experimental) A collection of algorithms for working with ValueVectors.
+ - Native
+ * - arrow-avro
+ - (Experimental) A library for converting Avro data to Arrow data.
+ - Native
+ * - arrow-compression
+ - (Experimental) A library for working with compression/decompression of Arrow data.
+ - Native
+ * - arrow-c-data
+ - Java implementation of `C Data Interface`_
+ - JNI
+ * - arrow-orc
+ - (Experimental) A JNI wrapper for the C++ ORC reader implementation.
+ - JNI
+ * - arrow-gandiva
+ - Java wrappers around the native Gandiva SQL expression compiler.
+ - JNI
+ * - arrow-dataset
+ - Java bindings to the Arrow Datasets library.
+ - JNI
+
+Arrow Java modules support working with data (1) in-memory, (2) at rest, and (3) on-the-wire.
+
+For more detail about how to install this modules please review `Installing Java Modules`.
Review comment:
Deleted
##########
File path: docs/source/java/quickstartguide.rst
##########
@@ -0,0 +1,311 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks. Data types describe the types of values;
+ValueVectors are sequences of typed values; fields describe the types of columns in
+tabular data; schemas describe a sequence of columns in tabular data, and
+VectorSchemaRoot represents tabular data. Arrow also provides readers and
+writers for loading data from and persisting data to storage.
+
+Create a ValueVector
+********************
+
+**ValueVectors** represent a sequence of values of the same type.
+They are also known as "arrays" in the columnar format.
+
+Example: create a vector of 32-bit integers representing ``[1, null, 2]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", allocator);
+ ){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [1, null, 2]
+
+
+Example: create a vector of UTF-8 encoded strings representing ``["one", "two", "three"]``:
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", allocator);
+ ){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+.. code-block:: shell
+
+ Vector created in memory: [one, two, three]
+
+Create a Field
+**************
+
+**Fields** are used to denote the particular columns of tabular data.
+They consist of a name, a data type, a flag indicating whether the column can have null values,
+and optional key-value metadata.
+
+Example: create a field named "document" of string type:
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document",
+ new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata),
+ /*children*/ null);
+ System.out.println("Field created: " + document + ", Metadata: " + document.getMetadata());
+
+.. code-block:: shell
+
+ Field created: document: Utf8, Metadata: {A=Id card, B=Passport, C=Visa}
+
+Create a Schema
+***************
+
+**Schemas** hold a sequence of fields together with some optional metadata.
+
+Example: Create a schema describing datasets with two columns:
+an int32 column "A" and a UTF8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), /*children*/ null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), /*children*/ null);
+ Schema schema = new Schema(asList(a, b), metadata);
+ System.out.println("Schema created: " + schema);
+
+.. code-block:: shell
+
+ Schema created: Schema<A: Int(32, true), B: Utf8>(metadata: {K1=V1, K2=V2})
+
+Create a VectorSchemaRoot
+*************************
+
+A **VectorSchemaRoot** combines ValueVectors with a Schema to represent tabular data.
+
+Example: Create a dataset of names (strings) and ages (32-bit signed integers).
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Field age = new Field("age",
+ FieldType.nullable(new ArrowType.Int(32, true)),
+ /*children*/null
+ );
+ Field name = new Field("name",
+ FieldType.nullable(new ArrowType.Utf8()),
+ /*children*/null
+ );
+ Schema schema = new Schema(asList(age, name), /*metadata*/ null);
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator);
+ IntVector ageVector = (IntVector) root.getVector("age");
+ VarCharVector nameVector = (VarCharVector) root.getVector("name");
+ ){
+ root.setRowCount(3);
+ ageVector.allocateNew(3);
+ ageVector.set(0, 10);
+ ageVector.set(1, 20);
+ ageVector.set(2, 30);
+ nameVector.allocateNew(3);
+ nameVector.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ nameVector.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ nameVector.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+ System.out.println("VectorSchemaRoot created: \n" + root.contentToTSVString());
+ }
+
+.. code-block:: shell
+
+ VectorSchemaRoot created:
+ age name
+ 10 Dave
+ 20 Peter
+ 30 Mary
+
+
+Interprocess Communication (IPC)
+********************************
+
+Arrow data can be written to and read from disk, and both of these can be done in
+a streaming and/or random-access fashion depending on application requirements.
+
+**Write data to an arrow file**
+
+Example: Write the dataset from the previous example to an Arrow random-access file.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.ipc.ArrowFileWriter;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import java.io.File;
+ import java.io.FileOutputStream;
+ import java.io.IOException;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Field age = new Field("age",
+ FieldType.nullable(new ArrowType.Int(32, true)),
+ /*children*/ null);
+ Field name = new Field("name",
+ FieldType.nullable(new ArrowType.Utf8()),
+ /*children*/ null);
+ Schema schema = new Schema(asList(age, name));
+ try(
+ BufferAllocator allocator = new RootAllocator();
+ VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator);
+ IntVector ageVector = (IntVector) root.getVector("age");
+ VarCharVector nameVector = (VarCharVector) root.getVector("name");
+ ){
+ ageVector.allocateNew(3);
+ ageVector.set(0, 10);
+ ageVector.set(1, 20);
+ ageVector.set(2, 30);
+ nameVector.allocateNew(3);
+ nameVector.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ nameVector.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ nameVector.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+ root.setRowCount(3);
+ File file = new File("random_access_file.arrow");
+ try (
+ FileOutputStream fileOutputStream = new FileOutputStream(file);
+ ArrowFileWriter writer = new ArrowFileWriter(root, /*provider*/ null, fileOutputStream.getChannel());
+ ) {
+ writer.start();
+ writer.writeBatch();
+ writer.end();
+ System.out.println("Record batches written: " + writer.getRecordBlocks().size()
+ + ". Number of rows written: " + root.getRowCount());
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+ }
+
+.. code-block:: shell
+
+ Record batches written: 1. Number of rows written: 3
+
+**Read data from an arrow file**
+
+Example: Read the dataset from the previous example to an Arrow random-access file.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.ipc.ArrowFileReader;
+ import org.apache.arrow.vector.ipc.message.ArrowBlock;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import java.io.File;
+ import java.io.FileInputStream;
+ import java.io.FileOutputStream;
+ import java.io.IOException;
+
+ try(
+ BufferAllocator allocator = new RootAllocator(Long.MAX_VALUE);
+ FileInputStream fileInputStream = new FileInputStream(new File("random_access_file.arrow"));
+ ArrowFileReader reader = new ArrowFileReader(fileInputStream.getChannel(), allocator);
+ ){
+ System.out.println("Record batches in file: " + reader.getRecordBlocks().size());
+ for (ArrowBlock arrowBlock : reader.getRecordBlocks()) {
+ reader.loadRecordBatch(arrowBlock);
+ VectorSchemaRoot root = reader.getVectorSchemaRoot();
+ System.out.println("VectorSchemaRoot read: \n" + root.contentToTSVString());
+ }
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+
+.. code-block:: shell
+
+ VectorSchemaRoot read:
Review comment:
Added
##########
File path: docs/source/java/index.rst
##########
@@ -18,12 +18,11 @@
Java Implementation
===================
-This is the documentation of the Java API of Apache Arrow. For more details
-on the Arrow format and other language bindings see the :doc:`parent documentation <../index>`.
-
Review comment:
Added
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] davisusanibar commented on a change in pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
davisusanibar commented on a change in pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#discussion_r839007858
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,218 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+Arrow Java modules is created using specification such as Columnar Format, Off-Heap
+Memory, Serialization and Inter Process Communication (IPC). Some of the Java modules
+was created with their own native implementations and others through bindings.
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
Review comment:
Deleted
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,218 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+Arrow Java modules is created using specification such as Columnar Format, Off-Heap
+Memory, Serialization and Inter Process Communication (IPC). Some of the Java modules
+was created with their own native implementations and others through bindings.
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+
+Arrow Java Modules
Review comment:
Deleted
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,218 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+Arrow Java modules is created using specification such as Columnar Format, Off-Heap
+Memory, Serialization and Inter Process Communication (IPC). Some of the Java modules
+was created with their own native implementations and others through bindings.
Review comment:
Changed
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,218 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+Arrow Java modules is created using specification such as Columnar Format, Off-Heap
+Memory, Serialization and Inter Process Communication (IPC). Some of the Java modules
+was created with their own native implementations and others through bindings.
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+
+Arrow Java Modules
+------------------
+
+Arrow Java are divided in these modules to offer in-memory columnar data structures:
Review comment:
Deleted
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,218 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+Arrow Java modules is created using specification such as Columnar Format, Off-Heap
+Memory, Serialization and Inter Process Communication (IPC). Some of the Java modules
+was created with their own native implementations and others through bindings.
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+
+Arrow Java Modules
+------------------
+
+Arrow Java are divided in these modules to offer in-memory columnar data structures:
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 50 25
+ :header-rows: 1
+
+ * - Module
+ - Description
+ - Root Decision
Review comment:
Changed
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,218 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+Arrow Java modules is created using specification such as Columnar Format, Off-Heap
+Memory, Serialization and Inter Process Communication (IPC). Some of the Java modules
+was created with their own native implementations and others through bindings.
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+
+Arrow Java Modules
+------------------
+
+Arrow Java are divided in these modules to offer in-memory columnar data structures:
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 50 25
+ :header-rows: 1
+
+ * - Module
+ - Description
+ - Root Decision
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ - Native implementation
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ - Native implementation
+ * - arrow-memory-unsafe
+ - Allocator and utils for allocating memory in Arrow based on sun.misc.Unsafe.
+ - Native implementation
+ * - arrow-memory-netty
+ - Netty allocator and utils for allocating memory in Arrow.
+ - Native implementation
+ * - arrow-vector
+ - An off-heap reference implementation for Arrow columnar data format.
+ - Native implementation
+ * - arrow-tools
+ - Java applications for working with Arrow ValueVectors.
+ - Native implementation
+ * - arrow-jdbc
+ - (Experimental) A library for converting JDBC data to Arrow data.
+ - Native implementation
+ * - arrow-plasma
+ - (Experimental) Java client for the Plasma object store.
+ - Native implementation
+ * - flight-core
+ - (Experimental) An RPC mechanism for transferring ValueVectors.
+ - Native implementation
+ * - flight-grpc
+ - (Experimental) Contains utility class to expose Flight gRPC service and client.
+ - Native implementation
+ * - flight-sql
+ - (Experimental) Contains utility classes to expose Flight SQL semantics for clients and servers over Arrow Flight.
+ - Native implementation
+ * - flight-integration-tests
+ - Integration tests for Flight RPC.
+ - Native implementation
+ * - arrow-performance
+ - JMH Performance benchmarks for other Arrow libraries.
+ - Native implementation
+ * - arrow-algorithm
+ - (Experimental) A collection of algorithms for working with ValueVectors.
+ - Native implementation
+ * - arrow-avro
+ - (Experimental) A library for converting Avro data to Arrow data.
+ - Native implementation
+ * - arrow-compression
+ - (Experimental) A library for working with the compression/decompression of Arrow data.
+ - Native implementation
+ * - arrow-c-data
+ - Java implementation of C Data Interface
+ - Bindings
+ * - arrow-orc
+ - (Experimental) A JNI wrapper for the C++ ORC reader implementation.
+ - Bindings
+ * - arrow-gandiva
+ - Java wrappers around the native Gandiva SQL expression compiler.
+ - Bindings
+ * - arrow-dataset
+ - Java implementation of Arrow Dataset API/Framework
+ - Bindings
+
+Arrow Java modules offer support to work data (1) in-memory, (2) at rest and (3) on-the-wire.
+
+For more detail about how to install this modules please review
+:doc:`Installing Java Modules <install>`.
Review comment:
Deleted
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] davisusanibar commented on a change in pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
davisusanibar commented on a change in pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#discussion_r839007111
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A VectorSchemaRoot
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
Review comment:
Added
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
Review comment:
Changed
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
Review comment:
Added
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
Review comment:
Changed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] lidavidm commented on a change in pull request #12739: ARROW-15972: [Java][Doc] Getting Started Section - Landing Page
Posted by GitBox <gi...@apache.org>.
lidavidm commented on a change in pull request #12739:
URL: https://github.com/apache/arrow/pull/12739#discussion_r838826608
##########
File path: docs/source/java/index.rst
##########
@@ -18,17 +18,9 @@
Java Implementation
===================
-This is the documentation of the Java API of Apache Arrow. For more details
-on the Arrow format and other language bindings see the :doc:`parent documentation <../index>`.
-
.. toctree::
:maxdepth: 2
- install
- memory
- vector
- vector_schema_root
- ipc
- algorithm
- dataset
+ getting_started
+ Examples <examples/index>
Review comment:
Why are we reorganizing this file?
##########
File path: docs/source/java/examples/index.rst
##########
@@ -0,0 +1,24 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+Examples
+========
+
+.. toctree::
+ :maxdepth: 1
+
+ quickstartguide
Review comment:
Why do we have this file? Why can't we just put the quick start at the top level with everything else?
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
Review comment:
don't use code formatting for things that aren't code. If you need emphasis, I would use italics here instead.
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
Review comment:
Can we format the code before pasting it into these code blocks?
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A VectorSchemaRoot
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
Review comment:
This doesn't explain what it is in principle. Look to the Python docs for inspiration on how we can talk about things. For instance the Python docs say
> Multiple arrays can be combined in tables to form the columns in tabular data when attached to a column name
So we can say "VectorSchemaRoot combines ValueVectors with a Schema to represent tabular data" or something like that. We can still say it's analogous to tables, but we can't use that as the primary explanation.
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
Review comment:
```suggestion
ValueVectors represent a sequence of values of the same type.
They are also known as "arrays" in the columnar format.
```
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A VectorSchemaRoot
+***************************
Review comment:
```suggestion
Create a VectorSchemaRoot
*************************
```
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
Review comment:
Can we break long lines to be more readable?
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A VectorSchemaRoot
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
Review comment:
Just `"age"` is more natural, no?
##########
File path: docs/source/java/getting_started.rst
##########
@@ -0,0 +1,33 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+User Guide
+==========
+
+.. toctree::
Review comment:
We don't need this. Just make "quickstartguide.rst" into "getting_started.rst" and revert the file hierarchy changes.
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A VectorSchemaRoot
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+
+ System.out.println("VectorSchemaRoot: \n" + vectorSchemaRoot.contentToTSVString());
+ }
+ }
+
+Interprocess Communication (IPC)
+********************************
+
+Arrow data can be written to and read from disk, and both of these can be done in
+a streaming and/or random-access fashion depending on application requirements.
+
+**Create a IPC File or Random Access Format**
+
+Write File or Random Access Format: Write to a file a dataset with metadata
+that contains integer age and string names of data.
Review comment:
"Example: write the dataset from the previous example to an Arrow random-access file:"
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
Review comment:
```suggestion
Create a Field
```
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A VectorSchemaRoot
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+
+ System.out.println("VectorSchemaRoot: \n" + vectorSchemaRoot.contentToTSVString());
+ }
+ }
+
+Interprocess Communication (IPC)
+********************************
+
+Arrow data can be written to and read from disk, and both of these can be done in
+a streaming and/or random-access fashion depending on application requirements.
+
+**Create a IPC File or Random Access Format**
+
+Write File or Random Access Format: Write to a file a dataset with metadata
+that contains integer age and string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.ipc.ArrowFileWriter;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.io.File;
+ import java.io.FileOutputStream;
+ import java.io.IOException;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+ // Arrow Java At Rest
Review comment:
The comment isn't very useful here
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A VectorSchemaRoot
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+
+ System.out.println("VectorSchemaRoot: \n" + vectorSchemaRoot.contentToTSVString());
+ }
+ }
+
+Interprocess Communication (IPC)
+********************************
+
+Arrow data can be written to and read from disk, and both of these can be done in
+a streaming and/or random-access fashion depending on application requirements.
+
+**Create a IPC File or Random Access Format**
+
+Write File or Random Access Format: Write to a file a dataset with metadata
+that contains integer age and string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.ipc.ArrowFileWriter;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.io.File;
+ import java.io.FileOutputStream;
+ import java.io.IOException;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+ // Arrow Java At Rest
+ File file = new File("randon_access_to_file.arrow");
+ try (FileOutputStream fileOutputStream = new FileOutputStream(file);
+ ArrowFileWriter writer = new ArrowFileWriter(vectorSchemaRoot, null, fileOutputStream.getChannel())
+ ) {
+ writer.start();
+ writer.writeBatch();
+ writer.end();
+ System.out.println("Record batches written: " + writer.getRecordBlocks().size() + ". Number of rows written: " + vectorSchemaRoot.getRowCount());
Review comment:
Can we break up long lines?
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
Review comment:
```suggestion
try(BufferAllocator allocator = new RootAllocator();
```
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,218 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+Arrow Java modules is created using specification such as Columnar Format, Off-Heap
+Memory, Serialization and Inter Process Communication (IPC). Some of the Java modules
+was created with their own native implementations and others through bindings.
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
Review comment:
We don't need to copy-paste this everywhere.
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
Review comment:
```suggestion
Example: create a vector of 32-bit integers representing ``[1, null, 2]``:
```
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,218 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+Arrow Java modules is created using specification such as Columnar Format, Off-Heap
+Memory, Serialization and Inter Process Communication (IPC). Some of the Java modules
+was created with their own native implementations and others through bindings.
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+
+Arrow Java Modules
+------------------
+
+Arrow Java are divided in these modules to offer in-memory columnar data structures:
Review comment:
I don't think the sentence adds anything
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,218 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+Arrow Java modules is created using specification such as Columnar Format, Off-Heap
+Memory, Serialization and Inter Process Communication (IPC). Some of the Java modules
+was created with their own native implementations and others through bindings.
Review comment:
"The Apache Arrow Java modules implement various specifications including the columnar format and IPC. Most modules are native Java implementations, but some modules are JNI bindings to the C++ library."
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
Review comment:
IMO, the bolding in each of these sections isn't useful. We already know we're talking about fields because that's the section we're reading. Just say "Example: create a column named "document"…" or something.
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
Review comment:
```suggestion
tabular data; ``schemas`` describe a sequence of columns in tabular data, and
```
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
Review comment:
```suggestion
Example: create a vector of UTF-8 encoded strings representing ``["one", "two", "three"]``:
```
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
Review comment:
We should consistently call these either just "vectors" or "ValueVectors". (I would probably prefer "ValueVector" to be consistent with existing documentation.)
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,218 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+Arrow Java modules is created using specification such as Columnar Format, Off-Heap
+Memory, Serialization and Inter Process Communication (IPC). Some of the Java modules
+was created with their own native implementations and others through bindings.
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+
+Arrow Java Modules
+------------------
+
+Arrow Java are divided in these modules to offer in-memory columnar data structures:
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 50 25
+ :header-rows: 1
+
+ * - Module
+ - Description
+ - Root Decision
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ - Native implementation
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ - Native implementation
+ * - arrow-memory-unsafe
+ - Allocator and utils for allocating memory in Arrow based on sun.misc.Unsafe.
+ - Native implementation
+ * - arrow-memory-netty
+ - Netty allocator and utils for allocating memory in Arrow.
Review comment:
```suggestion
- Memory management implementation based on Netty.
```
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,218 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+Arrow Java modules is created using specification such as Columnar Format, Off-Heap
+Memory, Serialization and Inter Process Communication (IPC). Some of the Java modules
+was created with their own native implementations and others through bindings.
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+
+Arrow Java Modules
+------------------
+
+Arrow Java are divided in these modules to offer in-memory columnar data structures:
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 50 25
+ :header-rows: 1
+
+ * - Module
+ - Description
+ - Root Decision
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ - Native implementation
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ - Native implementation
+ * - arrow-memory-unsafe
+ - Allocator and utils for allocating memory in Arrow based on sun.misc.Unsafe.
+ - Native implementation
+ * - arrow-memory-netty
+ - Netty allocator and utils for allocating memory in Arrow.
+ - Native implementation
+ * - arrow-vector
+ - An off-heap reference implementation for Arrow columnar data format.
+ - Native implementation
+ * - arrow-tools
+ - Java applications for working with Arrow ValueVectors.
+ - Native implementation
+ * - arrow-jdbc
+ - (Experimental) A library for converting JDBC data to Arrow data.
+ - Native implementation
+ * - arrow-plasma
+ - (Experimental) Java client for the Plasma object store.
+ - Native implementation
+ * - flight-core
+ - (Experimental) An RPC mechanism for transferring ValueVectors.
+ - Native implementation
+ * - flight-grpc
+ - (Experimental) Contains utility class to expose Flight gRPC service and client.
+ - Native implementation
+ * - flight-sql
+ - (Experimental) Contains utility classes to expose Flight SQL semantics for clients and servers over Arrow Flight.
+ - Native implementation
+ * - flight-integration-tests
+ - Integration tests for Flight RPC.
+ - Native implementation
+ * - arrow-performance
+ - JMH Performance benchmarks for other Arrow libraries.
+ - Native implementation
+ * - arrow-algorithm
+ - (Experimental) A collection of algorithms for working with ValueVectors.
+ - Native implementation
+ * - arrow-avro
+ - (Experimental) A library for converting Avro data to Arrow data.
+ - Native implementation
+ * - arrow-compression
+ - (Experimental) A library for working with the compression/decompression of Arrow data.
+ - Native implementation
+ * - arrow-c-data
+ - Java implementation of C Data Interface
Review comment:
Can we link to the C Data Interface page?
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
Review comment:
If we're printing things, can we also show the output?
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
Review comment:
```suggestion
Create a Schema
```
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,218 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+Arrow Java modules is created using specification such as Columnar Format, Off-Heap
+Memory, Serialization and Inter Process Communication (IPC). Some of the Java modules
+was created with their own native implementations and others through bindings.
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+
+Arrow Java Modules
+------------------
+
+Arrow Java are divided in these modules to offer in-memory columnar data structures:
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 50 25
+ :header-rows: 1
+
+ * - Module
+ - Description
+ - Root Decision
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ - Native implementation
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ - Native implementation
+ * - arrow-memory-unsafe
+ - Allocator and utils for allocating memory in Arrow based on sun.misc.Unsafe.
+ - Native implementation
+ * - arrow-memory-netty
+ - Netty allocator and utils for allocating memory in Arrow.
+ - Native implementation
+ * - arrow-vector
+ - An off-heap reference implementation for Arrow columnar data format.
+ - Native implementation
+ * - arrow-tools
+ - Java applications for working with Arrow ValueVectors.
+ - Native implementation
+ * - arrow-jdbc
+ - (Experimental) A library for converting JDBC data to Arrow data.
+ - Native implementation
+ * - arrow-plasma
+ - (Experimental) Java client for the Plasma object store.
+ - Native implementation
+ * - flight-core
+ - (Experimental) An RPC mechanism for transferring ValueVectors.
+ - Native implementation
+ * - flight-grpc
+ - (Experimental) Contains utility class to expose Flight gRPC service and client.
+ - Native implementation
+ * - flight-sql
+ - (Experimental) Contains utility classes to expose Flight SQL semantics for clients and servers over Arrow Flight.
+ - Native implementation
+ * - flight-integration-tests
+ - Integration tests for Flight RPC.
+ - Native implementation
+ * - arrow-performance
+ - JMH Performance benchmarks for other Arrow libraries.
+ - Native implementation
+ * - arrow-algorithm
+ - (Experimental) A collection of algorithms for working with ValueVectors.
+ - Native implementation
+ * - arrow-avro
+ - (Experimental) A library for converting Avro data to Arrow data.
+ - Native implementation
+ * - arrow-compression
+ - (Experimental) A library for working with the compression/decompression of Arrow data.
Review comment:
```suggestion
- (Experimental) A library for working with compression/decompression of Arrow data.
```
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,218 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+Arrow Java modules is created using specification such as Columnar Format, Off-Heap
+Memory, Serialization and Inter Process Communication (IPC). Some of the Java modules
+was created with their own native implementations and others through bindings.
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+
+Arrow Java Modules
+------------------
+
+Arrow Java are divided in these modules to offer in-memory columnar data structures:
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 50 25
+ :header-rows: 1
+
+ * - Module
+ - Description
+ - Root Decision
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ - Native implementation
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ - Native implementation
+ * - arrow-memory-unsafe
+ - Allocator and utils for allocating memory in Arrow based on sun.misc.Unsafe.
+ - Native implementation
+ * - arrow-memory-netty
+ - Netty allocator and utils for allocating memory in Arrow.
+ - Native implementation
+ * - arrow-vector
+ - An off-heap reference implementation for Arrow columnar data format.
+ - Native implementation
+ * - arrow-tools
+ - Java applications for working with Arrow ValueVectors.
+ - Native implementation
+ * - arrow-jdbc
+ - (Experimental) A library for converting JDBC data to Arrow data.
+ - Native implementation
+ * - arrow-plasma
+ - (Experimental) Java client for the Plasma object store.
+ - Native implementation
+ * - flight-core
+ - (Experimental) An RPC mechanism for transferring ValueVectors.
+ - Native implementation
+ * - flight-grpc
+ - (Experimental) Contains utility class to expose Flight gRPC service and client.
+ - Native implementation
+ * - flight-sql
+ - (Experimental) Contains utility classes to expose Flight SQL semantics for clients and servers over Arrow Flight.
+ - Native implementation
+ * - flight-integration-tests
+ - Integration tests for Flight RPC.
+ - Native implementation
+ * - arrow-performance
+ - JMH Performance benchmarks for other Arrow libraries.
Review comment:
```suggestion
- JMH benchmarks for the Arrow libraries.
```
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
Review comment:
```suggestion
Fields are used to denote the particular columns of tabular data.
They consist of a name, a data type, a flag indicating whether the column can have null values, and optional key-value metadata.
```
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A VectorSchemaRoot
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
Review comment:
We don't need metadata and I don't think we need to print out everything here.
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,218 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+Arrow Java modules is created using specification such as Columnar Format, Off-Heap
+Memory, Serialization and Inter Process Communication (IPC). Some of the Java modules
+was created with their own native implementations and others through bindings.
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+
+Arrow Java Modules
+------------------
+
+Arrow Java are divided in these modules to offer in-memory columnar data structures:
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 50 25
+ :header-rows: 1
+
+ * - Module
+ - Description
+ - Root Decision
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ - Native implementation
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ - Native implementation
+ * - arrow-memory-unsafe
+ - Allocator and utils for allocating memory in Arrow based on sun.misc.Unsafe.
+ - Native implementation
+ * - arrow-memory-netty
+ - Netty allocator and utils for allocating memory in Arrow.
+ - Native implementation
+ * - arrow-vector
+ - An off-heap reference implementation for Arrow columnar data format.
+ - Native implementation
+ * - arrow-tools
+ - Java applications for working with Arrow ValueVectors.
+ - Native implementation
+ * - arrow-jdbc
+ - (Experimental) A library for converting JDBC data to Arrow data.
+ - Native implementation
+ * - arrow-plasma
+ - (Experimental) Java client for the Plasma object store.
+ - Native implementation
+ * - flight-core
+ - (Experimental) An RPC mechanism for transferring ValueVectors.
+ - Native implementation
+ * - flight-grpc
+ - (Experimental) Contains utility class to expose Flight gRPC service and client.
+ - Native implementation
+ * - flight-sql
+ - (Experimental) Contains utility classes to expose Flight SQL semantics for clients and servers over Arrow Flight.
+ - Native implementation
+ * - flight-integration-tests
+ - Integration tests for Flight RPC.
+ - Native implementation
+ * - arrow-performance
+ - JMH Performance benchmarks for other Arrow libraries.
+ - Native implementation
+ * - arrow-algorithm
+ - (Experimental) A collection of algorithms for working with ValueVectors.
+ - Native implementation
+ * - arrow-avro
+ - (Experimental) A library for converting Avro data to Arrow data.
+ - Native implementation
+ * - arrow-compression
+ - (Experimental) A library for working with the compression/decompression of Arrow data.
+ - Native implementation
+ * - arrow-c-data
+ - Java implementation of C Data Interface
+ - Bindings
+ * - arrow-orc
+ - (Experimental) A JNI wrapper for the C++ ORC reader implementation.
+ - Bindings
+ * - arrow-gandiva
+ - Java wrappers around the native Gandiva SQL expression compiler.
+ - Bindings
+ * - arrow-dataset
+ - Java implementation of Arrow Dataset API/Framework
+ - Bindings
+
+Arrow Java modules offer support to work data (1) in-memory, (2) at rest and (3) on-the-wire.
+
+For more detail about how to install this modules please review
+:doc:`Installing Java Modules <install>`.
Review comment:
This is already in the sidebar, no need to repeat IMO
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,218 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+Arrow Java modules is created using specification such as Columnar Format, Off-Heap
+Memory, Serialization and Inter Process Communication (IPC). Some of the Java modules
+was created with their own native implementations and others through bindings.
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+
+Arrow Java Modules
+------------------
+
+Arrow Java are divided in these modules to offer in-memory columnar data structures:
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 50 25
+ :header-rows: 1
+
+ * - Module
+ - Description
+ - Root Decision
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ - Native implementation
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ - Native implementation
+ * - arrow-memory-unsafe
+ - Allocator and utils for allocating memory in Arrow based on sun.misc.Unsafe.
+ - Native implementation
+ * - arrow-memory-netty
+ - Netty allocator and utils for allocating memory in Arrow.
+ - Native implementation
+ * - arrow-vector
+ - An off-heap reference implementation for Arrow columnar data format.
+ - Native implementation
+ * - arrow-tools
+ - Java applications for working with Arrow ValueVectors.
+ - Native implementation
+ * - arrow-jdbc
+ - (Experimental) A library for converting JDBC data to Arrow data.
+ - Native implementation
+ * - arrow-plasma
+ - (Experimental) Java client for the Plasma object store.
+ - Native implementation
+ * - flight-core
+ - (Experimental) An RPC mechanism for transferring ValueVectors.
+ - Native implementation
+ * - flight-grpc
+ - (Experimental) Contains utility class to expose Flight gRPC service and client.
+ - Native implementation
+ * - flight-sql
+ - (Experimental) Contains utility classes to expose Flight SQL semantics for clients and servers over Arrow Flight.
+ - Native implementation
+ * - flight-integration-tests
+ - Integration tests for Flight RPC.
+ - Native implementation
+ * - arrow-performance
+ - JMH Performance benchmarks for other Arrow libraries.
+ - Native implementation
+ * - arrow-algorithm
+ - (Experimental) A collection of algorithms for working with ValueVectors.
+ - Native implementation
+ * - arrow-avro
+ - (Experimental) A library for converting Avro data to Arrow data.
+ - Native implementation
+ * - arrow-compression
+ - (Experimental) A library for working with the compression/decompression of Arrow data.
+ - Native implementation
+ * - arrow-c-data
+ - Java implementation of C Data Interface
+ - Bindings
+ * - arrow-orc
+ - (Experimental) A JNI wrapper for the C++ ORC reader implementation.
+ - Bindings
+ * - arrow-gandiva
+ - Java wrappers around the native Gandiva SQL expression compiler.
+ - Bindings
+ * - arrow-dataset
+ - Java implementation of Arrow Dataset API/Framework
Review comment:
```suggestion
- Java bindings to the Arrow Datasets library.
```
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
Review comment:
```suggestion
Arrow Java provides several building blocks. Data types describe the types of values;
```
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
Review comment:
```suggestion
Example: create a field named "document" of string type:
```
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
Review comment:
and ditto below.
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A VectorSchemaRoot
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
Review comment:
IMO let's not put metadata in the example here, let's try to keep the example minimal.
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A VectorSchemaRoot
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+
+ System.out.println("VectorSchemaRoot: \n" + vectorSchemaRoot.contentToTSVString());
+ }
+ }
+
+Interprocess Communication (IPC)
+********************************
+
+Arrow data can be written to and read from disk, and both of these can be done in
+a streaming and/or random-access fashion depending on application requirements.
+
+**Create a IPC File or Random Access Format**
Review comment:
Just "write data to an arrow file"?
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A VectorSchemaRoot
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+
+ System.out.println("VectorSchemaRoot: \n" + vectorSchemaRoot.contentToTSVString());
+ }
+ }
+
+Interprocess Communication (IPC)
+********************************
+
+Arrow data can be written to and read from disk, and both of these can be done in
+a streaming and/or random-access fashion depending on application requirements.
+
+**Create a IPC File or Random Access Format**
+
+Write File or Random Access Format: Write to a file a dataset with metadata
+that contains integer age and string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.ipc.ArrowFileWriter;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.io.File;
+ import java.io.FileOutputStream;
+ import java.io.IOException;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+ // Arrow Java At Rest
+ File file = new File("randon_access_to_file.arrow");
Review comment:
```suggestion
File file = new File("random_access_file.arrow");
```
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A VectorSchemaRoot
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+
+ System.out.println("VectorSchemaRoot: \n" + vectorSchemaRoot.contentToTSVString());
+ }
+ }
+
+Interprocess Communication (IPC)
+********************************
+
+Arrow data can be written to and read from disk, and both of these can be done in
+a streaming and/or random-access fashion depending on application requirements.
+
+**Create a IPC File or Random Access Format**
+
+Write File or Random Access Format: Write to a file a dataset with metadata
+that contains integer age and string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.ipc.ArrowFileWriter;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.io.File;
+ import java.io.FileOutputStream;
+ import java.io.IOException;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+ // Arrow Java At Rest
+ File file = new File("randon_access_to_file.arrow");
+ try (FileOutputStream fileOutputStream = new FileOutputStream(file);
+ ArrowFileWriter writer = new ArrowFileWriter(vectorSchemaRoot, null, fileOutputStream.getChannel())
+ ) {
+ writer.start();
+ writer.writeBatch();
+ writer.end();
+ System.out.println("Record batches written: " + writer.getRecordBlocks().size() + ". Number of rows written: " + vectorSchemaRoot.getRowCount());
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+ }
+ }
+
+**Read a IPC File or Random Access Format**
+
+Read File or Random Access Format: Mapping directly to memory a dataset file with metadata
+that contains integer age and string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.ipc.ArrowFileReader;
+ import org.apache.arrow.vector.ipc.message.ArrowBlock;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import java.io.File;
+ import java.io.FileInputStream;
+ import java.io.FileOutputStream;
+ import java.io.IOException;
+
+ try(RootAllocator rootAllocator = new RootAllocator(Long.MAX_VALUE)){
+ File file = new File("randon_access_to_file.arrow");
Review comment:
```suggestion
File file = new File("random_access_file.arrow");
```
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A VectorSchemaRoot
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+
+ System.out.println("VectorSchemaRoot: \n" + vectorSchemaRoot.contentToTSVString());
+ }
+ }
+
+Interprocess Communication (IPC)
+********************************
+
+Arrow data can be written to and read from disk, and both of these can be done in
+a streaming and/or random-access fashion depending on application requirements.
+
+**Create a IPC File or Random Access Format**
+
+Write File or Random Access Format: Write to a file a dataset with metadata
+that contains integer age and string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.ipc.ArrowFileWriter;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.io.File;
+ import java.io.FileOutputStream;
+ import java.io.IOException;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+ // Arrow Java At Rest
+ File file = new File("randon_access_to_file.arrow");
+ try (FileOutputStream fileOutputStream = new FileOutputStream(file);
+ ArrowFileWriter writer = new ArrowFileWriter(vectorSchemaRoot, null, fileOutputStream.getChannel())
+ ) {
+ writer.start();
+ writer.writeBatch();
+ writer.end();
+ System.out.println("Record batches written: " + writer.getRecordBlocks().size() + ". Number of rows written: " + vectorSchemaRoot.getRowCount());
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+ }
+ }
+
+**Read a IPC File or Random Access Format**
+
+Read File or Random Access Format: Mapping directly to memory a dataset file with metadata
+that contains integer age and string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.ipc.ArrowFileReader;
+ import org.apache.arrow.vector.ipc.message.ArrowBlock;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import java.io.File;
+ import java.io.FileInputStream;
+ import java.io.FileOutputStream;
+ import java.io.IOException;
+
+ try(RootAllocator rootAllocator = new RootAllocator(Long.MAX_VALUE)){
+ File file = new File("randon_access_to_file.arrow");
+ try (FileInputStream fileInputStream = new FileInputStream(file);
+ ArrowFileReader reader = new ArrowFileReader(fileInputStream.getChannel(), rootAllocator)
+ ){
+ System.out.println("Record batches in file: " + reader.getRecordBlocks().size());
+ for (ArrowBlock arrowBlock : reader.getRecordBlocks()) {
+ reader.loadRecordBatch(arrowBlock);
+ VectorSchemaRoot vectorSchemaRootRecover = reader.getVectorSchemaRoot();
Review comment:
```suggestion
VectorSchemaRoot root = reader.getVectorSchemaRoot();
```
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ IntVector intVector = new IntVector("fixed-size-primitive-layout", rootAllocator)){
+ intVector.allocateNew(3);
+ intVector.set(0,1);
+ intVector.setNull(1);
+ intVector.set(2,2);
+ intVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + intVector);
+ }
+
+**Varchar Vector**: Create an value vector of string like this [one, two, three]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.VarCharVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VarCharVector varCharVector = new VarCharVector("variable-size-primitive-layout", rootAllocator)){
+ varCharVector.allocateNew(3);
+ varCharVector.set(0, "one".getBytes());
+ varCharVector.set(1, "two".getBytes());
+ varCharVector.set(2, "three".getBytes());
+ varCharVector.setValueCount(3);
+ System.out.println("Vector created in memory: " + varCharVector);
+ }
+
+Create A Field
+**************
+
+Fields are used to denote the particular columns of tabular data.
+
+**Field**: Create a column "document" of string type with metadata.
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("A", "Id card");
+ metadata.put("B", "Passport");
+ metadata.put("C", "Visa");
+ Field document = new Field("document", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadata), /*children*/ null);
+
+Create A Schema
+***************
+
+Schema holds a sequence of fields together with some optional metadata.
+
+**Schema**: Create a schema describing datasets with two columns:
+a int32 column "A" and a utf8-encoded string column "B"
+
+.. code-block:: Java
+
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadata = new HashMap<>();
+ metadata.put("K1", "V1");
+ metadata.put("K2", "V2");
+ Field a = new Field("A", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("B", FieldType.nullable(new ArrowType.Utf8()), null);
+ Schema schema = new Schema(asList(a, b), metadata);
+
+Create A VectorSchemaRoot
+***************************
+
+VectorSchemaRoot is somewhat analogous to tables and record batches in the other
+Arrow implementations.
+
+**VectorSchemaRoot**: Create a dataset with metadata that contains integer age and
+string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+
+ System.out.println("VectorSchemaRoot: \n" + vectorSchemaRoot.contentToTSVString());
+ }
+ }
+
+Interprocess Communication (IPC)
+********************************
+
+Arrow data can be written to and read from disk, and both of these can be done in
+a streaming and/or random-access fashion depending on application requirements.
+
+**Create a IPC File or Random Access Format**
+
+Write File or Random Access Format: Write to a file a dataset with metadata
+that contains integer age and string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+ import org.apache.arrow.vector.VarCharVector;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import org.apache.arrow.vector.ipc.ArrowFileWriter;
+ import org.apache.arrow.vector.types.pojo.ArrowType;
+ import org.apache.arrow.vector.types.pojo.Field;
+ import org.apache.arrow.vector.types.pojo.FieldType;
+ import org.apache.arrow.vector.types.pojo.Schema;
+
+ import java.io.File;
+ import java.io.FileOutputStream;
+ import java.io.IOException;
+ import java.nio.charset.StandardCharsets;
+ import java.util.HashMap;
+ import java.util.Map;
+
+ import static java.util.Arrays.asList;
+
+ Map<String, String> metadataField = new HashMap<>();
+ metadataField.put("K1-Field", "K1F1");
+ metadataField.put("K2-Field", "K2F2");
+ Field a = new Field("Column-A-Age", FieldType.nullable(new ArrowType.Int(32, true)), null);
+ Field b = new Field("Column-B-Name", new FieldType(true, new ArrowType.Utf8(), /*dictionary*/ null, metadataField), null);
+ Map<String, String> metadataSchema = new HashMap<>();
+ metadataSchema.put("K1-Schema", "K1S1");
+ metadataSchema.put("K2-Schema", "K2S2");
+ Schema schema = new Schema(asList(a, b), metadataSchema);
+ System.out.println("Field A: " + a);
+ System.out.println("Field B: " + b + ", Metadata: " + b.getMetadata());
+ System.out.println("Schema: " + schema);
+ try(BufferAllocator rootAllocator = new RootAllocator();
+ VectorSchemaRoot vectorSchemaRoot = VectorSchemaRoot.create(schema, rootAllocator)){
+ vectorSchemaRoot.setRowCount(3);
+ try(IntVector intVectorA = (IntVector) vectorSchemaRoot.getVector("Column-A-Age");
+ VarCharVector varCharVectorB = (VarCharVector) vectorSchemaRoot.getVector("Column-B-Name")) {
+ intVectorA.allocateNew(3);
+ intVectorA.set(0, 10);
+ intVectorA.set(1, 20);
+ intVectorA.set(2, 30);
+ varCharVectorB.allocateNew(3);
+ varCharVectorB.set(0, "Dave".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(1, "Peter".getBytes(StandardCharsets.UTF_8));
+ varCharVectorB.set(2, "Mary".getBytes(StandardCharsets.UTF_8));
+ // Arrow Java At Rest
+ File file = new File("randon_access_to_file.arrow");
+ try (FileOutputStream fileOutputStream = new FileOutputStream(file);
+ ArrowFileWriter writer = new ArrowFileWriter(vectorSchemaRoot, null, fileOutputStream.getChannel())
+ ) {
+ writer.start();
+ writer.writeBatch();
+ writer.end();
+ System.out.println("Record batches written: " + writer.getRecordBlocks().size() + ". Number of rows written: " + vectorSchemaRoot.getRowCount());
+ } catch (IOException e) {
+ e.printStackTrace();
+ }
+ }
+ }
+
+**Read a IPC File or Random Access Format**
+
+Read File or Random Access Format: Mapping directly to memory a dataset file with metadata
+that contains integer age and string names of data.
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.ipc.ArrowFileReader;
+ import org.apache.arrow.vector.ipc.message.ArrowBlock;
+ import org.apache.arrow.vector.VectorSchemaRoot;
+ import java.io.File;
+ import java.io.FileInputStream;
+ import java.io.FileOutputStream;
+ import java.io.IOException;
+
+ try(RootAllocator rootAllocator = new RootAllocator(Long.MAX_VALUE)){
Review comment:
```suggestion
try(BufferAllocator allocator = new RootAllocator(Long.MAX_VALUE)){
```
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,218 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+Arrow Java modules is created using specification such as Columnar Format, Off-Heap
+Memory, Serialization and Inter Process Communication (IPC). Some of the Java modules
+was created with their own native implementations and others through bindings.
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+
+Arrow Java Modules
Review comment:
We should have either the table or the prose, but I don't think we need both. I think just the table is enough.
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,218 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+Arrow Java modules is created using specification such as Columnar Format, Off-Heap
+Memory, Serialization and Inter Process Communication (IPC). Some of the Java modules
+was created with their own native implementations and others through bindings.
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+
+Arrow Java Modules
+------------------
+
+Arrow Java are divided in these modules to offer in-memory columnar data structures:
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 50 25
+ :header-rows: 1
+
+ * - Module
+ - Description
+ - Root Decision
Review comment:
Name this column "Implementation", and the rows can either be "Native" or "JNI".
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,218 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+Arrow Java modules is created using specification such as Columnar Format, Off-Heap
+Memory, Serialization and Inter Process Communication (IPC). Some of the Java modules
+was created with their own native implementations and others through bindings.
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+
+Arrow Java Modules
+------------------
+
+Arrow Java are divided in these modules to offer in-memory columnar data structures:
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 50 25
+ :header-rows: 1
+
+ * - Module
+ - Description
+ - Root Decision
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ - Native implementation
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ - Native implementation
+ * - arrow-memory-unsafe
+ - Allocator and utils for allocating memory in Arrow based on sun.misc.Unsafe.
Review comment:
```suggestion
- Memory management implementation based on sun.misc.Unsafe.
```
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
Review comment:
(Oxford comma)
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,218 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+Arrow Java modules is created using specification such as Columnar Format, Off-Heap
+Memory, Serialization and Inter Process Communication (IPC). Some of the Java modules
+was created with their own native implementations and others through bindings.
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+
+Arrow Java Modules
+------------------
+
+Arrow Java are divided in these modules to offer in-memory columnar data structures:
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 50 25
+ :header-rows: 1
+
+ * - Module
+ - Description
+ - Root Decision
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ - Native implementation
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ - Native implementation
+ * - arrow-memory-unsafe
+ - Allocator and utils for allocating memory in Arrow based on sun.misc.Unsafe.
+ - Native implementation
+ * - arrow-memory-netty
+ - Netty allocator and utils for allocating memory in Arrow.
+ - Native implementation
+ * - arrow-vector
+ - An off-heap reference implementation for Arrow columnar data format.
+ - Native implementation
+ * - arrow-tools
+ - Java applications for working with Arrow ValueVectors.
+ - Native implementation
+ * - arrow-jdbc
+ - (Experimental) A library for converting JDBC data to Arrow data.
+ - Native implementation
+ * - arrow-plasma
+ - (Experimental) Java client for the Plasma object store.
+ - Native implementation
+ * - flight-core
+ - (Experimental) An RPC mechanism for transferring ValueVectors.
+ - Native implementation
+ * - flight-grpc
+ - (Experimental) Contains utility class to expose Flight gRPC service and client.
+ - Native implementation
+ * - flight-sql
+ - (Experimental) Contains utility classes to expose Flight SQL semantics for clients and servers over Arrow Flight.
+ - Native implementation
+ * - flight-integration-tests
+ - Integration tests for Flight RPC.
+ - Native implementation
+ * - arrow-performance
+ - JMH Performance benchmarks for other Arrow libraries.
+ - Native implementation
+ * - arrow-algorithm
+ - (Experimental) A collection of algorithms for working with ValueVectors.
+ - Native implementation
+ * - arrow-avro
+ - (Experimental) A library for converting Avro data to Arrow data.
+ - Native implementation
+ * - arrow-compression
+ - (Experimental) A library for working with the compression/decompression of Arrow data.
+ - Native implementation
+ * - arrow-c-data
+ - Java implementation of C Data Interface
+ - Bindings
+ * - arrow-orc
+ - (Experimental) A JNI wrapper for the C++ ORC reader implementation.
+ - Bindings
+ * - arrow-gandiva
+ - Java wrappers around the native Gandiva SQL expression compiler.
+ - Bindings
+ * - arrow-dataset
+ - Java implementation of Arrow Dataset API/Framework
+ - Bindings
+
+Arrow Java modules offer support to work data (1) in-memory, (2) at rest and (3) on-the-wire.
Review comment:
```suggestion
Arrow Java modules support working with data (1) in-memory, (2) at rest, and (3) on-the-wire.
```
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
Review comment:
```suggestion
Create a ValueVector
```
##########
File path: docs/source/java/examples/quickstartguide.rst
##########
@@ -0,0 +1,277 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+=================
+Quick Start Guide
+=================
+
+.. contents::
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+Create A Value Vector
+*********************
+
+Also known as "arrays" in the columnar format. Represent a one-dimensional
+sequence of homogeneous values.
+
+**Int Vector**: Create an value vector of int32s like this [1, null, 2]
+
+.. code-block:: Java
+
+ import org.apache.arrow.memory.BufferAllocator;
+ import org.apache.arrow.memory.RootAllocator;
+ import org.apache.arrow.vector.IntVector;
+
+ try(BufferAllocator rootAllocator = new RootAllocator();
Review comment:
and ditto below
##########
File path: docs/source/java/overview.rst
##########
@@ -0,0 +1,218 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+.. or more contributor license agreements. See the NOTICE file
+.. distributed with this work for additional information
+.. regarding copyright ownership. The ASF licenses this file
+.. to you under the Apache License, Version 2.0 (the
+.. "License"); you may not use this file except in compliance
+.. with the License. You may obtain a copy of the License at
+
+.. http://www.apache.org/licenses/LICENSE-2.0
+
+.. Unless required by applicable law or agreed to in writing,
+.. software distributed under the License is distributed on an
+.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+.. KIND, either express or implied. See the License for the
+.. specific language governing permissions and limitations
+.. under the License.
+
+.. default-domain:: java
+.. highlight:: java
+
+===================
+High-Level Overview
+===================
+
+.. contents::
+
+Arrow Java modules is created using specification such as Columnar Format, Off-Heap
+Memory, Serialization and Inter Process Communication (IPC). Some of the Java modules
+was created with their own native implementations and others through bindings.
+
+Arrow Java provides several building blocks; ``data types`` describe the types of values;
+``ValueVectors`` are sequences of typed values; ``fields`` describe the types of columns in
+tabular data; ``schemas`` describe a sequence of columns in tabular data and
+``VectorSchemaRoot`` represents tabular data. Arrow also provides ``readers`` and
+``writers`` for loading data from and persisting data to storage.
+
+
+Arrow Java Modules
+------------------
+
+Arrow Java are divided in these modules to offer in-memory columnar data structures:
+
+.. list-table:: Arrow Java Modules
+ :widths: 25 50 25
+ :header-rows: 1
+
+ * - Module
+ - Description
+ - Root Decision
+ * - arrow-format
+ - Generated Java files from the IPC Flatbuffer definitions.
+ - Native implementation
+ * - arrow-memory-core
+ - Core off-heap memory management libraries for Arrow ValueVectors.
+ - Native implementation
+ * - arrow-memory-unsafe
+ - Allocator and utils for allocating memory in Arrow based on sun.misc.Unsafe.
+ - Native implementation
+ * - arrow-memory-netty
+ - Netty allocator and utils for allocating memory in Arrow.
Review comment:
Can we also link to Netty?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org