You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@arrow.apache.org by ko...@apache.org on 2025/12/01 01:41:57 UTC
(arrow) branch main updated: GH-48287: [Ruby] Add minimum pure Ruby Apache Arrow reader implementation (#48288)
This is an automated email from the ASF dual-hosted git repository.
kou pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow.git
The following commit(s) were added to refs/heads/main by this push:
new d63eef986a GH-48287: [Ruby] Add minimum pure Ruby Apache Arrow reader implementation (#48288)
d63eef986a is described below
commit d63eef986ab90aba26e3c91cc635aab3aab39007
Author: Sutou Kouhei <ko...@clear-code.com>
AuthorDate: Mon Dec 1 10:41:51 2025 +0900
GH-48287: [Ruby] Add minimum pure Ruby Apache Arrow reader implementation (#48288)
### Rationale for this change
We want to implement step by step. Let's add only minimum implementation and base configuration as the first step. This doesn't have full reader implementation.
### What changes are included in this PR?
This includes the followings:
* Prepare package
* We use red-arrow-format as package ID
* We use Red Arrow Format as package name
* Auto generated FlatBuffers code
* It's generated by red-flatbuffers
* Add minimum file format reader
* It can read only UInt8 array
* CI configuration
### Are these changes tested?
Yes.
### Are there any user-facing changes?
Yes.
* GitHub Issue: #48287
Authored-by: Sutou Kouhei <ko...@clear-code.com>
Signed-off-by: Sutou Kouhei <ko...@clear-code.com>
---
.dockerignore | 3 +
.pre-commit-config.yaml | 1 +
dev/release/rat_exclude_files.txt | 1 +
ruby/Rakefile | 8 +-
ruby/red-arrow-cuda/red-arrow-cuda.gemspec | 2 +-
ruby/red-arrow-dataset/red-arrow-dataset.gemspec | 2 +-
.../red-arrow-flight-sql.gemspec | 2 +-
ruby/red-arrow-flight/red-arrow-flight.gemspec | 2 +-
ruby/red-arrow-format/.gitattributes | 18 ++
ruby/red-arrow-format/.gitignore | 19 ++
ruby/red-arrow-format/Gemfile | 26 +++
ruby/red-arrow-format/LICENSE.txt | 202 +++++++++++++++++++++
ruby/red-arrow-format/NOTICE.txt | 2 +
ruby/red-arrow-format/README.md | 61 +++++++
ruby/red-arrow-format/Rakefile | 67 +++++++
ruby/red-arrow-format/lib/arrow-format.rb | 19 ++
ruby/red-arrow-format/lib/arrow-format/array.rb | 31 ++++
ruby/red-arrow-format/lib/arrow-format/error.rb | 28 +++
ruby/red-arrow-format/lib/arrow-format/field.rb | 26 +++
.../lib/arrow-format/file-reader.rb | 162 +++++++++++++++++
.../org/apache/arrow/flatbuf/binary.rb | 21 +++
.../org/apache/arrow/flatbuf/binary_view.rb | 27 +++
.../arrow-format/org/apache/arrow/flatbuf/block.rb | 38 ++++
.../org/apache/arrow/flatbuf/body_compression.rb | 47 +++++
.../arrow/flatbuf/body_compression_method.rb | 31 ++++
.../arrow-format/org/apache/arrow/flatbuf/bool.rb | 20 ++
.../org/apache/arrow/flatbuf/buffer.rb | 38 ++++
.../org/apache/arrow/flatbuf/compression_type.rb | 22 +++
.../arrow-format/org/apache/arrow/flatbuf/date.rb | 36 ++++
.../org/apache/arrow/flatbuf/date_unit.rb | 22 +++
.../org/apache/arrow/flatbuf/decimal.rb | 48 +++++
.../org/apache/arrow/flatbuf/dictionary_batch.rb | 50 +++++
.../apache/arrow/flatbuf/dictionary_encoding.rb | 64 +++++++
.../org/apache/arrow/flatbuf/dictionary_kind.rb | 26 +++
.../org/apache/arrow/flatbuf/duration.rb | 30 +++
.../org/apache/arrow/flatbuf/endianness.rb | 24 +++
.../org/apache/arrow/flatbuf/feature.rb | 46 +++++
.../arrow-format/org/apache/arrow/flatbuf/field.rb | 92 ++++++++++
.../org/apache/arrow/flatbuf/field_node.rb | 43 +++++
.../org/apache/arrow/flatbuf/fixed_size_binary.rb | 27 +++
.../org/apache/arrow/flatbuf/fixed_size_list.rb | 27 +++
.../org/apache/arrow/flatbuf/floating_point.rb | 30 +++
.../org/apache/arrow/flatbuf/footer.rb | 74 ++++++++
.../arrow-format/org/apache/arrow/flatbuf/int.rb | 33 ++++
.../org/apache/arrow/flatbuf/interval.rb | 30 +++
.../org/apache/arrow/flatbuf/interval_unit.rb | 23 +++
.../org/apache/arrow/flatbuf/key_value.rb | 36 ++++
.../org/apache/arrow/flatbuf/large_binary.rb | 22 +++
.../org/apache/arrow/flatbuf/large_list.rb | 22 +++
.../org/apache/arrow/flatbuf/large_list_view.rb | 22 +++
.../org/apache/arrow/flatbuf/large_utf8.rb | 22 +++
.../arrow-format/org/apache/arrow/flatbuf/list.rb | 20 ++
.../org/apache/arrow/flatbuf/list_view.rb | 23 +++
.../arrow-format/org/apache/arrow/flatbuf/map.rb | 52 ++++++
.../org/apache/arrow/flatbuf/message.rb | 68 +++++++
.../org/apache/arrow/flatbuf/message_header.rb | 39 ++++
.../org/apache/arrow/flatbuf/metadata_version.rb | 36 ++++
.../arrow-format/org/apache/arrow/flatbuf/null.rb | 21 +++
.../org/apache/arrow/flatbuf/precision.rb | 23 +++
.../org/apache/arrow/flatbuf/record_batch.rb | 93 ++++++++++
.../org/apache/arrow/flatbuf/run_end_encoded.rb | 25 +++
.../org/apache/arrow/flatbuf/schema.rb | 68 +++++++
.../arrow/flatbuf/sparse_matrix_compressed_axis.rb | 22 +++
.../arrow/flatbuf/sparse_matrix_index_csx.rb | 96 ++++++++++
.../org/apache/arrow/flatbuf/sparse_tensor.rb | 92 ++++++++++
.../apache/arrow/flatbuf/sparse_tensor_index.rb | 29 +++
.../arrow/flatbuf/sparse_tensor_index_coo.rb | 93 ++++++++++
.../arrow/flatbuf/sparse_tensor_index_csf.rb | 129 +++++++++++++
.../org/apache/arrow/flatbuf/struct_.rb | 23 +++
.../org/apache/arrow/flatbuf/tensor.rb | 74 ++++++++
.../org/apache/arrow/flatbuf/tensor_dim.rb | 38 ++++
.../arrow-format/org/apache/arrow/flatbuf/time.rb | 51 ++++++
.../org/apache/arrow/flatbuf/time_unit.rb | 24 +++
.../org/apache/arrow/flatbuf/timestamp.rb | 152 ++++++++++++++++
.../arrow-format/org/apache/arrow/flatbuf/type.rb | 55 ++++++
.../arrow-format/org/apache/arrow/flatbuf/union.rb | 44 +++++
.../org/apache/arrow/flatbuf/union_mode.rb | 22 +++
.../arrow-format/org/apache/arrow/flatbuf/utf8.rb | 21 +++
.../org/apache/arrow/flatbuf/utf8view.rb | 27 +++
.../lib/arrow-format/record-batch.rb | 36 ++++
ruby/red-arrow-format/lib/arrow-format/schema.rb | 24 +++
ruby/red-arrow-format/lib/arrow-format/type.rb | 46 +++++
ruby/red-arrow-format/lib/arrow-format/version.rb | 26 +++
.../red-arrow-format.gemspec} | 39 ++--
ruby/red-arrow-format/test/helper.rb | 23 +++
ruby/red-arrow-format/test/run.rb | 50 +++++
ruby/red-arrow-format/test/test-file-reader.rb | 49 +++++
ruby/red-arrow/red-arrow.gemspec | 4 +-
ruby/red-gandiva/red-gandiva.gemspec | 2 +-
ruby/red-parquet/red-parquet.gemspec | 2 +-
90 files changed, 3509 insertions(+), 27 deletions(-)
diff --git a/.dockerignore b/.dockerignore
index 1f1715d8e8..1eda9f5e54 100644
--- a/.dockerignore
+++ b/.dockerignore
@@ -40,6 +40,9 @@
!ruby/red-arrow-cuda/Gemfile
!ruby/red-arrow-cuda/lib/arrow-cuda/version.rb
!ruby/red-arrow-cuda/red-arrow-cuda.gemspec
+!ruby/red-arrow-format/Gemfile
+!ruby/red-arrow-format/lib/arrow-format/version.rb
+!ruby/red-arrow-format/red-arrow-format.gemspec
!ruby/red-gandiva/Gemfile
!ruby/red-gandiva/lib/gandiva/version.rb
!ruby/red-gandiva/red-gandiva.gemspec
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index a354d76458..186277edf4 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -251,6 +251,7 @@ repos:
exclude: >-
(
?^dev/tasks/homebrew-formulae/.*\.rb$|
+ ?^ruby/red-arrow-format/lib/arrow-format/org/.*\.rb$|
)
- repo: https://github.com/cheshirekow/cmake-format-precommit
rev: v0.6.13
diff --git a/dev/release/rat_exclude_files.txt b/dev/release/rat_exclude_files.txt
index 5212dc47e0..5bd53436a2 100644
--- a/dev/release/rat_exclude_files.txt
+++ b/dev/release/rat_exclude_files.txt
@@ -99,4 +99,5 @@ r/inst/include/cpp11/*.hpp
r/tools/nixlibs-allowlist.txt
.gitattributes
ruby/red-arrow/.yardopts
+ruby/red-arrow-format/lib/arrow-format/org/*
.github/pull_request_template.md
diff --git a/ruby/Rakefile b/ruby/Rakefile
index acb8019324..b62043ca67 100644
--- a/ruby/Rakefile
+++ b/ruby/Rakefile
@@ -24,8 +24,12 @@ base_dir = File.join(__dir__)
packages = []
Dir.glob("#{base_dir}/*/*.gemspec") do |gemspec|
package = File.basename(File.dirname(gemspec))
- glib_package_name = package.gsub(/\Ared-/, "") + "-glib"
- next unless PKGConfig.exist?(glib_package_name)
+ if package == "red-arrow-format"
+ next if RUBY_VERSION < "3.1.0"
+ else
+ glib_package_name = package.gsub(/\Ared-/, "") + "-glib"
+ next unless PKGConfig.exist?(glib_package_name)
+ end
packages << package
end
diff --git a/ruby/red-arrow-cuda/red-arrow-cuda.gemspec b/ruby/red-arrow-cuda/red-arrow-cuda.gemspec
index 7bb34c6c2f..0039860d64 100644
--- a/ruby/red-arrow-cuda/red-arrow-cuda.gemspec
+++ b/ruby/red-arrow-cuda/red-arrow-cuda.gemspec
@@ -29,7 +29,7 @@ Gem::Specification.new do |spec|
]
spec.version = version_components.compact.join(".")
spec.homepage = "https://arrow.apache.org/"
- spec.authors = ["Apache Arrow Developers"]
+ spec.authors = ["The Apache Software Foundation"]
spec.email = ["dev@arrow.apache.org"]
spec.summary = "Red Arrow CUDA is the Ruby bindings of Apache Arrow CUDA"
diff --git a/ruby/red-arrow-dataset/red-arrow-dataset.gemspec b/ruby/red-arrow-dataset/red-arrow-dataset.gemspec
index 0a60925e4c..5aa99b82b3 100644
--- a/ruby/red-arrow-dataset/red-arrow-dataset.gemspec
+++ b/ruby/red-arrow-dataset/red-arrow-dataset.gemspec
@@ -29,7 +29,7 @@ Gem::Specification.new do |spec|
]
spec.version = version_components.compact.join(".")
spec.homepage = "https://arrow.apache.org/"
- spec.authors = ["Apache Arrow Developers"]
+ spec.authors = ["The Apache Software Foundation"]
spec.email = ["dev@arrow.apache.org"]
spec.summary = "Red Arrow Dataset is the Ruby bindings of Apache Arrow Dataset"
diff --git a/ruby/red-arrow-flight-sql/red-arrow-flight-sql.gemspec b/ruby/red-arrow-flight-sql/red-arrow-flight-sql.gemspec
index bbb737b1b1..bb866baf50 100644
--- a/ruby/red-arrow-flight-sql/red-arrow-flight-sql.gemspec
+++ b/ruby/red-arrow-flight-sql/red-arrow-flight-sql.gemspec
@@ -29,7 +29,7 @@ Gem::Specification.new do |spec|
]
spec.version = version_components.compact.join(".")
spec.homepage = "https://arrow.apache.org/"
- spec.authors = ["Apache Arrow Developers"]
+ spec.authors = ["The Apache Software Foundation"]
spec.email = ["dev@arrow.apache.org"]
spec.summary =
diff --git a/ruby/red-arrow-flight/red-arrow-flight.gemspec b/ruby/red-arrow-flight/red-arrow-flight.gemspec
index efe868ca74..19967559b4 100644
--- a/ruby/red-arrow-flight/red-arrow-flight.gemspec
+++ b/ruby/red-arrow-flight/red-arrow-flight.gemspec
@@ -29,7 +29,7 @@ Gem::Specification.new do |spec|
]
spec.version = version_components.compact.join(".")
spec.homepage = "https://arrow.apache.org/"
- spec.authors = ["Apache Arrow Developers"]
+ spec.authors = ["The Apache Software Foundation"]
spec.email = ["dev@arrow.apache.org"]
spec.summary = "Red Arrow Flight is the Ruby bindings of Apache Arrow Flight"
diff --git a/ruby/red-arrow-format/.gitattributes b/ruby/red-arrow-format/.gitattributes
new file mode 100644
index 0000000000..f5d0eefc98
--- /dev/null
+++ b/ruby/red-arrow-format/.gitattributes
@@ -0,0 +1,18 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+lib/arrow-format/org/**/*.rb linguist-generated
diff --git a/ruby/red-arrow-format/.gitignore b/ruby/red-arrow-format/.gitignore
new file mode 100644
index 0000000000..afd93a1686
--- /dev/null
+++ b/ruby/red-arrow-format/.gitignore
@@ -0,0 +1,19 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+/Gemfile.lock
+/pkg/
diff --git a/ruby/red-arrow-format/Gemfile b/ruby/red-arrow-format/Gemfile
new file mode 100644
index 0000000000..2307252d9e
--- /dev/null
+++ b/ruby/red-arrow-format/Gemfile
@@ -0,0 +1,26 @@
+# -*- ruby -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+source "https://rubygems.org/"
+
+gemspec
+
+gem "rake"
+gem "red-arrow", path: "../red-arrow"
+gem "test-unit"
diff --git a/ruby/red-arrow-format/LICENSE.txt b/ruby/red-arrow-format/LICENSE.txt
new file mode 100644
index 0000000000..d645695673
--- /dev/null
+++ b/ruby/red-arrow-format/LICENSE.txt
@@ -0,0 +1,202 @@
+
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
diff --git a/ruby/red-arrow-format/NOTICE.txt b/ruby/red-arrow-format/NOTICE.txt
new file mode 100644
index 0000000000..1b218efe16
--- /dev/null
+++ b/ruby/red-arrow-format/NOTICE.txt
@@ -0,0 +1,2 @@
+Apache Arrow
+Copyright 2025 The Apache Software Foundation
diff --git a/ruby/red-arrow-format/README.md b/ruby/red-arrow-format/README.md
new file mode 100644
index 0000000000..0152df6f86
--- /dev/null
+++ b/ruby/red-arrow-format/README.md
@@ -0,0 +1,61 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Red Arrow Format - Apache Arrow Format Ruby
+
+Red Arrow Format is the pure Ruby Apache Arrow format serializer and deserializer implementation. This provides only serialize/deserialize features. If you want to process Apache Arrow data not only serialize/desrialize Apache Arrow data, you should use Red Arrow not Red Arrow Format.
+
+[Apache Arrow](https://arrow.apache.org/) is an in-memory columnar data store. It's used by many products for data analytics.
+
+## Install
+
+If you want to install Red Arrow Format by Bundler, you can add the followings to your `Gemfile`:
+
+```ruby
+gem "red-arrow-format"
+```
+
+If you want to install Red Arrow Format by RubyGems, you can use the following command line:
+
+```console
+$ gem install red-arrow-format
+```
+
+## Usage
+
+```ruby
+require "arrow-format"
+
+File.open("/dev/shm/data.arrow", "rb") do |input|
+ reader = ArrowFormat::FileReader.new(input)
+ reader.each do |record_batch|
+ # Use record_batch
+ end
+end
+```
+
+## Development
+
+You can run tests by the following command lines:
+
+```console
+$ cd ruby/red-arrow-format
+$ bundle install
+$ bundle exec rake test
+```
diff --git a/ruby/red-arrow-format/Rakefile b/ruby/red-arrow-format/Rakefile
new file mode 100644
index 0000000000..f56c4c7935
--- /dev/null
+++ b/ruby/red-arrow-format/Rakefile
@@ -0,0 +1,67 @@
+# -*- ruby -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+require "bundler/gem_helper"
+require "pathname"
+require "tmpdir"
+
+base_dir = Pathname(__dir__)
+
+helper = Bundler::GemHelper.new(base_dir.to_s)
+helper.install
+spec = helper.gemspec
+
+release_task = Rake::Task["release"]
+release_task.prerequisites.replace(["release:rubygem_push"])
+
+task default: :test
+
+desc "Run tests"
+task :test do
+ cd(base_dir.to_s) do
+ ruby("test/run.rb")
+ end
+end
+
+namespace :flatbuffers do
+ desc "Generate FlatBuffers code"
+ task :generate do
+ Dir.mktmpdir do |tmp_dir|
+ format_dir = base_dir.parent.parent + "format"
+ output_dir = base_dir + "lib" + "arrow-format"
+ rm_rf((output_dir + "org").to_s)
+ ["File", "Message"].each do |fb_name|
+ fbs = format_dir + "#{fb_name}.fbs"
+ bfbs = File.join(tmp_dir, "#{fb_name}.bfbs")
+ sh("flatc",
+ "-o", tmp_dir,
+ # This arguments order is important.
+ "--binary",
+ "--schema",
+ "--bfbs-builtins",
+ "--bfbs-comments",
+ fbs.to_s)
+ sh("rbflatc",
+ "--output-dir", output_dir.to_s,
+ "--outer-namespaces", "ArrowFormat",
+ bfbs)
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format.rb b/ruby/red-arrow-format/lib/arrow-format.rb
new file mode 100644
index 0000000000..aea210bfb1
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format.rb
@@ -0,0 +1,19 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+require_relative "arrow-format/file-reader"
+require_relative "arrow-format/version"
diff --git a/ruby/red-arrow-format/lib/arrow-format/array.rb b/ruby/red-arrow-format/lib/arrow-format/array.rb
new file mode 100644
index 0000000000..dacc1d8a79
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/array.rb
@@ -0,0 +1,31 @@
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+module ArrowFormat
+ class UInt8Array
+ attr_reader :size
+ def initialize(size, validity_buffer, values_buffer)
+ @size = size
+ @validity_buffer = validity_buffer
+ @values_buffer = values_buffer
+ end
+
+ def to_a
+ # TODO: Check @validity_buffer
+ @values_buffer.values(:U8, 0, @size)
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/error.rb b/ruby/red-arrow-format/lib/arrow-format/error.rb
new file mode 100644
index 0000000000..125c0cbcf8
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/error.rb
@@ -0,0 +1,28 @@
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+module ArrowFormat
+ class Error < StandardError
+ end
+
+ class ReadError < StandardError
+ attr_reader :buffer
+ def initialize(buffer, message)
+ @buffer = buffer
+ super("#{message}: #{@buffer}")
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/field.rb b/ruby/red-arrow-format/lib/arrow-format/field.rb
new file mode 100644
index 0000000000..a5e146bcbd
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/field.rb
@@ -0,0 +1,26 @@
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+module ArrowFormat
+ class Field
+ attr_reader :name
+ attr_reader :type
+ def initialize(name, type)
+ @name = name
+ @type = type
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/file-reader.rb b/ruby/red-arrow-format/lib/arrow-format/file-reader.rb
new file mode 100644
index 0000000000..ffff36a393
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/file-reader.rb
@@ -0,0 +1,162 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+require_relative "array"
+require_relative "error"
+require_relative "field"
+require_relative "record-batch"
+require_relative "schema"
+require_relative "type"
+
+require_relative "org/apache/arrow/flatbuf/footer"
+require_relative "org/apache/arrow/flatbuf/message"
+require_relative "org/apache/arrow/flatbuf/schema"
+
+module ArrowFormat
+ class FileReader
+ include Enumerable
+
+ MAGIC = "ARROW1".b
+ MAGIC_BUFFER = IO::Buffer.for(MAGIC)
+ START_MARKER_SIZE = MAGIC_BUFFER.size
+ END_MARKER_SIZE = MAGIC_BUFFER.size
+ CONTINUATION = "\xFF\xFF\xFF\xFF".b
+ CONTINUATION_BUFFER = IO::Buffer.for(CONTINUATION)
+ # <magic number "ARROW1">
+ # <empty padding bytes [to 8 byte boundary]>
+ STREAMING_FORMAT_START_OFFSET = 8
+ INT32_SIZE = 4
+ FOOTER_SIZE_SIZE = INT32_SIZE
+ METADATA_SIZE_SIZE = INT32_SIZE
+
+ def initialize(input)
+ case input
+ when IO
+ @buffer = IO::Buffer.map(input, nil, 0, IO::Buffer::READONLY)
+ when String
+ @buffer = IO::Buffer.for(input)
+ else
+ @buffer = input
+ end
+
+ validate
+ @footer = read_footer
+ end
+
+ def each
+ offset = STREAMING_FORMAT_START_OFFSET
+ schema = nil
+ continuation_size = CONTINUATION_BUFFER.size
+ # streaming format
+ loop do
+ continuation = @buffer.slice(offset, continuation_size)
+ unless continuation == CONTINUATION_BUFFER
+ raise ReadError.new(@buffer, "No valid continuation")
+ end
+ offset += continuation_size
+
+ metadata_size = @buffer.get_value(:u32, offset)
+ offset += METADATA_SIZE_SIZE
+ break if metadata_size.zero?
+
+ metadata_data = @buffer.slice(offset, metadata_size)
+ offset += metadata_size
+ metadata = Org::Apache::Arrow::Flatbuf::Message.new(metadata_data)
+
+ body = @buffer.slice(offset, metadata.body_length)
+ header = metadata.header
+ case header
+ when Org::Apache::Arrow::Flatbuf::Schema
+ schema = read_schema(header)
+ when Org::Apache::Arrow::Flatbuf::RecordBatch
+ n_rows = header.length
+ columns = []
+ buffers = header.buffers
+ schema.fields.each do |field|
+ columns << read_column(field, n_rows, buffers, body)
+ end
+ yield(RecordBatch.new(schema, n_rows, columns))
+ end
+
+ offset += metadata.body_length
+ end
+ end
+
+ private
+ def validate
+ minimum_size = STREAMING_FORMAT_START_OFFSET +
+ FOOTER_SIZE_SIZE +
+ END_MARKER_SIZE
+ if @buffer.size < minimum_size
+ raise ReadError.new(@buffer,
+ "Input must be larger than or equal to " +
+ "#{minimum_size}: #{@buffer.size}")
+ end
+
+ start_marker = @buffer.slice(0, START_MARKER_SIZE)
+ if start_marker != MAGIC_BUFFER
+ raise ReadError.new(@buffer, "No start marker")
+ end
+ end_marker = @buffer.slice(@buffer.size - END_MARKER_SIZE, END_MARKER_SIZE)
+ if end_marker != MAGIC_BUFFER
+ raise ReadError.new(@buffer, "No end marker")
+ end
+ end
+
+ def read_footer
+ footer_size_offset = @buffer.size - END_MARKER_SIZE - FOOTER_SIZE_SIZE
+ footer_size = @buffer.get_value(:u32, footer_size_offset)
+ footer_data = @buffer.slice(footer_size_offset - footer_size, footer_size)
+ Org::Apache::Arrow::Flatbuf::Footer.new(footer_data)
+ end
+
+ def read_schema(fb_schema)
+ fields = fb_schema.fields.collect do |fb_field|
+ fb_type = fb_field.type
+ case fb_type
+ when Org::Apache::Arrow::Flatbuf::Int
+ case fb_type.bit_width
+ when 8
+ if fb_type.signed?
+ type = Int8Type.new
+ else
+ type = UInt8Type.new
+ end
+ end
+ end
+ Field.new(fb_field.name, type)
+ end
+ Schema.new(fields)
+ end
+
+ def read_column(field, n_rows, buffers, body)
+ case field.type
+ when UInt8Type
+ validity_buffer = buffers.shift
+ if validity_buffer.length.zero?
+ validity = nil
+ else
+ validity = body.slice(validity_buffer.offset, validity_buffer.length)
+ end
+
+ values_buffer = buffers.shift
+ values = body.slice(values_buffer.offset, values_buffer.length)
+ UInt8Array.new(n_rows, validity, values)
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/binary.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/binary.rb
new file mode 100644
index 0000000000..b1c8e57178
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/binary.rb
@@ -0,0 +1,21 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # Opaque binary data
+ class Binary < ::FlatBuffers::Table
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/binary_view.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/binary_view.rb
new file mode 100644
index 0000000000..3a8fb8cc74
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/binary_view.rb
@@ -0,0 +1,27 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # Logically the same as Binary, but the internal representation uses a view
+ # struct that contains the string length and either the string's entire data
+ # inline (for small strings) or an inlined prefix, an index of another buffer,
+ # and an offset pointing to a slice in that buffer (for non-small strings).
+ #
+ # Since it uses a variable number of data buffers, each Field with this type
+ # must have a corresponding entry in `variadicBufferCounts`.
+ class BinaryView < ::FlatBuffers::Table
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/block.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/block.rb
new file mode 100644
index 0000000000..5f6894039d
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/block.rb
@@ -0,0 +1,38 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //File.fbs
+# Rooting type: org.apache.arrow.flatbuf.Footer (//File.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ class Block < ::FlatBuffers::Struct
+ # Length of the data (this is aligned so there can be a gap between this and
+ # the metadata).
+ def body_length
+ field_offset = 16
+ @view.unpack_long(field_offset)
+ end
+
+ # Length of the metadata
+ def meta_data_length
+ field_offset = 8
+ @view.unpack_int(field_offset)
+ end
+
+ # Index to the start of the RecordBlock (note this is past the Message header)
+ def offset
+ field_offset = 0
+ @view.unpack_long(field_offset)
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/body_compression.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/body_compression.rb
new file mode 100644
index 0000000000..dccfc97a38
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/body_compression.rb
@@ -0,0 +1,47 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Message.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+require_relative "../../../apache/arrow/flatbuf/compression_type"
+require_relative "../../../apache/arrow/flatbuf/body_compression_method"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # Optional compression for the memory buffers constituting IPC message
+ # bodies. Intended for use with RecordBatch but could be used for other
+ # message types
+ class BodyCompression < ::FlatBuffers::Table
+ # Compressor library.
+ # For LZ4_FRAME, each compressed buffer must consist of a single frame.
+ def codec
+ field_offset = @view.unpack_virtual_offset(4)
+ if field_offset.zero?
+ enum_value = 0
+ else
+ enum_value = @view.unpack_byte(field_offset)
+ end
+ ::ArrowFormat::Org::Apache::Arrow::Flatbuf::CompressionType.try_convert(enum_value) || enum_value
+ end
+
+ # Indicates the way the record batch body was compressed
+ def method
+ field_offset = @view.unpack_virtual_offset(6)
+ if field_offset.zero?
+ enum_value = 0
+ else
+ enum_value = @view.unpack_byte(field_offset)
+ end
+ ::ArrowFormat::Org::Apache::Arrow::Flatbuf::BodyCompressionMethod.try_convert(enum_value) || enum_value
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/body_compression_method.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/body_compression_method.rb
new file mode 100644
index 0000000000..fbbada04f4
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/body_compression_method.rb
@@ -0,0 +1,31 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Message.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # Provided for forward compatibility in case we need to support different
+ # strategies for compressing the IPC message body (like whole-body
+ # compression rather than buffer-level) in the future
+ class BodyCompressionMethod < ::FlatBuffers::Enum
+ # Each constituent buffer is first compressed with the indicated
+ # compressor, and then written with the uncompressed length in the first 8
+ # bytes as a 64-bit little-endian signed integer followed by the compressed
+ # buffer bytes (and then padding as required by the protocol). The
+ # uncompressed length may be set to -1 to indicate that the data that
+ # follows is not compressed, which can be useful for cases where
+ # compression does not yield appreciable savings.
+ BUFFER = register("BUFFER", 0)
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/bool.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/bool.rb
new file mode 100644
index 0000000000..5e0f6cd38a
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/bool.rb
@@ -0,0 +1,20 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ class Bool < ::FlatBuffers::Table
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/buffer.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/buffer.rb
new file mode 100644
index 0000000000..76bcf13988
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/buffer.rb
@@ -0,0 +1,38 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # ----------------------------------------------------------------------
+ # A Buffer represents a single contiguous memory segment
+ class Buffer < ::FlatBuffers::Struct
+ # The absolute length (in bytes) of the memory buffer. The memory is found
+ # from offset (inclusive) to offset + length (non-inclusive). When building
+ # messages using the encapsulated IPC message, padding bytes may be written
+ # after a buffer, but such padding bytes do not need to be accounted for in
+ # the size here.
+ def length
+ field_offset = 8
+ @view.unpack_long(field_offset)
+ end
+
+ # The relative offset into the shared memory page where the bytes for this
+ # buffer starts
+ def offset
+ field_offset = 0
+ @view.unpack_long(field_offset)
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/compression_type.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/compression_type.rb
new file mode 100644
index 0000000000..4e503ba410
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/compression_type.rb
@@ -0,0 +1,22 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Message.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ class CompressionType < ::FlatBuffers::Enum
+ LZ4_FRAME = register("LZ4_FRAME", 0)
+ ZSTD = register("ZSTD", 1)
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/date.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/date.rb
new file mode 100644
index 0000000000..41f07de906
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/date.rb
@@ -0,0 +1,36 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+require_relative "../../../apache/arrow/flatbuf/date_unit"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # Date is either a 32-bit or 64-bit signed integer type representing an
+ # elapsed time since UNIX epoch (1970-01-01), stored in either of two units:
+ #
+ # * Milliseconds (64 bits) indicating UNIX time elapsed since the epoch (no
+ # leap seconds), where the values are evenly divisible by 86400000
+ # * Days (32 bits) since the UNIX epoch
+ class Date < ::FlatBuffers::Table
+ def unit
+ field_offset = @view.unpack_virtual_offset(4)
+ if field_offset.zero?
+ enum_value = 1
+ else
+ enum_value = @view.unpack_short(field_offset)
+ end
+ ::ArrowFormat::Org::Apache::Arrow::Flatbuf::DateUnit.try_convert(enum_value) || enum_value
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/date_unit.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/date_unit.rb
new file mode 100644
index 0000000000..d3d1299a09
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/date_unit.rb
@@ -0,0 +1,22 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ class DateUnit < ::FlatBuffers::Enum
+ DAY = register("DAY", 0)
+ MILLISECOND = register("MILLISECOND", 1)
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/decimal.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/decimal.rb
new file mode 100644
index 0000000000..1bdb27e2a6
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/decimal.rb
@@ -0,0 +1,48 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # Exact decimal value represented as an integer value in two's
+ # complement. Currently 32-bit (4-byte), 64-bit (8-byte),
+ # 128-bit (16-byte) and 256-bit (32-byte) integers are used.
+ # The representation uses the endianness indicated in the Schema.
+ class Decimal < ::FlatBuffers::Table
+ # Number of bits per value. The accepted widths are 32, 64, 128 and 256.
+ # We use bitWidth for consistency with Int::bitWidth.
+ def bit_width
+ field_offset = @view.unpack_virtual_offset(8)
+ return 128 if field_offset.zero?
+
+ @view.unpack_int(field_offset)
+ end
+
+ # Total number of decimal digits
+ def precision
+ field_offset = @view.unpack_virtual_offset(4)
+ return 0 if field_offset.zero?
+
+ @view.unpack_int(field_offset)
+ end
+
+ # Number of digits after the decimal point "."
+ def scale
+ field_offset = @view.unpack_virtual_offset(6)
+ return 0 if field_offset.zero?
+
+ @view.unpack_int(field_offset)
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/dictionary_batch.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/dictionary_batch.rb
new file mode 100644
index 0000000000..247f33449a
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/dictionary_batch.rb
@@ -0,0 +1,50 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Message.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+require_relative "../../../apache/arrow/flatbuf/record_batch"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # For sending dictionary encoding information. Any Field can be
+ # dictionary-encoded, but in this case none of its children may be
+ # dictionary-encoded.
+ # There is one vector / column per dictionary, but that vector / column
+ # may be spread across multiple dictionary batches by using the isDelta
+ # flag
+ class DictionaryBatch < ::FlatBuffers::Table
+ def data
+ field_offset = @view.unpack_virtual_offset(6)
+ return nil if field_offset.zero?
+
+ @view.unpack_table(::ArrowFormat::Org::Apache::Arrow::Flatbuf::RecordBatch, field_offset)
+ end
+
+ def id
+ field_offset = @view.unpack_virtual_offset(4)
+ return 0 if field_offset.zero?
+
+ @view.unpack_long(field_offset)
+ end
+
+ # If isDelta is true the values in the dictionary are to be appended to a
+ # dictionary with the indicated id. If isDelta is false this dictionary
+ # should replace the existing dictionary.
+ def delta?
+ field_offset = @view.unpack_virtual_offset(8)
+ return false if field_offset.zero?
+
+ @view.unpack_bool(field_offset)
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/dictionary_encoding.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/dictionary_encoding.rb
new file mode 100644
index 0000000000..60ad3f4a8e
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/dictionary_encoding.rb
@@ -0,0 +1,64 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+require_relative "../../../apache/arrow/flatbuf/dictionary_kind"
+require_relative "../../../apache/arrow/flatbuf/int"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ class DictionaryEncoding < ::FlatBuffers::Table
+ def dictionary_kind
+ field_offset = @view.unpack_virtual_offset(10)
+ if field_offset.zero?
+ enum_value = 0
+ else
+ enum_value = @view.unpack_short(field_offset)
+ end
+ ::ArrowFormat::Org::Apache::Arrow::Flatbuf::DictionaryKind.try_convert(enum_value) || enum_value
+ end
+
+ # The known dictionary id in the application where this data is used. In
+ # the file or streaming formats, the dictionary ids are found in the
+ # DictionaryBatch messages
+ def id
+ field_offset = @view.unpack_virtual_offset(4)
+ return 0 if field_offset.zero?
+
+ @view.unpack_long(field_offset)
+ end
+
+ # The dictionary indices are constrained to be non-negative integers. If
+ # this field is null, the indices must be signed int32. To maximize
+ # cross-language compatibility and performance, implementations are
+ # recommended to prefer signed integer types over unsigned integer types
+ # and to avoid uint64 indices unless they are required by an application.
+ def index_type
+ field_offset = @view.unpack_virtual_offset(6)
+ return nil if field_offset.zero?
+
+ @view.unpack_table(::ArrowFormat::Org::Apache::Arrow::Flatbuf::Int, field_offset)
+ end
+
+ # By default, dictionaries are not ordered, or the order does not have
+ # semantic meaning. In some statistical, applications, dictionary-encoding
+ # is used to represent ordered categorical data, and we provide a way to
+ # preserve that metadata here
+ def ordered?
+ field_offset = @view.unpack_virtual_offset(8)
+ return false if field_offset.zero?
+
+ @view.unpack_bool(field_offset)
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/dictionary_kind.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/dictionary_kind.rb
new file mode 100644
index 0000000000..e7634ecf32
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/dictionary_kind.rb
@@ -0,0 +1,26 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # ----------------------------------------------------------------------
+ # Dictionary encoding metadata
+ # Maintained for forwards compatibility, in the future
+ # Dictionaries might be explicit maps between integers and values
+ # allowing for non-contiguous index values
+ class DictionaryKind < ::FlatBuffers::Enum
+ DENSE_ARRAY = register("DenseArray", 0)
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/duration.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/duration.rb
new file mode 100644
index 0000000000..bf91a2ab0a
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/duration.rb
@@ -0,0 +1,30 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+require_relative "../../../apache/arrow/flatbuf/time_unit"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ class Duration < ::FlatBuffers::Table
+ def unit
+ field_offset = @view.unpack_virtual_offset(4)
+ if field_offset.zero?
+ enum_value = 1
+ else
+ enum_value = @view.unpack_short(field_offset)
+ end
+ ::ArrowFormat::Org::Apache::Arrow::Flatbuf::TimeUnit.try_convert(enum_value) || enum_value
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/endianness.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/endianness.rb
new file mode 100644
index 0000000000..adfe3555f7
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/endianness.rb
@@ -0,0 +1,24 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # ----------------------------------------------------------------------
+ # Endianness of the platform producing the data
+ class Endianness < ::FlatBuffers::Enum
+ LITTLE = register("Little", 0)
+ BIG = register("Big", 1)
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/feature.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/feature.rb
new file mode 100644
index 0000000000..55da686715
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/feature.rb
@@ -0,0 +1,46 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # Represents Arrow Features that might not have full support
+ # within implementations. This is intended to be used in
+ # two scenarios:
+ # 1. A mechanism for readers of Arrow Streams
+ # and files to understand that the stream or file makes
+ # use of a feature that isn't supported or unknown to
+ # the implementation (and therefore can meet the Arrow
+ # forward compatibility guarantees).
+ # 2. A means of negotiating between a client and server
+ # what features a stream is allowed to use. The enums
+ # values here are intended to represent higher level
+ # features, additional details may be negotiated
+ # with key-value pairs specific to the protocol.
+ #
+ # Enums added to this list should be assigned power-of-two values
+ # to facilitate exchanging and comparing bitmaps for supported
+ # features.
+ class Feature < ::FlatBuffers::Enum
+ # Needed to make flatbuffers happy.
+ UNUSED = register("UNUSED", 0)
+ # The stream makes use of multiple full dictionaries with the
+ # same ID and assumes clients implement dictionary replacement
+ # correctly.
+ DICTIONARY_REPLACEMENT = register("DICTIONARY_REPLACEMENT", 1)
+ # The stream makes use of compressed bodies as described
+ # in Message.fbs.
+ COMPRESSED_BODY = register("COMPRESSED_BODY", 2)
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/field.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/field.rb
new file mode 100644
index 0000000000..c748e94948
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/field.rb
@@ -0,0 +1,92 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+require_relative "../../../apache/arrow/flatbuf/key_value"
+require_relative "../../../apache/arrow/flatbuf/dictionary_encoding"
+require_relative "../../../apache/arrow/flatbuf/type"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # ----------------------------------------------------------------------
+ # A field represents a named column in a record / row batch or child of a
+ # nested type.
+ class Field < ::FlatBuffers::Table
+ # children apply only to nested data types like Struct, List and Union. For
+ # primitive types children will have length 0.
+ def children
+ field_offset = @view.unpack_virtual_offset(14)
+ return nil if field_offset.zero?
+
+ element_size = 4
+ @view.unpack_vector(field_offset, element_size) do |element_offset|
+ @view.unpack_table(::ArrowFormat::Org::Apache::Arrow::Flatbuf::Field, element_offset)
+ end
+ end
+
+ # User-defined metadata
+ def custom_metadata
+ field_offset = @view.unpack_virtual_offset(16)
+ return nil if field_offset.zero?
+
+ element_size = 4
+ @view.unpack_vector(field_offset, element_size) do |element_offset|
+ @view.unpack_table(::ArrowFormat::Org::Apache::Arrow::Flatbuf::KeyValue, element_offset)
+ end
+ end
+
+ # Present only if the field is dictionary encoded.
+ def dictionary
+ field_offset = @view.unpack_virtual_offset(12)
+ return nil if field_offset.zero?
+
+ @view.unpack_table(::ArrowFormat::Org::Apache::Arrow::Flatbuf::DictionaryEncoding, field_offset)
+ end
+
+ # Name is not required (e.g., in a List)
+ def name
+ field_offset = @view.unpack_virtual_offset(4)
+ return nil if field_offset.zero?
+
+ @view.unpack_string(field_offset)
+ end
+
+ # Whether or not this field can contain nulls. Should be true in general.
+ def nullable?
+ field_offset = @view.unpack_virtual_offset(6)
+ return false if field_offset.zero?
+
+ @view.unpack_bool(field_offset)
+ end
+
+ # This is the type of the decoded value if the field is dictionary encoded.
+ def type
+ type = type_type
+ return nil if type.nil?
+
+ field_offset = @view.unpack_virtual_offset(10)
+ return nil if field_offset.zero?
+ @view.unpack_union(type.table_class, field_offset)
+ end
+
+ def type_type
+ field_offset = @view.unpack_virtual_offset(8)
+ if field_offset.zero?
+ enum_value = 0
+ else
+ enum_value = @view.unpack_utype(field_offset)
+ end
+ ::ArrowFormat::Org::Apache::Arrow::Flatbuf::Type.try_convert(enum_value) || enum_value
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/field_node.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/field_node.rb
new file mode 100644
index 0000000000..83eeb6719c
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/field_node.rb
@@ -0,0 +1,43 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Message.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # ----------------------------------------------------------------------
+ # Data structures for describing a table row batch (a collection of
+ # equal-length Arrow arrays)
+ # Metadata about a field at some level of a nested type tree (but not
+ # its children).
+ #
+ # For example, a List<Int16> with values `[[1, 2, 3], null, [4], [5, 6], null]`
+ # would have {length: 5, null_count: 2} for its List node, and {length: 6,
+ # null_count: 0} for its Int16 node, as separate FieldNode structs
+ class FieldNode < ::FlatBuffers::Struct
+ # The number of value slots in the Arrow array at this level of a nested
+ # tree
+ def length
+ field_offset = 0
+ @view.unpack_long(field_offset)
+ end
+
+ # The number of observed nulls. Fields with null_count == 0 may choose not
+ # to write their physical validity bitmap out as a materialized buffer,
+ # instead setting the length of the bitmap buffer to 0.
+ def null_count
+ field_offset = 8
+ @view.unpack_long(field_offset)
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/fixed_size_binary.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/fixed_size_binary.rb
new file mode 100644
index 0000000000..4659403a92
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/fixed_size_binary.rb
@@ -0,0 +1,27 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ class FixedSizeBinary < ::FlatBuffers::Table
+ # Number of bytes per value
+ def byte_width
+ field_offset = @view.unpack_virtual_offset(4)
+ return 0 if field_offset.zero?
+
+ @view.unpack_int(field_offset)
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/fixed_size_list.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/fixed_size_list.rb
new file mode 100644
index 0000000000..6191a8425c
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/fixed_size_list.rb
@@ -0,0 +1,27 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ class FixedSizeList < ::FlatBuffers::Table
+ # Number of list items per value
+ def list_size
+ field_offset = @view.unpack_virtual_offset(4)
+ return 0 if field_offset.zero?
+
+ @view.unpack_int(field_offset)
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/floating_point.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/floating_point.rb
new file mode 100644
index 0000000000..7072ce4269
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/floating_point.rb
@@ -0,0 +1,30 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+require_relative "../../../apache/arrow/flatbuf/precision"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ class FloatingPoint < ::FlatBuffers::Table
+ def precision
+ field_offset = @view.unpack_virtual_offset(4)
+ if field_offset.zero?
+ enum_value = 0
+ else
+ enum_value = @view.unpack_short(field_offset)
+ end
+ ::ArrowFormat::Org::Apache::Arrow::Flatbuf::Precision.try_convert(enum_value) || enum_value
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/footer.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/footer.rb
new file mode 100644
index 0000000000..6ef0c100be
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/footer.rb
@@ -0,0 +1,74 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //File.fbs
+# Rooting type: org.apache.arrow.flatbuf.Footer (//File.fbs)
+
+require "flatbuffers"
+require_relative "../../../apache/arrow/flatbuf/key_value"
+require_relative "../../../apache/arrow/flatbuf/block"
+require_relative "../../../apache/arrow/flatbuf/schema"
+require_relative "../../../apache/arrow/flatbuf/metadata_version"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # ----------------------------------------------------------------------
+ # Arrow File metadata
+ #
+ class Footer < ::FlatBuffers::Table
+ # User-defined metadata
+ def custom_metadata
+ field_offset = @view.unpack_virtual_offset(12)
+ return nil if field_offset.zero?
+
+ element_size = 4
+ @view.unpack_vector(field_offset, element_size) do |element_offset|
+ @view.unpack_table(::ArrowFormat::Org::Apache::Arrow::Flatbuf::KeyValue, element_offset)
+ end
+ end
+
+ def dictionaries
+ field_offset = @view.unpack_virtual_offset(8)
+ return nil if field_offset.zero?
+
+ element_size = 24
+ @view.unpack_vector(field_offset, element_size) do |element_offset|
+ @view.unpack_struct(::ArrowFormat::Org::Apache::Arrow::Flatbuf::Block, element_offset)
+ end
+ end
+
+ def record_batches
+ field_offset = @view.unpack_virtual_offset(10)
+ return nil if field_offset.zero?
+
+ element_size = 24
+ @view.unpack_vector(field_offset, element_size) do |element_offset|
+ @view.unpack_struct(::ArrowFormat::Org::Apache::Arrow::Flatbuf::Block, element_offset)
+ end
+ end
+
+ def schema
+ field_offset = @view.unpack_virtual_offset(6)
+ return nil if field_offset.zero?
+
+ @view.unpack_table(::ArrowFormat::Org::Apache::Arrow::Flatbuf::Schema, field_offset)
+ end
+
+ def version
+ field_offset = @view.unpack_virtual_offset(4)
+ if field_offset.zero?
+ enum_value = 0
+ else
+ enum_value = @view.unpack_short(field_offset)
+ end
+ ::ArrowFormat::Org::Apache::Arrow::Flatbuf::MetadataVersion.try_convert(enum_value) || enum_value
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/int.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/int.rb
new file mode 100644
index 0000000000..2b932aac3b
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/int.rb
@@ -0,0 +1,33 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ class Int < ::FlatBuffers::Table
+ def bit_width
+ field_offset = @view.unpack_virtual_offset(4)
+ return 0 if field_offset.zero?
+
+ @view.unpack_int(field_offset)
+ end
+
+ def signed?
+ field_offset = @view.unpack_virtual_offset(6)
+ return false if field_offset.zero?
+
+ @view.unpack_bool(field_offset)
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/interval.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/interval.rb
new file mode 100644
index 0000000000..4ec4b13eed
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/interval.rb
@@ -0,0 +1,30 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+require_relative "../../../apache/arrow/flatbuf/interval_unit"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ class Interval < ::FlatBuffers::Table
+ def unit
+ field_offset = @view.unpack_virtual_offset(4)
+ if field_offset.zero?
+ enum_value = 0
+ else
+ enum_value = @view.unpack_short(field_offset)
+ end
+ ::ArrowFormat::Org::Apache::Arrow::Flatbuf::IntervalUnit.try_convert(enum_value) || enum_value
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/interval_unit.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/interval_unit.rb
new file mode 100644
index 0000000000..f4866d5287
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/interval_unit.rb
@@ -0,0 +1,23 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ class IntervalUnit < ::FlatBuffers::Enum
+ YEAR_MONTH = register("YEAR_MONTH", 0)
+ DAY_TIME = register("DAY_TIME", 1)
+ MONTH_DAY_NANO = register("MONTH_DAY_NANO", 2)
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/key_value.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/key_value.rb
new file mode 100644
index 0000000000..0faed748e2
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/key_value.rb
@@ -0,0 +1,36 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # ----------------------------------------------------------------------
+ # user defined key value pairs to add custom metadata to arrow
+ # key namespacing is the responsibility of the user
+ class KeyValue < ::FlatBuffers::Table
+ def key
+ field_offset = @view.unpack_virtual_offset(4)
+ return nil if field_offset.zero?
+
+ @view.unpack_string(field_offset)
+ end
+
+ def value
+ field_offset = @view.unpack_virtual_offset(6)
+ return nil if field_offset.zero?
+
+ @view.unpack_string(field_offset)
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/large_binary.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/large_binary.rb
new file mode 100644
index 0000000000..ef9bfe050e
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/large_binary.rb
@@ -0,0 +1,22 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # Same as Binary, but with 64-bit offsets, allowing to represent
+ # extremely large data values.
+ class LargeBinary < ::FlatBuffers::Table
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/large_list.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/large_list.rb
new file mode 100644
index 0000000000..e64d3b25a7
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/large_list.rb
@@ -0,0 +1,22 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # Same as List, but with 64-bit offsets, allowing to represent
+ # extremely large data values.
+ class LargeList < ::FlatBuffers::Table
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/large_list_view.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/large_list_view.rb
new file mode 100644
index 0000000000..8ed57eaef5
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/large_list_view.rb
@@ -0,0 +1,22 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # Same as ListView, but with 64-bit offsets and sizes, allowing to represent
+ # extremely large data values.
+ class LargeListView < ::FlatBuffers::Table
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/large_utf8.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/large_utf8.rb
new file mode 100644
index 0000000000..3d147edf20
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/large_utf8.rb
@@ -0,0 +1,22 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # Same as Utf8, but with 64-bit offsets, allowing to represent
+ # extremely large data values.
+ class LargeUtf8 < ::FlatBuffers::Table
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/list.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/list.rb
new file mode 100644
index 0000000000..1155706222
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/list.rb
@@ -0,0 +1,20 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ class List < ::FlatBuffers::Table
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/list_view.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/list_view.rb
new file mode 100644
index 0000000000..057f59f687
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/list_view.rb
@@ -0,0 +1,23 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # Represents the same logical types that List can, but contains offsets and
+ # sizes allowing for writes in any order and sharing of child values among
+ # list values.
+ class ListView < ::FlatBuffers::Table
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/map.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/map.rb
new file mode 100644
index 0000000000..366833e52f
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/map.rb
@@ -0,0 +1,52 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # A Map is a logical nested type that is represented as
+ #
+ # List<entries: Struct<key: K, value: V>>
+ #
+ # In this layout, the keys and values are each respectively contiguous. We do
+ # not constrain the key and value types, so the application is responsible
+ # for ensuring that the keys are hashable and unique. Whether the keys are sorted
+ # may be set in the metadata for this field.
+ #
+ # In a field with Map type, the field has a child Struct field, which then
+ # has two children: key type and the second the value type. The names of the
+ # child fields may be respectively "entries", "key", and "value", but this is
+ # not enforced.
+ #
+ # Map
+ # ```text
+ # - child[0] entries: Struct
+ # - child[0] key: K
+ # - child[1] value: V
+ # ```
+ # Neither the "entries" field nor the "key" field may be nullable.
+ #
+ # The metadata is structured so that Arrow systems without special handling
+ # for Map can make Map an alias for List. The "layout" attribute for the Map
+ # field must have the same contents as a List.
+ class Map < ::FlatBuffers::Table
+ # Set to true if the keys within each value are sorted
+ def keys_sorted?
+ field_offset = @view.unpack_virtual_offset(4)
+ return false if field_offset.zero?
+
+ @view.unpack_bool(field_offset)
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/message.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/message.rb
new file mode 100644
index 0000000000..b4d1d76689
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/message.rb
@@ -0,0 +1,68 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Message.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+require_relative "../../../apache/arrow/flatbuf/key_value"
+require_relative "../../../apache/arrow/flatbuf/message_header"
+require_relative "../../../apache/arrow/flatbuf/metadata_version"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ class Message < ::FlatBuffers::Table
+ def body_length
+ field_offset = @view.unpack_virtual_offset(10)
+ return 0 if field_offset.zero?
+
+ @view.unpack_long(field_offset)
+ end
+
+ def custom_metadata
+ field_offset = @view.unpack_virtual_offset(12)
+ return nil if field_offset.zero?
+
+ element_size = 4
+ @view.unpack_vector(field_offset, element_size) do |element_offset|
+ @view.unpack_table(::ArrowFormat::Org::Apache::Arrow::Flatbuf::KeyValue, element_offset)
+ end
+ end
+
+ def header
+ type = header_type
+ return nil if type.nil?
+
+ field_offset = @view.unpack_virtual_offset(8)
+ return nil if field_offset.zero?
+ @view.unpack_union(type.table_class, field_offset)
+ end
+
+ def header_type
+ field_offset = @view.unpack_virtual_offset(6)
+ if field_offset.zero?
+ enum_value = 0
+ else
+ enum_value = @view.unpack_utype(field_offset)
+ end
+ ::ArrowFormat::Org::Apache::Arrow::Flatbuf::MessageHeader.try_convert(enum_value) || enum_value
+ end
+
+ def version
+ field_offset = @view.unpack_virtual_offset(4)
+ if field_offset.zero?
+ enum_value = 0
+ else
+ enum_value = @view.unpack_short(field_offset)
+ end
+ ::ArrowFormat::Org::Apache::Arrow::Flatbuf::MetadataVersion.try_convert(enum_value) || enum_value
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/message_header.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/message_header.rb
new file mode 100644
index 0000000000..13e3f9b1f1
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/message_header.rb
@@ -0,0 +1,39 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Message.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # ----------------------------------------------------------------------
+ # The root Message type
+ # This union enables us to easily send different message types without
+ # redundant storage, and in the future we can easily add new message types.
+ #
+ # Arrow implementations do not need to implement all of the message types,
+ # which may include experimental metadata types. For maximum compatibility,
+ # it is best to send data using RecordBatch
+ class MessageHeader < ::FlatBuffers::Union
+ NONE = register("NONE", 0, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::Utf8View", "../../../apache/arrow/flatbuf/utf8view")
+ SCHEMA = register("Schema", 1, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::Schema", "../../../apache/arrow/flatbuf/schema")
+ DICTIONARY_BATCH = register("DictionaryBatch", 2, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::DictionaryBatch", "../../../apache/arrow/flatbuf/dictionary_batch")
+ RECORD_BATCH = register("RecordBatch", 3, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::RecordBatch", "../../../apache/arrow/flatbuf/record_batch")
+ TENSOR = register("Tensor", 4, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::Tensor", "../../../apache/arrow/flatbuf/tensor")
+ SPARSE_TENSOR = register("SparseTensor", 5, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::SparseTensor", "../../../apache/arrow/flatbuf/sparse_tensor")
+
+
+ private def require_table_class
+ require_relative @require_path
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/metadata_version.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/metadata_version.rb
new file mode 100644
index 0000000000..d144c82d62
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/metadata_version.rb
@@ -0,0 +1,36 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ class MetadataVersion < ::FlatBuffers::Enum
+ # 0.1.0 (October 2016).
+ V1 = register("V1", 0)
+ # 0.2.0 (February 2017). Non-backwards compatible with V1.
+ V2 = register("V2", 1)
+ # 0.3.0 -> 0.7.1 (May - December 2017). Non-backwards compatible with V2.
+ V3 = register("V3", 2)
+ # >= 0.8.0 (December 2017). Non-backwards compatible with V3.
+ V4 = register("V4", 3)
+ # >= 1.0.0 (July 2020). Backwards compatible with V4 (V5 readers can read V4
+ # metadata and IPC messages). Implementations are recommended to provide a
+ # V4 compatibility mode with V5 format changes disabled.
+ #
+ # Incompatible changes between V4 and V5:
+ # - Union buffer layout has changed. In V5, Unions don't have a validity
+ # bitmap buffer.
+ V5 = register("V5", 4)
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/null.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/null.rb
new file mode 100644
index 0000000000..2bfab0246d
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/null.rb
@@ -0,0 +1,21 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # These are stored in the flatbuffer in the Type union below
+ class Null < ::FlatBuffers::Table
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/precision.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/precision.rb
new file mode 100644
index 0000000000..9196ffcd42
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/precision.rb
@@ -0,0 +1,23 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ class Precision < ::FlatBuffers::Enum
+ HALF = register("HALF", 0)
+ SINGLE = register("SINGLE", 1)
+ DOUBLE = register("DOUBLE", 2)
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/record_batch.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/record_batch.rb
new file mode 100644
index 0000000000..80b70454a3
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/record_batch.rb
@@ -0,0 +1,93 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Message.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+require_relative "../../../apache/arrow/flatbuf/buffer"
+require_relative "../../../apache/arrow/flatbuf/body_compression"
+require_relative "../../../apache/arrow/flatbuf/field_node"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # A data header describing the shared memory layout of a "record" or "row"
+ # batch. Some systems call this a "row batch" internally and others a "record
+ # batch".
+ class RecordBatch < ::FlatBuffers::Table
+ # Buffers correspond to the pre-ordered flattened buffer tree
+ #
+ # The number of buffers appended to this list depends on the schema. For
+ # example, most primitive arrays will have 2 buffers, 1 for the validity
+ # bitmap and 1 for the values. For struct arrays, there will only be a
+ # single buffer for the validity (nulls) bitmap
+ def buffers
+ field_offset = @view.unpack_virtual_offset(8)
+ return nil if field_offset.zero?
+
+ element_size = 16
+ @view.unpack_vector(field_offset, element_size) do |element_offset|
+ @view.unpack_struct(::ArrowFormat::Org::Apache::Arrow::Flatbuf::Buffer, element_offset)
+ end
+ end
+
+ # Optional compression of the message body
+ def compression
+ field_offset = @view.unpack_virtual_offset(10)
+ return nil if field_offset.zero?
+
+ @view.unpack_table(::ArrowFormat::Org::Apache::Arrow::Flatbuf::BodyCompression, field_offset)
+ end
+
+ # number of records / rows. The arrays in the batch should all have this
+ # length
+ def length
+ field_offset = @view.unpack_virtual_offset(4)
+ return 0 if field_offset.zero?
+
+ @view.unpack_long(field_offset)
+ end
+
+ # Nodes correspond to the pre-ordered flattened logical schema
+ def nodes
+ field_offset = @view.unpack_virtual_offset(6)
+ return nil if field_offset.zero?
+
+ element_size = 16
+ @view.unpack_vector(field_offset, element_size) do |element_offset|
+ @view.unpack_struct(::ArrowFormat::Org::Apache::Arrow::Flatbuf::FieldNode, element_offset)
+ end
+ end
+
+ # Some types such as Utf8View are represented using a variable number of buffers.
+ # For each such Field in the pre-ordered flattened logical schema, there will be
+ # an entry in variadicBufferCounts to indicate the number of number of variadic
+ # buffers which belong to that Field in the current RecordBatch.
+ #
+ # For example, the schema
+ # col1: Struct<alpha: Int32, beta: BinaryView, gamma: Float64>
+ # col2: Utf8View
+ # contains two Fields with variadic buffers so variadicBufferCounts will have
+ # two entries, the first counting the variadic buffers of `col1.beta` and the
+ # second counting `col2`'s.
+ #
+ # This field may be omitted if and only if the schema contains no Fields with
+ # a variable number of buffers, such as BinaryView and Utf8View.
+ def variadic_buffer_counts
+ field_offset = @view.unpack_virtual_offset(12)
+ return nil if field_offset.zero?
+
+ element_size = 8
+ @view.unpack_vector(field_offset, element_size) do |element_offset|
+ @view.unpack_long(element_offset)
+ end
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/run_end_encoded.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/run_end_encoded.rb
new file mode 100644
index 0000000000..7141da0f31
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/run_end_encoded.rb
@@ -0,0 +1,25 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # Contains two child arrays, run_ends and values.
+ # The run_ends child array must be a 16/32/64-bit integer array
+ # which encodes the indices at which the run with the value in
+ # each corresponding index in the values child array ends.
+ # Like list/struct types, the value array can be of any type.
+ class RunEndEncoded < ::FlatBuffers::Table
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/schema.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/schema.rb
new file mode 100644
index 0000000000..5997c409a9
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/schema.rb
@@ -0,0 +1,68 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+require_relative "../../../apache/arrow/flatbuf/key_value"
+require_relative "../../../apache/arrow/flatbuf/endianness"
+require_relative "../../../apache/arrow/flatbuf/field"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # ----------------------------------------------------------------------
+ # A Schema describes the columns in a row batch
+ class Schema < ::FlatBuffers::Table
+ def custom_metadata
+ field_offset = @view.unpack_virtual_offset(8)
+ return nil if field_offset.zero?
+
+ element_size = 4
+ @view.unpack_vector(field_offset, element_size) do |element_offset|
+ @view.unpack_table(::ArrowFormat::Org::Apache::Arrow::Flatbuf::KeyValue, element_offset)
+ end
+ end
+
+ # endianness of the buffer
+ # it is Little Endian by default
+ # if endianness doesn't match the underlying system then the vectors need to be converted
+ def endianness
+ field_offset = @view.unpack_virtual_offset(4)
+ if field_offset.zero?
+ enum_value = 0
+ else
+ enum_value = @view.unpack_short(field_offset)
+ end
+ ::ArrowFormat::Org::Apache::Arrow::Flatbuf::Endianness.try_convert(enum_value) || enum_value
+ end
+
+ # Features used in the stream/file.
+ def features
+ field_offset = @view.unpack_virtual_offset(10)
+ return nil if field_offset.zero?
+
+ element_size = 8
+ @view.unpack_vector(field_offset, element_size) do |element_offset|
+ @view.unpack_long(element_offset)
+ end
+ end
+
+ def fields
+ field_offset = @view.unpack_virtual_offset(6)
+ return nil if field_offset.zero?
+
+ element_size = 4
+ @view.unpack_vector(field_offset, element_size) do |element_offset|
+ @view.unpack_table(::ArrowFormat::Org::Apache::Arrow::Flatbuf::Field, element_offset)
+ end
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/sparse_matrix_compressed_axis.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/sparse_matrix_compressed_axis.rb
new file mode 100644
index 0000000000..ad8b63e895
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/sparse_matrix_compressed_axis.rb
@@ -0,0 +1,22 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //SparseTensor.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ class SparseMatrixCompressedAxis < ::FlatBuffers::Enum
+ ROW = register("Row", 0)
+ COLUMN = register("Column", 1)
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/sparse_matrix_index_csx.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/sparse_matrix_index_csx.rb
new file mode 100644
index 0000000000..f1ab5be5f8
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/sparse_matrix_index_csx.rb
@@ -0,0 +1,96 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //SparseTensor.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+require_relative "../../../apache/arrow/flatbuf/sparse_matrix_compressed_axis"
+require_relative "../../../apache/arrow/flatbuf/buffer"
+require_relative "../../../apache/arrow/flatbuf/int"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # Compressed Sparse format, that is matrix-specific.
+ class SparseMatrixIndexCSX < ::FlatBuffers::Table
+ # Which axis, row or column, is compressed
+ def compressed_axis
+ field_offset = @view.unpack_virtual_offset(4)
+ if field_offset.zero?
+ enum_value = 0
+ else
+ enum_value = @view.unpack_short(field_offset)
+ end
+ ::ArrowFormat::Org::Apache::Arrow::Flatbuf::SparseMatrixCompressedAxis.try_convert(enum_value) || enum_value
+ end
+
+ # indicesBuffer stores the location and size of the array that
+ # contains the column indices of the corresponding non-zero values.
+ # The type of index value is long.
+ #
+ # For example, the indices of the above X is:
+ # ```text
+ # indices(X) = [1, 2, 2, 1, 3, 0, 2, 3, 1].
+ # ```
+ # Note that the indices are sorted in lexicographical order for each row.
+ def indices_buffer
+ field_offset = @view.unpack_virtual_offset(12)
+ return nil if field_offset.zero?
+
+ @view.unpack_struct(::ArrowFormat::Org::Apache::Arrow::Flatbuf::Buffer, field_offset)
+ end
+
+ # The type of values in indicesBuffer
+ def indices_type
+ field_offset = @view.unpack_virtual_offset(10)
+ return nil if field_offset.zero?
+
+ @view.unpack_table(::ArrowFormat::Org::Apache::Arrow::Flatbuf::Int, field_offset)
+ end
+
+ # indptrBuffer stores the location and size of indptr array that
+ # represents the range of the rows.
+ # The i-th row spans from `indptr[i]` to `indptr[i+1]` in the data.
+ # The length of this array is 1 + (the number of rows), and the type
+ # of index value is long.
+ #
+ # For example, let X be the following 6x4 matrix:
+ # ```text
+ # X := [[0, 1, 2, 0],
+ # [0, 0, 3, 0],
+ # [0, 4, 0, 5],
+ # [0, 0, 0, 0],
+ # [6, 0, 7, 8],
+ # [0, 9, 0, 0]].
+ # ```
+ # The array of non-zero values in X is:
+ # ```text
+ # values(X) = [1, 2, 3, 4, 5, 6, 7, 8, 9].
+ # ```
+ # And the indptr of X is:
+ # ```text
+ # indptr(X) = [0, 2, 3, 5, 5, 8, 10].
+ # ```
+ def indptr_buffer
+ field_offset = @view.unpack_virtual_offset(8)
+ return nil if field_offset.zero?
+
+ @view.unpack_struct(::ArrowFormat::Org::Apache::Arrow::Flatbuf::Buffer, field_offset)
+ end
+
+ # The type of values in indptrBuffer
+ def indptr_type
+ field_offset = @view.unpack_virtual_offset(6)
+ return nil if field_offset.zero?
+
+ @view.unpack_table(::ArrowFormat::Org::Apache::Arrow::Flatbuf::Int, field_offset)
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/sparse_tensor.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/sparse_tensor.rb
new file mode 100644
index 0000000000..ade483a487
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/sparse_tensor.rb
@@ -0,0 +1,92 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //SparseTensor.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+require_relative "../../../apache/arrow/flatbuf/buffer"
+require_relative "../../../apache/arrow/flatbuf/tensor_dim"
+require_relative "../../../apache/arrow/flatbuf/sparse_tensor_index"
+require_relative "../../../apache/arrow/flatbuf/type"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ class SparseTensor < ::FlatBuffers::Table
+ # The location and size of the tensor's data
+ def data
+ field_offset = @view.unpack_virtual_offset(16)
+ return nil if field_offset.zero?
+
+ @view.unpack_struct(::ArrowFormat::Org::Apache::Arrow::Flatbuf::Buffer, field_offset)
+ end
+
+ # The number of non-zero values in a sparse tensor.
+ def non_zero_length
+ field_offset = @view.unpack_virtual_offset(10)
+ return 0 if field_offset.zero?
+
+ @view.unpack_long(field_offset)
+ end
+
+ # The dimensions of the tensor, optionally named.
+ def shape
+ field_offset = @view.unpack_virtual_offset(8)
+ return nil if field_offset.zero?
+
+ element_size = 4
+ @view.unpack_vector(field_offset, element_size) do |element_offset|
+ @view.unpack_table(::ArrowFormat::Org::Apache::Arrow::Flatbuf::TensorDim, element_offset)
+ end
+ end
+
+ # Sparse tensor index
+ def sparse_index
+ type = sparse_index_type
+ return nil if type.nil?
+
+ field_offset = @view.unpack_virtual_offset(14)
+ return nil if field_offset.zero?
+ @view.unpack_union(type.table_class, field_offset)
+ end
+
+ def sparse_index_type
+ field_offset = @view.unpack_virtual_offset(12)
+ if field_offset.zero?
+ enum_value = 0
+ else
+ enum_value = @view.unpack_utype(field_offset)
+ end
+ ::ArrowFormat::Org::Apache::Arrow::Flatbuf::SparseTensorIndex.try_convert(enum_value) || enum_value
+ end
+
+ # The type of data contained in a value cell.
+ # Currently only fixed-width value types are supported,
+ # no strings or nested types.
+ def type
+ type = type_type
+ return nil if type.nil?
+
+ field_offset = @view.unpack_virtual_offset(6)
+ return nil if field_offset.zero?
+ @view.unpack_union(type.table_class, field_offset)
+ end
+
+ def type_type
+ field_offset = @view.unpack_virtual_offset(4)
+ if field_offset.zero?
+ enum_value = 0
+ else
+ enum_value = @view.unpack_utype(field_offset)
+ end
+ ::ArrowFormat::Org::Apache::Arrow::Flatbuf::Type.try_convert(enum_value) || enum_value
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/sparse_tensor_index.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/sparse_tensor_index.rb
new file mode 100644
index 0000000000..a857b7fe6e
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/sparse_tensor_index.rb
@@ -0,0 +1,29 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //SparseTensor.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ class SparseTensorIndex < ::FlatBuffers::Union
+ NONE = register("NONE", 0, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::Utf8View", "../../../apache/arrow/flatbuf/utf8view")
+ SPARSE_TENSOR_INDEX_COO = register("SparseTensorIndexCOO", 1, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::SparseTensorIndexCOO", "../../../apache/arrow/flatbuf/sparse_tensor_index_coo")
+ SPARSE_MATRIX_INDEX_CSX = register("SparseMatrixIndexCSX", 2, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::SparseMatrixIndexCSX", "../../../apache/arrow/flatbuf/sparse_matrix_index_csx")
+ SPARSE_TENSOR_INDEX_CSF = register("SparseTensorIndexCSF", 3, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::SparseTensorIndexCSF", "../../../apache/arrow/flatbuf/sparse_tensor_index_csf")
+
+
+ private def require_table_class
+ require_relative @require_path
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/sparse_tensor_index_coo.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/sparse_tensor_index_coo.rb
new file mode 100644
index 0000000000..0807721a04
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/sparse_tensor_index_coo.rb
@@ -0,0 +1,93 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //SparseTensor.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+require_relative "../../../apache/arrow/flatbuf/buffer"
+require_relative "../../../apache/arrow/flatbuf/int"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # ----------------------------------------------------------------------
+ # EXPERIMENTAL: Data structures for sparse tensors
+ # Coordinate (COO) format of sparse tensor index.
+ #
+ # COO's index list are represented as a NxM matrix,
+ # where N is the number of non-zero values,
+ # and M is the number of dimensions of a sparse tensor.
+ #
+ # indicesBuffer stores the location and size of the data of this indices
+ # matrix. The value type and the stride of the indices matrix is
+ # specified in indicesType and indicesStrides fields.
+ #
+ # For example, let X be a 2x3x4x5 tensor, and it has the following
+ # 6 non-zero values:
+ # ```text
+ # X[0, 1, 2, 0] := 1
+ # X[1, 1, 2, 3] := 2
+ # X[0, 2, 1, 0] := 3
+ # X[0, 1, 3, 0] := 4
+ # X[0, 1, 2, 1] := 5
+ # X[1, 2, 0, 4] := 6
+ # ```
+ # In COO format, the index matrix of X is the following 4x6 matrix:
+ # ```text
+ # [[0, 0, 0, 0, 1, 1],
+ # [1, 1, 1, 2, 1, 2],
+ # [2, 2, 3, 1, 2, 0],
+ # [0, 1, 0, 0, 3, 4]]
+ # ```
+ # When isCanonical is true, the indices is sorted in lexicographical order
+ # (row-major order), and it does not have duplicated entries. Otherwise,
+ # the indices may not be sorted, or may have duplicated entries.
+ class SparseTensorIndexCOO < ::FlatBuffers::Table
+ # The location and size of the indices matrix's data
+ def indices_buffer
+ field_offset = @view.unpack_virtual_offset(8)
+ return nil if field_offset.zero?
+
+ @view.unpack_struct(::ArrowFormat::Org::Apache::Arrow::Flatbuf::Buffer, field_offset)
+ end
+
+ # Non-negative byte offsets to advance one value cell along each dimension
+ # If omitted, default to row-major order (C-like).
+ def indices_strides
+ field_offset = @view.unpack_virtual_offset(6)
+ return nil if field_offset.zero?
+
+ element_size = 8
+ @view.unpack_vector(field_offset, element_size) do |element_offset|
+ @view.unpack_long(element_offset)
+ end
+ end
+
+ # The type of values in indicesBuffer
+ def indices_type
+ field_offset = @view.unpack_virtual_offset(4)
+ return nil if field_offset.zero?
+
+ @view.unpack_table(::ArrowFormat::Org::Apache::Arrow::Flatbuf::Int, field_offset)
+ end
+
+ # This flag is true if and only if the indices matrix is sorted in
+ # row-major order, and does not have duplicated entries.
+ # This sort order is the same as of Tensorflow's SparseTensor,
+ # but it is inverse order of SciPy's canonical coo_matrix
+ # (SciPy employs column-major order for its coo_matrix).
+ def canonical?
+ field_offset = @view.unpack_virtual_offset(10)
+ return false if field_offset.zero?
+
+ @view.unpack_bool(field_offset)
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/sparse_tensor_index_csf.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/sparse_tensor_index_csf.rb
new file mode 100644
index 0000000000..5e86ade5c5
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/sparse_tensor_index_csf.rb
@@ -0,0 +1,129 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //SparseTensor.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+require_relative "../../../apache/arrow/flatbuf/buffer"
+require_relative "../../../apache/arrow/flatbuf/int"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # Compressed Sparse Fiber (CSF) sparse tensor index.
+ class SparseTensorIndexCSF < ::FlatBuffers::Table
+ # axisOrder stores the sequence in which dimensions were traversed to
+ # produce the prefix tree.
+ # For example, the axisOrder for the above X is:
+ # ```text
+ # axisOrder(X) = [0, 1, 2, 3].
+ # ```
+ def axis_order
+ field_offset = @view.unpack_virtual_offset(12)
+ return nil if field_offset.zero?
+
+ element_size = 4
+ @view.unpack_vector(field_offset, element_size) do |element_offset|
+ @view.unpack_int(element_offset)
+ end
+ end
+
+ # indicesBuffers stores values of nodes.
+ # Each tensor dimension corresponds to a buffer in indicesBuffers.
+ # For example, the indicesBuffers for the above X is:
+ # ```text
+ # indicesBuffer(X) = [
+ # [0, 1],
+ # [0, 1, 1],
+ # [0, 0, 1, 1],
+ # [1, 2, 0, 2, 0, 0, 1, 2]
+ # ].
+ # ```
+ def indices_buffers
+ field_offset = @view.unpack_virtual_offset(10)
+ return nil if field_offset.zero?
+
+ element_size = 16
+ @view.unpack_vector(field_offset, element_size) do |element_offset|
+ @view.unpack_struct(::ArrowFormat::Org::Apache::Arrow::Flatbuf::Buffer, element_offset)
+ end
+ end
+
+ # The type of values in indicesBuffers
+ def indices_type
+ field_offset = @view.unpack_virtual_offset(8)
+ return nil if field_offset.zero?
+
+ @view.unpack_table(::ArrowFormat::Org::Apache::Arrow::Flatbuf::Int, field_offset)
+ end
+
+ # indptrBuffers stores the sparsity structure.
+ # Each two consecutive dimensions in a tensor correspond to a buffer in
+ # indptrBuffers. A pair of consecutive values at `indptrBuffers[dim][i]`
+ # and `indptrBuffers[dim][i + 1]` signify a range of nodes in
+ # `indicesBuffers[dim + 1]` who are children of `indicesBuffers[dim][i]` node.
+ #
+ # For example, the indptrBuffers for the above X is:
+ # ```text
+ # indptrBuffer(X) = [
+ # [0, 2, 3],
+ # [0, 1, 3, 4],
+ # [0, 2, 4, 5, 8]
+ # ].
+ # ```
+ def indptr_buffers
+ field_offset = @view.unpack_virtual_offset(6)
+ return nil if field_offset.zero?
+
+ element_size = 16
+ @view.unpack_vector(field_offset, element_size) do |element_offset|
+ @view.unpack_struct(::ArrowFormat::Org::Apache::Arrow::Flatbuf::Buffer, element_offset)
+ end
+ end
+
+ # CSF is a generalization of compressed sparse row (CSR) index.
+ # See [smith2017knl](http://shaden.io/pub-files/smith2017knl.pdf)
+ #
+ # CSF index recursively compresses each dimension of a tensor into a set
+ # of prefix trees. Each path from a root to leaf forms one tensor
+ # non-zero index. CSF is implemented with two arrays of buffers and one
+ # arrays of integers.
+ #
+ # For example, let X be a 2x3x4x5 tensor and let it have the following
+ # 8 non-zero values:
+ # ```text
+ # X[0, 0, 0, 1] := 1
+ # X[0, 0, 0, 2] := 2
+ # X[0, 1, 0, 0] := 3
+ # X[0, 1, 0, 2] := 4
+ # X[0, 1, 1, 0] := 5
+ # X[1, 1, 1, 0] := 6
+ # X[1, 1, 1, 1] := 7
+ # X[1, 1, 1, 2] := 8
+ # ```
+ # As a prefix tree this would be represented as:
+ # ```text
+ # 0 1
+ # / \ |
+ # 0 1 1
+ # / / \ |
+ # 0 0 1 1
+ # /| /| | /| |
+ # 1 2 0 2 0 0 1 2
+ # ```
+ # The type of values in indptrBuffers
+ def indptr_type
+ field_offset = @view.unpack_virtual_offset(4)
+ return nil if field_offset.zero?
+
+ @view.unpack_table(::ArrowFormat::Org::Apache::Arrow::Flatbuf::Int, field_offset)
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/struct_.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/struct_.rb
new file mode 100644
index 0000000000..0b2808716a
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/struct_.rb
@@ -0,0 +1,23 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # A Struct_ in the flatbuffer metadata is the same as an Arrow Struct
+ # (according to the physical memory layout). We used Struct_ here as
+ # Struct is a reserved word in Flatbuffers
+ class Struct < ::FlatBuffers::Table
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/tensor.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/tensor.rb
new file mode 100644
index 0000000000..b8097cec22
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/tensor.rb
@@ -0,0 +1,74 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Tensor.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+require_relative "../../../apache/arrow/flatbuf/buffer"
+require_relative "../../../apache/arrow/flatbuf/tensor_dim"
+require_relative "../../../apache/arrow/flatbuf/type"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ class Tensor < ::FlatBuffers::Table
+ # The location and size of the tensor's data
+ def data
+ field_offset = @view.unpack_virtual_offset(12)
+ return nil if field_offset.zero?
+
+ @view.unpack_struct(::ArrowFormat::Org::Apache::Arrow::Flatbuf::Buffer, field_offset)
+ end
+
+ # The dimensions of the tensor, optionally named
+ def shape
+ field_offset = @view.unpack_virtual_offset(8)
+ return nil if field_offset.zero?
+
+ element_size = 4
+ @view.unpack_vector(field_offset, element_size) do |element_offset|
+ @view.unpack_table(::ArrowFormat::Org::Apache::Arrow::Flatbuf::TensorDim, element_offset)
+ end
+ end
+
+ # Non-negative byte offsets to advance one value cell along each dimension
+ # If omitted, default to row-major order (C-like).
+ def strides
+ field_offset = @view.unpack_virtual_offset(10)
+ return nil if field_offset.zero?
+
+ element_size = 8
+ @view.unpack_vector(field_offset, element_size) do |element_offset|
+ @view.unpack_long(element_offset)
+ end
+ end
+
+ # The type of data contained in a value cell. Currently only fixed-width
+ # value types are supported, no strings or nested types
+ def type
+ type = type_type
+ return nil if type.nil?
+
+ field_offset = @view.unpack_virtual_offset(6)
+ return nil if field_offset.zero?
+ @view.unpack_union(type.table_class, field_offset)
+ end
+
+ def type_type
+ field_offset = @view.unpack_virtual_offset(4)
+ if field_offset.zero?
+ enum_value = 0
+ else
+ enum_value = @view.unpack_utype(field_offset)
+ end
+ ::ArrowFormat::Org::Apache::Arrow::Flatbuf::Type.try_convert(enum_value) || enum_value
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/tensor_dim.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/tensor_dim.rb
new file mode 100644
index 0000000000..452c1dd60c
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/tensor_dim.rb
@@ -0,0 +1,38 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Tensor.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # ----------------------------------------------------------------------
+ # Data structures for dense tensors
+ # Shape data for a single axis in a tensor
+ class TensorDim < ::FlatBuffers::Table
+ # Name of the dimension, optional
+ def name
+ field_offset = @view.unpack_virtual_offset(6)
+ return nil if field_offset.zero?
+
+ @view.unpack_string(field_offset)
+ end
+
+ # Length of dimension
+ def size
+ field_offset = @view.unpack_virtual_offset(4)
+ return 0 if field_offset.zero?
+
+ @view.unpack_long(field_offset)
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/time.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/time.rb
new file mode 100644
index 0000000000..eb5c0b2705
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/time.rb
@@ -0,0 +1,51 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+require_relative "../../../apache/arrow/flatbuf/time_unit"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # Time is either a 32-bit or 64-bit signed integer type representing an
+ # elapsed time since midnight, stored in either of four units: seconds,
+ # milliseconds, microseconds or nanoseconds.
+ #
+ # The integer `bitWidth` depends on the `unit` and must be one of the following:
+ # * SECOND and MILLISECOND: 32 bits
+ # * MICROSECOND and NANOSECOND: 64 bits
+ #
+ # The allowed values are between 0 (inclusive) and 86400 (=24*60*60) seconds
+ # (exclusive), adjusted for the time unit (for example, up to 86400000
+ # exclusive for the MILLISECOND unit).
+ # This definition doesn't allow for leap seconds. Time values from
+ # measurements with leap seconds will need to be corrected when ingesting
+ # into Arrow (for example by replacing the value 86400 with 86399).
+ class Time < ::FlatBuffers::Table
+ def bit_width
+ field_offset = @view.unpack_virtual_offset(6)
+ return 32 if field_offset.zero?
+
+ @view.unpack_int(field_offset)
+ end
+
+ def unit
+ field_offset = @view.unpack_virtual_offset(4)
+ if field_offset.zero?
+ enum_value = 1
+ else
+ enum_value = @view.unpack_short(field_offset)
+ end
+ ::ArrowFormat::Org::Apache::Arrow::Flatbuf::TimeUnit.try_convert(enum_value) || enum_value
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/time_unit.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/time_unit.rb
new file mode 100644
index 0000000000..b01d95cdc3
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/time_unit.rb
@@ -0,0 +1,24 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ class TimeUnit < ::FlatBuffers::Enum
+ SECOND = register("SECOND", 0)
+ MILLISECOND = register("MILLISECOND", 1)
+ MICROSECOND = register("MICROSECOND", 2)
+ NANOSECOND = register("NANOSECOND", 3)
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/timestamp.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/timestamp.rb
new file mode 100644
index 0000000000..6d898a456a
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/timestamp.rb
@@ -0,0 +1,152 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+require_relative "../../../apache/arrow/flatbuf/time_unit"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # Timestamp is a 64-bit signed integer representing an elapsed time since a
+ # fixed epoch, stored in either of four units: seconds, milliseconds,
+ # microseconds or nanoseconds, and is optionally annotated with a timezone.
+ #
+ # Timestamp values do not include any leap seconds (in other words, all
+ # days are considered 86400 seconds long).
+ #
+ # Timestamps with a non-empty timezone
+ # ------------------------------------
+ #
+ # If a Timestamp column has a non-empty timezone value, its epoch is
+ # 1970-01-01 00:00:00 (January 1st 1970, midnight) in the *UTC* timezone
+ # (the Unix epoch), regardless of the Timestamp's own timezone.
+ #
+ # Therefore, timestamp values with a non-empty timezone correspond to
+ # physical points in time together with some additional information about
+ # how the data was obtained and/or how to display it (the timezone).
+ #
+ # For example, the timestamp value 0 with the timezone string "Europe/Paris"
+ # corresponds to "January 1st 1970, 00h00" in the UTC timezone, but the
+ # application may prefer to display it as "January 1st 1970, 01h00" in
+ # the Europe/Paris timezone (which is the same physical point in time).
+ #
+ # One consequence is that timestamp values with a non-empty timezone
+ # can be compared and ordered directly, since they all share the same
+ # well-known point of reference (the Unix epoch).
+ #
+ # Timestamps with an unset / empty timezone
+ # -----------------------------------------
+ #
+ # If a Timestamp column has no timezone value, its epoch is
+ # 1970-01-01 00:00:00 (January 1st 1970, midnight) in an *unknown* timezone.
+ #
+ # Therefore, timestamp values without a timezone cannot be meaningfully
+ # interpreted as physical points in time, but only as calendar / clock
+ # indications ("wall clock time") in an unspecified timezone.
+ #
+ # For example, the timestamp value 0 with an empty timezone string
+ # corresponds to "January 1st 1970, 00h00" in an unknown timezone: there
+ # is not enough information to interpret it as a well-defined physical
+ # point in time.
+ #
+ # One consequence is that timestamp values without a timezone cannot
+ # be reliably compared or ordered, since they may have different points of
+ # reference. In particular, it is *not* possible to interpret an unset
+ # or empty timezone as the same as "UTC".
+ #
+ # Conversion between timezones
+ # ----------------------------
+ #
+ # If a Timestamp column has a non-empty timezone, changing the timezone
+ # to a different non-empty value is a metadata-only operation:
+ # the timestamp values need not change as their point of reference remains
+ # the same (the Unix epoch).
+ #
+ # However, if a Timestamp column has no timezone value, changing it to a
+ # non-empty value requires to think about the desired semantics.
+ # One possibility is to assume that the original timestamp values are
+ # relative to the epoch of the timezone being set; timestamp values should
+ # then adjusted to the Unix epoch (for example, changing the timezone from
+ # empty to "Europe/Paris" would require converting the timestamp values
+ # from "Europe/Paris" to "UTC", which seems counter-intuitive but is
+ # nevertheless correct).
+ #
+ # Guidelines for encoding data from external libraries
+ # ----------------------------------------------------
+ #
+ # Date & time libraries often have multiple different data types for temporal
+ # data. In order to ease interoperability between different implementations the
+ # Arrow project has some recommendations for encoding these types into a Timestamp
+ # column.
+ #
+ # An "instant" represents a physical point in time that has no relevant timezone
+ # (for example, astronomical data). To encode an instant, use a Timestamp with
+ # the timezone string set to "UTC", and make sure the Timestamp values
+ # are relative to the UTC epoch (January 1st 1970, midnight).
+ #
+ # A "zoned date-time" represents a physical point in time annotated with an
+ # informative timezone (for example, the timezone in which the data was
+ # recorded). To encode a zoned date-time, use a Timestamp with the timezone
+ # string set to the name of the timezone, and make sure the Timestamp values
+ # are relative to the UTC epoch (January 1st 1970, midnight).
+ #
+ # (There is some ambiguity between an instant and a zoned date-time with the
+ # UTC timezone. Both of these are stored the same in Arrow. Typically,
+ # this distinction does not matter. If it does, then an application should
+ # use custom metadata or an extension type to distinguish between the two cases.)
+ #
+ # An "offset date-time" represents a physical point in time combined with an
+ # explicit offset from UTC. To encode an offset date-time, use a Timestamp
+ # with the timezone string set to the numeric timezone offset string
+ # (e.g. "+03:00"), and make sure the Timestamp values are relative to
+ # the UTC epoch (January 1st 1970, midnight).
+ #
+ # A "naive date-time" (also called "local date-time" in some libraries)
+ # represents a wall clock time combined with a calendar date, but with
+ # no indication of how to map this information to a physical point in time.
+ # Naive date-times must be handled with care because of this missing
+ # information, and also because daylight saving time (DST) may make
+ # some values ambiguous or nonexistent. A naive date-time may be
+ # stored as a struct with Date and Time fields. However, it may also be
+ # encoded into a Timestamp column with an empty timezone. The timestamp
+ # values should be computed "as if" the timezone of the date-time values
+ # was UTC; for example, the naive date-time "January 1st 1970, 00h00" would
+ # be encoded as timestamp value 0.
+ class Timestamp < ::FlatBuffers::Table
+ # The timezone is an optional string indicating the name of a timezone,
+ # one of:
+ #
+ # * As used in the Olson timezone database (the "tz database" or
+ # "tzdata"), such as "America/New_York".
+ # * An absolute timezone offset of the form "+XX:XX" or "-XX:XX",
+ # such as "+07:30".
+ #
+ # Whether a timezone string is present indicates different semantics about
+ # the data (see above).
+ def timezone
+ field_offset = @view.unpack_virtual_offset(6)
+ return nil if field_offset.zero?
+
+ @view.unpack_string(field_offset)
+ end
+
+ def unit
+ field_offset = @view.unpack_virtual_offset(4)
+ if field_offset.zero?
+ enum_value = 0
+ else
+ enum_value = @view.unpack_short(field_offset)
+ end
+ ::ArrowFormat::Org::Apache::Arrow::Flatbuf::TimeUnit.try_convert(enum_value) || enum_value
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/type.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/type.rb
new file mode 100644
index 0000000000..da89a89830
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/type.rb
@@ -0,0 +1,55 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # ----------------------------------------------------------------------
+ # Top-level Type value, enabling extensible type-specific metadata. We can
+ # add new logical types to Type without breaking backwards compatibility
+ class Type < ::FlatBuffers::Union
+ NONE = register("NONE", 0, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::Utf8View", "../../../apache/arrow/flatbuf/utf8view")
+ NULL = register("Null", 1, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::Null", "../../../apache/arrow/flatbuf/null")
+ INT = register("Int", 2, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::Int", "../../../apache/arrow/flatbuf/int")
+ FLOATING_POINT = register("FloatingPoint", 3, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::FloatingPoint", "../../../apache/arrow/flatbuf/floating_point")
+ BINARY = register("Binary", 4, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::Binary", "../../../apache/arrow/flatbuf/binary")
+ UTF8 = register("Utf8", 5, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::Utf8", "../../../apache/arrow/flatbuf/utf8")
+ BOOL = register("Bool", 6, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::Bool", "../../../apache/arrow/flatbuf/bool")
+ DECIMAL = register("Decimal", 7, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::Decimal", "../../../apache/arrow/flatbuf/decimal")
+ DATE = register("Date", 8, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::Date", "../../../apache/arrow/flatbuf/date")
+ TIME = register("Time", 9, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::Time", "../../../apache/arrow/flatbuf/time")
+ TIMESTAMP = register("Timestamp", 10, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::Timestamp", "../../../apache/arrow/flatbuf/timestamp")
+ INTERVAL = register("Interval", 11, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::Interval", "../../../apache/arrow/flatbuf/interval")
+ LIST = register("List", 12, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::List", "../../../apache/arrow/flatbuf/list")
+ STRUCT_ = register("Struct_", 13, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::Struct", "../../../apache/arrow/flatbuf/struct_")
+ UNION = register("Union", 14, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::Union", "../../../apache/arrow/flatbuf/union")
+ FIXED_SIZE_BINARY = register("FixedSizeBinary", 15, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::FixedSizeBinary", "../../../apache/arrow/flatbuf/fixed_size_binary")
+ FIXED_SIZE_LIST = register("FixedSizeList", 16, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::FixedSizeList", "../../../apache/arrow/flatbuf/fixed_size_list")
+ MAP = register("Map", 17, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::Map", "../../../apache/arrow/flatbuf/map")
+ DURATION = register("Duration", 18, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::Duration", "../../../apache/arrow/flatbuf/duration")
+ LARGE_BINARY = register("LargeBinary", 19, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::LargeBinary", "../../../apache/arrow/flatbuf/large_binary")
+ LARGE_UTF8 = register("LargeUtf8", 20, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::LargeUtf8", "../../../apache/arrow/flatbuf/large_utf8")
+ LARGE_LIST = register("LargeList", 21, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::LargeList", "../../../apache/arrow/flatbuf/large_list")
+ RUN_END_ENCODED = register("RunEndEncoded", 22, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::RunEndEncoded", "../../../apache/arrow/flatbuf/run_end_encoded")
+ BINARY_VIEW = register("BinaryView", 23, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::BinaryView", "../../../apache/arrow/flatbuf/binary_view")
+ UTF8VIEW = register("Utf8View", 24, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::Utf8View", "../../../apache/arrow/flatbuf/utf8view")
+ LIST_VIEW = register("ListView", 25, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::ListView", "../../../apache/arrow/flatbuf/list_view")
+ LARGE_LIST_VIEW = register("LargeListView", 26, "::ArrowFormat::Org::Apache::Arrow::Flatbuf::LargeListView", "../../../apache/arrow/flatbuf/large_list_view")
+
+
+ private def require_table_class
+ require_relative @require_path
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/union.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/union.rb
new file mode 100644
index 0000000000..83b73e2209
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/union.rb
@@ -0,0 +1,44 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+require_relative "../../../apache/arrow/flatbuf/union_mode"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # A union is a complex type with children in Field
+ # By default ids in the type vector refer to the offsets in the children
+ # optionally typeIds provides an indirection between the child offset and the type id
+ # for each child `typeIds[offset]` is the id used in the type vector
+ class Union < ::FlatBuffers::Table
+ def mode
+ field_offset = @view.unpack_virtual_offset(4)
+ if field_offset.zero?
+ enum_value = 0
+ else
+ enum_value = @view.unpack_short(field_offset)
+ end
+ ::ArrowFormat::Org::Apache::Arrow::Flatbuf::UnionMode.try_convert(enum_value) || enum_value
+ end
+
+ def type_ids
+ field_offset = @view.unpack_virtual_offset(6)
+ return nil if field_offset.zero?
+
+ element_size = 4
+ @view.unpack_vector(field_offset, element_size) do |element_offset|
+ @view.unpack_int(element_offset)
+ end
+ end
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/union_mode.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/union_mode.rb
new file mode 100644
index 0000000000..f4d862a430
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/union_mode.rb
@@ -0,0 +1,22 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ class UnionMode < ::FlatBuffers::Enum
+ SPARSE = register("Sparse", 0)
+ DENSE = register("Dense", 1)
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/utf8.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/utf8.rb
new file mode 100644
index 0000000000..916c5dd3e9
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/utf8.rb
@@ -0,0 +1,21 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # Unicode with UTF-8 encoding
+ class Utf8 < ::FlatBuffers::Table
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/utf8view.rb b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/utf8view.rb
new file mode 100644
index 0000000000..d627cdd8f1
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/org/apache/arrow/flatbuf/utf8view.rb
@@ -0,0 +1,27 @@
+# Automatically generated. Don't modify manually.
+#
+# Red FlatBuffers version: 0.0.3
+# Declared by: //Schema.fbs
+# Rooting type: org.apache.arrow.flatbuf.Message (//Message.fbs)
+
+require "flatbuffers"
+
+module ArrowFormat
+ module Org
+ module Apache
+ module Arrow
+ module Flatbuf
+ # Logically the same as Utf8, but the internal representation uses a view
+ # struct that contains the string length and either the string's entire data
+ # inline (for small strings) or an inlined prefix, an index of another buffer,
+ # and an offset pointing to a slice in that buffer (for non-small strings).
+ #
+ # Since it uses a variable number of data buffers, each Field with this type
+ # must have a corresponding entry in `variadicBufferCounts`.
+ class Utf8View < ::FlatBuffers::Table
+ end
+ end
+ end
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/record-batch.rb b/ruby/red-arrow-format/lib/arrow-format/record-batch.rb
new file mode 100644
index 0000000000..1258c62bab
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/record-batch.rb
@@ -0,0 +1,36 @@
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+module ArrowFormat
+ class RecordBatch
+ attr_reader :schema
+ attr_reader :n_rows
+ attr_reader :columns
+ def initialize(schema, n_rows, columns)
+ @schema = schema
+ @n_rows = n_rows
+ @columns = columns
+ end
+
+ def to_h
+ hash = {}
+ @schema.fields.zip(@columns) do |field, column|
+ hash[field.name] = column
+ end
+ hash
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/schema.rb b/ruby/red-arrow-format/lib/arrow-format/schema.rb
new file mode 100644
index 0000000000..5e58f68280
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/schema.rb
@@ -0,0 +1,24 @@
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+module ArrowFormat
+ class Schema
+ attr_reader :fields
+ def initialize(fields)
+ @fields = fields
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/type.rb b/ruby/red-arrow-format/lib/arrow-format/type.rb
new file mode 100644
index 0000000000..7f6ccc46fa
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/type.rb
@@ -0,0 +1,46 @@
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+module ArrowFormat
+ class Type
+ attr_reader :name
+ def initialize(name)
+ @name = name
+ end
+ end
+
+ class IntType < Type
+ attr_reader :bit_width
+ attr_reader :signed
+ def initialize(name, bit_width, signed)
+ super(name)
+ @bit_width = bit_width
+ @signed = signed
+ end
+ end
+
+ class Int8Type < IntType
+ def initialize
+ super("Int8", 8, true)
+ end
+ end
+
+ class UInt8Type < IntType
+ def initialize
+ super("UInt8", 8, false)
+ end
+ end
+end
diff --git a/ruby/red-arrow-format/lib/arrow-format/version.rb b/ruby/red-arrow-format/lib/arrow-format/version.rb
new file mode 100644
index 0000000000..389bd4dc5e
--- /dev/null
+++ b/ruby/red-arrow-format/lib/arrow-format/version.rb
@@ -0,0 +1,26 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+module ArrowFormat
+ VERSION = "23.0.0-SNAPSHOT"
+
+ module Version
+ numbers, TAG = VERSION.split("-")
+ MAJOR, MINOR, MICRO = numbers.split(".").collect(&:to_i)
+ STRING = VERSION
+ end
+end
diff --git a/ruby/red-arrow-flight-sql/red-arrow-flight-sql.gemspec b/ruby/red-arrow-format/red-arrow-format.gemspec
similarity index 55%
copy from ruby/red-arrow-flight-sql/red-arrow-flight-sql.gemspec
copy to ruby/red-arrow-format/red-arrow-format.gemspec
index bbb737b1b1..babe1466cf 100644
--- a/ruby/red-arrow-flight-sql/red-arrow-flight-sql.gemspec
+++ b/ruby/red-arrow-format/red-arrow-format.gemspec
@@ -17,36 +17,41 @@
# specific language governing permissions and limitations
# under the License.
-require_relative "lib/arrow-flight-sql/version"
+require_relative "lib/arrow-format/version"
Gem::Specification.new do |spec|
- spec.name = "red-arrow-flight-sql"
+ spec.name = "red-arrow-format"
+
version_components = [
- ArrowFlightSQL::Version::MAJOR.to_s,
- ArrowFlightSQL::Version::MINOR.to_s,
- ArrowFlightSQL::Version::MICRO.to_s,
- ArrowFlightSQL::Version::TAG,
+ ArrowFormat::Version::MAJOR.to_s,
+ ArrowFormat::Version::MINOR.to_s,
+ ArrowFormat::Version::MICRO.to_s,
+ ArrowFormat::Version::TAG,
]
spec.version = version_components.compact.join(".")
spec.homepage = "https://arrow.apache.org/"
- spec.authors = ["Apache Arrow Developers"]
+ spec.authors = ["The Apache Software Foundation"]
spec.email = ["dev@arrow.apache.org"]
- spec.summary =
- "Red Arrow Flight SQL is the Ruby bindings of Apache Arrow Flight SQL"
+ spec.summary = "Red Arrow Format is the pure Ruby Apache Arrow implementation"
spec.description =
- "Apache Arrow Flight is a protocol for interacting with SQL databases " +
- "using the Apache Arrow in-memory format and Apache Arrow Flight."
+ "Apache Arrow is a common in-memory columnar data store. " +
+ "It's useful to share and process large data efficiently. " +
+ "Red Arrow Format provides only serialize/deserialize features. " +
+ "If you want to process Apache Arrow data efficiently, " +
+ "use Red Arrow instead."
spec.license = "Apache-2.0"
spec.files = ["README.md", "Rakefile", "Gemfile", "#{spec.name}.gemspec"]
spec.files += ["LICENSE.txt", "NOTICE.txt"]
spec.files += Dir.glob("lib/**/*.rb")
- spec.test_files += Dir.glob("test/**/*")
- spec.extensions = ["dependency-check/Rakefile"]
+ spec.files += Dir.glob("doc/text/*")
- spec.add_runtime_dependency("red-arrow-flight", "= #{spec.version}")
+ spec.add_runtime_dependency("red-flatbuffers")
- spec.add_development_dependency("bundler")
- spec.add_development_dependency("rake")
- spec.add_development_dependency("test-unit")
+ github_url = "https://github.com/apache/arrow"
+ spec.metadata = {
+ "bug_tracker_uri" => "#{github_url}/issues",
+ "changelog_uri" => "#{github_url}/releases/tag/apache-arrow-#{spec.version}",
+ "source_code_uri" => github_url,
+ }
end
diff --git a/ruby/red-arrow-format/test/helper.rb b/ruby/red-arrow-format/test/helper.rb
new file mode 100644
index 0000000000..394d92d0dd
--- /dev/null
+++ b/ruby/red-arrow-format/test/helper.rb
@@ -0,0 +1,23 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+require "tmpdir"
+
+require "test-unit"
+
+require "arrow"
+require "arrow-format"
diff --git a/ruby/red-arrow-format/test/run.rb b/ruby/red-arrow-format/test/run.rb
new file mode 100755
index 0000000000..48d2c49e1d
--- /dev/null
+++ b/ruby/red-arrow-format/test/run.rb
@@ -0,0 +1,50 @@
+#!/usr/bin/env ruby
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+$VERBOSE = true
+
+require "pathname"
+
+(ENV["ARROW_DLL_PATH"] || "").split(File::PATH_SEPARATOR).each do |path|
+ RubyInstaller::Runtime.add_dll_directory(path)
+end
+
+base_dir = Pathname.new(__dir__).parent.expand_path
+arrow_base_dir = base_dir.parent + "red-arrow"
+
+lib_dir = base_dir + "lib"
+test_dir = base_dir + "test"
+
+arrow_lib_dir = arrow_base_dir + "lib"
+arrow_ext_dir = arrow_base_dir + "ext" + "arrow"
+
+build_dir = ENV["BUILD_DIR"]
+if build_dir
+ arrow_build_dir = Pathname.new(build_dir) + "red-arrow"
+else
+ arrow_build_dir = arrow_ext_dir
+end
+
+$LOAD_PATH.unshift(arrow_build_dir.to_s)
+$LOAD_PATH.unshift(arrow_lib_dir.to_s)
+$LOAD_PATH.unshift(lib_dir.to_s)
+
+require_relative "helper"
+
+exit(Test::Unit::AutoRunner.run(true, test_dir.to_s))
diff --git a/ruby/red-arrow-format/test/test-file-reader.rb b/ruby/red-arrow-format/test/test-file-reader.rb
new file mode 100644
index 0000000000..ef0a4a7200
--- /dev/null
+++ b/ruby/red-arrow-format/test/test-file-reader.rb
@@ -0,0 +1,49 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+class TestFileReader < Test::Unit::TestCase
+ def setup
+ Dir.mktmpdir do |tmp_dir|
+ table = Arrow::Table.new(uint8: Arrow::UInt8Array.new([1, 2, 3]))
+ @path = File.join(tmp_dir, "data.arrow")
+ table.save(@path)
+ File.open(@path, "rb") do |input|
+ @reader = ArrowFormat::FileReader.new(input)
+ yield
+ @reader = nil
+ end
+ GC.start
+ end
+ end
+
+ def read
+ @reader.to_a.collect do |record_batch|
+ record_batch.to_h.tap do |hash|
+ hash.each do |key, value|
+ hash[key] = value.to_a
+ end
+ end
+ end
+ end
+
+ def test_uint8
+ assert_equal([
+ {"uint8" => [1, 2, 3]},
+ ],
+ read)
+ end
+end
diff --git a/ruby/red-arrow/red-arrow.gemspec b/ruby/red-arrow/red-arrow.gemspec
index 53e8d8bc08..2487568b1d 100644
--- a/ruby/red-arrow/red-arrow.gemspec
+++ b/ruby/red-arrow/red-arrow.gemspec
@@ -33,13 +33,13 @@ Gem::Specification.new do |spec|
]
spec.version = version_components.compact.join(".")
spec.homepage = "https://arrow.apache.org/"
- spec.authors = ["Apache Arrow Developers"]
+ spec.authors = ["The Apache Software Foundation"]
spec.email = ["dev@arrow.apache.org"]
spec.summary = "Red Arrow is the Ruby bindings of Apache Arrow"
spec.description =
"Apache Arrow is a common in-memory columnar data store. " +
- "It's useful to share and process large data."
+ "It's useful to share and process large data efficiently."
spec.license = "Apache-2.0"
spec.files = ["README.md", "Rakefile", "Gemfile", "#{spec.name}.gemspec"]
spec.files += ["LICENSE.txt", "NOTICE.txt"]
diff --git a/ruby/red-gandiva/red-gandiva.gemspec b/ruby/red-gandiva/red-gandiva.gemspec
index ec4db8913e..dad1766c36 100644
--- a/ruby/red-gandiva/red-gandiva.gemspec
+++ b/ruby/red-gandiva/red-gandiva.gemspec
@@ -29,7 +29,7 @@ Gem::Specification.new do |spec|
]
spec.version = version_components.compact.join(".")
spec.homepage = "https://arrow.apache.org/"
- spec.authors = ["Apache Arrow Developers"]
+ spec.authors = ["The Apache Software Foundation"]
spec.email = ["dev@arrow.apache.org"]
spec.summary = "Red Gandiva is the Ruby bindings of Gandiva"
diff --git a/ruby/red-parquet/red-parquet.gemspec b/ruby/red-parquet/red-parquet.gemspec
index dffafed193..cb794044cb 100644
--- a/ruby/red-parquet/red-parquet.gemspec
+++ b/ruby/red-parquet/red-parquet.gemspec
@@ -29,7 +29,7 @@ Gem::Specification.new do |spec|
]
spec.version = version_components.compact.join(".")
spec.homepage = "https://arrow.apache.org/"
- spec.authors = ["Apache Arrow Developers"]
+ spec.authors = ["The Apache Software Foundation"]
spec.email = ["dev@arrow.apache.org"]
spec.summary = "Red Parquet is the Ruby bindings of Apache Parquet"