You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "kou (via GitHub)" <gi...@apache.org> on 2025/12/15 05:06:02 UTC
[PR] GH-48537: [Ruby] Add support for reading fixed size binary array [arrow]
kou opened a new pull request, #48538:
URL: https://github.com/apache/arrow/pull/48538
### Rationale for this change
It's a fixed size variant of binary array.
### What changes are included in this PR?
* Add `ArrowFormat::FixedSizeBinaryType`
* Add `ArrowFormat::FixedSizeBinaryArray`
### Are these changes tested?
Yes.
### Are there any user-facing changes?
Yes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48537: [Ruby] Add support for reading fixed size binary array [arrow]
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #48538:
URL: https://github.com/apache/arrow/pull/48538#issuecomment-3659166356
After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit fdcc6472bd94626a423f192d8fa067c59eacdc70.
There were no benchmark performance regressions. 🎉
The [full Conbench report](https://github.com/apache/arrow/runs/58169226824) has more details. It also includes information about 1 possible false positive for unstable benchmarks that are known to sometimes produce them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48537: [Ruby] Add support for reading fixed size binary array [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #48538:
URL: https://github.com/apache/arrow/pull/48538#issuecomment-3653009969
:warning: GitHub issue #48537 **has been automatically assigned in GitHub** to PR creator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48537: [Ruby] Add support for reading fixed size binary array [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou merged PR #48538:
URL: https://github.com/apache/arrow/pull/48538
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48537: [Ruby] Add support for reading fixed size binary array [arrow]
Posted by "hiroyuki-sato (via GitHub)" <gi...@apache.org>.
hiroyuki-sato commented on code in PR #48538:
URL: https://github.com/apache/arrow/pull/48538#discussion_r2618613889
##########
ruby/red-arrow-format/lib/arrow-format/file-reader.rb:
##########
@@ -256,6 +258,16 @@ def read_column(field, nodes, buffers, body)
values_buffer = buffers.shift
values = body.slice(values_buffer.offset, values_buffer.length)
field.type.build_array(length, validity, values)
+ when VariableSizeBinaryType
Review Comment:
Just for interest. What sort order are you use? Why you moved `VariableSizeBinaryType` here?
(I asked for furture implementatation)
##########
ruby/red-arrow-format/lib/arrow-format/array.rb:
##########
@@ -229,6 +229,32 @@ def encoding
end
end
+ class FixedSizeBinaryArray < Array
+ def initialize(type, size, validity_buffer, values_buffer)
+ super(type, size, validity_buffer)
+ @values_buffer = values_buffer
+ end
+
+ def to_a
+ byte_width = @type.byte_width
+ values = 0.step(@size * byte_width - 1, byte_width).collect do |offset|
+ @values_buffer.get_string(offset, byte_width)
+ end
+ apply_validity(values)
+ end
+ end
+
+ class BinaryArray < VariableSizeBinaryLayoutArray
+ private
+ def buffer_type
+ :s32 # TODO: big endian support
+ end
+
+ def encoding
+ Encoding::ASCII_8BIT
+ end
+ end
+
Review Comment:
```suggestion
```
```
/path/to/arrow/ruby/red-arrow-format/lib/arrow-format/array.rb:249: warning: method redefined; discarding old buffer_type
/path/to/arrow/ruby/red-arrow-format/lib/arrow-format/array.rb:201: warning: previous definition of buffer_type was here
/path/to/arrow/ruby/red-arrow-format/lib/arrow-format/array.rb:253: warning: method redefined; discarding old encoding
/path/to/arrow/ruby/red-arrow-format/lib/arrow-format/array.rb:205: warning: previous definition of encoding was here
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48537: [Ruby] Add support for reading fixed size binary array [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on code in PR #48538:
URL: https://github.com/apache/arrow/pull/48538#discussion_r2621276242
##########
ruby/red-arrow-format/lib/arrow-format/array.rb:
##########
@@ -229,6 +229,32 @@ def encoding
end
end
+ class FixedSizeBinaryArray < Array
+ def initialize(type, size, validity_buffer, values_buffer)
+ super(type, size, validity_buffer)
+ @values_buffer = values_buffer
+ end
+
+ def to_a
+ byte_width = @type.byte_width
+ values = 0.step(@size * byte_width - 1, byte_width).collect do |offset|
+ @values_buffer.get_string(offset, byte_width)
+ end
+ apply_validity(values)
+ end
+ end
+
+ class BinaryArray < VariableSizeBinaryLayoutArray
+ private
+ def buffer_type
+ :s32 # TODO: big endian support
+ end
+
+ def encoding
+ Encoding::ASCII_8BIT
+ end
+ end
+
Review Comment:
Oh, sorry for the garbage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] GH-48537: [Ruby] Add support for reading fixed size binary array [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on code in PR #48538:
URL: https://github.com/apache/arrow/pull/48538#discussion_r2621276525
##########
ruby/red-arrow-format/lib/arrow-format/file-reader.rb:
##########
@@ -256,6 +258,16 @@ def read_column(field, nodes, buffers, body)
values_buffer = buffers.shift
values = body.slice(values_buffer.offset, values_buffer.length)
field.type.build_array(length, validity, values)
+ when VariableSizeBinaryType
Review Comment:
I used primitive types -> nested types order.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org