You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@fluss.apache.org by "leekeiabstraction (via GitHub)" <gi...@apache.org> on 2026/02/12 22:42:08 UTC
[PR] Fix issue where FixedSizedBinary type cannot be build in arrow [fluss-rust]
leekeiabstraction opened a new pull request, #304:
URL: https://github.com/apache/fluss-rust/pull/304
### Purpose
Fix bug where use of `DataTypes::binary(a)` fails as Arrow's FixedSizedBinaryBuilder is not used causing data write error.
### Tests
Updated IT
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix issue where FixedSizedBinary type cannot be build in arrow [fluss-rust]
Posted by "leekeiabstraction (via GitHub)" <gi...@apache.org>.
leekeiabstraction commented on PR #304:
URL: https://github.com/apache/fluss-rust/pull/304#issuecomment-3895885123
Addressed comments, PTAL!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix issue where FixedSizedBinary type cannot be build in arrow [fluss-rust]
Posted by "fresh-borzoni (via GitHub)" <gi...@apache.org>.
fresh-borzoni commented on code in PR #304:
URL: https://github.com/apache/fluss-rust/pull/304#discussion_r2801820224
##########
crates/fluss/src/row/datum.rs:
##########
@@ -493,7 +494,21 @@ impl Datum<'_> {
Datum::Float32(v) => append_value_to_arrow!(Float32Builder, v.into_inner()),
Datum::Float64(v) => append_value_to_arrow!(Float64Builder, v.into_inner()),
Datum::String(v) => append_value_to_arrow!(StringBuilder, v.as_ref()),
- Datum::Blob(v) => append_value_to_arrow!(BinaryBuilder, v.as_ref()),
+ Datum::Blob(v) => {
+ append_value_to_arrow!(BinaryBuilder, v.as_ref());
+ if let Some(b) = builder
+ .as_any_mut()
+ .downcast_mut::<FixedSizeBinaryBuilder>()
+ {
+ return b.append_value(v.as_ref()).map_err(|e| RowConvertError {
+ message: format!("Failed to append FixedSizeBinary: {e}"),
Review Comment:
Can we have smth like - FixedSizeBinary length mismatch: expected {n}, got {}.
I mean we know FixedSizeBinary length and also know the length of the bytes we are writing, if they don't match - we can just short circuit and it be a lot easier to reason.
Actually in Arrow, it's the only thing that can fail in this case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix issue where FixedSizedBinary type cannot be build in arrow [fluss-rust]
Posted by "leekeiabstraction (via GitHub)" <gi...@apache.org>.
leekeiabstraction commented on PR #304:
URL: https://github.com/apache/fluss-rust/pull/304#issuecomment-3893793769
@luoyuxia @fresh-borzoni Appreciate a review here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix issue where FixedSizedBinary type cannot be build in arrow [fluss-rust]
Posted by "leekeiabstraction (via GitHub)" <gi...@apache.org>.
leekeiabstraction commented on code in PR #304:
URL: https://github.com/apache/fluss-rust/pull/304#discussion_r2803140143
##########
crates/fluss/src/row/datum.rs:
##########
@@ -493,7 +494,21 @@ impl Datum<'_> {
Datum::Float32(v) => append_value_to_arrow!(Float32Builder, v.into_inner()),
Datum::Float64(v) => append_value_to_arrow!(Float64Builder, v.into_inner()),
Datum::String(v) => append_value_to_arrow!(StringBuilder, v.as_ref()),
- Datum::Blob(v) => append_value_to_arrow!(BinaryBuilder, v.as_ref()),
+ Datum::Blob(v) => {
+ append_value_to_arrow!(BinaryBuilder, v.as_ref());
+ if let Some(b) = builder
+ .as_any_mut()
+ .downcast_mut::<FixedSizeBinaryBuilder>()
+ {
+ return b.append_value(v.as_ref()).map_err(|e| RowConvertError {
+ message: format!("Failed to append FixedSizeBinary: {e}"),
Review Comment:
I've made it so that arrow error is surfaced instead
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix issue where FixedSizedBinary type cannot be build in arrow [fluss-rust]
Posted by "fresh-borzoni (via GitHub)" <gi...@apache.org>.
fresh-borzoni commented on code in PR #304:
URL: https://github.com/apache/fluss-rust/pull/304#discussion_r2803445163
##########
crates/fluss/src/row/datum.rs:
##########
@@ -493,7 +494,21 @@ impl Datum<'_> {
Datum::Float32(v) => append_value_to_arrow!(Float32Builder, v.into_inner()),
Datum::Float64(v) => append_value_to_arrow!(Float64Builder, v.into_inner()),
Datum::String(v) => append_value_to_arrow!(StringBuilder, v.as_ref()),
- Datum::Blob(v) => append_value_to_arrow!(BinaryBuilder, v.as_ref()),
+ Datum::Blob(v) => {
+ append_value_to_arrow!(BinaryBuilder, v.as_ref());
+ if let Some(b) = builder
+ .as_any_mut()
+ .downcast_mut::<FixedSizeBinaryBuilder>()
+ {
+ return b.append_value(v.as_ref()).map_err(|e| RowConvertError {
+ message: format!("Failed to append FixedSizeBinary: {e}"),
Review Comment:
Sure, it's marginal
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix issue where FixedSizedBinary type cannot be build in arrow [fluss-rust]
Posted by "leekeiabstraction (via GitHub)" <gi...@apache.org>.
leekeiabstraction commented on code in PR #304:
URL: https://github.com/apache/fluss-rust/pull/304#discussion_r2803137508
##########
crates/fluss/src/row/datum.rs:
##########
@@ -493,7 +494,21 @@ impl Datum<'_> {
Datum::Float32(v) => append_value_to_arrow!(Float32Builder, v.into_inner()),
Datum::Float64(v) => append_value_to_arrow!(Float64Builder, v.into_inner()),
Datum::String(v) => append_value_to_arrow!(StringBuilder, v.as_ref()),
- Datum::Blob(v) => append_value_to_arrow!(BinaryBuilder, v.as_ref()),
+ Datum::Blob(v) => {
Review Comment:
Good call!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix issue where FixedSizedBinary type cannot be build in arrow [fluss-rust]
Posted by "fresh-borzoni (via GitHub)" <gi...@apache.org>.
fresh-borzoni commented on code in PR #304:
URL: https://github.com/apache/fluss-rust/pull/304#discussion_r2801797755
##########
crates/fluss/src/row/datum.rs:
##########
@@ -493,7 +494,21 @@ impl Datum<'_> {
Datum::Float32(v) => append_value_to_arrow!(Float32Builder, v.into_inner()),
Datum::Float64(v) => append_value_to_arrow!(Float64Builder, v.into_inner()),
Datum::String(v) => append_value_to_arrow!(StringBuilder, v.as_ref()),
- Datum::Blob(v) => append_value_to_arrow!(BinaryBuilder, v.as_ref()),
+ Datum::Blob(v) => {
Review Comment:
Maybe we should use arrow_schema::DataType to dispatch?
##########
crates/fluss/src/row/datum.rs:
##########
@@ -493,7 +494,21 @@ impl Datum<'_> {
Datum::Float32(v) => append_value_to_arrow!(Float32Builder, v.into_inner()),
Datum::Float64(v) => append_value_to_arrow!(Float64Builder, v.into_inner()),
Datum::String(v) => append_value_to_arrow!(StringBuilder, v.as_ref()),
- Datum::Blob(v) => append_value_to_arrow!(BinaryBuilder, v.as_ref()),
+ Datum::Blob(v) => {
+ append_value_to_arrow!(BinaryBuilder, v.as_ref());
+ if let Some(b) = builder
+ .as_any_mut()
+ .downcast_mut::<FixedSizeBinaryBuilder>()
+ {
+ return b.append_value(v.as_ref()).map_err(|e| RowConvertError {
+ message: format!("Failed to append FixedSizeBinary: {e}"),
Review Comment:
Can we have smth like - FixedSizeBinary length mismatch: expected {n}, got {}?
I mean we know FixedSizeBinary length and also know the length of the bytes we are writing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix issue where FixedSizedBinary type cannot be build in arrow [fluss-rust]
Posted by "luoyuxia (via GitHub)" <gi...@apache.org>.
luoyuxia commented on code in PR #304:
URL: https://github.com/apache/fluss-rust/pull/304#discussion_r2803050561
##########
crates/fluss/src/row/datum.rs:
##########
@@ -493,7 +494,21 @@ impl Datum<'_> {
Datum::Float32(v) => append_value_to_arrow!(Float32Builder, v.into_inner()),
Datum::Float64(v) => append_value_to_arrow!(Float64Builder, v.into_inner()),
Datum::String(v) => append_value_to_arrow!(StringBuilder, v.as_ref()),
- Datum::Blob(v) => append_value_to_arrow!(BinaryBuilder, v.as_ref()),
+ Datum::Blob(v) => {
Review Comment:
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix issue where FixedSizedBinary type cannot be build in arrow [fluss-rust]
Posted by "luoyuxia (via GitHub)" <gi...@apache.org>.
luoyuxia merged PR #304:
URL: https://github.com/apache/fluss-rust/pull/304
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org