You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2023/11/08 19:06:14 UTC
[I] Unsupported functions in protobuf serializatin [arrow-datafusion]
alamb opened a new issue, #8098:
URL: https://github.com/apache/arrow-datafusion/issues/8098
### Describe the bug
Inspired by https://github.com/apache/arrow-datafusion/pull/8089/files from @Syleechan I checked if there were other functions not covered by our serialization logic. It turns out that yes there are:
### To Reproduce
Remove the catchall in parse_expr:
```diff
diff --git a/datafusion/proto/src/logical_plan/from_proto.rs b/datafusion/proto/src/logical_plan/from_proto.rs
index cdb0fe9bda..52a8480146 100644
--- a/datafusion/proto/src/logical_plan/from_proto.rs
+++ b/datafusion/proto/src/logical_plan/from_proto.rs
@@ -1645,9 +1645,6 @@ pub fn parse_expr(
)),
ScalarFunction::Isnan => Ok(isnan(parse_expr(&args[0], registry)?)),
ScalarFunction::Iszero => Ok(iszero(parse_expr(&args[0], registry)?)),
- _ => Err(proto_error(
- "Protobuf deserialization error: Unsupported scalar function",
- )),
}
}
ExprType::ScalarUdfExpr(protobuf::ScalarUdfExprNode { fun_name, args }) => {
```
The compiler will tell you there are several more functions not covered *like `ToTimestamp` and `ArrowTypeOf`:
```
error[E0004]: non-exhaustive patterns: `datafusion::ScalarFunction::ToTimestamp`, `datafusion::ScalarFunction::StructFun`, `datafusion::ScalarFunction::ArrowTypeof` and 2 more not covered
--> datafusion/proto/src/logical_plan/from_proto.rs:1303:19
|
1303 | match scalar_function {
| ^^^^^^^^^^^^^^^ patterns `datafusion::ScalarFunction::ToTimestamp`, `datafusion::ScalarFunction::StructFun`, `datafusion::ScalarFunction::ArrowTypeof` and 2 more not covered
|
note: `datafusion::ScalarFunction` defined here
```
### Expected behavior
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Unsupported functions in protobuf serializatin [arrow-datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #8098:
URL: https://github.com/apache/arrow-datafusion/issues/8098#issuecomment-1802489839
I think this is a good first issue as it has an example of how to fix it, and would get someone exposed to the various logical plan structures
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Unsupported functions in protobuf serializatin [arrow-datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #8098: Unsupported functions in protobuf serializatin
URL: https://github.com/apache/arrow-datafusion/issues/8098
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org