You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@arrow.apache.org by "Will Jones (Jira)" <ji...@apache.org> on 2022/10/11 22:59:00 UTC
[jira] [Commented] (ARROW-17994) [C++] Add overflow argument is required when it shouldn't be
[ https://issues.apache.org/jira/browse/ARROW-17994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616096#comment-17616096 ]
Will Jones commented on ARROW-17994:
------------------------------------
Example plan that is broken:
{code:json}
{
"extensionUris": [
{
"extensionUriAnchor": 1
}
],
"extensions": [
{
"extensionFunction": {
"extensionUriReference": 1,
"functionAnchor": 1,
"name": "equal"
}
},
{
"extensionFunction": {
"extensionUriReference": 1,
"functionAnchor": 2,
"name": "add"
}
}
],
"relations": [
{
"root": {
"input": {
"project": {
"input": {
"read": {
"baseSchema": {
"names": [
"a",
"b",
"c"
],
"struct": {
"types": [
{
"fp64": {
"nullability": "NULLABILITY_NULLABLE"
}
},
{
"i64": {
"nullability": "NULLABILITY_NULLABLE"
}
},
{
"fp64": {
"nullability": "NULLABILITY_NULLABLE"
}
}
],
"nullability": "NULLABILITY_REQUIRED"
}
},
"namedTable": {
"names": [
"table0"
]
}
}
},
"expressions": [
{
"scalarFunction": {
"functionReference": 2,
"outputType": {
"fp64": {
"nullability": "NULLABILITY_NULLABLE"
}
},
"arguments": [
{
"value": {
"selection": {
"directReference": {
"structField": {}
},
"rootReference": {}
}
}
},
{
"value": {
"selection": {
"directReference": {
"structField": {
"field": 2
}
},
"rootReference": {}
}
}
}
]
}
}
]
}
},
"names": [
"a",
"b",
"c",
"v"
]
}
}
]
}
{code}
Example plan that works:
{code:json}
{
"extensionUris": [
{
"extensionUriAnchor": 1
}
],
"extensions": [
{
"extensionFunction": {
"extensionUriReference": 1,
"functionAnchor": 1,
"name": "equal"
}
},
{
"extensionFunction": {
"extensionUriReference": 1,
"functionAnchor": 2,
"name": "add"
}
}
],
"relations": [
{
"root": {
"input": {
"project": {
"input": {
"read": {
"baseSchema": {
"names": [
"a",
"b",
"c"
],
"struct": {
"types": [
{
"fp64": {
"nullability": "NULLABILITY_NULLABLE"
}
},
{
"i64": {
"nullability": "NULLABILITY_NULLABLE"
}
},
{
"fp64": {
"nullability": "NULLABILITY_NULLABLE"
}
}
],
"nullability": "NULLABILITY_REQUIRED"
}
},
"namedTable": {
"names": [
"table0"
]
}
}
},
"expressions": [
{
"scalarFunction": {
"functionReference": 2,
"outputType": {
"fp64": {
"nullability": "NULLABILITY_NULLABLE"
}
},
"arguments": [
{
"enum": {
"specified": "ERROR"
}
},
{
"value": {
"selection": {
"directReference": {
"structField": {}
},
"rootReference": {}
}
}
},
{
"value": {
"selection": {
"directReference": {
"structField": {
"field": 2
}
},
"rootReference": {}
}
}
}
]
}
}
]
}
},
"names": [
"a",
"b",
"c",
"v"
]
}
}
]
}
{code}
> [C++] Add overflow argument is required when it shouldn't be
> ------------------------------------------------------------
>
> Key: ARROW-17994
> URL: https://issues.apache.org/jira/browse/ARROW-17994
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Reporter: Will Jones
> Priority: Major
> Labels: acero, substrait
> Fix For: 11.0.0
>
>
> If I pass a substrait plan that contains an add function, but don't provide the nullablity argument, I get the following error:
> {code:none}
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "pyarrow/_substrait.pyx", line 140, in pyarrow._substrait.run_query
> File "pyarrow/error.pxi", line 144, in pyarrow.lib.pyarrow_internal_check_status
> File "pyarrow/error.pxi", line 100, in pyarrow.lib.check_status
> pyarrow.lib.ArrowInvalid: Expected Substrait call to have an enum argument at index 0 but the argument was not an enum.
> /Users/willjones/Documents/arrows/arrow/cpp/src/arrow/engine/substrait/extension_set.cc:684 call.GetEnumArg(arg_index)
> /Users/willjones/Documents/arrows/arrow/cpp/src/arrow/engine/substrait/extension_set.cc:702 ParseEnumArg(call, 0, kOverflowParser)
> /Users/willjones/Documents/arrows/arrow/cpp/src/arrow/engine/substrait/relation_internal.cc:332 FromProto(expr, ext_set, conversion_options)
> /Users/willjones/Documents/arrows/arrow/cpp/src/arrow/engine/substrait/serde.cc:156 FromProto(plan_rel.has_root() ? plan_rel.root().input() : plan_rel.rel(), ext_set, conversion_options)
> /Users/willjones/Documents/arrows/arrow/cpp/src/arrow/engine/substrait/util.cc:106 engine::DeserializePlans(substrait_buffer, consumer_factory, registry, nullptr, conversion_options_)
> /Users/willjones/Documents/arrows/arrow/cpp/src/arrow/engine/substrait/util.cc:130 executor.Init(substrait_buffer, registry)
> {code}
> Yet in the spec, this argument is supposed to be optional: https://github.com/substrait-io/substrait/blob/f3f6bdc947e689e800279666ff33f118e42d2146/extensions/functions_arithmetic.yaml#L11
> If I modify the plan to include the argument, it works as expected.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)